Add native SPM role source enrichment and release verification - #894
Merged
Conversation
This was referenced Sep 9, 2026
MaxGhenis
marked this pull request as ready for review
September 9, 2026 16:23
…ection-seed moves only the draw (#355) A --dataset-households run replaces its calibration product with the compact refit, but the dense joint solve it was cut from is byte-identical to a standalone dense run on the same inputs, seed and epochs. Keeping it lets a size candidate be compared with its own dense reference without a second full-pool run: - local_rowwise: the evidence labelling (alignment by name, exact target values, local/national diagnostics, past-cap censuses, initial loss) moves into _doctrine_solve_evidence and runs for both results; the solve product carries `dense_reference` (weights, initial weights, diagnostics, losses, censuses). - driver: a size run writes dense_reference_diagnostics.csv (every target's dense estimate, local rows then national, with a `grain` column) and dataset_size_selection.csv (pool row index, household id, clone index, design weight, inclusion probability, certainty, Horvitz–Thompson baseline, refit weight), lists both under outputs with digests, and summarises the reference under solve.dataset_size (losses, fit by family, uk_weight_summary). Dense runs are unchanged; the two files are size-run-only in the publish order. - --selection-seed (requires --dataset-households; defaults to --seed) seeds only the informed L0 search, the exact-count draw and the refit, threaded through the holdout as well, so two selections compare on one pool and one dense reference; recorded as parameters.selection_seed. Tests: the doctrine solve keeps a reference equal to the standalone dense solve and a selection seed leaves it untouched; the CLI size run writes both sidecars, lists them, and records both seeds; --selection-seed without a size is refused. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd receipt (#355) The first licensed size rehearsal (55,000 of 792,690 at 100 epochs) was refused inside select_exact_k: "degenerate boundary mass … adjust pi_hi or k". With pi_hi=1.0 every gate below one stays in the boundary and its open probability is scaled to the remaining draw size m, so the design is feasible only when m * max(pi_boundary) <= sum(pi_boundary); the L0 budget search stops on the count of not-fully-closed gates, which sits above the open-probability mass while gates are only partly polarised. The refusal carried no numbers. refit_uk_dataset_size now measures selection_feasibility before the draw — certainties and boundary draw at pi_hi=1, boundary mass and largest boundary gate, feasibility, the largest household count feasible at pi_hi=1, the smallest feasible pi_hi on a fixed grid, gate quantiles and counts, the budget search's n_nonzero and lambda — and attaches it to the ValueError on refusal and to the size receipt on success. Nothing is clamped or promoted; the ruling on pi_hi stays a reviewed decision, now made from measured mass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…evaluate_uk_dataset_size steps 00-90, uk_fit_by_family lifted (#355) Every evaluation piece for a --dataset-households candidate now has one name and one place: - uk_runtime/size_evaluation.py — pure measurement over a run directory: load_run (tolerant of pre-#877 dense runs), run_acceptance (the size-run checklist, not_applicable on dense runs), fit_tables (by grain and family), weight_tables (Kish ESS, distinct sources, stretch vs the Horvitz–Thompson baseline and vs pool design from the selection sidecar or a spine, self-checked against the manifest), area_support_tables (per-grain floors and breaches), gate_table (six ids with criticality), paired_targets (join on name; wins/ties/losses; the reference's red rows tracked through family/area/metric), dense_reference_deltas (size-only effect from the run's own dense reference), frozen_vs_recomputed (national rows against the incumbent-surface evaluator), footprint, and summarize with PRE_REGISTERED_OUTCOMES_V1 flags. No subprocess, no engine, no licensed data in tests. - tools/evaluate_uk_dataset_size.py — the one command: steps 00-run-acceptance, 10-dense-reference, 20-vs-reference/<label>, 30-incumbent-score (score_uk_local_candidate), 40-incumbent-surface (evaluate_uk_incumbent_surface --engine-blocks 1 + frozen vs recomputed), 50-downstream (uk-candidate-eval T3/T4/T5 and the #731 scorecard under this venv, each timed with /usr/bin/time -l), 90-summary (EVALUATION.md + JSON with the pre-registered table and a file index); every step writes receipt.json (inputs and outputs with digests, argv, wall, exit) and is skipped with a reason when its inputs are absent; re-running a step rewrites only its folder. - _fit_by_family lifted from the driver into uk_runtime/diagnostics.py::uk_fit_by_family unchanged; the driver imports it. Tests: test_uk_size_evaluation.py (synthetic run dirs in both manifest shapes, table-format H5s, time -l parsing, red-row mapping, stretch self-check, breaches, flags, refusals) and test_uk_evaluate_dataset_size_tool.py (tree, receipts, manifest, summary; skipped steps; a failing downstream stub recorded without aborting; single-step re-run). Implemented by Codex from .codex-work/PLAN.md; verified here: ruff, format, ci_test_groups --verify, 88 tests across the four UK rowwise/evaluation files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…/spine-p, S0, S1/S1b refusals with the feasibility numbers (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t, as the wheels venv is (#355) The wheels lane runs the suite against installed wheels in a venv without pytables; the size CLI test read the exported candidate through pd.HDFStore and failed there (the only failure in 7,963 tests). The rowwise candidate test file's convention is importorskip("tables") and importorskip("h5py") at the top of every CLI test; the size test now follows it, and the two evaluation test files, which write PyTables-format H5 fixtures, carry the module-level importorskip the same lane relies on. With pytables present nothing is skipped (41 tests run); with the import blocked the modules skip instead of erroring. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…b: --selection-pi-hi (#355) refit_uk_dataset_size takes pi_hi (default 1.0, the protected-carriers-only design); the driver exposes it as --selection-pi-hi, candidate-only and bounded to (0, 1], threaded through the holdout, recorded as parameters.selection_pi_hi and in the size receipt (selection_pi_hi, the draw receipt's pi_hi, and the feasibility measurement's requested_pi_hi / feasible_at_requested_pi_hi). The refusal message names the requested threshold. A threshold below one promotes learned near-certain gates; the plan doc records the measured reason and the 2026-09-08 ruling (smoke at the measured feasible count; the 55,000 candidate at pi_hi 0.95, the US exact-k ladder's setting, and 2,000 epochs). Nothing changes for dense runs or for the default. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…355) The 45,800 smoke on spine-p (the tree rebased onto #879) was refused inside contribution_initialization: "nonzero target at row 20648 has no support", a national row whose constraint row is all zeros on the cloned pool. The dense solve tolerates such a row as a miss; a size selection cannot carry it, and the row index alone is not actionable. refit_uk_dataset_size now lists every nonzero target with no supporting household by name before initialisation and refuses with the list, so the binding defect is fixed upstream or the row is excluded with a signed reason, never selected around. unsupported_nonzero_targets() is the reusable check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…macOS's /usr/bin/time -l (#355) The evaluation command timed each downstream script with `/usr/bin/time -l`, a BSD flag GNU time rejects with exit 125, so on the linux CI runners a failing stub could not surface its own exit code (test_downstream_continues_after_a_failing_stub: 125 != 7). The timed form now runs the script in a child interpreter through a shim that executes it in-process and reports the child's own peak resident set (bytes on every platform) on stderr at exit; wall time is measured by the caller and the exit code is the script's own. The marker line is stripped from the recorded stderr tail. Same receipt shape on macOS and linux. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s above £27,600 (#355, ruling 2026-09-08) Under #879's relationship-based uc_calibration_family_type no benefit unit in the spine is a couple without children with an annual UC award in the 27,600-28,800 or 28,800-30,000 band: both rows sit at -100% on #879's corrected comparison and have no carrier on the K=15 rowwise pool of spine-p, where the #355 size selection refuses them by name. The dense seam tolerated them as capped misses. Signed into the reviewed measure exclusion register on the zero-support-channel precedent (approved by juaristi22, 2026-09-08, expiring 2026-12-08; catalogue and top-band repair tracked in microcosm#736; the sibling 26,400-27,600 band was already excluded). Register census pins move with it: 18 payment-band exclusions, 82 active payment bands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…aunch (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… smoke4 at pi_hi 0.7 (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t binding defect (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… never write one (#355) score_uk_local_candidate needs the frozen compiled register R14 built separately; --scoring-registry names it, with <run>/ and each reference's scoring_registry_compiled.json as fallbacks, and the skip reason says so. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t; re-pin the identities it moves (#355, ruling 2026-09-08) The exact-count draw can only draw from the gates' open-probability mass, but the L0 budget search stopped on the count of not-fully-closed weights, which sat 10-13% above that mass in every licensed measurement; the draw then refused, and the "feasible count" moved with every request. calibrate() gains budget_basis ("nonzero_count", the historical default, or "open_probability_mass": each probe is measured by sum(pi), the gates' expected open count) and records it in options; the size refit uses the mass basis and records selection_budget_basis, and the feasibility measurement names its basis. On a 60-record toy problem the mass basis lands sum(pi) 20.2 for a budget of 20 where the count basis gives count 19 with mass 18.5. solve.py is attested, so the identities move again: the am/be/uk country-bundle digests, the loader golden vector, the seed-protocol and seed-map digests with the regenerated US coverage report, the H1 calibrate parity fixture (regenerated on the authoring platform; direct bytes unchanged), and the US spec digest pinned in the multispine pool-tool test. 245 tests across the pin and parity files pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…onthly rent x A17-uprated private-renter households (#355, ruling 2026-09-08) ons.rent.private_rent (314 local-authority rows, ONS PIPR, active since #874's re-pin) bound a mean monthly rent to rent/private_rent, a weighted annual total, so every row sat at a relative error of 1e5-1e6 and at the loss cap on the rowwise pool. The calibration is linear in the weights, so the mean's linear form is a total: uk_private_rent_mean_to_total, applied in uk_local_target_surface after the A15/A17 uprating, composes each row's value as 12 x mean x the same authority's bound tenure/private_rent count, keeps the mean and the count in the row's metadata, refuses by area when the count is missing or non-positive, and records every cell under cross_grain.private_rent_mean_to_total. The metric is unchanged. The census rationale for the PIPR source no longer claims the feed carries only 2026-06 (it carries the 2025 months the reference averages); the committed census artifact is regenerated. Implemented by Codex from .codex-work/PLAN-rent.md; verified here: ruff clean, 137 tests across the ledger-target, local rowwise, rowwise candidate and census files. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… on 2026-09-08 (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…un, the S2 launch (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e ±5% band diagnosis, options (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… checkpoint the solve before the draw (#355) S2 (55,000 at pi_hi 0.95, 2,000 epochs) was refused at the exact-count draw after 4.8 hours: the mass-basis budget search stopped inside its +/-5% band 166 rows under the request, the draw's condition is one-sided, and nothing was persisted, so the dense solve was lost with the refusal. Fix A. exact_k_design_feasibility(pi, k, pi_hi) is the draw's inequality without the draw (verdicts feasible / certainties_exceed_k / boundary_short_of_draw / boundary_mass_short; random polarised designs agree with select_exact_k on every one). calibrate(..., feasible_draw_pi_hi=h) on the mass basis makes the budget search accept a probe only when a draw of target_records at h is feasible on its gate probabilities, steers an infeasible probe like a count miss, prefers feasible probes for the returned run, and records every probe under options["budget_search"]. The UK size selection passes its threshold; the size receipt carries the search receipt and each scan threshold's verdict. Fix C. refit_uk_dataset_size is split into select_uk_dataset_size (the search) and the draw + refit, which accepts an existing selection. uk_runtime/size_checkpoint.py persists the dense solve and the selection with the identity of the pool, the target surface and the solve settings; load rebuilds both through the new calibrate.rebuild_calibration_result on the resumed run's compiled pool and refuses by name on identity, pool or surface drift. The doctrine solve and the candidate driver write the checkpoint into --out by default on size runs (--no-size-checkpoint) and --resume-size-checkpoint DIR continues at the draw, at any threshold, with both thresholds and Logbook phases recorded. Attested calibrate modules moved: country-bundle digests, seed protocol and map, loader golden vector, US spec digest (multispine pool-tool test), the regenerated coverage report and the H1 calibrate parity fixture. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…to the pre-registered table; P95b launched on the fixed code (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…et probes with their drawability verdict, the search stop (#355) A size run was silent between "solving ... under the doctrine..." and its manifest, hours later. The budget search now emits one budget_probe event per finished probe (penalty, measure, certainties, boundary draw and mass, verdict) and a budget_search_done event beside the per-epoch stream, on the calibrator's existing progress_callback seam. The UK doctrine solve takes a progress line sink (uk_runtime/solve_progress.py formats timestamped lines: a loss every hundred epochs and at the last epoch of the dense solve, of each probe and of the refit; one line per probe; one when the search stops), the size search and refit are tagged by phase, and the candidate driver writes the lines to stderr so they land in the run log. solve.py is attested: the country-bundle digests, seed protocol and map, loader golden vector, US spec digest, coverage report and H1 calibrate parity fixture are re-cut. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… receipts: one-engine scorecard, incumbent-score finding, exclusion audit (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…chine (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…6 replays against the enhanced FRS (#355) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…oint guard, attested and frozen gates, honest draw and stretch keys (#355) 1. The size-checkpoint identity carries the solve doctrine and the doctrine solve asserts the restored max_weight_ratio, mass and target_loss_cap against today's doctrine after a resume; the writing run's code pin and build id ride in the checkpoint's provenance and are reported, not compared. 2. An --out that already holds a size checkpoint is refused before the solve. 3. microcosm.calibrate.gates and microcosm.calibrate.initialization join the seed-protocol implementation digest, and the pre-best-iterate oracle runs with a frozen copy of the gate module loaded as pre_best_iterate_gates, so a gate-behaviour change surfaces as a numeric mismatch on the gated control path instead of being absorbed on both sides. 4. The size receipt reports certainty_share, boundary_draws and zero_target_rows; the manifest distinguishes the realized stretch against the refit's reference from the stretch against the pool design weights. 5. The written-checkpoint receipt carries no timestamp and no absolute path. Rebased over #891. The attestation change moved the seed protocol, the pointer inventory, the field and mode counts and the resolved_seed_protocol claim, the country-bundle digests, the loader golden vector and the US spec digest; all re-cut, the coverage report regenerated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tialization (#355) Attesting microcosm.calibrate.gates and microcosm.calibrate.initialization in seeds.py added two resolved-binding fields (9,770 -> 9,772; ledger 42,154 -> 42,156; compiler_semantic 27,715 -> 27,717; no_generation0_effect 3,678 -> 3,680). field_usage.py and the coverage tool test were re-pinned in c2ee134; this sibling test only runs in the rest, engine-shared and wheels lanes and was missed. Values recomputed from the ledger, not transcribed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…able-final-20260909
UK dataset sizes: informed L0 and exact-count candidates (#355)
Cancel outdated PR CI runs while preserving main pushes
Align UK CGT calibration with FY2024-25 individual observations
…ISRA's published lookup (#887) - Re-pin the UK Chronicle consumer feed 6fb700e -> ec7169b (131,450 facts, facts sha 4a50ee95); no national or local reference changed status or value. - Northern Ireland: replace the active-postcode modal DZ2021->PARLCON24 inference with NISRA's published administrative lookup (v3, April 2025) and rebuild the OA ladder: only constituency_code moves, at 10 Data Zones; NI sums land within 16 households of NISRA's 18 published counts (was 694). A ladder-vs-Chronicle household dispersion receipt scores every build. - Declare ons.census.households as the 34th UK local contract target (650 constituency + 361 local-authority cells) compiled from the two households record-set specs; delete the ladder append loop; A15 uprating is now per grain from the Chronicle sums (29,003,000 / 28,061,271 and / 28,061,277); A17 tenure cells take the local-authority factor. - Close the hatches: cross_grain refuses external: sides and skips the exact-signature key for a reviewed-unbound bridge; UK bridges must name a contract lower side; the pinned_in_ladder status and the 1,011 ladder-derived battery exclusions are removed; guard tests pin all of it. - Registers and docs: census disclosure fence and adjudication re-worded to per-area dispersion (approver and dates unchanged, pending ratification); receipts in experiments/887-census-households-receipts.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Vahid (CHANGES_REQUESTED at 0a8787d): - Blocker 1: the candidate manifest keys this PR renamed are read under their new, Chronicle-named shape by the release preflight, the dense-release assembler, the incumbent-surface evaluator and the microcosm-data release contract: identity.targets.chronicle, census_household_uprating with per-grain factors, ladder_assignment_provenance, household_dispersion (ladder_vs_chronicle_household_dispersion / chronicle_households); manifest and dry-run plans are schema_version 3; the contract names a missing identity instead of silently skipping the surface evaluation. Frozen ledger-named surfaces stay per the chronicle_epoch rule. - Blocker 2: rebased onto main 141bfac (previous commit); national references regenerate byte-identical to main, the local surface is unchanged; main's stale national parity receipts refresh on its own pin. - Should-fix 3: the census source rows cite the artifacts Chronicle pinned (Nomis NM_2059_1 TYPE172/TYPE424, NRS UV404 CSVs on UKDS CKAN, NISRA PARLCON24/LGD14). - Should-fix 4: the receipt states the A17 rule is unchanged while its factor moves from 1.0335759 to the LA-grain 1.0335595; all 1,011 cells before and after in docs/evidence/uk-887/census-households-cells.json. The f100 re-measure is deferred by ruling. - Should-fix 5: local_area_support_exclusions.json is schema 3 with a signed bound_despite_support_floor.census_households block; the generator reads excluded areas and binding families from the register. - Questions: ons.census.households carries the occupied-household measurement filter and the exact-signature skip is reverted, so the census cells pair only through the composition bridge, never with dwelling stock; fence and adjudication say so. - Nits: touched files formatted; households-only identities joined by area code; the NISRA lookup sha pinned in local_area_crosswalk.json. Max (source review at 0a8787d): - The households-only scope now runs through uk_local_target_surface, so it applies the per-grain A15 factor and carries the uprating receipt. - openpyxl is declared in the microcosm-build[uk] extra with a named import error in the NISRA loader; the workbook tests skip without it. CI on the rebased commit: main's #891 seam-driver test now asserts the reviewed local pin, and the workbook tests importorskip openpyxl before the loader imports it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ty (#887) Declaring openpyxl in the microcosm-build[uk] extra re-resolved uv.lock, and the primary-QRF worker identity refuses an unapproved lock digest by design (engine-us us-qs: 13 failures + 324 errors, one cause). Same one-line approval as 45833f1 for the merged #834 tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cle-887 Bind UK census households from Chronicle and map NI Data Zones with NISRA's published lookup (#887)
Combine SPM PR #894 with main after UK census PR #900. Preserve both changesets through Git's automatic merge and retain the source-enrichment publication guards. Local validation passes 734 SPM tests, 218 shared contract/H5 tests, and 64 worker-identity tests, with four existing absent-engine skips. All six wheels build and install under locked constraints; lint and formatting pass. The upstream contract and lock hashes changed. Historical producer/native qualification remains historical and requires an approved fresh execution path before publication.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build P lacks a native SPM independence-role input, so age-only classification misses independent minors. This adds a source enrichment builder that reconstructs the role from three complete, pinned Census ASEC files and appends it to the existing native person table. It verifies every source unit and preserves existing IDs, memberships, ages, weights, variables and HDF metadata.
The release contract preserves the parent’s original schema-5 calibration evidence and requires exhaustive HDF comparison, clean producer source, and measured country/Core/wrapper/calculator wheel compatibility before publication. Ordinary releases retain their schema-6 gate and reject enrichment-only arguments. Publication and CLI preflight share the same guards. A builder-to-certification-to-publication regression exercises the emitted manifest through the real preparation and publisher functions with the existing FakeHub.
Validation on
f5c5367667453b609fd9cc2738a1512a007aa413, incorporating main6f7571e1ab8c516154289bd8ebf75cb2646f03b5: 1,016 targeted tests pass (734 SPM, 218 shared contract/H5, 64 worker identity), with four existing skips because the test environment does not install PolicyEngine-US. Partition, formatting, lint and dependency checks pass. Six shard wheels build and pass clean constrained import checks. Independent integration review found no merge defects. Hosted CI for this head is pending.The integration preserves both changesets. Twenty of the 21 SPM paths remain byte-identical; the shared contract also includes main’s UK checks. That whole-file producer hash and the dependency lock have changed. The historical producer manifest therefore fails the existing source guard from this tree: a fresh reviewed producer and native qualification are required before publication. Historical certificates and numerical receipts retain their original identities. These local tests do not certify population results or a dataset release.
This PR changes source only. No H5, dataset tag, new calibration or package is published by merging it. Final source review and CI remain required; the coordinated release separately gates old-consumer protection, native/scientific qualification and the single immutable dataset publication. Related launch consolidation: #893.