Publish calibration hierarchy schema 8 - #855
Conversation
476e097 to
790766f
Compare
790766f to
7ca4a5d
Compare
Repair UC capital graph inputs and document calibration support gaps
…tialization (#355) Attesting microcosm.calibrate.gates and microcosm.calibrate.initialization in seeds.py added two resolved-binding fields (9,770 -> 9,772; ledger 42,154 -> 42,156; compiler_semantic 27,715 -> 27,717; no_generation0_effect 3,678 -> 3,680). field_usage.py and the coverage tool test were re-pinned in c2ee134; this sibling test only runs in the rest, engine-shared and wheels lanes and was missed. Values recomputed from the ledger, not transcribed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review: request changesReviewed head: ab66edc C1 — Must fix: stack this work on #900 and preserve its Chronicle household contractPR #855 independently migrates the census-household target away from runtime ladder injection, but retains the synthetic contract ID external:census_households/households. Its generic local-authority support-floor path then defers E06000053 and E09000001. The generated artifacts show the resulting difference:
I recommend rebasing/updating #900 onto current main, then stacking #855 on #900. During conflict resolution, preserve #900's Chronicle contract and feed pin, NISRA mapping, 1,011-cell set, refusal of external target IDs, and diagnostic-only OA ladder. Then reapply the schema-8 hierarchy work and regenerate the reference, membership, parity, and golden artifacts. In particular, replace the #855 test that approves the two household deferrals with an invariant asserting that all 1,011 cells and both named IDs remain present. Relevant code: generic support-floor handling. The two branches currently have 19 conflicted paths, so this needs deliberate conflict resolution rather than a blind rebase. A1 — Should address: fail closed for required source-owned labelsThe schema-8 hierarchy direction is good, but the implementation humanizes stable identifiers when Chronicle labels are missing. The same fallback also masks conflicting labels: when selected facts do not produce exactly one unique nonblank label, the code substitutes identifier-derived display text. That can make a broken upstream contract look valid. I suggest:
Relevant code: hierarchy label fallback. GeographyA narrow display-name fallback is reasonable only after an explicit valid geography identity has been supplied. For example, country/K02000001 may resolve to “United Kingdom” through a shared authoritative catalog. I would not interpret missing geography as UK. “National” can mean UK, Great Britain, England and Wales, England, or another coverage universe. If a source adapter guarantees UK coverage, encode country/K02000001 explicitly at that boundary; unknown or absent geography identities should fail schema-8 validation. Validation
Overall: the typed hierarchy model and all-or-none schema-8 validation are directionally strong, but I would not merge until #855 is stacked on #900 and the label fallback contract is tightened. |
UK dataset sizes: informed L0 and exact-count candidates (#355)
Cancel outdated PR CI runs while preserving main pushes
Align UK CGT calibration with FY2024-25 individual observations
…ISRA's published lookup (#887) - Re-pin the UK Chronicle consumer feed 6fb700e -> ec7169b (131,450 facts, facts sha 4a50ee95); no national or local reference changed status or value. - Northern Ireland: replace the active-postcode modal DZ2021->PARLCON24 inference with NISRA's published administrative lookup (v3, April 2025) and rebuild the OA ladder: only constituency_code moves, at 10 Data Zones; NI sums land within 16 households of NISRA's 18 published counts (was 694). A ladder-vs-Chronicle household dispersion receipt scores every build. - Declare ons.census.households as the 34th UK local contract target (650 constituency + 361 local-authority cells) compiled from the two households record-set specs; delete the ladder append loop; A15 uprating is now per grain from the Chronicle sums (29,003,000 / 28,061,271 and / 28,061,277); A17 tenure cells take the local-authority factor. - Close the hatches: cross_grain refuses external: sides and skips the exact-signature key for a reviewed-unbound bridge; UK bridges must name a contract lower side; the pinned_in_ladder status and the 1,011 ladder-derived battery exclusions are removed; guard tests pin all of it. - Registers and docs: census disclosure fence and adjudication re-worded to per-area dispersion (approver and dates unchanged, pending ratification); receipts in experiments/887-census-households-receipts.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Vahid (CHANGES_REQUESTED at 0a8787d): - Blocker 1: the candidate manifest keys this PR renamed are read under their new, Chronicle-named shape by the release preflight, the dense-release assembler, the incumbent-surface evaluator and the microcosm-data release contract: identity.targets.chronicle, census_household_uprating with per-grain factors, ladder_assignment_provenance, household_dispersion (ladder_vs_chronicle_household_dispersion / chronicle_households); manifest and dry-run plans are schema_version 3; the contract names a missing identity instead of silently skipping the surface evaluation. Frozen ledger-named surfaces stay per the chronicle_epoch rule. - Blocker 2: rebased onto main 141bfac (previous commit); national references regenerate byte-identical to main, the local surface is unchanged; main's stale national parity receipts refresh on its own pin. - Should-fix 3: the census source rows cite the artifacts Chronicle pinned (Nomis NM_2059_1 TYPE172/TYPE424, NRS UV404 CSVs on UKDS CKAN, NISRA PARLCON24/LGD14). - Should-fix 4: the receipt states the A17 rule is unchanged while its factor moves from 1.0335759 to the LA-grain 1.0335595; all 1,011 cells before and after in docs/evidence/uk-887/census-households-cells.json. The f100 re-measure is deferred by ruling. - Should-fix 5: local_area_support_exclusions.json is schema 3 with a signed bound_despite_support_floor.census_households block; the generator reads excluded areas and binding families from the register. - Questions: ons.census.households carries the occupied-household measurement filter and the exact-signature skip is reverted, so the census cells pair only through the composition bridge, never with dwelling stock; fence and adjudication say so. - Nits: touched files formatted; households-only identities joined by area code; the NISRA lookup sha pinned in local_area_crosswalk.json. Max (source review at 0a8787d): - The households-only scope now runs through uk_local_target_surface, so it applies the per-grain A15 factor and carries the uprating receipt. - openpyxl is declared in the microcosm-build[uk] extra with a named import error in the NISRA loader; the workbook tests skip without it. CI on the rebased commit: main's #891 seam-driver test now asserts the reviewed local pin, and the workbook tests importorskip openpyxl before the loader imports it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ty (#887) Declaring openpyxl in the microcosm-build[uk] extra re-resolved uv.lock, and the primary-QRF worker identity refuses an unapproved lock digest by design (engine-us us-qs: 13 failures + 324 errors, one cause). Same one-line approval as 45833f1 for the merged #834 tree. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cle-887 Bind UK census households from Chronicle and map NI Data Zones with NISRA's published lookup (#887)
Main's diagnostics schema 8 (#855) requires a complete hierarchy for every registry-backed target, so the `national.demographic_calibration` node failed on the merge ref: its S0101 registry carried none. Each activated cell now declares the only hierarchy it may have — provider `census_acs`, category `census_acs.population_by_age` (S0101) or `census_acs.population_by_sex_and_age` (B01001), both labels added to the reviewed US variable-label catalog, the activation's own national publisher row (0100000US, "United States", level country), one `publisher_cell` dimension whose value is the cell and whose label is the published one the band binds, and the cell itself as the target so its id equals the spec name. Both S0101 spec builders attach it; the registry declaration round- trips it; the registry validator refuses a spec without exactly the hierarchy rebuilt from its table, name, label and geography; the survey diagnostics verifier's defensive pin moves from schema 6 to 8. Controls: the invented seven-node graph's diagnostics are schema 8 with a hierarchy on every row; the registry JSON round-trips every hierarchy; missing, foreign-cell, foreign-geography and foreign-category hierarchies are refused; the label catalog pin lists the two new rows. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#904) Main merged the calibration target hierarchy (schema 8, #855) after round 2. The eight #890 targets now declare a catalog category and a label (four new providers: ORR, Welsh Government, DfI Northern Ireland, NITHC; six new categories; the cars fuel duty under obr.efo_receipts), and the committed references carry the contract-derived hierarchy lookups the generator emits for the 196 contract targets with an active reference. What the rebase exposes and this commit does not fix: main's hierarchy completion requires dimension labels on every selected Chronicle fact (chronicle#261), which no artifact carries, so on main and here alike the generators and compile_uk_target_registry return nothing against the pinned feed (0 compiled / 424 unsupported). The reference surfaces are the round-2 regenerations plus those lookups; the feed-backed regeneration and compile tests skip in CI and fail locally until #261 or the interim fallback that issue announces. Vendored resources still regenerate byte-identical. The nine English regions join Microcosm's UK geography label catalog: the London fact carries no geography name and the catalog had only country codes, so the region leg would be refused at compile once labels arrive; a test checks the national region roster is covered. Conflicts: measure-exclusion counts to the merged truth (47 entries, tranche 35); the UK spec digest re-pinned; the target-fit register and its tests are main's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Summary
TargetSpec, registry serialization, runtime targets, and diagnosticsmainValidation
uv run ruff check .Fixes #854