Consolidate the US survey graph build and enrichment pipeline

[MaxGhenis]

The US builder needs a reproducible path from original ACS and ASEC sources to one complete population, imputed inputs, calibration and verified analysis files. This draft consolidates that implementation with shared graph, atomic-geography, artifact/replay and calibration changes. It is an integration branch; its full completion is not a prerequisite for independent UK work.

The intended order, corrected on 10 September, is ACS + ASEC → complete initial multispine including support/PUF clones → assign one Census block per resulting household → derive larger geographies → remaining imputation and derivation → national/CD calibration → full and pruned exports. Distinct initial clones can receive different constrained draws, keyed by stable source identity plus a clone discriminator. Their locations then remain fixed. The corrected nine-node prefix now clones before block assignment, with nineteen-node financial and twelve-node age extensions. Its 70 inventoried controls plus a separate default-path compatibility check pass on invented inputs; the small native post-clone pilot did not complete final verification within its reviewed resource budget and remains unaccepted. Earlier native acceptance remains explicitly scoped to the prior order.

The PUF operator acts on the combined survey population. Social Security values and unknownness remain source-owned; the two PUF55 routes use nine predictors when the selected total is known and eight when it is unknown. Prior wages are excluded from the launch input profile. National and local exports must preserve the same constructed record values and relationships; calibration weights, scope and sparsity may vary.

Parallel review and dependencies

Reviewer	Useful review now	Work that should continue
Anthony	Shared graph/Frame and calibration compatibility; target hierarchy/diagnostics and dashboard metadata, including #855	Merged #900 clears target/dashboard changes; UK staging #896 can proceed independently
Maria	Register every existing UK pipeline stage as a graph node using this branch's machinery; verify UK behavior and coverage	With #900 merged, UK graph-node registration deliberately stacks on #893, followed by stage reordering in a separate PR; source/imputation and candidate validation continue
US integration owner	Resolve overlap, demonstrate compatibility, extract agreed shared prerequisites and complete the US build/release checks	US source preparation, imputation, quality diagnostics and progressive local builds

Coordination corrected after the 10 September team meeting: Maria intentionally stacks her UK graph-node PR on #893; pin its parent revision and coordinate shared-interface changes. Her #900 source/target reconciliation merged on 10 September at 14:54 UTC, clearing that dependency for Anthony's #855 hierarchy/diagnostics and dashboard work. Anthony's staging #896 proceeds independently. Anthony considers #885 superseded; it is reference material, not another runtime implementation to retain or merge. This does not claim the PR is closed or that he has approved #893.

The US integration owner will support Maria's graph with the shared atomic-area adapter, not build a competing UK graph. Keep UK graph-node registration and stage reordering as separate PRs. Record shared dependencies as specific changes and pinned revisions, extracting smaller prerequisites where useful while preserving Maria's intentional stack. Full native US PUF completion does not gate UK source/target repairs, staging or candidate validation. Full graph coverage in both countries remains a requirement for the broader advertised launch.

Verification and remaining work

This branch incorporates main through 6f7571e1ab8c516154289bd8ebf75cb2646f03b5, including Maria's merged #900 target/lookup reconciliation, UK CGT observation-period work #889 and PR CI scheduling #899. The integration preserves both parent histories and introduces exactly the upstream changed paths; dependency metadata parses and the updated worker lock digest matches. No dependency installation or new runtime acceptance is implied. The shared solver retains UK ordinary/informed behavior and the US grouped-bound path; unsupported combinations fail explicitly.

• The small native survey/block/financial pilot passed cold execution and required replay: 1,584 source households / 3,464 people became 3,168 / 6,928 after cloning. All 20 nodes replayed; four non-manifest exports matched cold bytes and the portable manifest retained its content. This evidence uses the earlier pre-clone geography order.
• Corrected post-clone geography preserves stable origin/clone draw identity, source weights, missingness and retained populations, with a typed geography validation dependency before financial enrichment. The 70 scoped controls are bound to the adopted source through exact selected-function/fixture AST evidence; maintained-module-path CI remains separate. Fable reviewed the production/test change, and root verified the applied hashes. Ten corrected artifact-reader controls also pass. The native nineteen-node runner returned and export readback ran, but final validation did not complete; the last sample recorded 7,216 CPU seconds against a 7,200-second cap and about 11.5 GB peak RSS. There is no final acceptance receipt. Repeated source/catalogue qualification is being optimized under separate controls; intermediate exports are not release candidates.
• All 28 focused PUF cross-store replay controls pass with independent source/resource verification. The comparison now uses directional Frame/ledger replay for independently decoded stores while retaining physical mutation fences for live objects. Two positive controls of the actual 245-node host now pass on a separately frozen source snapshot, using 64 invented donors, both Social Security routes, 110 fits and 110 applications, and complete cold/required runs. Its 18 adversarial controls are running separately; maintained adoption, native donor fitting and release remain unverified.
• The two-route numerical layer passed 46 invented controls with real QRF fitting/applications and whole-cohort-finalizer comparisons. Its receipt binds the finalized candidate and rejects mutation during encoding. All 47 canonical donor CREATE/converter controls passed, including cold execution and required replay. Independent source/resource checks passed; both source changes are included here. These controls do not qualify native donor/recipient fitting or the complete PUF attachment.
• Public recipient checks cover 28 distinct controls across the original 27 passing cases and a corrected five-case run with four overlaps, not one 28-case invocation. Earlier failed runs remain preserved. Donor-helper and full65/PUF55 cold/required chains have separate frozen-snapshot evidence.
• All 149 selected invented calibration controls passed on the combined US/UK solver snapshot, with independent source and unique-result checks. This verifies solver behavior, not native national/CD calibration.
• Changed-file Ruff, workflow actionlint and tracked test inventory checks passed on their identified revisions. Independent review scopes and corrections are recorded in the review guide. Current-head CI remains separate.

The CI diagnostic preserves missing event merge metadata as empty while requiring checkout/event equality and head/base identities. Forty-one environment-only controls were added; the bootstrap test step passed on a preceding commit, while spec-identity derivation still encounters restricted dependency probes. No source-access or bootstrap permissions were broadened.

Remaining release requirements include native execution/replay of the corrected geography order, current-head CI/spec identities, complete native PUF and other model-input coverage, a small full candidate, model/holdout quality, national/CD calibration, progressive scaling, full/pruned readback and a dashboard tied to the exact release file. No dataset release, default consumer promotion or deployment is claimed.

See the review and evidence index, shared US/UK release-path reference, historical native replay record, and companion site/methods draft #72. Typed-model #873 still needs reconciliation; #885 is retained as historical reference following Anthony's supersession decision.