Declare survey- and calibration-year rules for UK FRS parameter reads (#862)

[juaristi22]

Closes #862. Microcosm's half of uk-data#475, uk-data#476 and uk-data#478, which uk-data PR #480 fixed on the incumbent side on 2026-09-04.

What changes

• Resolver. frs_release.resolve_uk_year_rule maps a declared year_rule (survey_year | calibration_year) to the release year from frs_release.json; any other string fails closed. year_rule was declaration-only before (a string equality check on the student-loan operations).
• frs_disability. Categories and flags both read the fiscal-converted gov.dwp tree at the survey year through one cached tree per year. Categories previously read the pre-conversion baseline clone, which is the previous fiscal year's table (AA higher 101.75 vs 108.55 at 2024). Both derive operations declare year_rule: survey_year and the stage binds them closed-world.
• spi_income. The post-redraw disability refresh now calls derive_frs_disability with the same rates instead of carrying a second copy of the derivation (which divided categories by 52 rather than 365.25/7). build_period is dropped from impute_uk_spi_income_support.
• frs_education_grants. Grant capacities materialise at the survey year; the three DSA eligibility booleans and the DfE maximum at the calibration year, so the DSA seed is no longer a zero vector. The frame stays stamped at the survey year. The stage declares three operations (grant predictors at survey_year, DSA predictors at calibration_year, the derive at calibration_year).
• Schema and manifests. year_rule is a closed enum on derive, materialize_rules_engine_predictors and the two student-loan operation kinds; sources.yaml and source_stages.json updated in lockstep.
• Register. disabled_students_allowance_eligible_expenses is signed as a candidate-only column (dsa-eligible-expenses-seeded-at-calibration-year, class defect_fix) against the parity reference, which was exported before uk-data#480 and lists the column as zero-share. The reference itself is untouched.
• Locks regenerated. Coverage-manifest source-manifest sha, the uk spec digest pin, the graph parity fixture.

Stage-level receipt (licensed FRS 2024-25 tabs, experiments/862-policy-year-rule-receipts.md)

The seven FRS stages through the grant split, run on the pinned SN 9563 tabs on main and on this branch (34,966 persons, 68.25m weighted, policyengine-uk 2.92.1):

measure	main	this branch
disability category bands (5 columns, all bands)	see receipt	identical, row for row
is_disabled / is_enhanced_disabled / is_severely_disabled weighted	6,184,013 / 3,013,472 / 3,588,886	identical
DSA column records > 0	0	1
DSA weighted people / GBP	0 / 0	4,654 / 83,772,000 (row value 18,000, the DfE maximum)
education_grants residual GBP	2,816,276,654	2,732,504,654

This matches uk-data#480's receipt on the same tabs (categories and flags identical; DSA 0 → 1 record, 4.7k weighted, GBP 83.8m).

Review round 1 (Vahid, at 19172e8) — addressed in c651c20

• Disability year resolution is now discriminated engine-free: a test stamps a frame 2023 and asserts both readers still receive 2024 from the resolver (test_disability_stage_resolves_the_survey_year_not_the_frame_period).
• Formatting: the three test files this PR had drifted are formatted; the two runtime files ruff format --check still flags (frs_disability.py, frs_education_grants.py) were already drifted on main and are left alone to keep the diff to the change.
• Criterion 2 of UK FRS stages evaluate policy parameters at the survey period: declare a policy-year rule (uk-data#475, uk-data#478) #862 is met by inference, not by a calibrated build: the receipt runs the seven FRS stages through the grant split and shows no category or flag row moves, so the two two-child-limit disability cells (UK spine: child disability under-identification blocks the TCL disability cells (−56.5% / −40.0% across three spine vintages) #808) cannot move. A full licensed spine + calibration run stays available if María wants the cells signed on a build.
• Relabel versus uprating is measured, not asserted: on the licensed frame the three DSA eligibility booleans are identical on every one of the 34,966 persons under the adapter's fiscal_year=2025 relabel and under uk-data's path (2024-stamped dataset, engine uprates, calculate at 2025), with 11 jointly eligible people either way; the receipt has the table and the docstring cites it.
• The SPI refresh inherits frs_disability.YEAR_RULE by design: it re-derives the FRS stage's eight columns for redrawn rows, so it must follow that stage's declaration; a second declaration on hmrc_spi_income_spine could only drift from it (and would also break the frozen June manifest lockstep).
• Closed-world is per-stage, and the generic materializer now honours a declared year_rule (source_runtime.py), refusing an unknown one, so a future declaration is never silently ignored. Making the rule the default rather than the exception for the eight stages that still materialise at the frame period is a follow-up issue.
• Register entry: self-adjudicated on the drafting date; it stands only with María's sign-off (unchanged posture).
• Nits: student_loans resolves its fallback year through the resolver; the grant stage's pre-2025 branch honours an injected policy again; the receipt names the commit it was repeated at and explains the education_grants residual level difference against uk-data#480 (upstream split difference; the delta this change owns matches).

Verification

ruff check . clean (ruff format clean on every file this PR touches except the two pre-existing runtime drifts noted above); the listed suite 425 passed, 1 skipped at c651c20; tools/ci_test_groups.py --verify ok. The package-wide -m requires_uk lane has two failures that reproduce on main unchanged: test_cached_reference_regeneration_matches_committed_surface and test_cached_candidate_regeneration_matches_committed_evidence, both local-cache checks of licensed artifacts regenerated at engine 2.92.1 against surfaces committed at 2.89.0.

Not in this PR

• Re-measuring the two two-child-limit disability cells (UK spine: child disability under-identification blocks the TCL disability cells (−56.5% / −40.0% across three spine vintages) #808 lineage) needs a full licensed spine + calibration run. The receipt shows no category or flag row moves, so those cells cannot move; a full run is available as a follow-up if wanted signed on a build.
• Re-minting efrs_parity_reference.json needs a post-uk-data#480 incumbent export; until then the DSA column is populated here and zero-share there, which the register entry signs.
• uk-data#481 (the GBP 1 gap against the April uprating step) is untouched.
• Draft Correct SPI income inputs and UK calibration measurements #879 edits spi_income.py, the manifests, the schema and the parity fixtures in other hunks; whichever lands second rebases (the coverage-manifest sha, the spec digest and the schema lines will conflict trivially).

The adjudicated_on date on the register entry is the drafting date; it stands only with María's sign-off on this PR.

🤖 Generated with Claude Code

[vahid-ahmadi]

Review pass at 19172e8f (Claude Code, high effort; worktree checkout, diff read against #862's acceptance criteria and against uk-data #480, which I reviewed on the incumbent side; without the engine: 316 passed, 6 skipped across the ten build suites in scope, 229 passed in the data contract tests; ruff check clean; both census tools, the coverage manifest, spec_engine_coverage and ci_test_groups --verify current).

The port is faithful

resolve_uk_year_rule (frs_release.py:76-89) maps a declared survey_year or calibration_year to the release and rejects any other string; both stages resolve through it rather than the frame period (frs_disability.py:100, frs_education_grants.py:76-77), and the closed-world operation asserts on both stages refuse a missing, extra or unknown year_rule, with tests. No .baseline read remains in uk_runtime. Categories and flags share one cached parameters(year).gov.dwp tree, the £1/week tolerance is preserved in survey weeks on both paths, and the SPI refresh now calls derive_frs_disability instead of carrying its own copy — that is the better fix for uk-data #476's real content, since it removes the fork that produced the 0.34% disagreement rather than aligning a constant. The grant stage materialises grant capacities at the survey year and the three DSA booleans plus the DfE maximum at the calibration year while the frame stays stamped 2024, mirroring #480's year / policy_year split; the DSA column is signed as a candidate-only defect_fix with its mass recorded (1 record, 4,654 weighted, £83.77m), which is #862's acceptance criterion 3, and the deferrals (the TCL re-measure, the parity reference re-mint) are stated rather than hidden. Manifests, schema enum, spec sha, coverage manifest and graph parity fixture all moved in lockstep.

Findings

1. Should-fix — no engine-free test discriminates the disability stage's year resolution. The grant stage has one (test_transform_materializes_survey_and_calibration_years pins [(grants, 2024), (DSA, 2025)] against a stub engine). For frs_disability, the only value pin is the requires_uk reader test (108.55 versus 101.75), and the stub readers in test_uk_frs_spine.py:1062 ignore their period argument. Because survey year and frame period are both 2024, asserting 2024 would not discriminate; a stub frame stamped 2023 asserting the readers still receive 2024 would prove the resolver, not the frame, chose the year.

2. Should-fix — ruff format --check fails on three test files this PR drifted (test_uk_frs_education_grants.py:82, 140, test_uk_signed_differences.py, test_uk_source_stages.py), plus two runtime files already drifted on main. CI runs only ruff check, so not a gate, but "ruff clean" in the body is the weaker claim.

Questions

• Criterion 2 is met by inference plus one signing. The receipt runs seven FRS stages through the grant split; the two TCL disability cells (#808) are inferred unchanged because no category or flag row moves, and that inference is sound, but the criterion says "measured on a licensed build". Fine if María accepts the inference; say so.
• The register entry is self-adjudicated (adjudicator: juaristi22, adjudicated_on: 2026-09-07); the body defers to María's sign-off, which is the right posture, and the matching semantics are correct — the DSA column sits in the reference only under zero_share_input_columns_excluded, which nothing gates a candidate against.
• The mechanism differs from #480 and that lives in a docstring. uk-data calls sim.calculate(var, policy_year); microcosm materialises the DSA variables at 2025 on the 2024 frame, and the adapter's fiscal_year=int(period) is a relabel with no uprating. The class docstring argues the three booleans are invariant to uprating; that is asserted, not tested, and belongs in the body. Same one-record outcome on the licensed receipt, so consistent in result.
• The SPI refresh inherits frs_disability.YEAR_RULE rather than declaring its own; fine in this build order, but the declaration is inherited rather than stated on hmrc_spi_income_spine.
• Closed-world is per-stage, not schema-wide. year_rule stays optional on derive and materialize_rules_engine_predictors, eight other stages still materialise at uk_time_period(frame) with no rule, and the generic materializer in source_runtime.py:132-165 ignores year_rule entirely — a future op declaring calibration_year there would be silently ignored. Within #862's two-stage scope this is acceptable; worth a follow-up issue so the rule becomes the default rather than the exception.

Nits

student_loans.py:121-123 still reads calibration_year directly rather than through the resolver; the grant stage's pre-2025 branch now discards an injected policy (harmless, capacity is zero); the receipt cites 7119f220 while the head is two commits later (neither touches the seven receipt stages, so the figures stand — say so); the receipt's "matches uk-data#480" holds for the DSA delta but the education_grants residual level differs (pre-existing).

Needs the licensed engine

The 108.55 / 101.75 reader pin, every receipt figure, the relabel-invariance claim, and the two pre-existing cache failures the body reports on main.

---

Approve once 1 lands; 2 is a formatter run. Criteria 1 and 3 of #862 are met outright and criterion 2 by sound inference. Genuinely good: de-duplicating the SPI refresh instead of aligning a constant, the parametrised drift tests, and the survey-year frame stamp preserved throughout.

[juaristi22]

Thanks for the round, Vahid. Rebased on main e07a4735 (after #874, #879 and #870), now at cbadfa81; the round-1 changes are in 8ff697cb.

Should-fixes

1. Engine-free discrimination of the disability stage's year: test_disability_stage_resolves_the_survey_year_not_the_frame_period stamps a frame 2023 and asserts both readers still receive 2024, so the resolver, not the frame, picks the year.
2. ruff format on the three test files this PR had drifted. The two runtime files it still flags (frs_disability.py, frs_education_grants.py) were already drifted on main, so they stay out of this diff.

Questions

• Relabel versus uprating is now measured rather than asserted: on the licensed frame the three DSA eligibility booleans are identical on every one of the 34,966 persons under the adapter's fiscal_year=2025 relabel and under uk-data's uprated path, 11 jointly eligible either way. The receipt has the table; the docstring cites it.
• The generic materializer (source_runtime.py) now honours a declared year_rule over the runtime period and refuses an unknown one, with two tests, so a future declaration cannot be silently ignored. Making the rule the default for the eight stages that still materialise at the frame period is drafted as a follow-up issue for María.
• Criterion 2 by inference, the register entry's self-adjudication (stands only with María's sign-off) and the SPI refresh inheriting frs_disability.YEAR_RULE by design are stated in the body.

Nits
student_loans resolves its fallback year through the resolver; the grant stage's pre-2025 branch honours an injected policy again; the receipt names the commit it was repeated at and explains the education_grants residual level against uk-data#480 (upstream split difference; the delta this change owns matches).

One merge note from the rebase
#879 re-uses build_period on impute_uk_spi_income_support to rebase SPI incomes and adds a cached uprating helper; this branch had removed that parameter and the functools.cache import on the premise that the disability refresh was their only consumer, and git merged both sides without a textual conflict into an unimportable module. Both are restored in cbadfa81; the disability refresh itself still takes its year from the declared rule. Listed suite 441 passed, 1 skipped on the rebased tree.

🤖 Generated with Claude Code

[vahid-ahmadi]

Second pass at cbadfa81 (Claude Code, high effort; worktree checkout, the round-one commit and the merge repair read against the dispositions; the ten suites from round one plus the materializer tests and the data contract: 554 passed, 15 skipped, 0 failed; ruff check clean, ruff format clean on the changed test files with the two runtime files still drifted exactly as on main; both census tools, the coverage manifest and ci_test_groups --verify current; the diff against main is 26 files, all within the dispositions).

1. Verified. test_disability_stage_resolves_the_survey_year_not_the_frame_period stamps the frame 2023, drives UKFRSDisabilityStageTransform.__call__ with recording readers, and asserts both receive 2024 with the frame period untouched. It passes on the old head too, which is expected — the old head already resolved through the rule — so it is a regression guard rather than a discriminator, and that is what I asked for.

2. Verified. The three test files are formatted; frs_disability.py and frs_education_grants.py are flagged on main identically.

3. Verified, and this one discriminates. source_runtime.py:148-159 resolves a declared year_rule through the resolver, re-raises an unknown rule as SourceRuntimeError, and leaves the rule-less path on the runtime period unchanged. The two new tests fail on the old head ([2023] != [2025]; DID NOT RAISE).

4. Verified. The receipt's relabel-versus-uprating table shows the three DSA booleans identical on all 34,966 persons under relabel-at-2025, uprated-to-2025 and at-2024, with 11 jointly eligible; the docstring cites it; the receipt names 8ff697cb as the commit it was repeated at and explains the education_grants residual level against uk-data #480 (upstream split difference, the £83.77m delta matching). cbadfa81 touches none of the seven measured stages, so the figures stand.

5. Verified. student_loans falls back through the resolver; the grant stage's pre-2025 branch honours an injected policy.

6. Verified in code; one test gap. spi_income.py imports cleanly at this head, build_period is back on the signature and used for #879's rebasing, the functools.cache helper is restored, and main at e07a4735 was never broken — the unimportable state existed only on the merged branch, and the repair is exactly the two restored lines. The disability refresh does take its year from frs_disability.YEAR_RULE (:1283-1288). But no test pins that: the only test reaching _refresh_disability_derived_inputs injects both rate tables and so bypasses the resolver branch, and neither SPI test file references the rule. A ten-line test mirroring item 1 — call the refresh with no rates on a 2023-stamped frame and assert the readers receive 2024 — would close the last untested year source in the PR.

7. Verified. The body states criterion 2 by inference, the SPI refresh inheriting the rule by design, and the register entry standing only with María's sign-off.

---

Approving. The materializer change is the substantive addition this round and it is the right generalisation; item 6's test is the one thing I would still add.