Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/uc-calibration-best-iterate.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Preserve the best feasible iterate within the requested budget for unregularized, free-mass Adam calibration, including a better warm start, and record the selected update in diagnostics. Free-mass national and rowwise-local calibration weights can change when an earlier iterate fits better; this does not establish national fit stability. Conserved-mass, L2-regularized and L0-gated paths retain their closing-state behavior, checked against exact pre-change output bytes.
1 change: 1 addition & 0 deletions changelog.d/uk-uc-claim-target-contract.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Measure UK two-child-limit targets over each benefit unit's own qualifying children and claimant roles, separating statistical affected-child proxies from element denial. Correct Scottish UC claims with a child under one to benefit-unit grain and record remaining annual-period, birth-year and positive-award approximations in measurement receipts.
3 changes: 3 additions & 0 deletions changelog.d/uk-uc-couple-donor-predictors.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Use retained FRS claimant and partner roles to classify couples in UC capital donor cells and reported-UC predictors, including cohabiting couples and excluding older dependent children. Preserve legal marital status, emit an explicit is_uc_claimant input from the raw FRS spine, and share the claimant-role derivation with UC calibration measurement.

Preserve the Boolean claimant flag through graph storage, refresh the synthetic UK spine fixture, and bind the shared relationship helper into the three consuming stages' implementation hashes.
1 change: 1 addition & 0 deletions changelog.d/uk-uc-development-replays.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add private, pinned UC relationship and target-matrix diagnostic replays with source and overlay hashes, fixed-weight comparisons, and baseline reproduction checks.
1 change: 1 addition & 0 deletions changelog.d/uk-uc-released-claimant-input.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Require PolicyEngine-UK 2.97.0 for the UK extras, allow recorded UC claimant roles through export, and refuse model loading that drops or changes those roles. Preserve historical candidate evidence at its recorded model version.
1 change: 1 addition & 0 deletions changelog.d/uk-uc-source-month-coverage.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Declare the April–December 2025 source window for national UC monthly targets and reject missing, extra or repeated months during UK target compilation. Record the actual observation months and averaging denominator without changing target values or treating the window as a full calendar year.
4 changes: 4 additions & 0 deletions changelog.d/uk-uc-target-geography-and-finite-tail.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Align national UC calibration measures with their Great Britain source geography,
projecting household geography through benefit-unit links without changing family
composition. Keep the published £2,400.01–£2,500 monthly payment band finite instead
of counting the separately published, unbound higher-payment category in that row.
16 changes: 8 additions & 8 deletions docs/evidence/spec-engine/us-f0-coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -1656,13 +1656,13 @@
"compiler_ir.node_slices"
],
"expected": {
"map_sha256": "6b4902d9a640dd459942e588ae2cc7fb937f1c42ad814365c7c1616c98ce1b68",
"protocol_sha256": "e63bbfa0f05302a672acd9914e3652c151cc7ee690182d8823dfb85329b1e911"
"map_sha256": "c94a5af8eb24866156b8cd4b77b93dc497b7df5b025204fe1022f80b1d8446ea",
"protocol_sha256": "1c53f1d9b3e185a41181fbe861f9b08d56c83880710fd1e18d7a4ceb5d6a354e"
},
"failures": [],
"observed": {
"map_sha256": "6b4902d9a640dd459942e588ae2cc7fb937f1c42ad814365c7c1616c98ce1b68",
"protocol_sha256": "e63bbfa0f05302a672acd9914e3652c151cc7ee690182d8823dfb85329b1e911"
"map_sha256": "c94a5af8eb24866156b8cd4b77b93dc497b7df5b025204fe1022f80b1d8446ea",
"protocol_sha256": "1c53f1d9b3e185a41181fbe861f9b08d56c83880710fd1e18d7a4ceb5d6a354e"
},
"status": "covered"
},
Expand All @@ -1677,7 +1677,7 @@
"compiler_ir.seed_stream_map"
],
"expected": {
"implementation_sha256": "e63bbfa0f05302a672acd9914e3652c151cc7ee690182d8823dfb85329b1e911",
"implementation_sha256": "1c53f1d9b3e185a41181fbe861f9b08d56c83880710fd1e18d7a4ceb5d6a354e",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand All @@ -1698,7 +1698,7 @@
},
"failures": [],
"observed": {
"implementation_sha256": "e63bbfa0f05302a672acd9914e3652c151cc7ee690182d8823dfb85329b1e911",
"implementation_sha256": "1c53f1d9b3e185a41181fbe861f9b08d56c83880710fd1e18d7a4ceb5d6a354e",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand Down Expand Up @@ -2599,7 +2599,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "9db29b4d33424fbb21a83c63927c7de55ba9a333d631f6323935f67a496eee46"
"spec_sha256": "8b1546bc97b540afd525fe88f36a315de18b4d57bd9fab73035dacd8962e302e"
}
},
"report_schema_version": 3,
Expand All @@ -2609,7 +2609,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "9db29b4d33424fbb21a83c63927c7de55ba9a333d631f6323935f67a496eee46"
"spec_sha256": "8b1546bc97b540afd525fe88f36a315de18b4d57bd9fab73035dacd8962e302e"
},
"status": "pass"
}
2 changes: 1 addition & 1 deletion packages/microcosm-build/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ us = [
# The UK extra adds the rules engine for local metric generation from a
# Microcosm UK H5. Target tables remain explicit inputs, and the base package
# still does not import policyengine-uk at import time.
uk = ["policyengine-uk>=2.93", "h5py>=3", "tables>=3"]
uk = ["policyengine-uk>=2.97.0", "h5py>=3", "tables>=3"]

[project.scripts]
microcosm-export-us-l0-refit-h5 = "microcosm.build.us_runtime.l0_refit_export:main"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -359,8 +359,8 @@
"late_schedule": "e59c019d3d454eac99ac0ac209b6c5b6faaf9bdfcaeee18c36a25be19bf7da2f",
"ownership": "5f64f0aac49e2313177564f71876bffc8c81b3ded4df701e70930e60e9c98356",
"primary_tuples": "987b501c695e31f45521c4a178528f75ab3df22c09bc407b182213b2de99ee57",
"seed_map": "6b4902d9a640dd459942e588ae2cc7fb937f1c42ad814365c7c1616c98ce1b68",
"seed_protocol": "e63bbfa0f05302a672acd9914e3652c151cc7ee690182d8823dfb85329b1e911",
"seed_map": "c94a5af8eb24866156b8cd4b77b93dc497b7df5b025204fe1022f80b1d8446ea",
"seed_protocol": "1c53f1d9b3e185a41181fbe861f9b08d56c83880710fd1e18d7a4ceb5d6a354e",
"source_manifest": "cd5ba8924d64da5425ee14cca82a774e3f4b2bb5aabe06df291cc3cc457287a9",
"take_up": "fa186daea0f8dd641cc470e41d1a2953f887d45282ec990201298f47bedf8d4d",
"tail": "ac92829c88a1a4fb6460d61190918d5d99c6c377fc8dd8f62f02b332d09bf59c",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,9 @@
# sibling's lower edge within the same compiled contract target. Edges must
# never derive from an exclusion-pruned roster, or excluding a band silently
# widens its lower neighbour; only the compiled register's top band runs to
# infinity.
# infinity unless the binding declares a source-unit ``band_upper_bound``.
# Entity-count indicators: a value of one per record of the owning entity.
_COUNT_VALUE_VARIABLES = frozenset(
{"household_count", "person_count", "benunit_count"}
)
_COUNT_VALUE_VARIABLES = frozenset({"household_count", "person_count", "benunit_count"})

_BAND_LOWER_BOUND_SUFFIX = "_lower_bound"
_LEDGER_FILTER_PREFIX = "ledger_filter_"
Expand Down Expand Up @@ -182,9 +180,7 @@ def resolve_target_measures(
"skipped": [skip.__dict__ for skip in skipped],
}
receipt["rounds"].append(round_receipt)
all_skips.extend(
{**skip.__dict__, "round": round_index} for skip in skipped
)
all_skips.extend({**skip.__dict__, "round": round_index} for skip in skipped)
if not skipped:
return MeasureResolution(
measure_inputs=MappingProxyType(dict(measure_inputs)),
Expand Down Expand Up @@ -242,6 +238,10 @@ def resolve_target_measures(
all_skips,
)
measure_inputs[key] = np.asarray(values)
# Providers can learn the comparison contract while resolving
# inputs (for example, UC claim-state/date proxies). Preserve the
# post-computation receipt, not only the pre-resolution snapshot.
receipt["provider"] = _measure_provider_receipt(provider)
provided_this_round.add(key)
receipt["attached"][f"{entity}.{variable}"] = route
progressed = True
Expand Down Expand Up @@ -308,9 +308,7 @@ def _raise_measure_resolution(
raise MeasureResolutionError(message, receipt=receipt)


def _band_lower_edge(
spec: Any, binding: Mapping[str, Any]
) -> float | None:
def _band_lower_edge(spec: Any, binding: Mapping[str, Any]) -> float | None:
"""This spec's band lower edge in model units, or None if unbanded.

Reads the compiled spec's Ledger filter metadata: a ``*_lower_bound``
Expand Down Expand Up @@ -441,6 +439,30 @@ def _band_bounds(
if edge > lower:
upper = edge
break
# A consumer may bind only the finite portion of a published histogram.
# Declare its source-unit ceiling instead of allowing the last included
# row to absorb a separately published, unbound top-coded category.
if "band_upper_bound" in binding:
inclusive = binding.get("band_upper_bound_inclusive")
factor = float(binding.get("band_period_factor", 1))
declared_upper = float(binding["band_upper_bound"]) * factor
if (
not isinstance(inclusive, bool)
or not math.isfinite(factor)
or factor <= 0
or not math.isfinite(declared_upper)
or declared_upper <= lower
):
raise ValueError(
"band_upper_bound requires a finite bound above the lower "
"edge, a positive period factor and an explicit boolean "
"band_upper_bound_inclusive"
)
if inclusive:
declared_upper = np.nextafter(declared_upper, math.inf)
upper = min(upper, declared_upper)
elif "band_upper_bound_inclusive" in binding:
raise ValueError("band_upper_bound_inclusive requires band_upper_bound")
return lower, upper


Expand Down
1 change: 1 addition & 0 deletions packages/microcosm-build/src/microcosm/build/uk/gates.json
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,7 @@
"person.is_before_universal_credit_qualifying_young_person_terminal_date",
"person.is_in_non_advanced_education",
"person.is_parent",
"person.is_uc_claimant",
"person.legacy_jobseeker_proxy",
"person.outpatient_visits",
"person.pension_contributions_via_salary_sacrifice",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,7 @@
"capital_gains"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "HMRC Capital Gains Tax statistics, July 2025, Table 2.1a",
"survey": "HMRC Capital Gains Tax statistics Table 2.1a and Advani-Summers capital-gains incidence"
Expand All @@ -496,7 +496,7 @@
"capital_gains"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "Advani and Summers (2020), Capital Gains and UK Inequality, CAGE Working Paper 465",
"survey": "Family Resources Survey 2024-25, SPI synthetic support, and Advani-Summers capital-gains incidence"
Expand Down Expand Up @@ -525,7 +525,7 @@
"required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
"rewrites": [],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab, DfT rail fare index, and public NHS activity/cost table.",
"survey": "Effects of Taxes and Benefits 1977-2024 and NHS age-gender public table"
Expand All @@ -545,7 +545,7 @@
"required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
"rewrites": [],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "UK Data Service SN 8856 Effects of Taxes and Benefits household tab and cited VAT anchor resource.",
"survey": "Effects of Taxes and Benefits 1977-2024"
Expand Down Expand Up @@ -576,7 +576,7 @@
"superseded_by": {
"reason": "The FRS spine build executes hmrc_cgt_gains_spine, which applies the same HMRC Table 3 amounts redraw directly in source_stages.json before calibration.",
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"stage": "hmrc_cgt_gains_spine"
}
},
Expand All @@ -596,7 +596,7 @@
"capital_gains"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"hmrc_surface": "2023-24",
"mapped_build_period": "2024"
Expand All @@ -610,7 +610,7 @@
"base_candidate_tier": "frs",
"calibration_permitted": false,
"canonical_source_manifest": "source_stages.json",
"canonical_source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"canonical_source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"effective_mass_requirements": {
"charitable_investment_gifts": {
"mass_share_denominator": "all_person_effective_mass",
Expand Down Expand Up @@ -693,7 +693,7 @@
"superseded_by": {
"reason": "The FRS spine build executes hmrc_spi_income_spine, which supersedes the June retained-leaves/hmrc_spi_income pair inside source_stages.json.",
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"stage": "hmrc_spi_income_spine"
}
},
Expand Down Expand Up @@ -727,7 +727,7 @@
"required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
"rewrites": [],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "UK Data Service SN 9468 Living Costs and Food Survey 2023-24 household/person tabs, NEED 2023 headline energy tables, Ofgem Q2 2026 unit rates, and WAS round-8 bridge donor.",
"survey": "Living Costs and Food Survey 2023-24"
Expand All @@ -748,7 +748,7 @@
"property_wealth"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "MHCLG dwellings and ONS UK House Price Index December 2025 regional average prices.",
"survey": "Public regional property reference"
Expand All @@ -772,7 +772,7 @@
"employee_pension_contributions"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "HMRC, Salary sacrifice reform for pension contributions effective from 6 April 2029",
"survey": "Family Resources Survey 2024-25 salary-sacrifice respondents and HMRC salary-sacrifice reform analysis"
Expand All @@ -794,7 +794,7 @@
"student_loan_plan"
],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "Explore Education Statistics Table 6a, Higher education total",
"survey": "Family Resources Survey 2024-25 and Student Loans Company borrower forecasts for England"
Expand Down Expand Up @@ -829,7 +829,7 @@
"required_mass_change_reason": "E5 source-stage transform preserves household rows and typed household weights; total household mass is conserved.",
"rewrites": [],
"source_manifest": "source_stages.json",
"source_manifest_sha256": "d83310aa9ebb309b4a9c6409a35916cc49390e330827cce8d7321c613679cd84",
"source_manifest_sha256": "f1bb134b5456be67c10f2e09b5dc9999c75b1bf3c47165092e260cbd79a54732",
"source_vintages": {
"source": "Office for National Statistics Wealth and Assets Survey, UK Data Service SN 7215, DOI 10.5255/UKDA-SN-7215-20; local licensed 2006-22 household tab.",
"survey": "Wealth and Assets Survey round 8"
Expand Down
Loading
Loading