test(ci): derive the hardware-docs guard from the workflows

[volen-silo]

Part of #294 — acceptance criterion 4 ("adding a new lane without updating docs/ci-hardware-testing.md fails CI"). The drift fixes and the composite-action extraction follow separately.

Summary

• hardware_testing_docs_cover_all_self_hosted_platforms now derives its expectations from e2e-selfhosted.yml / nightly.yml / ci.yml instead of comparing the doc against literals in the test source.
• Two surfaces gain a guard they never had: the runner-labels column, which was parsed and then discarded (.map(|row| (row[0], row[2]))), and the dispatch-choices list, which had none.
• No documentation changes — see below.

Why the old guard could not work

Both sides were hand-maintained. Adding a lane changed neither the doc nor the vec!, so the test stayed green with the table silently short a row. It only fired when someone edited one side without the other — which fails the person fixing the docs, not the person who made them wrong. #278 shows the shape: the author had to update the workflow, the doc, the README and the test literals in lockstep, and the test went red mid-way through doing so.

New assertions

Derived per self-hosted job, in workflow order: table rows, runner labels vs actual runs-on, artifact list (via uploaded_e2e_artifacts across all three workflows), workflow_dispatch.inputs.platform.options vs the documented choices, and both "the self-hosted jobs (…)" prose enumerations. The Platform column stays free prose — it isn't derivable from YAML — and is asserted non-empty only.

Correcting the issue on two points

The docs are not currently stale. #294 says adding a lane in #278 "left the doc table, the artifact list, two sentences and the dispatch-options list stale, with the test green throughout". #278 (01ba5257) in fact updated docs/ci-hardware-testing.md (22 lines), tests/e2e-cucumber/README.md (6) and the test literals (13) in the same commit. All six derived assertions pass against the doc exactly as it is on main, which is why this PR touches no documentation. The defect is structural — hardcoded expectations cannot catch future drift — not a divergence sitting in the tree today.

"Three of those lanes" (ci-hardware-testing.md:188) is not stale either. Exactly three lanes carry strix-halo. Left alone.

Non-obvious decisions

• No YAML crate. The file deliberately uses line-scanning helpers and documents why; this matches that style.
• runs_on_values generalized to flattened_values(text, key) so the same continuation-joining logic can read options:. runs_on_values is now a wrapper and its existing extractor guard passes untouched.
• uploaded_e2e_artifacts lived inside #[cfg(test)] mod tests in e2e_report.rs and had to be hoisted to #[cfg(test)] pub(crate) to be shared. Body unchanged; both existing consumers still pass.
• Job-id scanning is scoped to top_level_block(text, "jobs"), so push: / group: / contents: at the same indent cannot be mistaken for jobs. Covered by a dedicated extractor guard alongside the file's other helper tests.

Verification

cargo fmt --all -- --check = 0 · cargo clippy --workspace --all-targets -- -D warnings = 0 · cargo test --workspace --all-targets = 0 · python scripts/smoke_local.py = 0.

A guard whose failure mode is "passes while wrong" proves nothing by passing, so it was mutation-tested. Each of these fails, and the tree restores byte-identical:

Mutation	Result
Delete a lane row from the doc table	FAIL — job-column assertion, naming the missing lane
Change a Runner labels cell	FAIL — labels assertion, showing documented vs actual
Blank a Platform cell	FAIL — non-empty assertion
Remove one dispatch choice from the doc	FAIL — dispatch-choices assertion
Add a sixth self-hosted lane, docs untouched	FAIL, and it keeps failing through each surface in turn — table row, then artifact list, then dispatch options, then the first prose enumeration, then the second — going green only after all six follow

Load-bearing tests confirmed still passing: ci_yml_schedules_no_self_hosted_job, workflows_use_distinct_concurrency_groups, self_hosted_workflow_owns_the_gpu_lanes, all four *_prebuilt_e2e_lanes_*, and both consumers of uploaded_e2e_artifacts.

Scope

CI/test-only; no CLI-observable behaviour changes, so no Gherkin scenario is required (AGENTS.md §3 exempts CI plumbing and tests).

One CI-visibility note worth stating: xtask/src/affected.rs forces a full-workspace test run for any .github/workflows/ change, so this test runs on PRs that touch the workflows. A docs-only PR selects nothing (is_ignorable drops docs/), so the doc half is enforced on push/merge_group and by windows-build-and-test rather than on the PR itself.

[rominf]

I reviewed this PR (diff-only, plus building and running the affected xtask tests locally against the PR's actual worktree content) and didn't find anything blocking.

What I checked in particular, since this is a CI guard whose whole point is not to pass silently when it shouldn't:

• Ran hardware_testing_docs_cover_all_self_hosted_platforms and the rest of workflow_contract/e2e_report against the PR head — all 21 workflow-contract tests and the full xtask suite (110 tests) pass as-is, and cargo fmt --check / cargo clippy --all-targets -D warnings are clean.
• Reproduced a few of the mutation cases described in the PR body myself (adding a 6th self-hosted lane without touching the docs, adding a dispatch option, and drifting a runner-labels cell) — each one fails the new guard with a clear, specific assertion message naming the mismatch, so the derived assertions do catch the drift they claim to.
• Traced the new helpers (flattened_values, flattened_list_items, self_hosted_e2e_jobs, the nested_block reuse for workflow_dispatch.inputs.platform.options) against the actual e2e-selfhosted.yml content — the job-id scan is correctly scoped to the jobs: block, and the label/list comparisons work for both the flow ([a, b]) and block (- a) YAML spellings.
• The PR's claim that the docs aren't currently stale and that this touches no documentation checks out — git diff confirms only xtask/src/e2e_report.rs and xtask/src/workflow_contract.rs changed.

This is a diff-and-local-test read, not a substitute for CI or a maintainer's own pass — but from that read, this looks safe to merge once CI is green.

[r0x0r]

This is a solid structural fix — deriving the expectations from the workflows is the right call, and the mutation table (especially "add a sixth lane, docs untouched" failing through each surface in turn) is convincing evidence the guard actually bites now rather than passing-while-wrong.

One forward-looking note on flattened_list_items: for the block form it splits the " - "-joined string that flattened_values produced back on " - ". That round-trip is lossy — it's safe for runner labels and the current dispatch options: values (none contain " - "), but now that flattened_values is generalized over arbitrary keys, a future derived value that legitimately contained " - " would mis-split silently. It's a reasonable assumption to make given the inputs; worth a one-line comment stating that the helper assumes label-like tokens with no embedded " - ", so the constraint is explicit for whoever reuses it next.

Everything else — scoping the job-id scan to top_level_block(text, "jobs"), hoisting uploaded_e2e_artifacts to shared with the body unchanged, and keeping runs_on_values as a thin wrapper so its existing guard still applies — reads well.