feat: add deterministic contribution IDs and specify artifact command

[nicolehaugen]

Consolidates the work from #4261 and #4267 into a single PR against main.

Why consolidated

#4261 (contribution IDs) and #4267 (specify artifact command) were originally opened as a stack, but the underlying identifier grammar and resolver changes evolved together through review — splitting them cleanly would leave both PRs with incomplete stories. This PR ships the full, cohesive story in one place.

What's in it

Contribution IDs (from #4261):

• Deterministic {layer}:{source_id}:{kind}:{name} identifier grammar for every core / preset / extension / project-override contribution.
• PresetResolver.collect_all_layers() returns each layer with a stable lookupId so the same ID round-trips through the resolver.
• _identifier.py module owns the grammar; consumers never re-derive it via string prefixes.

Artifact command (from #4267):

• New specify artifact command that surfaces composition stacks as JSON, keyed by the same lookup IDs above.
• _script_variants.py module: single place that knows the bash/, powershell/, python/ runtime subdir layout for core scripts.
• Shared _locate_core_asset_dir in _assets.py — one source of truth for locating core templates/, commands/, and scripts/ across wheel-installed and source-checkout layouts.

Fixes surfaced during review:

• collect_all_layers() uses the on-disk directory name as the extension sourceId, matching how iter_extensions_by_priority() and the artifact catalog enumerate extensions — closes a gap where a manifest with a declared id different from its directory produced a lookupId the resolver would not itself hand back.
• Preset-manifest projection branches on manifest kind rather than folding all preset contributions into provides.templates, so an installed override like speckit.constitution no longer becomes an ambiguous template: row.
• Project-override layer is serialized with its own project: layer prefix instead of being labeled as a preset with sourceId _.

Fixes

Fixes #4210
Fixes #4212

Supersedes

Supersedes #4261 (closed)
Supersedes #4267 (closed)

Testing

1132 tests passing on Windows (11 skipped, 127 deselected are pre-existing Windows symlink-privilege failures unrelated to these changes, confirmed by running the same tests against unmodified origin/main).

---

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, supervised).

[Copilot]

Review details

Suppressed comments (4)

Previously missed (2) — in code that hasn't changed since the last review.

extensions/EXTENSION-API-REFERENCE.md:866

• This identity distinction is now stale. collect_all_layers() uses the validated manifest ID for manifest-declared preset/extension layers even when the installation directory differs, so those lookupId values do join directly to iter_contributions(); only convention-only contributions use the directory identity. Update this paragraph and the repeated statements in the Python API/determinism subsections.

Every command, template, script, and hook contributed by an extension (or a preset, or the core layer) is addressable at read time by a deterministic opaque identifier. Resolved artifact-stack layers carry a matching `lookupId` field that shares this grammar and identifies the layer's stack position. Manifest contribution `id` values and resolver `lookupId` values are **not always equal** — they only join directly when the installed directory name matches the manifest-declared `id:` (see [Determinism guarantees](#determinism-guarantees) below). Identifiers are **computed on demand from author-declared manifest content** and are **never persisted** to `.specify/` or to any cache file.

src/specify_cli/artifacts/init.py:635

• ExtensionRegistry(...) can raise OSError while reading an unreadable .registry before is_corrupt() is reached, bypassing the artifact command's JSON error envelope. Translate constructor I/O failures to ArtifactResolutionError, as with malformed registry content.

This issue also appears on line 654 of the same file.

    if ExtensionRegistry(extensions_dir).is_corrupt():
        raise ArtifactResolutionError()

src/specify_cli/artifacts/init.py:655

• PresetRegistry(...) reads .registry in its constructor before is_corrupt() runs. A directory at .registry (or an unreadable file) therefore raises OSError here instead of producing the promised ArtifactResolutionError JSON envelope. Catch constructor I/O failures and translate them before calling is_corrupt().

    if PresetRegistry(presets_dir).is_corrupt():
        raise ArtifactResolutionError()

src/specify_cli/presets/init.py:5269

• The return-value documentation is stale: ext_id is the on-disk directory identity, but manifest-declared layers now use ExtensionManifest.id in lookupId. Clarify that the directory name is used only for lookup/provenance and convention-only contribution IDs.

        Each entry is ``(priority, ext_id, metadata_or_none)`` where ``ext_id``
        is always the on-disk directory name used in lookup identifiers.

• Files reviewed: 17/17 changed files
• Comments generated: 6
• Review effort level: Balanced

[Copilot]

Fixed in 1c4319c. The tier-5 branch now delegates to the existing _find_bundled_core helper, which already routes all three asset families through _locate_core_asset_dir — so a core_pack without scripts/ falls back to the source checkout in resolve() exactly as it does in collect_all_layers(). Added test_core_fallback_uses_shared_asset_resolver covering that case and asserting both APIs return the same path.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

Fixed in 1c4319c. Added _describe_artifact_file, which routes to the existing _extract_frontmatter_description / _extract_script_description extractors, and _iter_project_override_artifacts now yields that value instead of "". Since descriptions are picked from the highest-priority layer that has one, an active override with metadata now reports its own description; a metadata-free override still falls back to the hidden layer. Both cases are covered by new tests.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

Updated in 1c4319c. The paragraph now states the manifest-declared join is guaranteed by the implementation so consumers can key off lookupId directly, and scopes the registry/directory fallback to convention-only layers that have no manifest contribution to join to.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

Added in 1c4319c: docs/reference/artifacts.md documents both JSON shapes (list rows and the info object), stack semantics including active/hidden being independent labels, core rows, lookup IDs and their join to the preset contribution grammar, and the JSON error envelope with its pinned messages and exit codes. Linked from docs/toc.yml and docs/reference/overview.md, and referenced from the _commands.py module docstring.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

PR description updated in 1c4319c. The bullet now states that manifest-declared preset/extension layers derive their sourceId from the manifest's validated id:, with the registry key / on-disk directory name used only as the convention-only fallback.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

Fixed in 1c4319c. Both iter_presets_by_priority and iter_extensions_by_priority docstrings now say the registry key / on-disk directory name identifies where the pack lives — used for lookup, provenance, and as the sourceId of convention-only contribution IDs — while manifest-declared layers take their lookupId sourceId from PresetManifest.id / ExtensionManifest.id.

Posted on behalf of @nicolehaugen by GitHub Copilot (model: claude-opus-4.7, autonomous).

[Copilot]

Review details

Suppressed comments (5)

Previously missed (2) — in code that hasn't changed since the last review.

src/specify_cli/artifacts/init.py:618

• This guard does not fully fail closed: ExtensionRegistry.is_corrupt() accepts a present JSON object with no extensions key, after which _load() normalizes it to empty and the resolver admits every on-disk directory as an unregistered extension. The preset check has the analogous false-inventory behavior. Require the collection key itself to be present and mapping-valued whenever a registry file exists.

    if ExtensionRegistry(extensions_dir).is_corrupt():

src/specify_cli/presets/init.py:5270

• This says the directory identity is used in lookup identifiers, but manifest-declared extension layers now use extension.id; only convention-only layers use ext_id. Update the docstring so callers do not rely on the wrong identity.

        """Return extension directories in resolver lookup order.

        Each entry is ``(priority, ext_id, metadata_or_none)`` where ``ext_id``
        is always the on-disk directory name used in lookup identifiers.
        """

src/specify_cli/presets/init.py:5435

• This tier still bypasses _locate_core_asset_dir. If core_pack exists but lacks scripts/, candidate stays None and resolve() never tries the source checkout, while collect_all_layers() does fall back through the shared helper. Route tier-5 resolution through _locate_core_asset_dir for all asset families so path resolution and stack reporting cannot disagree.

                candidate = next(
                    (path for path in script_variant_paths(_core_pack / "scripts", template_name) if path.exists()),
                    None,

src/specify_cli/artifacts/_commands.py:35

• The linked #4212 acceptance criteria requires a CLI reference section documenting the full JSON shapes and stack semantics, but the documentation tree contains no specify artifact reference. The Typer help strings do not document fields such as active, hidden, manifestPath, or lookupId; add the required reference documentation before shipping the new public command.

artifact_app = typer.Typer(
    name="artifact",
    help="Introspect commands, templates, and scripts SpecKit exposes.",
    no_args_is_help=True,
)

src/specify_cli/presets/init.py:5117

• This new docstring is inaccurate for manifest-declared contributions: pack_id is the registry/on-disk identity, but their lookup IDs use the validated manifest ID. Reserve the statement about pack_id in lookup identifiers for convention-only contributions.

        """Return preset directories in resolver lookup order.

        Each entry is ``(pack_id, metadata)`` where ``pack_id`` is the registry
        key/directory name used in lookup identifiers.
        """

• Files reviewed: 17/17 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

src/specify_cli/presets/init.py:675

• A registry JSON object that omits presets is accepted as healthy, even though _load() silently normalizes it to an empty map and drops every installed preset from the artifact inventory. Treat a missing key the same as any other non-mapping presets value so this fail-closed probe fulfills its contract.

        if "presets" in data and not isinstance(data["presets"], dict):
            return True

src/specify_cli/artifacts/_commands.py:150

• The info path also lets filesystem OSErrors escape the JSON error contract—for example, an unreadable registry raises during construction before the corruption probe runs. Convert these read failures to ArtifactResolutionError, as the command promises for registry/manifest resolution failures.

    except PresetError:
        _emit_error_and_exit(ArtifactResolutionError())
        return  # pragma: no cover

src/specify_cli/_identifier.py:128

• The “complete shape” check accepts invalid kinds: core:_:bogus:name currently returns core, and callers then treat that malformed ID as authoritative. Validate the kind enum before selecting the expected component count so only named artifact kinds and hooks classify successfully.

    expected_len = 5 if parts[2] == "hook" else 4
    if len(parts) != expected_len:
        return None

src/specify_cli/presets/init.py:5731

• The PR description says extension sourceId now uses the on-disk directory name, but this implementation intentionally uses ExtensionManifest.id for manifest-declared layers and only falls back to the directory for convention-only entries. The code, tests, and lookup round-trip contract agree here; update the PR description so consumers are not given the opposite identity rule.

                            ext_manifest = ExtensionManifest(ext_manifest_path)
                            if isinstance(ext_manifest.id, str) and ext_manifest.id:
                                source_id_for_lookup = ext_manifest.id

• Files reviewed: 20/20 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (7)

Previously missed (4) — in code that hasn't changed since the last review.

docs/reference/presets.md:223

• layer_kind_from_lookup_id() cannot distinguish manifest-declared contributions from convention-only contributions; both use the same preset or extension layer prefix. It only classifies the source layer, so this guidance would lead consumers to make a distinction the API cannot provide.

`PresetResolver.collect_all_layers()` returns layer dicts that each include a `lookupId` field that identifies the layer. For manifest-declared preset and extension layers, the `lookupId`'s `sourceId` component is the manifest's validated `id:` field, so it joins directly to the `id` used by `PresetManifest.iter_contributions()` / `ExtensionManifest.iter_contributions()` even when the installed directory was renamed. That join is guaranteed by the implementation, so consumers can key off `lookupId` directly rather than re-deriving the contribution id. Convention-only layers (undeclared in any manifest) have no manifest id to consult, so their `lookupId`'s `sourceId` falls back to the resolver's registry key or on-disk directory name instead; those layers have no manifest contribution to join to. Use `layer_kind_from_lookup_id` to tell the two cases apart rather than parsing the string yourself. Project-local overrides in `.specify/templates/overrides/` are a resolver-only concept — they carry a synthetic `project:_:{kind}:{name}` `lookupId` that intentionally does not match any manifest contribution.

docs/reference/artifacts.md:125

• The command does not fail when a manifest cannot be read: both managers treat invalid manifests as unavailable, and the resolver may continue with convention-based files. Only registry/read-resolution failures are converted to this message, so listing unreadable manifests as a guaranteed cause documents behavior the implementation does not provide.

| `artifact resolution failed`                        | The preset/extension registries or a manifest could not be read   |

src/specify_cli/artifacts/init.py:812

• get_artifact_info() rebuilds the complete inventory repeatedly: _find_matches() calls list_artifacts() here (and once above), then _describe() calls it a third time. Each rebuild walks every pack and resolves every artifact stack, so a single info request scales poorly for the wizard use case. Build the inventory once and reuse the matching Artifact for validation and description.

        if not any(kind_name == resolved_kind for kind_name, _ in self._find_matches(validated_name)):

src/specify_cli/artifacts/init.py:458

• A convention-only layer can belong to a pack that has a valid manifest which does not declare this artifact. This code still returns that manifest's path, so manifestPath falsely identifies it as the declaring manifest even though the layer's lookupId intentionally has no manifest contribution to join. Carry whether the resolver match was manifest-declared and return None here for convention fallback layers.

    if not manifest_path.is_file():
        return None

src/specify_cli/artifacts/init.py:654

• PresetRegistry(...) executes _load() before this corruption probe. Because _load() neither guards a non-regular .registry path nor catches OSError, a registry directory or unreadable file raises before is_corrupt() runs and bypasses the artifact command's JSON error envelope. Convert that construction failure to ArtifactResolutionError.

    if PresetRegistry(presets_dir).is_corrupt():

src/specify_cli/_identifier.py:128

• This classifier accepts unknown contribution kinds such as core:_:bogus:name as valid core lookup IDs, even though its contract says malformed IDs return None. Callers then treat a value outside the identifier grammar as authoritative; validate the kind before selecting the expected component count.

        return None
    if parts[2] not in _CONTRIBUTION_KINDS:
        return None

src/specify_cli/presets/init.py:5731

• The current PR description still says collect_all_layers() uses the on-disk extension directory as sourceId, but this implementation deliberately substitutes the validated manifest ID for manifest-declared layers. The code, tests, and round-trip contract agree on the manifest ID, so update the PR description to avoid publishing the opposite identifier contract.

                                ValidationError as ExtValidationError,
                            )
                            ext_manifest = ExtensionManifest(ext_manifest_path)

• Files reviewed: 20/20 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

src/specify_cli/artifacts/init.py:812

• get_artifact_info() currently rebuilds the complete inventory up to three times: here, again during kind validation, and again in _describe(). Each build scans every contribution and calls collect_all_layers() for every artifact, so one info lookup multiplies the most expensive filesystem work—especially costly for the wizard's intended repeated lookups. Build the inventory once and reuse its matching row and description.

        if resolved_kind is None:
            matches = self._find_matches(bare)

docs/reference/artifacts.md:125

• This cause is not implemented: malformed or unreadable manifests are caught and skipped by the resolver/catalog, which may then use convention fallback, so they do not produce artifact resolution failed. Narrow this row to the failures that actually reach the error envelope (corrupt registries or uncaught inventory filesystem access) to avoid promising fail-closed manifest validation.

| `artifact resolution failed`                        | The preset/extension registries or a manifest could not be read   |

src/specify_cli/presets/init.py:5733

• The PR description says extension stack IDs use the on-disk directory name as sourceId, but this code deliberately substitutes ExtensionManifest.id; the tests and API docs also require the manifest ID so lookupId joins to iter_contributions(). Update the PR description to state that directory identity is only the convention-only fallback.

                            ext_manifest = ExtensionManifest(ext_manifest_path)
                            if isinstance(ext_manifest.id, str) and ext_manifest.id:
                                source_id_for_lookup = ext_manifest.id

• Files reviewed: 20/20 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

Review details

Suppressed comments (2)

src/specify_cli/presets/init.py:5598

• This exclusion is only applied in collect_all_layers(). The earlier resolve() convention loops still return <preset>/README.md or <extension>/README.md as a root-level template, so preset resolve README can succeed while artifact list/info omit the same artifact. Apply the same case-insensitive root-level README check in both resolve() loops to keep resolution and inventory consistent.

                    if template_name.lower() == "readme":
                        continue

src/specify_cli/presets/init.py:5733

• The current PR description says extension sourceId uses the on-disk directory name, but this implementation deliberately uses ExtensionManifest.id for manifest-declared layers (with the directory only for convention fallback). The code matches the round-trip requirement and tests, so update that PR-description bullet to publish the actual contract.

                            ext_manifest = ExtensionManifest(ext_manifest_path)
                            if isinstance(ext_manifest.id, str) and ext_manifest.id:
                                source_id_for_lookup = ext_manifest.id

• Files reviewed: 20/20 changed files
• Comments generated: 0 new
• Review effort level: Balanced