feat(instructions): add overlay metadata annotation guidance and skill#17890
Open
liunan-ms wants to merge 2 commits into
Open
feat(instructions): add overlay metadata annotation guidance and skill#17890liunan-ms wants to merge 2 commits into
liunan-ms wants to merge 2 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR is documentation-only. It adds guidance and an on-demand skill for annotating Azure Linux overlays with metadata (a category, plus optional provenance/upstreamable fields). It fits the repo's existing .github/instructions/* + .github/skills/* structure, giving agents a canonical reference for the closed set of overlay categories and a workflow to classify overlays, find grounding upstream commits/bugs, and write metadata without changing rendered specs.
Changes:
- New
overlay-metadata.instructions.mddocumenting the 10-valuecategoryset,commits/bugs/upstreamablesemantics, the per-file.overlay.tomlvs. inline TOML forms, and file layout/ordering rules. - New
skill-annotate-overlaysskill defining the classify → hunt-provenance → judge-upstreamability → write → validate workflow. - Registers the new skill in the
AGENTS.mdskills table.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.github/instructions/overlay-metadata.instructions.md |
New instructions on picking a category and writing overlay metadata; the multi-overlay TOML example misuses spec-remove-subpackage and has a mismatched package value. |
.github/skills/skill-annotate-overlays/SKILL.md |
New skill defining the annotation workflow; references an azldev component diff-sources subcommand that isn't documented or used elsewhere in the repo. |
AGENTS.md |
Adds a skills-table row for the new skill; format is consistent with surrounding rows. |
Comment on lines
+120
to
+132
| # One logical change: drop the maven-local-openjdk21 subpackage. | ||
| [metadata] | ||
| category = "azl-pruning" | ||
|
|
||
| [[overlays]] | ||
| description = "Remove maven-local-openjdk21 %package section — AZL ships no java-21-openjdk" | ||
| type = "spec-remove-subpackage" | ||
| package = "maven-local-openjdk21" | ||
|
|
||
| [[overlays]] | ||
| description = "Remove maven-local-openjdk21 %files section — subpackage above is gone" | ||
| type = "spec-remove-subpackage" | ||
| package = "javapackages-local-openjdk21" |
Comment on lines
+240
to
+242
| azldev component diff-sources -p <name> -q > ./base/build/work/scratch/<name>-diff-before.txt # before edits | ||
| # … move overlays into files / add metadata … | ||
| azldev component diff-sources -p <name> -q > ./base/build/work/scratch/<name>-diff-after.txt # after edits |
liunan-ms
force-pushed
the
liunan/overlay_annotation_instructions
branch
from
July 13, 2026 20:13
9922f66 to
e8a8cc1
Compare
Comment on lines
+102
to
+103
| If any of these turns up the keywords, classify as `azl-disable-flaky-tests`; | ||
| otherwise treat the skipped test as `azl-disable-unsupported-tests`. |
|
|
||
| If any of these turns up the keywords, classify as `azl-disable-flaky-tests`; | ||
| otherwise treat the skipped test as `azl-disable-unsupported-tests`. | ||
| - Fedora→AZL name/path or RHEL-alignment edits → `azl-branding-policy`. |
| URL you discovered and verified is not "inventing" metadata; an unverified guess | ||
| is.** Never fabricate a `bugs` entry. | ||
|
|
||
| Search in this order, stopping at the first that yields a **verifiable** commit: |
Comment on lines
+209
to
+213
| - **Move *all* of a component's overlays into files.** Overlays loaded via | ||
| `overlay-files` are **appended after** any inline overlays left on the | ||
| component, so a partial migration silently moves the file-loaded overlays to the | ||
| end of the sequence. Leave no inline `[[overlays]]` behind (or preserve the | ||
| exact global order via ordinals across both). |
Comment on lines
+240
to
+244
| **Order check without a full render.** If `render` is unavailable (e.g. the mock | ||
| build root is locked in a shared environment), use `diff-sources`, which applies | ||
| the overlays in order and shows the resulting patch. Capture it before and after | ||
| the migration and confirm they are byte-identical — this directly proves the apply | ||
| order was preserved: |
|
|
||
| If `render` fails, re-check the per-file rules in the instructions — common | ||
| mistakes: per-overlay `metadata` in a `.overlay.toml`; a file-level `[metadata]` | ||
| missing `category`; an `overlay-files` glob that matches no files; leftover inline |
Comment on lines
+218
to
+220
| Preserve any explanatory top-of-file comments and the existing `description` text | ||
| verbatim; if a `description` is missing, leave it absent unless the user asks you | ||
| to write one. |
Comment on lines
+243
to
+246
| If you added or modified the overlay's actual behavior (not just metadata), | ||
| follow the normal build + smoke-test protocol in | ||
| [`AGENTS.md`](../../AGENTS.md) and re-run `azldev comp update -p <name>` before | ||
| opening a PR. Metadata-only edits need neither a rebuild nor a lock refresh. |
Collaborator
There was a problem hiding this comment.
This contradiction with what is in AGENTS.md might cause LLM issues. Please resolve.
Contributor
There was a problem hiding this comment.
update is super fast, really no cost to running it always. I think the suggestion would probably work.
christopherco
requested changes
Jul 18, 2026
Comment on lines
+243
to
+246
| If you added or modified the overlay's actual behavior (not just metadata), | ||
| follow the normal build + smoke-test protocol in | ||
| [`AGENTS.md`](../../AGENTS.md) and re-run `azldev comp update -p <name>` before | ||
| opening a PR. Metadata-only edits need neither a rebuild nor a lock refresh. |
Collaborator
There was a problem hiding this comment.
This contradiction with what is in AGENTS.md might cause LLM issues. Please resolve.
dmcilvaney
reviewed
Jul 21, 2026
| @@ -0,0 +1,246 @@ | |||
| --- | |||
| applyTo: "**/*.comp.toml, **/*.overlay.toml" | |||
Contributor
There was a problem hiding this comment.
Lately I'm a fan of putting as much effort into the skill file as possible. Unfortunately the instruction files are not cross agent (ie claude code won't read them). Skills are generally the globally accepted medium. You can just add a direction here "When doing any of the following: a, b, c, always load skill X"
In this case, maybe two skills if there isn't enough overlap?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds documentation and an on-demand skill for annotating Azure Linux overlays with
metadata(category, provenance, upstreamability).Commits
overlay-metadata.instructions.md— source of truth for the closed set of overlaycategoryvalues, field semantics (commits,bugs,upstreamable), the per-file.overlay.tomlvs. inline TOML forms, and overlay file layout/naming.skill-annotate-overlays— targeted skill that, for new or user-specified overlays, reads each change, suggests a category (with rationale), hunts for grounding upstream commits/bugs, judgesupstreamable, and writes the metadata per the instructions. Registered inAGENTS.md.Key principles
[metadata]block per logical change, with one or more[[overlays]]..overlay.tomllayout for new work, even single-overlay changes.Validation