Skip to content

fix: Make netlify-fetch immune to stale cached release branches - #905

Open
lfrancke wants to merge 2 commits into
mainfrom
fix/netlify-fetch-stale-refs
Open

fix: Make netlify-fetch immune to stale cached release branches#905
lfrancke wants to merge 2 commits into
mainfrom
fix/netlify-fetch-stale-refs

Conversation

@lfrancke

@lfrancke lfrancke commented Jul 28, 2026

Copy link
Copy Markdown
Member

Description

Production docs builds fail with FATAL (antora): Could not find e9b0f5de... (branch: release-26.7).

Root cause

e9b0f5de is not a commit, it is a blob: ui/NOTICE on release-26.7.
And it could not be found because (see the fix section below) we excluded ui from checkout because it used to be a submodule. And the isomorphic-git used by Antora can't dynamically fetch missing content so.... it was missing.

The fix

-             git -c gc.auto=0 checkout -q "$$branch" -- ':(exclude)ui' . ; \
+             git -c gc.auto=0 checkout -q "$$branch" -- . ; \

Do not exclude UI anymore... simple as that :(

Separate preemptive fixes/hardening

These were "discovered" during the research...would not really help the root cause but good to have anyway.

  • gc.auto=0 for the loop commands, so background gc cannot race the ref updates.
  • set -e + no -q on fetch: to just fail where the failure actually happens to get a better error message
  • git fetch --all --prune to delete stale stuff from the cache, not only add or update

The first build after a new release branch can also fail because the branch list that is used uses the last cached git state from the previous run. That doesn't have the new release branch so it can fail. This now uses git ls-remote --heads origin (instead of git branch -r) to make sure we have the up-to-date state.

Verified by simulation (by Claude): blobless clone (like netlify's), stale release-26.7 ref planted pointing at an outdated commit, make netlify-fetch run - the ref force-updates to the real remote tip, all content blobs of every release tip are present (only the deliberately excluded ui/ blobs stay missing, which antora never reads), working tree clean, exit 0.

@netlify

netlify Bot commented Jul 28, 2026

Copy link
Copy Markdown

Deploy Preview for stackable-docs ready!

Name Link
🔨 Latest commit 04f2864
🔍 Latest deploy log https://app.netlify.com/projects/stackable-docs/deploys/6a68d27df740ba0008ae6075
😎 Deploy Preview https://deploy-preview-905--stackable-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@lfrancke lfrancke self-assigned this Jul 28, 2026
@lfrancke lfrancke moved this to Development: Waiting for Review in Stackable Engineering Jul 28, 2026
Comment thread Makefile Outdated
NickLarsenNZ
NickLarsenNZ previously approved these changes Jul 28, 2026

@NickLarsenNZ NickLarsenNZ left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a comment, but in any case, it's an improvement.

@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Jul 28, 2026
Production builds failed with antora unable to find object e9b0f5d on
release-26.7. That object is the ui/NOTICE blob: release-26.7 is the
first release branch cut after the UI was vendored into this repo, so
ui/ is regular content there instead of a submodule gitlink - and the
blob-materializing loop excluded ui/ from its pathspec checkouts, a
rule written when every release branch still had the submodule. Antora
reads the whole tree of a content branch with isomorphic-git, which
cannot fetch missing objects on demand, and died on the first ui blob.
Reproduced and verified against a fresh blobless clone: with the
exclusion the exact production error appears, without it the build
passes and no tip objects are missing.

Dropping the exclusion is harmless for the old branches: a gitlink has
no blobs to fetch.

Additional hardening while in here:
- branch list from git ls-remote instead of the cached remote-tracking
  refs (make expanded the old list before the fetch even ran, so a
  brand-new release branch was invisible to its first build)
- no more silent failures: set -e and no -q on fetch abort the build
  at the failing command instead of surfacing later as a confusing
  antora error
- gc.auto=0 inside the loop and git fetch --all --prune keep the
  long-lived netlify build cache tidy and race-free
Review feedback (set -e at the top / pipefail): make runs every recipe
line in its own shell, so a leading set -e would only govern itself,
and the shell is sh, which has no pipefail - but the spirit of the
comment found a real gap: a failing ls-remote inside the command
substitution would have silently emptied the loop. The list is now
captured and checked explicitly, and set -eu replaces set -e.
@lfrancke
lfrancke requested a review from NickLarsenNZ July 28, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Development: In Review

Development

Successfully merging this pull request may close these issues.

2 participants