Skip to content

rules_go: make instrumented artifacts deterministic - #251

Merged
tonyredondo merged 6 commits into
mainfrom
fix/rules-go-stdlib-determinism
Sep 11, 2026
Merged

rules_go: make instrumented artifacts deterministic#251
tonyredondo merged 6 commits into
mainfrom
fix/rules-go-stdlib-determinism

Conversation

@tonyredondo

Copy link
Copy Markdown
Member

Summary

  • remove sandbox and module-cache paths from instrumented stdlib and synthetic helper archives
  • normalize variable Go package build IDs only in the declared helper-bundle copies, leaving Go's internal cache untouched
  • publish synthetic packagefile paths relative to the execroot
  • compare action keys and every declared output byte across two isolated cold builds for GoStdlib, GoSyntheticTestmainHelpers, testmain GoCompilePkg, and GoLink
  • run that reproducibility gate for every supported rules_go version on Linux x86-64 and ARM64

Why

Reprise showed stable-looking Bazel actions whose declared outputs still differed between cold roots. The differences came from temporary module-cache paths in Go and assembly objects plus action-specific package build IDs. Reusing those outputs through remote cache was therefore unsafe even though the build graph itself was stable.

The fix is limited to Test Optimization builds. Plain stdlib compilation and ordinary assembler invocations retain their previous behavior, and no remote-download or test-execution policy changes are included.

Validation

  • functional determinism verification passed for v0.60.0, v0.61.1, v0.62.0, and v0.63.0; the first three parallel commands were interrupted only during temporary-directory cleanup after reporting verified
  • python3.12 -m unittest discover -s tools/tests/python -p test_rules_go_profile_tools.py (14 passed)
  • ./bazelw test //tools/tests/python:python_tools_test --noexperimental_split_xml_generation --test_env=PYTHON=/Users/tony.redondo/.pyenv/shims/python3.12 (504 passed, 1 platform skip)
  • uncached compilepkg_test and orchestrion_test passed for all four vendored forks
  • ./bazelw test //... --noexperimental_split_xml_generation --test_env=PYTHON=/Users/tony.redondo/.pyenv/shims/python3.12 --java_runtime_version=remotejdk_21 (491 passed)
  • Bzlmod monorepo consumer E2E passed with the local archive and rules_go v0.63.0, including disabled mode, instrumented tests, cache replay, payload validation, doctor, uploader, and static .topt targets
  • fork-map generation, materialization, module-version alignment, release-archive contents, Python compilation, gofmt, YAML parsing, JSON parsing, and git diff --check passed

@tonyredondo
tonyredondo requested a review from a team as a code owner September 9, 2026 02:11

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The change removes variable paths and build IDs only from Test Optimization outputs. The check compares action keys and all declared output bytes for each affected action type.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit b1741e5 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@datadog-prod-us1-4

This comment has been minimized.

@tonyredondo
tonyredondo merged commit 76f2e4c into main Sep 11, 2026
62 checks passed
@tonyredondo
tonyredondo deleted the fix/rules-go-stdlib-determinism branch September 11, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants