Skip to content

fix: handle Windows paths in adk eval#6419

Open
sgoel2be24-cyber wants to merge 2 commits into
google:mainfrom
sgoel2be24-cyber:fix-6415-windows-eval-paths
Open

fix: handle Windows paths in adk eval#6419
sgoel2be24-cyber wants to merge 2 commits into
google:mainfrom
sgoel2be24-cyber:fix-6415-windows-eval-paths

Conversation

@sgoel2be24-cyber

Copy link
Copy Markdown

Summary

  • detect Windows drive-qualified eval-set paths before parsing case selectors
  • search for the selector delimiter only after the drive prefix
  • preserve existing POSIX-path, eval-set-ID, blank-selector, multiple-input,
    and duplicate-merging behavior
  • add platform-independent regression coverage for Windows path variants

Fixes #6415

Root cause

parse_and_get_evals_to_run() treated the colon in a Windows drive prefix
such as C:\ as the eval-case selector separator. This reduced the eval-set
path to C and discarded selectors appearing after a later colon.

Testing

  • 9 passed — focused parser regression cases
  • 1 passed — existing
    test_cli_eval_with_eval_set_file_path regression
  • 11 passed — complete parser test module
  • 42 passed — adjacent CLI test module
  • Ruff passed
  • isort check passed
  • Pyink check passed
  • git diff --check passed

The complete repository suite was not rerun because the native Windows
baseline contains known unrelated platform failures.

Two aggregate pre-commit hooks use POSIX executable/shebang paths that native
Windows could not launch directly. Their underlying checks passed through the
repository’s available Git Bash and uv routes; no hook configuration was
changed.

Scope

The change is limited to the eval-set parser and focused unit tests. It does
not depend on filesystem existence and does not change evaluation execution,
result storage, or CLI syntax.

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label Jul 17, 2026
@sgoel2be24-cyber

Copy link
Copy Markdown
Author

Thanks for updating the branch with the latest main.

I checked the resulting CI failures. All unit-test jobs fail on the same unrelated Antigravity test, test_resumed_replayed_steps_are_skipped, because its sess_456_agy fixture is shorter than the current 32-character conversation_id requirement.

Current upstream main fails the same test with the same validation error across Python 3.10–3.14, and this PR does not modify the affected Antigravity source or test files.

On the updated PR head, the contribution-specific checks still pass:

  • 9 focused parser regression tests
  • existing Windows CLI regression
  • complete parser module: 11 tests
  • adjacent CLI module: 42 tests
  • Ruff, isort, Pyink, and git diff --check

The triage-agent failure also appears transient because Gemini returned 503 UNAVAILABLE. I don’t see any contributor-side code change needed for this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: adk eval misparses Windows absolute eval-set paths as case selectors

4 participants