Skip to content

chore(parallel tests): revamp parallel test outputs for searchability and clarity - #17940

Open
chalmerlowe wants to merge 8 commits into
mainfrom
feat/parallel-output-revamp
Open

chore(parallel tests): revamp parallel test outputs for searchability and clarity#17940
chalmerlowe wants to merge 8 commits into
mainfrom
feat/parallel-output-revamp

Conversation

@chalmerlowe

@chalmerlowe chalmerlowe commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Warning

  • Revert Before Merge: The changes to setup.py files and the dummy test in google-cloud-dns are only for verification and will be reverted before this PR is merged, hence the do-not-merge label.

Problem

The current parallel system test output can be difficult to read and search, especially when dealing with many packages or mixed results (where some test sessions pass and others fail). Logs for successful packages were often hidden, and searching for specific package results required scanning through large blocks of text.

Solution

This PR revamps the output format of parallel system tests to improve readability and searchability (CTRL+F).

Specific Changes

  • Standardized Segments: Clearly segregates Succeeded and Failed packages.
  • Search Tokens: Introduces unique, consistent tags (@SUMMARY, @FAILED, @SUCCEEDED, @PACKAGE) for quick navigation (i.e. via CTRL+F) to key sections of the logs.
  • Robust Logging: Ensures logs for both succeeded and failed packages are displayed correctly.
  • Noise Reduction: Removes redundant wrapper chatter (like Configuring environment...).
  • Mixed Results Handling: Packages with mixed results (some sessions succeed, some fail) are grouped under the @FAILED section to ensure visibility, with clear labels indicating which session(s) need attention.

Note

  • Temporary Test Setup: This PR includes temporary changes to 4 fast packages (google-cloud-dns, google-cloud-translate, google-cloud-speech, google-cloud-access-approval) to demonstrate the new layout and verify handling of mixed results.
  • Focus on system.sh: The core of this PR is the changes to .kokoro/system.sh.
  • Lint failures: are due to the minor triggers added and will disappear when the triggers are reverted.

🔍 Search Cheat Sheet for Reviewers

Goal Search Term (CTRL+F)
Jump to the Summary Table @SUMMARY
Jump to Section for Failed Packages @FAILED
Jump to Section for Succeeded Packages @SUCCEEDED
Jump to a specific package <package-name>
Jump directly to a Failed package @PACKAGE (FAILED): <package-name>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request revamps the parallelization output in the Kokoro system test script, renaming 'passed' packages to 'succeeded' and adding detailed log outputs for succeeded packages. It also introduces temporary configurations and a dummy test in google-cloud-dns to verify mixed test results. Feedback on these changes suggests removing the temporary experimental configurations and dummy test before merging. Additionally, it is recommended to avoid printing detailed logs for succeeded packages to prevent excessively large and noisy CI/CD build logs, opting instead to list only their names.

Comment thread .kokoro/system.sh
Comment thread .kokoro/system.sh
Comment thread packages/google-cloud-dns/tests/system/test_system.py Outdated
@chalmerlowe chalmerlowe changed the title feat: revamp parallel test outputs for searchability and clarity chore(parallel tests): revamp parallel test outputs for searchability and clarity Jul 30, 2026
@chalmerlowe chalmerlowe self-assigned this Jul 30, 2026
@chalmerlowe chalmerlowe added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Jul 30, 2026
@chalmerlowe

Copy link
Copy Markdown
Contributor Author

Do-not-merge added as a pre-emptive barrier to ensure that all fake tests and similar triggers are removed before merge.

@chalmerlowe
chalmerlowe marked this pull request as ready for review July 30, 2026 19:46
@chalmerlowe
chalmerlowe requested a review from a team as a code owner July 30, 2026 19:46

@daniel-sanche daniel-sanche left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, after trigger comments are removed

@chalmerlowe chalmerlowe left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing trigger comments. Reckon I am gonna leave this unmerged until Monday since this has a broad ranging the can impact troubleshooting, debugging. Would like to be there when this first starts being used broadly to handle any unforeseen issues.

Comment thread packages/google-cloud-speech/setup.py Outdated
Comment thread packages/google-cloud-translate/setup.py Outdated
Comment thread packages/google-cloud-dns/tests/system/test_system.py Outdated
Comment thread packages/google-cloud-access-approval/setup.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Indicates a pull request not ready for merge, due to either quality or timing.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants