[TRTLLM-15376][test] expand Gemma 4 MTP QA coverage

[jieli-matrix]

Dev Engineer Review

• Updated TestGemma4_26B_A4B to use the is_sm_100f() architecture guard.
• Replaced the previous B200-only gating.
• Added TestGemma4_26B_A4B::test_nvfp4 to the QA test list.
• The QA entry uses the correct format.
• No API, performance, error-handling, configuration, or duplicate-entry issues were identified.

QA Engineer Review

• Modified TestGemma4_26B_A4B.
• Added TestGemma4_26B_A4B::test_nvfp4.
• The test is covered in tests/integration/test_lists/qa/llm_function_core.txt.
• CBTS selection coverage data is unavailable.
• Verdict: needs follow-up.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 37381275-36fb-4658-8b40-1558cb657bc0

📥 Commits

Reviewing files that changed from the base of the PR and between b08deaa and d62d5e6.

📒 Files selected for processing (2)

• tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py
• tests/integration/test_lists/qa/llm_function_core.txt

🚧 Files skipped from review as they are similar to previous changes (2)

• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The PR updates Gemma 4 26B A4B multimodal test gating to use the SM100-family device check and adds the NVFP4 accuracy test to the QA test list.

Changes

Gemma 4 multimodal QA

Layer / File(s)	Summary
Update test gating and QA registration tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py, tests/integration/test_lists/qa/llm_function_core.txt	The test imports and uses is_sm_100f() for device gating. The skip reason documents the shared-KV MTP FlashInfer TRTLLM-Gen requirement. The NVFP4 test is added to the QA list.

Estimated code review effort: 2 (Simple) | ~5 minutes

Merge Risk: ⚪ Minimal · up to d62d5

This localized change expands Gemma 4 MTP QA coverage without altering production behavior, and no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: aswinvisva, stanleysun639

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The required sections are present, but the Description and Test Coverage sections contain no substantive information. The checklist is marked as reviewed, but it does not replace the missing explanati…	Add a concise description of the issue and solution. List the relevant tests, including the Gemma 4 MTP multimodal accuracy test and its execution gating or coverage changes.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the ticket, change type, and primary change: expanding Gemma 4 MTP QA coverage.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The required sections are present, but the Description and Test Coverage sections contain no substantive information. The checklist is marked as reviewed, but it does not replace the missing explanation and test details.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[jieli-matrix]

/bot skip --comment "Only add qa tests"

[tensorrt-cicd]

PR_Github #69277 [ skip ] triggered by Bot. Commit: 35797a9 Link to invocation

[crazydemo]

Please run /bot run --post-merge to check the changes, along with the cbts selection logic

[crazydemo]

recheck the pytest marker, remove the over-deisgned part

[tensorrt-cicd]

PR_Github #69277 [ skip ] completed with state SUCCESS. Commit: 35797a9
Skipping testing for commit 35797a9

Link to invocation

[jieli-matrix]

/bot run --post-merge

[crazydemo]

LGTM

[tensorrt-cicd]

PR_Github #69377 [ run ] triggered by Bot. Commit: 147f879 Link to invocation

[tensorrt-cicd]

PR_Github #69377 Bot args parsing error: CI requested by --post-merge or an approval-gated stage selector (*, *Post-Merge*, or *PerfSanity*) requires the ci: post-merge approved PR label. Ask a member of NVIDIA/trt-llm-ci-approvers to apply it.

Link to invocation

[jieli-matrix]

/bot run

[tensorrt-cicd]

PR_Github #69384 [ run ] triggered by Bot. Commit: 147f879 Link to invocation

[github-actions]

Removed the "ci: post-merge approved" label because @crazydemo could not be verified as an active member of NVIDIA/trt-llm-ci-approvers. Ask a member of that team to apply it.

[tensorrt-cicd]

PR_Github #69384 [ run ] completed with state FAILURE. Commit: 147f879
/LLM/main/L0_MergeRequest_PR pipeline #56722 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Ask a member of NVIDIA/trt-llm-ci-approvers to add the label, then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[jieli-matrix]

/bot run --extra-stage "DGX_B200-PyTorch-Post-Merge-1,DGX_B200-PyTorch-Post-Merge-2"

[tensorrt-cicd]

PR_Github #69575 [ run ] triggered by Bot. Commit: 82fc696 Link to invocation

[tensorrt-cicd]

PR_Github #69575 [ run ] completed with state SUCCESS. Commit: 82fc696
/LLM/main/L0_MergeRequest_PR pipeline #56893 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jieli-matrix]

/bot run --extra-stage "DGX_B200-PyTorch-Post-Merge-1,DGX_B200-PyTorch-Post-Merge-2"

[tensorrt-cicd]

PR_Github #69641 [ run ] triggered by Bot. Commit: 82fc696 Link to invocation

[tensorrt-cicd]

PR_Github #69641 [ run ] completed with state SUCCESS. Commit: 82fc696
/LLM/main/L0_MergeRequest_PR pipeline #56947 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[jieli-matrix]

/bot run --extra-stage "DGX_B200-PyTorch-Post-Merge-1,DGX_B200-PyTorch-Post-Merge-2"

[tensorrt-cicd]

PR_Github #69857 [ run ] triggered by Bot. Commit: d62d5e6 Link to invocation

[tensorrt-cicd]

PR_Github #69857 [ run ] completed with state SUCCESS. Commit: d62d5e6
/LLM/main/L0_MergeRequest_PR pipeline #57147 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[jieli-matrix]

/bot run --stage-list "DGX_B200-PyTorch-Post-Merge-1,DGX_B200-PyTorch-Post-Merge-2"

[tensorrt-cicd]

PR_Github #69940 [ run ] triggered by Bot. Commit: d62d5e6 Link to invocation