Skip to content

fix(e2e): keep localdns reply-direction NOTRACK rules allowlisted - #9258

Open
Ganeshkumar Ashokavardhanan (ganeshkumarashok) wants to merge 2 commits into
mainfrom
ganesh/e2e-restore-localdns-reply-notrack-allowlist
Open

fix(e2e): keep localdns reply-direction NOTRACK rules allowlisted#9258
Ganeshkumar Ashokavardhanan (ganeshkumarashok) wants to merge 2 commits into
mainfrom
ganesh/e2e-restore-localdns-reply-notrack-allowlist

Conversation

@ganeshkumarashok

@ganeshkumarashok Ganeshkumar Ashokavardhanan (ganeshkumarashok) commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind bug
/kind cleanup

What this PR does / why we need it

Agentbaker GPU E2E has been red on main since #9247 merged. All 18 real-node GPU scenarios (A100 x2, H100, A10 x2, NC, RTXPro6000, GridDriver, NoDriver, NvidiaDevicePlugin, MIG, DRA) fail on the same assertion from ValidateIPTablesCompatibleWithCiliumEBPF:

Rule in table raw did not match any pattern:
-A OUTPUT -s 169.254.10.10/32 -p tcp -m comment --comment "localdns: skip conntrack" -m tcp --sport 53 -j NOTRACK
...
Rules found that do not match any of the given patterns.

Root cause

#9247 reverted #9230. That revert removed the reply-direction NOTRACK rules from localdns.sh and removed the matching entry from the e2e iptables allowlist. Those two removals do not take effect at the same time:

  • localdns.sh is baked into the VHD by the packer file provisioner (vhdbuilder/packer/*.json). It is not delivered through CSE custom data. A change to it only reaches a node when a new VHD is published.
  • The e2e allowlist lives in the test code and takes effect on the next pipeline run.

So every VHD published while #9230 was on main still installs the reply-direction rules, and those VHDs stay in support for 6 months — but the test stopped accepting them immediately.

Timeline (UTC):

When What
08-17 22:44 ea63b859a0 (#9230) adds reply rules + allowlist entry
08-18 08:29 VHD 1.1787040551.32430 published — rules baked in
08-18 18:44 build 177153237 green
08-19 01:51 7716637881 (#9247) reverts both
08-19 02:00+ builds 177223847, 177234019, 177266063 red

main may partially self-heal as post-revert VHDs roll out, but any run pinned to VHD_BUILD_ID in the 08-18 → 08-19 window, and any release branch still on those images, stays red. The allowlist is also required again the moment #9230 is re-landed.

Changes

e2e/validation.go — restore the reply-direction pattern in the raw table. The allowlist describes what a node is permitted to have, not what main currently installs, so it must cover the whole supported VHD window.

e2e/validators.go — extract the matching loop into iptablesRuleMatchesAnyPattern and stop discarding the regexp.MatchString error. Previously a malformed allowlist entry silently degraded to "no match", which surfaces as a confusing "unsupported iptables rule" failure on every scenario rather than pointing at the bad pattern. The loop also assigned to the outer matched variable; the extraction removes that hazard. Matching semantics (regex OR literal substring, blank patterns skipped) are unchanged — note in particular that a pattern which is not valid regex syntax still gets its literal-substring chance, since the allowlist is deliberately fed rules pasted out of iptables -S (a comment like --comment "kube-proxy (v1.31" is not a valid regex). The compile error is only surfaced when the rule went unmatched overall, i.e. exactly when the malformed pattern is the likely culprit.

.github/workflows/go-test.ymle2e/ is a separate Go module, so make test in the root module never reaches it and check-coverage.yml explicitly filters it out. Without this, the new tests could only fail inside the ADO E2E pipeline they exist to protect, and could not block the PR causing the regression. Added an e2e-unit-test job modelled on the existing version-consistency job in validate-components.yml, scoped to tests needing neither Azure credentials nor network.

e2e/validation_iptables_test.go (new) — pure unit tests, no Azure resources needed:

  • every global and per-table allowlist pattern compiles
  • both request- and reply-direction LocalDNS rules are allowlisted (the regression guard)
  • the allowlist still fails closed on an unknown rule, so the guard can't be satisfied by an overly broad pattern
  • table-driven coverage of the new helper, including the invalid-pattern error path

Does this PR introduce a user-facing change?

NONE

Testing / verification

Product behaviour is unchanged — this only touches e2e test code.

$ cd e2e && go build ./... && go vet ./...
$ go test -run 'Test_iptables' -v ./
--- PASS: Test_iptablesAllowlist_patternsCompile
--- PASS: Test_iptablesAllowlist_localdnsRules
--- PASS: Test_iptablesAllowlist_rejectsUnknownRule
--- PASS: Test_iptablesRuleMatchesAnyPattern (5 subtests)
ok  github.com/Azure/agentbaker/e2e

Negative check — deleting the restored pattern reproduces the pipeline failure exactly:

--- FAIL: Test_iptablesAllowlist_localdnsRules
    LocalDNS rule is not allowlisted for eBPF host routing: -A OUTPUT -s 169.254.10.10/32 -p tcp ... --sport 53 -j NOTRACK
    (x4, matching the 4 unmatched rules per node in build 177266063)

Refs #9230, #9247. ADO build 177266063 (definition 443878, Agentbaker GPU E2E).

PR #9247 reverted #9230, which removed the reply-direction localdns NOTRACK
rules from localdns.sh *and* removed the matching entry from the eBPF
host-routing iptables allowlist in e2e. Those two things do not take effect at
the same time.

localdns.sh is baked into the VHD by the packer `file` provisioner, it is not
delivered through CSE custom data. So every VHD published while #9230 was on
main still installs the reply-direction rules, and those VHDs remain in support
for 6 months. The e2e allowlist, by contrast, changed the moment the revert
merged. The result is that ValidateIPTablesCompatibleWithCiliumEBPF (run from
ValidateCommonLinux) fails on any scenario booting one of those VHDs:

    Rule in table raw did not match any pattern:
    -A OUTPUT -s 169.254.10.10/32 -p tcp -m comment --comment "localdns: skip conntrack" -m tcp --sport 53 -j NOTRACK

That took out all 18 real-node GPU scenarios in Agentbaker GPU E2E build
177266063 (A100, H100, A10, NC, MIG, DRA, device-plugin, ...), starting with
build 177223847, the first run after the revert landed.

Changes:

* e2e/validation.go: restore the reply-direction pattern in the "raw" table.
  The allowlist describes what a node may legally have, not what main currently
  installs, so it has to cover the whole supported VHD window. This also means
  the entry is already in place if #9230 is re-landed.

* e2e/validators.go: extract the matching loop into
  iptablesRuleMatchesAnyPattern and stop discarding the regexp.MatchString
  error. Previously a malformed allowlist entry silently became "no match",
  which surfaces as a confusing "unsupported iptables rule" failure on every
  scenario instead of pointing at the bad pattern.

* e2e/validation_iptables_test.go: unit tests that assert every allowlist
  pattern compiles, that both localdns rule directions are allowlisted, that
  the allowlist still fails closed for an unknown rule, and that the helper
  behaves as documented. Removing the restored pattern makes
  Test_iptablesAllowlist_localdnsRules fail with the exact rules seen in the
  pipeline, so this is a real regression guard.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   13 suites   1m 21s ⏱️
404 tests 404 ✅ 0 💤 0 ❌
407 runs  407 ✅ 0 💤 0 ❌

Results for commit ebc1453.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restores compatibility for supported VHDs containing LocalDNS reply-direction NOTRACK rules and improves allowlist diagnostics.

Changes:

  • Restores the LocalDNS reply-rule allowlist pattern.
  • Extracts matching logic with explicit regex error handling.
  • Adds regression and helper unit tests.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
e2e/validation.go Restores reply-direction NOTRACK allowlisting.
e2e/validators.go Adds reusable, error-aware rule matching.
e2e/validation_iptables_test.go Adds allowlist regression and matching tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Addresses two review findings.

1. The previous commit changed behaviour for patterns that are not valid
   regular expressions. regexp.MatchString returns (false, err) on a compile
   error, so the original inline loop fell through to the strings.Contains
   check and such a pattern could still match literally. Returning early on the
   error removed that fallback, which matters because this allowlist is
   deliberately fed rules pasted out of `iptables -S` -- a comment such as
   --comment "kube-proxy (v1.31" is not valid regex syntax.

   iptablesRuleMatchesAnyPattern now records the first compile error but keeps
   evaluating, so the literal-substring path still runs. The error is only
   returned when the rule went unmatched overall, which is exactly the case
   where the malformed pattern is the likely culprit. Three test cases cover
   this: literal match despite an invalid regex, no error when a later pattern
   matches, and the error surfacing when nothing matches.

2. e2e is a separate Go module, so `make test` in the root module never runs
   it and check-coverage.yml explicitly filters it out. The new tests could
   therefore only fail inside the ADO E2E pipeline they exist to protect, and
   could not block the PR that caused the regression. Added an e2e-unit-test
   job to the Go Unit Tests workflow, following the existing precedent in
   validate-components.yml, scoped to tests that need neither Azure
   credentials nor network.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@ganeshkumarashok

Copy link
Copy Markdown
Contributor Author

E2E evidence: the fix works

Agentbaker GPU E2E build 177368786 ran against this PR. Comparing it to the last main run before the fix, build 177266063:

main (177266063) this PR (177368786)
Failed tests 18 2
Occurrences of Rules found that do not match any of the given patterns every failure 0
Test_Ubuntu2204_GPUA100 Failed Passed
Test_Ubuntu2404_GPU_H100 Failed Passed

Every one of the 18 failures on main was the iptables allowlist assertion. That signature is completely absent from the PR run — grep -c "do not match any of the given patterns" over all three task logs returns 0.

The 2 remaining failures are unrelated infrastructure flakes

Both are Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG{,_MultiGPU}, and neither reaches validation — they fail during VM creation:

bastionssh.go:323: Attempt 1/5 SSH handshake failed: ssh: handshake failed: failed to get reader: context deadline exceeded
...  (5/5 attempts, across 3 re-runs)
test_helpers.go:102: FAIL: create vmss "...": failed to start bastion tunnel

Test_Ubuntu2404_GPU_A100 shows as skipped for an unrelated reason too — vmss.go:683: skipping scenario SKU quota exceeded.

The other two pipelines are likewise unrelated to this change: Agentbaker E2E failed on Test_RCV1P_ACL and Test_Ubuntu2204_ArtifactStreaming_TrustedLaunch; Agentbaker Windows E2E failed on Test_Windows2022_VHDCaching_LegacyTLSBootstrap — this PR only touches Linux e2e validation code, so it cannot affect Windows.

All three pipelines have been re-queued (GPU 177378215, E2E 177378257, Windows 177378268).

@ganeshkumarashok

Copy link
Copy Markdown
Contributor Author

Re-run results — the two non-GPU pipelines are now explained

Pipeline Re-run Result
Agentbaker E2E 177378257 succeeded
Agentbaker GPU E2E 177378215 1 failure (was 18 on main)
Agentbaker Windows E2E 177378268 pre-existing main failure

GPU E2E: 18 → 1, and the iptables signature is still gone

grep -c "do not match any of the given patterns" over all task logs of the re-run: 0.

The single remaining failure is Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG_MultiGPU, which never reaches validation — it fails during VM creation:

FAIL: create vmss ...: handshake failed: failed to get reader: context deadline exceeded

Test_Ubuntu2404_NvidiaDevicePluginRunning_MIG — which failed in the first run for the same bastion reason — passed on the re-run, confirming this is an infrastructure flake rather than a code issue.

Windows E2E failure is pre-existing on main

Test_Windows2022_VHDCaching_LegacyTLSBootstrap also fails on main in build 177273329. This PR changes only Linux e2e validation code (e2e/validation.go, e2e/validators.go) plus a test file and a GitHub workflow, so it cannot affect Windows provisioning.

GPU E2E queued once more as 177384709 to try to clear the bastion flake.

@ganeshkumarashok

Copy link
Copy Markdown
Contributor Author

Final CI evidence across three GPU E2E runs

The iptables regression this PR fixes is gone in every run. The residual failures are an Azure Bastion infrastructure problem that is also present on main, so GPU E2E cannot go fully green right now for reasons unrelated to this change.

Build Branch Failing tests do not match any of the given patterns handshake failed: failed to get reader
177266063 main (pre-fix) 18 present in all 18 10
177368786 this PR 2 0 21
177378215 this PR 1 0 18
177384709 this PR 4 0 6

Two independent signals:

  1. The allowlist assertion never fires again. 18 → 0, sustained across three runs.
  2. The bastion timeout is not new. It occurs 10 times in the pre-fix main build, before this branch existed. Every residual failure dies at create vmss ... failed to start bastion tunnel, i.e. before the node is ever provisioned, so ValidateCommonLinux — and therefore the code this PR touches — is never reached.

The set of affected tests also shuffles between runs (MIG failed then passed; NvidiaDevicePluginRunning passed twice then failed), which is the signature of an environment flake rather than a deterministic code fault.

Other pipelines:

  • Agentbaker E2E 177378257 — ✅ succeeded
  • Agentbaker Windows E2ETest_Windows2022_VHDCaching_LegacyTLSBootstrap fails identically on main (177273329); this PR touches no Windows code
  • All 23 GitHub checks pass, including the new e2e-unit-test job, actionlint and zizmor

@ganeshkumarashok

Copy link
Copy Markdown
Contributor Author

Does this re-introduce what #9247 reverted? No — and here is the proof

Reasonable objection, so stating it explicitly. #9247 removed four things. This PR restores exactly one of them, and it is the only one with permissive rather than prescriptive semantics.

Removed by #9247 (7716637881) Effect Restored here?
localdns.sh — emission of -s <ip> --sport 53 -j NOTRACK product behaviour ❌ no
localdns_spec.sh — shellspec assertions for reply rules test ❌ no
ValidateLocalDNSConntrackRules (e2e/validators.go) requires reply rules exist (test "$response_rule_count" = "4") ❌ no
allowlist regex in getIPTablesRulesCompatibleWithEBPFHostRouting permits reply rules if present yes, only this

git diff --name-only origin/main...HEAD touches e2e/validation.go, e2e/validators.go, e2e/validation_iptables_test.go, .github/workflows/go-test.yml. localdns.sh is not among themgrep -c "sport 53" parts/linux/cloud-init/artifacts/localdns.sh returns 0 on this branch, exactly as on main. ValidateLocalDNSConntrackRules does not exist anywhere in this branch.

The polarity is the whole point

An allowlist entry cannot cause a node to install a rule. It only says "if a node has this rule, do not fail it." Restoring it is incapable of re-enabling #9230's product behaviour.

What actually triggered the revert was the other validator

Build 177145051 (main, inside the #9230 window, 18 failures):

localdns should install request and response direction NOTRACK rules   → 85 occurrences
expected 4 response-direction localdns NOTRACK rules
"do not match any of the given patterns"                                →  0 occurrences

So #9230 broke CI via ValidateLocalDNSConntrackRules, which required rules that VHDs did not yet have. The revert correctly removed that requirement — and this PR does not bring it back.

This PR fixes the mirror image of that same VHD-skew bug: the test now forbids rules that VHDs still do have. The two fixes are complementary, not opposed:

Both are needed because localdns.sh ships only in the VHD image, so test-side expectations must tolerate every VHD in the 6-month support window — in both directions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants