Skip to content

fix(security): normalize concealed instruction text - #408

Merged
mohgupta-ship-it merged 20 commits into
mainfrom
codex/security-text-normalization
Aug 31, 2026
Merged

fix(security): normalize concealed instruction text#408
mohgupta-ship-it merged 20 commits into
mainfrom
codex/security-text-normalization

Conversation

@Spectorian

@Spectorian Spectorian commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • normalize security-analysis text with version-pinned default-ignorable handling
  • reconstruct bounded inter-character separator runs with exact gap signatures and raw source-offset attribution
  • add a scored analysis-evasion signal for security-relevant condensed terms that cannot be reconstructed safely
  • treat confirmed inter-character obfuscation as a nonfatal interpretation limit so strict CLI and MCP consumers fail closed
  • preserve benign multilingual text, emoji, punctuation, notation, URLs, e-mail addresses, tables, and code through negative controls

Validation

  • sanitized static-only coverage for ASCII whitespace, punctuation, symbols, middle dot, and bullet separators
  • analyzer, SARIF location, cross-window, resource-limit, and false-positive regressions
  • completeness parity across reports, default and strict CLI modes, and MCP install verdicts
  • branch-wide non-integration/non-provider suite
  • Ruff lint, format, targeted mypy, and git diff --check

@rng1995
rng1995 marked this pull request as ready for review August 24, 2026 18:15

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the normalizer still has two fail-open concealed-instruction paths. Each current-head repro yields only the raw view, no P1/AE6 finding, risk 0, SAFE, complete analysis, and safe_to_install=True.

Comment thread src/skillspector/artifacts.py Outdated
Comment thread src/skillspector/artifacts.py
Signed-off-by: Nir Paz <npaz@nvidia.com>
@mohgupta-ship-it

mohgupta-ship-it commented Aug 25, 2026

Copy link
Copy Markdown
Member

Powered by Codex: five-lens GPT-5.6-sol council correction on current head c0b233a.

  • Rating: critical fix
  • Disposition: NEEDS_CHANGES
  • Critical gap: yes — additional current-head evasion paths still publish SAFE / safe_to_install=True.
  • CI/live state: all five hosted checks are green; mergeable is true, but review state is blocked.
  • Council method: independent spec/regression, security/trust-boundary, runtime/architecture, standards/YAGNI, and focused-test lenses; an evidence-only judge then adjudicated reproduced claims.

New, non-duplicate findings:

  1. Mixed adjacent/separated text (i g n o re ...) and digit-interleaving both bypass the reconstruction/AE6 boundary and remain complete + SAFE.
  2. Mixed separators plus a Greek confusable remain LOW/SAFE while the ASCII equivalent is AE6/PARTIAL.
  3. Benign algebra/spelling that condenses to an allowlisted word becomes HIGH AE6 and incomplete.

The earlier allowlist issue remains tracked in its existing discussion and is not repeated here. New anchored review: #408 (review)

Recommended action: do not merge until the false-SAFE cases fail closed and the benign notation controls remain SAFE/complete.

Signed-off-by: Nir Paz <npaz@nvidia.com>
Signed-off-by: Nir Paz <npaz@nvidia.com>
@Spectorian

Copy link
Copy Markdown
Collaborator Author

Fixes pushed. please review and confirm.

@mohgupta-ship-it mohgupta-ship-it left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Powered by Codex: inline blocker from delta re-review on current head c0b233a.

Comment thread src/skillspector/nodes/analyzers/artifact_integrity.py

@mohgupta-ship-it mohgupta-ship-it left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Powered by Codex: five-lens GPT-5.6-sol delta council — REQUEST CHANGES on c0b233a. These are new findings beyond the already-tracked allowlist issue: two additional false-SAFE reconstruction paths, a confusable parity failure, and a confirmed benign false positive.

Comment thread src/skillspector/artifacts.py
Comment thread src/skillspector/nodes/analyzers/artifact_integrity.py Outdated
Comment thread src/skillspector/nodes/analyzers/artifact_integrity.py
Signed-off-by: Nir Paz <npaz@nvidia.com>
@Spectorian

Copy link
Copy Markdown
Collaborator Author

Fixes pushed. please review and confirm.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-review: the two previously requested normalization paths are now covered and their focused regressions pass, but this head is not ready to approve. I still reproduce no finding for i g n o re previous instructions. and digit-interleaved i0g0n0o0r0e ..., while benign spelling/math forms such as r e s t r i c t i o n s and b + y + p + a + s + s emit AE6. These current fail-open/false-positive issues are already anchored in live review threads, and the branch also conflicts with main.

mohgupta-ship-it and others added 4 commits August 30, 2026 17:32
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
fix(security): detect declared-marker obfuscation

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it
mohgupta-ship-it force-pushed the codex/security-text-normalization branch from 032bf53 to 958ace2 Compare August 30, 2026 15:24
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it
mohgupta-ship-it force-pushed the codex/security-text-normalization branch from 958ace2 to 41eea29 Compare August 30, 2026 19:01
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it

mohgupta-ship-it commented Aug 30, 2026

Copy link
Copy Markdown
Member

Rebase and security-fix update

Powered by Codex

PR #408 is rebased on current origin/main and now points to ede4176. GitHub reports the branch as conflict-free and mergeable.

What the follow-up fix closes

  • Replaces runner-dependent shell timing assertions with deterministic bounds on scan calls, cumulative examined span, cache reuse, and runtime callbacks.
  • Makes pinned default-ignorable lookup constant-time and skips semantically identical Unicode gap runs without changing source-offset, emoji, keycap, newline, or targeted-filler behavior.
  • Preserves P1 detection and line 2 attribution for 300,000-character U+FE0F and mixed default-ignorable cross-window attacks.
  • Prevents same-line P2 format-control floods from exhausting the finding budget before higher-value P1 analysis, while retaining one P2 signal per affected logical line and continuing after safe emoji ZWJ sequences.
  • Keeps the production 30-second limit unchanged and retains fail-closed PARTIAL behavior on forced timeouts.

Review and validation evidence

  • All six prior review discussions were answered and resolved; no unresolved review threads remain.
  • Three independent 5.6-sol council reviews returned READY with no blocker or high-severity finding.
  • Exact normalization/source-offset equivalence: 66,429 constructed strings.
  • Contextual/compact offset equivalence: 177,155 constructed strings.
  • DP accelerated-versus-reference equivalence: 333,333 constructed cases.
  • Full local CI: 3,946 passed, 14 skipped, 4 expected xfails, 0 unexpected failures.
  • Ruff, formatting, mypy, byte-compilation, DCO, and diff checks passed.
  • Fresh hosted CI: all five checks passed, including the full unit/coverage suite: https://github.com/NVIDIA/SkillSpector/actions/runs/33331997101

@rng1995, please re-review the new head. The only expected remaining merge gate is replacement of the prior CHANGES_REQUESTED review.

@mohgupta-ship-it

Copy link
Copy Markdown
Member

LGTM. We are yet to make some more minor fixes to improve the safety nets. Will be raising a separate PR

@mohgupta-ship-it
mohgupta-ship-it merged commit 539c115 into main Aug 31, 2026
5 checks passed
@mohgupta-ship-it
mohgupta-ship-it deleted the codex/security-text-normalization branch August 31, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants