Skip to content

static_patterns_tool_misuse: common bash idioms trigger static_parse_limit, forcing partial scans and a HIGH AE1 finding #628

Closed

Description

@raphabruno7

Summary

has_bounded_parse_exhaustion in static_patterns_tool_misuse.py reports parse exhaustion on ordinary, benign bash scripts once they are longer than ~4 KB. The scan is then marked partial with static_parse_limit, and AE1 ("Incomplete referenced artifact analysis") is raised at HIGH with confidence 1.0. A script with no destructive command at all cannot reach SAFE. Longer real-world skills with bash helpers typically end up at DO_NOT_INSTALL, because the AE1 points stack with the other findings.

Version: SkillSpector v2.12.0 (installed via uv tool install git+https://github.com/NVIDIA/skillspector.git), --no-llm and claude_cli provider both affected.

Minimal end-to-end repro

repro/
  SKILL.md
  scripts/run.sh

SKILL.md:

---
name: repro
description: Minimal repro.
---
Run `scripts/run.sh`.

scripts/run.sh has 3 lines of code, followed by ~5.5 KB of comment padding:

#!/bin/bash
out="$(date)"
echo "$out"
# padding line 1 ............................................
# ... (90 comment lines in total)
$ skillspector scan repro --no-llm
score 25, CAUTION, coverage 50%, status partial
  ledger: static_parse_limit scripts/run.sh
  AE1 HIGH (confidence 1.0) SKILL.md:5  scripts/run.sh (partial)

Change only out="$(date)" to the equivalent out=$(date):

score 0, SAFE, coverage 100%, status complete

Isolated triggers

Calling has_bounded_parse_exhaustion(content, lambda: None) directly, where pad(n) is n # characters plus a newline:

content result
x="$(y)"\n + pad(4093) False
x="$(y)"\n + pad(4094) True
x=$(y)\n + pad(6000) False
v="a (b)"\n + pad(6000) True
v="a b"\n + pad(6000) False
v="|$a|"\n + pad(6000) True
v=":$a:"\n + pad(6000) False

So there are three independent triggers, each one active only when more than ~4 KB (_SHELL_COMMAND_WORD_CHARS = 4096) of content follows it:

  1. A quoted command substitution in any position: an assignment x="$(cmd)", an argument printf '%s' "$(f "$x")", a test [ "$(f)" = true ], or a here-string <<< "$(f)".
  2. A literal ( or ) inside a double-quoted string, e.g. "deleted in $sha ($date)".
  3. A literal | inside a double-quoted string, e.g. seen="|$item|".

It looks like the command-word tokenizer does not track double-quote context in these cases. A quote after the substitution, paren or pipe is treated as the start of a new word, and the unterminated "word" then runs to the end of the file and exceeds the budget. The padding can be comments. They are scanned too, so a comment that mentions "$(cmd)" triggers it as well.

Impact

  • These are idiomatic, recommended bash forms. ShellCheck (SC2046/SC2086) actively pushes authors toward quoting "$(...)".
  • Because of the fail-closed rule in report.py, a partial scan can never be SAFE, and AE1 alone adds 25 points. Any skill that ships a bash helper over ~4 KB is penalized no matter what the script contains.
  • In the project where I found this, a 12 KB read-only audit script plus its test suite caused the scan to report 100 / CRITICAL / DO_NOT_INSTALL. After I rewrote the scripts to avoid the three forms above, with byte-identical output and no behavioral change, the same skill scans 0 / SAFE / complete. That rewrite is available for reference: raphabruno7/drift-doc@947adf1

Expected

Double-quoted strings, and command substitutions inside them, should be tokenized as a single word. Those three forms should not exhaust the parser budget, or at least the exhaustion should not be attributed when no destructive command word (rm, etc.) is actually present in the unresolved span.

Environment

  • SkillSpector v2.12.0
  • macOS 26 (arm64), Python 3.14 via uv
  • Reproduced with --no-llm and with SKILLSPECTOR_PROVIDER=claude_cli

Activity

  1. rng1995 commented on Sep 28, 2026

    @rng1995
    Collaborator

    Implementation is in progress in PR #634, which tracks ownership of successfully parsed shell words to avoid revisiting quoted assignment content and producing false parse-limit failures. It retains incomplete coverage for genuinely unresolved or over-budget work. The PR remains open and unmerged.

  2. Spectorian commented on Sep 29, 2026

    @Spectorian
    Collaborator

    The same false partial-analysis result also occurs with ordinary Python, HTML and JavaScript syntax. Direct calls to has_bounded_parse_exhaustion returned True for:

    Python:

            cmd = ["timeout", str(timeout)] + cmd
    

    HTML:

        <button class="btn btn-export" onclick="exportEvalSet()">Export Eval Set</button>
    

    JavaScript:

    const name = `graph_${blocks.length + 1}`;
    

    For the HTML case, the exact scanner input has four leading spaces and contains the class attribute. The minimized <button onclick="exportEvalSet()">Export</button> returned False; the full input matters. The Python probe has eight leading spaces.

    These inputs were parsed, not executed. The helper's false exhaustion result feeds partial-coverage reporting; a containing skill's final recommendation also depends on its other findings.

    Expected behavior: ordinary host-language syntax should not count as exhausted shell inspection. Continue detecting embedded executable substitutions, dangerous commands and actual budget exhaustion, with accurate source locations and coverage reporting.

    PR #634 handles quoted shell assignments, PR #627 handles Markdown ownership, and PR #626 handles larger JSON. The Python, HTML and JavaScript cases above remain outside those fixes. Existing scoring and completeness rules should stay unchanged.

    Relevant code

    static_patterns_tool_misuse.py:54.

  3. rng1995 commented on Oct 4, 2026

    @rng1995
    Collaborator

    Merged-scope update: PR #634 fixes the originally reported quoted-shell-assignment ownership cases, and PR #686 additionally handles value-only parameter expansions. The focused shell-quote, parameter-expansion, and ledger regressions pass on current main.

    This does not claim the broader host-language cases added in the later comment are resolved: the supplied Python, HTML, and JavaScript helper probes still report bounded parse exhaustion on 33c9afe. Further non-shell/source ownership cases remain tracked in #694; Markdown and larger-JSON work remains open in PR #627 and PR #626. Recording the scope of the automatic closure, not treating those follow-up cases as fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions