AE3/AE4 analyze binary assets the ledger already marks opaque_content (85-byte PNG -> HIGH, DO_NOT_INSTALL) #413

Summary

A skill containing a single 85-byte PNG scores 55/100, HIGH, DO_NOT_INSTALL with --no-llm, despite having no scripts, no executables, and eight lines of markdown. Three findings fire, two of them against the PNG's decoded bytes.

The notable part is that the same report already knows the file is unreadable. The completeness ledger records:

Ledger exceptions
  - opaque_content assets/diagram.png: Artifact contents could not be fully interpreted.

So one subsystem classifies the artifact as opaque and excludes it, while AE3 and AE4 analyze its decoded content anyway and report findings at 90% and 80% confidence. Those two conclusions cannot both be right.

This is the same class as #144 (closed COMPLETED 2026-06-24), which covered binary content reaching static pattern analyzers via MP2. The MP2 path appears fixed; the AE* analyzers were not covered by that fix.

Environment

• SkillSpector 2.9.6, installed from main at 698e2bf29c7d32aa8211ada677382460c01900d7
• Windows 11, Python 3.14, --no-llm (static only)

Minimal reproduction

mkdir -p binary-repro/assets
cat > binary-repro/SKILL.md <<'EOF'
---
name: binary-repro
description: A skill that ships one small PNG as reference material.
---

# Binary repro

Describe the diagram in assets/diagram.png to the user.
EOF

# 85-byte 1x1 PNG
python3 -c "import base64; open('binary-repro/assets/diagram.png','wb').write(base64.b64decode('iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAHElEQVQoz2NgGAWjYBSMglEwCkbBKBgFo2AUAAAHkgABfXRPtQAAAABJRU5ErkJggg=='))"

skillspector scan binary-repro --no-llm

Actual result

 Risk Assessment
 Score           55/100
 Severity        HIGH
 Recommendation  DO NOT INSTALL

Issues (3)
  HIGH:   AE1 - Referenced artifact was not completely inspected...  SKILL.md:8            conf 100%
  HIGH:   AE3 - Text artifact contains embedded NUL bytes...         assets/diagram.png:3  conf 90%
  MEDIUM: AE4 - Suspicious Unicode normalization or mixed-script...  assets/diagram.png:1  conf 80%

 Inspection Completeness
 Coverage              50.0%
 Entirely uninspected  1
Ledger exceptions
  - opaque_content assets/diagram.png: Artifact contents could not be fully interpreted.

Fifteen analyzers additionally report degraded under Limitations.

Note the component inventory lists assets/diagram.png as Lines: 8 — the PNG is being line-counted as text, which is where the NUL-byte and mixed-script observations come from.

Expected result

A PNG shipped as reference material is not a security finding. Specifically:

• An artifact the ledger classifies as opaque_content should be excluded from content analyzers, not analyzed and reported on. Whatever check produces the ledger exception should gate the analyzers, so the two cannot disagree.
• AE3 "embedded NUL bytes" is a tautology for any binary format. NUL bytes are evidence a file is binary, not evidence it is malicious.
• AE4 mixed-script detection over bytes that were never text has no meaningful signal.
• If AE1 is intended to flag genuinely uninspectable references, a recognized binary asset type is not the case it should fire HIGH at 100% confidence on.

Impact

Severity drives the score, so a single HIGH sets the verdict regardless of finding count. Any skill shipping a screenshot, diagram, or icon — a common and entirely legitimate pattern — inherits DO_NOT_INSTALL. For anyone using the MCP server to gate installs (scan_skill → safe_to_install), this blocks benign skills automatically.

It also degrades the tool's signal in the direction that matters most: users who see DO_NOT_INSTALL on a skill they know is fine learn to discount the verdict, which is the failure mode #144's original report described.

General rule

Worth applying beyond these two pattern IDs: an analyzer should never emit a content finding about an artifact the inspection ledger has already classified as uninterpretable. The ledger is the tool's own record of what it could and could not read — treating it as authoritative for analyzer applicability would close this whole class in one place, rather than per-pattern as it is discovered.

That framing may also cover #367 (incomplete skip denylist, CWE-184) and relates to #389 (coverage computed but never affecting the exit code): coverage, ledger exceptions, and analyzer applicability are three views of the same question, and they currently disagree.

Related

• [Bug] Binary/PDF files trigger mass false positives in static pattern analysis #144 — binary/PDF false positives via MP2 (closed COMPLETED, same class, different analyzers)
• [SECURITY] SC8 presence check does not fix incomplete skip denylist (CWE-184 / CWE-693) (same class as __pycache__ still applies elsewhere) #367 — incomplete skip denylist
• Coverage is computed but never affects the exit code: a partially-blind scan is indistinguishable from a complete one in CI #389 — coverage computed but never affects exit code
• Not Git sample hooks make safe skills fail SC9 #411 — that one is .git/hooks/*.sample; this repro deliberately has no git repository

Happy to test a fix against this repro, or against a wider corpus of skills shipping images if that is useful.