Repository navigation
fix(cli): print recursive SARIF to stdout and keep status text off it - #565
Merged
Merged
Conversation
A recursive scan with --format sarif and no --output printed only the human-readable progress and summary to stdout; the merged SARIF log was written only when --output was set. Other status lines (the multi-skill and no-sub-skill advisories, --verbose "Running scan...", and transitive failure warnings) were also printed to stdout ahead of SARIF, because NVIDIA#467 routed them to stderr for JSON only. Treat SARIF like JSON for stdout routing, and print the merged recursive SARIF log when no --output is given. Co-Authored-By: Claude Opus 5 <[email protected]> Signed-off-by: kevin9327 <[email protected]>
rng1995
approved these changes
Sep 16, 2026
rng1995
left a comment
Collaborator
There was a problem hiding this comment.
[SkillSpector Review]
Approved at commit 8e1882b.
I traced the affected single-skill, recursive, verbose, and transitive paths. JSON and SARIF are now treated consistently as machine-readable output: status text is routed away from stdout, recursive SARIF without --output emits one bounded and validated log, and the existing file-output, exit-code, cleanup, and completeness behavior is preserved.
The added tests cover the reported regressions, including aggregate SARIF output and transitive failure warnings. All current CI jobs pass.
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
skillspector scan <collection> --recursive --format sarif > results.sarifdoes not produce a SARIF file. stdout contains only the human-readable progress and summary, and the merged SARIF log is never printed:The
--helptext for--outputsays "If not specified, prints to stdout." #467 fixed this for--format json(combined document to stdout, status text to stderr). SARIF is the other format tools parse from stdout, and it was left on the old routing, so several single-skill paths also print status text ahead of the log:--no-llm --format sarif)mainscan ./multi --recursiveMulti-skill directory detected: 2 skills found(and no SARIF follows)scan ./multi(two sub-skills, no--recursive)Warning: Found 2 skills in this directory. Use --recursive to scan eachscan ./no-skill-md --recursiveWarning: --recursive specified but no sub-skills detected. Scanning as singlescan ./skill --verboseRunning scan...scan ./skill --transitivewhen a child scan fails (reproduced through the unit test below)Warning: Transitive scan failed for https://github.com/...In each case
json.loads(stdout)fails at character 0.Fix
_MACHINE_READABLE_FORMATS = {json, sarif}and use it at the five places that currently checkformat == FormatChoice.jsonto send status text to stderr._scan_multi_skill, build the merged SARIF log whether or not--outputis set. With--outputit is written to the file as before; without it, it is printed. This mirrors what fix(cli): emit recursive JSON reports to stdout #467 did for the JSON branch.--outputbehaviour, the SARIF content, the output bound and exit codes are unchanged. Markdown and terminal routing are untouched.Tests (
tests/unit/test_cli.py)test_recursive_single_skill_advisory_does_not_pollute_stdoutandtest_multi_skill_advisory_does_not_pollute_stdout: the existing JSON tests from fix(cli): emit recursive JSON reports to stdout #467, now parametrized overjsonandsarif(thejsonin their names was dropped).test_recursive_sarif_without_output_writes_only_the_log_to_stdout: recursive, verbose, no--output. stdout must passvalidate_sarif_reportand contain the aggregate run; progress, "Running scan" and the summary must be on stderr.test_transitive_failure_warning_stays_off_sarif_stdout: SARIF variant oftest_transitive_resolver_failure_preserves_direct_report.Against unmodified
cli.py(Windows 11, Python 3.12.10,mainat4148ab3):With the fix:
6 passed. Every changed line runs in these tests (checked with branch coverage).I also re-ran the first four rows against the real CLI. After the fix, stdout passes
validate_sarif_reportin every row (for./multi --recursiveit holds 3 runs: two skills plus the aggregate), and the status text shows up on stderr.Suite
pytest -m "not integration and not provider" -p no:randomly, same machine:maintests/unit,tests/opencode,tests/test_*.pytests/nodesThe +4 are the new SARIF cases. The failures and errors are the same on both sides and Windows-only (release and
compare_scan_accuracyharnesses, CRLF fixtures, a backslash inside a POSIX file name, and parametrized test IDs over the Windows environment-variable length limit).ruff check src/ tests/: All checks passed.ruff format --check src/ tests/: 246 files already formatted.🤖 Generated with Claude Code