Skip to content

Feature/gemini support and cli ux - #7

Merged
rng1995 merged 10 commits into
NVIDIA:mainfrom
saeed-vayghan:feature/gemini-support-and-cli-ux
Sep 16, 2026
Merged

rng1995 merged 10 commits into
NVIDIA:mainfrom
saeed-vayghan:feature/gemini-support-and-cli-ux

Conversation

@saeed-vayghan

Copy link
Copy Markdown
Contributor

Feature: Gemini Support & CLI UX Improvements

Overview

This PR adds support for gemini-3.5-flash and significantly improves the CLI by adding real-time visual progress tracking for long scans.

Key Changes

  • Live Progress & File Tree: Added a real-time progress bar and a visual file discovery tree so users aren't left waiting on a blank screen.
  • Clean JSON Exports: Safely routed all progress UI to stderr, ensuring stdout remains perfectly clean when using --format json.
  • Gemini Support: Officially registered gemini-3.5-flash to prevent token limit errors and added a setup example to the README.md.
  • Cleanup: Suppressed noisy Pydantic warnings, cleaned up error stack traces, and added report* to .gitignore.

Testing

  • Run skillspector scan ./my-skill/ to see the new progress bar.
  • Run skillspector scan ./my-skill/ --format json > out.json to verify the JSON output remains uncorrupted.

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes — the CLI streaming rewrite itself is correct and the stdout/stderr split is a genuine improvement, but two concrete issues should be fixed before merge, and bundling several unrelated changes makes this harder to review/revert.

What's good ✅

  • The non-verbose streaming path is functionally sound: result accumulates exactly the keys the CLI consumes downstream — report_body/sarif_report (_write_result), temp_dir_for_cleanup (_cleanup_result), and risk_score (exit code). It keys on output presence rather than node names, so it stays robust to graph changes. total_steps = 4 + len(ANALYZER_NODE_IDS) matches the actual non-analyzer nodes (resolve_input, build_context, meta_analyzer, report), and ANALYZER_NODE_IDS is exported. Existing tests/unit/test_cli.py runs the default (non-verbose) path, so output correctness is covered.
  • Routing the progress UI / file tree to a stderr console so stdout stays clean for --format json is the right call.

Please address (blocking)

  1. .gitignore: report* is unanchored. A gitignore entry without a leading slash matches that basename at any depth, so report* also matches src/skillspector/nodes/report.py and will silently ignore any future report* source/test file. Please anchor to the generated artifacts, e.g. /report* at repo root, or better, target the actual outputs (/report*.json, /report*.sarif).
  2. Trailing whitespace fails ruff check. pyproject.toml selects the W ruleset (only E501 is ignored), and the new CLI block adds ~8 blank lines with trailing whitespace (W293). Please strip them so the change is lint-clean.

Non-blocking suggestions

  • Scope/bundling: this PR mixes Gemini model-registry entries, the CLI UX rewrite, a .gitignore change, and an exception-logging tweak. Splitting them would make review and rollback safer.
  • warnings.filterwarnings("ignore", category=UserWarning, module="pydantic") is a process-wide, persistent filter. Consider a scoped with warnings.catch_warnings(): around the graph run (or narrowing the message) so unrelated warnings aren't hidden.
  • mcp_tool_poisoning.py TP4 handler: switching from exc_info=True to ": %s" drops the traceback. For a security analyzer, keeping exc_info=True (optionally gated on debug level) preserves debuggability when the LLM check fails.
  • nv_build/model_registry.yaml: adding gemini-3.5-flash to the nv_build provider registry reads as semantically off, since Gemini is reached via the OpenAI-compatible path; harmless for token budgeting but potentially misleading.
  • Minor: the .gitignore (and a couple of the YAML files) end without a trailing newline; the inline imports in the else block could move to module top.

Thanks for the work here — happy to re-review once the gitignore pattern and the lint issues are addressed.

@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

@rng1995 Thanks for the thorough review! I've addressed everything in two commits:

Blocking issues (fixed):

  • .gitignore: Anchored the pattern to repo root with specific extensions — now /report*.md, /report*.json, /report*.sarif. Source files like report.py are no longer matched.
  • Trailing whitespace: Stripped all 8 blank lines with hidden spaces. ruff check passes clean now.

Non-blocking suggestions (also fixed):

  • Scoped warnings: Wrapped the filterwarnings call inside with warnings.catch_warnings(): so it only applies during the graph run, not process-wide.
  • TP4 exc_info: Restored exc_info=True so we keep the full traceback when the LLM check fails.
  • Gemini in nv_build: Removed gemini-3.5-flash from the nv_build registry. It stays in the openai/ and root registries where it belongs.
  • Inline imports: Moved all inline imports (warnings, rich.progress, rich.tree, ANALYZER_NODE_IDS) to module level. Ruff I001 passes.
  • Trailing newlines: All files now end with a proper newline.

On scope/bundling: Good point — I'll keep this in mind for future PRs. For this one, the changes are already pushed as separate commits so they can be reverted independently if needed.

Ready for re-review when you get a chance.

@saeed-vayghan
saeed-vayghan force-pushed the feature/gemini-support-and-cli-ux branch from 3591d39 to 17883c1 Compare June 23, 2026 21:24
@saeed-vayghan
saeed-vayghan requested a review from rng1995 June 23, 2026 21:31

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Re-review: both blockers resolved — approving.

  1. .gitignore is now anchored to repo root with specific extensions (/report*.md|json|sarif), so src/skillspector/nodes/report.py is no longer matched.
  2. The trailing-whitespace block in cli.py is cleaned (ruff W293).

All non-blocking items were also handled: the Pydantic warnings filter is now scoped via with warnings.catch_warnings() (no longer process-wide), the TP4 exc_info change was reverted (file no longer in the diff), and gemini-3.5-flash moved out of the nv_build registry into the OpenAI-compatible path. The non-verbose streaming path still accumulates exactly the keys the CLI consumes and routes UI to stderr. Minor cosmetic nit: .gitignore has no trailing newline. Approving.

@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

@rng1995 It seems for some reason I have no access to Merge Button. Could you please do it?

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — approving. Since my last review the substantive addition is the non-verbose streaming progress UI (graph.stream + Rich progress/file-tree), plus the Gemini-via-OpenAI-compat example and registry entry. The streaming path correctly accumulates the keys the CLI consumes downstream (report_body, sarif_report, risk_score, temp_dir_for_cleanup), and the pydantic warning suppression is scoped to the run.

Non-blocking nits:

  • import warnings is inserted before import sys in cli.py (not import-sorted) — likely trips the ruff format/lint check; run ruff format/ruff check --fix.
  • .gitignore is missing a trailing newline.
  • Consider a small test that the non-verbose path produces the same result keys as the verbose (graph.invoke) path, so the two can't drift.

@saeed-vayghan saeed-vayghan left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments have been reviewed and implemented.

@saeed-vayghan
saeed-vayghan requested a review from rng1995 July 2, 2026 13:23

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Re-review: approved. The PR-authored change after the prior approval is the requested stream/invoke parity integration test plus import-order/newline cleanup. It exercises all four CLI-consumed result keys, and the current diff preserves the already-approved stderr streaming path and Gemini registry/docs behavior.

@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

@rng1995 Thank you for review.
It still shows me this message: "1 workflow awaiting approval - This workflow requires approval from a maintainer."

Who else needs to approve and merge?

rng1995
rng1995 previously approved these changes Aug 12, 2026
@rng1995

rng1995 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

@saeed-vayghan - I've approved the PR. Please fix CI issues, so we can merge the PR.

@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

@saeed-vayghan - I've approved the PR. Please fix CI issues, so we can merge the PR.

Hi — I pushed a fix for the DCO sign-off and ruff lint failures on this branch. Could someone with write access approve the pending CI run? Thanks!

rng1995
rng1995 previously approved these changes Aug 14, 2026

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review approved at the current head. The prior review feedback remains addressed and there are no unresolved review threads. Merge remains blocked: the merge-updated head fails formatting and nine unit tests around CLI JSON/exit-code behavior, plus DCO; the owner must repair CI before merge.

@rng1995

rng1995 commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator

@saeed-vayghan Re-review is approved, but please fix the current CI regressions before merge. Ruff would reformat src/skillspector/cli.py and tests/integration/test_stream_invoke_parity.py. The merge-updated head also fails 9 unit tests: test_sc8_single_pyc_blocks_install_and_cli_exit; test_cli_writes_report_then_exits_two_for_execution_failure; test_cli_scan_slack_p6_pe3_regression; test_cli_scan_required_table_keeps_malicious_pe3; test_cli_baseline_generate_then_scan_round_trip; test_cli_scan_excludes_selected_baseline_inside_skill; test_cli_scan_excludes_only_the_selected_baseline; test_recursive_single_skill_scan_still_accepts_baseline; and test_cli_scan_json_preserves_single_skill_contract. The failures cluster around CLI JSON output and exit-code behavior after merging current main. The latest merge commit also needs a DCO Signed-off-by trailer. Please push the fixes and get every required check green; I will not merge until then.

@saeed-vayghan
saeed-vayghan force-pushed the feature/gemini-support-and-cli-ux branch from cee6013 to b2197eb Compare August 14, 2026 21:05
@saeed-vayghan
saeed-vayghan force-pushed the feature/gemini-support-and-cli-ux branch from 5febbb3 to f82d19e Compare August 14, 2026 21:13
@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

@rng1995 It says you need to review again, every time this branch gets rebased some thing breaks. Hope this time would be working.

@saeed-vayghan
saeed-vayghan force-pushed the feature/gemini-support-and-cli-ux branch from f82d19e to 4f1b9e3 Compare August 20, 2026 10:28

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SkillSpector Review]

Re-review approved. The new corrective commit propagates execution_successful through the default streaming path, restores the incomplete-execution exit-code contract, and updates JSON assertions for Click 8.2's stdout/stderr behavior. Focused regressions and all required checks pass.

@rng1995

rng1995 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

@saeed-vayghan - Thanks for your patience and contribution. CI looks good but merge conflicts need to be fixed. Please fix it in order to merge the PR.

@mohgupta-ship-it

mohgupta-ship-it commented Aug 25, 2026 •

Copy link
Copy Markdown
Member

Powered by Codex: PR council review result.

This is a triage signal, not a maintainer approval.

  • Rating: new feature
  • Confidence: medium
  • Status read: Approved, green checks, conflicting
  • Review method: fresh GitHub metadata/body/files/reviews/checks plus selected diffs; council lenses were spec fit, dead-code/reachability, YAGNI/scope, design/coupling, and code standards/tests.
  • Council assessment: Gemini support and CLI UX changes are approved/green, but branch is stale and overlaps provider/CLI evolution.
  • Recommended action: Rebase and keep only still-relevant delta.

@rng1995
rng1995 enabled auto-merge (squash) September 16, 2026 19:56
@rng1995
rng1995 disabled auto-merge September 16, 2026 19:56
This commit adds 'gemini-3.5-flash' to the model registries across various providers to prevent token-limit warnings. It also updates the exception handling in mcp_tool_poisoning to gracefully catch non-fatal LLM check failures without printing noisy tracebacks.

Signed-off-by: Saeed <[email protected]>
@rng1995

rng1995 commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the changes @saeed-vayghan ! Sorry for the prolonged delay. I've rebased all changes and will merge your PR. Thanks for your contributions!

saeed-vayghan and others added 9 commits September 16, 2026 13:06
…overy

- Swapped the synchronous graph invoke for a streamed execution when verbose mode is off.

- Integrated rich.progress to provide immediate feedback to users during long-running security scans.

- Added a file tree breakdown right after the context is built, so users know exactly which files are being picked up for analysis.

- Explicitly suppressed Pydantic's serializer warnings that were cluttering the terminal output during structured LLM generation.

- The UI now actively ticks off the analyzer rules as they finish, removing the guesswork when waiting on slower LLM checks.

Signed-off-by: Saeed <[email protected]>
- Provided a clear snippet for using the Gemini API via the OpenAI compatibility layer.

Signed-off-by: Saeed <[email protected]>
- Added `report*` to .gitignore to prevent accidentally committing local scan reports.

Signed-off-by: Saeed <[email protected]>
…in cli.py

- .gitignore: anchor report* to repo root with specific extensions
  (/report*.md, /report*.json, /report*.sarif) so source files like
  src/skillspector/nodes/report.py are no longer silently ignored
- cli.py: strip trailing whitespace from blank lines to pass ruff W293

Signed-off-by: Saeed <[email protected]>
…mini entry

- cli.py: replace process-wide warnings.filterwarnings() with scoped
  warnings.catch_warnings() so Pydantic suppression only lives during
  the graph run; move inline imports to module level (ruff I001 clean)
- mcp_tool_poisoning.py: restore exc_info=True on TP4 exception handler
  to preserve full tracebacks when LLM check fails
- nv_build/model_registry.yaml: remove gemini-3.5-flash — Gemini is
  accessed via OpenAI-compatible path, not NVIDIA build endpoints;
  entry remains in openai/ and root registries where it belongs

Signed-off-by: Saeed <[email protected]>
… outputs and clean up imports in cli.py

Signed-off-by: Saeed <[email protected]>
…output/stdout split in tests

The default (non-verbose) scan path streams graph updates via
graph.stream() and only copied a fixed set of scalar fields into the
CLI result. execution_successful was missing from that set, so a
failed execution could never trigger exit code 2 outside --verbose
mode. Track it alongside the other accumulated fields.

Click 8.2 changed CliRunner.Result.output to mix stdout and stderr
instead of proxying stdout, breaking tests that parsed JSON from
result.output for commands that also write progress/tree output to
stderr. Switch those assertions to result.stdout, and update two
tests that mocked graph.invoke/graph directly with a stream()
implementation matching the CLI's actual non-verbose execution path.

Signed-off-by: Saeed <[email protected]>
@rng1995
rng1995 force-pushed the feature/gemini-support-and-cli-ux branch from 3875661 to 3dd12b5 Compare September 16, 2026 20:39

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after rebase onto current main: approved. Conflict resolution preserves current CLI, transitive-analysis, and completeness contracts; streamed scans retain the full graph state; terminal output is escaped and bounded; and all local regression gates plus fresh CI pass.

@rng1995
rng1995 merged commit 4148ab3 into NVIDIA:main Sep 16, 2026
6 checks passed
@saeed-vayghan

Copy link
Copy Markdown
Contributor Author

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants