Skip to content

LLM finding schemas emit JSON-schema minimum/maximum, which some structured-output endpoints reject #132

Closed

Description

@wernerkasselman-au

Summary

The structured-output response models for LLM analysis, LLMFinding (src/skillspector/llm_analyzer_base.py) and MetaAnalyzerFinding (src/skillspector/nodes/meta_analyzer.py), express their numeric ranges with Pydantic ge/le (start_line >= 1, confidence in [0.0, 1.0]). Pydantic emits those as JSON-schema minimum / maximum. Some OpenAI-compatible structured-output and tool-calling endpoints validate the response schema and reject those keywords, so the LLM call fails before any finding is returned.

Impact

On an affected endpoint, every LLM-backed analyzer (semantic discovery and the meta-analyzer filter/enrich pass) fails the structured-output call, so --llm runs produce no LLM findings even though static analysis still works.

Proposed fix

Drop the ge/le Field bounds, convey the ranges in the field descriptions, and enforce them with runtime field_validators that clamp into range. The emitted schema then carries no minimum / maximum, while the in-code [0, 1] / >= 1 guarantee is unchanged.

Raised while splitting the bounded-reads PR (#19), where @rng1995 asked for this to land as its own change with rationale.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions