Repository navigation
fix: keep post-tool final_answer brief — don't re-list tool output (closes #181) - #203
Merged
Merged
Conversation
…loses #181) The original report: ask "list plugins", get the structured plugin list rendered as an ability_result UI block, but the LLM's final_answer text ALSO lists every plugin again. With 20+ plugins this is pure duplication. Two prompt nudges fix this for Qwen 3 1.7B: 1. buildToolResultMessage suffix now tells the model "The user already sees the tool result above — keep final_answer brief (one or two sentences). Don't re-list items the tool already returned." 2. The system prompt's RULES section gains an explicit no-re-listing rule, and the worked example is updated to demonstrate a brief summary ("You have 2 plugins installed. One is active.") instead of the verbose restatement style ("You have 2 plugins: Akismet (active) and Hello Dolly (inactive).") This is prompt-engineering, not new code — the LLM can still produce a verbose answer if it wants to, but the prompt now strongly nudges toward brief acknowledgement when the tool result is structured. Tests: 96 passing, unchanged (mock-LLM tests don't exercise prompt content directly). Lint: clean. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The original report: ask "list plugins", get the structured plugin list rendered as an `ability_result` UI block in the chat — but the LLM's `final_answer` text also lists every plugin again. With 20+ plugins it's pure duplication.
Two prompt nudges fix this for Qwen 3 1.7B (no new code, just better instructions to the model).
Net: +9 / -4 lines, 1 file touched.
What changed
1. `buildToolResultMessage` suffix
The message we send to the LLM right after a tool returns now ends with:
(Previously just told it to call another tool or provide a final_answer, without the brevity nudge.)
2. System prompt RULES + EXAMPLE
The RULES section gained an explicit no-re-listing rule. The worked example was updated to show the desired brief style:
Before:
```json
{"action": "final_answer", "content": "You have 2 plugins: Akismet (active) and Hello Dolly (inactive)."}
```
After:
```json
{"action": "final_answer", "content": "You have 2 plugins installed. One is active."}
```
Why this is prompt-engineering, not code
The LLM can still produce a verbose answer if it wants to — these are nudges, not guarantees. Small models like Qwen 3 1.7B respond well to:
If a user explicitly asks "list each plugin one by one and tell me what each does," the LLM should still comply — we don't want to suppress legitimate verbose responses, just the redundant restatement that happens by default.
Verification
This is the kind of fix that's best validated with the actual model. To test in Playground:
Test plan
🤖 Generated with Claude Code