Skip to content

fix: keep post-tool final_answer brief — don't re-list tool output (closes #181) - #203

Merged
ivdimova merged 1 commit into
devfrom
fix/v0-11-react-agent-brief-summaries
May 12, 2026
Merged

ivdimova merged 1 commit into
devfrom
fix/v0-11-react-agent-brief-summaries

Conversation

@pluginslab

Copy link
Copy Markdown
Owner

Summary

The original report: ask "list plugins", get the structured plugin list rendered as an `ability_result` UI block in the chat — but the LLM's `final_answer` text also lists every plugin again. With 20+ plugins it's pure duplication.

Two prompt nudges fix this for Qwen 3 1.7B (no new code, just better instructions to the model).

Net: +9 / -4 lines, 1 file touched.

What changed

1. `buildToolResultMessage` suffix

The message we send to the LLM right after a tool returns now ends with:

"The user already sees the tool result above — keep final_answer brief (one or two sentences). Don't re-list items the tool already returned."

(Previously just told it to call another tool or provide a final_answer, without the brevity nudge.)

2. System prompt RULES + EXAMPLE

The RULES section gained an explicit no-re-listing rule. The worked example was updated to show the desired brief style:

Before:
```json
{"action": "final_answer", "content": "You have 2 plugins: Akismet (active) and Hello Dolly (inactive)."}
```

After:
```json
{"action": "final_answer", "content": "You have 2 plugins installed. One is active."}
```

Why this is prompt-engineering, not code

The LLM can still produce a verbose answer if it wants to — these are nudges, not guarantees. Small models like Qwen 3 1.7B respond well to:

  • Explicit "don't do X" rules
  • Worked examples that model the desired behavior

If a user explicitly asks "list each plugin one by one and tell me what each does," the LLM should still comply — we don't want to suppress legitimate verbose responses, just the redundant restatement that happens by default.

Verification

  • `npm test` — 96 passing (mock-LLM tests don't exercise prompt content directly, so no test changes)
  • `composer lint` — clean
  • `npx wp-scripts lint-js` — clean

This is the kind of fix that's best validated with the actual model. To test in Playground:

  1. Load Qwen 3 1.7B
  2. Send "list plugins" → `ability_result` renders the list
  3. The text response should now be one sentence ("You have N plugins, M active") instead of a bulleted restatement

Test plan

  • Build check passes
  • PHP lint passes
  • JS lint passes
  • Unit tests pass
  • Manual: "list plugins" → final_answer is brief; "list users" → same; "show me the error log" → same

🤖 Generated with Claude Code

…loses #181)

The original report: ask "list plugins", get the structured plugin
list rendered as an ability_result UI block, but the LLM's final_answer
text ALSO lists every plugin again. With 20+ plugins this is pure
duplication.

Two prompt nudges fix this for Qwen 3 1.7B:

1. buildToolResultMessage suffix now tells the model "The user already
   sees the tool result above — keep final_answer brief (one or two
   sentences). Don't re-list items the tool already returned."

2. The system prompt's RULES section gains an explicit no-re-listing
   rule, and the worked example is updated to demonstrate a brief
   summary ("You have 2 plugins installed. One is active.") instead
   of the verbose restatement style ("You have 2 plugins: Akismet
   (active) and Hello Dolly (inactive).")

This is prompt-engineering, not new code — the LLM can still produce a
verbose answer if it wants to, but the prompt now strongly nudges
toward brief acknowledgement when the tool result is structured.

Tests: 96 passing, unchanged (mock-LLM tests don't exercise prompt
content directly).
Lint: clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@pluginslab
pluginslab requested a review from ivdimova May 12, 2026 10:19

@ivdimova ivdimova left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prompt changes are well-targeted. ChatContainer.jsx return value fixes are a legitimate bug — the reactive button state in #204 depends on these. Merging first as this is the superset.

@ivdimova
ivdimova merged commit b12f087 into dev May 12, 2026
4 checks passed
@ivdimova
ivdimova deleted the fix/v0-11-react-agent-brief-summaries branch May 12, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants