Skip to content

error-log-read: fails to summarize when debug.log has many entries #107

Description

@pluginslab

Problem

When WP_DEBUG is enabled and debug.log contains entries, the agent returns "I had trouble understanding how to help" instead of showing the log contents.

Reported by @wpbullet in #92.

What works

  • Selection: Tool selection now routes correctly (fixed in PR fix: wpbullet ability selection regressions + 23 new test cases #102, test passes)
  • PHP backend: Code is solid — reads last N lines via SplFileObject, handles all edge cases (missing file, debug disabled, empty log)
  • Small results: When debug.log is empty or debug is disabled, the ability works (small JSON response)

Root cause

The failure happens on the second LLM iteration — after the tool executes and returns data. With a populated debug.log, the tool returns up to 50 lines of PHP errors. Even after truncation to 2000 chars (maxToolResultLength), the combined context of:

  • System prompt (~800 tokens with 31 tools listed)
  • User message
  • First LLM response (tool_call JSON)
  • Tool result (2000 chars of error log entries)
  • JSON format reminder suffix

…likely exceeds the 4096-token context window of Qwen 3 1.7B, causing the model to produce malformed output that fails JSON parsing → triggers the "I had trouble understanding" fallback (line 273 of react-agent.js).

Proposed fixes

A. Use interpretResult to pre-summarize before sending to LLM

The interpretResult function already exists in the JS ability and produces a concise summary like "Found 50 error entries (out of 150 total lines)." Instead of sending the raw truncated JSON to the LLM, use interpretResult to compress the result before the second iteration. This would drastically reduce token usage.

This is potentially a general improvement for all abilities — using interpretResult as a pre-filter for the LLM's tool result message, especially on the 1.7B model with its tight context window.

B. Reduce default line count for 1.7B context

The PHP backend defaults to 50 lines. For a 4096-token model, even 10 lines of verbose PHP errors can be too much after truncation. Consider:

  • Reducing the default to 20 lines
  • Or dynamically adjusting based on the model's context size

C. Smarter truncation in the tool result

Instead of blindly truncating at 2000 chars (which may cut mid-entry), truncate at entry boundaries and include a count: "Showing 5 of 50 entries: [entry1], [entry2], ..."

Files involved

  • src/extensions/services/react-agent.js — buildToolResultMessage() (line 774), maxToolResultLength config
  • src/extensions/abilities/error-log-read.js — interpretResult() already exists
  • includes/abilities/error-log-read.php — default line count

Scenarios from #92

Scenario Expected Actual
No WP_DEBUG, no debug.log "No debug.log found" "I had trouble understanding" (selection fail, now fixed)
No WP_DEBUG, empty debug.log Debug disabled message Works correctly
WP_DEBUG on, populated debug.log Show error entries "I had trouble understanding" (context overflow)

Refs

Refs #92 (selection part fixed in PR #102, this covers the result processing)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions