fix: fold transcript entries incrementally instead of replaying each turn - #715

What & why

With several long-running agent turns (e.g. one orchestrator with 10 subagents), the daemon stalled: steer/queue messages sat on "Sending…", "Clear all" and Fork/Continue never returned, and resuming a session failed with ACP initialize handshake failed … context deadline exceeded.

Root cause: every event append in sessiondb.writeEventBatch re-projected the whole active assistant entry. It re-read every event already assigned to the turn (loadAssignedEvents) and re-decoded each one through secret redaction (ProjectAssignedEntry). A turn is a single transcript entry, so per-append cost grew with turn length. On real session data:

~83% of that was regex redaction inside decodeStoredEvent, which also redacted the parsed event twice. Ten subagents in long turns pinned all 16 cores (load avg 61), and session write transactions were held for 30–157 s (sqlite write transaction held for at least one second … writeEventBatch). Every request that appends to a session waits on that session's FIFO writer: the steer's user_message, the queue-clear marker, the stop/cancel markers. That is why they hung. claude-agent-acp then could not answer initialize within 30 s on the starved machine; that step normally takes ~1 s.

The fix:

• transcript.EntryFold: an incremental reduction of one assistant entry, equivalent to ProjectAssignedEntry, which now uses it, so there is a single implementation.
• The session writer keeps up to two folds (writer-goroutine owned, no lock). Each append checks the cached fold against the durable assigned-event prefix (GetTranscriptEntryEventBounds: count + max sequence) and applies only events after it (ListEventsForTranscriptEntryAfter). If the check fails (rolled-back write, generation change, out-of-order event), it falls back to a full replay. The cache is reset on archive, rewind, and clear.
• decodeStoredEvent no longer redacts parsed a second time; parseEvent already redacts on every path.

How you verified it

• go test ./internal/transcript/... ./internal/store/sessiondb/... (also with -race) and go test ./internal/session/... ./internal/api/...: 4,643 tests pass. golangci-lint on the changed packages: 0 issues.
• New TestSessionDBTranscriptProjection/Should_keep_entries_appended_one_event_at_a_time_equal_to_a_full_replay: a long turn written one event per transaction (thoughts, chunked text, tool calls, missing results, plus a late tool result routed back to the completed entry) must equal transcript.BuildProjection of the same log. Mutating the fold to drop its tail makes it fail.
• BenchmarkSessionDBAppendLongAssistantTurn (an append to a 600-event turn): 927 ms/op on main → 25 ms/op (seeding the turn went from 288 s to 8.5 s).
• Real data, from three sessions of the incident (1,217 / 1,080 / 739 events), replayed one append at a time into a fresh SessionDB:
  • all 1,217 events in 32 s, ~57 ms per append at the end of the turn (vs ~8 s on main);
  • the incremental projection equals a full BuildProjection for every entry;
  • the full-projection SHA-256 is identical between main and this branch for all three sessions, so removing the duplicate redaction changes no bytes. The full rebuild itself is 1.4–2× faster.

Impact

No impact on CLI commands, HTTP/UDS routes, config.toml keys, or web surfaces. Internal only. Two package-private sqlc queries were added to the session DB; there is no schema or migration change and transcript output is byte-identical. No docs update needed.

Known residual, left for a follow-up: an append still rewrites the whole entry row (multi-MB JSON for very long turns, ~20–60 ms), because the transcript stores one row per assistant turn. Changing that is a storage-contract change.

• make gate passes locally; this PR is delivered only after its required CI checks are green
• New or changed behavior is covered by tests, or I explained above why not
• If an agent wrote or co-wrote this, I named it above and verified the result myself: written by Claude Code (Opus 5.5) and verified with the tests, benchmark, and real-data replays above

🤖 Generated with Claude Code

Summary by CodeRabbit

• Performance
  • Transcript entries are updated incrementally as events are recorded, reducing repeated processing in long conversations.
• Bug Fixes
  • Late-arriving tool results are reflected in the correct transcript entry.
  • Parsed events continue to be redacted when stored events are decoded.