Take filtered context from a bounded window next to each seed (fixes #1712) - #1713

Purpose of the change

Fixes #1712. On PostgreSQL, a context read with a property filter could read every row on the seed's side of its partition and sort it: the planner has no statistics for property predicates, estimates about one match per seed, and plans a bitmap over that side instead of an ordered walk that stops at the context size. Requests took seconds on large partitions.

Description

• Filtered context comes from a bounded window. With a property filter, get_segment_contexts takes each seed's context on each side from the 1,000 segments nearest it on that side, read with no filter; the filter and the context size apply over those rows (_context_rows_query). The inner read has no filter for the planner to misestimate, so it stays an ordered index scan and stops once the context is found or the window ends. No segment further than 1,000 segments from its seed is context, filtered or not; the SegmentStore ABC now allows an implementation-defined bound on every context read. The bound is fixed in the store; callers do not choose it.
• Unfiltered reads keep their statement, which already reads exactly the context size; the bound caps that size at 1,000, which needs no window and costs nothing.
• SQLite takes the same window, so both dialects return the same context. Its per-seed reads now build each direction's statement once per request and bind each seed's position (first commit): building a statement costs more than SQLite takes to run it, so building one per seed and direction dominated the read. The SQLite table below shows the first commit on its own.
• Both dialects build their context statements in _context_rows_query: the walk past the seed on one side, the partition's liveness check, and, with a property filter, the window over the walk. They differ only in where the seed's position comes from: the seeds subquery's columns on PostgreSQL, bound parameters on SQLite. On PostgreSQL the liveness check moves from the seeds subquery into the walk, where the plan evaluates it once per statement; SQLite's statements are byte-identical to before.
• PostgreSQL loads context rows through the ORM, as SQLite does, instead of passing ten keyword arguments per row to SegmentRow's constructor, as main does.

Measurements

Measured on 2026-09-25 with this PR at 38d255252 against main (whose segment store is unchanged since 66d4a653d, #1661): 20 seeds per request, 2 segments back and 4 forward unless noted, the rows of 21 partitions interleaved on disk; property filters matching 20% and 1% of segments. p50 per request with one client, requests/s with concurrent clients. Arms alternated, on AC power, while another session's unit tests shared the CPU (1-minute load 2.3-6.8). This PR ran twice per table and the medians are shown; main ran twice on SQLite and once on PostgreSQL, where its filtered reads take seconds, and a morning run of main on a quiet host agreed within 10% on those rows.

PostgreSQL 16 (128 MB shared buffers), one 1,000,000-segment partition and twenty 20,000-segment ones:

On small PostgreSQL partitions, where main's plan is already cheap, this PR is as fast or faster. Medians of five runs of main and four of this PR; main first, this PR second:

The window adds a subquery to a filtered statement, but PostgreSQL now loads context rows through the ORM instead of building each by hand (6c7f41f6b), so building a request's two statements takes 0.95-0.98 ms with a filter against main's 0.88-0.96 ms, and 0.46-0.48 ms without one against 0.77-0.80 ms.

SQLite, one 200,000-segment partition and twenty 20,000-segment ones; p50 in ms with one client and with four, for main, this PR's first commit, and this PR:

SQLite's planner already walked the index in order, so there the window costs 1.0-1.25x over the first commit. With one client this PR is as fast as main or faster in every row; with four, a 1% filter is 3-13% slower than main. What the window buys on SQLite is the same context as PostgreSQL and a bounded worst case: with a property that only the 20 seeds carry, a request takes 37 ms instead of 613-617 ms on the 200,000-segment partition (and 36 ms instead of 70-78 ms on a 20,000-segment one), because the walk no longer runs toward the end of the partition.

For the same seeds, both dialects returned the same context as main in every scenario but one, where the bound applies: 12 forward with the 1% filter, which matches every 100th segment, returns the 10 matches within 1,000 segments.

Choosing the window's size

The bound is a cost limit that keeps what filters ask for, not a relevance cutoff. The window counts the segments the walk passes, so how much it keeps depends on whether the walk stays inside the seed's conversation.

The walk stays inside the seed's conversation. It does in this store when a partition holds one conversation, and in #1684's schema, which keeps many conversations in a partition with a session column the walk follows. On a real store of coding-agent sessions in #1684's schema (PostgreSQL, 20 seeds, 4 back and 12 forward), a 1,024-segment window returned what the current statement returns for a user-and-assistant-message filter (15.2 of 16 neighbors per seed) and 94% of it for filters on user messages alone, in less time than the current statement. A smaller window cuts sparse filters off sooner: 128 returned half as much for user messages alone.

The walk crosses conversations. It does in this store when a partition holds many conversations: with no session column, the conversation is only a filter, and the walk passes other conversations' segments. #1684 is meant to replace that layout. On a snapshot of such a store (SQLite, 917,610 segments from 2,039 conversations in one partition; 5,000 random message seeds), a 1,000-segment window kept this share of what an unbounded walk within the seed's conversation returns:

On that store, the unbounded filtered walk (that store's own build of the segment store, the same walk as #1661's) ran to the partition's start or end for 7.5-9.2% of seeds: whenever the seed's conversation had fewer matches left on a side than requested, it read on through every other conversation's segments. The worst seeds took 1.0-1.5 s; typical ones 1.0-1.5 ms at p50. The window stops such a walk after 1,000 segments, about 1-1.6 ms at the 1.1-1.6 µs per segment measured there.

The window is 1,000, the round number next to the 1,024 measured; nothing depends on a power of two.

Commits

Branched from main at 66d4a653d, where #1661 merged; dd9851a4c merges main at d57f5cb36 (#1671 and #1541), which touches none of this PR's files. This PR is its twenty-three commits: 5a4addc3c, 97c708767, 9055c2ad8, ac96db241, ab58e5072, 9f5a5c8cf, 040f3fd55, e13a259a0, d47909804, b0ff12e49, 35414350a, f92524a9a, 74e91b678 and 4b6d25e39, then after the merge 155a5a523, 745c67921, 6c7f41f6b, 08420008f, 38d255252, 7189e656a, 9aac74f77, 862f15ae2 and dddaecf86.

Verification

At each of the first fourteen commits, on main at 66d4a653d: ruff check and ruff format --check clean; ty check clean as CI runs it (uv run --frozen --all-extras ty check --project packages/server); the full server suite without integration tests passes, 1982 tests at the first commit, 1983 from the second (the window's test) and 1986 from d47909804 (the model test). The event memory and long-term memory integration tests pass against PostgreSQL at 35414350a and 74e91b678, 107 with 3 skipped.

The window's test sets the bound to 4 segments and checks that a match beyond it is not context and that an unfiltered read stops at it; it fails on both dialects with the window's LIMIT off by one or removed and with the unfiltered cap removed. d47909804 adds test_random_context_reads_agree_with_a_model: random context reads, several seeds at a time, with filters of every node type, many segments sharing a timestamp, and another partition's segments at the same times, compared with an in-memory model of the timeline; every partition is far smaller than the bound. The existing context tests pass with the lateral's walk left uncorrelated to its seed and with ties ordered by the wrong columns; the new test fails on both dialects with ties misordered, with the window ordered the same way on either side, and with the filter resolved against the table instead of the window, and on PostgreSQL with the walk uncorrelated. It passes on both dialects at #1661's last commit, so outputs there and here agree wherever every match is within the bound.

The refactors change no results. ab58e5072 builds each statement's walk in _context_rows_query: SQLite's context statements compile byte-identical to before it, on PostgreSQL only the liveness check moves (the plan evaluates it once per statement), and p50s stay within 0.2 ms of ac96db241's over five alternating rounds. Before and after 040f3fd55 (keyword-only flags and bounds) and b0ff12e49 (statements built from Core columns, SQLite rows loaded through select(SegmentRow).from_statement()), every context statement compiles identically on both dialects; b0ff12e49 cut the Python that builds a filtered PostgreSQL request's statements from 1.69-1.71 ms to 1.13-1.24 ms, measured on 2026-09-25 before 6c7f41f6b lowered it further. 35414350a annotates with the documented ColumnCollection instead of the undocumented ReadOnlyColumnCollection. f92524a9a caps unfiltered reads at the same bound. 9055c2ad8, 9f5a5c8cf, e13a259a0, 74e91b678 and 4b6d25e39 rename a helper or change only comments and docstrings; ac96db241 sets the bound to 1,000.

After the merge: 155a5a523 builds SQLite's statements only for the sides a read asks for; 745c67921 had each model-test read set the bound at random, down to one segment, and 7189e656a reverts it: the bound's size is not part of the contract and bounds of a few segments are not realistic use, while the window's test already catches the bound's failures and the model test, without a bound, the multi-seed one (a walk uncorrelated to its seed on PostgreSQL); 6c7f41f6b loads PostgreSQL's context rows through the ORM. 6c7f41f6b, against the commit before it in one quiet window: p50 at one client 14-22% lower with a filter on the 1M-segment partition and on the 20k one with the 20% filter, and requests/s at 16 clients 26-39% higher in every scenario, with identical contexts. 08420008f resolves filter fields against columns only, and 38d255252 reads SQLite's context one direction at a time, as PostgreSQL does; both leave the SQL unchanged on both dialects (SQLite runs the same statements, all backward before forward), and against 6c7f41f6b in the measurement window above, SQLite p50s are within 3% either way and PostgreSQL's equal or lower, with identical contexts. At 745c67921, 6c7f41f6b and 38d255252: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes, 1994 tests (the merge brought in #1541's); the event memory and long-term memory integration tests pass against PostgreSQL, 107 with 3 skipped. At 08420008f: ruff, ty and the event memory tests without integration (193) pass. At 7189e656a (test file only): ruff and ty clean, and the segment store tests pass, 101 on SQLite and 104 against PostgreSQL. 9aac74f77 drew the model test's identifiers from its seeded generator, and 862f15ae2 reverts it: the seeded generator already makes every run test the same cases, and identifiers stay random UUIDs; at 862f15ae2 the test file is as at 7189e656a, and the segment store tests pass, 101 on SQLite and 104 against PostgreSQL. dddaecf86 gives the model test's events random UUIDs in sorted order, so events sharing a timestamp come in the same order every run and every run tests the same cases; at it, ruff and ty are clean, the segment store tests pass (101 and 104), and the model test still fails on both dialects with ties misordered, with the window ordered the same way on either side and with the filter resolved against the table, and on PostgreSQL with the walk uncorrelated.

🤖 Generated with Claude Code