Repository navigation
fix(event-backend): make expand_context return timeline-neighbor episodes - #1541
Conversation
4d003a2 to
4f62903
Compare
7dc58e1 to
6306991
Compare
edwinyyyu
changed the title
edwinyyyu
changed the title
edwinyyyu
changed the title
6306991 to
368b221
Compare
edwinyyyu
changed the title
| assert len(expanded) <= 3 | ||
|
|
||
|
|
||
| async def test_expand_context_zero_returns_matches_in_score_order( |
There was a problem hiding this comment.
test_expand_context_zero_returns_matches_in_score_order cannot fail against the pre-fix code: at expand_context=0 the new clamp is the identity and the if expand_context > 0: branch is not taken, and the source change is purely additive, so pre- and post-fix execute the same statements. The other two expansion tests do discriminate; the description's "all three expansion tests fail against the code they cover" is two. Same on #1547.
368b221 to
cde0657
Compare
marvinyu-memverge
left a comment
marvinyu-memverge
left a comment
There was a problem hiding this comment.
Reviewed at cde0657, base main @ 4ad28de. The fold itself checks out; two asks, both on the same seam.
1. expand_context is spent in segments, but every published surface documents it in episodes.
EventMemory._query derives the window straight from it: max_backward_segments = expand_context // 3, max_forward_segments = expand_context - max_backward_segments (event_memory.py:450). The API describes that same integer as episodes in three places - the OpenAPI field description ("The number of additional episodes to include around each matched episode", doc.py:213), the Python client (client/memory.py:385), and MemMachine.query_search (memmachine.py:997).
Those two units coincide only under PassthroughSegmenter, which the body says. But TextSegmenterConf ships and is config-selectable (episodic_config.py:141, type: "text", max_chunk_length default 500), and under it the new clamp claims parity in episode terms while bounding a segment budget. Concretely, at max_chunk_length: 500 with episodes of about 2,000 characters (4 segments each), top_k=10, expand_context=9 clamps to 9, splits 3 backward / 6 forward, and returns roughly 2 neighbour episodes for the 9 that were asked for. No error, and nothing in the response tells the caller.
The conflation predates this PR - it lives in event_memory, not in your diff. What changes here is that expand_context stops being inert, so this is the first release where it is visible. Is the clamp meant to be in segment space, or is the episode wording meant to be qualified to the passthrough case?
2. Nothing exercises more than one segment per episode through search_scored.
Every fixture in test_event_backend_wiring.py builds with PassthroughSegmenter() (lines 192, 449, 510, 707), and TextSegmenter shows up only in the event_memory tests. That is precisely the configuration in which segments and episodes cannot come apart, so the window-to-episode fold, the dedup in _episode_uid_context, and the clamp's parity claim are each pinned only where they are trivially true. One wiring test at TextSegmenter with a chunk length small enough to split each episode would cover the dedup path and turn whatever you decide on ask 1 into an assertion. (Separate from the zero-test point you already have open on line 747.)
Verified. _unify_scored_uid_contexts is a faithful mirror of DeclarativeMemory._unify_scored_anchored_episode_contexts (declarative_memory.py:617, with _weighted_index_proximity at :674): same break condition, same whole-context fit test, same setdefault score retention, same (proximity - 0.5) / 2 forward weighting, same final chronological sort. The one divergence - applying the zero floor after the quota clamp rather than before - is the safer direction, and your comment names it. "Best windows first" holds under both metric directions, since EventMemory._query sorts with reverse=higher_is_better and that flag accounts for both the reranker and the collection's similarity metric. num_episodes_limit=0 returns empty rather than one episode: the seed loop's break fires after the first append and would return one, but vector_search_limit is 0 as well, so there are no seeds - which is also why assert windows still holds in the clamp test, as get_segment_contexts is called unconditionally with an empty seed list. And the assumption the proximity fill rests on is real: the SQLAlchemy store assembles the window as [*reversed(backward_rows), seed_row, *forward_rows] (sqlalchemy_segment_store.py:374).
cde0657 to
6b080cb
Compare
Review on MemMachine#1541: every wiring fixture uses PassthroughSegmenter, the one configuration in which segments and episodes cannot come apart, so the window-to-episode fold was exercised through search_scored only where it is trivially one-to-one; and expand_context is spent in segments (EventMemory splits it into the segment store's backward/forward window) while the API describes it in episodes. The unit stays segments. EventMemory and the segment store know segments, not episodes; windowing by true episodes would need new store operations, and the passthrough segmenter already gives the declarative backend's behavior, one segment per episode. A splitting segmenter trades that for chunked retrieval: the same window then reaches fewer episodes, the ones its segments belong to. The clamp comment says so, since the clamp bounds episodes as well as segments (a segment belongs to one episode). The new wiring test runs the timeline fixture under both segmenters at expand_context=3, limit 4, with TextSegmenter(max_chunk_length=9) splitting every episode into three segments. Episodes keep the score of the first window that contributed them, so the episodes at the match's score are the match's own window: three neighbours under passthrough, one under the splitting segmenter for any backward/forward split, and the fold dedups one episode's segments into it. It fails against the pre-fix long_term_memory.py (no neighbours at all). _make_ltm_with_metric becomes _make_ltm(embedder, episodes, *, segmenter) so the test can build self-contained stores per segmenter; the three metric tests pass their FakeEmbedder explicitly. Mechanical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
marvinyu-memverge
left a comment
marvinyu-memverge
left a comment
There was a problem hiding this comment.
Approving at eecee28 (base main @ 3809561). Both asks from my last round are settled.
- Segments it is. That's a fine call: passthrough is the default (
episodic_config.py:238), so the API's "episodes" wording holds out of the box, and the clamp comment now says what a splitting segmenter changes. - The new splitting-segmenter test does the job. I ran it: 24/24 pass at head, and with
long_term_memory.pyswapped back to its pre-fix version, all three window-passing tests fail, the new one included. Using the match's score to identify the match's own window is sound, since onlytok-3embeds at cosine 1.0 and every other segment scores strictly lower.
Optional, non-blocking: the only place that says a type: text segmenter shrinks what expand_context reaches is the code comment. One clause in TextSegmenterConf's description would put that in front of the operator who opts in. Your call.
Commit 1 is a pure rebase of what I reviewed at cde0657 (identical added/removed lines), so my earlier verified-parity notes still stand.
Review on MemMachine#1541: every wiring fixture uses PassthroughSegmenter, the one configuration in which segments and episodes cannot come apart, so the window-to-episode fold was exercised through search_scored only where it is trivially one-to-one; and expand_context is spent in segments (EventMemory splits it into the segment store's backward/forward window) while the API describes it in episodes. The unit stays segments. EventMemory and the segment store know segments, not episodes; windowing by true episodes would need new store operations, and the passthrough segmenter already gives the declarative backend's behavior, one segment per episode. A splitting segmenter trades that for chunked retrieval: the same window then reaches fewer episodes, the ones its segments belong to. The clamp comment says so, since the clamp bounds episodes as well as segments (a segment belongs to one episode). The new wiring test runs the timeline fixture under both segmenters at expand_context=3, limit 4, with TextSegmenter(max_chunk_length=9) splitting every episode into three segments. Episodes keep the score of the first window that contributed them, so the episodes at the match's score are the match's own window: three neighbours under passthrough, one under the splitting segmenter for any backward/forward split, and the fold dedups one episode's segments into it. It fails against the pre-fix long_term_memory.py (no neighbours at all). _make_ltm_with_metric becomes _make_ltm(embedder, episodes, *, segmenter) so the test can build self-contained stores per segmenter; the three metric tests pass their FakeEmbedder explicitly. Mechanical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
b996862 to
b90214b
Compare
…odes (speedkick) (MemMachine#1547) * fix(event-backend): make expand_context return timeline-neighbor episodes Fixes MemMachine#1540. On the event backend, expand_context was silently inert: EventMemory fetched and materialized the expanded segment windows, but LongTermMemory._search_scored_event read only the seed segment's _episode_uid and score from each window, and the response schema has no context field - so responses were byte-identical for expand_context 0 and 5 while every request paid the LATERAL fetch. The declarative backend, by contrast, folds neighbor episodes into the returned list (_unify_scored_anchored_episode_contexts). This brings the event backend to parity: - Each scored window now contributes the episodes its segments belong to (chronological within the window, the seed's episode as nucleus). - Windows are unified best-score-first with the same fill algorithm as the declarative backend: taken whole while they fit within num_episodes_limit, then filled by weighted index-proximity to the nucleus (forward recall preferred) until the limit is met; an episode keeps the score of the first window that contributed it. - The unified context is returned chronologically, matching the declarative backend's ordering contract for expanded results. - expand_context is clamped to num_episodes_limit - 1 (declarative parity). expand_context == 0 behavior is unchanged (score-ordered seeds, exactly as before). Reranked configurations gain the same folding on top of reranker-scored windows. Tests: end-to-end via the in-memory event-backend wiring (neighbors returned, chronological order, limit respected) and unit tests for the window-to-episode-uid extraction and the unification algorithm (whole-context fit, overflow proximity with forward preference, first-window score retention). Co-Authored-By: Claude Fable 5 <[email protected]> * style: ruff format Co-Authored-By: Claude Fable 5 <[email protected]> * fix(event-backend): clamp expand_context above zero, and make the expansion tests actually discriminate Self-review of the two commits above turned up one defect and one hole. Defect: the quota clamp `min(max(0, expand_context), num_episodes_limit - 1)` goes negative when `num_episodes_limit == 0` -- reachable, since `SearchMemoriesSpec.top_k` carries no lower bound. `EventMemory._query` then derives `max_backward_segments = -1 // 3 = -1` and hands the segment store a negative window, which the SegmentStorePartition contract does not define: the SQLAlchemy store happens to short-circuit on `<= 0`, the in-memory store computes an empty slice and drops the seed. Apply the floor last so the clamp can only ever produce a non-negative window. Hole: neither end-to-end test could tell the fix from its absence -- both pass unmodified against the pre-fix `long_term_memory.py`. `FakeEmbedder` maps text to `[len(text), -len(text)]`, so under cosine every document scores exactly 1.0 against every query; all seven timeline episodes become seeds of equal rank, ties keep insertion order (which is chronological), and `num_episodes_limit=7` returns all seven with or without expansion. The "expansion adds episodes" assertion compared a limit-2 search against a limit-7 one, so the limit alone explained the difference. Embed on a keyword instead: only `tl-3` matches the query, so `tl-4` and `tl-5` -- which score zero -- can reach the result only through the expansion. The tests now pin the exact window (`[tl-3, tl-4, tl-5]`, chronological, each keeping the window's score), the clamp against an oversized `expand_context`, and the non-negative window above. All three fail against the code they cover. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01PVtg6Zea292Pb9L7GXnTJp * test(event-backend): assert the expansion contract, not a ranking The tests added in the previous commit discriminate, but they pin an outcome: an exact episode list (`[tl-3, tl-4, tl-5]`) and exact score values. Both are properties of the fixture's ranking and of the backward/forward split `expand_context // 3`, neither of which the fix claims -- change the split or the scoring and the tests fail while the behaviour under test is still correct. Restate them as the contract. Each episode now gets its own similarity from an explicit search rank, with the match's four timeline neighbours ranked last, so: - no correct top-k can return those neighbours, and any nonzero window around the match reaches at least one of them whatever the split. The assertion is "expansion returned a neighbour the search itself would not", plus chronological order and the episode limit. - the clamp is asserted on the call made to the segment store (0 <= backward + forward <= limit - 1, over several limit/expand_context pairs) rather than on which episodes come back. - `expand_context == 0` is asserted as "matches only, best score first", without naming them. Exact lists and score values stay in the unit tests, which own the fill algorithm and the score-retention rule and are meant to track them. All three expansion tests still fail against the code they cover. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01PVtg6Zea292Pb9L7GXnTJp --------- Co-authored-by: Claude Fable 5 <[email protected]>
Review on MemMachine#1541: every wiring fixture uses PassthroughSegmenter, the one configuration in which segments and episodes cannot come apart, so the window-to-episode fold was exercised through search_scored only where it is trivially one-to-one; and expand_context is spent in segments (EventMemory splits it into the segment store's backward/forward window) while the API describes it in episodes. The unit stays segments. EventMemory and the segment store know segments, not episodes; windowing by true episodes would need new store operations, and the passthrough segmenter already gives the declarative backend's behavior, one segment per episode. A splitting segmenter trades that for chunked retrieval: the same window then reaches fewer episodes, the ones its segments belong to. The clamp comment says so, since the clamp bounds episodes as well as segments (a segment belongs to one episode). The new wiring test runs the timeline fixture under both segmenters at expand_context=3, limit 4, with TextSegmenter(max_chunk_length=9) splitting every episode into three segments. Episodes keep the score of the first window that contributed them, so the episodes at the match's score are the match's own window: three neighbours under passthrough, one under the splitting segmenter for any backward/forward split, and the fold dedups one episode's segments into it. It fails against the pre-fix long_term_memory.py (no neighbours at all). _make_ltm_with_metric becomes _make_ltm(embedder, episodes, *, segmenter) so the test can build self-contained stores per segmenter; the three metric tests pass their FakeEmbedder explicitly. Mechanical. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
cad4770 to
74c720f
Compare
3 participants
Fixes #1540.
Problem
On the event backend,
expand_contextwas silently inert on the search path:EventMemory.queryfetched and materialized the expanded segment windows (a LATERAL query per direction), butLongTermMemory._search_scored_eventread only the seed segment's_episode_uidand score from each window, and the response schema has no context field. Responses were byte-identical forexpand_context0 and 5, while every request paid for the expansion.Fix
Bring the event backend to parity with the declarative backend's context handling (
DeclarativeMemory._unify_scored_anchored_episode_contexts):_episode_uid, so segment-space windows map directly onto episode-space context; with the passthrough segmenter this is 1:1 timeline neighbors.)num_episodes_limit, then filled by weighted index-proximity to the nucleus (forward recall preferred over backward, same weighting as declarative) until the limit is met. An episode keeps the score of the first window that contributed it.expand_contextis clamped tonum_episodes_limit - 1(declarative parity).expand_context == 0behavior is unchanged — score-ordered seed episodes exactly as before, so existing callers see no difference unless they pass the parameter. Reranked configurations gain the same folding on top of reranker-scored windows (which already consumed the segments for scoring).Follow-up self-review
Two things the first version got wrong:
expand_contextcould go negative. The quota clampmin(max(0, expand_context), num_episodes_limit - 1)is-1whennum_episodes_limit == 0, whichSearchMemoriesSpec.top_kallows (no lower bound).EventMemory._querythen derivesmax_backward_segments = -1and asks the segment store for a negative window — undefined by theSegmentStorePartitioncontract; the SQLAlchemy store happens to short-circuit on<= 0, the in-memory store computes an empty slice and drops the seed. The floor is now applied last.long_term_memory.py.FakeEmbeddermaps text to[len(text), -len(text)], so under cosine every document scores exactly 1.0 against every query: all seven timeline episodes are seeds of equal rank, ties keep insertion order (chronological), andnum_episodes_limit=7returns all seven with or without expansion. The "expansion adds episodes" assertion compared a limit-2 search against a limit-7 one, so the limit alone explained the difference.The expansion tests now give each episode its own similarity from an explicit search rank, with the match's four timeline neighbours ranked last. That lets them assert the contract rather than an outcome — an exact episode list or exact score values would be pinned to the fixture's ranking and to the
expand_context // 3split, neither of which the fix claims:0 <= backward + forward <= limit - 1, over several limit/expand_contextpairs) rather than on which episodes come back.expand_context == 0is asserted as "matches only, best score first", without naming them.Exact lists and score values stay in the unit tests, which own the fill algorithm and the score-retention rule. The two expansion tests that pass a window fail against the code they cover; the zero-window test pins the path the change leaves alone.
Tests
End-to-end through the in-memory event-backend wiring: expansion returns neighbor episodes beyond the plain matches, chronologically ordered, and never exceeding
num_episodes_limit.Unit tests for the window→episode-uid extraction (dedup, nucleus identification) and the unification algorithm (whole-context fit, overflow proximity with forward preference, first-window score retention).
The two window-passing expansion tests fail against the pre-fix
long_term_memory.py; the unit tests cover the window→episode-uid extraction and the unification algorithm directly.ruff(pinned 0.16.8 on this base) format + check clean;ty check packages/serverreports no diagnostics from the touched files. Episodic + server test suites pass (514 passed, 1 skipped).The same change, based on
speedkick, is #1547.🤖 Generated with Claude Code
Stack
23 PRs on
main: 5 independent ones, and three consecutive stages numbered on their own: the vector store scale-out, the SQLite store fixes and the vector store contract changes. A stacked PR's diff on GitHub is cumulative until the PRs under it merge.Independent PRs, each directly on
main; review and merge in any order:The vector store stages, consecutive: each stacked on the one below. [vector store scale-out 1/5] is directly on
mainand 2/5 on it alone; they do not depend on the independent PRs. From 3/5 up, the chain's history also carries the independent PRs' commits beneath it, since the later stages were written on top of them ([vector store contract 2/6] on #1624's session policy, for one); so a merge of an independent PR rebases the chain without conflict, and until they merge those PRs' changes show in a stacked PR's diff.speedkick, #1588)speedkick, #1589)This PR is its one commit,
6b080cb4b, directly onmain, independent of the others: review and merge it in any order among the independent PRs (a later one touching the same file rebases). The stages' history carries the same change from [vector store scale-out 3/5] up, so their diffs include it until it merges.Verification
At its one commit, on 2026-09-21:
ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server); the full server suite without integration tests passes (pytest packages/server/server_tests -m "not integration"), 1921 tests at its head6b080cb4b.