Skip to content

(Depends on #1597) Evict near-duplicate derivatives at ingest (speedkick) - #1617

Closed
edwinyyyu wants to merge 68 commits into
MemMachine:speedkickfrom
edwinyyyu:feat/event-memory-eviction-speedkick
Closed

edwinyyyu wants to merge 68 commits into
MemMachine:speedkickfrom
edwinyyyu:feat/event-memory-eviction-speedkick

Conversation

@edwinyyyu

Copy link
Copy Markdown
Contributor

The eviction part of design/event_memory_handoff.md (branch design/tenant-lifecycle, PR #1579), split out of #1597 for review. Stacked on #1597: its commits come first, and this PR is the last commit. #1611 is stacked on this one.

What changes

Options (event_memory/data_types.py, EventMemoryParams)

  • EvictionOptions(cosine_similarity_threshold, search_limit, target_size): the cosine similarity at or above which eviction is considered, how many stored derivatives at or above it are fetched per new derivative, and how many of a new derivative and those stored ones are kept when there are more. EventMemoryParams.eviction: EvictionOptions | None; None keeps every derivative and issues no query.

Encode (event_memory.py), after embedding and before any write:

  • The batch is sorted by (timestamp, uuid) first, so what a batch evicts among its own derivatives is what serial ingestion would.
  • _compute_batch_predecessors: for each derivative, the earlier batch indices whose cosine similarity is at or above the threshold, from the batch's own embeddings.
  • One batched neighbor query with min_cosine_similarity=cosine_similarity_threshold and limit=search_limit; the stored neighbors' timestamps are read from their segments (get_segment_uuids_by_derivative_uuids, then get_segments), since on Answer with cosine scores and uuids, not vectors and stale properties (speedkick) #1598's contract the vector store answers uuids and scores only. A neighbor whose segment is gone is not a member.
  • _select_eviction_targets: per derivative, the members are its stored neighbors not already displaced in this batch, its batch predecessors not already skipped, and itself; over target_size, they are sorted by timestamp and the temporal middle is trimmed, keeping the earliest target_size // 2 and the latest remainder. Stored members in the middle are displaced, batch members skipped.
  • Displaced records are deleted from the vector store and unlinked with delete_derivatives; skipped derivatives are never written to either store.

Store (segment_store.py, sqlalchemy_segment_store.py, the in-memory fake)

  • delete_derivatives(derivative_uuids): removes link rows by derivative uuid and leaves the segments. Locks the partition for write and, on PostgreSQL, the link rows in uuid order, the same order as delete_segments.

Decisions

  1. Two indexed reads per batch for timestamps. The vector store returns no properties, so a stored neighbor's timestamp comes from its segment. A timestamp-only read can follow if ingest profiling asks for it.
  2. The serialize_encode lock from the branch is not ported, as Add session, source and expansion to EventMemory (speedkick) #1597 says.

Verification

🤖 Generated with Claude Code

https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE

edwinyyyu and others added 9 commits September 14, 2026 16:35
Implements the session/source/expansion/eviction part of
design/event_memory_handoff.md from the tenant-lifecycle branch. The
context-part and block-kind model and the segmenter/deriver tables
follow in a second change; this one keeps the producer/null context
union and the `block_type` discriminator as they are.

Data models. `Event`, `Segment` and `Derivative` carry `session_id`
and `source_id` as nullable fields, copied verbatim down the
pipeline; null is encoded as a missing record key, `None` in a typed
id list selects it, and property values stay `None`-free. `SearchHit(score, seed, segments)` replaces
`ScoredSegmentContext` and `QueryResult`; `Neighborhood(before,
after)` and `EvictionOptions` are added.

Reserved keys. `common/property_keys.py` reserves the `memmachine_`
namespace; every system value a search filters on at the vector stage
sits in the record under a reserved key (`event_timestamp`,
`event_session`, `event_source`, `block_kind`). `utils.py` owns the
translation between the typed filters (`since`, `until`, `session_ids`,
`source_ids`, `block_kinds`) and filter trees; a caller key in the
namespace is rejected before any segment is written.

Segment store. `segment_store_sg` gains `session_id`, `source_id` and
`block_kind` columns, a session-led ordering index and a source index; the total order is
`(timestamp, event_uuid, index, offset)`, windows and neighborhoods
are confined to the seed's session, and a null session is one stream.
`get_segment_windows` takes `before`/`after` and the typed filters;
`get_segment_neighborhoods` returns the neighbors and never the seed, as two
lists; `delete_derivatives` unlinks without touching segments; PG
lateral reads run one statement pair per seed session. No migration:
`startup()` keeps `create_all`, and an existing speedkick database is
recreated; schema migration waits for the lifecycle/DDL changes.

EventMemory. `encode_events` forgets the batch first, so a repeat
leaves one copy, and runs eviction from the agentic_expansion branch,
cosine only: batch predecessors, one bounded neighbor query per
derivative, a cluster over `target_size` trimmed from its temporal
middle, displaced records and their links deleted, skipped ones never
written. `query` is the vector stage and returns hits with the seed's
index in its window; `rerank` is the second stage, static, for a
caller with a reranker; `expand` walks a neighborhoods from a segment
or event anchor; `render` replaces the string formatters. The reranker
and the per-call format options leave the constructor and the call,
respectively; the deriver's format is fixed per memory.

Server. `LongTermMemory` sets `Event.source_id` from the producer id,
leaves the session null and keeps the producer context; it reranks after `query` and reads
hits.

Tests: the branch's neighbor and eviction tests ported to the new
shapes on both dialects, plus session confinement, half-open bounds,
instant comparison of zoned bounds on SQLite, source and kind filters,
and link deletion. Each new store assertion was checked to fail
against a mutated store (no session predicate, an inclusive `until`, a
filtered neighborhoods seed, an unnormalized bound).

Rebased 2026-09-10 onto speedkick after MemMachine#1598 merged, adopting its
post-review names (`segment_by_derivative`, `seed_cosine_similarities`).
`common/property_keys.py` and its test, which MemMachine#1598 did not carry into
its merge, are included here.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01MuAu353FiSmCJjLX1LWDQW
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Every field of Event, Segment and Derivative carries a description;
the byte bound on ids is a validator on the model, so
EventMemory._validate_events checks only property keys; positions are
validated non-negative. SearchHit.seed is seed_index, checked to lie
inside segments, and the window around a seed is a segment window.
EvictionOptions.similarity_threshold is cosine_similarity_threshold,
the threshold at or above which eviction is considered; the other two
options say what is fetched and what is kept.

The store contract's class docstring only contrasts the two reads;
the details live on each method. Typed id lists hold ids only, and a
seed with no session walks every session: events in no session do not
belong together, so "no session" is not a value a list can name and
the only timeline to show around such a seed is everything. The
timestamp ordering index segment_store_sg__in_ts_ev_ix_of is restored
for that walk beside the session-led one.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
The memory makes no use of either bound, so the caller discards hits
itself; the vector stage keeps its limit because the store uses it.
Every hit comes back rescored, in descending score.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
cosine_similarity_matrix in eviction, and cosine similarity in every
docstring, comment and test name that named a bare similarity.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
…the store

Properties are set at ingest and never edited; a changed event is
forgotten and encoded again under the same uuid, with new segment and
derivative uuids. Event, EventMemory and SegmentStorePartition each
say so in their own terms, and the store's add is pinned as an insert
that rejects a stored uuid, so the vector record's copy of the
declared properties stays exact by construction and a future update
operation has to argue with the contract.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
get_segment_windows and get_segment_neighborhoods differed only in
whether the seed was filtered, which is a lookup concern, not a
walk's. get_segments(uuids, filters) returns the segments the
partition holds that pass; get_segment_neighborhoods(segments, ...)
walks the order around segments the caller holds and looks nothing
up, so a segment can be walked from even after its row is gone. The
rule for a caller: filters select what a read returns, and only a
segment obtained first can be walked from. query fetches its seeds
with the filters and walks from those; expand fetches its anchor
unfiltered and walks; eviction reads timestamps with the lookup. No
walk is asked for when expand_context is zero.

The seed keys reach PostgreSQL as typed bound parameters in a row set
rather than a VALUES list, which SQLAlchemy would recompile on every
call, since the data would be part of the statement's cache key.
Measured against the previous head, alternating runs on one
PostgreSQL container with a fresh analyzed schema per run, the
search-shaped read is a wash within noise and the SQLite read pays
one more connection checkout per call; the numbers are in the pull
request.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
…orhood)

A hit carried a flat list and an index into it, which the lookup and
the walk had to be flattened into and the reader had to index back
out of. QueryHit holds the seed the query matched and the Neighborhood
around it, the shape expand returns, so walking further from a hit
composes with expand and there is no index invariant to keep. window()
gives the flat list where one is wanted: rendering and the server's
episode folding.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
The contract is a store's, not a table's: segments are immutable, and
a walk starts from a segment, not from a lookup of it.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from fe73899 to 01e9f7e Compare September 15, 2026 20:22
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from 01e9f7e to 3d6a11a Compare September 15, 2026 20:27
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from 3d6a11a to f00cf68 Compare September 15, 2026 21:36
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from f00cf68 to 57a64ae Compare September 15, 2026 21:37
Two `if`s say what a loop over a tuple of names said; `_in_values` is a
static method under `_row_conditions`, the one place that calls it; and
the comment on an empty lookup says what the registry read is for.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from 57a64ae to fec385c Compare September 15, 2026 21:52
The order that put the incarnation second removed a competitor from the
planner's candidates for the link table's foreign-key check; it did not
make the planner choose well, statistics do. A fresh PostgreSQL table
misplans until its first ANALYZE whatever the indexes (the lookup by
uuid runs a sequential scan in that window too), and autovacuum's first
pass, or an ANALYZE after an initial import, ends it for the table's
lifetime. So the indexes read as the rest of the store does, scoped by
the incarnation first, and the PR body states the practice.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from fec385c to 065a6b9 Compare September 15, 2026 22:25
The source-pinned walk index served a walk filtered by one source, and
no measured workload runs one; the session-led index serves every walk,
and a filter on source or kind scans past the session's other rows, tens
of microseconds on a 200,000-row table. An index is paid for on every
insert for as long as it exists, and adding one later is a one-off
background build (2.5 s per million rows on PostgreSQL, concurrently),
so the store indexes the reads it has: the primary key, lookup by
event, and the walk. A walk filtered by source or by kind earns its
index when a workload shows it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from 065a6b9 to ca91a0f Compare September 16, 2026 16:43
@edwinyyyu
edwinyyyu force-pushed the feat/event-memory-eviction-speedkick branch from ca91a0f to d1cde08 Compare September 16, 2026 17:02
edwinyyyu and others added 2 commits September 16, 2026 16:06
`encode_events` no longer forgets the batch's events before encoding
them: every call adds, and encoding an event a second time stores a
second copy, as speedkick's encode does. The forget-first step widened a
race that speedkick already has: a forget that runs between an encode's
segment commit and its vector upsert orphans the encode's records, and
with encode forgetting first, a concurrent re-encode of one event took
that path too. A follow-up moves the upsert inside the write transaction
and rejects a reused event uuid at the store; until then encode is a
plain insert.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Ported from the agentic_expansion branch, cosine only. EventMemory
takes an optional EvictionOptions: a cosine similarity threshold at or
above which eviction is considered, the number of stored derivatives
at or above it fetched per new derivative, and how many of a new
derivative and those stored ones to keep when there are more. None
keeps every derivative and issues no query.

At encode, after embedding: the batch's predecessors of each
derivative are computed from the batch's own embeddings, earlier
indices only, so a batch evicts what serial ingestion would; one
batched neighbor query fetches the stored derivatives at or above the
threshold; the stored ones' timestamps come from their segments
through the store's lookup, since the vector store answers uuids and
scores only, and a neighbor whose segment is gone is not a member;
then, per derivative, the members over target_size are trimmed from
the temporal middle, the earliest target_size // 2 and the latest
remainder kept. Displaced stored records are deleted from the vector
store and unlinked from their segments with delete_derivatives, which
the store contract, the SQLAlchemy store and the fake gain; skipped
batch derivatives are never written. The batch is sorted by timestamp
first, so the predecessor rule matches serial order.

Tests: the branch's eviction tests on the new shapes, and
delete_derivatives on both dialects, including on a stale handle.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
@edwinyyyu

Copy link
Copy Markdown
Contributor Author

Closed in favor of its main port, #1685, which carries this PR's content at its final tip; the review history stays here. See the "Source and review" section of #1685 for what was reviewed and what was adapted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant