Repository navigation
Conversation
This was referenced Sep 17, 2026
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
from
September 17, 2026 21:44
dd14448 to
04a159e
Compare
This was referenced Sep 17, 2026
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
2 times, most recently
from
September 18, 2026 00:08
206a1d6 to
cd1b1b6
Compare
edwinyyyu
marked this pull request as draft
September 18, 2026 16:45
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
from
September 18, 2026 19:50
cd1b1b6 to
ee9fca5
Compare
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
from
September 28, 2026 18:02
ee9fca5 to
d79171f
Compare
This was referenced Sep 28, 2026
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
from
October 2, 2026 00:18
d79171f to
c6c82e1
Compare
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
2 times, most recently
from
October 7, 2026 20:11
9ee6c4c to
db0b63b
Compare
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
5 times, most recently
from
October 9, 2026 23:05
b3526ef to
800308e
Compare
The event backend wrote every property of an event into its vector record and mapped the caller's whole filter onto the vector store, so a user key that a deployment never declared was both stored and filtered there: on Qdrant and Milvus as unindexed payload a filtered query scans for. The segment store already holds every property and already receives the whole filter for the context windows, so the vector side only duplicated work the segment store does anyway. The vector record now carries the keys the collection declares: EventMemory's reserved timestamp, the `_`-prefixed system properties an adapter stamps on the event, and the keys a project's `properties_schema` declares. The vector store is queried with the conjuncts of the filter that name only such fields; a conjunct is dropped whole when any field under it is undeclared, so dropping only ever widens the vector search, and the segment store narrows it back on the windows. An undeclared key never reaches the vector store, so a tenant's undeclared properties cannot shape what it stores or scans. `filter_fields` joins the filter parser: every field name a tree addresses. Rebased onto MemMachine#1631, where the vector record no longer carries the segment uuid (the segment store maps a derivative to its segment). Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
…ck) (MemMachine#1606) * Regenerate the OpenAPI document under the locked FastAPI `docs/openapi.json` predates the FastAPI release in `uv.lock` (0.141.1), whose `ValidationError` component carries `input` and `ctx`; regenerating the document with `docs/tools/generate_openapi.py` adds the two fields and changes nothing else. Separate from the API changes above it so their diffs of this file show only what they change. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn * Remove per-project filterable properties A project could declare `properties_schema`, a set of caller property keys with types, on its long-term memory configuration; the event backend merged it into the vector store collection's indexed schema and rejected filters on any other `m.<key>`. That let a tenant create database resources (indexes, columns) by naming them in a request, which is what forced per-collection native resources named by a hash of their schema on the backends that limit them. The option is removed from the server configuration, the project API and the memory-configuration API, the Python SDK, the sample configurations, the configuration docs and the OpenAPI document. A filter may name any `m.<key>`; the stores evaluate it on the properties they hold. What a store indexes is decided by the deployment, not per project. A breaking API change on `speedkick`. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn Rebased onto MemMachine#1631: the per-project schema also leaves MemMachine#1631's service locator, which creates the session's collection in a retry loop, and the commented option goes from the event sample configuration MemMachine#1698 added. --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> (cherry picked from commit a8322a7)
…n, and use serial commas MemMachine#1734's, MemMachine#1735's, and MemMachine#1736's latest round, for this PR's partition registry and stores; no behavior changes: - The queue's `failed_rounds` column is `consecutive_failed_rounds`, in the code, the tests, the documents, and the dead-letter log's hint. - The registry keeps its durations as seconds. `_has_elapsed(seconds, since=)` answers whether a duration has passed on the database clock, for the retention, the backoff, and the lease, and `_purge_retry_backoff_seconds()` computes each tombstone's capped backoff. - `run_purge_round` runs named steps: `_claim_oldest_due_tombstone` answers a `_TombstoneClaim`, an `_UnendedPurgeRound`, or None, and the round's outcome goes to `_count_failed_purge_round`, `_end_tombstone_claim_after_cancellation`, or `_record_purge_round`. `_insert` is `_insert_pending_partition`, `_claim_releases` `_tombstone_claim_endings`, and the base store's `_cancellations` `_reservation_cancellations`. - The Milvus filter helpers are named for what they produce, comments are shorter, and lists in the documents, comments, and docstrings take a serial comma. The same changes as MemMachine#1734's 2a3d87c and 503687c, MemMachine#1735's 106e106, and MemMachine#1736's 3681d09 and bc6d4f9, for this PR's partitions. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
A round that never ended was counted by the next claim, which took a CASE-shaped update that both recorded the lost round and ended its claim; the count lived apart from the claims that made it. Each claim now counts its attempt: the claim is a plain update that increments `consecutive_attempts` (renamed from `consecutive_failed_rounds`), stamps `claimed_at`, and bumps the generation, and `_TombstoneClaim.attempt` carries the count. A tombstone is claimable when it has no attempts, when no claim is open and the backoff has passed since its last failure, or when an open claim has outlived its lease plus the backoff. A raised round ends its claim and stamps `last_failed_at`; a cancelled round ends its claim and takes its attempt back; a round that found records resets the attempts. After `_MAX_PURGE_ATTEMPTS` attempts a tombstone is dead-lettered, and a last attempt that raises is reported. A retry logs which attempt it is, and a raised round's error names its incarnation and attempt. The registry tests, the purge design document, and the registry design document follow. The same change as MemMachine#1734's 1732038, for this PR's partition registry. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
`consecutive_attempts` counted the purge rounds claimed since a round last found records, but its name said only that they were in a row. It is `attempts_without_progress`, and `_MAX_PURGE_ATTEMPTS` is `_MAX_PURGE_ATTEMPTS_WITHOUT_PROGRESS`. The column's comment, the claim's attempt, the backoff parameter's description, the dead-letter log, the class docstring, the tests, the purge design document, and the registry design document's table follow; nothing else changes. The same change as MemMachine#1734's 96a8b5b, for this PR's partition registry. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
…urrent claim The measured cost of the claim came from earlier forms of it, which held a transaction across the round. The section now gives the figures measured on 2026-10-06 against the claim this registry ports, naming the MemMachine#1734 commits they were taken at: the backoff scan with 1k, 10k, and 100k tombstones backing off, and interactive throughput and liveness latency beside two sweepers on PostgreSQL and on SQLite. The same change as MemMachine#1734's bab379b, for this PR's purge document. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
MemMachine#1736's review round, for this PR's Milvus store: - Filter strings reach Milvus as UTF-8, so a character outside the Basic Multilingual Plane parses, and an undeclared property's condition requires the stored type tag to be one the value compares with. - Startup's already-exists guard goes, with its mocked-client test: Milvus answers a create of an existing collection with the same schema with success. - The store no longer re-sorts search results, which Milvus returns best first. - The mocked-client purge test disposes its registry engine when it fails. The same changes as MemMachine#1736's 1a7fffa, d6fc219, 9fd7f76, 6fcd8c5, and c264b3d, for this PR's store. MemMachine#1736's 28e8c50 and b3ef4f1 have no counterpart here: startup prepares the store's one native collection before any purge round runs, and the store refuses an unsupported metric at construction, before anything is reserved. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
The declared path counted an int and a float as comparable either way, so
a float filter value on a declared int property reached Milvus as a float
literal against an INT64 field, which Milvus refuses to parse ("cannot
cast value to Int64", code 1100). A float now compares only with a float
property, and matches no int one, as a value of another type matches
nothing; an int still compares with a float property by value.
The new test filters a declared float property with an int and a declared
int property with a float; it fails with the float counted as comparable
with an int property.
The same change as MemMachine#1736's 31fc7c4, for this PR's store.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
…y name MemMachine#1736's baabf48 creates Milvus native collections at Bounded by name, and this PR's base carries that into startup's create, so the read level the purge and the tombstone retention rely on no longer comes from pymilvus's default. The consistency test now spies create_collection and checks the level it names, as MemMachine#1736's test does; it fails with the level dropped from the create. The design document and the store's docstring say the store creates its one native collection at Bounded. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
MemMachine#1736's f56686f halves a Milvus upsert refused with RESOURCE_EXHAUSTED, and this PR's base carries the halving into the partition handle's `_upsert`. Its tests come here in partition terms: the integration test upserts 1,200 records with a 60,000-character property, about 72 MB, and fails with RESOURCE_EXHAUSTED without the halving; the mocked-client tests pin the halving, a single refused entity raising, and a timeout or another refusal sent once, on a handle `_partition_on` builds, which the mocked delete test now uses too. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
…nt alone MemMachine#1736's 49b9ec2 drops the offset field beside a declared Milvus datetime, keeping its UTC instant alone, and this PR's base carries that into the store. The tests follow in partition terms: the native collection's fields are exactly the fixed ones and one per declared property; a declared datetime is stored as its instant in UTC; and a store whose declared datetime has the longest property key, 32 bytes, writes its one field, reads it back, and matches the record by its instant, the test MemMachine#1736's b015229 adds. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
The test of MemMachine#1813, merged into this PR's base, in partition terms: the server defaults new collections to strict mode, the store's collection reads back with it off, and a filter on a partition's unindexed property is served. Co-Authored-By: Claude Opus 5.5 <[email protected]>
A partition stored every property of a record and filtered on any key, which made a caller's arbitrary keys part of the store's schema: the SQLite stores kept them in a JSON column and filtered with json_extract, Milvus the keys its schema did not declare in a JSON field, and a filter on a key the store never indexed scanned. Since EventMemory routes a filter on an undeclared key to the segment store, the vector store need not hold undeclared keys at all. A partition now stores the properties its store declares and no others. `upsert` raises UndeclaredPropertyKeyError before anything is sent for a record naming an undeclared key, and PropertyTypeMismatchError for a value of another type than its key declares; `query` raises UndeclaredPropertyKeyError for a filter naming an undeclared key and UnsupportedFilterError for a node outside the partition's `supported_filter_nodes`. Both SQLite stores keep one typed, indexed, nullable column per declared key on the records table (sql_columns.py); sqlite-vec 0.1.9 rejects NULL in a vec0 metadata column and a declared key is optional per record, so that store keeps the columns on the records table and hands the KNN a `rowid IN (SELECT ...)` allowlist, evaluating the filter during the search instead of after it. Qdrant drops the JSON copy and keeps a payload field per declared key; Milvus drops its JSON field and keeps its typed field per declared key. Datetimes are stored as microseconds since the epoch where a backend has no datetime type. Since every key a filter may name is now indexed, the Qdrant store creates its collection in strict mode (`unindexed_filtering_retrieve` and `_update` false, Qdrant Cloud's default): a filter on an unindexed key is refused by the server instead of scanned for. A leaf whose value is of another type than its key declares matches nothing, as on the SQL stores; the Qdrant compiler answers it with a filter no point satisfies, since the server would refuse the condition for the field's index, and Milvus no longer compares an int with a float key. Local mode does not record the setting, so a unit test checks the request and integration tests the server's answer. `declared_schema_contract.py` states the contract every backend's test module runs: which records a filtered search admits, over fixtures small enough that every backend searches them exactly, checked after each upsert so an approximate index fails on recall, by name, and not on the filter. On the registry-backed stores, the checks are the base handle's: `upsert` runs require_declared_properties in place of the type check it ran, and `query` runs require_supported_filter against the subclass's `supported_filter_nodes`, which each subclass now implements. A datetime column on the SQLite stores has a `tz_<key>` column beside it holding the UTC offset in seconds, written with the value and read by no filter, so a stored datetime is the value written, as on Milvus, Qdrant and the segment store; the microseconds column alone would keep only the instant. The Qdrant and Milvus design documents describe the declared-only properties and Qdrant's strict mode. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
A declared datetime on the SQLite stores had a `tz_<key>` column beside its microseconds column, holding the UTC offset in seconds that no filter reads. The vector stores keep a datetime property's instant only, as MemMachine#1736 now does for Milvus and MemMachine#1788 for Qdrant, and the segment store keeps the offset, so the column, `offset_column_name`, and the value written to it go: each declared property is one column, and a datetime stays microseconds since the epoch. The tests read a stored datetime back as its instant in UTC, and the roundtrip test checks that the stored instant equals the written one. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Every embedder MemMachine ships produces vectors meant to be compared by cosine -- OpenAI hard-coded it, Bedrock defaulted to it, SentenceTransformer only reported what the model declared -- while every layer that touched a score paid for the other three metrics in direction flags, threshold directions, and per-backend tables mapping the enum onto native metric names. `SimilarityMetric` is gone; scores are cosine similarities in [-1, 1] and the names say so: `QueryMatch.score` and `SearchMatch.score` become `cosine_similarity`, and `query(score_threshold=)` becomes `query(min_cosine_similarity=)`, which no longer needs a direction to be meaningful and is refused when not finite, as the threshold was. A store and its partitions no longer have a `similarity_metric`, the schema a partition is registered under no longer records one, and a search engine factory takes the dimensions alone. The Bedrock embedder's `similarity_metric` config key and semantic memory's `vector_similarity_metric` go with it, and the install and configuration docs drop them. The vector graph stores carried a metric per stored embedding, as a companion property beside every vector; that is gone and `Node.embeddings` holds plain vectors. NebulaGraph's `cosine()` cannot take `APPROXIMATE` and its vector indexes offer only L2 and IP. Cosine similarity between unit vectors is their inner product, so embeddings are normalized on the way in and compared with `inner_product()` against an IP index. The cosine half of MemMachine#1598 (`abf92a3a4` on speedkick), re-derived on the one-collection store: the rest of MemMachine#1598 (queries answer record UUIDs and scores, no `get`, semantic memory's `vector_uuid`) and all of MemMachine#1603 are in registry-backed base and its partition handle lose the metric, the stores' own threshold checks become `require_valid_min_cosine_similarity`, and the design documents describe cosine scoring and a schema without a metric. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
…hine#1597, without session) The part of MemMachine#1597 that is not session: events and segments carry a nullable source id; query and expand take since/until and source_ids; expand walks outward from a seed, which is an address located whatever the filters say, and returns a Neighborhood that excludes it; query answers QueryHits (score, seed, neighborhood); rendering is render_segments with a DateTimeFormat; the segment store answers get_segments and get_segment_neighborhoods in place of get_segment_contexts; the v2 adapter lifts timestamp/created_at bounds and producer_id conjuncts out of the filter into the typed parameters; reserved property keys live in their own module and a caller cannot write one; the timestamp column holds a UTC instant. On this base: - A neighborhood is MemMachine#1713's walk: the partition's order next to the seed, filters applied inside its window of 1,000 segments per side. The ordering index keeps main's shape. - The vector stage gets MemMachine#1684's predicates on the reserved keys and, joined with AND, the conjuncts of the property filter the vector store declares, as MemMachine#1702's routing has it; the segment store still gets the whole filter. A record's declared properties come from its derivative's segment. - Episode uids are UUIDs since MemMachine#1707: the search path reads each hit's `_episode_uid` back as a UUID, and the tests key their episodes by `_uid(name)` as main's do. - Session and block kind are split out: the block kind column, parameter and record key go to the kinds PR, which is their first consumer, and session goes to its own PR. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
The segment store gains a write transaction, `write()`, whose block the caller may fill with work of its own: the event memory upserts its vector records inside it, so segments commit only once the vector store has acknowledged their records, and a failed upsert rolls them back. A forget can then only see links whose records exist, and its delete is issued after the upsert's acknowledgment, which closes the orphan race between encode and forget rather than repairing after it. An event is held at most once: a new event table, keyed by incarnation and event uuid, is inserted first with ON CONFLICT DO NOTHING RETURNING, and a batch naming a held event is rejected whole with `SegmentStoreEventAlreadyStoredError` before anything is stored. Segments carry a cascading foreign key to the event row; `delete_events` replaces forget's by-event path, `delete_segments` stays for eviction and leaves the event held, and `get_derivative_uuids_by_event_uuids` replaces the two lookups whose only caller was forget. The purge reclaims event rows after the segments, on the same budget. The residue a crash can leave, a record acknowledged by the vector store whose commit never happened, is repaired on retrieval: a hit without a link is re-checked under `write(exclusive=True)`, which waits for every write in flight, and a record still without a link is deleted after the fence is released. An upsert that fails deletes the same ids before the error propagates, since it may have been applied first. On this base the change meets MemMachine#1661's final store: the writer's inserts are the store's own, moved; the leaked-link reclaim MemMachine#1661 dropped stays dropped; and the event rows purge the way the segments do, after them, continuing from a cursor of their own on the queue entry, events_purged_through, since a call that ends exactly on the last segment leaves purged_through naming a segment. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Deleting the segment partition first waits for every write in flight, whose records land before it commits, and blocks new ones, so the vector partition's deletion that follows removes every record that could ever have landed in it. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Mechanical: the store now holds events, segments and links, so its name follows its dependent. `SegmentStore` and every derived name become `EventMemoryStore` and theirs, the module and test directories move with them, the tables and indexes take the `event_memory_store_` prefix, and the configuration key `segment_store` becomes `event_memory_store` in the API spec, the client, the docs, the sample configs, the Helm chart and the compose files. No behavior changes; tables are recreated. Applied by the same substitutions to this tree rather than rebased. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Ported from the agentic_expansion branch, cosine only. EventMemory takes an optional EvictionOptions: a cosine similarity threshold at or above which eviction is considered, the number of stored derivatives at or above it fetched per new derivative, and how many of a new derivative and those stored ones to keep when there are more. None keeps every derivative and issues no query. At encode, after embedding, `_decide_eviction` settles the whole decision before anything is written: the batch's predecessors of each derivative are computed from the batch's own embeddings, earlier indices only, so a batch evicts what serial ingestion would; one batched neighbor query fetches the stored derivatives at or above the threshold; the stored ones' timestamps come from their segments through the store's lookup, since the vector store answers uuids and scores only, and a neighbor whose segment is gone is not a member; then, per derivative, the members over target_size are trimmed from the temporal middle, the earliest target_size // 2 and the latest remainder kept. The batch is sorted by timestamp first, so the predecessor rule matches serial order. The encode's write() block then carries both link writes: it adds the surviving links and unlinks the displaced derivatives through delete_derivatives, which the writer interface, the SQLAlchemy writer and the fake writer gain, and upserts the surviving records inside the same transaction, with the compensating delete an upsert failure already had. The displaced records leave the vector store only after that block commits: a delete that fails then leaves records no link names, which read repair reclaims, rather than links naming records that are gone. Skipped batch derivatives are never written, and a displaced derivative's segment stays stored. Tests: the branch's eviction tests on the new shapes, and delete_derivatives through write() on both dialects, including an unlink rolled back with its block. On this base the surviving records carry their segment's declared properties, as every record does since the routing change, and the eviction's neighbor query and deletes go to the vector store partition. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Moved here from MemMachine#1684's port, where every block was text and the column would have held one value with nothing dispatching on it; this PR's kind-keyed segmenter and deriver tables are its first consumer. The segment row carries its block's kind in a column of its own, filled by the store from the block, since the codec's bytes are opaque to SQL; query, expand, get_segments and get_segment_neighborhoods take block_kinds; and the vector record carries the kind under its reserved key, which the vector stage filters with an In predicate. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
…s (speedkick)
The second half of the event memory handoff, on top of the
source/expansion/eviction change: the content model and the processing
steps become registered families keyed by kind.
Blocks. `Block` is an ABC with `kind` as the discriminator every
registered family uses and `render(options) -> str | None`; the union
stays closed over `TextBlock`; the segment row's `block_kind` reads
`block.kind`.
Context. `Context = Mapping[str, ContextPart]`, keyed by part kind,
never None; `Author` is the first registered kind, an unregistered
kind decodes to `UnknownPart` and round-trips unchanged; `with_part` composes one. `ProducerContext`, `NullContext` and the
discriminated union go; the codec encodes a context as `{kind:
fields}`. The server puts the producer id in an `Author` part beside
`source_id`, so rendering keeps its `producer: text` shape.
Segmenter and deriver. `Segmenter` and `Deriver` are tables from block
kind to handler, built from handlers in order with a later handler
replacing an earlier one for its kind. `BlockSegmenter[B]` and `BlockDeriver[B]` are the one-kind
handler contracts, typed by the block class: `split(event, block) ->
list[Piece]` and `derive(segment, block) -> list[str]`. The table
builds every envelope; a handler decides pieces or texts and nothing
else. A kind with no handler passes through as one segment and derives
nothing; nothing raises on a kind. `TextSegmenter`, `WholeTextDeriver`
and `SentenceTextDeriver` keep their names as `text` handlers;
`PassthroughSegmenter` and its configuration name go: one segment per
block is the identity segmentation, and a default gets no name, so an
omitted `segmenter` means it.
Composition. A derivative is text (`Derivative.text`, the text to
embed, plus the segment's `block_kind` for the record), so every
derivative gets the same context processing. `format_header` is the
one composition point for the embedded text and the rendered header:
the timestamp, then the context parts `parts` names in that order
(default `("author",)`), then the content. Parts carry no order; a part
not listed contributes nothing; a new kind is placed by listing it.
`parts` is a parameter of the composers, the handler and `render`,
not a field of `DatetimeFormat`, which writes a timestamp and nothing
else. A handler owns the composition of what it embeds, its
`datetime_format` and `parts`, so a kind or a handler can embed under
its own; the Deriver table composes each derivative's text with them;
rendering for display takes the caller's per call.
Tests: table dispatch, override order, envelope copying; text handlers returning bare
content; composition order, unlisted parts, kind-name validation, and
the anchor over a derivative; context and block round-trips including
an unregistered part kind.
On this base events carry no session, so no segment, derivative or
record copies one; the test harness builds records on the vector store
partition; main's splitting-segmenter test compares the default
segmenter with a text handler; and the event sample configuration main
added documents the text handler as the other sample configurations do.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
RegisteredBlock as the field type left three type errors where a handler's or the codec's Block was assigned to the field. The fields are typed Block, decoded through the registered union by a before-validator and serialized as the concrete kind; the handler tests keep a typed block in hand instead of narrowing the field. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
A context was a mapping the caller keyed by hand, and the rule that
the key equals the part's kind lived in prose. Context is a class now:
built from parts, read by kind, so there is no key to get wrong; two
parts of one kind are rejected at construction. It owns its wire form
through a Pydantic core schema, so a model field of the type accepts
an instance or `{kind: fields}` and serializes to `{kind: fields}`,
and the per-model validators, `with_part`, `encode_context` and
`decode_context` go.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
The kind-keyed tables removed its config, its service-locator branch and its test, and the module survived with no consumer. The long-term memory's over-fetch comment named it too; it now says why a segmenter can carry one episode several times. The expand_context comment main added names it as well; it now says "no segmenter handler". Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
A coding agent's transcript carries more than messages: the tool calls the agent makes, what those tools return, and text that entered the conversation without a user typing it. Each is a block kind of its own, registered so the v1 events route accepts it and the timeline holds it. `tool_call` carries the tool's name and its input. The input is a JSON object rather than a mapping of property values: a tool's arguments nest, and a block is content the event carries, not something the event is filtered by. `tool_result` carries the tool's name, its output, and whether the tool failed instead of returning; a result says nothing about failure unless it says so. `injected` carries the text and what put it there: a hook, a skill, a compaction, a reminder, a command, or something else. Each kind renders on one line, through the same `render` a timeline reader already gets for a message: a call's input as compact JSON, a failed result behind an `[error]` marker, injected text behind its source. A window mixing messages and tool events reads as one timeline. None of the three declares a segmenter or a deriver, so the tables' fallbacks decide: one segment per block, however long, and no derivative and no vector record. A tool event is on the timeline and off the search surface, reached by expanding from a message, which is the capture policy the coding-agent integration settled on. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
The kinds' round trip through the codec, their encoded form, and what each renders: a call as its tool and its input on one line, a result as its tool and its output with an `[error]` marker when it failed, and injected text as its source and its text. An empty or overlong tool name, an unlisted injection source and an unregistered kind are rejected. Segmentation and derivation are asserted where the tables decide them: a long tool result is one segment while the text handler splits the message beside it, and the text deriver derives nothing from any of the three. End to end, an event carrying a message, a call and a result is three segments and one vector record, the query reaches the message, and expanding from it returns the two tool events. The route's tests go with the v1 route, which sits above this change. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Capture holds everything the session log holds, never what a model
selected, so the agent's reasoning is a kind of its own rather than
something dropped on the way in: `{"kind": "thinking", "text": ...}`,
holding the reasoning as the transcript recorded it.
It is treated as the tool events are. It declares no segmenter and no
deriver, so reasoning is one segment however long and yields no
derivative and no vector record: it is on the timeline and off the
search surface, reached by expanding from a message. It renders as
`thinking: <text>`, one line of the same timeline a message and a tool
event render into.
The v1 route, which sits above this change, accepts it through the
registered union and names it with the other kinds.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
An event belongs to a session, a conversation's own timeline, and a walk never leaves it. Event, Segment and Derivative carry a required, bounded session_id; the segment row holds it in a NOT NULL column, the ordering index's second, after the incarnation (event_memory_store_sg__in_se_ts_ev_ix_of), and a neighborhood's walk pins the seed's session, on SQLite as a bound parameter per seed and on PostgreSQL through the lateral join, inside the same context window. query, expand and get_segments take session_ids, an empty list keeping nothing and None every session; the vector record carries the session under its reserved key, so the vector stage evaluates session_ids too; expand raises LookupError for a seed outside the named sessions, which it checks through get_segments. The v2 adapter has no conversation id to give, so every event it ingests is in one reserved session, memmachine_default. On this base the session goes onto the event memory store's names, the write transaction's segment insert and the kind-keyed segmenter and deriver tables, which copy it into every segment and derivative they build. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
`render` takes any segments, drops one given twice, and lays them out as a block per session, the sessions in the order of their latest timestamps with a blank line between, each block in the store's order, one line per run of adjacent pieces of one event. Nothing in the text says whether two lines are adjacent in the store, since a filtered walk can omit a neighbor and no segment carries a position. `ids` marks what a reader can name back: `"session"` heads each block with `[session:"<id>"]`, the id JSON-quoted since a session id is any string; `"segment"` starts each line with `[segment:<hex>]`, or `[segments:<first>..<last>]` when the line holds more than one, so the marker says which id opens the event and which closes it without a word of prompting, and a one-segment event carries one id. Uuids are 32 hex digits. With no ids and one session the text is what `render` produced before, which `rerank` relies on. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
edwinyyyu
force-pushed
the
port/event-memory-render-ids-main
branch
from
October 10, 2026 00:33
800308e to
35b1047
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Source and review
speedkick, closed in favor of this PR; its tip 5954753 is the content ported here._conjunctsand whose tests sit before [event memory 5/7] Block kinds for tool calls, tool results, injected text and thinking #1692's rendering test.Port of #1632 to
main, on #1715. Content, the marker grammar and tests are described on #1632, with two notes on its body: "exactly whatrender_segmentsproduced before" holds for store-ordered, duplicate-free input, which is what the reranker's windows are; and the helpers behind the rendering (_runs,_render_run,_store_order,_segment_marker,IdKind) are implementation detail it does not list.Stack
Two stacks, one line of branches. Every PR but #1693 targets
feat/horizontal-scaling, so a diff shows everything below it on that branch until that merges; #1693 is client-only, branches frommainand targets it. Rebuilt on 2026-09-28: #1684 split into time bounds and sources (#1684) and sessions (#1715), the block kind moved to #1687, and each PR restacked in dependency order. Rebased on 2026-10-01 after #1713 merged, dropping the merge commit that carried it; on 2026-10-06 after #1733 was squash-merged intofeat/horizontal-scaling; on 2026-10-07 after that branch took main's #1707, which makes episode uids UUIDs; and on 2026-10-09, after #1736 was squash-merged intofeat/horizontal-scaling, onto #1663's head c0a2bbd, and the same day onto fb38a72, after #1628 moved beneath #1663 and #1813 was squash-merged intofeat/horizontal-scaling. #1715 and everything above it stay deferred with the coding-agent features.Event memory, on #1663:
tool_call,tool_result,injected,thinkingsession_ids(deferred)Coding agents, slices of
design/coding_agent_integration.md(#1579), on the event memory stack; 3/3 shares no code with the server, so its branch is onmain, but it configures the endpoint 2/3 serves and writes the kinds #1692 registers, so it merges after both:memory_queryandmemory_expandserved at/v1/mcpStophook, and captureAt this head, on #1663 fb38a72 (2026-10-09), which carries #1628's declared-only stores: ruff, ruff format, ty as CI runs it on
packages/serveranduv lock --checkpass, anddocs/openapi.jsonmatches its generator. Unit and integration suites run again when this PR comes up for review; they last passed on 2026-10-07 on #1663 e0dedbd: 2203 server tests, 136 PostgreSQL store integration tests and 258 client tests.🤖 Generated with Claude Code
https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE