Repository navigation
Conversation
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
14 times, most recently
from
September 11, 2026 17:37
212ba7f to
e618650
Compare
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
12 times, most recently
from
September 12, 2026 00:58
5c774da to
c6583a8
Compare
edwinyyyu
marked this pull request as draft
September 14, 2026 17:25
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 14, 2026 18:50
c6583a8 to
bfe69ad
Compare
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 15, 2026 20:27
9e96a53 to
f747e98
Compare
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 15, 2026 21:36
f747e98 to
620730f
Compare
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 15, 2026 21:37
620730f to
fa616d3
Compare
Two `if`s say what a loop over a tuple of names said; `_in_values` is a static method under `_row_conditions`, the one place that calls it; and the comment on an empty lookup says what the registry read is for. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 15, 2026 21:53
fa616d3 to
18770cf
Compare
The order that put the incarnation second removed a competitor from the planner's candidates for the link table's foreign-key check; it did not make the planner choose well, statistics do. A fresh PostgreSQL table misplans until its first ANALYZE whatever the indexes (the lookup by uuid runs a sequential scan in that window too), and autovacuum's first pass, or an ANALYZE after an initial import, ends it for the table's lifetime. So the indexes read as the rest of the store does, scoped by the incarnation first, and the PR body states the practice. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
edwinyyyu
force-pushed
the
feat/event-memory-blocks-speedkick
branch
from
September 15, 2026 22:25
18770cf to
b18c2ee
Compare
The source-pinned walk index served a walk filtered by one source, and no measured workload runs one; the session-led index serves every walk, and a filter on source or kind scans past the session's other rows, tens of microseconds on a 200,000-row table. An index is paid for on every insert for as long as it exists, and adding one later is a one-off background build (2.5 s per million rows on PostgreSQL, concurrently), so the store indexes the reads it has: the primary key, lookup by event, and the walk. A walk filtered by source or by kind earns its index when a workload shows it. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
`encode_events` no longer forgets the batch's events before encoding them: every call adds, and encoding an event a second time stores a second copy, as speedkick's encode does. The forget-first step widened a race that speedkick already has: a forget that runs between an encode's segment commit and its vector upsert orphans the encode's records, and with encode forgetting first, a concurrent re-encode of one event took that path too. A follow-up moves the upsert inside the write transaction and rejects a reused event uuid at the store; until then encode is a plain insert. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Ported from the agentic_expansion branch, cosine only. EventMemory takes an optional EvictionOptions: a cosine similarity threshold at or above which eviction is considered, the number of stored derivatives at or above it fetched per new derivative, and how many of a new derivative and those stored ones to keep when there are more. None keeps every derivative and issues no query. At encode, after embedding: the batch's predecessors of each derivative are computed from the batch's own embeddings, earlier indices only, so a batch evicts what serial ingestion would; one batched neighbor query fetches the stored derivatives at or above the threshold; the stored ones' timestamps come from their segments through the store's lookup, since the vector store answers uuids and scores only, and a neighbor whose segment is gone is not a member; then, per derivative, the members over target_size are trimmed from the temporal middle, the earliest target_size // 2 and the latest remainder kept. Displaced stored records are deleted from the vector store and unlinked from their segments with delete_derivatives, which the store contract, the SQLAlchemy store and the fake gain; skipped batch derivatives are never written. The batch is sorted by timestamp first, so the predecessor rule matches serial order. Tests: the branch's eviction tests on the new shapes, and delete_derivatives on both dialects, including on a stale handle. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
…s (speedkick)
The second half of design/event_memory_handoff.md, on top of the
session/source/expansion/eviction change: the content model and the
processing steps become registered families keyed by kind.
Blocks. `Block` is an ABC with `kind` as the discriminator every
registered family uses and `render(options) -> str | None`; the union
stays closed over `TextBlock`; the segment row's `block_kind` reads
`block.kind`.
Context. `Context = Mapping[str, ContextPart]`, keyed by part kind,
never None; `Author` is the first registered kind, an unregistered
kind decodes to `UnknownPart` and round-trips unchanged; `with_part` composes one. `ProducerContext`, `NullContext` and the
discriminated union go; the codec encodes a context as `{kind:
fields}`. The server puts the producer id in an `Author` part beside
`source_id`, so rendering keeps its `producer: text` shape.
Segmenter and deriver. `Segmenter` and `Deriver` are tables from block
kind to handler, built from handlers in order with a later handler
replacing an earlier one for its kind. `BlockSegmenter[B]` and `BlockDeriver[B]` are the one-kind
handler contracts, typed by the block class: `split(event, block) ->
list[Piece]` and `derive(segment, block) -> list[str]`. The table
builds every envelope; a handler decides pieces or texts and nothing
else. A kind with no handler passes through as one segment and derives
nothing; nothing raises on a kind. `TextSegmenter`, `WholeTextDeriver`
and `SentenceTextDeriver` keep their names as `text` handlers;
`PassthroughSegmenter` and its configuration name go: one segment per
block is the identity segmentation, and a default gets no name, so an
omitted `segmenter` means it.
Composition. A derivative is text (`Derivative.text`, the text to
embed, plus the segment's `block_kind` for the record), so every
derivative gets the same context processing. `format_header` is the
one composition point for the embedded text and the rendered header:
the timestamp, then the context parts `parts` names in that order
(default `("author",)`), then the content. Parts carry no order; a part
not listed contributes nothing; a new kind is placed by listing it.
`parts` is a parameter of the composers, the handler and `render`,
not a field of `DatetimeFormat`, which writes a timestamp and nothing
else. A handler owns the composition of what it embeds, its
`datetime_format` and `parts`, so a kind or a handler can embed under
its own; the Deriver table composes each derivative's text with them;
rendering for display takes the caller's per call.
Tests: table dispatch, override order, envelope copying; text handlers returning bare
content; composition order, unlisted parts, kind-name validation, and
the anchor over a derivative; context and block round-trips including
an unregistered part kind.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
RegisteredBlock as the field type left three type errors where a handler's or the codec's Block was assigned to the field. The fields are typed Block, decoded through the registered union by a before-validator and serialized as the concrete kind; the handler tests keep a typed block in hand instead of narrowing the field. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
A context was a mapping the caller keyed by hand, and the rule that
the key equals the part's kind lived in prose. Context is a class now:
built from parts, read by kind, so there is no key to get wrong; two
parts of one kind are rejected at construction. It owns its wire form
through a Pydantic core schema, so a model field of the type accepts
an instance or `{kind: fields}` and serializes to `{kind: fields}`,
and the per-model validators, `with_part`, `encode_context` and
`decode_context` go.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE
This was referenced Sep 17, 2026
Contributor
Author
edwinyyyu
commented
Sep 17, 2026
edwinyyyu
left a comment
Contributor
Author
There was a problem hiding this comment.
packages/server/src/memmachine_server/episodic_memory/event_memory/segmenter/passthrough_segmenter.py survives this change with no consumer: the diff removes PassthroughSegmenterConf, the type: passthrough branch in service_locator.py and test_passthrough_segmenter.py, but not the module, so git grep PassthroughSegmenter at this head hits only its own definition. The description says "PassthroughSegmenter is gone"; the module should go with it. long_term_memory.py (_episode_to_event neighborhood) also still says "under non-passthrough segmenters". Same finding on the port, #1687.
Contributor
Author
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The second half of
design/event_memory_handoff.md(branchdesign/tenant-lifecycle, PR #1579), split out of #1597 on 2026-09-10. Stacked on #1597 and, on it, #1617 (eviction): their commits come first, and this PR's are the last four: the rebased original, the block-typing fix, the context class, and the table wording on the memory's parameters. Design:design/components/blocks.md("Processing") anddesign/components/context.md("Rendering").What changes
Blocks (
event_memory/data_types.py)Blockis an ABC withkindas the discriminator every registered family uses andrender(options) -> str | None; the union stays closed overTextBlock. The segment row'sblock_kindcolumn readsblock.kind.Event.blocksandSegment.blockare typedBlock, the ABC: a before-validator decodes encoded data through the registered union, andSerializeAsAnyserializes the concrete kind, so a handler's typed block assigns without narrowing. (RegisteredBlockas the field type left the branch with three type errors in the store and the segmenter, now gone.)Context parts (
event_memory/data_types.py)Contextis a class built from parts and read by kind (Context(Author(...)),context.get("author"),context.with_part(part)), neverNone; a caller never writes a part's key, so cannot write a wrong one, and two parts of one kind are rejected at construction.Authoris the first registered kind; an unregistered kind decodes toUnknownPartand round-trips unchanged. The class owns its wire form,{kind: fields}, through a Pydantic core schema, so a model field of the type accepts an instance or the encoded form and serializes to the encoded form.ProducerContext,NullContextand the discriminated union are gone.long_term_memory.py): the producer id is anAuthorpart besidesource_id, so rendered segments keep theirproducer: textshape.Segmenter and deriver (
segmenter/,deriver/)SegmenterandDeriverare tables from block kind to handler, built from handlers in order with a later handler replacing an earlier one for its kind. A kind with no handler passes through as one segment, unchanged, and derives nothing; a step never raises on a kind.BlockSegmenter[B]andBlockDeriver[B]are the handler contracts, one kind each (kind: ClassVar[str]), typed by the block class:split(event, block) -> list[Piece]andderive(segment, block) -> list[str]. The table builds every envelope, so a handler decides only the pieces (Piece(offset, block)) or the texts.TextSegmenter,WholeTextDeriverandSentenceTextDeriverkeep their names astexthandlers.PassthroughSegmenteris gone, and so istype: passthrough: one segment per block is the identity segmentation, the complete answer for any kind, and a default gets no name.segmenteromitted in configuration means it;segmenter: {type: text, max_chunk_length: N}selects the splitter.Composition (
event_memory.py,formatting.py)Derivative.blockbecomestext, plusblock_kind(the segment's) for the vector record. Every derivative gets the same context processing.format_header(formatting.py) is the one composition point for the embedded text and the rendered header: the timestamp, written per aDateTimeFormat, then the context partspartsnames in that order (default("author",)), then the content. Parts carry no order of their own; a part not listed contributes nothing; a new kind is placed by listing it.partsis a parameter of the composers, not a field ofDateTimeFormat, which writes a timestamp and nothing else. ABlockDeriverhandler owns itsdatetime_formatandparts, so a kind or a handler can embed under its own composition, and theDerivertable composes each derivative's text with them;Derivative.textis the text to embed.render_segmentstakes the caller'sdatetime_formatandpartsper call.EventMemoryParams.format_optionsis gone.Decisions
NotImplementedErrorbranch this removes; kind and block class are one-to-one, so the handler'sBgives it a typed block.partsorder what it embeds, a caller's order the display.blocks.md, proposed); raising at ingest would fail a batch on replayed history whose kind the server no longer registers.event_memory/utils.py(moved fromsystem_filters.pyin Add session, source and expansion to EventMemory (speedkick) #1597's history).memmachine.block_kindsentry points,UnknownBlock, base-from-defaults and per-kind tenant options (blocks.md, proposed).Verification
uv run --frozen pytest packages/server/server_tests/memmachine_server/episodic_memory packages/server/server_tests/memmachine_server/common/filter packages/server/server_tests/memmachine_server/common/test_property_keys.py packages/server/server_tests/memmachine_server/common/configuration: 620 passed (SQLite).... pytest packages/server/server_tests/memmachine_server/episodic_memory/event_memory/segment_store -m integration: 111 passed (PostgreSQL via testcontainers).ty check --project packages/server,ruff check,ruff format --check: clean.QueryHit(seed_index,cosine_similarity_threshold, id lists withoutNone, the unconfined walk from a seed with no session,rerankwithoutlimitandmin_score); the same suites rerun at the counts above.TextSegmenter's splitter keepsstrip_whitespace=True, so pieces rebuild a block only up to boundary whitespace; the test asserts that and the one-line fix changes embedded chunk text, so it is a separate decision.🤖 Generated with Claude Code
https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE