Design: MemMachine server redesign (tenants, event memory API, configuration, lifecycle, schema management) - #1579

Summary

A design proposal, no code: design/server_redesign.md and one specification per component under design/components/. It started as a tenant lifecycle design and now covers the server, because the lifecycle cannot be fixed inside the current API, configuration and wiring. Against speedkick, which carries the segment store overhaul it refers to (#1548, the merged copy of #1545). Tracking: #1574. Line references are to speedkick at 7752e4c. Settings are named, never given numeric defaults. The premise: this is a clean slate; the first deployment carries no data forward.

Decisions the document records

• Tenant and components have separate lifecycles. A tenant is a name, an id and a state (active, deleting, deleted); it exists the moment its row is inserted and holds nothing by itself. Each component's per-tenant resources (the event store's partition, a memory subsystem's stores) are enabled, configured and disabled on their own through PUT/DELETE /v1/tenants/{id}/components/{name}, each with its own state, jobs and tombstone; a template is the set of sections a create enables at once. Deleting a tenant disables every component and retires the id. The id is a UUID minted per lifetime and is the physical key in every store (no incarnation anywhere); a deleted tenant's row keeps only the id and deleted_at (no former name, so no personal data outlives the tenant in the registry).
• Tombstones. A component tombstone is swept in rounds: a round that purges something is followed by another; a round that finds nothing on its first batch stamps clean_at; after a retention of at least a day on the database clock a verification round runs, and only if it finds nothing again is the tombstone removed (finding something clears clean_at and the rounds start over). The tenant row goes when its last component tombstone does. Every remote client must have a request timeout, which is what makes the retention safe.
• Reconciler without held locks. A claim is an atomic state transition on the job row (pending to running), one job per claim, on its own connection; the hook runs with no database lock held on any row. A running job whose claim is older than reclaim_after is taken over, which is the one thing time decides in the reconciler and is liveness only: a slow step may run twice, and idempotent hooks plus store fences make the second run a wasted write. Ordering comes from the state machine on the component row, idempotent hooks, strict creates, and the sweep step calling delete before it purges, so a resource a late provision created is still unlinked and removed. Eligibility is computed at claim from last_run_at, last_outcome and attempts (backoff capped, reset on success; a replay with more log continues at once); reset_replay updates existing rows only.
• Events are the caller's ingestion type, with session_id and source_id beside timestamp (system fields with server semantics: the one total order is within a session; the source is the filterable identity, its readable name a context part) and a Context that is a mapping from part kind to one registered Pydantic part. Blocks are a registered family of kinds; a segment is one block, so a block's kind is a system field of the segment. Event and Context are reshaped for one reason: to prohibit dynamic index creation. The event store keeps events and a per-tenant log of additions and deletions with commit-ordered positions; every field round-trips, the timestamp with its ingested offset. Ingest responds 202 once durable and the replay job is pending; processing is the replay job per memory subsystem, one consumer each, which advances its watermark only after both derived stores hold the batch, and ?wait= polls the watermarks. The job carries no content and no ids: the log holds them. No rebuild of derived data: a reprocessing is a new ingestion into a new tenant. Log compaction is an operator command.
• Keys and fencing. Every store fences on its own registry: the SQL-backed stores on a row beside their data plus a purge queue written in the delete's transaction (a row is live, a queue entry is dropping), so their fence is in-statement and they depend on no second component; the others on a three-state row in the shared key registry, injected as a view scoped to one store. Stores whose data is elsewhere read the row, perform the remote write acknowledged as applied, and read the row again; no lock spans remote I/O. Creates are strict; idempotency is the component's provision, which knows the key's provenance. Data consumers hold stateless handles (EventPartition, SegmentPartition, VectorCollection), the store's operations bound to one key with no method taking a key, so a wrong key is unrepresentable past construction.
• Vector store: containers are one per embedder the composition built, named by the embedder id, provisioned by the schema command; a native per-tenant object is named by the key's hex; the store rejects undeclared property keys; datetimes are stored as integer microseconds where a backend has no datetime type. A table surveys Qdrant, Milvus, pgvector, Pinecone, S3 Vectors, Weaviate, Chroma and the two SQLite stores; the SQLite stores move to shared tables.
• Concurrency scope (process < host < cluster; Feat: Declare concurrency scope and data-plane contracts on storage ABCs (collection registry stack 4/5) #1531 is the reference): every resource computes its scope from its constructor arguments, the deployment declares the scope it runs at, and startup refuses anything narrower. The reconciler is a role.
• Composition and settings, five rules. Fixed topology with pluggable slots chosen by kind; composition is one Python function; settings are data, ServerSettings the root of the file; scoped views wherever a shared resource serves several holders; three scopes told apart by identity (resources, configured objects, call arguments). EpisodicMemoryManager is the resource; it builds one EpisodicMemory per request from the tenant's handles, embedder, format and cached segmenter and deriver, and runs the search stages.
• Search is two stages. EpisodicMemory.query is the vector stage with one limit and one threshold (min_similarity, cosine) and returns hits in descending score, each a context window with the matched segment marked; the manager runs the optional reranking stage over the rendered windows with its own candidate count and threshold, so over-fetching is one limit set above another. SearchOptions is one model: the tenant's search section with every field set, the request's overrides with every field optional. Events are not returned by search. Expansion walks the anchor's session in the one total order (session, timestamp, event position, index, offset), counted in segments, the one unit, and returns the two sides without the anchor, which the caller holds, so a neighborhood survives an anchor that fails the filter (Let a filtered-out seed still anchor its segment context #1498); MCP tools expose no expansion counts to the model.
• Eviction, from agentic_expansion where it runs in production: with it on, encode treats near-duplicate derivatives as a cluster (stored neighbors above a cosine threshold, earlier derivatives of the same batch, itself) and trims a cluster over the target size from the temporal middle, deleting stored derivatives and skipping new ones; segments and events are untouched, so the loss is to search only. A mutable tenant option whose threshold is calibrated per embedder.
• Properties and filtering. System fields are typed parameters stored under a reserved memmachine_ key namespace. User properties are scalar, bounded, immutable, copied verbatim to derived data; filter indexes are declared once per vector store, never dynamically; undeclared keys are filtered in the segment store, whose expression indexes a deployment names in settings and the schema command creates. A filter is a constructed closed-union tree, a JSON object at the API and in MCP. Routing by backend capability: the declared part is evaluated during the search, the undeclared part afterward by the segment store with bounded widening; the store scores nothing by id. Every count is a maximum.
• API /v1/tenants, /v1/tenants/{id}/components/{name}, /v1/tenants/{id}/events, /v1/tenants/{id}/episodic-memory/*, one error handler, closed error-code set including component_not_enabled and component_not_active, no tracebacks in responses.
• Schema management: Alembic per component with its own version table and an include_object limited to its tables; memmachine schema upgrade is the only thing that runs DDL that is not tenant-specific, creates the declared property indexes outside its transaction, and provisions containers; serve fails only when a database is behind its code, so a previous release may restart after a migration under expand/contract.
• Not wired in: short-term memory, semantic memory, declarative memory and the graph stores, the retrieval agent.

Component specifications

design/components/: one file per component (tenant service with registry, component enablement, jobs, reconciler and tombstone pass; key registry; event store; segment store; vector store; EpisodicMemory; EpisodicMemoryManager; context; blocks; ingest service; filters and properties; server and settings), each with its API, storage, fencing, settings and the changes required of an existing component; identifiers are typed UUID throughout. Race matrices tabulate every concurrent pair on a tenant or component row and on the data path. Every SQL-backed component's schema is given with SQLAlchemy types mapped to PostgreSQL and SQLite, constraints and indexes. Every contract is an ABC; every store is two, the store (lifecycle, and constructing handles) and the handle (data). Types are few and each answers one need (StoredEvent, IngestResult, LogEntry, SearchHit, Tenant, SearchOptions); listings take the last position or name as the cursor. Job kinds are four (provision; delete, the unlink; sweep; replay), all defined by the tenant service; MemorySubsystem extends TenantComponent with replay and watermark.

One naming scheme for values: settings, tenant configuration (per component row, with options), templates, overrides, defaults, partition configuration, request parameters, job arguments; in identifiers Settings and Config.

What must be built first

A section names what the first deployment freezes and so must precede the first tenant (the key scheme, including the physical key inside each store, and the tenant tables with tombstones; the event store with positions; stored-record conventions; a registry row per vector key; the public surface; Alembic from the first migration), what must land before churn, what can follow at any time, and the narrowest first deployment.

Relation to issues

Listed in the document's "Relation to open issues": #1548, #1530, #1531, #1571, #1575, #1576, #1577, #1572, #1573, #1564, #1565, #1537, #1563, #1535, #1570, #1542, and #1574 as the tracking issue.

Open questions

Hierarchy (flat proposed; an org-level share if semantic memory returns); event size limits; readable metadata as a json block kind or a designed field; retention by age, source or session as a job kind; which vector backends the first release implements; selecting the ungrouped stream in a search; log compaction as a scheduled duty.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MbYdqGZsuws6Z2WHYfCCR5