Skip to content

[vector store alt 2/9] Create a session's storage when the session is created, never on a request (speedkick) - #1637

Closed
edwinyyyu wants to merge 2 commits into
MemMachine:speedkickfrom
edwinyyyu:alt/session-owned-storage-speedkick
Closed

edwinyyyu wants to merge 2 commits into
MemMachine:speedkickfrom
edwinyyyu:alt/session-owned-storage-speedkick

Conversation

@edwinyyyu

@edwinyyyu edwinyyyu commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Alternative to the [vector store N/13] chain (#1597, #1622..#1616): the two exclude each other. This chain keeps per-project properties_schema (user-defined metadata indexes) and conforms Qdrant to the incarnation lifecycle; SQLite and Milvus follow later. Every PR is a draft.

Purpose of the change

Port of #1622 onto this chain. Every path that creates a session row runs through EpisodicMemoryManager._create_session, which creates the session's segment store partition and vector store collection with the row; the request path binds handles with the stores' lookups and raises SessionPartitionMissingError when a partition is absent. The session's vector store collection is created with the system keys and the project's own properties_schema, resolved and merged as the request path did before (the one place this port differs from #1622). The semantic manager creates its one collection at first use; nothing calls the stores' open-or-create.

Stack

Slice 2 of 9, every PR targeting speedkick; merge bottom-up.

# PR change
1 #1636 Restore per-project filterable properties
2 #1637 (this PR) Create a session's storage when the session is created, never on a request
3 #1638 Make the scripts and examples that write to a project create it first
4 #1639 Make no memory request create a project
5 #1640 Remove open-or-create and close from both stores
6 #1641 Rename both stores' lookup to get_collection and get_partition
7 #1642 Remove custom sharding from the Qdrant store, and disable strict mode on its collections
8 #1643 Mint an incarnation per collection life on Qdrant; delete logically, reclaim by purge
9 #1644 Bound every request to a remote vector store by a configured timeout

This PR's own change is its commit da4f2a1d9; the rest of its diff is the slices under it, and drops out as they merge. Stacked on slice 1; slice 3 is stacked on it.

Verification

ruff check, ruff format --check, ty check, pytest packages/server/server_tests (integration deselected): server 1929 passed, 3 skipped; run at this slice's own commit on 2026-09-15.

🤖 Generated with Claude Code

edwinyyyu and others added 2 commits September 15, 2026 12:00
Reverts the second commit of MemMachine#1606 (a8322a7), "Remove per-project
filterable properties", and keeps its first, the OpenAPI regeneration
under the locked FastAPI 0.141.1: docs/openapi.json is regenerated here
with the same tool, so it differs from the pre-MemMachine#1606 document only by the
ValidationError fields that regeneration added.

A project declares `properties_schema`, a set of caller property keys with
types, on its long-term memory configuration; the event backend merges it
into the vector store collection's indexed schema and rejects filters on
any other `m.<key>`. Under this design the schema is fixed for the life of
the project, a schema change is a delete and a create, and the vector
store maps every distinct schema to one physical collection that all
projects declaring it share. What that costs, per backend, is written
where a tenant will see it: a warning in the configuration docs and the
field descriptions on the server configuration, the project API, the
memory-configuration API and the Python SDK.

Alternative to the one-collection chain (MemMachine#1622..MemMachine#1616): the two exclude
each other.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…quest

The event backend created a session's vector store collection and segment
store partition on the first request that opened the session, so a search
or a write for an unknown session created storage as a side effect, and
the service locator was the only place that knew both stores' create paths.

The owner is the session. Every path that creates a session row runs
through EpisodicMemoryManager._create_session, which inserts the row and,
when the row is new, creates the session's partitions in its segment store
and its vector store (create_episodic_memory_storage); an equivalent
re-create accepts the row and leaves the storage as it is. The request
path binds handles with the stores' lookups and raises
SessionPartitionMissingError when a partition is absent: a session without
its storage is broken, not new. Deleting a session with no open instance
deletes its partitions by key, so a session whose storage was never fully
created can still be deleted. MemMachine.create_session goes through the
manager for the same reason.

The session's vector store collection is created with the system keys and
the project's own `properties_schema`, resolved and merged as the request
path did before.

The semantic manager owns its one collection and creates it, once, at the
storage's first use. With that, nothing calls the stores' open-or-create.

The API is unchanged: the manager's open-or-create still creates a session
a memory request names, now through the same path.

Ported from MemMachine#1622 (4266b09) onto the chain that keeps per-project
filterable properties.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@edwinyyyu

Copy link
Copy Markdown
Contributor Author

Reopen if stack chosen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant