Skip to content

[vector store alt 6/9] Rename both stores' lookup to get_collection and get_partition (speedkick) - #1641

Closed
edwinyyyu wants to merge 6 commits into
MemMachine:speedkickfrom
edwinyyyu:alt/rename-lookup-get-speedkick
Closed

edwinyyyu wants to merge 6 commits into
MemMachine:speedkickfrom
edwinyyyu:alt/rename-lookup-get-speedkick

Conversation

@edwinyyyu

@edwinyyyu edwinyyyu commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Alternative to the [vector store N/13] chain (#1597, #1622..#1616): the two exclude each other. This chain keeps per-project properties_schema (user-defined metadata indexes) and conforms Qdrant to the incarnation lifecycle; SQLite and Milvus follow later. Every PR is a draft.

Purpose of the change

The open -> get half of #1626: a lookup that answers None is a Python get. Identifiers only, by the script in the commit message. The collection -> partition half is not carried: its premise is a store that is one collection.

Stack

Slice 6 of 9, every PR targeting speedkick; merge bottom-up.

# PR change
1 #1636 Restore per-project filterable properties
2 #1637 Create a session's storage when the session is created, never on a request
3 #1638 Make the scripts and examples that write to a project create it first
4 #1639 Make no memory request create a project
5 #1640 Remove open-or-create and close from both stores
6 #1641 (this PR) Rename both stores' lookup to get_collection and get_partition
7 #1642 Remove custom sharding from the Qdrant store, and disable strict mode on its collections
8 #1643 Mint an incarnation per collection life on Qdrant; delete logically, reclaim by purge
9 #1644 Bound every request to a remote vector store by a configured timeout

This PR's own change is its commit 48955b5b6; the rest of its diff is the slices under it, and drops out as they merge. Stacked on slice 5; slice 7 is stacked on it.

Verification

ruff check, ruff format --check, ty check, pytest packages/server/server_tests (integration deselected): server 1918 passed, 3 skipped; run at this slice's own commit on 2026-09-15.

🤖 Generated with Claude Code

edwinyyyu and others added 6 commits September 15, 2026 12:00
Reverts the second commit of MemMachine#1606 (a8322a7), "Remove per-project
filterable properties", and keeps its first, the OpenAPI regeneration
under the locked FastAPI 0.141.1: docs/openapi.json is regenerated here
with the same tool, so it differs from the pre-MemMachine#1606 document only by the
ValidationError fields that regeneration added.

A project declares `properties_schema`, a set of caller property keys with
types, on its long-term memory configuration; the event backend merges it
into the vector store collection's indexed schema and rejects filters on
any other `m.<key>`. Under this design the schema is fixed for the life of
the project, a schema change is a delete and a create, and the vector
store maps every distinct schema to one physical collection that all
projects declaring it share. What that costs, per backend, is written
where a tenant will see it: a warning in the configuration docs and the
field descriptions on the server configuration, the project API, the
memory-configuration API and the Python SDK.

Alternative to the one-collection chain (MemMachine#1622..MemMachine#1616): the two exclude
each other.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
…quest

The event backend created a session's vector store collection and segment
store partition on the first request that opened the session, so a search
or a write for an unknown session created storage as a side effect, and
the service locator was the only place that knew both stores' create paths.

The owner is the session. Every path that creates a session row runs
through EpisodicMemoryManager._create_session, which inserts the row and,
when the row is new, creates the session's partitions in its segment store
and its vector store (create_episodic_memory_storage); an equivalent
re-create accepts the row and leaves the storage as it is. The request
path binds handles with the stores' lookups and raises
SessionPartitionMissingError when a partition is absent: a session without
its storage is broken, not new. Deleting a session with no open instance
deletes its partitions by key, so a session whose storage was never fully
created can still be deleted. MemMachine.create_session goes through the
manager for the same reason.

The session's vector store collection is created with the system keys and
the project's own `properties_schema`, resolved and merged as the request
path did before.

The semantic manager owns its one collection and creates it, once, at the
storage's first use. With that, nothing calls the stores' open-or-create.

The API is unchanged: the manager's open-or-create still creates a session
a memory request names, now through the same path.

Ported from MemMachine#1622 (4266b09) onto the chain that keeps per-project
filterable properties.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
The simple chatbot example, the TypeScript REST demo and the Dify plugin's
add-memory tool wrote to a project without creating it, relying on the
write to create it. Each now creates its project before its first memory
request and accepts 409 as the project already existing. No behavior
changes for them; they stop depending on a write creating a project.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Adding memories to, or searching, a project that did not exist created it,
with the server's default configuration, without the caller's knowledge.
Now only the create-project request creates a project: a write or a
search opens the session and answers 404 for an unknown project, as the
search endpoint already promised; the manager's open-or-create goes.

Two callers depended on the implicit creation. `org_id` and `project_id`
default to `universal`, so the API promises the project
`universal/universal`; the server creates it, once, at startup, and leaves
one that already exists as it is. The MCP add tool names its own project
and has no create-project counterpart, so it creates the project it writes
to, once, and says so. The API doc strings and the OpenAPI document say
which requests create a project.

A breaking API change on `speedkick`.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Nothing calls them since a session's storage is created with the session:
`open_or_create_collection` and `close_collection` leave the vector store
interface and its four backends, `open_or_create_partition` and
`close_partition` leave the segment store interface and its
implementation, and the two config-mismatch errors that only open-or-create
raised go with them. A store creates on `create_*`, strictly, and looks up
on `open_*`, answering None; create-if-absent is the owner's, where the
key's provenance is known.

Source changes are deletions only. The tests that exercised
open-or-create as a fixture use a test-side create-if-absent instead, and
the tests of its own semantics go.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
A lookup that answers None for an absent collection or partition is a
Python get, not an open. Identifiers only, produced by the script below,
plus the two docstring lines that said "opened"; every contract is as it
was.

```sh
set -e
cd "$(git rev-parse --show-toplevel)"
git ls-files -z 'packages/server/*.py' | xargs -0 perl -0pi -e '
  s/\bopen_collection\b/get_collection/g;
  s/\bopen_partition\b/get_partition/g;
'
uv run ruff check --fix --quiet packages/server
uv run ruff format --quiet packages/server
```

The `open -> get` half of MemMachine#1626 (e9783d9); the `collection -> partition`
half is not carried, because its premise is a store that is one
collection.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@edwinyyyu

Copy link
Copy Markdown
Contributor Author

Reopen if stack chosen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant