Repository navigation
Conversation
15ee80d to
1639f5e
Compare
15ee80d to
e1075ec
Compare
a1025f9 to
9b3c734
Compare
9b3c734 to
a8c4b22
Compare
d21a5d0 to
ffcdd2c
Compare
edwinyyyu
left a comment
There was a problem hiding this comment.
- The no-implicit-project guarantee has a gap for non-episodic requests (line comment).
memmachine-server --stdio(app.py) callsinitialize_resource()directly and neverinit_global_memory, soensure_default_projectdoes not run for that entrypoint; "at startup" covers the HTTP lifespan, mcp_http and mcp_stdio only. Same on the closed speedkick counterpart, #1639.
packages/server/src/memmachine_server/main/memmachine.py:710 -- The 404 for an unknown project is raised only inside if MemoryType.Episodic in target_memories. A write with types: ["semantic"] (any request without episodic) to an unknown project is answered 200: episode_storage.add_episodes a few lines up has already inserted rows under the unknown session key (no FK to the sessions table), and the semantic manager ingests without a session lookup. Even with episodic included, the episode rows are persisted before this check runs, so the 404 arrives after the write. The description's "a write or a search opens the session and answers 404 for an unknown project" holds only for episodic requests. Same on #1639.
e1b6052 to
4761a70
Compare
The simple chatbot example, the TypeScript REST demo and the Dify plugin's add-memory tool wrote to a project without creating it, relying on the write to create it. Each now creates its project before its first memory request and accepts 409 as the project already existing. No behavior changes for them; they stop depending on a write creating a project. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 4cae58a)
Adding memories to, or searching, a project that did not exist created it, with the server's default configuration, without the caller's knowledge. Now only the create-project request creates a project: a write or a search opens the session and answers 404 for an unknown project, as the search endpoint already promised; the manager's open-or-create goes. Two callers depended on the implicit creation. `org_id` and `project_id` default to `universal`, so the API promises the project `universal/universal`; the server creates it, once, at startup, and leaves one that already exists as it is. The MCP add tool names its own project and has no create-project counterpart, so it creates the project it writes to, once, and says so. The API doc strings and the OpenAPI document say which requests create a project. A breaking API change on `speedkick`. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit d21a5d0)
…anything, on every entry point A write persisted its episodes before opening episodic memory, and a write or search that targeted only semantic memory never opened it, so a request naming a project nobody created could still insert episode rows under that key and answer 200. Every write now checks the registry first, and a search that does not open episodic memory checks it too; the check is one registry read, and the episodic open, which refuses an unknown project itself, is unchanged. `memmachine-server --stdio` built its resources without setting or starting the module-level MemMachine the tools read, and so never created the default project either; it now starts and stops through the same calls as the HTTP servers. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
4761a70 to
5efe2a3
Compare
…uilt by the composition root A vector store was a factory of logical collections, each identified by a (namespace, name) pair and created with its own dimensions, metric and schema; Qdrant and Milvus shared one native collection among logical collections of equal configuration, under a name derived from a hash of that configuration, and the registry mapped (namespace, name) to it. A store is now one collection: `VectorStore(vector_store_name, vector_dimensions, similarity_metric, indexed_properties)` names its one native collection (or its tables and index files) at construction, every partition of it shares the store's dimensions, metric and schema, and `provision()` creates the store's durable resources idempotently, before `startup`. `create_partition(key)`, `open_or_create_partition(key)`, `get_partition(key)` and `delete_partition(key)` take a string key; a partition is the records carrying its incarnation in the store's native collection (Qdrant, Milvus) or a pair of tables (the SQLite stores), and the registry records what each partition was created under, so a store built with other dimensions, another metric or another schema raises `VectorStorePartitionSchemaMismatchError` instead of reading columns and vectors that are not there. "Collection" names only the native Qdrant or Milvus collection. Vector store names match `[a-z0-9_]+` and are at most 32 bytes, the rule partition and property keys follow, so every name works on every backend: the Qdrant store names its collection by the vector store name, and the Milvus store by `sys_` followed by it, since Milvus requires a leading letter or underscore and names its own internals with a leading underscore. The hash-derived native names go, and with them the per-partition config. The partition registry is keyed by partition key within one store: its tables, `partition_registry_pt` and `partition_registry_gc`, are shared by every store on a database and keyed by vector store name, so the registries of several stores share one database and nothing else. `mark_live` and `unregister_incarnation` take the incarnation alone. On the registry-backed base, the storage a store's partitions share is prepared by `provision()` (`_prepare_storage()`), and the storage a partition keeps of its own by `_prepare_partition_storage(key, incarnation)`, between its registration and its mark; the Qdrant and Milvus stores keep nothing per partition. A purge round works on one incarnation in the store's own collection. `DatabaseManager.get_vector_store(backend, vector_store_name=, vector_dimensions=, similarity_metric=, indexed_properties=)` builds and caches one store per (backend, vector store name), provisioning its registry and then the store; asking for a name again with other dimensions, another metric or other keys is a `VectorStoreConfigurationError`. The event backend's store is named by a UUIDv5 of its embedder id in the UUIDv5 of its backend's key in a namespace fixed for the event backend, written as 32 hexadecimal digits, so neither the key nor the id is constrained; stores on different backends never share a name, so their registries stay apart when the backends share a registry database, since a store name identifies one store among all the stores whose registries share it. Semantic memory's one store is named `semantic_memory` whatever its embedder and holds every org's features in one partition, `semantic_memory`, as main's one collection does. The event backend opens a session's partition with `open_or_create_partition`, which waits on one another worker is still creating. The SQLite stores change shape only: their registry tables become `vector_store_sqlite_pt` and `vector_store_sqlite_vec_pt`, keyed by vector store name and partition key, the pending-operation log is keyed the same way, and per-partition table names embed the vector store name. No migration is provided; existing SQLite vector data is orphaned. Rebuilt as one change on MemMachine#1631: this PR's earlier history carried copies of MemMachine#1631's commits as of `38684b385`, its own commits on them, and mirrors of MemMachine#1631's later commits in its terms. Its code is that history's final tree, which was verified, merged with main and without MemMachine#1624's commits. The design documents are still MemMachine#1631's and describe collections. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Purpose of the change
The simple chatbot example, the TypeScript REST demo and the Dify plugin's add-memory tool wrote to a project without creating it, relying on the write to create it. Each now creates its project before its first memory request and accepts 409 as the project already existing. No behavior changes for them; they stop depending on a write creating a project.
Adding memories to, or searching, a project that did not exist created it, with the server's default configuration, without the caller's knowledge. Now only the create-project request creates a project: a write or a search opens the session and answers 404 for an unknown project, as the search endpoint already promised; the manager's open-or-create goes.
Two callers depended on the implicit creation.
org_idandproject_iddefault touniversal, so the API promises the projectuniversal/universal; the server creates it, once, at startup, and leaves one that already exists as it is. The MCP add tool names its own project and has no create-project counterpart, so it creates the project it writes to, once, and says so. The API doc strings and the OpenAPI document say which requests create a project.A breaking API change on
main.Adaptation for
main:add_memorieskeeps main's default of all memory types (#1555 is not ported); onmaina session's storage is still created lazily by the first open, which #1622 in the chain moves to creation. The deleted-session test keeps only its open assertion: onspeedkickit also pinned that re-creating a session being deleted is refused, and #1677'screate_or_validate_sessionaccepts one whose data matches (#1539, which it ports, refused it); reported on #1677 rather than fixed here.Stack
21 open PRs: three independent PRs, and the vector store tree of short parallel branches. Every PR in the tree has
feat/horizontal-scalingas its GitHub base, and the independent PRs havemain. The branches are in a fork, and a pull request can target only this repository's branches, so the on column gives the order the PRs build on each other. A stacked PR's diff on GitHub includes the PRs under it until they merge.Independent of the vector store tree, directly on
main:mainmainmainThe vector store tree. Each PR builds on the one in its on column; PRs on the same parent are parallel branches and do not depend on each other. #1631 is closed, superseded by #1733–#1736, which hold its changes split in four, with review changes since. #1702 and #1670 sit beneath #1627, whose code depends on them. Until the PRs under it merge, their changes show in a stacked PR's diff.
mainfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingThis PR is its 3 commits,
555ef9cde,17f12e541,5efe2a3a2, directly onmain, independent of the vector store tree: review and merge it in any order.Verification
At every one of its 3 commits, on 2026-09-21:
ruff checkandruff format --checkclean;ty checkclean as CI runs it (uv run --frozen --all-extras ty check --project packages/server); the full server suite without integration tests passes (pytest packages/server/server_tests -m "not integration"), 1921 tests at its head5efe2a3a2.🤖 Generated with Claude Code
https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn