[vector store scale-out] Mint an incarnation per collection life in a SQL-arbitrated registry, so any process may serve any Qdrant or Milvus collection by edwinyyyu · Pull Request #1631 · MemMachine/MemMachine

edwinyyyu · 2026-09-15T17:26:15Z

Purpose of the change

Summary

Any number of server processes can now share one Qdrant or Milvus backend.

Each store used to keep its catalog of collections inside the backend, which has no transactions or unique constraints. So two processes creating the same collection could both succeed, a handle in one process kept writing into a collection another process had deleted and re-created (#1563), and a deletion raced the writes in flight. The VectorStore contract answered this by requiring that a collection be managed by at most one process at a time, leaving the consumer to shard names across processes, which no consumer did.

What this PR does:

A SQL registry arbitrates collections. VectorStoreCollectionRegistry, implemented on PostgreSQL or SQLite by SQLAlchemyVectorStoreCollectionRegistry, holds each collection's name, configuration and incarnation: a UUID minted for each life of the collection. Its primary key decides creation races across processes.
Every record carries its collection's incarnation. A collection deleted and re-created under the same name starts empty, and a handle to the deleted one raises VectorStoreCollectionHandleStaleError.
Deletion takes effect at once; storage is reclaimed later. Deleting a collection queues a tombstone, and the new purge_deleted_collections, run by a sweeper the resource manager starts, reclaims its records after a retention (a day by default), in bounded rounds.
Qdrant and Milvus share one base class, RegistryBackedVectorStore, which makes every registry call; each store supplies its storage preparation, its handle and a purge round.
The Milvus store is reworked: the async client, a typed field per declared property, AUTOINDEX's HNSW_SQ index with a tuned search, and reads at Milvus's default consistency, Bounded.

Also in this PR:

Queries answer record UUIDs and scores only; VectorStoreCollection.get, close_collection, return_vector and return_properties are removed. Fixes Semantic memory's update_feature reads its vector back to rewrite it: concurrent updates lose an embedding, and a missed read fails the update half-applied #1721.
A record's UUID names it within its collection only.
The stores refuse a declared property of another type, a vector of another width, a non-finite query vector and a non-finite score threshold; Record refuses a non-finite vector coordinate.
QdrantConf and MilvusConf gain collection_registry (required), tombstone_retention_seconds and request_timeout_seconds (Bound every request to a remote vector store by a configured timeout #1630, folded in); MilvusConf gains max_varchar_length and purge_batch_size and loses consistency_level.

Breaking, with no migration (pre-GA): existing Qdrant and Milvus data is orphaned, and an existing Milvus native collection must be dropped; every Qdrant and Milvus configuration must name a collection_registry; the semantic feature table gains a vector_uuid column.

Reviewing: start with design/vector_store_horizontal_scaling.md, which links the documents for the registry, the purge, consistency, isolation, and each store. The PRs stacked on this one (see Stack) build on it; this PR does not depend on them.

The registry and a collection's lifecycle

Tables. Every vector store's registry lives in two tables of a relational database, keyed by vector_store_name (the backend's key under resources.databases): collection_registry_ct, a row per registered collection (name, incarnation, configuration, whether it is live, and when it was registered, on the database clock), and collection_registry_gc, the purge queue of tombstones. Every store whose client connects to the same Qdrant, or the same Milvus database, must use the same registry: the same collection_registry database and the same key under resources.databases. The caller starts the registry and hands it to the store in its params. On SQLite the registry needs 3.35 or newer, for RETURNING.
Create. register inserts the collection, pending, under a freshly minted incarnation; the store prepares its storage (on Qdrant and Milvus, the native collection its namespace and configuration share); mark_live, conditional on the incarnation, makes it live. A pending collection holds its name: opening it raises VectorStoreCollectionPendingError, which says since when it has been pending, and creating it raises VectorStoreCollectionAlreadyExistsError. A preparation that raises, or is cancelled, unregisters the pending collection when the registry can; otherwise, as after a crash, it stays pending until deleted like any other. open_or_create_collection waits on a pending collection of its configuration, a second apart, up to 10 attempts; the event backend's service locator, which composes open and a strict create itself, likewise waits, up to 10 attempts a second apart.
Use. A handle checks the registry before each upsert and query, and after each upsert and delete, so a write that raced a deletion raises instead of reporting success; no lock spans a call to the backend. The base class makes these checks, with the input validation, in its handle's upsert, query and delete; each store implements only the backend calls.
Delete. One registry transaction removes the row, by name (unregister) or by incarnation (unregister_incarnation, how a failed creation takes back its own), and queues the incarnation's tombstone. The incarnation is never minted again while its tombstone is queued.
The logic follows the segment store's registry (Overhaul segment store: shared tables with incarnation-scoped tenant keys (port of #1548) #1661) wherever it can: the bounded mint loop, the locking re-check of the queue, the idempotent deletion, the claim under FOR UPDATE SKIP LOCKED.

Purge

A tombstone comes due once tombstone_retention_seconds (a day by default) has passed since the deletion, on the database clock; the retention must outlast any write in flight, so the configuration refuses one below 10 x request_timeout_seconds + 300 s. A too-short retention leaks storage; it never returns a wrong result.
Each call to purge_deleted_collections runs one round on the oldest due tombstone and returns whether it ran one, so calling it until False drains the queue. A round that finds records keeps the tombstone due; one that finds none removes it. A round that raises is retried after a backoff of 30 s doubled per consecutive failure, up to 1 h, and after 10 in a row the tombstone is dead-lettered and reported.
Qdrant deletes the whole incarnation with one filter-delete per round: about 1.3 s per 1M points on 1.19.1, stalling writes and scrolls on the shard but never searches. Deleting in batches collapsed once a vacuum rebuilt the segments mid-purge (7 of 7 runs).
Milvus lists and deletes up to purge_batch_size (10,000) primary keys per round, about 100 ms each through a 1M purge, stalling other tenants at most 0.3 s; one filter-delete of 1M points stalled every tenant for 2.7-9 s (Milvus 3.0.2, reads at Session). A delete raises unless Milvus accepted every key.
The SQLite stores reclaim storage when a collection is deleted, so their purge does nothing.

Contract changes

Consistency: an upsert or delete is durable once it returns, and queries may not reflect it right away; a store that guarantees more says so. Milvus reads at Bounded, at most the server's common.gracefulTime (5 s by default) behind. Qdrant states no more than the contract, as Milvus offers no read-your-writes: its writes wait to be applied, so on one node a query does reflect them at once, but no caller may rely on it.
Queries answer record UUIDs and scores. Event memory resolves a hit through the segment store's derivative rows (new get_segment_uuids_by_derivative_uuids), and semantic memory through the new vector_uuid column on its feature row. get is removed because a stale read in semantic memory's update_feature became a lasting wrong write (Semantic memory's update_feature reads its vector back to rewrite it: concurrent updates lose an embedding, and a missed read fails the update half-applied #1721).
Opening a collection still being created raises VectorStoreCollectionPendingError; None means only that no collection holds the name. The SQLite stores, which have no pending state, never raise it.
Stale handles: operations through a handle whose collection was deleted may raise VectorStoreCollectionHandleStaleError; the Qdrant and Milvus stores always do.
Record UUIDs are scoped to their collection. A Qdrant point's id is now uuid5(incarnation, record UUID), with the record UUID in the payload; the SQLite stores and Milvus already scoped it.
Validation happens before anything is sent, in every store: declared property types (a bool is not an int, and an int is not a float), a record's vector width, a query vector's width and finiteness, and a score threshold that is not finite (None is no threshold). Record requires a vector and refuses a non-finite coordinate.

The Milvus store

Schema: a nullable typed field per declared property (_p_<name>, a datetime as TIMESTAMPTZ with its UTC offset beside it), undeclared properties in one JSON field, and the incarnation as the partition key with partitionkey.isolation.
Index: HNSW_SQ with AUTOINDEX's own parameters (M=18, efConstruction=240, SQ4U codes rescored against FP16), named so every server builds it: AUTOINDEX builds float32 HNSW on 2.6.8 and 2.6.9. A search sets only refine_k = 8. On the 100k-row tenant of a 180k-vector test set (Milvus 2.6.24), recall@10 is 0.988-0.990 against 0.73 with no search parameters, at 2.2-2.3 ms per search; HNSW_SQ uses 0.8 GB less than float32 HNSW at 600k vectors of 768 dimensions (3.0.2).
Client: AsyncMilvusClient. The synchronous client under asyncio.to_thread had let slow reads exhaust the shared executor: with 32 Strong gets in flight beside 8 searchers, 12-21 searches per second against 670-752 on the async client.
Consistency: Milvus's default, Bounded. Session had stalled every search behind the same process's writes (a 5.3-5.8 s p99, against 39-58 ms at Bounded).
Supported servers: Milvus 2.6.8 and later; Milvus Lite is refused. CI runs 2.6.24; the store's tests also pass on 2.6.8 and 3.0.2.

The Qdrant store

Graphs per tenant (m=0, payload_m=16), with the incarnation as a tenant-keyed payload index.
An upsert Qdrant refuses with a 400 or 413 is split in half until it fits; any other error, a timeout included, raises at once.
Upserts and deletes wait for Qdrant to apply them. Not waiting saved about 1 ms at the median and much of the write tail (p99 7-10 ms against 119-170 ms at 800 points/s), but changed neither searches, CPU nor what Qdrant sustains; waiting keeps the writes Qdrant has accepted but not applied to the store's writes in flight, whose callers see the wait as latency, and reports a failure to apply (measurements in the Qdrant design document).

Configuration and deployment

QdrantConf and MilvusConf: collection_registry (required: a relational database under resources.databases), tombstone_retention_seconds (default 86,400), request_timeout_seconds (default 30, bounding every request to the backend). MilvusConf also gets max_varchar_length (65,535) and purge_batch_size (10,000), each within a server setting, and its uri defaults to http://localhost:19530.
The wizard points the registry at its SQLite database, the sample configurations at profile_storage, and the Helm chart at its PostgreSQL database, db_postgres; the configuration docs list the new keys.

Tests

collection_lifecycle_contract.py, mixed into the Qdrant (local, REST, gRPC) and Milvus tests: stale handles, empty re-creation, lost creation races, idempotent deletion, purge, and a write landing under a dead incarnation.
test_sqlalchemy_collection_registry.py runs the registry on SQLite and PostgreSQL, including concurrent creators, pending collections and doubly claimed tombstones.
test_registry_backed_vector_store.py covers the creation flow: invisible while pending, a failed preparation freeing the name, and a deletion during preparation.

Stack

22 open PRs: one independent PR, and the vector store tree of short parallel branches. Every PR's GitHub base is main, since the branches are in a fork and a pull request can target only this repository's branches; the on column gives the order the PRs build on each other instead. A stacked PR's diff on GitHub includes the PRs under it until they merge.

Independent of the vector store tree, directly on main:

#	PR	change	on
—	#1624	Make no memory request create a project	`main`

The vector store tree. Each PR builds on the one in its on column; PRs on the same parent are parallel branches and do not depend on each other. #1631 is maybe superseded by #1733–#1736, which hold the same changes split in four, with review changes since; it stays open, and nothing builds on it. #1670 and #1702 sit beneath #1627, whose code depends on them. Until the PRs under it merge, their changes show in a stacked PR's diff.

#	PR	change	on
[vector store scale-out 1/6]	#1671 (merged)	Remove custom sharding from the Qdrant store (port of #1654)	`main`
[vector store scale-out 2/6]	#1733	Answer vector store queries with record UUIDs and scores, and refuse invalid inputs	`main`
[vector store scale-out 3/6]	#1734	Arbitrate vector store collections in a SQL registry, with an incarnation per collection life and a purge	#1733
[vector store scale-out 4/6]	#1735	Move the Qdrant store onto the collection registry	#1734
[vector store scale-out 5/6]	#1736	Move the Milvus store onto the collection registry, against a Milvus server	#1735
—	#1631 (this PR)	Maybe superseded by #1733–#1736, which hold its changes split in four: mint an incarnation per collection life in a SQL-arbitrated registry, so any process may serve any Qdrant or Milvus collection	—
[user properties 1/2]	#1670	Remove per-project filterable properties (port of #1606)	#1736
[user properties 2/2]	#1702	Keep user properties out of the vector store	#1670
[vector store scale-out 6/6]	#1627	Make a vector store one collection, with string-keyed partitions	#1702
[session storage 1/2]	#1622	Create a session's storage with the session, never on a request	#1627
[session storage 2/2]	#1625	Remove open-or-create from both stores, and close from the segment store	#1622
[search results]	#1663	Score every vector search by cosine similarity, and name scores for it (port of #1598's cosine half)	#1627
[sqlite store fixes 1/7]	#1460	Publish vector index files atomically (but not durably)	#1663
[sqlite store fixes 2/7]	#1469	Never reuse a row id in SQLiteVectorStore	#1460
[sqlite store fixes 3/7]	#1672	Own the search engine's concurrency in the store, not in each engine (port of #1612)	#1469
[sqlite store fixes 4/7]	#1673	Serialize a partition's writes so the engine sees them in order (port of #1607)	#1672
[sqlite store fixes 5/7]	#1674	Refuse a pending row replay cannot honor, instead of dropping it (port of #1608)	#1673
[sqlite store fixes 6/7]	#1675	Take SQLite's write lock at BEGIN, not at the first write (port of #1609)	#1674
[sqlite store fixes 7/7]	#1676	Give every write a fresh row id, so a key names one version (port of #1610)	#1675
[qdrant options]	#1618	Let a deployment tune a Qdrant collection's HNSW, optimizers and quantization	#1663
[declared schema 1/2]	#1628	Make a vector store filter only on the properties it declares	#1663
[declared schema 2/2]	#1616	Close the filter union, and make negation the complement on every backend	#1628

This PR is its 162 commits and 1 merge of main, 16705bb5c, b8a5bcc4a, 7861ec149, b3cda6efc, 095e8d571, 48e80e2e0, c09b0509a, 2f582a182, 001523159, e5b96b93f, 137ff2d00, 2f82c80ad, 5d80939a0, 482dbaa10, 531b9cf86, 6786d0a80, e682be1db, cc6d75679, c5b3d94e3, 8d01d8bbe, e14df9c4d, 5c6e6c19a, 52f613c76, 8d5cfe9f6, ee5a5bab2, 9c8c7e70f, f5f345c2d, 96a9607af, fd12174d8, abe226b1d, a2be45026, a1c52389d, f3a1d65ba, 6bcc6e4bf, ec8ab75fa, abd11ea7c, 806f35a42, 21d117ee3, ce08dfd70, 27ac1133f, 979d27fe2, 8b2515443, 931e0791e, 2d50a843e, ea35e3be8, 89c38366e, b0c844b21, dc7a5e034, 75d15e47f, 5a208ac74, cb0e1cc48, 1d7cb9fec, 2435f0a49, 29b9f1659, 729f5b60d, 38684b385, aa981d86e, 2ba949b6e, 146be2f9e, d034a1b15, e61c8c457, a3d84a46f, 28fa41ae4, ce2e9656e, ffd25a7a3, fef880281, 1aaa809ba, 72b8aa3cb, 53d41d1c8, 1889c5370, d3231dbad, 4f9874fa5, ec159da44, e201c3edf, 03e612859, 87f37ff85, 955ce216e, af6320b1c, 26b578d2a, 16d29ecd1, 1483bf16b, fa01a92e8, 20c799c47, 5fc8d6cc0, cf93f5151, 8a983f0b3, 0b10e316c, b35a92129, 7ee4de4ea, 83c16e97f, c7d34652c, a03804ad2, 2a7cb941f, 05ea97595, 14eea7d35, 679f9f6d5, 4719c6d0d, ef4d4df82, e09ba662c, e5e2d6191, 4417815ed, 22d501903, f8280cf09, baf8bc0a0, 0e12706c9, 875bfce15, 61d05b652, 0bdbffb25, f8752839b, 985696691, 389a27ac4, 6f7580a16, d0fd7e6bf, d40c75242, e0c8f9648, 88443f350, 79b176ff8, d556452b4, afe7734c6, b62c0da25, b0f4dbe9b, e557a5af3, 2f154bcfd, 69dbab973, 5467b240b, 9eae61807, 18baa1618, 62def515b, af4a17ad6, 9e473927d, 6a099089e, 632ae439c, 3013d32bc, 7a017292d, d06425d56, 640ca4c21, 53f46873a, 2ff5150be, fb772500b, 7e290ca7c, 0cfc4f684, ea9b8ef0d, e101638d1, fb6b0e040, a598759ea, 4a4e27f4a, 58c3438bb, 05aea74bf, cdfbdf6d6, 11c92db93, 1bd021abb, 1d6ef628a, 277559eb6, 10aaa9801, 1a35a3975, 53e410734, 18ee574f0, ba644cc0f, 635da2ac6, 2fb2ce22b, c497574ad, fbb37c3a0, 89ba59636 (a merge of main), directly on main. Nothing builds on it: #1733–#1736 hold the same changes split in four.

Verification

At the head 89ba59636 (2026-10-01), the merge of main at 82b6c6f58 (#1713, #1541):

ruff check, ruff format --check and ty check clean, as CI runs them (uv run --frozen --all-extras ty check --project packages/server).
The full server suite without integration tests passes (2083).
The merge's integration tests (segment store, event memory, long-term memory) pass against PostgreSQL (113, 3 skipped); the vector store figures below are from 2fb2ce22b, and the merge touched no vector store file.
In test containers: the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (272), and with the long-term memory and vector-store semantic storage ones at ba644cc0f (277, 3 skipped); the episodic and semantic memory integration tests pass at e101638d1 (1026, 91 skipped for want of credentials or servers).
The Milvus store's tests passed against Milvus 2.6.8, 2.6.24 and 3.0.2 at 18baa1618 (54 each).
Where a commit message says a test fails against the code before it, that was run.

Per-commit verification log

On this base (main at d556f4b38), 2026-09-25: at every one of the 58 commits through 2ba949b6e, ruff check and ruff format --check clean and ty check clean as CI runs it (uv run --frozen --all-extras ty check --project packages/server). The full server suite without integration tests passes at each commit where the rebase onto #1661 and #1682 needed a resolution (b3cda6efc, 095e8d571, 2f82c80ad, 482dbaa10, 8d5cfe9f6, ee5a5bab2) and at 729f5b60d, 38684b385 and 2ba949b6e, 2013 tests at 2ba949b6e. 146be2f9e renames claim_due to claim_purgeable_incarnation: ruff check, ruff format --check and ty check clean, and the vector store tests without integration tests pass there. d034a1b15 adds only a comment on the purge queue's name column: ruff check and ruff format --check clean. From the 2026-09-28 code review: e61c8c457 gives the Helm chart's and configuration.event.yml's Qdrant store a collection_registry (the event sample fails validation without it and parses with it; the chart was not rendered, Helm not installed); a3d84a46f refuses a float for a Milvus property declared int (the new case fails before it); 28fa41ae4 gives the Milvus timeout test its own namespace (with the shared one it fails when run after TestCollectionLifecycle::test_create_open_delete); ce2e9656e dead-letters a tombstone after 10 consecutive failed purge rounds (its four behavior tests fail before it); ffd25a7a3 completes a Milvus native collection a failed creation left part-built (create, index and load as separate steps, each only when missing; its two tests, a creation refused at the index step and at the load step, fail before it; the Milvus store tests pass against 2.6.24 and 3.0.2, 49 each). fef880281 writes incarnations as hyphenated UUIDs, 1aaa809ba lists a dead Milvus incarnation by its incarnation field (57-63 ms per round of 10,000 against 70-75 ms by key range, 1.4M rows, Milvus 2.6.24), and the head 72b8aa3cb backs a failing tombstone off from its last failed round (its four tests fail before it; claim cost and the alternatives measured in its commit message). At each of the three: ruff check, ruff format --check and ty check clean; at fef880281 the Qdrant and Milvus integration tests pass (216); at the head the registry tests pass on SQLite (33) and PostgreSQL (30), the vector store and resource manager integration tests against Qdrant 1.19.1 and Milvus 2.6.24 (245) and the Milvus store's against 3.0.2 (49), and the full suite without integration tests (2020). At each: ruff check, ruff format --check and ty check clean. At the head, the registry tests pass on SQLite (30) and PostgreSQL (27), the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (241), and the common and installation tests without integration tests passed at 28fa41ae4 (1051). At the head, the integration tests pass in test containers: the registry against PostgreSQL (23), the Qdrant store against Qdrant 1.19.1 (167), the Milvus store against Milvus 2.6.24 (47) and, with the fixture's image swapped, 3.0.2 (47), at 38684b385; at the head, the Milvus store and resource manager integration tests against Milvus 2.6.24 and the Milvus store's against 3.0.2 (46 each: the store no longer refuses an oversized limit itself); and the long-term memory integration tests (3, with 3 skipped for want of a NebulaGraph server). 2026-09-29: 53d41d1c8 and 1889c5370 test both outcomes of losing an open-or-create race (the winner opened, or refused for another configuration), each asserting exactly one lost registration; d3231dbad states in the upsert contract that record UUIDs are the service's own; 4f9874fa5 moves the Milvus store to AsyncMilvusClient (its store tests pass against Milvus 2.6.24, 51); ec159da44 adds only the design documents. At the head ec159da44: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2022); the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (252). Then e201c3edf removes get (its three semantic tests fail on the code before it), 03e612859 states the consistency and reads Milvus at Bounded, and 87f37ff85 restructures the design documents. At e201c3edf and 03e612859: ruff check, ruff format --check and ty check clean. At the head 87f37ff85: the full server suite without integration tests passes (2010); the vector store, resource manager and semantic memory integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (1151), and the Milvus store's tests pass twice in a row with the store at Bounded (50). Then af6320b1c shortens the consistency contract, 26b578d2a answers queries with UUIDs and scores, and 16d29ecd1 updates the design documents. At the head 16d29ecd1: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2001); the vector store, resource manager, semantic memory and episodic memory integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (1249). Then 1483bf16b keeps Milvus's offset fields (its new test checks a declared datetime is stored with its offset), fa01a92e8 removes close_collection, and 20c799c47 and 5fc8d6cc0 touch the design documents only. At fa01a92e8: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2001); the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (230). Then cf93f5151 names the Milvus UUID and key lengths exactly (the key's field is 73 long), 8a983f0b3 derives Qdrant point ids and scopes record UUIDs to their collection (its new test fails with bare ids), and 0b10e316c updates the design documents. At 8a983f0b3: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2002); the vector store, resource manager, episodic memory and semantic memory integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (1252). Then b35a92129 touches the design documents only, and 7ee4de4ea requires a vector on Record. At 7ee4de4ea: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2004); of three runs of the vector store, resource manager, episodic memory and semantic memory integration tests against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24, one failed TestCollectionLifecycleAcrossWorkers::test_two_workers_creating_at_once_agree_on_one_collection once (1251 passed) and the rerun passed (1252); that test did not fail again in 33 targeted runs, alone, in its file, and under concurrent load, and its failure message was not captured. 2026-09-29, from the review of the Milvus and Qdrant stores: 83c16e97f ports #1663's tests of get_segment_uuids_by_derivative_uuids, whose source was already identical to #1663's; a03804ad2 rewrites the Milvus filter compiler with match in #1616's arm order; 679f9f6d5 checks declared property types in every store (its refusal test runs on all four stores); 4719c6d0d and e09ba662c rename PurgeClaim.found to found_any_records; e5e2d6191 gives the registry a params model and a module-level schema; 4417815ed removes consistency_level and defaults the Milvus store's sizes; 22d501903 searches Milvus with ef = max(limit, 64), with a new test of a query for 100 results (Milvus refused ef=64 at k=100 in the benchmark its commit message reports); c7d34652c, 2a7cb941f, 05ea97595, 14eea7d35, ef4d4df82, f8280cf09 and baf8bc0a0 change docstrings, comments and design documents. At the head baf8bc0a0: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2025); every integration test was run in test containers against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24: 1416 passed, 266 skipped for want of credentials or servers, and 8 failed. The 8 are pgvector semantic storage tests (test_get_set_ids_with_older_than*, test_filter_features_by_created_at_with_other_filters) that compare a cutoff read from the host's clock with times the database stamps, 10 ms apart; they pass when run alone at the head and at the merge base (22 each), this PR does not change that storage, and the Docker VM's clock measured 90-460 ms off the host's. The two-workers Qdrant test passed in this run. Qdrant creates collections under one global lock (toc/create_collection.rs, v1.19.1), so a losing creator can wait past its client's timeout; that is the leading hypothesis for its one failure, not verified. Then 0e12706c9 and 875bfce15 record in the Milvus design document how the purge behaves with its listing at Bounded (Milvus 2.6.24, a 200,000-entity incarnation, two runs each): at the resource manager's one-second busy pause as Session did, 20 rounds and none listed twice, with the lowest search p99 for other tenants (4.2-4.5 ms); back to back, 83-87 rounds re-deleting entities not yet applied, against Session's 20. Keeping Bounded is proposed there, not accepted. Then: 61d05b652 annotates the registry's claim as returning AsyncGenerator (typeshed deprecates the AsyncIterator form for @asynccontextmanager; basedpyright reports it, ty does not); 0bdbffb25 renames the claim's outcome any_records_found; f8752839b rewrites the design documents without status markers; 985696691, 389a27ac4 and 6f7580a16 state which stores share a registry and make every params field's description its docstring's text; d0fd7e6bf indexes Milvus with float32 HNSW, M=16, efConstruction=128, searched with ef = max(limit, 128); d40c75242 raises ValueError for a declared property of another type. At the head d40c75242: ruff check, ruff format --check and ty check clean; the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (246). 2026-09-30: e0c8f9648 builds every test Qdrant client with QdrantConf's timeout (30 s) instead of qdrant-client's 5 s default. Under disk load on the Docker VM, 2 of 200 concurrent Qdrant collection creations failed with that 5 s timeout, on creations taking up to 5.0 s (qdrant_creation_race.py). That fits the one unexplained failure of the two-workers test. 88443f350 tests that a schema over Milvus's field cap leaves no registry row and no native collection behind (it passes against Milvus 2.6.24). 79b176ff8 and b0f4dbe9b touch the design documents only; the second records the recall of the new Milvus index and the 1M purge at Bounded. d556452b4 and e557a5af3 let the registry's and the segment store's callers decide to mint again, instead of their inserts. afe7734c6 and b62c0da25 build Qdrant points and Milvus entities in one method each, checking declared types there. At b62c0da25, the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (247). At the head e557a5af3, ruff check, ruff format --check and ty check are clean, and the segment store tests pass on SQLite and PostgreSQL (212). Then 2f154bcfd raises when Milvus does not accept the delete of every primary key sent, in the store's delete and in the purge (its mocked-client test fails before it; a new integration test deletes records the collection does not hold, which Milvus counts as accepted), and 69dbab973 refuses a tombstone_retention_seconds below 10 x request_timeout_seconds + 300 for Qdrant and Milvus (its test fails before it). At the head 69dbab973: ruff check, ruff format --check and ty check clean; the Milvus store tests pass against Milvus 2.6.24 (56); the configuration, resource manager and installation tests without integration tests pass (259). Then 5467b240b moves the Qdrant and Milvus stores' registry calls into the base class RegistryBackedVectorStore: at it, ruff check, ruff format --check and ty check clean, the vector store, resource manager and configuration tests without integration tests pass (531), and the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (248). 9eae61807 indexes Milvus with AUTOINDEX's HNSW_SQ again and sets only refine_k = 8 (recall and latency in its commit message and the Milvus design document); at it, ruff check, ruff format --check and ty check clean and the same integration tests pass (248). 18baa1618 removes the Milvus field-cap test of 88443f350: the contract test of a failed native creation covers its registry half, and its 64 properties were under Milvus 3.0's cap of 256 fields, so it failed against 3.0.2. At the head 18baa1618, the Milvus store tests pass against Milvus 2.6.8, 2.6.24 and 3.0.2 (54 each). Then 62def515b halves a Qdrant upsert only when it is refused with a 400 or a 413, and raises every other error, a timeout included, at once (it had halved and resent on any failure); its two tests of a timeout and a 500 fail against the code before it. At the head 62def515b: ruff check, ruff format --check and ty check clean; the vector store tests without integration tests pass (319), and the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (247). Then af4a17ad6 refuses, with ValueError and in every store, a vector whose width is not the collection's or with a non-finite coordinate, on upsert and on query (before it, on SQLite, a NaN vector matched every usearch query at 1.0, a wrong-width hnswlib query answered, and a NaN sqlite-vec query answered nothing; Qdrant 1.19.1 refused all four cases with a 400); its test runs on all four stores, and its eight SQLite cases fail against the code before it. At the head af4a17ad6: ruff check, ruff format --check and ty check clean; the vector store, episodic memory and semantic memory tests without integration tests pass (988); the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (259), and the episodic and semantic memory integration tests (1026, 91 skipped for want of credentials or servers). Then 9e473927d moves the finiteness check of a record's vector into Record (list[FiniteFloat], refused when the record is built, at 9.15 against 9.10 microseconds per 1,536-dimension record) out of every store's upsert, where it had cost about 11 microseconds per record, a sixth of a Milvus 2.6.24 upsert; the stores keep a record's width check and a query vector's width and finiteness check. Its Record test fails against the type before it. At the head 9e473927d: ruff check, ruff format --check and ty check clean; the common, episodic memory and semantic memory tests without integration tests pass (1690); the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (259). Then 6a099089e refuses a NaN score threshold in every store's query, which Milvus and the SQLite stores had answered with nothing and Qdrant, receiving it as null over REST, with every match; its test runs on all four stores, and its two SQLite cases fail against the code before it. At the head 6a099089e: ruff check, ruff format --check and ty check clean; the common, episodic memory and semantic memory tests without integration tests pass (1693); the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (262). Then 632ae439c states in the base classes' docstrings which underscore attributes form their subclass API and has the Qdrant and Milvus handles read the public config, and 3013d32bc changes only a comment. At 632ae439c: ruff check, ruff format --check and ty check clean; the vector store tests without integration tests pass (337), and the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (262). At 3013d32bc: ruff check and ruff format --check clean. Then 7a017292d gives the base class's three abstract methods full contracts (docstrings only); at 7a017292d: ruff check, ruff format --check and ty check clean. Then d06425d56 renames the storage hook _prepare_storage and makes the base's and the registry's docstrings layout-neutral; at d06425d56: ruff check, ruff format --check and ty check clean, the vector store tests without integration tests pass (337), and the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (262). Then 640ca4c21 and 53f46873a rename the retention floor's constants and check (_RETENTION_FLOOR_TIMEOUT_FACTOR, _RETENTION_FLOOR_EXTRA_SECONDS, _require_retention_floor); at each: ruff check and ruff format --check clean, and the configuration tests pass (140). Then 2ff5150be registers a collection pending, prepares its storage, then marks it live, and removes is_live: the registry tests pass on SQLite and PostgreSQL, the new creation-flow tests (test_registry_backed_vector_store.py) pass, the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (265), the common, episodic memory and semantic memory tests without integration tests pass (1704), and the episodic and semantic memory integration tests pass (1026, 91 skipped). fb772500b and 7e290ca7c correct and tighten docstrings, comments, test names and documents after a review, with the same tests passing. 0cfc4f684 has the event backend's service locator wait for a racing creator's session collection to become live; its test now opens a collection that is pending twice before it is live, and a new test gives up on one that stays pending. ea9b8ef0d has the purge return whether it ran a round; its new drain test fails against the code before it. e101638d1 keys mark_live and the new unregister_incarnation by incarnation alone, and fb6b0e040 refuses a score threshold that is not finite (NaN, +inf, -inf) in every store. Then a598759ea moves the handles' operation tracking, input validation and liveness checks into the base class's upsert, query and delete, the stores implementing only _upsert, _query and _delete (every operation checks liveness even with nothing to send, and a limit of 0 is answered without calling the backend, which Qdrant had been sent); the vector store tests without integration tests pass there (355). 4a4e27f4a raises VectorStoreCollectionPendingError, carrying the registry's new registered_at, when a pending collection is opened, and folds the service locator's open, strict create and wait into one loop; its new locator test (a winner's creation undone, so the loser creates) and its pending-open tests fail against the code before it. 58c3438bb unregisters a pending collection whose preparation is cancelled, in a shielded task; its two cancellation tests fail against the code before it. 05aea74bf refuses a SQLite runtime older than 3.35 in the registry's params. Then cdfbdf6d6 has semantic storage's three deletions read the deleted features' vector UUIDs with one DELETE ... RETURNING instead of a second query binding every feature id in an IN list, which SQLite refuses past 32,766 parameters; its new test, deleting 40,000 features through delete_all and through delete_feature_set, fails with "too many SQL variables" against the code before it and passes on SQLite and PostgreSQL. At cdfbdf6d6: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2075). Then, 2026-10-01: 11c92db93 checks a delete's liveness once, after its call, saving a registry round trip per delete (the check-count contract test now expects one); the vector store and resource manager integration tests pass at it against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (274). 1d6ef628a lets a registry's name be any database key, in a 255-character column, instead of the identifier rule for namespaces and names, which a configuration valid on main with a key like qdrant-main failed at startup. 1bd021abb, 277559eb6, 10aaa9801, 1a35a3975, 53e410734, 18ee574f0 and ba644cc0f come from an audit of the PR's comments, docstrings, user docs and design documents: they drop measured figures, unchecked third-party versions, history, PR references and implementation details from contracts, and correct statements that had drifted from the code (Milvus's negated null check, the purge claim being a row lock on PostgreSQL only, the delete's single check, Qdrant's absent load step); 1a35a3975 also lets VectorStoreCollection allow a store that cannot detect a stale handle, which the Qdrant and Milvus stores still do. At ba644cc0f: ruff check, ruff format --check and ty check clean; the full server suite without integration tests passes (2071). Then 635da2ac6 reflows a docstring, and 2fb2ce22b keeps Qdrant's upserts and deletes waiting for Qdrant to apply them, now passed explicitly, while the store stops stating that a query reflects a write at once, which Milvus at Bounded does not offer either. Waiting was measured against not waiting (Qdrant 1.19.1, 4 CPUs / 4 GB, a fresh collection per phase of 100 tenants x 1,000 points of 768 dimensions, two rounds; table in the Qdrant design document): not waiting saved about 1 ms of write latency at the median and most of its tail at load, but changed neither searches, CPU nor what Qdrant sustains. At 2fb2ce22b: ruff check, ruff format --check and ty check clean; the vector store tests without integration tests pass (354), and the vector store and resource manager integration tests pass against PostgreSQL, Qdrant 1.19.1 and Milvus 2.6.24 (272). Then c497574ad drops from the Qdrant design document one observation, waiting writes stalling past a 120 s client timeout, which 2fb2ce22b's message repeats: it came from a run whose host slept from 17:54 to 19:47, so it measured the sleep. The table's measurements come from a later run with the host awake on AC throughout. fbb37c3a0 rewords what waiting bounds: the writes Qdrant has accepted but not applied, at most the store's writes in flight, rather than how fast writers together send. 89ba59636 merges main at 82b6c6f58 (#1713's bounded filtered-context windows and #1541's timeline neighbors), which touched the segment store and long-term memory without conflicting: ruff check, ruff format --check and ty check clean, the full server suite without integration tests passes (2083), and the segment store, event memory and long-term memory integration tests pass against PostgreSQL (113, 3 skipped).

Before the rebase, on #1671's previous head (hashes below are the rebased counterparts of the commits verified):

At every one of its 45 commits (on #1671), on 2026-09-21 and 2026-09-22: ruff check and ruff format --check clean; ty check clean as CI runs it (uv run --frozen --all-extras ty check --project packages/server). The full server suite without integration tests passes (pytest packages/server/server_tests -m "not integration") at b3cda6efc, 2f582a182, 2f82c80ad, c5b3d94e3, 8d01d8bbe, e14df9c4d, 52f613c76, 8d5cfe9f6, 21d117ee3, 27ac1133f and its head ea35e3be8, 1984 tests at the head. The vector store, resource manager, configuration, installation and config-service tests pass at each of the review commits between them; from 5c6e6c19a on, the long-term memory tests too, with their integration tests: the registry against PostgreSQL, and the stores against Qdrant and Milvus Lite, in test containers. At the head the vector store and resource manager integration tests pass (192). Every test added from ee5a5bab2 on was run with the behavior it pins removed, and fails there. The tests write every time they store in the queue by the database, as the code does, so on SQLite a test's queue holds only the text form the code writes. Of the code review's new tests, the two that remain (client not opened on a bad collection_registry, the SQLite double claim) fail on the commit before their fix; its test of the refusal after close went with the closed gate in 8d5cfe9f6; the SQLite double-claim test went with the clean stamp it guarded in 75d15e47f.

Commits 89c38366e through 29b9f1659 (2026-09-24 and 2026-09-25): ruff check, ruff format --check and ty check clean at each. The Milvus store tests pass against a Milvus 3.0.2 server at b0c844b21, dc7a5e034, cb0e1cc48 and 1d7cb9fec, and against both 2.6.24 and 3.0.2 at 2435f0a49 and the head (47); the registry tests pass on SQLite and PostgreSQL at 75d15e47f, and the configuration, resource manager and installation tests at 5a208ac74. At the head the full server suite without integration tests passes (1946). The five registry tests of the retention rule and the six Milvus store tests of 2435f0a49 fail against the code before their commits.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

The failed-preparation path caught Exception, so a CancelledError during _prepare_storage (a shutdown, a cancelled request) skipped the unregistration and left the collection pending until deleted. It now catches BaseException, runs unregister_incarnation in a task awaited through asyncio.shield, so a second cancellation does not cut it short, and re-raises. The store holds the task until it finishes, as asyncio's documentation requires of a task nothing else awaits. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Unregistration and counting a failed purge round use RETURNING, which SQLite added in 3.35; on an older runtime they fail with a syntax error at the first deletion. The registry's params now refuse such a runtime when the store is built, as the segment store's params do. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

26b578d had semantic storage's delete_all and delete_feature_set read every matching feature's id, then read their vector UUIDs with a second query binding every id in an IN list. SQLite refuses a statement with more than 32,766 parameters, so either deletion failed with "too many SQL variables" once it matched more than that many features (its new test fails so with 40,000 features); PostgreSQL drivers cap a statement's parameters too. Each deletion, delete_features included, is now one DELETE ... RETURNING vector_uuid: no IN list beyond the caller's own ids, one statement fewer, and the vector records deleted are exactly those of the rows deleted. The store already needs RETURNING for add_feature. The two lookup helpers lose their last callers and go. The new test runs on SQLite and PostgreSQL. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

A delete checked the registry before and after the backend call. The check before an upsert keeps a stale handle from writing records under a dead incarnation after its tombstone is purged, where no purge would find them; a delete adds no records, and a stale handle's delete reaches only its own incarnation's, since Qdrant point ids and Milvus primary keys carry the incarnation. The check after alone still raises VectorStoreCollectionHandleStaleError for a delete through a stale handle or one that raced a deletion, and saves one registry round trip per delete. Upserts keep both checks and queries their one, before. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

The comment named the two operations that use RETURNING, which goes stale as soon as another one does. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

The registry checked vector_store_name against the identifier rule for namespaces and collection names, [a-z0-9_]+ and at most 32 bytes, which exists because Qdrant and Milvus embed namespaces in native collection names. The store name is the backend's key under resources.databases, which main leaves unconstrained, and it reaches no native name: it is only the value that keeps registries apart in their shared tables. So a configuration valid on main with a key like qdrant-main failed at startup with an error about a "vector store name" its author never set. The rule goes. The name's columns are 255 characters wide, as the segment store's key column is. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

From an audit against the PR's writing rules: comments and docstrings in the tests MemMachine#1631 changed that described production behavior or third-party behavior a test does not exercise (pymilvus's timeouts and delete results, Qdrant's tenant index, MEMMACHINE_WORKERS, the segment store's row), named a backend the module does not test, narrated a removed behavior, or stated a negative the code contradicts (a helper "not" decided by elapsed time that times out after 30 seconds). The lifecycle mixin's docstring now names all three hooks it requires, and an assertion message no longer blames a cause the registry rules out. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

From an audit against the PR's writing rules: - databases.mdx no longer states the registry key rule that the registry dropped, nor a reason for the Milvus version floor that the Milvus design document contradicts. - The service locator's comments describe what it calls, VectorStore, rather than the registry inside two of its implementations, and name the retry delay's constant rather than restating its value. - The segment store ABC states which derivatives its lookup answers for as part of the mapping, and its registry error says what it is. - The Milvus URI check and the wizard state their reasons without a negative or a copy of MilvusConf's default. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

From an audit against the PR's writing rules: - VectorStoreCollection promised that a stale handle's operations raise, then let a store opt out in its own contract. It now allows both outcomes, raising VectorStoreCollectionHandleStaleError or acting on a collection created again under the name, and a store that detects stale handles states that itself. The query's Raises section lists the invalid filter key every store refuses. - The registry ABC named Qdrant, Milvus and SQL inserts, and promised a claim goes to one purger while saying elsewhere that two may hold one. It now states the guarantee alone, and leaves backoff and dead-lettering to the implementation that has them. - The shared base's comments said a primary key decides creation races, that a pending collection is invisible, that every write is checked before and after (a delete is checked after only), and that tombstones are claimed oldest first with a backoff: the registry's own business. _upsert and _delete now state durability on return and the partial writes a raising call may leave. - declared_properties' docstring no longer describes its callers. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

From an audit against the PR's writing rules: - Milvus's filter compiler claimed every negation holds where a property has no value; a negated null check does not, so it now says a negated comparison or membership test. - The registry described a purge claim as a row lock everywhere; SQLAlchemy's SQLite dialect drops FOR UPDATE, so it is one on PostgreSQL only. The module docstring no longer describes its consumers' backends. - Guarantees stated as negatives (point ids and primary keys never shared, a deleted collection's records not part of a new one) now state what holds, and the Milvus index comment drops the server versions and defaults no code checks. - The Milvus purge comment drops the alternative it rejected, and the SQLite stores state what a stale handle does instead of that it goes undetected. - The Qdrant system-key comment points at the rule it relies on, Record's identifier keys, since this file no longer imports the regex it named. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

The params docstring and the registry design document stated the name column's 255-character width as a rule. No rule on names has been decided, and the width is storage, so the contract leaves it unstated. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

From an audit against the PR's writing rules, across the seven vector store design documents: - They narrated changes and PRs (what "is gone", what the configuration "gains" and "loses", what MemMachine#1627, MemMachine#1663, MemMachine#1625, MemMachine#1661 and MemMachine#1671 do, and "the previous design"), and claimed MemMachine#1468 fixed by two PRs that are open. Each now states the design, keeping links to open issues. - They had drifted from the code: a delete checks liveness once, after its call; Qdrant has no load step and Milvus loads on every preparation; the handle's writes run with qdrant-client's default wait rather than passing it; the Qdrant store states no replicated read delay; and the purge contract promises bounded work per call, while oldest-first rounds are the registry-backed stores'. - Repetition across documents goes: the clients section, the replicated Qdrant measurements and the Qdrant id alternatives now live in one document each, linked from the others; consistency cites the Strong wait's measurement instead of a figure without its conditions; the async-client measurement names its Milvus version. - The UUID text form, a string format rather than a design decision, is gone from the isolation document. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

The Qdrant store stated that on one node a query reflects a write as soon as it returns, which Milvus at its default Bounded level does not offer. The contract needs only a write durable on return, which Qdrant gives once the write is in its write-ahead log, so the store no longer states more than Milvus: a caller relies on neither to see its own writes at once. Whether to keep waiting for Qdrant to apply each write (wait=True) was measured (Qdrant 1.19.1, 4 CPUs / 4 GB; a fresh collection per phase of 100 tenants x 1,000 points of 768 dimensions in the store's per-tenant layout; async gRPC; two rounds): - one writer, p50 / p99, waiting against not: 1 point 1.5-1.7 / 6-8 ms against 0.6 / 2.6-2.8 ms; 10 points 3.4-3.5 / 16-25 against 2.6-2.7 / 7-9; 100 points 23-24 / 88-181 against 21-22 / 33-76. - 800 points/s in batches of 10 on a fixed schedule beside 4 searchers: writes 4.9-5.0 / 119-170 ms against 3.6 / 7-10 ms; searches 1.6 / 5 against 1.7 / 4 ms; a write not waited for was visible to a retrieve within 5-8 ms at p99, 71-145 ms at most. - 2,400 points/s (three times the workload's peak): writes 3.7-4.1 / 712-835 ms waiting; not waiting, 3.6 / 17 ms in one round and 3.5 / 3,969 ms with a 4.4 s stall in the other; searches 4.2 / 17-24 ms either way; CPU the same. - 8 writers back to back, waiting: some writes stalled past a 120 s client timeout. Not waiting saves about a millisecond at the median and much of the write tail, and changes neither searches, CPU nor what Qdrant sustains. A tenant writes a few small batches at a time, served well either way, so the store keeps waiting, now passed explicitly: the writer is paced to Qdrant's apply rate and a failure to apply reaches it. The Qdrant design document records the measurements. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

2fb2ce2's design text and message say that with 8 writers sending back to back, some waiting writes stalled past a 120 s client timeout. That run's host was unplugged and slept from 17:54 to 19:47 (pmset log), so its later phases, those stalls included, measured the sleep, not Qdrant. The table's measurements come from a later run with the host awake on AC throughout, and stand. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

"A writer is paced to what Qdrant applies" read as a limit on how fast writers together send. Waiting bounds the writes Qdrant has accepted but not applied to the store's writes in flight, each one's wait seen by its caller as latency; with many writers in flight, that is many writes. The design document and the upsert's comment now say that. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

edwinyyyu · 2026-10-01T22:40:26Z

Superseded by #1733, #1734, #1735, #1736

edwinyyyu force-pushed the feat/vector-store-incarnations-speedkick branch 5 times, most recently from 7749e57 to c808040 Compare September 15, 2026 19:16

edwinyyyu mentioned this pull request Sep 15, 2026

[vector store alt 8/9] Mint an incarnation per collection life on Qdrant; delete logically, reclaim by purge (speedkick) #1643

Closed

edwinyyyu force-pushed the feat/vector-store-incarnations-speedkick branch 4 times, most recently from 09586e3 to 74c28b7 Compare September 15, 2026 20:30

edwinyyyu mentioned this pull request Sep 15, 2026

Vector store indexing and mapping decision: tenant-defined property indexes, runtime-created native resources, and the logical-to-native mapping (folds #1528, #1534, #1535, #1562, #1564, #1565, #1572, #1573) #1648

Open

edwinyyyu force-pushed the feat/vector-store-incarnations-speedkick branch from 74c28b7 to 5f20796 Compare September 15, 2026 21:40

edwinyyyu marked this pull request as draft September 15, 2026 21:52

edwinyyyu force-pushed the feat/vector-store-incarnations-speedkick branch 2 times, most recently from 4b77ff7 to a37d5e0 Compare September 16, 2026 17:30

edwinyyyu mentioned this pull request Sep 16, 2026

[vector store 2/13] Remove custom sharding from the Qdrant store (speedkick) #1654

Merged

edwinyyyu changed the title ~~[vector store 9/13] Mint an incarnation per partition life; delete logically, reclaim by purge (speedkick)~~ [vector store 10/13] Mint an incarnation per partition life; delete logically, reclaim by purge (speedkick) Sep 16, 2026

edwinyyyu and others added 16 commits September 30, 2026 16:43

State the registry's SQLite minimum without naming its callers

1bd021a

The comment named the two operations that use RETURNING, which goes stale as soon as another one does. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Reflow the Milvus filter compiler's docstring

635da2a

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

edwinyyyu mentioned this pull request Oct 1, 2026

[Bug]: Deleting a session whose memory is not open creates its storage first, only to delete it #1728

Open

Merge branch 'main' into feat/vector-store-incarnations-speedkick

89ba596

Navigation Menu

Repository navigation

[vector store scale-out] Mint an incarnation per collection life in a SQL-arbitrated registry, so any process may serve any Qdrant or Milvus collection - #1631

[vector store scale-out] Mint an incarnation per collection life in a SQL-arbitrated registry, so any process may serve any Qdrant or Milvus collection#1631
edwinyyyu wants to merge 163 commits into
MemMachine:mainfrom
edwinyyyu:feat/vector-store-incarnations-speedkick

edwinyyyu commented Sep 15, 2026 •

edited

Loading

Uh oh!

edwinyyyu commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

edwinyyyu commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the change

Summary

The registry and a collection's lifecycle

Purge

Contract changes

The Milvus store

The Qdrant store

Configuration and deployment

Tests

Stack

Verification

Uh oh!

edwinyyyu commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

edwinyyyu commented Sep 15, 2026 •

edited

Loading