Bound every request to a remote vector store by a configured timeout by edwinyyyu · Pull Request #1630 · MemMachine/MemMachine

edwinyyyu · 2026-09-14T21:39:19Z

Purpose of the change

A remote vector store request had no bound on this side: qdrant-client bounds its own at 5 s by default, pymilvus not at all, so a Milvus write could hang a request indefinitely, and a Qdrant deployment could not choose its bound.

request_timeout_seconds on QdrantConf and MilvusConf: an integer number of seconds, default 30, required positive (zero would expire every request on arrival), named for its unit like max_retry_interval_seconds on the embedder and language-model configurations. Why the two stores differ in the diff: AsyncQdrantClient(timeout=) is a client-wide default, the httpx request timeout on REST and the deadline of every gRPC call, so the one kwarg where the client is built bounds every request QdrantVectorStore makes and the store itself does not change (qdrant-client takes an integer and applies math.ceil to it). MilvusClient(timeout=) bounds connecting and reconnecting only; each data call takes its own timeout=, so the Milvus store carries the value and passes it on every request it makes. A cfg.yml that predates the option loads unchanged. The sample configurations, the configuration docs' parameter table and their Qdrant and Milvus examples show the option.

Stack

23 PRs on main: 5 independent ones, and three consecutive stages numbered on their own: the vector store scale-out, the SQLite store fixes and the vector store contract changes. A stacked PR's diff on GitHub is cumulative until the PRs under it merge.

Independent PRs, each directly on main; review and merge in any order:

#	PR	change
—	#1541	fix(event-backend): make expand_context return timeline-neighbor episodes
—	#1661	Overhaul segment store: shared tables with incarnation-scoped tenant keys (port of #1548)
—	#1630 (this PR)	Bound every request to a remote vector store by a configured timeout
—	#1682	Hand the Qdrant vector store a metrics factory, so its tracker emits (port of #1532)
—	#1624	Make no memory request create a project

The vector store stages, consecutive: each stacked on the one below. [vector store scale-out 1/5] is directly on main and 2/5 on it alone; they do not depend on the independent PRs. From 3/5 up, the chain's history also carries the independent PRs' commits beneath it, since the later stages were written on top of them ([vector store contract 2/6] on #1624's session policy, for one); so a merge of an independent PR rebases the chain without conflict, and until they merge those PRs' changes show in a stacked PR's diff.

#	PR	change
		Stage 1, the vector store scale-out: what horizontal scalability without sharding requires.
[vector store scale-out 1/5]	#1671	Remove custom sharding from the Qdrant store (port of #1654)
[vector store scale-out 2/5]	#1631	Mint an incarnation per collection life in a SQL-arbitrated registry, so any process may serve any Qdrant or Milvus collection
[vector store scale-out 3/5]	#1670	Remove per-project filterable properties (port of #1606)
[vector store scale-out 4/5]	#1702	Keep user properties out of the vector store
[vector store scale-out 5/5]	#1627	Make a vector store one collection, with string-keyed partitions
		Stage 2, the SQLite store fixes, on [vector store scale-out 5/5]: re-derived on the one-collection store.
[sqlite store fixes 1/7]	#1460	Publish vector index files atomically (but not durably) (as merged into `speedkick`, #1588)
[sqlite store fixes 2/7]	#1469	Never reuse a row id in SQLiteVectorStore (as merged into `speedkick`, #1589)
[sqlite store fixes 3/7]	#1672	Own the search engine's concurrency in the store, not in each engine (port of #1612)
[sqlite store fixes 4/7]	#1673	Serialize a partition's writes so the engine sees them in order (port of #1607)
[sqlite store fixes 5/7]	#1674	Refuse a pending row replay cannot honor, instead of dropping it (port of #1608)
[sqlite store fixes 6/7]	#1675	Take SQLite's write lock at BEGIN, not at the first write (port of #1609)
[sqlite store fixes 7/7]	#1676	Give every write a fresh row id, so a key names one version (port of #1610)
		Stage 3, the vector store contract changes, on [sqlite store fixes 7/7].
[vector store contract 1/6]	#1663	Answer with cosine scores and uuids, not vectors and stale properties, and require a vector on the record (port of #1598 and #1603)
[vector store contract 2/6]	#1622	Create a session's storage with the session, never on a request
[vector store contract 3/6]	#1625	Remove open-or-create and close from both stores
[vector store contract 4/6]	#1618	Let a deployment tune a Qdrant collection's HNSW, optimizers and quantization
[vector store contract 5/6]	#1628	Make a vector store filter only on the properties it declares
[vector store contract 6/6]	#1616	Close the filter union, and make negation the complement on every backend

This PR is its 6 commits, f4b5aea9c, 95dab3075, b9f19fe3d, 6a7be31de, 874ec4707, ce97ac053, directly on main, independent of the others: review and merge it in any order among the independent PRs (a later one touching the same file rebases). The stages' history carries the same change from [vector store scale-out 3/5] up, so their diffs include it until it merges.

Verification

At every one of its 6 commits, on 2026-09-21: ruff check and ruff format --check clean; ty check clean as CI runs it (uv run --frozen --all-extras ty check --project packages/server); the full server suite without integration tests passes (pytest packages/server/server_tests -m "not integration"), 1916 tests at its head ce97ac053.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

The Qdrant and Milvus clients were built without a timeout, so a remote write could hang a request indefinitely. `request_timeout` on QdrantConf and MilvusConf, in seconds, is passed to the client; it is required, with no default, so a deployment states how long it is willing to wait, and the configuration wizard supplies 30 seconds as the starting point. The sample configurations and the configuration docs show the option. A breaking configuration change on `speedkick`. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 5c669ee)

A required field bounds only the deployments that added it and fails the rest at load; a default bounds every deployment, including one whose cfg.yml predates the option, and matches every other field on QdrantConf and MilvusConf. The wizard no longer carries the value: it constructs the confs and the field supplies it. Zero and negative values are rejected at load rather than handed to httpx as the request timeout and to pymilvus as the gRPC deadline, where zero expires every request on arrival. Without the option, qdrant-client already bounded a request at 5 seconds (httpx's default for REST, DEFAULT_GRPC_TIMEOUT for gRPC); pymilvus passed no deadline, so a Milvus request could wait forever. The default applies 30 seconds to both. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 98f09b2)

Neither client's keyword carries the unit, and the field mirrors neither (both take `timeout`), so it follows max_retry_interval_seconds on the embedder and language model configurations instead. The sample configurations lose their "seconds" comments, which the name now carries. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 0fe31cd)

MilvusClient's constructor timeout is the time it waits for the channel to become ready, at construction and on reconnect; a request is bounded only by the timeout passed to that request, and pymilvus keeps no default for it, so every request the store made had no deadline. The store now takes request_timeout_seconds and passes it on every request; the client keeps it as its connection bound. A test wraps every request method and checks the timeout reaches each call. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 92f536e)

qdrant-client takes its timeout as an int and rounds a fraction up, so a fractional value was honored by pymilvus and silently changed for Qdrant. An int is honored exactly by both, and matches max_retry_interval_seconds on the embedder and language model configurations. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 5e9f363)

…e the stores are configured MilvusVectorStoreCollection.get() was the one client request without `timeout=`; the spy test now exercises it, so a request without the timeout fails the test. The configuration parameter table gains `request_timeout_seconds`, the databases page's Milvus example carries it, and the configuration page gains a Qdrant example beside the Milvus one. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

edwinyyyu · 2026-09-22T20:16:47Z

@malatewang These changes (or similar) will get folded into #1631 since a timeout is required for #1631. This was mistakenly taken out of the chain.

The Qdrant and Milvus clients were built without a timeout, so a remote write could hang a request indefinitely. `request_timeout` on QdrantConf and MilvusConf, in seconds, is passed to the client; it is required, with no default, so a deployment states how long it is willing to wait, and the configuration wizard supplies 30 seconds as the starting point. The sample configurations and the configuration docs show the option. A breaking configuration change on `speedkick`. Folded from MemMachine#1630 into MemMachine#1631, whose purge relies on a bounded request: request_timeout sits beside collection_registry in the configuration, the tests and the samples, and MemMachine#1630's is_distributed and registry_replication_factor lines are dropped (both fields are gone below). Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 5c669ee) (cherry picked from commit f4b5aea)

A required field bounds only the deployments that added it and fails the rest at load; a default bounds every deployment, including one whose cfg.yml predates the option, and matches every other field on QdrantConf and MilvusConf. The wizard no longer carries the value: it constructs the confs and the field supplies it. Zero and negative values are rejected at load rather than handed to httpx as the request timeout and to pymilvus as the gRPC deadline, where zero expires every request on arrival. Without the option, qdrant-client already bounded a request at 5 seconds (httpx's default for REST, DEFAULT_GRPC_TIMEOUT for gRPC); pymilvus passed no deadline, so a Milvus request could wait forever. The default applies 30 seconds to both. Folded from MemMachine#1630 into MemMachine#1631: the timeout argument leaves the configurations that also name a collection_registry, which stays; the new positivity checks name a collection_registry, which MemMachine#1631 requires. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 98f09b2) (cherry picked from commit 95dab30)

Neither client's keyword carries the unit, and the field mirrors neither (both take `timeout`), so it follows max_retry_interval_seconds on the embedder and language model configurations instead. The sample configurations lose their "seconds" comments, which the name now carries. Folded from MemMachine#1630 into MemMachine#1631: the rename reaches the configurations and tests that also name a collection_registry. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 0fe31cd) (cherry picked from commit b9f19fe)

MilvusClient's constructor timeout is the time it waits for the channel to become ready, at construction and on reconnect; a request is bounded only by the timeout passed to that request, and pymilvus keeps no default for it, so every request the store made had no deadline. The store now takes request_timeout_seconds and passes it on every request; the client keeps it as its connection bound. A test wraps every request method and checks the timeout reaches each call. Folded from MemMachine#1630 into MemMachine#1631, whose Milvus store no longer keeps its registry in Milvus: the timeout reaches every request this store makes, the existence check under the native-creation lock and the purge's existence check, query and delete included, and the per-namespace registry calls this commit bounded are gone. The test follows the store's requests, the purge's among them, and no longer expects the registry's insert. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 92f536e) (cherry picked from commit 6a7be31)

qdrant-client takes its timeout as an int and rounds a fraction up, so a fractional value was honored by pymilvus and silently changed for Qdrant. An int is honored exactly by both, and matches max_retry_interval_seconds on the embedder and language model configurations. Folded from MemMachine#1630 into MemMachine#1631: the whole seconds reach the configurations and tests that also name a collection_registry. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit 5e9f363) (cherry picked from commit 874ec47)

…e the stores are configured MilvusVectorStoreCollection.get() was the one client request without `timeout=`; the spy test now exercises it, so a request without the timeout fails the test. The configuration parameter table gains `request_timeout_seconds`, the databases page's Milvus example carries it, and the configuration page gains a Qdrant example beside the Milvus one. Folded from MemMachine#1630 into MemMachine#1631: the parameter table and the Milvus example show the timeout beside collection_registry, and the every-request test spies on get again, now that get carries the timeout. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn (cherry picked from commit ce97ac0)

edwinyyyu · 2026-09-22T23:19:53Z

Folded into #1631 as its own six commits (c5c0efc1d, e443dac65, aedcad290, 914dcd78b, 92a38429e, 7de506272), since the tombstone retention there only has to exceed the longest a request to the backend can be in flight, which this bound sets. Review the timeout in #1631.