Skip to content

Require a vector on the record type nothing reads back (speedkick) - #1603

Merged
edwinyyyu merged 1 commit into
MemMachine:speedkickfrom
edwinyyyu:feat/vector-store-strict-record-speedkick
Sep 10, 2026
Merged

edwinyyyu merged 1 commit into
MemMachine:speedkickfrom
edwinyyyu:feat/vector-store-strict-record-speedkick

Conversation

@edwinyyyu

@edwinyyyu edwinyyyu commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the change

Record is input-only since #1598: a collection stores vectors to search them and properties to filter on them, and answers a query with QueryMatch. Nothing reads a record back.

Its type never caught up. vector: list[float] | None = None says a caller may omit the vector, which no caller may do — so every store re-stated the rule at its own upsert, four copies of the same message:

if record.vector is None:
    raise ValueError(f"Record {record.uuid} has vector=None, which is not allowed on input.")

properties was optional the same way, leaving Qdrant with record.properties if record.properties is not None else {}.

Description

  • vector: list[float] — required.
  • properties: dict[str, PropertyValue] = Field(default_factory=dict) — never None.
  • The four runtime checks go, in sqlite, sqlite-vec, Qdrant and Milvus, along with Qdrant's None coalescing.
  • Rejection moves to the model, so a caller meets it at construction rather than at a write.

Tests

The two test_upsert_rejects_none_vector tests asserted that a store raises. They now assert the model does — pytest.raises(ValidationError) on a Record built without a vector — plus a new test that properties defaults to {}. The four _make_record helpers require a vector.

Verification

  • pytest packages/server/server_tests: 1913 passed, 3 skipped, 1 failed.
  • ty check --project packages/server: clean — and clean on the base, so this introduces none.
  • ruff check / ruff format --check: clean.

The one failure is test_get_version, pre-existing on speedkick: it rejects the .post segment git describe puts into the derived version. Nothing here touches versioning or tags.


🤖 Generated with Claude Code

https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

@edwinyyyu
edwinyyyu force-pushed the feat/vector-store-strict-record-speedkick branch 4 times, most recently from 3b0723d to 3713bc9 Compare September 10, 2026 17:25
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 10, 2026
@edwinyyyu
edwinyyyu force-pushed the feat/vector-store-strict-record-speedkick branch 4 times, most recently from 5e05262 to 00f9abf Compare September 10, 2026 18:11
`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL
@edwinyyyu
edwinyyyu force-pushed the feat/vector-store-strict-record-speedkick branch from 00f9abf to 814ded7 Compare September 10, 2026 18:45
@edwinyyyu
edwinyyyu merged commit cdeef88 into MemMachine:speedkick Sep 10, 2026
39 checks passed
This was referenced Sep 10, 2026
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 17, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 17, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 17, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.


Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 17, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.


Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 18, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
(cherry picked from commit cdeef88)
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 18, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
(cherry picked from commit cdeef88)
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 21, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
(cherry picked from commit cdeef88)
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 21, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
(cherry picked from commit cdeef88)
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 25, 2026
…emMachine#1603)

Require a vector on the record type nothing reads back

`Record` is written and never returned, so `vector` being optional
described nothing a caller could do. Every store rejected `None` at its
own `upsert` with the same message, four copies of one rule that the type
could state once. `properties` was optional in the same way, so Qdrant
carried `record.properties if record.properties is not None else {}`.

The vector is required and the properties default to `{}`. The four
checks go, and rejection moves to the model, where a caller finds it at
construction rather than at a write.

Claude-Session: https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
(cherry picked from commit cdeef88)
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 29, 2026
Record is input-only since queries answer UUIDs and scores, but its types
still made both fields optional: `vector: list[float] | None = None` and
`properties: ... | None = None`, the `None` once meaning "not requested"
on a returned record. So every store re-checked the vector at upsert
(four copies of "has vector=None, which is not allowed on input") and
Milvus and Qdrant coalesced `None` properties to `{}`.

As MemMachine#1603 did (MemMachine#1663 on main): `vector: list[float]` is required, and
`properties` defaults to `{}` through `Field(default_factory=dict)`. The
model rejects a missing vector at construction, so the four checks and
the coalescing go, along with the in-memory test collection's.

The tests of a store refusing a `None` vector become tests that the model
refuses a missing one, built with `Record.model_validate` rather than a
type ignore, plus one that properties default to `{}` and one that a
record without properties is stored. The stores' `_make_record` helpers
require a vector.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Oct 10, 2026
Every embedder MemMachine ships produces vectors meant to be compared by
cosine -- OpenAI hard-coded it, Bedrock defaulted to it, SentenceTransformer
only reported what the model declared -- while every layer that touched a
score paid for the other three metrics in direction flags, threshold
directions, and per-backend tables mapping the enum onto native metric
names. `SimilarityMetric` is gone; scores are cosine similarities in
[-1, 1] and the names say so: `QueryMatch.score` and `SearchMatch.score`
become `cosine_similarity`, and `query(score_threshold=)` becomes
`query(min_cosine_similarity=)`, which no longer needs a direction to be
meaningful and is refused when not finite, as the threshold was. A store
and its partitions no longer have a `similarity_metric`, the schema a
partition is registered under no longer records one, and a search engine
factory takes the dimensions alone. The Bedrock embedder's
`similarity_metric` config key and semantic memory's
`vector_similarity_metric` go with it, and the install and configuration
docs drop them.

The vector graph stores carried a metric per stored embedding, as a
companion property beside every vector; that is gone and `Node.embeddings`
holds plain vectors. NebulaGraph's `cosine()` cannot take `APPROXIMATE` and
its vector indexes offer only L2 and IP. Cosine similarity between unit
vectors is their inner product, so embeddings are normalized on the way in
and compared with `inner_product()` against an IP index.

The cosine half of MemMachine#1598 (`abf92a3a4` on speedkick), re-derived on the
one-collection store: the rest of MemMachine#1598 (queries answer record UUIDs and
scores, no `get`, semantic memory's `vector_uuid`) and all of MemMachine#1603 are in
registry-backed base and its partition handle lose the metric, the stores'
own threshold checks become `require_valid_min_cosine_similarity`, and the
design documents describe cosine scoring and a schema without a metric.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant