Skip to content

Add VectorSearchEngine backed by turbovec (speedkick) - #1599

Draft
edwinyyyu wants to merge 1 commit into
MemMachine:speedkickfrom
edwinyyyu:feat/turbovec-engine-on-cosine-speedkick
Draft

edwinyyyu wants to merge 1 commit into
MemMachine:speedkickfrom
edwinyyyu:feat/turbovec-engine-on-cosine-speedkick

Conversation

@edwinyyyu

@edwinyyyu edwinyyyu commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the change

A VectorSearchEngine over turbovec: an in-memory index of 4-bit TurboQuant codes.

It scans every vector, like the engines beside it, so it changes no asymptotics. What it changes is the memory traffic an exhaustive scan is bound by, which 4-bit codes cut about eightfold.

Why it lands after #1598

Under the engine contract before #1598 this engine had to supply get_vectors, and it could only ever raise: turbovec keeps quantized codes and does not retain the originals. Adding a method that exists to refuse, and retiring it a PR later, is churn that says nothing. On the contract #1598 left behind there is no keyed vector access to refuse, so every method this engine declares, it implements.

Description

  • Scores are clamped onto [-1, 1]: quantization can carry an inner product slightly outside it, and a similarity reading 1.0000001 is a lie about the range the contract promises.
  • Vectors are zero-padded to a multiple of eight — the width the index can hold — and normalized on the way in, so the inner product it computes is cosine.
  • turbovec has a native allowlist search, but it needs the allowed ids enumerated, and the engine contract's allowed_keys only answers membership. So the engine fetches unrestricted, drops what the filter rejects, and widens the fetch until limit survive or the whole index has been scanned, as usearch does. Handing turbovec the ids directly waits for the contract to carry them (DEFERRED (until lifecycle/DDL changes): Route filtered queries by selectivity, and score allowlists exactly (speedkick) #1602, deferred).
  • Like the engines beside it since Own the search engine's concurrency in the store, not in each engine (speedkick) #1612, it holds no lock of its own: the store serializes access.
  • turbovec>=1.0.0 as an optional extra; 1.0.0 is upstream's first stable release and commits to the on-disk format this persists through.

Verification

  • pytest packages/server/server_tests: 1963 passed, 3 skipped.
  • ty check --project packages/server: clean.
  • ruff check / ruff format --check / uv lock --check: clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NKmF9xNph9QH3ozNw3ZnJL

@edwinyyyu
edwinyyyu force-pushed the feat/turbovec-engine-on-cosine-speedkick branch 14 times, most recently from c44e683 to 64c0dff Compare September 10, 2026 00:47
@edwinyyyu
edwinyyyu force-pushed the feat/turbovec-engine-on-cosine-speedkick branch 9 times, most recently from f9aaa4e to 8640ea0 Compare September 10, 2026 18:09
@edwinyyyu
edwinyyyu marked this pull request as draft September 10, 2026 20:26
An in-memory index over 4-bit TurboQuant codes. It scans every vector like
the engines beside it, so it changes no asymptotics; what it changes is the
memory traffic that an exhaustive scan is bound by, which 4-bit codes cut
about eightfold.

Landing after the read-back removal is deliberate: the engine does not keep
the vectors it is given -- the codes are lossy and the originals are not
retained -- so against the older contract it would have had to supply a
`get_vectors` that could only raise. The one method a conforming engine was
allowed to refuse would have been introduced and retired inside one stack.

Scores are clamped onto [-1, 1]: quantization can carry an inner product
slightly outside it, and a similarity that reads 1.0000001 is a lie about
the range the contract promises.

Vectors are zero-padded to a multiple of eight, which is the width the
index can hold, and normalized on the way in so the inner product it
computes is cosine.

turbovec has a native allowlist search, but it needs the allowed ids
enumerated, and the engine contract's `allowed_keys` only answers
membership. So the engine fetches unrestricted, drops what the filter
rejects, and widens the fetch until `limit` survive or the whole index has
been scanned, as usearch does. Handing turbovec the ids directly waits for
the contract to carry them (MemMachine#1602).

Like the engines beside it since MemMachine#1612, it holds no lock of its own: the
store serializes access.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn
@edwinyyyu
edwinyyyu force-pushed the feat/turbovec-engine-on-cosine-speedkick branch from 8640ea0 to 196ccef Compare September 11, 2026 03:32

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant