Never reuse a row id in SQLiteVectorStore (speedkick) - #1589

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps, per
collection, the engine together with its lock, created and dropped with
the engine, and every engine call sits under it: searches on the read
side, mutations, loads, and the index save on the write side. The save's
trim runs after the lock is released, so readers wait for the file write
and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store wraps each
collection's engine in a _SynchronizedEngine, created and dropped with
the engine, whose methods apply the rule: search on the read side;
replace, remove, save, and load on the write side. A rewrite is one
replace. The save's trim runs after the lock is released, so readers
wait for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

…(speedkick) (#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from #1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.


Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.


Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

Never reuse a row id in SQLiteVectorStore

The records table used a plain INTEGER PRIMARY KEY, which is SQLite's
rowid, assigned as max(rowid) + 1: deleting the highest row freed its id
for the very next insert. query() scores keys in the search engine and
resolves them to rows in a second step, holding nothing in between, so a
reused id let a record that was never scored come back wearing the score
of the record that was. Nothing about that result looks wrong: the record
exists and the score is in range.

Declare the table with sqlite_autoincrement=True so ids are never reused.
A stale engine key then matches no row and is dropped.

Both tests fail without the flag: one pins the id policy directly, the
other parks a query between scoring and row lookup, retires the scored
record, inserts another, and asserts the query returns nothing.

First half of MemMachine#1468.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>

…(speedkick) (MemMachine#1612)

Own the search engine's concurrency in the store, not in each engine

Each engine wrapped its five methods in its own read-write lock and the
abstract class promised concurrent use. The store is the only caller,
and it already knows which calls must exclude which: a search shares the
engine with other searches, and everything else runs alone. Holding that
lock in the store puts every exclusion in the one file that reads the
engine, and lets a rule the engines could not express hold: a rewrite's
remove and add are one step to a reader, where before a search could run
between them and see neither version.

Engines drop their lock and keep only the index calls; the abstract
class now states that the owner serializes. The store keeps one
read-write lock per collection beside the engine, kept for the store's
lifetime like the engine's other per-collection state, and takes it at
every engine call: searches on the read side; mutations, loads, and the
index save on the write side. A rewrite's remove and add sit under one
hold. The save's trim runs after the lock is released, so readers wait
for the file write and never for SQL.

The row-id regression test from MemMachine#1589 parked inside a wrapper engine's
search, outside the real engine's lock; under the store's lock that
parks the read side, and the writes it then awaits cannot proceed. It
now parks where it meant to, between the engine search and the row
lookup. One new test pins the one-step rewrite; it fails against the
engines' own locks.

The turbovec engine in flight carries the same lock and needs the same
subtraction.

Claude-Session: https://claude.ai/code/session_01ESpWYTmCR7X3bJEpoA8SAn

Co-authored-by: Claude Fable 5.1 <[email protected]>