(Depends on #1606) Make filter negation the complement on every backend, and name absence IsNull (speedkick) - #1615

…te undeclared predicates to the segment store (speedkick)

The records-and-queries side of the vector store redesign
(design/vector_store_handoff.md), written against `VectorStoreCollection`
as it is today so the lifecycle work can later replace how a handle is
obtained without touching any of this.

Declared, not dynamic, indexes. A store declares one
`indexed_properties: Mapping[str, PropertyType]` for every collection it
holds, from deployment configuration merged with the system keys of the
one service that uses it, at the point the store is built for that
service (`DatabaseManager.get_vector_store(name, indexed_properties=...)`).
`indexed_properties_schema` leaves `VectorStoreCollectionConfig`, and no
request creates an index. EventMemory keeps
`expected_vector_store_collection_schema` as its declaration of the four
reserved keys and raises `InvalidCollectionSchemaError` at construction
when the collection's store does not declare them. Services do not share
a vector store: a second service whose keys the built store does not
declare is refused with `VectorStoreConfigurationError`.

Undeclared keys are rejected. `upsert` raises `UndeclaredPropertyKeyError`
before anything is sent when a record carries a key the store has not
declared, and `PropertyTypeMismatchError` when a declared key holds a
value of another type, since a typed column or index cannot hold it
without coercing it. `query` raises `UndeclaredPropertyKeyError` when a
filter names an undeclared key and `UnsupportedFilterError` when it uses a
node outside `supported_filter_nodes`. An undeclared key therefore never
exists in the vector store.

The filter language. `common/filter/filter_expression.py` is the closed
union from the `default` branch (822ccb6): `Equals`, `NotEquals`,
`Ordering` over numbers and datetimes, non-empty homogeneous `In`,
`IsMissing`, n-ary `And` and `Or`, `Not`. `Comparison` and `IsNull` go;
every compiler is an exhaustive `match`. The string parser stays as the
server's translation into the union, since the HTTP API still speaks it;
a same-operator chain parses to one n-ary node. `split_declared`,
`conjoin`, `conjuncts`, `filter_fields` and `filter_nodes` are the tree
helpers the routing needs. Ordering strings is no longer expressible,
which retires two semantic-storage tests that ordered string values.

Semantics. A predicate matches only a record holding a value of the
compared type: the SQL column compiler renders a leaf of another type as
FALSE rather than letting affinity coerce it, and `Not` is the complement
of a match (`NOT COALESCE(x, FALSE)`), so `Not(Equals)` keeps records
holding no value while `NotEquals` does not. Qdrant compiles `NotEquals`
as "not the value and not empty", since `must_not` alone admits points
lacking the field; Milvus pushes `Not` to the leaves by De Morgan, since
its own `not` excludes entities lacking the field (measured on Milvus
Lite). A datetime is normalized to a UTC instant at node construction.

Datetimes. Where a backend has no datetime type, a datetime property is
stored as an integer of microseconds since the epoch: the two SQLite
stores keep one typed, indexed, nullable column per declared key on the
collection's records table (`sql_columns.py`) and store datetimes that
way, so a `since` or `until` bound evaluates identically everywhere.

sqlite-vec. The handoff planned vec0 metadata columns, on which sqlite-vec
evaluates only comparisons joined by AND. sqlite-vec 0.1.9 rejects NULL in
a metadata column ("Expected text for TEXT metadata column name, received
NULL"), and a declared key is optional per record (`memmachine_event_session`
on its own), so the store keeps the declared columns on the records table
and hands the KNN `rowid IN (SELECT rowid ... WHERE <filter>)`, which vec0
takes into the search: with k=1 and a filter excluding the nearest rows the
admitted row is returned, and every node is evaluated during the search,
so the store reports the full node set.

Engine-backed store. The declared columns replace the JSON properties
column, each with its own index, and the per-candidate SQL predicate
compiles over them; MemMachine#1602's selectivity routing is deferred to the
lifecycle work and can rebase onto these columns. Both SQLite stores
record the schema a collection was created under and raise
`IndexedPropertiesMismatchError` when a store built with another schema
opens it, since nothing here migrates a column.

Clients. `request_timeout` is a required field of `QdrantConf` and
`MilvusConf`, passed to the client at construction. Qdrant's
`hnsw_config`, `optimizers_config` and `quantization_config` come over
from the `default` branch (a0753d3) as plain mappings validated against
the qdrant models when the store is built; `hnsw_config.m` must be 0 or
unset, and `payload_m` is the knob. Milvus drops the properties JSON
field nothing read.

EventMemory. One plan in `query`: the vector store gets the system
predicates and the part of the caller's filter naming declared keys; the
segment store, which holds every property, applies the whole filter to
the seeds and their windows afterward. When the filter has an undeclared
part, a seed the segment store drops leaves the search short and the
vector limit is widened by four each time, up to
`limit * FilterOptions.max_overfetch_factor`, where the search returns
what survived; with no undeclared part the first query is the last. An
empty id or kind list admits nothing and issues no query. Every count is
a maximum.

Not in this change: keys, registries, strict create, logical delete and
purge, stateless handles, one container per embedder, the SQLite stores
on shared tables, and the removal of content-addressed names and Qdrant
shard keys (the collections-and-keys side of the seam). Native Qdrant and
Milvus collection names still hash the collection config, which no longer
carries a schema, so existing native collections are not reused.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01DtbsPQafKpBUn7ksnDU2UY

…e IsNull (speedkick)

The filter grammar is two-valued: a property has a value or is absent, a
leaf predicate on a field with no value is false, `IsNull` on it is true,
and `And`, `Or` and `Not` are the ordinary boolean connectives over the
records. `Not` is therefore the exact complement of a match, and
`Not(Not(x))` is `x`. Three of the pieces did not follow that rule.

The absence node is renamed `IsMissing` to `IsNull`, the name the textual
filter language already uses for it and the one the design settles on.

`NotEquals` is removed. The parser maps `!=` and `<>` to
`Not(Equals(...))`, as it already mapped `NOT IN` to `Not(In(...))` and
`IS NOT NULL` to `Not(IsNull(...))`, so all three negations mean the same
thing. A caller who wants only records that hold some other value conjoins
`Not(IsNull(field))`. Every compiler arm and every `_SUPPORTED_FILTER_NODES`
entry for the node goes with it; the union is closed, so the type checker
finds them all. The dead `!=` branch of short-term memory's value
comparison, which only the removed arm reached, goes too.

`compile_sql_filter` compiled `Not` to `~inner`, which is SQL three-valued
logic: `NOT (k = v)` on a row whose `k` is NULL is unknown and the row is
dropped, so a negated filter silently excluded rows that hold no value.
It now compiles to `~func.coalesce(inner, false())`, making the operand
total before negating it, which is what the vector stores' column compiler
already did. The segment store, the episode store, both SQLAlchemy semantic
storages and short-term memory read through that compiler.

The vector store conformance contract now asserts the complement law over a
fixture of records that hold every declared key, records that lack each key
in turn, and a record that holds none: for each of a list of filters
covering `Equals`, `Ordering`, `In`, `IsNull`, `And`, `Or` and a nested
`Not`, the records `Not(F)` admits are every record minus the ones `F`
admits, and `Not(Not(F))` admits exactly what `F` does. It runs on SQLite,
sqlite-vec, Milvus Lite and Qdrant. The same law is asserted for
`compile_sql_filter` over its column, raw-JSON and properties-JSON
encodings, on SQLite and on PostgreSQL, and the segment store gains a
context query whose negated filter keeps a segment carrying no property at
all.

The Neo4j and NebulaGraph graph stores and the Neo4j semantic storage get
the rename and the node removal only. Each still emits `NOT (<comparison>)`,
which on a missing property is Cypher's and nGQL's unknown and drops the
row, so their negation is not yet the complement; there is no backend for
them in this test environment, so the fix is left for a change that can be
tested against one.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01YBbQgZiCqeoLu83EkbEFHE