You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: Metadata-filtered search fails on Qdrant Cloud strict mode #1776
On the event backend with Qdrant, an episodic memory search that carries a user metadata filter (metadata.<key>=...) sends that filter to Qdrant as a payload filter on a key the collection has no index for. Qdrant Cloud turns strict mode on for every new collection with unindexed_filtering_retrieve: false, so it refuses such a query instead of scanning, and the search fails. A search without a filter is unaffected.
The path, at main (ad8ff24b0) and on feat/horizontal-scaling (d45d8dd):
EventMemory maps the caller's whole property_filter onto vector-record properties (map_filter_fields(property_filter, EventMemory._to_vector_record_property)) and passes it to vector_store_collection.query(..., property_filter=...) in packages/server/src/memmachine_server/episodic_memory/event_memory/event_memory.py.
QdrantVectorStore creates payload indexes only for the collection's indexed_properties_schema, which is core's own fields plus the keys a deployment lists in properties_schema (packages/server/src/memmachine_server/common/vector_store/qdrant_vector_store.py). A user's metadata key is in neither unless the deployment declared it.
Declaring the keys is not a way out: metadata keys are whatever each client sends, one collection serves every project with the same embedding configuration, and Qdrant Cloud's strict mode also caps a collection at 100 payload indexes.
Qdrant documents the defaults in "Configure Qdrant Cloud Clusters": strict mode is activated by default for new collections, with unindexed_filtering_retrieve and unindexed_filtering_update set to false and max_payload_index_count set to 100. Self-hosted Qdrant leaves strict mode off, which is why a local install does not show this.
Steps to reproduce
Run qdrant/qdrant:v1.19.1 with Qdrant Cloud's strict-mode defaults for new collections, mounted as /qdrant/config/production.yaml:
Build LongTermMemory on the event backend as service_locator does: QdrantVectorStore against that Qdrant (on feat/horizontal-scaling, with a SQLAlchemyVectorStoreCollectionRegistry), and a collection whose indexed_properties_schema is EventMemory.expected_vector_store_collection_schema() plus EVENT_BACKEND_SYSTEM_FIELDS, with no properties_schema. The segment store, episode store and embedder are the in-memory and fake ones test_event_backend_wiring.py uses.
Add three episodes whose metadata and filterable_metadata are {"topic": "alpha"}, {"topic": "beta"} and {"topic": "gamma"}.
Result, at main (ad8ff24b0) and at feat/horizontal-scaling (d45d8dd): the unfiltered search returns all three episodes, and the filtered one raises UnexpectedResponse:
Bad request: Index required but not found for "topic" of one of the following types: [keyword]. Help: Create an index for this key or use a different filter.
Against the same Qdrant image without the strict-mode config, the filtered search returns only the alpha episode at both commits.
This drove core's search path, not the HTTP server: the request through /api/v2/memories/search was not run.
Expected behavior
The search returns the matching episode, as it does on a Qdrant without strict mode.
Qdrant Cloud documents that strict mode can be disabled per collection (PUT /collections/{name} with strict_mode_config.enabled: false), but not as a cluster default, and recommends keeping it on.
Describe the bug
On the event backend with Qdrant, an episodic memory search that carries a user metadata filter (
metadata.<key>=...) sends that filter to Qdrant as a payload filter on a key the collection has no index for. Qdrant Cloud turns strict mode on for every new collection withunindexed_filtering_retrieve: false, so it refuses such a query instead of scanning, and the search fails. A search without a filter is unaffected.The path, at
main(ad8ff24b0) and onfeat/horizontal-scaling(d45d8dd):EventMemorymaps the caller's wholeproperty_filteronto vector-record properties (map_filter_fields(property_filter, EventMemory._to_vector_record_property)) and passes it tovector_store_collection.query(..., property_filter=...)inpackages/server/src/memmachine_server/episodic_memory/event_memory/event_memory.py.QdrantVectorStorecreates payload indexes only for the collection'sindexed_properties_schema, which is core's own fields plus the keys a deployment lists inproperties_schema(packages/server/src/memmachine_server/common/vector_store/qdrant_vector_store.py). A user's metadata key is in neither unless the deployment declared it.Declaring the keys is not a way out: metadata keys are whatever each client sends, one collection serves every project with the same embedding configuration, and Qdrant Cloud's strict mode also caps a collection at 100 payload indexes.
Qdrant documents the defaults in "Configure Qdrant Cloud Clusters": strict mode is activated by default for new collections, with
unindexed_filtering_retrieveandunindexed_filtering_updateset to false andmax_payload_index_countset to 100. Self-hosted Qdrant leaves strict mode off, which is why a local install does not show this.Steps to reproduce
Run
qdrant/qdrant:v1.19.1with Qdrant Cloud's strict-mode defaults for new collections, mounted as/qdrant/config/production.yaml:Build
LongTermMemoryon the event backend asservice_locatordoes:QdrantVectorStoreagainst that Qdrant (onfeat/horizontal-scaling, with aSQLAlchemyVectorStoreCollectionRegistry), and a collection whoseindexed_properties_schemaisEventMemory.expected_vector_store_collection_schema()plusEVENT_BACKEND_SYSTEM_FIELDS, with noproperties_schema. The segment store, episode store and embedder are the in-memory and fake onestest_event_backend_wiring.pyuses.Add three episodes whose
metadataandfilterable_metadataare{"topic": "alpha"},{"topic": "beta"}and{"topic": "gamma"}.Call
search_scored("launch", num_episodes_limit=10, property_filter=parse_filter("metadata.topic=alpha")).Result, at
main(ad8ff24b0) and atfeat/horizontal-scaling(d45d8dd): the unfiltered search returns all three episodes, and the filtered one raisesUnexpectedResponse:Against the same Qdrant image without the strict-mode config, the filtered search returns only the
alphaepisode at both commits.This drove core's search path, not the HTTP server: the request through
/api/v2/memories/searchwas not run.Expected behavior
The search returns the matching episode, as it does on a Qdrant without strict mode.
Additional context
PUT /collections/{name}withstrict_mode_config.enabled: false), but not as a cluster default, and recommends keeping it on.