What happens
Measured on main at ad8ff24 with MilvusVectorStore, records holding a string property s of 'plain', 'caf茅', and '馃榾', and a filter s = <value>:
| value |
Milvus 2.6.24 |
Milvus Lite 3.2.1 |
'plain' |
matches |
matches |
'caf茅' |
matches |
MilvusException: (code=6, message=unknown escape sequence \u at column 42 |
'馃榾' |
MilvusException: (code=1100, message=failed to create query plan: cannot parse expression: (partition_key == "str") && (_p_s == "馃榾"): invalid syntax: invalid parameter) |
MilvusException: (code=6, message=unknown escape sequence \u at column 39 |
Every comparison operator on such a string fails the same way, so a stored value holding a character outside the Basic Multilingual Plane (on Milvus Lite, any non-ASCII character) cannot be filtered on. SQLiteVectorStore, SQLiteVecVectorStore, and QdrantVectorStore match s = 'caf茅 馃榾' against the record holding it.
Why
The store renders a string literal with json.dumps(value) (packages/server/src/memmachine_server/common/vector_store/milvus_vector_store.py:71-73, used by _literal at :76-84). Its default ensure_ascii=True writes each non-ASCII character as a \uXXXX escape, and a character outside the Basic Multilingual Plane as a UTF-16 surrogate pair ("馃榾"). Milvus 2.6.24's expression parser accepts 茅 and refuses the surrogate pair; Milvus Lite refuses every \u escape.
Expected
A filter on any string value matches the records holding that value on Milvus and Milvus Lite, as on the other stores.
Fix
#1736 renders string literals with json.dumps(value, ensure_ascii=False), so characters reach Milvus as UTF-8.
馃 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.
What happens
Measured on
mainat ad8ff24 withMilvusVectorStore, records holding a string propertysof'plain','caf茅', and'馃榾', and a filters = <value>:'plain''caf茅'MilvusException: (code=6, message=unknown escape sequence \u at column 42'馃榾'MilvusException: (code=1100, message=failed to create query plan: cannot parse expression: (partition_key == "str") && (_p_s == "馃榾"): invalid syntax: invalid parameter)MilvusException: (code=6, message=unknown escape sequence \u at column 39Every comparison operator on such a string fails the same way, so a stored value holding a character outside the Basic Multilingual Plane (on Milvus Lite, any non-ASCII character) cannot be filtered on.
SQLiteVectorStore,SQLiteVecVectorStore, andQdrantVectorStorematchs = 'caf茅 馃榾'against the record holding it.Why
The store renders a string literal with
json.dumps(value)(packages/server/src/memmachine_server/common/vector_store/milvus_vector_store.py:71-73, used by_literalat:76-84). Its defaultensure_ascii=Truewrites each non-ASCII character as a\uXXXXescape, and a character outside the Basic Multilingual Plane as a UTF-16 surrogate pair ("馃榾"). Milvus 2.6.24's expression parser accepts茅and refuses the surrogate pair; Milvus Lite refuses every\uescape.Expected
A filter on any string value matches the records holding that value on Milvus and Milvus Lite, as on the other stores.
Fix
#1736 renders string literals with
json.dumps(value, ensure_ascii=False), so characters reach Milvus as UTF-8.馃 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.