What happens
Measured on main at ad8ff24 through each store's own code, and through POST /api/v2/memories with the real episode store, segment store, and semantic storage on SQLite and on PostgreSQL, and SQLiteVectorStore. The value is a message's content:
| store |
"a\u0000b" |
"a\ud800b" (lone surrogate) |
| episode store, SQLite |
stored |
refused (UnicodeEncodeError) |
| episode store, PostgreSQL |
refused (CharacterNotInRepertoireError) |
refused (DataError) |
| segment store, SQLite and PostgreSQL |
stored |
stored |
| vector stores (the derived text's embedding) |
stored |
stored |
Through the route on PostgreSQL, content "a\u0000b" answers 500 with the episode in the segment store, the vector store, and semantic memory's three sets, and not in the episode store, which the search path reads episodes back from.
Why
MemoryMessage.content and EpisodeEntry.content take any string.
- The episode store keeps content in a text column (
packages/server/src/memmachine_server/common/episode_store/episode_sqlalchemy_store.py:86), which PostgreSQL refuses U+0000 in and which UTF-8 encoding refuses a lone surrogate in, while the segment store keeps the block as bytes (packages/server/src/memmachine_server/episodic_memory/event_memory/segment_store/sqlalchemy_segment_store.py:177).
MemMachine.add_episodes writes the stores concurrently (packages/server/src/memmachine_server/main/memmachine.py:795-819).
Expected
The request refuses such content with 422 before any store writes.
Fix
Not in #1792, which covers property values. The PropertyStr check it adds would refuse such content if content used it.
🤖 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.
What happens
Measured on
mainat ad8ff24 through each store's own code, and throughPOST /api/v2/memorieswith the real episode store, segment store, and semantic storage on SQLite and on PostgreSQL, andSQLiteVectorStore. The value is a message'scontent:"a\u0000b""a\ud800b"(lone surrogate)UnicodeEncodeError)CharacterNotInRepertoireError)DataError)Through the route on PostgreSQL, content
"a\u0000b"answers 500 with the episode in the segment store, the vector store, and semantic memory's three sets, and not in the episode store, which the search path reads episodes back from.Why
MemoryMessage.contentandEpisodeEntry.contenttake any string.packages/server/src/memmachine_server/common/episode_store/episode_sqlalchemy_store.py:86), which PostgreSQL refuses U+0000 in and which UTF-8 encoding refuses a lone surrogate in, while the segment store keeps the block as bytes (packages/server/src/memmachine_server/episodic_memory/event_memory/segment_store/sqlalchemy_segment_store.py:177).MemMachine.add_episodeswrites the stores concurrently (packages/server/src/memmachine_server/main/memmachine.py:795-819).Expected
The request refuses such content with 422 before any store writes.
Fix
Not in #1792, which covers property values. The
PropertyStrcheck it adds would refuse such content ifcontentused it.🤖 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.