Skip to content

Message content with U+0000 or a lone surrogate is refused by the episode store after the other stores have written it #1799

Description

@edwinyyyu

What happens

Measured on main at ad8ff24 through each store's own code, and through POST /api/v2/memories with the real episode store, segment store, and semantic storage on SQLite and on PostgreSQL, and SQLiteVectorStore. The value is a message's content:

store "a\u0000b" "a\ud800b" (lone surrogate)
episode store, SQLite stored refused (UnicodeEncodeError)
episode store, PostgreSQL refused (CharacterNotInRepertoireError) refused (DataError)
segment store, SQLite and PostgreSQL stored stored
vector stores (the derived text's embedding) stored stored

Through the route on PostgreSQL, content "a\u0000b" answers 500 with the episode in the segment store, the vector store, and semantic memory's three sets, and not in the episode store, which the search path reads episodes back from.

Why

  • MemoryMessage.content and EpisodeEntry.content take any string.
  • The episode store keeps content in a text column (packages/server/src/memmachine_server/common/episode_store/episode_sqlalchemy_store.py:86), which PostgreSQL refuses U+0000 in and which UTF-8 encoding refuses a lone surrogate in, while the segment store keeps the block as bytes (packages/server/src/memmachine_server/episodic_memory/event_memory/segment_store/sqlalchemy_segment_store.py:177).
  • MemMachine.add_episodes writes the stores concurrently (packages/server/src/memmachine_server/main/memmachine.py:795-819).

Expected

The request refuses such content with 422 before any store writes.

Fix

Not in #1792, which covers property values. The PropertyStr check it adds would refuse such content if content used it.

🤖 Written by Claude Code (Claude Opus 5.5) on behalf of @edwinyyyu.

Activity

  1. added theissue type on Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions