[Bug]: Background semantic ingestion crashes/restarts when semantic storage references missing episode_ids

Describe the bug

Reported error - https://github.com/MemMachine/MemMachine/actions/runs/21885425744/job/63179096417

The job is failing because the ingestion process encountered invalid episode_ids for set_id user-b, specifically ['4'], resulting in this error:

ValueError: Failed to retrieve messages. Invalid episode_ids exist for set_id user-b: ['4']
This error is raised in src/memmachine/semantic_memory/semantic_ingestion.py at line 110, when messages cannot be retrieved for the given episode_ids.

Steps to reproduce

Review any Pull Request that failed the test.

Expected behavior

Unit tests should pass.

Environment

OS: Mac OS X
MemMachine: current commit

Additional context

Proposed fix

Investigate why episode_id '4' is invalid or missing from the episode storage. Check if it's being deleted prematurely or not added correctly.

If an episode was deleted or failed to save, ensure all references to that episode_id are removed or handled gracefully.

You can modify the ingestion logic to skip invalid episode_ids rather than fail entirely.
In src/memmachine/semantic_memory/semantic_ingestion.py, update the _process_single_set method as follows:

# Replace
if len(none_h_ids) != 0:
    raise ValueError(
        "Failed to retrieve messages. Invalid episode_ids exist for set_id "
        f"{set_id}: {none_h_ids}"
    )
# With
if len(none_h_ids) != 0:
    logger.warning(f"Skipping invalid episode_ids for set_id {set_id}: {none_h_ids}")
    raw_messages = [m for m in raw_messages if m is not None]
    # Optionally, mark as ingested or delete references to prevent repeated failures

Additionally, audit your data insertion and deletion routines to prevent inconsistent episode references for set_id user-b.

This change will prevent complete ingestion failure and allow processing of valid messages.

@o-love

Hi @o-love , i hit a lot of errors today after set prompt to

prompt:
  default_org_categories: []
  default_project_categories: []
  default_user_categories:
  - profile_prompt

errors are

2026-02-24 23:53:49,628 [ERROR] memmachine.semantic_memory.semantic_ingestion - Failed to process set_id mem_user_set_org_MemVerge_1_7e8d0b20c039__producer_id_Mel
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/memmachine/semantic_memory/semantic_ingestion.py", line 61, in _run
    await self._process_single_set(set_id)
  File "/app/.venv/lib/python3.12/site-packages/memmachine/semantic_memory/semantic_ingestion.py", line 110, in _process_single_set
    raise ValueError(
ValueError: Failed to retrieve messages. Invalid episode_ids exist for set_id mem_user_set_org_MemVerge_1_7e8d0b20c039__producer_id_Mel: ['2', '4']
2026-02-24 23:53:49,629 [ERROR] memmachine.semantic_memory.semantic_ingestion - Failed to process set_id mem_user_set_org_MemVerge_1_7e8d0b20c039__producer_id_Sarah
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/memmachine/semantic_memory/semantic_ingestion.py", line 61, in _run
    await self._process_single_set(set_id)
  File "/app/.venv/lib/python3.12/site-packages/memmachine/semantic_memory/semantic_ingestion.py", line 110, in _process_single_set
    raise ValueError(
ValueError: Failed to retrieve messages. Invalid episode_ids exist for set_id mem_user_set_org_MemVerge_1_7e8d0b20c039__producer_id_Sarah: ['10', '7', '9']

Added priority label since the error can be reproduced after using user level profile prompt.

[GH-1102] Clean invalid episode_ids during ingestion

[GH-1102] Fix dangling episode_ids

[GH-1102] Clean invalid episode_ids during ingestion

[GH-1102] Clean invalid episode_ids during ingestion (#1148)

[GH-1102] Fix dangling episode_ids

[GH-1102] Fix dangling episode_ids (#1151)

@o-love

Thank you @o-love for the fix, i don't see this error anymore in my test!

Navigation Menu

Repository navigation

[Bug]: Background semantic ingestion crashes/restarts when semantic storage references missing episode_ids #1102

Describe the bug

Steps to reproduce

Expected behavior

Environment

Additional context

Proposed fix

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug]: Background semantic ingestion crashes/restarts when semantic storage references missing episode_ids #1102

Description

Describe the bug

Steps to reproduce

Expected behavior

Environment

Additional context

Proposed fix

Activity

jgong commented on Feb 25, 2026

jgong commented on Feb 25, 2026

jgong commented on Mar 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions