When the LLM returns a context length exceeded error, MemMachine’s retry logic fails to activate. The root cause is that ExternalServiceAPIError is raised without including the original error message from the LLM. As a result, _is_exceed_context_window_error() cannot detect keywords like “context length” or “token limit”, skipping the split-and-retry mechanism.
This leads to silent failure with “Summarization failed due to unexpected error”, risking data loss.
2026-03-18 19:43:14,833 [ERROR] memmachine_server.common.language_model.openai_chat_completions_language_model - [call uuid: e3758a9d-e2ef-481c-b14c-f381949b03e1] Giving up generating response after failed attempt 1 due to non-retryable BadRequestError
Traceback (most recent call last):
File "/path/to/mm/packages/server/src/memmachine_server/common/language_model/openai_chat_completions_language_model.py", line 219, in _generate_response
response = await self._client.chat.completions.create(**args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py", line 2676, in create
return await self._post(
^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1884, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1669, in request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "You passed 202753 input tokens and requested 0 output tokens. However, the model's context length is only 202752 tokens, resulting in a maximum input length of 202752 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=202753)", 'type': 'BadRequestError', 'param': 'input_tokens', 'code': 400}}
2026-03-18 19:43:14,836 [ERROR] memmachine_server.episodic_memory.short_term_memory.short_term_memory - Summarization failed due to unexpected error
Traceback (most recent call last):
File "/path/to/mm/packages/server/src/memmachine_server/common/language_model/openai_chat_completions_language_model.py", line 219, in _generate_response
response = await self._client.chat.completions.create(**args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py", line 2676, in create
return await self._post(
^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1884, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1669, in request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "You passed 202753 input tokens and requested 0 output tokens. However, the model's context length is only 202752 tokens, resulting in a maximum input length of 202752 tokens. Please reduce the length of the input prompt. (parameter=input_tokens, value=202753)", 'type': 'BadRequestError', 'param': 'input_tokens', 'code': 400}}
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/path/to/mm/packages/server/src/memmachine_server/episodic_memory/short_term_memory/short_term_memory.py", line 606, in _create_summary
result = await self._model.generate_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/packages/server/src/memmachine_server/common/language_model/openai_chat_completions_language_model.py", line 160, in generate_response
output, function_calls_arguments, _, _ = await self._generate_response(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/path/to/mm/packages/server/src/memmachine_server/common/language_model/openai_chat_completions_language_model.py", line 259, in _generate_response
raise ExternalServiceAPIError(error_message) from e
memmachine_server.common.data_types.ExternalServiceAPIError: [call uuid: e3758a9d-e2ef-481c-b14c-f381949b03e1] Giving up generating response after failed attempt 1 due to non-retryable BadRequestError
I fixed this myself and it’s working well in our environment. If you think it’s safe, feel free to merge it directly — it’s just one line.
(Not sure if this bug also exists when using the official OpenAI cloud service — but since we’re on-prem, this fix is critical for us.)
Describe the bug
When the LLM returns a context length exceeded error, MemMachine’s retry logic fails to activate. The root cause is that ExternalServiceAPIError is raised without including the original error message from the LLM. As a result, _is_exceed_context_window_error() cannot detect keywords like “context length” or “token limit”, skipping the split-and-retry mechanism.
This leads to silent failure with “Summarization failed due to unexpected error”, risking data loss.
Steps to reproduce
Expected behavior
Environment
Additional context
I fixed this myself and it’s working well in our environment. If you think it’s safe, feel free to merge it directly — it’s just one line.
(Not sure if this bug also exists when using the official OpenAI cloud service — but since we’re on-prem, this fix is critical for us.)
Root Cause:
openai_chat_completions_language_model.py, ExternalServiceAPIError is raised with a generic message:→
_is_exceed_context_window_error()receives no keywords → returns False → retry logic skipped.Fix:
Verification: