Skip to content

Add runtime configuration API, graceful startup, and client SDK - #993

Merged
mwqgithub merged 1 commit into
mainfrom
bugfix/allowEmptyModel
Jan 30, 2026
Merged

mwqgithub merged 1 commit into
mainfrom
bugfix/allowEmptyModel

Conversation

@jealous

@jealous jealous commented Jan 28, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the change

MemMachine currently requires all configured resources (embedders, language models, databases) to be available at startup. If any resource is unavailable — for example, a model is not yet deployed or an API key is not set — the server crashes. This makes it impossible to start the server in a partially configured state and add resources later.

This PR adds a runtime configuration API, makes startup graceful, and provides a client SDK module so users can:

  1. Start the server even when some resources are unavailable
  2. Add, remove, and retry resources at runtime via REST API
  3. Update memory configuration (episodic/semantic) to point at newly added resources
  4. Get clear status reporting on which resources are ready, failed, or pending
  5. Use the Python SDK to manage configuration programmatically

Description

Graceful startup

  • Resource managers (EmbedderManager, LanguageModelManager, RerankerManager) now track build failures instead of crashing. Failed resources are recorded with status FAILED and can be retried later.
  • Extracted shared resource lifecycle logic into BaseResourceManager to reduce duplication across the three manager types.
  • SemanticMemoryConf gains an enabled flag (default True) with a model_validator that auto-disables when required fields (database, llm_model, embedding_model) are empty.
  • MemMachine._initialize_default_episodic_configuration() now gracefully disables long-term or short-term episodic memory when the required embedder/reranker is not configured, instead of raising an error.
  • MemMachine.start() and stop() skip the semantic service when semantic memory is disabled.

Runtime configuration API (/api/v2/config/...)

Method Endpoint Description
GET /config View configuration with resource status
GET /config/resources View all resources and their status
PUT /config/memory Update episodic and/or semantic memory configuration
POST /config/resources/embedders Add a new embedder
POST /config/resources/language_models Add a new language model
DELETE /config/resources/embedders/{name} Remove an embedder
DELETE /config/resources/language_models/{name} Remove a language model
POST /config/resources/embedders/{name}/retry Retry building a failed embedder
POST /config/resources/language_models/{name}/retry Retry building a failed language model
POST /config/resources/rerankers/{name}/retry Retry building a failed reranker
  • Configuration changes are persisted to the YAML config file when one is loaded.
  • Error handling extracted into RestError in a new exceptions.py module.

Client SDK (rest_client/config.py)

Added a Config class to the Python SDK that wraps all configuration API endpoints:

  • client.config() — convenience accessor on MemMachineClient
  • get_config() / get_resources() — read current configuration and resource status
  • update_memory_config() — update episodic/semantic memory settings
  • add_embedder() / add_language_model() — add new resources
  • delete_embedder() / delete_language_model() — remove resources
  • retry_embedder() / retry_language_model() / retry_reranker() — retry failed resources

Follows the same patterns as the existing Project and Memory SDK classes (closed-client checks, timeout forwarding, Pydantic response parsing, exception logging).

Fixes/Closes

Fixes #974

Type of change

  • New feature (non-breaking change which adds functionality)
  • Refactor (does not change functionality, e.g., code style improvements, linting)

How Has This Been Tested?

  • Unit Test

890+ unit tests pass (pytest -m "not integration and not slow"), including:

  • test_config_router.py — 27 tests covering all API endpoints including error cases
  • test_config_service.py — 18 tests covering service-layer logic (add/remove/retry resources, memory config updates, persistence)
  • test_configuration.py — 5 new tests for config save/load and config_file_path tracking
  • test_resource_manager.py — 3 new tests for save_config and config property access
  • test_config.py — 23 tests covering the client SDK Config class (happy paths, closed-client errors, timeout forwarding, exception propagation)

Checklist

  • My code follows the style guidelines of this project (See STYLE_GUIDE.md)
  • I have performed a self-review of my own code
  • My changes generate no new warnings
  • I have added unit tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have checked my code and corrected any misspellings

Maintainer Checklist

  • Confirmed all checks passed
  • Contributor has signed the commit(s)
  • Reviewed the code
  • Run, Tested, and Verified the change(s) work as expected

Screenshots/Gifs

N/A

Further comments

None

@jealous jealous self-assigned this Jan 28, 2026
@jealous
jealous marked this pull request as draft January 28, 2026 02:00
@jealous
jealous force-pushed the bugfix/allowEmptyModel branch from 22ebb3f to 1492646 Compare January 28, 2026 22:23
@jealous
jealous requested a review from Copilot January 28, 2026 22:23
@jealous jealous changed the title Allow runtime resource configuration Add runtime configuration API and graceful startup Jan 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds runtime resource configuration capabilities to the MemMachine server, enabling dynamic management of embedders, language models, rerankers, and memory configurations without requiring server restarts.

Changes:

  • Introduces REST API endpoints for runtime configuration management
  • Implements configuration persistence to YAML files
  • Adds error tracking and retry mechanisms for failed resource initialization

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/memmachine/common/api/config_spec.py Defines API request/response models for configuration operations
src/memmachine/common/api/doc.py Adds documentation strings for configuration API endpoints
src/memmachine/common/configuration/init.py Adds config file path tracking and save() method to Configuration
src/memmachine/common/configuration/embedder_conf.py Adds null-safety check for embedder parsing
src/memmachine/common/configuration/language_model_conf.py Adds null-safety check for language model parsing
src/memmachine/common/errors.py Introduces ResourceNotReadyError for failed resource initialization
src/memmachine/common/resource_manager/base_manager.py Creates base class with shared resource management logic
src/memmachine/common/resource_manager/embedder_manager.py Refactors to use BaseResourceManager, adds add/remove operations
src/memmachine/common/resource_manager/language_model_manager.py Refactors to use BaseResourceManager, adds add/remove operations
src/memmachine/common/resource_manager/reranker_manager.py Refactors to use BaseResourceManager pattern
src/memmachine/common/resource_manager/resource_manager.py Adds manager property accessors and save_config() method
src/memmachine/main/memmachine.py Refactors default resolution logic, adds resource_manager property
src/memmachine/server/api_v2/config_router.py Implements configuration API endpoints
src/memmachine/server/api_v2/config_service.py Implements configuration service logic
src/memmachine/server/api_v2/exceptions.py Extracts RestError class to separate module
src/memmachine/server/api_v2/router.py Removes RestError class, adds config_router registration
tests/memmachine/common/api/test_version.py Updates version regex to support additional local version identifiers
tests/memmachine/common/configuration/test_configuration.py Adds tests for config file path tracking and save() method
tests/memmachine/common/resource_manager/test_resource_manager.py Adds tests for save_config() functionality
tests/memmachine/server/api_v2/test_config_router.py Adds comprehensive tests for configuration API endpoints
tests/memmachine/server/api_v2/test_config_service.py Adds tests for configuration service functions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jealous
jealous force-pushed the bugfix/allowEmptyModel branch 4 times, most recently from 299adeb to 4ee0fc9 Compare January 28, 2026 23:19
@jealous
jealous marked this pull request as ready for review January 28, 2026 23:28
@jealous
jealous force-pushed the bugfix/allowEmptyModel branch 2 times, most recently from 694fa05 to f92feae Compare January 29, 2026 18:56
- Add REST API endpoints for managing resources at runtime
  (add/remove/retry embedders, language models; view status)
- Add PUT /api/v2/config/memory endpoint to update episodic
  and semantic memory configuration
- Introduce graceful startup: resources that fail to build are
  tracked as FAILED instead of crashing the server
- Add enabled/disabled flag for semantic memory with auto-disable
  when required fields (database, llm_model, embedding_model) are
  missing
- Make episodic memory initialization resilient to missing
  embedder/reranker configuration
- Extract shared resource manager logic into BaseResourceManager
- Persist configuration changes to YAML file on updates
- Add rest_client Config class wrapping all /api/v2/config endpoints
  with MemMachineClient.config() convenience accessor
- Add comprehensive tests for config router, config service, and
  client SDK config module

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@jealous
jealous force-pushed the bugfix/allowEmptyModel branch from f92feae to 34704c9 Compare January 29, 2026 22:37
@jealous jealous changed the title Add runtime configuration API and graceful startup Add runtime configuration API, graceful startup, and client SDK Jan 29, 2026
@mwqgithub
mwqgithub merged commit b63b1c0 into main Jan 30, 2026
48 checks passed
edwinyyyu pushed a commit to edwinyyyu/MemMachine that referenced this pull request Feb 2, 2026
…achine#993)

- Add REST API endpoints for managing resources at runtime
  (add/remove/retry embedders, language models; view status)
- Add PUT /api/v2/config/memory endpoint to update episodic
  and semantic memory configuration
- Introduce graceful startup: resources that fail to build are
  tracked as FAILED instead of crashing the server
- Add enabled/disabled flag for semantic memory with auto-disable
  when required fields (database, llm_model, embedding_model) are
  missing
- Make episodic memory initialization resilient to missing
  embedder/reranker configuration
- Extract shared resource manager logic into BaseResourceManager
- Persist configuration changes to YAML file on updates
- Add rest_client Config class wrapping all /api/v2/config endpoints
  with MemMachineClient.config() convenience accessor
- Add comprehensive tests for config router, config service, and
  client SDK config module

Co-authored-by: Claude Opus 4.5 <[email protected]>
edwinyyyu added a commit to edwinyyyu/MemMachine that referenced this pull request Sep 10, 2026
Every pull request against speedkick fails the "Run Python unit tests"
job on ubuntu and macos with

    FAILED .../common/api/test_version.py::test_get_version
    AssertionError: Invalid server version: 0.3.9.post2.dev14+g2c69bb83e

The annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from
speedkick, after v0.3.9") sits on speedkick's ancestry and is the
nearest tag for every commit since, so setuptools-scm renders each
build as 0.3.9.post2.devN+g<sha>. The test's hand-written pattern had
no post-release segment. main is unaffected because the tag is not
reachable from it.

The test is deleted rather than patched. get_version() is a wrapper
around importlib.metadata.version for two distribution names; nothing
in it produces a version shape. The shape comes from setuptools-scm,
the repository's tags and the working tree, so the test could only
ever fail for environmental reasons, and that is its whole history:
the regex was widened in MemMachine#993 (dotted local segment), MemMachine#1038
(two-component tag) and would have been again here, while the function
under test has not changed since MemMachine#951 wrote it. It also rejected a
documented output: MemMachine#951 specifies client_version as "not available"
when memmachine-client is not installed, which is the case for a wheel
install of the server, whose package does not depend on the client.
The test passed only because the uv workspace installs every member.
test_version_string stays; it tests the model's own string form.

The check moves to the layer that owns it. The Test Server Package
workflow already builds the wheels and installs them; a new step runs
the installed memmachine-server --version and requires the reported
server version to equal the version in the wheel's filename. That ties
the binary to the artifact it shipped in without asserting anything
about the version's format.

Verified locally on this branch: the trimmed test file passes and is
clean under ruff check and ruff format --check; the new step, run
verbatim against wheels built from this tree and installed into a
fresh virtualenv, passes.

The windows unit-test jobs report success only because the step runs
under pwsh, which does not stop on the first command's non-zero exit
and returns the last command's status (the client suite). Their server
run fails this test plus two windows-only tests
(test_nebula_graph_vector_literals charmap decode, test_index_persistence
flush guard). That masking is left as is: forcing bash on the step
would turn windows red until those two are fixed.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_019zu1CSWVso2NA4bWPYmw8Z
edwinyyyu added a commit that referenced this pull request Sep 10, 2026
…kick) (#1604)

Verify the version in the package workflow, not in a unit test

Every pull request against speedkick fails the "Run Python unit tests"
job on ubuntu and macos with

    FAILED .../common/api/test_version.py::test_get_version
    AssertionError: Invalid server version: 0.3.9.post2.dev14+g2c69bb83e

The annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from
speedkick, after v0.3.9") sits on speedkick's ancestry and is the
nearest tag for every commit since, so setuptools-scm renders each
build as 0.3.9.post2.devN+g<sha>. The test's hand-written pattern had
no post-release segment. main is unaffected because the tag is not
reachable from it.

The test is deleted rather than patched. get_version() is a wrapper
around importlib.metadata.version for two distribution names; nothing
in it produces a version shape. The shape comes from setuptools-scm,
the repository's tags and the working tree, so the test could only
ever fail for environmental reasons, and that is its whole history:
the regex was widened in #993 (dotted local segment), #1038
(two-component tag) and would have been again here, while the function
under test has not changed since #951 wrote it. It also rejected a
documented output: #951 specifies client_version as "not available"
when memmachine-client is not installed, which is the case for a wheel
install of the server, whose package does not depend on the client.
The test passed only because the uv workspace installs every member.
test_version_string stays; it tests the model's own string form.

The check moves to the layer that owns it. The Test Server Package
workflow already builds the wheels and installs them; a new step runs
the installed memmachine-server --version and requires the reported
server version to equal the version in the wheel's filename. That ties
the binary to the artifact it shipped in without asserting anything
about the version's format.

Verified locally on this branch: the trimmed test file passes and is
clean under ruff check and ruff format --check; the new step, run
verbatim against wheels built from this tree and installed into a
fresh virtualenv, passes.

The windows unit-test jobs report success only because the step runs
under pwsh, which does not stop on the first command's non-zero exit
and returns the last command's status (the client suite). Their server
run fails this test plus two windows-only tests
(test_nebula_graph_vector_literals charmap decode, test_index_persistence
flush guard). That masking is left as is: forcing bash on the step
would turn windows red until those two are fixed.


Claude-Session: https://claude.ai/code/session_019zu1CSWVso2NA4bWPYmw8Z

Co-authored-by: Claude Fable 5.1 <[email protected]>
malatewang added a commit that referenced this pull request Sep 19, 2026
test_get_version asserts that get_version() returns a version matching a
hand-written pattern: release, optional .devN, optional local segment.
On speedkick that assertion fails on every pull request, because the
annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from
speedkick, after v0.3.9") sits on that branch's ancestry and
setuptools-scm renders each build as 0.3.9.post2.devN+g<sha>, a shape
the pattern never allowed. main passes today only because that tag is
not yet reachable from it; it becomes reachable the moment speedkick
merges, and any post-release tag would do the same.

The test is deleted rather than patched. get_version() is a wrapper
around importlib.metadata.version for two distribution names; nothing
in it produces a version shape. The shape comes from setuptools-scm,
the repository's tags and the working tree, so the test could only
ever fail for environmental reasons, and that is its whole history:
the regex was widened in #993 (dotted local segment), #1038
(two-component tag) and would have been again here, while the function
under test has not changed since #951 wrote it. It also rejected a
documented output: #951 specifies client_version as "not available"
when memmachine-client is not installed, which is the case for a wheel
install of the server, whose package does not depend on the client.
The test passed only because the uv workspace installs every member.
test_version_string stays; it tests the model's own string form.

The check moves to the layer that owns it. The Test Server Package
workflow already builds the wheels and installs them; a new step runs
the installed memmachine-server --version and requires the reported
server version to equal the version in the wheel's filename. That ties
the binary to the artifact it shipped in without asserting anything
about the version's format.

Verified locally on this branch: the trimmed test file passes and is
clean under ruff check and ruff format --check; the new step, run
verbatim against wheels built from this tree and installed into a
fresh virtualenv, passes (and reports "client: not available", which
the deleted test would have rejected).

Unrelated but worth knowing: the pytest workflow's test step has no
shell key, so on windows it runs under pwsh, which does not stop on the
first command's non-zero exit and returns the last command's status
(the client suite). A server-suite failure on windows would not fail
the job. On main the windows server run passes today (1830 passed on
#1584's run), so nothing is hidden yet; on speedkick two windows-only
failures are. Forcing bash there is left for when those are fixed.

Same change as #1604 on speedkick, applied to main.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
Co-authored-by: Shu Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Sometimes the LLM is stuck in an infinite loop when ingesting semantic memories

5 participants