Repository navigation
Fix local failing tests and use a single flag for integration tests - #1038
Merged
Merged
Conversation
Signed-off-by: Edwin Yu <[email protected]>
edwinyyyu
force-pushed
the
weird_tests
branch
from
February 4, 2026 00:36
b539820 to
e839d0f
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes failing tests that occurred when sentence_transformers is not installed or Docker is unavailable. The changes replace the command-line --integration flag with pytest markers and environment checks to gracefully skip tests when dependencies are missing.
Changes:
- Removed the
--integrationcommand-line option in favor of automatic test skipping based on available dependencies - Added helper functions and decorators to detect Docker availability and
sentence_transformersinstallation - Modified test fixtures to use lazy imports for optional dependencies
- Updated the version test regex pattern to handle additional version formats
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/memmachine/conftest.py | Removed --integration CLI option; added is_docker_available(), requires_sentence_transformers, and requires_docker helpers |
| tests/memmachine/episodic_memory/long_term_memory/test_long_term_memory.py | Moved imports inside fixtures and added skip decorators for sentence_transformers-dependent tests |
| tests/memmachine/episodic_memory/declarative_memory/test_declarative_memory.py | Moved imports inside fixtures and added skip decorators for sentence_transformers-dependent tests |
| tests/memmachine/common/vector_graph_store/test_neo4j_vector_graph_store.py | Added Docker availability check to neo4j_connection_info fixture |
| tests/memmachine/common/resource_manager/test_reranker_manager.py | Added skip decorators for sentence_transformers-dependent tests |
| tests/memmachine/common/resource_manager/test_embedder_manager.py | Added skip decorators for sentence_transformers-dependent tests |
| tests/memmachine/common/reranker/test_cross_encoder_reranker.py | Added pytest.importorskip and simplified fixtures |
| tests/memmachine/common/embedder/test_sentence_transformer_embedder.py | Added pytest.importorskip and simplified fixtures |
| tests/memmachine/common/api/test_version.py | Updated version regex to support major.minor format in addition to major.minor.patch |
| pyproject.toml | Added default pytest option to exclude integration tests |
| .github/workflows/pytest-integration.yml | Removed --integration flag from pytest commands |
| .github/workflows/installation-test.yml | Removed --integration flag from pytest commands |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
malatewang
approved these changes
Feb 4, 2026
o-love
approved these changes
Feb 4, 2026
edwinyyyu
added a commit
to edwinyyyu/MemMachine
that referenced
this pull request
Sep 10, 2026
Every pull request against speedkick fails the "Run Python unit tests"
job on ubuntu and macos with
FAILED .../common/api/test_version.py::test_get_version
AssertionError: Invalid server version: 0.3.9.post2.dev14+g2c69bb83e
The annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from
speedkick, after v0.3.9") sits on speedkick's ancestry and is the
nearest tag for every commit since, so setuptools-scm renders each
build as 0.3.9.post2.devN+g<sha>. The test's hand-written pattern had
no post-release segment. main is unaffected because the tag is not
reachable from it.
The test is deleted rather than patched. get_version() is a wrapper
around importlib.metadata.version for two distribution names; nothing
in it produces a version shape. The shape comes from setuptools-scm,
the repository's tags and the working tree, so the test could only
ever fail for environmental reasons, and that is its whole history:
the regex was widened in MemMachine#993 (dotted local segment), MemMachine#1038
(two-component tag) and would have been again here, while the function
under test has not changed since MemMachine#951 wrote it. It also rejected a
documented output: MemMachine#951 specifies client_version as "not available"
when memmachine-client is not installed, which is the case for a wheel
install of the server, whose package does not depend on the client.
The test passed only because the uv workspace installs every member.
test_version_string stays; it tests the model's own string form.
The check moves to the layer that owns it. The Test Server Package
workflow already builds the wheels and installs them; a new step runs
the installed memmachine-server --version and requires the reported
server version to equal the version in the wheel's filename. That ties
the binary to the artifact it shipped in without asserting anything
about the version's format.
Verified locally on this branch: the trimmed test file passes and is
clean under ruff check and ruff format --check; the new step, run
verbatim against wheels built from this tree and installed into a
fresh virtualenv, passes.
The windows unit-test jobs report success only because the step runs
under pwsh, which does not stop on the first command's non-zero exit
and returns the last command's status (the client suite). Their server
run fails this test plus two windows-only tests
(test_nebula_graph_vector_literals charmap decode, test_index_persistence
flush guard). That masking is left as is: forcing bash on the step
would turn windows red until those two are fixed.
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_019zu1CSWVso2NA4bWPYmw8Z
edwinyyyu
added a commit
that referenced
this pull request
Sep 10, 2026
…kick) (#1604) Verify the version in the package workflow, not in a unit test Every pull request against speedkick fails the "Run Python unit tests" job on ubuntu and macos with FAILED .../common/api/test_version.py::test_get_version AssertionError: Invalid server version: 0.3.9.post2.dev14+g2c69bb83e The annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from speedkick, after v0.3.9") sits on speedkick's ancestry and is the nearest tag for every commit since, so setuptools-scm renders each build as 0.3.9.post2.devN+g<sha>. The test's hand-written pattern had no post-release segment. main is unaffected because the tag is not reachable from it. The test is deleted rather than patched. get_version() is a wrapper around importlib.metadata.version for two distribution names; nothing in it produces a version shape. The shape comes from setuptools-scm, the repository's tags and the working tree, so the test could only ever fail for environmental reasons, and that is its whole history: the regex was widened in #993 (dotted local segment), #1038 (two-component tag) and would have been again here, while the function under test has not changed since #951 wrote it. It also rejected a documented output: #951 specifies client_version as "not available" when memmachine-client is not installed, which is the case for a wheel install of the server, whose package does not depend on the client. The test passed only because the uv workspace installs every member. test_version_string stays; it tests the model's own string form. The check moves to the layer that owns it. The Test Server Package workflow already builds the wheels and installs them; a new step runs the installed memmachine-server --version and requires the reported server version to equal the version in the wheel's filename. That ties the binary to the artifact it shipped in without asserting anything about the version's format. Verified locally on this branch: the trimmed test file passes and is clean under ruff check and ruff format --check; the new step, run verbatim against wheels built from this tree and installed into a fresh virtualenv, passes. The windows unit-test jobs report success only because the step runs under pwsh, which does not stop on the first command's non-zero exit and returns the last command's status (the client suite). Their server run fails this test plus two windows-only tests (test_nebula_graph_vector_literals charmap decode, test_index_persistence flush guard). That masking is left as is: forcing bash on the step would turn windows red until those two are fixed. Claude-Session: https://claude.ai/code/session_019zu1CSWVso2NA4bWPYmw8Z Co-authored-by: Claude Fable 5.1 <[email protected]>
malatewang
added a commit
that referenced
this pull request
Sep 19, 2026
test_get_version asserts that get_version() returns a version matching a hand-written pattern: release, optional .devN, optional local segment. On speedkick that assertion fails on every pull request, because the annotated tag v0.3.9-post1 (2026-08-31, "Post-release build from speedkick, after v0.3.9") sits on that branch's ancestry and setuptools-scm renders each build as 0.3.9.post2.devN+g<sha>, a shape the pattern never allowed. main passes today only because that tag is not yet reachable from it; it becomes reachable the moment speedkick merges, and any post-release tag would do the same. The test is deleted rather than patched. get_version() is a wrapper around importlib.metadata.version for two distribution names; nothing in it produces a version shape. The shape comes from setuptools-scm, the repository's tags and the working tree, so the test could only ever fail for environmental reasons, and that is its whole history: the regex was widened in #993 (dotted local segment), #1038 (two-component tag) and would have been again here, while the function under test has not changed since #951 wrote it. It also rejected a documented output: #951 specifies client_version as "not available" when memmachine-client is not installed, which is the case for a wheel install of the server, whose package does not depend on the client. The test passed only because the uv workspace installs every member. test_version_string stays; it tests the model's own string form. The check moves to the layer that owns it. The Test Server Package workflow already builds the wheels and installs them; a new step runs the installed memmachine-server --version and requires the reported server version to equal the version in the wheel's filename. That ties the binary to the artifact it shipped in without asserting anything about the version's format. Verified locally on this branch: the trimmed test file passes and is clean under ruff check and ruff format --check; the new step, run verbatim against wheels built from this tree and installed into a fresh virtualenv, passes (and reports "client: not available", which the deleted test would have rejected). Unrelated but worth knowing: the pytest workflow's test step has no shell key, so on windows it runs under pwsh, which does not stop on the first command's non-zero exit and returns the last command's status (the client suite). A server-suite failure on windows would not fail the job. On main the windows server run passes today (1830 passed on #1584's run), so nothing is hidden yet; on speedkick two windows-only failures are. Forcing bash there is left for when those are fixed. Same change as #1604 on speedkick, applied to main. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: Shu Wang <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of the change
Version test was failing locally. Tests have been failing in main for ages. This should make it somewhat better.
Tests were failing when sentence_transformers was not installed.
Tests were failing when Docker is not up.
Description
Used Claude Code.
Now tests should just work after clone.
Unit tests:
uv run pytestoruv run --all-extras pytestIntegration tests:
uv run pytest -m integrationoruv run --all-extras pytest -m integrationAll tests:
uv run pytest -m ""oruv run --all-extras pytest -m ""Limitations:
Type of change
How Has This Been Tested?
All tests pass locally or are skipped.
Checklist
Maintainer Checklist