Repository navigation
Refuse a filter value of the wrong type for a datetime column, and answer an invalid list filter with 422 (port of #1620) - #1786
Draft
edwinyyyu wants to merge 1 commit into
Conversation
…swer an invalid list filter with 422 (port of MemMachine#1620) * Refuse a filter value of the wrong type for a datetime column before the statement is built (speedkick) A column-encoded leaf now pairs a DateTime column with a datetime value and nothing else; any other pairing raises ValueError in compile_sql_filter, on every SQL-backed store, with a message that names the field and points at date('...'). SQLAlchemy types a bind by its Python value, not by the column, so created_at > '2026-01-01T00:00:00Z' reached PostgreSQL as timestamp with time zone > character varying and failed as a ProgrammingError, which the routes answered with 500; on SQLite the same statement ran as a text comparison. The reverse pairing, a date() literal against a non-datetime column, failed the same way. Tests: the compile-level cases (str, int and IN against a datetime column; date() against a string column; the ordering that does compile) and the episode store's created_at filter on both engines. The normalization test's fixture column becomes a DateTime, which is what it models. Refs MemMachine#1619 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01FNHipcrwW9JByUwhg3JVcj * Answer an invalid filter on /memories/list with 422, as /memories/search does (speedkick) list_memories mapped no error to a status, so a FilterParseError, an unknown field or a value of the wrong type was a 500 there and a 422 on search. The same ValueError -> 422 mapping now applies to both. Refs MemMachine#1619 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01FNHipcrwW9JByUwhg3JVcj * Document the date() literal on the filter field (speedkick) The filter description names date('...') as the form a datetime field such as created_at compares with, and says a quoted string or a number there is an invalid argument. docs/openapi.json carries the same two descriptions; the rest of a local regeneration is drift the main workflow owns. Refs MemMachine#1619 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01FNHipcrwW9JByUwhg3JVcj --------- Ported onto main from 8d7b832 on speedkick. /memories/list on main lists every memory type when none is given; the port keeps that default and maps a ValueError from listing to 422 around it. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: Shu Wang <[email protected]> Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
This was referenced Oct 7, 2026
Draft
[session storage 2/2] Remove open-or-create from both stores, and close from the segment store
#1625
Draft
[qdrant options] Let a deployment tune a Qdrant collection's HNSW, optimizers and quantization
#1618
Draft
edwinyyyu
marked this pull request as draft
October 7, 2026 20:49
This was referenced Oct 7, 2026
Draft
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1619.
Port
Copy of #1620, merged into
speedkickas8d7b832a1, ontomain, which has the same parser, compiler, and route, so the same defect. The one conflict is inlist_memories:mainlists every memory type when a request names none, and the port keeps that default inside #1620's 422 mapping.Purpose of the change
A filter such as
created_at > '2026-01-01T00:00:00Z'reached PostgreSQL astimestamp with time zone > character varying, failed there, and came back as HTTP 500 from/memories/list(episodic and semantic) and from/memories/searchwithtypes: ["semantic"]. The same filter ran on SQLite as a text comparison. SQLAlchemy types a bind by its Python value, so the mismatch was invisible until the database rejected it.Three changes, one per concern, in one commit here as #1620 merged squashed:
compile_sql_filterrefuses the pairing before a statement is built. A column-encoded leaf pairs aDateTimecolumn with a datetime value and nothing else; astr, anint, or anINlist againstcreated_at(ortimestampon the segment store), and adate()literal against a non-datetime column, raiseValueErrorwith a message that names the field and points atdate('...'). This holds on every SQL-backed store that resolves a datetime column: the episode store, the segment store, and both semantic feature stores.list_memoriesmapsValueErrorto 422, assearch_memoriesalready did. Before, every invalid filter on list (a syntax error, an unknown field, this mismatch) was a 500.filterfield's description (SpecDoc.FILTER_MEM, and the two copies indocs/openapi.json) namesdate('...')as the form a datetime field compares with. Only those two descriptions change in the spec; a full local regeneration also drifts onValidationErrorandinfo.version, which the main workflow owns.The value is rejected rather than coerced: the filter language types a value by its literal,
date()is the datetime literal, and aproperties_jsonleaf already matches only a value of the declared type. Coercing a string here would make the column encoding disagree with the others.Relation to #1616: that PR makes the parser reject a string in an ordering comparison, which covers
>/</>=/<=with astr. This change is at the column boundary, so it also covers=,!=, an integer, and anINlist, and it does not depend on the vector store stack. The two compose; the rebase of either over the other is a few lines in_compile_column_leaf.Stack
21 open PRs: three independent PRs, and the vector store tree of short parallel branches. Every PR in the tree has
feat/horizontal-scalingas its GitHub base, and the independent PRs havemain. The branches are in a fork, and a pull request can target only this repository's branches, so the on column gives the order the PRs build on each other. A stacked PR's diff on GitHub includes the PRs under it until they merge.Independent of the vector store tree, directly on
main:mainmainmainThe vector store tree. Each PR builds on the one in its on column; PRs on the same parent are parallel branches and do not depend on each other. #1631 is closed, superseded by #1733–#1736, which hold its changes split in four, with review changes since. #1702 and #1670 sit beneath #1627, whose code depends on them. Until the PRs under it merge, their changes show in a stacked PR's diff.
mainfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scaling)feat/horizontal-scalingfeat/horizontal-scalingfeat/horizontal-scalingThis PR is its one commit,
899719556, directly onmain, independent of the vector store tree: review and merge it in any order.Verification
As measured on
speedkickfor #1620:test_sql_filter_util.py: the fixture model gains aDateTimecolumn; four mismatched filters raise with "holds a datetime",name > date(...)raises with "does not hold a datetime", andcreated_at > date(...)/>= date(...)compile and select the expected rows.test_bounds_are_normalized_to_utc's fixture column becomes aDateTime, which is what it models.test_episode_storage.py:created_at > '...',created_at = '...'andcreated_at > 5raiseValueErrorfrom bothget_episode_messagesandget_episode_messages_count, on SQLite and (under-m integration) PostgreSQL.test_router.py:/memories/listanswers aValueErrorfromlist_searchwith 422.uv run pytest packages/server/server_tests: 1937 passed, 3 skipped.-m integrationontest_episode_storage.py(PostgreSQL via testcontainers): 24 passed.ruff check,ruff format --check, andty check --project packages/serverclean.End to end on a server backed by pgvector/pgvector:pg16 (
pool_size: 2,max_overflow: 0): the text form on list (episodic, semantic) and on semantic search answers 422 withField 'created_at' holds a datetime; compare it with date('...'), not str; nine of them in a row leavepg_stat_activityunchanged and the next valid request answers 200 in 4 ms. The same run on the unpatched tree answers 500 at each of those points. The connection leak in the original report did not reproduce on either tree; details in #1619.The port, on
mainatad8ff24b0:ruff check,ruff format --check, andty checkare clean, anduv lock --checkpasses; the full server suite without integration tests passes (2114 passed, 3 skipped); in test containers, the episode store's, the filter compiler's, and the server routes' integration tests pass against PostgreSQL (92 passed). It merges withfeat/horizontal-scalingwithout conflicts.🤖 Generated with Claude Code