Repository navigation
fix(gfql): stop WITH re-entry mutating the caller's Plottable (#1786) - #1793
Merged
Merged
Conversation
`g.gfql("MATCH (a {kind:'a'}) WITH a MATCH (a)-[*]->(b) RETURN count(*)")`
left the re-entry seed on `g` itself, so the NEXT, entirely unrelated query on
the same object was answered against it -- no error, just the previous query's
count. Engine-independent (wrong on pandas too, so an engine A/B never surfaces
it) and a direct violation of the repo's pure-functional contract: users
reasonably reuse one Plottable for many queries.
Leak sites (every `_gfql_*` field assigned during execution was audited):
* gfql_unified `_execute_compiled_query_chain_non_union` -- the reported bug.
`_seeded_dispatch_graph` returns `base_graph` ITSELF when the compiled query
has no seed rows, so `dispatch_graph._gfql_start_nodes = ...` wrote straight
onto the user's graph.
* gfql_unified `gfql()` -- `dispatch_self = self` then
`_gfql_shortest_path_backend = shortest_path_backend`: a per-CALL argument
persisted on the caller's graph and silently became the default for its next
query.
* chain `_handle_boundary_calls` and its polars twin `_run_calls_polars` --
same in-place writes of `_gfql_start_nodes` / `_gfql_rows_base_graph`. Their
target is an internal graph on today's paths rather than the caller's, but
they are the same pattern one refactor away from the same bug, and without
the clear below the polars one rides out on the RESULT.
Approach: carry the state on an INTERNAL COPY, not a try/finally restore. The
per-execution context idea already exists here -- `gfql/index/handoff.py`
attaches its boundary decision with `g.bind()` -- so `gfql/exec_context.py` is
the same shape for the row context, and every site now attaches on the way in
and clears on the way out. Threading the seed through `ExecutionContext` instead
would be the purest fix but it is read deep inside the row pipeline and both
polars pattern appliers, so it is a large refactor; a `finally` restore was
rejected because it still mutates a shared object for the duration of the call
(anything holding `g` concurrently sees the seed) and it cannot fix the second
half of the defect, where the state escapes on the RESULT graph and poisons
queries run against that.
Clearing on the way out matters independently: the result of a WITH query used
to carry the seed, so a follow-up query on that result -- a different graph
entirely -- got the same wrong answer one hop removed.
Mutation-checked: reverting the gfql_unified seed write fails 8 of the new
tests, reverting the shortest-path-backend write fails 2, removing the three
clears fails 3, and a full revert fails 13. Tests run on both pandas and polars
since the defect is engine-independent.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
lmeyerov
added a commit
that referenced
this pull request
Jul 28, 2026
…failure The first version of this file's engine probe wrapped a smoke query in a bare `except Exception: return False`. Two things went wrong with that, both found by trying rather than by argument (on the sibling #1788/#1790 parametrization): * a probe that runs THE SHAPE UNDER TEST disarms its own file — reverting a production guard made the probe raise and every parameter reported SKIPPED instead of failing; * a probe that swallows EVERYTHING is worse — a transient `MemoryError: ... cudaErrorMemoryAllocation` in a fresh GPU container silently dropped cuDF from a run that otherwise passed, and a skipped GPU parameter reads as evidence of passing. So the check now CLASSIFIES: a missing module skips, a recognisable GPU-stack error skips WITH ITS TEXT QUOTED in the reason, and any other failure propagates. The smoke query stays a plain traversal, never the shape under test. GPU receipts — dgx GB10, `graphistry/test-rapids-official:26.02-gfql-polars`, `docker run --gpus all`, cuDF 26.02.01 / polars 1.35.2: 43 passed, 0 skipped (this file plus the #1793 suite), repeated. The marker list is duplicated from the copy landing in `polars_test_utils.py` alongside the #1788/#1790 parametrization; collapse to one definition once both land. Kept duplicated for now so the two PRs stay independently mergeable. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
This was referenced Jul 28, 2026
lmeyerov
added a commit
that referenced
this pull request
Jul 28, 2026
… review) Review asked whether `clear_row_exec_context` should RESTORE the value the graph carried on entry rather than NULLing it — a fair question, because `attach_row_exec_context` INHERITS on the way in (a `None` argument keeps what `g` already carries), so an outer scope's value can reach an inner execution and be dropped by it. Determination: NULL is correct and restore would reintroduce #1786. Three reasons, each now carried by a test rather than by argument: 1. `clear` is PURE. It returns `g.bind()` and never writes through to the object it was handed, so an outer scope that set the field still has it afterwards. There is no caller state to save. That is exactly what separates this from #1786, which WAS an in-place write onto the caller's own graph — restore is the fix for a mutation, and there is no mutation. 2. The only channel restore would change is the RETURN VALUE, and putting the seed back there IS the second half of #1786 ("the result of a WITH query carries the seed, and a follow-up query on that result is answered against it"). Measured: hand-restoring the seed onto a result changes the answer of the next query on it (7 -> 2 -> 1 rows). 3. No execution frame inherits a context it did not set. Instrumenting `attach` over `graphistry/tests/compute` recorded 3907 calls and ZERO inheriting ones. 53 DID enter on a graph already carrying a seed (nested boundary frames) but each was handed the IDENTICAL `start_nodes` parameter, so the frame still owns what it sets. The cross-segment WITH seed travels as the explicit `start_nodes` PARAMETER (`chain_impl(..., start_nodes=)`, `_compiled_query_reentry_state`), never through the graph field, so the field's lifetime is exactly one boundary-call run. `test_exec_context_scoping.py` re-runs that ownership measurement as an assertion over a corpus of the shapes that reach every attach site, so a future path that starts relying on inheritance reopens this decision loudly instead of silently losing an outer value. MUTATION-CHECKED IN THE DECIDING DIRECTION: implementing save/restore at all three attach sites (chain, native polars chain, gfql_unified) fails 4 of the new tests on every runnable engine — while the existing #1793 suite (`test_reentry_caller_graph_immutability.py`) passes UNCHANGED. Those tests could not distinguish the two designs; this file can, which is why it exists. Engine-parametrized over pandas/polars/cuDF/polars-gpu with a fixed engine list plus a runtime probe (not `available_nonpandas_engines()`, which silently shrinks): a non-runnable engine reports SKIPPED, it does not vanish from the report. Also added to `bin/test-polars.sh` — the file has no module-level `importorskip`, so nothing would otherwise have flagged that its polars params run in no lane (#1795 class). RUNTIME DELTA: zero. No production line changed; the only non-test edits are the `clear_row_exec_context` docstring and a CHANGELOG entry, so no pyg-bench lane run is required (CB5). Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
lmeyerov
added a commit
that referenced
this pull request
Jul 28, 2026
…failure The first version of this file's engine probe wrapped a smoke query in a bare `except Exception: return False`. Two things went wrong with that, both found by trying rather than by argument (on the sibling #1788/#1790 parametrization): * a probe that runs THE SHAPE UNDER TEST disarms its own file — reverting a production guard made the probe raise and every parameter reported SKIPPED instead of failing; * a probe that swallows EVERYTHING is worse — a transient `MemoryError: ... cudaErrorMemoryAllocation` in a fresh GPU container silently dropped cuDF from a run that otherwise passed, and a skipped GPU parameter reads as evidence of passing. So the check now CLASSIFIES: a missing module skips, a recognisable GPU-stack error skips WITH ITS TEXT QUOTED in the reason, and any other failure propagates. The smoke query stays a plain traversal, never the shape under test. GPU receipts — dgx GB10, `graphistry/test-rapids-official:26.02-gfql-polars`, `docker run --gpus all`, cuDF 26.02.01 / polars 1.35.2: 43 passed, 0 skipped (this file plus the #1793 suite), repeated. The marker list is duplicated from the copy landing in `polars_test_utils.py` alongside the #1788/#1790 parametrization; collapse to one definition once both land. Kept duplicated for now so the two PRs stay independently mergeable. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
lmeyerov
added a commit
that referenced
this pull request
Jul 28, 2026
Both landed without a CHANGELOG entry. #1793 is the one that matters: it fixes a SILENT WRONG ANSWER that is user-visible on every engine. Reproduced on the pre-fix tree while auditing the merge, so the entry states a measured effect rather than a description: `MATCH (a {grp:1}) WITH a MATCH (a)-[]->(b) RETURN count(*)` followed by a plain `MATCH (a)-[]->(b) RETURN count(*)` on the SAME graph object returned 60 instead of 134, 69 instead of 136 and 61 instead of 143 across seeds. On polars the poisoned graph then raised `NotImplementedError` for EVERY subsequent query — a failed query left the user's object broken. Both are fixed on master; neither was written down. #1792's graphviz half (`KeyError: None` -> an actionable `ValueError` on a bound-but-unlabelled frame) is smaller but is still a user-facing error-message change, so it gets an entry too. DOCS ONLY: no code touched, so runtime delta is zero and no pyg-bench lane run is required (CB5) — checkable from the diff, which is one file. NOT INCLUDED, deliberately. The audit also flagged #1792's `dtype: object` -> `dtype: DType` as a typing WIDENING (`DType = Any`, which is strictly weaker than `object`, and mypy is indifferent between them here — verified). I am not reverting it: `graphistry/compute/typing.py` declares `DType` as the repo's engine-agnostic dtype alias with the comment "Honestly Any -- the concrete type is engine-dependent", and these parameters do receive pandas/cuDF/polars dtypes, so the named alias is the documented convention even though it checks as `Any`. Reverting on the audit's reading would churn against a stated convention on a judgement call that belongs to the owner. Reported, not silently decided. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB
lmeyerov
added a commit
that referenced
this pull request
Jul 28, 2026
* test(gfql): decide #1793's clear-vs-restore question, and pin it (#1793 review) Review asked whether `clear_row_exec_context` should RESTORE the value the graph carried on entry rather than NULLing it — a fair question, because `attach_row_exec_context` INHERITS on the way in (a `None` argument keeps what `g` already carries), so an outer scope's value can reach an inner execution and be dropped by it. Determination: NULL is correct and restore would reintroduce #1786. Three reasons, each now carried by a test rather than by argument: 1. `clear` is PURE. It returns `g.bind()` and never writes through to the object it was handed, so an outer scope that set the field still has it afterwards. There is no caller state to save. That is exactly what separates this from #1786, which WAS an in-place write onto the caller's own graph — restore is the fix for a mutation, and there is no mutation. 2. The only channel restore would change is the RETURN VALUE, and putting the seed back there IS the second half of #1786 ("the result of a WITH query carries the seed, and a follow-up query on that result is answered against it"). Measured: hand-restoring the seed onto a result changes the answer of the next query on it (7 -> 2 -> 1 rows). 3. No execution frame inherits a context it did not set. Instrumenting `attach` over `graphistry/tests/compute` recorded 3907 calls and ZERO inheriting ones. 53 DID enter on a graph already carrying a seed (nested boundary frames) but each was handed the IDENTICAL `start_nodes` parameter, so the frame still owns what it sets. The cross-segment WITH seed travels as the explicit `start_nodes` PARAMETER (`chain_impl(..., start_nodes=)`, `_compiled_query_reentry_state`), never through the graph field, so the field's lifetime is exactly one boundary-call run. `test_exec_context_scoping.py` re-runs that ownership measurement as an assertion over a corpus of the shapes that reach every attach site, so a future path that starts relying on inheritance reopens this decision loudly instead of silently losing an outer value. MUTATION-CHECKED IN THE DECIDING DIRECTION: implementing save/restore at all three attach sites (chain, native polars chain, gfql_unified) fails 4 of the new tests on every runnable engine — while the existing #1793 suite (`test_reentry_caller_graph_immutability.py`) passes UNCHANGED. Those tests could not distinguish the two designs; this file can, which is why it exists. Engine-parametrized over pandas/polars/cuDF/polars-gpu with a fixed engine list plus a runtime probe (not `available_nonpandas_engines()`, which silently shrinks): a non-runnable engine reports SKIPPED, it does not vanish from the report. Also added to `bin/test-polars.sh` — the file has no module-level `importorskip`, so nothing would otherwise have flagged that its polars params run in no lane (#1795 class). RUNTIME DELTA: zero. No production line changed; the only non-test edits are the `clear_row_exec_context` docstring and a CHANGELOG entry, so no pyg-bench lane run is required (CB5). Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB * test(gfql): classify engine availability instead of swallowing every failure The first version of this file's engine probe wrapped a smoke query in a bare `except Exception: return False`. Two things went wrong with that, both found by trying rather than by argument (on the sibling #1788/#1790 parametrization): * a probe that runs THE SHAPE UNDER TEST disarms its own file — reverting a production guard made the probe raise and every parameter reported SKIPPED instead of failing; * a probe that swallows EVERYTHING is worse — a transient `MemoryError: ... cudaErrorMemoryAllocation` in a fresh GPU container silently dropped cuDF from a run that otherwise passed, and a skipped GPU parameter reads as evidence of passing. So the check now CLASSIFIES: a missing module skips, a recognisable GPU-stack error skips WITH ITS TEXT QUOTED in the reason, and any other failure propagates. The smoke query stays a plain traversal, never the shape under test. GPU receipts — dgx GB10, `graphistry/test-rapids-official:26.02-gfql-polars`, `docker run --gpus all`, cuDF 26.02.01 / polars 1.35.2: 43 passed, 0 skipped (this file plus the #1793 suite), repeated. The marker list is duplicated from the copy landing in `polars_test_utils.py` alongside the #1788/#1790 parametrization; collapse to one definition once both land. Kept duplicated for now so the two PRs stay independently mergeable. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB --------- Co-authored-by: Claude Opus 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1786.
The defect
WITHre-entry wrote its seed onto the caller'sPlottableand never cleared it, so the next -- entirely unrelated -- query on the same object was answered against the stale seed. Fails open: no error, just the previous query's count. Engine-independent, so an engine A/B never surfaces it.Leak sites (all
_gfql_*fields assigned during execution were audited)gfql_unified._execute_compiled_query_chain_non_union_gfql_start_nodes_seeded_dispatch_graphhands backbase_graphITSELF when the compiled query has no seed rows, so the write lands on the user's graph. This is the reported bug.gfql_unified.gfql(dispatch_self = self)_gfql_shortest_path_backendchain._handle_boundary_calls_gfql_start_nodes,_gfql_rows_base_graphg_tempis always rebuilt by the middle ops first) -- same pattern, converted for uniformity.polars.chain._run_calls_polars_gfql_start_nodes,_gfql_rows_base_graphg, but onto the result on the all-calls path -- a follow-up query on that result is then answered against another graph's seed.Left alone deliberately:
row/pipeline.py's_gfql_native_sp_cache(an edges-identity-keyed memo, semantically neutral) and the compiled-string-query memo cache (now explicitly owned byself, so it still works).Approach and why
Carry the state on an internal copy, not a
try/finallyrestore. This mechanism already exists in the codebase --gfql/index/handoff.pyattaches its boundary decision withg.bind()-- sogfql/exec_context.pyis the same shape for the row context, and every site now attaches on the way in and clears on the way out.ExecutionContextwould be the purest fix, but it is read deep inside the pandas row pipeline and both polars pattern appliers; that is a large refactor, not a bug fix.finallyrestore was rejected: it still mutates a shared object for the duration of the call, and it cannot fix the second half of the defect -- the state escaping on the result graph.Clearing on the way out matters independently: the result of a
WITHquery used to carry the seed, so a follow-up query on that result got the same wrong answer one hop removed.No banned constructs (
Any,cast,getattr,setattr, barelist,Dict[str, Any],object) -- the fields are already declared onPlottable, so it is ordinary typed attribute access throughout.Tests
graphistry/tests/compute/gfql/test_reentry_caller_graph_immutability.py, 9 cases x {pandas, polars}: two independent queries on one graph, the re-entry answer itself, repeated re-entry, nested (twoWITHs), order-independence, the caller's fields after a query, the result's fields, the pure-call chain result, and the shortest-path-backend argument.Mutation-checked (each leak reintroduced in-container, suite re-run, then restored):
dispatch_graph._gfql_start_nodes = start_nodesdispatch_self._gfql_shortest_path_backend = ...clear_row_exec_contextcallsclear_row_exec_contextaloneThe two attach sites that are not independently observable (chain / polars) are documented as such above rather than claimed as tested.
Verification
All on dgx-spark in
graphistry/test-rapids-official:26.02-gfql-polarswith--gpus all.graphistry/tests/compute: masterc76f23a3= 9 failed, 7050 passed, 90 skipped, 16 xfailed; this branch = 9 failed, 7068 passed, 90 skipped, 16 xfailed. Identical failure set (7 igraph-cudf round-trip + 1 cudf coercion + 1 dask, all pre-existing); +18 = the new tests.🤖 Generated with Claude Code
https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB