Repository navigation
perf(gfql): validate a query's ops once per call, not twice - #2088
Merged
Merged
Conversation
Every AST object is validated TWICE inside a single execution. `gfql` builds a Chain from the caller's ops, which validates them; the execution path then constructs a throwaway Chain purely to validate them again, and deliberately unwraps an already-validated Chain to do it. Measured redundancy factor 1.7-1.8x by counting ASTSerializable.validate calls in one execution. This is not a benchmark artifact, and that was checked before building. The SNB harness reuses AST objects across repetitions, so a memo keyed on object identity would show a large benchmark win and do nothing for a real caller; that was rejected. What is removed here is redundant WITHIN a single call, so a caller that passes a plain list and reuses nothing pays both passes today. The re-validation exists to catch a caller that built a Chain, mutated its ops, and then executed it. That caller is unaffected and still raises, pinned for both entry points. The signal is explicit and narrow: Chain.gfql_validated(), set only by gfql on a Chain it just built, never inferred from the Chain itself, and it cannot be forged onto a Chain constructed with validate=False. Two sites needed it -- the polars branch, and the pandas path, which unwraps the Chain before validating and so loses any marker. Measured in-process, arms alternated, 6 rounds x 31 reps, each against its own A/A control (LDBC SNB SF0.1, polars): seed-lookup 0.995 -> 0.897 ms (-0.098, A/A floor 0.034) message-replies 1.714 -> 1.640 ms (-0.074, A/A floor 0.011) recent-replies 3.488 -> 3.390 ms (-0.098, A/A floor 0.022) Three to seven times its own floor on each. It is a fixed per-query cost, so it helps every query and every engine. No end-to-end claim: a cross-process A/B against master cannot resolve 0.1 ms, its own A/A floor running +3.3% to +12.5%. Tests: 8 new, covering both sides -- gfql validates once, chain called directly still re-validates, a Chain mutated after construction is still caught through both entry points, the mark cannot be forged, and malformed queries raise identical error codes. Validation suites 746 pass. Broad compute suite 14980 pass / 32 fail, an identical failure set to untouched master on this box. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
lmeyerov
commented
Sep 17, 2026
| # Construct a fresh validator: the constructor validates children once. Skipped | ||
| # only when `gfql` built this Chain in this same call, where re-validating cannot | ||
| # observe a change -- every other caller may have mutated its ops since. | ||
| if not getattr(ops, "_gfql_validated_in_call", False): |
Contributor
Author
There was a problem hiding this comment.
aviod dynamic typing
…ehaviour boundary
Review feedback, both items.
DYNAMIC ACCESS. The skip was decided by `getattr(ops, "_gfql_validated_in_call", False)`
-- an attribute probe used as control flow, which the review skill rejects: control-flow
checks must use structured signals. `chain` accepts a list or a Chain and only the latter
can carry the mark, so the check is now a typed predicate on the known type,
`Chain.ops_were_validated_in_this_call(ops)`, doing an isinstance narrow and reading a
declared bool attribute. Both call sites use it; no probe remains.
TESTS ASSERTED IMPLEMENTATION, NOT BEHAVIOUR. The old pins counted Chain constructions.
The real boundary is "could these ops have changed since they were validated?", and the
suite now tests what a CALLER observes on each side of it:
could have changed -> a Chain mutated after construction still raises, through BOTH
entry points; a Chain reused across calls and mutated between
them still raises; a plain list reused and mutated still raises;
three malformed shapes raise the same code through both entries
could not have -> well-formed ops return the same rows through both entry points,
and gfql and chain agree on the same ops
One implementation assertion remains, labelled as such: the engagement pin that fails if
the optimization is removed entirely. Without it the behaviour tests would all pass with
the change reverted.
Verified in both directions rather than assumed. Over-applying the skip (skipping for
every caller) fails `test_a_chain_mutated_after_construction_still_raises[chain]`.
Removing the optimization fails the engagement pin. 15 pass as shipped.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
Comments-only. The predicate `ops_were_validated_in_this_call` already says what the check is, and the tests already say what the contract is, so the paragraphs explaining both were doing no work. Removed: the two-line attribute comment (the attribute names say it), the five-line `gfql_validated` docstring (now one line), the four-line predicate docstring (the name is the documentation), and the appended halves of the two call-site comments -- those are back to the one-line originals they had before this branch touched them. Test file: module docstring cut from twelve lines to three, six per-test docstrings that restated their own test name deleted, two trimmed, three section banners removed. Behaviour unchanged: chain.py is 3 insertions / 21 deletions, all prose. Contract and reuse suites 43 pass; lint, guards and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
This was referenced Sep 17, 2026
lmeyerov
added a commit
that referenced
this pull request
Sep 17, 2026
…tree The vendored numbers were pre-release and four runs were 52-61 compute commits past their measuring commit, against a policy limit of 12 -- this PR's own drift gate was failing on them. seed-lookup published 1.018 ms at SF0.1 where the shipped tree measures 0.218, and 1.363 at SF1 where it measures 0.281. Re-vendored from pyg-bench with SNB measured on pygraphistry f283a30 (master with #2084, #2086, #2087, #2088, #2090) and GraphBench on 24c0b1e, both under the DGX idle gate and host perf lock with canonical rows asserted identical across engines, scales and repetitions. docs/test_bench_numbers.py now passes: 37 passed, 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Every AST object in a query is validated twice inside a single execution.
gfqlbuilds aChainfrom the caller's ops, which validates them; the execution path then constructs a throwawayChainpurely to validate them again, and deliberately unwraps an already-validatedChainto do it.Measured redundancy factor 1.7–1.8x, by counting
ASTSerializable.validatecalls within one execution:This is not a benchmark artifact
Worth stating, because the obvious version of this change would be one. The SNB harness reuses AST objects across repetitions, so a memo keyed on object identity would show a large benchmark win and do nothing for a real caller. That was rejected.
What is removed here is redundant within a single call. A caller that passes a plain list to
gfql()and reuses nothing pays both passes today.What is preserved
The re-validation exists to catch a caller that built a
Chain, mutated its ops, and then executed it. That caller is unaffected and still raises — pinned for both entry points.The signal is explicit and narrow:
Chain.gfql_validated(), set only bygfqlon aChainit just built in the same call. It is never inferred from theChainitself, and it cannot be forged onto one constructed withvalidate=False. Two sites needed it: the Polars branch, and the pandas path, which unwraps theChainbefore validating and so loses any marker.Measured
In-process, arms alternated per round, 6 rounds × 31 reps, each against its own A/A control. LDBC SNB SF0.1, Polars.
Three to seven times its own floor on each. It is a fixed per-query cost, so it helps every query and every engine, not just these.
No end-to-end claim is made here. A cross-process A/B against master cannot resolve 0.1 ms: its own A/A floor ran +3.3% to +12.5%, larger than the effect. Anyone quoting a benchmark number from this needs to measure it in combination, on the bench.
Tests
Eight new, covering both sides of the boundary:
gfqlvalidates once;chaincalled directly still re-validates.Chainmutated after construction is still caught, through both entry points.Chain.gfql(three shapes).Validation suites: 746 pass. Broad compute suite: 14980 pass / 32 fail, an identical failure set to untouched master on this box (24 Polars-conformance, 8 umap, both pre-existing and verified by running the same selection on a clean checkout of
c9d7d5360).Based on current master
c9d7d5360. Independent of #2084/#2086/#2087.🤖 Generated with Claude Code
https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp