Skip to content

perf(gfql): validate a query's ops once per call, not twice - #2088

Merged
lmeyerov merged 3 commits into
masterfrom
perf/gfql-skip-redundant-chain-validation
Sep 17, 2026
Merged

lmeyerov merged 3 commits into
masterfrom
perf/gfql-skip-redundant-chain-validation

Conversation

@lmeyerov

Copy link
Copy Markdown
Contributor

Problem

Every AST object in a query is validated twice inside a single execution. gfql builds a Chain from the caller's ops, which validates them; the execution path then constructs a throwaway Chain purely to validate them again, and deliberately unwraps an already-validated Chain to do it.

Measured redundancy factor 1.7–1.8x, by counting ASTSerializable.validate calls within one execution:

query validate() calls distinct objects
seed-lookup 12 7
recent-replies 24 13

This is not a benchmark artifact

Worth stating, because the obvious version of this change would be one. The SNB harness reuses AST objects across repetitions, so a memo keyed on object identity would show a large benchmark win and do nothing for a real caller. That was rejected.

What is removed here is redundant within a single call. A caller that passes a plain list to gfql() and reuses nothing pays both passes today.

What is preserved

The re-validation exists to catch a caller that built a Chain, mutated its ops, and then executed it. That caller is unaffected and still raises — pinned for both entry points.

The signal is explicit and narrow: Chain.gfql_validated(), set only by gfql on a Chain it just built in the same call. It is never inferred from the Chain itself, and it cannot be forged onto one constructed with validate=False. Two sites needed it: the Polars branch, and the pandas path, which unwraps the Chain before validating and so loses any marker.

Measured

In-process, arms alternated per round, 6 rounds × 31 reps, each against its own A/A control. LDBC SNB SF0.1, Polars.

query before after saved A/A floor
seed-lookup 0.995 ms 0.897 ms −0.098 ms (−9.8%) 0.034 ms
message-replies 1.714 ms 1.640 ms −0.074 ms (−4.3%) 0.011 ms
recent-replies 3.488 ms 3.390 ms −0.098 ms (−2.8%) 0.022 ms

Three to seven times its own floor on each. It is a fixed per-query cost, so it helps every query and every engine, not just these.

No end-to-end claim is made here. A cross-process A/B against master cannot resolve 0.1 ms: its own A/A floor ran +3.3% to +12.5%, larger than the effect. Anyone quoting a benchmark number from this needs to measure it in combination, on the bench.

Tests

Eight new, covering both sides of the boundary:

  • gfql validates once; chain called directly still re-validates.
  • A Chain mutated after construction is still caught, through both entry points.
  • The mark cannot be forged onto an unvalidated Chain.
  • Malformed queries raise identical error codes through gfql (three shapes).

Validation suites: 746 pass. Broad compute suite: 14980 pass / 32 fail, an identical failure set to untouched master on this box (24 Polars-conformance, 8 umap, both pre-existing and verified by running the same selection on a clean checkout of c9d7d5360).

Based on current master c9d7d5360. Independent of #2084/#2086/#2087.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp

Every AST object is validated TWICE inside a single execution. `gfql` builds a Chain
from the caller's ops, which validates them; the execution path then constructs a
throwaway Chain purely to validate them again, and deliberately unwraps an
already-validated Chain to do it. Measured redundancy factor 1.7-1.8x by counting
ASTSerializable.validate calls in one execution.

This is not a benchmark artifact, and that was checked before building. The SNB harness
reuses AST objects across repetitions, so a memo keyed on object identity would show a
large benchmark win and do nothing for a real caller; that was rejected. What is removed
here is redundant WITHIN a single call, so a caller that passes a plain list and reuses
nothing pays both passes today.

The re-validation exists to catch a caller that built a Chain, mutated its ops, and then
executed it. That caller is unaffected and still raises, pinned for both entry points.
The signal is explicit and narrow: Chain.gfql_validated(), set only by gfql on a Chain it
just built, never inferred from the Chain itself, and it cannot be forged onto a Chain
constructed with validate=False. Two sites needed it -- the polars branch, and the pandas
path, which unwraps the Chain before validating and so loses any marker.

Measured in-process, arms alternated, 6 rounds x 31 reps, each against its own A/A
control (LDBC SNB SF0.1, polars):

  seed-lookup      0.995 -> 0.897 ms  (-0.098, A/A floor 0.034)
  message-replies  1.714 -> 1.640 ms  (-0.074, A/A floor 0.011)
  recent-replies   3.488 -> 3.390 ms  (-0.098, A/A floor 0.022)

Three to seven times its own floor on each. It is a fixed per-query cost, so it helps
every query and every engine.

No end-to-end claim: a cross-process A/B against master cannot resolve 0.1 ms, its own
A/A floor running +3.3% to +12.5%.

Tests: 8 new, covering both sides -- gfql validates once, chain called directly still
re-validates, a Chain mutated after construction is still caught through both entry
points, the mark cannot be forged, and malformed queries raise identical error codes.
Validation suites 746 pass. Broad compute suite 14980 pass / 32 fail, an identical
failure set to untouched master on this box.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
Comment thread graphistry/compute/chain.py Outdated
# Construct a fresh validator: the constructor validates children once. Skipped
# only when `gfql` built this Chain in this same call, where re-validating cannot
# observe a change -- every other caller may have mutated its ops since.
if not getattr(ops, "_gfql_validated_in_call", False):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aviod dynamic typing

…ehaviour boundary

Review feedback, both items.

DYNAMIC ACCESS. The skip was decided by `getattr(ops, "_gfql_validated_in_call", False)`
-- an attribute probe used as control flow, which the review skill rejects: control-flow
checks must use structured signals. `chain` accepts a list or a Chain and only the latter
can carry the mark, so the check is now a typed predicate on the known type,
`Chain.ops_were_validated_in_this_call(ops)`, doing an isinstance narrow and reading a
declared bool attribute. Both call sites use it; no probe remains.

TESTS ASSERTED IMPLEMENTATION, NOT BEHAVIOUR. The old pins counted Chain constructions.
The real boundary is "could these ops have changed since they were validated?", and the
suite now tests what a CALLER observes on each side of it:

  could have changed  -> a Chain mutated after construction still raises, through BOTH
                         entry points; a Chain reused across calls and mutated between
                         them still raises; a plain list reused and mutated still raises;
                         three malformed shapes raise the same code through both entries
  could not have      -> well-formed ops return the same rows through both entry points,
                         and gfql and chain agree on the same ops

One implementation assertion remains, labelled as such: the engagement pin that fails if
the optimization is removed entirely. Without it the behaviour tests would all pass with
the change reverted.

Verified in both directions rather than assumed. Over-applying the skip (skipping for
every caller) fails `test_a_chain_mutated_after_construction_still_raises[chain]`.
Removing the optimization fails the engagement pin. 15 pass as shipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
Comments-only. The predicate `ops_were_validated_in_this_call` already says what the
check is, and the tests already say what the contract is, so the paragraphs explaining
both were doing no work.

Removed: the two-line attribute comment (the attribute names say it), the five-line
`gfql_validated` docstring (now one line), the four-line predicate docstring (the name is
the documentation), and the appended halves of the two call-site comments -- those are
back to the one-line originals they had before this branch touched them.

Test file: module docstring cut from twelve lines to three, six per-test docstrings that
restated their own test name deleted, two trimmed, three section banners removed.

Behaviour unchanged: chain.py is 3 insertions / 21 deletions, all prose. Contract and
reuse suites 43 pass; lint, guards and mypy clean.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov
lmeyerov merged commit 7ce5315 into master Sep 17, 2026
24 of 25 checks passed
lmeyerov added a commit that referenced this pull request Sep 17, 2026
…tree

The vendored numbers were pre-release and four runs were 52-61 compute commits
past their measuring commit, against a policy limit of 12 -- this PR's own
drift gate was failing on them. seed-lookup published 1.018 ms at SF0.1 where
the shipped tree measures 0.218, and 1.363 at SF1 where it measures 0.281.

Re-vendored from pyg-bench with SNB measured on pygraphistry f283a30 (master
with #2084, #2086, #2087, #2088, #2090) and GraphBench on 24c0b1e, both under
the DGX idle gate and host perf lock with canonical rows asserted identical
across engines, scales and repetitions.

docs/test_bench_numbers.py now passes: 37 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant