Repository navigation
perf(gfql): serve seeded node lookups and two-alias projections on the fast paths - #2035
Conversation
…e fast paths
- seeded_node_lookup fast path: MATCH (p {id}) RETURN p / RETURN p.a, p.b resolves the
seed via the resident node-id index, a resident node-property index, or one scalar
scan and projects the matched rows directly (pandas, polars, cuDF)
- seeded typed hop: RETURN may project from both pattern aliases (IS1 shape), one row
per matched edge, built from the seed and destination rows the reduction holds
- shared seed resolution (_seed_node_rows) so property-indexed seeds engage on the
seeded hop as well
- pins: engagement map, cross-alias cases flip from decline to engage, contract gaps
cleared, new test_seeded_node_lookup_fastpath.py (parity, oracle, index, declines)
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_01WwMmVFo44ADiRRj5cxh7i1
…; gfql_explain no longer reports a served fast path as an index Negative pins: whole-row RETURN with a suffix, expression items, mixed whole-row and property items, LazyFrame nodes, a requested engine that is not the frames' engine, and a destination-side extension dtype all decline to the full path with parity; seed-side extension and datetime columns keep parity; the scan reason (index_missing / served / index_stale) and policy off are visible in gfql_explain. Under index_policy="off" a fast path may still serve, so gfql_explain now derives used_index and the decision from index steps only and reports policy_off. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
318a221 to
f8f2e73
Compare
|
Ready for review at f8f2e73 (rebased on master after #2024/#2028; 77/77 checks green incl. tck-gfql, RTD green). Hardening per the review bar (audit table in the private plan,
Suggested order after this: #2036 next (green; it is what keeps the #2028 scaling pin stable — a loaded runner just measured 30.6x vs the 30x bound on a branch without it), then #2037 → #2038 → #2040. |
|
READY (f8f2e73): 77/77 checks + RTD; boundary audit above; real-GPU 727/0/1 opt-in skip; mypy clean; sentinel + acceptance gate lock-ins. |
…hop, not only its endpoints Edge-alias properties now ride the per-matched-edge frame with the seed and destination lookups, keeping their source dtypes like the full path; a projection that touches any non-destination alias takes that frame, so a no-match seed with an edge property projects an empty frame instead of reading node columns. Pins: all three aliases, edge-only incl. endpoints, twelve properties across three aliases, parallel edges, no match; two hops and an absent edge property decline with parity. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
|
Fair challenge on "two projections". In the code, "two" was never a projection count: it meant the two node endpoints of the single hop, and the path already took any number of properties from either endpoint (an eight-property RETURN is served). The benchmark-shaped gap was real though: an edge-alias property (
What still declines by construction: anything beyond one hop (a different fast path's job) and anything with side channels (policy, WHERE, OPTIONAL). Local: fast-path + index + latency + lowering suites 548 + 1765 passed (the two cuDF t6 lowering failures are the dev-box libcublas environment, identical on master); ruff/guards/mypy clean, no new Any/ignore/cast. Real-GPU receipt for the new shapes follows on this PR. #2037/#2038/#2040 are restacked on this head. |
…alias marker The full path's chain output puts the edge alias' boolean marker first on the edge frame; the seeded projection now emits the same empty frame. The connected-bindings kernel parity test disables the seeded fast paths so the kernel stays the route under test. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
…w a column of their frame An edge alias named like an edge column (or a node alias named like a node column) keeps the full path's collision contract instead of the fast path inserting a marker over the column. Pins on pandas/polars/cuDF for edge and node collisions, projected or not, and the whole-row RETURN. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
… too The guard now sits after the shape checks and before the projection branches, so a colliding alias declines on every RETURN form. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
|
Scenario review (include/exclude, per your note; matrix in the private plan
Real-GPU at 37ada01: 766 passed / 0 failed; rerun on 7e3f983 in flight. #2037/#2038/#2040 restacked on this head. |
…endpoint bindings are not edge properties A random-shape differential (aliases, property subsets from every side, seed hit or miss, suffixes) found that projecting an edge's endpoint binding served the ids where the full path resolves an absent property; the fast path now declines those items. The latency contract gains the all-aliases shape against a two-lookup plain-frame floor. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
|
Quality pass per the review checklist, landed as aad106c:
Suites: lookup + typed-hop + indexed-bindings + engagement + latency-contract + index + alias-scoping = 825 passed locally; guards/mypy clean. #2037 ab3a32e / #2038 6312e18 / #2040 ab74105 restacked. Real-GPU run of aad106c in flight; CI running. |
|
Real-GPU receipt at aad106c (dgx-spark, RAPIDS 26.02 image, cudf 26.02.01 / polars 1.35.2): lookup + typed-hop + engagement + latency-contract + index + indexed-bindings + polars-gpu suites = 829 passed, 0 failed, 1 opt-in skip. |
…as categorical rows Round-002 amplification of the seeded fast paths found two dtype divergences against the full Cypher path: - an empty projection kept the node id column at int64 where the full path widens it to float64 (no-match seed, destination label mismatch) - a whole-row RETURN on pandas frames with a categorical column kept the neighbouring bool/int columns narrow where the full path widens them; the typed-hop path now declines those frames (polars/cuDF keep parity and stay on the fast path) Pins: two empty shapes in the two-alias parity matrix and a categorical matrix over pandas/polars/cudf x scan/indexed x four shapes. Co-Authored-By: Claude Fable 5.1 <[email protected]> Claude-Session: https://claude.ai/code/session_01QztW7jYsDd66e8rb8pJNQA
|
READY + MERGING (7db7968) — owner-authorized after the round-002 test-amplification pass. Round-002 (timeboxed 1–1.5 h, hypotheses not pre-constrained): a 90-cell differential matrix (30 shapes × pandas/polars/cuDF, default route vs fast paths disabled vs pandas oracle) over 13 hypothesis families (categorical / string / nullable dtypes, stale indexes,
Post-fix matrix: 0 DIFF / 81 parity / 9 both-raise (same exception class on both routes). Artifacts and probe outputs in the plan's Receipts at 7db7968
Restacked on this head with identical patch-ids: #2037 ff818e1, #2038 0664c3a (also drops a duplicated #2033 CHANGELOG bullet a union rebase had left), #2040 682c072. |
Stacked on #2031 (contract pins) → #2030 (dtype memo). Closes the two fast-path coverage gaps behind the SNB point-lookup regression (plan T2.10.2).
What changes
seeded_node_lookupfast path:MATCH (p {id}) RETURN p/RETURN p.a AS x, p.bresolves the seed through the resident node-id index, a resident node-property index (gfql_index_node_props), or one scalar scan, and projects the matched rows directly. Before, this shape ran the full two-pass chain over the node table with no index consulted (the alias on the node op disqualified the native single-op shortcut).RETURN m.id, p.firstName, the LDBC IS1 form), one row per matched edge, built from the seed and destination rows the reduction already holds. Before, it fell to the row pipeline's connected-bindings frame builder.chain_fast_paths._seed_node_rows): node-id index → property index → scan, with the ORIGINAL filter dict re-applied through the canonical filter (filter_by_dict/filter_by_dict_polars) so typed errors, temporal and mixed-type comparisons match the full path exactly. The seeded hop now also benefits from a property-indexed seed._indexed_kernel_admitsmirrors its seed admission and frontier/gather cost gates).Evidence (SNB SF0.1 fixture, local box, not published; pyg-bench sentinel #245): pandas IS1 official 19.1 → 4.0 ms, node-only lookup 13.7 → 1.2 ms; polars IS1 7.9 → 5.7 ms; cuDF IS1 78 → 39 ms; every basic shape now reports
fast_path servedon pandas, polars and cuDF. Contract pins intest_gfql_latency_contract.pyflip from strict-xfail to pass.Tests:
test_seeded_node_lookup_fastpath.py(lookup shapes × scan/indexed × 3 engines, independent oracle, property-index engagement with thenode_lookuptrace seam, out-of-shape declines, policy decline, duplicate-id rows, two-alias shapes incl. int/bool from both aliases, no-match dtypes, non-binding indexed seeds, bag multiplicity, multi-seed, hub seed over the frontier gate); cross-alias pins intest_seeded_typed_hop_fastpath.pyflip from decline to engage-with-parity; engagement map extended;test_use_policy_sparse_serves_dense_declinesmoved to a 2-hop shape. Fullgraphistry/tests/compute/gfqlsweep on pandas/polars/cuDF: 11,896 passed, 5 failures = pre-existing locallibcublas.so.12import errors (identical on master). Guards (ruff, mypy, comment density, type hygiene, Cypher surface) green.Found on the way: #2034 (full path multiplies rows for duplicate node ids and joins null ids), #2033 (polars native chain skips the seeded fast path).
🤖 Generated with Claude Code
https://claude.ai/code/session_01WwMmVFo44ADiRRj5cxh1i1