Repository navigation
fix(gfql): explicit polars-gpu runs the Cypher OLAP fast paths on GPU or raises - #1979
Merged
Merged
Conversation
… or raises The fast-path call sites pinned the lazy execution target to CPU whatever engine the caller asked for, and the connected-match-join two-star arms were never wrapped in a target at all. Every fast-path-served OLAP shape therefore collected on CPU polars under an `engine='polars-gpu'` label -- the exact mislabelling `lazy._engine_for`'s `raise_on_fail=True` NO-CHEATING contract exists to prevent, and undetectable downstream. One shared seam, `_run_fast_path_on_requested_target`, now runs every fast-path arm on the requested engine's target, so the five lazy collect sites in `gfql_fast_paths` cannot drift apart. A plan node cudf-polars cannot execute surfaces as `NotImplementedError` and is treated as a fast-path DECLINE; the generic route -- itself GPU-or-raise -- answers, so nothing is quietly served on CPU. `polars`, `auto`, `pandas` and `cudf` keep the CPU target, and an NIE there is still a real error rather than a decline. Fixes #1824 Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01AjbKuKheqDu78oapRT5AYm
Contributor
Author
|
Second failure-set comparison, added after the PR body was written — the full
Failure SETS identical in both directions — zero new, zero accidentally fixed. All 7 are pre-existing and unrelated to this change: The pass-count delta (+18) is the new pins in this PR. |
This was referenced Sep 12, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1824
The rule being implemented
Owner ruling: auto may switch modes; an explicit engine selection may not. An explicitly requested engine either runs as requested or raises — it is never silently run somewhere else and reported as the requested engine.
What was broken (verified on current master,
af494becc)The issue's line references have moved. Re-read against master:
Engine.py:33POLARS_ENGINES = (Engine.POLARS, Engine.POLARS_GPU)graphistry/Engine.py:40lazy/__init__.py:186-198NO-CHEATING contractgraphistry/compute/gfql/lazy/__init__.py:233-255(_engine_for), error text at:258-266gfql_fast_paths.py:1688/1868/2036requested_engine = resolve_engine(...)graphistry/compute/gfql_fast_paths.py:2371 / 3039 / 3325gfql_fast_paths.py:1070/1449/1519/1713/1907if requested_engine in POLARS_ENGINES:graphistry/compute/gfql_fast_paths.py:1077 / 1463 / 1540 / 1621 / 1699gfql_fast_paths.py:871,878bare.collect()gfql.lazy.collect; the module has zero bare.collect()calls today (the one grep hit at:2112is inside a comment)So the collect-routing half of #1824 had landed — but
gfql.lazy.collectpicks its engine fromactive_target(), and the fast-path routes never set that target to GPU:gfql_unified.py:1373was a function named_fast_path_execution_target_ignoring_requested_enginethat returnedExecutionTarget.CPUunconditionally ("Not GPU until every fast-path arm is GPU-or-decline");_apply_connected_match_join's two-star arms (_connected_join_two_star_fast_grouped_count,_connected_join_two_star_fast_rows, and the fused lane behind them) were a second call site that was never wrapped in a target at all.Result: every fast-path-served OLAP shape collected on CPU polars while the caller had asked for — and any benchmark would label — GPU.
AUTO can now reach POLARS_GPU — the issue's premise is partially refuted
Engine.py:34still says "Explicit opt-in only — AUTO never selects it", andresolve_enginenever returnsPOLARS_GPUforAUTO. Butgfql_unified.py:2298/2323/2426(the owner-directed cuDF arm, 2026-08-02) re-entersgfql()withengine=_AUTO_CUDF_ROUTE_ENGINE == 'polars-gpu'when every bound frame is cuDF and the GPU target probes usable. So a resolvedPOLARS_GPUno longer proves the user typed it.Testing the resolved engine is still exactly right, and still exactly the owner's rule: that arm wraps its recursion in
except NotImplementedError -> legacy CUDF path, so the mode switch happens in the AUTO layer (allowed) while the inner run stays GPU-or-raise (required). Values are unchanged either way — see the evidence below.The fix
One shared seam rather than per-site edits, so the ~5 lazy collect sites cannot drift apart:
_fast_path_execution_target(engine)— GPU iff the requested engine ispolars-gpu, CPU otherwise._run_fast_path_on_requested_target(engine, run)— runs a fast-path arm under that target and returns(result, reason). On the GPU target a non-GPU-executable plan surfaces asNotImplementedError(vialazy._gpu_raise) and becomes a decline, so the generic route — itself GPU-or-raise — answers. On the CPU target an NIE stays a real error.Both call sites now go through it:
_try_fast(same_path / row_pipeline arms) and_apply_connected_match_join's two two-star arms.Evidence
Measured on a real GPU (RTX 3080 Ti, cudf 25.10, cudf-polars-cu12 25.10 + polars 1.32.3), on the fused two-star lane — the graph-benchmark q7 anatomy:
engine='polars-gpu'engine='polars-gpu'engine='polars'/'auto'Values identical in every cell.
Red at merge-base: the new pins were re-run at
af494beccwith one shim line (_fast_path_execution_target = _fast_path_execution_target_ignoring_requested_engine) and everything else byte-identical — 9 failed / 26 passed with the GPU stack, 8 failed / 26 passed / 1 skipped on a CPU-only box (the real-GPU pin skips). Green here: 35 passed (GPU), 34 passed + 1 skipped (CPU-only).Per-site mutation (all non-vacuous): target decision forced back to CPU -> 9 red;
_try_fastsite unwrapped -> 2 red; connected-join grouped-count arm unwrapped -> 3 red; connected-join rows arm unwrapped -> 2 red.Failure-set comparison:
bin/test-polars.sh(serial) at merge-base and at this head, same box, same env — identical failure SETS in both directions (5 pre-existing, unrelated: 3 cuDF degree-consult, 1 viz categorical, 1 dask coercion). 7304 passed here vs 7286 at master (the delta is the new pins).polars/autounchanged: covered by the identical failure sets above; by the CPU-target and NIE-is-a-real-error pins parametrized overpolarsandauto; and by an explicit A/B ofengine='auto'on a cuDF graph (the arm that routes through POLARS_GPU) againstcudfandpandas— byte-identical results at master and at this head.bin/ci_comment_density_guard.py,bin/lint.sh,bin/typecheck.shall rc=0. Correctness only; no performance measurements were run. Expectpolars-gputo get slower on these shapes — the old numbers were CPU numbers wearing a GPU label.🤖 Generated with Claude Code
https://claude.ai/code/session_01AjbKuKheqDu78oapRT5AYm