fix(gfql): pin sum/avg-over-BOOLEAN return types across engines (#1820 verdict) - #1982

Implements the verdict the owner issued on 2026-07-28 in #1820 — "adopt the rule, with three amendments — measured, not assumed" — including all three amendments. This is not a re-litigation of the design question; it is the work the verdict specifies.

sum()/avg() over BOOLEAN is adopted as a documented extension. Cypher rejects it ("expected Float, Integer or Duration but was Boolean"); GFQL serves it. It is a strict superset — it accepts only input Cypher rejects outright, so no Cypher-valid query changes meaning.

Fixes #1820

The three amendments

1. min/max are an ORDERING, not an AND/OR fold. min == AND / max == OR is a derivation from false < true, not a definition, and a logical fold gets the empty case backwards: the conventional identity of AND over zero elements is true and of OR over zero elements is false, while every engine answers NULL. Stated and pinned as "standard boolean ordering false < true, NULL when there are no non-null values" — the same order ORDER BY already gives booleans.

2. sum -> 0 over zero rows is conformance, not a compromise. Cypher's sum() returns 0 where SQL's returns NULL, and Cypher's avg() returns null. Described that way in both the module contract and the docs.

3. Dtypes are pinned, not just values — the gap that was still live.

sum(BOOLEAN)   -> INTEGER (int64)     count of true, nulls skipped; 0 over zero non-null
avg(BOOLEAN)   -> FLOAT   (float64)   true_count / non_null_count; NULL over zero non-null
min(BOOLEAN)   -> BOOLEAN             ordering false < true; NULL over zero non-null
max(BOOLEAN)   -> BOOLEAN             ordering false < true; NULL over zero non-null
count(BOOLEAN) -> INTEGER (int64)     non-null count

Before / after, measured on this box

Input pd.array([...], dtype="boolean"), whole-table and grouped (identical); dtypes read natively (a to_pandas() round-trip turns a cuDF bool-with-nulls column into object and fabricates a divergence).

Two findings beyond the reported polars UInt32, both from arms the verdict flagged as unverified or unexamined:

• cuDF answered a WRONG VALUE, not just a wrong type. Its grouped sum answers a group with no non-null values with NULL, where Cypher says 0 and pandas/polars already said 0 — on BOOLEAN, Int64 and float64 alike. This is the verdict's [null, null] -> sum 0 row, and it is exactly what "cuDF was not probed" was hiding.
• cuDF answered count(DISTINCT ...) with int32, where pandas answers int64.

Plus two polars ones the sweep turned up: the all-null substitution emitted a bare pl.lit(0), which is Int32, and the OLAP fast path's value_counts count plan — a fourth count formulation — was left at UInt32 after its group_by sibling was conformed, silently making two lanes that document themselves as interchangeable type-different.

Scoping: what widened, and the evidence

sum is boolean-scoped by construction, not by a guard. Measured on polars 1.42: sum over Int8/Int64/UInt8 already returns Int64, over Float64 returns Float64, over Duration returns Duration. Boolean is the only input whose sum is not already the contract type, so the sum half of the cast can only fire on a boolean column. polars_agg_result_cast declines Float64 and Duration explicitly, and a test pins those declines — widening a FLOAT or DURATION sum to Int64 would be a silent wrong answer.

count DID widen, deliberately. Polars answers count() with UInt32 for every input dtype; pandas and cuDF answer int64 for every input dtype; Cypher declares count INTEGER. Fixing it only for boolean input would have left the same divergence one query away and made the contract arbitrary. Evidence that the widening is correct and expected: the failure-set diff below shows the only tests it moved were the value_counts lane's own schema-equality pins, which compare that lane against a locally-built pl.len() twin — and those pass once the twin is built the way production now builds it.

Sites

Seven aggregate implementations, all conformed off the one contract module (agg_types.py):

1. pandas/cuDF row pipeline
2. native polars row pipeline
3. OLAP fast path — fused polars lane
4. OLAP fast path — value_counts low-cardinality count plan
5. OLAP fast path — eager polars twin (serves when the fused lane declines)
6. OLAP fast path — pandas/cuDF twin
7. the all-null substitution literal (both branches)

Receipts

Anti-vacuity — 30 RED at the merge-base. The new/strengthened pins were run at e4ad11c6b in a throwaway worktree: 30 failed, 323 passed, 0 errors. Green with the fix (353 passed).

Per-site mutation — 17 sites, 0 vacuous. Each site was reverted to its pre-fix form and the pin file re-run:

Three sites mutated to zero on the first pass and the pins were strengthened rather than accepted: the two all-null literals (their tests compared values only), and the fused lane's pl.len() (the value_counts plan wins for a pure count(*), so reaching it needs a count(*) sharing its RETURN with another aggregate). An eighth candidate site — an integer widen on the fast path's pandas count — could not be made to bite because cuDF's grouped count is already int64, so it was removed rather than kept as a vacuous no-op.

Failure SET comparison — graphistry/tests/compute, merge-base e4ad11c6b vs this branch, same box, same interpreter:

• pre-existing at both: 7 (5 cuDF index/lowering, 1 categorical searchany, 1 dask coercion — all unrelated)
• fixed by this PR: 0 (the divergences were unpinned, so nothing was red)
• regressions: 0 after the value_counts twin update (7 before it, all in that one file, all the schema-equality pins described above)

Guards — run inside the worktree: bin/ci_comment_density_guard.py rc=0 (no growth; 2 files now below baseline), ./bin/lint.sh rc=0, ./bin/typecheck.sh rc=0 (mypy clean, 333 files).

Engine arms actually exercised

The cuDF arm is the one the verdict called out as unverified; it is now exercised for real, and it is where the wrong-value bug came from.

Known gap left OPEN, deliberately

The 0-row ungrouped identity row carries no type evidence, so avg/min/max over an empty result land on pandas object / polars Null rather than their contract dtypes. The values are correct on every engine (sum 0, avg/min/max NULL, count 0) and are pinned; the dtypes are not, and the test says so in place rather than pinning today's behaviour. This is not boolean-specific — avg over an empty INTEGER column loses its dtype identically — and closing it means plumbing the aggregate's contract dtype through the identity-row fill. Registered on #1665 and #1664 rather than folded into this PR.

Same for cuDF's 0-row grouped schema, which types every aggregate column with the input dtype.

Registered

Commented on #1665 (per-engine capability/semantics matrix) and #1664 (openCypher conformance tracker) so the extension reads as a deliberate, typed divergence rather than an oversight.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AjbKuKheqDu78oapRT5AYm