GFQL: track open native-Polars NIE surfaces + a per-engine capability/semantics doc matrix

Summary

Two related gaps to track:

Open NotImplementedError (NIE) surfaces in the native Polars GFQL engine — features that currently raise NotImplementedError on engine='polars' (and 'polars-gpu', which inherits them) and fall to "use engine='pandas'". They're honest declines (parity-or-NIE, never silent wrong answers), but they're coverage gaps to close where feasible.
Per-engine capability/semantics documentation — GFQL runs on four engines (pandas, cudf, polars, polars-gpu) with different coverage and (until the openCypher-conformance work in GFQL: conform to openCypher/SQL semantics — track, fix & document violations (3VL nulls, …) #1664 lands) different semantics. There is no user-facing matrix of "what each engine supports / declines / diverges on." We need one.

Context: found while building/extending the native polars engine. pandas is the differential oracle and the most complete; cudf mostly mirrors pandas (with the cross-engine divergences in #1663); polars is parity-or-NIE; polars-gpu (cudf_polars) inherits the polars engine's coverage. Recently made native (so NOT on this list): fixed multi-hop, forward/reverse to_fixed_point, undirected fixed multi-hop, forward/reverse min_hops>1 (in chain()), toFloat, collect/collect_distinct, WHERE … IN.

Open polars-engine NIE surfaces

Feasible to implement (port from the pandas path; rank ~MEDIUM)

include_zero_hop_seed — emit seed nodes at hop 0 (pandas compute/hop.py seed-union).
label_node_hops / label_edge_hops / label_seeds — emit hop-number columns. The min_hops path already computes per-edge hop labels internally; needs exact label-VALUE + NA-dtype parity (see GFQL: cuDF cross-engine result divergences (list-literal order, toString(float), min_hops seed hop-label, group_by Series-truthiness) #1663 Document API with pydoc #3 for the dtype contract — a prerequisite).
output_min_hops / output_max_hops output slicing — post-traversal hop-range mask; depends on the labels above.
unwind of a list COLUMN / expression (literal-list unwind is already native) — explode with null/empty-element semantics to prove vs pandas.
List scalar functions head / tail / reverse / range — element order/repr parity care.
Multi-entity binding projection (rows(binding_ops=…), edge entity-text) — n.x-prefixed multi-entity columns.
call() as a chain PREFIX (row-table → graph re-entry) — polars traversal can't currently consume a row-table input.
Direct hop(min_hops>1) — native in chain()/gfql() but NIE as a bare hop() (it needs pandas' separate un-labeled direct-hop node output + target_wave_front threading; validated this session that it silently diverges otherwise, e.g. drops a genuinely-reachable node).
Duplicate output column names — polars .select rejects dup names pandas tolerates.
tz-aware DateTimeValue/TimeValue predicates — localize/convert semantics (naive Datetime DateValue is already native).

Keep-NIE (genuinely blocked or a correct decline)

Undirected to_fixed_point and undirected min_hops>1 — need connected-components + 2-core seed retention (compute/hop.py:817-887); no vectorized polars primitive.
*_query / node query= — pandas df.query() eval syntax (pandas-specific).
Numeric-vs-string comparison — polars raises; correct decline (no silent coercion).
List-column scalar comparison (non-labels) — pandas compares the whole list; a contains-membership port would be wrong.
structured=False float/temporal/nested entity-text — pandas float repr diverges (1e+20); structured=True (default) already flattens any dtype.
QuantifierExpr (any/all/none/single), ListComprehension, Match/Fullmatch custom regex flags — HARD / narrow.

(Each has a file:line in the engine source; happy to expand any into its own task.)

Per-engine documentation deliverable

A user-facing capability + semantics matrix across pandas / cudf / polars / polars-gpu:

Which traversal/row/predicate/scalar/aggregation features are native, NIE (declines), or divergent per engine.
The openCypher semantics notes (3VL nulls, integer division, ORDER BY null position, NaN-vs-null) and where engines currently differ — cross-link GFQL: conform to openCypher/SQL semantics — track, fix & document violations (3VL nulls, …) #1664.
The cross-engine divergences (list-literal order, toString/properties float repr, NA-dtype) — cross-link GFQL: cuDF cross-engine result divergences (list-literal order, toString(float), min_hops seed hop-label, group_by Series-truthiness) #1663.
Guidance: when to pick which engine (this exists partially in the engine-selection docs — extend it with the support matrix).

This is the documentation half of the openCypher-conformance effort (#1664) plus the polars-engine coverage story; the matrix should be generated/maintained alongside the conformance ledger so it can't silently drift.

Related: #1664 (openCypher conformance), #1663 (cuDF cross-engine divergences).

fix(gfql): ne()/<> and IN on NULL follow openCypher/SQL 3-valued logi…

Registering a deliberate, now-TYPED divergence: `sum`/`avg` over `BOOLEAN`

Filing this against the per-engine capability/semantics matrix (item 2) so it lands there as a choice with a contract, not an oversight, per the owner's 2026-07-28 verdict on #1820. Implemented in #1982.

The extension

sum()/avg() over a BOOLEAN column is a type error in Cypher (Neo4j 5.26.26: "expected Float, Integer or Duration but was Boolean"; Kuzu rejects at bind time) and is served on every GFQL engine. It is a strict superset — it only accepts input Cypher rejects outright, so no Cypher-valid query changes meaning under it.

The contract the matrix should carry (values AND return types, all four engines)

sum(BOOLEAN)   -> INTEGER (int64)     count of true, nulls skipped; 0 over zero non-null
avg(BOOLEAN)   -> FLOAT   (float64)   true_count / non_null_count; NULL over zero non-null
min(BOOLEAN)   -> BOOLEAN             ordering false < true; NULL over zero non-null
max(BOOLEAN)   -> BOOLEAN             ordering false < true; NULL over zero non-null
count(BOOLEAN) -> INTEGER (int64)     non-null count

min/max are an ordering, not an AND/OR fold. The fold reading agrees on populated input but predicts the conventional empty identities (true/false) where every engine answers NULL.

Divergences this closed (all were cross-engine, all now conformed)

Measured on this repo's four-arm sweep, dtypes read natively (a to_pandas() round-trip turns a cuDF bool-with-nulls column into object and fabricates a divergence — worth noting for anyone else building a matrix):

divergence	engine	was	now
`sum(BOOLEAN)` return type	polars	`UInt32`	`Int64`
`count(<any type>)` return type	polars	`UInt32`	`Int64`
all-null `sum` substitution literal	polars	`Int32` (bare `pl.lit(0)`)	`Int64`
`count(DISTINCT ...)` return type	cuDF	`int32`	`int64`
`sum` over a group with no non-null values	cuDF	`NULL`	`0`

The last one is a wrong VALUE, not a wrong type, and it was not boolean-specific — cuDF answered NULL for BOOLEAN, Int64 and float64 alike, where Cypher says 0 and pandas/polars already said 0. It surfaced only because the verdict required exercising the cuDF arm, which had never been probed.

Engine-arm coverage of the new pins

arm	status
pandas	exercised
polars 1.42	exercised
cuDF 25.10	exercised on a real GPU (RTX 3080 Ti, cupy 13.6, compiled kernel verified)
polars-gpu	NOT covered — `cudf_polars` absent on the box; every param skips with the named reason. Needs a run on the RAPIDS 26.02 image.

Still open, and belongs on this matrix rather than in that PR

The 0-row ungrouped identity row carries no type evidence. avg/min/max over an empty result land on pandas object / polars Null instead of their contract dtypes. The values are correct and pinned on every engine (sum 0, avg/min/max NULL, count 0); the dtypes are not. This is not boolean-specific — avg over an empty INTEGER column loses its dtype identically — and closing it means plumbing the aggregate's contract dtype through the identity-row fill.

Related: cuDF's 0-row grouped schema types every aggregate output column with the input dtype (so avg over an empty boolean group reports bool), where pandas and polars keep the per-aggregate dtype.

Executable form of all of the above: graphistry/tests/compute/gfql/test_aggregate_type_contract.py; single source of the rule: graphistry/compute/gfql/agg_types.py.

Navigation Menu

Repository navigation

GFQL: track open native-Polars NIE surfaces + a per-engine capability/semantics doc matrix #1665

Summary

Open polars-engine NIE surfaces

Feasible to implement (port from the pandas path; rank ~MEDIUM)

Keep-NIE (genuinely blocked or a correct decline)

Per-engine documentation deliverable

Registering a deliberate, now-TYPED divergence: `sum`/`avg` over `BOOLEAN`

The extension

The contract the matrix should carry (values AND return types, all four engines)

Divergences this closed (all were cross-engine, all now conformed)

Engine-arm coverage of the new pins

Still open, and belongs on this matrix rather than in that PR

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

GFQL: track open native-Polars NIE surfaces + a per-engine capability/semantics doc matrix #1665

Description

Summary

Open polars-engine NIE surfaces

Feasible to implement (port from the pandas path; rank ~MEDIUM)

Keep-NIE (genuinely blocked or a correct decline)

Per-engine documentation deliverable

Activity

lmeyerov commented on Aug 20, 2026

Registering a deliberate, now-TYPED divergence: sum/avg over BOOLEAN

The extension

The contract the matrix should carry (values AND return types, all four engines)

Divergences this closed (all were cross-engine, all now conformed)

Engine-arm coverage of the new pins

Still open, and belongs on this matrix rather than in that PR

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Registering a deliberate, now-TYPED divergence: `sum`/`avg` over `BOOLEAN`