Skip to content

GFQL: a DANGLING DESTINATION is matched by pandas/cuDF and dropped by polars — silent cross-engine divergence #1808

Description

@lmeyerov

Minimal repro

import pandas as pd, polars as pl, graphistry
nodes = pd.DataFrame({"id": [0, 1, 2]})
q = "MATCH (a)-[]->(b) RETURN count(*) AS c"

# edge (1 -> 9): destination 9 is NOT in the node table
edges = pd.DataFrame({"s": [0, 1], "d": [1, 9]})
case edges pandas cuDF polars
dangling source (9 -> 2) 2 1 1 1
dangling destination (1 -> 9) 2 2 2 1
both 3 2 2 1

A dangling source is dropped by all three engines. A dangling destination is dropped only by polars — pandas and cuDF match it and bind b to an endpoint that is not a node.

Which is right

MATCH (a)-[]->(b) binds b to a node. If the destination id has no row in the node table there is nothing to bind, so the pattern must not match. polars is correct; pandas and cuDF are wrong — and pandas is this repo's oracle, so the cross-engine parity suites are calibrated against the wrong answer for this shape.

It is also internally inconsistent: the same engine drops a dangling source and keeps a dangling destination, so it is an asymmetry in the endpoint gate rather than a deliberate "edges are authoritative" policy.

Scope

No error, no warning — just a different count. Found while auditing merged PRs #1782/#1783/#1784/#1790/#1792/#1793 with a pandas-vs-polars differential harness; it reproduces identically on 84be35fb (before that stack) and on 233b64c8, so it is pre-existing and not caused by any of them. It reproduces at every graph size tried and on every seed.

Why it has not been caught

The polars parity suites compare against the pandas oracle, so a shape where pandas is the wrong one shows up as "polars diverges" and is easy to read as a polars gap. Nothing in the corpus builds a graph with a dangling destination on purpose.

Suggested fix direction

Make the pandas/cuDF endpoint gate symmetric — require BOTH endpoints to resolve to node rows — then re-baseline any parity fixture that silently encoded the old count. Worth pinning with an explicit dangling-endpoint fixture on all four engines rather than relying on random fuzz to regenerate it.

Activity

  1. lmeyerov commented on Aug 13, 2026

    @lmeyerov
    ContributorAuthor

    Canvas disposition (release 2026-08): CONFIRMED reproducing at the #1873 stack head (pandas c=2 vs polars c=1 on dangling destination — the exact filed asymmetry). polars is the Cypher-correct side. Deferred to next cycle with an executable pin: test_known_cross_engine_divergences.py::test_1808_dangling_destination_pandas_matches_polars (strict xfail, lands via #1878) — the fix (symmetric pandas/cuDF endpoint gate + CSR-path parity + parity-fixture re-baselining) touches hop core and is too broad for a release-eve patch on degenerate input. Flipping the xfail is the fix PR's acceptance test.

  2. lmeyerov commented on Aug 14, 2026

    @lmeyerov
    ContributorAuthor

    Round-002 update: this issue's premise is now PARTIALLY STALE at the #1883 head — the polars ROW path matches pandas (keeps the dangling destination, the side this issue argues is wrong); only the polars AGGREGATE path retains the Cypher-correct drop, so polars now disagrees with itself (projection 2 vs count 1). Absorbed into the endpoint-closure contract design issue, which fixes all five surfaces to one rule.

  3. added a commit that references this issue on Aug 16, 2026
  4. lmeyerov commented on Aug 18, 2026

    @lmeyerov
    ContributorAuthor

    Verified fixed on master e6625ed28 (post correctness-stack merge) — recommending close after review.

    Repro: nodes {0,1,2}, MATCH (a)-[]->(b) RETURN count(*) AS c with an edge whose source, destination, or both dangle (endpoint id not present in the nodes frame). Hand-computed: both endpoints must resolve to node rows, so each case has exactly 1 matching edge → c=1.

    Output on e6625ed28 (pandas 2.3.3 / polars 1.42.0 / cudf 25.10):

    {'case': 'dangling_source', 'expected': 1, 'pandas': 1, 'polars': 1, 'cudf': 1}
    {'case': 'dangling_dest',   'expected': 1, 'pandas': 1, 'polars': 1, 'cudf': 1}
    {'case': 'both',            'expected': 1, 'pandas': 1, 'polars': 1, 'cudf': 1}
    

    The filed divergence (pandas/cuDF matching the dangling-destination edge → 2 while polars dropped it) is gone; all three engines converged on the endpoint-closure answer, not on the old pandas over-match.

  5. lmeyerov commented on Aug 18, 2026

    @lmeyerov
    ContributorAuthor

    Closing: verified fixed on master e6625ed28 by the 2026-08 stack — empirical repro + hand-computed oracle in the verification comment above. Residual sub-items, where any, are tracked in the successor issues named there (#1916, #1908, #1906, #1934-#1938).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions