Skip to content

Cypher/GFQL cannot lower MATCH + OPTIONAL MATCH reply-author flag queries after a connected MATCH #996

Closed
#1061
Closed
Cypher/GFQL cannot lower MATCH + OPTIONAL MATCH reply-author flag queries after a connected MATCH#996
#1061
Labels
gfqlp2

Description

@lmeyerov
lmeyerov
opened on Mar 31, 2026

Status

Still open as of April 2, 2026.

This remains the active interactive-short-7 / OPTIONAL MATCH lane.

Related landed work:

  • #1016 fixed the separate #880 open-range bindings replay bug.
  • #1018 added follow-on amplification and DRY cleanup around that same #880 replay boundary.
  • The #880 residual bindings-table lane is now considered done and should not be treated as remaining scope for this issue.

Summary

Direct Cypher execution currently rejects MATCH ... OPTIONAL MATCH ... RETURN CASE ... reply-author flag queries with:

[unsupported-cypher-query] Only node-only pre-binding MATCH clauses are supported before the final connected MATCH in this phase

This is now a measured benchmark blocker from pyg-bench for LDBC SNB Interactive interactive-short-7 / IS7.

Repro Shape

MATCH (m:Message {id: $messageId })<-[:REPLY_OF]-(c:Comment)-[:HAS_CREATOR]->(p:Person)
OPTIONAL MATCH (m)-[:HAS_CREATOR]->(a:Person)-[r:KNOWS]-(p)
RETURN c.id AS commentId,
    c.content AS commentContent,
    c.creationDate AS commentCreationDate,
    p.id AS replyAuthorId,
    p.firstName AS replyAuthorFirstName,
    p.lastName AS replyAuthorLastName,
    CASE r
        WHEN null THEN false
        ELSE true
    END AS replyAuthorKnowsOriginalMessageAuthor
ORDER BY commentCreationDate DESC, replyAuthorId

Actual

Compiler/runtime rejects the query before execution with:

[unsupported-cypher-query] Only node-only pre-binding MATCH clauses are supported before the final connected MATCH in this phase

Expected

The query should lower and execute, preserving row semantics for the connected reply rows while null-extending the optional KNOWS binding for the boolean flag.

Benchmark Impact

  • blocks direct Cypher coverage for SNB Interactive interactive-short-7
  • this is the first measured query in our translation queue that clearly requires OPTIONAL MATCH + CASE semantics rather than only the already-known row/materialization gaps

Relationship To Existing Issues

  • likely related to the broader row-carrier / seeded-row architecture work in #989
  • distinct from #981, which covers multi-binding projection problems after lowering/execution begins
  • distinct from #994, which is an undirected peer-binding wrong-answer bug on a query that already executes
  • distinct from #880, which covered the residual bindings-table / replay lane and is now closed after #1016 and #1018

Activity

  1. lmeyerov
    added
    p2
    gfql
    p3
    and removed
    p2
    p3
    on Apr 1, 2026

  2. lmeyerov commented on Apr 1, 2026

    @lmeyerov
    lmeyerov
    on Apr 1, 2026
    ContributorAuthor

    Benchmark update from pyg-bench on 2026-04-01 after rerunning interactive-short-7 / message-replies against isolated origin/master e0396cbd3 (v0.53.16).

    Artifact:

    • results/runs/dgx-spark-snb-interactive-is7-origin-master-e0396cbd3-reuse-r5/

    Current measured result:

    • GFQL: partial via workaround under #880
    • direct Cypher: primitive_gap under #996

    Direct Cypher note from the probe:

    • Direct Cypher lowering does not yet support the measured MATCH + OPTIONAL MATCH row-seeded shape used by the reply-author flag query.

    So #996 remains the correct active issue for the current IS7 lane on latest measured master.

  3. lmeyerov
    added 2 commits that reference this issue on Apr 2, 2026

    feat(cypher): support connected MATCH + OPTIONAL MATCH (#996)

    89b5787

    feat(cypher): support connected MATCH + OPTIONAL MATCH (#996) (#1021)

    14dafaf

  4. lmeyerov commented on Apr 2, 2026

    @lmeyerov
    lmeyerov
    on Apr 2, 2026
    ContributorAuthor

    Addressed in #1021 (squash-merged).

    What landed:

    • Connected MATCH ... OPTIONAL MATCH ... RETURN queries now compile and execute. The compiler lowers each clause independently, left-outer-joins binding-row tables on shared node aliases, and delegates RETURN / ORDER BY / SKIP / LIMIT to the standard row pipeline.
    • 19 regression tests covering expression breadth, join edge cases, and post-projection ops.
    • GPU-validated on DGX with RAPIDS 26.02 (886 passed across full GFQL surface).

    Remaining scope (out of this PR):

    • WHERE clauses on OPTIONAL MATCH results (e.g., WHERE c:C) — still rejected
    • Multiple OPTIONAL MATCH clauses
    • 3+ MATCH clauses with mixed OPTIONAL
    • OPTIONAL MATCH with UNWIND/WITH/SKIP/LIMIT between clauses

    The IS7 query shape (MATCH connected OPTIONAL MATCH ... RETURN ... CASE ... ORDER BY) is now supported. The full IS7 query with property predicates and labels needs the graph to have the right label/property columns but the compiler no longer rejects the shape.

  5. lmeyerov commented on Apr 2, 2026

    @lmeyerov
    lmeyerov
    on Apr 2, 2026
    ContributorAuthor

    #996 is now the umbrella issue for OPTIONAL MATCH support.

    Landed:

    Remaining sub-issues:

  6. lmeyerov commented on Apr 3, 2026

    @lmeyerov
    lmeyerov
    on Apr 3, 2026
    ContributorAuthor

    Benchmark follow-up from pyg-bench on current origin/master cd7c7bc23ca26a502d2c9f02419f7cc567f8110a after #1021 / the current #996 landing:

    Pinned launcher artifact:

    • results/runs/dgx-spark-snb-interactive-is7-cypher-origin-master-cd7c7bc23-no-warmup-r1/

    Command:

    • uv run python scripts/run_dgx_spark_suite.py --suite snb-interactive --config configs/suites/snb-interactive-is7-cypher-conformance-sf1-no-warmup.yaml --output-dir results/runs/dgx-spark-snb-interactive-is7-cypher-origin-master-cd7c7bc23-no-warmup-r1 --graphistry-repo-path /tmp/pygraphistry-origin-master-cd7c7bc23 --host-uv

    What changed:

    • this no longer fails in the old direct-Cypher lowering/compiler bucket
    • fixture load completes on the narrow core_fast path
    • direct Cypher probe actually starts on the official interactive-short-7 / message-replies lane

    What still happens on sf1:

    • remote launcher returns 137
    • dgx.execute_remote_command duration_ms = 187901.489
    • snb.load_dataset_fixture duration_ms = 13030.692

    I then reran the same staged remote directory directly under /usr/bin/time -v using the existing .venv/bin/python to separate “launcher problem” from runtime behavior. That follow-up was also killed:

    • terminated by signal 9
    • elapsed wall time 3:28.95
    • max RSS 7811192 kbytes
    • shell-reported EXIT_CODE=137

    So the benchmark read is now:

    • #996 did move IS7 past the old lowering rejection
    • but official IS7 on current sf1 is not benchmark-closed yet; the direct-Cypher lane still dies during execution with a runtime/perf/memory kill rather than a compiler error

    I am not filing a separate issue yet because #1030 is the next obvious OPTIONAL MATCH follow-on and may materially change this lane again. If it lands and the same runtime kill remains, that would justify splitting a dedicated execution/perf issue from the old compiler umbrella.

  7. lmeyerov commented on Apr 4, 2026

    @lmeyerov
    lmeyerov
    on Apr 4, 2026
    ContributorAuthor

    Update: #1030 merged (closes #1024 + #1025).

    Now landed:

    Remaining:

  8. lmeyerov commented on Apr 4, 2026

    @lmeyerov
    lmeyerov
    on Apr 4, 2026
    ContributorAuthor

    #1026 investigation started — WIP PR #1044.

    Two sub-issues identified:

    1. Simple WITH + OPTIONAL MATCH gives wrong results (inner join, no null-fill) — bug on current master
    2. Multi-alias WITH + OPTIONAL MATCH fails to compile — TCK match-where6-5

    Root cause: _compile_bounded_reentry_query() doesn't inject OPTIONAL semantics into the suffix compilation. Plan at plans/issue-1026-optional-match-with-stages/plan.md.

  9. lmeyerov commented on Apr 4, 2026

    @lmeyerov
    lmeyerov
    on Apr 4, 2026
    ContributorAuthor

    #1026 merged via #1044. Single-alias WITH + OPTIONAL MATCH now null-fills correctly. Multi-alias WITH tracked in #1047.

  10. lmeyerov
    added 8 commits that reference this issue on Apr 5, 2026

    feat(cypher): stub — OPTIONAL MATCH + CASE execution for interactive-…

    e2c7e48

    feat(cypher): stub — OPTIONAL MATCH + CASE execution for interactive-…

    9403bad

    fix(cypher): CASE x WHEN null correctly matches null via __cypher_cas…

    8bb0a2d

    test(cypher): add DGX smoke test for #996 OPTIONAL MATCH + CASE null

    9296938

    test(cypher): amplify #996 CASE WHEN null + OPTIONAL MATCH boundaries…

    f40a8d5

    fix(changelog): move #996 entries from released 0.54.0 to [Development]

    f7b6982

    test(cypher): add cudf path to #996 DGX smoke test

    898a352

    feat(cypher): OPTIONAL MATCH + CASE execution for interactive-short-7 (…

    b7843b7
  11. lmeyerov
    closed this as completedin #1061on Apr 5, 2026

  12. lmeyerov commented on Apr 5, 2026

    @lmeyerov
    lmeyerov
    on Apr 5, 2026
    ContributorAuthor

    Fixed in #1061 (merged). Root cause: __cypher_case_eq__ used pd.Series == None which always returns False in pandas; fixed by returning the null-mask when either operand is a scalar null literal. Also fixed a cudf incompatibility in _apply_connected_optional_match (marker.apply(lambda) → marker.notna()). 12 new regression tests added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    gfqlp2

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions