Skip to content

GFQL/Cypher: admit carried-endpoint rebind via single-MATCH flatten (#1341) - #1351

Merged
lmeyerov merged 12 commits into
masterfrom
1341-rebind-secondary-alias
May 8, 2026
Merged

lmeyerov merged 12 commits into
masterfrom
1341-rebind-secondary-alias

Conversation

@lmeyerov

@lmeyerov lmeyerov commented May 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Closes Cypher MATCH after WITH rejects re-binding a carried secondary alias as a node variable in trailing MATCH #1341.
  • Adds an AST pre-pass flattener (graphistry/compute/gfql/cypher/reentry/flatten.py) that detects a narrow IC1-style rebind shape and rewrites the WITH stage away into a comma-separated single-MATCH so the existing two-endpoint shortestPath and shared-alias paths execute it directly.
  • Repurposes the prior failfast test to a positive admit test (with a no-back-edge empty-result companion) and adds a dedicated IC1 shortestPath((p)-[:KNOWS*1..3]-(friend)) regression with cuDF parity.

Approach

The flattener is intentionally narrow — admit only when:

  • exactly one prefix MATCH and one trailing MATCH
  • exactly one WITH stage that is a pure bare-alias carry (no DISTINCT, no aggregation, no <expr> AS <alias>, no WHERE/ORDER BY/SKIP/LIMIT)
  • no UNWIND (prefix or trailing), no CALL, no row sequence, no OPTIONAL on the trailing MATCH, no reentry WHEREs
  • every node alias bound by trailing patterns is among the carried set
  • every carried alias was bound by the prefix MATCH
  • every trailing pattern adds structural constraints (≥ 1 RelationshipPattern); a pure single-node re-reference like WITH a MATCH (a) RETURN a is left to the existing reentry path so the merged form does not introduce duplicate-alias bindings that downstream lowering rejects

The blanket reject at lowering.py (Cypher MATCH after WITH does not yet support re-binding a carried secondary alias as a node variable in the trailing MATCH) is preserved for residual non-admitted shapes.

Test plan

  • ./bin/ruff.sh graphistry/compute/gfql/cypher/reentry/flatten.py graphistry/compute/gfql/cypher/lowering.py graphistry/tests/compute/gfql/cypher/test_lowering.py
  • ./bin/mypy.sh graphistry/compute/gfql/cypher/reentry/flatten.py graphistry/compute/gfql/cypher/lowering.py
  • pytest graphistry/tests/compute/gfql/cypher/test_lowering.py — 825 passed
  • pytest graphistry/tests/compute/gfql/ — 1852 passed
  • pytest graphistry/tests/compute/ — 2670 passed
  • CI green
  • cuDF parity test passes on GPU runner

🤖 Generated with Claude Code

@lmeyerov
lmeyerov force-pushed the 1341-rebind-secondary-alias branch 2 times, most recently from 62ce645 to 4c2e869 Compare May 8, 2026 03:44
lmeyerov and others added 5 commits May 7, 2026 20:44
…1341)

Flatten the narrow IC1 reentry shape — single prefix MATCH, single pure
bare-alias WITH, single trailing MATCH whose patterns reference only
carried aliases AND add structural edges — into a comma-separated
single-MATCH clause that the existing two-endpoint shortestPath /
shared-alias paths handle directly. The blanket reject at
``lowering.py`` is preserved for residual non-admitted shapes.

Repurposed the prior failfast test to a positive admit test plus a
no-back-edge empty-result test, and added a dedicated IC1
``shortestPath((p)-[:KNOWS*1..3]-(friend))`` regression with cuDF parity.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…AL prefix (#1341)

Address review wave 1:

- Disqualify ``OPTIONAL MATCH`` prefix in ``flatten_carried_endpoint_rebind``
  to keep the merged single MATCH from over-broadening optional scope onto
  trailing-derived patterns.
- Comment the ``parser.py`` invariant that mirrors top-level WHERE into
  ``match_clauses[-1].where`` so the inline-WHERE check covers both shapes.
- Add ``test_flatten_carried_endpoint_rebind.py`` — 13 direct unit tests
  anchoring each early-return branch (DISTINCT, alias rename, ORDER BY,
  LIMIT, fresh trailing alias, OPTIONAL prefix/trailing, multi-WITH,
  reentry WHERE, no-edge trailing, no-reentry baseline) plus the IC1 +
  simple rebind admit cases.
- Note the ``MatchClause.pattern_alias_kinds`` default-tuple invariant in
  ``_normalized_kinds`` so a future ``PathPatternKind`` rebase doesn't
  silently mis-fill.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
)

Wave-2 review surfaced a scope-leak: when WITH dropped a prefix-bound
alias and post-WITH RETURN/ORDER BY referenced it, the existing reentry
path emitted a clean "Unknown Cypher alias" scope error, but flatten
silently re-admitted the merged single MATCH where the dropped alias
remained in scope. Tighten the flatten check from ``carried.issubset``
to ``carried == prefix_aliases`` so partial-carry shapes fall through to
the reentry path's correct rejection.

Also clarify the defensive ``trailing_match.where`` branch (parser routes
post-WITH WHEREs to ``reentry_wheres``; the check guards AST-built
inputs) and tighten the ``reentry_where_present`` test docstring.

Adds direct regression ``test_flatten_disqualifies_partial_carry_drops_prefix_alias``.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Rebased onto master after #1348 (#1343 aggregate fix) landed; the
narrow-shape flatten wiring at compile_cypher_query adds 12 lines to
lowering.py, so 8688 → 8700.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…HERE preservation (#1341)

Wave 3 cleanup:
- Comment recursion termination invariant at ``compile_cypher_query``
  (flatten output has empty ``reentry_matches``; the recursion cannot
  re-enter the flatten branch).
- Add ``test_flatten_preserves_prefix_match_where_on_merged_match``: asserts
  the prefix MatchClause's WhereClause object is preserved by-reference on
  the merged single MATCH (also covers the parser's top-level mirror onto
  ``match_clauses[-1].where``).
- Tighten ``_parse`` typing in the unit suite so the AST narrowing is
  explicit.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@lmeyerov
lmeyerov force-pushed the 1341-rebind-secondary-alias branch from 4c2e869 to b55787f Compare May 8, 2026 03:47
lmeyerov and others added 3 commits May 7, 2026 20:56
#1341)

Wave-4 review surfaced a relationship-variable scope leak: when prefix
bound a named relationship variable (``[r:R]``) and WITH dropped it,
post-WITH RETURN/ORDER BY references to ``r.<prop>`` would land in scope
under the merged single MATCH but produce a clean scope error on the
existing reentry path. Direct repro:

    MATCH (a:A)-[r:R]->(b:B) WITH a, b MATCH (b)-[:S]->(a) RETURN r.weight

Flatten silently admitted and returned ``r.weight``; the existing reentry
path correctly rejects.

Fix: extend ``_node_aliases`` with ``_all_pattern_aliases`` (nodes +
relationships) and key the equality check off the union, so dropping a
named relationship variable forces fall-through to the reentry-path
rejection.

Adds two regression tests:
- ``test_flatten_disqualifies_drops_relationship_variable``
- ``test_flatten_admits_when_relationship_variable_is_carried``

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Wave-5 review surfaced a defense-in-depth gap: ``MatchClause.pattern_aliases``
(path aliases like ``MATCH path = (a)-->(b)``) were not part of the
prefix-alias set, so a query like

    MATCH path = (a:A)-[:R]->(b:B) WITH a, b MATCH (b)-[:S]->(a) RETURN length(path)

passed the carry-set equality even though WITH dropped ``path``. No
user-visible leak today (the row pipeline rejects ``length(path)`` /
``nodes(path)`` etc.), but a future row-pipeline path-function expansion
would silently re-admit ``path`` into post-WITH scope through the merged
single MATCH.

Tighten the equality check to union path aliases from ``pattern_aliases``
into the prefix-alias set. Adds ``test_flatten_disqualifies_drops_prefix_path_alias``.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…racle (#1341)

Test-amplification skill round-001. Five wave reviews on PR #1351 found 4
bugs all of one taxonomy class: carry-equality boundary scope-leaks via
missing alias kinds and missing semantic disqualifications. Round-001
locks in the equivalence class so future drift is caught at test time
rather than via another review wave.

New tests:
- ``test_flatten_alias_kind_enumeration_locks_in_with_ast`` —
  introspection lock-in: enumerates ``typing.get_args(PatternElement)`` and
  ``dataclasses.fields(MatchClause)``; asserts ``_all_pattern_aliases``
  collects each variable-bearing pattern subtype's ``variable``, and that
  the ``MatchClause.pattern_aliases`` field is observed in the prefix-
  alias union via a parsed admit/disqualify probe.
- ``test_flatten_disqualifies_partial_carry_for_each_alias_kind`` —
  parametrized partial-carry sweep over (node, rel, path) alias kinds.
- ``test_flatten_disqualifies_optional_match_combinations`` —
  parametrized OPTIONAL matrix asserting only (F,F) admits.
- ``test_flatten_disqualifies_both_inline_wheres_via_built_ast`` —
  hand-built AST input that reaches the otherwise-unreachable defensive
  ``prefix.where AND trailing.where`` branch.
- ``test_string_cypher_flatten_admit_matches_hand_flattened_oracle_ic1`` /
  ``..._simple_rebind`` — admit-side correctness oracle: result equality
  between WITH-form (flatten path) and the manually-flattened single-
  MATCH form.
- ``test_string_cypher_executes_with_match_reentry_relationship_variable_carried_on_cudf`` —
  cuDF parity for the rel-var-carried admit case (mirrors the existing
  IC1 cuDF parity test).

Total: 6 new test functions / ~9 parametrized cases; no source changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
lmeyerov and others added 4 commits May 8, 2026 03:26
…1341)

Validation on dgx-spark via rapids 25.02 + 26.02 (docker/test-rapids-
official-matrix.sh) surfaced two pre-existing cuDF row-pipeline issues
that round-001 lock-in tests exposed:

- ``length(path)`` projection on shortestPath returns object/string dtype
  on cuDF instead of numeric (pandas baseline = float64). Both flatten-on
  and hand-flattened single-MATCH forms produce identical (broken) cuDF
  output → not introduced by the new flatten module. Filed as #1354.
  Workaround: ``test_string_cypher_executes_ic1_shortest_path_with_carried_
  endpoints_rebound_on_cudf`` now extracts friend IDs and int-coerces dist
  values for comparison instead of asserting full dict equality.

- Shared-alias multi-pattern with relationship-property projection
  segfaults (exit 139) on cuDF — affects both the WITH-form (flatten path)
  and the hand-flattened single-MATCH form. C-level crash; out of scope
  for this PR. Filed as #1355. ``test_string_cypher_executes_with_match_
  reentry_relationship_variable_carried_on_cudf`` is ``@pytest.mark.skip``'d
  with the issue reference; the pandas equivalent (``test_flatten_admits_
  when_relationship_variable_is_carried``) still anchors the wave-4 fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Wave-7 review found two real test-quality gaps in the round-001
amplification additions:

- ``test_flatten_alias_kind_enumeration_locks_in_with_ast`` was vacuous
  on existing ``PatternElement`` subtypes. Both ``NodePattern`` and
  ``RelationshipPattern`` have required fields without defaults
  (``labels``, ``properties``, ``direction``, ``types``); the prior
  ``kwargs={}; break; continue`` skip caused the assertion to never
  execute against either subtype. Replace with a ``typing.get_origin``-
  driven filler that supplies empty tuples for ``Tuple[...]`` fields and
  the first allowed value for ``Literal`` fields, plus a final
  ``asserted_subtype`` guard so the test fails loudly if no
  variable-bearing subtype is exercised. Verified via sabotage probe:
  monkey-patching ``_all_pattern_aliases`` to ignore RelationshipPattern
  now fails the test with ``must collect variable from
  RelationshipPattern (got set())``.

- ``test_string_cypher_executes_ic1_shortest_path_with_carried_
  endpoints_rebound_on_cudf`` int-coerced ``r["dist"]`` directly, which
  would ``ValueError`` on ``'1.0'`` strings or ``TypeError`` on
  ``None``/``NaN`` if cuDF behavior evolves. Replace with
  ``int(float(r["dist"]))`` two-step coercion plus an explicit
  ``len(rows) == 3`` shape check. Cite #1354 by issue number in the
  comment.

Polish: link #1355 by issue number in the rel-var-rel-prop cuDF skip
reason (was generic "filed as a separate cuDF parity issue"), and note
why ``skip`` rather than ``xfail`` is the correct mechanism for a
SIGSEGV failure mode.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…subtypes (#1341)

Wave-8 review SUGGESTION: the lock-in test's ``"variable" not in
type_hints: continue`` skip would let the test pass vacuously if a
future refactor moves ``variable`` off both ``NodePattern`` and
``RelationshipPattern`` (e.g. into a base mixin). Add an explicit
``{"NodePattern", "RelationshipPattern"}.issubset(asserted_subtype_names)``
final guard so the lock-in always anchors against today's concrete
subtypes by name.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…IP/non-bare tests (#1341)

Wave-9 review surfaced:

- IMPORTANT: missing CHANGELOG entry under ``## [Development] / ###
  Changed`` (PR converts a previously-rejected query shape into an
  admitted, executable shape; recent ``## [Development]`` peers all
  have entries). Added.

- Documentation drift in ``flatten.py`` module docstring claiming
  "fresh trailing aliases ... disqualify the pattern". Only fresh
  trailing *node* aliases disqualify; fresh trailing relationship and
  path aliases admit (they are legitimately in scope post-WITH).
  Tightened the docstring.

- Two parser-reachable disqualification branches lacked direct tests:
  ``WITH a, b SKIP 5`` and ``WITH a, b.id``. Added
  ``test_flatten_disqualifies_with_skip`` and
  ``test_flatten_disqualifies_non_bare_with_item``.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@lmeyerov
lmeyerov merged commit a075abd into master May 8, 2026
137 checks passed
@lmeyerov
lmeyerov deleted the 1341-rebind-secondary-alias branch May 8, 2026 20:58
lmeyerov added a commit that referenced this pull request May 9, 2026
* feat(cypher): cross-kind alias rebind guard at binder layer (#1357)

Centralize entity_kind alias-scope enforcement in FrontendBinder so a MATCH
pattern that re-uses an existing scalar/path/edge alias as a node variable
(or vice versa) is rejected at the binder layer with a structured
GFQLValidationError carrying existing_kind / new_kind / new_role context,
instead of being silently merged into a wrong-kind BoundVariable.

Previously a downstream re-entry compile-time check at
reentry/compiletime.py:274 caught the scalar→node case for one specific
WITH-prefix shape with the message string "Cypher MATCH after WITH
scalar-only prefix aliases cannot be reused as node variables". The new
binder-level guard fires earlier and covers all four cross-kind transitions
(node↔edge, edge↔node, path↔node, path↔edge) uniformly across MATCH and
re-entry MATCH paths. The downstream guard is preserved as defense-in-depth.

Updates two test_cycle_policy.py assertions and one test_lowering.py
assertion to match the new error surface (binder-level instead of re-entry
compile-time). The intent is unchanged — reject scalar→node rebind across
WITH — only the rejection site and message moved.

Surfaced by PR #1351 round-001 agent-03 binder-parity audit. This commit
ships the safe portion of #1357 (cross-kind dimension); full
strict_name_resolution flip at lowering.py:8400 is deferred — discovery in
plans/1357-binder-strict-name-resolution/research/ classified 75 failures
across multiple binder coverage gaps (namespaced functions, quantifiers,
list comprehensions, CALL/YIELD scope, post-WITH UNWIND traversal) that
need their own fixes first.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(cypher): cross-kind alias rebind matrix at binder + compile entrypoint (#1357)

14 tests pinning the four cross-kind transitions (node↔edge, edge↔node,
path↔node, path↔edge) plus same-kind admit cases. Exercises both
FrontendBinder().bind() directly and compile_cypher_query so the guard's
reach is locked at both layers.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(cypher): regression-pin baselines for future strict_name_resolution rollout (#1357)

13 tests pinning current loose-mode admits at the binder + compile entry
points for the five binder-coverage gaps the discovery flip surfaced
(namespaced builtins, quantifier predicates, post-WITH UNWIND traversal,
CALL/YIELD scope, and unresolved-alias rejections currently routed
through downstream guards).

When a follow-up PR closes a gap and is ready to flip strict mode, the
corresponding test here flips from "admits" to "raises". The
parameterized cases serve as the gap-by-gap readiness ledger.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(cypher): validator/runtime strict-mode parity baselines (#1357)

6 tests pinning where gfql_validate(strict=True) and compile_cypher_query
agree (cross-kind rebind — parity ships in this PR) vs diverge
(quantifier predicates, post-WITH UNWIND — known binder gaps blocking
the post-normalize strict flip).

Each divergence case becomes a parity case once the corresponding binder
gap is closed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(changelog): cross-kind alias rebind guard at binder layer (#1357)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test+docs(cypher): wave-1 review follow-ups for #1357

- test_binder_cross_kind_rebind.py: extend coverage with scalar-origin
  matrix (WITH expression projection, WITH literal projection — both
  routed through reentry MATCH path), dedicated reentry-MATCH cross-kind
  test (MATCH...WITH a MATCH ()-[a]->()), and a schema-vs-cross-kind
  precedence test pinning E301 (label miss) raises before E204 when
  both apply on the same MATCH.
- test_binder_strict_compile_baseline.py: tighten exception assertions
  from broad Exception+substring to GFQLValidationError + context["value"]
  / structured str(...) — survives a future strict-mode move from E108
  to E204 without going vacuous.
- CHANGELOG.md: clarify wording — the guard covers all three cross-kind
  pairs (node↔edge, node↔scalar, edge↔scalar), where path aliases bind
  as entity_kind="scalar" alongside other scalar carriers (UNWIND,
  WITH-expr projection, CALL/YIELD).

NOTE for follow-ups: UNWIND-then-MATCH cases are NOT reliably rejected
today — _bind_graph_sequence binds ast.matches before ast.unwinds, so
MATCH (x) binds x as a fresh node before UNWIND silently overwrites
entity_kind. Tracked in #1371 P1.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(cypher): cover graph-query cross-kind binder paths

* GFQL: enforce integer division for literal cypher row arithmetic

* GFQL: satisfy mypy for integer-division scalar path

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cypher MATCH after WITH rejects re-binding a carried secondary alias as a node variable in trailing MATCH

1 participant