refactor(gfql/cypher): switch to Earley parser; retire LALR pattern-shape cruft (#1031 slice 1) - #1217

…hape cruft (#1031 slice 1)

Per tech-lead direction (1-day timebox) following an LALR(1) state-collapse
on ``<comparable> AND <pattern>`` shapes that could not be resolved by
grammar restructure.

Earley parser swap eliminates four LALR-induced workarounds:

1. The 3 dedicated pattern-shape ``where_clause`` grammar alternatives
   (``where_pattern_and_expr_clause`` / ``expr_and_where_pattern_clause``
   / ``where_pattern_only_clause``) — collapse to the unified ``expr``
   route via a new ``WHERE_PATTERN -> pattern_atom`` leaf in ``?primary``.
2. ``_canonicalize_where_single_pattern_and_expr`` — regex source-rewrite
   that reordered ``expr AND pattern AND expr`` to ``pattern AND <rest>``
   so the LALR grammar could match.  Earley accepts patterns at any
   AND-spine position natively.
3. ``_mixed_where_pattern_expr_error`` — pre-flight rejector for shapes
   LALR could parse but the IR couldn't represent.  Replaced with a
   structural lift step that inspects the parsed boolean tree.
4. The ``except LarkError`` retry block in ``parse_cypher`` that fed
   the canonicalize-then-reparse path.

What this PR adds:

- ``BooleanExpr.op`` literal extended with ``"pattern"``; new
  ``BooleanExpr.pattern: Optional[Tuple[PatternElement, ...]]`` field.
- Parser ``pattern_atom`` transformer: wraps ``WHERE_PATTERN`` token as
  ``BooleanExpr(op="pattern", pattern=parsed)``; reuses the existing
  ``_parse_where_pattern_predicate_text`` helper.
- ``_split_top_level_and_pattern_leaves`` walker + ``_rebuild_and_tree``
  helper + ``_build_where_with_pattern_lift`` that extracts pattern
  leaves from ``expr_tree`` into ``WhereClause.predicates`` as
  ``WherePatternPredicate`` entries before lowering.
- Parser swap: ``Lark(parser="earley")`` for both ``_parser`` and
  ``_pattern_parser``.

What this PR retires:

- 3 grammar alternatives, 3 transformer methods, 1 helper
  (``_mixed_where_clause``), 2 pre-flight functions, 3 derived regex
  constants (``_WHERE_PATTERN_SEQUENCE_RE``, ``_WHERE_PATTERN_THEN_EXPR_RE``,
  ``_WHERE_EXPR_THEN_PATTERN_RE``), 2 helper utilities
  (``_WHERE_CLAUSE_BODY_RE``, ``_BOOLEAN_KEYWORD_RE``,
  ``_line_and_column_from_offset``), and the ``parse_cypher``
  ``except LarkError`` retry block.

Net diff: +204 / -144.

Strict-improvement consequences (Earley accepts what LALR rejected):

- ``WHERE expr OR expr`` now parses as a structured ``or`` tree.
- ``WHERE expr AND (expr OR expr)`` (Shape B) now parses as
  ``and(left, or(...))``.
- ``WHERE n:Label AND n.prop = X`` now routes through structured
  ``where_predicates`` (was raw expr); label narrowing applies.
- ``WHERE prop OP 'string'`` mixed with label predicates now hits
  ``ComparisonPredicate`` for strings — separately fixed in the
  preceding commit by extending EQ's string-accepting overrides
  to NE/GT/LT/GE/LE.

#1031 milestones in this PR:

- ✅ Retires ``_canonicalize_where_single_pattern_and_expr``.
- ✅ Removes one of three ``split_top_level_and`` callers (parser side).
  Remaining caller: ``predicate_pushdown._split_conjuncts``, retired
  separately under #1200.  Together this brings ``expr_split.py``
  deletion within reach.
- ⏭️  ``NOT pattern`` lowering (anti-semi-join) — slice 2 territory;
  emits explicit ``unsupported`` error today.
- ⏭️  Multi-positive patterns — slice 3 territory; rejected at lift step.
- ⏭️  ``OR``/``XOR`` around patterns — slice 4 territory; rejected at lift step.

Tests updated:

- 7 tests adjusted to match Earley's strict-improvement routing
  (``test_or_where_not_yet_supported``, conformance Shape B parser +
  binder, conformance Shape F parser + binder, label-narrowing mixed
  case, logical planner WITH-stage attachment).
- ``test_parse_supports_where_pattern_predicate_and_expr_mix`` paren-
  preservation case updated to expect ``boolean_expr_to_text``-
  reconstructed paren-stripped form (semantically equivalent).

1539 / 1539 GFQL tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

…ngs entries

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

…#1217)

Pre-#1217: LALR rejected OR / NOT inside WHERE bodies, so disjunctive
and negation shapes had zero native end-to-end test coverage.  Earley
accepts them and the runtime evaluates via pandas, but until now the
correctness was implicit — only the (separate-repo) tck-gfql harness
exercised these paths via xfail-marked scenarios.

Adds 5 native execution tests in test_lowering.py:

- ``test_string_cypher_executes_disjunctive_property_predicate_returns_union``
  mirrors TCK match-where1-10: ``WHERE n.p1 = 12 OR n.p2 = 13`` returns
  the union of matches (a, b), not just one.
- ``test_string_cypher_executes_disjunctive_same_alias_property_predicate``
  ``WHERE n.p1 = 12 OR n.p1 = 99`` — same alias on both sides.
- ``test_string_cypher_executes_negation_property_predicate_returns_complement``
  ``WHERE NOT n.p1 = 12`` — locks pandas's NaN-NOT-NaN semantics
  (NaN is falsy under NOT, so rows with null p1 are excluded).
- ``test_string_cypher_executes_disjunctive_then_conjunction``
  ``WHERE (n.p1 = 12 OR n.p2 = 13) AND n.id = 'a'`` — mixed OR-AND
  tree narrowing.
- ``test_string_cypher_executes_disjunction_returns_correct_count_with_more_rows``
  larger fixture (6 rows) confirming OR doesn't silently union too many
  or too few rows.

Pairs with the tck-gfql sibling branch ``issue-1031-grammar-mixed-where-pattern-expr``
which registers ``match-where1-10`` as ``success_matches_expected`` in
the xfail-contract (kept as ``xfail`` rather than promoted to
``supported`` until broader OR-disjunction conformance is validated).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

IMPORTANT findings from multi-dimension review:

1. Dead-code in OR test (test_lowering.py:3199-3205) — the
   ``if False else`` made the first comprehension permanently dead;
   the fallback branch then asserted on either of two contradictory
   shapes.  Replaced with a single deterministic assertion: 5 rows
   expected, every kept row carries either ``p1: 1`` or ``p2: 2`` in
   its rendered form.
2. EQ duplicated the new mixin's overrides (comparison.py:245-263).
   The mixin docstring already invited the retrofit ("EQ predates this
   mixin with an inline override").  EQ now uses
   ``_StringAllowingComparisonMixin`` for its single-source contract.

SUGGESTIONS deferred (documented in plan):
- NaN-NOT semantics test name (locks pandas-not-Cypher behavior;
  acknowledged in test docstring + #1219 link).
- ``_rebuild_and_tree`` invents source-spans across non-adjacent
  residuals (cosmetic, error-reporting only).
- No deeply-nested pattern-under-multi-level-boolean test
  (``_has_pattern_descendant`` handles correctly by inspection).
- PEP-8 spacing nit after constant deletes.
- Defensive ``_split_top_level_and_pattern_leaves`` bad-flag drop on
  malformed AND nodes (#1214 invariant prevents the case).

1680 GFQL + predicates tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

…der Earley ambiguity (#1031)

Earley with ambiguous grammar parsed `MATCH ... WITH ... WHERE ... RETURN`
two ways: (a) `with_stage[with_clause + with_where_clause]` (correct), or
(b) `stage[with_clause] + standalone where_clause` (wrong — query_body
then attached the where to the preceding MATCH at parser.py:1582-1587).

Earley's default ambiguity resolution silently picked the wrong parse:
the predicate ended up populated on both matches[0].where and top-level
where, with with_stages[0].where = None.  Surfaced by Wave 2 fresh-eyes
review of #1217 — the renamed/loosened
test_logical_planner_applies_predicates_attached_to_with_stage_filter
masked the regression.

Fix: bump Lark rule priority on with_where_clause so Earley prefers it
during ambiguity resolution.  Smoke-trace post-fix:
  match.where=0 preds, top.where=0 preds, with[0].where=ExpressionText(...)

Validated on dgx-spark:
- graphistry/tests/compute/gfql/cypher/  → 1038 passed
- graphistry/tests/compute/gfql/         → 1543 passed
- graphistry/tests/compute/              → 2441 passed

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ring mixin

Two Wave-2 review fixes:

1. test_logical_planner_applies_predicates_attached_to_with_part —
   restored original test name + WITH-part predicate assertion. The
   previous rename (..._to_with_stage_filter) and "predicate anywhere
   in plan" loosening had MASKED the WITH-WHERE re-routing regression
   that the previous commit (parser priority bump) fixed.  Now the
   binder correctly attaches the predicate to the WITH query_part, so
   the original assertion holds again.

2. New test_comparison_strings.py — direct unit tests for
   _StringAllowingComparisonMixin on EQ/NE/GT/LT/GE/LE.  Each op
   verified for: validate() acceptance of raw-string val, JSON
   round-trip preserving string val, lexicographic Series application
   matching pandas semantics.  Closes the wave-2 IMPORTANT gap that
   only end-to-end coverage existed for these ops.

Validated on dgx-spark: graphistry/tests/compute/ → 2458 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…p tests

DRY pass on #1217 surface area:
- parser.py: drop pre/post-#1031 narration on _split_top_level_and_pattern_leaves,
  _has_pattern_descendant, _rebuild_and_tree, _build_where_with_pattern_lift,
  pattern_atom, and the Earley parser comment.  Replace defensive
  'pattern_atom invariant' branch with an assert.
- test_lowering.py: trim 9-line OR/NOT section header + per-test docstrings
  to one structural line each; pandas NaN/Cypher semantics covered once
  at the section level.
- test_parser.py / test_where_bool_conformance.py: drop 'Pre-#1031 LALR /
  Post-#1031 Earley' history comments — assertions speak for themselves.
- test_comparison_strings.py: parametrize the per-op lex-apply tests via
  a callable→expected-list mapping; 5 near-identical functions → 1
  parametrized test.

Net -140 lines; no behavior change.  Validated on dgx-spark: 1078 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Adds 4 native end-to-end tests locking the Earley-admitted row-boolean
shapes that were previously deferred to #1219:

1. cross_alias_or_returns_correct_union — `WHERE a.x = 1 OR b.y = 2`
   against a fixture where wrong-rows would surface if the OR were
   silently split past a join.  Pushdown correctly leaves the OR intact;
   2 expected rows returned.
2. optional_match_or_with_existing_validator — `OPTIONAL MATCH ... WHERE
   ... OR b IS NULL` raises a `GFQLValidationError` from the existing
   OPTIONAL-MATCH-projection validator before WHERE evaluates.  Locks
   the gate fires regardless of OR shape.
3. mixed_type_or_via_ast_evaluator_gate — `WHERE n.var > 'te' OR n.var > 0`
   against a mixed-type Series.  The row-where AST evaluator emits an
   explicit `GFQLTypeError` rather than silently propagating pandas's
   raw TypeError.  Locks the loud-failure mode.
4. homogeneous_or_returns_correct_union — sanity check that simple
   homogeneous-type OR continues working post-#1217.

CHANGELOG entry rewritten: removes "complex compositions ... unverified"
language; replaces with empirical coverage summary.  #1219 still tracks
the deeper compositional frontier beyond current fixtures.

Validated on dgx-spark: 1547 passed (1543 + 4 new).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…onesty

Wave-3 review on commit aacb2a6 surfaced two IMPORTANTs:

1. Shape 2 test name + docstring overstated what it gates.  The
   OPTIONAL-MATCH-projection validator fires regardless of WHERE shape
   (or absence of WHERE).  Renamed to *_seed_only_projection,
   parametrized over `("", "WHERE b.x = 1", "WHERE b.x = 1 OR b IS NULL")`
   so the test now empirically *proves* OR-equivalence — the gate fires
   identically across all three WHERE shapes.

2. Shape 3 narrative implied a "type-mismatch gate"; in reality the call
   executor wraps any non-typed exception as GFQLTypeError(E303,
   "unsupported row expression").  Tightened pytest.raises to drop the
   fallback `TypeError` (unreachable empirically) and pin the substring
   "unsupported row expression" so a future regression that drops the
   wrapper actually fails the test.  CHANGELOG language softened
   accordingly: "the call executor wraps pandas's TypeError as
   GFQLTypeError(E303) via the generic unsupported-row-expression path".

Validated on dgx-spark: 1549 passed (1547 + 2 from shape-2 parametrize).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…ixture, CHANGELOG accuracy

Wave-4 (targeted-amplification + full-fresh-eyes) found 6 IMPORTANTs.
Addressed:

1. Shape-2 `match="OPTIONAL MATCH"` could match any of 6 distinct
   error sites — pinned to "seed alias are not yet supported" so the
   test cannot drift onto an unintended validator.

2. Shape-3 `match="unsupported row expression"` was an inner-ValueError
   substring with 30+ emit sites in the row pipeline — pinned to the
   stable error code "invalid-node-reference" (E303) instead, which
   is the actual contract the wrapper enforces.

3. Cross-alias-OR fixture (test_string_cypher_executes_cross_alias_or_*)
   used 1:1 join topology where multiple wrong-pushdown variants
   coincidentally produce the same answer.  Rebuilt with 2:2 join
   so wrong variants discriminate.  Expected rows changed from 2 to 3.

4. 5-row OR test (`test_string_cypher_executes_disjunction_returns_correct_count_with_more_rows`)
   used `len == 5` + `all(... in r ...)` — could pass on wrong row
   identities.  Tightened to sorted-id assertion.

5. CHANGELOG drift — claimed 1539 tests + only `match-where1-10`
   changed on tck-gfql side.  Rewrote to 1549 + the actual 8-refinement
   contract scope (TYPE_ERROR_KEYS, MATCHES_EXPECTED, WRONG_ROW_KEYS,
   PROMOTION demotion).

6. Coverage gap — `test_parse_rejects_mixed_where_pattern_predicates_as_unsupported`
   only covered variable-length patterns (`-[:R*]->`).  Added the
   simple non-variable-length variant so future slice 2/3/4 lifts
   can't silently regress the common case.

Validated on dgx-spark: 1551 passed (was 1549; +2 from non-VL params).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

Wave-5 SUGGESTION: the +2 non-VL parametrize entries from 912a4ac
brought the count to 1551 but the CHANGELOG still said 1549.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

…mixed-where-pattern-expr