Skip to content

refactor(#1213 sub-PR A): lift boolean_expr_to_text + establish expr_tree invariant - #1214

Merged
lmeyerov merged 2 commits into
masterfrom
feat/issue-1213-sub-pr-a-lift-helper
Apr 25, 2026
Merged

lmeyerov merged 2 commits into
masterfrom
feat/issue-1213-sub-pr-a-lift-helper

Conversation

@lmeyerov

Copy link
Copy Markdown
Contributor

Summary

Sub-PR A of #1213. Two tightly-related steps that unblock the rest of the migration:

  1. Lift helpers: Move _boolean_expr_to_text and _flatten_top_level_ands from frontends/cypher/binder.py to new shared module graphistry/compute/gfql/cypher/_boolean_expr_text.py. Subsequent slices (lowering, ast_normalizer migrations) can now reconstruct WHERE-body text from expr_tree without depending on binder internals.

  2. Establish parser invariant: (WhereClause.expr is None) == (WhereClause.expr_tree is None). Three sites previously violated this by populating expr without expr_tree:

    • _mixed_where_clause (parser.py:984): WHERE pattern AND expr
    • where_clause non-structured branch (parser.py:1121): atom-shaped WHERE bypassing and_op/or_op
    • generic_where_clause single-atom fallback (parser.py:1176): items[0] not a BooleanExpr

    All three now synthesize a single-atom BooleanExpr(op="atom", atom_text=expr_text, ...). Behavior-preserving: binder's primary expr_tree path emits one BoundPredicate per top-level AND conjunct; a single-atom tree yields one BoundPredicate with expression == atom_text, identical to the legacy expr.text fallback the binder takes today.

Adds test_where_clause_expr_tree_invariant.py with 11 tests covering structured, label-narrowed, atom, OR, XOR, NOT, and mixed-pattern shapes.

Test plan

  • 11/11 invariant tests pass
  • Full GFQL suite locally: 1535 passed, 0 failed (no regressions)
  • CI green

Refs

🤖 Generated with Claude Code

lmeyerov and others added 2 commits April 25, 2026 04:57
…tree invariant

Move `_boolean_expr_to_text` and `_flatten_top_level_ands` from binder.py to
new shared module `cypher/_boolean_expr_text.py` so lowering and ast_normalizer
can reconstruct WHERE-body text from `expr_tree` in subsequent slices.

Establish parser invariant `(WhereClause.expr is None) == (WhereClause.expr_tree is None)`
by synthesizing single-atom `BooleanExpr` at three sites that previously left
`expr_tree` unset while populating `expr`:

  - `_mixed_where_clause` (parser.py): WHERE pattern AND expr / expr AND pattern
  - `where_clause` non-structured branch: atom-shaped WHERE bypassing and_op/or_op
  - `generic_where_clause` single-atom fallback: items[0] is not a BooleanExpr

Behavior-preserving: binder primary path emits one `BoundPredicate` per
top-level AND conjunct; a single-atom tree yields one `BoundPredicate` with
`expression == atom_text`, identical to the legacy `expr.text` fallback.

Add `test_where_clause_expr_tree_invariant.py` covering all three previously-
broken paths plus the structured-predicates and parenthesized-OR shapes.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@lmeyerov
lmeyerov marked this pull request as ready for review April 25, 2026 12:06
@lmeyerov
lmeyerov merged commit 1f90c57 into master Apr 25, 2026
133 checks passed
@lmeyerov
lmeyerov deleted the feat/issue-1213-sub-pr-a-lift-helper branch April 25, 2026 12:06
lmeyerov added a commit that referenced this pull request Apr 25, 2026
Compose with #1214's parser invariant `(expr is None) == (expr_tree is None)`:

- `generic_where_clause` now relies on #1214's single-atom synthesis to
  guarantee `expr_tree` is always a `BooleanExpr`. The walker
  (`_lift_label_only_and_spine`) runs over that uniform tree.
- Drops the `_ExpressionSlice` branch this PR added — redundant now that
  #1214 wraps non-BooleanExpr operands as single-atom trees upstream of
  this point.
- Merged CHANGELOG: kept both entries (this PR's #1194 walker entry +
  #1214's invariant entry); updated this PR's entry to note the
  composition with #1214.

Verified: 940 targeted (parser/binder/boolean_expr/conformance/
binder_expr_tree/where_clause_expr_tree_invariant/lowering) passed
including #1214's 11 new invariant tests; 1539 full GFQL sweep passed.
mypy on parser.py clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
lmeyerov added a commit that referenced this pull request Apr 26, 2026
…ree readers (#1213 sub-PR B) (#1216)

First read-side slice of #1213 (umbrella #1200 slice 5).  Migrate all
text-only readers of `WhereClause.expr.text` to consume
`boolean_expr_to_text(WhereClause.expr_tree)` directly, leveraging the
parser invariant `(expr is None) == (expr_tree is None)` established by
#1214.

Sites migrated:

- `frontends/cypher/binder.py::_where_predicates` — drop the dead
  `elif where.expr is not None` fallback.  Per #1214's invariant the
  branch is unreachable for parser-produced WhereClauses; it was
  defensive code for hand-built fixtures only.
- `cypher/lowering.py::_reject_unsupported_where_expr_forms` — gate +
  `expr.text.strip()` access.
- `cypher/lowering.py::_reject_unsupported_variable_length_where_pattern_predicates`
  — error-reporting `value=expr.text` access.
- `cypher/lowering.py::_check_expr` callers (three sites — `query.where`,
  `reentry_where`, and the post-binding `query.where` re-check) —
  gate + `expr.text` access.

Out of scope (left for sub-PR C):
- `ExpressionText` passthrough callers (`rewrite_expr` in lowering at
  7515/7533, `_extract_relationship_type_where` at 8131,
  `pre_join_filters.append` at 7998, `row_where` assignment at 6322-6329,
  `_rewrite_shortest_path_expr_text` at ast_normalizer.py:432-436).
- Span-access readers at lowering.py:8143-8144 (entangled with the
  passthrough at 8131; migrate together in C).

Tests: removed `test_where_predicates_with_expr_tree_none_falls_back_to_expr_text`
which exercised the now-dropped binder fallback via a hand-built
`WhereClause(expr=..., expr_tree=None)` fixture — that shape violates
the parser invariant and is unreachable from any real input.  The
sibling test `test_where_predicates_with_no_expr_at_all_returns_empty`
still covers the legitimate no-expr-tree case.

Verified:
- 1538/1538 GFQL tests pass (was 1539, minus deleted test).
- mypy clean on `cypher/lowering.py` and `frontends/cypher/binder.py`.

Coordination: this is sub-PR B per the split agreed on #1213
(reader-side: us; writer-side D+E: colleague who landed #1214).

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
lmeyerov added a commit that referenced this pull request Apr 26, 2026
…WhereClause.expr → expr_tree readers (#1213 sub-PR C) (#1218)

Second read-side slice of #1213.  Migrate all `ExpressionText`-passthrough
readers of `WhereClause.expr` to consume `WhereClause.expr_tree`-derived
values via two new helpers in `cypher/lowering.py`:

- `_where_clause_expr_text(where)` — synthesizes a fresh `ExpressionText`
  from `where.expr_tree` using `where.span` (which equals the original
  `where.expr.span` for parser-produced WhereClauses, both derived from
  the same `_span_from_meta(meta)` of `generic_where_clause`) so error
  positions match master's behavior.
- `_rewrite_where_clause_and_resync(where, rewrite, field)` — applies a
  text rewrite via the existing `(ExpressionText, str) -> ExpressionText`
  callback shape and resynchronizes `expr_tree` to a single-atom
  `BooleanExpr` carrying the rewritten text (Option B of the C decision
  posted on #1213; documented in plan.md).

Side-effect fix: closes the latent text/tree staleness bug on master
post-#1214 where `replace(where, expr=rewrite(where.expr, ...))` left
`expr_tree` pointing at the pre-rewrite text.  Now both fields stay in
sync after rewriting.

Sites migrated:

- `lowering.py:6322` — dynamic row-where extraction
  (`_extract_relationship_type_where(query.where.expr, ...)` + `row_where`
  passthrough → use `_where_clause_expr_text`).
- `lowering.py:7530-7538` — reentry-where rewrite of remaining wheres
  (use `_rewrite_where_clause_and_resync`).
- `lowering.py:7550-7553` — reentry-where rewrite of primary
  `query.reentry_where` (same helper).
- `lowering.py:7998` — `pre_join_filters.append(query.where.expr)` →
  append the synthesized ExpressionText.
- `lowering.py:8176-8191` — connected-OPTIONAL-MATCH where lowering
  (`_extract_relationship_type_where` + `_unsupported` error reporting
  with `.text` and `.span.line/.column`) → all consume the synthesized
  ExpressionText uniformly.
- `ast_normalizer.py::_rewrite_where` — shortest-path expr rewrite via
  `_rewrite_shortest_path_expr_text` (inline equivalent of the lowering
  helper since this module only has one callsite).

Out of scope (deferred to D + E):

- `ast_normalizer.py:520-526` constructor passthrough (`expr=query.where.expr`
  inside a `WhereClause(...)` re-construction) — left for sub-PR D to drop
  along with the other writer-side construction-site updates in parser.py.
  The gate at L520 is migrated (`expr is not None` → `expr_tree is not None`).

Verified:
- 1538/1538 GFQL tests pass; no regressions.
- `mypy` clean on `cypher/lowering.py` and `cypher/ast_normalizer.py`.

Coordination: this is sub-PR C per the split agreed on #1213.  Ready for
the colleague's combined sub-PRs D + E (drop `expr=` from `WhereClause(...)`
constructions; remove the field) once this lands.

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
lmeyerov added a commit that referenced this pull request Apr 26, 2026
IMPORTANT findings from multi-dimension review:

1. Dead-code in OR test (test_lowering.py:3199-3205) — the
   ``if False else`` made the first comprehension permanently dead;
   the fallback branch then asserted on either of two contradictory
   shapes.  Replaced with a single deterministic assertion: 5 rows
   expected, every kept row carries either ``p1: 1`` or ``p2: 2`` in
   its rendered form.
2. EQ duplicated the new mixin's overrides (comparison.py:245-263).
   The mixin docstring already invited the retrofit ("EQ predates this
   mixin with an inline override").  EQ now uses
   ``_StringAllowingComparisonMixin`` for its single-source contract.

SUGGESTIONS deferred (documented in plan):
- NaN-NOT semantics test name (locks pandas-not-Cypher behavior;
  acknowledged in test docstring + #1219 link).
- ``_rebuild_and_tree`` invents source-spans across non-adjacent
  residuals (cosmetic, error-reporting only).
- No deeply-nested pattern-under-multi-level-boolean test
  (``_has_pattern_descendant`` handles correctly by inspection).
- PEP-8 spacing nit after constant deletes.
- Defensive ``_split_top_level_and_pattern_leaves`` bad-flag drop on
  malformed AND nodes (#1214 invariant prevents the case).

1680 GFQL + predicates tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
lmeyerov added a commit that referenced this pull request Apr 26, 2026
* refactor(#1213 sub-PR D+E): drop WhereClause.expr field

Final slice of #1200 / #1213. Removes the legacy raw-text WHERE field now
that all production readers (binder, lowering, ast_normalizer) consume
expr_tree-derived text via boolean_expr_to_text() per sub-PR B/C.

Sub-PR D — drop expr= from writer construction sites:
  - parser.py: _mixed_where_clause, where_clause non-structured branch,
    generic_where_clause fallback (the three sites that synthesized
    BooleanExpr atoms in #1214); also strip expr=None from the three
    structured-path constructions
  - ast_normalizer.py: drop expr= forwarder + switch gate to expr_tree

Sub-PR E — remove the field:
  - ast.py: drop WhereClause.expr; update docstring to two-shape contract
    (structured vs tree, with mixed pattern+expr as the third combination)
  - test_where_clause_expr_tree_invariant.py: rewrite from the now-vacuous
    (expr is None) == (expr_tree is None) symmetry to the surviving
    routing-shape contract (structured / tree / mixed)
  - test_parser.py, test_boolean_expr.py, test_ast_normalizer.py,
    test_where_bool_conformance.py: replace `where.expr is None/not None`
    with `where.expr_tree is None/not None`; replace `where.expr.text`
    with `boolean_expr_to_text(where.expr_tree)`; drop the slice-1
    backward-compat test that asserted both fields were populated

NOT YET REBASED ONTO POST-C MASTER. This commit produces the expected
93 test failures from lowering/ast_normalizer readers that still access
.expr; sub-PR C migrates them. Rebase onto post-C master before pushing
or running CI.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* docs(#1213 sub-PR D+E): pre-stage CHANGELOG entry for field removal

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(#1213 sub-PR D+E): drop leftover expr= from sub-PR C's replace() calls

Sub-PR C (#1218) introduced two ``replace(where, expr=rewritten, expr_tree=new_tree)``
calls in ``ast_normalizer.py::_rewrite_where`` and ``lowering.py::_rewrite_where_clause_and_resync``.
These passed both fields because the field still existed at C's merge time.

After this PR removes the field, the ``expr=`` arg becomes invalid keyword
to the dataclass constructor. Drop it from both sites; ``expr_tree`` is the
single source of truth post-D+E.

Also drops the now-unused ``expr=...`` and ``expr=None`` args from three
hand-built ``WhereClause(...)`` test fixtures in ``test_binder_expr_tree.py``
that broke for the same reason, plus the now-unused ``ExpressionText`` import.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* docs(#1213 sub-PR D+E): apply review findings — docstring + test coverage

Two review waves (focus + general per the new review policy) converged on:

1. **WhereClause docstring missing the "mixed" routing shape** (IMPORTANT, both
   waves). The new docstring listed only 2 shapes; reality is 3 (and arguably
   subdivided further by which kind of structured `predicates` populated).
   Rewrote to enumerate all three observable shapes, distinguishing
   `WherePredicate` from `WherePatternPredicate` in the structured case.

2. **Invariant test missing parser construction paths** (IMPORTANT, focus
   wave). Added `MATCH (a) WHERE (a)-[]->(:Admin) RETURN a` covering
   `where_pattern_only_clause` (structured, no AND, single
   `WherePatternPredicate`).  `expr_and_where_pattern_clause` (expr-first
   mixed) doesn't currently parse; `where_clause` _ExpressionSlice branch is
   Lark-internal routing already exercised by integration tests.

3. **Stale binder comment claiming `predicates`/`expr_tree` mutually exclusive**
   (SUGGESTION, general wave).  They aren't — the mixed shape populates
   both — and the binder body correctly handles all three shapes.  Updated
   the comment to match the AST docstring's 3-shape contract.

4. **CHANGELOG polish** (SUGGESTIONS, both waves): clarified count
   ("9 query forms across the 3 shapes" rather than the Cartesian-reading
   "3 × 8"), added an API note that `WhereClause` is exported and external
   constructors must drop `expr=` kwarg.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
lmeyerov added a commit that referenced this pull request Apr 27, 2026
…hape cruft (#1031 slice 1) (#1217)

* fix(predicates): allow raw strings in NE/GT/LT/GE/LE comparison predicates

EQ already accepts raw strings via inline ``_normalize_value`` +
``_validate_fields`` overrides.  The other comparison ops lacked the
same overrides and rejected raw strings unconditionally — their
``__call__`` methods also explicitly raised ``TypeError`` on strings.

This was latent: LALR couldn't unify label-predicate and
property-predicate alternatives in Cypher's ``where_predicates``, so
queries like ``WHERE n.name <> 'value' AND n:Label`` routed through
raw expr text and never reached ``ComparisonPredicate.__call__``.

#1031's Earley swap unifies the route, surfacing the strictness on
idiomatic Cypher patterns.  Pandas Series supports lexicographic
``>``/``<``/``!=`` on strings natively, so the fix is to extend EQ's
pre-existing string acceptance to siblings.

Implementation:
- Extract ``_StringAllowingComparisonMixin`` with the
  ``_normalize_value`` + ``_validate_fields`` overrides.
- Apply mixin to NE/GT/LT/GE/LE.
- Extend each ``__call__`` to handle ``isinstance(self.val, str)``.

The strict raw-string rejection on the base ``ComparisonPredicate``
stays — it still applies to ``Between`` (and any future direct-IR
constructors) where the datetime-vs-string ambiguity matters for
direct-IR users.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor(gfql/cypher): switch to Earley parser; retire LALR pattern-shape cruft (#1031 slice 1)

Per tech-lead direction (1-day timebox) following an LALR(1) state-collapse
on ``<comparable> AND <pattern>`` shapes that could not be resolved by
grammar restructure.

Earley parser swap eliminates four LALR-induced workarounds:

1. The 3 dedicated pattern-shape ``where_clause`` grammar alternatives
   (``where_pattern_and_expr_clause`` / ``expr_and_where_pattern_clause``
   / ``where_pattern_only_clause``) — collapse to the unified ``expr``
   route via a new ``WHERE_PATTERN -> pattern_atom`` leaf in ``?primary``.
2. ``_canonicalize_where_single_pattern_and_expr`` — regex source-rewrite
   that reordered ``expr AND pattern AND expr`` to ``pattern AND <rest>``
   so the LALR grammar could match.  Earley accepts patterns at any
   AND-spine position natively.
3. ``_mixed_where_pattern_expr_error`` — pre-flight rejector for shapes
   LALR could parse but the IR couldn't represent.  Replaced with a
   structural lift step that inspects the parsed boolean tree.
4. The ``except LarkError`` retry block in ``parse_cypher`` that fed
   the canonicalize-then-reparse path.

What this PR adds:

- ``BooleanExpr.op`` literal extended with ``"pattern"``; new
  ``BooleanExpr.pattern: Optional[Tuple[PatternElement, ...]]`` field.
- Parser ``pattern_atom`` transformer: wraps ``WHERE_PATTERN`` token as
  ``BooleanExpr(op="pattern", pattern=parsed)``; reuses the existing
  ``_parse_where_pattern_predicate_text`` helper.
- ``_split_top_level_and_pattern_leaves`` walker + ``_rebuild_and_tree``
  helper + ``_build_where_with_pattern_lift`` that extracts pattern
  leaves from ``expr_tree`` into ``WhereClause.predicates`` as
  ``WherePatternPredicate`` entries before lowering.
- Parser swap: ``Lark(parser="earley")`` for both ``_parser`` and
  ``_pattern_parser``.

What this PR retires:

- 3 grammar alternatives, 3 transformer methods, 1 helper
  (``_mixed_where_clause``), 2 pre-flight functions, 3 derived regex
  constants (``_WHERE_PATTERN_SEQUENCE_RE``, ``_WHERE_PATTERN_THEN_EXPR_RE``,
  ``_WHERE_EXPR_THEN_PATTERN_RE``), 2 helper utilities
  (``_WHERE_CLAUSE_BODY_RE``, ``_BOOLEAN_KEYWORD_RE``,
  ``_line_and_column_from_offset``), and the ``parse_cypher``
  ``except LarkError`` retry block.

Net diff: +204 / -144.

Strict-improvement consequences (Earley accepts what LALR rejected):

- ``WHERE expr OR expr`` now parses as a structured ``or`` tree.
- ``WHERE expr AND (expr OR expr)`` (Shape B) now parses as
  ``and(left, or(...))``.
- ``WHERE n:Label AND n.prop = X`` now routes through structured
  ``where_predicates`` (was raw expr); label narrowing applies.
- ``WHERE prop OP 'string'`` mixed with label predicates now hits
  ``ComparisonPredicate`` for strings — separately fixed in the
  preceding commit by extending EQ's string-accepting overrides
  to NE/GT/LT/GE/LE.

#1031 milestones in this PR:

- ✅ Retires ``_canonicalize_where_single_pattern_and_expr``.
- ✅ Removes one of three ``split_top_level_and`` callers (parser side).
  Remaining caller: ``predicate_pushdown._split_conjuncts``, retired
  separately under #1200.  Together this brings ``expr_split.py``
  deletion within reach.
- ⏭️  ``NOT pattern`` lowering (anti-semi-join) — slice 2 territory;
  emits explicit ``unsupported`` error today.
- ⏭️  Multi-positive patterns — slice 3 territory; rejected at lift step.
- ⏭️  ``OR``/``XOR`` around patterns — slice 4 territory; rejected at lift step.

Tests updated:

- 7 tests adjusted to match Earley's strict-improvement routing
  (``test_or_where_not_yet_supported``, conformance Shape B parser +
  binder, conformance Shape F parser + binder, label-narrowing mixed
  case, logical planner WITH-stage attachment).
- ``test_parse_supports_where_pattern_predicate_and_expr_mix`` paren-
  preservation case updated to expect ``boolean_expr_to_text``-
  reconstructed paren-stripped form (semantically equivalent).

1539 / 1539 GFQL tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* docs(changelog): add #1031/#1217 Earley parser swap + comparison strings entries

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* test(gfql/cypher): native end-to-end coverage for OR/NOT WHERE shapes (#1217)

Pre-#1217: LALR rejected OR / NOT inside WHERE bodies, so disjunctive
and negation shapes had zero native end-to-end test coverage.  Earley
accepts them and the runtime evaluates via pandas, but until now the
correctness was implicit — only the (separate-repo) tck-gfql harness
exercised these paths via xfail-marked scenarios.

Adds 5 native execution tests in test_lowering.py:

- ``test_string_cypher_executes_disjunctive_property_predicate_returns_union``
  mirrors TCK match-where1-10: ``WHERE n.p1 = 12 OR n.p2 = 13`` returns
  the union of matches (a, b), not just one.
- ``test_string_cypher_executes_disjunctive_same_alias_property_predicate``
  ``WHERE n.p1 = 12 OR n.p1 = 99`` — same alias on both sides.
- ``test_string_cypher_executes_negation_property_predicate_returns_complement``
  ``WHERE NOT n.p1 = 12`` — locks pandas's NaN-NOT-NaN semantics
  (NaN is falsy under NOT, so rows with null p1 are excluded).
- ``test_string_cypher_executes_disjunctive_then_conjunction``
  ``WHERE (n.p1 = 12 OR n.p2 = 13) AND n.id = 'a'`` — mixed OR-AND
  tree narrowing.
- ``test_string_cypher_executes_disjunction_returns_correct_count_with_more_rows``
  larger fixture (6 rows) confirming OR doesn't silently union too many
  or too few rows.

Pairs with the tck-gfql sibling branch ``issue-1031-grammar-mixed-where-pattern-expr``
which registers ``match-where1-10`` as ``success_matches_expected`` in
the xfail-contract (kept as ``xfail`` rather than promoted to
``supported`` until broader OR-disjunction conformance is validated).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* docs(changelog): cross-link #1219 row-boolean validation gap

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* refactor+test: wave-1 review fixes for #1217

IMPORTANT findings from multi-dimension review:

1. Dead-code in OR test (test_lowering.py:3199-3205) — the
   ``if False else`` made the first comprehension permanently dead;
   the fallback branch then asserted on either of two contradictory
   shapes.  Replaced with a single deterministic assertion: 5 rows
   expected, every kept row carries either ``p1: 1`` or ``p2: 2`` in
   its rendered form.
2. EQ duplicated the new mixin's overrides (comparison.py:245-263).
   The mixin docstring already invited the retrofit ("EQ predates this
   mixin with an inline override").  EQ now uses
   ``_StringAllowingComparisonMixin`` for its single-source contract.

SUGGESTIONS deferred (documented in plan):
- NaN-NOT semantics test name (locks pandas-not-Cypher behavior;
  acknowledged in test docstring + #1219 link).
- ``_rebuild_and_tree`` invents source-spans across non-adjacent
  residuals (cosmetic, error-reporting only).
- No deeply-nested pattern-under-multi-level-boolean test
  (``_has_pattern_descendant`` handles correctly by inspection).
- PEP-8 spacing nit after constant deletes.
- Defensive ``_split_top_level_and_pattern_leaves`` bad-flag drop on
  malformed AND nodes (#1214 invariant prevents the case).

1680 GFQL + predicates tests pass.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

* fix(parser): prefer with_where_clause over query_item where_clause under Earley ambiguity (#1031)

Earley with ambiguous grammar parsed `MATCH ... WITH ... WHERE ... RETURN`
two ways: (a) `with_stage[with_clause + with_where_clause]` (correct), or
(b) `stage[with_clause] + standalone where_clause` (wrong — query_body
then attached the where to the preceding MATCH at parser.py:1582-1587).

Earley's default ambiguity resolution silently picked the wrong parse:
the predicate ended up populated on both matches[0].where and top-level
where, with with_stages[0].where = None.  Surfaced by Wave 2 fresh-eyes
review of #1217 — the renamed/loosened
test_logical_planner_applies_predicates_attached_to_with_stage_filter
masked the regression.

Fix: bump Lark rule priority on with_where_clause so Earley prefers it
during ambiguity resolution.  Smoke-trace post-fix:
  match.where=0 preds, top.where=0 preds, with[0].where=ExpressionText(...)

Validated on dgx-spark:
- graphistry/tests/compute/gfql/cypher/  → 1038 passed
- graphistry/tests/compute/gfql/         → 1543 passed
- graphistry/tests/compute/              → 2441 passed

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(#1217 wave-2): restore WITH-part assertion + cover comparison-string mixin

Two Wave-2 review fixes:

1. test_logical_planner_applies_predicates_attached_to_with_part —
   restored original test name + WITH-part predicate assertion. The
   previous rename (..._to_with_stage_filter) and "predicate anywhere
   in plan" loosening had MASKED the WITH-WHERE re-routing regression
   that the previous commit (parser priority bump) fixed.  Now the
   binder correctly attaches the predicate to the WITH query_part, so
   the original assertion holds again.

2. New test_comparison_strings.py — direct unit tests for
   _StringAllowingComparisonMixin on EQ/NE/GT/LT/GE/LE.  Each op
   verified for: validate() acceptance of raw-string val, JSON
   round-trip preserving string val, lexicographic Series application
   matching pandas semantics.  Closes the wave-2 IMPORTANT gap that
   only end-to-end coverage existed for these ops.

Validated on dgx-spark: graphistry/tests/compute/ → 2458 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor+test: wave-2 trim narrative comments + parametrize string-cmp tests

DRY pass on #1217 surface area:
- parser.py: drop pre/post-#1031 narration on _split_top_level_and_pattern_leaves,
  _has_pattern_descendant, _rebuild_and_tree, _build_where_with_pattern_lift,
  pattern_atom, and the Earley parser comment.  Replace defensive
  'pattern_atom invariant' branch with an assert.
- test_lowering.py: trim 9-line OR/NOT section header + per-test docstrings
  to one structural line each; pandas NaN/Cypher semantics covered once
  at the section level.
- test_parser.py / test_where_bool_conformance.py: drop 'Pre-#1031 LALR /
  Post-#1031 Earley' history comments — assertions speak for themselves.
- test_comparison_strings.py: parametrize the per-op lex-apply tests via
  a callable→expected-list mapping; 5 near-identical functions → 1
  parametrized test.

Net -140 lines; no behavior change.  Validated on dgx-spark: 1078 passed.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test(#1217): pull #1219's row-boolean validation gap into-scope

Adds 4 native end-to-end tests locking the Earley-admitted row-boolean
shapes that were previously deferred to #1219:

1. cross_alias_or_returns_correct_union — `WHERE a.x = 1 OR b.y = 2`
   against a fixture where wrong-rows would surface if the OR were
   silently split past a join.  Pushdown correctly leaves the OR intact;
   2 expected rows returned.
2. optional_match_or_with_existing_validator — `OPTIONAL MATCH ... WHERE
   ... OR b IS NULL` raises a `GFQLValidationError` from the existing
   OPTIONAL-MATCH-projection validator before WHERE evaluates.  Locks
   the gate fires regardless of OR shape.
3. mixed_type_or_via_ast_evaluator_gate — `WHERE n.var > 'te' OR n.var > 0`
   against a mixed-type Series.  The row-where AST evaluator emits an
   explicit `GFQLTypeError` rather than silently propagating pandas's
   raw TypeError.  Locks the loud-failure mode.
4. homogeneous_or_returns_correct_union — sanity check that simple
   homogeneous-type OR continues working post-#1217.

CHANGELOG entry rewritten: removes "complex compositions ... unverified"
language; replaces with empirical coverage summary.  #1219 still tracks
the deeper compositional frontier beyond current fixtures.

Validated on dgx-spark: 1547 passed (1543 + 4 new).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test+docs(#1217): wave-3 review fixes — calibrate shape-2 / shape-3 honesty

Wave-3 review on commit aacb2a6 surfaced two IMPORTANTs:

1. Shape 2 test name + docstring overstated what it gates.  The
   OPTIONAL-MATCH-projection validator fires regardless of WHERE shape
   (or absence of WHERE).  Renamed to *_seed_only_projection,
   parametrized over `("", "WHERE b.x = 1", "WHERE b.x = 1 OR b IS NULL")`
   so the test now empirically *proves* OR-equivalence — the gate fires
   identically across all three WHERE shapes.

2. Shape 3 narrative implied a "type-mismatch gate"; in reality the call
   executor wraps any non-typed exception as GFQLTypeError(E303,
   "unsupported row expression").  Tightened pytest.raises to drop the
   fallback `TypeError` (unreachable empirically) and pin the substring
   "unsupported row expression" so a future regression that drops the
   wrapper actually fails the test.  CHANGELOG language softened
   accordingly: "the call executor wraps pandas's TypeError as
   GFQLTypeError(E303) via the generic unsupported-row-expression path".

Validated on dgx-spark: 1549 passed (1547 + 2 from shape-2 parametrize).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* test+docs(#1217): wave-4 review fixes — pin substrings, adversarial fixture, CHANGELOG accuracy

Wave-4 (targeted-amplification + full-fresh-eyes) found 6 IMPORTANTs.
Addressed:

1. Shape-2 `match="OPTIONAL MATCH"` could match any of 6 distinct
   error sites — pinned to "seed alias are not yet supported" so the
   test cannot drift onto an unintended validator.

2. Shape-3 `match="unsupported row expression"` was an inner-ValueError
   substring with 30+ emit sites in the row pipeline — pinned to the
   stable error code "invalid-node-reference" (E303) instead, which
   is the actual contract the wrapper enforces.

3. Cross-alias-OR fixture (test_string_cypher_executes_cross_alias_or_*)
   used 1:1 join topology where multiple wrong-pushdown variants
   coincidentally produce the same answer.  Rebuilt with 2:2 join
   so wrong variants discriminate.  Expected rows changed from 2 to 3.

4. 5-row OR test (`test_string_cypher_executes_disjunction_returns_correct_count_with_more_rows`)
   used `len == 5` + `all(... in r ...)` — could pass on wrong row
   identities.  Tightened to sorted-id assertion.

5. CHANGELOG drift — claimed 1539 tests + only `match-where1-10`
   changed on tck-gfql side.  Rewrote to 1549 + the actual 8-refinement
   contract scope (TYPE_ERROR_KEYS, MATCHES_EXPECTED, WRONG_ROW_KEYS,
   PROMOTION demotion).

6. Coverage gap — `test_parse_rejects_mixed_where_pattern_predicates_as_unsupported`
   only covered variable-length patterns (`-[:R*]->`).  Added the
   simple non-variable-length variant so future slice 2/3/4 lifts
   can't silently regress the common case.

Validated on dgx-spark: 1551 passed (was 1549; +2 from non-VL params).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(changelog): fix #1217 test count drift (1549 → 1551)

Wave-5 SUGGESTION: the +2 non-VL parametrize entries from 912a4ac
brought the count to 1551 but the CHANGELOG still said 1549.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(ci): sync workflow security files from master

---------

Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant