Skip to content

feat(gfql): index DDL and a query in one gfql() call (#2119) - #2130

Closed
lmeyerov wants to merge 5 commits into
masterfrom
feat/gfql-2119-fused-index-ddl
Closed

feat(gfql): index DDL and a query in one gfql() call (#2119)#2130
lmeyerov wants to merge 5 commits into
masterfrom
feat/gfql-2119-fused-index-ddl

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Closes #2119. CREATE GFQL INDEX ... was matched whole-string, so building an index and querying it were two gfql() calls (the indexing.rst "two calls today" note). Now leading CREATE GFQL INDEX FOR <kind> / DROP GFQL INDEX ... statements separated by ; build the indexes in order and the query that follows runs on them; the caller's g is unchanged:

out = g.gfql(
    "CREATE GFQL INDEX FOR edge_out_adj; "
    "CREATE GFQL INDEX FOR node_id; "
    "MATCH (m {id: 0})-[e]->(p) RETURN p"
)
g.gfql_explain(same_string)["used_index"]   # True
  • cypher_ddl.py: split_top_level_statements (; outside quotes/brackets) and parse_index_ddl_prefix, gated by an unanchored (CREATE|DROP|SHOW) GFQL INDEX search so a plain query pays nothing; a lone DDL statement returns None and keeps today's whole-string path (and today's "Malformed" error).
  • ComputeMixin.gfql: applies the leading ops with apply_index_op, then runs the remainder on the indexed graph (or returns the indexed graph for a DDL-only list), honoring index_policy/engine.
  • Typed errors: DDL after the query, SHOW GFQL INDEXES inside a list (it returns a table), malformed DDL. ; inside a string literal is not a statement break.
  • indexing.rst: the two-calls sentence becomes the fused example (an unmarked block, so docs/test_doc_examples.py runs it).

Design note and the deferred GRAPH { CALL graphistry.index.* } form are in plans/gfql-benchmark-numbers/plan.md ("QUEUE ITEM #2119"): the issue's "or a statement list in gfql()" is met here; the CALL form needs a new procedure backend + safelist entry and is left as an owner call.

Test plan

  • graphistry/tests/compute/gfql/index/test_index_ddl_statement_list_2119.py (9): fused query takes the index and leaves the caller untouched, DDL-only list, literal ;, four typed rejections, lone statements keep their paths, splitter/prefix contract
  • gfql/index + cypher parser + test_chain: 1401 passed, 0 failed; docs/test_doc_examples.py -k indexing: passed
  • changed-line coverage 94.4%, bin/lint.sh, mypy clean
  • CI green at the head

🤖 Generated with Claude Code

https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp

lmeyerov and others added 5 commits October 3, 2026 01:26
@lmeyerov @claude
feat(gfql): index DDL and a query in one gfql() call (#2119)
9c95cfb 
Leading CREATE/DROP GFQL INDEX statements separated by ';' build the indexes in order and
the query that follows runs on them; the caller's g is unchanged. ';' inside literals is
not a break; DDL after the query or SHOW inside a list is a typed error; a lone DDL
statement keeps its path. indexing.rst's two-calls note becomes the fused example.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov @claude
fix(gfql): register the DDL-anywhere regex cache with the cache registry
ec843dd 
Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov @claude
test(gfql): split the fused-DDL engagement pin from the result pin fo…
886dff7 
…r the routes-off replay

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov @claude
refactor(gfql): one DDL word pattern, one trailing-DDL raise; shorter…
cb72c04 
… docs sentences

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov @claude
test(gfql): run the fused-DDL result pin on every available engine; r…
4f72cd6 
…egister it in the polars lane

Co-Authored-By: Claude Fable 5.1 <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
@lmeyerov

lmeyerov commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Real-GPU receipt (dgx-spark, graphistry/test-rapids-official:26.02-gfql-polars, cudf 26.02.01 / cupy 13.6.0 / polars 1.35.2) at 4f72cd6: 327 passed, 2 xfailed (statement-list pins incl. the cuDF parameter + test_index).

@lmeyerov

lmeyerov commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Closing as superseded. Master landed the same feature independently in cdac877 (#2119 is marked done in the 0.59.1 changelog): split_leading_index_ddl + looks_like_index_ddl dispatch in ComputeMixin.gfql, with pins covering what this PR pinned — leading DDL then query (pandas/polars/cuDF), a ; inside the query body not splitting, misplaced/malformed DDL as a typed error, and gfql_explain engagement parity with the two-step form (test_index_ddl_in_one_call.py).

The one residual this branch still carried was the docs/source/gfql/indexing.rst paragraph that said "building and querying are two calls today ... not accepted yet"; that correction is folded into #2131 (614d646), which already touches that page.

@lmeyerov lmeyerov closed this Oct 4, 2026
@lmeyerov lmeyerov mentioned this pull request Oct 4, 2026
Merged
2 of 3 tasks
@lmeyerov

lmeyerov commented Oct 4, 2026

Copy link
Copy Markdown
Contributor Author

Read-only review (parallel session) — supersession audit vs master #2132

Nothing here touches the branch; findings only, fixes deferred.

Mode: findings, fixes deferred. Read-only; the branch is the colleague's. Method: #2130's own
test cases (test_index_ddl_statement_list_2119.py) replayed as queries against MASTER through
g.gfql(str) (its API names parse_index_ddl_prefix / split_top_level_statements do not exist on
master, so the import-level tests cannot run; the BEHAVIOR is what matters). Probe:
scratchpad/probe_2130_on_master.py.

#2130 case master behaviour parity
CREATE ...; MATCH ... (lead + query) runs, 3 rows, indexes built same
two DDL + query runs same
DDL-only list, with and without trailing ; runs, returns indexed graph same
; inside a string literal in the query not split, 1 row same (master never splits the remainder)
SHOW GFQL INDEXES; MATCH ... ValueError: SHOW GFQL INDEXES answers a table and cannot be combined same class, different text
malformed leading DDL (FOR bogus) ValueError: Malformed GFQL INDEX DDL same
lowercase keywords, leading newlines, DROP ... IF EXISTS lead run same
query whose string literal CONTAINS CREATE GFQL INDEX ordinary query, not treated as DDL same
DDL after the query (MATCH ...; CREATE ...) GFQLSyntaxError [invalid-cypher-syntax] from the Cypher parser delta: #2130 raises a typed ValueError("GFQL INDEX DDL must lead the statement list ...")
DDL on both sides same generic syntax error same delta

Verdict: superseded. Every accepted shape in #2130 is accepted by master with the same result, and
every rejected shape is rejected. The only delta is the quality of one error message: trailing DDL
after a query surfaces as a generic Cypher syntax error on master instead of a pointed "DDL must lead"
error. That is a SUGGESTION-sized follow-up (detect looks_like_index_ddl on a trailing ;-separated
statement in split_leading_index_ddl and raise the pointed error), not a reason to keep #2130.

The docs paragraph fix the colleague wants to carry is already going into #2131 per their note.

Recommendation: close #2130 with a comment pointing at #2132 and the one-line error-message follow-up
(file it as an issue or fold it into #2131 since that PR already edits cypher_ddl.py).

Draft close comment:

Superseded by #2132 (master dd6daca), which landed the same one-call CREATE GFQL INDEX ...; <query>
shape. Replayed this PR's cases against master: every accepted and rejected shape matches, including
; inside string literals, DDL-only lists, SHOW-in-list and malformed DDL. One delta worth keeping:
DDL placed AFTER the query gets a generic invalid-cypher-syntax on master rather than this PR's
"DDL must lead the statement list" error — small follow-up in split_leading_index_ddl. Closing.

🤖 Generated with Claude Code

https://claude.ai/code/session_017ropeBMLJUuy6ViYwy15ud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

GFQL: index DDL cannot be fused with a query in one gfql() call (no CREATE GFQL INDEX inside GRAPH { }, no CreateIndex chain/let stage)

1 participant

@lmeyerov