Repository navigation
GFQL/Cypher: ORDER BY on stringified-list columns uses Cypher list-orderability (#1359) - #1362
Merged
Merged
Conversation
…ability (#1359) Issue #1359 (meta #1353 item #1) — when a list-valued property is stored as a string column (CSV/Arrow round-trip), `ORDER BY <col>` previously fell back to lex string sort, mishandling negatives ("-" < "2" in ASCII puts "[1, -20]" before "[1, 2]"). Add `order_detect_stringified_list_series` + `parse_stringified_list_series` in graphistry/compute/gfql/row/ordering.py; route the row pipeline through the existing `build_list_sort_columns` after `ast.literal_eval` parsing when the column is fully list-shaped. Track the parsed aux column in `aux_drop_cols` and drop it after `sort_values` so it doesn't leak into downstream stages. Python-list-typed columns continue through the existing list-aware path unchanged. Add pygraphistry-side regression tests on both Python-list and stringified-list inputs. Matching TCK port-level fixture/runner fixes that flip the 14 wrong-row scenarios to success_matches_expected are tracked in tck-gfql#36. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…1359) Apply review-wave findings inline: - W1-I1: bridge cuDF index in parse_stringified_list_series via host_index.to_pandas(), mirroring pipeline.py _gfql_eval_dynamic_list_subscript canonical pattern. - W2-A1: validate_order_series_vector_safe now splits the str family into list_string (list-shaped) vs plain str and raises mixed-family on the boundary, instead of silently lex-sorting partial-list-shape columns. Also extended to dtype="string" (StringDtype) columns. - W1-I4/I5/I6: add 4 behavioral tests pinning null-row tolerance, malformed-string fallback, multi-key ORDER BY (stringified-list + scalar) with aux-col leak guard, and partial-list-shape mixed-family raise. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
lmeyerov
force-pushed
the
issue-1359-with-orderby-wrong-rows
branch
from
May 9, 2026 01:18
f3bb6a2 to
de10711
Compare
Wave-4 review surfaced a misleading docstring on test_string_cypher_order_by_stringified_list_with_nulls — it claimed the parsed-list path runs through, but actually pipeline.py:3955-3967 skips that branch when has_null=True (mirroring the Python-list path), and we fall through to lex sort. Rewrite the docstring to describe what really happens and rename to ..._returns_top_k_without_error to remove ambiguity. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This was referenced May 9, 2026
Contributor
Author
|
Audit follow-ups filed as #1373 — (1) latent cuDF host-bridge gap in |
This was referenced May 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes pygraphistry-side semantic gap from #1359 (item #1 of meta-issue #1353 — Comparator + ORDER BY semantics tranche).
When a list-valued property is stored as a string column (e.g. round-tripped through CSV / Arrow string columns),
ORDER BY <col>previously fell back to lex string sort, which mishandles negative numbers because'-'(0x2D) sorts before'2'(0x32) — so'[1, -20]'came before'[1, 2]', putting the wrong row inLIMIT Nresults.What changed
graphistry/compute/gfql/row/ordering.pyorder_detect_stringified_list_series(series)— returns True when every non-null entry is an actual string matching^\[.*\]$. Complementsorder_detect_list_series(which only matches Python-list values).parse_stringified_list_series(series)— parses the column viaast.literal_eval, returning a Python-listSeries(orNoneon parse failure).graphistry/compute/gfql/row/pipeline.pyorder_by: after the existinglist_candidatecheck, if the column is a stringified-list series with no nulls, parse into a fresh__gfql_sort_listparsed_*__aux column and re-enter the existing list-sort path (build_list_sort_columns).aux_drop_colsand drop them aftersort_valuesso they don't leak into downstream stages (regression-tested via existingtest_string_cypher_supports_labels_projection_*andtest_string_cypher_supports_graph_functions_on_list_wrapped_entities).Python-list-typed columns continue through the existing path unchanged.
Tests
graphistry/tests/compute/gfql/cypher/test_lowering.pytest_string_cypher_order_by_python_list_column_uses_list_orderability— pins ASC/DESC top-3 on a Python-list column.test_string_cypher_order_by_stringified_list_column_uses_list_orderability— same query,dtype="string"list column; pre-fix produced E ([2,-2,100]) in top-3 instead of A ([2,-2]); post-fix the SET matches openCypher.Test plan
./bin/lint.shclean./bin/mypy.sh graphistry/compute/gfql/row/ordering.py graphistry/compute/gfql/row/pipeline.pycleanpytest graphistry/tests/compute/gfql/cypher/— 1224 passed, 90 skipped, 15 xfailed (no regressions)with-orderbykeys: Cluster Awith-orderby1-32-{1,2}(list ORDER BY DESC) flip tosuccess_matches_expected; remaining 12 are pygraphistry-SET-correct, blocked at runner levelsuccess_matches_expectedOut of scope (handled in tck-gfql#36)
tests/cypher_tck/parse_cypher.py::_parse_propertiesdoesn't parse list literals (lists become strings in fixtures).tests/cypher_tck/test_tck_runner.py::_rows_orderedenforces strict order even when the openCypher feature usesThen the result should be, in any order:.Cross-reference
with-orderbysuccess_wrong_rowscases🤖 Generated with Claude Code