Repository navigation
feat(arrow): add opt-in mixed type coercion - #1591
Merged
Merged
Conversation
lmeyerov
force-pushed
the
issue-867-auto-coerce-arrow
branch
4 times, most recently
from
May 21, 2026 06:16
668ccce to
ab99fc5
Compare
lmeyerov
force-pushed
the
issue-867-auto-coerce-arrow
branch
from
May 21, 2026 06:36
ab99fc5 to
07cc9dc
Compare
This was referenced May 21, 2026
perf: graphistry wheel size (625 kB) — GFQL engine files are large, investigate modularization
#1058
Closed
Contributor
Author
This was referenced May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #867.
Summary
auto_coerce=Falsetoplot(),upload(), andto_arrow()so Arrow conversion remains strict by default.auto_coerce=True, retry failed pandas/cuDF Arrow conversion by probing object columns one at a time and coercing only Arrow-incompatible columns to strings.Plottableprotocol, and anchored regression tests.Coercion heuristic
Graphistry first runs the normal Arrow conversion. If pandas/cuDF conversion fails with
ArrowTypeError/ArrowInvalidandauto_coerce=True, it tests object columns individually with the same Arrow conversion path. Only columns that fail individual Arrow conversion are coerced to strings; Arrow-compatible object columns are left alone. Pandas coercion uses nullable string dtype so missing values remain Arrow nulls. cuDF uses its string dtype path. This keeps the fallback narrow and mirrors the user workaround from #867 while preserving strict default behavior.Info-log sample:
LOC buckets
Compiler-plan surface touched: no. This is public Arrow-conversion API surface only; no GFQL compiler-plan routes, dispatch contracts, error-code/defer-code context, source spans, pass/verifier hooks, schema/type-system hooks, remote schema hooks, or compat executor paths were touched.
Validation
python3 -m pytest -q graphistry/tests/test_plotter.py::TestPlotterArrowConversions-> 25 passed, 7 skipped, 1 xfailedpython3 -m pytest -q graphistry/tests/test_plotter.py graphistry/tests/test_validate_settings.py graphistry/tests/test_schema_artifacts.py graphistry/tests/test_viz_settings.py-> 83 passed, 10 skipped, 4 xfailedpython3 bin/changed_line_coverage.py --data-file /tmp/pr1591.coverage --base-ref origin/master --head-ref HEAD --min-percent 80 --output-dir /tmp/pr1591-changed-coverage-> passed, 19/23 = 82.61%python3 -m graphistry.devschemas.export --check./bin/ruff.sh graphistry/PlotterBase.py graphistry/Plottable.py graphistry/exceptions.py graphistry/tests/test_plotter.py./bin/typecheck.sh graphistry/PlotterBase.py graphistry/Plottable.py graphistry/exceptions.pygit diff --checkssh dgx-spark 'cd repos/pygraphistry-issue-867-dgx && RAPIDS_VERSION=25.02 PROFILE=basic WITH_GPU=1 WITH_IMAGE_BUILD=0 TEST_FILES="graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_table_to_arrow_cudf_auto_coerce_smoke graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_api3_cudf_to_arrow_memoization graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_api3_cudf_to_arrow_memoization_forgets" docker/test-rapids-official-local.sh'-> cudf 25.02.02, cuml 25.02.01, Python 3.12.9, 3 passedssh dgx-spark 'cd repos/pygraphistry-issue-867-dgx && RAPIDS_VERSION=26.02 PROFILE=basic WITH_GPU=1 WITH_IMAGE_BUILD=0 TEST_FILES="graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_table_to_arrow_cudf_auto_coerce_smoke graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_api3_cudf_to_arrow_memoization graphistry/tests/test_plotter.py::TestPlotterArrowConversions::test_api3_cudf_to_arrow_memoization_forgets" docker/test-rapids-official-local.sh'-> cudf 26.02.01, cuml 26.02.00, Python 3.13.12, 3 passedTypecheck note:
./bin/typecheck.sh ... graphistry/tests/test_plotter.pystill hits existing test typing gaps (pyteststubs missing and unrelatedCOMPLEX_EMPTYannotation). Production touched files typecheck cleanly.