Skip to content

feat(pyspark and db): db notebook and pyspark df - #277

Merged
lmeyerov merged 11 commits into
masterfrom
dev/databricks
Nov 22, 2021
Merged

lmeyerov merged 11 commits into
masterfrom
dev/databricks

Conversation

@lmeyerov

@lmeyerov lmeyerov commented Nov 21, 2021 •

Copy link
Copy Markdown
Contributor

Feat:

  • new plot params extra_html and override_html_style
  • passing in a pyspark dataframe will get auto-handled, including memoization
  • plotting from a databricks notebook will Just Work
  • databricks dashboards may require you to play with override_html_style

Docs:

  • Demo notebook of spark df ingest, databricks notebook plotting, and databricks dashboard plotting
  • Examples on main readme

Ex: Databricks Spark graphistry notebook:

displayHTML(
  graphistry
    .edges(devices.sample(fraction=0.1), 'device_name', 'cca3')
    .settings(url_params={'strongGravity': 'true'})
    .plot()
)

Ex: Extra formatting for Databricks Graphistry dashboard:

displayHTML(
    g
        .settings(url_params={'splashAfter': 'false'})  # extends existing setting
        .plot(override_html_style="""
            border: 1px #DDD dotted;
            width: 50em; height: 50em;
        """)
)

image

@lmeyerov
lmeyerov merged commit 2937c42 into master Nov 22, 2021
@lmeyerov
lmeyerov deleted the dev/databricks branch November 22, 2021 06:39
lmeyerov added a commit that referenced this pull request Mar 13, 2022
* feat(pyspark and db): db notebook and pyspark df

* fix(databricks): global imports

* fix(databricks): more settings detection

* feat(height): support strings

* feat(height): support strings

* feat(iframe extra_html)

* feat(override_html_style)

* docs(spark)

* docs(dev)

* fix(docs): trim spark output

* docs(databricks): tweak
pull Bot referenced this pull request in admariner/pygraphistry Mar 13, 2022
* feat(pyspark and db): db notebook and pyspark df

* fix(databricks): global imports

* fix(databricks): more settings detection

* feat(height): support strings

* feat(height): support strings

* feat(iframe extra_html)

* feat(override_html_style)

* docs(spark)

* docs(dev)

* fix(docs): trim spark output

* docs(databricks): tweak
lmeyerov added a commit that referenced this pull request Sep 17, 2026
The previous vendor could not be regenerated. Its own generated_by named pyg-bench
`baf58bd8`, which is not an ancestor of main -- it came from the lineage of closed PR #271 --
and the `results/graphbench-*-24c0b1eba` directories it depended on existed on no surviving
branch. The drift gate could not see this: that board sat at drift 4 against a limit of 12.
It also published GraphBench from the four-PR tree while publishing SNB from the shipped one.

Re-vendored from pyg-bench main `e7c49f44`, where GraphBench is now published from the shipped
release tree (pyg-bench #277). All four measured runs sit at compute-drift 0; the remaining
five carry standing waivers. Every vendored run's artifact directory is present on pyg-bench
main, so these numbers are reproducible by anyone.

docs/test_bench_numbers.py: 37 passed, 1 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_012Me1E7ZdDuGqJGu3mMEzhp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant