UMAP upload sends incorrect edge bindings after creating hypergraph

Environment

• PyGraphistry version: 0.45.0
• Python version: 3.10+
• Installation: pip install graphistry[all]==0.45.0

Summary

After running UMAP via .gfql(), the resulting Plottable correctly updates internal bindings (_source, _destination) to match the new hypergraph columns (_src_implicit, _dst_implicit), but when calling .upload(), the serialized metadata contains the original bindings instead of the updated ones. This causes server-side validation to fail because the bindings reference columns that no longer exist.

Impact

• ❌ Cannot upload UMAP results to Graphistry server
• ❌ Breaks remote GFQL workflows that persist results
• ❌ Affects any shape-changing operation that creates new edge columns

Steps to Reproduce

import pandas as pd
import graphistry
from graphistry import call

# 1. Create a graph with explicit bindings
nodes_df = pd.DataFrame({
    'id': range(20),
    'col1': range(20),
    'col2': range(20, 40),
    'col3': range(40, 60),
})

edges_df = pd.DataFrame({
    'src': [0, 1, 2, 3, 4] * 4,
    'dst': [1, 2, 3, 4, 5] * 4,
})

g = graphistry.edges(edges_df, 'src', 'dst').nodes(nodes_df, 'id')

print(f"Before UMAP - bindings: {g._source}, {g._destination}")
print(f"Before UMAP - columns: {list(g._edges.columns)}")
# Output: bindings: src, dst
# Output: columns: ['src', 'dst']

# 2. Run UMAP to create hypergraph
g_umap = g.gfql([
    call('umap', {
        'n_neighbors': 5,
        'X': ['col1', 'col2', 'col3']
    })
], engine='pandas')

print(f"\nAfter UMAP - bindings: {g_umap._source}, {g_umap._destination}")
print(f"After UMAP - columns: {list(g_umap._edges.columns)}")
# Output: bindings: _src_implicit, _dst_implicit  ✓ CORRECT
# Output: columns: ['_src_implicit', '_dst_implicit', '_weight']  ✓ CORRECT

# 3. Try to upload (requires configured server)
graphistry.register(api=3, server='https://hub.graphistry.com', token='YOUR_TOKEN')
result = g_umap.upload()
# ❌ FAILS: Server validation error
# "Edge binding 'source' points to a column named 'src' that does not exist"
# Available columns: ['_src_implicit', '_dst_implicit', '_weight']

Expected Behavior

After UMAP creates a hypergraph, .upload() should serialize the current bindings:

{
  "edge_encodings": {
    "bindings": {
      "source": "_src_implicit",
      "destination": "_dst_implicit"
    }
  }
}

Actual Behavior

.upload() serializes the original bindings:

{
  "edge_encodings": {
    "bindings": {
      "source": "src",
      "destination": "dst"
    }
  }
}

This causes server validation to fail because columns src and dst don't exist in the DataFrame.

Evidence from Production

We confirmed this bug by inspecting actual uploaded data on a Graphistry server:

Uploaded edge data (processed/edges_parquet):

['_src_implicit', '_dst_implicit', '_weight']  # 286 edges

Uploaded metadata (raw/dataset.json):

{
  "edge_encodings": {
    "bindings": {
      "source": "src",
      "destination": "dst"
    }
  }
}

Server error log:

ValueError: Edge binding 'source' points to a column named 'src' that does not exist,
is there a typo in column name?
Available columns: Index(['_src_implicit', '_dst_implicit', '_weight'], dtype='object')

Root Cause Analysis

The bug is in the upload serialization code. While UMAP correctly updates the Plottable's internal state (._source, ._destination), the .upload() method appears to serialize stored/cached metadata rather than reading the current bindings from the Plottable instance.

What works ✓:

• UMAP correctly updates g._source = '_src_implicit'
• UMAP correctly updates g._destination = '_dst_implicit'
• DataFrame has correct columns matching updated bindings

What's broken ✗:

• .upload() serializes old bindings from somewhere (cached metadata?)
• Server receives mismatched bindings and columns
• Validation correctly fails

Proposed Fix

The upload serialization should use the current binding attributes:

# In Plottable.upload() or serialization helper
def _get_edge_encodings_for_upload(self):
    return {
        'bindings': {
            'source': self._source,      # Use CURRENT binding
            'destination': self._destination  # Use CURRENT binding
        }
    }

Not from cached/stored metadata.

Workaround

For Graphistry server deployments, we added a server-side workaround to detect and fix binding mismatches before validation:

def persist_dataset(g: Plottable) -> str:
    # Auto-detect and fix UMAP implicit bindings
    if g._edges is not None:
        edge_cols = list(g._edges.columns)
        if '_src_implicit' in edge_cols and '_dst_implicit' in edge_cols:
            if g._source != '_src_implicit':
                # Fix bindings before upload
                g = g.edges(g._edges, '_src_implicit', '_dst_implicit')

    return g.upload(memoize=False)

But this only helps server-side workflows, not client-side uploads.

Additional Context

• This affects UMAP and likely other shape-changing operations
• Issue exists in PyGraphistry v0.45.0 (latest)
• Reproducible in both CPU (pandas) and GPU (cudf) modes
• Server-side validation is working correctly - the bug is in PyGraphistry upload

Related

• Similar to issue where shape-changing operations might not preserve metadata
• May affect other operations that create new column names (hypergraph, custom transforms, etc.)