Repository navigation
embedding api improvements - #2249
Merged
Merged
Conversation
# Conflicts: # Cargo.lock # Cargo.toml # raphtory-graphql/schema.graphql # raphtory-graphql/src/python/server/server.rs # raphtory-graphql/src/server.rs # raphtory/Cargo.toml
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'GraphQL Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: d31223c | Previous: 4707062 | Ratio |
|---|---|---|---|
addNode |
20 req/s |
1302 req/s |
65.10 |
This comment was automatically generated by workflow using github-action-benchmark.
ricopinazo
force-pushed
the
embedding-api
branch
from
February 4, 2026 12:34
38606ee to
a706b46
Compare
ricopinazo
force-pushed
the
embedding-api
branch
from
February 4, 2026 15:05
204a599 to
efba505
Compare
ricopinazo
commented
Feb 4, 2026
ricopinazo
force-pushed
the
embedding-api
branch
from
February 4, 2026 18:12
c5de4cc to
d11e49a
Compare
# Conflicts: # raphtory-graphql/src/main.rs
fabianmurariu
left a comment
fabianmurariu
left a comment
Collaborator
There was a problem hiding this comment.
Some minor changes before we merge this in?
Contributor
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rust Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.
| Benchmark suite | Current: d31223c | Previous: 4707062 | Ratio |
|---|---|---|---|
lotr_graph/num_nodes |
2 ns/iter (± 0) |
0 ns/iter (± 0) |
+∞ |
This comment was automatically generated by workflow using github-action-benchmark.
ricopinazo
commented
Feb 10, 2026
miratepuffin
added a commit
that referenced
this pull request
Apr 16, 2026
* embedding api improvements (#2249) * embedding api improvements * disable embeddings by default * fix compilation error for enable_embeddings * fix compilation error in main.rs * sort milvus integration * lancedb vector storage implementation and rename score to distance * like this almost compiles but not yet * force lancedb to work with other version of chrono but still polars-arrow complaining * this seems to compile with all features but storage * put pometry-storage back into place * sort new multi-embedding vector cache * still some rust tests failing * remove outdated comment * all rust tests now passing with the new custom openai server * some compilation errors caused by teh server future not being Sync * fixing some python tests * wip * trying to avoid the drop of the tempdir but still not working * fix compilation error * fix bug caused by a temp dir being dropped too soon * fix python tests * fix dependency conflicts * format * fix rust test * change rust version * started implementing context manager for PyEmbeddingServer * context manager for embedding server * all graphql vector tests are passing now * re-indexing, graphql vectorise, and minor fixes * big cleanup * update Cargo.lock * handle all unwraps in lancedb.rs * fix cache tests * fix benchmark compilation error * fix python graphql vector tests * fix more graphql tests * this should fix all the compilation errors on the tests * make vector cache lazy * fix rust doc tests * fix compilation error on a test and expose reindex api * handle python embedding errors * try a different way of making the vector cache lazy * rely only on id and actual vector, not distance, for lancedb index test * fix mutable graph test * remove unused comments * try different ports for mutable_graph tests * fix test_server_start_with_failing_embedding test * remove unused imports * disable all features for lancedb * tune cargo in rust test action * try to fix cargo in the python test action as well * fix python tests * disable cargo tuning for python tests * try to avoid the no space left on devide when running the graphql bench * add df -h in all steps * try removing target a bit earlier * replace one-off vectors with random smaller ones * apply same fix no space left on device for stress test * chore: apply tidy-public auto-fixes * chore: apply tidy-public auto-fixes * remove outdated comments * change custom embedding server api * remove outdated comment * change Embedding to be Float32Array removes some needless iter and copy * implement vectorise_all_graphs in python * fix insertion bug cause by missing append call * change imports to satisfy cargo check * address some of the comments * remove one unwrap out of closure in lancedb.rs * replace collect appropriately * wrap raphtory access with blocking_compute * revert back ci changes * remove unused prints on python tests * put back CARGO_BUILD_JOBS: 2 * chore: apply tidy-public auto-fixes * changes to caching * fixes for tokio * fixes for stable embedding cache * remove printlns * remove useless print and extract vector cache * fix the python tests * more python tests * chore: apply tidy-public auto-fixes * minor changes before merge --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Fabian Murariu <[email protected]> Co-authored-by: Ben Steer <[email protected]> * fix auto release action (#2498) * forbid weird characters on graph names or namespaces (#2496) * forbid weird characters on graph names or namespaces * only disable hidden paths and improve tests * Update UI to c8c6dbdf4 (v0.2.1) (#2505) Update UI to v0.2.1 Co-authored-by: rachchan <[email protected]> * GenericNodeState (#2342) * add GenericNodeState and TypedNodeState; add python-facing OutputNodeState; convert degree centrality to use new nodestate; add conversion method to Prop * add python iterators to output_node_state * cleanup and to parquet for GenericNodeState * add id column when writing NodeState to parquet * in progress nodestate merging * checkpoint of nodestate functionality * update from master * update algorithm signatures * update label prop and update algorithm docstrings * convert lcc batch to use new nodestate * convert primitive returns types to structs for serialization to nodestate * lcc should pick all nodes if no nodes provided * lcc bugfix * formatting * update node op and lazy nodestate to return new nodestate * re-add deserialize requirement for generic node state types * learning to spell * convert all algos to return new node state * algorithms all compile * update all algo tests; fix labelprop * wrangle nasty rebase * rewrite prop conversion; sorting out python tests had claude rewrite some of the prop conversion with the num_traits crate. Reviewing. * checkpoint; playing with function pointers * nodestates work again * add GenericNodeState and TypedNodeState; add python-facing OutputNodeState; convert degree centrality to use new nodestate; add conversion method to Prop * add python iterators to output_node_state * cleanup and to parquet for GenericNodeState * add id column when writing NodeState to parquet * in progress nodestate merging * checkpoint of nodestate functionality * update from master * update algorithm signatures * update label prop and update algorithm docstrings * convert lcc batch to use new nodestate * convert primitive returns types to structs for serialization to nodestate * lcc should pick all nodes if no nodes provided * lcc bugfix * formatting * update node op and lazy nodestate to return new nodestate * re-add deserialize requirement for generic node state types * learning to spell * convert all algos to return new node state * algorithms all compile * update all algo tests; fix labelprop * wrangle nasty rebase * rewrite prop conversion; sorting out python tests had claude rewrite some of the prop conversion with the num_traits crate. Reviewing. * checkpoint; playing with function pointers * nodestates work again * sorting out tests * cleanup and bugfixing * getting mapped nodestates to work in python * update python deserialization * rust tests for algorithms pass now * cleanup * all python tests updated; cleanup * ensure node cols gets merged correctly; cleaning up merge; starting from_parquet * add from_parquet * cleanup * add default values to merge * address final corner case for from_parquet * add from parquet to output nodestate * add some notes for docstrings * documentation; cleanup; bugfixes * cleanup; resolve component filtering merge; fix partialeq for lazynodestate and nodestate * fix failing tests * fix bug with making genericnodestates from subgraphs; cleanup * resync with master * eliminate terminal return char * cleanup and remove unused imports * fix issue with fp precision in typednodestate deserialization; cleanup * adjust precision of python algo tests; cleanup * serde untagged prop wrapper for typednodestate * custom deserialize implementation for PropUntagged * remove unneeded import from python algo test; add conditional compilation to nodestate block * cleanup * add sort, top_k, and groups to typednodestate * cleanup * cleanup * fixing doc examples * doc example fix * switch all errors to GraphErrors I'm just using IOErrorMsg for now * chore: apply tidy-public auto-fixes * cleanup; update outputnodestate docstrings * chore: apply tidy-public auto-fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Ben Steer <[email protected]> * Features/nodestatefilter (#2483) * add GenericNodeState and TypedNodeState; add python-facing OutputNodeState; convert degree centrality to use new nodestate; add conversion method to Prop * add python iterators to output_node_state * cleanup and to parquet for GenericNodeState * add id column when writing NodeState to parquet * in progress nodestate merging * checkpoint of nodestate functionality * update from master * update algorithm signatures * update label prop and update algorithm docstrings * convert lcc batch to use new nodestate * convert primitive returns types to structs for serialization to nodestate * lcc should pick all nodes if no nodes provided * lcc bugfix * formatting * update node op and lazy nodestate to return new nodestate * re-add deserialize requirement for generic node state types * learning to spell * convert all algos to return new node state * algorithms all compile * update all algo tests; fix labelprop * wrangle nasty rebase * rewrite prop conversion; sorting out python tests had claude rewrite some of the prop conversion with the num_traits crate. Reviewing. * checkpoint; playing with function pointers * nodestates work again * add GenericNodeState and TypedNodeState; add python-facing OutputNodeState; convert degree centrality to use new nodestate; add conversion method to Prop * add python iterators to output_node_state * cleanup and to parquet for GenericNodeState * add id column when writing NodeState to parquet * in progress nodestate merging * checkpoint of nodestate functionality * update from master * update algorithm signatures * update label prop and update algorithm docstrings * convert lcc batch to use new nodestate * convert primitive returns types to structs for serialization to nodestate * lcc should pick all nodes if no nodes provided * lcc bugfix * formatting * update node op and lazy nodestate to return new nodestate * re-add deserialize requirement for generic node state types * learning to spell * convert all algos to return new node state * algorithms all compile * update all algo tests; fix labelprop * wrangle nasty rebase * rewrite prop conversion; sorting out python tests had claude rewrite some of the prop conversion with the num_traits crate. Reviewing. * checkpoint; playing with function pointers * nodestates work again * sorting out tests * cleanup and bugfixing * getting mapped nodestates to work in python * update python deserialization * rust tests for algorithms pass now * cleanup * all python tests updated; cleanup * ensure node cols gets merged correctly; cleaning up merge; starting from_parquet * add from_parquet * cleanup * add default values to merge * address final corner case for from_parquet * add from parquet to output nodestate * add some notes for docstrings * documentation; cleanup; bugfixes * cleanup; resolve component filtering merge; fix partialeq for lazynodestate and nodestate * fix failing tests * fix bug with making genericnodestates from subgraphs; cleanup * resync with master * eliminate terminal return char * cleanup and remove unused imports * fix issue with fp precision in typednodestate deserialization; cleanup * adjust precision of python algo tests; cleanup * serde untagged prop wrapper for typednodestate * impl node state filter, add test * add review suggestions * custom deserialize implementation for PropUntagged * remove unneeded import from python algo test; add conditional compilation to nodestate block * cleanup * impl py * fix toml * fix grapherror * chore: apply tidy-public auto-fixes --------- Co-authored-by: wyatt-joyner-pometry <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Update UI to a3d8781b1 (v0.3.0) (#2510) Update UI to v0.3.0 Co-authored-by: ricopinazo <[email protected]> * Update UI to 9ddf8253e (v0.3.0) (#2514) Update UI to v0.3.0 Co-authored-by: ricopinazo <[email protected]> * Release v0.18.0 (#2519) chore: Release Co-authored-by: Pometry-Team <[email protected]> * Fix rust build on release action (#2518) * Fix rust build on release action * add toolchain file * set up rust 1.91.0 inside the linu docker container * hardcode the rust version on the python linux build * try using rustup override set * remove wrong override set stable-msvc * add build target for arm * try just using maturin rust-toolchain option --------- Co-authored-by: Ben Steer <[email protected]> * Fix all current dependency issues within graph bench and raphtory (#2512) * updated graphql-bench * cargo.toml * update rust version * fixed package * fixing * working auth code * 91.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.191.1 * chore: apply tidy-public auto-fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * 404 in readme * Create dependabot.yml * fix merge * update lock file * update test to not depend on node order * maybe not getting latest xcode helps... * bump dependencies * add basic optimisations for build-fast to shrink file sizes * fix features for jsonwebtoken * fix algorithms test * restore lock file * fix versions * add LayerId everywhere * update version * make display for LayerId transparent * bring back missing import * add basic optimisations * fmt * remove profile * fix nodes output in node state * clean up unused code * make nodestate merge actually useable from rust and add test * fix nodestate merge * clean up comment * fix degree centrality for graph without edges * fix doc string * fix python node state compilation * make sure test is independent of node order * chore: apply tidy-public auto-fixes --------- Co-authored-by: Pedro Rico Pinazo <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Fabian Murariu <[email protected]> Co-authored-by: Ben Steer <[email protected]> Co-authored-by: rachchan <[email protected]> Co-authored-by: wyatt-joyner-pometry <[email protected]> Co-authored-by: Shivam <[email protected]> Co-authored-by: ricopinazo <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Pometry-Team <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change? If yes is this documented?
How was this patch tested?
Are there any further changes required?