Repository navigation
Investigate using DataFusion #780
Description
Activity
Note that it is also possible to rewrite SHACL to SPARQL and SQL: Compiling SHACL into SQL
Maxime Jakubowski et alHi @Tpt ! Firstly, thanks for Oxigraph!
I just stumbled across this issue. Maybe https://github.com/tobixdev/rdf-fusion can give you some insights. We basically implemented SPARQL on-top of DataFusion.
RDF Fusion started as a fork from Oxigraph and we use a very similar Store API (just with async). So I guess here is already an alignment.
We also support custom storage backends. While we have a backend for in-memory stores, we currently have nothing that persists the data. I can imagine that we could implement a storage backend for Rocks DB that is compatible with existing Oxigraph DBs.
Maybe we could have a discussion with joining our efforts?
Arrow DataFusion is a modular query engine toolkit focused on OLAP use cases.
Oxigraph query evaluation system is currently very primitive, it might be interesting to evaluate if DataFusion can be used as a base for a new query evaluation system, making use of all the work done in DataFusion around optimization and vectorized evaluation.
Some challenges: