New query evaluator using DataFusion - #1527

Merging this PR will degrade performance by 6.15%

❌ 1 regressed benchmark
✅ 15 untouched benchmarks
⏩ 78 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

_{Comparing fusion (f0d6c84) with main (27ee297)²}

Wow @Tpt this looks amazing. This looks like it was an enormous amount of work.
Do you think this will (eventually) become the default query evaluator?
Are there any downsides, eg, particularly with regard to DataFusion using Apache Arrow in-memory structures, and the whole Oxigraph->Arrow conversion overhead?

@ashleysommer Thank you! Was much easier than the JSON-LD streaming parser (I have not finished it...). It currently has a few downsides:

• much larger dependency footprint
• currently much slower on "BSBM explore 5000 query in memory" it takes 150s vs <1s. My guess is that having better join reordering and use for loop joins where it makes sense might help a lot
• the conversion to Arrow is indeed very costly. I am considering writing a new storage backend on top of Parquet, likely for read-only workloads first
• EXISTS support is currently very spotty and it will require significant work to get it done properly

Some missing elements in this MR:

• RocksDB iter is quite slow, making large reads not very good. Tweaking RocksDB evaluation to do less hash/merge joins and more "for loop" joins is needed to keep the same performances on top of RocksDB (and nudges more toward rewriting the storage to be more read-friendly at the cost of maybe being less write-friendly)
• EXISTS and LATERAL are written as correlated queries. Sadly DataFusion does not know how to evaluate them without decorating the queries first. We should either change the emitted logical plan in something DataFusion is able to decorellate, write a better decorrelation algorithm or add correlated evaluation to DataFusion (this is sorted by personal order of preference).