fix(query): place BIND and FILTER after the triple that binds their variable - #1995

Summary

A BIND or FILTER could run before the triple that binds its variable, whenever that variable was listed in the upstream schema but unbound on some rows. Three things leave a variable in that state: an UNDEF cell in a VALUES table, an unmatched OPTIONAL, and a per-row seed. The planner treated "in the schema" as "bound".

This came from a dashboard report on ccv-claims:

SELECT ?payer ?name WHERE {
  VALUES ?payer { UNDEF }
  ?c ex:payer ?payer ; ex:patientAge ?a .
  FILTER(?a > 0)
  BIND(STR(?payer) AS ?name)
}

• ?name came back unbound.
• A FILTER reading ?payer (for example FILTER(BOUND(?payer))) dropped every row.
• Trigger: a FILTER that pushes a range bound (?a > 0) moves ?c ex:patientAge ?a ahead of ?c ex:payer ?payer. The BIND or FILTER was then inlined into the first join step, before ?payer was bound.
• An unmatched OPTIONAL followed by a triple that binds the same variable failed the same way.
• FLUREE_DISABLE_QUERY_FAST_PATHS returned the same wrong rows, so the bug is in the plan both lanes share.

The same readiness logic had been reimplemented in several places, so this PR also consolidates it. The review turned up two more bugs of the same kind, and both are fixed here.

Changes

One commit per concern, in this order:

1. fix(query): hold BIND and FILTER until the triple that binds their variable has run.
   • Reorder: BIND now waits on its variables' binders, as FILTER/EXISTS already did (attach_filter_binders).
   • Join chain: build_sequential_join_block keeps a BIND or FILTER off a step while a triple not yet joined can still bind one of its variables. This covers inline ops, deferred BINDs/FILTERs and range-semijoin folds.
   • Seeds: Operator::bound_in_every_row reports which variables a seed binds on every row. It is exact for SeedOperator, BatchSeedOperator, BatchReplayOperator and the OPTIONAL MaterializedSeedOperator.
   • UNDEF columns: a VALUES column that is UNDEF in every row is dropped before planning when another pattern binds its variable. VALUES ?x { UNDEF } then plans exactly like the query without it.
2. refactor(query): one must-bind analysis.
   • must_bind_vars(pattern, BindTargets) makes every caller say whether a BIND target counts as bound on every row. An erroring expression leaves it unbound.
   • values_bound_in_every_row / values_column_all_undef replace three copies of the same VALUES column test.
   • collect_guaranteed_vars (which was plain produced_vars) is renamed produced_vars_of.
3. fix(query): OPTIONAL barriers settle only on seed variables bound on every row.
   • SeedVars { schema, bound_in_every_row } feeds the reorder.
   • The VALUES after OPTIONAL cross-products instead of joining (silent wrong results) #1690 VALUES barrier and the Planner hoists a triple above a preceding OPTIONAL that binds a shared variable, returning wrong rows #1924 left-join barrier now use bound_in_every_row instead of the full seed schema.
   • Before, in a per-row UNION/OPTIONAL/EXISTS plan whose seed row left a variable unbound, a triple reading it was hoisted above the OPTIONAL that introduces it, and the left join kept every pair.
4. refactor(query): one readiness type. Settled { bound, unsettled } with admits replaces a pair of loose sets that was threaded through six helpers, plus an inline copy in the range-semijoin fold.
5. fix(query): a property-join star applies every FILTER collected with it.
   • build_property_join_block used to discard the BINDs and FILTERs it could not inline: let (child, _, _) = apply_deferred_patterns(..).
   • On an anchored star of three or more triples, FILTER(?a > 14 || EXISTS {…}) and FILTER(?unbound = 1) were silently dropped, so the query returned the rows the FILTER rejects. This reproduces on main in both SPARQL and JSON-LD.
   • Both block builders now end with apply_all_remaining.
6. fix(query): UNWIND waits for the triple that binds its variable. The reorder places UNWIND by its dependencies, as it does BIND, but UNWIND was missing from waits_for_binders. After a VALUES UNDEF cell, JSON-LD ["unwind", "?x", "(list (str ?payer))"] unwound a null on every row that came through the cell. This reproduces on main.
7. perf(query): prune with a guarded copy of a FILTER held for a later triple.
   • A FILTER that reads a variable a later triple of the block can still bind waits for that triple. A BIND's target is always such a variable, so in ?s ex:num ?num . BIND(IRI(CONCAT(…, STR(?num))) AS ?t) ?t ex:val ?v . FILTER(STRENDS(STR(?t), "77")) every row paid for its ex:val lookup before the FILTER dropped it: about 3× the fuel of the same FILTER written on ?num.
   • When unsettled variables are all that hold a FILTER, and the expression is safe to evaluate twice (expression_is_duplication_safe), the step also runs !BOUND(?v) || … || filter, once per FILTER. A join can't change a variable a row already binds, so the copy drops only rows the FILTER drops. A row where the variable is unbound passes, and the FILTER decides after the triple.
   • This also restores early pruning on the rows that do bind the variable in the VALUES UNDEF, OPTIONAL and per-row-seed shapes.

Behavior changes to be aware of

• BIND targets: in the join chain, a BIND's target no longer counts as bound on every row, whether the BIND is at the top level or nested inside UNION, GRAPH, SERVICE or a subquery. A BIND whose expression errors leaves it unbound, and a later triple of the block can bind it. A BIND or FILTER reading it waits for that triple; a FILTER also prunes early through the guarded copy (change 7). On main, no BIND target was held.
• BIND written before its binder: now sees the bound value, for example BIND(STR(?payer) AS ?name) written before the triple. This matches how BIND was already placed by its dependencies whenever no VALUES was involved.
• Planner sites that still count BIND targets as bound: the reorder's barrier and binder sites still pass BindTargets::Bound, as before. The known BIND-then-OPTIONAL hole documented on must_bind_vars is unchanged. Tightening it changes placement and should get its own tests.
• Opaque seeds: seeds that cannot see their rows return None from bound_in_every_row, for example the default-graph edge-annotation lane. They keep the previous assumption that every schema variable is bound. The annotation lane returns wrong rows because of it, on main too (see Follow-ups).

Query-plan verification

The consolidation commits were meant to be plan-neutral, and the fix commits were meant to change plans only in the broken shapes. I checked this across the test corpus by diffing physical plans between the commit before the consolidation (aa685685f) and this head (f8fe559c5).

Method (temporary instrumentation, not committed):

• An env-gated wrapper around build_where_operators_seeded_with_needed logged every where-plan, including per-row sub-plans: hash(patterns, seed schema, seed bound_in_every_row, needed vars, group-by, dedup flag, required vars) → describe() JSON.
• describe() does not show where BIND/FILTER are inlined, so the patch also added inline_ops to the plan details of NestedLoopJoinOperator, PropertyJoinOperator, ScanDatasetBuilder and BinaryScanOperator, and expr to BindOperator/FilterOperator. None of those files differ between the two commits, so the identical patch applied to both.
• Corpus: the full cargo test -p fluree-db-api plus the testsuite-sparql W3C suite, run twice on the head (to measure noise) and once on the base.

Results:

Limits:

• This shows no unexplained differences; it does not prove none exist.
• About 15% of plan keys can't be compared from run to run: required_where_vars reaches the planner in hash order, and nested sub-plans get filed under different parent keys.
• No test exercises the nested-BIND change above, so its absence from the diff says nothing about it.
• The diff predates changes 6 and 7, and I did not re-run it for them. They change plans only where UNWIND reads a variable a later triple can still bind, and where a FILTER is held by unsettled variables (an extra inline filter at an earlier step).

Tests

New tests. Each one fails without its fix; I checked this by reverting each mechanism in turn, and against main.

• it_values_undef_placement.rs (grp_query_sparql):
  • The reported shape, on a mixed { UNDEF ex:payer1 } table, in novelty and indexed ledgers. It asserts the range-bounded triple drives the chain, so the test cannot pass vacuously.
  • FILTER on the UNDEF variable (BOUND, STRSTARTS).
  • BIND after an unmatched OPTIONAL.
  • A per-row UNION branch.
  • The Planner hoists a triple above a preceding OPTIONAL that binds a shared variable, returning wrong rows #1924 barrier under an unbound seed variable.
  • All-UNDEF VALUES plans like the query without it (plan comparison).
  • JSON-LD twin of the reported shape.
  • A FILTER on a BIND's target, over an index, burns the same fuel as the same FILTER on the BIND's input (1.03 each; 3.01 without the guarded copy).
  • A row whose BIND errored keeps the targets the later triple binds. It fails if the copy is placed without its guard.
  • JSON-LD UNWIND after an UNDEF cell.
• it_property_join_filters.rs (grp_query_sparql): FILTER with EXISTS, and FILTER on a variable nothing binds, on a property-join star. Each asserts the PropertyJoinOperator precondition. Includes a JSON-LD twin.
• Planner unit tests: left_join_barrier_ignores_a_seed_var_unbound_on_some_rows, and values_barrier_ignores_a_seed_var_unbound_on_some_rows, which asserts both directions. Join-chain unit tests: filter_on_an_unsettled_var_runs_a_guarded_copy_once and filter_that_differs_per_evaluation_gets_no_guarded_copy.

Suites, on this head with main merged in (#1966, #1988, #1990):

• cargo test -p fluree-db-query --lib: 1647 passed.
• cargo test -p fluree-db-api: 3,902 passed, 0 failed. it_limit_stops_work::distinct_limit_stops_a_chain_after_one_small_probe is a pre-existing flake that passed this run: it fails 1 in 4 parallel runs with main's query crate, passes in isolation, and its thread-local tracing capture sometimes sees no events.
• Two other tests failed once each under heavy parallel load during the plan runs and passed on rerun: it_annotation_filter_pushdown::annotation_body_threshold_reduces_scan_work_on_both_surfaces and it_bounded_overlay_translation::warm_whole_product_short_circuits_and_respects_epoch. The latter also failed on the base commit.
• testsuite-sparql: green; no register entry changed state.
• cargo fmt --all and cargo clippy --all-targets --no-deps are clean for fluree-db-query and fluree-db-api.

Follow-ups (not in this PR)

• Subquery join keys: self_produced_vars (triples and paths only) and the planner's must_bind_vars decide "does the body bind this on every row" differently. Unifying them changes hash-key selection and needs its own tests.
• Poisoned vs Unbound: the two are treated inconsistently across operators. The Cypher sequential write driver maps Poisoned to Unbound when it threads rows into VALUES (cypher_seq.rs normalize_binding). That is confirmed in code; no wrong answer has been reproduced yet.
• Hand-written variable walkers with gaps: collect_var_stats has no arm for SERVICE or the search adapters, and SPARQL collect_bound_variables omits the GRAPH ?g and annotations.
• Default-graph edge-annotation lane: it plans on top of a child that returns None from bound_in_every_row, so after OPTIONAL { ?a ex:joined ?since }, a FILTER on ?since drops the rows where only ?a ex:knows ?b {| ex:since ?since |} binds it. This predates this PR. Follow-up: FILTER over an OPTIONAL variable drops rows an RDF-star annotation binds #1999
• Test isolation: tests flip FAST_PATHS_DISABLED and call set_var("FLUREE_CYCLIC_BGP") at runtime, which changes plans for other tests running in parallel in the same binary.

Partially addresses #1986: seed variables are now settled only when the seed binds them on every row. Cost-based placement of MINUS/EXISTS/NOT EXISTS remains.