fix(transact): retractions name stored facts; RDF text is parsed once - #2008

…tion LRU alone

binary_range_eq_v3 sent every probe through the cross-call translation
LRU when the overlay reported a content version, and NoOverlay reports
one. Staging reads the persisted index through NoOverlay once per
retraction group, and the LRU key includes to_t, so each commit's first
such probe per (graph, index order) inserted an empty entry into an
eight-slot cache and could evict a live query's novelty translation.
The translation step moves into probe_overlay, which returns an empty
translation for an effectively empty overlay before touching the cache.

A retraction staged for commit must name a fact that is currently
stored. Today staging rebuilds retractions from query bindings, templates
and edge keys, which drops language tags and list positions and invents
retractions of facts that do not exist. This adds the read those sites
will share: per (graph, subject, predicate) slot, one seek into the
persisted index through NoOverlay plus a Sid-space novelty seek bounded
by the slot's sentinels, lifecycle-resolved, graph stamped. A graph whose
novelty is small next to the slots asked of it is walked once instead.

StoredFact (private fields, built only here) and Retraction (private
field, built from StoredFact::retract) carry the invariant in the types;
the next commits route staging through them.

The upsert wave queried each (graph, subject, predicate) with
execute_pattern and rebuilt every retraction from the ?o binding with
m: None. A language-tagged value was retracted without its tag and a list
entry without its position, so neither retraction matched anything: the
old values stayed, and an identical refresh of such values committed
two flakes every time (#1976). A named graph without a range provider
also walked the graph's whole novelty once per slot.

The wave now asks CurrentFacts for each slot's current facts and retracts
them as stored. The #1549 absence pre-check and its counters are
unchanged; pattern_queries now counts slot reads. binding_to_flake_object,
query_novelty_for_graph and the per-group materializer are gone.

The replacement unit is one function, upsert_replacement_slots: every
current value of each (graph, subject, predicate) the payload names, all
languages and list positions together. Blank subjects join the wave when
the skolem scope is deterministic (the upsert payload scope), so an
identical re-upsert of a document with blank nodes is now a no-op too.

…rrentFacts

The whole-graph scan moves from scan_graph_flakes to
CurrentFacts::whole_graph, which returns StoredFacts with the graph
stamped. Graph sync, CLEAR/DROP and the COPY/MOVE destination and source
clears now build their retractions with StoredFact::retract instead of
flipping op and t on scanned flakes. Behavior is unchanged: the scan,
its limit, the stamping and the policy model are the same.

A DELETE template instantiated into a retraction of a fact the ledger does
not hold used to be committed anyway. DELETE DATA of an absent triple
committed a new t with one retraction, which docs/transactions/retractions.md
says is a no-op. Worse, DELETE {x} INSERT {x} WHERE {} with x absent
committed nothing and never wrote x: the phantom retraction cancelled the
insert in the accumulator, against SPARQL 1.1 Update's (DS - D) ∪ I. Both
twins (SPARQL and JSON-LD) behaved the same.

Every DELETE instantiation is now a RetractIntent, and the accumulator
accepts only Retraction, which CurrentFacts produces:
- An intent whose template a WHERE triple witnesses (delete_witness: same
  subject, predicate and object variable, the object bound nowhere else,
  the same graph) is the decode of a fact the WHERE just matched. On a
  ledger that holds no @list position (ListFree) it retracts as decoded with
  no read, which keeps bulk filtered deletes at zero lookups. On a
  list-bearing ledger it reads its slot for the list position, under the
  rule hydration had: the first stored list entry with the same term.
- Every other intent (template constants, DELETE DATA, values bound by
  another pattern) is matched against the stored facts of its slot: exact
  datatype (#1990's rule, one function), case-insensitive language tag, the
  named list position or else the first list entry, else the plain value.
  An intent that names no stored fact stages nothing.

hydrate_list_index_meta_for_retractions is gone, and so is the unused
generate::cancellation module, which pushed arbitrary flakes as
retractions. The delete_witnessed routing stamp reports which lane ran.

The it_fast_stats_1391 fixture relied on a DELETE writing a no-op
retraction; it now hand-builds that commit, the shape legacy and replayed
commits still carry.

…r graph

The cascade that follows a retracted edge to its annotations rebuilt each
f:reifies* bundle from the decoded EdgeKey, and read annotation subjects
with range_with_overlay(&novelty). Its body-cleanup pass (a by-id
annotation delete in LPG mode) did not stamp the graph on index-resident
rows, so the body's retraction landed in the default graph and the body
survived in its named graph. And nothing retracted the bundle's
f:reifiesGraph anchor, which the by-id delete does not name, so it stayed
behind describing nothing.

All three passes now read annotation subjects with CurrentFacts::of_subject,
graph stamped, and retract bundles and bodies as stored. A complete by-id
bundle retract also retracts every other stored slot of that bundle,
unless the transaction re-points the reifier.

A take-source merge or revert, and a take-branch rebase, retract the
losing side's current values on each conflicting (subject, predicate,
graph). current_asserted_for_key read them with a range read and kept the
rows whose graph equalled the key's, but index-resident rows come back
without their graph, so in a named graph every indexed value was dropped:
the take-source merge left the target's value next to the source's, and
merge_preview showed no target value. Novelty-resident values and the
default graph were unaffected.

current_asserted_for_key now reads the slot through CurrentFacts, which
stamps the graph, and the retractions come from StoredFact::retract.

Upserts written before the retraction resolver retracted "a"@en as a
tagless "a"^^rdf:langString. That retraction matched nothing and is inert
for reads and indexing, but a revert inverts every flake of the reverted
commits, and inverting it asserted "a" as an rdf:langString with no language
tag (LANG ""), a literal that cannot exist.

The undo fold now skips any retraction whose shape no stored fact has: an
rdf:langString without a tag, or a tag on another datatype. It logs how
many it skipped. Legacy retractions of list entries without their position
are well-formed plain literals, so no shape check can catch them; telling
those apart needs a historical stored-fact read and is left as a follow-up.

…only stored facts

upsert.md: the replacement unit is the whole (graph, subject, predicate), all
languages and list positions together, and an identical refresh commits
nothing, blank nodes included. The "Empty Replacement" and comparison-table
text said an upsert removes predicates the payload does not name; it never
did, and the page's own summary said so. retractions.md: a DELETE of a triple
that is not stored commits nothing, names terms exactly (datatype exact,
language tag case-insensitive), and a DELETE/INSERT of an absent triple
inserts it. update-where-delete-insert.md: the same, and the comparison
table's upsert granularity.

With #1988, SPARQL UPDATE lowers `"1"^^xsd:int`, `xsd:long` and
`xsd:dateTime` literals as JSON-LD does, so a DELETE DATA intent names the
stored term exactly and the retraction resolver deletes it, over novelty
and over an index. Before #1988 the intent named a string, and the delete
removed nothing.

The persisted index keeps less than some terms carry. An integer too large
for i64 is keyed by value alone (NUM_BIG), with no XSD subtype, and a
dateTime or time is keyed to the microsecond. So an indexed fact's decode
can differ from the term that names it. The index decode says xsd:integer
where a big xsd:nonNegativeInteger was written, a query row says
xsd:decimal, and a nanosecond literal decodes to its microsecond.

Matching by term alone therefore missed these facts on an indexed ledger.
DELETE DATA, a JSON-LD delete or a constant DELETE template naming a big
integer under its subtype committed nothing. So did a DELETE row whose
value an OPTIONAL, UNION or BIND bound. Cypher SET kept the old big value
next to the new one, DETACH DELETE left it behind, and a sub-microsecond
dateTime or time survived DELETE DATA. main deleted each of these, because
its retraction carried the same index key.

A fact read from the index is now also named by its index key: the
intent's object encoded as the index keys it (o_type + o_key), compared
with the fact's. Terms with one key are one fact to the index, which
merges every retraction by that key. The resolver still retracts the fact
as read, so every retraction names a stored fact. Term identity stays
first, and novelty facts keep it alone. The key comparison runs only
when no stored term matches, so a DELETE of a present term costs what it
did. The witnessed-row list lookup uses the same rule.

BinaryRangeProvider::persisted_object_key exposes the encoding the
overlay merge already applies to retractions.

A DELETE intent that names no stored fact stages nothing. Under a policy,
that also kept it from the modify-policy check, which only saw the staged
flakes. A delete the identity may not perform was then refused when its
target was stored and committed nothing when it was not.

Intents that name no stored fact now go through the modify-policy check
with the staged flakes, and are still dropped from the commit. A delete
the identity may not perform is refused the same way whether or not its
target is stored, as it was before intents were resolved against storage.
A permitted delete of an absent value still commits nothing. Only a
non-root policy context collects the unmatched intents.

…L and UNION rows included

Two changes to the rule that lets a DELETE row retract without a read.

Graphs compare as ledger graph ids. The rule used to compare the WHERE
default's IRI with the template's IRI, and a dataset alias by name. It now
takes the graph ids the WHERE dataset actually reads: its default graph
when that is one ledger graph (the ledger's own with no USING, WITH or
from, else the one graph those name resolves to, where the ledger's own
address is its default graph), and each GRAPH <name>'s graph. It compares
them with the graph the ledger has under the template's IRI. A name the
WHERE resolves to one graph and a template writes to another is never a
witness. USING or WITH naming the ledger's own address reads the default
graph, while a GRAPH <address> template writes a graph registered under
that name. A test covers a ledger holding such a graph.

OPTIONAL and UNION rows can witness. A triple that is the only pattern of
a top-level OPTIONAL, or of one branch of a top-level UNION (or of a UNION
an OPTIONAL holds alone), witnesses a template whose object variable
nothing else binds. A row that binds that variable is the decode of the
fact the triple matched, and a row that does not emits no intent. These
rows, which include Cypher SET and the outbound half of DETACH DELETE,
now retract without reading every slot. A UNION whose other branch binds
the object from another predicate still goes through the resolver.

…h less detail

An identical re-upsert commits nothing, except for a value stored with less
detail than it was written: an integer beyond 64 bits written with an XSD
subtype, or a dateTime or time with digits past the microsecond. Re-upserting
such a value as written commits a retraction and an assertion of the same
stored fact, and the value can be lost. This is not new in this branch. The
upsert page now says so and says how to write such values. A second ignored
U5 test pins the temporal case next to the big-integer one.

Also:
- A test pins that a refresh upsert of every language's label satisfies
  sh:uniqueLang, and that a second label in a language is still refused.
- The of_slots doc no longer claims duplicate slots are read once.
- A stale comment about SPARQL typed literals is removed.

Adds three columns over the existing people data: upsert_turtle (the
Turtle bodies upserted into a fresh ledger), upsert_turtle_replace (the
same upserts over a ledger that already holds them, so every subject's
values are read and restated), and trig_mixed (a TriG insert of each body
with its second half in a trailing graph block, the documents the
streaming Turtle parse stops on). These are the RDF-text lanes the next
commits move off the JSON-LD round trip; this records them first.

The locator finds each graph block's label, as written, and its extent by
reading tokens only. It never interprets a term, so it cannot disagree
with the parser about what the text means: a brace or the word "graph"
inside a literal is not a block. It blanks the block syntax in place, so
every segment it returns is plain Turtle at the document's own byte
offsets, and a parse error inside a block can report where it is in the
document.

It names the construct it refuses (a nested block, a directive inside a
block, a blank-node label, an unclosed block, a stray `}`) as a Turtle
parse error at the offending token. `has_graph_blocks` exposes it for
callers that must tell TriG from Turtle before choosing a lane.

`parse_rdf_text` reads Turtle and TriG with the conformant parser into one
`TemplateSink`, for every verb that writes RDF text. Plain Turtle takes a
single parse. A TriG document is parsed segment by segment in document
order from the locator's segments, each seeded with the prefix and base
declarations made before it: a redefinition applies to what follows it and
to nothing before it. A block label resolves through the parser, under the
declarations in force where the block appears; parse errors report the
document's byte offset.

The sink converts literals exactly as `FlakeSink` does (lenient: an
ill-typed lexical form is kept with its declared datatype), keeps list
positions, takes blank-node and literal `rdf:type` objects like any other
predicate's, and applies the reserved-predicate firewall. Blank-node labels
are document-scoped across default statements and blocks; anonymous nodes
get fresh `-b{N}` labels. RDF 1.2 annotations become the `f:reifies*`
bundle through `bundle_templates`, which states no `f:reifiesGraph`: the
graph scope that places a template in a named graph adds the anchor, here
through a local stand-in with the contract of the graph-scope emitter.

`<#txn-meta>` (with no base in force) becomes commit metadata with the
refusals the TriG txn-meta parser applies. `content_id` is a commutative
sum of per-statement 128-bit hashes, so it ignores statement order;
`parse_rdf_text_txn` derives the upsert blank-node skolem scope from it.
`Placement::Into` homes every statement in one graph for graph-scoped
writes and refuses a block for another graph or a mix of blocks and
default-graph statements, naming the graph.

Turtle and TriG upserts were parsed into a graph, re-encoded as JSON-LD and
expanded again, and TriG blocks went through a second parser. The round
trip refused IRIs whose scheme the compact-IRI guard took for an undefined
prefix (`tag:`, `kb:`), stored collections as unordered values, refused an
annotated `rdf:type` edge, and parsed literals more strictly than insert.

Every upsert entry point (`Fluree::upsert_turtle*`, the owned and cached
builders with or without a policy, the graph builder) now stages from
`parse_rdf_text_txn`, parsed against the state it is staged on:
`OpPlan::Rdf` carries the text through the cached-handle retry loop. The
upsert stores what a Turtle insert of the same document stores, blank
nodes are scoped to the document's content, the raw transaction is the
text, and a parse error is a Turtle parse error at the document's offset.
`Fluree::upsert_turtle` now reads TriG as the builders do.

Fuel is charged as for any upsert: the baseline plus one unit per staged
flake, reported in the tally on every builder lane.

A document the streaming Turtle parser stops on is read as TriG when the
locator finds a graph block in it (`<#txn-meta>` included), and is then
parsed once into templates, as a TriG upsert is, and staged with insert
semantics. It used to go through the phase-1 rewrite and the JSON-LD round
trip, so a TriG insert refused `tag:`/`kb:` IRIs and applied a prefix or
base redefinition to the statements before it as well as after. Plain
Turtle still streams to flakes, and a document with no graph block keeps
its Turtle error; a malformed block is reported as one, by the locator.

The routing stamp is unchanged: `turtle_insert` proceeds for plain Turtle
and falls back for TriG. The directive-order cases are the ones the
phase-1 fix pins, for both lanes.

A graph-scoped RDF body (graph sync, the graph store routes) had its TriG
blocks unwrapped and the whole text converted to JSON-LD, so it refused
`tag:`/`kb:` IRIs and stored a collection as unordered values. It is now
parsed once into templates homed on the target graph
(`Placement::Into`): a block must name the target, a body holds the
target's statements either in blocks or as default-graph statements, and
the refusals keep their messages and their 400. An empty body is refused
as before unless a sync confirms it.

Sync keeps its graph-scoped blank-node scope, and the parse labels blank
nodes as the JSON-LD conversion did, so a graph last synced through the
conversion re-syncs from the same text without a commit.

With every RDF-text lane staging from the one parse, nothing reads a TriG
document as JSON-LD any more. `convert_named_graphs_to_templates`, the
trig-meta and named-graph staging variants (`stage_transaction_with_*`,
`transact_with_*`, now `stage_transaction_tracked` for JSON-LD), the block
hashing in `upsert_payload_id`, `extract_trig_txn_meta` and
`unwrap_trig_graph_blocks` go. The TriG reader stays for bulk import only
(phase 1 plus `resolve_trig_meta`); its tests now read documents that way,
so they keep covering the import lane.

What the deleted converter pinned is pinned on the parse: a hand-written
`f:reifies*` statement in a block is refused as a transaction error, an
undefined prefix in a block is a Turtle parse error, and a stable `_:fdb-`
id in a block addresses the stored node.

`parse_to_json` is documented as lossy, and the transact and api crates
disallow it in their clippy.toml, so no transaction lane can go back to it
without saying why.

`fluree sync` converted Turtle to JSON-LD client-side, so a Turtle export
lost collection order and refused `tag:`/`kb:` IRIs before it reached the
server. RDF text is now sent as written: locally as a `GraphPayload::Rdf`,
remotely as `text/turtle` (or `application/trig` for a body with graph
blocks), and parsed once where it is staged. Only a server from before
`/sync` read RDF bodies answers 415, before staging anything; for that one
a Turtle body is converted to JSON-LD, as older CLIs did, and sent again.
TriG has no JSON-LD form for its blocks and is not converted.

TriG detection (`is_trig_body`, used by sync, `validate` and `--shacl`) is
the locator's: a token pass, so a block holding `[ … ]` or a collection is
TriG, not "not TriG" as the old content parse made it.

The harness loads named-graph data as TriG through `upsert_turtle`, and
the five eval-triple-terms tests registered as "blocked on TriG
GRAPH-block parsing" now load their data. None of them passes yet; each is
re-attributed to the failure it now reaches: graphs-1/2 to the non-
isomorphic results of the reifier model, expr-1 to the unlowered
triple-term functions, update-1 to quoted triples in SPARQL UPDATE
lowering, and update-2 to an annotation valued by a `GRAPH ?g` binding.
The harness notes no longer describe the phase-1 reader.

turtle.md describes how every transaction endpoint now reads Turtle and
TriG: directives in document order, relative IRIs against the base in
force (an error without one), literals kept as written, any IRI scheme,
collections in order, document-scoped blank-node labels, errors at the
document's offset. It narrows the #1930 limitation to bulk import, and
documents the upsert blank-node scope with a query that lists blank nodes
nothing references, pinned by a test.

The ledger-config recipe upserts as written again (tested); `fluree sync`
docs describe RDF bodies and the 415 fallback; edge-annotations no longer
lists upsert and sync among the paths that refuse an annotated
`rdf:type` edge; the compact-IRI troubleshooting entry notes that Turtle
and TriG never raise it.

The TriG insert fallback asked the locator whether the document has a
graph block and then parsed it, which located it again: two token passes
over every TriG insert. `parse_trig_txn` locates once and returns `None`
for a document with no block, so the streaming parser's Turtle error
stands as before. The routing stamp now fires once the document parses as
TriG.

Cross-ledger configuration names its model ledger by id, as an IRI
reference in the config graph's block (`f:ledger <org/governance:main>`,
the form docs/security/cross-ledger-policy.md uses), and an id with a `/`
before its `:` is a relative reference. The TriG block reader kept such a
reference as written when no `@base` was in force; the one parse refused
it, which broke every TriG cross-ledger config.

A new `ParserOptions::relative_iris` (default `Resolve`, so every other
caller stays strict) lets the RDF-text driver keep a relative reference
as written (`Verbatim`) in a labeled block's contents when no base is in
force: upsert and TriG insert as before, and sync bodies now alike. The
default graph and block labels still resolve against the base and are
refused without one, and a base in force still resolves a reference in a
block.

The parity test also syncs its fixture: sync stores what insert stores.

With no `@base` in force, a block's label keeps a relative reference
exactly as written, as the block's contents do and as the TriG reader
always did: `GRAPH <cfg/local> { … }` names the graph `cfg/local`. A
ledger id is a relative reference, so a label that names a ledger reaches
staging as written, and what it denotes is decided there with every other
write graph name. The default graph still refuses a relative reference
without a base, and a base in force still resolves label and contents.

…ne parser

The driver used to start a fresh parser for every segment of a TriG
document. Each one was seeded from a copy of every prefix declared so far,
and each block label was resolved by its own probe parse. Each segment was
also lexed a second time, after the locator had lexed the whole document to
find the blocks. A document of many small blocks paid all of it per block:
100k one-triple blocks under 20 prefixes ingested 1.6 to 1.9 times slower
than the phase-1 reader.

fluree-graph-turtle gains SegmentParser, one parser over the pieces of a
document, fed the tokens the locator lexed. The parser is generic over its
token source, a streaming lexer by default, so the lexer path compiles as
before. Declarations, the base, the term caches and the sink carry from
piece to piece as they do between statements, the relative-IRI mode is set
per piece, and error positions are the document's. The locator hands over
each segment's tokens instead of an edited copy of the text, so nothing is
lexed twice or copied per block. The driver resolves each distinct label
once per declaration epoch, which a prefix or base declaration advances,
so a label under a redefined prefix still names the new graph.

A TriG upsert leaves orphaned blank nodes in the graphs its blocks write,
where the documented default-graph query cannot see them. turtle.md adds
the same check over GRAPH ?g, and the documented-query test pins it on an
edited block.

upsert.md's Idempotency section says that a value stored with less detail
than it was written is the one exception to a retry-safe upsert. The
page's opening line, its behavior list and its comparison table, the
transactions README, and the WHERE/DELETE/INSERT comparison table still
called replace mode idempotent without it. They now name the exception and
point to that section, and the README's retry checklist says how to write
such values.

…SPARQL too

The routing test's doc named the SPARQL shapes but ran only JSON-LD. It now
runs SPARQL OPTIONAL and UNION twins, and a SPARQL UNION whose other branch
binds the object from another predicate, which stays on the resolver. It
collects every mismatch before asserting, so one run shows each case.

A blank-node label names one node across a TriG document, so a node
described in one graph can be referenced from another graph or from the
default graph. The per-graph orphan query checked references only within
the node's own graph and listed such a node, and the default-graph query
missed references from named graphs. Both now look for references in every
graph. A test pins a node referenced from another graph and one referenced
from a named graph, and shows the own-graph checks listing them.

…not view

A write that restates a stored value as it is stages nothing: the
retraction of the stored fact and the assertion of the same fact cancel
before the modify check, so such a write was not checked against modify
policy.

Under a non-root policy, the accumulator now reports the pairs that
cancel, and both sides of each pair whose stored fact the identity cannot
view go to the modify check. Restating a value the identity can view
still commits nothing without a check, root is unaffected, and a write
with no restated value builds no view check. This covers upsert, a DELETE
and INSERT of one value on any surface, and graph sync.

The policy docs say how restatements are checked, and that granting
modify without view does not keep a value confidential from that
identity.