Repository navigation
fluree doc ingest: add an RDF 1.2 annotation output mode for extracted relations #1860
Description
fluree doc ingest records everything needed to treat an extracted relation as an attributed claim — the sentence it came from, the document and chunk, the verdict against the supplied ontology, whether it was asserted — but writes it only as RDF 1.1 reification. There is no mode that emits the RDF 1.2 annotation form the rest of the product recommends.
Where it stands today
--relations offers three modes (fluree-db-doc/src/extract.rs:38):
direct— reified relation nodes, plus a plain edge for every admitted onereified— reified relation nodes only, for review before anything becomes an edgeoff— entities only
All three emit the relation as rdf:subject / rdf:predicate / rdf:object (extract.rs:594):
{ "@id": "<doc>/relation/0", "@type": "doc:Relation",
"rdf:subject": {"@id": "ex:alice"},
"rdf:predicate": {"@id": "ex:knows"},
"rdf:object": {"@id": "ex:bob"},
"doc:excerpt": "Alice knows Bob Ramirez professionally",
"doc:sourceDocument": {"@id": "urn:fluree:doc:memo-a.md"},
"doc:sourceChunk": {"@id": "…/chunk/0"},
"doc:verdict": "valid", "doc:asserted": true, "doc:extractedBy": "llm" }and, under direct, a separate ex:alice ex:knows ex:bob. The two are not linked to each other, so a traversal over the edge cannot see the provenance, and {| … |} patterns against that edge return nothing.
There is no annotation emission anywhere in fluree-db-doc.
Why it is worth a mode rather than a note
The staged-then-promote flow is a good design and worth keeping — reified explicitly exists so relations can be reviewed before they become edges. The gap is that there is no supported final step. Getting from the review layer to an attributed edge is a SPARQL UPDATE the user has to write:
INSERT { ?s ?p ?o ~ ?rel {| doc:sourceDocument ?src ; doc:excerpt ?excerpt ; … |} }
WHERE { ?rel a doc:Relation ; rdf:subject ?s ; rdf:predicate ?p ; rdf:object ?o ;
doc:asserted true .
OPTIONAL { ?rel doc:sourceDocument ?src } … }That works — variable predicate, variable reifier and the OPTIONAL provenance fields all behave — and it is a reasonable thing to hand someone who wants to filter what gets promoted. But every user who wants document-derived data in the recommended claim shape has to be told about it, and has to get the property list right.
Suggested shape
A fourth --relations mode — annotated, or a --promote flag on the existing ones — that attaches the provenance to the edge as a reifier at ingest time, with the same set of doc:* properties the review node carries. reified stays the default for review-first workflows; annotated serves the case where the ontology verdict is trusted and the graph should be immediately walkable.
Worth pairing with the ability to carry a confidence: the model already reports one per relation, and confidence is the single most common thing a claim-based traversal filters on.
Environment
main @ 3d89e8f, release build, macOS.
Activity
aaj3f commentedon Sep 16, 2026
Coming back to this from a different angle — I went digging through
fluree-db-docfor something adjacent and came out with two things I'd add and one I need to walk back.The walkback is the confidence line at the end of the description. I wrote that "the model already reports one per relation," and that isn't true.
grep -rn "confidence" fluree-db-doc/src/comes back empty:LlmRelation(fluree-db-doc/src/extract.rs:82-95) asks the model forsubjectName/predicate/predicateLabel/objectName/objectIsLiteral/contextand nothing else, andSYSTEM_PROMPT_TEMPLATEnever asks for a confidence either. So whatever shape this mode ends up taking, on its own it can only carry thedoc:*properties the review node already has — the confidence pairing isn't a small add-on to this issue, it's a separate change. Filed it as #1864; whichever of the two lands first, the other should pick the field up.Leaving the description as-written rather than editing it, since the reasoning is worth keeping visible — but that one sentence shouldn't be relied on by anyone picking this up.
The rest of the description still holds as far as I can tell — I re-checked
extract.rs:38for the three modes,:594for the emission, and the "no annotation emission anywhere influree-db-doc" claim, and all three are right.Two things I'd add, both more open questions than objections:
Re-ingest.
doc:assertedexists so that "a re-ingest knows which edges to reconsider" (fluree-db-doc/src/vocab.rs:63-65), which works today because underdirectthe edge is the only thing written outside the review layer. An annotation is a second thing to reconsider: if a document is re-ingested and a relation is no longer extracted, something has to retract the reifier as well as the edge, and I don't think there's anything in the current re-ingest path that would. That may be the difference between this being a flag and being a project, so it seems worth settling before anyone starts rather than after.annotatedvs--promote. The description offers both without picking, and having sat with it I'd lean toward the flag. A fourth enum value is ambiguous about whether the plain edge is still written — and since a 1.2 reifier entails its base triple,annotatedwould more or less subsumedirect, which makes the four modes non-orthogonal and muddies whatdoc:assertedmeans.--relations reified --promotereads unambiguously as "review nodes, plus annotated edges for the admitted ones," and composes with the existing modes instead of competing with them. Not a strong position, though — if there's a reason the enum is cleaner that I'm not seeing, happy to be talked out of it.And one small one: whichever shape it takes, promotion probably wants to key on
doc:asserted truethe way the hand-writtenINSERT … WHEREdoes, orrejectedrelations ride along with everything else.