📚 SEC 10-K RAG Analyst: grounded, cited answers over annual reports

A production-style RAG assistant over SEC 10-K filings: hybrid retrieval (BM25 + dense) → cross-encoder reranking → grounded answers with inline [n] citations, plus a labeled eval that measures section-recall and isolates the reranker's lift. FastAPI service + Streamlit UI. Runs offline end-to-end (synthetic filing + extractive fallback); plug in a Claude key and a real ticker for the full experience.

10-Ks are the hard case for RAG, 100+ pages, dense tables, repetitive boilerplate, and cross-references. Naive "embed everything + top-k cosine" retrieves plausible-but-wrong passages. This repo does the parts that actually matter in production: section-aware chunking, hybrid lexical+dense retrieval, reranking, citations, and measurement.

Interactive/exportable version: docs/assets/architecture.html.

SEC EDGAR 10-K
     │  src/ingest.py     (ticker -> CIK -> latest 10-K -> strip HTML -> Item sections)
     ▼
  sections.jsonl
     │  src/chunk.py      (section-aware sliding word windows w/ overlap)
     ▼
  chunks.jsonl
     │  src/index.py      (BM25 lexical + BGE dense embeddings -> FAISS)
     ▼
  data/index/  ──►  src/retrieve.py   BM25 ─┐
                                            ├─ RRF fuse ─► cross-encoder rerank ─► top-k
                                    dense ─┘
                                            │  src/generate.py  (Claude, grounded + cited)
                                            ▼
                        api/main.py  POST /ask   ·   app/streamlit_app.py

• Retrieval: rank-bm25 (lexical) + sentence-transformers BGE-small (dense) + FAISS; BAAI/bge-reranker-base cross-encoder for reranking.
• Generation: Claude claude-opus-4-8 via the anthropic SDK, grounded answers with inline [n] citations. Falls back to extractive (top passages) with no API key.
• Serving / UI: FastAPI + Streamlit. Ops: Docker.

Uses the conda personal env (per environment conventions, never base).

PY=~/miniconda3/envs/personal/bin/python
$PY -m pip install -r requirements.txt

# --- OFFLINE PATH (no network, no API key) -------------------------------------
$PY -m src.ingest --sample                                   # synthetic 10-K
$PY -m src.chunk  --in data/processed/SAMPLE.jsonl           # section-aware chunks
$PY -m src.index  --chunks data/processed/SAMPLE.chunks.jsonl # BM25 + dense index
$PY -m src.pipeline "What are the company's main risk factors?"
$PY -m src.evaluate                                          # recall@k + rerank ablation

# --- REAL DATA -----------------------------------------------------------------
# SEC requires a descriptive User-Agent with a contact:
$PY -m src.ingest --ticker AAPL --user-agent "yourname [email protected]"
$PY -m src.chunk  --in data/processed/AAPL.jsonl
$PY -m src.index  --chunks data/processed/AAPL.chunks.jsonl

# --- GROUNDED ANSWERS WITH CLAUDE ----------------------------------------------
export ANTHROPIC_API_KEY=sk-ant-...     # without this, answers are extractive
$PY -m src.pipeline "How did revenue and gross margin change year over year?"

# --- SERVE ---------------------------------------------------------------------
$PY -m uvicorn api.main:app --reload --port 8000   # POST /ask {"question": "...", "top_k": 5}
$PY -m streamlit run app/streamlit_app.py

make sample runs the whole offline build in one shot.

src/retrieve.py runs four stages so each one's contribution is measurable:

1. BM25 top-N, exact terms, numbers, Item references.
2. Dense top-N, paraphrase ("how profitable" ≈ "gross margin").
3. RRF fusion, rank-based merge; no fragile score normalization.
4. Cross-encoder rerank, reads (query, passage) jointly for precision.

Pass --no-rerank (CLI) or rerank=False to return the fused list, that's the ablation the eval uses to prove the reranker earns its latency.

src/evaluate.py scores a labeled question set (eval/questions.yaml):

• section recall@k, did retrieval surface a chunk from the Item that should answer it?
• reranker ablation, the same metric with rerank ON vs OFF → the lift.
• keyword grounding, fraction of expected facts present in retrieved context (a key-free proxy for "could the model answer faithfully?").
• faithfulness (opt-in), an LLM judge over answer-vs-context, when ANTHROPIC_API_KEY is set.

Resume bullet (fill after running on a real 10-K):

Built a production RAG assistant over SEC 10-K filings (hybrid BM25+dense retrieval, cross-encoder reranking, cited answers via Claude); reranking lifted section-recall@5 from [x] → [y] on a [N]-question eval; deployed as a FastAPI service + Streamlit UI.

sec-rag-analyst/
├── src/
│   ├── ingest.py    EDGAR fetch + Item segmentation (+ --sample offline filing)
│   ├── chunk.py     section-aware sliding-window chunking
│   ├── embed.py     BGE embedding wrapper (passages + query-side instruction)
│   ├── index.py     build/load hybrid index (BM25 + FAISS)
│   ├── retrieve.py  BM25 + dense → RRF → cross-encoder rerank
│   ├── generate.py  Claude grounded answer w/ citations (+ extractive fallback)
│   ├── pipeline.py  end-to-end RAGPipeline + CLI
│   └── evaluate.py  section-recall + rerank ablation + grounding
├── api/main.py      FastAPI /ask
├── app/streamlit_app.py
├── eval/questions.yaml
├── tests/           key-free unit tests (chunking, RRF, Item parsing)
├── requirements.txt · Dockerfile · Makefile

Stack note: conda personal env for builds, never base. SEC EDGAR requires a real contact in the User-Agent. Pin model + dataset revisions so results reproduce.