Add synchronized pinned worktree index views - #170

Summary

Layer 2 of shared-worktree indexing, dependent on #169 and part of native stack 172. Based on parent commit 4a1cdb9f8f98b40e9d9f59c5b2768d1e8dfa3354. Latest head: 2af22f08bfb74d61d11da75d5dc9d96976464f78. Automatic shared serving, native watchers, routing, and content caches belong to layer 3; this layer does not enable shared CLI routing.

Shared-worktree design, API contracts, and cost caveats

• Add tgrep_core::worktrees::{WorktreeView, WorktreeOptions, WorktreeSnapshot, WorktreeStatus, ReconcileStats, WorktreeError}. Each canonical worktree retains its exact Arc<Generation> and private whole-file overlay relative to that pin, including committed divergence, staging, untracked files, renames/deletions, sparse checkout and eligibility changes.
• Verify actual checkout bytes with the existing auto decoder and ContentId, rather than trusting Git status or stat data. Reuse matching base/overlay postings, and stream-copy verified content-identical base masks for renames/copies without extraction.
• Publish overlay, visibility and complete filename membership atomically behind readiness. Bounded file/subtree invalidations close the gate; overflow/unknown changes force full verification. Concurrent invalidation aborts publication with ChangedDuringReconcile; discovery/read errors remain not-ready. Reconciliation makes one attempt, never an unbounded churn loop.
• Expose guarded candidate/path resolution and files() membership without leaking live IDs or mutable HybridIndex. Final matching reads requesting-root handles through WorktreeSnapshot::open_file, or uses a private versioned cache.
• Save only the private delta using the existing directory-bound atomic checkpoint publisher. Its single file embeds the exact generation key/root/base binding; restoration always requires reconciliation before readiness. Checkpoint destinations must be directories, validated on construction, restore and save. No sidecar writes, base flushes, GC or generation deletion.
• Preserve ordinary walker membership for a linked worktree's plain .git pointer file, including hidden filename and content queries. Actual common_dir/git_dir metadata subtrees and generation/checkpoint/index storage remain excluded. The gitfile follows ordinary ignore/hidden/size rules and is not followed into the directory it names.
• Reject unrepresentable native file/directory names before lossy metadata-path conversion or visibility recording. Non-Unicode names and literal Unix backslashes return incomplete-discovery errors rather than aliasing another, potentially ignored file. Ordinary scans retain native paths.
• Normalize accepted invalidation hints, preserving subtree semantics for trailing/repeated separators and interior .. Invalid parent/absolute/leading . hints explicitly error and force full repair. Probe normalized paths and ancestor prefixes in a BTreeSet, rather than scanning all hints for every file; retain conservative ASCII case matching and non-ASCII full repair.
• Extract the existing serving reader into tgrep_core::rooted::RootedDir. Worktree views and ordinary ServerState each retain one reader instead of reopening/re-registering the root for every candidate. Unix uses component-relative no-follow/nonblocking opens and requires a regular final handle; Windows guards ancestors, rejects reparse points, and checks actual handle parent/containment. Revalidation uses rooted opens too.
• Update README API usage and the linked design's implementation status, safe final-read contract, root lifecycle, freshness and cost limitations.

Layer 3 contract

Construct/restore not-ready, then subscribe to changes before the first refresh(). Forward relative file/subtree hints through invalidate_path (both rename paths), and overflow/Git/ignore configuration or uncertain events through invalidate_all. refresh rewalks membership and handles hints; without hints it verifies all admitted contents. reconcile_full always verifies all content, including same-size/restored-mtime changes. Every reconciliation advances the epoch; successful publication acknowledges earlier invalidation tokens with an equal or later epoch.

with_snapshot holds readiness and overlay guards through candidate ID resolution. Snapshot methods expose root/epoch/visibility, candidates(plan, prefix, include_hidden) and files(prefix, include_hidden) as owned relative paths, plus open_file(relative: &str) -> std::io::Result<std::fs::File> through the retained reader. Do not reenter the view from the callback.

Pinned-root identity is verified automatically before and after every with_snapshot callback, including empty/file-only queries. Root verification or candidate-open failure returns an outer WorktreeError::Io, closes readiness, advances the invalidation epoch, clears hints and queues full repair before releasing the guard. The first candidate-open error is latched even if the callback handles it, performs a later successful open, or tries to return a success-shaped result.

A returned file handle can be read/matched outside the guard; the runtime must buffer results and perform its final snapshot/epoch check before publication. Later errors reading that handle remain caller-owned: report them and call invalidate_all() after leaving the guard, without publishing partial results. Neither handles nor epochs freeze same-inode contents. A refresh acknowledges processed inputs rather than promising an atomic filesystem snapshot; periodic full checks and scan fallback remain necessary.

The additive RootedDir::{open(root), open_file(relative), verify_root()} helper is also available. No extra view reader accessor is needed. Windows directory guards persist until their owners are dropped: detach/release shared registrations and stop ordinary serving before removing or renaming those roots.

Cost evidence

Real fixtures assert three identical tracked files reuse the shared reader with zero trigram extractions. A linked worktree also has a plain .git file absent from the committed base: this costs one additional private extraction initially. Four clean CRLF/smudge-transformed tracked files need four private extractions plus the gitfile, for five total; their next unchanged reconciliation extracts none. Positional/next-byte masks make LF/CRLF normalization unsafe. Widespread transformations may require an all-file private overlay; no cross-view transformed-content cache is claimed.

A hinted pass can avoid rereading unaffected, previously verified contents but still walks metadata and opens regular-file handles safely. ReconcileStats.hint_lookups counts ordered-set probes, bounded by path depth per file (each probe logarithmic in queue size). The regression queues 2,001 hints for two files and performs five probes, finds a same-size/restored-mtime subtree edit, and avoids rereading the unrelated file. Full/non-ASCII repair performs no hint probes. Ordinary serving retains its root registration instead of paying constructor work per file; identity verification still costs filesystem operations.

Validation

Final head 2af22f08bfb74d61d11da75d5dc9d96976464f78 passed locally on Windows and real Linux via WSL:

• cargo test -p tgrep-core: 361 Windows / 375 Linux tests passed.
• Entire CLI binary unit suite, including serve::tests and serve::poll_tests: 266 Windows / 288 Linux passed, with one pre-existing ignored test per platform. The serving subset passed 129 / 151.
• cargo fmt --all -- --check and cargo clippy --workspace --all-targets -- -D warnings passed on both platforms.
• Final-head CI run passed Windows, Ubuntu and macOS workspace tests, Format and Clippy; CodeQL run passed.
• The completed final-head automated overview reports no findings, and all three actual inline review threads are resolved. It still recommends final human review of the cross-platform filesystem/concurrency/checkpoint surface.
• Linux tests were explicitly listed and executed: FIFO, root replacement, native-name collisions, hint normalization and symlink/executable/gitlink coverage are real top-level tests, not nested/unexecuted fixtures.
• Deterministic hooks cover directory-to-link/junction swaps before/after ancestor opening and root replacement during verification/open. Real-Git tests assert not-ready/full-retry behavior. Unix FIFO regressions use deadline-protected subprocesses so blocking regressions fail instead of hanging.
• Reproduced snapshot-open readiness failures on Windows and Linux before fixing them. Regressions cover directory-link swaps, missing candidates, swallowed errors, first-error preservation despite later opens, and mandatory full repair after an unrelated hint catches a same-size/restored-mtime edit. A dedicated real Linux snapshot FIFO test also executes under a deadline.
• Added checkpoint regular-file rejection tests for inside/outside-root destinations and a directory replaced before save, retaining destination bytes. Added final candidate-read link-swap and root-replacement checks, including empty snapshot callbacks and ordinary-serving root retention.
• Pinned root registration exposed two artificial missing-root polling fixtures. They now use an existing root plus a malformed-ignore discovery error, retaining failed-attempt, one-walk, completion-time and retry-cadence assertions. The previously omitted serve::poll_tests module now runs in the full CLI validation above.

Earlier regressions reproduced lossy native-path aliases on Linux (including ignored literal-U+FFFD/directory-path collisions), lost subtree hints on Windows after same-size/restored-mtime edits, and missing linked .git membership. Other fixtures cover divergent worktrees/current-root reads; scan parity and match removal; empty/short content; ignore/hidden/filename-only binary/size/storage rules; CRLF/smudge and UTF-16; assume-unchanged; sparse/skip-worktree; deterministic concurrent invalidation; read/discovery errors and overflow; checkpoint revalidation/bindings and immutable destination rejection. Windows hidden-attribute/exclusive-read coverage ran locally; Linux executed actual non-UTF-8 collisions that APFS may reject at fixture creation.

No unrelated Python scripts were changed. Parent fixes are inherited by rebase, not duplicated. Native stack membership is preserved; nothing is merged.