You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Start the tgrep-side foundation for reusing one content index across multiple worktrees, without changing existing CLI or server behavior.
Add tgrep_core::shared::SharedBase, which shares one validated Arc<IndexReader> and its path table across independent worktree-rooted HybridIndex views.
Save only per-worktree postings, masks, and deletion tombstones in atomically replaced checkpoints; never merge the overlay into the shared base.
Bind checkpoints to a fingerprint of the exact base path table, lookup table, and postings, plus the canonical worktree root. Reject incompatible, malformed, missing, and wrong-base/root checkpoints rather than silently falling back to the base.
Validate shared snapshots' physical sections, metadata counts, contiguous posting layout, valid trigrams, sorted/unique in-range posting IDs, and nonzero location masks, while accepting valid empty indexes and short files.
Prevent checkpoint publication into the base by directory identity. Unix staging/replacement/cleanup use an open directory handle; Windows retains non-delete-sharing handles on the canonical parent and all ancestors throughout publication.
Reject directory-shaped checkpoint paths instead of silently normalizing them into filenames.
Preserve existing Unicode-root checkpoint strings and encode non-Unicode roots losslessly using platform-native units.
Document the API, its immutability requirements, and its integration boundaries.
The design document uses the generic term agent runtime and includes diagrams for repository-scoped ownership, per-worktree search, and immutable base generations. It distinguishes the implemented core foundation from proposed Git discovery, synchronization, and daemon integration.
Recommended rollout: merge this backwards-compatible foundation independently once review and required checks are satisfied, then implement base-generation management, worktree synchronization, and daemon/agent runtime integration in focused follow-up PRs rather than expanding this PR into the entire feature.
Backward compatibility and scope
Existing CLI commands, RPC protocol, on-disk index format, and single-root server behavior remain unchanged. Existing HybridIndex::open and ordinary IndexReader::open/validate_lookup behavior is preserved; stricter integrity and coverage checks apply only to the new shared-base API.
This PR is a core-library increment, not automatic shared indexing in the CLI. Git-delta discovery, repository-scoped daemon registration, per-worktree watchers/visibility/caches, and runtime lifecycle integration remain follow-up work. Callers must supply all worktree differences and reconcile restored overlays before serving searches, and keep the base files immutable while readers reference them.
Checkpoint replacement is atomic visibility, not power-loss durability: file contents are synced, but the parent directory is not synced after replacement. Missing or stale checkpoints require reconciliation or rebuilding. Existing Unicode-root checkpoints remain compatible; older readers reject the new native-root representation.
Validation
Latest CI: full workspace builds, benchmark builds, and tests passed on Linux, macOS, and Windows; formatting and Clippy passed.
Local Windows targeted run: 264 core unit tests, 30 shared-worktree integration tests, and 7 snapshot consistency tests passed.
Regression coverage includes reader sharing, worktree isolation, masks/tombstones/empty files, exact-base identity, restoration, malformed sections/metadata/postings, duplicate or descending posting IDs with conflicting masks, native-root identity, repeated saves, Windows failure recovery, renamed parents/bases, symlink redirection, directory-shaped destination paths, cleanup, and private Unix checkpoint permissions.
The reason will be displayed to describe this comment to others. Learn more.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Introduces a new core API for sharing a single immutable base content index across multiple worktrees via independent live overlays and JSON checkpoints, without changing existing CLI/server behavior.
Changes:
Added tgrep_core::shared::SharedBase for shared base reader + per-worktree overlay creation, saving, and restoration with strict base/root identity checks.
Added IndexReader::snapshot_id() to fingerprint base lookup/postings/path table identity independent of directory.
Added regression tests and documentation for shared-worktree behavior and checkpoint invariants.
File
Description
tgrep-core/src/shared.rs
Adds the shared-base API, overlay checkpoint format, validation, and save/restore logic.
tgrep-core/src/reader.rs
Adds snapshot fingerprinting used to bind checkpoints to exact base bytes.
tgrep-core/src/hybrid.rs
Refactors reader opening to support SharedBase creating worktrees from a shared Arc<IndexReader>.
Support non-Unicode filesystem paths in checkpoint serialization
tgrep-core/src/shared.rs:168
Serializing PathBuf through Serde JSON fails for canonical roots containing non-UTF-8 Unix bytes or unpaired Windows UTF-16. create_worktree accepts those valid filesystem paths, but save_overlay can never checkpoint them, so this public API has an undocumented path restriction. Encode the platform-native root bytes losslessly in the checkpoint, or reject unsupported roots when creating the view and document that constraint.
Capture the proposed agent runtime integration, query flow, immutable base lifecycle, compatibility boundaries, and staged implementation plan with architecture diagrams.
Co-authored-by: Copilot App <[email protected]>
Reject mismatched empty sections and inconsistent metadata counts only for shared bases. Persist checkpoints using the validated canonical destination and preserve non-Unicode root identities with platform-native encodings while retaining legacy Unicode strings.
Co-authored-by: Copilot App <[email protected]>
Keep filesystem round-trips on Linux and Windows, and test Unix byte serialization independently of filesystem filename restrictions on macOS.
Co-authored-by: Copilot App <[email protected]>
Use handle-relative temporary files and replacement on Unix, and hold non-delete-sharing ancestor handles for Windows path-based publication. Exclude bases by directory identity and cover parent replacement, renamed bases, cleanup, private permissions, and Windows locking.
Co-authored-by: Copilot App <[email protected]>
Reject duplicate and descending file IDs before a base can reach query-time deduplication, preserving distinct masks rather than silently discarding them. Keep ordinary readers unchanged and cover conflicting-mask duplicates.
Co-authored-by: Copilot App <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Start the tgrep-side foundation for reusing one content index across multiple worktrees, without changing existing CLI or server behavior.
tgrep_core::shared::SharedBase, which shares one validatedArc<IndexReader>and its path table across independent worktree-rootedHybridIndexviews.Design
Shared worktree index architecture and rollout
The design document uses the generic term agent runtime and includes diagrams for repository-scoped ownership, per-worktree search, and immutable base generations. It distinguishes the implemented core foundation from proposed Git discovery, synchronization, and daemon integration.
Recommended rollout: merge this backwards-compatible foundation independently once review and required checks are satisfied, then implement base-generation management, worktree synchronization, and daemon/agent runtime integration in focused follow-up PRs rather than expanding this PR into the entire feature.
Backward compatibility and scope
Existing CLI commands, RPC protocol, on-disk index format, and single-root server behavior remain unchanged. Existing
HybridIndex::openand ordinaryIndexReader::open/validate_lookupbehavior is preserved; stricter integrity and coverage checks apply only to the new shared-base API.This PR is a core-library increment, not automatic shared indexing in the CLI. Git-delta discovery, repository-scoped daemon registration, per-worktree watchers/visibility/caches, and runtime lifecycle integration remain follow-up work. Callers must supply all worktree differences and reconcile restored overlays before serving searches, and keep the base files immutable while readers reference them.
Checkpoint replacement is atomic visibility, not power-loss durability: file contents are synced, but the parent directory is not synced after replacement. Missing or stale checkpoints require reconciliation or rebuilding. Existing Unicode-root checkpoints remain compatible; older readers reject the new native-root representation.
Validation
cargo test -p tgrep-core --lib --test shared_worktrees --test snapshot_consistency --locked --quietcargo clippy --workspace --all-targets --locked --quiet -- -D warningscargo fmt --all -- --checkgit diff --check