Skip to content

Fix inline case-insensitive index filtering and full-corpus stats - #163

Merged
Shengyu Fu (shengyfu) merged 4 commits into
mainfrom
shengyfu-inline-case-filtering
Sep 21, 2026
Merged

Shengyu Fu (shengyfu) merged 4 commits into
mainfrom
shengyfu-inline-case-filtering

Conversation

@shengyfu

Copy link
Copy Markdown
Member

Summary

Fixes #162.

  • Apply command-line and inline case flags during regex parsing, then recover ASCII-folded literal runs from character classes for trigram planning. This covers scoped flags and relaxed -P patterns without excluding Unicode case-fold matches.
  • Report query plans, raw candidates, filtered candidates, and corpus size for server-backed --stats, matching local-index reporting. Annotate summaries with no index narrowing whenever raw candidates cover the nonempty corpus, even for a usable trigram plan.
  • Count active files using the query's reader snapshot and live-overlay changes rather than walking the entire corpus for statistics. Preserve compatibility with older servers and keep default search output unchanged.
  • Document the behavior and add planner, CLI, RPC, and snapshot regression coverage. The issue's query selects 2 of 258 fixture files with either inline (?i) or -i, locally and through the server.

Validation

  • Reproduced the original MatchAll regression before the fix.
  • cargo test -p tgrep-core --quiet
  • cargo test -p tgrep-cli --test ripgrep_compat --test concurrent_search --quiet
  • cargo test -p tgrep-cli --bin tgrep search::tests --quiet
  • cargo fmt --all -- --check
  • cargo clippy --workspace --benches -- -D warnings

All listed checks passed on Windows.

Honor inline and scoped case flags in trigram planning, preserve Unicode matches, and report full-corpus candidate selection for server and local searches.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings September 20, 2026 19:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Server and local filtered-candidate counts diverge when a stricter --max-filesize is used.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Fixes inline case-insensitive trigram planning and expands indexed-search statistics across local and server backends.

Changes:

  • Parses case flags into regex HIR and safely recovers ASCII-foldable literals.
  • Adds snapshot-aware corpus counts and server candidate statistics.
  • Documents and tests candidate narrowing behavior.
File Description
tgrep-core/​src/​query.rs Improves case-insensitive query planning.
tgrep-core/​src/​live.rs Exposes overlay-shadowed paths.
tgrep-core/​src/​hybrid.rs Adds snapshot-aware file counting.
tgrep-core/​tests/​snapshot_consistency.rs Tests overlay and snapshot totals.
tgrep-cli/​src/​serve.rs Returns server-side index statistics.
tgrep-cli/​src/​search.rs Displays consistent candidate statistics.
tgrep-cli/​tests/​ripgrep_compat.rs Adds CLI, server, and RPC regressions.
README.md Documents indexed statistics and case flags.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tgrep-cli/src/serve.rs Outdated
Count candidates after scope, visibility, glob and type filtering but before max-filesize checks. Cover server/local parity when the size cap excludes some or all candidates, and document the reporting stage.

Co-authored-by: Copilot App <[email protected]>
Copilot AI review requested due to automatic review settings September 21, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation is consistent across backends and has comprehensive regression coverage.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Co-authored-by: Copilot App <[email protected]>
Copilot AI review requested due to automatic review settings September 21, 2026 16:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation is consistent across backends and has comprehensive planner, snapshot, CLI, and RPC regression coverage.

Review effort: Balanced
Findings: None

Treat killpg EPERM as an exit race only after confirming the child has finished. Always close query pipes, preserve permission failures for live children, and cover both cases with deterministic subprocess regressions.

Co-authored-by: Copilot App <[email protected]>
Copilot AI review requested due to automatic review settings September 21, 2026 17:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation is consistent across local and server paths and includes focused regression coverage.

Review effort: Balanced
Findings: None

@shengyfu
Shengyu Fu (shengyfu) merged commit 56f8c62 into main Sep 21, 2026
12 checks passed
@shengyfu
Shengyu Fu (shengyfu) deleted the shengyfu-inline-case-filtering branch September 21, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inline (?i) degenerates the candidate set to the full corpus: ~1000x slower, and the summary still claims (via server)

2 participants