Skip to content

fix: prevent read-only watcher events from causing overflow loops - #152

Merged
Shengyu Fu (shengyfu) merged 2 commits into
mainfrom
shengyfu-watcher-event-filtering
Sep 11, 2026
Merged

Shengyu Fu (shengyfu) merged 2 commits into
mainfrom
shengyfu-watcher-event-filtering

Conversation

@shengyfu

Copy link
Copy Markdown
Member

Summary

  • Filter notifications before they enter the bounded watcher queue, sharing the worker's create/modify/remove predicate.
  • Preserve native rescan signals and recovery requests raised during reconciliation; do not discard concurrent edits by draining the queue after a scan.
  • Add deterministic queue/recovery coverage and a Linux regression using 128 watched directories with an eight-event queue.
  • Document queue filtering and bump the workspace and both lockfiles to 1.0.7.

Fixes #151.

Validation

  • Reproduced the read-only overflow on Linux before the fix; the regression passes afterward and confirms real edits remain searchable.
  • Targeted watcher tests pass on Linux and Windows.
  • Targeted Linux Clippy and Rust formatting checks pass.
  • CLI reports tgrep 1.0.7.

Ignore read-only notifications before the bounded watcher queue while preserving native rescan signals and overflow recovery for real changes.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings September 11, 2026 00:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The concurrency-sensitive, platform-dependent watcher recovery changes warrant final human review despite strong targeted coverage.

Review tier: Balanced
Findings: None

What changed in this PR

Prevents read-only watcher events from overflowing the bounded queue while preserving recovery signals and real filesystem changes.

Changes:

  • Filters non-index-changing events before queueing.
  • Preserves concurrent overflow and native rescan recovery requests.
  • Adds regression tests, documentation, and version 1.0.7 updates.
File Description
tgrep-cli/​src/​serve.rs Implements watcher filtering and recovery handling.
tgrep-cli/​src/​serve/​poll_tests.rs Tests overflow during reconciliation.
tgrep-cli/​tests/​watcher_watch_registration.rs Adds Linux overflow regression coverage.
README.md Documents queue filtering behavior.
Cargo.toml Bumps workspace version.
Cargo.lock Updates workspace package versions.
fuzz/​Cargo.lock Updates the fuzz dependency version.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@shengyfu
Shengyu Fu (shengyfu) merged commit c2fdb3e into main Sep 11, 2026
10 checks passed
@shengyfu
Shengyu Fu (shengyfu) deleted the shengyfu-watcher-event-filtering branch September 11, 2026 01:15
@desarrollo-badabun

Copy link
Copy Markdown

Reviewed, and ran it on the tree from #151. Thanks for the quick turnaround.

Empirical check — v1.0.7 on the original repro tree (25,360 files, 5,709 watched directories, watch budget 8192), with --watcher-queue-cap removed so it runs at the default 16,384:

  • Three full read sweeps of the tree: 201,726 file opens in 6s, roughly 12x the queue cap per sweep. Zero watcher queue overflowed lines and no recovery walks in the log.
  • Real changes still land: a new file is searchable ~2s after creation, and a deletion drops out of the index in ~2s.
  • Before 1.0.7 this machine needed --watcher-queue-cap 131072 to stay out of the ~1.5s overflow loop. That workaround is now gone from my systemd unit.

Code — the placement looks right. Two things I checked specifically:

  • need_rescan() is evaluated before the kind filter. inotify delivers IN_Q_OVERFLOW as EventKind::Any with no paths, so the new filter would otherwise have turned a real kernel overflow into silence. Good to see a test pinning that down.
  • overflowed.swap(false, ...) happens before the stale walk rather than after, so edits landing during reconciliation re-arm recovery instead of being swallowed.

Sharing event_changes_index between the callback and handle_fs_event is also what keeps this from regressing later: one predicate instead of two that can drift apart.

One residual item, not a blocker: notify 7.0.0 hardcodes WatchMask::OPEN in its inotify mask (notify/src/inotify.rs#L418-L425) purely to synthesize the Access(Open) events tgrep now always discards. The kernel still queues that traffic, and fs.inotify.max_queued_events is 16,384 by default on most distros. If the notify reader thread ever falls behind on a large tree, IN_Q_OVERFLOW → need_rescan() → recovery walk → the same flood: same loop shape, much harder to reach. Dropping OPEN from the mask would remove the traffic at the source, but that needs an upstream change in notify (or using the inotify crate directly). Happy to open a separate issue if it's worth tracking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serve loops forever on Linux: its own stale checks overflow the watcher queue (IN_OPEN events are enqueued unfiltered)

4 participants