Coalesce duplicate file watcher events - #130

Summary

• collect adjacent native watcher notifications into a 25 ms quiet-window burst with a 100 ms hard deadline
• deduplicate by path in first-seen order while retaining whether any event could introduce a directory
• emit one synthetic event per path so final filesystem classification still handles removals and forced reads still catch same-size/same-timestamp rewrites
• cap each burst at 1,024 events and 4,096 paths; split an oversized individual event into ordered pending chunks and carry any cap-rejected event into the next batch
• preserve idle-only queue-overflow recovery so a queued backlog triggers one stale reconcile after draining rather than one per capped batch
• suppress the remaining cross-burst duplicate: a save whose second notification lands after the burst closes no longer re-commits the identical bytes or logs a second reindex: modified

Cross-burst duplicate suppression

Time-based coalescing cannot help when the two notifications for one save are separated by more than the burst window, which is what a Substrate repository save of CertificateUtil.cs showed. Every concrete watcher event still forces a reindex on purpose, because that is what catches a same-size, same-coarse-timestamp rewrite, so the second event repeats the full overlay remove/reinsert and the log line.

The fix suppresses only a provable no-op, and never weakens the forced read:

• RecentReindexCache (tgrep-cli/src/serve.rs) keeps, per normalized rel path, the exact raw Vec<u8> of the last successful commit together with the live-overlay ID that commit produced. It is bounded three ways — 4,096 entries, 64 MiB total, 64 MiB per entry — with exact byte accounting across replacement, LRU count eviction, byte-budget eviction, explicit pop and clear. A replacement that cannot be admitted still evicts the previous record for that key, so oversized writes can never leave stale bytes behind.
• LiveIndex::file_id_for_path (tgrep-core/src/live.rs) is the only new core API: a read of the active overlay ID. IDs are minted per upsert, so any direct or bulk upsert, delete/recreate, prune, reconcile or later commit invalidates a record automatically. No trigram-map comparator is involved. Replacing the whole HybridIndex restarts the ID space, so both replacement sites clear the cache under the index write lock.
• reindex_file is unchanged through open, containment-safe fresh read, decode, trigram computation and the final file_still_has_bytes byte/version verification. Only after all of that does it look up a candidate, copying the recorded ID out from under the evidence lock. Under the index write lock it revalidates that ID against index.live.file_id_for_path(rel_path). commit_upsert and the reindex: modified line are skipped only when the bytes are exactly equal and the ID still matches.
• A skip is not a no-op elsewhere: the filename-only marker is still removed (and the sidecar marked dirty if it was), the decoded content cache is invalidated and its generation advanced unconditionally (an A → temporary B search → A sequence can leave B cached), and the read-bound FileStamp is written, replacing any retry sentinel. Real commits record fresh evidence after every index, filename, cache and stamp lock is released; drops and filename-only transitions forget it, including on their no-op early returns.
• Lock order is documented: recent_reindexes is a leaf, taken last and never held while acquiring index, filename_extra_paths, cache or file_stamps.

Nothing trusts FileVersion, FileStamp, a hash, decoded text or the trigram representation as content identity, and force=false is never substituted. Different bytes of the same length, with a manually matched coarse stamp, still commit and become searchable.

Tests

• deterministic production-path receiver coverage for prequeued duplicate collapse, cap-rejected pending rollover, oversized-event ordered chunking without path loss, and zero-max-duration queue preservation
• unit coverage for distinct path ordering, directory-introduction preservation, and event/path caps
• separate_forced_events_suppress_only_exact_current_overlay_duplicate: two separate forced reindexes of unchanged bytes preserve the overlay ID and dirty count, while a deliberately stale DecodedFile planted in the content cache is evicted with a generation bump and a {u64::MAX, u64::MAX} stamp sentinel is replaced by the real stamp
• concrete_event_reindexes_equal_length_rewrite_with_matching_stamp: strengthened to assert a new overlay ID and an incremented dirty count for equal-length different bytes under a matching persisted stamp
• exact_reindex_evidence_does_not_suppress_a_to_b_to_a: three distinct overlay IDs, three dirty increments, and only the final content searchable
• overlay_id_mismatch_prevents_exact_byte_suppression: a direct live.upsert_file after the record is written makes the next forced reindex commit rather than trust the stale ID
• replacing_the_whole_index_forgets_exact_byte_evidence: after an index replacement recycles the first overlay ID, the same bytes still commit
• recent_reindex_cache_bounds_bytes_capacity_and_oversized_replacements: capacity eviction, byte-budget eviction and an unadmitted oversized replacement all leave exact accounting and no usable stale record
• mutation controls: removing the duplicate-path merge produced two outputs; dropping a cap-rejected pending event failed next-invocation ordering; disabling oversized-event splitting returned all paths in the first burst; forcing the exact-match skip to false failed the cross-burst test with left: Some(2147483649) vs right: Some(2147483648), i.e. a second overlay entry was created — restored and re-run green
• cargo fmt --all -- --check
• cargo clippy --workspace --all-targets -- -D warnings
• cargo test --workspace