Add AGENTS.md: usage guide for AI coding agents - #143

Adds an AI-agent usage guide for tgrep and links it from the README.

Changes:

• Documents setup, search behavior, output formats, freshness, and troubleshooting.
• Adds a README pointer to the guide.

Suppressed comments (1)

AGENTS.md:21

• The server is not guaranteed to be “always current.” The README documents that filesystem notifications can be missed and drift may remain until periodic reconciliation; --no-watch disables both mechanisms. Please avoid presenting server results as unconditionally fresh.

1. **Server** running for this tree: query it over TCP. Fastest, and always
   current because the server watches the filesystem.

• Files reviewed: 2/2 changed files
• Comments generated: 4
• Review effort level: Balanced

Suppressed comments (4)

AGENTS.md:103

• This sample is not output from the shown command: an explicit . is preserved in output paths, and the referenced fn main is in tgrep-cli/src/main.rs. The current bytes_printed values also do not equal the displayed JSON bytes (249 before end, not 269). Keep the original path and corresponding byte counts so agents can rely on this as a real sample.

{"data":{"path":{"text":"src/main.rs"}},"type":"begin"}
{"data":{"absolute_offset":38226,"line_number":998,"lines":{"text":"fn main() {\n"},"path":{"text":"src/main.rs"},"submatches":[{"end":7,"match":{"text":"fn main"},"start":0}]},"type":"match"}
{"data":{"binary_offset":null,"path":{"text":"src/main.rs"},"stats":{"bytes_printed":269,"bytes_searched":50644,"elapsed":{"human":"0.000019s","nanos":18541,"secs":0},"matched_lines":1,"matches":1,"searches":1,"searches_with_match":1}},"type":"end"}
{"data":{"elapsed_total":{"human":"0.000834s","nanos":833792,"secs":0},"stats":{"bytes_printed":529,"bytes_searched":50644,"elapsed":{"human":"0.000834s","nanos":833792,"secs":0},"matched_lines":1,"matches":1,"searches":1,"searches_with_match":1}},"type":"summary"}

AGENTS.md:80

• Quoting does not stop Clap from treating an exact subcommand name as a subcommand: for example, tgrep "serve" . starts the server instead of searching for serve. Agents should consistently terminate option parsing before the pattern, as the tool sketch does below.

- **Quote the pattern** and pass the search root explicitly (`.` or a path).

AGENTS.md:33

• Indexing: complete only reports that the server's initial/background build has finished; it does not prove that every later watcher notification was received. Since this guide already acknowledges missed notifications, waiting for this status cannot guarantee an exhaustive search, and rebuilding the on-disk index does not refresh an already-running server's in-memory index. Recommend --no-index when a current filesystem scan is required.

An agent never has to choose between these; the command is the same. Results
can differ, though: an on-disk index omits changes since its last build, and a
server that is still building its index answers from partial data. For an
exhaustive search, check `tgrep status .` and wait for `Indexing: complete`,
or rebuild with `tgrep index .`.

AGENTS.md:188

• This is not an object-argument JSON Schema: pattern, path, and flags must be nested under properties; as written they are unknown schema keywords, so the schema does not constrain or describe the tool arguments. Also mark pattern required so the model cannot issue an argument-less search.

  "parameters": {
    "pattern": {"type": "string"},
    "path": {"type": "string", "default": "."},
    "flags": {"type": "array", "items": {"type": "string"}}
  }

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Balanced

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

AGENTS.md:203

• Returning only stdout hides the fallback warnings described above and every diagnostic accompanying exit code 2, so the model cannot distinguish a clean search from degraded or failed execution. Include stderr and the exit code in the tool result.

AGENTS.md:112

• This universal compatibility claim conflicts with the documented invalid-UTF-8 behavior in README.md:613-626. tgrep emits repaired lines.text and decoded-text submatch offsets where ripgrep emits base64 lines.bytes and source offsets, so a parser that derives source positions using ripgrep semantics can produce incorrect offsets. Qualify compatibility to UTF-8 input and point readers to the divergence.

Any parser written for `rg --json` works as is.

AGENTS.md:158

• --no-max-filesize also defines index membership and must be aligned across index creation, serving, and search. If an index was built with the default 64 MiB cap, a search using --no-max-filesize does not bypass that index and can silently miss larger files. Include both forms of the size-limit option in this rule.

- `--index-path`, `--max-filesize`, `--no-require-git`: `index`, `serve` and
  every search.

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

Suppressed comments (4)

Previously missed (1) — in code that hasn't changed since the last review.

AGENTS.md:203

• Returning only stdout hides the diagnostics this guide relies on, including bad-regex errors and the no-index warning, and prevents the wrapper from reliably distinguishing exit code 2 from empty results. Return stderr and the exit code as well, and surface code 2 as an error.

AGENTS.md:32

• A cold server build does not expose a growing partial index: bootstrap_index_build deliberately keeps queries on an empty index until publication (tgrep-cli/src/serve.rs:6086-6089). Distinguish this from resuming a partial index so agents understand that a cold-start search can return zero results until indexing completes.

can differ, though: an on-disk index omits changes since its last build, and a
server that is still building its index answers from partial data. `tgrep
status .` shows `Indexing: complete` once the initial build is done, but that

AGENTS.md:62

• The on-disk index becomes stale after any relevant edit, not only after large changes. An ordinary agent edit can add a trigram that is absent from the old postings and therefore be missed, so the rebuild guidance should cover every change that a later search must observe.

If your agent framework cannot keep a background process alive, skip `serve`
and run `tgrep index .` instead. Searches then use the on-disk index. Re-run
`tgrep index .` after large changes (branch switch, generated code).

AGENTS.md:143

• Watcher processing is asynchronous: filesystem callbacks enqueue events for a worker (tgrep-cli/src/serve.rs:2341-2347), so an edit followed immediately by a search can still race with index updating. Avoid guaranteeing that recent edits are already visible; --no-index is the documented option when immediate freshness is required.

  event. Edits made a moment ago are visible.

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Balanced

Suppressed comments (3)

AGENTS.md:37

• --no-index only bypasses the index; it still honors ignore/hidden-file rules, binary detection, and the default 64 MiB limit. Therefore it reflects current contents only for the files selected by those rules, not the filesystem “exactly” or every file.

watcher events missed later. When a search must reflect the
filesystem exactly as it is now, pass `--no-index`. It scans every file and
is slow on large trees, so use it deliberately.

AGENTS.md:87

• The reserved subcommand list is incomplete: search and count-files are also subcommands, so using either as the first bare pattern is misparsed for the same reason. Include all subcommand names so agents know when -- is required.

- **Put `--` before the pattern** and pass the search root explicitly. Shell
  quotes do not stop the parser from reading a bare `serve`, `index` or
  `status` as a subcommand; `tgrep -- serve .` searches for the word.

AGENTS.md:182

• This warning can also mean a server is running under a custom --index-path that the search omitted; the client then checks the default location and emits this message. The current cause/fix incorrectly tells users to start a second server instead of passing the matching path.

| `warning: no index at ... - scanning every file` | No index and no server | Run `tgrep index .` or `tgrep serve .` |

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

Suppressed comments (2)

AGENTS.md:139

• This overstates which options automatically bypass the index. --no-ignore-messages and --no-ignore-files do not enter the bypass condition, and -E auto remains index-compatible. As written, agents may unnecessarily assume these options force a slow scan.

- `--hidden`, `--no-ignore` and variants, `-u`/`-uu`/`-uuu`

AGENTS.md:25

• A previously running server can persist watcher updates into .tgrep/ via its auto-save loop, so freshness is not necessarily tied to the last explicit tgrep index run. Describe this as the last persisted update to avoid misstating what the local index contains.

2. **On-disk index** but no server: read `.tgrep/` directly. Fast, but only as
   fresh as the last `tgrep index` run.

• Files reviewed: 2/2 changed files
• Comments generated: 3
• Review effort level: Balanced

Suppressed comments (5)

Previously missed (3) — in code that hasn't changed since the last review.

AGENTS.md:25

• A persisted index is not necessarily a completed tgrep index snapshot. If serve is interrupted after one of its checkpoint flushes, lookup.bin remains with complete = false, and ordinary local search still opens it without checking that flag. This tier can therefore return incomplete results, not merely stale ones; advise agents to rebuild or resume serve before relying on an exhaustive search.
  AGENTS.md:35
• On a warm start, serve launches reconciliation of the existing disk index in a background thread without setting the indexing status flag. Consequently, status can report Indexing: complete while startup reconciliation is still in progress, so this is not a reliable freshness signal even before considering later missed watcher events.
  AGENTS.md:118
• The unconditional parser-compatibility claim conflicts with the documented invalid-UTF-8 behavior in README.md:615-626: tgrep emits repaired lines.text, while ripgrep emits base64 lines.bytes, so consumers that rely on ripgrep's byte representation will not work unchanged.

AGENTS.md:166

• --no-ignore also determines index membership and must match between index and serve. If an index built with --no-ignore is served without it, startup can treat the ignored entries as deleted and remove them from the index (the existing warning is documented in README.md:187-197). A search that passes --no-ignore is different: it bypasses the index, as described above.

- `--exclude <DIR>`: `index` and `serve` only. Use the same value on both.
- `--index-path`, `--max-filesize`, `--no-require-git`: `index`, `serve` and
  every search.

AGENTS.md:187

• This diagnosis is not valid when a server is already running: a new file can be absent while the initial build is incomplete or while its watcher event is queued, and starting/retaining a server does not make that query current. Re-running index also does not update the running server's in-memory view. The troubleshooting step should distinguish server lag from a stale disk-only index and point freshness-sensitive searches to --no-index.

| A new file is not found | On-disk index is stale | Run `tgrep index .` or use a server |

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Balanced

Suppressed comments (3)

AGENTS.md:36

• The server does not remain empty until the entire build completes. It inserts each 500-file batch into LiveIndex (tgrep-cli/src/serve.rs:6477-6524), so searches transition from empty to partial results throughout the build. This wording should distinguish the initial empty window from the later partial-results window.

server started with no index at all answers from an empty index, so every
search returns nothing, until the first build completes. A server resuming a
partial index answers from what it has so far. `tgrep status .` shows

AGENTS.md:129

• Invalid UTF-8 is repaired once per maximal invalid subsequence, not once per bad byte (tgrep-core/src/encoding.rs:236-239). For malformed multi-byte input, the current wording gives consumers the wrong number of replacement characters and therefore the wrong expected lines.text offsets.

that is not valid UTF-8, ripgrep emits base64 `lines.bytes`, while tgrep
always emits `lines.text` with each bad byte replaced by U+FFFD. A consumer
that depends on the raw bytes of such lines will see repaired text instead.

AGENTS.md:204

• This diagnosis omits two supported cases where waiting for the indexing worker does not fix the symptom: --no-watch disables both updates and periodic reconciliation, and a silently missed event may wait up to the reconciliation deadline. Include these causes and recommend restarting serve (or using --no-index) when no watcher/reconciliation can update the running index.

| A new file is not found, server running | First build still in progress, or the watcher event is still queued | Wait, or pass `--no-index` for this search; re-running `tgrep index .` does not update a running server |

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

AGENTS.md:129

• This is not the only invalid-UTF-8 difference: paths are also converted with to_string_lossy() before JSON serialization (search.rs:1331-1336, 1783-1796), so tgrep emits repaired path.text where ripgrep can emit base64 path.bytes. Document both cases so consumers that preserve raw filenames do not treat the replacement path as exact.

AGENTS.md:231

• -q suppresses the missing-index warning (search.rs:747-749), so the warning does not always arrive with code 0 or 1. Since this guide recommends -q, the integration guidance should state that callers needing fallback diagnostics must avoid quiet mode.

Run `tgrep <flags...> -- <pattern> <path>` and return stdout, stderr and the
exit code together. Treat `1` as "no results", not as a failure. Treat `2` as
an error; stderr then carries the cause, such as a bad regex. Always pass
stderr through regardless of the exit code: the "no index" warning arrives
with code `0` or `1` and explains why a search was slow.

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

Suppressed comments (2)

AGENTS.md:162

• This freshness guarantee only applies when watching is enabled. With --no-watch, the server starts no watcher or periodic reconciliation loop, so edits after the startup refresh remain stale indefinitely; waiting for asynchronous processing will not help. Please document the unwatched-server case explicitly.

- With a **server**, results reflect the last watcher event the indexing
  worker has processed. Events are queued and applied asynchronously, so a
  search issued right after an edit can run before the index has caught up.
  Use `--no-index` when the very latest edit must be visible.

AGENTS.md:204

• This row omits --no-watch, for which “Wait” is not a valid fix: that mode disables both watcher updates and periodic reconciliation, so the new file remains absent. Include this cause and direct users to --no-index or a server restart.

| A new file is not found, server running | First build still in progress, or the watcher event is still queued | Wait, or pass `--no-index` for this search; re-running `tgrep index .` does not update a running server |

• Files reviewed: 2/2 changed files
• Comments generated: 0 new
• Review effort level: Balanced