Ledger rules classify diffs they do not describe, and a rule that stops explaining anything is invisible #372

tools/differential/compare.py classifies a diff by the first ledger rule that matches, where fields matches by subset (compare.py:469) and name_regex rules sort ahead of fields-only ones (compare.py:114). Nothing checks that the rule which claims a diff actually describes it, so a broad rule silently explains changes it says nothing about — and the run exits 0.

Found while reviewing #370, which hit it twice: once as a discovery, once by reproducing it.

Confirmed instances

1. fix(suffix-routing) claims the whole corpus. expected_since_1.4.0.toml:166 — the only fields-only rule in any ledger, fields = ["given","family","suffix"], no regex, so it matches all 751 names. Tallying classifier-of-record over every (name × field) pair, it owns 1639 of 5257 — 31% of the space. #370's diff landed on it before a rule was written:

classify('Mr. Van Nguyen', {'given','family'})
  without a #367 rule -> fix(suffix-routing) "two-token name with unambiguous trailing suffix stays suffix"

2. fix(comma-family) matches on a bare comma. expected_since_1.4.0.toml:118 — name_regex = ",", fields = ["given","title","suffix"], reaching 236 corpus names and 715 (name × field) pairs. It latently absorbs #367's own comma-bearing class:

'Dr. Do Van Johnson, MD'   moved ['given','title']  -> classified fix(comma-family)

No such name is in the corpus today, so this is latent — it becomes a green-on-regression the day one is added. That rule's prose already concedes it absorbed five CJK honorific rows it "has nothing to do with", which is documented; the bare-comma reach is not.

3. A blanket CJK rule shadows five to seven specific ones. expected_since_1.4.0.toml:62 and the 2.0.0 twin — name_regex is a bare CJK/Hangul/Kana character class, fields = ["given","middle","family"], matching all 97 CJK corpus names. It mechanically shadows fix(cjk-maiden-marker), fix(cjk-comma-compound), fix(cjk-honorific-suffix), fix(cjk-delimited-nickname), fix(cjk-fullwidth-paren-nickname), and in the 2.0.0 ledger also fix(#308/#312/#319/#320) and fix(#307/#308/#320). A future regression in Japanese or Korean given/family splitting is labelled with 2.1-era issue numbers and exits 0.

The structural gap: a rule that stops explaining anything is invisible

_CORPUS_CLAIMS in tests/v2/test_ledger_guards.py records what a rule's regex matches, which is independent of the parser. Reverting #370's fix and re-running showed 5 unit tests failing (good) but no ledger or differential failure — the 2.1.0 baseline reports 0 diffs, 0 unexplained, exit 0, with its only rule now explaining nothing. Deleting a rule is caught (test_every_rule_claims_the_recorded_share_of_the_corpus); a rule going inert is not.

Two cheap mechanical checks

Full over-claim detection is human judgement — no code decides whether "lone post-comma piece routes to suffix/title" describes Dr. Do Van Johnson, MD. But two checks are not judgement, and either would have surfaced instances 1 and 2 at authoring time:

1. Report ambiguity. classify() returns on first match. Have it collect every matching rule and print AMBIGUOUS 'Mr. Van Nguyen' -> [fix(#367), fix(suffix-routing)] alongside the classification. Exit code unchanged; the operator sees the overlap without having to delete a rule to discover it.
2. A specificity floor for fields-only rules. validate_rules (compare.py:362) already rejects a fields list naming all seven roles, but permits a fields-only rule with six. A fields-only rule reaching every corpus name is the shape most worth a warning, or an explicit declaration of unbounded reach.

A --strict mode asserting every rule fired at least once would close the third gap.

Note

#370 tightened its own fix(#367) rule after this was found — it originally reached 11 corpus names including Vincent van Gogh, which AGENTS.md:148 names as a standard regression canary — so the new rule is not an instance. The three above are pre-existing.