Repository navigation
perf(security): memoize the pure text predicates behind security views - #570
Merged
Merged
Conversation
The obfuscation scan classifies every character of every file, several times per scan, through `_is_token_gap_character`. That predicate calls `_is_unconditionally_ignored`, which runs a compiled-regex `fullmatch` and a `unicodedata.category` lookup per character. Profiling one 237 KB skill showed 18.3M `fullmatch` calls, 20.4M `unicodedata.category` calls, and 218.8M Python function calls in total. Two changes, both behaviour-preserving: - Resolve ASCII characters from frozensets built at import. The tables are derived from the predicates themselves, so they cannot drift from them. - Skip `_token_bridging_gap_spans` entirely when the text contains no character that could be a gap. No printable ASCII code point is a token-gap character — below 0x80 only the 30 control code points qualify — so pure-ASCII prose has no spans to yield and the per-character walk is pure overhead. Measured on 901 real skills, two alternating runs per arm on an idle 8-core host: p95 -14.9% (5.60s -> 4.77s), p99 -21.2% (18.28s -> 14.41s), total CPU -5.0%. Findings are byte-identical across the corpus (8,152 on both arms) and so are coverage-ledger outcomes (176 on both), with zero errors. Correctness is pinned by tests asserting the fast paths agree with the underlying computation for every ASCII code point and across the non-ASCII ranges that carry format, separator and ignorable characters, and that the whole-string skip does not change the spans yielded. Verified exhaustively over all 1,114,112 code points while developing: zero mismatches. This is a partial improvement, not a full fix. The remaining cost is spread across the multi-pass structure of `_obfuscated_instruction_matches` and `_obfuscated_action_completions` rather than any single hot predicate. Signed-off-by: Steven Moy <[email protected]>
A scan runs every analyzer over the same file content, and each one asks the
same questions about that text again. Instrumenting one 237 KB skill:
_obfuscated_instruction_matches 832 calls over 23 distinct texts 36.2x
_has_letter_spacing_run 780 calls over 23 distinct texts 33.9x
_requires_normalized_security_view 650 calls over 20 distinct texts 32.5x
These are pure functions of the text returning immutable values, so the answers
are memoized with a bounded `lru_cache` rather than recomputed.
`_obfuscated_instruction_matches` is deliberately left alone: it is a generator,
and caching it would hand an exhausted iterator to the second consumer. Only the
existence check that `security_text_views` needs is memoized, as a new
`_has_obfuscated_instruction` returning a plain bool. The two call sites that
iterate the matches in full, including the one that passes a runtime budget, are
untouched.
The cache is bounded (64 entries) so a long-lived process cannot retain file
content indefinitely.
Measured on 901 real skills, two alternating runs per arm on an idle 8-core
host, against upstream main:
p95 p99 CPU
NVIDIA#569 -10.0% -21.9% -5.4%
+ this -28.5% -42.4% -17.8%
Sweep wall-clock over the corpus falls from 313s to 250s. Findings are
byte-identical across all three arms (8,152), as are coverage-ledger outcomes
(176) and error counts (0).
Tests assert the memoized predicates agree with the underlying generator and
span scans, that repeated calls return what the first uncached call returned,
that distinct texts do not share an answer, that the views built from them are
unchanged, and that the caches stay bounded. Full suite: 5739 passed.
Signed-off-by: Steven Moy <[email protected]>
smoy
added a commit
to smoy/SkillSpector
that referenced
this pull request
Sep 23, 2026
… a regex class
Three whole-text fast paths in the security scan, all behaviour-preserving.
1. Token-gap scanning seeks the next candidate with a compiled regex instead of
stepping through the text one character at a time in Python. Every token-gap
character lies outside printable ASCII and tab/newline/carriage-return, so the
seek class is a superset and each hit is still confirmed by the exact
predicate. Verified: no code point in Unicode is a gap character that the
seek class fails to match.
2. `_ASCII_CONFUSABLE_PATTERN` is replaced by a frozenset membership test. The
question asked of it is only ever "does this text contain any of these", and
the class holds 1,515 code points spanning 528 disjoint ranges, so as a regex
it costs a bounded scan per character. On 180 KB of ASCII prose:
regex character class 164.30 ms
range-compressed class 44.97 ms
frozenset.isdisjoint 0.70 ms
The class contains no ASCII code point at all, so ordinary text answers with
a single disjointness check. This lifts `_requires_normalized_security_view`
from 166.31 ms to 2.78 ms on that text, a 60x improvement, and the regex
accounted for 164 of those 166 ms.
3. `normalized_security_view` and the letter-spacing span scan are memoized, in
the manner of NVIDIA#570. The view is a frozen dataclass whose `source_offsets`
array is only ever read -- sliced, or copied into a fresh array -- so callers
can share one instance. A caller passing `check_runtime` bypasses the cached
path so runtime budgets are still enforced.
Measured on 901 real skills, two alternating runs per arm on an idle 8-core
host, against upstream main: p95 -8.8%, p99 -12.1%, total scan time -4.1%.
Findings are byte-identical (8,152 on both arms), as are coverage-ledger
outcomes (176) and error counts (0).
The corpus-level gain is much smaller than the microbenchmarks because the
predicate that improves 60x is already memoized by NVIDIA#570, so it runs about 23
times per scan rather than 780. It is still the single most expensive thing left
in that predicate, and the regex class would cost proportionally more on larger
files.
Tests assert the seek class covers every token-gap code point in Unicode, that
gap spans are unchanged across 1,500 randomized texts, that confusable
membership matches the original character class across 1,500 more, that the
memoized spans equal the uncached scan, and that a runtime budget is still
honoured. Full suite: 5808 passed.
Signed-off-by: Steven Moy <[email protected]>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
A scan runs every analyzer over the same file content, and each analyzer independently asks the same questions about that text. Instrumenting one 237 KB skill:
_obfuscated_instruction_matches_has_letter_spacing_run_requires_normalized_security_viewThese are pure functions of the text returning immutable values, so their answers are memoized with a bounded
lru_cacheinstead of recomputed._obfuscated_instruction_matchesis deliberately not cached. It is a generator, and caching it would hand an exhausted iterator to the second consumer. Only the existence check thatsecurity_text_viewsneeds is memoized, as a new_has_obfuscated_instructionreturning a plainbool. The two call sites that iterate the matches in full — including the one inartifact_integritythat passes a runtime budget — are untouched.The cache is bounded at 64 entries so a long-lived process cannot retain file content indefinitely.
Root Cause Analysis
security_text_viewsis the common entry point for the obfuscation reconstruction, and it recomputes_has_letter_spacing_run,_requires_normalized_security_viewand the obfuscated-instruction existence check on every call. Because each analyzer reaches it independently with the same file content, the underlying scans run dozens of times over identical input within a single scan.This is the structural half of the 2.11.1 scan-time increase. #569 addressed the per-character constant factor; this addresses the repetition. Function calls on the same 237 KB skill:
mainTest Coverage & Verification
New
tests/nodes/analyzers/test_security_text_predicate_cache.py, over 12 texts covering letter spacing, dotted and hyphenated separators, zero-width joiners, soft hyphens, bidi overrides, precomposed accents, fullwidth forms and CRLF:security_text_viewsbuilds the same views across repeated calls.Full suite:
5739 passed, 14 skipped, 39 deselected, 4 xfailed.ruff check,ruff format --checkandmypyclean.End-to-end on real skills. 901 real skills, two alternating runs per arm on an idle 8-core host:
mainAgainst upstream
main: p95 −28.5%, p99 −42.4%, total CPU −17.8%. Sweep wall-clock over the corpus falls from 313s to 250s.Findings are byte-identical across all three arms — 8,152 — as are coverage-ledger outcomes (176 units) and error counts (0).
Scope
Together with #569 this recovers most of the tail regression introduced in 2.11.1. On this corpus p95 per scan was 2.53s at 2.11.0 and 5.30s at 2.11.1; these two changes bring
mainto 3.84s. The remainder is the genuine cost of the additional analysis rather than repeated or per-character overhead, so I have stopped here rather than reaching for the multi-pass structure of_obfuscated_action_completionsitself.