Repository navigation
[RESP] Fix InlinePing SIMD-scan regression and MGET per-key boxing allocation - #1988
Merged
Badrish Chandramouli (badrishc) merged 5 commits intoJul 25, 2026
Merged
Conversation
…ion) PR #1658 added an unconditional SIMD Vector128 scan at the top of FastParseCommand that runs whenever the receive buffer has >= 16 bytes. Every SIMD pattern is RESP array-framed ("*N\r\n$L\r\nCMD\r\n"), so an inline command such as "PING\r\n" (which starts with a command letter, not '*') can never match any pattern. In a batched inline workload the buffer is always >= 16 bytes, so each inline command paid a NoInlining call plus ~18 vector comparisons before falling through to FastParseInlineCommand. Gate the SIMD scan on a leading '*' so inline commands skip it and go straight to FastParseInlineCommand, as they did before #1658. This loses no SIMD matches (all patterns require '*') and adds only a single byte compare to the array-framed hot path. BDN Operations.BasicOperations.InlinePing (None), net10.0: pre-#1658 1.64us -> #1658 2.22us (+35%) -> fixed 1.73us. Array-framed commands (GetNotFound/GetFound) are unaffected. Co-authored-by: Copilot <[email protected]> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
MGetReadArgBatch_SG is a struct implementing IReadArgBatch, whose InitialIORecordSize and ReadCopyOptions members are default interface methods. The struct did not implement them, so calling them through the generic TBatch constraint in ContextReadWithPrefetch boxed the struct on every invocation. InitialIORecordSize is read once per key inside the read loop, so a 100-key MGET boxed 100 times (plus one box for ReadCopyOptions) -> 4848 B allocated per MGET, tripping the BDN allocation gate (expected 0) for Cluster.ClusterOperations.MGet and Cluster.ClusterMigrate.MGet. Explicitly implement both members on the struct (returning the same values as the interface defaults) so the calls bind directly and no longer box. This matches the existing VectorReadBatch implementation. Result: MGet allocation 4848 B -> 0 B; mean also improves ~13.3 -> 11.95 us. ScatterGatherMGet (disk/async completion) tests still pass. Co-authored-by: Copilot <[email protected]> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
Copilot started reviewing on behalf of
Badrish Chandramouli (badrishc)
July 25, 2026 00:14
View session
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses two RESP-layer performance regressions identified by benchmark CI: (1) avoids unnecessary SIMD scanning for inline (non-*-framed) commands in the command parser, and (2) removes per-key allocations in MGET scatter/gather by implementing IReadArgBatch default-interface members directly on the struct batch type.
Changes:
- Gate the SIMD
SimdFastParsepath inFastParseCommandon a leading*, so inline commands fall directly toFastParseInlineCommand. - Implement
InitialIORecordSizeandReadCopyOptionsonMGetReadArgBatch_SGto prevent struct boxing when invoked through the genericTBatch : IReadArgBatch<...>constraint.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| libs/server/Resp/Parser/RespCommand.cs | Skips SIMD pattern scan unless the buffer is array-framed (*), restoring inline parsing performance without affecting SIMD-matchable commands. |
| libs/server/Resp/MGetReadArgBatch.cs | Adds struct implementations for IReadArgBatch DIM members to eliminate per-key boxing/allocation in MGET scatter/gather. |
Ted Hart (TedHartMS)
approved these changes
Jul 25, 2026
Condense the InitialIORecordSize/ReadCopyOptions comment to state the reason for explicit implementation (direct binding, no struct boxing). Co-authored-by: Copilot <[email protected]> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
Add a coding-style note: comments should describe what the code does and why, in the present tense, and avoid historical narration or references to issues/PRs/prior problems. Co-authored-by: Copilot <[email protected]> Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
Ted Hart (TedHartMS)
approved these changes
Jul 25, 2026
Badrish Chandramouli (badrishc)
deleted the
badrishc/fix-inlineping-mget-perf
branch
July 25, 2026 02:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two independent performance fixes for regressions surfaced by the BDN CI benchmarks, both in
libs/server/Resp/. Split into two commits (one per fix).1. InlinePing SIMD-scan regression —
RespCommand.csRoot cause: #1658 added an unconditional SIMD
Vector128scan at the top ofFastParseCommand, gated only onremainingBytes >= 16. Every SIMD pattern is RESP array-framed (*N\r\n$L\r\nCMD\r\n), so an inline command such asPING\r\n(starts with a command letter, not*) can never match. In a batched inline workload the buffer is always ≥ 16 bytes, so each inline command paid aNoInliningcall plus ~18 vector comparisons before falling through toFastParseInlineCommand.Fix: Gate the SIMD scan on a leading
*. Inline commands skip it and go straight toFastParseInlineCommand(pre-#1658 behavior). This loses no SIMD matches (all patterns require*) and adds only a single byte compare to the array-framed hot path.Measured —
Operations.BasicOperations.InlinePing (None), net10.0:Array-framed commands (
GetNotFound/GetFound) verified unaffected.2. MGET per-key struct boxing allocation —
MGetReadArgBatch.csRoot cause:
MGetReadArgBatch_SGis astructimplementingIReadArgBatch, whoseInitialIORecordSizeandReadCopyOptionsare default interface methods. The struct did not implement them, so calling them through the genericTBatchconstraint inContextReadWithPrefetchboxed the struct on every call.InitialIORecordSizeis read once per key in the read loop, so a 100-key MGET boxed 100× (plus one box forReadCopyOptions) → 4848 B allocated per MGET, tripping the BDN allocation gate (expected 0) forCluster.ClusterOperations.MGetandCluster.ClusterMigrate.MGet.Fix: Explicitly implement both members on the struct (returning the same values as the interface defaults) so the calls bind directly and no longer box. This matches the existing
VectorReadBatchimplementation.Measured: MGet allocation 4848 B → 0 B; mean also improves ~13.3 → 11.95 µs.
Key technical details
Affected types:
RespServerSession.FastParseCommand(libs/server/Resp/Parser/RespCommand.cs)MGetReadArgBatch_SG:IReadArgBatch(libs/server/Resp/MGetReadArgBatch.cs)Both fixes are behavior-preserving: identical parse results, and the MGET members return the same values as the interface defaults.
What NOT to do (for future agents)
*-framed — inline commands can never match array-framed patterns, so the scan is pure overhead.IReadArgBatchdefault interface members from a struct batch — calling a default interface method on a value type through a generic constraint boxes it (once per call). Implement the members explicitly.Testing
RespTests(347),RespParseFuzzRegressionTests(4), andRespGetLowMemoryTests.ScatterGatherMGet(disk/async completion, 30/300/3000 keys) all pass.InlinePingrecovered to ~pre-[RESP] Optimize command parsing with SIMD fast path and O(1) hash table lookup #1658;ClusterOperations.MGet/ClusterMigrate.MGetallocation now0.dotnet format --verify-no-changesclean; Release build has no new warnings.Issues Fixed
These regressions were surfaced by the BDN charts / CI perf gate rather than tracked issues. Link an issue here if one exists.