Skip to content

[RESP] Fix InlinePing SIMD-scan regression and MGET per-key boxing allocation - #1988

Merged
Badrish Chandramouli (badrishc) merged 5 commits into
mainfrom
badrishc/fix-inlineping-mget-perf
Jul 25, 2026
Merged

Badrish Chandramouli (badrishc) merged 5 commits into
mainfrom
badrishc/fix-inlineping-mget-perf

Conversation

@badrishc

Copy link
Copy Markdown
Collaborator

Summary

Two independent performance fixes for regressions surfaced by the BDN CI benchmarks, both in libs/server/Resp/. Split into two commits (one per fix).


1. InlinePing SIMD-scan regression — RespCommand.cs

Root cause: #1658 added an unconditional SIMD Vector128 scan at the top of FastParseCommand, gated only on remainingBytes >= 16. Every SIMD pattern is RESP array-framed (*N\r\n$L\r\nCMD\r\n), so an inline command such as PING\r\n (starts with a command letter, not *) can never match. In a batched inline workload the buffer is always ≥ 16 bytes, so each inline command paid a NoInlining call plus ~18 vector comparisons before falling through to FastParseInlineCommand.

Fix: Gate the SIMD scan on a leading *. Inline commands skip it and go straight to FastParseInlineCommand (pre-#1658 behavior). This loses no SIMD matches (all patterns require *) and adds only a single byte compare to the array-framed hot path.

Measured — Operations.BasicOperations.InlinePing (None), net10.0:

Version Mean
pre-#1658 1.64 µs
#1658 (regressed) 2.22 µs (+35%)
fixed 1.73 µs

Array-framed commands (GetNotFound/GetFound) verified unaffected.


2. MGET per-key struct boxing allocation — MGetReadArgBatch.cs

Root cause: MGetReadArgBatch_SG is a struct implementing IReadArgBatch, whose InitialIORecordSize and ReadCopyOptions are default interface methods. The struct did not implement them, so calling them through the generic TBatch constraint in ContextReadWithPrefetch boxed the struct on every call. InitialIORecordSize is read once per key in the read loop, so a 100-key MGET boxed 100× (plus one box for ReadCopyOptions) → 4848 B allocated per MGET, tripping the BDN allocation gate (expected 0) for Cluster.ClusterOperations.MGet and Cluster.ClusterMigrate.MGet.

Fix: Explicitly implement both members on the struct (returning the same values as the interface defaults) so the calls bind directly and no longer box. This matches the existing VectorReadBatch implementation.

Measured: MGet allocation 4848 B → 0 B; mean also improves ~13.3 → 11.95 µs.


Key technical details

Affected types:

  • RespServerSession.FastParseCommand (libs/server/Resp/Parser/RespCommand.cs)
  • MGetReadArgBatch_SG : IReadArgBatch (libs/server/Resp/MGetReadArgBatch.cs)

Both fixes are behavior-preserving: identical parse results, and the MGET members return the same values as the interface defaults.

What NOT to do (for future agents)

  • ❌ Don't run the SIMD command scan for buffers that aren't *-framed — inline commands can never match array-framed patterns, so the scan is pure overhead.
  • ❌ Don't rely on IReadArgBatch default interface members from a struct batch — calling a default interface method on a value type through a generic constraint boxes it (once per call). Implement the members explicitly.

Testing

Issues Fixed

These regressions were surfaced by the BDN charts / CI perf gate rather than tracked issues. Link an issue here if one exists.

…ion)

PR #1658 added an unconditional SIMD Vector128 scan at the top of
FastParseCommand that runs whenever the receive buffer has >= 16 bytes.
Every SIMD pattern is RESP array-framed ("*N\r\n$L\r\nCMD\r\n"), so an
inline command such as "PING\r\n" (which starts with a command letter,
not '*') can never match any pattern. In a batched inline workload the
buffer is always >= 16 bytes, so each inline command paid a NoInlining
call plus ~18 vector comparisons before falling through to
FastParseInlineCommand.

Gate the SIMD scan on a leading '*' so inline commands skip it and go
straight to FastParseInlineCommand, as they did before #1658. This loses
no SIMD matches (all patterns require '*') and adds only a single byte
compare to the array-framed hot path.

BDN Operations.BasicOperations.InlinePing (None), net10.0:
  pre-#1658 1.64us -> #1658 2.22us (+35%) -> fixed 1.73us.
Array-framed commands (GetNotFound/GetFound) are unaffected.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
MGetReadArgBatch_SG is a struct implementing IReadArgBatch, whose
InitialIORecordSize and ReadCopyOptions members are default interface
methods. The struct did not implement them, so calling them through the
generic TBatch constraint in ContextReadWithPrefetch boxed the struct on
every invocation. InitialIORecordSize is read once per key inside the
read loop, so a 100-key MGET boxed 100 times (plus one box for
ReadCopyOptions) -> 4848 B allocated per MGET, tripping the BDN
allocation gate (expected 0) for Cluster.ClusterOperations.MGet and
Cluster.ClusterMigrate.MGet.

Explicitly implement both members on the struct (returning the same
values as the interface defaults) so the calls bind directly and no
longer box. This matches the existing VectorReadBatch implementation.

Result: MGet allocation 4848 B -> 0 B; mean also improves ~13.3 ->
11.95 us. ScatterGatherMGet (disk/async completion) tests still pass.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
Copilot AI review requested due to automatic review settings July 25, 2026 00:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses two RESP-layer performance regressions identified by benchmark CI: (1) avoids unnecessary SIMD scanning for inline (non-*-framed) commands in the command parser, and (2) removes per-key allocations in MGET scatter/gather by implementing IReadArgBatch default-interface members directly on the struct batch type.

Changes:

  • Gate the SIMD SimdFastParse path in FastParseCommand on a leading *, so inline commands fall directly to FastParseInlineCommand.
  • Implement InitialIORecordSize and ReadCopyOptions on MGetReadArgBatch_SG to prevent struct boxing when invoked through the generic TBatch : IReadArgBatch<...> constraint.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
libs/server/Resp/Parser/RespCommand.cs Skips SIMD pattern scan unless the buffer is array-framed (*), restoring inline parsing performance without affecting SIMD-matchable commands.
libs/server/Resp/MGetReadArgBatch.cs Adds struct implementations for IReadArgBatch DIM members to eliminate per-key boxing/allocation in MGET scatter/gather.

Condense the InitialIORecordSize/ReadCopyOptions comment to state the
reason for explicit implementation (direct binding, no struct boxing).

Co-authored-by: Copilot <[email protected]>
Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
Add a coding-style note: comments should describe what the code does and
why, in the present tense, and avoid historical narration or references
to issues/PRs/prior problems.

Co-authored-by: Copilot <[email protected]>
Copilot-Session: e66a8a41-4525-473a-9095-a5e614d95e1a
@badrishc
Badrish Chandramouli (badrishc) merged commit b6f14b9 into main Jul 25, 2026
161 checks passed
@badrishc
Badrish Chandramouli (badrishc) deleted the badrishc/fix-inlineping-mget-perf branch July 25, 2026 02:09
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants