zstd: decode at full speed into exact-size destinations - #1246

DecodeAll into a destination whose capacity is exactly the decoded size ran much slower than the same call with 16 bytes to spare. This is the natural buffer for a caller that knows the decompressed size, for example from the frame header or its own metadata.

(1) zstd: bound the decodeSync slack check by the block, not the frame (7ef7a21). When the frame content size is known, useSafeDecodeSync asked for compressedBlockOverAlloc bytes of slack beyond the whole rest of the frame (maxSyncLen). With an exact-size destination that is never true, so every block ran the bounds-exact copies; on Neoverse N1 that path was 60% of decode time. The extended copies only need the slack past the block being decoded, which produces at most min(maxSyncLen, maxBlockSize) bytes. The assembly still checks every sequence against cap-compressedBlockOverAlloc on that path, so a frame that lies about its size fails as before. Test: TestUseSafeDecodeSyncFrameBound in zstd/seqdec_test.go.

The same change fixes multi-frame input decoded in one DecodeAll call. maxSyncLen starts at FrameContentSize + len(dst), so for every frame after the first it overstated what was left by all the output already written. The old whole-frame check then sent later frames to the bounds-exact copies even when the buffer had plenty of room.

(2) zstd: don't reallocate the output for missing slack in executeSimple (1720e5a). executeSimple grew s.out whenever the block plus compressedBlockOverAlloc did not fit. With an exact-size destination that happens on the frame's last block: the growth copied the entire decoded frame into a new buffer about twice its size, and DecodeAll returned that copy instead of the caller's buffer. It now grows only when the block itself does not fit, and uses the bounds-exact copies when only the slack is missing. DecodeAll reaches this through the two-pass path, which runs on arm64 for windows of 1 MiB and up. Test: TestDecodeAllExactCapacity in zstd/decoder_test.go, which forces the two-pass path so it also runs on amd64. A panic planted in the new branch confirmed the test reaches it.

Both tests fail on master and pass here. go test ./zstd passes in the default, noasm and nounsafe builds on amd64 and arm64.

Performance

Data: 64 MiB slices of apache.log, github-june-2days-2019.json, gob-stream and nyc-taxi-data-10M.csv from https://klauspost.com/files/compress/, in two shapes:

• single frames encoded by this package at SpeedDefault and SpeedBetterCompression;
• 4 MiB frames with a 4 MiB window, concatenated.

All CRC off. BenchmarkCorpus's DecodeAll preallocates exactly the decoded size. Each run is 10 interleaved rounds pinned to one core, compared with benchstat.

This PR vs master, B/s geomean over the 16 files. No regressed cells (0 of 64).

For comparison on N1, giving master's destination 16 spare bytes recovers +10.4% on the single frames (the fix matches all but the last block, which still needs the exact copies), but nothing on the multi-frame files.

Callers that already leave slack are unaffected, because both changes make the same choices for them. That includes DecodeAll with a nil or too-small dst, which allocates FrameContentSize+16 itself. Measured with per-frame DecodeAll into a reused buffer on the same data, with PCALIGN $64 in both builds:

• Graviton5 flat;
• Graviton3 +0.2…+0.5% in 6 of 8 cells;
• Graviton2 −0.5…−0.8% in 3 cells;
• N1 −0.6…−1.4% in 6 cells.

The only added work is one compare per block, so I believe that spread is the placement of the Go code, which PCALIGN doesn't pin, but I haven't isolated it.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved compressed-data decoding when the output buffer has exactly enough space for the decoded data, including when it already contains a prefix.
  • Reduced unnecessary buffer growth while maintaining safe decoding when available space is limited.