Implement Adaptive Cumulator - #16731
Merged
Merged
Conversation
Contributor
|
Some related unfinished work: #16076 |
shivaspeaks
marked this pull request as ready for review
April 30, 2026 11:48
sauravzg
reviewed
May 4, 2026
shivaspeaks
force-pushed
the
adaptive-cumulator
branch
from
May 12, 2026 09:55
4ce24d0 to
bec6047
Compare
Contributor
Author
|
While I ask for the re-review, letting you know that I'm working on your comment to test the public behaviors and make private functions as truly private by changing the access modifier to |
Contributor
Author
|
@normanmaurer would you mind taking a look at this when you have a moment please. |
yawkat
reviewed
May 15, 2026
yawkat
reviewed
May 15, 2026
ejona86
reviewed
Jun 4, 2026
ejona86
left a comment
Member
There was a problem hiding this comment.
Sending what I have, since these comments are already getting old. Still need to look at it more.
sauravzg
approved these changes
Jun 22, 2026
chrisvest
reviewed
Jun 24, 2026
chrisvest
reviewed
Jun 24, 2026
Member
|
More checkstyle issues |
Contributor
Author
|
@chrisvest can you take a look now and help me merge it if everyhting looks good? |
chrisvest
reviewed
Jun 30, 2026
chrisvest
approved these changes
Jul 1, 2026
Member
|
Very nice, thanks! |
Contributor
|
Auto-port PR for 5.0: #17033 |
chrisvest
pushed a commit
that referenced
this pull request
Jul 2, 2026
Auto-port of #16731 to 5.0 Cherry-picked commit: bc4b983 --- **Motivation:** This PR introduces the `AdaptiveCumulator`, a specialized Cumulator implementation for `ByteToMessageDecoder` that dynamically switches between merge (memory copy) and compose (zero-copy) strategies based on the size of incoming data. Standard cumulators in Netty present a trade-off: the `MERGE_CUMULATOR` is safe but can consume up to 10% of CPU time due to excessive copies during high-throughput transfers. Conversely, the `COMPOSITE_CUMULATOR` provides zero-copy benefits but is vulnerable to "Single Byte Attacks," where an attacker sends messages one byte at a time to force the server to allocate excessive object metadata. The `AdaptiveCumulator` applies a heuristic to achieve the best of both worlds: - **Merge (Copy):** If the total size of the tail component and the incoming buffer is below a configurable threshold (default 1024 bytes), the data is copied into a new buffer to minimize object overhead. - **Compose (Zero-Copy):** Once the threshold is reached, the buffer is added as a new component without copying, maximizing throughput. **Modification:** - Added `AdaptiveCumulator.java` to the `io.netty.handler.codec` package. - Added `AdaptiveCumulatorTest.java`, a comprehensive test suite modernized for JUnit 5. This PR is an attempt to upstream what we were trying to do in gRPC grpc/grpc-java#9558, but the behaviour of `duplicate()` method on slices was changed/corrected in #14093 which invalidated previous manual index workarounds and since then it has been a tech debt in grpc-java. This implementation is "PR #14093-aware" and correctly handles the capacity-capping introduced by that change by verifying the state of the tail component before expansion. This implementation is specifically designed to address historical failure modes associated with `CompositeByteBuf` manipulation: - **Modernized Index Synchronization with componentSlice()** Previous attempts to implement this logic relied on fragile internal hacks like `internalComponent().duplicate()` to correct stale indices returned by `CompositeByteBuf.component()`. This PR leverages the `componentSlice()` method (introduced in Netty 4.2.4) to obtain a correctly indexed view through a public, supported API. - **Protection Against "Resurrected" Bytes** To prevent silent data corruption, we have introduced a capacity safety check: `tail.capacity() == componentView.capacity()`. If a mismatch is detected, it indicates the component is a partial slice containing hidden "discarded" bytes. In such cases, the logic correctly falls back to a safe reallocation to "cleanse" the data, preventing discarded bytes from reappearing in the message. This particular problem was caught in the unit test `mergeWithCompositeTail_detectsIndicesInNestedComposite()` while trying to invalidate the use of previous way (`internalComponent.duplicate()`) as in grpc/grpc-java#9558. - **Prevention of "Russian Doll" Nesting** The implementation strictly uses `addFlattenedComponents` to ensure the `CompositeByteBuf` remains a flat, linear structure. This avoids the recursive deallocation crashes (StackOverflow) that occurred in previous versions when buffers were nested thousands of layers deep. - **Single-Owner Lifecycle Management** The lifecycle of the input buffer is managed using a single-owner pattern in the top-level cumulate method. Sub-methods like `mergeWithCompositeTail` no longer release the input buffer in their finally blocks, which we were doing otherwise in grpc/grpc-java#9558. This prevents `IllegalReferenceCountException` (double-release) crashes during error scenarios. **Usage** Downstream projects can enable the `AdaptiveCumulator` by calling the `setCumulator()` method on any handler that extends `ByteToMessageDecoder`. For example, to use the adaptive strategy with a 1KB threshold: `setCumulator(new AdaptiveCumulator(1024));` CC: @ejona86, @sauravzg Co-authored-by: MV Shiva <[email protected]>
mergify Bot
added a commit
to ArcadeData/arcadedb
that referenced
this pull request
Jul 8, 2026
…l [skip ci] Bumps [io.netty:netty-all](https://github.com/netty/netty) from 4.2.15.Final to 4.2.16.Final. Release notes *Sourced from [io.netty:netty-all's releases](https://github.com/netty/netty/releases).* > netty-4.2.16.Final > ------------------ > > What's Changed > -------------- > > * Document Java 9 requirement for io\_uring by [`@jchambers`](https://github.com/jchambers) in [netty/netty#16904](https://redirect.github.com/netty/netty/pull/16904) > * Add BlockHound exception for DnsQueryIdSpace by [`@violetagg`](https://github.com/violetagg) in [netty/netty#16896](https://redirect.github.com/netty/netty/pull/16896) > * Fix incorrect bounds in error message of HpackDecoder.setMaxHeaderListSize by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16901](https://redirect.github.com/netty/netty/pull/16901) > * Add epoch-based chunk cache purge with ring buffer for thread-local reuse by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16766](https://redirect.github.com/netty/netty/pull/16766) > * Use Splittable/ThreadLocalRandom to generate bulk data in tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16808](https://redirect.github.com/netty/netty/pull/16808) > * Auto-port 4.2: SingleThreadEventExecutor: document Throwable safety contract on run() by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16909](https://redirect.github.com/netty/netty/pull/16909) > * Auto-port 4.2: Make HTTP/2 frame hashCode consistent with equals by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16910](https://redirect.github.com/netty/netty/pull/16910) > * Auto-port 4.2: MQTT: Make the decodeProperties early-REPLAY check actually fire by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16919](https://redirect.github.com/netty/netty/pull/16919) > * IoUring: fix io\_uring datagram writes with non-zero readerIndex by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16905](https://redirect.github.com/netty/netty/pull/16905) > * Exclude internal events from IoHandler.run() return value in epoll, io\_uring and kqueue by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16848](https://redirect.github.com/netty/netty/pull/16848) > * IoUring: Pass IORING\_ENTER\_NO\_IOWAIT to report accurate CPU usage by [`@wineway`](https://github.com/wineway) in [netty/netty#16739](https://redirect.github.com/netty/netty/pull/16739) > * Avoid logging exceptions that tests ignore by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16891](https://redirect.github.com/netty/netty/pull/16891) > * Reject control characters at the boundary of HTTP method names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16723](https://redirect.github.com/netty/netty/pull/16723) > * IoUring: fix TCP Fast Open initial writes with readerIndex and composites by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16929](https://redirect.github.com/netty/netty/pull/16929) > * Try to fix/stabilize a number of flaky tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16934](https://redirect.github.com/netty/netty/pull/16934) > * Fix propagation of startTls for client SslContext handlers by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16931](https://redirect.github.com/netty/netty/pull/16931) > * Update to latest tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16936](https://redirect.github.com/netty/netty/pull/16936) > * Move test to shared testsuite by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16928](https://redirect.github.com/netty/netty/pull/16928) > * [Refactor] Useful helper method getOrDefault & cleaner abstraction by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#16927](https://redirect.github.com/netty/netty/pull/16927) > * Make permessage-deflate server window size and memLevel configurable by [`@fru1tworld`](https://github.com/fru1tworld) in [netty/netty#16809](https://redirect.github.com/netty/netty/pull/16809) > * Return early in DnsQueryContext.writeQuery when the query ID space is exhausted by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16950](https://redirect.github.com/netty/netty/pull/16950) > * Fix HTTP 2 PUSH\_PROMISE stream association validation by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16952](https://redirect.github.com/netty/netty/pull/16952) > * Fix GZIP FEXTRA extra-field handling in JdkZlibDecoder by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16951](https://redirect.github.com/netty/netty/pull/16951) > * Http3FrameCodec handle fragmented payloads when skipping unknown frames by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16960](https://redirect.github.com/netty/netty/pull/16960) > * Add opt-in validation of mandatory pseudo-header fields for HTTP/2 by [`@hyperxpro`](https://github.com/hyperxpro) in [netty/netty#16932](https://redirect.github.com/netty/netty/pull/16932) > * Strictly validate MQTT UTF-8 Encoded String by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16939](https://redirect.github.com/netty/netty/pull/16939) > * Stop DateFormatter trailing token from running past the parse end by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16958](https://redirect.github.com/netty/netty/pull/16958) > * IpFilter: Deprecate constructor which use accept by default by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16961](https://redirect.github.com/netty/netty/pull/16961) > * Add RFC 10008 QUERY Method support by [`@desiderantes`](https://github.com/desiderantes) in [netty/netty#16966](https://redirect.github.com/netty/netty/pull/16966) > * Correctly release and fail queued traffic-shaping writes on close by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16959](https://redirect.github.com/netty/netty/pull/16959) > * Reject control characters at the boundary of the HTTP version token by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16971](https://redirect.github.com/netty/netty/pull/16971) > * FlowControlHandler: respect auto-read when toggled while dequeueing by [`@schiemon`](https://github.com/schiemon) in [netty/netty#16949](https://redirect.github.com/netty/netty/pull/16949) > * Fix leak in ReferenceCountedOpenSslEngine.addCredential by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16979](https://redirect.github.com/netty/netty/pull/16979) > * IdleStateHandler: reset firstWriter/ReaderIdleEvent in resetWriteTimeout/resetReadTimeout by [`@husseinvr97`](https://github.com/husseinvr97) in [netty/netty#16982](https://redirect.github.com/netty/netty/pull/16982) > * Fix typo in AbstractSniHandler Javadoc by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#16988](https://redirect.github.com/netty/netty/pull/16988) > * Fix client/server inconsistency in SslCredential support matrix by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16990](https://redirect.github.com/netty/netty/pull/16990) > * Reconcile `AbstractCoalescingBufferQueue` readableBytes when it drains, and fail stuck HTTP/2 streams instead of spinning empty DATA frames by [`@gavinbunney`](https://github.com/gavinbunney) in [netty/netty#16947](https://redirect.github.com/netty/netty/pull/16947) > * Use Ticker in Http2MaxRstFrameListener for testability by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16993](https://redirect.github.com/netty/netty/pull/16993) > * Reset UTF-8 decode state on CR in StompSubframeDecoder by [`@vasiliy-mikhailov`](https://github.com/vasiliy-mikhailov) in [netty/netty#16991](https://redirect.github.com/netty/netty/pull/16991) > * FastLz: Guard decompression against truncated input by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17000](https://redirect.github.com/netty/netty/pull/17000) > * Reject non-token characters in HTTP/2 header names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16762](https://redirect.github.com/netty/netty/pull/16762) > * Auto-port 4.2: Fix SelfSignCertificate initialization in tests by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#17029](https://redirect.github.com/netty/netty/pull/17029) > * Enable extension of Http3ClientConnectionHandler to support higher-level protocols such as WebTransport. by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#17027](https://redirect.github.com/netty/netty/pull/17027) > * Implement Adaptive Cumulator by [`@shivaspeaks`](https://github.com/shivaspeaks) in [netty/netty#16731](https://redirect.github.com/netty/netty/pull/16731) > * Allow WebSocket extension negotiation to be disabled per response by [`@mkurz`](https://github.com/mkurz) in [netty/netty#17030](https://redirect.github.com/netty/netty/pull/17030) > * Support QPACK sensitivity detector for Never Indexed header fields by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17026](https://redirect.github.com/netty/netty/pull/17026) > * Fix maxAllocation for brotli-encoded content in HttpContentDecompressor by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17037](https://redirect.github.com/netty/netty/pull/17037) > * Pin github actions to reduce risk by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17043](https://redirect.github.com/netty/netty/pull/17043) ... (truncated) Commits * [`3703d79`](netty/netty@3703d79) [maven-release-plugin] prepare release netty-4.2.16.Final * [`63bbb2c`](netty/netty@63bbb2c) Update rust toolchain - add required parameters * [`ac06c1b`](netty/netty@ac06c1b) Update rust toolchain * [`5b68c61`](netty/netty@5b68c61) Merge branches from forks ([#17063](https://redirect.github.com/netty/netty/issues/17063)) * [`de5d276`](netty/netty@de5d276) Update lz4-java to 1.11.1 ([#17061](https://redirect.github.com/netty/netty/issues/17061)) * [`da22048`](netty/netty@da22048) Pin github actions to reduce risk ([#17043](https://redirect.github.com/netty/netty/issues/17043)) * [`0332676`](netty/netty@0332676) Fix maxAllocation for brotli-encoded content in HttpContentDecompressor ([#17037](https://redirect.github.com/netty/netty/issues/17037)) * [`7364401`](netty/netty@7364401) Support QPACK sensitivity detector for Never Indexed header fields ([#17026](https://redirect.github.com/netty/netty/issues/17026)) * [`06faf18`](netty/netty@06faf18) Allow WebSocket extension negotiation to be disabled per response ([#17030](https://redirect.github.com/netty/netty/issues/17030)) * [`bc4b983`](netty/netty@bc4b983) Implement Adaptive Cumulator ([#16731](https://redirect.github.com/netty/netty/issues/16731)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.15.Final...netty-4.2.16.Final) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
mergify Bot
added a commit
to ArcadeData/arcadedb
that referenced
this pull request
Jul 8, 2026
…ip ci] Bumps `netty.version` from 4.2.15.Final to 4.2.16.Final. Updates `io.netty:netty-transport` from 4.2.15.Final to 4.2.16.Final Release notes *Sourced from [io.netty:netty-transport's releases](https://github.com/netty/netty/releases).* > netty-4.2.16.Final > ------------------ > > What's Changed > -------------- > > * Document Java 9 requirement for io\_uring by [`@jchambers`](https://github.com/jchambers) in [netty/netty#16904](https://redirect.github.com/netty/netty/pull/16904) > * Add BlockHound exception for DnsQueryIdSpace by [`@violetagg`](https://github.com/violetagg) in [netty/netty#16896](https://redirect.github.com/netty/netty/pull/16896) > * Fix incorrect bounds in error message of HpackDecoder.setMaxHeaderListSize by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16901](https://redirect.github.com/netty/netty/pull/16901) > * Add epoch-based chunk cache purge with ring buffer for thread-local reuse by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16766](https://redirect.github.com/netty/netty/pull/16766) > * Use Splittable/ThreadLocalRandom to generate bulk data in tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16808](https://redirect.github.com/netty/netty/pull/16808) > * Auto-port 4.2: SingleThreadEventExecutor: document Throwable safety contract on run() by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16909](https://redirect.github.com/netty/netty/pull/16909) > * Auto-port 4.2: Make HTTP/2 frame hashCode consistent with equals by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16910](https://redirect.github.com/netty/netty/pull/16910) > * Auto-port 4.2: MQTT: Make the decodeProperties early-REPLAY check actually fire by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16919](https://redirect.github.com/netty/netty/pull/16919) > * IoUring: fix io\_uring datagram writes with non-zero readerIndex by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16905](https://redirect.github.com/netty/netty/pull/16905) > * Exclude internal events from IoHandler.run() return value in epoll, io\_uring and kqueue by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16848](https://redirect.github.com/netty/netty/pull/16848) > * IoUring: Pass IORING\_ENTER\_NO\_IOWAIT to report accurate CPU usage by [`@wineway`](https://github.com/wineway) in [netty/netty#16739](https://redirect.github.com/netty/netty/pull/16739) > * Avoid logging exceptions that tests ignore by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16891](https://redirect.github.com/netty/netty/pull/16891) > * Reject control characters at the boundary of HTTP method names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16723](https://redirect.github.com/netty/netty/pull/16723) > * IoUring: fix TCP Fast Open initial writes with readerIndex and composites by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16929](https://redirect.github.com/netty/netty/pull/16929) > * Try to fix/stabilize a number of flaky tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16934](https://redirect.github.com/netty/netty/pull/16934) > * Fix propagation of startTls for client SslContext handlers by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16931](https://redirect.github.com/netty/netty/pull/16931) > * Update to latest tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16936](https://redirect.github.com/netty/netty/pull/16936) > * Move test to shared testsuite by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16928](https://redirect.github.com/netty/netty/pull/16928) > * [Refactor] Useful helper method getOrDefault & cleaner abstraction by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#16927](https://redirect.github.com/netty/netty/pull/16927) > * Make permessage-deflate server window size and memLevel configurable by [`@fru1tworld`](https://github.com/fru1tworld) in [netty/netty#16809](https://redirect.github.com/netty/netty/pull/16809) > * Return early in DnsQueryContext.writeQuery when the query ID space is exhausted by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16950](https://redirect.github.com/netty/netty/pull/16950) > * Fix HTTP 2 PUSH\_PROMISE stream association validation by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16952](https://redirect.github.com/netty/netty/pull/16952) > * Fix GZIP FEXTRA extra-field handling in JdkZlibDecoder by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16951](https://redirect.github.com/netty/netty/pull/16951) > * Http3FrameCodec handle fragmented payloads when skipping unknown frames by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16960](https://redirect.github.com/netty/netty/pull/16960) > * Add opt-in validation of mandatory pseudo-header fields for HTTP/2 by [`@hyperxpro`](https://github.com/hyperxpro) in [netty/netty#16932](https://redirect.github.com/netty/netty/pull/16932) > * Strictly validate MQTT UTF-8 Encoded String by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16939](https://redirect.github.com/netty/netty/pull/16939) > * Stop DateFormatter trailing token from running past the parse end by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16958](https://redirect.github.com/netty/netty/pull/16958) > * IpFilter: Deprecate constructor which use accept by default by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16961](https://redirect.github.com/netty/netty/pull/16961) > * Add RFC 10008 QUERY Method support by [`@desiderantes`](https://github.com/desiderantes) in [netty/netty#16966](https://redirect.github.com/netty/netty/pull/16966) > * Correctly release and fail queued traffic-shaping writes on close by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16959](https://redirect.github.com/netty/netty/pull/16959) > * Reject control characters at the boundary of the HTTP version token by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16971](https://redirect.github.com/netty/netty/pull/16971) > * FlowControlHandler: respect auto-read when toggled while dequeueing by [`@schiemon`](https://github.com/schiemon) in [netty/netty#16949](https://redirect.github.com/netty/netty/pull/16949) > * Fix leak in ReferenceCountedOpenSslEngine.addCredential by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16979](https://redirect.github.com/netty/netty/pull/16979) > * IdleStateHandler: reset firstWriter/ReaderIdleEvent in resetWriteTimeout/resetReadTimeout by [`@husseinvr97`](https://github.com/husseinvr97) in [netty/netty#16982](https://redirect.github.com/netty/netty/pull/16982) > * Fix typo in AbstractSniHandler Javadoc by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#16988](https://redirect.github.com/netty/netty/pull/16988) > * Fix client/server inconsistency in SslCredential support matrix by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16990](https://redirect.github.com/netty/netty/pull/16990) > * Reconcile `AbstractCoalescingBufferQueue` readableBytes when it drains, and fail stuck HTTP/2 streams instead of spinning empty DATA frames by [`@gavinbunney`](https://github.com/gavinbunney) in [netty/netty#16947](https://redirect.github.com/netty/netty/pull/16947) > * Use Ticker in Http2MaxRstFrameListener for testability by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16993](https://redirect.github.com/netty/netty/pull/16993) > * Reset UTF-8 decode state on CR in StompSubframeDecoder by [`@vasiliy-mikhailov`](https://github.com/vasiliy-mikhailov) in [netty/netty#16991](https://redirect.github.com/netty/netty/pull/16991) > * FastLz: Guard decompression against truncated input by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17000](https://redirect.github.com/netty/netty/pull/17000) > * Reject non-token characters in HTTP/2 header names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16762](https://redirect.github.com/netty/netty/pull/16762) > * Auto-port 4.2: Fix SelfSignCertificate initialization in tests by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#17029](https://redirect.github.com/netty/netty/pull/17029) > * Enable extension of Http3ClientConnectionHandler to support higher-level protocols such as WebTransport. by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#17027](https://redirect.github.com/netty/netty/pull/17027) > * Implement Adaptive Cumulator by [`@shivaspeaks`](https://github.com/shivaspeaks) in [netty/netty#16731](https://redirect.github.com/netty/netty/pull/16731) > * Allow WebSocket extension negotiation to be disabled per response by [`@mkurz`](https://github.com/mkurz) in [netty/netty#17030](https://redirect.github.com/netty/netty/pull/17030) > * Support QPACK sensitivity detector for Never Indexed header fields by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17026](https://redirect.github.com/netty/netty/pull/17026) > * Fix maxAllocation for brotli-encoded content in HttpContentDecompressor by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17037](https://redirect.github.com/netty/netty/pull/17037) > * Pin github actions to reduce risk by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17043](https://redirect.github.com/netty/netty/pull/17043) ... (truncated) Commits * [`3703d79`](netty/netty@3703d79) [maven-release-plugin] prepare release netty-4.2.16.Final * [`63bbb2c`](netty/netty@63bbb2c) Update rust toolchain - add required parameters * [`ac06c1b`](netty/netty@ac06c1b) Update rust toolchain * [`5b68c61`](netty/netty@5b68c61) Merge branches from forks ([#17063](https://redirect.github.com/netty/netty/issues/17063)) * [`de5d276`](netty/netty@de5d276) Update lz4-java to 1.11.1 ([#17061](https://redirect.github.com/netty/netty/issues/17061)) * [`da22048`](netty/netty@da22048) Pin github actions to reduce risk ([#17043](https://redirect.github.com/netty/netty/issues/17043)) * [`0332676`](netty/netty@0332676) Fix maxAllocation for brotli-encoded content in HttpContentDecompressor ([#17037](https://redirect.github.com/netty/netty/issues/17037)) * [`7364401`](netty/netty@7364401) Support QPACK sensitivity detector for Never Indexed header fields ([#17026](https://redirect.github.com/netty/netty/issues/17026)) * [`06faf18`](netty/netty@06faf18) Allow WebSocket extension negotiation to be disabled per response ([#17030](https://redirect.github.com/netty/netty/issues/17030)) * [`bc4b983`](netty/netty@bc4b983) Implement Adaptive Cumulator ([#16731](https://redirect.github.com/netty/netty/issues/16731)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.15.Final...netty-4.2.16.Final) Updates `io.netty:netty-codec` from 4.2.15.Final to 4.2.16.Final Release notes *Sourced from [io.netty:netty-codec's releases](https://github.com/netty/netty/releases).* > netty-4.2.16.Final > ------------------ > > What's Changed > -------------- > > * Document Java 9 requirement for io\_uring by [`@jchambers`](https://github.com/jchambers) in [netty/netty#16904](https://redirect.github.com/netty/netty/pull/16904) > * Add BlockHound exception for DnsQueryIdSpace by [`@violetagg`](https://github.com/violetagg) in [netty/netty#16896](https://redirect.github.com/netty/netty/pull/16896) > * Fix incorrect bounds in error message of HpackDecoder.setMaxHeaderListSize by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16901](https://redirect.github.com/netty/netty/pull/16901) > * Add epoch-based chunk cache purge with ring buffer for thread-local reuse by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16766](https://redirect.github.com/netty/netty/pull/16766) > * Use Splittable/ThreadLocalRandom to generate bulk data in tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16808](https://redirect.github.com/netty/netty/pull/16808) > * Auto-port 4.2: SingleThreadEventExecutor: document Throwable safety contract on run() by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16909](https://redirect.github.com/netty/netty/pull/16909) > * Auto-port 4.2: Make HTTP/2 frame hashCode consistent with equals by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16910](https://redirect.github.com/netty/netty/pull/16910) > * Auto-port 4.2: MQTT: Make the decodeProperties early-REPLAY check actually fire by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16919](https://redirect.github.com/netty/netty/pull/16919) > * IoUring: fix io\_uring datagram writes with non-zero readerIndex by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16905](https://redirect.github.com/netty/netty/pull/16905) > * Exclude internal events from IoHandler.run() return value in epoll, io\_uring and kqueue by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16848](https://redirect.github.com/netty/netty/pull/16848) > * IoUring: Pass IORING\_ENTER\_NO\_IOWAIT to report accurate CPU usage by [`@wineway`](https://github.com/wineway) in [netty/netty#16739](https://redirect.github.com/netty/netty/pull/16739) > * Avoid logging exceptions that tests ignore by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16891](https://redirect.github.com/netty/netty/pull/16891) > * Reject control characters at the boundary of HTTP method names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16723](https://redirect.github.com/netty/netty/pull/16723) > * IoUring: fix TCP Fast Open initial writes with readerIndex and composites by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16929](https://redirect.github.com/netty/netty/pull/16929) > * Try to fix/stabilize a number of flaky tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16934](https://redirect.github.com/netty/netty/pull/16934) > * Fix propagation of startTls for client SslContext handlers by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16931](https://redirect.github.com/netty/netty/pull/16931) > * Update to latest tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16936](https://redirect.github.com/netty/netty/pull/16936) > * Move test to shared testsuite by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16928](https://redirect.github.com/netty/netty/pull/16928) > * [Refactor] Useful helper method getOrDefault & cleaner abstraction by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#16927](https://redirect.github.com/netty/netty/pull/16927) > * Make permessage-deflate server window size and memLevel configurable by [`@fru1tworld`](https://github.com/fru1tworld) in [netty/netty#16809](https://redirect.github.com/netty/netty/pull/16809) > * Return early in DnsQueryContext.writeQuery when the query ID space is exhausted by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16950](https://redirect.github.com/netty/netty/pull/16950) > * Fix HTTP 2 PUSH\_PROMISE stream association validation by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16952](https://redirect.github.com/netty/netty/pull/16952) > * Fix GZIP FEXTRA extra-field handling in JdkZlibDecoder by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16951](https://redirect.github.com/netty/netty/pull/16951) > * Http3FrameCodec handle fragmented payloads when skipping unknown frames by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16960](https://redirect.github.com/netty/netty/pull/16960) > * Add opt-in validation of mandatory pseudo-header fields for HTTP/2 by [`@hyperxpro`](https://github.com/hyperxpro) in [netty/netty#16932](https://redirect.github.com/netty/netty/pull/16932) > * Strictly validate MQTT UTF-8 Encoded String by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16939](https://redirect.github.com/netty/netty/pull/16939) > * Stop DateFormatter trailing token from running past the parse end by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16958](https://redirect.github.com/netty/netty/pull/16958) > * IpFilter: Deprecate constructor which use accept by default by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16961](https://redirect.github.com/netty/netty/pull/16961) > * Add RFC 10008 QUERY Method support by [`@desiderantes`](https://github.com/desiderantes) in [netty/netty#16966](https://redirect.github.com/netty/netty/pull/16966) > * Correctly release and fail queued traffic-shaping writes on close by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16959](https://redirect.github.com/netty/netty/pull/16959) > * Reject control characters at the boundary of the HTTP version token by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16971](https://redirect.github.com/netty/netty/pull/16971) > * FlowControlHandler: respect auto-read when toggled while dequeueing by [`@schiemon`](https://github.com/schiemon) in [netty/netty#16949](https://redirect.github.com/netty/netty/pull/16949) > * Fix leak in ReferenceCountedOpenSslEngine.addCredential by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16979](https://redirect.github.com/netty/netty/pull/16979) > * IdleStateHandler: reset firstWriter/ReaderIdleEvent in resetWriteTimeout/resetReadTimeout by [`@husseinvr97`](https://github.com/husseinvr97) in [netty/netty#16982](https://redirect.github.com/netty/netty/pull/16982) > * Fix typo in AbstractSniHandler Javadoc by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#16988](https://redirect.github.com/netty/netty/pull/16988) > * Fix client/server inconsistency in SslCredential support matrix by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16990](https://redirect.github.com/netty/netty/pull/16990) > * Reconcile `AbstractCoalescingBufferQueue` readableBytes when it drains, and fail stuck HTTP/2 streams instead of spinning empty DATA frames by [`@gavinbunney`](https://github.com/gavinbunney) in [netty/netty#16947](https://redirect.github.com/netty/netty/pull/16947) > * Use Ticker in Http2MaxRstFrameListener for testability by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16993](https://redirect.github.com/netty/netty/pull/16993) > * Reset UTF-8 decode state on CR in StompSubframeDecoder by [`@vasiliy-mikhailov`](https://github.com/vasiliy-mikhailov) in [netty/netty#16991](https://redirect.github.com/netty/netty/pull/16991) > * FastLz: Guard decompression against truncated input by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17000](https://redirect.github.com/netty/netty/pull/17000) > * Reject non-token characters in HTTP/2 header names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16762](https://redirect.github.com/netty/netty/pull/16762) > * Auto-port 4.2: Fix SelfSignCertificate initialization in tests by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#17029](https://redirect.github.com/netty/netty/pull/17029) > * Enable extension of Http3ClientConnectionHandler to support higher-level protocols such as WebTransport. by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#17027](https://redirect.github.com/netty/netty/pull/17027) > * Implement Adaptive Cumulator by [`@shivaspeaks`](https://github.com/shivaspeaks) in [netty/netty#16731](https://redirect.github.com/netty/netty/pull/16731) > * Allow WebSocket extension negotiation to be disabled per response by [`@mkurz`](https://github.com/mkurz) in [netty/netty#17030](https://redirect.github.com/netty/netty/pull/17030) > * Support QPACK sensitivity detector for Never Indexed header fields by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17026](https://redirect.github.com/netty/netty/pull/17026) > * Fix maxAllocation for brotli-encoded content in HttpContentDecompressor by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17037](https://redirect.github.com/netty/netty/pull/17037) > * Pin github actions to reduce risk by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17043](https://redirect.github.com/netty/netty/pull/17043) ... (truncated) Commits * [`3703d79`](netty/netty@3703d79) [maven-release-plugin] prepare release netty-4.2.16.Final * [`63bbb2c`](netty/netty@63bbb2c) Update rust toolchain - add required parameters * [`ac06c1b`](netty/netty@ac06c1b) Update rust toolchain * [`5b68c61`](netty/netty@5b68c61) Merge branches from forks ([#17063](https://redirect.github.com/netty/netty/issues/17063)) * [`de5d276`](netty/netty@de5d276) Update lz4-java to 1.11.1 ([#17061](https://redirect.github.com/netty/netty/issues/17061)) * [`da22048`](netty/netty@da22048) Pin github actions to reduce risk ([#17043](https://redirect.github.com/netty/netty/issues/17043)) * [`0332676`](netty/netty@0332676) Fix maxAllocation for brotli-encoded content in HttpContentDecompressor ([#17037](https://redirect.github.com/netty/netty/issues/17037)) * [`7364401`](netty/netty@7364401) Support QPACK sensitivity detector for Never Indexed header fields ([#17026](https://redirect.github.com/netty/netty/issues/17026)) * [`06faf18`](netty/netty@06faf18) Allow WebSocket extension negotiation to be disabled per response ([#17030](https://redirect.github.com/netty/netty/issues/17030)) * [`bc4b983`](netty/netty@bc4b983) Implement Adaptive Cumulator ([#16731](https://redirect.github.com/netty/netty/issues/16731)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.15.Final...netty-4.2.16.Final) Updates `io.netty:netty-handler` from 4.2.15.Final to 4.2.16.Final Release notes *Sourced from [io.netty:netty-handler's releases](https://github.com/netty/netty/releases).* > netty-4.2.16.Final > ------------------ > > What's Changed > -------------- > > * Document Java 9 requirement for io\_uring by [`@jchambers`](https://github.com/jchambers) in [netty/netty#16904](https://redirect.github.com/netty/netty/pull/16904) > * Add BlockHound exception for DnsQueryIdSpace by [`@violetagg`](https://github.com/violetagg) in [netty/netty#16896](https://redirect.github.com/netty/netty/pull/16896) > * Fix incorrect bounds in error message of HpackDecoder.setMaxHeaderListSize by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16901](https://redirect.github.com/netty/netty/pull/16901) > * Add epoch-based chunk cache purge with ring buffer for thread-local reuse by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16766](https://redirect.github.com/netty/netty/pull/16766) > * Use Splittable/ThreadLocalRandom to generate bulk data in tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16808](https://redirect.github.com/netty/netty/pull/16808) > * Auto-port 4.2: SingleThreadEventExecutor: document Throwable safety contract on run() by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16909](https://redirect.github.com/netty/netty/pull/16909) > * Auto-port 4.2: Make HTTP/2 frame hashCode consistent with equals by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16910](https://redirect.github.com/netty/netty/pull/16910) > * Auto-port 4.2: MQTT: Make the decodeProperties early-REPLAY check actually fire by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#16919](https://redirect.github.com/netty/netty/pull/16919) > * IoUring: fix io\_uring datagram writes with non-zero readerIndex by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16905](https://redirect.github.com/netty/netty/pull/16905) > * Exclude internal events from IoHandler.run() return value in epoll, io\_uring and kqueue by [`@franz1981`](https://github.com/franz1981) in [netty/netty#16848](https://redirect.github.com/netty/netty/pull/16848) > * IoUring: Pass IORING\_ENTER\_NO\_IOWAIT to report accurate CPU usage by [`@wineway`](https://github.com/wineway) in [netty/netty#16739](https://redirect.github.com/netty/netty/pull/16739) > * Avoid logging exceptions that tests ignore by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16891](https://redirect.github.com/netty/netty/pull/16891) > * Reject control characters at the boundary of HTTP method names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16723](https://redirect.github.com/netty/netty/pull/16723) > * IoUring: fix TCP Fast Open initial writes with readerIndex and composites by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#16929](https://redirect.github.com/netty/netty/pull/16929) > * Try to fix/stabilize a number of flaky tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#16934](https://redirect.github.com/netty/netty/pull/16934) > * Fix propagation of startTls for client SslContext handlers by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16931](https://redirect.github.com/netty/netty/pull/16931) > * Update to latest tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16936](https://redirect.github.com/netty/netty/pull/16936) > * Move test to shared testsuite by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16928](https://redirect.github.com/netty/netty/pull/16928) > * [Refactor] Useful helper method getOrDefault & cleaner abstraction by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#16927](https://redirect.github.com/netty/netty/pull/16927) > * Make permessage-deflate server window size and memLevel configurable by [`@fru1tworld`](https://github.com/fru1tworld) in [netty/netty#16809](https://redirect.github.com/netty/netty/pull/16809) > * Return early in DnsQueryContext.writeQuery when the query ID space is exhausted by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16950](https://redirect.github.com/netty/netty/pull/16950) > * Fix HTTP 2 PUSH\_PROMISE stream association validation by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16952](https://redirect.github.com/netty/netty/pull/16952) > * Fix GZIP FEXTRA extra-field handling in JdkZlibDecoder by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16951](https://redirect.github.com/netty/netty/pull/16951) > * Http3FrameCodec handle fragmented payloads when skipping unknown frames by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16960](https://redirect.github.com/netty/netty/pull/16960) > * Add opt-in validation of mandatory pseudo-header fields for HTTP/2 by [`@hyperxpro`](https://github.com/hyperxpro) in [netty/netty#16932](https://redirect.github.com/netty/netty/pull/16932) > * Strictly validate MQTT UTF-8 Encoded String by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16939](https://redirect.github.com/netty/netty/pull/16939) > * Stop DateFormatter trailing token from running past the parse end by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16958](https://redirect.github.com/netty/netty/pull/16958) > * IpFilter: Deprecate constructor which use accept by default by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#16961](https://redirect.github.com/netty/netty/pull/16961) > * Add RFC 10008 QUERY Method support by [`@desiderantes`](https://github.com/desiderantes) in [netty/netty#16966](https://redirect.github.com/netty/netty/pull/16966) > * Correctly release and fail queued traffic-shaping writes on close by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16959](https://redirect.github.com/netty/netty/pull/16959) > * Reject control characters at the boundary of the HTTP version token by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#16971](https://redirect.github.com/netty/netty/pull/16971) > * FlowControlHandler: respect auto-read when toggled while dequeueing by [`@schiemon`](https://github.com/schiemon) in [netty/netty#16949](https://redirect.github.com/netty/netty/pull/16949) > * Fix leak in ReferenceCountedOpenSslEngine.addCredential by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16979](https://redirect.github.com/netty/netty/pull/16979) > * IdleStateHandler: reset firstWriter/ReaderIdleEvent in resetWriteTimeout/resetReadTimeout by [`@husseinvr97`](https://github.com/husseinvr97) in [netty/netty#16982](https://redirect.github.com/netty/netty/pull/16982) > * Fix typo in AbstractSniHandler Javadoc by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#16988](https://redirect.github.com/netty/netty/pull/16988) > * Fix client/server inconsistency in SslCredential support matrix by [`@jmcrawford45`](https://github.com/jmcrawford45) in [netty/netty#16990](https://redirect.github.com/netty/netty/pull/16990) > * Reconcile `AbstractCoalescingBufferQueue` readableBytes when it drains, and fail stuck HTTP/2 streams instead of spinning empty DATA frames by [`@gavinbunney`](https://github.com/gavinbunney) in [netty/netty#16947](https://redirect.github.com/netty/netty/pull/16947) > * Use Ticker in Http2MaxRstFrameListener for testability by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#16993](https://redirect.github.com/netty/netty/pull/16993) > * Reset UTF-8 decode state on CR in StompSubframeDecoder by [`@vasiliy-mikhailov`](https://github.com/vasiliy-mikhailov) in [netty/netty#16991](https://redirect.github.com/netty/netty/pull/16991) > * FastLz: Guard decompression against truncated input by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17000](https://redirect.github.com/netty/netty/pull/17000) > * Reject non-token characters in HTTP/2 header names by [`@daguimu`](https://github.com/daguimu) in [netty/netty#16762](https://redirect.github.com/netty/netty/pull/16762) > * Auto-port 4.2: Fix SelfSignCertificate initialization in tests by [`@netty-project-bot`](https://github.com/netty-project-bot) in [netty/netty#17029](https://redirect.github.com/netty/netty/pull/17029) > * Enable extension of Http3ClientConnectionHandler to support higher-level protocols such as WebTransport. by [`@sanjomo`](https://github.com/sanjomo) in [netty/netty#17027](https://redirect.github.com/netty/netty/pull/17027) > * Implement Adaptive Cumulator by [`@shivaspeaks`](https://github.com/shivaspeaks) in [netty/netty#16731](https://redirect.github.com/netty/netty/pull/16731) > * Allow WebSocket extension negotiation to be disabled per response by [`@mkurz`](https://github.com/mkurz) in [netty/netty#17030](https://redirect.github.com/netty/netty/pull/17030) > * Support QPACK sensitivity detector for Never Indexed header fields by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17026](https://redirect.github.com/netty/netty/pull/17026) > * Fix maxAllocation for brotli-encoded content in HttpContentDecompressor by [`@skyguard1`](https://github.com/skyguard1) in [netty/netty#17037](https://redirect.github.com/netty/netty/pull/17037) > * Pin github actions to reduce risk by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17043](https://redirect.github.com/netty/netty/pull/17043) ... (truncated) Commits * [`3703d79`](netty/netty@3703d79) [maven-release-plugin] prepare release netty-4.2.16.Final * [`63bbb2c`](netty/netty@63bbb2c) Update rust toolchain - add required parameters * [`ac06c1b`](netty/netty@ac06c1b) Update rust toolchain * [`5b68c61`](netty/netty@5b68c61) Merge branches from forks ([#17063](https://redirect.github.com/netty/netty/issues/17063)) * [`de5d276`](netty/netty@de5d276) Update lz4-java to 1.11.1 ([#17061](https://redirect.github.com/netty/netty/issues/17061)) * [`da22048`](netty/netty@da22048) Pin github actions to reduce risk ([#17043](https://redirect.github.com/netty/netty/issues/17043)) * [`0332676`](netty/netty@0332676) Fix maxAllocation for brotli-encoded content in HttpContentDecompressor ([#17037](https://redirect.github.com/netty/netty/issues/17037)) * [`7364401`](netty/netty@7364401) Support QPACK sensitivity detector for Never Indexed header fields ([#17026](https://redirect.github.com/netty/netty/issues/17026)) * [`06faf18`](netty/netty@06faf18) Allow WebSocket extension negotiation to be disabled per response ([#17030](https://redirect.github.com/netty/netty/issues/17030)) * [`bc4b983`](netty/netty@bc4b983) Implement Adaptive Cumulator ([#16731](https://redirect.github.com/netty/netty/issues/16731)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.15.Final...netty-4.2.16.Final) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
This PR introduces the
AdaptiveCumulator, a specialized Cumulator implementation forByteToMessageDecoderthat dynamically switches between merge (memory copy) and compose (zero-copy) strategies based on the size of incoming data.Standard cumulators in Netty present a trade-off: the
MERGE_CUMULATORis safe but can consume up to 10% of CPU time due to excessive copies during high-throughput transfers. Conversely, theCOMPOSITE_CUMULATORprovides zero-copy benefits but is vulnerable to "Single Byte Attacks," where an attacker sends messages one byte at a time to force the server to allocate excessive object metadata.The
AdaptiveCumulatorapplies a heuristic to achieve the best of both worlds:Modification:
AdaptiveCumulator.javato theio.netty.handler.codecpackage.AdaptiveCumulatorTest.java, a comprehensive test suite modernized for JUnit 5.This PR is an attempt to upstream what we were trying to do in gRPC grpc/grpc-java#9558, but the behaviour of
duplicate()method on slices was changed/corrected in #14093 which invalidated previous manual index workarounds and since then it has been a tech debt in grpc-java. This implementation is "PR #14093-aware" and correctly handles the capacity-capping introduced by that change by verifying the state of the tail component before expansion.This implementation is specifically designed to address historical failure modes associated with
CompositeByteBufmanipulation:Modernized Index Synchronization with componentSlice() Previous attempts to implement this logic relied on fragile internal hacks like
internalComponent().duplicate()to correct stale indices returned byCompositeByteBuf.component(). This PR leverages thecomponentSlice()method (introduced in Netty 4.2.4) to obtain a correctly indexed view through a public, supported API.Protection Against "Resurrected" Bytes To prevent silent data corruption, we have introduced a capacity safety check:
tail.capacity() == componentView.capacity(). If a mismatch is detected, it indicates the component is a partial slice containing hidden "discarded" bytes. In such cases, the logic correctly falls back to a safe reallocation to "cleanse" the data, preventing discarded bytes from reappearing in the message. This particular problem was caught in the unit testmergeWithCompositeTail_detectsIndicesInNestedComposite()while trying to invalidate the use of previous way (internalComponent.duplicate()) as in netty: create adaptive cumulator grpc/grpc-java#9558.Prevention of "Russian Doll" Nesting The implementation strictly uses
addFlattenedComponentsto ensure theCompositeByteBufremains a flat, linear structure. This avoids the recursive deallocation crashes (StackOverflow) that occurred in previous versions when buffers were nested thousands of layers deep.Single-Owner Lifecycle Management The lifecycle of the input buffer is managed using a single-owner pattern in the top-level cumulate method. Sub-methods like
mergeWithCompositeTailno longer release the input buffer in their finally blocks, which we were doing otherwise in netty: create adaptive cumulator grpc/grpc-java#9558. This preventsIllegalReferenceCountException(double-release) crashes during error scenarios.Usage
Downstream projects can enable the
AdaptiveCumulatorby calling thesetCumulator()method on any handler that extendsByteToMessageDecoder. For example, to use the adaptive strategy with a 1KB threshold:setCumulator(new AdaptiveCumulator(1024));CC: @ejona86, @sauravzg