Skip to content

[improve][build] Upgrade data structure and compression libraries - #26357

Merged
nodece merged 1 commit into
apache:masterfrom
lhotari:lh-improve-datastructure-libs
Aug 18, 2026
Merged

nodece merged 1 commit into
apache:masterfrom
lhotari:lh-improve-datastructure-libs

Conversation

@lhotari

@lhotari lhotari commented Aug 17, 2026

Copy link
Copy Markdown
Member

Motivation

The primitive-collection and compression libraries are behind. Two of them are constrained by
Pulsar's Java 17 baseline (options.release.set(17) in pulsar.java-conventions), which is worth
recording explicitly so future bumps do not silently overshoot it.

Modifications

gradle/libs.versions.toml:

library from to
fastutil 8.5.18 8.5.19
jctools 4.0.6 4.0.7
RoaringBitmap 1.6.9 1.6.20
hppc 0.9.1 0.10.0 (not 0.11.1)
zstd-jni 1.5.7-3 1.5.7-15
lz4-java 1.11.1 1.11.2

Plus the corresponding jar names in the server and shell distribution LICENSE.bin.txt files.

hppc is capped at 0.10.0 by the Java 17 baseline

Class file versions read directly from the published jars:

hppc bytecode
0.9.1 (current) Java 8
0.10.0 (this PR) Java 11
0.11.1 (latest) Java 21

hppc is an api dependency of pulsar-broker and is used by PersistentTopic,
NonPersistentTopic and AbstractDispatcherMultipleConsumers, so Java 21 class files would fail to
load on a Java 17 JVM. A comment recording the cap is added to the catalog.

DataSketches is left unchanged for the same reason

Not part of this PR, but worth stating since it looks stale: datasketches-java 7.0.1 and
datasketches-memory 4.1.0 are already the newest releases still compiled for Java 17.
datasketches-java 8.0.0 is Java 21 and 9.0.0 is Java 25; datasketches-memory 5.0.0 and 6.0.0 are
Java 21, and 7.0.0 publishes no plain jar.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Verified locally with ./gradlew sanityCheck and ./gradlew checkBinaryLicense.

Does this pull request potentially affect one of the following parts:

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

- fastutil       8.5.18  -> 8.5.19
- jctools        4.0.6   -> 4.0.7
- RoaringBitmap  1.6.9   -> 1.6.20
- hppc           0.9.1   -> 0.10.0
- zstd-jni       1.5.7-3 -> 1.5.7-15
- lz4-java       1.11.1  -> 1.11.2

hppc is intentionally taken to 0.10.0 rather than the latest 0.11.1: 0.11.x is compiled to
Java 21 bytecode and Pulsar targets Java 17 (options.release is 17). 0.10.0 is Java 11
bytecode. hppc is an api dependency of pulsar-broker and is used by PersistentTopic,
NonPersistentTopic and AbstractDispatcherMultipleConsumers, so a Java 21 class file
version would fail at runtime on a Java 17 JVM.

DataSketches is left unchanged for the same reason and is not part of this change:
datasketches-java 7.0.1 and datasketches-memory 4.1.0 are the newest releases still
compiled for Java 17 (8.0.0 is Java 21, 9.0.0 is Java 25).

Assisted-by: Claude Code (Opus 5)
@lhotari lhotari added the area/dependency Pull requests that update a dependency file label Aug 17, 2026
@lhotari lhotari added this to the 5.0.0-M2 milestone Aug 17, 2026
@nodece
nodece merged commit ee1c18d into apache:master Aug 18, 2026
65 of 79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/dependency Pull requests that update a dependency file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants