Repository navigation
Building Pulsar (incl. docker images) from source with Java 9+ (Java 11, Java 15, etc.) causes silent NoSuchMethodErrors at runtime #8445
Description
Building Pulsar (incl. docker images) from source with Java 11 causes silent NoSuchMethodErrors at runtime
To Reproduce
Steps to reproduce the behavior:
Use Java 11.
Then, compile v2.6.2-candidate-1 docker images locally:
git clone --depth 1 -b v2.6.2-candidate-1 https://github.com/apache/pulsar pulsar-2.6
cd pulsar-2.6
mvn -DskipTests clean install
./docker/build.sh
Start pulsar in docker
docker run -p 6650:6650 pulsar:2.6.2 bin/pulsar standalone --no-functions-worker -nss
Compile & run this Pulsar code with required libraries:
import org.apache.pulsar.client.api.*;
public class PulsarJava11Problem {
public static void main(String[] args) throws PulsarClientException {
PulsarClient pulsarClient = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer<String> producer = pulsarClient.newProducer(Schema.STRING).topic("mytopic").create();
}
}The producer never gets created and it times out.
What happens on a server is a java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
This happens at
java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
at org.apache.pulsar.common.schema.LongSchemaVersion.bytes(LongSchemaVersion.java:44)
at org.apache.pulsar.common.protocol.Commands.newProducerSuccessCommand(Commands.java:377)
at org.apache.pulsar.broker.service.PulsarCommandSenderImpl.sendProducerSuccessResponse(PulsarCommandSenderImpl.java:95)
at org.apache.pulsar.broker.service.ServerCnx.lambda$handleProducer$19(ServerCnx.java:1137)
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.lambda$createSchemaLocator$28(BookkeeperSchemaStorage.java:504)
at org.apache.bookkeeper.util.ZkUtils$1.processResult(ZkUtils.java:80)
at org.apache.bookkeeper.zookeeper.ZooKeeperClient$10$1.processResult(ZooKeeperClient.java:737)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:636)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)
This exception never gets printed to the log, this stacktrace was captured using the debugger.
Expected behavior
Compiling with an unsupported Java version should result in an error and the NoSuchMethodError that happens in Pulsar shouldn't get "swallowed".
It should be possible to compile the code with Java 9+ without running into issues when running with Java 8.
There's an explanation of the issue with ByteBuffer at jetty/jetty.project#3244 . A solution is also described:
The solution is to cast the
ByteBuffertoBufferwhen calling those methods:((Buffer)byteBuffer).position(0);
Activity
lhotari commentedon Nov 4, 2020
@eolivelli I noticed that you had done some changes to allow compiling with JDK 14+. Have you come across the ByteBuffer incompatibilities?
wolfstudy commentedon Nov 4, 2020
This problem should not only exist in 2.6.2, but it should also be a master problem, @zymap helped to verify the logic with JDK 1.8, it seems that everything is OK. It looks like we need to support different JDK versions to build Apache Pulsar.
eolivelli commentedon Nov 4, 2020
@lhotari
my efforts about building with JDK14+ are about using JDK14 for development,
if you build it with JDK>8 you cannot run with JDK8.
We should add maven.compiler.release=8 in our pom.xml, it must be done in a profile activated when you are on jdk9+, it is a quite easy fix.
that's a very common problem:
java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
lhotari commentedon Nov 4, 2020
@eolivelli I think it would be valuable. It's easy to miss that you must compile Pulsar with Java 8 since the compilation passes without errors and even at runtime the errors are swallowed and not logged (at least in this particular case). Very hard to debug.
eolivelli commentedon Nov 4, 2020
I guess that if you build with JDK14+ and you run everything on JDK14+ you won't hit problems.
probably you are not going to run Pulsar on JDK8, as newer JVMs are generally (I don't want to start a discussion now) more efficient and provide better performances.
In BK community we are switching to JDK11+ (but still next release will work on JDK8)
lhotari commentedon Dec 21, 2020
There is a blog post with an easy solution: https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/ .
This property should be defined in pom.xml:
eolivelli commentedon Dec 21, 2020
You have to add it in a profile, because if you add only this property the build will break on jdk8 because the 'release' option is not valid with jdk8
Set release in maven-compiler-plugin configuration and enforce JDK 11…