This repository was archived by the owner on Sep 25, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 25, 2026. It is now read-only.
google cloud storage: NPE from BlobId.java:119 #709
Description
Environment details
- google cloud storage
- OS type and version: Ubuntu 18.04.4, running in docker container on k8s.
- Java version:
java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode) - storage version(s):
google-cloud-storage-1.113.9.jar
google-cloud-core-1.94.0.jar
google-cloud-core-http-1.94.0.jar
Steps to reproduce
- Upload content per code below
Code example
BlobId blobId = BlobId.of(bucketName, key);
String md5;
try (InputStream inputStream = getInputStream()) {
md5 = Base64.encodeBase64String(DigestUtils.md5(inputStream));
}
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType(getContentType())
.setContentDisposition(String.format("attachment; filename=%s", getDownloadName()))
.setMd5(md5)
.build();
storage.createFrom(blobInfo, getInputStream(), Storage.BlobWriteOption.md5Match());Stack trace
Caused by: java.lang.NullPointerException
at com.google.cloud.storage.BlobId.fromPb(BlobId.java:119)
at com.google.cloud.storage.BlobInfo.fromPb(BlobInfo.java:1160)
at com.google.cloud.storage.Blob.fromPb(Blob.java:958)
at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:267)
at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:253)
at org.ambraproject.rhino.service.impl.GCSObjectStorageServiceImpl.storeFile(GCSObjectStorageServiceImpl.java:72)
at org.ambraproject.rhino.service.impl.GCSObjectStorageServiceImpl.lambda$null$0(GCSObjectStorageServiceImpl.java:78)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
This is an intermittent issue - I’ve only seen it once, as far as I know. I can’t see how the NPE could be coming from our code. I can’t seem to find any similar issue in the issues list or online.
Thanks for any help you can provide!
Activity
egh commentedon Feb 16, 2021
Please let me know if there is any more information I can provide.
tritone commentedon Feb 17, 2021
Thanks for your report.
From digging into the stack a bit it looks like the
StorageObjectproto is null here where the client is trying to construct the object for for the newly created Blob. This seems like a bug; it should not be something that can happen if the upload was successful and I would expect a different error if the upload failed (not NPE).@frankyn @BenWhitehead any ideas how this could have happened? I know there have been some recent updates to the upload logic.
egh commentedon Feb 17, 2021
We saw this for a second time yesterday. I have updated the stack trace slightly to reflect that this is coming from a ThreadPoolExecutor. We have found that we need to send content to GCS in parallel in order to get acceptable speeds. I'm not sure if that affects anything.
frankyn commentedon Feb 17, 2021
Hi @egh,
Did you have these issues with
1.113.3or is your usage relatively new?I think this is not a concurrency issue. It is most likely due an edge case that was not handled correctly in PR #604.
I'm writing a unit test to confirm, but I think the case is:
egh commentedon Feb 17, 2021
Hi @frankyn -
We were previously using 1.113.6 - before that we were not using GCS.
We did encounter #666 (
com.google.cloud.storage.StorageException: Resumable upload is already complete.) when using that version.frankyn commentedon Feb 17, 2021
Thanks for clarifying and for filing this issue, @egh. This is a relatively new change which tried to make the library more reliable but introduced a few issues.
One recommendation if you need to get unblocked now is to revert to the earlier version
1.113.4while we work out this issue.fix: retrying get remote offset and recover from last chunk failures. (…