Skip to content
This repository was archived by the owner on Sep 25, 2026. It is now read-only.
This repository was archived by the owner on Sep 25, 2026. It is now read-only.

StorageException: Resumable upload is already complete. #666

Closed
#677
Closed
StorageException: Resumable upload is already complete.#666
#677
Assignees
frankyn
Labels
api: storageIssues related to the googleapis/java-storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Description

@n12c
n12c
opened on Dec 16, 2020

Following #647, we no longer sometimes see:

java.lang.NullPointerException: null
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.getCurrentUploadOffset(HttpStorageRpc.java:783)
	at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:125)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:70)
	at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
	at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:265)
...snip...

This appeared in 1.113.4 but no longer appears in 1.113.6. Thanks for the fix.

Instead (in 1.113.6) we now sometimes see:

com.google.cloud.storage.StorageException: Resumable upload is already complete.
	at com.google.cloud.storage.spi.v1.HttpStorageRpc.getCurrentUploadOffset(HttpStorageRpc.java:769)
	at com.google.cloud.storage.BlobWriteChannel$1.run(BlobWriteChannel.java:125)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
	at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
	at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
	at com.google.cloud.storage.BlobWriteChannel.flushBuffer(BlobWriteChannel.java:70)
	at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
	at com.google.cloud.storage.StorageImpl.createFrom(StorageImpl.java:265)
...snip...

What do you think? More tweaks to #604 required?

Thanks for your time.

Activity

  1. product-auto-label
    added
    api: storageIssues related to the googleapis/java-storage API.
    on Dec 16, 2020
  2. frankyn
    added
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Dec 16, 2020
  3. frankyn
    self-assigned this
    on Dec 16, 2020
  4. frankyn
    added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    on Dec 16, 2020

  5. frankyn commented on Dec 16, 2020

    @frankyn
    frankyn
    on Dec 16, 2020
    Contributor

    Hi @n12c!

    Thanks for filing the issue. Do you have an example use case that causes this issue?

  6. n12c commented on Dec 16, 2020

    @n12c
    n12c
    on Dec 16, 2020
    Author

    I don't have a self-contained reproducer; this seems quite intermittent. Generally we just call com.google.cloud.storage.Storage#createFrom(com.google.cloud.storage.BlobInfo, java.io.InputStream, int, com.google.cloud.storage.Storage.BlobWriteOption...) with content in the range of 100 bytes to a few MiB in length. I added retry logic to my own code, which repeats this call unmodified and without delay (with a fresh java.io.InputStream having the exact same content); this always seems to succeed on the second try. We always use BlobWriteOption#disableGzipContent and sometimes use BlobWriteOption#encryptionKey(java.security.Key). The only instances I have seen have all occurred while CSEK was in use, but that doesn't necessarily prove it doesn't also happen without CSEK.

    I can try to write a reproducer but it would basically consist of uploading random objects using this API in a tight loop until (hopefully) encountering the same issue.

  7. frankyn commented on Dec 16, 2020

    @frankyn
    frankyn
    on Dec 16, 2020
    Contributor

    Thanks! I'll try to reproduce the issue, it might be a retry an accidental retry when there shouldn't be one.

  8. n12c commented on Dec 16, 2020

    @n12c
    n12c
    on Dec 16, 2020
    Author

    Update: I just saw this happen without CSEK (no encryption of any kind, just the default Google managed keys). So, CSEK is not a factor.

  9. richiesgr commented on Jan 3, 2021

    @richiesgr
    richiesgr
    on Jan 3, 2021

    Hi
    Any update regarding what cause this ?

  10. frankyn commented on Jan 11, 2021

    @frankyn
    frankyn
    on Jan 11, 2021
    Contributor

    Hi @richiesgr I'm working to release a fix today. Apologies for the delay.

  11. gcf-merge-on-green
    closed this as completedin #677on Jan 12, 2021
  12. gcf-merge-on-green
    added a commit that references this issue on Jan 12, 2021

    fix: last chunk is retriable (#677)

    44f49e0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: storageIssues related to the googleapis/java-storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions