Skip to content

Gcloud::Storage::File#signed_url does not work when using Google Cloud SDK credentials #181

Closed
#196
Closed
Gcloud::Storage::File#signed_url does not work when using Google Cloud SDK credentials#181
#196
Assignees
blowmage
Labels
🚨This issue needs some love.api: storageIssues related to the Cloud Storage API.triage meI really want to be triaged.

Description

@premist
premist
opened on Jul 1, 2015

I tried using Gcloud::Storage::File#signed_url to generate object url with query strings, but I can't get it to work. I used the following code:

s = Gcloud.storage "my-project-name"
buk = s.find_bucket "my-bucket"
file = buk.find_file "myfile.txt"
file.signed_url

It returns the following error:

[10] pry(main)> file.signed_url
NoMethodError: undefined method `sign' for nil:NilClass
from /Users/premist/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/gcloud-0.1.1/lib/gcloud/storage/file.rb:502:in `signed_url'

I tried to investigate this and found out that signing_key on Gcloud::Credentials is not being set properly.

I used gcloud auth credential from my computer initially, and I tried to explicitly provide keyfile for authentication too, but it didn't work.

Activity

  1. blowmage commented on Jul 1, 2015

    @blowmage
    blowmage
    on Jul 1, 2015
    Contributor

    Hey @premist, the File#signed_url method expects that the Signet::OAuth2::Client object used for authenticating returns a valid object when calling it's signing_key method. According to the documentation, signing_key can return a OpenSSL::PKey (which we expect), or a string (which we don't expect yet, but I'm not sure we will see since we create a OpenSSL::PKey object). But in your case nil is being returned, which is very strange.

    I am having a hard time reproducing this. Are you available to pair remotely so I can see what you are doing in hopes of creating a reliable reproduction?

  2. blowmage
    self-assigned this
    on Jul 1, 2015

  3. premist commented on Jul 2, 2015

    @premist
    premist
    on Jul 2, 2015
    ContributorAuthor

    I will try to replicate this on clean Linux machine soon and let you know. If this works properly on clean environment, there should be some misconfiguration on my machine which causes the issue.

  4. blowmage commented on Jul 6, 2015

    @blowmage
    blowmage
    on Jul 6, 2015
    Contributor

    @premist Any luck reproducing this?

  5. premist commented on Jul 13, 2015

    @premist
    premist
    on Jul 13, 2015
    ContributorAuthor

    Hi @blowmage, sorry for late reply. Will try this later today and I'll let you know how it goes.

  6. blowmage commented on Jul 13, 2015

    @blowmage
    blowmage
    on Jul 13, 2015
    Contributor

    Thanks! Looking forward to it. :)

  7. blowmage commented on Jul 13, 2015

    @blowmage
    blowmage
    on Jul 13, 2015
    Contributor

    Hey @premist, when you provide the keyfile explicitly, are you giving the absolute path to the file? e.g /Users/premist/gcloud.json and not ~/gcloud.json?

  8. premist commented on Jul 15, 2015

    @premist
    premist
    on Jul 15, 2015
    ContributorAuthor

    I created a clean Ubuntu (15.04 x64) VM, and did the following:

    • Install Ruby 2.2.2 via rbenv
    • Install Google Cloud SDK
    • Used $ gcloud auth login to sign in
    • Install gcloud gem

    I tried the code above again and it failed. Seems like signing_key doesn't work as intended when gcloud gem authenticates using Cloud SDK credentials.

  9. premist
    changed the title [-]Gcloud::Storage::File#signed_url does not work[/-] [+]Gcloud::Storage::File#signed_url does not work when using Google Cloud SDK credentials[/+] on Jul 15, 2015

  10. blowmage commented on Jul 15, 2015

    @blowmage
    blowmage
    on Jul 15, 2015
    Contributor

    @premist Thanks! You also said:

    I used gcloud auth credential from my computer initially, and I tried to explicitly provide keyfile for authentication too, but it didn't work.

    I can't get it to fail when explicitly providing a keyfile, but I assume that the path you provided didn't resolve and so it fell back to the cloud SDK credentials. Any chance you can confirm?

  11. premist commented on Jul 15, 2015

    @premist
    premist
    on Jul 15, 2015
    ContributorAuthor

    I tried providing JSON keyfile explicitly to Gcloud.storage, and it worked. Providing an invalid path like ~/Downloads/keyfile.json raised a proper error, which is the following:

    RuntimeError: The keyfile '~/Downloads/keyfile.json' is not a valid file.

    I'm not sure why I was unable to get signed url when I provided keyfile explicitly, but seems like it works as intended.

  12. blowmage commented on Jul 15, 2015

    @blowmage
    blowmage
    on Jul 15, 2015
    Contributor

    It looks like that the code that derives credentials from the cloud SDK doesn't return "service account" credentials, which is missing the private key needed to sign the URLs. Right now we're trying to determine how we can convert "authorized user" credentials to "service account" credentials.

  13. premist commented on Jul 15, 2015

    @premist
    premist
    on Jul 15, 2015
    ContributorAuthor

    I found something that might be related to this on gsutil documentation. In order to use gsutil signurl command, you need to explicitly provide private key file regardless of Google Cloud SDK authentication.

    I checked gcloud-python and gcloud-node, seems like those libraries don't support authentication by Cloud SDK. Also I couldn't find a code which converts 'authorized user' credentials to 'service account' credentials.

  14. jgeewax commented on Jul 15, 2015

    @jgeewax
    jgeewax
    on Jul 15, 2015

    I'm pretty sure both node and python will pull the creds set by the cloud SDK. CC @stephenplusplus @dhermes to comment.

  15. dhermes commented on Jul 15, 2015

    @dhermes
    dhermes
    on Jul 15, 2015

    Yes, we have a tight integration with https://github.com/google/oauth2client and don't implement much of the signing ourselves, just proxy it out to the auth library.

  16. stephenplusplus commented on Jul 15, 2015

    @stephenplusplus
    stephenplusplus
    on Jul 15, 2015

    Same as @dhermes said, gcloud-node's auth library is https://github.com/google/google-auth-library-nodejs which respects the cloud sdk session.

  17. premist commented on Jul 15, 2015

    @premist
    premist
    on Jul 15, 2015
    ContributorAuthor

    Ah, thanks for the correction. Then it should be possible to generate signed url using Cloud SDK credentials.

  18. blowmage commented on Jul 15, 2015

    @blowmage
    blowmage
    on Jul 15, 2015
    Contributor

    @dhermes @stephenplusplus Do your auth libraries give you credentials with the private key when authenticating with the cloud SDK? Or do you have a way to exchange the "authorized user" credentials for "service account" credentials?

  19. blowmage commented on Jul 15, 2015

    @blowmage
    blowmage
    on Jul 15, 2015
    Contributor

    The signed url documentation is clear about requiring service account authentication for signing the url. We need the private key to sign the url.

  20. stephenplusplus commented on Jul 15, 2015

    @stephenplusplus
    stephenplusplus
    on Jul 15, 2015

    No, haven't found a way to do this yet: googleapis/google-cloud-node#211

  21. dhermes commented on Jul 15, 2015

    @dhermes
    dhermes
    on Jul 15, 2015

    There is no way to sign with "authorized user" credentials, it must be a service account. Our code paths fail on non-service account credentials (as well as on GCE service account credentials).

  22. blowmage commented on Jul 15, 2015

    @blowmage
    blowmage
    on Jul 15, 2015
    Contributor

    @stephenplusplus Ah, thanks for the link. I had missed than in my searches. Makes sense.

    @dhermes Thank you for the confirmation.

    @premist I have some ideas. I'll try to get a PR up today.

  23. blowmage
    added
    api: storageIssues related to the Cloud Storage API.
    on Jul 15, 2015
  24. blowmage
    added a commit that references this issue on Jul 21, 2015

    Add File#signed_url issuer/siging_key options

    f4935c3
  25. quartzmo
    closed this as completedin #196on Jul 22, 2015
  26. yoshi-automation
    added
    🚨This issue needs some love.
    triage meI really want to be triaged.
    on Apr 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

🚨This issue needs some love.api: storageIssues related to the Cloud Storage API.triage meI really want to be triaged.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions