Skip to content

google-cloud-ruby does not work with Application Default Credentials #951

Closed
Closed
google-cloud-ruby does not work with Application Default Credentials#951
Assignees
blowmage
Labels
🚨This issue needs some love.authpriority: p2Moderately-important priority. Fix may not be included in next release.status: acknowledgedtype: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Description

@beccasaurus
beccasaurus
opened on Sep 30, 2016

See also: google-cloud-node does not work with Application Default Credentials

Problem

After running gcloud beta auth application-default login I should be able to use google-cloud-ruby to access Google Cloud Platform services, but I can't. I must set the GOOGLE_CLOUD_PROJECT env var or explicitly pass a project ID to a client for google-cloud-ruby to work.

Steps to Reproduce

  1. Install the Google Cloud SDK and Ruby
  2. Run gcloud init, you'll be prompted to set a default project
  3. Run gcloud beta auth application-default login and follow the OAuth 2.0 flow to login
  4. Run gem install google-cloud-storage to install a google-cloud-ruby client library
  5. Run irb and use the google-cloud-storage gem to interact with Google Cloud Storage. In this case, I am following the example code snippet for Retrieving Buckets found on the website for google-cloud-ruby.
>> require "google/cloud"
=> true
>> gcloud = Google::Cloud.new
=> #<Object:0x007fc27b105f78 @project=nil, @keyfile=nil, @retries=nil, @timeout=nil>
>> storage = gcloud.storage
ArgumentError: project is missing
    from /Users/remily/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/google-cloud-storage-0.20.0/lib/google-cloud-storage.rb:110:in `storage'
    from /Users/remily/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/google-cloud-storage-0.20.0/lib/google-cloud-storage.rb:66:in `storage'
    from (irb):3
    from /Users/remily/.rbenv/versions/2.3.1/bin/irb:11:in `<main>'

Expected

After a user runs gcloud beta auth application-default login, google-cloud-ruby client libraries should work without explicitly setting a project ID via the code or an environment variable.

Activity

  1. blowmage commented on Sep 30, 2016

    @blowmage
    blowmage
    on Sep 30, 2016
    Contributor

    Thanks for the issue, @remi. I was not aware this was being discussed on the other language projects.

    In googleapis/google-cloud-node#1653 it seems like the consensus is that this functionality should be provided by the authentication library, and not the google-cloud-node library. For ruby, that is the googleauth gem. The googleauth gem does sniff for both the GCE environment, and the gcloud SDK Application Default Credentials. But when we asked them to support retrieving the default project_id for GCE they declined, saying that it was not something the library should provide.

    What is your opinion? Is this something that googleauth should provide, as it looks to be in future releases for python and node? Or something that google-cloud should provide?

  2. blowmage
    added
    auth
    type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.
    on Sep 30, 2016

  3. beccasaurus commented on Oct 1, 2016

    @beccasaurus
    beccasaurus
    on Oct 1, 2016
    Author

    Because this feature is so tightly coupled to the Google Cloud SDK command-line tool, I think it makes sense for this feature to be implemented in google-cloud rather than the generalized Ruby Google API client.

    For Python, project ID detection is currently implemented in google-cloud-python, not pushed down into the general Python Google API client.

    I rather like Python's project detection:

    1. GOOGLE_CLOUD_PROJECT environment variable
    2. GOOGLE_APPLICATION_CREDENTIALS JSON file
    3. Get default service project from $ gcloud beta auth application-default login
    4. Google App Engine application ID (not relevant for Ruby)
    5. Google Compute Engine project ID (from metadata server)

    Also, I see that googleauth sniffs the Google Compute Engine metadata server for an auth token, which is very relevant to auth, I think that it makes the most sense for google-cloud-ruby to lookup the project ID.

    What do you think?

  4. beccasaurus commented on Oct 1, 2016

    @beccasaurus
    beccasaurus
    on Oct 1, 2016
    Author

    Follow-up comment: I have been discussing this offline with Python and Node.js stakeholders. The referenced Node.js issue also includes discussion about upcoming changes to the the Python Google Auth library to move project detection into the authentication library.

    Reasoning / benefits:

    1. That Auth Library already has code that is specific to Google Cloud use-cases, such as Google Compute Engine use-cases and default credential lookup
    2. Having the code in one place is ideal for if/when any changes are made, eg. to config file format/location/etc
    3. Having this logic live outside of an API client library will allow any client-library to make use of this functionality, eg. the existing Google API client for Ruby will be able to make use of this feature as well

    The benefits of putting this code into the Google Auth library for Ruby are great and will benefit all libraries that make use of googleauth

    I'll follow-up with an issue files on the googleauth library. We can block this pending support for project detection in googleauth?

    /cc @jonparrott @jmdobry

  5. blowmage commented on Oct 1, 2016

    @blowmage
    blowmage
    on Oct 1, 2016
    Contributor

    Sounds good to me! We noticed that the newer keyfile JSON files also include the project_id. Since the google-cloud libs already provide googleauth with the JSON file I think it would be great if it gave back the project_id as well as the credentials object.

  6. landrito
    added
    status: blockedResolving the issue is dependent on other work.
    status: acknowledged
    priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.
    on Mar 1, 2017
  7. landrito
    assigned
    quartzmo
    on Mar 1, 2017
  8. quartzmo
    added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    and removed
    priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.
    on Nov 9, 2017

  9. blowmage commented on Jan 8, 2018

    @blowmage
    blowmage
    on Jan 8, 2018
    Contributor

    This issue is blocked by googleapis/google-auth-library-ruby#90, so I am moving this issue to the feature backlog until we are unblocked.

  10. blowmage
    closed this as completedon Jan 8, 2018

  11. AteqEjaz commented on Sep 4, 2018

    @AteqEjaz
    AteqEjaz
    on Sep 4, 2018

    Still having the same issue with Ruby version 2.4.4 thanks

  12. TheRoyalTnetennba
    reopened this on Oct 24, 2018
  13. JustinBeckwith
    added
    🚨This issue needs some love.
    on Oct 24, 2018
  14. blowmage
    assigned
    blowmage
    and unassigned
    quartzmo
    on Oct 24, 2018
  15. blowmage
    removed
    status: blockedResolving the issue is dependent on other work.
    on Oct 24, 2018

  16. blowmage commented on Oct 24, 2018

    @blowmage
    blowmage
    on Oct 24, 2018
    Contributor

    The recently released googleauth 0.7.0 will support the functionality to extract the project_id from service account credentials, but both google-api-client and google-gax pin the dependency on googleauth to a version lower than 0.7.0. We will have to coordinate the loosening of the downstream dependencies before we can start making use of this feature in the google-cloud-* gems.

  17. blowmage commented on Oct 24, 2018

    @blowmage
    blowmage
    on Oct 24, 2018
    Contributor

    I'm going to reclose this issue and open a new issue for the remaining work.

  18. blowmage
    closed this as completedon Oct 24, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

🚨This issue needs some love.authpriority: p2Moderately-important priority. Fix may not be included in next release.status: acknowledgedtype: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions