Skip to content

Project ID required even if running script from Compute Engine #1908

Description

var datastore = require('@google-cloud/datastore')();

The line above used on compute engine (not app engine) will still throw the error below -

[Error: Sorry, we cannot connect to Google Cloud Services without a project ID. You may specify one with an environment variable named "GCL
OUD_PROJECT". See https://googlecloudplatform.github.io/google-cloud-node/#//docs/guides/authentication for a detailed guide on creating an
 authenticated connection.]

Yet in the documentation it says -

If you are running your app on Google Compute Engine, you won't need to worry about supplying connection configuration options to @google-cloud/datastore— we figure that out for you.

That is not the case.

To reproduce create a google Compute engine instance and run a node.js script without a supplied project ID it will fail.

Activity

  1. stephenplusplus commented on Jan 8, 2017

    @stephenplusplus
    Contributor

    Thanks for reporting. Which version of @google-cloud/datastore are you running?

  2. Splaktar commented on Jan 11, 2017

    @Splaktar
    Contributor

    I'm seeing this today on App Engine Flex.

    Error: Sorry, we cannot connect to Google Cloud Services without a project ID. You may specify one with an environment variable named "GCLOUD_PROJECT". See https://googlecloudplatform.github.io/google-cloud-node/#//docs/guides/authentication for a detailed guide on creating an authenticated connection.
    

    But I have an environment variable set for GCLOUD_PROJECT=myproject. This was working fine on January 3rd when I last deployed. But it is blocking me from deploying today.

    Note that I'm still on "google-cloud": "0.41.2". Will upgrading to the latest solve this or is it a bug in App Engine Flex?

    I opened #1918 to fix the broken link in the error message.

  3. Splaktar commented on Jan 11, 2017

    @Splaktar
    Contributor

    I was on gcloud 138.0.0, but I found this:

    139.0.0 (2017/01/11)
      Breaking Changes
    - (Google App Engine) Flexible deployments now skip files matching `node_modules` and `.git` by default, but no longer skip other hidden files by default. Standard deployments are not changing.
    

    We're using https://github.com/motdotla/dotenv for our ENV variables and we use .env and .env-prod files that were apparently being stripped out of our container by the builder service in 138.0.0.

    I tried again with gcloud 139.0.0 and we're able to deploy our app again. Though this doesn't seem like it will resolve the OP's issue unfortunately :(

  4. michaelfreund commented on Jan 25, 2017

    @michaelfreund

    This also happens on non app or compute engine environments with >= 0.45.1. Downgrading to 0.45.0 resolves the issue.

    const Gcloud = require('google-cloud')({
      keyFilename: Config.GCLOUD.KEYFILE
    });

    @stephenplusplus any ideas?

  5. stephenplusplus commented on Jan 25, 2017

    @stephenplusplus
    Contributor

    Looking into now, sorry for the delay!

  6. stephenplusplus commented on Jan 25, 2017

    @stephenplusplus
    Contributor

    @HighNoonSolutions I haven't been able to reproduce on GCE. I set up a micro box, installed Node, and tried against both [email protected] and @google-cloud/[email protected].

    @michaelfreund Using [email protected] locally (non-GAE or GCE) didn't throw for me.

    I must be failing to mimic both of your environments in some way, so please share anything else I can do to reproduce. In general, be sure that you're starting from a clean node_modules directory and un- and then re-install google-cloud or any of the sub-modules you might be using, so you're getting the latest copies.

  7. michaelfreund commented on Jan 27, 2017

    @michaelfreund

    @stephenplusplus completely removed and reinstalled node modules. I still get the same error. If I output the error message from https://github.com/GoogleCloudPlatform/google-cloud-node/blob/b1c6b210106bb064b0662173e2dbcd41ad211f89/packages/common/src/util.js#L355 I can see a Maximum call stack size exceeded running on node v6.9.2.

    Taking a look at https://github.com/GoogleCloudPlatform/google-cloud-node/blob/b1c6b210106bb064b0662173e2dbcd41ad211f89/packages/common/src/util.js#L519 I figured out that iteration is running 47933 times.

    bildschirmfoto 2017-01-27 um 08 27 39

  8. arbesfeld commented on Feb 15, 2017

    @arbesfeld
    Contributor

    Hey @michaelfreund, did you find a fix fox this? We're experiencing the same problem

  9. stephenplusplus commented on Feb 16, 2017

    @stephenplusplus
    Contributor

    This may have been resolved with a recent change. Please try upgrading to [email protected].

  10. arbesfeld commented on Feb 17, 2017

    @arbesfeld
    Contributor

    Unfortunately this didn't fix it. We are using projectID like this:

    import makeGcloud from 'google-cloud';
    
    const gcloud = makeGcloud({
      projectId: 'my-project',
    });
    const bigtable = gcloud.bigtable()
    

    While we are also running in App Engine. Would this be expected to work?

  11. added
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    priority: p2Moderately-important priority. Fix may not be included in next release.
    on Mar 1, 2017
  12. 12 remaining items

  13. bjwatson commented on Mar 7, 2017

    @bjwatson

    Ok, thanks. I dropped this priority to P1. Let's close it in another week if we don't hear any other complaints about this.

  14. dan-petty commented on Mar 11, 2017

    @dan-petty

    Getting this with 0.48.0 on App Engine nodejs runtime.
    var gcloud = require('google-cloud')({ projectId: 'my-project-id-here', keyFilename: './keyfile.json' });

    Log error when trying to access gcs bucket:

    "Error: Sorry, we cannot connect to Cloud Services without a project ID. You may specify one with an environment variable named "GCLOUD_PROJECT". See https://googlecloudplatform.github.io/google-cloud-node/#/docs/guides/authentication for a detailed guide on creating an authenticated connection."

    EDIT: Reverting to:

    `"gcloud": "^0.37.2"`
    

    resolves issue with no changes to code.

    EDIT 2: Also resolved by:

    var storage = require('@google-cloud/storage');
    var gcs = storage({
      projectId: 'my-project-id-here',
      keyFilename: './keyfile.json'
    });
  15. stephenplusplus commented on Mar 13, 2017

    @stephenplusplus
    Contributor

    Thanks @mixmatch. I'm about to kick off a publish-fest. At the end of it, we'll have [email protected], and hopefully you can test for us again.

  16. stephenplusplus commented on Mar 14, 2017

    @stephenplusplus
    Contributor

    The publish fest is over-- I believe everyone should be covered now if you un- and re-install the latest dependencies that you need. Let us know if there are still any issues.

  17. xyk2 commented on Mar 15, 2017

    @xyk2

    I am still having the same problems on 0.49.0 after doing a clean uninstall and reinstall. Downgrading back to 0.45.0 resolves this issue.

    var google_cloud = require('google-cloud')({
    	projectId: 'PROJECT',
    	keyFilename: 'KEYFILE.json'
    });
    
    var gcs = google_cloud.storage();
    var bucket = gcs.bucket('SOURCE-BUCKET');
    var dest_bucket = gcs.bucket('DESTINATION-BUCKET');

    Strangely the error only occurs on the upload stage to dest_bucket.

  18. stephenplusplus commented on Mar 15, 2017

    @stephenplusplus
    Contributor

    What environment are you running in @xyk2? Also, could you show the code you're using to make the API request that fails?

  19. xyk2 commented on Mar 16, 2017

    @xyk2

    Apologies should have included that earlier - Node 6.10.0 on Mac OSX 10.11.6 using restify.

    var restify = require('restify');
    var server = restify.createServer({
    	name: 'ffmpeg-runner'
    });
    
    var google_cloud = require('google-cloud')({
    	projectId: 'PROJECT-ID',
    	keyFilename: 'keys/KEYFILE.json'
    });
    
    var gcs = google_cloud.storage();
    var dest_bucket = gcs.bucket('BUCKET_NAME');
    
    server.get('/download', download);
    server.get('/upload', upload);
    
    function download(req, res, next) {
    	dest_bucket.file('sandbox/index.m3u8').download({destination: 'index.m3u8'}, function(err) {
    		if(err) return console.log(err);
    
    		res.send("Download successful");
    		console.log("Download successful");
    	});
    }
    
    function upload(req, res, next) {
    	dest_bucket.upload('index.m3u8', {destination: 'sandbox/index.m3u8'}, function(err, gFileObj) {
    		if(err) return console.log(err);
    
    		res.send('Upload successful');
    		console.log('Upload successful');
    	});
    }
    
    server.listen(8080, function() {
    	console.log('%s listening at %s', server.name, server.url);
    });

    The code where the error occurs is in the restify upload handler. When there is a GET request /download, the file downloads fine from GCS. The upload handler throws Error: Sorry, we cannot connect to Cloud Services without a project ID. The same upload code works when placed outside the Restify handler though, so I may be missing something very stupid...

  20. stephenplusplus commented on Mar 16, 2017

    @stephenplusplus
    Contributor

    Actually, I think that was exactly what I needed to see the problem. We have been iterating over a larger object than I believe we should be. I'm putting a fix through and I think we'll all be happier people after it thanks to you 👍

  21. stephenplusplus commented on Mar 17, 2017

    @stephenplusplus
    Contributor

    PR sent in #2100.

  22. added a commit that references this issue on Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

corepriority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions