Repository navigation
Integration with Gapic-generated code #1463
Description
Activity
Hello! Is there any "getting started" material for those projects? Example code would be very helpful as well.
Sorry, we don't 😞
We are working on it, but for now, I want to generate on my local setup and send the generated files as a PR.Hmm, I've noticed that the handling of grpc-client is significantly different.
Our code generator will generate the code assuming to obtain
grpc.load()results and inject some features (like retries or converting paged responses to a stream) through the GAX library, whereas gcloud-node's lib/common provides another abstraction layer with features like retries.Let me keep the existing scheme for now, but probably this common library and GAX would converge in some way in the future.
No problem, I'm excited to see the PR.
but probably this common library and GAX would converge in some way in the future.
Yeah, that would be great! I'd really like if we deferred to GAX for retries/error handling best practices.
Proposals that came out of my meeting with @jmuk, using the Speech API as an example:
Variant 1 - expose auto-gen on exported module
Importing
// Initialize hand-written client: var Speech = require('@google-cloud/speech'); var speech = Speech({ keyFilename: '/path/to/keyfile.json', projectId: 'grape-spaceship-123' }); // Use an idiomatic convenience method, which was implemented using // the v2 "bare metal" method shown below speech.createDetectJob(...); // Initialize auto-generated client for v2: var speech_v2 = Speech.v2(); // Use a "bare metal" grpc method speech_v2.asyncrecognize(...); //Initialize auto-generated client for v1: var speech_v1 = Speech.v1(); // Use a "bare metal" grpc method speech_v1.asyncrecognize(...);
This requires that the exported
Speechmodule has the auto-gen clients attached to it:function Speech (...) {...} Speech.v2 = require('./v2'); Speech.v1 = require('./v1'); module.exports = Speech;
Getting auto-gen updates integrated into gcloud-node would require a little more
engineering in the generated PRs, as the rootindex.jsfile has to be edited.File structure
packages/ speech/ src/ index.js // hand-written layer, uses auto-gen under the hood v1/ index.js // auto-generated v2/ index.js // auto-generated package.jsonpackage.jsoncontains"main": "./src/index.js".What if the API doesn't have or need a hand-written layer?
The root
index.jsfile would just export one of the auto-gen clients.// Initialize hand-written client: var simpleapi = require('@google-cloud/simpleapi')(); // Initialize auto-generated client for v2 of simpleapi: var simpleapi_v2 = simpleapi.v2(); assert.deepEqual(simpleapi, simpleapi_v2)
Variant 2 - user imports auto-gen on separate path
Importing
// Initialize hand-written client: var speech = require('@google-cloud/speech')({ keyFilename: '/path/to/keyfile.json', projectId: 'grape-spaceship-123' }); // Use an idiomatic convenience method, which was implemented using // the v2 "bare metal" method shown below speech.createDetectJob(...); // Initialize auto-generated client for v2: var speech_v2 = require('@google-cloud/speech/v2')(); // Use a "bare metal" grpc method speech_v2.asyncrecognize(...); // Initialize auto-generated client for v1: var speech_v1 = require('@google-cloud/speech/v1')(); // Use a "bare metal" grpc method speech_v1.asyncrecognize(...);
This does NOT require the exported
Speechmodule to have the auto-gen clients attached to it:function Speech (...) {...} module.exports = Speech;
Instead the user gets a hold of an auto-gen client by importing it from a deeper path:
var speech_v1 = require('@google-cloud/speech/v1')();
The updated folder structure in this variant allows
require's resolution algorithm
to find the correct file within the package.Getting auto-gen updates integrated into gcloud-node would be a little easier,
as the rootindex.jsdoes not have to be edited. auto-gen clients for new API
versions could be picked up automatically.File structure
packages/ speech/ index.js // hand-written layer, uses auto-gen under the hood v1/ index.js // auto-generated v2/ index.js // auto-generated package.jsonpackage.jsonomits themainfield.What if the API doesn't have or need a hand-written layer?
The root
index.jsfile would just export one of the auto-gen clients.// Initialize hand-written client: var simpleapi = require('@google-cloud/simpleapi')(); // Initialize auto-generated client for v2 of simpleapi: var simpleapi_v2 = require('@google-cloud/simpleapi/v2')(); assert.deepEqual(simpleapi, simpleapi_v2)
Variant 1 has my vote. That makes the transition seamless to the users that just want the convenience API, then uses API versions as the concise, logical entry points to the users that need full access. 👍
One additional thought I had is that the hand-written layer inevitably must instantiate an auto-gen client in order to use it to implement idiomatic methods, thus, the exposed hand-written client should make its auto-gen client methods available on itself, for example:
// Initialize hand-written client: var Speech = require('@google-cloud/speech'); var speech = Speech({ keyFilename: '/path/to/keyfile.json', projectId: 'grape-spaceship-123' }); // use hand-written method speech.createDetectJob(...); // now use bare-metal method, a pass-thru to auto-gen client speech.asyncrecognize(...); // This is wasteful, as the v2 auto-gen client has now been initialized twice. // I shouldn't HAVE to do this just to get to that bare-metal method var speech_v2 = Speech.v2(); speech_v2.asyncrecognize(...);
I actually prefer the separation, but we also have name clashing to watch out for. An alternative to a call to "version()" could be just dropping the methods inside a "version" property on the instantiated object:
pubsub.getTopics()
pubsub.v1beta1.getTopics()Actually not sure about that ^. If required scopes change between API releases, we couldn't re-use an auth client. So without a function call to create a new "v2()", we would have to use some kind of lazy-instantiating code that creates a new auth-client. That often means confusing code and unintended side effects.
I personally don't mind the "v2()" call and like the separation implied by that hierarchy, but happy to hear other thoughts on the matter.
I personally don't mind the "v2()" call and like the separation implied by that hierarchy, but happy to hear other thoughts on the matter.
You probably right. If someone wants to use a bare-metal method, they can just instantiate an auto-gen client. However, documentation for it is hard to generalize because the auto-gen clients are namespaced by a version. The user would have to be aware of the API version the want to use in order to do it, which is an extra thing to have to think about.
Hi,
I pushed the initial patch for Speech API client as jmuk@ec6ecf7
Please take a look for how it goes. A minor difference is -- we generate a 'class' (or a service object) per grpc service. Because an API can consist of multiple services, it will be:
var Speech = require('@google-cloud/speech'); // hand-written layere var speech = Speech({ keyFilename: ... projectId: ... }); // auto-gen layer var api = new Speech.v1.SpeechApi(); api.nonStreamingRecognize(...);rathrer than
Speech.v1(...)or something. A better example would be pubsub -- it would consist ofpubsub.v1.PublisherApiandpubsub.v1.SubscriberApi. I believe that won't affect the conclusion though.Regarding the auth-tokens (sorry I'm late on the response): GAX depends on the auth library ("google-auth-library"), the generated code would (and should) support the customized authentication as the auth library accepts.
11 remaining items
- added a commit that references this issue
on Jul 23, 2025 - added a commit that references this issue
on Oct 22, 2025 - added 2 commits that reference this issue
on Oct 29, 2025 - added a commit that references this issue
on Feb 25, 2026 - added a commit that references this issue
on Mar 27, 2026 - added a commit that references this issue
on May 5, 2026
I'm in charge of Gapic codegen and GAX for NodeJS
It was an independent effort, but recently we reached to an agreement that bringing the generated code into a part of gcloud itself.
It's still unclear to me where the generated code will be located and how these files are accessible from users, so this is an issue to discuss about these things.