Repository navigation
BigTable - local instance #1506
Description
Activity
@callmehiphop we can probably do this the same way we do with Datastore, with an 'apiEndpoint' option. Want to tack it onto your v2 PR?
Sure!
On second thought I think we should do this post-merge - this feature would have to wait for v2 completion anyways and it would ship around the same time regardless of whether or not we lump them together.
@lesv is it possible to connect to a Bigtable emulator host through gRPC?
I tried to make a request, and the emulator prints:
[bigtable] 2016/09/02 12:28:41 transport: http2Server.HandleStreams received bogus greeting from client: "\x16\x03\x01\x00\xa6\x01\x00\x00\xa2\x03\x03\x9bp|Mb)z\xf7\xe8\x17\xba3\xbe" [bigtable] 2016/09/02 12:28:46 transport: http2Server.HandleStreams failed to receive the preface from client: EOFAnd output from the script that made the request:
E0902 12:28:41.194034000 123145321807872 ssl_transport_security.c:945] Handshake failed with fatal error SSL_ERROR_SSL: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number. E0902 12:28:41.194212000 123145321807872 handshake.c:241] Handshake failed with error TSI_PROTOCOL_FAILURE E0902 12:28:41.194218000 123145321807872 secure_channel_create.c:99] Secure handshake failed with error 1. E0902 12:28:46.549982000 123145338609664 tcp_client_posix.c:191] failed to connect to 'ipv4:127.0.0.1:8051': timeout occurred@stephenplusplus The emulator only supports plaintext negotiation, try to disable TLS. Maybe this is
grpc.ChannelCredentials.createInsecure()?Thank you, I forgot about that! I've run into the next error which is returned from gRPC:
Error: unknown service google.bigtable.admin.v2.BigtableInstanceAdmin at Function.GrpcService.decorateError_ (node_modules/@google-cloud/common/src/grpc-service.js:494:34) at Object.callback (node_modules/@google-cloud/common/src/grpc-service.js:253:35) at node_modules/grpc/src/node/src/client.js:420:14This is trying to create an instance-- are there any methods/operations/services that aren't accessible through the emulator?
It doesn't support any instance-related services. So just use any project/instance and start by creating a table.
Hi, I finally got around to giving this a shot and seems to be hanging on this RPC:
const client = bigtable({ projectId: config.get('gcloud.projectId'), zone: config.get('bigtable.zone'), maxRetries: 6, credentials, }).instance(config.get('bigtable.instance')); ... async init() { this.table = await new Promise((resolve, reject) => { client.table('events').get({ autoCreate: true, families: [{ name: 'event', rule: { versions: 1, }, }], }, (err, table) => { if (err) return void reject(err); resolve(table); }); });I'm not seeing any logging in the bigtable process:
➜ ~ gcloud beta emulators bigtable start --host-port localhost:8721 --quiet Executing: /Users/mata/Downloads/google-cloud-sdk/platform/bigtable-emulator/cbtemulator --host=localhost --port=8721 [bigtable] Cloud Bigtable emulator running on 127.0.0.1:8721Looks like there's a discrepancy between the emulator and the upstream API. When
autoCreateis set to true we first attempt to get the object in question and in the event of a 404 we'll attempt to create said object. The emulator appears to be returning a 500 error instead of a 404.@garye do we know who's in charge of the emulator?
@arbesfeld In the meantime you could try something like
async init() { const table = client.table('events'); this.table = await new Promise((resolve, reject) => { table.get(err => { if (!err) return resolve(table); table.create({ families: [...] }, err => { if (err) return void reject(err); resolve(table); }); }); }); }
@callmehiphop I am :) The emulator is actually just this:
https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/bigtable/bttest/inmem.goLet me look into changing the error code. So the
autoCreateoption callsGetTable?and grpc
NotFoundgets treated as a 404, right?So the autoCreate option calls GetTable?
Correct! First we call
GetTableand if that returns a 404 (grpc 5) then we callCreateTableand grpc NotFound gets treated as a 404, right?
Yep! Sorry I didn't mention that up front, we map grpc codes to HTTP codes to make it easier for the user.
There's a fix in for this: googleapis/google-cloud-go@1d0367c
Will take at least a week to make its way into the gcloud emulator. Let me know if you're up for trying against the go app itself.
@callmehiphop @arbesfeld Any issues left connecting to the emulator or are things looking good here?
It works! Thank you
Reacted by Gary Elliott, Stephen, Dave and Todd Morrison- added a commit that references this issue
on Jul 23, 2025 - added a commit that references this issue
on Feb 3, 2026 - added a commit that references this issue
on Mar 5, 2026
Is there a way to specify the host/port of the BigTable instance? We'd like to use https://github.com/spotify/docker-bigtable for local testing.