Figure out some integration with Cloud Bigtable?

@stephenplusplus

@stephenplusplus : Any ideas ? Are there any popular existing HBase libraries for NodeJS?

https://www.npmjs.com/package/hbase-client (https://github.com/alibaba/node-hbase-client) seems to be the most popular.

https://www.npmjs.com/package/node-hbase (https://github.com/wdavidw/node-hbase) is also up to date, but not as popular on npm

Can we get some more clarification on how we communicate with the remote server? Most of the documentation I'm seeing talks about setting up a local server

@dhermes

/cc @dhermes @maxluebbe @carterpage

Do you mean bigtable.googleapis.com? Or something else? I think all you should need is the server name and use the gRPC package. (HTTP/2) Am I misunderstanding the Q?

@lesv

@lesv yeah, I'm referring to bigtable.googleapis.com, the node clients we're looking at have some default settings I'm not sure apply to us and I didn't see any documentation specific to communicating with the remote server outside of command line usage.

Take a look at https://io2015codelabs.appspot.com/codelabs/gRPC it's a local critter, but I think it has how to query another server from it. If not, I can ask for you.

@lesv

@lesv thanks, will do!

Take a look at https://github.com/googlesamples/io2015-codelabs/blob/master/gRPC/complete/client.js#L5

@lesv

@lesv that definitely helps, thanks for linking me to that.

@jgeewax I don't think either of the node libraries mentioned support gRPC, however, one of them linked to a coffeescript library that has protobuf support. Do you know if the .proto file we need to use already exists and if so where?

Those libraries definitely won't support gRPC -- we'd need to swap out the transport layer to be gRPC.

Protos are here: https://github.com/GoogleCloudPlatform/cloud-bigtable-client/tree/master/bigtable-protos (I think those are the right ones). You'll need protoc v3 :-/ ... aka, get ready to type make and such...

FYI

I've done a bunch of work around getting the gRPC C/C++ core installed as well as making the calls to BigTable. I'm happy to have a chat if needed.

@dhermes

@dhermes awesome, thanks! I may just take you up on that 😄

I'm running into an issue (could be a user error) where I try to load bigtable_service.proto but the grpc module does not know how to interpret the rpc options.

JS

var grpc = require('grpc');

var bigtable = grpc.load({
  root: __dirname,
  file: '/google/bigtable/v1/bigtable_service.proto'
});

Ouput

Error: Illegal option value in message undefined, option (google.api.http) at line 37: {
    at Error (native)
    at ProtoBuf.DotProto.ParserPrototype._parseOption

If I remove the rpc options everything works properly.

I'm sort of at a loss on how to proceed -- I was able to build the proto files in python, etc. via protoc but there's not a JavaScript option, I'm assuming because the standard is to just pass the .proto files directly to the module.

@nathanielmanistaatgoogle

@nathanielmanistaatgoogle @tbetbetbe Can you refer us to the folks working on the grpc Node library?

Thanks for all of the help, everyone!

Looks like protobuf.js doesn't understand this syntax:

option (google.api.http) = { post: "/v1/{table_name=projects/*/zones/*/clusters/*/tables/*}/rows:read" body: "*" };

But it does understand this syntax:

option (google.api.http).post = "/v1/{table_name=projects/*/zones/*/clusters/*/tables/*}/rows:read";

I had to change that in the proto files just to get grpc not to crash. I haven't made it further than that, so I don't know if it's going to cause problems down the line. Put it up in a repo here: https://github.com/stephenplusplus/bigtable-grpc-playground so we can keep charging forward.