Skip to content

BigTable asynchronous row handling #1715

Description

@arbesfeld

Is there a way to asynchronously handle rows with a BigTable range query? What I'm trying to do is something like this:

table.getRows().on('data', async row => {
  await doSomeExpensiveWork(row);
});

Activity

  1. added
    type: questionRequest for information or clarification. Not an issue.
    api: bigtableIssues related to the Bigtable API.
    on Oct 18, 2016
  2. callmehiphop commented on Oct 18, 2016

    @callmehiphop
    Contributor

    What about something like this?

    $ npm install --save through2
    var through = require('through2');
    
    table.createReadStream()
      .on('error', console.error)
      .pipe(through.obj((row, _, next) => {
        doSomeExpensiveWork()
          .then(() => next(null), next);
      })
      .on('finish', function() {
        // all done!
      });
  3. stephenplusplus commented on Oct 18, 2016

    @stephenplusplus
    Contributor

    Yep, that's what I would say. It will not pull more data from the stream until you're done processing the last result.

    In case users of other APIs see this, this will work for any object stream from our APIs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

api: bigtableIssues related to the Bigtable API.type: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions