Skip to content

Bigtable this.end() does not stop the stream? #2271

Description

@arbesfeld

Using Bigtable 0.9.1 - I seem to be getting more than 1 result after calling this.end(). Is this expected behavior?

Activity

  1. stephenplusplus commented on May 2, 2017

    @stephenplusplus
    Contributor

    I recall seeing this discussed in another issue, and it might be an issue with one of our dependencies. I will look into it. Thanks for reporting.

  2. stephenplusplus commented on May 2, 2017

    @stephenplusplus
    Contributor

    The issue I was thinking of doesn't actually relate to the Bigtable API, so throw that theory out the window.

    I tried to reproduce, but couldn't. My script and results:

    var bigtable = require('@google-cloud/bigtable')()
    
    bigtable.getInstancesStream()
      .on('data', function (instance) {
        console.log('instance')
        this.end()
      })
      .on('end', function () {
        console.log('over')
      })
    
    // output:
    instance
    over

    Without the this.end():

    var bigtable = require('@google-cloud/bigtable')()
    
    bigtable.getInstancesStream()
      .on('data', function (instance) {
        console.log('instance')
      })
      .on('end', function () {
        console.log('over')
      })
    
    // output:
    instance
    instance
    instance
    over

    Are you using a different method? Could there be other code interfering with when this.end() is called? Seeing your code might help.

  3. arbesfeld commented on May 3, 2017

    @arbesfeld
    ContributorAuthor

    We're using table.createReadStream(). Here's what it looks like:

        await new Promise((resolve, reject) => {
          table
            .createReadStream({
              decode: true,
              start: 'foo',
              end: 'bar'
              filter: [{
                column: {
                  cellLimit: 1,
                },
              }],
            })
            .on('data', function onData(row) {
              this.end();
              resolve();
            })
            .on('end', resolve)
            .on('error', reject);
        });

    Other methods could be using table at the same time, if that affects something?

  4. stephenplusplus commented on May 3, 2017

    @stephenplusplus
    Contributor

    Would it be a problem that resolve() is called from the data handler and end? this.end() will gracefully exit the stream, which means that the end event will be called.

  5. arbesfeld commented on May 3, 2017

    @arbesfeld
    ContributorAuthor

    That would be acceptable for our application. We found that onData would continued to be called even after this.end(). We're on Node 7.9.0, and we found a workaround by introducing another variable outside of the promise.

  6. added
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on May 3, 2017
  7. stephenplusplus commented on May 3, 2017

    @stephenplusplus
    Contributor

    Found the issue. PR incoming.

  8. stephenplusplus commented on May 4, 2017

    @stephenplusplus
    Contributor

    PR sent: #2276

  9. added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    on Aug 7, 2017
  10. lukesneeringer commented on Oct 31, 2017

    @lukesneeringer
    Contributor

    This issue was moved to googleapis/nodejs-bigtable#6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

api: bigtableIssues related to the Bigtable API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions