Skip to content

Add support for DATETIME, TIME, and BYTES query parameters #1099

Description

@blowmage

BigQuery has DATETIME, TIME, and BYTES data types, but the current query parameter implementation does not yet support them. Make it so.

Activity

  1. added
    api: bigqueryIssues related to the BigQuery API.
    release blockingRequired feature/issue must be fixed prior to next release.
    on Dec 2, 2016
  2. self-assigned this
    on Dec 2, 2016
  3. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    Supporting BYTES should be fairly easy. We support bytes on other libraries and we can check for an IO-ish object. So users can provide a File or StringIO object containing the bytes they want written.

    Supporting TIME is not going to be as easy. Ruby doesn't have an object that only represents time. So how will users format an object to indicate that the value should be a TIME type? What about using a Time object that is less than 24 hours from the epoch? Would that work?

  4. changed the title [-]Add support for DATE and BYTES query parameters[/-] [+]Add support for TIME and BYTES query parameters[/+] on Dec 2, 2016
  5. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    Another option is to use Ruby's DateTime for BigQuery's TIMESTAMP, Ruby's Time for BigQuery's TIME, and Ruby's Date for BigQuery's DATE. This is what ActiveRecord does.

  6. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    @quartzmo Do you have an opinion on this? Thoughts?

  7. quartzmo commented on Dec 2, 2016

    @quartzmo
    Member

    @blowmage What about BigQuery's DATETIME type? Also Ruby's Time, correct?

    I like this solution. Isn't this how Rails maps these types?

  8. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    Ugh, I'm blind. I was going off a different set of types and didn't see those until I started looking for documentation. I agree it would be best to keep Ruby's DateTime for BigQuery's DATETIME, and Ruby's Time for BigQuery's TIMESTAMP. But then we are back to the original question of how do we represent BigQuery's TIME?

  9. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    Supporting TIME is looking more and more difficult. I'm starting to think we will need to add a new object to represent it. Perhaps a Google::Cloud::Bigquery::Time class or struct, and a Project#time helper to make creating a TIME easier.

    require "google/cloud/bigquery"
    
    bigquery = Google::Cloud::Bigquery.new
    
    data = bigquery.query "SELECT * FROM tbl WHERE time_of_day > @time},
                          params: { time: bigquery.time(Time.now) }

    Thoughts?

  10. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    Rails uses Ruby's Time for TIME because some databases can store TIME values larger than 24 hours, so it makes sorta sense to use Ruby's Time for that. But BigQuery isn't that way, so I think Time for TIME is sorta wasted. Implementation-wise, Ruby's Time is really a TIMESTAMP, so I think it best we allow it to represent that and create something new for TIME.

  11. blowmage commented on Dec 2, 2016

    @blowmage
    ContributorAuthor

    We might also need a second pass on the documentation to make sure users understand how values passed in for query parameters will be converted to BigQuery data types.

  12. quartzmo commented on Dec 2, 2016

    @quartzmo
    Member

    @blowmage I also thought of a new object, and this seems OK to me given the situation.

  13. changed the title [-]Add support for TIME and BYTES query parameters[/-] [+]Add support for DATETIME, TIME, and BYTES query parameters[/+] on Dec 2, 2016
  14. added a commit that references this issue on Dec 8, 2016
  15. added a commit that references this issue on Dec 8, 2016
    7548d0c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.release blockingRequired feature/issue must be fixed prior to next release.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions