Skip to content

Integer64 columns not assigned NA when doing a left merge #1459

Description

@abielr

Handle NA integer64 properly in:

  • merge
  • rbindlist (following @jangorecki's comment below)
require(data.table)
require(bit64)
x1 <- data.table(id=1, value=as.integer64(1))
x2 <- data.table(id=c(1,2))
merge(x2, x1, by="id", all.x=T)

Yields

#    id               value
#1:  1                   1
#2:  2 9218868437227407266

This is with data.table 1.9.6.

Activity

  1. dlithio commented on Dec 4, 2015

    @dlithio

    I think this is a duplicate.

  2. arunsrinivasan commented on Dec 4, 2015

    @arunsrinivasan
    Member

    Thanks for linking to the old issue @dlithio. Maybe we can fix it internally..

  3. jangorecki commented on Apr 23, 2016

    @jangorecki
    Member

    The same happens when rbindlist(fill=TRUE). This probably can be fixed in the same place as merge so I'm not creating new issue for that, let me know if you want separate one for rbindlist fill.

    library(data.table)
    library(bit64)
    rbindlist(list(
        data.table(a=1:2, b=as.integer64(c(1,NA))),
        data.table(a=3L)
    ), fill=TRUE)
    #   a                   b
    #1: 1                   1
    #2: 2                  NA
    #3: 3 9218868437227407266
  4. added this to the v1.9.8 milestone on Jul 21, 2016
  5. added a commit that references this issue on Jul 21, 2016
  6. modified the milestones: , v1.9.8 on Jul 25, 2016
  7. andrewkho commented on Oct 2, 2017

    @andrewkho

    I'm not sure if I should create a new issue for this, but I am having this problem when using the := operator, but not when using merge.

    > packageVersion("data.table")
    [1] ‘1.10.4’
    > packageVersion("bit64")
    [1] ‘0.9.7’
    > dt1 <- data.table(id=c(1,2), x=bit64::as.integer64(c(100,200)))
    > dt2 <- data.table(id=c(1,3))
    > dt1
       id   x
    1:  1 100
    2:  2 200
    > dt2
       id
    1:  1
    2:  3
    > merge(dt1,dt2, all=T)
       id   x
    1:  1 100
    2:  2 200
    3:  3  NA
    > dt2[J(dt1), x := i.x, on=.(id)]
    > dt2
       id                   x
    1:  1                 100
    2:  3 9218868437227407266
    

    Update: I guess this is the expected behaviour:

    > merge(dt1, dt2, all.y = T)
       id   x
    1:  1 100
    2:  3  NA
    

    I have a lot of code using the := method, and I'm not sure if it's feasible to switch everything to use merge instead

  8. MichaelChirico commented on Oct 3, 2017

    @MichaelChirico
    Member
  9. removed this from the milestone on May 10, 2018
  10. added a commit that references this issue on Jul 23, 2019
  11. added this to the 1.12.4 milestone on Aug 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions