Merge (outer join) fails to set NAs for integer64 #1385

Activity

1. arunsrinivasan commented on Oct 11, 2015

   

   arunsrinivasan

   on Oct 11, 2015

   Member

   More actions

   I'm not sure what we can do about this.. But good to know, thanks.

   Matt, if you think we should fix this, feel free to reopen.
2. arunsrinivasan

   closed this as completedon Oct 11, 2015
3. arunsrinivasan

   added

   wontfix

   on Oct 11, 2015
4. dlithio commented on Oct 12, 2015

   

   dlithio

   on Oct 12, 2015

   Author

   More actions

   Thanks for the response. I don't have strong feelings either way, but I wonder if it would be appropriate to provide a warning of some sort either in fread or in data.table's merge. Just to be clear, the reason for this would be that using base R to read files and merge will give expected results (at least in terms of what is NA)

   df1 = read.table(header = TRUE, text = "
                    x y
                    1 30000000001
                    ")
   
   df2 = read.table(header = TRUE, text = "
                    x
                    1
                    2
                    ")
   
   merge(df1,df2,all=TRUE)
   
   #  x     y
   #1 1 3e+10
   #2 2    NA

   While reading the same two files with fread raises the issue I described above

   require(data.table)
   
   dt1 = fread("x y
                1 30000000001")
   
   #Warning message:
   #  In fread("x y\n             1 30000000001") :
   #  Some columns have been read as type 'integer64' but package bit64 isn't loaded. Those columns will display as strange looking floating point data. There is no need to reload the data. Just require(bit64) to obtain the integer64 print method and print the data again.
   
   require(bit64)
   
   dt2 = fread("x
                    1
                    2")
   
   setkey(dt1,x)
   setkey(dt2,x)
   
   merge(dt1,dt2,all=TRUE)
   
   #x                   y
   #1: 1         30000000001
   #2: 2 9218868437227407266

   Even if you do require(bit64) after the merge the same issue still persists.
5. arunsrinivasan

   added

   enhancement

   and removed

   wontfix

   on Jul 21, 2016
6. arunsrinivasan

   self-assigned this

   on Jul 21, 2016
7. arunsrinivasan

   added this to the v1.9.8 milestone on Jul 21, 2016
8. arunsrinivasan

   added a commit that references this issue on Jul 21, 2016

   Closes #1385, partly #1459. Joins handle no match for int64 columns c…

   ...

   This issue will close once commit 7bc0f17 is merged into the 'master' branch.

   7bc0f17