I didn't think get() copies, why is fun2() slowest here? #727

Activity

1. mattdowle

   added

   question

   on Jul 10, 2014
2. arunsrinivasan commented on Jul 15, 2014

   

   arunsrinivasan

   on Jul 15, 2014

   Member

   More actions

   I'm almost sure it's because of this part in [.data.table:

   if (is.null(irows)) {
       for (s in seq_along(xcols)) {  # xcols means non-join x columns, since join columns come from i
           target = xcolsAns[s]
           source = xcols[s]
           ans[[target]] = x[[source]]
           if (address(ans[[target]]) == address(x[[source]])) ans[[target]] = copy(ans[[target]])
       }
   } else {
       for (s in seq_along(xcols)) {
           target = xcolsAns[s]
           source = xcols[s]
           ans[[target]] = .Call(CsubsetVector,x[[source]],irows)   # i.e. x[[source]][irows], but guaranteed new memory even for singleton logicals from R 3.1
       }
   }

   get() fetches all columns and here is.null(irows) == TRUE and therefore runs the for-loop under the if-clause looping through each one of columns, copying each time, before assigning ans to .SD later on.
3. MichaelChirico commented on Sep 2, 2019

   

   MichaelChirico

   on Sep 2, 2019

   Member

   More actions

   I think that without .SDcols get has to copy. So I'm not sure there's any fix to this besides supplying .SDcols... OK to close?
4. jangorecki commented on Mar 22, 2020

   

   jangorecki

   on Mar 22, 2020

   Member

   More actions

   I think that solution proposed #4304 is a proper way to address this issue. It removes the burden of maintaining get optimisation. I think it is is reasonable to advise to use substitution rather than get, thus closing. SO should be updated when we will have it in master.
5. jangorecki

   closed this as completedon Mar 22, 2020