Skip to content

Multiple column assignment using :=(...) with shift() creates list column types in 1.14.3 #5403

Description

@ben519

Minimal Reprex

The following code works as expected

dt <- data.table(grp = c(1,1,1,2,2,2), x = 1:6)
dt[, x_prev := shift(x = x, type = "lag", n = 1), by = grp]
dt[, x_next := shift(x = x, type = "lead", n = 1), by = grp]

print(dt)
     grp     x x_prev x_next
   <num> <int>  <int>  <int>
1:     1     1     NA      2
2:     1     2      1      3
3:     1     3      2     NA
4:     2     4     NA      5
5:     2     5      4      6
6:     2     6      5     NA

but when I try to create x_prev and x_next in one fell swoop, the resulting fields are lists of vectors

dt <- data.table(grp = c(1,1,1,2,2,2), x = 1:6)
dt[, `:=`(
  x_prev = shift(x = x, type = "lag", n = 1),
  x_next = shift(x = x, type = "lead", n = 1)
), by = grp]

print(dt)
     grp     x            x_prev            x_next
   <num> <int>            <list>            <list>
1:     1     1 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA
2:     1     2 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA
3:     1     3 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA
4:     2     4 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA
5:     2     5 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA
6:     2     6 NA, 1, 2,NA, 4, 5  2, 3,NA, 5, 6,NA

The same code works properly on data.table 1.14.2, so this appears to be a bug introduced by 1.14.3.


> sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: x86_64-apple-darwin21.3.0 (64-bit)
Running under: macOS Monterey 12.4

Matrix products: default
LAPACK: /usr/local/Cellar/r/4.2.0/lib/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.14.3

loaded via a namespace (and not attached):
[1] compiler_4.2.0  httr_1.4.3      R6_2.5.1        generics_0.1.2  tools_4.2.0     lubridate_1.8.0

Activity

  1. added
    GForceissues relating to optimized grouping calculations (GForce)
    on Jun 5, 2022
  2. ben-schwen commented on Jun 5, 2022

    @ben-schwen
    Member

    Thanks for the report. This is another case of #5245. What happens is that gshift is allowed to return a list contrary to all other gforce functions.

    This list is then wrongly unpacked and assigned.

    Second case which errors in 1.14.3 but works in 1.14.2 is when returning a list with gshift on purpose.

    dt = data.table(grp = c(1,1,2,2), x = 1:4)
    dt[, `:=`(
      x1 = shift(x, type = "lag", n = c(1,-1))
    ), by = grp]
  3. added a commit that references this issue on Jun 5, 2022
  4. jangorecki commented on Jun 16, 2022

    @jangorecki
    Member

    Thanks for working this out. Also when reporting devel bugs please include git sha.

  5. added this to the 1.14.3 milestone on Jul 19, 2022
  6. modified the milestones: 1.14.9, 1.15.0 on Oct 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    GForceissues relating to optimized grouping calculations (GForce)bugdev

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions