patterns in .SDcols #1878

Activity

1. eantonya

   added

   feature request

   on Oct 14, 2016
2. ksavin commented on Oct 26, 2016

   

   ksavin

   on Oct 26, 2016

   More actions

   I'd like to add, that patterns would be super useful in j as well.

   It is often needed to select columns with grep and the only way is to refer it via names(), e.g.
   veryLongDataTableName[, grep('lag', names(veryLongDataTableName), with = FALSE]

   or to remove multiple columns, e.g.
   dt[, (grep('lag', names(dt)) := NULL]

   or to make new column names:
   dt[, paste0(grep('^test', names(dt), value = TRUE), '_sqrt') := lapply(.SD, sqrt), .SDcols = grep('^test', names(dt), value = TRUE)]

   These would be much shorter with patterns available in j as well:
   veryLongDataTableName[, patterns('lag'), with = FALSE]
dt[, patterns('lag') := NULL]
dt[, paste0(patterns('^test'), '_sqrt') := lapply(.SD, sqrt), .SDcols = patterns('^test')]

   Alternatively, it would be handy to have a special symbol for column names, selected in .SDcols, e.g. .NM
3. MichaelChirico commented on Oct 26, 2016

   

   MichaelChirico

   on Oct 26, 2016

   Member

   More actions

   names(.SD) should suffice...

   On Oct 26, 2016 12:37 PM, "ksavin" [email protected] wrote:

   I'd like to add, that patterns would be super useful in j as well.

   It is often needed to select columns with grep and the only way is to
   refer it via names(), e.g.
   veryLongDataTableName[, grep('lag', names(veryLongDataTableName), with =
   FALSE]

   or to remove multiple columns, e.g.
   dt[, (grep('lag', names(dt)) := NULL]

   or to make new column names:
   dt[, paste0(grep('^test', names(dt), value = TRUE), '_sqrt') :=
   lapply(.SD, sqrt), .SDcols = grep('^test', names(dt), value = TRUE)]

   These would be much shorter with patterns available in j as well:
   veryLongDataTableName[, patterns('lag'), with = FALSE]
   dt[, patterns('lag') := NULL]
   dt[, paste0(patterns('^test'), '_sqrt') := lapply(.SD, sqrt), .SDcols =
   patterns('^test')]

   Alternatively, it would be handy to have a special symbol for column
   names, selected in .SDcols, e.g. .NM

   —
   You are receiving this because you are subscribed to this thread.
   Reply to this email directly, view it on GitHub
   #1878 (comment),
   or mute the thread
   https://github.com/notifications/unsubscribe-auth/AHQQdapOU-mUdVF-4ADOsKkuD4cSC3Lvks5q34G7gaJpZM4KXfit
   .
4. ksavin commented on Oct 26, 2016

   

   ksavin

   on Oct 26, 2016

   More actions

   Forgot I am in fact using names(.SD) for these cases :)
   Still, way cleaner and shorter with patterns.
5. MichaelChirico commented on Oct 26, 2016

   

   MichaelChirico

   on Oct 26, 2016

   Member

   More actions

   I do regularly things like grep(pattern, names(.SD)) (and maybe add value = TRUE)... Maybe I'm just used to this setup.
6. mbacou commented on Dec 8, 2016

   

   mbacou

   on Dec 8, 2016

   More actions

   Another one I tend to use is .SDcols=names(.SD) %like% "mypattern", a little ugly. Upvoting this FR as well.
7. hannes101 commented on Mar 24, 2017

   

   hannes101

   on Mar 24, 2017

   Last edited by hannes101

   More actions

   Just as a reference to a SO question, please update it there also if it got implemented :-)
   https://stackoverflow.com/questions/42999949/select-data-table-columns-with-grep-like-partial-matching
8. franknarf1 commented on Nov 28, 2017

   

   franknarf1

   on Nov 28, 2017

   Last edited by franknarf1

   Contributor

   More actions

   SO q to update: https://stackoverflow.com/questions/47535845/perform-operations-on-data-table-columns-based-on-regex?noredirect=1

   A to update: https://stackoverflow.com/a/51331981/
9. HughParsonage commented on May 18, 2018

   

   HughParsonage

   on May 18, 2018

   Member

   More actions

   I wrote select_grep in package hutils before realizing this was an outstanding issue:

   library(hutils)
   library(data.table)
   dt <- data.table(x1 = 1, x2 = 2, y = 0)
   select_grep(dt, "x")
   #>    x1 x2
   #> 1:  1  2
       select_grep(dt, "x", .and = "y")
   #>    x1 x2 y
   #> 1:  1  2 0
       select_grep(dt, "x", .and = "y", .but.not = "x2")
   #>    x1 y
   #> 1:  1 0

   Created on 2018-05-19 by the reprex package (v0.2.0).
10. MichaelChirico

    added a commit that references this issue on Dec 5, 2018

    Closes #1878 and #3185 -- .SDcols accepts patterns

    This issue will close once commit b24f490 is merged into the 'master' branch.

    b24f490
11. mattdowle

    added this to the 1.12.0 milestone on Dec 14, 2018
12. mattdowle

    closed this as completedin #3186on Dec 14, 2018