Repository navigation
Improve how fread's drop/select deal with duplicate column names #1899
Description
Activity
Poking around a bit; seems this comes down to the return behavior of
chmatch:https://github.com/Rdatatable/data.table/blob/master/src/freadR.c#L275
if (isString(dropSxp)) itemsInt = PROTECT(chmatch(dropSxp, colNamesSxp, NA_INTEGER, FALSE))This is the same as the behavior of
basematch:match('x1', c('x1', 'x1', 'y')) # [1] 1To skirt changing how
chmatchworks, I guess we have to (?) reverse the arguments and look forNA:match(c('x1', 'x1', 'y'), 'x1') # [1] 1 1 NA # ^ know to drop anything that's not NA. # IIUC we can do this more easily setting `in = TRUE` in arguments to chmatch @ C levelI know how I would do this in R (
!names(x) %in% drop), but not sure the right way to go about it in C
Consider
x1is duplicated. If bothx1are chosen byselect, both should be excluded bydrop. At the very least, there should be a warning about multiplex1detected.Use case is I have a .csv with a lot of columns called
fillerwhich are pointless white space. It would be nice to just saydrop = "filler"and exclude all of them.Originally filed on SO.