Repository navigation
fread's nrow argument could accept -ve values to skip last 'n' rows #1643
Description
Activity
Note that this would break the current default of
-1L.Other than that, it seems like it shouldn't be too hard to implement, maybe one or two extra lines in this branch of
fread.c:https://github.com/Rdatatable/data.table/blob/master/src/fread.c#L956-L1018
Yeah, I could use this. I'm currently reading in csvs that often have an incomplete last line (not enough fields as inferred from commas), which reliably causes fread to crash R.
It would be nice to set
skip.last=1Lto avoid this. Becausenrowalready allows a negative value as Michael mentioned, I think it would be cleaner as a separate arg or allowing theskiparg to have a length of two (with the second component of the vector taking on this role when present).I would prefer the way mentioned by Arun, as it would be consistent to linux
headandtailway of handling negative values. If negativeskipis currently being used, and cannot be easily changed, then it make sense to allowskipof length two, soskip=c(0, 1)would skip just the last line.
Just for completeness current workaround:fread("head -n -1 filename.csv")Reacted by Frank, Michael Chirico, Artem Klevtsov and Yuri DiasOn a related note, it would be nice if we could pass a list of indicies (also as part of the 'skip' parameter) to explicitly read in the rows you want, like we can do in python's Pandas. If the list of indicies is random, it is a nice way to create a random sample of a data frame which is too large to be read onto a local machine, for instance.
@lanceculnane see also #583
Reacted by lanceculnane
could be useful for this post: http://stackoverflow.com/q/36558437/559784