Repository navigation
change the order of file.exists(input) and grep('\\n|\\r', input) in fread.R - #2630
Conversation
in `fread.R` in order to speed up execution time when `input` is the actual data. `file.exists` is very slow, so checking first for new lines speeds up the process. See also discussion in Rdatatable#2531
|
it seems like an R bug that length(grep(.)) could be faster than grep(.),
no? isn't that the point of having a function like gepl in the first place??
|
|
@MichaelChirico |
|
True, though checking It would seem |
|
@HughParsonage see my comment in #2531 with the benchmark of If on the other hand then |
Codecov Report
@@ Coverage Diff @@
## master #2630 +/- ##
=======================================
Coverage 93.03% 93.03%
=======================================
Files 61 61
Lines 12115 12115
=======================================
Hits 11271 11271
Misses 844 844
Continue to review full report at Codecov.
|
mattdowle
left a comment
There was a problem hiding this comment.
Thanks for investigating and fixing. Yes, I thought file.exists() would be fast to return false on large input. Interesting it isn't. And it's good to avoid too many OS calls too I suppose, say if fread is being calling a lot in a loop with direct character input.. Nice fix.
change the order of
file.exists(input)andgrep('\\n|\\r', input)in
fread.Rin order to speed up execution time wheninputis theactual data.
file.existsis very slow, so checking firstfor new lines speeds up the process.
For benchmark on
file.existssee also this comment in #2531Not adding additional tests, as no change in functionality, just speed up of execution. (current tests all pass locally).
Not adding NEWS entry, as there is already one for #2531 and this PR is just a trivial addition.