Skip to content

fread error with explicit 'select = NULL' #2007

Closed
Closed
fread error with explicit 'select = NULL'#2007
Labels
bugfread
Milestone
v1.11.0

Description

@rcapell
rcapell
opened on Jan 30, 2017

data.table version: 1.10.0
R version 3.3.2
(full session info at bottom of post)

Default value for fread argument select is NULL according to the function's help file.

I get an error when i explicitly pass select = NULL to fread (I use fread in a wrapper function and pass select to the wrapper).

Example, first two lines work as expected, third line yields an error:

# 1: read some data
fread("a,b\n1,2")
# 2: same but select just the second column
fread("a,b\n1,2", select = 2)
# 3: explicitly make no selection (should yield same result as 1)
fread("a,b\n1,2", select = NULL, verbose = T)

Input contains a \n (or is ""). Taking this to be text input (not a filename)
Detected eol as \n only (no \r afterwards), the UNIX and Mac standard.
Positioned on line 1 after skip or autostart
This line is the autostart and not blank so searching up for the last non-blank ... line 1
Detecting sep ... ','
Detected 2 columns. Longest stretch was from line 1 to line 2
Starting data input on line 1 (either column names or first row of data). First 10 characters: a,b
All the fields on line 1 are character fields. Treating as the column names.
Count of eol: 1 (including 0 at the end)
Count of sep: 1
nrow = MIN( nsep [1] / (ncol [2] -1), neol [1] - endblanks [0] ) = 1
Type codes (point 0): 11
Type codes: 11 (after applying colClasses and integer64)
Type codes: 11 (after applying drop or select (if supplied)
Allocating 2 column slots (2 - 0 dropped)
Read 1 rows. Exactly what was estimated and allocated up front
0.000s ( 0%) Memory map (rerun may be quicker)
0.000s ( 0%) sep and header detection
0.000s ( 0%) Count rows (wc -l)
0.000s ( 0%) Column type detection (100 rows at 10 points)
0.000s ( 0%) Allocation of 1x2 result (xMB) in RAM
0.000s ( 0%) Reading data
0.000s ( 0%) Allocation for type bumps (if any), including gc time if triggered
0.000s ( 0%) Coercing data already read in type bumps (if any)
0.000s ( 0%) Changing na.strings to NA
0.000s Total
Error in setcolorder(ans, reorder) :
neworder is length 0 but x has 2 columns.

Session info:

sessionInfo()

R version 3.3.2 (2016-10-31)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Red Hat Enterprise Linux Client 7.3 (Maipo)

locale:
[1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8 LC_MONETARY=en_GB.UTF-8
[6] LC_MESSAGES=en_GB.UTF-8 LC_PAPER=en_GB.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] data.table_1.10.0 HYPEtools_0.4-4 hydroGOF_0.3-8 zoo_1.7-14

loaded via a namespace (and not attached):
[1] Rcpp_0.12.9 lattice_0.20-34 intervals_0.15.1 FNN_1.1 reshape_0.8.6 spacetime_1.2-0 class_7.3-14 gstat_1.1-4
[9] grid_3.3.2 plyr_1.8.4 e1071_1.6-7 pbapply_1.3-1 sp_1.2-4 xts_0.9-7 tools_3.3.2 automap_1.0-14
[17] parallel_3.3.2 hydroTSM_0.4-2-1

Activity

  1. MichaelChirico commented on Jan 30, 2017

    @MichaelChirico
    MichaelChirico
    on Jan 30, 2017
    Member

    I guess this line should be changed:

    if (!missing(select)) {

    To

    if (!is.null(select)) {

    Otherwise no default value for select should be given.

  2. jangorecki
    added
    fread
    on Mar 5, 2017
  3. st-pasha
    added
    bug
    on Jul 6, 2017

  4. MichaelChirico commented on Sep 25, 2017

    @MichaelChirico
    MichaelChirico
    on Sep 25, 2017
    Member

    @dselivanov i was just going to sit down and crank out this PR, unless you've already started?

  5. dselivanov commented on Sep 25, 2017

    @dselivanov
    dselivanov
    on Sep 25, 2017 via email

  6. MichaelChirico commented on Sep 25, 2017

    @MichaelChirico
    MichaelChirico
    on Sep 25, 2017 via email
    Member

  7. dselivanov commented on Sep 25, 2017

    @dselivanov
    dselivanov
    on Sep 25, 2017 via email

  8. MichaelChirico commented on Sep 25, 2017

    @MichaelChirico
    MichaelChirico
    on Sep 25, 2017 via email
    Member

  9. dselivanov commented on Sep 26, 2017

    @dselivanov
    dselivanov
    on Sep 26, 2017 via email

  10. MichaelChirico commented on Sep 26, 2017

    @MichaelChirico
    MichaelChirico
    on Sep 26, 2017 via email
    Member

  11. MichaelChirico commented on Oct 17, 2017

    @MichaelChirico
    MichaelChirico
    on Oct 17, 2017
    Member

    The fix appears to have come through setcolorder (I think #592, fixed in PR #1792), which no longer errors as before (even though that line is still hit).

    The missing line should be replaced with is.null anyway.

    I also find it odd that select is sent as a parameter to fread, since select is not touched on the R side before the C call -- perhaps @st-pasha knows better, but I think that means that either 1) select is not actually used within C or 2) all the type-checking, etc. that's done with select is done within C?

  12. mattdowle
    added this to the milestone on Oct 19, 2017
  13. mattdowle
    closed this as completedin 39c137don Oct 19, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugfread

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions