Repository navigation
fread reads in empty fields as logical NA #1159
Description
Activity
Why should it be read as character by default? Why not specify
colClasses = ..instead.Because an empty column does not fit in a logical type without loss of information. After reading it in, you can't know anymore if you had an all NA column in your csv or all empty.
colClassesis not a good solution imo, as it requires you knowing too much info before even reading the file, which I don't think should be necessary.I see. Then presence of any empty element should result in a character column?
fread("a,b\n1,\n2,NA") fread("a,b\n1,\n2,5")
In the last case, I guess you'd agree it makes sense to have it as integer column, even though (strictly speaking) that is also loss of information...?
I didn't consider that, but I think you're right and presence of any empty element should result in a character column by default (at least as long as
NA's are written as "NA" by default, which they are forwrite.csv).It feels like
na.stringsshould be used somehow to makefreadread in that empty element as either character or smth else in your examples, but I can't seem to be able to make that work (I've never usedna.stringsbefore tbh).Thanks, I get it now.
na.strings = ""would convert all""toNA. So the current functionality seems to implicitly assumena.strings = "".. (whereas the default value is"NA"). That might a way to look at this issue.- added this to the milestone
on Mar 6, 2016 I'm hoping PR #2652 resolves this one.
( Aside: An all empty column is read as type logical because that's the lowest type. The thinking behind that is type-bumping which always bumps upwards. Now in dev, an automatic reread happens to ensure absolutely no loss where for example '000' was read as '0L' after the bump should have been read as character. That used to be warning and now is automatic re-read.)- modified the milestones: ,
on Mar 3, 2018
The following results in very unpleasant behavior if you read and subsequently write a csv:
Second column is read in as a logical column, which means empty values get converted to NA. Those columns should instead be read in as character.