Repository navigation
Possible regression with numpy 1.14.0 and recfromcsv #10394
Description
Hello,
I couldn't find this on a quick glance, but there may be a small regression on numpy 1.14.0 with the recfromcsv function.
Numpy 1.13.0. I'm running on python 2.7.... (not sure if same problem exists on python 3.)
import numpy as np
from StringIO import StringIO
csv = """
id,low,age,lwt,race,smoke,ptl,ht,ui,ftv,bwt
85,0,19,182,"black",0,0,0,1,0,2523
86,0,33,155,"other",0,0,0,0,3,2551
87,0,20,105,"white",1,0,0,0,1,2557
"""
np.recfromcsv(StringIO(csv), converters={4: lambda x: x.strip('\"')})Gives this:
rec.array([(85, 0, 19, 182, 'black', 0, 0, 0, 1, 0, 2523),
(86, 0, 33, 155, 'other', 0, 0, 0, 0, 3, 2551),
(87, 0, 20, 105, 'white', 1, 0, 0, 0, 1, 2557)],
dtype=[('id', '<i8'), ('low', '<i8'), ('age', '<i8'), ('lwt', '<i8'), ('race', 'S5'), ('smoke', '<i8'), ('ptl', '<i8'), ('ht', '<i8'), ('ui', '<i8'), ('ftv', '<i8'), ('bwt', '<i8')])When you upgrade to 1.14.0, notice the race column becomes blank.
rec.array([(85, 0, 19, 182, '', 0, 0, 0, 1, 0, 2523),
(86, 0, 33, 155, '', 0, 0, 0, 0, 3, 2551),
(87, 0, 20, 105, '', 1, 0, 0, 0, 1, 2557)],
dtype=[('id', '<i8'), ('low', '<i8'), ('age', '<i8'), ('lwt', '<i8'), ('race', 'S'), ('smoke', '<i8'), ('ptl', '<i8'), ('ht', '<i8'), ('ui', '<i8'), ('ftv', '<i8'), ('bwt', '<i8')])xref statsmodels/statsmodels#4205
Thanks.
Activity
eric-wieser commentedon Jan 14, 2018
On python 3, this fails with
TypeError: a bytes-like object is required, not 'str'Why do
converterstake bytes as an input, even if the source is unicode? (related to #10054 )eric-wieser commentedon Jan 14, 2018
A simpler repro:
eric-wieser commentedon Jan 14, 2018
Bug is here: https://github.com/numpy/numpy/pull/10054/files#r161386985
BUG: Resize bytes_ columns in genfromtxt