Structured array fields with str dtype have length 0, not 1

See this SO question for the original source.

I have reproduced this with current master, the original report was on 1.8.1:

>>> a = np.zeros((10,), dtype=[('x', int), ('y', str)])
>>> a
array([(0, ''), (0, ''), (0, ''), (0, ''), (0, ''), (0, ''), (0, ''),
       (0, ''), (0, ''), (0, '')],
      dtype=[('x', '<i4'), ('y', 'S')])
>>> a['y'][a['x'] == 0] = 'hello'
>>> a
array([(0, ''), (104, ''), (104, ''), (104, ''), (104, ''), (104, ''),
       (104, ''), (104, ''), (104, ''), (104, '')],
      dtype=[('x', '<i4'), ('y', 'S')])
>>> ord('h')
104

What seems to be going on is that, when using str as the field type, it is created with type S which apparently means S0, not S1. Assigning any length to the string field makes it work:

>>> a = np.zeros((10,), dtype=[('x', int), ('y', 'S1')])
>>> a['y'][a['x'] == 0] = 'hello'
>>> a
array([(0, 'h'), (0, 'h'), (0, 'h'), (0, 'h'), (0, 'h'), (0, 'h'),
       (0, 'h'), (0, 'h'), (0, 'h'), (0, 'h')],
      dtype=[('x', '<i4'), ('y', 'S1')])

This does not happen with regular arrays:

>>> a = np.zeros((5,), dtype=str)
>>> a
array(['', '', '', '', ''],
      dtype='|S1')

Probably also related to this:

>>> mdtype = dict(names=['x','y'],formats=['int64','a'])
>>> np.dtype(mdtype)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: data type not understood
>>> mdtype = dict(names=['x','y'],formats=['int64','a1'])
>>> np.dtype(mdtype)
dtype([('x', '<i8'), ('y', 'S1')])

maybe that is what you mean, note that:

In [1]: a = np.zeros((10,), dtype=[('x', int), ('y', str)])

In [2]: a['y']
Out[2]: 
array(['', '', '', '', '', '', '', '', '', ''], 
      dtype='|S1')

a lot of numpy assumes S0 does not exist.

Try assigning anything to a['y'], even though it reports it as |S1, values assigned to it bleed to the next int:

>>> a = np.zeros((10,), dtype=[('x', int), ('y', str)])
>>> a['y'][3] = 'h'
>>> a['y']
array(['', '', '', 'h', '', '', '', '', '', 'j'],
      dtype='|S1')
>>> a
array([(0, ''), (0, ''), (0, ''), (0, ''), (104, ''), (0, ''), (0, ''),
       (0, ''), (0, ''), (0, '')],
      dtype=[('x', '<i4'), ('y', 'S')])

Of course. Anyway, I think the actual thing is, S0 does not exist and must not exist. The type should be S1 in the first place, and S0 in a structured type must be an error.

Or in other words, subtypes must not have a size of 0. Now one could argue S0 is valid, so maybe we can fix those places that assume that no S0 or 0-sized dtype exist (no idea how much that might actually be).

Good, we agree! My original thought was that 'a', or str should always be converted to 'S1', which does not seem to be the case in structured arrays. Will take a look at what is going on

BUG: Fix numerous bugs related to zero-width string arrays (numpy#473, …

Fixed by #6430

Navigation Menu

Repository navigation

Structured array fields with str dtype have length 0, not 1 #4955

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Structured array fields with str dtype have length 0, not 1 #4955

Description

Activity

jaimefrio commented on Sep 10, 2014

seberg commented on Sep 10, 2014

jaimefrio commented on Sep 10, 2014

seberg commented on Sep 10, 2014

jaimefrio commented on Sep 10, 2014

ahaldane commented on Jun 2, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions