Questionable behaviour in MaskedArray.__eq__ on flexible arrays

@pierregm

These lines, introduced in a5da87c by @pierregm, appear multiple times in numpy/ma/core.py, such as MaskedArray.__eq__:

try:
    mask = mask.view((bool_, len(self.dtype))).all(axis)
except ValueError:
    mask = np.all([[f[n].all() for n in mask.dtype.names]
      for f in mask], axis=axis)

But where is this ValueError expected to come from? The .view, or the .all, due to an invalid axis.

It seems like there is a bug in the code that prepares axis here, and this bug is being silenced by an overzealous try/catch.

@mhvk

@mhvk @ahaldane Thoughts.

In the commit pointed too, in another similar change, there is a comment: "# In case we have nested fields..."

So, at least that gives a hint why this was introduced. Indeed, one of the test cases shows that it is at least one thing the ValueError is meant to catch:

ndtype = [('A', int), ('B', [('BA', int), ('BB', int)])]
a = np.ma.array([(1, (1, 1)), (2, (2, 2))], mask=[(0, (1, 0)), (0, (0, 1))], dtype=ndtype)
a.mask
# array([(False, ( True, False)), (False, (False,  True))], 
#       dtype=[('A', '?'), ('B', [('BA', '?'), ('BB', '?')])])
a.mask.view((np.bool_, len(a.dtype)))
# ValueError: new type not compatible with array.

In principle, I guess one should really write

try:
    mask = mask.view((bool_, len(self.dtype)))
except ValueError:
    mask = np.all(....)
else:
    mask = mask.all(axis)

But where does the indexing go wrong? I must say having a default of axis=1 seems very odd; surely, this should be axis=-1?!?!

Just checking now: my try/except/else indeed throws the same errors as found in #8584, and those disappear when I (correctly) set axis=-1. PR to come soon.

Does this mean there is a bug with masked 2D record arrays?

Does this mean there is a bug with masked 2D record arrays?

Yes. What a code... Anyway, fix almost done.

See #8590 for a fix.

@mhvk

@mhvk: We can close this as of #8590, right?

Yes!

Navigation Menu

Repository navigation

Questionable behaviour in MaskedArray.eq on flexible arrays #8589

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Questionable behaviour in MaskedArray.__eq__ on flexible arrays #8589

Description

Activity

charris commented on Feb 9, 2017

mhvk commented on Feb 9, 2017

mhvk commented on Feb 9, 2017

eric-wieser commented on Feb 9, 2017

mhvk commented on Feb 9, 2017

mhvk commented on Feb 9, 2017

eric-wieser commented on Apr 4, 2017

mhvk commented on Apr 5, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Questionable behaviour in MaskedArray.eq on flexible arrays #8589