Skip to content

Questionable behaviour in MaskedArray.__eq__ on flexible arrays #8589

Description

@eric-wieser

These lines, introduced in a5da87c by @pierregm, appear multiple times in numpy/ma/core.py, such as MaskedArray.__eq__:

try:
    mask = mask.view((bool_, len(self.dtype))).all(axis)
except ValueError:
    mask = np.all([[f[n].all() for n in mask.dtype.names]
      for f in mask], axis=axis)

But where is this ValueError expected to come from? The .view, or the .all, due to an invalid axis.

It seems like there is a bug in the code that prepares axis here, and this bug is being silenced by an overzealous try/catch.

Activity

  1. charris commented on Feb 9, 2017

    @charris
    Member

    @mhvk @ahaldane Thoughts.

  2. mhvk commented on Feb 9, 2017

    @mhvk
    Contributor

    In the commit pointed too, in another similar change, there is a comment: "# In case we have nested fields..."

    So, at least that gives a hint why this was introduced. Indeed, one of the test cases shows that it is at least one thing the ValueError is meant to catch:

    ndtype = [('A', int), ('B', [('BA', int), ('BB', int)])]
    a = np.ma.array([(1, (1, 1)), (2, (2, 2))], mask=[(0, (1, 0)), (0, (0, 1))], dtype=ndtype)
    a.mask
    # array([(False, ( True, False)), (False, (False,  True))], 
    #       dtype=[('A', '?'), ('B', [('BA', '?'), ('BB', '?')])])
    a.mask.view((np.bool_, len(a.dtype)))
    # ValueError: new type not compatible with array.
    

    In principle, I guess one should really write

    try:
        mask = mask.view((bool_, len(self.dtype)))
    except ValueError:
        mask = np.all(....)
    else:
        mask = mask.all(axis)
    

    But where does the indexing go wrong? I must say having a default of axis=1 seems very odd; surely, this should be axis=-1?!?!

  3. mhvk commented on Feb 9, 2017

    @mhvk
    Contributor

    Just checking now: my try/except/else indeed throws the same errors as found in #8584, and those disappear when I (correctly) set axis=-1. PR to come soon.

  4. eric-wieser commented on Feb 9, 2017

    @eric-wieser
    MemberAuthor

    Does this mean there is a bug with masked 2D record arrays?

  5. mhvk commented on Feb 9, 2017

    @mhvk
    Contributor

    Does this mean there is a bug with masked 2D record arrays?

    Yes. What a code... Anyway, fix almost done.

  6. mhvk commented on Feb 9, 2017

    @mhvk
    Contributor

    See #8590 for a fix.

  7. eric-wieser commented on Apr 4, 2017

    @eric-wieser
    MemberAuthor

    @mhvk: We can close this as of #8590, right?

  8. mhvk commented on Apr 5, 2017

    @mhvk
    Contributor

    Yes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions