BUG: change in behavior of assert_equal in numpy '1.13.0.dev0+c25db39'

This change in behavior resulted in failure of a scipy test.

numpy 1.12.1

In [3]: from numpy.testing import assert_equal

In [4]: assert_equal(np.complex128(-0j), 0)

The assertion succeeds.

numpy '1.13.0.dev0+c25db39'

In [20]: from numpy.testing import assert_equal

In [21]: assert_equal(np.complex128(-0j), 0)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_equal(actual, desired, err_msg, verbose)
    366         try:
--> 367             assert_equal(actualr, desiredr)
    368             assert_equal(actuali, desiredi)

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_equal(actual, desired, err_msg, verbose)
    392             if not signbit(desired) == signbit(actual):
--> 393                 raise AssertionError(msg)
    394     # If TypeError or ValueError raised while using isnan and co, just handle

AssertionError: 
Items are not equal:
 ACTUAL: -0.0
 DESIRED: 0

During handling of the above exception, another exception occurred:

AssertionError                            Traceback (most recent call last)
<ipython-input-21-8adfca176832> in <module>()
----> 1 assert_equal(np.complex128(-0j), 0)

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_equal(actual, desired, err_msg, verbose)
    368             assert_equal(actuali, desiredi)
    369         except AssertionError:
--> 370             raise AssertionError(msg)
    371 
    372     # Inf/nan/negative zero handling

AssertionError: 
Items are not equal:
 ACTUAL: (-0-0j)
 DESIRED: 0

Since it emerged from my PR, I'll take a look into this. Given what you showed earlier, I wonder if we want to revisit the decision with this commit here in which this comparison fails:

assert_equal(-0.0, 0)

But this one does not?

assert_equal(np.array(-0.0), 0)

Obviously, the quickest solution is to revert my original PR, but I personally find this inconsistency confusing. Perhaps someone who has worked with this part of the code more can explain? In particular, why do we care about the sign-bit for scalars but not for arrays (which is why we have this inconsistency)?

Also, xref scipy/scipy#7227 (to complete the circle of cross-referencing)

Hmm, before doing any work, it seems we need to decide whether we consider -0 and +0 equal or not. If we think they are different, then the previous behaviour was in fact a bug that is now solved.

it seems we need to decide whether we consider -0 and +0 equal or not

Ideally assert_(x == 0) and assert_equal(x, 0) should give the same result.

@person142

@person142 - assert_equal is much stricter than ==. For instance, for ndarray, it is equivalent to assert_array_equal which insists that the shapes of the arrays match (i.e., no broadcasting is done). I must admit I find this somewhat confusing but in a testing environment it does make sense to have the possibility to check complete equality. After all, if one does not care about that, one can just do assert_(np.all(x==y)).

@mhvk

@mhvk wrote:

we need to decide whether we consider -0 and +0 equal or not.

As far as assert_equal goes when applied to scalars, that decision was made years ago: assert_equal considers them equal. And all the complex permutations of signed zeros (0+0j, 0-0j, -0+0j and -0-0j) are also equal to 0. The system python on my laptop has numpy 1.6.2, and the assertion assert_equal(np.complex128(-0j), 0) passes, as it does in numpy 1.12.1.

There is definitely a use-case for a "fussy" comparison to zero, in which the sign matters. That is exactly what the scipy test that triggered this issue does. After the initial "sloppy" comparison of the computed value to 0, it checks the sign bit of the real and imaginary parts of the value. It would be nice to have a fussy comparison to 0 included in a numpy test function, but changing the default behavior of assert_equal is too drastic.

Arghhh.... sorry, I have to backtrack already. As noted above, assert_equal(-0.0, 0) fails, as does assert_equal(np.float64(-0.0), 0), and that behavior also goes back as far as numpy 1.6.2. So it is messier than I thought.

@mhvk

@mhvk wrote

assert_equal is much stricter than ==. For instance, for ndarray, it is equivalent to assert_array_equal which insists that the shapes of the arrays match (i.e., no broadcasting is done).

That's true if both arguments are, in fact, ndarrays. If one is a scalar, it does broadcast:

In [11]: np.__version__
Out[11]: '1.13.0.dev0+c25db39'

In [12]: x = np.array([10, 10, 10])

In [13]: assert_equal(x, 10)

In [14]: assert_equal(x, np.int_(10))

In [15]: assert_equal(x, np.array(10))

All those assertions pass.

When the second value is actually an array with a nontrivial shape, the assertion fails:

In [16]: assert_equal(x, np.array([10]))
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-16-d2f3dd205067> in <module>()
----> 1 assert_equal(x, np.array([10]))

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_equal(actual, desired, err_msg, verbose)
    340     from numpy.lib import iscomplexobj, real, imag
    341     if isinstance(actual, ndarray) or isinstance(desired, ndarray):
--> 342         return assert_array_equal(actual, desired, err_msg, verbose)
    343     msg = build_err_msg([actual, desired], err_msg, verbose=verbose)
    344 

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_array_equal(x, y, err_msg, verbose)
    850     __tracebackhide__ = True  # Hide traceback for py.test
    851     assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
--> 852                          verbose=verbose, header='Arrays are not equal')
    853 
    854 

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+c25db39-py3.5-macosx-10.6-x86_64.egg/numpy/testing/utils.py in assert_array_compare(comparison, x, y, err_msg, verbose, header, precision, equal_nan, equal_inf)
    723                                 verbose=verbose, header=header,
    724                                 names=('x', 'y'), precision=precision)
--> 725             raise AssertionError(msg)
    726 
    727         if isnumber(x) and isnumber(y):

AssertionError: 
Arrays are not equal

(shapes (3,), (1,) mismatch)
 x: array([10, 10, 10])
 y: array([10])

That's off-topic for this issue, but sometimes I can't help being pedantic. 😄

Just an update: the scipy issue has been successfully closed, but the question of how to handle this inconsistency on the numpy end remains...

I think the basic issue is that signed zeros are treated differently in assert_equal depending on whether the comparison is between scalars or an array in involved, scalars being the pedantic case. For numpy 1.12 and earlier complex numbers were an exception to that rule, I assume because they went through the ufunc. From that point of view the numpy behavior in 1.13 is a bugfix, although the larger issue could be labeled a long standing bug in assert_equal. I'm inclined to keep the current behavior pending a decision on the larger issue. If I had a preference, we would fix the array case, but that would no doubt bring a lot of error reports. Maybe the thing to do is create assert_pedantically_equal or some such.

We really need to take a look at creating some new test functions that have better defined behavior.

Pushing this off to 1.14. If problems are reported during the release process we can deal with them then.

Is this still a problem, or should we just accept current behavior? Pushing off to 1.17 as time is short for 1.16.

Since this has now been pushed off for 3 releases, my tendency would be to close this. I think the new behaviour is slightly better, although it remains true that perhaps we should look at all our functions more carefully.

I agree, current behavior is fine (a little better), and no point in reverting after 3 releases anymore

Navigation Menu

Repository navigation

BUG: change in behavior of `assert_equal` in numpy '1.13.0.dev0+c25db39' #8848

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BUG: change in behavior of assert_equal in numpy '1.13.0.dev0+c25db39' #8848

Description

Activity

gfyoung commented on Mar 27, 2017

mhvk commented on Mar 27, 2017

person142 commented on Mar 27, 2017

mhvk commented on Mar 27, 2017

WarrenWeckesser commented on Mar 27, 2017

WarrenWeckesser commented on Mar 27, 2017

WarrenWeckesser commented on Mar 27, 2017

gfyoung commented on Mar 27, 2017

charris commented on May 7, 2017

charris commented on May 8, 2017

charris commented on Dec 4, 2018

mhvk commented on Dec 4, 2018

rgommers commented on Dec 5, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

BUG: change in behavior of `assert_equal` in numpy '1.13.0.dev0+c25db39' #8848