Repository navigation
np.bool_ causes insidious behavior; some suggestions #7913
Description
Example:
In [1]: import numpy as np
In [2]: a = np.int64(1)
In [3]: (a > 0) is True
Out[3]: False
Obviously I understand the reason is because is is doing an object comparison and the result of (a > 0) is still type numpy.bool_. However, for scalar bools, this is really annoying bordering on mistake-inviting due to the fact that is and == are almost idiomatically interchangeable for bools in native Python (again, I understand the difference).
There are also odd promotion rules, to wit:
In [4]: b = np.bool_(True)
In [5]: type(b and True)
Out[5]: bool
In [6]: type(True and b)
Out[6]: numpy.bool_
In [7]: type(b or True)
Out[7]: numpy.bool_
In [8]: type(True or b)
Out[8]: bool
In [9]: type(not b)
Out[9]: bool
In [10]: type(~b)
Out[10]: numpy.bool_
It appears __and__ is doing some kind of inverted __or__. Once more, this is understandable, but it introduces many insidious bugs in existing code that already depends on the assumption that bools in Python work a certain way with operators.
Now, suggestion. One possibility, short of simply converting scalar numpy.bool_ to Python bool is at least for operators like __and__, __or__ whose other operand is a Python bool to return always a Python bool. That would at least isolate the "problem" of unexpected behavior to segments of the code where only numpy code is, and therefore visible to the writer, instead of a case where a library writer may return a numpy.bool_ and a library user may use only the return value and Python bools, neither knowing there is some hidden interaction. May I also suggest, so long as they are different, that __repr__ for numpy.bool_ return something like true and false rather than masquerading as internal True and False?
Activity
pv commentedon Aug 6, 2016
prints scalars using the same formatters as array elements
thanatos commentedon Jun 9, 2017
More insanity that my team hit with this today:
Took us a while to realize that the
reprwas just flat-out lying to us. (x, above, is anumpy.bool_:)
numpy.bool_should justrepritself correctly:'numpy.bool_(False)'or'numpy.bool_(True)'. It's not__str__: if I'mrepr'ing something, it is because I want to know what it is.charris commentedon Jun 9, 2017
Pickle works fine, what is ujson doing differently?
The problem isn't just
bool_njsmith commentedon Jun 9, 2017
ujson (and json serializers in general) work totally differently than pickle: they only support a few specific types (basically dict / list / str / int / float / bool / None), so they need to be able to look at an object and figure out which of these types it is. Unfortunately this means that there's basically no way for json libraries handle numpy scalars except by hard-coding numpy support into the json library. Python doesn't have a
standard "json protocol", so there's not much we can do on the numpy side.
All the suggestions raised at the top of this issue are unfortunately impossible (in particular, python doesn't allow classes to customize
andororbecause they're short-circuiting), so I'm going to close this issue.It's true that we could consider printing more accurate reprs. And it's especially weird for types with non-standard sizes (e.g., why does
repr(np.int8(1)) == repr(np.uint64(1)) == repr(1)?). But if someone wants to take that up (please do!) then it should be its own issue / mailing list discussion instead of tagging onto the bottom of this issue.