Repository navigation
BUG subclass with __numpy_ufunc__ segfaults with recarray comparison #4855
Description
If I define a ndarray subclass that implements __numpy_ufunc__, it segfaults whenever it contains a record array and a comparison is done (see below). Note that any direct calls to ufunc give the usual TypeError, and the segfault can be avoided by defining one's own __eq__, __ne__.
import numpy as np
class MyA(np.ndarray):
def __numpy_ufunc__(self, ufunc, method, i, inputs, **kwargs):
return getattr(ufunc, method)(*(input.view(np.ndarray)
for input in inputs), **kwargs)
a = np.arange(12.).reshape(4,3)
ma = a.view(MyA)
ma == ma[0]
# works as expected
ra = a.view(dtype=('f8,f8,f8')).squeeze()
ra == ra[0]
# works as expected: array([ True, False, False, False], dtype=bool)
mra = ra.view(MyA)
mra == ra[0]
# Segmentation fault
Activity
juliantaylor commentedon Jul 26, 2014
what should the result be here?
T, F, F, ForTypeError?mra < ra[0]raises TypeError whilera < ra[0]is true so I guess it should raiseTypeError?pv commentedon Jul 26, 2014
The expected behavior probably is
T, F, F, F.If we decide to raise a TypeError, the workaround is just to define
def __eq__(self, other): return self.view(np.ndarray) == other.The problem here is that ufuncs use a
NotImplementedreturn value as an indicator for some record array inputs (these are intended to be interpreted specially by binops):The
def __numpy_ufunc__above does not consider this (undocumented implementation detail) issue; and so it generates a TypeError.One might argue that ndarray.eq et al. should do the equivalent of
Similar things with record arrays are probably present also in other binops. There one probably gets TypeErrors anyway since there are no fallbacks such as here, so it is not a problem.
mhvk commentedon Jul 26, 2014
It mostly should not segfault! But I would say it should indeed return a masked boolean array (with in this case no mask set, and data equal to
T,F,F,F).I should add I'm still slightly surprised why defining a
__numpy_ufunc__that effectively just calls the normal ufunc would ever be a problem. Is there a "built-in" wrapper around the C-code ufunc that does deal withNotImplemented? (Or doesndarray.__eq__deal with it?)pv commentedon Jul 26, 2014
The problems occur because for record arrays, part of the logic that in principle should be in
ndarray.__eq__is insidenp.equal, and it is communicated back with the same NotImplemented special value as used in__numpy_ufunc__decision mechanism.mhvk commentedon Jan 19, 2015
Checking the 1.10 blockers that turned up by my
__numpy_ufunc__trials: in1.10.0.dev-256d256, the code at the top still leads to a segfault.mhvk commentedon Jun 11, 2015
@ahaldane - would your work on
recarrayalso solve these segmentation faults? (FWIW, it still happens with current master,1.10.0.dev0+83f36c5ahaldane commentedon Jun 11, 2015
I don't think so. The two PRs I made that fix segfaults (#5947 and #5636) aren't merged yet, but they have to do with getfield/setfield which aren't used here at first glance.
Btw, this issue involves "structured arrays" rather than recarrays (recarrays are a special subclass
np.recarrayyou have to specifically ask for).Also, the example code above doesn't segfault for me in 1.9.2 or in current master, in python2 and 3.
mhvk commentedon Jun 11, 2015
@ahaldane - OK, thanks; should have thought of the difference between structured and record arrays.
No surprise it doesn't segfault in 1.9.2, since that didn't have
__numpy_ufunc__, but are you sure that it works with the currentdevand with python3? I'm using linux, are you using a different OS?ahaldane commentedon Jun 11, 2015
Oh, I was mistaken. It does segfault in master, on linux. I must have pasted into the wrong terminal. My PRs don't fix it.
charris commentedon Jun 14, 2015
Fixed by #5964, so closing.
mhvk commentedon Jun 14, 2015
Great that this is solved; thanks, @njsmith, @charris.