Repository navigation
Backwards compatible mixin like NDArrayOperatorsMixin as a separate project #9016
Closed as not planned
Description
As a follow up on #8247, I would like to write another backwards compatible mixin class like NDArrayOperatorsMixin suitable for vendoring in third-party libraries such as xarray and dask (and maybe pandas) that want to support __array_ufunc__ but aren't yet willing to require NumPy v1.13.
My intended usage pattern is as a drop-in replacement:
try:
from numpy.lib.mixins import NDArrayOperatorsMixin
except ImportError:
from my_library.legacy_numpy_compat import NDArrayOperatorsMixin
class MyArray(NDArrayOperatorsMixin):
def __numpy_ufunc__(self, ufunc, method, *inputs, **kwargs):
...
For simplicity and ease of use (not to mention decoupling from the NumPy release cycle), I think this should be a separate repository (e.g., "legacy-ndarray-operators-mixin") in the numpy github organization. I'm happy to help put this together, but wanted to make sure this makes sense to others first.
Activity
njsmith commentedon Apr 29, 2017
Makes sense to me.
Can I nudge you to consider bumping
xarrays minimum required version to 1.13 though? :-)shoyer commentedon Apr 30, 2017
This might actually be a better idea. These days, upgrading numpy is easier than ever :). So after some contemplation, I'm no longer convinced this is essential.
It also occurs to me that many existing projects will just add a
__array_ufunc__method without adjusting their special operator overloads, and that's probably OK. (Even though many/most existing special method implementations are broken in various subtle ways, in practice that isn't a big deal.)@mrocklin would you find such a mixin useful for dask?
mrocklin commentedon Apr 30, 2017
What reusable logic would this mixin have? I think that the competition to this idea is just to rebuild this logic each time in each codebase. Hopefully this logic isn't that complex. I haven't yet looked at the full spec though so I don't know if this is overly naive.
My initial inclination (again without heavy research) would be to build the
__array_ufunc__method in one project first (likely xarray is my guess) and then use that as an example for other projects to copy. If we find that we're rebuilding almost exactly the same code then we would choose to coordinate on an external package. I think that you have a deeper understanding of__array_ufunc__than I do though.shoyer commentedon May 1, 2017
Mostly this is pretty routine: defining every special operator in terms of it's matching numpy ufunc: https://github.com/numpy/numpy/blob/master/numpy/lib/mixins.py
Sounds good. We'll probably defer for this now, then.
njsmith commentedon May 1, 2017
@mrocklin: it's mostly that the logic is a bit tiresome (defining all the binop special methods), has a few gotchas if you want to correctly interoperate with arbitrary third-party classes, and is exactly the same for every object that wants to quack like an ndarray, so it makes sense to have a standard implementation everyone shares. But the standard implementation that ships with numpy assumes that you have a recent numpy (both to have the mixin, and because that implementation assumes that it's being used with a numpy that supports
__array_ufunc__). So @shoyer has been wondering if we should make another version that does something slightly-wrong-but-not-too-wrong and that works with older numpys. Anyway it sounds like you have a plan but that's what the question was :-)mrocklin commentedon May 1, 2017
OK, if there is a great deal of boiler plate then yes, I'd probably appreciate having it around.
I guess my general answer is "I'm not able to answer this without better understanding the costs."
shoyer commentedon May 20, 2022
I never got around to this and at this point it isn't worth the trouble -- NumPy 1.13 was released almost 5 years ago!