Repository navigation
BUG: Segfault When __array_ufunc__ Called With Garbage #24023
Description
Describe the issue:
Thanks for the great work!
To try to detect bugs in a system which caches off numpy arrays, I have a test which calls every method returned by dir on a numpy array with many permutations of random data of various types. In the last few days CI has started segfaulting and I have been able to reproduce locally after upgrading to numpy=1.25.0 when calling array.__array_ufunc__(2.3, 1).
Admittedly calling every function with every permutation of random garbage data isn't real use case and happy to close this and just skip the __array_ufunc__ in this test (after which my test went back to passing). But it just started happening, reproduced pretty well and none of the other methods segfault under any inputs I could put together so I thought it might be interesting.
Reproduce the code example:
mikedh@luna:tests$ python -X faulthandler
Python 3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> a = np.empty(1)
>>> a.__array_ufunc__(2.3, 1)
Fatal Python error: Segmentation fault
Current thread 0x00007efca4ce1000 (most recent call first):
File "<stdin>", line 1 in <module>
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator (total: 13)
Segmentation fault (core dumped)Error message:
Fatal Python error: Segmentation fault
Current thread 0x00007efca4ce1000 (most recent call first):
File "<stdin>", line 1 in <module>
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator (total: 13)
Segmentation fault (core dumped)Runtime information:
I have seen this in Github actions, Docker, and my Parallels desktop Ubuntu 22.04 virtual.
My desktop VM:
In [7]: import sys, numpy; print(numpy.__version__); print(sys.version)
1.25.0
3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]
In [8]: print(numpy.show_runtime())
WARNING: `threadpoolctl` not found in system! Install it by `pip install threadpoolctl`. Once installed, try `np.show_runtime` again for more detailed build information
[{'numpy_version': '1.25.0',
'python': '3.11.0rc1 (main, Aug 12 2022, 10:02:14) [GCC 11.2.0]',
'uname': uname_result(system='Linux', node='luna', release='5.19.0-45-generic', version='#46~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Jun 7 15:06:04 UTC 20', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}}]
None
Context for the issue:
It doesn't effect my work, I already skipped this in tests, I thought the team might find inducable hard segfaults interesting.
Here is the "call every method with garbage" test that induced the segfault
Activity
skip numpy method for numpy/numpy#24023
mikedh commentedon Jun 23, 2023
There was one more segfault which didn't reproduce at all locally, but on every Github Actions run with
ubuntu-latestthe following produced a segfault:When trying to reproduce either locally, or locally in docker I get the expected
TypeError: astype() missing required argument 'dtype' (pos 0)but for some reason on the Github actions runners it segfaults. I would have thought it would reproduce with this but I couldn't get it to happen locally even on a bunch of base images:I'm skipping
astypeand__array_ufunc__and it's passing locally and in CI.seberg commentedon Jun 23, 2023
Thanks! I am not too worried about it since its not really user-facing, but the
__array_ufunc__should be easy to track down and fix. Theastype()is also a real bug, but harder to find because its uninitialized stack values.ngoldbaum commentedon Jun 23, 2023
In my debug build of python:
So there's probably a reference counting error in an error path somewhere.
ngoldbaum commentedon Jun 23, 2023
And it's in code I wrote 🙃. I'll have a patch shortly.
seberg commentedon Jun 23, 2023
I think might have been me, my think the initialization is just missing for the things we have been working on. I suspect the other issue is just not accounting for
kwargsbeingNULL.ngoldbaum commentedon Jun 23, 2023
Adding such a test to the numpy test suite seems worthwhile. Error paths have poor coverage in numpy's tests and this would make the tests hit more of them.
ngoldbaum commentedon Jun 23, 2023
Yup:
I'll leave that one to you since I'm not familiar with the
__array_ufunc__internals.pypi-trimesh: Autospec creation for update from version 3.22.1 to ver…
mikedh commentedon Jul 13, 2023
Awesome thanks for the quick fixes!!
__array_ufunc__andastypeare now passing for me on the1.25.1release, and I PR'd the fuzzing test in #24175 if the project thinks it'd be useful happy to contribute.