Repository navigation
ctypeslib test_struct_array_pointer test failing with Python 3.12.0rc1 #24399
Copy link
Copy link
Closed
Milestone
Description
This is showing up in all the Python 3.12 wheel build jobs now, see gh-24398 (that tests with an empty commit on top of main). I think it came in with the update to cibuildwheel 2.15.0 in gh-24372 two days ago; the wheel build jobs were not actually run on that PR.
____________________ TestAsArray.test_struct_array_pointer _____________________
self = <numpy.tests.test_ctypeslib.TestAsArray object at 0x117f99a90>
def test_struct_array_pointer(self):
from ctypes import c_int16, Structure, pointer
class Struct(Structure):
_fields_ = [('a', c_int16)]
Struct3 = 3 * Struct
c_array = (2 * Struct3)(
Struct3(Struct(a=1), Struct(a=2), Struct(a=3)),
Struct3(Struct(a=4), Struct(a=5), Struct(a=6))
)
expected = np.array([
[(1,), (2,), (3,)],
[(4,), (5,), (6,)],
], dtype=[('a', np.int16)])
def check(x):
assert_equal(x.dtype, expected.dtype)
assert_equal(x, expected)
# all of these should be equivalent
check(as_array(c_array))
check(as_array(pointer(c_array), shape=()))
> check(as_array(pointer(c_array[0]), shape=(2,)))
Struct = <class 'numpy.tests.test_ctypeslib.TestAsArray.test_struct_array_pointer.<locals>.Struct'>
Struct3 = <class 'numpy.tests.test_ctypeslib.Struct_Array_3'>
Structure = <class '_ctypes.Structure'>
c_array = <Struct_Array_3_Array_2 object at 0x121ddd9d0>
c_int16 = <class 'ctypes.c_short'>
check = <function TestAsArray.test_struct_array_pointer.<locals>.check at 0x10239f100>
expected = array([[(1,), (2,), (3,)],
[(4,), (5,), (6,)]], dtype=[('a', '<i2')])
pointer = <built-in function pointer>
self = <numpy.tests.test_ctypeslib.TestAsArray object at 0x117f99a90>
../venv-test/lib/python3.12/site-packages/numpy/tests/test_ctypeslib.py:241:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../venv-test/lib/python3.12/site-packages/numpy/tests/test_ctypeslib.py:236: in check
assert_equal(x, expected)
expected = array([[(1,), (2,), (3,)],
[(4,), (5,), (6,)]], dtype=[('a', '<i2')])
x = array([(1,), (2,), (3,)], dtype=[('a', '<i2')])
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (<built-in function eq>, array([(1,), (2,), (3,)], dtype=[('a', '<i2')]), array([[(1,), (2,), (3,)],
[(4,), (5,), (6,)]], dtype=[('a', '<i2')]))
kwds = {'err_msg': '', 'header': 'Arrays are not equal', 'strict': False, 'verbose': True}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E AssertionError:
E Arrays are not equal
E
E (shapes (3,), (2, 3) mismatch)
E x: array([(1,), (2,), (3,)], dtype=[('a', '<i2')])
E y: array([[(1,), (2,), (3,)],
E [(4,), (5,), (6,)]], dtype=[('a', '<i2')])
args = (<built-in function eq>, array([(1,), (2,), (3,)], dtype=[('a', '<i2')]), array([[(1,), (2,), (3,)],
[(4,), (5,), (6,)]], dtype=[('a', '<i2')]))
func = <function assert_array_compare at 0x10dba2de0>
kwds = {'err_msg': '', 'header': 'Arrays are not equal', 'strict': False, 'verbose': True}
self = <contextlib._GeneratorContextManager object at 0x10d9b9af0>
This will block uploading of 3.12 wheels, so it needs either investigating or xfail-ing to deal with later. Cc @charris for visibility.
Reactions are currently unavailable
Activity
charris commentedon Aug 11, 2023
Curious error.
There has been a fair amount of churn in ctypes in the past couple of weeks of Python 3.12.0 development, we may want to open an issue.
seberg commentedon Aug 12, 2023
On first sight, the test does look right so I think we should indeed upstream as it seems like a likely Python bug. I don't have python 3.12 ready right now, but one remaining check might to reproduce it via
memoryview()(digging into ourctypeslib.as_array()):compared to using
memoryview(obj)at the end there (and then maybe trying to figure out where the shape info got dropped)? @eric-wieser might actually be interested in it, since I think some of this may be based on his Python PRs years ago.mattip commentedon Aug 13, 2023
+1 for xfailing for now, there is time before the actual 3.12 release and it would be good to get going with rc1 wheels.
mattip commentedon Aug 14, 2023
The tests is checking unsafe behaviour in
ctypes.cast: given a 2d ctypes array ofc_array = some_class([[1, 2, 3], [4, 5, 6]]), what happens when we castpointer(c_array[0])back totype(c_array)?The test is casting a data pointer to the same p_arr_type (the type of c_array), which will always be a 2x3 array.
_ctype_ndarray()is producsing the correct 2x3 ctypes type. The cast also works well, even when the rhs obj isc_array[0]thecastis a 3x2 object. But when taking thecontents, something happens and only the originalc_array[0]content is returned.mattip commentedon Aug 14, 2023
Here is a smaller reproducer:
mattip commentedon Aug 14, 2023
Was this failing in the 3.12 alpha/beta releases? I think it may have to do with a PR that went in to rc1 just 2 weeks ago.
rgommers commentedon Aug 14, 2023
I don't think it was failing before. We don't have a regular 3.12-dev CI job, so it's hard to be sure. But my impression is that it happened recently indeed.
ZupoLlask commentedon Sep 16, 2023
I guess this is now fixed, as Python team reverted the commit causing this issue, right?