Repository navigation
ctypeslib test_struct_array_pointer test failing with Python 3.12.0rc1 #24399
Description
Activity
Curious error.
There has been a fair amount of churn in ctypes in the past couple of weeks of Python 3.12.0 development, we may want to open an issue.
On first sight, the test does look right so I think we should indeed upstream as it seems like a likely Python bug. I don't have python 3.12 ready right now, but one remaining check might to reproduce it via
memoryview()(digging into ourctypeslib.as_array()):p_arr_type = ctypes.POINTER(_ctype_ndarray(obj._type_, shape)) obj = ctypes.cast(obj, p_arr_type).contents asarray(obj)compared to using
memoryview(obj)at the end there (and then maybe trying to figure out where the shape info got dropped)? @eric-wieser might actually be interested in it, since I think some of this may be based on his Python PRs years ago.This will block uploading of 3.12 wheels, so it needs either investigating or xfail-ing to deal with later.
+1 for xfailing for now, there is time before the actual 3.12 release and it would be good to get going with rc1 wheels.
Reacted by Ralf Gommers and Łukasz LangaThe tests is checking unsafe behaviour in
ctypes.cast: given a 2d ctypes array ofc_array = some_class([[1, 2, 3], [4, 5, 6]]), what happens when we castpointer(c_array[0])back totype(c_array)?The test is casting a data pointer to the same p_arr_type (the type of c_array), which will always be a 2x3 array.
_ctype_ndarray()is producsing the correct 2x3 ctypes type. The cast also works well, even when the rhs obj isc_array[0]thecastis a 3x2 object. But when taking thecontents, something happens and only the originalc_array[0]content is returned.Here is a smaller reproducer:
from ctypes import Structure, pointer, c_int16, cast, POINTER class Struct(Structure): _fields_ = [('a', c_int16)] Struct3 = 3 * Struct c_array = (2 * Struct3)( Struct3(Struct(a=1), Struct(a=2), Struct(a=3)), Struct3(Struct(a=4), Struct(a=5), Struct(a=6)) ) assert c_array[0][0].a == 1 p_obj = cast(pointer(c_array), POINTER(pointer(c_array)._type_)) obj = p_obj.contents # Works on 3.12, obj is 2d assert obj[0][0].a == 1 p_obj = cast(pointer(c_array[0]), POINTER(pointer(c_array)._type_)) obj = p_obj.contents # Fails on 3.12, obj is 2d assert obj[0][0].a == 1
Was this failing in the 3.12 alpha/beta releases? I think it may have to do with a PR that went in to rc1 just 2 weeks ago.
I don't think it was failing before. We don't have a regular 3.12-dev CI job, so it's hard to be sure. But my impression is that it happened recently indeed.
I guess this is now fixed, as Python team reverted the commit causing this issue, right?
This is showing up in all the Python 3.12 wheel build jobs now, see gh-24398 (that tests with an empty commit on top of main). I think it came in with the update to
cibuildwheel2.15.0 in gh-24372 two days ago; the wheel build jobs were not actually run on that PR.This will block uploading of 3.12 wheels, so it needs either investigating or xfail-ing to deal with later. Cc @charris for visibility.