Repository navigation
BUG: Stuctured field dtype assignment can modify data not included in dtype #12686
Description
There are probably more examples for this, but I will include the following tests:
"""Tests subarray fields which contain sparse dtypes so that
not all memory is used by the dtype work. Such dtype's should
leave the underlying memory unchanged.
"""
dtype = np.dtype([('a', {'names':['aa', 'ab'], 'formats':['f', 'f'],
'offsets':[0, 4]}, (2, 3))])
sparse_dtype = np.dtype([('a', {'names':['ab'], 'formats':['f'],
'offsets':[4]}, (2, 3))])
def test_sparse_field_assignment():
arr = np.zeros(3, dtype)
sparse_arr = arr.view(sparse_dtype)
sparse_arr[...] = np.finfo(np.float32).max
# dtype is reduced when accessing the field, so shape is (3, 2, 3):
print(repr(arr["a"]["aa"]))
def test_sparse_field_assignment_fancy():
# Fancy assignment goes to the copyswap function for comlex types:
arr = np.zeros(3, dtype)
sparse_arr = arr.view(sparse_dtype)
sparse_arr[[0, 1, 2]] = np.finfo(np.float32).max
# dtype is reduced when accessing the field, so shape is (3, 2, 3):
print(repr(arr["a"]["aa"]))
test_sparse_field_assignment()
test_sparse_field_assignment_fancy()Results should all be 0, first one fails on master, the second one will be fixed in gh-12650
I doubt this is really much of a priority, but it seems definitely wrong, mostly creating an issue for discussionan to reference from the test.
Activity
seberg commentedon Mar 24, 2019
(moving from Matti's PR) @ahaldane since the structured field stuff is no in 1.16, that sounds like this issue may get much more relevant?
ahaldane commentedon Mar 24, 2019
This appears to be that longstanding bug in subarray assignment, which has existed forever.
It's come up in other issues that subarray assignment (and other operations) are quite broken except in the simplest cases. The subarray code needs a good rework.
See #9313, #10442, #11946
KevinKobi commentedon Apr 16, 2020
is there progress with this?
seberg commentedon Apr 16, 2020
No, nobody is working on it specifically right now that I know of. But at least we have tests that show the issues. Out of curiosity, I opened this bug while adding tests, do you have actual code running into this as an issue?
seberg commentedon Feb 13, 2021
Just to note, this is a different bug than the ones listed above and it will be closed by gh-18398.
seberg commentedon Feb 13, 2021
Going to add the test there, and close this before I forget it.