Skip to content

MAINT: do not use copyswap in flatiter internals - #24176

Merged
seberg merged 3 commits into
numpy:mainfrom
ngoldbaum:flatiter-copyswap
Jul 18, 2023
Merged

seberg merged 3 commits into
numpy:mainfrom
ngoldbaum:flatiter-copyswap

Conversation

@ngoldbaum

@ngoldbaum ngoldbaum commented Jul 13, 2023 •

Copy link
Copy Markdown
Member

Fixes #23486.

Updates all the code paths used by the flatiter getter and setter to use single-element casts instead of copyswap. This allows new dtypes to use these code paths without seg faulting.

I ran the flatiter benchmarks and didn't see any significant performance changes, but there seem to only be benchmarks for boolean indexing. I could probably substantially improve performance by only using copyswap in situations where it's required but using a hand-unrolled memcpy like fasttake otherwise but only want to fix the seg faults for now.

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits, mostly leave or take (or probably just leave because the below comment makes them not matter anymore).

One larger comment. It seems to me we can pass in the function into iter_ass_sub_int, iter_ass_sub_Bool, etc. Which means we can liberate getting the transfer function to before the if (PySlice_Check(ind)).
That will mean we only to the code once rather than 3 times and the cleanup is fully outside of those helpers.

The same should work for the iter_subscript* paths, with the added finish: label and goto finish.

Comment thread numpy/core/src/multiarray/iterators.c Outdated
Comment thread numpy/core/src/multiarray/iterators.c
Comment thread numpy/core/src/multiarray/iterators.c
Comment thread numpy/core/tests/test_custom_dtypes.py Outdated
Comment thread numpy/core/src/multiarray/iterators.c
@ngoldbaum
ngoldbaum force-pushed the flatiter-copyswap branch from 70d33bf to e2259b2 Compare July 17, 2023 20:12
@ngoldbaum

Copy link
Copy Markdown
Member Author

Thanks for the review @seberg! Simplified so there's only one cast setup that is passed to the boolean and integer indexing helpers. Let me know if you spot any additional issues.

@seberg seberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for that and even doing some of the smaller things. Cleanup now looks all good to me, so lets put it in.

@seberg
seberg merged commit 7dceee5 into numpy:main Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: ndarray.flat is incompatible with new-style dtypes

2 participants