ENH: pass copy=True to __array__ #26208

Activity

1. ngoldbaum

   added

   01 - Enhancement

   on Apr 3, 2024
2. mtsokol

   self-assigned this

   on Apr 4, 2024
3. mtsokol commented on Apr 4, 2024

   

   mtsokol

   on Apr 4, 2024

   Member

   More actions

   Hi @ngoldbaum @seberg,

   I followed np.array implementation and, if I understood it correctly, we need to prevent numpy from making a copy after __array__ is called rather than before.

   def __array__() (with copy argument) is only called deep inside PyArray_DiscoverDTypeAndShape which is meant to discover ndims. I think the result of __array__ is stored in a cache passed to that function (first copy, 1. in the photo), but then this cache is passed in ctors.c::1622:

   arr = (PyArrayObject *)(cache->arr_or_sequence);
   /* we may need to cast or assert flags (e.g. copy) */
   PyObject *res = PyArray_FromArray(arr, dtype, flags);

   where flags still has "copy always" (second copy?, 2. in the photo).

   My question: Would it be sufficient to avoid the second copy by changing "copy always" flag to "copy if needed" in flags, as the cache that was provisioned in Py_Array_Discover... already holds a copy?

   
4. seberg commented on Apr 4, 2024

   

   seberg

   on Apr 4, 2024

   Member

   More actions

   Well, there are two things to note I guess:

   • When the discovery finds an array immediately, we may or may not already get a copy if we pass it on with copy=True. But that of course only works for __array__ and none of the others, so would need to add the copy logic also for others or indicate that a copy was made.
   • When the discovery finds a sequence (of arrays), a copy has to be made in any case and it doesn't make sense to keep passing on copy=True to __array__. (also we must fail for copy=False, but I think we do)
5. mtsokol commented on Apr 4, 2024

   

   mtsokol

   on Apr 4, 2024

   Member

   More actions

   Then I can try with copy was made indicator that is set by Py_Array_Discover..., that would remove that flag for further processing.
6. mtsokol

   linked a pull request that will close this issueAPI: Enforce one copy for ``__array__`` when ``copy=True`` #26215on Apr 5, 2024
7. mtsokol

   changed the title [-]ENH: pass copy=True to __array__[/-] [+]ENH: pass `copy=True` to `__array__`[/+] on Apr 5, 2024
8. pitrou

   added a commit that references this issue on Apr 15, 2024

   GH-41098: [Python] Add copy keyword in Array.__array__ for numpy 2.0+…

   ...

   80258fe
9. raulcd

   added a commit that references this issue on Apr 15, 2024

   GH-41098: [Python] Add copy keyword in Array.__array__ for numpy 2.0+…

   ...

   0d3af8d
10. mtsokol

    closed this as completedin #26215on Apr 19, 2024
11. vibhatha

    added a commit that references this issue on May 25, 2024

    apacheGH-41098: [Python] Add copy keyword in Array.__array__ for nump…

    ...

    28e281b