Skip to content

np.vectorize fails with output dtype datetime64[ns] #25936

Open

Description

@gcaria

Describe the issue:

This came out from @mathause 's investigation in pydata/xarray#8802 (comment)

Reproduce the code example:

import numpy as np
otype = "datetime64[ns]"
arr = np.array(['2024-01-01', '2024-01-02', '2024-01-03'], dtype='datetime64[ns]')
np.vectorize(lambda x: x, signature="(i)->(j)", otypes=[otype])(arr)

Error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 4
      2 otype = "datetime64[ns]"
      3 arr = np.array(['2024-01-01', '2024-01-02', '2024-01-03'], dtype='datetime64[ns]')
----> 4 np.vectorize(lambda x: x, signature="(i)->(j)", otypes=[otype])(arr)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2372, in vectorize.__call__(self, *args, **kwargs)
   2369     self._init_stage_2(*args, **kwargs)
   2370     return self
-> 2372 return self._call_as_normal(*args, **kwargs)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2365, in vectorize._call_as_normal(self, *args, **kwargs)
   2362     vargs = [args[_i] for _i in inds]
   2363     vargs.extend([kwargs[_n] for _n in names])
-> 2365 return self._vectorize_call(func=func, args=vargs)

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2446, in vectorize._vectorize_call(self, func, args)
   2444 """Vectorized call to `func` over positional `args`."""
   2445 if self.signature is not None:
-> 2446     res = self._vectorize_call_with_signature(func, args)
   2447 elif not args:
   2448     res = func()

File ~/envs/test-xarray/lib/python3.10/site-packages/numpy/lib/function_base.py:2506, in vectorize._vectorize_call_with_signature(self, func, args)
   2502         outputs = _create_arrays(broadcast_shape, dim_sizes,
   2503                                  output_core_dims, otypes, results)
   2505     for output, result in zip(outputs, results):
-> 2506         output[index] = result
   2508 if outputs is None:
   2509     # did not call the function even once
   2510     if otypes is None:

ValueError: Cannot convert from specific units to generic units in NumPy datetimes or timedeltas

Python and NumPy Versions:

1.26.4
3.10.1 (main, Dec 15 2021, 17:45:54) [GCC 9.3.0]

Runtime Environment:

[{'numpy_version': '1.26.4',
  'python': '3.10.1 (main, Dec 15 2021, 17:45:54) [GCC 9.3.0]',
  'uname': uname_result(system='Linux', node='ip-172-34-24-37', release='5.15.0-1051-aws', version='#56~20.04.1-Ubuntu SMP Tue Nov 28 15:43:31 UTC 2023', machine='x86_64')},
 {'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
                      'found': ['SSSE3',
                                'SSE41',
                                'POPCNT',
                                'SSE42',
                                'AVX',
                                'F16C',
                                'FMA3',
                                'AVX2',
                                'AVX512F',
                                'AVX512CD',
                                'AVX512_SKX'],
                      'not_found': ['AVX512_KNL',
                                    'AVX512_KNM',
                                    'AVX512_CLX',
                                    'AVX512_CNL',
                                    'AVX512_ICL']}},
 {'architecture': 'SkylakeX',
  'filepath': '/home/ubuntu/envs/test-xarray/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so',
  'internal_api': 'openblas',
  'num_threads': 4,
  'prefix': 'libopenblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.23.dev'}]
None

Context for the issue:

No response

Activity

  1. mathause commented on Mar 5, 2024

    @mathause

    I think this happens because numpy creates the target array with np.dtype("datetime64[ns]").char, i.e. dtype('<M8') and not dtype('<M8[ms]')

    otypes = ''.join([_nx.dtype(x).char for x in otypes])

  2. ngoldbaum commented on Mar 5, 2024

    @ngoldbaum
    Member

    I wonder what breaks if you drop the char there. Looking at the Vectorize class, it seems to only pass otypes entries as values for dtype keywords, so dtype instances should be fine.

  3. francisco-sousa1 commented on Mar 21, 2024

    @francisco-sousa1
    Contributor

    I will work on this.

  4. added a commit that references this issue on Apr 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions