np.prod returns 0 in case of integer overflow (Trac #1629)

Original ticket http://projects.scipy.org/numpy/ticket/1629 on 2010-10-10 by trac user kilian, assigned to unknown.

I am using np.prod to calculate the number of elements of a sparse matrix (np.prod(C.shape)) and I noticed the following behavior: In case the result is greater than 2**31, zero is returned. Instead, the result should be converted to int long int (or at least an exception should be raised).

>>> import numpy as np
>>> np.version.version
'1.4.0'
>>> np.prod([2**16,2**16])
0

@charris

@charris wrote on 2010-10-10

I don't see this for current svn on my 64 bit system:

In [3]: array([2**16,2**16]).prod()
Out[3]: 4294967296

In [4]: array([2**16,2**16]).prod(dtype=int64)
Out[4]: 4294967296

In [7]: array([2**16,2**16]).dtype
Out[7]: dtype('int64')

However, it could be that the default accumator on your system is int32? What platform are you on? In any case, you should specify the type of the result, numpy can't tell in advance how big the product is going to be and it could easily be out of range for anything but arbitrary precision integers.

@rgommers

@rgommers wrote on 2010-10-10

It does depend on whether the platform is 32/64-bit:

In [10]: np.prod([2**16,2**16])
Out[10]: 0

In [11]: np.prod([2**16,2**16]).dtype
Out[11]: dtype('int32')

But the documentation explains this issue in detail, quoting relevant parts:

    Parameters
    ----------
...
    dtype : data-type, optional
        The data-type of the returned array, as well as of the accumulator
        in which the elements are multiplied.  By default, if `a` is of
        integer type, `dtype` is the default platform integer. (Note: if
        the type of `a` is unsigned, then so is `dtype`.)  Otherwise,
        the dtype is the same as that of `a`.
...
    Notes
    -----
    Arithmetic is modular when using integer types, and no error is
    raised on overflow.  That means that, on a 32-bit platform:

    >>> x = np.array([536870910, 536870910, 536870910, 536870910])
    >>> np.prod(x) #random
    16

There probably was a good reason to implement and document it like this, but that's best asked on the mailing list. I think this ticket can be closed.

@pv

@pv wrote on 2010-10-10

The good reason is that the alternative is checking integer overflow manually, and that the output type of an ufunc should depend only on the input type, not on the input values.

@pv

Milestone changed to 1.5.1 by @pv on 2010-10-10

trac user kilian wrote on 2010-10-11

Ok, thanks for all the comments! I should have read the doc string more carefully first...

ENH: Add meson skeleton from numpygh-2225

ENH: ``meson`` backend for ``f2py`` (#24532)

ENH: ``meson`` backend for ``f2py`` (numpy#24532)

Merge pull request #24647 from charris/backport-24532

ENH: ``meson`` backend for ``f2py`` (numpy#24532)

Navigation Menu

Repository navigation

np.prod returns 0 in case of integer overflow (Trac #1629) #2225

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

np.prod returns 0 in case of integer overflow (Trac #1629) #2225

Description

Activity

numpy-gitbot commented on Oct 19, 2012

numpy-gitbot commented on Oct 19, 2012

numpy-gitbot commented on Oct 19, 2012

numpy-gitbot commented on Oct 19, 2012

numpy-gitbot commented on Oct 19, 2012

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions