Repository navigation
Bug in ravel_multi_index for big indexes #7546
Description
I'm working with big sparse multidimensional arrays using scipy and numpy and I've found a bug in numpy.ravel_multi_index. Specifically, for big index values (greater than 2^32) sometimes the returned value is invalid or even negative (it should be always nonnegative using the default ‘raise’ mode). Probably due to casting integers to int32 in ravel_multi_index.
Sample code that reproduces the problem:
>>> numpy.ravel_multi_index(([0], [1], [3], [19], [2379], [2], [0]), (41, 7, 120, 36, 2706, 8, 6))
array([577726140], dtype=int64)
OK
>>> numpy.ravel_multi_index(([1], [3], [3], [19], [2379], [2], [0]), (41, 7, 120, 36, 2706, 8, 6))
array([1332804284], dtype=int64)
Invalid result, should be [5627771580]
>>> numpy.ravel_multi_index(([29], [5], [117], [2], [1284], [2], [1]), (41, 7, 120, 36, 2706, 8, 6))
array([-7294480627], dtype=int64)
Invalid result, should be [117259570957]
I know that in such a case the maximum index value is huge (1,6104E+11), but not that huge.. I get the same error even when explicitly specifying all integers as numpy.int64. Setting modes {‘raise’, ‘wrap’, ‘clip’} does not have any effect either.
Tested using numpy 1.9.2 (Python 3.4.3 64bit) on Windows, and numpy 1.11.0 (Python 3.4.3 64bit) on Ubuntu.
Activity
charris commentedon Apr 14, 2016
Hmm... I think this should not be too difficult to fix.
jaimefrio commentedon Apr 14, 2016
There's an
int svariable declared at the begining ofarr_ravel_multi_indexincompiled_base.c, see here, that should be anpy_intp s. We should probably also be checking for overflow, i.e. dimensions define too big an array, when usingsto fill up the strides, similarly to how @seberg did in #7463 usingnpy_mul_with_overflow_intp.charris commentedon Jul 10, 2016
Still broken as of july 10, 2016.
Merge pull request #8109 from skwbc/issue#7546
lagru commentedon Jan 30, 2020
This issue seems to have been fixed in #8109 and can be closed now.
mattip commentedon Jan 30, 2020
Thanks