Repository navigation
100 times poorer performance indexing scalar record array (Trac #1386) #1984
Description
Original ticket http://projects.scipy.org/numpy/ticket/1386 on 2010-01-31 by trac user oscar.bristol, assigned to unknown.
Assigning to a field in a scalar record array like this
B['a'] = A['a']
takes around 50 times as long as assigning like this:
B['a'][:] = A['a']
and 100 times as long as assigning to a size 1 record array.
I've attached a script (test.py) that compares scalars and size 1 arrays and the two different methods of indexing above. On my machine, the results are:
$ python test.py 100000
array, B['a'][:] = A['a'] 0.25 seconds
array, B['a'] = A['a'] 0.213 seconds
scalar, B['a'][:] = A['a'] 0.395 seconds
scalar, B['a'] = A['a'] 22.0 seconds
Activity
thouis commentedon Oct 19, 2012
Attachment added by trac user oscar.bristol on 2010-01-31: test.py
thouis commentedon Oct 19, 2012
Attachment added by trac user jpeel on 2011-01-12: 0001-BF-poor-performance-indexing-scalar-record-array.patch
thouis commentedon Oct 19, 2012
trac user oscar.bristol wrote on 2010-01-31
(Second attempt)
Assigning to a field in a scalar record array like this:
takes around 50 times as long as assigning like this:
and 100 times as long as assigning to a size 1 record array. I've attached a script (test.py) that compares scalars and size 1 arrays and the two different methods of indexing above. On my machine, the results are:
thouis commentedon Oct 19, 2012
trac user jpeel wrote on 2011-01-12
The problem was that the scalar void type's setfield method makes a copy of the input array even if it is already an array. I made a fast track for when the right hand side value is an array. The fast track uses the getfield method (which is used in the B['a'][:] scalar example above) to get a Numpy array for B['a']. I then just use PyArray_CopyObject to do the transfer. Now, this last method of assignment is right with the others in time (actually faster than scalar B['a'][:] because a slice is not done).
thouis commentedon Oct 19, 2012
Milestone changed to
NumPy 2.0by trac user jpeel on 2011-01-12BUG: fix numpy#1984 (generate correct version.py).
charris commentedon Feb 20, 2014
Now it's only 10x as fast ;) @seberg @juliantaylor Either of you interested in taking a look at this?