Repository navigation
DISCUSS: What should the default integer type/dtype be #24890
Description
Activity
Maybe I did not follow along closely enough with your proposal @seberg. When you proposed to move away from
long, which was mainly a problem on Windows, I naturally assumed that the goal was to get toint64everywhere by default.int64is always available, so if there's a blocker somewhere, I missed that.To make sure I understand the difference:
- On 64-bit platforms it doesn't matter, since
intpis an alias forin64, right? - On 32-bit platforms, it's still
int32, and the tradeoff is indexing performance vs. overflows?
64-bit Windows is by far the biggest issue, so both changes are capturing the bulk of the benefit. So maybe it doesn't matter that much? If we'd start from scratch I'd aim for
int64everywhere, but if it's a much larger implementation effort, it may not be worth it?- On 64-bit platforms it doesn't matter, since
Well, I am sure I always suggested, go to
intpto change it for windows. Not that I think it is a better default conceptually.But it removes a whole dimension of worry (along indexing, but also related to use returning
intparrays in places). And it fixes the biggest issues, which to me seem mostly about libraries written by linux devs that don't port well to windows (these probably don't care about 32bit, otherwise they would port to windows probably!). Maybe this is due to few users, but I have rarely seen anyone complain about the default on 32bit platforms, and in practice the code run on 32bit platforms is probably not the same as that on 64bit anymore.Now, clearly I didn't try that (or maybe I did a bit, but it is so long ago I don't remember). I suspect getting a 64bit default, but using a 32bit
intpin many places will create problems, so that most places in NumPy that currently useintpmight have to use 64bit (possibly including all of the indexing code as the main path, making 32bit just a potential fast-path).
That is a larger change. For the windows change we know we align it to a well-tested platform (where we don't we fix something that gets a lot of complaints), for changing 32bit platforms that is not the case making it a bigger unknown for me.Well, I am sure I always suggested, go to
intpto change it for windows. Not that I think it is a better default conceptually.Yeah, I just think I saw the change on Windows, thought "great idea, +1", and translated it in my head to
int64.I agree with what you wrote - and you've got the
intpchange pretty much ready to go. So let's close this issue and get your PR merged?it fixes the biggest issues, which to me seem mostly about libraries written by linux devs that don't port well to windows
definitely
but I have rarely seen anyone complain about the default on 32bit platforms
I think it's more a "fact of life, we'll just deal with it". I don't think many people care about performance, mostly about keeping the tests passing for correctness and to stem the flow of bug reports for niche platforms.
Looks like this is resolved now, intp it is for pragmatism’s sake.
This has been discussed many times before in various places e.g. gh-9464, gh-17640, and maybe most importantly gh-12332.
There are always some opinions voiced the default should always be int64. Because our indexing is specialized for int32, and we have quite a bit of functions that use
intpexplicitly, I had at some point decided thatintpwould be a far more pragmatic change:intp. 32bit platforms would get a performance regression in all indexing related things unless significant work gets into specializing for more types.Thus, I have for a long time always considered the path forward to change to intp and hoped that 2.0 would be a point where to do it (mentioned in all meetings/notes around that as: change the default integer on windows). That is implemented in gh-24224.
However, maybe the consensus actually goes into a different direction, since there is an overlapping discussion in gh-24794.