CPython 3.13 support (tracker issue)

This issue is to plan and keep track on the effort to add support for CPython 3.13 .
See PEP 719 for the release schedule.
At the time of opening, version 3.13.0b2 is available, and NumPy is already publishing nightly wheels for it.

Here's the chronological task list to complete before we can declare support for it.

I'll also list which of our core dependencies already have compatible and stable wheels on PyPI, but note that it isn't a necessary condition to make our own.

pyerfa (abi3)
numpy v2.1.0 (2024-08-18)
PyYAML 6.0.2 final release checklist for Python 3.13 support yaml/pyyaml#818

Supporting the experimental free-threading (a.k.a "nogil") build is a non-goal for this issue (although I note that it's currently blocked by liberfa/pyerfa#152)
Relevant PRs are to be linked in this message.

#16596 currently fixes all first 3 points. The last one needs to wait until 2024-07-30 (about a month from now) when 3.13.0rc1 is (hopefully) available.

Maybe a little too early for a PR? According to https://peps.python.org/pep-0719/#schedule , stable release not till Oct 2024. I think we can wait for RC.

It is definitely earlier than previous years (because numpy is getting better at catching up with Python's development), but my reasoning is that, as always, the cost of fixing problems increases with time; if we wait for rc1 then we renounce reporting a whole category of avoidable bugs to CPython folks (i.e. bugs that aren't fixable without breaking ABI). The chance that we encounter such bugs is admittedly very small, but I still think it's worth testing now, especially considering how simple the patch currently needs to be.

In other words, my opinion is that we should start testing as soon as our dependencies allow us to.

Another thing to keep in mind is that Python 3.13 will be the first version to come in two flavors: GIL edition and free-threading edition. The latter is an entirely new kind of beast in the Python world and I have no idea how long it'll take for us to build compatibility for it, so the sooner we can establish we're at least compatible with the GIL build, the better.

Also, anecdotally, last year we found a serious memory leak while testing yt on Python 3.12 betas, which turned out to be a bug in Cython, but we only found out like 2 weeks before 3.12.0 final was scheduled. It was much more stressful because of how short a window we had to even look for it, so I don't look forward to getting this kind of late surprises this year.

I think testing as soon as our required dependencies support a prerelease seems reasonable.

as soon as our required dependencies

Strictly speaking, that is only numpy and pyerfa. So you might not want to use all the devdeps?

wait I think devdeps is indeed restricted to pyerfa, numpy and scipy (which is now also making cp313 wheels btw) ?

Oh I get it. devdeps indeed includes matplotlib (not ready) by default but it's really controlled by the noscipy factor 🙃

Yes, welcome to Dependency Hell™️ !

FWIW https://labs.quansight.org/blog/free-threaded-python-rollout

Also FYI; I just opened liberfa/pyerfa#154, which will be needed if we plan to experiment/test against free-threaded Python sometimes this year.

For the item "publish cp313 wheels to PyPI" -- Is that for after Python 3.13 stable is out?

Not necessarily. The first release candidate is sufficient because it's guaranteed to be ABI stable. We should be able to provide cp313 wheels whenever our next release is.

Y'all probably know about this but here is a bit of useful info on how to adapt + compile the CPython extensions for the free-threaded python - https://py-free-threading.github.io/porting/#__tabbed_1_1

Effectively add the function PyUnstable_Module_SetGIL() to enable GIL on free-threaded python (though GIL enabled seems to be set as the default)

PyMODINIT_FUNC
PyInit__module(void)
{
    PyObject *mod = PyModule_Create(&module);
    if (mod == NULL) {
        return NULL;
    }

#ifdef Py_GIL_DISABLED
    PyUnstable_Module_SetGIL(mod, Py_MOD_GIL_USED); 
#endif
}

@manodeep

Thanks @manodeep ! Though as I wrote in the OP, freethreading is out of scope for this issue. I mean to open a separate tracker issue for it soon, but it's a pretty wide topic and we could probably get a couple easy wins now.
Was it the initial motivation for your work on WCS' thread safety ?

Yeah taking advantage of free-threading while not introducing data races will probably be tricky. Definitely needs a wider discussion (or waiting for a bit).

No the motivation for WCS' thread-safety came from the reproject memory race error (that turned out to be wcs thread-race error). (Which reminds me - I need to review Mark's latest patch for the thread-safe wcs code)

Re:free-threading: the dedicated issue is #16916

Navigation Menu

Repository navigation

CPython 3.13 support (tracker issue) #16595

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CPython 3.13 support (tracker issue) #16595

Description

Activity

neutrinoceros commented on Jun 21, 2024

pllim commented on Jun 21, 2024

neutrinoceros commented on Jun 22, 2024

neutrinoceros commented on Jun 22, 2024

neutrinoceros commented on Jun 22, 2024

astrofrog commented on Jun 22, 2024

pllim commented on Jun 24, 2024

neutrinoceros commented on Jun 25, 2024

neutrinoceros commented on Jun 25, 2024

pllim commented on Jun 25, 2024

pllim commented on Jul 16, 2024

neutrinoceros commented on Jul 16, 2024

pllim commented on Aug 20, 2024

neutrinoceros commented on Aug 20, 2024

manodeep commented on Aug 21, 2024

neutrinoceros commented on Aug 21, 2024

manodeep commented on Aug 21, 2024

neutrinoceros commented on Sep 2, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions