[BUG]: Python atexit broken with pybind11 version >= 2.9.2

Required prerequisites

Make sure you've read the documentation. Your issue may be addressed there.
Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

2.10.2

Problem description

We want to upgrade from pybind11 release 2.9.0 to 2.10.2, but we encountered an issue in the Python atexit (https://docs.python.org/3/library/atexit.html) functions, that our custom types are not registered anymore. With delta debugging we found that this PR caused the issue: #3744

In my opinion the

    detail::get_local_internals().registered_types_cpp.clear();    
    detail::get_local_internals().registered_exception_translators.clear();

calls should be after the Py_Finalize function call.

When I change the orders of those function calls, the problem is gone.

Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

2.9.1

@rwgk

FYI @rwgk

@chrreisinger

@chrreisinger could you help by

creating a PR that moves the .clear() calls after the Py_Finalize() call,
and ideally adds a unit test for your situation (in tests/test_embed/test_interpreter.cpp)?

The change makes sense to me. My understanding is: the Py_Finalize() invalidates all PyObject* pointers in the local_internals, but looking at the struct local_internals code, it seems to me the .clear() calls don't ever use those pointers. There are a bunch of non-trivial destructors that will run, but none of them will call into the Python C API.

I will give it a try on the weekend.

I will give it a try on the weekend.

Awesome.

If you create a PR with just the lines moved, that'll be a great start. It will be good to confirm that valgrind does not find any issues. We can work on the test together from there.

@chrreisinger

@chrreisinger Any updates on this?

Keep registered types until after Py_Finalize(). Fix pybind#4459

bugfix: Keep registered types until after Py_Finalize(). Fix #4459 (#…

Revert "bugfix: Keep registered types until after Py_Finalize(). Fix p…

Oh boy, this broke some more things.

Fix pybind#4459 (cpp atexit callbacks) without segfault

🤖 AI text below 🤖

The finalization order was rewritten by #5564. embed.h now calls Py_Finalize() before it destroys the internals, which is what this issue asked for. Python atexit handlers therefore run while the bindings are still alive.

If you can still reproduce this with current master (v3), please comment and we can reopen.

Navigation Menu

Repository navigation

[BUG]: Python atexit broken with pybind11 version >= 2.9.2 #4459

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

Problem description

Reproducible example code

Is this a regression? Put the last known working version here if it is.

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[BUG]: Python atexit broken with pybind11 version >= 2.9.2 #4459

Description

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

Problem description

Reproducible example code

Is this a regression? Put the last known working version here if it is.

Activity

Skylion007 commented on Jan 18, 2023

rwgk commented on Jan 19, 2023

chrreisinger commented on Jan 19, 2023

rwgk commented on Jan 19, 2023

Skylion007 commented on Jan 29, 2023

Skylion007 commented on Feb 8, 2023

henryiii commented on Jul 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions