As discussed in #129033, opening a new issue at the request of @vstinner. See #129033 (comment) for the reasons this API is needed.
In short, a way to run python code and insert a meta importer into sys.meta_path before the "main" phase of initialization runs is needed. This allows, for instance, to bundle all python code as pre-compiled bytecode into a single mmapped file and directly load it from there for improved performance and hygiene.
I am aware that the state of the interpreter after "core" initialization is not entirely specified, however, in practice it is complete enough to write a meta path importer.
This facility has been used for many years by at least two projects that I know of: bypy (link to source code) https://github.com/kovidgoyal/bypy/blob/master/bypy/freeze/bypy-freeze.h#L835 and PyOxidizer: https://github.com/indygreg/PyOxidizer/blob/1ceca8664c71f39e849ce4873e00d821504b32bd/pyembed/src/technotes.rs#L155
I dont much care about this API in particular, but I would like some way to hook into the initialization process after "core" initialization and before "main" initialization.
Some context about use case. bypy is used in the calibre and kitty programs, both of which embed a "frozen" Python interpreter. _Py_InitializeMain() allows that frozen importer to import all its python code, including stdlib code from a single mmaped file contianing pre-compiled bytecode. This has obvious performance benefits especially on windows.
Linked PRs
As discussed in #129033, opening a new issue at the request of @vstinner. See #129033 (comment) for the reasons this API is needed.
In short, a way to run python code and insert a meta importer into sys.meta_path before the "main" phase of initialization runs is needed. This allows, for instance, to bundle all python code as pre-compiled bytecode into a single mmapped file and directly load it from there for improved performance and hygiene.
I am aware that the state of the interpreter after "core" initialization is not entirely specified, however, in practice it is complete enough to write a meta path importer.
This facility has been used for many years by at least two projects that I know of: bypy (link to source code) https://github.com/kovidgoyal/bypy/blob/master/bypy/freeze/bypy-freeze.h#L835 and PyOxidizer: https://github.com/indygreg/PyOxidizer/blob/1ceca8664c71f39e849ce4873e00d821504b32bd/pyembed/src/technotes.rs#L155
I dont much care about this API in particular, but I would like some way to hook into the initialization process after "core" initialization and before "main" initialization.
Some context about use case. bypy is used in the calibre and kitty programs, both of which embed a "frozen" Python interpreter. _Py_InitializeMain() allows that frozen importer to import all its python code, including stdlib code from a single mmaped file contianing pre-compiled bytecode. This has obvious performance benefits especially on windows.
Linked PRs