Repository navigation
Embedded initialization ignores Py_SetProgramName() #75530
Description
| BPO | 31349 |
|---|---|
| Nosy | @pfmoore, @vstinner, @tjguk, @zware, @zooba, @chrullrich |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-09-26.01:14:47.432>
created_at = <Date 2017-09-05.12:45:00.505>
labels = ['type-bug', '3.8', '3.9', 'OS-windows']
title = 'Embedded initialization ignores Py_SetProgramName()'
updated_at = <Date 2019-09-26.01:14:47.431>
user = 'https://github.com/chrullrich'bugs.python.org fields:
activity = <Date 2019-09-26.01:14:47.431>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-26.01:14:47.432>
closer = 'vstinner'
components = ['Windows']
creation = <Date 2017-09-05.12:45:00.505>
creator = 'chrullrich'
dependencies = []
files = []
hgrepos = []
issue_num = 31349
keywords = []
message_count = 8.0
messages = ['301306', '301307', '301345', '301346', '301371', '313342', '313343', '353248']
nosy_count = 6.0
nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'zach.ware', 'steve.dower', 'chrullrich']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue31349'
versions = ['Python 3.8', 'Python 3.9']
Activity
chrullrich commentedon Sep 5, 2017
I'm trying to do something that may be slightly tricky, and I think I just found a vintage 1997 bug. Please correct me if I'm wrong.
This is PC/getpathp.c from current master:
430: static void
431: get_progpath(void)
432: {
433: extern wchar_t *Py_GetProgramName(void);
434: wchar_t *path = _wgetenv(L"PATH");
435: wchar_t *prog = Py_GetProgramName();
436:
437: #ifdef Py_ENABLE_SHARED
438: extern HANDLE PyWin_DLLhModule;
439: /* static init of progpath ensures final char remains \0 */
440: if (PyWin_DLLhModule)
441: if (!GetModuleFileNameW(PyWin_DLLhModule, dllpath, MAXPATHLEN))
442: dllpath[0] = 0;
443: #else
444: dllpath[0] = 0;
445: #endif
446: if (GetModuleFileNameW(NULL, progpath, MAXPATHLEN))
447: return;
448: if (prog == NULL || *prog == '\0')
449: prog = L"python";
Lines 446-447 have been like this ever since Guido wrote them twenty years ago. I think the logic is the wrong way around. As it is now, the function always returns on line 447 because GetModuleFileNameW() *always* succeeds (returns nonzero). Hence, everything below line 447 is dead code.
GetModuleFileNameW(NULL, ...) fills the buffer with the path to the executable the current process was created from. It returns the number of characters put into the buffer, or zero on error. However, the only way *this* call could fail is if the code was being executed outside of any process, something that clearly cannot happen.
Prior to August 1997, the relevant bit of code looked like this:
156: if (!GetModuleFileName(NULL, progpath, MAXPATHLEN))
157: progpath[0] = '\0'; /* failure */
This bug, if it is one, would only manifest when initializing an embedded interpreter using a different argv[0] than that of the actual host process, because if the host process is a python.exe, it very likely runs inside a standard installation or venv anyway. What I am trying is to make an interpreter running in a third-party host process take site-packages from a venv.
Doing this (argv[0] different from actual host process) may not be entirely proper itself, but then again, why would Py_SetProgramName() even exist otherwise?
Suggested fix:
Since the call to GetModuleFileNameW() cannot possibly fail, there is no real point in checking its return value.
chrullrich commentedon Sep 5, 2017
That should have been
instead, of course, without the negation.
zooba commentedon Sep 5, 2017
You're right, though I disagree with the fix as that would cause the process filename to be ignored.
A correct fix would check whether the name has been set explicitly and if so, either use it as-is (if absolute) or get the actual name and use its directory (since there's no point searching PATH as we'll likely just find some other executable and hence not be able to initialize).
As this is a subtle change in behaviour, we should only do it for 3.7. For 3.6 and earlier I'd suggest naming your executable the name that you'd like it to be, as that will work fine in this case and anywhere else we assume that the process name is the name of the process. (The fact that the behaviour is wrong in earlier versions is overridden IMO by the fact that it has been wrong for a decade and nobody has complained, and there is a trivial workaround.)
chrullrich commentedon Sep 5, 2017
Not quite. Looking a bit further down get_progname()'s weird logic, we see that it works like this:
prog = "python"
progpath = prog
So it uses the host process name from step 2 (i.e. progpath) whenever Py_SetProgramName() has not been used and step 3 has set prog to a value not containing a directory separator (or if it *has* been used to set something with no such separator in it).
The logic makes sense, I think, but it is quite impenetrable. Any chance of PEP-432 ("Restructuring the CPython startup sequence") happening this century?
zooba commentedon Sep 5, 2017
People are working on PEP-432 this week at the sprints, so yeah, it's likely.
zooba commentedon Mar 6, 2018
I just hit this myself (and embarrassingly, I recently touched this code everyone and forgot to fix this).
Hopefully I get a chance to get to it, but patches are certainly welcome and I'll happily aim to get them into 3.6 onwards.
zooba commentedon Mar 6, 2018
(That said, I didn't try again 3.7, so it may already be fixed there. But since we're still fixing problems with 3.6, we should do that one too.)
vstinner commentedon Sep 26, 2019
Modules/getpath.c and PC/getpathp.c have been deeply reworked with the implementation of the PEP-587 in Python 3.8. Parameters which are already set (ex: Py_SetProgramName() or using the new PyConfig API) are no longer overriden. I added some tests on the "path configuration".
The most reliable way to configure the path configuration is now to use the new initialization API using PyConfig.
https://docs.python.org/dev/c-api/init_config.html