Repository navigation
custom PYTHONPATH may break apps embedding Python #59161
Description
| BPO | 14956 |
|---|---|
| Nosy | @brettcannon, @ncoghlan, @vstinner, @merwok, @bitdancer, @davidmalcolm, @meadori, @Romain-Geissler-1A |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-05-27.14:51:32.617>
created_at = <Date 2012-05-29.19:56:51.840>
labels = ['type-bug', 'docs']
title = 'custom PYTHONPATH may break apps embedding Python'
updated_at = <Date 2019-05-27.14:51:32.615>
user = 'https://bugs.python.org/jankratochvil'bugs.python.org fields:
activity = <Date 2019-05-27.14:51:32.615>
actor = 'vstinner'
assignee = 'docs@python'
closed = True
closed_date = <Date 2019-05-27.14:51:32.617>
closer = 'vstinner'
components = ['Documentation']
creation = <Date 2012-05-29.19:56:51.840>
creator = 'jankratochvil'
dependencies = []
files = []
hgrepos = []
issue_num = 14956
keywords = []
message_count = 16.0
messages = ['161906', '161907', '161908', '161909', '161911', '161912', '161913', '161918', '161919', '161955', '161957', '161959', '161970', '194137', '328727', '343633']
nosy_count = 11.0
nosy_names = ['brett.cannon', 'ncoghlan', 'vstinner', 'eric.araujo', 'r.david.murray', 'dmalcolm', 'meador.inge', 'jankratochvil', 'docs@python', 'Kevin.Harms', 'RomainGeissler1A']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14956'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']
Activity
jankratochvil commentedon May 29, 2012
People use custom Python builds setting PYTHONHOME and PYTHONPATH to these custom builds. This may be expected to break running system Python binary but it also unexpectedly breaks various applications which also embed Python:
It is not obvious to the user who even already forgot about custom PYTHONPATH why GDB breaks.
Standard Fedora applications may link with system libpython incompatible with the PYTHONHOME/PYTHONPATH files for the other Python version. Python does not keep the scripts too compatible across Python versions.
This issues was discussed first for GDB upstream:
[RFA] ignore PYTHONHOME environment variable.
http://sourceware.org/ml/gdb-patches/2010-11/msg00328.html
http://sourceware.org/ml/gdb-patches/2010-12/msg00214.html
http://sourceware.org/ml/gdb-patches/2011-01/msg00307.html
http://sourceware.org/ml/gdb-patches/2012-05/msg00710.html
(first mail of the thread in each month)
Then also in Fedora:
https://fedorahosted.org/fesco/ticket/858
This Python issue would not be filed by the user if it gets resolved:
bpo-12140
Exactly the same problem exists for many other apps, yum for all
Various proposal have been, applicable possibly in some combination together:
or Recover from Python startup failure and disable the Python support, if possible.
How should get PYTHONHOME/PYTHONPATH inherited into child processes, if any (debuggees in the GDB case).
Another question is whether the behavior should be the same if upstream application has been linked with non-system libpython location.
merwok commentedon May 29, 2012
An alternate idea would be to recommend more strongly in the doc that people should not shadow standard library module names (in your example, site).
jankratochvil commentedon May 29, 2012
This site.py is only an example how it can happen. In reality it is due to .py files intended for incompatible version of Python. I am not a Python programmer to have some appropriate incompatible code at hand.
merwok commentedon May 29, 2012
[-]custom PYTHONPATH breaks Python-embedded apps[/-][+]custom PYTHONPATH may break apps embedding Python[/+]bitdancer commentedon May 29, 2012
I think it is up to each embedding application to decide if it wants to respect the environment variables or not. I don't think that's a decision that core can make.
Now, if we aren't providing an easy way for an embedding application to make that decision, that might be something we could fix. But I suspect we are (I don't know the C API well enough to say for sure).
jankratochvil commentedon May 29, 2012
I fully agree with site.py/os.py/spam.py but I find it offtopic for this Issue.
I do not find too important if some unsetenv/setenv gets called by the app or by libpython. But the rules should apply for every embedded app the same, the Fedora ticket is suggesting new PEP (Informational) for it.
bitdancer commentedon May 29, 2012
An informational PEP sounds like exactly what I said: each application needs to decide. The PEP would provide the information on which to base that decision, and suggestions for how to do it.
An informational PEP on best practices for embedding sounds like a great idea to me. I hope someone will write one.
ncoghlan commentedon May 29, 2012
It wouldn't be a PEP, it would be an addition to the embedding docs:
http://docs.python.org/extending/embedding.html
ncoghlan commentedon May 29, 2012
For the more general breakage due to PYTHONHOME and PYTHONPATH, yeah, global environment variables are bad, particularly when an OS relies on tools written in (or embedding) Python.
That's the reason virtualenv (and 3.3's forthcoming venv) are a preferred alternative - they give you a space to play in that shouldn't break your system Python or apps that embed it.
merwok commentedon May 30, 2012
I don’t understand this message :) There is nothing to agree with or judge on or off-topic; I was trying to understand the root of the bug and really asking you in good faith to try two things to see if my idea was right.
jankratochvil commentedon May 30, 2012
While it should be documented this is not only a docs issue. It should be solved in some way during runtime.
bitdancer commentedon May 30, 2012
No it shouldn't. As mentioned in the Fedora thread you linked, this is no different than the user setting LD_LIBRARY_PATH to something that screws up a system installed program.
ncoghlan commentedon May 30, 2012
If we don't expose the mechanism behind -E to embedding applications via
the C API, then a non-docs change may be needed. However, writing (or at
least trying to write) the relevant docs is a good way to check whether or
not that is the case.
KevinHarms commentedon Aug 1, 2013
As far as I can tell, this issue is not resolved. It would be very nice for the C API to have the equivalent for -E for security reasons. I would like to make a setgid binary and do not want users to be able to set PYTHON* variables to force loading of custom code.
Romain-Geissler-1A commentedon Oct 28, 2018
Hi,
Just updating this 6 year old bug report. The last comment says it's not possible to ignore environment when using the C-API. I don't know back then, but today it is possible. I have just built gdb 8.2 against python 3.7.1, and patched gdb with this simple patch, which apparently works:
--- gdb/python/python.c
+++ gdb/python/python.c
@@ -1726,6 +1726,9 @@
#endif
#endif
Cheers,
Romain
vstinner commentedon May 27, 2019
This issue has been fixed in 36763 by the PEP-587 which provides a new "Isolated Configuration" and a way to tune the Python configuration without impacting subprocesses. For example, the Isolated Configuration ignores environment variables by default.
In Python 3.7 and older, you can set Py_IgnoreEnvironmentFlag to 1 (the flag exists also in Python 2.7), or even set Py_IsolatedFlag to 1 (new in Python 3.4).