Repository navigation
Python contextvars don't persist across cells in notebook #11565
Description
Activity
That's weird. ContextVars are like thread-locals until someone explicitly calls
Context.run... is the kernel single-threaded? Does it useContext.run? Is some other lib calling it?I believe tornado on py3 uses the asyncio loop by default, and if you're on py 3.7 then asyncio will automatically use different contexts for different tasks. And probably each cell execution counts as a different task as far as tornado/aio is concerned.
Sounds like the kernel should be explicitly holding a
Contextobject and reusing it for each cell execution, instead of letting tornado/aio set theContextimplicitly.Reacted by Sofie Van Landeghem, Mark Dickinson, Saul Shanabrook and Sylvain CorlayAlso, related to this. It seems like context vars are not correctly propagated when initializing or calling code that get/sets vars in differently in async functions.
Here is some code to display the problem:
from IPython.terminal.embed import InteractiveShellEmbed from traitlets.config.loader import Config import contextvars cvar = contextvars.ContextVar('foo') def run(): cvar.set('bar') def foo(): cvar.set('foofunc') async def afoo(): cvar.set('afoofunc') ipshell = InteractiveShellEmbed(config=Config()) ipshell() if __name__ == '__main__': run()Here is in ipython prompt:
In [1]: cvar.get() Out[1]: 'bar' In [2]: foo() In [3]: cvar.get() Out[3]: 'foofunc' In [4]: await afoo() In [6]: cvar.get() == 'afoofunc' Out[6]: False- added 6 commits that reference this issue
on Feb 24, 2020 I recently had a package that was build with
contextvarsthat suddenly broke in jupyter notebooks. It would be great if this issue could be somehow addressed, as it might become a real problem if more packages start to rely oncontextvarsand simply break unexpectedly when used within ipython/jupyter.Two observations that I made are:
- The problem does not occur, when
Run all cellsis used. - The problem also does not seem to occur when Python 3.6 and the backport of
contextvarsis used, but that might well be related to what @njsmith described above.
Reacted by Sofie Van Landeghem- The problem does not occur, when
- added a commit that references this issue
on Feb 15, 2021 This produces surprising results when changing the
decimalprecision (or other aspects of the decimal context) in notebooks, sincedecimalnow uses a PEP 567 context variable for the decimal context.Related: ipython/ipykernel#494, jupyter/notebook#5260
Reacted by Saul Shanabrook, Evan Burke, Yang Yang and Shamir AlaviNote:
This does not impact IPython, or the xeus-python kernel.- added a commit that references this issue
on Aug 16, 2024
Python 3.7 added a new
contextvarspackage to "manage, store, and access context-local state".However, if I set a
ContextVarin one cell and then try to get it in another, it isn't persisted:This may be an antipattern (not sure if there is a good use case to set a variable across many cells) and I worked around it by setting
default, as recommended. However it is surprising and is different than howipythonworks.