Repository navigation
np.errstate is not asyncio-safe #9444
Description
While it is threadsafe, the with statement gives the false impression that it's always safe. It is not:
import asyncio
import numpy as np
def divide_by_zero():
np.float64(1) / 0
async def foo():
for j in range(3):
with np.errstate(divide='raise'):
for i in range(2):
try:
divide_by_zero()
except FloatingPointError:
pass
else:
print("foo: Failed to raise!")
await asyncio.sleep(0.15)
async def bar():
for j in range(3):
with np.errstate(divide='ignore'):
for i in range(2):
try:
divide_by_zero()
except FloatingPointError:
print("bar: raised anyway!")
await asyncio.sleep(0.11)
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(foo(), bar()))
loop.close()gives:
foo: Failed to raise!
bar: raised anyway!
foo: Failed to raise!
because the error-state is thread-local, but asyncio tasks share a thread.
I don't know if there is any way this can be fixed - it seems like this might be a python bug, and context managers might need an __yield_enter__ and __yield_leave__ mechanism, to be notified when control is leaving the suite via a yield/await
Activity
NeilGirdhar commentedon Jul 20, 2017
Your solution seems reasonable except that it adds a lot of overhead since you need to call
__yield_enter__on the entire stack of active asynchronous context managers.eric-wieser commentedon Jul 20, 2017
Using
tasklocalsfrom pypi would also solve the problemeric-wieser commentedon Jul 20, 2017
@njsmith is two years ahead of me here with PEP 521, but reassuringly has come to a similar conclusion to how this would need to be fixed
NeilGirdhar commentedon Jul 20, 2017
Yeah, he even explains the differences between that proposal and the task locals. So, it seems that one day Python will have this sorted out. Until then, there is probably no point in investing time in a "deprecation context manager". Still having a list of what you would deprecate would be really useful to motivate its creation one day?
NeilGirdhar commentedon Aug 19, 2017
FYI: A competing proposal, PEP 550, is being discussed on python-ideas.
1st1 commentedon Jan 29, 2018
PEP 567—Context Variables—has been accepted and implemented in Python 3.7. With its new APIs it's now possible to fix this issue.
NeilGirdhar commentedon Apr 26, 2018
The new
np.printoptionsshould probably be fixed at the same time.seberg commentedon Nov 10, 2021
I think this could be attacked now (and would also close gh-19006). Might be nice to see that it doesn't have any performance downsides, but it seemd a bit like contextvars are faster than "
TheDict".This goes way into the depth of the Python C-API, but (as far as I understand right now!) the work used in allocators should outline how this can be achieved: gh-17582