asyncio hang in subprocess wait_closed() on Windows, BrokenPipeError

@asvetlov

BPO	45074
Nosy	@asvetlov, @1st1
Files	wait_closed.py: demonstrate hang in stream.wait_closed()

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-01.05:10:33.552>
labels = ['type-bug', '3.9', '3.10', 'expert-asyncio']
title = 'asyncio hang in subprocess wait_closed() on Windows, BrokenPipeError'
updated_at = <Date 2021-09-01.05:10:33.552>
user = 'https://bugs.python.org/byllyfish'

bugs.python.org fields:

activity = <Date 2021-09-01.05:10:33.552>
actor = 'byllyfish'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2021-09-01.05:10:33.552>
creator = 'byllyfish'
dependencies = []
files = ['50250']
hgrepos = []
issue_num = 45074
keywords = []
message_count = 1.0
messages = ['400810']
nosy_count = 3.0
nosy_names = ['asvetlov', 'yselivanov', 'byllyfish']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45074'
versions = ['Python 3.9', 'Python 3.10']

I have a reproducible case where stdin.wait_closed() is hanging on
Windows. This happens in response to a BrokenPipeError. The same code
works fine on Linux and MacOS.

Please see the attached code for the demo.

I believe the hang is related to this debug message from the logs:

DEBUG <_ProactorWritePipeTransport closing fd=632>: Fatal write error on pipe transport
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.9.6\x64\lib\asyncio\proactor_events.py", line 379, in _loop_writing
    f.result()
  File "C:\hostedtoolcache\windows\Python\3.9.6\x64\lib\asyncio\windows_events.py", line 812, in _poll
    value = callback(transferred, key, ov)
  File "C:\hostedtoolcache\windows\Python\3.9.6\x64\lib\asyncio\windows_events.py", line 538, in finish_send
    return ov.getresult()
BrokenPipeError: [WinError 109] The pipe has been ended

It appears that the function that logs "Fatal write error on pipe transport" also
calls _abort on the stream. If _abort is called before stdin.close(), everything is okay.
If _abort is called after stdin.close(), stdin.wait_closed() will hang.

Please see issue bpo-44428 for another instance of a similar hang in wait_closed().

This bug is another instance of protocol.connection_lost callback not getting called correctly. With #98572 the callback is always called which fixes the issue.

With fix output:

Running Debug|x64 interpreter...
BrokenPipe 1 - correct
BrokenPipe 2 - correct

GH-89237: fix hang in proactor `subprocess.wait_closed()` (#98572)

pythonGH-89237: fix hang in proactor `subprocess.wait_closed()` (pyth…

GH-89237: fix hang in proactor `subprocess.wait_closed()` (GH-98572)

[3.11] GH-89237: fix hang in proactor `subprocess.wait_closed()` (GH-…

Navigation Menu

Repository navigation

asyncio hang in subprocess wait_closed() on Windows, BrokenPipeError #89237

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

asyncio hang in subprocess wait_closed() on Windows, BrokenPipeError #89237

Description

Activity

byllyfish commented on Sep 1, 2021

kumaraditya303 commented on Oct 23, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions