Repository navigation
QT backend crashes IPython #14815
Description
Specifically 334fabc introduced in #14700 causes _exec(_event_loop) to be called with _event_loop = None. Reverting it causes the problem to go away, but maybe someone with better understanding of qt would like to actually fix it.
OS: Arch linux
Known affected versions:
8.33.0(Backport PR #14700 on branch 8.x (Improve Qt object management and performance) #14714)9.0.09.0.1
Crash log
Python 3.13.2 (main, Feb 5 2025, 08:05:21) [GCC 14.2.1 20250128]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.0.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: You can change the edditing mode of IPython to behave more like vi, or emacs.
In [1]: import matplotlib.pyplot as plt
...:
...: plt.plot([0], [0])
Out[1]: [<matplotlib.lines.Line2D at 0x7cd42f7e9950>]
In [2]: Traceback (most recent call last):
File "/usr/bin/ipython", line 8, in <module>
sys.exit(start_ipython())
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/IPython/__init__.py", line 139, in start_ipython
return launch_new_instance(argv=argv, **kwargs)
File "/usr/lib/python3.13/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/IPython/terminal/ipapp.py", line 325, in start
self.shell.mainloop()
~~~~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/IPython/terminal/interactiveshell.py", line 1010, in mainloop
self.interact()
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/site-packages/IPython/terminal/interactiveshell.py", line 995, in interact
code = self.prompt_for_code()
File "/usr/lib/python3.13/site-packages/IPython/terminal/interactiveshell.py", line 945, in prompt_for_code
text = self.pt_app.prompt(
default=default,
inputhook=self._inputhook,
**self._extra_prompt_options(),
)
File "/usr/lib/python3.13/site-packages/prompt_toolkit/shortcuts/prompt.py", line 1035, in prompt
return self.app.run(
~~~~~~~~~~~~^
set_exception_handler=set_exception_handler,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
inputhook=inputhook,
^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3.13/site-packages/prompt_toolkit/application/application.py", line 978, in run
result = loop.run_until_complete(coro)
File "/usr/lib/python3.13/asyncio/base_events.py", line 712, in run_until_complete
self.run_forever()
~~~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/asyncio/base_events.py", line 683, in run_forever
self._run_once()
~~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/asyncio/base_events.py", line 2002, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.13/site-packages/prompt_toolkit/eventloop/inputhook.py", line 150, in select
self.inputhook(InputHookContext(self._r, input_is_ready))
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/IPython/terminal/pt_inputhooks/qt.py", line 86, in inputhook
_exec(_eventloop)
~~~~~^^^^^^^^^^^^
File "/usr/lib/python3.13/site-packages/IPython/terminal/pt_inputhooks/qt.py", line 16, in _exec
obj.exec() if hasattr(obj, "exec") else obj.exec_()
^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'exec_'
Activity
[-]QT backend crashes IPython 9[/-][+]QT backend crashes IPython[/+]Carreau commentedon Mar 5, 2025
We would need to ping @jdranczewski and decide whether to revert or if a fix is possible..
jdranczewski commentedon Mar 6, 2025
I will look into it today, I thought it was not possible to reach that line without an eventloop being created - let me check on it.
Explicitly create QEventLoop separately from QApplication
jdranczewski commentedon Mar 6, 2025
It seems I don't check explicitly whether the eventloop exists because I assumed the inputhook function is the only way a Qt application would be created in the IPython context - you can see that based on this assumption I create the QApplication and QEventLoop together at
ipython/IPython/terminal/pt_inputhooks/qt.py
Lines 52 to 53 in 334fabc
If a QtApplication was created in some other way (which I must assume is what's happening here), then we will never enter that block of code, and the QEventLoop won't be created.
This is likely related to how matplotlib starts the Qt integration? In my testing I always entered the loop explicitly through
%gui qt, but it seems that callingimport matplotlib.pyplot as pltdoes it too. I can do a bit more testing to try to figure out why exactly this happens.A very quick fix then is to make sure the QEventLoop exists separately from the QApplication (see the commit linked above). @gtsiam can you check that my fork works for you? (https://github.com/jdranczewski/ipython) I am unable to reproduce the crash on Windows, but I'm spinning up a Python environment on my Steam Deck to test it on Linux 😅
jdranczewski commentedon Mar 6, 2025
Can confirm a couple of things:
%gui qtor%matplotlib qtis not called, butimport matplotlib.pyplot as pltand thenplt.plot()is.gtsiam commentedon Mar 6, 2025
Just checked - your quick fix does indeed appear to solve the problem.
jdranczewski commentedon Mar 6, 2025
Thank you for checking!
I still can't wrap my head around why this fails specifically on Linux, or what the exact logic matplotlib applies is - on Windows it seems to create a self-contained application and event loop and blocks the IPython process, on Linux it creates an application too but then defers to IPython to run the event loop as an inputhook. I'm failing to locate the code responsible for this logic in the matplotlib codebase unfortunately.
Either way, checking for the eventloop separately from the application seems like a good fix for this edge case. I'd be happy to open a pull request. Alternatively we could return to the previous handling of QObjects while still fixing #14581 by reverting my changes to object management but still changing the timer to be 10ms instead of 50ms. I would prefer the first option, just to avoid any potential memory leak issues still. @Carreau how would you like to proceed on this?
On a different note, apologies for introducing this bug in the first place. I was running on the assumption that clients interacting with IPython/IPykernel would leave the creation of QApps to IPython, as there can only be one at a time. The lesson for me is to not assume what the client code will do!
gtsiam commentedon Mar 7, 2025
FWIW, matplotlib seems to call ipython's enable_gui() right after initializing its backend. The matplotlib qt application is created here when the first canvas is created. I can't find any platform specific code in either matplotlib or ipython (only backend/toolkit specific code). So I'm inclined to believe that the code causing the difference is in neither of these projects, but in one of the dependencies.
Maybe windows calls the inputhook once immediately and linux doesn't for some reason? This seems most likely but I don't have access to a windows pc right now. Or maybe there's some weird mixing of QT major versions? In any case, the input hook should be able to handle existing qt application instances.
Also: I'm not sure why IPython needs to create QT applications in the first place. If one already exists, we should be able to just block the inputhook. But if none does, can't we just return immediately?
Carreau commentedon Mar 7, 2025
I don't have much preferences, I'll trust you, I don't have enough qt experience.
No need to apologise, you helped fixed others, and it happens to everyone to write imperfect code.
jdranczewski commentedon Mar 8, 2025
Thanks for creating the PR @Carreau, I got caught up with other things yesterday! I'm happy with that being merged as is.
Explicitly create QEventLoop separately from QApplication (#14828)
jdranczewski commentedon Mar 8, 2025
@gtsiam thanks for looking into the matplotlib side! I did spot the call to
enable_guibut couldn't trace the logic of whyinstall_repl_displayhookwould be called in the context of creating a Figure object (on Linux) - unless there's an automatic call toion()somewhere that I'm not seeing. To be clear, Windows doesn't seem to call the inputhook at all unless%matplotlib qtis explicitly enabled.I may try to dive into this with a debugger attached sometime soon for my own satisfaction, but the core cause of the problem is identified and resolved I think - my code didn't support a case where someone else created the QApplication, but still relied on IPython to run the QEventLoop.
Carreau commentedon Mar 8, 2025
No worries, 9.0 and 8.34 should be out, I hope that will fix all the critical issues.
gtsiam commentedon Mar 24, 2025
I'd say there is no reason to keep this open then - thanks for addressing it!