Repository navigation
Memory leak with %matplotlib qt, with simple fix #14240
Description
When using matplotlib event loop integration (e.g. %matplotlib) with pyqt, there is a memory leak caused by a misunderstanding of the lifespan of QObjects.
I found it because I'm trying to debug slowdowns in long-running IPython sessions with autoreload. I'm not sure that this is actually the cause, but it's certainly ugly. Within a session which has been open for a couple of weeks I found:
>>> collections.Counter(map(type, gc.get_objects())).most_common(5)
[(PyQt5.QtCore.QEventLoop, 1718061),
(dict, 1006864),
(list, 702602),
(ast.Name, 267460),
(ast.Attribute, 96929)]
There are 1.7m QEventLoop instances kicking around. They are being created by IPython.terminal.pt_inputhooks.qt.inputhook on line 58, with event_loop = QtCore.QEventLoop(app). But QEventLoop is a QObject subclass, which means that passing app in the constructor parents the instance to app, so even when it exits the scope it won't be deleted, even though it's no longer needed by that point.
The fix is simple: just add a line at the end of that function containing event_loop.setParent(None).
Hopefully it's faster for you to just add that single line of code rather than me forking, making a pull request etc, but please tell me if you need me to do so.
Activity
ccordoba12 commentedon Nov 13, 2023
Thanks for finding a fix @pag!
Unfortunately, it's not because we're busy with other things. So, please submit a pull request to fix this issue.
Fix memory leak in Qt event loop integration (ipython#14240)
hperrey commentedon Nov 23, 2023
Thanks @pag for the suggested fix! We just discovered a memory leak on our machines related to IPython and this issue helped a lot to track down the problem!
I took the liberty of creating a PR, citing you.
pag commentedon Nov 23, 2023
Thanks for that @hperrey.
The other memory leak I found was related to those ast objects you can see in the count, basically vast numbers of objects are created and kept alive till the end of the session each time that an exception is raised with 'xmode Verbose'. They're being kept alive in the stack_data module. Swapping to simpler tracebacks with e.g. xmode Context fixes that. With that done and the Qt fix above, long-running sessions use constant space and autoreload doesn't slow down!
hperrey commentedon Nov 23, 2023
@pag, interesting, I had not noted that but will keep an eye out for that source of leaks!
Fix memory leak in Qt event loop integration (#14240) (#14251)
Clean up Qt object management
Improve Qt object management and performance (#14700)