Repository navigation
Pytest keeps reference to attributes of failed test cases for too long #12198
Description
Activity
this is a unfortunate side-effect of ensuring staying debugable
there was no effort yet to provide a api/hook to purge tracebakcs/stacktraces of expensive resources
in this case im considering the array a "resource" as its something that puts possibly large pressure onto the system and ought to be purged if disposable
Hi @RonnyPfannschmidt,
Thanks a lot for the follow-up.
Wondering what was the reason for not providing an api/hook to remove the traceback?
Or the pytest team does not have this in plans at all?
Is it complicated for a beginner to pytest repo like me to try to implement it?For others who also encounter this issue in their tests,
I found a work-aroundimport weakref import numpy as np import pytest N = 20_000_000 def get_data(n): d = np.random.rand(n,100) return d def process_data(d): return np.diff(d) def test1(data_provider): d = data_provider.get(N) process_data(d) assert False def test2(): d = get_data(N) process_data(d) class DataFlyweight: def __init__(self): self._data =[] def get(self,n): if n not in self._data: self._data.append(get_data(n)) return weakref.proxy(self._data[-1]) def clear(self): self._data.clear() @pytest.fixture def data_provider(): data_flyweight = DataFlyweight() yield data_flyweight data_flyweight.clear()
Memory does not explode and the second test passed as expected.
Multiple reasons
Initially tracebacks where not attached to exceptions and extracting ex info typically lost a number of locals automatically
Also for most test suites memory intensive variables where no issue
So there was no incentive/pressure to work on a feature with limited win and excessive possible edge cases for a long time
Hi,
While running some tests with large numpy arrays, I figured out that there is a difference in how pytest cleans up the data for the failed and passed test cases.
For the failed tests, the references to local variables are kept, leading to the accumulation of many large arrays in the memory and eventually out-memory error.
It could probably be best explained with examples
I also logged memory consumption using the following pytest fixture
And here is the result. Nothing is unexpected.
But if one test fails, the second one will fail too as there is no longer enough memory in the python process.
I used
objgraphto trace the reference to the array. It looks like a frame used by the traceback hold onto it.I think this could be a bug in pytest and may be there is a known work around for this issue.
Pytest 8.1.1
Python 3.11.0
OS: Ubuntu 22.04.3 LTS
Install Packages
Package Version
iniconfig 2.0.0
numpy 1.26.4
packaging 24.0
pip 23.3.1
pluggy 1.4.0
psutil 5.9.8
pytest 8.1.1
setuptools 68.2.2
wheel 0.41.2
pip listfrom the virtual environment you are using