Repository navigation
Memory leak when calling PIL.Image.Image.__arrow_c_array__() #8950
Description
What did you do?
I'm calling the PIL.Image.Image.__arrow_c_array__ method on images to access their data.
What did you expect to happen?
max rss should stay constant after a while
What actually happened?
memory gets leaked
The output of the reproduction script below:
RSS: 88656
RSS: 88836
RSS: 88908
=== accessing __arrow_c_array__ ===
RSS: 395212
RSS: 713164
=== with arro3.core.Array ===
RSS: 1031376
RSS: 1349072
=== with pyarrow.array ===
RSS: 1667588
RSS: 1986052
What are your OS, Python and Pillow versions?
- OS: Arch Linux
- Python: Python 3.13.3
- Pillow: 11.2.1
$ python3 -m PIL --report
--------------------------------------------------------------------
Pillow 11.2.1
Python 3.13.3 (main, Apr 9 2025, 07:44:25) [GCC 14.2.1 20250207]
--------------------------------------------------------------------
Python executable is /home/josh/code/an-website/venv/bin/python3
Environment Python files loaded from /home/josh/code/an-website/venv
System Python files loaded from /usr
--------------------------------------------------------------------
Python Pillow modules loaded from /home/josh/code/an-website/venv/lib/python3.13/site-packages/PIL
Binary Pillow modules loaded from /home/josh/code/an-website/venv/lib/python3.13/site-packages/PIL
--------------------------------------------------------------------
--- PIL CORE support ok, compiled for 11.2.1
--- TKINTER support ok, loaded 8.6
--- FREETYPE2 support ok, loaded 2.13.3
--- LITTLECMS2 support ok, loaded 2.17
--- WEBP support ok, loaded 1.5.0
*** AVIF support not installed
--- JPEG support ok, compiled for libjpeg-turbo 3.1.0
--- OPENJPEG (JPEG2000) support ok, loaded 2.5.3
--- ZLIB (PNG/ZIP) support ok, loaded 1.3.1, compiled for zlib-ng 2.2.4
--- LIBTIFF support ok, loaded 4.7.0
--- RAQM (Bidirectional Text) support ok, loaded 0.10.1, fribidi 1.0.16, harfbuzz 11.0.1
*** LIBIMAGEQUANT (Quantization method) support not installed
--- XCB (X protocol) support ok
--------------------------------------------------------------------
Reproduction script
Doesn't need to be run with uv. If run with cpython, make sure the dependencies are installed>
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "arro3-core",
# "numpy",
# "pillow",
# "pyarrow",
# ]
# ///
import gc, io, urllib.request, resource
import numpy, pyarrow
from arro3.core import Array
from PIL import Image
COUNT = 400
# the exact image doesn't matter
with urllib.request.urlopen("https://testimages.org/img/testimages_screenshot.jpg") as file:
data = file.read()
del file
# 3 times without arrow
for _ in range(3):
for _ in range(COUNT):
image = Image.open(io.BytesIO(data))
image.load()
numpy.asarray(image)
tuple(image.getdata())
image.close()
del image
gc.collect()
print(f"RSS: {resource.getrusage(resource.RUSAGE_SELF).ru_maxrss: 10}")
print("=== accessing __arrow_c_array__ ===")
for _ in range(2):
for _ in range(COUNT):
image = Image.open(io.BytesIO(data))
image.load()
a, b = image.__arrow_c_array__()
image.close()
del image, a, b
gc.collect()
print(f"RSS: {resource.getrusage(resource.RUSAGE_SELF).ru_maxrss: 10}")
print("=== with arro3.core.Array ===")
for _ in range(2):
for _ in range(COUNT):
image = Image.open(io.BytesIO(data))
image.load()
Array(image)
image.close()
del image
gc.collect()
print(f"RSS: {resource.getrusage(resource.RUSAGE_SELF).ru_maxrss: 10}")
print("=== with pyarrow.array ===")
for _ in range(2):
for _ in range(COUNT):
image = Image.open(io.BytesIO(data))
image.load()
pyarrow.array(image)
image.close()
del image
gc.collect()
print(f"RSS: {resource.getrusage(resource.RUSAGE_SELF).ru_maxrss: 10}")
Activity
wiredfool commentedon May 11, 2025
Ok, Looking at this in valgrind is clearly showing that the image memory is retained. This is the first arrow section with fewer iterations:
Looks like an
off by one in the refcount implementationnot releasing the Image when the pycapsule used child arrays, with a fix I'm seeing flat memory usage. This is for all three arrow sections.radarhere commentedon Jun 19, 2025
#8954 has been merged to fix this.