JPEG encoding with Pillow PyPI wheel is 6x slower than distro package

What did you do?

Run the attached script to encode a RGB image to JPEG

What did you expect to happen?

The encoding should take about 1ms

What actually happened?

The encoding takes about 1ms with distribution provided package, but ~6ms with Pillow-8.0.1-cp3*-cp3*-manylinux1_x86_64.whl from PyPI

What are your OS, Python and Pillow versions?

I have tested on ArchLinux, Ubuntu and CentOS. Here are the results (all from same computer):

Archlinux

# python-pillow
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0]
Pillow 7.2.0
0.8895997999999963 ms/iter

# python-pillow-git (619a7b9f)
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0]
Pillow 8.1.0.dev0
0.8732269799998903 ms/iter

# Pillow from PyPI
Python 3.8.6 (default, Sep 30 2020, 04:00:38)
[GCC 10.2.0]
Pillow 8.0.1
5.752080059999571 ms/iter

Ubuntu

# python3-pillow
Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0]
Pillow 7.0.0
1.8287872446002438 ms/iter

# Pillow from PyPI
Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0]
Pillow 8.0.1
8.810945131699555 ms/iter

CentOS

# python3-pillow
Python 3.6.8 (default, Apr 16 2020, 01:36:27)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Pillow 5.1.1
2.030805503507145 ms/iter

# Pillow from PyPI
Python 3.6.8 (default, Apr 16 2020, 01:36:27)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Pillow 8.0.1
8.873232635483145 ms/iter

Code

#!/bin/python3

import timeit
import PIL
import PIL.Image
import io
import sys

print('Python', sys.version)
print('Pillow', PIL.__version__)
im = PIL.Image.open('lena.png').convert('RGB')  # http://optipng.sourceforge.net/pngtech/img/lena.png
def test():
    byio = io.BytesIO()
    im.save(byio, 'JPEG')
print(timeit.timeit(test, number=10000) / 10, 'ms/iter')

I'm pretty sure the difference is due to the fact that wheels use the libjpeg library, while the distro version most likely uses libjpeg-turbo.

In that case can we switch to libjpeg-turbo in future PyPI build? People usually use the PyPI package to install a newer version than the distro provided one. And they might be surprised by the significant performance drop.

libjpeg-turbo is also under the IJG license if we only use libjpeg interface. It is long used by major linux distros as default libjpeg implementation, and is faster than libjpeg. It is already in python-pillow/pillow-depends. It is also used in CI build.

I've created python-pillow/pillow-wheels#237 to resolve this by switching to libjpeg-turbo in the macOS and Linux wheels.

Navigation Menu

Repository navigation

JPEG encoding with Pillow PyPI wheel is 6x slower than distro package #5073

What did you do?

What did you expect to happen?

What actually happened?

What are your OS, Python and Pillow versions?

Code

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

JPEG encoding with Pillow PyPI wheel is 6x slower than distro package #5073

Description

What did you do?

What did you expect to happen?

What actually happened?

What are your OS, Python and Pillow versions?

Code

Activity

nulano commented on Dec 2, 2020

wangqr commented on Dec 2, 2020

radarhere commented on Dec 31, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions