Skip to content

32-bit Unicode code points don't work on Windows #3777

Description

@nulano

When rendering with TrueType fonts on Windows, Unicode code points outside the 16-bit range are rendered as a pair of unknown characters (usually a box with a question mark). I suspect this is caused by one of the following lines:

Py_UNICODE* p = PyUnicode_AS_UNICODE(string);

Py_UNICODE *text = PyUnicode_AS_UNICODE(string);

According to Python docs, Py_UNICODE is a typedef for wchar_t, which is 16-bit on Windows, but typically 32-bit on Linux. The docs suggest using the type Py_UCS4 and function PyUnicode_AsUCS4Copy (introduced in Python 3.3).
https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUCS4Copy

I would attempt changing this myself, but I'm not sure how to compile native Python libraries from source.

Code to reproduce:

import sys
from PIL import Image, ImageDraw, ImageFont

size = int(sys.argv[2]) if len(sys.argv) >= 3 else 48
fontname = sys.argv[1] if len(sys.argv) >= 2 else 'Symbola.ttf'
font = ImageFont.truetype(fontname, size=size)

text = ''' Unicode pictogram test:
Letters: P H BUS
Sans-serif: 𝗣 𝗛 𝗕𝗨𝗦
Crosses: † ✝
Amenity: 🚻 🖂 ⛲ ⛹ 🎓 🎒
Entertainment: 🎨 🎡 🎢 🎜 🎬 🎭 🎰 🎲
Sport: 🎳 🎾 🏓 🏸 🚩
Restaurants: 🍽 🍴 🍵 🍶 🍷 🍺 🍾
Transport: ⛽ 🚂 🚁 🚃 🚋 🚌 🚖 🛪
'''.splitlines()

image = Image.new('RGB', (1000, 600), '#fff')
draw = ImageDraw.Draw(image, 'RGBA')
for i, line in enumerate(text):
    draw.text((0, i * size), line, fill='#000', font=font)
image.save(fontname + '.png')

Font used: https://www.fontspace.com/unicode-fonts-for-ancient-scripts/symbola

Result on Windows (Pillow 5.4.1, Python 3.7.0 32-bit, Windows 7 64-bit):

win-Symbola ttf

Result on Linux (Pillow 5.4.1, Python 3.4.3, Linux Mint 17.3 64-bit):

linux-Symbola ttf

Activity

  1. nulano commented on Apr 7, 2019

    @nulano
    ContributorAuthor

    I think #3047 might be a duplicate, although with the wrong conclusion. The font I am using does support the characters I am using (both with Pillow on Linux and in Microsoft Word on Windows), but not the problematic character in #3047.

  2. nulano commented on Apr 11, 2019

    @nulano
    ContributorAuthor

    This is also a duplicate of #3344 .

  3. added a commit that references this issue on Jun 25, 2019
  4. radarhere commented on Jun 30, 2019

    @radarhere
    Member

    #3780 has been merged, which fixes this issue for Python 3. This can either be closed now, or after Pillow 6.2.0, when Python 2 support is no longer needed.

  5. hugovk commented on Jun 30, 2019

    @hugovk
    Member

    This report was for Python 3, let's close it. Thanks all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions