Skip to content

test_file_jpeg2k fails with IOError: broken data stream when reading image file on Windows #584

Closed
Closed
test_file_jpeg2k fails with IOError: broken data stream when reading image file on Windows#584
Labels
BugAny unexpected behavior, until confirmed feature.
Milestone
2.4.0

Description

@cgohlke
cgohlke
opened on Mar 29, 2014

Using Pillow af4424e linked to openjpeg 2.0svn on Windows, the following tests fail on all Python versions.

Tests\test_file_jpeg2k.py:35: im.load() failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 35, in test_sanity
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:47: im.load() failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 47, in test_lossless
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:53: im.load() failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 53, in test_lossy_tiled
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:59: im = roundtrip(test_card) failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 59, in test_lossless_rt
    im = roundtrip(test_card)
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:63: im = roundtrip(test_card, quality_layers=[20]) failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 63, in test_lossy_rt
    im = roundtrip(test_card, quality_layers=[20])
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:67: im = roundtrip(test_card, tile_size=(128, 128)) failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 67, in test_tiled_rt
    im = roundtrip(test_card, tile_size=(128, 128))
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file






Tests\test_file_jpeg2k.py:72: offset=(32, 32)) failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 72, in test_tiled_offset_rt
    offset=(32, 32))
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:76: im = roundtrip(test_card, irreversible=True, quality_layers=[20]) failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 76, in test_irreversible_rt
    im = roundtrip(test_card, irreversible=True, quality_layers=[20])
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:80: im = roundtrip(test_card, quality_layers=[60, 40, 20], progression='LRCP') failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 80, in test_prog_qual_rt
    im = roundtrip(test_card, quality_layers=[60, 40, 20], progression='LRCP')
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:84: im = roundtrip(test_card, num_resolutions=8, progression='RLCP') failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 84, in test_prog_res_rt
    im = roundtrip(test_card, num_resolutions=8, progression='RLCP')
  File "Tests\test_file_jpeg2k.py", line 25, in roundtrip
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:92: im.load() failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 92, in test_reduce
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file
Tests\test_file_jpeg2k.py:103: im.load() failed:
Traceback (most recent call last):
  File "Tests\test_file_jpeg2k.py", line 103, in test_layers
    im.load()
  File "D:\Build\Pillow\Pillow-git\PIL\Jpeg2KImagePlugin.py", line 181, in load
    ImageFile.ImageFile.load(self)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 234, in load
    raise_ioerror(e)
  File "D:\Build\Pillow\Pillow-git\PIL\ImageFile.py", line 56, in raise_ioerror
    raise IOError(message + " when reading image file")
IOError: broken data stream when reading image file

Activity

  1. wiredfool commented on Mar 29, 2014

    @wiredfool
    wiredfool
    on Mar 29, 2014
    Member

    @al45tair? Ideas?

  2. aclark4life
    added
    Bug
    on Mar 29, 2014
  3. aclark4life
    added this to the 2.4.0 milestone on Mar 29, 2014

  4. aclark4life commented on Mar 30, 2014

    @aclark4life
    aclark4life
    on Mar 30, 2014
    Member

    FYI: release is Tuesday

  5. al45tair commented on Mar 30, 2014

    @al45tair
    al45tair
    on Mar 30, 2014
    Contributor

    I'll take a look first thing tomorrow (Monday); I suspect it's something silly and easily fixed.

  6. al45tair commented on Mar 31, 2014

    @al45tair
    al45tair
    on Mar 31, 2014
    Contributor

    OK, so I've taken a look at this, and after fixing a problem introduced by e7e103b (I'm using the OpenJPEG binaries installed using the installer from the OpenJPEG website, not building my own, so forcing it to statically link does not work), and after removing a line from the test script that tried to save a file in /tmp (my fault, shouldn't have been left in, sorry), I'm unable to reproduce this problem.

    The most likely reason for this to happen is that Pillow was built against a different version of the Microsoft C runtime than the one that Python was linked with. As a result, when Python passes an fd to the code in Pillow, it won't work properly (for those who don't already know, fds on Windows are simulated by the C runtime — Windows’ native handle type is not int; the problem is that there are multiple versions of the C runtime, and if you link together code that expects different C runtimes, it will load all of the C runtimes separately into the process, which means each of them has a separate handle table).

    Of course, I can't be certain that's the problem because I don't have the @cgohlke's build environment to look at, but it seems likely.

    To be clear: I’ve built Pillow with Visual Studio 2008 (VC9) using Python 2.7 to test this. I haven't tried Python 3 on Windows (IIRC 3.4 needs a newer version of Visual Studio, but not the latest version).

  7. cgohlke commented on Apr 1, 2014

    @cgohlke
    cgohlke
    on Apr 1, 2014
    ContributorAuthor

    Closing.

    Turns out that using openjpeg sources from svn\tags\version.2.0 instead of svn\branches\openjpeg-2.0 fixes these errors. This might come up again with the next 2.0.1 or 2.1 release.

    I assume this is the line containing the hard coded /tmp/ that should be fixed before the release?

  8. cgohlke
    closed this as completedon Apr 1, 2014

  9. al45tair commented on Apr 1, 2014

    @al45tair
    al45tair
    on Apr 1, 2014
    Contributor

    @cgohlke Yes, that's the line in question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugAny unexpected behavior, until confirmed feature.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions