Repository navigation
Sun Raster files with RLE encoding and depth 1 fail to load #2231
Description
Activity
This is an error that's triggered from within the decoder, where it's being requested to write more into a buffer than remaining space in that buffer. It's checking, failing, and returning an error message.
RLE is run length encoding, in this case there's a marker for RLE, then a byte of how many, then a byte for the value. e.g.
0x80 0xff 0x00is the code for 255 zerosWhat's going on here, at least initially, is that the Sun RLE decoder is expecting that when you're unpacking 255 items into a buffer, that there's at least 255 bytes of space in that buffer.
if (state->x + n > state->bytes) { /* FIXME: is this correct? */ state->errcode = IMAGING_CODEC_OVERRUN; return -1; }This is in general, a correct requirement, but the buffer that's getting passed in is width/8 since it's a 1 bit buffer. So, 80 bytes. Hence the overflow.
Modifying that so that we're expecting a byte per pixel buffer, we get past the first error, then on the third RLE bit, it's requesting another 255 items, but the width is 640, so there's another overflow.
So. Time to go to the RLE docs. http://www.fileformat.info/format/sunraster/egff.htm
Note also that the Sun Raster bitmap is read as if it is a single stream of data. Therefore, the encoding of pixel runs does not stop at the end of each scan line.
Also, from the same page:
Bitmap files with a Depth of 1 contain 2-color image data. [...] Each bit in the bitmap represents a pixel, with a value of 0 representing black and a value of 1 representing white (the bits are stored most significant bit first within each byte).
So, the above guess that we were dealing with a byte/bit confusion is wrong, but what we need to do is wrap the scan lines when writing. So, if we've got an 80 byte buffer, and we're requesting a 255 byte write, it's
xxxxxxxx (80) xxxxxxxx (80) xxxxxxxx (80) x (15)As an aside, this code is 2 months shy of 20 years old, and the header at the top says:
/* * THIS IS WORK IN PROGRESS *Sigh.
Reacted by Kit SundeFWIW, I'm getting this from imagemagick:
display: unexpected end-of-file `sunraster.im1': @ error/sun.c/ReadSUNImage/568.I've got it parsing now, but it still looks like there's a bug somewhere. And we're definitely hitting some sort of parsing error towards the end, it's reporting a truncated file but we're not reading the last 178 bytes.
I'm not sure about that file in particular, but I'm also pretty sure that SunRLE has never worked. Fixing the multiple issues I've found where the parsing didn't match the spec, and I've got something that's losing a few bytes per row:
So there's still one more bug somewhere in this. (at least)
fwiw, target is:
Code is at https://github.com/wiredfool/Pillow/tree/sunrle
And finally, the testing code only tests open, not the load. And the load fails.
And now I've got it matching, and coming up with exactly the right number of bytes.
I think it's actually a GIMP bug where they've got an off by one error in the encoding:
from https://git.gnome.org/browse/gimp/tree/plug-ins/common/file-sunras.c
/* Write uncompressed character to RLE-stream */ static gint rle_fputc (gint val, FILE *ofp) { ... if (rlebuf.val == val) /* Same value in the buffer ? */ { (rlebuf.n)++; if (rlebuf.n == 257) /* Can not be encoded in a single run ? */ { retval = rle_putrun (256, rlebuf.val, ofp); // Attempting to write a run of 256?? if (retval < 0) return retval; rlebuf.n -= 256; } return val; }and slightly later:
/* Write out a run with 0 < n < 257 */ static gint rle_putrun (gint n, gint val, FILE *ofp) { int retval, flag = 0x80; /* Useful to write a 3 byte run ? */ if ((n > 2) || ((n == 2) && (val == flag))) { putc (flag, ofp); putc (n-1, ofp); // here we subtract 1 from the run to fit in UINT8 retval = putc (val, ofp); }Where the specifications that I've found clearly state: (http://www.fileformat.info/format/sunraster/egff.htm)
For example, a run of 100 pixels with the value of 0Ah would encode as the values 80h 64h 0Ah. A single pixel value of 80h would encode as the values 80h 00h. The four unencoded bytes 12345678h would be stored in the RLE stream as 12h 34h 56h 78h.
100 pixels, n=100, not 100 pixels, n=99.
But Wait! There's More! (http://www.fileformat.info/format/sunraster/spec/598a59c4fac64c52897585d390d86360/view.htm)
If the first byte is 0x80, and the second byte is not zero, the
record is three bytes long. The second byte is a count and the
third byte is a value. Output (count+1) pixels of that value.The nice thing about specifications is that there's so many to choose from.
Where the specifications that I've found clearly state: (http://www.fileformat.info/format/sunraster/egff.htm)
For example, a run of 100 pixels with the value of 0Ah would encode as the values 80h 64h 0Ah. A single >>pixel value of 80h would encode as the values 80h 00h. The four unencoded bytes 12345678h would be >>stored in the RLE stream as 12h 34h 56h 78h.
From what I've seen in other Sun Raster files, the above specification is incorrect and the count byte really means you have n+1 of the values, which is what your second specification says.
But Wait! There's More! (http://www.fileformat.info/format/sunraster/spec/598a59c4fac64c52897585d390d86360/view.htm)
If the first byte is 0x80, and the second byte is not zero, the
record is three bytes long. The second byte is a count and the
third byte is a value. Output (count+1) pixels of that value.This is the rule I used when writing out Sun Raster files in a project I worked on and the old SunOS systems from 20 years ago were able to read them correctly.
I agree. At least, imagemagick and gimp both use the n+1 standard from inspecting their code.
What did you do?
Tried to open a 1-bit RLE encoded Sun Raster file
What did you expect to happen?
Expected image to load
What actually happened?
Got a 'buffer overrun when reading image file' error
What versions of Pillow and Python are you using?
Python 2.7.8, Pillow 3.3.1
Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, django, or buildout, try to replicate the issue just using Pillow.
Use image in attached ZIP file or create a new image in GIMP, 640x400 with white background. Select black brush, draw a line, convert Image->Mode to Indexed, and select black and white (1-bit) palette. Save image as Sun Raster file with RLE encoding.
sunraster.zip