Start using unittest for tests - #693

Coverage decreased (-0.47%) when pulling 34db40d on hugovk:unittest0 into 6858aef on python-pillow:master.

Coverage decreased (-0.11%) when pulling 34db40d on hugovk:unittest0 into 6858aef on python-pillow:master.

The reason coverage decreased 0.47% is because for some reason ImageGrab.py wasn't being included in the report with the old test runner. After, when being tested via unititest, it's included in the report.

Before coverage was 72.86%. After, when including ImageGrab.py, that file's 25% coverage brought the average down.

Before: https://coveralls.io/builds/835904
After: https://coveralls.io/builds/837087

(The reason for the second decrease (and failure) is an occasional Fatal RPython error with PyPy for test_imagedraw.py, meaning it's coverage didn't all get included. I've seen this problem with test_imagedraw.py on the old test runner too, so it's not directly related to using unittest.)

Coverage decreased (-0.11%) when pulling 07aa1a5 on hugovk:unittest0 into 6858aef on python-pillow:master.

@aclark4life I'll merge it and update this PR tomorrow.

Great, thanks

Coverage decreased (-0.12%) when pulling 0940f0b on hugovk:unittest0 into 8beb664 on python-pillow:master.

@aclark4life OK ready, more tests added and merged with upstream.

I'll add mores tests in another branch and create a new PR later. Still TODO:

• Caching in lena().
• Implement tempfile(). This will basically be the same as before.
• Convert the rest of the tests.

But new tests, if tempfile() isn't needed, can be created based on unittest. If not, I can convert them later.

Thanks!

So... Where's the test runner for these?

Ok, found nose. Appears broken.

(vpy27)erics@builder-1204-x64:~/Pillow$ python setup.py install
running install
running bdist_egg
running egg_info
writing Pillow.egg-info/PKG-INFO

...

Installed /home/erics/vpy27/lib/python2.7/site-packages/Pillow-2.4.0-py2.7-linux-x86_64.egg
Processing dependencies for Pillow==2.4.0
Finished processing dependencies for Pillow==2.4.0

(vpy27)erics@builder-1204-x64:~/Pillow$ nosetests 2>&1 | more
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE...........EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE.ESEEEEESEEEEE
======================================================================
ERROR: Failure: ImportError (cannot import name _imaging)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/nose/loader.py", line 414, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/erics/vpy27/local/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/erics/Pillow/Tests/cms_test.py", line 7, in <module>
    from PIL import Image
  File "/home/erics/Pillow/PIL/Image.py", line 54, in <module>
    from PIL import _imaging as core
ImportError: cannot import name _imaging
...

Also, now we've lost all the SCM context of the tests -- git blame and git log are essentially showing a bunch of brand new files.

Try python setup.py test with b7f94e3

@wiredfool How is it possible to lose context? Any idea what caused the loss of context? Maybe @hugovk can look into it…

Ok, python setup.py test works.

I think that the context loss is basically due to the combination of moving the files and changing all of the lines at the same time due to hoisting them into classes.

I'm not good enough at git to know if there's a good solution, but it would be nice if there was a way to connect the history of the files together.

Ah I see, that is annoying

To run the an individual test, use either:

• python test/test_something.py
• nosetests test/test_something.py

To run the whole batch (e.g. like on Travis CI) use nosetests:

• No coverage: nosetests test/
• With coverage: coverage run --append --include=PIL/* -m nose test/

All these can take a -v verbose flag at the end.

Re: history

I did git mv, update the test, then git add and git commit so I'd hoped history was preserved, but it appears not.

Git does have some rename support, and there's a git log --follow option, but that's not showing the history.

Here's an idea:

If you want to back up this PR out of master (there's been no unrelated commits afterwards) then when I have all the tests converted, I can make a fresh PR from a fresh branch by just replacing the files and leaving them in the Tests folder with no git mv.

Current status:

Branch: https://github.com/hugovk/Pillow/tree/unittest2tempfile

I'm almost done converting all the files, just have the last one to fix. test_cffi.py fails after conversion:

ERROR: test_get_vs_c (test_cffi.TestCffi)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/hugovk/Pillow/test/test_cffi.py", line 64, in test_get_vs_c
    self._test_get_access(lena('RGB'))
  File "/home/travis/build/hugovk/Pillow/test/test_cffi.py", line 55, in _test_get_access
    caccess = im.im.pixel_access(False)
AttributeError: 'NoneType' object has no attribute 'pixel_access'

It seems this is because the original lena() caches files and an earlier test modifies the image so it has an im.im. The converted test has caching disabled for now and always returns a freshly opened file (specifically to make sure tests are independent and repeatable) whose im.im is None.

(Well, there's one more I've not yet done -- test_001_archive.py -- but I'll remove it because it's going through a list of files in ../pil-archive/* and that directory doesn't exist.)

How far back should I go, just: 001b46c or further?

@aclark4life To the one just before that, so 8beb664 will be the latest.

I've made a new branch based on 8beb664 and am adding the tests in with fresh commits so they remain in the Tests/ directory with preserved history.

OK, new branch is ready with preserved history.
https://github.com/hugovk/Pillow/tree/unittest3
https://travis-ci.org/hugovk/Pillow/builds/27206742
e.g. https://github.com/hugovk/Pillow/commits/unittest3/Tests/test_imagecms.py

I'll create a new PR.

K, working on the revert(s)

@hugovk Let me know how the revert looks and if OK I'll merge your new PR

Looks like PyPy testing failed on Travis but we can probably fix that up

@aclark4life PR here: #699

@hugovk Thanks, so my revert looks OK?

@aclark4life Yes, all merged smoothly.

Thanks, done!