Repository navigation
Overly verbose test result output #4164
Description
A minor annoyance I'm finding since the switch to pytest 2.7.3 in #4027 is that during test running the results for each module list the full path to the module, as found it the temporary test directory, rather than a relative path, like:
//tmp/astropy-test-k8ciiv35/lib.linux-x86_64-3.4/astropy/_erfa/tests/test_erfa.py .....
//tmp/astropy-test-k8ciiv35/lib.linux-x86_64-3.4/astropy/analytic_functions/tests/test_blackbody.py ....
//tmp/astropy-test-k8ciiv35/lib.linux-x86_64-3.4/astropy/config/tests/test_configs.py .................
//tmp/astropy-test-k8ciiv35/lib.linux-x86_64-3.4/astropy/constants/tests/test_constant.py ........
...
as opposed to
astropy/_erfa/tests/test_erfa.py .....
astropy/analytic_functions/tests/test_blackbody.py ....
astropy/config/tests/test_configs.py .................
astropy/constants/tests/test_constant.py ........
...
The double slash at the beginning is odd too. It might in some way be related to a new line showing up at the beginning of the test output that wasn't there before:
rootdir: /, inifile:
Activity
embray commentedon Sep 17, 2015
[If this can't be fixed expediently we can change the label to Affects-release and fix it later, but I'm going to take a quick look]
embray commentedon Sep 17, 2015
This doesn't seem to be a problem when I run
astropy.test()from the interpreter, in the source checkout. In that case it shows:so for whatever reason I think it's rootdir that doesn't get found correctly when we do the copy into the temp build directory.
(Note: astropy/astropy above refers to the root of the astropy git checkout, not the astropy directory inside the git checkout (I also have astropy/astropy-helpers astropy/astropy-tools, etc.).
embray commentedon Sep 17, 2015
Ah, this explains it: https://pytest.org/latest/customize.html#initialization-determining-rootdir-and-inifile
When we run
./setup.py testit ends up sending args like:to
pytest.main()It should still find
/path/to/my/home/src/astropy/astropyas the common ancestor directory, but it fails (and ends up finding/as the rootdir) due to the use of/tmp. I think we should just make the temp dir relative to the source checkout instead of/tmp.embray commentedon Sep 17, 2015
This is a change that needs to be made in astropy_helpers it seems.
mdboom commentedon Sep 17, 2015
Ah -- good find. I was so focused on just getting the tests to pass I didn't even notice!
mdboom commentedon Sep 17, 2015
If we make the temp dir relative to the source checkout, won't that clutter the source checkout? Kind of annoying, and why I'd prefer using
/tmp. (Plus, on most modern Linuxes/tmpis a ramdisk so it's blazingly fast). But I can't think of any better solution to the output problem -- we could tell pytest what the rootdir is, but we basically have two roots (one for the code and one for the docs). We could copy the docs to the tempdir as well and test them from there, I guess...embray commentedon Sep 18, 2015
It shouldn't clutter the source checkout because the temp dir is deleted at the end of the test run, though it could still be a problem with random crashes, etc. Though usually that isn't even a problem, since the tests are run in a subprocess. So as long as the subprocess doesn't take down the parent process as well the temp dir still gets deleted. I experimented with this a bit and had a hard time getting it to stick around. So no, clutter not a problem.
The ramdisk issue is a better point though. That's in part why I left this configurable. That said, let me dig around a bit more in pytest and see if there's a way to force a particular "rootdir" on it. I'll also experiment with just copying the docs as well :/
mhvk commentedon Sep 18, 2015
Would a symbolic link to
/tmp/from the source checkout solve this?18 remaining items
embray commentedon Sep 29, 2015
As @bsipocz points out this is still broken on OSX #4194 (comment)
If you look at the build log for #4194, on Linux everything looks fine, but not on the OSX build: https://travis-ci.org/astropy/astropy/jobs/82570188
I don't know why this would be any different on OSX, but it appears to find different test paths there.
embray commentedon Sep 30, 2015
Turns out the reason we run into problems on OSX is that
tempfile.tempdiron OSX returns a path, by default, under/var. But on OSX/varis usually a symlink to/private/var. However,tempfile.tempdirdoesn't return the real full path with the symlink derferenced. Later, this gets compared to the full path by py.test, and it fails to find the correct common rootdir for the tests.This is kinda a bug in py.test too, since when finding the common rootdir for some test paths it should take symlinks into account better.
embray commentedon Sep 30, 2015
(Didn't mean to close.)
Update to astropy_helpers that should fix astropy#4164 on OSX. The ch…
Update to astropy_helpers that should fix astropy#4164 on OSX. The ch…
Merge pull request #4204 from embray/testing/4164-2
Merge pull request #4178 from embray/testing/4164
Update to astropy-helpers version to incorporate the fixes from astro…
Update to astropy_helpers that should fix astropy#4164 on OSX. The ch…
Update to astropy-helpers version to incorporate the fixes from astro…
Update to astropy_helpers that should fix astropy#4164 on OSX. The ch…