Repository navigation
Overly verbose test result output #4164
Description
Activity
[If this can't be fixed expediently we can change the label to Affects-release and fix it later, but I'm going to take a quick look]
This doesn't seem to be a problem when I run
astropy.test()from the interpreter, in the source checkout. In that case it shows:rootdir: /path/to/my/home/src/astropy/astropy, inifile: setup.cfgso for whatever reason I think it's rootdir that doesn't get found correctly when we do the copy into the temp build directory.
(Note: astropy/astropy above refers to the root of the astropy git checkout, not the astropy directory inside the git checkout (I also have astropy/astropy-helpers astropy/astropy-tools, etc.).
Ah, this explains it: https://pytest.org/latest/customize.html#initialization-determining-rootdir-and-inifile
When we run
./setup.py testit ends up sending args like:['/tmp/astropy-test-k8ciiv35/lib.linux-x86_64-3.4/astropy', '/path/to/my/home/src/astropy/astropy/docs', '--doctest-rst']
to
pytest.main()It should still find
/path/to/my/home/src/astropy/astropyas the common ancestor directory, but it fails (and ends up finding/as the rootdir) due to the use of/tmp. I think we should just make the temp dir relative to the source checkout instead of/tmp.This is a change that needs to be made in astropy_helpers it seems.
Ah -- good find. I was so focused on just getting the tests to pass I didn't even notice!
If we make the temp dir relative to the source checkout, won't that clutter the source checkout? Kind of annoying, and why I'd prefer using
/tmp. (Plus, on most modern Linuxes/tmpis a ramdisk so it's blazingly fast). But I can't think of any better solution to the output problem -- we could tell pytest what the rootdir is, but we basically have two roots (one for the code and one for the docs). We could copy the docs to the tempdir as well and test them from there, I guess...It shouldn't clutter the source checkout because the temp dir is deleted at the end of the test run, though it could still be a problem with random crashes, etc. Though usually that isn't even a problem, since the tests are run in a subprocess. So as long as the subprocess doesn't take down the parent process as well the temp dir still gets deleted. I experimented with this a bit and had a hard time getting it to stick around. So no, clutter not a problem.
The ramdisk issue is a better point though. That's in part why I left this configurable. That said, let me dig around a bit more in pytest and see if there's a way to force a particular "rootdir" on it. I'll also experiment with just copying the docs as well :/
Would a symbolic link to
/tmp/from the source checkout solve this?18 remaining items
As @bsipocz points out this is still broken on OSX #4194 (comment)
If you look at the build log for #4194, on Linux everything looks fine, but not on the OSX build: https://travis-ci.org/astropy/astropy/jobs/82570188
I don't know why this would be any different on OSX, but it appears to find different test paths there.
Turns out the reason we run into problems on OSX is that
tempfile.tempdiron OSX returns a path, by default, under/var. But on OSX/varis usually a symlink to/private/var. However,tempfile.tempdirdoesn't return the real full path with the symlink derferenced. Later, this gets compared to the full path by py.test, and it fails to find the correct common rootdir for the tests.This is kinda a bug in py.test too, since when finding the common rootdir for some test paths it should take symlinks into account better.
(Didn't mean to close.)
A minor annoyance I'm finding since the switch to pytest 2.7.3 in #4027 is that during test running the results for each module list the full path to the module, as found it the temporary test directory, rather than a relative path, like:
as opposed to
The double slash at the beginning is odd too. It might in some way be related to a new line showing up at the beginning of the test output that wasn't there before: