Implement --only-docs for testing #2048

Activity

1. eteq commented on Feb 5, 2014

   

   eteq

   on Feb 5, 2014

   Member

   More actions

   By "test only docs" you mean the doctests, right, @astrofrog? I definitely agree that's a good idea. Sort of the "anti-#2004 "...
2. astrofrog commented on Feb 5, 2014

   

   astrofrog

   on Feb 5, 2014

   MemberAuthor

   More actions

   @eteq - yeah, the tests in docs (not the tests in docstrings which should still be tested by the -t option)
3. mdboom commented on Feb 5, 2014

   

   mdboom

   on Feb 5, 2014

   Contributor

   More actions

   Not sure I follow:

   currently if I work on say astropy.modeling and want to check the docs tests I have to run the full test suite.

   > python setup.py test -P modeling
   collected 249 items / 1 skipped 
   
   astropy/modeling/core.py ...
   astropy/modeling/functional_models.py ..
   astropy/modeling/tests/test_constraints.py ssssssss..........
   astropy/modeling/tests/test_core.py .....
   astropy/modeling/tests/test_fitters.py ..s..sss.ssss
   astropy/modeling/tests/test_functional_models.py .
   astropy/modeling/tests/test_input.py ...............ssss........
   astropy/modeling/tests/test_models.py .........s..............................ssssssssssssss..s.........s.......ssssssssssssssssssssssssssssssssss...
   astropy/modeling/tests/test_parameters.py ...........................
   astropy/modeling/tests/test_polynomial.py ssssssssssssss
   astropy/modeling/tests/test_projections.py .................
   astropy/modeling/tests/test_rotations.py ....
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/algorithms.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/design.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/fitting.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/index.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/models.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/new.rst .
   ../../../home/mdboom/Work/builds/astropy/docs/modeling/parameters.rst .
   
4. eteq commented on Feb 5, 2014

   

   eteq

   on Feb 5, 2014

   Member

   More actions

   @mdboom - I think he wants to be able to do that, but not have it run the actual package tests - only the things in docs. And also be able to do all the docs doctests, but without any of the package tests.
5. mdboom commented on Feb 5, 2014

   

   mdboom

   on Feb 5, 2014

   Contributor

   More actions

   I just think we should have a strong argument for this feature -- our test framework is so flexible at this point it's easy to miss things if not running in particular modes etc. I'd prefer a test framework with as few operating modes as possible.
6. astrofrog commented on Feb 5, 2014

   

   astrofrog

   on Feb 5, 2014

   MemberAuthor

   More actions

   For me the motivation is that if I develop something in astropy.modeling, I can run and re-run the tests in a matter of seconds, which makes it fast to develop. If I change something in docs, I have to wait for 5 minutes for all the tests to run. Maybe, rather than add options, we could simply make it that:

   python setup.py test -t docs/modeling
   

   would work - that is, we aren't adding any additional modes, just making the above work.
7. mdboom commented on Feb 5, 2014

   

   mdboom

   on Feb 5, 2014

   Contributor

   More actions

   I guess I still don't understand: you don't have to run all of the doctests -- you can run only the ones in modeling (along with all of the unit tests in modeling). I think breaking up by package like this is much safer and more natural with respect to likelihood of interdependencies.

   I'm not opposed to making -t work for .rst files, though.
8. astrofrog commented on Feb 5, 2014

   

   astrofrog

   on Feb 5, 2014

   MemberAuthor

   More actions

   @mdboom - just to be clear, how do I run only the tests in the RST files indocs/modeling at the moment? I agree I can run the doctests in astropy/modeling, but not those in docs/modeling.
9. mdboom commented on Feb 5, 2014

   

   mdboom

   on Feb 5, 2014

   Contributor

   More actions

   If you run setup.py test -P modeling, it will run the Python tests, the Python doctests and the .rst doctests in the modeling package only. You don't need to run the entire test suite to run the .rst doctests.
10. astrofrog commented on Feb 5, 2014

    

    astrofrog

    on Feb 5, 2014

    MemberAuthor

    More actions

    @mdboom - Oh, I didn't know about that! You have proven your point that we already have too many options ;) In that case, I rescind my original suggestion.
11. astrofrog

    closed this as completedon Feb 5, 2014
12. mdboom commented on Feb 5, 2014

    

    mdboom

    on Feb 5, 2014

    Contributor

    More actions

    No worries. I still created #2050 so we can address making -t work for .rst files.