Skip to content

Running tests directly from pytest is broken聽#6424

Description

@drdavella

Sorry to add to the stack of testing issues 馃槩 .

First of all, is this even technically supported? Issue #5402 suggests that I'm not the only person who has expected this to work, but I'm guessing I'm the only person who has tried it in a long time. If it is supported, it could be a potential solution to #6392 since all pytest functionality is available directly from the pytest interface.

Here's what I get when I try to run pytest (using version 3.1.3 for the record):

../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:336: in _getconftestmodules
    return self._path2confmods[path]
E   KeyError: local('/home/ddavella/astropy/astropy')

During handling of the above exception, another exception occurred:
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:367: in _importconftest
    return self._conftestpath2mod[conftestpath]
E   KeyError: local('/home/ddavella/astropy/astropy/conftest.py')

During handling of the above exception, another exception occurred:
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/py/_path/common.py:372: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/py/_path/common.py:410: in gen
    dirs = self.optsort([p for p in entries
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/py/_path/common.py:411: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/main.py:686: in _recurse
    ihook = self.gethookproxy(path)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/main.py:590: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:350: in _getconftestmodules
    mod = self._importconftest(conftestpath)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:386: in _importconftest
    self.consider_conftest(mod)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:409: in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:264: in register
    self.consider_module(plugin)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:415: in consider_module
    self._import_plugin_specs(getattr(mod, 'pytest_plugins', []))
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:420: in _import_plugin_specs
    self.import_plugin(import_spec)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:452: in import_plugin
    self.register(mod, modname)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/config.py:258: in register
    ret = super(PytestPluginManager, self).register(plugin, name)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:371: in register
    hook._maybe_apply_history(hookimpl)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:768: in _maybe_apply_history
    res = self._hookexec(self, [method], kwargs)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
../miniconda3/envs/astropy-pytest/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
    res = hook_impl.function(*args)
astropy/tests/pytest_doctestplus.py:187: in pytest_configure
    config.getini('doctest_rst') or config.option.doctest_rst),
E   AttributeError: 'CmdOptions' object has no attribute 'doctest_rst'

I can patch this by making a quick fix to lines 185-187 in astropy/tests/pytest_doctestplus.py:

    config.pluginmanager.register(                                              
         DoctestPlus(DocTestModulePlus, DocTestTextfilePlus, False),             
                     #config.getini('doctest_rst') or config.option.doctest_rst),
         'doctestplus') 

After this, the tests will run, but they cause hundreds of errors and warnings, and also seem to want to use all of the memory on my machine.

Activity

  1. changed the title [-]Running tests directly from pytests is broken[/-] [+]Running tests directly from pytest is broken[/+] on Aug 4, 2017
  2. drdavella commented on Aug 8, 2017

    @drdavella
    ContributorAuthor

    Well, I think I've convinced myself that this is actually a duplicate of #5402.

    I think I've also convinced myself that this is just not the way that pytest expects to be used, which is really unfortunate. It seems like doing something perfectly reasonable like running pytest from the top-level directory should not cause completely meaningless and baffling errors.

    From the top-level directory, running pytest astropy seems to work fine (although I have doubts about whether options like --doctest-rst are working as expected).

    This can probably be closed but I'd really like to try to address these issues if/when we manage to refactor the test stuff into a separate package.

  3. pllim commented on Aug 8, 2017

    @pllim
    Member

    If you have plans to address this in the future, we should keep this open for now.

  4. drdavella commented on Aug 8, 2017

    @drdavella
    ContributorAuthor

    I'm a little bit skeptical about whether it is actually addressable given the constraints imposed by pytest, but I think it's at least worth looking at again.

  5. saimn commented on Aug 29, 2017

    @saimn
    Contributor

    I sometimes use pytest directly, but it doesn't always work. I think it works only if you specify a directory or test file, to avoid the issues with plugins configuration. So I would be very happy if this can fixed ! Also concerning use all of the memory on my machine, I noticed and fixed a similar issue not so long ago (#6228).

  6. drdavella commented on Dec 6, 2017

    @drdavella
    ContributorAuthor

    This was fixed as a consequence of #6606 and can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions