Repository navigation
Feedback about --import-mode=importlib in pytest 6.0 #7245
Description
Activity
- added 2 commits that reference this issue
on May 23, 2020 - added 3 commits that reference this issue
on Jun 7, 2020 It looks like the
pytest-bddplugin is incompatible with--import-mode=importlib:_______________________________ ERROR collecting tests/end2end/features/test_backforward_bdd.py _______________________________ tests/end2end/features/test_backforward_bdd.py:21: in <module> bdd.scenarios('backforward.feature') .tox/py38-pyqt515/lib/python3.8/site-packages/pytest_bdd/scenario.py:296: in scenarios features_base_dir = get_features_base_dir(module) .tox/py38-pyqt515/lib/python3.8/site-packages/pytest_bdd/scenario.py:241: in get_features_base_dir default_base_dir = os.path.dirname(caller_module.__file__) E AttributeError: 'NoneType' object has no attribute '__file__'def scenarios(*feature_paths, **kwargs): frame = inspect.stack()[1] module = inspect.getmodule(frame[0]) # <- this is None features_base_dir = kwargs.get("features_base_dir") if features_base_dir is None: features_base_dir = get_features_base_dir(module)
Any idea why this could be happening?
I think
bdd.scenariosassumes the module given as arguments are available insys.path, which is not the case when--import-mode=importlib.Perhaps this will work (untested):
module = inspect.getmodule(frame[0]) + if module is None: + module = __import__(frame[0])
Maybe this issue should be transfered here?
Thanks @schperplata, but it seems that's a different issue however (I replied there).
refs: pytest-dev/py#203 (comment)
for me relative imports are broken when using
--import-mode=importlibhttps://github.com/graingert/relative-imports-import-mode-importlib/blob/default/tests/test_foo.py#L1============================================================================================================================== test session starts =============================================================================================================================== platform linux -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 rootdir: /home/graingert/projects/relative-imports-import-mode-importlib, configfile: pytest.ini plugins: forked-1.3.0, cov-2.9.0, xdist-1.34.0, case-1.5.3, timeout-1.4.2 collected 0 items / 1 error ===================================================================================================================================== ERRORS ===================================================================================================================================== _______________________________________________________________________________________________________________________ ERROR collecting tests/test_foo.py _______________________________________________________________________________________________________________________ ImportError while importing test module '/home/graingert/projects/relative-imports-import-mode-importlib/tests/test_foo.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: tests/test_foo.py:1: in <module> from . import utils E ImportError: attempted relative import with no known parent packagewhen doing inline tests, I often use relative imports:
ImportError while importing test module '/home/graingert/projects/relative-imports-import-mode-importlib/graingertspkg/tests/test_business_logic.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: graingertspkg/tests/test_business_logic.py:1: in <module> from .. import business_logic E ImportError: attempted relative import with no known parent packageReacted by Arya Massaratfor me relative imports are broken
Not sure they are broken, imports using
importlibby design won't add modules tosys.modulesor changesys.path, which explains why it doesn't work for your use case.This however brings an important point, that
importlibis not recommended for tests included with package code if one wants to use relative imports.We should consider that as documentation, and also weight this use case in an eventual future decision of changing the default
--importmode.@nicoddemus would an
--importmode=pkgutilthat usespkgutil.resolve_namefor tests work with relative imports?Don't know, but should be easy to try, the logic is implemented in:
Line 445 in 91dbdb6
But one question is: what are you trying to fix by using a different import mode, and would that be fixed by using
pkgutil.resolve_name?I guess something like this, would work for inline tests:
if mode is ImportMode.append: if str(pkg_root) not in sys.path: sys.path.append(str(pkg_root)) elif mode is ImportMode.prepend: if str(pkg_root) != sys.path[0]: sys.path.insert(0, str(pkg_root)) elif mode is ImportMode.manual: return importlib.import_module(module_name) importlib.import_module(module_name)
44 remaining items
So how to allow it to feed plugins without importing the file?
Hmm sorry I'm not sure I understand the relation to that regarding
--import-mode...@bennyrowland I'm not a Pytest dev, but as a basic check, you might want to confirm that the problem disappears by just changing the
import-mode? If so, I imagine more precise detail on the exact structure of your project, both tests and code under test, and the full traceback and error message, would be helpful (and ideally, a minimal example repo).And yeah, I've never really understood the desire for running any of these static checks with Pytest (as opposed to Pre-Commit if you want a check runner and/or tox/nox/etc. if you want unified orchestration), just like you wouldn't run Pytest under Pre-Commit; there's a pretty clear conceptual and practical separation between the needs and goals of static checks vs dynamic tests. But I digress...
@The-Compiler, to be clear, these plugins are not my project, I am just a user. There are many plugins like: pytest-flake8, pytest-flakes, pytest-pylint, pytest-black, pytest-mypy etc. that all work in this way currently, and I understand are used by a good number of people. Changing to --import-mode=importlib will break all those tools for any repo that uses relative imports.
@CAM-Gerlach: I don't think that in this case an example repo is necessary - the code I provided in my original post is all that is necessary to break any of the above tools. Explicitly if pytest uses importlib to try to import a single source file that uses a relative import, it will fail by design. @nicoddemus points out earlier in this thread that that is (probably) correct behaviour for the test folder, but not for the main source tree. You are right that there are (possibly better) alternative ways to run these checkers and linters: I have found doing things this way to be convenient, but I will look into replacing things with pre-commit (which I do already use) and tox/nox (which I keep meaning to). However, all these plugin tools have worked correctly for a long time, a lot of people are using them and like the paradigm (as new plugins keep on proliferating for additional tools), it seems quite tough to just break the whole approach.
@nicoddemus, I think what I meant was this: using --import-mode=importlib is going to make importing source files for many packages impossible when they worked ok before. However, the tests that are being run on the source files (like mypy, pylint etc.) do not actually require those files to be imported, just passed to the external tools for processing. Therefore it would be possible to make all these plugins compatible with --importlib if the plugins could collect their test lists without the files having to be imported at all. I suspect that this may even be possible already, although none of the plugins I use regularly, or the others that I tried in encountering this issue worked in this case. So my question for you (and anybody else who can help) is whether this is something that needs to be fixed on the plugin side or on the pytest side in order for this to "just work" again.
can you provide a minimal example? I'm unable to reproduce
@asottile here is an example repo: https://github.com/bennyrowland/pytest_plugin_example. Python 3.9 in a brand new env, install requirements.txt (only pytest-mypy) and run
pytest --mypy --import-mode=importliband I get the following tracebackH:\Python\pytest_plugin_example>pytest --import-mode=importlib --mypy ======================================================================================================== test session starts ======================================================================================================== platform win32 -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 rootdir: H:\Python\pytest_plugin_example plugins: mypy-0.8.1 collected 3 items src\example\__init__.py .. [ 66%] src\example\example.py E [100%] ============================================================================================================== ERRORS =============================================================================================================== _____________________________________________________________________________________________ ERROR at setup of src\example\example.py ______________________________________________________________________________________________ self = <Package example> def _importtestmodule(self): # We assume we are only called once per module. importmode = self.config.getoption("--import-mode") try: > mod = import_path(self.fspath, mode=importmode) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\python.py:578: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ p = local('H:\\Python\\pytest_plugin_example\\src\\example\\__init__.py') def import_path( p: Union[str, py.path.local, Path], *, mode: Union[str, ImportMode] = ImportMode.prepend, ) -> ModuleType: """Import and return a module from the given path, which can be a file (a module) or a directory (a package). The import mechanism used is controlled by the `mode` parameter: * `mode == ImportMode.prepend`: the directory containing the module (or package, taking `__init__.py` files into account) will be put at the *start* of `sys.path` before being imported with `__import__. * `mode == ImportMode.append`: same as `prepend`, but the directory will be appended to the end of `sys.path`, if not already in `sys.path`. * `mode == ImportMode.importlib`: uses more fine control mechanisms provided by `importlib` to import the module, which avoids having to use `__import__` and muck with `sys.path` at all. It effectively allows having same-named test modules in different places. :raises ImportPathMismatchError: If after importing the given `path` and the module `__file__` are different. Only raised in `prepend` and `append` modes. """ mode = ImportMode(mode) path = Path(str(p)) if not path.exists(): raise ImportError(path) if mode is ImportMode.importlib: module_name = path.stem for meta_importer in sys.meta_path: spec = meta_importer.find_spec(module_name, [str(path.parent)]) if spec is not None: break else: spec = importlib.util.spec_from_file_location(module_name, str(path)) if spec is None: raise ImportError( "Can't find module {} at location {}".format(module_name, str(path)) ) mod = importlib.util.module_from_spec(spec) > spec.loader.exec_module(mod) # type: ignore[union-attr] C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\pathlib.py:498: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <_frozen_importlib_external.SourceFileLoader object at 0x03A3F430>, module = <module '__init__' from 'H:\\Python\\pytest_plugin_example\\src\\example\\__init__.py'> > ??? <frozen importlib._bootstrap_external>:850: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ f = <built-in function exec> args = (<code object <module> at 0x03A69840, file "H:\Python\pytest_plugin_example\src\example\__init__.py", line 1>, {'__bui...t__.cpython-39.pyc', '__doc__': None, '__file__': 'H:\\Python\\pytest_plugin_example\\src\\example\\__ init__.py', ...}) kwds = {} > ??? <frozen importlib._bootstrap>:228: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > from . import example E ImportError: attempted relative import with no known parent package src\example\__init__.py:1: ImportError The above exception was the direct cause of the following exception: cls = <class '_pytest.runner.CallInfo'>, func = <function call_runtest_hook.<locals>.<lambda> at 0x034B6898>, when = 'setup', reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>) @classmethod def from_call( cls, func: "Callable[[], TResult]", when: "Literal['collect', 'setup', 'call', 'teardown']", reraise: Optional[ Union[Type[BaseException], Tuple[Type[BaseException], ...]] ] = None, ) -> "CallInfo[TResult]": excinfo = None start = timing.time() precise_start = timing.perf_counter() try: > result: Optional[TResult] = func() C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\runner.py:311: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > lambda: ihook(item=item, **kwds), when=when, reraise=reraise ) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\runner.py:255: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <_HookCaller 'pytest_runtest_setup'>, args = (), kwargs = {'item': <MypyFileItem mypy>}, argname = 'item', firstresult = False def __call__(self, *args, **kwargs): if args: raise TypeError("hook calling supports only keyword arguments") assert not self.is_historic() # This is written to avoid expensive operations when not needed. if self.spec: for argname in self.spec.argnames: if argname not in kwargs: notincall = tuple(set(self.spec.argnames) - kwargs.keys()) warnings.warn( "Argument(s) {} which are declared in the hookspec " "can not be found in this hook call".format(notincall), stacklevel=2, ) break firstresult = self.spec.opts.get("firstresult") else: firstresult = False > return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\pluggy\_hooks.py:265: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <_pytest.config.PytestPluginManager object at 0x0158E640>, hook_name = 'pytest_runtest_setup' methods = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from 'C:\\Users\\Hallmarq.BEN-LAPTOP\\anaconda3\\envs\\sy...ture=None>>, <HookImpl plugin_name='logging-plugin', plugin=<_pytest.logging.LoggingPlugin object a t 0x033BA448>>, ...] kwargs = {'item': <MypyFileItem mypy>}, firstresult = False def _hookexec(self, hook_name, methods, kwargs, firstresult): # called from all hookcaller instances. # enable_tracing will set its own wrapping function at self._inner_hookexec > return self._inner_hookexec(hook_name, methods, kwargs, firstresult) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\pluggy\_manager.py:80: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hook_name = 'pytest_runtest_setup' hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from 'C:\\Users\\Hallmarq.BEN-LAPTOP\\anaconda3\\envs\\sy...ture=None>>, <HookImpl plugin_name='logging-plugin', plugin=<_pytest.logging.LoggingPlugin objec t at 0x033BA448>>, ...] caller_kwargs = {'item': <MypyFileItem mypy>}, firstresult = False def _multicall(hook_name, hook_impls, caller_kwargs, firstresult): """Execute a call into multiple python functions/methods and return the result(s). ``caller_kwargs`` comes from _HookCaller.__call__(). """ __tracebackhide__ = True results = [] excinfo = None try: # run impl and wrapper setup functions in a loop teardowns = [] try: for hook_impl in reversed(hook_impls): try: args = [caller_kwargs[argname] for argname in hook_impl.argnames] except KeyError: for argname in hook_impl.argnames: if argname not in caller_kwargs: raise HookCallError( f"hook call must provide argument {argname!r}" ) if hook_impl.hookwrapper: try: gen = hook_impl.function(*args) next(gen) # first yield teardowns.append(gen) except StopIteration: _raise_wrapfail(gen, "did not yield") else: res = hook_impl.function(*args) if res is not None: results.append(res) if firstresult: # halt further impl calls break except BaseException: excinfo = sys.exc_info() finally: if firstresult: # first result hooks return a single value outcome = _Result(results[0] if results else None, excinfo) else: outcome = _Result(results, excinfo) # run all wrapper post-yield blocks for gen in reversed(teardowns): try: gen.send(outcome) _raise_wrapfail(gen, "has second yield") except StopIteration: pass > return outcome.get_result() C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\pluggy\_callers.py:60: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <pluggy._result._Result object at 0x034AC940> def get_result(self): """Get the result(s) for this hook call. If the hook was marked as a ``firstresult`` only a single value will be returned otherwise a list of results. """ __tracebackhide__ = True if self._excinfo is None: return self._result else: ex = self._excinfo > raise ex[1].with_traceback(ex[2]) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\pluggy\_result.py:60: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ hook_name = 'pytest_runtest_setup' hook_impls = [<HookImpl plugin_name='nose', plugin=<module '_pytest.nose' from 'C:\\Users\\Hallmarq.BEN-LAPTOP\\anaconda3\\envs\\sy...ture=None>>, <HookImpl plugin_name='logging-plugin', plugin=<_pytest.logging.LoggingPlugin objec t at 0x033BA448>>, ...] caller_kwargs = {'item': <MypyFileItem mypy>}, firstresult = False def _multicall(hook_name, hook_impls, caller_kwargs, firstresult): """Execute a call into multiple python functions/methods and return the result(s). ``caller_kwargs`` comes from _HookCaller.__call__(). """ __tracebackhide__ = True results = [] excinfo = None try: # run impl and wrapper setup functions in a loop teardowns = [] try: for hook_impl in reversed(hook_impls): try: args = [caller_kwargs[argname] for argname in hook_impl.argnames] except KeyError: for argname in hook_impl.argnames: if argname not in caller_kwargs: raise HookCallError( f"hook call must provide argument {argname!r}" ) if hook_impl.hookwrapper: try: gen = hook_impl.function(*args) next(gen) # first yield teardowns.append(gen) except StopIteration: _raise_wrapfail(gen, "did not yield") else: > res = hook_impl.function(*args) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\pluggy\_callers.py:39: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ item = <MypyFileItem mypy> def pytest_runtest_setup(item: Item) -> None: _update_current_test_var(item, "setup") > item.session._setupstate.prepare(item) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\runner.py:150: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <_pytest.runner.SetupState object at 0x033BA850>, colitem = <MypyFileItem mypy> def prepare(self, colitem) -> None: """Setup objects along the collector chain to the test-method.""" # Check if the last collection node has raised an error. for col in self.stack: if hasattr(col, "_prepare_exc"): exc = col._prepare_exc # type: ignore[attr-defined] raise exc needed_collectors = colitem.listchain() for col in needed_collectors[len(self.stack) :]: self.stack.append(col) try: col.setup() except TEST_OUTCOME as e: col._prepare_exc = e # type: ignore[attr-defined] > raise e C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\runner.py:452: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <_pytest.runner.SetupState object at 0x033BA850>, colitem = <MypyFileItem mypy> def prepare(self, colitem) -> None: """Setup objects along the collector chain to the test-method.""" # Check if the last collection node has raised an error. for col in self.stack: if hasattr(col, "_prepare_exc"): exc = col._prepare_exc # type: ignore[attr-defined] raise exc needed_collectors = colitem.listchain() for col in needed_collectors[len(self.stack) :]: self.stack.append(col) try: > col.setup() C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\runner.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Package example> def setup(self) -> None: # Not using fixtures to call setup_module here because autouse fixtures # from packages are not called automatically (#4085). setup_module = _get_first_non_fixture_func( > self.obj, ("setUpModule", "setup_module") ) C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\python.py:644: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Package example> @property def obj(self): """Underlying Python object.""" obj = getattr(self, "_obj", None) if obj is None: > self._obj = obj = self._getobj() C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\python.py:291: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Package example> def _getobj(self): > return self._importtestmodule() C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\python.py:500: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Package example> def _importtestmodule(self): # We assume we are only called once per module. importmode = self.config.getoption("--import-mode") try: mod = import_path(self.fspath, mode=importmode) except SyntaxError as e: raise self.CollectError( ExceptionInfo.from_current().getrepr(style="short") ) from e except ImportPathMismatchError as e: raise self.CollectError( "import file mismatch:\n" "imported module %r has this __file__ attribute:\n" " %s\n" "which is not the same as the test file we want to collect:\n" " %s\n" "HINT: remove __pycache__ / .pyc files and/or use a " "unique basename for your test file modules" % e.args ) from e except ImportError as e: exc_info = ExceptionInfo.from_current() if self.config.getoption("verbose") < 2: exc_info.traceback = exc_info.traceback.filter(filter_traceback) exc_repr = ( exc_info.getrepr(style="short") if exc_info.traceback else exc_info.exconly() ) formatted_tb = str(exc_repr) > raise self.CollectError( "ImportError while importing test module '{fspath}'.\n" "Hint: make sure your test modules/packages have valid Python names.\n" "Traceback:\n" "{traceback}".format(fspath=self.fspath, traceback=formatted_tb) ) from e E _pytest.nodes.Collector.CollectError: ImportError while importing test module 'H:\Python\pytest_plugin_example\src\example\__init__.py'. E Hint: make sure your test modules/packages have valid Python names. E Traceback: E src\example\__init__.py:1: in <module> E from . import example E E ImportError: attempted relative import with no known parent package C:\Users\Hallmarq.BEN-LAPTOP\anaconda3\envs\symlink\lib\site-packages\_pytest\python.py:603: CollectError =============================================================================================================== mypy ================================================================================================================ Success: no issues found in 2 source files ====================================================================================================== short test summary info ====================================================================================================== ERROR src/example/example.py::mypy - _pytest.nodes.Collector.CollectError: ImportError while importing test module 'H:\Python\pytest_plugin_example\src\example\__init__.py'. ==================================================================================================== 2 passed, 1 error in 0.61s =====================================================================================================that points to a bug in pytest-mypy, it shouldn't be constructing
Package/Modulediscovery items (it doesn't use them, just passes the filename along tomypy)@asottile, thanks, that is useful information, I figured it was at least a 50/50 that the issue was in the plugins. I will see if I can figure out how to fix the issue there, although it is worth pointing out that there are at least 6 pytest plugins of this nature that I tested and all have the same issue. That may be because they are all copying each other, but it may also be worth taking a look at pytest's own documentation on creating plugins as they are all going wrong.
At least naively as I understand it, the broader issue seems to be that these plugins are hooking into Pytest's test discovery system to feed them the files to analyze, when it at least nominally isn't really primarily designed for this use case. Given it seems the legacy import-modes will stick around indefinitely as a non-default option (at least per the discussion above), it should still be possible for plugins to do this if users pass the appropriate config option/CLI flag, if they don't want to migrate to alternative file-discovery methods.
While it may affect these types of plugins generally, I'm not sure if it would make sense to do anything further on the Pytest side, given this isn't really the intended use case of test collection, as opposed to either the aforementioned workarounds or recommending a more appropriate tool, most notably @asottile 's superb Pre-Commit (particularly since you're already using it).
Protip: If you surround your huge code block with
<details>tags, it won't take up pages of space in the thread, andpython-tracebackwill give you nice formatting:Details
<details> ```python-traceback TRACEBACK HERE ``` </details>
Reacted by Ben RowlandI actually have trouble both with import-mode importlib as well as prepend/append, neither of them doing what I expect:
Assume I have a project including a package as well as an installed version of the same package in my environment.
The package includes an integrated tests package.project/python/mypackage project/python/mypackage/tests project/python/mypackage/tests/test_mypackage # includes an import mypackage install/mypackage install/mypackage/tests install/mypackage/tests/test_mypackage # includes an import mypackageNow, what shall I do when trying to
- test the code in my project I am working on
- test the installed code
Using importlib mode:
- Running plain
pytestin my project folder, pytest is unable to find packages/modules in my project since none of the necessary paths have been added to sys.path. (ModuleNotFoundError: mypackage) - Running
pytest --pyargs my_packageindeed runs the installed tests. Always. - So to get plain
pytestto work I tried to add paths manually to sys.path in pytest_configure() in conftest.py. But nowpytest --pyargs mypackageexecutesproject/python/mypackage/tests/test_mypackageif executed in the project folder.
Using prepend mode:
-
Running plain
pytestin my project folder correctly runs tests in my projectfolder. -
pytest --pyargs my_packageruns tests either on the installed code or on the code in the project folder, depending on where I do actually execute it. This is slightly worse than the first option, since I cannot be sure what happens. E.g.project/python > pytest --pyargs my_packagewill runproject/python/mypackage/tests/test_mypackageproject > pytest --pyargs my_packagewill runinstall/mypackage/tests/test_mypackage
So all import modes do not really work consistently and transparently.
See the Pytest docs, which explains much of this.
- Running plain
pytestin my project folder, pytest is unable to find packages/modules in my project since none of the necessary paths have been added to sys.path. (ModuleNotFoundError: mypackage)
Yes, this is exactly what you'd expect (and, generally, want) from Python's normal behavior, since you're using a de-facto
srclayout (just calling the directorypythoninstead, I'm assuming due to having a substantial non-Python component to your project). This means that, sincepythonis not itself a Python import package (nor should it be), the local non-installed copy of yourmypackagepackage doesn't get picked up. This is a good thing, because it means you can confidently test your installed code (as you should, to ensure the package actually works when installed) without the local copy interfering, which is one of the main benefits of thesrcdirectory layout.In order to test the local copy of your project, the standard best practice is just to install it in editable mode in your development environment (
pip install -e .), assuming your packaging config (pyproject.toml,setup.cfg, etc) is set up correctly.- Running
pytest --pyargs my_packageindeed runs the installed tests. Always.
Yes, as expected.
- So to get plain
pytestto work I tried to add paths manually to sys.path in pytest_configure() in conftest.py. But nowpytest --pyargs mypackageexecutesproject/python/mypackage/tests/test_mypackageif executed in the project folder.
You should basically never, ever munge paths like this, as it is a path that leads to the dark side. What you're effectively doing, if I'm interpreting what you're saying correctly, is psuedo-"installing" the package, which the modes
prependandappenddo for you anyway, but in a way not directly controlled by Pytest, leading to--pyargsnot being able to access the installed package, while not properly testing the actual package install, and potentially leading to other problems.(Sidenote: If you're specifying CLI options or other config every time, which you generally are if you're running your tests rigorously, you can and should simply add it to your
pytest.ini)- Running plain
pytestin my project folder correctly runs tests in my projectfolder.
It is fine for the tests to be run from your project folder, but the code under test should be from the installed package. To achieve that result, you should be able to do so with an editable install of your package, combined with the
importlibmode.-
pytest --pyargs my_packageruns tests either on the installed code or on the code in the project folder, depending on where I do actually execute it. This is slightly worse than the first option, since I cannot be sure what happens. E.g.project/python > pytest --pyargs my_packagewill runproject/python/mypackage/tests/test_mypackageproject > pytest --pyargs my_packagewill runinstall/mypackage/tests/test_mypackage
Yes, this is as expected, following the previous explanation. The current working directory gets added to
sys.path, which means that per the above, yourmy_packagepackage doesn't get picked up when you're in the project root directory (which you generally don't want), whereas it does if you happen tocdinto yoursrc(here namedpython) directory, since it is de-facto "installed" by being on the CWD path.So all import modes do not really work consistently and transparently.
Some recommendations to ensure consistent behavior for your case:
- Use the
importlibmode withaddopts = import-mode=importlibin yourpytest.ini - To test your local code, editable-install your package in your development env with
pip install -e . - To test your fully installed package (i.e. installed from a wheel built with
python -m build, PyPI, etc) activate an env with it installed.
- Running plain
Wow, thanks for the thorough answer.
My misconception was, that for testing the development code I could "just run pytest", since with import mode prepend/append this is what (apparently) happens.
Since prepend/append however do not solve the ambiguity of which code is actually tested, I agree that importlib mode is better suited as it "forces" you to test on "properly installed packages" in order to get rid of the possible ambiguity. Possibly in editable mode if it shall be the development code.Thanks for the clarification
Reacted by C.A.M. GerlachI'm hitting a strange issue when trying to test files that are not part of the actual project. Assume that the project is located at
/A/srcwith a fileA/src/conftest.pyand pytest config inA/src/tox.ini. With this setup, testing a fileA/src/test.pyworks fine. However, when I copy the same file to a folder/Bsomewhere else and runpytest /B/test.py --impot-mode=importlibthen this fails with the error messageERROR: not found: /B/test.py (no name '/B/test.py' in any of [])Does someone has an idea whats going on?
Well, unless the directory
Bis at the root (/) of your drive,pytestis not going to find it at/B/test.py, importlib or not. Did you test thatpytest /B/test.py --import-mode=prependworks?Also,
--impot-mode[sic] is not spelled correctly in the command invocation posted above.Finally, to note, with this current configuration
srcis your top-level (import) package directory (rather than your package being a directory undersrc), which if you're developing a Python package (as opposed to support code for something else), is probably not what you want...Reacted by Tobias DiezThanks everyone for the feedback.
Based on your feedback, it is clear that we should not change the
importmodedefault toimportlib, as it has its own set of drawbacks, not making it a clear winner, and changing the default would severely impact a lot of test suites.Closing this issue then, feel free to open new issues to discuss specific topics related to
--import-mode. 👍Reacted by C.A.M. GerlachReacted by Ofek Lev
This issue is a placeholder to gather feedback from users about the new
--import-mode=importlibintroduced in pytest 6.0.It is our intention to change the default to
importlibin future versions, and perhaps even eliminate the other--import-modeoptions (prependandappend).