Repository navigation
ModuleType.__repr__ can fail on py312+ #112414
Description
Activity
Maybe something like this would be a reasonable fix?
--- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -826,8 +826,10 @@ def _module_repr_from_spec(spec): if spec.origin is None: if spec.loader is None: return f'<module {name!r}>' - else: + elif hasattr(spec.loader, "_path"): return f'<module {name!r} (namespace) from {list(spec.loader._path)}>' + else: + return f'<module {name!r} ({spec.loader!r})>'
Maybe it'd be better to do an
isinstance()check. Then someone defining_pathin their own loader wouldn't have the repr mysteriously change. Seems like an entirely reasonable attribute to have, and if it was a string you'd get rather silly results.Reacted by Alex Waygood and Jelle ZijlstraMaybe it'd be better to do an
isinstance()check. Then someone defining_pathin their own loader wouldn't have the repr mysteriously change. Seems like an entirely reasonable attribute to have, and if it was a string you'd get rather silly results.Makes sense. That implies a diff like this:
--- a/Lib/importlib/_bootstrap.py +++ b/Lib/importlib/_bootstrap.py @@ -824,10 +824,16 @@ def _module_repr_from_spec(spec): """Return the repr to use for the module.""" name = '?' if spec.name is None else spec.name if spec.origin is None: - if spec.loader is None: + loader = spec.loader + if loader is None: return f'<module {name!r}>' + elif ( + _bootstrap_external is not None + and isinstance(loader, _bootstrap_external.NamespaceLoader) + ): + return f'<module {name!r} (namespace) from {list(loader._path)}>' else: - return f'<module {name!r} (namespace) from {list(spec.loader._path)}>' + # the loader could be a custom third-party object without a `_path` attribute + return f'<module {name!r} ({loader!r})>'
Another easy way to observe this bug: apply this diff to
main:diff --git a/Lib/test/test_importlib/import_/test___loader__.py b/Lib/test/test_importlib/import_/test___loader__.py index 858b37effc..4ae60574ed 100644 --- a/Lib/test/test_importlib/import_/test___loader__.py +++ b/Lib/test/test_importlib/import_/test___loader__.py @@ -23,6 +23,7 @@ def test___loader__(self): with util.uncache('blah'), util.import_state(meta_path=[loader]): module = self.__import__('blah') self.assertEqual(loader, module.__loader__) + repr(module)
And then run
python -m test test_importlib.import_.test___loader__ -v:Test output
Using random seed: 2645428552 0:00:00 Run 1 test sequentially 0:00:00 [1/1] test_importlib.import_.test___loader__ test___loader__ (test.test_importlib.import_.test___loader__.Frozen_SpecLoaderAttributeTests.test___loader__) ... ERROR test___loader__ (test.test_importlib.import_.test___loader__.Source_SpecLoaderAttributeTests.test___loader__) ... ERROR ====================================================================== ERROR: test___loader__ (test.test_importlib.import_.test___loader__.Frozen_SpecLoaderAttributeTests.test___loader__) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Users\alexw\coding\cpython\Lib\test\test_importlib\import_\test___loader__.py", line 26, in test___loader__ repr(module) File "<frozen importlib._bootstrap>", line 545, in _module_repr File "<frozen importlib._bootstrap>", line 830, in _module_repr_from_spec AttributeError: 'SpecLoaderMock' object has no attribute '_path' ====================================================================== ERROR: test___loader__ (test.test_importlib.import_.test___loader__.Source_SpecLoaderAttributeTests.test___loader__) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Users\alexw\coding\cpython\Lib\test\test_importlib\import_\test___loader__.py", line 26, in test___loader__ repr(module) File "<frozen importlib._bootstrap>", line 545, in _module_repr File "<frozen importlib._bootstrap>", line 830, in _module_repr_from_spec AttributeError: 'SpecLoaderMock' object has no attribute '_path' ---------------------------------------------------------------------- Ran 2 tests in 0.022s FAILED (errors=2) test test_importlib.import_.test___loader__ failed test_importlib.import_.test___loader__ failed (2 errors) == Tests result: FAILURE == 1 test failed: test_importlib.import_.test___loader__ Total duration: 254 ms Total tests: run=2 Total test files: run=1/1 failed=1 Result: FAILURE
I've filed #112425 to fix it.
- added a commit that references this issue
on Nov 27, 2023 - added a commit that references this issue
on Nov 27, 2023 - added a commit that references this issue
on Nov 28, 2023
Bug report
Bug description:
Calling
repr()on aModuleTypeinstance can fail withAttributeErroron py312+, due to changes made in #98870. This is because the implementation ofimportlib._bootstrap._module_repr_from_specassumes that all loaders will have a_pathattribute. But if the module had a custom loader, that won't necessarily hold true:_pathis an undocumented, internal, private attribute that CPython's import loaders have, but custom third-party loaders won't necessarily have.An easy way to see this is the following (in a py312 venv with
setuptoolsinstalled):setuptoolsimplements the customVendorImporterloader here: https://github.com/pypa/setuptools/blob/b1bf87be7869df40872e947e27296ef87e3125ae/setuptools/extern/__init__.py#L5-L70. Instances ofVendorImporterdo not have a_pathattribute.This kind of thing trips up external tools such as mypy's
stubtest.pyscript, which callsrepr()on arbitrary instances oftypes.ModuleTypeon certain code paths and (reasonably) expects that to always succeed: https://github.com/python/mypy/blob/1200d1d956e589a0a33c86ef8a7cb3f5a9b64f1f/mypy/stubtest.py#L224Cc. @FFY00 and @jaraco, as participants in #98139 (the issue #98870 was linked to)
CPython versions tested on:
3.12
Operating systems tested on:
Windows
Linked PRs
AttributeErrorwhen callingrepr()on a namespace package imported with a custom loader #112425AttributeErrorwhen callingrepr()on a namespace package imported with a custom loader (GH-112425) #112440repr()on a namespace package #112475repr()on a namespace package (GH-112475) #112480