Refactor optional dependencies flags into utils module

#4606 has many instances of:

try:
    import pathlib
except ImportError:
    HAS_PATHLIB = False
else:
    HAS_PATHLIB = True

This idiom gets repeated in many other places for other optional dependencies. Might it make sense to refactor all these into a single module (perhaps opt_deps.py) in utils? Then application code would be:

from ...utils.opt_deps import HAS_PATHLIB

That would also make is easier to find and change if the optional dependency changes, is removed (e.g. we decide in the future that scipy becomes a hard dependency).

In Glue, I also made decorators that can be used directly on the test functions, e.g. @requires_pathlib, so we could do that here too optionally?

@astrofrog

@astrofrog - is this @requires_X mapped to pytest.mark.skipif so pytest is doing the right thing? It seems like @requires could take an argument of one or more optional deps, e.g. @test_requires('pathlib', 'scipy', 'beautifulsoup'). I think it makes sense to have test in the decorator name if it is specifically targeted at tests.

@taldcroft

@taldcroft - yes, you can see what I have here:

https://github.com/glue-viz/glue/blob/master/glue/tests/helpers.py

However, it's hard to do boolean logic with that, so it might be over-engineering, and the constants you are suggesting might be enough.

Glue's decorator seems a good way to go.
https://github.com/glue-viz/glue/blob/master/glue/tests/helpers.py#L14
It provides both a decorator which wraps pytest's skipif, and a ...._INSTALLED variable to check if a module is available. (Should be renamed to HAS_... ?)

pathlib is now provided by the minversion of Python that we support, so that if ... else ... can be taken out altogether. For testing, pytest has importorskip, so can this issue be closed?

I think it might still be worth considering centralizing the HAS_... variables for other optional dependencies?

I started working on this and then realized that we cannot really lazy import until minversion of Python is 3.7. I think something like this might work based on PEP 562, otherwise we either over-engineer or incur a penalty of checking for everything at import time.

Lazy import is important because some of these HAS_* checks occur inside real modules, not just tests.

I called this file astropy/utils/compat/optional_deps.py but nothing is set in stone yet:

# Licensed under a 3-clause BSD style license - see LICENSE.rst
"""Checks for optional dependencies using lazy import from
`PEP 562 <https://www.python.org/dev/peps/pep-0562/>`_.

"""
import importlib

__all__ = [  # noqa
    'HAS_BS4', 'HAS_BLEACH', 'HAS_HTML5LIB', 'HAS_JPLEPHEM', 'HAS_LXML',
    'HAS_PANDAS', 'HAS_PIL', 'HAS_SCIPY', 'HAS_SKYFIELD', 'HAS_YAML']


def __getattr__(name):
    if name in __all__:
        module_name = name.split('_')[1]
        if module_name not in ('PIL', ):
            module_name = module_name.lower()
        try:
            importlib.import_module(module_name)
        except (ImportError, ModuleNotFoundError):
            return False
        return True

    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

Using this, calling HAS_NOTSUCHPACKAGE raises ImportError or AttributeError depending on how the import is actually done.

p.s. I welcome better ideas.

@nstarman

Finding this after @nstarman re-raised the issue: I think this is doable now! Aren't we Python 3.7+?

Theoretically, yes... but I have bigger fish to fry. Maybe someone else wanna give it a stab this time?

You're just trying to trick me into contributing to astropy.utils... :)

But I have some time to try this out shortly!

Navigation Menu

Repository navigation

Refactor optional dependencies flags into utils module #4630

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Refactor optional dependencies flags into utils module #4630

Description

Activity

hamogu commented on Feb 23, 2016

astrofrog commented on Feb 23, 2016

taldcroft commented on Feb 23, 2016

astrofrog commented on Feb 23, 2016

saimn commented on Feb 23, 2016

pllim commented on May 11, 2020

astrofrog commented on May 11, 2020

pllim commented on May 12, 2020

adrn commented on Apr 6, 2021

pllim commented on Apr 6, 2021

adrn commented on Apr 6, 2021

pllim commented on Apr 6, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions