Repository navigation
convolution cannot be imported #8032
Description
Activity
Well, @pllim pointed out I can't be in the source library to import convolution, at least a bit less cryptic error message would be nice to avoid causing a heart attack.
Building in the source dir also doesn't solve all the issues, now I see another one:
In [2]: >>> from astropy.convolution import convolve ...: >>> convolve([1, 4, 5, 6, 5, 7, 8], [0.2, 0.6, 0.2]) ...: ...: --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-2-843c8031be97> in <module>() ----> 1 from astropy.convolution import convolve 2 convolve([1, 4, 5, 6, 5, 7, 8], [0.2, 0.6, 0.2]) ~/munka/devel/astropy/astropy/convolution/__init__.py in <module>() 8 try: 9 # Not guaranteed available at setup time ---> 10 from .convolve import convolve, convolve_fft, interpolate_replace_nans, convolve_models 11 except ImportError: 12 if not _ASTROPY_SETUP_: ~/munka/devel/astropy/astropy/convolution/convolve.py in <module>() 23 # Turn the faulthandler ON to help catch any signals, e.g. segfaults 24 # or asserts. This doesn't, currently, work with Jupyter Notebook. ---> 25 faulthandler.enable() 26 27 # Find and load C convolution library AttributeError: '_StdoutProxy' object has no attribute 'fileno'I had none of these on the LTS branch....
This is due to the refactoring of the convolution code. I think we should just remove the call to faulthandler, it's not actually needed for the convolution code. And I agree that we should basically make sure the error message when importing for the source tree is the same as for other extensions. I'll open a PR.
Reacted by P. L. LimI also cannot import convolve running MacOS 10.13.6. This is from last night on master, git hash 064dac1
In [1]: import astropy In [2]: astropy.__version__ Out[2]: '3.2.dev23162' In [3]: from astropy.convolution import Kernel2D --------------------------------------------------------------------------- OSError Traceback (most recent call last) <ipython-input-3-0d49cdbc4685> in <module> ----> 1 from astropy.convolution import Kernel2D ~/miniconda3/envs/jwst_dev/lib/python3.7/site-packages/astropy/convolution/__init__.py in <module> 8 try: 9 # Not guaranteed available at setup time ---> 10 from .convolve import convolve, convolve_fft, interpolate_replace_nans, convolve_models 11 except ImportError: 12 if not _ASTROPY_SETUP_: ~/miniconda3/envs/jwst_dev/lib/python3.7/site-packages/astropy/convolution/convolve.py in <module> 30 libConvolve = ctypes.windll.LoadLibrary(lib_path) 31 else: ---> 32 libConvolve = ctypes.cdll.LoadLibrary(lib_path) 33 34 ~/miniconda3/envs/jwst_dev/lib/python3.7/ctypes/__init__.py in LoadLibrary(self, name) 432 433 def LoadLibrary(self, name): --> 434 return self._dlltype(name) 435 436 cdll = LibraryLoader(CDLL) ~/miniconda3/envs/jwst_dev/lib/python3.7/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error) 354 355 if handle is None: --> 356 self._handle = _dlopen(self._name, mode) 357 else: 358 self._handle = handle OSError: dlopen(/Users/jdavies/miniconda3/envs/jwst_dev/lib/python3.7/site-packages/astropy/convolution/lib_convolve.py, 6): no suitable image found. Did find: /Users/jdavies/miniconda3/envs/jwst_dev/lib/python3.7/site-packages/astropy/convolution/lib_convolve.py: file too short /Users/jdavies/miniconda3/envs/jwst_dev/lib/python3.7/site-packages/astropy/convolution/lib_convolve.py: file too shortJust to be clear #8035 does not fix the error @jdavies-st reported.
@jamienoss - any ideas about @jdavies-st's issue?
I'll walk over and find out...
Regarding the "more", are there other issues that I should be aware of? I just looked but couldn't find any open (or very recently closed).
more, in addition to the last commit done in that PR. My first (wrong) thought was that this is due to some remnants of openMP, but apparently ctypes messes with my pure python life just as much.
@nden just walked me through their findings and I'm just in the process of trying to reproduce on my box... done.
In gist, I never tested the build via
./setup install, I only ever used./setup buildand then imported explicitly from the build dir. I only tested the install with conda-build.So the issue is that the install generates the extra
lib_convolve.py. The grep forlib_convolve*picks that up instead of the.soor.dll. The grep could be made more explicit, it could loop through the returned list and exclude the*pyor be explicit conditional on OS, e.g.lib_convolve*.so.The content of
lib_convolve.pyis:def __bootstrap__(): global __bootstrap__, __loader__, __file__ import sys, pkg_resources, imp __file__ = pkg_resources.resource_filename(__name__, 'lib_convolve.cpython-36m-darwin.so') __loader__ = None; del __bootstrap__, __loader__ imp.load_dynamic(__name__,__file__) __bootstrap__()
The other approach is to determine why it is even there. I will also sanity check the conda-build method again.
To be clear
./setup builddoesn't producelib_convolve.pythis is generated as part of the install stage. Perhaps it's an egg thing?conda build and install works great for me, if you ignore the numpy 1.11 -> 1.15 issue and the failing erfa test:
Traceback (most recent call last): File "//anaconda/conda-bld/astropy_1540921946927/test_tmp/run_test.py", line 6, in <module> import astropy._erfa._core ModuleNotFoundError: No module named 'astropy._erfa._core'
conda-build doesn't generate the superfluous file,
lib_conovlve.py. Which is interesting since the build script only executes$PYTHON setup.py install --offline --old-and-unmanageable.Looks like the trick is
--old-and-unmanageablethus preventing the egg build. Guess it is an egg thing?Should we split this issue out into two?
lib_convolve.pyis also produced on windows via./setup install.
