Repository navigation
Download cache shelve file is same on Python 2.7 and 3.5 => crash #4426
Description
When downloading a file with caching, first on Python 2.7 and then subsequently on Python 3.5, it generates an exception. This is apparently related to 2f1a83e, which adds the suffix '.db' to the file name for Python 3 only, whereas is appears the shelve backend I happen to be using does the same on Python 2.
# PYTHON 2.7
In [1]: astro
1.1
In [2]: from astropy.utils import data
In [5]: ls /Users/aldcroft/.astropy/cache
In [10]: !rm -rf /Users/aldcroft/.astropy/cache
In [11]: f = data.download_file('http://google.com', cache=True)
Downloading http://google.com [Done]
In [12]: ls -l /Users/aldcroft/.astropy/cache
total 32
drwxr-xr-x 3 aldcroft staff 102 Jan 1 08:36 download/
-rw-r--r-- 1 aldcroft staff 16384 Jan 1 08:36 download_urlmap.db <== SAME as for Python 3
# PYTHON 3.5
In [2]: from astropy.utils import data
In [3]: f = data.download_file('http://google.com', cache=True)
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-3-0fbf18f99ba3> in <module>()
----> 1 f = data.download_file('http://google.com', cache=True)
/Users/aldcroft/anaconda/envs/py35/lib/python3.5/site-packages/astropy/utils/data.py in download_file(remote_url, cache, show_progress, timeout)
1024 if cache:
1025 # We don't need to acquire the lock here, since we are only reading
-> 1026 with _open_shelve(urlmapfn, True) as url2hash:
1027 if url_key in url2hash:
1028 return url2hash[url_key]
/Users/aldcroft/anaconda/envs/py35/lib/python3.5/site-packages/astropy/utils/data.py in _open_shelve(shelffn, withclosing)
1324 shelf = shelve.open(shelffn, protocol=2)
1325 elif six.PY3:
-> 1326 shelf = shelve.open(shelffn + '.db', protocol=2)
1327
1328 if withclosing:
/Users/aldcroft/anaconda/envs/py35/lib/python3.5/shelve.py in open(filename, flag, protocol, writeback)
241 """
242
--> 243 return DbfilenameShelf(filename, flag, protocol, writeback)
/Users/aldcroft/anaconda/envs/py35/lib/python3.5/shelve.py in __init__(self, filename, flag, protocol, writeback)
225 def __init__(self, filename, flag='c', protocol=None, writeback=False):
226 import dbm
--> 227 Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
228
229
/Users/aldcroft/anaconda/envs/py35/lib/python3.5/dbm/__init__.py in open(file, flag, mode)
86 elif result == "":
87 # db type cannot be determined
---> 88 raise error[0]("db type could not be determined")
89 elif result not in _modules:
90 raise error[0]("db type is {0}, but the module is not "
error: db type could not be determined
Activity