Skip to content

Fix two problems related to clear_download_cache - #4810

Merged
mhvk merged 4 commits into
astropy:masterfrom
taldcroft:utils-download
May 12, 2016
Merged

mhvk merged 4 commits into
astropy:masterfrom
taldcroft:utils-download

Conversation

@taldcroft

@taldcroft taldcroft commented Apr 27, 2016 •

Copy link
Copy Markdown
Member

This fixes two problems:

Unfortunately I don't know of any automated test that can verify that the second is fixed, but I have done this manually.

The download cache directory / file structure is changed from

<root_cache_dir>/download/  <download files>
<root_cache_dir>/urlmap.db

to:

<root_cache_dir>/download/py2/  <download files>
<root_cache_dir>/download/py2/urlmap.db  # actual name depends on shelf backend
<root_cache_dir>/download/py3/  <download files>
<root_cache_dir>/download/py3/urlmap.db

The 2 or 3 is just sys.version_info.major (so this is Python 4 compatible).

cc: @eteq

@taldcroft

Copy link
Copy Markdown
Member Author

Marked as v1.2.0 since it changes where files are cached. If people think this should be backported I would not object though.

@pllim

pllim commented Apr 27, 2016

Copy link
Copy Markdown
Member

Need a change log.

it changes where files are cached

If the cache location is changing, shouldn't it be documented in the docs?

@taldcroft

Copy link
Copy Markdown
Member Author

@pllim - about documenting the change in cache location, the cache directory does not appear to be anywhere in the docs currently, so there is nothing to change. I think this is by design, i.e. the exact location of the cache should not be something people worry about. So this is a subtle case where it is not clear if this represents an API change or just a plain bug fix.

I usually wait for tests to pass and possibly get feedback before the changelog. Maybe @eteq would say this is a terrible idea and the PR gets scrapped. 😄

@pllim

pllim commented Apr 27, 2016

Copy link
Copy Markdown
Member

Can you at least comment on this PR on how the path changed exactly? Would be nice to document it here if not in the actual doc. Thanks!

@taldcroft

Copy link
Copy Markdown
Member Author

@pllim - the initial description has been updated to describe the change in file names.

@taldcroft

Copy link
Copy Markdown
Member Author

@eteq - I have tested backporting this to v1.1.2 and v1.0.9. There were no conflicts and tests passed (at least --package=utils) on Python 3, so I think we can backport if it seems like the right thing to do.

I'll wait on updating CHANGES.rst and the milestone until there is a decision.

@astrofrog

Copy link
Copy Markdown
Member

I personally think we should backport this since it fixes two bugs, and the cache location should be irrelevant for the user. Worst case, some users are going to be redownloading some files.

Comment thread astropy/utils/data.py
Opens a shelf file. If `withclosing` is True, it will be opened with closing,
allowing use like:

with _open_shelve('somefile',True) as s:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yikes, this cries out for a little class with an __enter__ and __exit__ method -- but fine to ignore here!

@mhvk mhvk modified the milestones: v1.0.10, v1.2.0 May 11, 2016
@mhvk

mhvk commented May 11, 2016

Copy link
Copy Markdown
Contributor

This looks good to me. I agree with @astrofrog that one might as well backport, since this is a bug. So I changed the milestone. Feel free to merge once the changelog entry is in the right place!

@astrofrog

Copy link
Copy Markdown
Member

@taldcroft - can you update this and add a changelog in the v1.0.10 section in this case? Thanks!

@taldcroft

Copy link
Copy Markdown
Member Author

@astrofrog - changelog entry added.

@mhvk mhvk self-assigned this May 12, 2016
@mhvk

mhvk commented May 12, 2016

Copy link
Copy Markdown
Contributor

OK, all set, merging!

@mhvk
mhvk merged commit adc4edc into astropy:master May 12, 2016
astrofrog pushed a commit that referenced this pull request May 16, 2016
Fix two problems related to clear_download_cache
astrofrog pushed a commit that referenced this pull request May 16, 2016
Fix two problems related to clear_download_cache
astrofrog pushed a commit that referenced this pull request May 16, 2016
Fix two problems related to clear_download_cache
@taldcroft
taldcroft deleted the utils-download branch February 25, 2019 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

clear_download_cache() does not work in Python 2.7 Download cache shelve file is same on Python 2.7 and 3.5 => crash

4 participants