Skip to content

download_file() bug with cached results from a mirror #6982

Closed

Description

@adrn

This is something I stumbled on when I was having connection issues with data.astropy.org but not the mirror: when using download_file(), my download times were very long because it tries to connect several times before giving up and switching to the mirror. When it then finally decides to use the mirror, it works, downloads the file, and caches it. But because the file gets cached using the mirror url, if I run download_file() again with the same exact filename, it won't read it from the cache because the first URL it tries is with data.astropy.org, so then the same timeout issues happen again.

This isn't a serious issue, because I suspect the failure mode is pretty rare: this will only happen if connections to data.astropy.org timeout. If the connection is explicitly blocked, it should quickly fall back to the mirror. But it should probably first check the cache for all possible URLs before doing any internet connection.

cc @saimn @pllim

Activity

  1. pllim commented on Dec 14, 2017

    @pllim
    Member

    I might have a fix. PR forthcoming after I figured out how to write a test for this.

  2. self-assigned this
    on Dec 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions