Skip to content

initialize data url alias for data.astropy.org - #9187

Closed
dstndstn wants to merge 1 commit into
astropy:masterfrom
dstndstn:data-url-alias
Closed

initialize data url alias for data.astropy.org#9187
dstndstn wants to merge 1 commit into
astropy:masterfrom
dstndstn:data-url-alias

Conversation

@dstndstn

@dstndstn dstndstn commented Aug 30, 2019

Copy link
Copy Markdown

To address #8676

@astropy-bot astropy-bot Bot added the utils label Aug 30, 2019
@dstndstn dstndstn mentioned this pull request Aug 30, 2019
Closed
pllim
pllim requested changes Aug 30, 2019
View reviewed changes

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but unfortunately, this needs a bit more thought. First of all, the STScI server is no longer a thing (this just happened and was not widely advertised), so the mirror situation needs some figuring out among core maintainers. Secondly, this completely bypasses the config below, which concerns me.

Perhaps a less controversial approach is an option to disable alias caching. But I have not had time to think about this too much. And it looks like @aarchiba is onto something in her PRs.

@aarchiba

aarchiba commented Aug 30, 2019

Copy link
Copy Markdown
Contributor

If I understand the code's current logic, the point of the alias mechanism is to treat URLs that are requested from https://astropy.stsci.edu/data/ be stored under the same key as if they had been requested from http://data.astropy.org/ . No other URL rewriting happens. This seems pointless, at best it reduces some data duplication. It doesn't help with mirroring. What is the user experience you want from the alias mechanism?

More to the point might be an ability for download_file to accept a list of mirrors, perhaps configured in the conffile. Then when http://data.astropy.org/thing is requested, it would know internally that if there were some problem with data.astropy.org it could also request http://astropy.stsci.edu/data/thing and store the result under http://data.astropy.org/thing.

Perhaps I'll try adding that to PR #9182 . It's not really the same topic but I can't easily make it a separate pull request.

@aarchiba

aarchiba commented Aug 30, 2019

Copy link
Copy Markdown
Contributor

Actually, because this is a much easier decision to merge, why not just strip out the alias mechanism entirely?

@dstndstn

dstndstn commented Aug 30, 2019

Copy link
Copy Markdown
Author

Thanks for your reply, @aarchiba . Please feel free to close this PR if you're willing to address the issue in #9182! I totally agree with your questioning of the desired user experience of the aliasing mechanism (and suggestion of removing it).

@pllim

pllim commented Aug 30, 2019

Copy link
Copy Markdown
Member

The alias was added in #6987 to address #6982 . FYI.

@aarchiba aarchiba mentioned this pull request Sep 16, 2019
Merged
@pllim

pllim commented Oct 30, 2019

Copy link
Copy Markdown
Member

Superseded by #9182. Thanks anyway!

@pllim pllim closed this Oct 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pllim pllim pllim requested changes

Assignees

No one assigned

Labels

utils

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dstndstn @aarchiba @pllim