Skip to content

Downloading data from web should be cached #68

Description

@lshep

If a package is downloading data from the web it should be cached using BiocFileCache

Activity

  1. lcolladotor commented on May 15, 2020

    @lcolladotor
    Contributor

    Hm.... I think this one is really tricky.

    I can see the value in caching, which I appreciate and I should probably do it more in a few of my packages. Now, in some cases, I download the data to a specific directory in a specific structure, so I would need to learn how to copy the data from the BiocFileCache to the location I want it in.

    Also, what happens when you are accessing say BigWig files from the web? The idea is to just access a small chunk of the file on the web so you don't have to download the whole thing. I'm assuming that would still be ok.

  2. lshep commented on May 15, 2020

    @lshep
    ContributorAuthor

    Downloading data to a users system in a specific location is never really recommended because you never know what RW ability a user will have. We use default caching location of a users system that should be writable.

    Depending on the download link I think it should be possible. There is also the ability in BiocFileCache to do processing on a downloaded file and save the result rather than the raw file itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    IntermediateIntermediate level Hackathon project

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions