Repository navigation
Config woes meta-ticket - add ASTROPY_CONFIG_DIR and ASTROPY_CACHE_DIR #10738
Description
Description
This ticket does a few things:
- Attempts to synthesize various open tickets related to configuration system woes
- Proposes adding support for
ASTROPY_CONFIG_DIRandASTROPY_CACHE_DIRenvironment variables, allowing total customization of these paths by users if desired - De-emphasizes use of
XDG_environment variables for configuring where Astropy-related files go, as it is a somewhat system-biased solution and could cause problems for other apps when used by unwary users (while still maintaining support for the XDG directory specification and documenting it in more "intermediate" config documentation) - Ditto implementation of Consider using appdirs for config #8245 to vendor the appdirs module rather than maintaining our own config path code (or at least a more minimal amount).
I believe this proposal would, in one stroke, resolve:
- XDG_CONFIG_HOME failing silently caused confusion #6511
- Why doesn't astropy use ~/.config/astropy by default (on Unix-like systems)? #6803
- Use user-specified custom cache directory #10244
and maybe some others. Also related is #10090 which I don't think is in conflict with this proposal--it would still just suppress loading any config file. Originally I thought of incorporating #8245 into this, but upon further reflection I think it would be counter-productive.
Detailed proposal
Much of this proposal is already in line with the status quo, but I wanted to write out the logic clearly:
-
By default all Astropy-related directories remain under the path
~/.astropy/. This includes~/.astropy/config/and~/.astropy/cache; this is the current status quo, and is applicable to all platforms, where its platform-specific expansion depends on howos.path.expanduserworks on that platform (see also MNT: Change config home path logic #10705) -
Astropy will honor the XDG base directory specification if and only if
XDG_CONFIG_HOMEand/orXDG_CACHE_HOMEare explicitly set in the environment. Otherwise it falls back the the default behavior in 1. If either one of theXDG_*_HOMEvariables is set we commit to the XDG specification. For example ifXDG_CONFIG_HOMEis set, then the astropy config dir is$XDG_CONFIG_HOME/astropy, and the astropy cache dir is~/.cache/astropyand not~/.astropy/cache.- Use of
XDG_*environment variables will be de-emphasized in the docs, though this logic will be made clear in an admonition box for Linux/BSD users.
- Use of
-
If
ASTROPY_CONFIG_DIRis set, it overrides both 1. and 2. For example settingASTROPY_CONFIG_DIR=/etc/astropyfixes that path and overrides any other processing. Same forASTROPY_CACHE_DIR. These variables can both be used independently.- These will be documented as the recommended way to change astropy's default directories.
-
Suppress config loading with ASTROPY_SUPPRESS_CONFIG #10090 is maintained within this scheme as well. If
ASTROPY_SUPPRESS_CONFIGis set then all config file loading is suppressed, overriding all of 1., 2., and 3.
Activity
embray commentedon Sep 16, 2020
As an alternative to 1. and 2., using
appdirsà la #8245 could be considered as well. In this case the rest of the logic doesn't change much:ASTROPY_CONFIG_DIRandASTROPY_CACHE_DIRwould still completely override 1. (i.e. it would bypass whateverappdirsdoes on the current platform).ASTROPY_SUPPRESS_CONFIGwould still override all of the above.I remain 👎 on this counter-proposal, but it's still worth offering up for discussion.
pllim commentedon Sep 16, 2020
I am okay with dropping
appdirsas an option. No one ever stepped up to do it in all these years, so obviously the problem it was meant to fix isn't that bad.embray commentedon Sep 18, 2020
I mostly made mention of that since I did run across the issue for it, and for a hot minute it looked like a good idea until I changed my mind. Just listing it here since, assuming others agree, we can decide to close that issue (since it would be obsoleted by the other solution I proposed).
embray commentedon Sep 22, 2020
@Cadair any thoughts on this, since you opened #8245? @astrofrog you seemed interested as well.
saimn commentedon Oct 14, 2020
Following a discussion on Slack with @pllim (cc @astrofrog and @Cadair as well), I would like to push on the idea (already proposed in #10090) of stopping writing and updating config files by default, and doing nothing when trying to read it if it is missing.
The current mechanism is unnecessarily complicated, causing some issues (warnings on the v4.0.X branch #10854, race condition if multiple process attempt to read/write the config file #7290), for no real benefit. The only benefit is for the users to have an example config file with the possible parameters, but this can be solved with documentation (already done in #10148, https://docs.astropy.org/en/latest/config/astropy_config.html) and a way to generate the file if needed.
Also, doing nothing if the file is missing has zero impact. And it actually already works on the dev version, deleting it will cause no error or warning. The default file is only generated for released versions.
astrofrog commentedon Oct 14, 2020
I agree that we should not write the config file by default, and users should have to opt in to creating it if they want to customize anything.
astrofrog commentedon Oct 15, 2020
@saimn it might be worth just asking on astropy-dev if anyone has objections to your idea (mentioning the benefits) and doing a PR if not.
saimn commentedon Oct 16, 2020
I would be happy to do so, but I think we must first find an agreement here since there is also #10090 which is going in another direction ? I started a branch and can push it for discussion.
astrofrog commentedon Oct 16, 2020
In my view both are needed: a way to suppress any loading of config files, and avoiding writing them by default, so I see the two PRs as complementary.
saimn commentedon Oct 17, 2020
@astrofrog - But if the file is not written by default, the natural way to suppress loading the config file is to delete or rename it. I'm not sure we need an environment variable for this ?
astrofrog commentedon Oct 17, 2020
One might want to just temporarily suppress the loading in which case renaming or removing would be too heavy handed?
saimn commentedon Oct 18, 2020
Renaming the file is simple enough, and that's what I would think to do for any software.
Also there is the proposal above for
ASTROPY_CONFIG_DIR, which could be also used for this, if allowing it to point to a non-existing file.