config folder/file not found #10661

Activity

1. Phillip-M-Feldman commented on Aug 16, 2020

   

   Phillip-M-Feldman

   on Aug 16, 2020

   Author

   More actions

   Because the import fails, it seems as though there is no way to determine where Astropy is looking for the config folder (or where it got that path from).
2. pllim

   added

   Windows

   config

   question

   on Aug 17, 2020
3. pllim commented on Aug 17, 2020

   

   pllim

   on Aug 17, 2020

   Member

   More actions

   C:\Users.astropy\config -- Is the period supposed to be there?
4. Phillip-M-Feldman commented on Aug 17, 2020

   

   Phillip-M-Feldman

   on Aug 17, 2020 via email

   Author

   More actions

   I believe that the period should be there, but the text that I entered somehow got corrupted. The path should be `C:\Users\<USERID>\.astropy\config`.

   …

   On Sun, Aug 16, 2020 at 5:14 PM P. L. Lim ***@***.***> wrote: C:\Users.astropy\config -- Is the period supposed to be there? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10661 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEDRD5E7D3GMZA4KDK6TLSBBY5PANCNFSM4QASBQJA> .
5. saimn commented on Aug 17, 2020

   

   saimn

   on Aug 17, 2020

   Contributor

   More actions

   Your config file should be in $XDG_CONFIG_HOME/astropy.
6. embray commented on Aug 17, 2020

   

   embray

   on Aug 17, 2020

   Member

   More actions

   As @saimn noted, if XDG_CONFIG_HOME is set, the config file would be under $XDG_CONFIG_HOME/astropy. However, $XDG_CONFIG_HOME is more of a UNIX standard and doesn't make a lot of sense to use on Windows. You're better off not setting it at all, since by default C:\Users\<Username>\.astropy\config will be the right path for the config files on Windows.
7. pllim commented on Aug 17, 2020

   

   pllim

   on Aug 17, 2020

   Member

   More actions

   @Phillip-M-Feldman , did the answers resolve your issue? If so, please close this issue. If not, please let us know what else isn't working. Thanks!
8. saimn commented on Aug 17, 2020

   

   saimn

   on Aug 17, 2020

   Contributor

   More actions

   There is maybe another issue if the default (without setting XDG_CONFIG_HOME) did not work. And I guess the error message does not help to identify the problem ;) (FileNotFoundError:2 on None warn(ConfigurationMissingWarning(msg)) is a bit cryptic).
9. pllim commented on Aug 17, 2020

   

   pllim

   on Aug 17, 2020

   Member

   More actions

   On my Windows machine, I never had to set any special variables. I use Miniconda + Git Bash, install astropy (either released or dev) into a non-base environment there, and things Just Work ™️ .
10. Phillip-M-Feldman commented on Aug 17, 2020

    

    Phillip-M-Feldman

    on Aug 17, 2020 via email

    Author

    More actions

    Here's what I'm seeing:

    >> from astropy import units as u

    C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astropy\config\configuration.py:557: ConfigurationMissingWarning: Configuration defaults will be used due to FileNotFoundError:2 on None warn(ConfigurationMissingWarning(msg)) So, my configuration file is not being found, although it does exist in the default location.

    …

    On Mon, Aug 17, 2020 at 1:13 PM P. L. Lim ***@***.***> wrote: On my Windows machine, I never had to set any special variables. I use Miniconda + Git Bash, install astropy (either released or dev) into a non-base environment there, and things Just Work ™️ . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10661 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEDRAKBL2DO7W76XZSTALSBGFPBANCNFSM4QASBQJA> .
11. Phillip-M-Feldman commented on Aug 18, 2020

    

    Phillip-M-Feldman

    on Aug 18, 2020 via email

    Last edited by saimn

    Author

    More actions

    The more serious problem for me is that my script dies when it attempts to import (directly or indirectly) from astropy: ``` C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astropy\config\configuration.py:557: ConfigurationMissingWarning: Configuration defaults will be used due to FileNotFoundError:2 on None warn(ConfigurationMissingWarning(msg)) WARNING: CacheMissingWarning: Remote data cache could not be accessed due to FileNotFoundError: [WinError 53] The network path was not found: '\\\\<omitted>\\.astropy' [astropy.utils.data] Downloading https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat |===========================================| 1.3k/1.3k (100.00%) 0s WARNING: ConfigurationMissingWarning: Configuration defaults will be used due to FileNotFoundError:2 on None [astropy.config.configuration] Traceback (most recent call last): File "STK.py", line 49, in <module> from poliastro.twobody import Orbit File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\poliastro\twobody\__init__.py", line 1, in <module> from .orbit import Orbit File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\poliastro\twobody\orbit.py", line 15, in <module> from astroquery.jplhorizons import Horizons File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astroquery\jplhorizons\__init__.py", line 216, in <module> from .core import Horizons, HorizonsClass File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astroquery\jplhorizons\core.py", line 1302, in <module> Horizons = HorizonsClass() File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astroquery\jplhorizons\core.py", line 86, in __init__ super(HorizonsClass, self).__init__() File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astroquery\query.py", line 162, in __init__ paths.get_cache_dir(), 'astroquery', File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astropy\config\paths.py", line 173, in get_cache_dir return os.path.abspath(_find_or_create_root_dir('cache', linkto, rootname)) File "C:\ProgramData\Anaconda3\envs\py38\lib\site-packages\astropy\config\paths.py", line 298, in _find_or_create_root_dir os.mkdir(innerdir) FileNotFoundError: [WinError 53] The network path was not found: '\\\\<omitted>\\UserID\\.astropy' ``` It is unclear to my why Astropy is searching a network path.
12. embray commented on Aug 18, 2020

    

    embray

    on Aug 18, 2020

    Member

    More actions

    The warning is bubbling up from this call to get_config_dir():

    astropy/astropy/config/configuration.py

    Line 574 in be39265

    cfgfn = path.join(get_config_dir(rootname=rootname), pkgname + '.cfg')

    That's why the warning message says None. Because if an exception occurred at this line, cfgfn is None. This alone is definitely a bug.
13. embray commented on Aug 18, 2020

    

    embray

    on Aug 18, 2020

    Member

    More actions

    @Phillip-M-Feldman Could you please paste the output of

    import os, pprint
    pprint.pprint(os.environ)

    from your Python interpreter? Make sure to censor out anything private that might be in your environment.

    I wonder if this is related to #10570.
14. 9 remaining items

    Load more

15. Phillip-M-Feldman commented on Sep 2, 2020

    

    Phillip-M-Feldman

    on Sep 2, 2020 via email

    Author

    More actions

    After talking to some of our multi-platform developers, I understand the problem. I was told that HOMESHARE should be used on Linux, and USERPROFILE should be used on Windows. This is apparently a fairly common mistake for developers who work exclusively on Linux. Is there any possibility of getting this changed? (HOMESHARE is defined on some Windows machines, but is set via Active Directory and essentially hardwired, so that users--even with administrative rights--cannot change the value).

    …

    On Tue, Sep 1, 2020 at 9:31 AM P. L. Lim ***@***.***> wrote: So, in astropy, this is only a warning and doesn't stop anything from running. But astroquery promotes it to an exception because astroquery likes caching. And based on this Windows logic: https://github.com/astropy/astropy/blob/659ea8c708379a3e76f445f50bfbe1e4270ef668/astropy/config/paths.py#L42-L48 It tries to use HOMESHARE (as far as I can see based on what is posted above) and fails because your network drive is unavailable for whatever reason. So, here are my suggestions (better ones are certainly welcome): 1. Define MSYSTEM *and* HOME. Point HOME to some local drive where you want to cache to be stored; *-- OR --* 2. Change HOMESHARE to point to some local drive where you want to cache to be stored. On my Windows in a Git Bash shell, this is what it looks like for me: $ echo $MSYSTEM MINGW64 $ echo $HOME /c/Users/username — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10661 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEDREQ5EKYX7Y5ROCFDWLSDUOUZANCNFSM4QASBQJA> .
16. Phillip-M-Feldman commented on Sep 2, 2020

    

    Phillip-M-Feldman

    on Sep 2, 2020 via email

    Author

    More actions

    If I go the MSYSTEM and HOME route, how should MSYSTEM be set? Note that neither of these is a standard Windows environment variable. On Tue, Sep 1, 2020 at 8:31 PM Phillip Feldman <[email protected]> wrote:

    …

    After talking to some of our multi-platform developers, I understand the problem. I was told that HOMESHARE should be used on Linux, and USERPROFILE should be used on Windows. This is apparently a fairly common mistake for developers who work exclusively on Linux. Is there any possibility of getting this changed? (HOMESHARE is defined on some Windows machines, but is set via Active Directory and essentially hardwired, so that users--even with administrative rights--cannot change the value). On Tue, Sep 1, 2020 at 9:31 AM P. L. Lim ***@***.***> wrote: > So, in astropy, this is only a warning and doesn't stop anything from > running. But astroquery promotes it to an exception because astroquery > likes caching. > > And based on this Windows logic: > > > https://github.com/astropy/astropy/blob/659ea8c708379a3e76f445f50bfbe1e4270ef668/astropy/config/paths.py#L42-L48 > > It tries to use HOMESHARE (as far as I can see based on what is posted > above) and fails because your network drive is unavailable for whatever > reason. So, here are my suggestions (better ones are certainly welcome): > > 1. Define MSYSTEM *and* HOME. Point HOME to some local drive where > you want to cache to be stored; *-- OR --* > 2. Change HOMESHARE to point to some local drive where you want to > cache to be stored. > > On my Windows in a Git Bash shell, this is what it looks like for me: > > $ echo $MSYSTEM > MINGW64 > $ echo $HOME > /c/Users/username > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#10661 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAIEDREQ5EKYX7Y5ROCFDWLSDUOUZANCNFSM4QASBQJA> > . >
17. pllim commented on Sep 2, 2020

    

    pllim

    on Sep 2, 2020

    Member

    More actions

    Re: MSYSTEM

    Try set it to MINGW64 and see if anything breaks. If nothing on your machine is using it, it shouldn't matter what the value is. It just have to exist.

    Re: HOMESHARE vs USERPROFILE

    Fixing this might break some other person's set up, so any fix won't be in a backport anyway (you won't see it released soon). I do agree that maybe the logic should check for all the local stuff first before trying the network. I'll see if I can work out a PR.

    We apologize for any inconvenience caused!
18. pllim commented on Sep 2, 2020

    

    pllim

    on Sep 2, 2020

    Member

    More actions

    Also, seems like #8731 might be related.
19. pllim commented on Sep 2, 2020

    

    pllim

    on Sep 2, 2020

    Member

    More actions

    Do you want to see if #10705 would work for you? With that patch, if MSYSTEM isn't there, it would have checked for HOMEDRIVE + HOMEPATH next, which for your case, it would be P:\\.
20. embray commented on Sep 5, 2020

    

    embray

    on Sep 5, 2020

    Last edited by embray

    Member

    More actions

    On #10570 there was some discussion as to whether all of the messy logic in the _find_home function could simply be replaced with a call to os.path.expanduser('~') on both POSIX systems and Windows.

    No consensus was immediately reached on this before the PR was merged, so the use of os.path.expanduser('~') was only targeted to the case where os.name == 'posix', so this is of no immediate benefit to @Phillip-M-Feldman .

    The open question was whether the behavior of os.path.expanduser('~') is also an appropriate default on Windows. As I explained in the PR, I believe it is, and that all the Windows-specific logic should be removed and delegated to the default behavior provided by Python (rather than checking all these different environment variables ourselves). I guess this will require a follow-up PR.
21. embray

    self-assigned this

    on Sep 5, 2020
22. embray commented on Sep 5, 2020

    

    embray

    on Sep 5, 2020

    Member

    More actions

    @pllim Oops, I see you opened #10705 for this. I got my wires crossed since #10705 and #10570 are permutations of each other :)

    😄React with 😄1Reacted by P. L. Lim
23. embray

    closed this as completedin #10705on Sep 22, 2020
24. embray commented on Sep 22, 2020

    

    embray

    on Sep 22, 2020

    Member

    More actions

    With #10705 merged I think we can close this issue.

    To summarize:

    • By default, on Windows, the astropy config directory will go to ~/.astropy/config where how ~ is interpreted by default on Windows is documented at: https://docs.python.org/3.7/library/os.path.html#os.path.expanduser

    • HOMESHARE will not be used when determining the home directory on Windows.

    • You can still change the location of the config, even on Windows, by setting XDG_CONFIG_HOME, but this should be something like C:\Users\<Username>\config, and under that the C:\Users\<Username>\config\astropy directory should exist, but the \astropy is not part of the environment variable value (You can use any path you want though, I just used ``C:\Users<Username>\config` as an example.

    @Phillip-M-Feldman Feel free to follow up here if you have any more questions or concerns.
25. Phillip-M-Feldman commented on Sep 25, 2020

    

    Phillip-M-Feldman

    on Sep 25, 2020 via email

    Author

    More actions

    This is excellent news. Thanks! When I search via anaconda.org, the most recent version of astropy that I see is 4.0.1.post1, which I already have. How can I install the more recent version?

    …

    On Tue, Sep 22, 2020 at 10:23 AM E. M. Bray ***@***.***> wrote: With #10705 <#10705> merged I think we can close this issue. To summarize: - By default, on Windows, the astropy config directory will go to ~/.astropy/config where how ~ is interpreted by default on Windows is documented at: https://docs.python.org/3.7/library/os.path.html#os.path.expanduser - HOMESHARE will not be used when determining the home directory on Windows. - You can still change the location of the config, even on Windows, by setting XDG_CONFIG_HOME, but this should be something like C:\Users\<Username>\config, and under that the C:\Users\<Username>\config\astropy directory should exist, but the \astropy is not part of the environment variable value (You can use any path you want though, I just used ``C:\Users<Username>\config` as an example. @Phillip-M-Feldman <https://github.com/Phillip-M-Feldman> Feel free to follow up here if you have any more questions or concerns. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10661 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIEDRFMSIJ3RFGPXKXE7QLSHDMRJANCNFSM4QASBQJA> .
26. pllim commented on Sep 25, 2020

    

    pllim

    on Sep 25, 2020

    Member

    More actions

    If you insist to use conda install only the version released to conda, then I am afraid you have to wait a few months for 4.2. If you are more adventurous, you can try pip install git+https://github.com/astropy/astropy.git@master. For the latter, I recommend you try it out in a fresh and independent conda environment first.
27. ReeceHumphreys commented on Mar 30, 2022

    

    ReeceHumphreys

    on Mar 30, 2022

    More actions

    Stumbled onto this, if you need the latest astropy build from GitHub using pip the command is pip install git+https://github.com/astropy/astropy.git@main

    👍React with 👍1Reacted by james8128
28. james8128 commented on Jan 7, 2025

    

    james8128

    on Jan 7, 2025

    More actions

    Stumbled onto this, if you need the latest astropy build from GitHub using pip the command is pip install git+https://github.com/astropy/astropy.git@main

    After hours of googling, only your solution could solve my problem. Thanks a lot
29. neutrinoceros commented on Jan 7, 2025

    

    neutrinoceros

    on Jan 7, 2025

    Contributor

    More actions

    @james8128 did you encounter this issue with astropy 7.0.0 ?