Skip to content

API: avoid creating cache directories on discovery - #19631

Merged
astrofrog merged 1 commit into
astropy:mainfrom
neutrinoceros:tst/no-dir-pollution
May 1, 2026
Merged

API: avoid creating cache directories on discovery#19631
astrofrog merged 1 commit into
astropy:mainfrom
neutrinoceros:tst/no-dir-pollution

Conversation

@neutrinoceros

@neutrinoceros neutrinoceros commented May 1, 2026

Copy link
Copy Markdown
Contributor

Description

This resolves remaining pollution with (empty) cache and config directories being created at the user level from a test session.
Closes #19611
Note that none of the problems this solve were caused by #19575
I'm marking this as a breaking change, although I'm really unsure anyone really relies on the behavior being changed here: cache directories are still being created when they are actually needed. This branch only avoids creating them the instant paths are being discovered.

I'm opening this including a first, temporary commit, that acts as a safe guard and helped discovering where pollution happened. It is not meant to be merged, or even work on windows. But, short of changing the default of ensure_exists to False, which is probably much more impactful of a breaking change, this is the best I can come up with to verify that I wiped all problems in CI.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@neutrinoceros neutrinoceros added this to the v8.0.0 milestone May 1, 2026
@neutrinoceros neutrinoceros added testing API change PRs and issues that change an existing API, possibly requiring a deprecation period labels May 1, 2026
@github-actions

github-actions Bot commented May 1, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@neutrinoceros neutrinoceros mentioned this pull request May 1, 2026
Closed
@neutrinoceros

neutrinoceros commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

@pllim do you think this is viable, even this late in the game and even though it has nothing to do with #19575, which was your concern when you opened the ticket ?
There are simplifications I can make here, but I'd like the approach itself to be greenlit first.

@pllim

pllim commented May 1, 2026

Copy link
Copy Markdown
Member

My personal take is that if you can get this to a merge-able state in the next few hours, it might be good to get it in since #19575 is a breaking change anyway. It would be easier on users to adapt all at once rather than have to adapt again in the next release. @astrofrog , what do you think?

@pllim

pllim commented May 1, 2026

Copy link
Copy Markdown
Member

We would also need to run Extra CI to make sure your solution work across the board (as far as we can test).

@neutrinoceros

neutrinoceros commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

On it

@neutrinoceros
neutrinoceros force-pushed the tst/no-dir-pollution branch from df99e0c to 7ac086e Compare May 1, 2026 14:57
@neutrinoceros neutrinoceros added the Extra CI Run cron CI as part of PR label May 1, 2026
@neutrinoceros

neutrinoceros commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

Running again with a more refined version and extra CI enabled

neutrinoceros
neutrinoceros commented May 1, 2026
View reviewed changes
Comment thread astropy/config/configuration.py

# if the file already exists, check that it has not been modified
if cfgfn is not None and cfgfn.is_file():
if cfgfn.is_file():

@neutrinoceros neutrinoceros May 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the is not None part was always True (cfgfn is unconditionally initialized as a Path a couple lines above) so I simplified it out while I was in there.

@neutrinoceros

neutrinoceros commented May 1, 2026

Copy link
Copy Markdown
Contributor Author

Looks stable on all platforms for which tests finished up to now. I'll undraft now. 🤞

@neutrinoceros
neutrinoceros marked this pull request as ready for review May 1, 2026 15:23
pllim
pllim reviewed May 1, 2026
View reviewed changes
Comment thread astropy/config/configuration.py
interpolation=False,
)
cobj = configobj.ConfigObj(cfgfile, interpolation=False)
except OSError:

@pllim pllim May 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would OSError even happen now that it does not try to create the directory?

@neutrinoceros neutrinoceros May 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I'm not really sure I can test for the condition described in the accompanying inline comment though; unsetting HOME just immediately breaks pytest in my setup. Maybe I could run just this function though... but I'm out of time for today, so it'd need to be explored in a follow up PR, if that seems acceptable.

@pllim pllim May 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, looks like it got merged, so I guess I will make follow-up issue... 😅

@astrofrog
astrofrog merged commit e47d7e8 into astropy:main May 1, 2026
49 of 53 checks passed
@neutrinoceros
neutrinoceros deleted the tst/no-dir-pollution branch May 1, 2026 17:50
@pllim pllim mentioned this pull request May 1, 2026
Closed
pllim
pllim reviewed May 1, 2026
View reviewed changes
Comment thread docs/changes/utils/19631.api.rst

@pllim pllim May 1, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found a typo after merge. In the interest of time, I fixed it directly on main via f961612 . FYI.

@neutrinoceros neutrinoceros May 1, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you !

@neutrinoceros neutrinoceros mentioned this pull request May 4, 2026
Merged
1 task
@pllim

This comment was marked as outdated.

Sign in to view
@neutrinoceros neutrinoceros mentioned this pull request May 23, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pllim pllim pllim left review comments
@astrofrog astrofrog astrofrog approved these changes

Assignees

No one assigned

Labels

API change PRs and issues that change an existing API, possibly requiring a deprecation period config Extra CI Run cron CI as part of PR testing utils

Projects

None yet

Milestone

v8.0.0

Development

Successfully merging this pull request may close these issues.

Remaining cache and config overhaul issues for v8.0.0

3 participants

@neutrinoceros @pllim @astrofrog