Post-leap-second update to do list

@eteq

#9365 introduced the ability to update ERFA leap seconds. A few updates would still be nice

Document it in a bit more detail, in particular to tell how to avoid downloading with iers.conf.auto_download = False (both in iers docs and in LeapSeconds docstring).
Document how it can be used to simulate having missing leap seconds (@eteq).
Make the various URLs lists that include mirrors (easier with Cache preload #9182 merged).
Hard-code system files for ~~more~~ various operating systems.

System leap second file locations:

Debian/Ubuntu: '/usr/share/zoneinfo/leap-seconds.list'

Re: Hard-code system files for more operating systems. -- This is no longer necessary if you choose to remove the test for system file?

I think we'd want to have them in either case, since in most systems these will be up to date and hence the auto-update doesn't have to retrieve anything.

How can I inhibit downloading the leap second file similar to how the download of the IERS file can be inhibited?

@weaverba137

@weaverba137 - iers.conf.auto_download = False will also stop downloading leap-second files:

astropy/astropy/utils/iers/iers.py

Lines 993 to 994 in 8267d16

    
           if conf.auto_download: 
        
               trials += [(f, False) for f in files if urlparse(f).netloc]

Looking at the linked documentation, it clearly needs to discuss this is more detail. For now, I've just added an action item to the list...

That's useful to know, but I've already seen cases where the download appears to be happening at import time, rather than when I explicitly use the LeapSeconds class. I still need to track down exactly what is happening, but maybe you know of how that might happen?

Hmm, for example, would doing from astropy.time import Time cause LeapSeconds to be instantiated, in which case there's a sort of race condition where that import has to happen after conf.auto_download is set?

Yes, in fact, it's in astropy.time.core.Time.__new__().

Yes, it is the very first time a Time is initialized; I tried hard to avoid this but couldn't work out a secure way (there were rather horrible circular dependencies).

The hope, though, was that if the very first thing one does is

from astropy.utils import iers
iers.conf.auto_download = False

it should work. But am not sure if this is actually the case (if not, do let me/us know).

What should work is to set it inside .astropy/config (but I never tried that myself, I fear...)

p.s. Of course, the good solution is to have an up-to-date system file - which is possible for all linux/MacOS, but needs some auto-configuration.

Thanx, I'll run some tests soon.

In my tests, I am definitely setting auto_download = False before instantiating astropy.time.Time, but after astropy.time.Time is imported. That suggests that the configuration is already loaded into astropy.time.Time before instantiation. I thought the configuration was a singleton, but maybe it's not?

@weaverba137

@weaverba137 - yes, Time is used almost everywhere and your traceback makes clear why... I now recall that the main trouble I had was to avoid Table loading it too, since that would prevent the leap second Table from being created.

@weaverba137

Would #10125 fix your problem, @weaverba137 ?

That PR is a good start but:

Is that the only module-level call to Time() in astropy.coordinates? In astropy as a whole?
We still might call Time() in other contexts, so we still need to actively suppress downloads using other methods.

In other words, #10125 fixes an unexpected call to Time() but it doesn't fix every call to Time().

#10126 fixes the specific problem with auto_max_age = None.

I think one really useful thing would be to have a test that checks that importing all the modules of astropy does not cause any downloads...

I am surprised that our CI hasn't complained about it given that most of our jobs do not have remote data set. I'll investigate when I get the chance.

@pllim

@pllim - I was wondering too, but if the file included in astropy is up to date, it will not attempt to download anything. And the file we currently have expires Dec 2020 (I seem to have updated that as part of #9908).

belatedly, on:

We don't have control over the nodes like that, we have to treat them as blank slates.

But doesn't that mean you can install an up-to-date astropy, or set up an astropy cache that includes an up-to-date leap-second file?

@mhvk

@mhvk, no not really.

We need long-term version stability (for reproducibility among other things) so we change versions extremely infrequently. We are only just now starting to prepare for transitioning to Astropy 4. We can't just install the latest version of anything each time we run.

If we made significant structural changes to our pipeline, we could in principle prepare a node in advance with configuration files. However, the effort involved would be much greater than simply making configuration changes in code at runtime.

@weaverba137

@weaverba137 - fair enough, and best would be to make it easy to support. Of course, if you use an old version, and you need times and coordinates that are correct for the present, you will have to update IERS and leap seconds regardless, so possibly we should mostly be focussing on how to do that most easily. I can definitely see, though, that adjusting the configuration inside the python code that does all the work is easier than changing configuration files, so allowing to import the various conf without side effects is important regardless.

Re: #9479 (comment) on testing.

I cannot seem to get this test below to work properly when I run it with pytest. The behavior is different form when I run it line-by-line in IPython interpreter. 🤷‍♀

from pytest_remotedata.disable_internet import no_internet

from astropy.tests.helper import catch_warnings


def test_import_no_remote():
    from astropy.time import Time
    from astropy.utils.iers.iers import LeapSeconds, IERS_LEAP_SECOND_FILE

    LeapSeconds._auto_open_files[0] = ''  # Ignore ERFA
    LeapSeconds._auto_open_files[1] = ''  # Ignore package data

    with catch_warnings() as warns:
        with no_internet():
            assert Time('J2000').jd > 0

    assert len(warns) == 1 and 'An attempt was made to connect to the internet' in str(warns[0].message)

    LeapSeconds._auto_open_files[0] = 'erfa'
    LeapSeconds._auto_open_files[1] = IERS_LEAP_SECOND_FILE

Also see astropy/pytest-remotedata#17 (comment)

@mhvk

@mhvk is this issue still relevant?

None of the pieces have really been addressed, so in that sense, yes, still relevant. On the other hand, no one seems to have minded enough to address them, so maybe it is fine to just close it... I think the discussion here has certainly been separate enough not to be of much help, so let me just close...

Navigation Menu

Repository navigation

Post-leap-second update to do list #9479

18 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if conf.auto_download:
	trials += [(f, False) for f in files if urlparse(f).netloc]

Post-leap-second update to do list #9479

Description

Activity

pllim commented on Oct 28, 2019

mhvk commented on Oct 28, 2019

weaverba137 commented on Apr 6, 2020

mhvk commented on Apr 7, 2020

weaverba137 commented on Apr 7, 2020

weaverba137 commented on Apr 7, 2020

weaverba137 commented on Apr 7, 2020

mhvk commented on Apr 7, 2020

mhvk commented on Apr 7, 2020

weaverba137 commented on Apr 7, 2020

weaverba137 commented on Apr 7, 2020

18 remaining items

mhvk commented on Apr 7, 2020

pllim commented on Apr 8, 2020

weaverba137 commented on Apr 8, 2020

weaverba137 commented on Apr 8, 2020

mhvk commented on Apr 8, 2020

pllim commented on Apr 8, 2020

mhvk commented on Apr 8, 2020

mhvk commented on Apr 8, 2020

weaverba137 commented on Apr 8, 2020

mhvk commented on Apr 8, 2020

pllim commented on Apr 8, 2020

pllim commented on Jun 29, 2026

mhvk commented on Jun 30, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions