Fix IERS test following update of IERS-B - #5051

cc: @mhvk

@taldcroft - sorry to have messed up the IERS B upload! I think the main problem really was that the IERS B values themselves actually changed...

I don't know about the flag -- I don't think I would ever have realised this was needed, so perhaps instead we should force a remote data test generally before merging?

Still failing with:

_________________________________________________________________________________________________________ TestIERS_Auto.test_simple _________________________________________________________________________________________________________

self = <astropy.utils.iers.tests.test_iers.TestIERS_Auto instance at 0x7f768afeabd8>

    @remote_data
    def test_simple(self):
        iers_a_file_1 = os.path.join(os.path.dirname(__file__), 'finals2000A-2016-02-30-test')
        iers_a_file_2 = os.path.join(os.path.dirname(__file__), 'finals2000A-2016-04-30-test')
        iers_a_url_1 = 'file://' + os.path.abspath(iers_a_file_1)
        iers_a_url_2 = 'file://' + os.path.abspath(iers_a_file_2)

        iers.conf.iers_auto_url = iers_a_url_1

        dat = iers.IERS_Auto.open()
>       assert dat['MJD'][0] == 57359.0 * u.d
E       assert <Quantity 41684.0 d> == (57359.0 * Unit("d"))
E        +  where Unit("d") = u.d

@pllim - I suspect you have something cached in your test area. Can you do a git clean -fxd or similar to ensure fresh files?

@mhvk - I was remiss in not running this locally since I did know that --remote-data was needed. I have presumed that Travis does not allow getting remote data, but is that actually true? It does definitely get tested before a release as part of the process, but having a tag might help a bit and remind the reviewer to run locally with --remote-data.

Re: remote data on Travis

I have at least two packages that I had successfully run on Travis CI with --remote-data flag. I just don't turn it on because I don't want to burden the FTP site that supply the remote data.

@taldcroft , I cleaned the repo and ~/.astropy/cache, reinstalled, and reran the tests, but I still see the same failure. Did you try running it on your side on Python 2.7?

@taldcroft - Travis allows remote data, however there is a higher change to run overtime. Would be nice to have something like [skip ci] that allows them to run.

@pllim - can you run with --pdb and when it stops print out dat.meta and dat? I cannot reproduce on my mac nor linux. The start date of 41684 is suspicious of having the wrong file.

@bsipocz @pllim - good to know remote data are OK in theory. Can you think of a way to ensure that remote testing gets done for PRs that actually need it? Somehow it needs to be an on-demand sort of thing.

Re: remote test on demand -- The obvious thing to look for is the @remote_data decorator in the code diff. Maybe @astrofrog knows how to write a bot to sniff that out and post a comment saying that the affected PR needs --remote-data? For my own project, I have tried temporarily adding --remote-data to the Travis CI test command, look at the results, then remove it before merging; although I am not sure if that is the best way.

Re: pdb -- Things are just getting weirder on my end. See below...

Tests pass when I do python setup.py test --remote-data -P utils.iers but fail when I do python setup.py test --remote-data (all packages) or python setup.py test --remote-data -P time. Passed when I do python setup.py test --remote-data -P utils for the first try, but failed with HTTPError when I run it again (and again).

All the relevant pdb outputs is here:
pdb_output.txt

@pllim - I can reproduce at least some of your failures by forcing the iers.conf.iers_auto_url config var to a non-existent URL and then running tests. So it is likely that your failures are related to the server being down. Can you retry, or perhaps try pointing to the backup by editing the value of IERS_A_URL in astropy/utils/iers/iers.py?

BTW, what is happening here is that if you don't have any IERS-A file in the cache (always true at the beginning for testing) and it fails to download, then it falls back to the bundled IERS-B. But that is different in the details and tests fail.

I just pushed another improvement to the testing in 587a295. Previously the IERS auto-download URL was being changed globally. Now it is done within a context manager and so always reverts to the normal value after the IERS unit testing.

BTW, what is happening here is that if you don't have any IERS-A file in the cache (always true at the beginning for testing) and it fails to download, then it falls back to the bundled IERS-B.

Note that in this case a warning is issued so the user knows, but of course that is not seen in tests.

@taldcroft @mhvk - what is the status of this? can I merge and include in 1.2rc1?

The changes look good to me, so if @taldcroft is happy, so am I.

I ran the utils tests with and without --remote data on Py 2.7 and 3.3 and saw no fails. So I am good to merge this.

Thanks @taldcroft!

Just for info, I'm now seeing the following failure:

__________________________ TestIERS_Auto.test_simple ___________________________

self = <astropy.utils.iers.tests.test_iers.TestIERS_Auto object at 0x127c46cf8>

    @remote_data
    def test_simple(self):
        iers_a_file_1 = os.path.join(os.path.dirname(__file__), 'finals2000A-2016-02-30-test')
        iers_a_file_2 = os.path.join(os.path.dirname(__file__), 'finals2000A-2016-04-30-test')
        iers_a_url_1 = 'file://' + os.path.abspath(iers_a_file_1)
        iers_a_url_2 = 'file://' + os.path.abspath(iers_a_file_2)

        with iers.conf.set_temp('iers_auto_url', iers_a_url_1):

            dat = iers.IERS_Auto.open()
>           assert dat['MJD'][0] == 57359.0 * u.d
E           assert <Quantity 41684.0 d> == (57359.0 * Unit("d"))
E            +  where Unit("d") = u.d

astropy/utils/iers/tests/test_iers.py:164: AssertionError
======= 1 failed, 8506 passed, 56 skipped, 38 xfailed in 426.42 seconds ========

I guess this is related to the download URL being unreliable, but as @pllim said it doesn't happen if I run just the iers tests, so I don't think it's just an unreliable server.

This means at some point it is failing to download and defaulting to IERS-B. I'll investigate.

@taldcroft - I figured it out, see #5072 and #5074