Changing parametrized tests to be pytest 3.2.0 compatible - #6419

Hi there @bsipocz 👋 - thanks for the pull request! I'm just a friendly 🤖 that checks for issues related to the changelog and making sure that this pull request is milestoned and labelled correctly. This is mainly intended for the maintainers, so if you are not a maintainer you can ignore this, and a maintainer will let you know if any action is required on your part 😃.

Everything looks good from my point of view! 👍

The last release incompatible with this change was 3.0.7 and was released on 2017-03-14

I'm surprised we weren't affected by this sooner, wouldn't have CircleCI started to fail a while back?

Nope, they just did the change in pytest 3.2.0 released yesterday (even though they said it to be removed in pytest 4.0). A terribly short deprecation period I would say UPDATE: they haven't removed it yet, but their deprecation warning is raised as exception by us. The weird thing is that it happened during collection, so I haven't though about it as warnings raised as exceptions.

Anyway, this is now failing as circleCI already been set to run in parallel, so your fix is needed, too.

If we want to be less pytest version dependent, we could just remove these param things altogether and instead add if statements to the tests that do an explicit xfail - perhaps not as elegant, but in a way more readable.

@mhvk - I definitely like that idea for the bugfix branch. For master for 3.0, we may still want to require pytest 3.1+, not just for this issue, but for other slightly annoying warning about the cfg heading.

Indeed, better to stick with a standard rather than have a workaround for master, so let's require pytest >=3.1 for astropy 3.0.

@astrofrog @eteq - are you OK with the proposed pytest version requirement change for 3.0? #6419 (comment)

Changelog is added. The python3.4 will probably fail as the pytest 3.1 package is not yet built for it. As discussed above this will need a companion PR against the 2.0.x branch that rewrites the tests. I'll open that once the approach here is approved.

Looks good to me.

@astrofrog - This should be ready now. For the bugfix branch fix see #6430.

This one looks good to me too.

This needs pytest 3.1+ on py3.4 to fix the travis failure. I'll restart the job once it becomes available.

While reading pytest documentation, I found a way to filter this pytest warning during collection:

def pytest_collectstart(collector):
    warnings.filterwarnings(
        'ignore',
        message=r"Applying marks directly to parameters is deprecated",
        category=DeprecationWarning)

This could allow to keep pytest < 3.1 compatibility for some time, or maybe just for the maintenance branch ?

@saimn - This is already been fixed in the maintanance branch in #6430.

This PR is basically is hold up only because there is no new enough pytest available for python3.4 and we have been bogged down with some conda build issue.

@bsipocz - Ah ok, great !

Merging this now as it's now passing and got the approval long time ago.