Skip to content

Several changes to sigma clipping functions - #3595

Merged
crawfordsm merged 17 commits into
astropy:masterfrom
larrybradley:sigma_clip
May 21, 2015
Merged

crawfordsm merged 17 commits into
astropy:masterfrom
larrybradley:sigma_clip

Conversation

@larrybradley

Copy link
Copy Markdown
Member

This PR makes several changes to the sigma clipping functions:

  • Added sigma_lower and sigma_upper keywords to allow for unsymmetric clipping.
  • In sigma_clip, I renamed the sig keyword to sigma (if unspecified then sigma_lower and sigma_upper default to sigma).
  • I changed the sigma_clip input from varfunc to stdfunc. There are readily available alternate standard deviation estimators, e.g. astropy.stats.mad_std or astropy.stats.biweight_midvariance, but none that I'm aware of for an alternate variance function.
  • sigma_clip now uses the np.ma.masked_less and np.ma.masked_greater functions to prevent
    RuntimeWarning messages: "invalid value encountered in greater filtered_data.mask |= do * do > varfunc(filtered_data) * sig ** 2" that would occur for example when the data contain np.nan.
  • In sigma_clipped_stats, I changed the keyword from mask_val to mask_value.

In this PR, three parameter names have been changed, breaking the current API. Is there a good way to issue an AstropyDeprecationWarning about these parameter changes?

CC: @perrygreenfield, @hcferguson

@larrybradley

Copy link
Copy Markdown
Member Author

@embray Any suggestions about deprecation warnings?

@embray

embray commented Mar 25, 2015

Copy link
Copy Markdown
Member

I have a few suggestions on that, yeah. If you think they'll be necessary (they probably should be, since I know there are people actively using sigma_clipping).

Unfortunately if you do want to be able to support old arguments what you're probably going to have to do is change the argument signature for the function to accept (*args, **kwargs) instead of explicitly named arguments.

However, this is made somewhat easier by Python 3's new Signature handling. We have a version of this that is backported to work in Python 2 included in Astropy. You can read about the Signature spec here:

https://docs.python.org/3/library/inspect.html#introspecting-callables-with-the-signature-object

and the backported version in Astropy is found here:

https://github.com/astropy/astropy/blob/master/astropy/utils/compat/funcsigs.py

What you'll probably want to do create two Signature objects: One that represents the current function signature defined in your PR. You would then assign sigma_clipping.__signature__ = Signature...). The second Signature would support both the new arguments and the old arguments. You would use this one in the body of the sigma_clipping function itself to parse the (*args, **kwargs) with the bind method. Then you would unfortunately have to write some custom code to manually check if the user passed in any deprecated keyword arguments, and call warnings.warn(..., AstropyDeprecationWarning) for them. You would also have to make sure duplicates somehow aren't passed in (such as sig= and sigma= simultaneously).

Does that make any sense?

This reminds me I've wanted for a while to add a decorator specifically for this purpose...

@larrybradley
larrybradley force-pushed the sigma_clip branch 2 times, most recently from c45febf to 1552228 Compare April 7, 2015 18:53
@larrybradley

Copy link
Copy Markdown
Member Author

@embray This is ready for review.

@embray

embray commented May 12, 2015

Copy link
Copy Markdown
Member

Needs a changelog entry briefly describing the API changes.

@larrybradley

Copy link
Copy Markdown
Member Author

@embray I've updated the changelog and rebased.

@pllim pllim mentioned this pull request May 13, 2015
@embray embray added this to the v1.1.0 milestone May 13, 2015
@embray

embray commented May 13, 2015

Copy link
Copy Markdown
Member

👍 Is there an appropriate assignee to sign off on this? I'm just here for nitpicking, not actual useful comments about the functionality.

@embray

embray commented May 13, 2015

Copy link
Copy Markdown
Member

Note, once merged this will close #3236.

@larrybradley

Copy link
Copy Markdown
Member Author

fyi, I just fixed the changelog indentation. I was hoping someone would review this for functionality, but it's been 2 months now....

@embray

embray commented May 13, 2015

Copy link
Copy Markdown
Member

I don't know who the best person to ask is. You should ask on the astropy-dev mailing list. Possibly @adonath?

@larrybradley

Copy link
Copy Markdown
Member Author

I was thinking @astrofrog because he wrote the original sigma_clip function.

@crawfordsm

Copy link
Copy Markdown
Member

I've volunteered to help maintain the stats package but I am happy to take a look at the functionality of this tomorrow.

@crawfordsm

Copy link
Copy Markdown
Member

Functionality wise, it works as expected and passes all the tests, so from that end it looks good to me.

This could be done in a separate PR, , but I could also imagine that cenfunc, stdfunc, and axis could be included as parameters to sigma_clip_stats.

@embray should there be any tests added for the depreciation warning or the change in the sig parameter names?

@embray

embray commented May 18, 2015

Copy link
Copy Markdown
Member

No, it's fine.

@larrybradley

Copy link
Copy Markdown
Member Author

Thanks, @crawfordsm! I'll add those keywords to sigma_clip_stats in a separate PR.

@crawfordsm

Copy link
Copy Markdown
Member

I'm will go ahead and merge this in. Thanks for the contribution @larrybradley

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants