Skip to content

Fix sigma_clipped_stats to use the axis argument - #4726

Merged
pllim merged 1 commit into
astropy:masterfrom
saimn:sigma-clipping
Apr 12, 2016
Merged

pllim merged 1 commit into
astropy:masterfrom
saimn:sigma-clipping

Conversation

@saimn

@saimn saimn commented Mar 23, 2016

Copy link
Copy Markdown
Contributor

The axis argument was added in #3792 and is passed correctly to sigma_clip, but
then mean, median and std did not use axis and were computed on the whole array.
This commit changes this to use axis to compute the statistics.

Also another minor annoyance is that, with Numpy 1.10, np.ma.median always
return a masked array, even when axis is None. This was not the case in previous
versions (and will probably be fixed in the future), and mean and std return
a scalar. So if it happens here, we take the first element of the array.

cc @larrybradley

@pllim pllim added the stats label Mar 23, 2016
@larrybradley

Copy link
Copy Markdown
Member

Thanks, @saimn. Looks good to me.

@pllim

pllim commented Mar 23, 2016

Copy link
Copy Markdown
Member

@larrybradley , could you please tag the appropriate "Affect ..." and Milestone tags?

@larrybradley

Copy link
Copy Markdown
Member

@pllim I'd be happy to, except I don't have permission.

@larrybradley

Copy link
Copy Markdown
Member

@saimn Can you please add a change log entry?

@saimn

saimn commented Mar 23, 2016

Copy link
Copy Markdown
Contributor Author

@larrybradley - Yes, but which version, bugfix for 1.1 ?

@larrybradley

Copy link
Copy Markdown
Member

This fixes a released version, so it should go in the bug fix section of both 1.2 (next feature release) and 1.1.3 (next bug fix release).

@saimn

saimn commented Mar 23, 2016

Copy link
Copy Markdown
Contributor Author

@larrybradley - Ok, done.

@pllim pllim modified the milestones: v1.1.3, v1.2.0 Mar 23, 2016
@pllim

pllim commented Mar 23, 2016

Copy link
Copy Markdown
Member

I cannot tag multiple milestones, so I will just tag the bug fix.

@pllim

pllim commented Apr 7, 2016

Copy link
Copy Markdown
Member

Two more things before I can merge:

  1. Remove the bug fix entry from 1.2. Just listing it under the other section is enough.
  2. Squash all your commits into one.

@crawfordsm

Copy link
Copy Markdown
Member

I'd agree -- would be happy to see this merged as soon as @pllim suggested changes are made. Also should be back ported.

The axis argument was added in astropy#3792 and is passed correctly to sigma_clip, but
then mean, median and std did not use axis and were computed on the whole array.
This commit changes this to use axis to compute the statistics.

Also another minor annoyance is that, with Numpy 1.10, np.ma.median always
return a MaskedArray, even when axis is None. This was not the case in previous
version (and will probably be fixed in the future), and mean and std return
a scalar. So if it happens here, we take the first element of the array.
@saimn

saimn commented Apr 12, 2016

Copy link
Copy Markdown
Contributor Author

@pllim , @crawfordsm : ok, done.

@pllim

pllim commented Apr 12, 2016

Copy link
Copy Markdown
Member

Thanks. Merging. 😄

@pllim
pllim merged commit 1f72cec into astropy:master Apr 12, 2016
@saimn
saimn deleted the sigma-clipping branch April 12, 2016 13:56
@bsipocz

bsipocz commented Apr 27, 2016

Copy link
Copy Markdown
Member

This seems to brake a few tests in photutils https://travis-ci.org/astropy/photutils/jobs/125377151#L789

Should we treat it as a bug in photutils or as an API change here (meaning that it should go to 1.2, rather than a bugfix release)?

@saimn

saimn commented Apr 27, 2016

Copy link
Copy Markdown
Contributor Author

Oh sorry, the bug is in the code from this PR and the inconsistent behavior from np.ma.median: if a value is masked, and axis is None, it return an array with a shape (1,), but if no values are masked it returns an array with no dimensions and shape (). And this case is not handled in https://github.com/astropy/astropy/pull/4726/files#diff-9a14850376ba3cb948038a9ac71a2bf6R312

@astrofrog astrofrog mentioned this pull request May 17, 2016
2 tasks done
@astrofrog astrofrog modified the milestones: v1.2.0, v1.1.3 May 17, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants