Skip to content

Implement support for the -TAB algorithm in WCS - #9641

Merged
nden merged 1 commit into
astropy:masterfrom
mcara:wrap-tabprm
Jan 8, 2020
Merged

nden merged 1 commit into
astropy:masterfrom
mcara:wrap-tabprm

Conversation

@mcara

@mcara mcara commented Nov 21, 2019 •

Copy link
Copy Markdown
Contributor

This pull request is to implement support for the -TAB algorithm in FITS WCS as described in WCS Paper III.

This PR partially addresses #2362, #5462, #7883 in the sense that astropy.wcs now will be able to support -TAB in WCS if already defined in a HDUList object. However this PR does not provide "write" support for -TAB: the files must be created using external scripts.

@timj

timj commented Nov 21, 2019

Copy link
Copy Markdown
Contributor

This is great. LSST recently started writing files with -TAB for a small spectrograph we have. I can donate a sample file if you like? Also, wcsLib itself generates a rather complex -TAB example file. Have you tested with that file?

My test files are at: https://jira.lsstcorp.org/browse/DM-21797

@mcara

mcara commented Nov 21, 2019 •

Copy link
Copy Markdown
Contributor Author

@timj I got your files (Thanks!) and I was able to create astropy.wcs.WCS objects without issues for both of them. The numbers appear "reasonable" to me though I really have no idea whether the coordinates that I am getting are what you expect to see.

@nden

nden commented Nov 21, 2019

Copy link
Copy Markdown
Contributor

@timj Could someone on the LSST team test this PR as well?

@nden
nden requested a review from MSeifert04 November 21, 2019 17:21
@nden

nden commented Nov 21, 2019

Copy link
Copy Markdown
Contributor

@MSeifert04 Since you've looked at the C wrappers before do you think you can review this one?

@MSeifert04

MSeifert04 commented Nov 21, 2019 •

Copy link
Copy Markdown
Contributor

@mcara Is it feasible to undo the purely style-related changes? It makes the PR quite complicated to review and a bit of information is lost (the /*@null@*/, etc.) and I find some of the changes problematic, e.g. removal of curly braces.

Comment thread astropy/wcs/include/astropy_wcs/wcslib_wrap.h
Comment thread astropy/wcs/src/astropy_wcs.c Outdated
@MSeifert04

MSeifert04 commented Nov 21, 2019 •

Copy link
Copy Markdown
Contributor

This sounds like a great addition. Thank you @mcara . I'll do a full review soon, but I'm not sure if I will be able to do the review today/tomorrow.

Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wtbarr_wrap.c
Comment thread astropy/wcs/src/wcslib_wtbarr_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wtbarr_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
@mcara

mcara commented Nov 21, 2019 •

Copy link
Copy Markdown
Contributor Author

@MSeifert04 Thanks a lot for a thorough review. I hope I have addressed all your comments in the latest commit.

Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
(reprfunc)PyTabprm___str__, /*tp_repr*/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for removing the repr?

If that wasn't an accidental removal we probably should also remove the function implementation of PyTabprm___str__

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed __repr__ because I hated the output of it when I was debugging my code. However, it is really a matter of taste and whether people prefer excessive vs succinct output. To understand (=see for yourself) what I am talking about, try:

>>> from astropy import wcs
>>> 5 * [wcs.WCS().wcs]

and now please find the cdelt of the 2nd WCS in the printed output. So, for me it was easier to see how many tabprm or wtbarr structures I had in the members WCS.wcs.tab and WCS.wcs.wtb. If I wanted to see the content of one particular structure, I would do print(WCS.wcs.wtb[0]).

Maybe my above example is not appropriate for a single WCS object (and that is how it is used most of the time), but for lists of tabprm and wtbarr structures, it felt overwhelming. Maybe a more compact format can be found for __repr__ that is also more informative than a just object address.

@mcara

mcara commented Nov 22, 2019

Copy link
Copy Markdown
Contributor Author

@pllim Could you please be so kind as to help figuring out what is the issue with tests on Windows: https://travis-ci.org/astropy/astropy/jobs/615643939#L1980-L1990 Thank you!

@pllim

pllim commented Nov 22, 2019

Copy link
Copy Markdown
Member

I can tell you it probably segfaulted but I have no desire to debug it. You can try have the CI install pytest-faulthandler and see if that gives you anything useful. Otherwise, you might need to run it locally on a Windows machine with gdb or something equivalent. Maybe @jhunkeler got better ideas.

Comment thread astropy/wcs/src/wcslib_wrap.c Outdated
@mcara
mcara force-pushed the wrap-tabprm branch 3 times, most recently from 3be97d3 to 04835ef Compare January 7, 2020 04:40
@mcara

mcara commented Jan 7, 2020 •

Copy link
Copy Markdown
Contributor Author

@MSeifert04 @nden I believe now unit tests are all passing. The reported failures are unrelated to this PR. I found another bug in WCSLIB due to which an element with a very large index ((unsigned int) -1) before the allocated memory block was being accessed. I implemented fix in cextern/wcslib until we get a new release of WCSLIB.

I will report this bug.

@mcara
mcara requested a review from MSeifert04 January 7, 2020 05:59
@MSeifert04

Copy link
Copy Markdown
Contributor

@mcara Great 👍 I'm not sure what our policy is regarding extern libraries but I think we probably need to apply it as "patch" instead of fixing it in the extern code. Just to make it transparent what we applied and what is an extern library.

@nden

nden commented Jan 8, 2020

Copy link
Copy Markdown
Contributor

I don't think we have a policy about that but in the past we've applied "patches" to a library in cextern. And by "patch" I mean make the change in the same PR which updates wcslib. @MSeifert04 do you mean apply the change to wcslib in a separate PR or something more sophisticated?

@MSeifert04

Copy link
Copy Markdown
Contributor

do you mean apply the change to wcslib in a separate PR or something more sophisticated?

I am actually not sure how we should patch extern libraries. But I see that my comment was misleading (probably shouldn't comment when tired 😓 ). What I meant was that the commit that fixed wcslib also changed our code-base which I think is problematic, I think the fix for an extern code-base should be restricted to the extern code-base (so that we can easily revert it with git revert or similar) when the new version is out.

If the fix should be in a separate PR or how we should apply it (and what would be possible) is something I don't know.

@mcara

mcara commented Jan 8, 2020

Copy link
Copy Markdown
Contributor Author

Now WCSLIB fix is in a separate commit.

@nden

nden commented Jan 8, 2020

Copy link
Copy Markdown
Contributor

@mcara Actually do you mind applying it in a separate PR (to be merged before this one)? That way this PR contains only the -TAB implementation in astropy and the wcslib change is easy to find.

@mcara

mcara commented Jan 8, 2020

Copy link
Copy Markdown
Contributor Author

@nden Allright. Will do.

@nden

nden commented Jan 8, 2020

Copy link
Copy Markdown
Contributor

This looks ready to be merged. One last request @mcara, could you add an entry in the "what's new in 4.1" file? There was a recent decision to add entries there with the PR and not before a release.

@MSeifert04

MSeifert04 commented Jan 8, 2020 •

Copy link
Copy Markdown
Contributor

Yep, I think this looks okay now.

Thanks @mcara !

@nden

nden commented Jan 8, 2020

Copy link
Copy Markdown
Contributor

Thanks @mcara, @MSeifert04 and everyone else who contributed!
@MSeifert04 approved in a comment so I'm merging it.

@nden
nden merged commit 3fdf53b into astropy:master Jan 8, 2020
@mcara

mcara commented Jan 8, 2020

Copy link
Copy Markdown
Contributor Author

Thank you @nden and @MSeifert04 for constructive comments and bug finding work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants