Skip to content

wcs _TAB option: example please #7883

Closed
Closed
wcs _TAB option: example please#7883
Labels
wcs

Description

@knordsieck
knordsieck
opened on Oct 11, 2018

I have been unable to get the _TAB transformation class ("table lookup distortions") to work. I need it because of quite nonlinear spectral distortions I need to manage in objective prism images. Could someone point me to a successful use of this class? An actual fits file with the correct header keywords and table extension would be very valuable.

I am currently using astropy 2.0.2. I s this a problem?

Thanks

Activity

  1. pllim
    added
    wcs
    Needs-clarification
    on Oct 17, 2018

  2. pllim commented on Oct 17, 2018

    @pllim
    pllim
    on Oct 17, 2018
    Member

    Hello. Do you not have a FITS file already? If not, what format is your data in? What instrument did you use to get the data? Usually, an instrument comes with its own calibration software.

    Also, unless you are stuck with Python 2, we recommend using the latest release of 3.x series.

  3. knordsieck commented on Oct 17, 2018

    @knordsieck
    knordsieck
    on Oct 17, 2018 via email
    Author

  4. nden commented on Oct 17, 2018

    @nden
    nden
    on Oct 17, 2018
    Contributor

    @knordsieck There's no support for the -TAB convention in astropy yet. There's some initial scoping of work being done to see what needs to be implemented.

    The lookup table distortion is a different structure - it's the one described in the distortion paper. It's essentially an image extension with an optional interpolation applied.

  5. knordsieck commented on Oct 17, 2018

    @knordsieck
    knordsieck
    on Oct 17, 2018 via email
    Author

  6. pllim commented on Oct 17, 2018

    @pllim
    pllim
    on Oct 17, 2018
    Member

    Since you mentioned SALT and spectroscopy, cc @crawfordsm

  7. pllim
    removed
    Needs-clarification
    on Oct 17, 2018

  8. nden commented on Oct 17, 2018

    @nden
    nden
    on Oct 17, 2018
    Contributor

    astropy implements the Lookup table option from the distortion paper. An example file is in the astropy source directory, astropy/wcs/tests/data/dist_lookup.fits.gz

    It is used in HST imaging modes to describe part of the distortion in ACS and WFC3. Another source of files like that is the STScI archive.

  9. knordsieck commented on Oct 17, 2018

    @knordsieck
    knordsieck
    on Oct 17, 2018 via email
    Author

  10. nden commented on Oct 17, 2018

    @nden
    nden
    on Oct 17, 2018
    Contributor

    Yeah, it's not straightforward. In astropy, using the file from the tests directory (I used astropy master but this should work with the release as well):

    f = fits.open('dist_lookup.fits.gz') 
w = wcs.WCS(f[1].header, f)
w.p4_pix2foc(1,1,1) 
Out[5]: [array(0.96995641), array(1.01104154)]

    p4_pix2foc evaluates only the lookup table distortion.

    f.info() 
Filename: dist_lookup.fits.gz
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU       4   ()      
  1  SCI           1 ImageHDU       170   (100, 100)   float32   
  2  D2IMARR       1 ImageHDU        15   (4096, 1)   float32   
  3  WCSDVARR      1 ImageHDU        15   (65, 33)   float32   
  4  WCSDVARR      2 ImageHDU        15   (65, 33)   float32

    Lookup table extensions have EXTNAME=WCSDVARR. This file has two Lookup table extensions - one for each axis. Their size is smaller than the image in SCI (65, 33). A correction value is computed by bilinear interpolation. The best explanation for the format is the distortion paper. The paper explains what the keywords in the SCI header which point to the WCSDVARR extensions are.
    There are more details to all this...

    Again, in HST these are used for correcting residual distortions in images.
    This is different from the -TAB convention which is stored in a binary table, uses different keywords to connect the SCI extension with the lookup table extension and is approved convention. Hope some of this is helpful.

  11. knordsieck commented on Oct 17, 2018

    @knordsieck
    knordsieck
    on Oct 17, 2018 via email
    Author
  12. crawfordsm
    added this to the v3.2 milestone on Feb 8, 2019
  13. bsipocz
    removed this from the v3.2 milestone on May 30, 2019

  14. parejkoj commented on Oct 4, 2019

    @parejkoj
    parejkoj
    on Oct 4, 2019
    Contributor

    Joining onto this after trying specutils for this (see: astropy/specutils#528 ): I too would like to be able to write a -TAB spectral WCS, per C&G 2006.

    @nden says above that there is no support for the -TAB convention, but later says "astropy implements the Lookup table option from the distortion paper." I thought that's what the spectral -TAB format was?

    This is my specutils example for how I was going to write a per-pixel table spectral WCS (which doesn't work correctly in specutils currently). If there is a way to do this in astropy, I would be interested to see it.

    import astropy.units as u
import specutils

n_pixels=1000
wavelength = np.linspace(300, 1100, n_pixels)*u.nm
spectrum = np.random.random(n_pixels) * u.count

spec=specutils.Spectrum1D(flux=spectrum, spectral_axis=wavelength)
spec.write('outputfile.fits')

  15. nden commented on Oct 4, 2019

    @nden
    nden
    on Oct 4, 2019
    Contributor

    @parejkoj The -TAB convention is described in WCS Paper III (the paper for spectral coordinates). The LookupTable extension is a simpler convention described in the unpublished Distortion WCS paper. The latter is implemented in astropy.wcs and wcslib, the former is being worked on now.

    What you are attempting to do can be done using gwcs:

    import astropy.units as u
import gwcs
from astropy.modeling.tabular import Tabular1D

n_pixels=1000
wavelength = np.linspace(300, 1100, n_pixels)*u.nm
forward = Tabular1D(np.arange(n_pixels), lookup_table=wavelength, name='dispersion')

pixel_frame = gwcs.CoordinateFrame(naxes=1, axes_type=('detector',), axes_order=(0,))
spectral_frame = gwcs.SpectralFrame(axes_order=(0,), unit=wavelength.unit)
pipeline = [(pixel_frame, forward),
                    (spectral_frame, None)]

wcs = gwcs.wcs.WCS(pipeline)

    Of course, this only creates the WCS object. And it won't save it to fits. Are you sure specutils can save a gwcs object to fits file?

  16. parejkoj commented on Oct 4, 2019

    @parejkoj
    parejkoj
    on Oct 4, 2019
    Contributor

    No, if I pass the gwcs object to specutils, it writes it out in the same way: the examples given on the above specutils issue behave the same whether you pass in a gwcs object, or the table as the spectral axis directly.

    Is there an Issue or PR for the -TAB that you say "is being worked on now"? Is there a way I could contribute to that?

  17. nden commented on Oct 4, 2019

    @nden
    nden
    on Oct 4, 2019
    Contributor

    In the jwst pipeline we save this as pure asdf or as AsdfInFits (the asdf part is saved as a FITS extension). I don't know what specutils does but I doubt it can save to -TAB, that's just not implemented in astropy.

    The -TAB in astropy.wcs involves wrapping parts of wcslib. We are only now starting to look at what needs to be done and how feasible it is.

  18. pllim
    added
    Priority-High
    on Oct 5, 2019

  19. mcara commented on Nov 21, 2019

    @mcara
    mcara
    on Nov 21, 2019
    Contributor

    In PR #9641 I implemented support for the -TAB algorithm in FITS WCS defined in WCS Paper III.

  20. mcara commented on Jan 16, 2020

    @mcara
    mcara
    on Jan 16, 2020
    Contributor

    To those (still) interested: #9641 has been merged and WCS-TAB should now be supported in astropy.wcs. Please give it a try.

  21. pllim commented on Jan 16, 2020

    @pllim
    pllim
    on Jan 16, 2020
    Member

    Since the PR is merged, I am closing this as solved then. Thanks, @mcara !

  22. pllim
    closed this as completedon Jan 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    wcs

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions