Pyfits: Failure to write compressed np.int64 compressed images. #2153

Activity

1. embray commented on Feb 28, 2014

   

   embray

   on Feb 28, 2014

   Member

   More actions

   For whatever reason the FITS Tile Compression convention still does not allow compression of 64-bit integer data #931.

   This could still use a better error message or warning though.
2. embray

   added

   io.fits

   on Feb 28, 2014
3. embray

   self-assigned this

   on Feb 28, 2014
4. astrofrog

   added this to the v0.4.0 milestone on Mar 16, 2014
5. eteq commented on May 27, 2014

   

   eteq

   on May 27, 2014

   Member

   More actions

   @embray - should this be pushed to 0.4.1?
6. embray

   modified the milestones: v0.4.1, v0.4.0 on May 27, 2014
7. embray

   added

   effort-low

   on Jul 11, 2014
8. astrofrog

   added

   package-expert

   and removed

   package-novice

   on Jul 11, 2014
9. embray

   modified the milestones: v0.4.2, v0.4.3 on Sep 23, 2014
10. embray

    modified the milestone: v0.4.3 on Nov 21, 2014
11. embray commented on Oct 20, 2015

    

    embray

    on Oct 20, 2015

    Member

    More actions

    Although, it seems that according to the current tile compression spec, specifically section 6.1, RICE_1 compression should support 64-bit ints if I read correctly (even though other parts of the spec are a little vague about that).

    More importantly though, the current version of CFITSIO that is bundled in Astropy does not support this. I haven't checked CFITSIO updates in a while though so let me see if this has been improved...
12. embray commented on Oct 20, 2015

    

    embray

    on Oct 20, 2015

    Member

    More actions

    From the CFITSIO changelog it's not clear that this has been fixed yet. Might be worth trying out though. I don't remember the details of why this didn't work before (though I know it definitely didn't).

    Probably this should either give an error, or cast the data to 32-bit (where possible, with warnings and overflow errors).

    It would probably be good to update the convention and update CFITSIO to support this. Though if any effort were put into this I'd rather it be in the form of #3895.
13. embray commented on Jul 6, 2021

    

    embray

    on Jul 6, 2021

    Member

    More actions

    AFAICT CFITS (or at least the version vendored in Astropy) still does not support this, though it does give a slightly better error message:

    RuntimeError: writing TLONGLONG to compressed image is not supported
    
14. astrofrog commented on Jan 21, 2023

    

    astrofrog

    on Jan 21, 2023

    Member

    More actions

    This seems to have been fixed by #14252:

    In [11]: import numpy as np
    
    In [12]: from astropy.io import fits
    
    In [13]: orig = np.arange(100, dtype=int)
        ...: primary = fits.PrimaryHDU()
        ...: hdu = fits.CompImageHDU(orig)
        ...: hdulist = fits.HDUList([primary, hdu])
        ...: hdulist.writeto('test.fits', overwrite=True)
    
    In [14]: hdulist = fits.open('test.fits')
    
    In [15]: hdulist[1].data
    Out[15]: 
    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
           17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
           34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
           51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67,
           68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,
           85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99])
    
    In [16]: hdulist[1].data.dtype
    Out[16]: dtype('int64')
    
15. astrofrog

    closed this as completedon Jan 21, 2023
16. astrofrog

    added 2 commits that reference this issue on May 5, 2026

    prevent creating compressed files with (u)int64 data

    ...

    a36bb34

    prevent creating compressed files with (u)int64 data

    ...

    b916eb3