astropy.io.fits verification of MEAN?? keywords does not recognize NaN

If I try and read in and write out the following file:

http://mpia.de/~robitaille/python/SIRTF_I4_0012101376_0007_0000_03_levbflx.fits

I get the following error:

In [5]: hdulist = fits.open('SIRTF_I4_0012101376_0007_0000_03_levbflx.fits')

In [6]: hdulist.writeto('test.fits')
ERROR: VerifyError: 
HDU 0:
    Card 314:
        Card 'MEANC10' is not FITS standard (invalid value string: nan / mean of a 10x10 region at center of chip). [astropy.io.fits.verify]
---------------------------------------------------------------------------
VerifyError                               Traceback (most recent call last)
<ipython-input-6-120cce2982bd> in <module>()
----> 1 hdulist.writeto('test.fits')

/Users/tom/Library/Python/3.2/lib/python/site-packages/astropy-0.3.dev3538-py3.2-macosx-10.6-x86_64.egg/astropy/io/fits/hdu/hdulist.py in writeto(self, fileobj, output_verify, clobber, checksum)
    632             return
    633 
--> 634         self.verify(option=output_verify)
    635 
    636         # check if the file object is closed

/Users/tom/Library/Python/3.2/lib/python/site-packages/astropy-0.3.dev3538-py3.2-macosx-10.6-x86_64.egg/astropy/io/fits/verify.py in verify(self, option)
     76             warnings.warn('Note: Astropy uses zero-based indexing.\n')
     77         if opt == 'exception' and x:
---> 78             raise VerifyError('\n' + x)
     79 
     80 

VerifyError: 
HDU 0:
    Card 314:
        Card 'MEANC10' is not FITS standard (invalid value string: nan / mean of a 10x10 region at center of chip).

This is because MEANC10 is (correctly) set to nan. If you look at the image, the center of the image is indeed NaN, so should this not validate?

Wrong as it may seem, a strict reading of the FITS standard does not allow nan or inf keyword values:

4.2.4. Real floating-point number
If the value is a fixed-format real floating-point number, the
ASCII representation shall be right-justified in bytes 11 through
30.
A floating-point number is represented by a decimal number
followed by an optional exponent, with no embedded spaces.
A decimal number shall consist of a ‘+’ (decimal 43 or hex-
adecimal 2B) or ‘–’ (decimal 45 or hexadecimal 2D) sign, fol-
lowed by a sequence of ASCII digits containing a single decimal
point (‘.’), representing an integer part and a fractional part of
the floating-point number. The leading ‘+’ sign is optional. At
least one of the integer part or fractional part must be present.
If the fractional part is present, the decimal point must also be
present. If only the integer part is present, the decimal point may
be omitted, in which case the floating-point number is indistin-
guishable from an integer. The exponent, if present, consists of
an exponent letter followed by an integer. Letters in the exponen-
tial form (‘E’ or ‘D’)2 shall be upper case. The full precision of
64-bit values cannot be expressed over the whole range of values
using the fixed-format. This standard does not impose an upper
limit on the number of digits of precision, nor any limit on the
range of floating-point keyword values. Software packages that
read or write data according to this standard could be limited,
however, in the range of values and exponents that are supported
(e.g., to the range that can be represented by a 32-bit or 64-bit
floating-point number).
A free-format floating-point value follows the same rules as
a fixed-format floating-point value except that the ASCII repre-
sentation may occur anywhere within bytes 11 through 80.

So you need some application-specific way of handling this. Yay FITS!

Maybe we should also raise this with the FITS consortium, since it really does seem like an oversight? (not that things will change fast)

I fully agree that it looks like an oversight. I think the excuse is that not all floating point implementations support nan or inf. That said, as a practical matter, any current implementation should. I'll send this around the office too.

I think what I'm going to do for this is to bump in to v0.3 for now. I will go ahead and fix #954 by disallowing nan and inf from being assigned as header values--for now this should just not be allowed at all, rather than the current behavior that allows it but doesn't do anything sensible. Later, I will fix this issue by providing an option to recognized NAN and INF as valid header values.

@embray

@embray - do you still plan to try and do this for 0.3.0?

Nope

@embray

I'm going to remove the 1.0 milestone from this. @embray feel free to add it back if you think this will be solved (not sure if you are planning to do a merge of the latest pyfits changes?)

nan and inf as header values are disallowed since #954.

Navigation Menu

Repository navigation

astropy.io.fits verification of MEAN?? keywords does not recognize NaN #873

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

astropy.io.fits verification of MEAN?? keywords does not recognize NaN #873

Description

Activity

embray commented on Mar 20, 2013

astrofrog commented on Mar 20, 2013

embray commented on Mar 21, 2013

embray commented on May 15, 2013

astrofrog commented on Sep 20, 2013

embray commented on Sep 23, 2013

astrofrog commented on Jan 14, 2015

saimn commented on Oct 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions