Verification inconsistent when printing fits headers?

I read the following file
https://github.com/hamogu/specutils/blob/ioIRAF/specutils/io/tests/t/TRES.fits
when I ask ipython to show the header of the file the first time it fails with an exception, when I call the same command the second time it works (Full output is below).
This seems strange to me. Is this a feature that alerts me that my fits file is non-standard the first time it comes across the problem and then sets some flag so I don't get the same error again?

Python 2.7.3 (default, Aug  1 2012, 05:16:07) 
Type "copyright", "credits" or "license" for more information.

IPython 0.12.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import astropy.io.fits

In [2]: hdust = astropy.io.fits.open('TRES.fits')

In [3]: hdust[0].header['P.I.']
Out[3]: 'Latham'

In [4]: hdust[0].header
Out[4]: ERROR: VerifyError: 
Unfixable error: Illegal keyword name 'P.I.' [astropy.io.fits.verify]
---------------------------------------------------------------------------
VerifyError                               Traceback (most recent call last)
/home/moritz/Dropbox/code/python/specutils/specutils/io/tests/t/<ipython-input-4-49d3202ec7a2> in <module>()
----> 1 hdust[0].header

/usr/lib/python2.7/dist-packages/IPython/core/displayhook.pyc in __call__(self, result)
    236             self.start_displayhook()
    237             self.write_output_prompt()
--> 238             format_dict = self.compute_format_data(result)
    239             self.write_format_data(format_dict)
    240             self.update_user_ns(result)

/usr/lib/python2.7/dist-packages/IPython/core/displayhook.pyc in compute_format_data(self, result)
    148             MIME type representation of the object.
    149         """
--> 150         return self.shell.display_formatter.format(result)
    151 
    152     def write_format_data(self, format_dict):

/usr/lib/python2.7/dist-packages/IPython/core/formatters.pyc in format(self, obj, include, exclude)
    124                     continue
    125             try:
--> 126                 data = formatter(obj)
    127             except:
    128                 # FIXME: log the exception


/usr/lib/python2.7/dist-packages/IPython/core/formatters.pyc in __call__(self, obj)
    445                 type_pprinters=self.type_printers,
    446                 deferred_pprinters=self.deferred_printers)
--> 447             printer.pretty(obj)
    448             printer.flush()
    449             return stream.getvalue()

/usr/lib/python2.7/dist-packages/IPython/lib/pretty.pyc in pretty(self, obj)
    352                 if callable(obj_class._repr_pretty_):
    353                     return obj_class._repr_pretty_(obj, self, cycle)
--> 354             return _default_pprint(obj, self, cycle)
    355         finally:
    356             self.end_group()

/usr/lib/python2.7/dist-packages/IPython/lib/pretty.pyc in _default_pprint(obj, p, cycle)
    472     if getattr(klass, '__repr__', None) not in _baseclass_reprs:
    473         # A user-provided repr.

--> 474         p.text(repr(obj))
    475         return
    476     p.begin_group(1, '<')

/usr/local/lib/python2.7/dist-packages/astropy-0.3.dev3560-py2.7-linux-i686.egg/astropy/io/fits/header.pyc in __repr__(self)
    243 
    244     def __repr__(self):
--> 245         return self.tostring(sep='\n', endcard=False, padding=False)
    246 
    247     def __str__(self):

/usr/local/lib/python2.7/dist-packages/astropy-0.3.dev3560-py2.7-linux-i686.egg/astropy/io/fits/header.pyc in tostring(self, sep, endcard, padding)
    533         lines = []
    534         for card in self._cards:
--> 535             s = str(card)
    536             # Cards with CONTINUE cards may be longer than 80 chars; so break

    537             # them into multiple lines


/usr/local/lib/python2.7/dist-packages/astropy-0.3.dev3560-py2.7-linux-i686.egg/astropy/io/fits/card.pyc in __str__(self)
    451 
    452     def __str__(self):
--> 453         return self.image
    454 
    455     def __len__(self):

/usr/local/lib/python2.7/dist-packages/astropy-0.3.dev3560-py2.7-linux-i686.egg/astropy/io/fits/card.pyc in image(self)
    694     def image(self):
    695         if self._image and not self._verified:
--> 696             self.verify('fix')
    697         if self._image is None or self._modified:
    698             self._image = self._format_image()

/usr/local/lib/python2.7/dist-packages/astropy-0.3.dev3560-py2.7-linux-i686.egg/astropy/io/fits/verify.pyc in verify(self, option)
     67         x = str(self._verify(opt)).rstrip()
     68         if opt in ['fix', 'silentfix'] and 'Unfixable' in x:
---> 69             raise VerifyError('\n' + x)
     70         if opt not in ['silentfix', 'exception'] and x:
     71             warnings.warn(u'Output verification result:')

VerifyError: 
Unfixable error: Illegal keyword name 'P.I.'

In [5]: hdust[0].header
Out[5]: 
SIMPLE  =                    T / Written by IDL:  Tue Sep 25 11:55:33 2012      
BITPIX  =                  -32 / IEEE single precision floating point           
NAXIS   =                    2 /                                                
NAXIS1  =                 2304 /Number of positions along axis 1                
NAXIS2  =                   51 /Number of positions along axis 2                
EXTEND  =                    T / Fits standard                                  
OBJECT  = 'IQAur             ' / OBJECT NAME                                    
RA      = '05:19:00.0        ' / RA                                             
DEC     = '+33:44:54.        ' / DEC                                            
EPOCH   = '2000.0            ' / EPOCH                                          
AIR     = '1.00              ' / AIR MASS                                       
RRA     = '05:19:00.0        ' / RRA                                            
RDEC    = '+33:44:54.        ' / RDEC                                           
REPOCH  = '2000.0            ' / TEPOCH                                         
TRA     = '05:18:59.5        ' / TRA                                            
TDEC    = '+33:45:05.8       ' / TDEC                                           
TEPOCH  = '2000.000          ' / TEPOCH                                         
ST      = '05:26:27          ' / SIDEREAL TIME START                            
HA      = '+00:06:36         ' / HOUR ANGLE                                     
GFWHM   = '85                ' / GUIDE FWHM ARC SEC                             
SFOCUS  =               2.0100 / TRES FOCUS FLOAT                               
PERI    = 'OUT               ' / PERI POSITION                                  
COMPTIME= '0.0               ' / FLASH TIME                                     
FIBSIZE = '2.3               ' / TRES FIBER SIZE                                
IODINE  = 'OUT               ' / IODINE POSITION                                
CTEMPS  = ' -099.0C - 46%    ' / CHIP TEMP                                      
BTEMPS  = '23.271 23.143 21.768 19.223' / BENCH - BOT OPT HTR AC                
PRES_HUM= '776.64 25.49      ' / Air Pressure and Rel Humidity                  
DATE-OBS= '2012-09-25T12:29:03' / UT date/time at start                         
UTOPEN  = '12:29:03          ' / UT time at start                               
UTEND   = '12:31:03          ' / UT time at end                                 
MJD     =         56195.520174 / MODIFIED JULIAN DATE                           
OBSERVER= 'M Calkins         '                                                  
P.I.    = 'Latham            '                                                  
PROGRAM = '149 dwlmisc       '                                                  
OSSWITCH= ' NO               ' / OBJECT SKY SWAP

[... omitted some more keywords that are not relevant for this problem ..]

@hamogu

@hamogu - just for future, you can enclose code blocks in triple back quotes:

```
some code
```

(I edited your issue to fix this)

This is related to, but a different symptom of https://aeon.stsci.edu/ssb/trac/pyfits//ticket/205

Putting this off again just because I don't have an immediate obvious fix in mind. But I've heard a few other reports like this too and it sounds like something I should fix sooner rather than later.

@embray

Hello,
I'm having an issue that seems related to this one. I have a FITS file with an invalid keyword (IRAF-B/P). If I print the header I get only a warning, and then I can use the header with no error. But if I use directly the header (to create a WCS object for instance), I get a VerifyError error and traceback.
I have summarized this behavior here:
https://gist.github.com/saimn/f75f8926ec2153b2a0fa

If my understanding of the code is correct, in the first case self.verify is called with silentfix so it does not break and the file is marked as verified so the next calls to self.verify (with fix) will not be an issue. In the second case, self.verify is called with fix and it breaks.

@embray: Do you have any idea on the best way to fix this ? I would be happy to help. Maybe adding a keyword to specify the verify mode in fits.open ?

@saimn

@saimn Your understanding of the issue is correct. I haven't really given it any thought though--as I've said before my preference is to completely redo the verification system and that I do have plans for. It's just been low priority.

While the original data here no longer exists, it looks like the issue was fixed in #5054:

In [1]: import io
   ...: import numpy as np
   ...: from astropy.io import fits

In [2]: hdu = fits.PrimaryHDU(np.zeros((2, 2), dtype=np.float32))
   ...: buf = io.BytesIO()
   ...: hdu.writeto(buf)
   ...: raw = buf.getvalue()
   ...: cards = [
   ...:     "SIMPLE  =                    T / conforms to FITS standard                      ",
   ...:     "BITPIX  =                  -32 / array data type                                ",
   ...:     "NAXIS   =                    2 / number of array dimensions                     ",
   ...:     "NAXIS1  =                    2                                                  ",
   ...:     "NAXIS2  =                    2                                                  ",
   ...:     "P.I.    = 'Latham  '                                                            ",
   ...:     "END" + " " * 77,
   ...: ]
   ...: header_str = "".join(c.ljust(80) for c in cards)
   ...: pad = (-len(header_str)) % 2880
   ...: header_str = header_str + " " * pad
   ...: data = np.zeros((2, 2), dtype=">f4").tobytes()
   ...: datapad = (-len(data)) % 2880
   ...: data = data + b"\x00" * datapad
   ...: filebytes = header_str.encode("ascii") + data

In [3]: bio = io.BytesIO(filebytes)
   ...: hdul = fits.open(bio)
   ...: hdr = hdul[0].header
   ...: 

In [4]: repr(hdr)
WARNING: VerifyWarning: Verification reported errors: [astropy.io.fits.verify]
WARNING: VerifyWarning: Unfixable error: Illegal keyword name 'P.I.' [astropy.io.fits.verify]
WARNING: VerifyWarning: Note: astropy.io.fits uses zero-based indexing.
 [astropy.io.fits.verify]
Out[4]: "SIMPLE  =                    T / conforms to FITS standard                      \nBITPIX  =                  -32 / array data type                                \nNAXIS   =                    2 / number of array dimensions                     \nNAXIS1  =                    2                                                  \nNAXIS2  =                    2                                                  \nP.I.    = 'Latham  '                                                            "

In [5]: repr(hdr)
Out[5]: "SIMPLE  =                    T / conforms to FITS standard                      \nBITPIX  =                  -32 / array data type                                \nNAXIS   =                    2 / number of array dimensions                     \nNAXIS1  =                    2                                                  \nNAXIS2  =                    2

There is now a warning, and the keyword is removed, which I think is the desired behavior.

Navigation Menu

Repository navigation

Verification inconsistent when printing fits headers? #887

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Verification inconsistent when printing fits headers? #887

Description

Activity

astrofrog commented on Mar 18, 2013

embray commented on Mar 20, 2013

embray commented on Jul 19, 2013

saimn commented on Nov 28, 2014

embray commented on Nov 28, 2014

astrofrog commented on Jun 3, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions