You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It's a little weird to me--the only reason the file is left open is because by default it returns a memory mapping to the data (so the file has to be held open), but that should still be closed automatically when it gets garbage collected. I'm not sure at what stage Python raises those ResourceWarnings though, and if all objects are collected/deleted by that point. They may not be.
The point of ResourceWarnings is to ensure that resources are explicitly (via close, or a with statement) closed rather than by relying on the GC.
Here I think GC happens right after the function exits (if you add print(1) after getdata the print shows up after the warning). That's why I say no big deal (... until someone manages to run astropy on pypy, humhum).
I'm not really sure how that could be deal with except for manually closing the mmap object underlying each memmap array...
I have a number of plans for I/O rework in PyFITS, and perhaps one thing that it could at least try to do is track open mmaps. A utility function could be provided to close all open mappings, which could at least be used by tests to ensure they aren't held open at the end of each test.
mmap=True is a bit tricky to handle correctly, yes (basically the onus is on the user to indicate when the file can be closed); on the other hand in the mmap=False (=default) case the file can be closed explicitly at the end of getdata, right?
Ah, that much is definitely a bug. It looks like the flag that indicates whether or not to close the underlying file is getting a different value now. I seem to recall that its meaning changed when passed a filename. I'll have to look into it...
Below is the stacktrace and the warning. I am not sure whether this should have been fixed by #4786, probably not. I can open a new issue if necessary.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/hdu/hdulist.py", line 138, in fitsopen
return HDUList.fromfile(name, mode, memmap, save_backup, cache, **kwargs)
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/hdu/hdulist.py", line 280, in fromfile
save_backup=save_backup, cache=cache, **kwargs)
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/hdu/hdulist.py", line 838, in _readfrom
hdu = _BaseHDU.readfrom(ffo, **kwargs)
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/hdu/base.py", line 423, in readfrom
**kwargs)
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/hdu/base.py", line 483, in _readfrom_internal
header = Header.fromfile(data, endcard=not ignore_missing_end)
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/header.py", line 460, in fromfile
padding)[1]
File "/export/miniconda/miniconda/envs/sherpa3/lib/python3.5/site-packages/astropy/io/fits/header.py", line 529, in _from_blocks
raise IOError('Header missing END card.')
OSError: Header missing END card.
sys:1: ResourceWarning: unclosed file <_io.FileIO name='data.txt' mode='rb' closefd=True>
Activity
embray commentedon Aug 6, 2014
It's a little weird to me--the only reason the file is left open is because by default it returns a memory mapping to the data (so the file has to be held open), but that should still be closed automatically when it gets garbage collected. I'm not sure at what stage Python raises those ResourceWarnings though, and if all objects are collected/deleted by that point. They may not be.
anntzer commentedon Aug 6, 2014
The point of ResourceWarnings is to ensure that resources are explicitly (via
close, or awithstatement) closed rather than by relying on the GC.Here I think GC happens right after the function exits (if you add
print(1)aftergetdatathe print shows up after the warning). That's why I say no big deal (... until someone manages to run astropy on pypy, humhum).embray commentedon Aug 6, 2014
I'm not really sure how that could be deal with except for manually closing the
mmapobject underlying each memmap array...I have a number of plans for I/O rework in PyFITS, and perhaps one thing that it could at least try to do is track open mmaps. A utility function could be provided to close all open mappings, which could at least be used by tests to ensure they aren't held open at the end of each test.
anntzer commentedon Aug 6, 2014
mmap=Trueis a bit tricky to handle correctly, yes (basically the onus is on the user to indicate when the file can be closed); on the other hand in themmap=False(=default) case the file can be closed explicitly at the end of getdata, right?embray commentedon Aug 6, 2014
It is, though admittedly there should probably be a try/finally in there somewhere: https://github.com/astropy/astropy/blob/master/astropy/io/fits/convenience.py#L200
anntzer commentedon Aug 6, 2014
Yes, I see the line's here but clearly it doesn't -- otherwise Python wouldn't be complaining...
embray commentedon Aug 6, 2014
Ah, that much is definitely a bug. It looks like the flag that indicates whether or not to close the underlying file is getting a different value now. I seem to recall that its meaning changed when passed a filename. I'll have to look into it...
Fix FITS convenience functions to actually close files.
pllim commentedon Apr 27, 2016
Fixed in #4786.
olaurino commentedon Jul 27, 2016
I am getting a similar issue with astropy v1.1.2 on Python 3.5 (continuum) when running the following command:
python -Wall -c 'from astropy.io import fits; fits.open("data.txt")'Where
data.txtis a text file.Below is the stacktrace and the warning. I am not sure whether this should have been fixed by #4786, probably not. I can open a new issue if necessary.