Skip to content

problem with memmap when using latest astropy on mac - #3766

Merged
embray merged 2 commits into
astropy:masterfrom
embray:fits/issue-3766
Jul 14, 2015
Merged

problem with memmap when using latest astropy on mac#3766
embray merged 2 commits into
astropy:masterfrom
embray:fits/issue-3766

Conversation

@embray

@embray embray commented Jul 6, 2015

Copy link
Copy Markdown
Member

After upgrading astropy from version 0.4.2 to 1.0.2 I ran into a problem using memory mapped images which did not happen before

a fits file is being loaded using the following line:

f = astropy.open(inFile,memmap=True,do_not_scale_image_data=True)

or just:

f=astropy.open(weightsFile)

The code runs fine when working on fits files of 500Mb, however the code crashes when using a larger fitsfile (e.g. 2Gb)

The following error message occurred:

  File "/usr/local/lib/python2.7/site-packages/astropy/io/fits/hdu/image.py", line 595, in _get_scaled_image_data
    raise ValueError("Cannot load a memory-mapped image: "
ValueError: Cannot load a memory-mapped image: BZERO/BSCALE/BLANK header keywords present. Set memmap=False.

I am running my code on a Macbook pro with OS X 10.10
python 2.7.6
astropy 1.0.2

@apopping apopping changed the title problem with memmap when using latest satrapy on mac problem with memmap when using latest astropy on mac May 11, 2015
@embray embray added the io.fits label May 12, 2015
@embray

embray commented May 12, 2015

Copy link
Copy Markdown
Member

This is due to a deliberate change introduced in v1.0 via #2298.

The solution is not to explicitly request memmap=True in your fits.open() call--the best strategy for loading the image will be used automatically. I think the exception message could be clearer about that though.

@embray

embray commented May 12, 2015

Copy link
Copy Markdown
Member

Although, with do_not_scale_image_data=True you also should not be getting this exception, since in that case the scale factors are not being applied (except, perhaps, for BLANK). So this may be a bug after all. Could you post the header of the image it's crashing on?

@embray embray mentioned this pull request Jul 6, 2015
Merged
embray added a commit to embray/astropy that referenced this pull request Jul 6, 2015
@embray
Borrows a change that was also made in astropy#3916 which also happen…
1153e3d 
…s to fix astropy#3766; included a regression test.
embray added a commit to embray/astropy that referenced this pull request Jul 6, 2015
@embray
Add changelog entry for astropy#3766 [skip ci]
5949a72
@embray embray added this to the v1.0.4 milestone Jul 6, 2015
@embray
embray force-pushed the fits/issue-3766 branch from 5949a72 to b77541c Compare July 7, 2015 13:43
embray added a commit to embray/astropy that referenced this pull request Jul 7, 2015
@embray
Borrows a change that was also made in astropy#3916 which also happen…
8e5b9d1 
…s to fix astropy#3766; included a regression test.
embray added a commit to embray/astropy that referenced this pull request Jul 7, 2015
@embray
Add changelog entry for astropy#3766 [skip ci]
b77541c
@embray

embray commented Jul 7, 2015

Copy link
Copy Markdown
Member

Note to self, since I accidentally merged #3916 first, the commit that fixes this issue (after rebasing) no longer actually has the fix, and won't merge the fix into the v1.0.x branch. Instead use 1153e3d which is the pre-rebase version of this commit, which includes the actual fix.

@embray embray self-assigned this Jul 7, 2015
@embray
embray force-pushed the fits/issue-3766 branch from b77541c to b4b72c4 Compare July 7, 2015 21:37
embray added a commit to embray/astropy that referenced this pull request Jul 7, 2015
@embray
Borrows a change that was also made in astropy#3916 which also happen…
ea15092 
…s to fix astropy#3766; included a regression test.
embray added a commit to embray/astropy that referenced this pull request Jul 7, 2015
@embray
Add changelog entry for astropy#3766 [skip ci]
b4b72c4
@embray
embray force-pushed the fits/issue-3766 branch from b4b72c4 to 5cea5e0 Compare July 14, 2015 20:12
embray added a commit that referenced this pull request Jul 14, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
38a4691 
problem with memmap when using latest astropy on mac
@embray
embray merged commit 38a4691 into astropy:master Jul 14, 2015
@embray
embray deleted the fits/issue-3766 branch July 14, 2015 20:18
embray added a commit that referenced this pull request Jul 14, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
5b076e0 
problem with memmap when using latest astropy on mac
@embray

embray commented Jul 14, 2015

Copy link
Copy Markdown
Member

Backported to v1.0.x, including the actual fix.

embray added a commit that referenced this pull request Aug 5, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
8adf588 
problem with memmap when using latest astropy on mac
embray added a commit that referenced this pull request Aug 7, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
4c306a8 
problem with memmap when using latest astropy on mac
dhomeier pushed a commit to dhomeier/astropy that referenced this pull request Aug 11, 2015
@embray @dhomeier
Borrows a change that was also made in astropy#3916 which also happen…
567257b 
…s to fix astropy#3766; included a regression test.
dhomeier pushed a commit to dhomeier/astropy that referenced this pull request Aug 11, 2015
@embray @dhomeier
Add changelog entry for astropy#3766 [skip ci]
583ac7b
embray added a commit that referenced this pull request Aug 11, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
6239832 
problem with memmap when using latest astropy on mac
embray added a commit that referenced this pull request Aug 11, 2015
@embray
Merge pull request #3766 from embray/fits/issue-3766
19b0cab 
problem with memmap when using latest astropy on mac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@embray embray

Labels

Bug io.fits

Projects

None yet

Milestone

v1.0.4

Development

Successfully merging this pull request may close these issues.

1 participant

@embray