astropy.table.Table.read(file, format='ascii') occasionally segfaults

Hi all,

I have an error where I encounter segfaults sometimes when using `astropy.table.Table.read(file, format='ascii'). Since the segfault automatically exits the python interpreter, I don't have a traceback to provide. However, the error seems to occur more frequently when I call the reader multiple times, on somewhat large (~thousands of lines) ascii files.

At the interpreter, I get this:

/Users/tsrice/anaconda/bin/python.app: line 3: 37109 Segmentation fault: 11  /Users/tsrice/anaconda/python.app/Contents/MacOS/python "$@"

Mac OS X gave me a crash report, which I've pasted here: https://gist.github.com/tomr-stargazer/d9dde76e6bea5387dcf6

Please let me know if I can provide more information - again, I'm not used to reporting segfaults as python errors, so I'm a little out of my element here.

Also, version information:

➜  python
Python 2.7.9 |Anaconda 1.9.1 (x86_64)| (default, Dec 15 2014, 10:37:34)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import astropy
>>> astropy.__version__
u'1.0'
>>> import numpy
>>> numpy.__version__
'1.9.2'

@tomr-stargazer

@tomr-stargazer - could you provide example files that have definitely crashed in the past, even if the error is non-deterministic? Note that there were some segfaults that were fixed in astropy 1.0.1 so you might want to try updating: https://github.com/astropy/astropy/blob/master/CHANGES.rst#101-2015-03-06

Hi Tom,

I will definitely now update to the new astropy version, I didn't realize that my issue had been possibly already solved! Thanks for the quick reply!

There are two instances where I had been hitting the segfault: one where I was reading in a table from disk, and one where I was using astropy.table.Table.read() to turn a list of strings into an astropy table.

~~Table from disk~~:
~~https://gist.github.com/tomr-stargazer/50bd41566cb35434c21a/~~
Actually this one:
https://gist.github.com/tomr-stargazer/8617fee62c1190e81e62

Table from list of strings:
https://gist.github.com/tomr-stargazer/4c378e741ec70575b4da

Once I update, I'll let you know if I get this issue anymore. (The frustrating thing is that the segfault was definitely non-deterministic; sometimes I could run the literal same code twice, and have it crash on the second time after running it cleanly the first time.)

Okay, I've upgraded to astropy 1.0.1, but I am still able to reproduce the segfault in my production code.

>>> astropy.__version__
u'1.0.1'

So it seems that my bug was not fixed here.

@tomr-stargazer

@tomr-stargazer - thanks - does the issue happen systematically, or sporadically? Could you post here the header from python -c 'import astropy; astropy.test()', just so we can see all the system info, etc.?

➜  python -c 'import astropy; astropy.test()'
============================= test session starts ==============================
platform darwin -- Python 2.7.9 -- pytest-2.5.1

Running tests with Astropy version 1.0.1.
Running tests in /Users/tsrice/anaconda/lib/python2.7/site-packages/astropy.

Platform: Darwin-13.4.0-x86_64-i386-64bit

Executable: /Users/tsrice/anaconda/bin/python

Full Python Version:
2.7.9 |Anaconda 1.9.1 (x86_64)| (default, Dec 15 2014, 10:37:34)
[GCC 4.2.1 (Apple Inc. build 5577)]

encodings: sys: ascii, locale: UTF-8, filesystem: utf-8, unicode bits: 15
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.9.2
Scipy: 0.14.0
Matplotlib: 1.4.0
h5py: 2.4.0

The issue happens systematically in the following sense:
When I open up the ipython --pylab interpreter and run some production scripts that involve reading files in, I encounter a segfault on-average 50% of the times. The other 50% of the time, the code seems to execute as expected, with no error. (It's bizarre.)
Sometimes, the code will segfault on the first try; sometimes, it will work fine in the first time it's run in the interpreter, but then I re-run the same code (call the same function with the same arguments) and cause a segfault.

@tomr-stargazer

@tomr-stargazer - just to check, did you install astropy with conda?

Yes.

@mdmueller

@mdmueller @taldcroft - any idea how to go about debugging this?

One more comment: some of my scripts actually predate astropy, and I had been using atpy.Table(filename, type='ascii') without any segfaults. I understand that atpy now uses astropy internally, and it's only since that migration that I noticed these segfaults occurring.

Hm, it's probably something with the fast reader...I can't reproduce this myself. Have you tried reading with fast_reader=False?

Horrifically, it still segfaults with fast_reader=False:

# I have put a pdb.set_trace above the offending line of code --
-> kd_output = astropy.table.Table.read(lines_of_data, format='ascii', fast_reader=False)
(Pdb) c
/Users/tsrice/anaconda/bin/python.app: line 3: 40792 Segmentation fault: 11  /Users/tsrice/anaconda/python.app/Contents/MacOS/python "$@"

@taldcroft

Somewhat unrelated to this, but I just noticed something interesting--when format is unspecified for a basic text file, the slow reader takes precedence over the fast reader! @taldcroft - was this an intended change? If not, I can open a new PR to change it. The culprit seems to be this line. EDIT: regardless of whether or not this was intended, I think this is a good indication that we need a test somewhere ensuring that the fast reader is called when fast_reader=True.

In terms of this issue, I guess it isn't the fast reader at fault. I have no serious idea of what it could be at the moment--maybe a memory issue?

@mdmueller

I linked to a crash log above, with this possibly-relevant section excerpted:

Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000010beefd0d

VM Regions Near 0x10beefd0d:
    MALLOC_LARGE           000000010beb5000-000000010beda000 [  148K] rw-/rwx SM=COW  
--> 
    MALLOC_TINY            000000010bf00000-000000010c100000 [ 2048K] rw-/rwx SM=PRV

which seems to bolster the idea from @mdmueller that it's a memory issue. (Caveat: I don't work with C code or such crash logs very often, so I may have totally misinterpreted this.)

In the meantime, while this remains open, does anyone have suggestions for how I can turn a list of strings into an astropy.table.Table without having to rely on the buggy astropy.table.Table.read(..., format='ascii')? Perhaps numpy.loadtxt might help out?

Yes, numpy.loadtxt should work, though you will have to specify the data types in the table (or else use genfromtxt with dtype=None). Another thing to try, if you haven't already, is giving a specific format to Table.read, such as format="basic" or format="fixed_width". Not all formats have fast readers, but if you specify one of the "fast" formats it should go directly to the fast reader. I'm curious if this problem exists for the fast reader as well

Note that you have to add the prefix ascii. if you use Table.read, so e.g. format='ascii.fixed_width' (unlike if you use ascii.read where format='fixed_width' is fine)

@astrofrog

Oops my mistake, thanks @astrofrog.

OSX has a ton of memory allocation/management bugs. I wouldn't be surprised if this isn't an Astropy issue at all.

@astrofrog

Hi @astrofrog @mdmueller, when I use format='ascii.no_header' (a special case of ascii.basic if I understand correctly), my segfault issues completely go away (for one of the two cases where I was having problems). So - in that instance, switching ascii readers solved my problem.

In the other case, (where I had some old legacy code using atpy), the problem is more intermittent and it's not critical that I solve it. So - my immediate concerns are gone now.

@taldcroft

I think this is going to be related once again to the fact that guessing takes up a lot more memory than the final table will take, and I'm not sure if there's much we can do about that :-/

cc @taldcroft

@astrofrog

@astrofrog @taldcroft or anyone else: do you have a good idea what to do with these old issues? They may have been gone as there was no new comments in the last 4 years on this, it was difficult to reproduce though, so they still may be there.

I think we can close those, the code has changed a lot and there are new options like using chunked reading or the pandas backends.

Navigation Menu

Repository navigation

astropy.table.Table.read(file, format='ascii') occasionally segfaults #3611

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

astropy.table.Table.read(file, format='ascii') occasionally segfaults #3611

Description

Activity

tomr-stargazer commented on Mar 21, 2015

astrofrog commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

astrofrog commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

astrofrog commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

astrofrog commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

mdmueller commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

mdmueller commented on Mar 21, 2015

tomr-stargazer commented on Mar 21, 2015

tomr-stargazer commented on Mar 23, 2015

mdmueller commented on Mar 23, 2015

astrofrog commented on Mar 23, 2015

mdmueller commented on Mar 23, 2015

embray commented on Mar 24, 2015

tomr-stargazer commented on Mar 25, 2015

astrofrog commented on Mar 25, 2015

bsipocz commented on Mar 18, 2019

taldcroft commented on Mar 18, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions