Skip to content

Intermittent segfaults with ascii loader #3691

Closed
Closed
Intermittent segfaults with ascii loader#3691
Labels
Bugio.ascii

Description

@ChrisBeaumont
ChrisBeaumont
opened on Apr 16, 2015

My python interpreter crashes with a segfault about 10% of the time when I try to run

In [1]: from astropy.table import Table
In [2]: Table.read('test.csv', format='ascii')

when called using this file

the original file is about 10x larger, and it crashes everytime. I haven't been able to track down much more information than that. It doesn't crash if I don't set format='ascii'

Activity

  1. mdboom
    self-assigned this
    on Apr 16, 2015
  2. mdboom
    added
    io.ascii
    Bug
    on Apr 16, 2015

  3. mdboom commented on Apr 16, 2015

    @mdboom
    mdboom
    on Apr 16, 2015
    Contributor

    Hmm. Valgrind shows nothing amiss. Just to confirm -- is this with git master, or some other released version of astropy? What platform are you on, and what version of Python and Numpy?

  4. mdboom commented on Apr 16, 2015

    @mdboom
    mdboom
    on Apr 16, 2015
    Contributor

    There have been some recent fixes in this area: #3545, #3546, probably others...

  5. ChrisBeaumont commented on Apr 16, 2015

    @ChrisBeaumont
    ChrisBeaumont
    on Apr 16, 2015
    Author

    This was on conda-installed astropy 1.0.1, py 2.7.9, numpy 1.9.2, OSX. I'll check if this exists on master

  6. mdboom commented on Apr 16, 2015

    @mdboom
    mdboom
    on Apr 16, 2015
    Contributor

    FWIW, I can't reproduce this on astropy 1.0.1, python 2.7.9, numpy 1.9.2 on Linux (all from source). I don't know if OSX or anaconda matters.

    Do you have valgrind on your OSX box? The output of the following may be helpful:

    valgrind --tool=memcheck --log-file=output.vg `which python` script.py

    (where script.py contains the two lines at the top of this post).

  7. ChrisBeaumont commented on Apr 16, 2015

    @ChrisBeaumont
    ChrisBeaumont
    on Apr 16, 2015
    Author

    On master, I don't hit the segfault on the truncated gist file, but I still hit it on the full dataset. I don't have valgrind and it doesn't look like homebrew provides it for Yosemite :( . I'll try to find another way to install it and post the report here if I succeed.

  8. mdboom commented on Apr 16, 2015

    @mdboom
    mdboom
    on Apr 16, 2015
    Contributor

    Something that may also be helpful is a gdb traceback (though that's less helpful for figuring out where things first went wrong).

    gdb `which python`
> run script.py
... segfaults ...
> bt

  9. ChrisBeaumont commented on Apr 16, 2015

    @ChrisBeaumont
    ChrisBeaumont
    on Apr 16, 2015
    Author

    gdb traceback is below. Interestingly I installed valgrind but so far, the program doesn't crash or finish under valgrind

    #0  0x000000010775f3cc in htmlParseCharData () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
#1  0x000000010776652c in htmlParseTryOrFinish ()
   from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
#2  0x0000000107766b92 in htmlParseChunk () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
#3  0x0000000107592461 in __pyx_f_4lxml_5etree_11_FeedParser_close ()
   from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
#4  0x0000000107592ed3 in __pyx_pw_4lxml_5etree_11_FeedParser_3close ()
   from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
#5  0x00000001000c3f96 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#6  0x00000001000c541b in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#7  0x00000001000c541b in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#8  0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#9  0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#10 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#11 0x000000010001dfed in instancemethod_call ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#12 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#13 0x0000000100079588 in slot_tp_init () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#14 0x0000000100076405 in type_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#15 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#16 0x00000001000c0add in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#17 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#18 0x00000001000c40d0 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#19 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#20 0x00000001000c40d0 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#21 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#22 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#23 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#24 0x000000010001dfed in instancemethod_call ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#25 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#26 0x00000001000c0add in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#27 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#28 0x00000001000c40d0 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#29 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#30 0x00000001000c40d0 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#31 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#32 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#33 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#34 0x00000001000bedca in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#35 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#36 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#37 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
---Type <return> to continue, or q <return> to quit---
#38 0x00000001000bedca in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#39 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#40 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#41 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#42 0x00000001000bedca in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#43 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#44 0x00000001000c40d0 in PyEval_EvalFrameEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#45 0x00000001000c6063 in PyEval_EvalCodeEx ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#46 0x00000001000c6186 in PyEval_EvalCode () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#47 0x00000001000eb43e in PyRun_FileExFlags ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#48 0x00000001000eb6da in PyRun_SimpleFileExFlags ()
   from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#49 0x000000010010272d in Py_Main () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
#50 0x0000000100000f54 in start ()

  10. taldcroft commented on Apr 17, 2015

    @taldcroft
    taldcroft
    on Apr 17, 2015
    Member

    That's interesting, it looks like it is crashing in the HTML reader which uses libxml. This is consistent with what you've said:

    • With the format='ascii' option it goes through the full generic guess list, which will end up trying HTML before comma-separated.
    • With no ascii option then it uses the .csv suffix to determine it should call the Csv parser, which then succeeds.

    OT, about guessing again, this makes me wonder if the format keyword could be a list of acceptable formats. Right now it is either all possible formats or just one.

  11. mdboom commented on Apr 17, 2015

    @mdboom
    mdboom
    on Apr 17, 2015
    Contributor

    Interesting that the traceback is in lxml. libxml has notoriously
    difficult memory semantics, and the lxml wrappers that superceded the
    standard libxml bindings were supposed to address some of that. I didn't
    even realize io.ascii was using lxml...

    Do you know which version of lxml you have? That may explain why I can't
    reproduce...

    On Thu, Apr 16, 2015 at 7:14 PM Chris Beaumont [email protected]
    wrote:

    gdb traceback is below. Interestingly I installed valgrind but so far, the
    program doesn't crash or finish under valgrind

    #0 0x000000010775f3cc in htmlParseCharData () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #1 0x000000010776652c in htmlParseTryOrFinish ()
    from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #2 0x0000000107766b92 in htmlParseChunk () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #3 0x0000000107592461 in __pyx_f_4lxml_5etree_11_FeedParser_close ()
    from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
    #4 0x0000000107592ed3 in __pyx_pw_4lxml_5etree_11_FeedParser_3close ()
    from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
    #5 0x00000001000c3f96 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #6 0x00000001000c541b in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #7 0x00000001000c541b in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #8 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #9 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #10 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #11 0x000000010001dfed in instancemethod_call ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #12 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #13 0x0000000100079588 in slot_tp_init () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #14 0x0000000100076405 in type_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #15 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #16 0x00000001000c0add in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #17 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #18 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #19 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #20 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #21 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #22 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #23 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #24 0x000000010001dfed in instancemethod_call ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #25 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #26 0x00000001000c0add in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #27 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #28 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #29 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #30 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #31 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #32 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #33 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #34 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #35 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #36 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #37 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    ---Type to continue, or q to quit---
    #38 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #39 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #40 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #41 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #42 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #43 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #44 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #45 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #46 0x00000001000c6186 in PyEval_EvalCode () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #47 0x00000001000eb43e in PyRun_FileExFlags ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #48 0x00000001000eb6da in PyRun_SimpleFileExFlags ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #49 0x000000010010272d in Py_Main () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #50 0x0000000100000f54 in start ()

    —
    Reply to this email directly or view it on GitHub
    #3691 (comment).

  12. mdboom commented on Apr 17, 2015

    @mdboom
    mdboom
    on Apr 17, 2015
    Contributor

    And to make things even farther out of our control, astropy only uses lxml
    through BeautifulSoup. (I was wondering what it would take to replace lxml
    with our own expat-based pull parser that we already have, and
    unfortunately the answer would involve hacking BeautifulSoup...)

    On Thu, Apr 16, 2015 at 10:31 PM Michael Droettboom [email protected]
    wrote:

    Interesting that the traceback is in lxml. libxml has notoriously
    difficult memory semantics, and the lxml wrappers that superceded the
    standard libxml bindings were supposed to address some of that. I didn't
    even realize io.ascii was using lxml...

    Do you know which version of lxml you have? That may explain why I can't
    reproduce...

    On Thu, Apr 16, 2015 at 7:14 PM Chris Beaumont [email protected]
    wrote:

    gdb traceback is below. Interestingly I installed valgrind but so far,
    the program doesn't crash or finish under valgrind

    #0 0x000000010775f3cc in htmlParseCharData () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #1 0x000000010776652c in htmlParseTryOrFinish ()
    from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #2 0x0000000107766b92 in htmlParseChunk () from /Users/cbeaumont/anaconda/lib/libxml2.2.dylib
    #3 0x0000000107592461 in __pyx_f_4lxml_5etree_11_FeedParser_close ()
    from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
    #4 0x0000000107592ed3 in __pyx_pw_4lxml_5etree_11_FeedParser_3close ()
    from /Users/cbeaumont/anaconda/lib/python2.7/site-packages/lxml/etree.so
    #5 0x00000001000c3f96 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #6 0x00000001000c541b in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #7 0x00000001000c541b in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #8 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #9 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #10 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #11 0x000000010001dfed in instancemethod_call ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #12 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #13 0x0000000100079588 in slot_tp_init () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #14 0x0000000100076405 in type_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #15 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #16 0x00000001000c0add in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #17 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #18 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #19 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #20 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #21 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #22 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #23 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #24 0x000000010001dfed in instancemethod_call ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #25 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #26 0x00000001000c0add in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #27 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #28 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #29 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #30 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #31 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #32 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #33 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #34 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #35 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #36 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #37 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    ---Type to continue, or q to quit---
    #38 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #39 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #40 0x000000010003e4d0 in function_call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #41 0x000000010000cab2 in PyObject_Call () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #42 0x00000001000bedca in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #43 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #44 0x00000001000c40d0 in PyEval_EvalFrameEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #45 0x00000001000c6063 in PyEval_EvalCodeEx ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #46 0x00000001000c6186 in PyEval_EvalCode () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #47 0x00000001000eb43e in PyRun_FileExFlags ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #48 0x00000001000eb6da in PyRun_SimpleFileExFlags ()
    from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #49 0x000000010010272d in Py_Main () from /Users/cbeaumont/anaconda/lib/libpython2.7.dylib
    #50 0x0000000100000f54 in start ()

    —
    Reply to this email directly or view it on GitHub
    #3691 (comment).

  13. embray commented on Apr 17, 2015

    @embray
    embray
    on Apr 17, 2015
    Member

    This sounds almost exactly like #3611 and #3280, both of which have remained mysteries.

  14. embray commented on Apr 17, 2015

    @embray
    embray
    on Apr 17, 2015
    Member

    Just to confirm, looking at @tomr-stargazer's report in #3611, it's crashing in the exact same place.

  15. 22 remaining items

  16. embray commented on May 14, 2015

    @embray
    embray
    on May 14, 2015
    Member

    So it looks like libxml is only used if the lxml module is available (if not, BeautifulSoup has its own built-in fallbacks) so that's a start. Using libxml straight up doesn't crash on this file, but looking at the HTMLParser class in lxml it looks like it does some slightly complicated things. The problem might exist somewhere between lxml and libxml.

  17. embray commented on May 14, 2015

    @embray
    embray
    on May 14, 2015
    Member

    Okay, when I try to reproduce this, when io.ascii goes into beautifulsoup, the attempt to parse the "HTML" file just results in a UnicodeDecodeError:

    >>> soup = BeautifulSoup('\n'.join(lines))
ERROR: UnicodeDecodeError: 'utf8' codec can't decode byte 0x88 in position 0: invalid start byte [unknown]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/internal/1/root/home/embray/.virtualenvs/lxml/lib/python2.7/site-packages/bs4/__init__.py", line 172, in __init__
    self._feed()
  File "/internal/1/root/home/embray/.virtualenvs/lxml/lib/python2.7/site-packages/bs4/__init__.py", line 185, in _feed
    self.builder.feed(self.markup)
  File "/internal/1/root/home/embray/.virtualenvs/lxml/lib/python2.7/site-packages/bs4/builder/_lxml.py", line 175, in feed
    self.parser.close()
  File "parser.pxi", line 1322, in lxml.etree._FeedParser.close (src/lxml/lxml.etree.c:111100)
  File "parser.pxi", line 1352, in lxml.etree._FeedParser.close (src/lxml/lxml.etree.c:110937)
  File "parsertarget.pxi", line 142, in lxml.etree._TargetParserContext._handleParseResult (src/lxml/lxml.etree.c:126528)
  File "parsertarget.pxi", line 130, in lxml.etree._TargetParserContext._handleParseResult (src/lxml/lxml.etree.c:126294)
  File "lxml.etree.pyx", line 324, in lxml.etree._ExceptionContext._raise_if_stored (src/lxml/lxml.etree.c:10737)
  File "saxparser.pxi", line 499, in lxml.etree._handleSaxData (src/lxml/lxml.etree.c:120816)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x88 in position 0: invalid start byte

    which doesn't make a whole lot of sense either.

    We have the same version of libxml, but I wonder if I'm using a different version of lxml and/or BS?

  18. embray commented on May 14, 2015

    @embray
    embray
    on May 14, 2015
    Member

    @ChrisBeaumont Could you also check what lxml.etree.__version__ is? Thanks.

  19. tomr-stargazer commented on May 14, 2015

    @tomr-stargazer
    tomr-stargazer
    on May 14, 2015

    Since I was tagged above as reporting a similar issue, here's my version numbers (in case that's helpful - otherwise, sorry for the spam). My install is on a Mac OS X, via anaconda.

    In [1]: import lxml.etree
In [2]: print(lxml.etree.LIBXML_VERSION)
(2, 9, 0)

In [3]: print(lxml.etree.__version__)
3.3.1

  20. adrn commented on May 14, 2015

    @adrn
    adrn
    on May 14, 2015
    Member

    And since I have been experiencing this as well, here are my version:

    In [1]: import lxml.etree
In [2]: print(lxml.etree.LIBXML_VERSION)
(2, 9, 0)

In [3]: print(lxml.etree.__version__)
3.2.3

  21. embray commented on May 14, 2015

    @embray
    embray
    on May 14, 2015
    Member

    Thanks, that might help narrow it down more.

  22. embray commented on May 14, 2015

    @embray
    embray
    on May 14, 2015
    Member

    Well this is interesting--with lxml 3.2.3 I'm not getting a segfault, unfortunately, but I am getting that same UnicodeDecodeError as above. However, it occurs at different locations, sometimes with different invalid bytes (that shouldn't be appearing in the data at all), and sometimes it just works fine. I'm not sure why I don't get a segfault, but at least it's something. Are you all using Anaconda, and if so do you know what version/platform?

  23. tomr-stargazer commented on May 14, 2015

    @tomr-stargazer
    tomr-stargazer
    on May 14, 2015
    Python 2.7.9 |Anaconda 1.9.1 (x86_64)| (default, Dec 15 2014, 10:37:34)
Type "copyright", "credits" or "license" for more information.

IPython 2.4.1 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
Using matplotlib backend: MacOSX

  24. adrn commented on May 14, 2015

    @adrn
    adrn
    on May 14, 2015
    Member
    Python 2.7.9 |Anaconda 1.8.0 (x86_64)| (default, Dec 15 2014, 10:37:34)
Type "copyright", "credits" or "license" for more information.

IPython 3.1.0 -- An enhanced Interactive Python.

  25. ChrisBeaumont commented on May 14, 2015

    @ChrisBeaumont
    ChrisBeaumont
    on May 14, 2015
    Author

    Could you also check what lxml.etree.version is?

    3.4.2

    Python 3.4.3, Anaconda 2.2.0, IPython 3.0.0

  26. bsipocz commented on Mar 18, 2019

    @bsipocz
    bsipocz
    on Mar 18, 2019
    Member

    I suspect/hope this has gone away in the past years, too? @taldcroft @eteq what do you think, can we close?

  27. bsipocz
    unassigned
    mdboom
    on Mar 18, 2019

  28. taldcroft commented on Mar 18, 2019

    @taldcroft
    taldcroft
    on Mar 18, 2019
    Member

    Another option is the long-standing issue to improve guessing heuristics so a valid table file that is not an HTML table never gets to the HTML reader.

    This was done, and I think it would prevent this issue. In any case we have not gotten new reports in the mean time.

  29. taldcroft
    closed this as completedon Mar 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugio.ascii

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions