Skip to content

[Python] pyarrow 24.0.0 regression: co-loading a native extension bundling mimalloc v3 SIGSEGVs in bundled mimalloc at interpreter teardown (macOS arm64, py3.14); 22/23 unaffected #50428

Closed
Closed
[Python] pyarrow 24.0.0 regression: co-loading a native extension bundling mimalloc v3 SIGSEGVs in bundled mimalloc at interpreter teardown (macOS arm64, py3.14); 22/23 unaffected#50428
Assignees
pitrou
Labels
Component: Pythonbackport-candidate
Milestone
25.0.1

Description

@kkollsga
kkollsga
opened on Jul 8, 2026

Describe the bug, including details regarding any error messages, version, and platform.

Environment: macOS 15 arm64, CPython 3.14.3 (python.org), pyarrow from PyPI.

python3.14 -c "import pyarrow; import kglite; kglite.KnowledgeGraph()"
# pyarrow 24.0.0 → exit 139 (SIGSEGV at teardown); 23.0.1 / 22.0.0 → exit 0

kglite is a PyO3/abi3 Rust extension that statically links mimalloc v3 as its Rust global allocator (a very common Rust-wheel pattern). It links no Arrow code and exports one symbol. Import alone (no pyarrow API use, no kglite↔pyarrow data exchange) arms the crash; ARROW_DEFAULT_MEMORY_POOL=system does not help; os._exit(0) avoids it (teardown-only).

Crash: EXC_BAD_ACCESS inside libarrow's bundled mimalloc (_mi_theap_collect_retired + 124, lldb). Process census via MIMALLOC_VERBOSE: CPython 3.14 vendors mimalloc v3.3.2, the extension carries v3.3.1, libarrow 2400 carries a v2-line copy. Switching the extension's copy to v2 — or downgrading pyarrow to ≤23.0.1 — eliminates the crash; so something introduced in 24.0.0 (bundled-mimalloc bump? teardown/atexit registration change?) makes the pre-existing three-allocator situation fatal.

Impact: any Rust wheel shipping a mimalloc-v3 global allocator (the mimalloc crate defaults to v3 since libmimalloc-sys 0.1.47) can now crash any Python ≥3.13 process it shares with pyarrow 24 — at exit, i.e. flaky-looking CI failures and exit-139s with no traceback.

Cross-filed with microsoft/mimalloc (instance-isolation angle): microsoft/mimalloc#1327

Minimal repro wheels + lldb transcript available on request. A coexistence canary in Arrow's Python CI (import pyarrow alongside a mimalloc-v3 wheel, assert clean exit) would catch this class before release; we've added the mirror-image canary on our side.

Component(s)

Python

Activity

  1. github-actions
    added
    Component: Python
    on Jul 8, 2026
  2. kkollsga
    added a commit that references this issue on Jul 8, 2026

    release(0.12.12): pyarrow coexistence + dataset-fetch standardization

    c6c7d34

  3. raulcd commented on Jul 9, 2026

    @raulcd
    raulcd
    on Jul 9, 2026
    Member

    Could you share a gdb stacktrace?

  4. kkollsga commented on Jul 9, 2026

    @kkollsga
    kkollsga
    on Jul 9, 2026
    Author

    I'm on macOS (arm64), so here is the crash data from the macOS crash reporter, which unwinds the full teardown chain:

    EXC_BAD_ACCESS (SIGSEGV) KERN_INVALID_ADDRESS at 0x0000000000000017

#0 libarrow.2400.dylib  _mi_theap_collect_retired + 124
#1 libarrow.2400.dylib  mi_theap_collect_ex + 72
#2 libarrow.2400.dylib  mi_process_done + 76
#3 libsystem_c.dylib    __cxa_finalize_ranges + 480
#4 libsystem_c.dylib    exit + 44
#5 libdyld.dylib        dyld4::LibSystemHelpers::exit(int) const + 20
#6 dyld                 dyld4::LibSystemHelpersWrapper::exit(int) const + 172
#7 dyld                 start + 7232

    So libarrow's bundled mimalloc crashes in its mi_process_done atexit handler during __cxa_finalize, walking a thread-heap page list that contains a corrupted entry.

    Supporting detail from the same macOS crash report:

    • Faulting instruction (lldb disasm): ldrb w10, [x21, #0x16] with register x21 = 0x1 → fault address 0x17. The page pointer being walked is literally 0x1.
    • vmRegionInfo: "0x17 is not in any region" — a wild near-null pointer, not a mapped-but-protected page.
    • Register state at fault: x19 = 0x1195300c0, x20 = 0x1, x21 = 0x1, x24 = 0x9, x25 = 0x4a — consistent with a list walk hitting a sentinel-like value (0x1) where a mi_page_t* was expected. Possibly relevant: with multiple mimalloc instances in-process, one instance's static empty-page sentinel is a foreign address to the other.

    Two additional data points that may help localize it:

    • Under heavy allocation the same corruption also fires mid-operation rather than at exit (a second crash report shows the fault inside the co-loaded extension during object construction, KERN_PROTECTION_FAILURE on a stack guard page). Teardown is just the most reproducible manifestation.
    • Version matrix: the crash requires pyarrow 24.0.0 and a co-loaded extension linking mimalloc v3, on CPython ≥3.13 (which vendors mimalloc v3 itself). pyarrow 23.0.1/22.0.0 are clean with the identical extension, and rebuilding the extension on mimalloc v2 is also clean against 24.0.0 — so whatever changed in 24.0.0's bundled allocator (or its teardown registration) interacts specifically with a second v3 instance in the process.

  5. raulcd commented on Jul 9, 2026

    @raulcd
    raulcd
    on Jul 9, 2026
    Member

    cc @pitrou

  6. pitrou commented on Jul 9, 2026

    @pitrou
    pitrou
    on Jul 9, 2026
    Member

    PyArrow 25.0.0 will ship with mimalloc 3.3.1, so perhaps the issue will be solved then? (though it would be nice to ensure that this doesn't regress)

    You can install a development wheel of PyArrow to try it out from here:
    https://anaconda.org/channels/scientific-python-nightly-wheels/packages/pyarrow/overview

  7. kkollsga commented on Jul 9, 2026

    @kkollsga
    kkollsga
    on Jul 9, 2026
    Author

    Nightly (25.0.0.dev257) tested: crash gone. The extension that reliably SIGSEGVs with 24.0.0 is clean across repeated runs, both import orders.

    The mechanism is mimalloc ≥ 3.2.6's fixed macOS TLS slots, shared by all co-resident v3 instances with no ownership check (microsoft/mimalloc#1327). 24.0.0's 3.2.7 crashes against a 3.3.1 extension because their heap layouts differ; 25.0.0's 3.3.1 coexists because the layouts happen to match. A pairing where layouts diverge again would re-expose the crash. Bundled ≥ 3.2.6 is the exposure window, which is why 23.0.0 with 3.1.5 was unaffected.

  8. pitrou commented on Jul 9, 2026

    @pitrou
    pitrou
    on Jul 9, 2026
    Member

    Based on microsoft/mimalloc#1327 (comment) and the following messages, it seems we could enable MI_TLS_MODEL_LOCAL when building Arrow's mimalloc, at least on Apple.

  9. github-actions
    assigned
    pitrou
    on Jul 20, 2026
  10. raulcd
    added a commit that references this issue on Jul 20, 2026

    apacheGH-50428: [C++] Better mimalloc configuration on macOS

    5d4c429
  11. pitrou
    added a commit that references this issue on Jul 21, 2026

    GH-50428: [C++] Better mimalloc configuration on macOS (#50549)

    0077539
  12. pitrou
    added this to the 26.0.0 milestone on Jul 21, 2026

  13. pitrou commented on Jul 21, 2026

    @pitrou
    pitrou
    on Jul 21, 2026
    Member

    Issue resolved by pull request 50549
    #50549

  14. pitrou
    closed this as completedon Jul 21, 2026
  15. raulcd
    added
    backport-candidate
    on Jul 21, 2026
  16. raulcd
    modified the milestones: 26.0.0, 25.0.1 on Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: Pythonbackport-candidate

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions