Repository navigation
ports/embed: segfault in mp_compile() #11781
Description
Activity
This might precisely be the reason that the unix port does
mp_stack_ctrl_initas the very first thing.Remove mp_stack_ctrl_init() from mp_embed_init() and require the application itself to call it
Probably the way to go, though should be configurable perhaps, and in any case an error/exit should be generated immediately if someone forgets to call it since otherwise the resulting errors might be quite obscure.
FWIW I've been using a function similar to mp_embed_init for years and never had stack issues. Actually, also in light of #11430, looking at the current state where there's a myriad of variations on init/exec_str/exec_mpy/deinit/... spread over ports and utility functions I think it would be good to try and unify this into common functions which are either configurable at compiletime or else at runtime. Also by means of documentation. Take mp_embed_exec_str for example: it's there, it's an example, but once you actually want to use it it's going to take a couple of minutes and then you realize you need the return value and you're going to turn to one of th 5 or 10 other implementations which are all tiny variations on the same thing. Next you want to use MICROPYPATH and have to copy that from the unix port (or so), and so on.
I have a reproducer: cwalther@b404d15. It’s not a segfault but an assert this time (apparently the thing the pointer is overwritten with isn’t that outlandish, but still sufficiently wrong), but that should be enough to illustrate the problem. The added
printf()s are not important, they were just useful to nudge the garbage collections into the right spot.Build the
embeddingexample in the usual way withmake -f micropython_embed.mk && makeand run it.Actual result:
cwalther@tartan embedding % ./embed PARSE COMPILE RUN hello world! [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32]eol DONE PARSE COMPILE COLLECT Assertion failed: (ATB_GET_KIND(area, block) == AT_HEAD), function gc_free, file gc.c, line 776. zsh: abort ./embedExpected result:
cwalther@tartan embedding % ./embed PARSE COMPILE RUN hello world! [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32]eol DONE PARSE COMPILE COLLECT RUN iter 00000000 iter 00000001 iter 00000002 iter 00000003 iter 00000004 iter 00000005 iter 00000006 iter 00000007 iter 00000008 iter 00000009 caught exception ZeroDivisionError('divide by zero',) run GC collect COLLECT hello world! f 1 hello world! f 2 hello world! f 3 hello world! f 4 hello world! f 5 hello world! f 6 hello world! f 7 hello world! f 8 hello world! f 9 hello world! f 10 hello world! f 11 hello world! f 12 hello world! f 13 hello world! f 14 hello world! f 15 hello world! f 16 hello world! f 17 hello world! f 18 hello world! f 19 hello world! f 20 finish DONEThe expected result can be achieved by moving the call of
mp_stack_ctrl_init()as proposed: cwalther@60a889e- added a commit that references this issue
on Feb 14, 2024 - added a commit that references this issue
on Mar 19, 2024 - added a commit that references this issue
on Apr 21, 2024 - added a commit that references this issue
on Sep 21, 2024 - added a commit that references this issue
on Oct 19, 2025 - added a commit that references this issue
on May 23, 2026 - added a commit that references this issue
on Sep 21, 2026
There is a bug in the
embedport that can cause segfaults while compiling Python code in memory-constrained situations. I have not managed to construct a simple reproducer yet, since it depends on memory allocation patterns, but the root cause is easy to verify, so I’m reporting it already.mp_stack_ctrl_init(), called two levels deep inmain()→mp_embed_init()→mp_stack_ctrl_init(), captures a too lowstack_top, so that the local variableparse_treeofmp_embed_exec_str(), called only one level deep inmain()→mp_embed_exec_str(), ends up outside of the stack range looked at by the garbage collector ingc_helper_collect_regs_and_stack(). When garbage collection is invoked at just the wrong moment during compilation, the blocks pointed at by the contents ofparse_treeare therefore freed prematurely, and when they are later reused and overwritten, that corrupts the pointers inside them, which is likely to lead to memory access failures inmp_parse_tree_clear()at the latest.This is easily verified with
This assert fails for me on macOS 13 x86_64, compiling with clang 14.
I am not sure how to best fix this. Ideas so far:
mp_stack_ctrl_init()frommp_embed_init()and require the application itself to call it, at the same or higher level than it will later callmp_embed_exec_str().mp_stack_ctrl_init()apply some offset to the detected stack top to account for its own call depth. I have no idea how to generically and portably determine what that offset should be, though. (Underestimating will fail to fix the problem, overestimating might run into unmapped memory above the stack.)mp_embed_exec_str()could check whetherparse_treeis abovestack_topand if so adjust the latter.Any opinions?