Repository navigation
ESP32: can't access esp-idf from dynamic native module #5458
Description
Activity
I'm seeing that what I'm asking for is not currently possible because the dynamically loaded modules are not really linked against the full firmware image. I.e., the only externs that can be called are the ones in
mp_fun_table. For dev purposes it might be neat if one could pull additional symbols from a specific ELF file and add to mp_fun_table, but in general that seems like a can of worms...
Do I understand all this correctly?NB: it's odd that one has to use ARCH=xtensawin for the esp32... what does "win" mean?
It means that the system ABI calling convention uses windowed registers.
For dev purposes it might be neat if one could pull additional symbols from a specific ELF file and add to mp_fun_table, but in general that seems like a can of worms...
Do I understand all this correctly?That's correct. It shouldn't actually be that hard to add more symbols, eg from the IDF. Just add the ones you need to the end of
mp_fun_table(just asvoid*'s would be fine), then also add them intools/mpy_ld.py'sfun_tabledict in the same location. Thenmpy_ld.pyshould be able to pick these symbols up and provide relocations for them when the mpy is imported.- added a commit that references this issue
on Dec 27, 2019 If this ends up being a frequently requested feature, it might be a good idea to facilitate it by easily allowing port-specific extensions to the function linking table.
Sorry for commenting on a closed bug, but I'm having problems following its resolution. I'm trying to include
mp_obj_get_arrayfrompy/obj.cin thefun_table. So I changednativeglue.clike this:... &mp_stream_unbuffered_readline_obj, &mp_stream_write_obj, mp_obj_get_array, }; #endif // MICROPY_EMIT_NATIVEand the
nativeglue.hlike this:... const mp_obj_fun_builtin_var_t *stream_unbuffered_readline_obj; const mp_obj_fun_builtin_var_t *stream_write_obj; void (*mp_obj_get_array)(mp_obj_t o, size_t *len, mp_obj_t **items); } mp_fun_table_t; extern const mp_fun_table_t mp_fun_table; #endif // MICROPY_INCLUDED_PY_NATIVEGLUE_HThen I edited
tools/mpy_ld.pylike this:... 'mp_stream_unbuffered_readline_obj', 'mp_stream_write_obj', 'mp_obj_get_array', ]) } for sym in env.unresolved_syms: assert sym['st_value'] == 0 ...and still I can't get code that uses
mp_obj_get_arrayto compile:Traceback (most recent call last): File "../../../micropython/micropython/tools/mpy_ld.py", line 965, in <module> main() File "../../../micropython/micropython/tools/mpy_ld.py", line 962, in main do_link(args) File "../../../micropython/micropython/tools/mpy_ld.py", line 939, in do_link link_objects(env, len(native_qstr_vals), len(native_qstr_objs)) File "../../../micropython/micropython/tools/mpy_ld.py", line 720, in link_objects populate_got(env) File "../../../micropython/micropython/tools/mpy_ld.py", line 385, in populate_got assert 0, (got_entry.name, got_entry.sec_name) AssertionError: ('mp_obj_get_array', '.external.mp_fun_table') ../../../micropython/micropython/py/dynruntime.mk:138: recipe for target 'build/_stage.native.mpy' failedAny advice would be appreciated.
Sorry for commenting on a closed bug, but I'm having problems following its resolution.
@deshipu no problem, it can be a tricky feature to use. I think the problem here is that the custom linker doesn't know how to relocate calls to entries in the function table. Instead you can try just referencing the function in your code using
mp_fun_table.mp_obj_get_array(...), like it's done with the other macros inpy/dynruntime.hThat worked, thank you!
Just to follow up on that point: the linker cannot currently deal with directly calling functions listed in
mp_fun_tablebecause the compiler inserts normal calls to such functions and the dynamic loader that runs on the target device cannot relocate such calls (in part because the destination may not be in range of the call machine instruction). The way around this is for the (offline) linker to automatically generate stubs for such functions and include them in the final .mpy. Eg:// this is an automatically generated stub void mp_obj_get_array(...) { mp_fun_table.mp_obj_get_array(...); }This is part of the dynamically loadable native code, and so will be in range of a normal call (and the normal call will be relocated statically to point to the stub).
I'm trying to access some esp-idf function from a dynamic native module on the esp32 and I'm not sure how to get all the include paths so I can include the appropriate header files. I suppose I will start by copy-pasting from ports/esp32/Makefile but that seems like a bad solution. As far as I can tell, the port makefiles need to be split so a piece of each can be included into the dynruntime.mk
Or perhaps I'm missing some trick altogether...
NB: it's odd that one has to use ARCH=xtensawin for the esp32... what does "win" mean?