Repository navigation
Exposing more types to native modules #5641
Description
Activity
You need to make sure the changes are reflected in this table: https://github.com/micropython/micropython/blob/master/tools/mpy_ld.py#L666-L682
Yes, I've added the symbols to that (unfortunately it doesn't fix the issue, so I think I have a problem elsewhere).
I would have thought that without them I should hit the LinkError on line 704 anyway?
Just some more notes for me to refer to later:
after building micropython with
DEBUG=1,paholeshows that the exported symbol table has the correct structure:pahole --reorganize -C _mp_fun_table_t micropythongives... mp_obj_t (*obj_new_str)(const char *, size_t); /* 456 8 */ mp_obj_t (*obj_new_bytes)(const byte *, size_t); /* 464 8 */ mp_obj_t (*obj_new_bytearray_by_ref)(size_t, void *); /* 472 8 */ mp_obj_t (*obj_new_float_from_f)(float); /* 480 8 */ mp_obj_t (*obj_new_float_from_d)(double); /* 488 8 */ float (*obj_get_float_to_f)(mp_obj_t); /* 496 8 */ double (*obj_get_float_to_d)(mp_obj_t); /* 504 8 */ /* --- cacheline 8 boundary (512 bytes) --- */ void (*get_buffer_raise)(mp_obj_t, mp_buffer_info_t *, mp_uint_t); /* 512 8 */ const mp_stream_p_t * (*get_stream_raise)(mp_obj_t, int); /* 520 8 */ const mp_print_t * plat_print; /* 528 8 */ const mp_obj_type_t * type_type; /* 536 8 */ const mp_obj_type_t * type_int; /* 544 8 */ const mp_obj_type_t * type_float; /* 552 8 */ const mp_obj_type_t * type_str; /* 560 8 */ const mp_obj_type_t * type_bytes; /* 568 8 */ /* --- cacheline 9 boundary (576 bytes) --- */ const mp_obj_type_t * type_list; /* 576 8 */ const mp_obj_type_t * type_dict; /* 584 8 */ const mp_obj_type_t * type_fun_builtin_0; /* 592 8 */ const mp_obj_type_t * type_fun_builtin_1; /* 600 8 */ const mp_obj_type_t * type_fun_builtin_2; /* 608 8 */ const mp_obj_type_t * type_fun_builtin_3; /* 616 8 */ const mp_obj_type_t * type_fun_builtin_var; /* 624 8 */ const mp_obj_fun_builtin_var_t * stream_read_obj; /* 632 8 */ ...And indeed the table is still lined up with the
mp_ld.pyoffset of 67 * 8 = mp_type_type.modifying mp_ld.py to print out symbol offsets shows that
mp_type_fun_builtin_1is indeed being mapped to offset 576 in the native module, rather than the correct offset of 600.But in this process I actually just found my dumb mistake, see if you can spot it:
mp_fun_table_sec = Section('.external.mp_fun_table', b'', 0) fun_table = {key: 67 + idx for idx, key in enumerate([ 'mp_type_type', 'mp_type_int', 'mp_type_float' 'mp_type_str', 'mp_type_bytes' 'mp_type_list', 'mp_type_dict', 'mp_type_fun_builtin_0', 'mp_type_fun_builtin_1', 'mp_type_fun_builtin_2', 'mp_type_fun_builtin_3', 'mp_type_fun_builtin_var', 'mp_stream_read_obj', 'mp_stream_readinto_obj', 'mp_stream_unbuffered_readline_obj', 'mp_stream_write_obj', ]) }But in this process I actually just found my dumb mistake, see if you can spot it:
Ok, so you missed commas!
It might be a good idea to generate this table in
mpy_ld.pyautomatically (when that script runs) by parsingpy/nativeglue.h.Ah, didn't see this in time. I just created a PR with all of the types from obj.h.
I'll need to think a bit about parsing, because from memory
pycparsingdoesn't like macros and micropython obviously uses them a lot.
I'm trying to expose the
mp_type_int,mp_type_floatandmp_type_bytesto native modules. I have done the following things:mp_fun_table_tinnativeglue.hto add the types:mp_fun_tableinnativeglue.cto set the pointer values:Unfortunately I seem to be getting an odd error where the functions are no longer hooked up correctly, I suspect that this is because there is an offset in the exported function table somewhere which has not been updated, so
mp_type_fun_builtin_1is now mapped tomp_type_list.I also noticed that in
mpy_ld.pythere is a section which specifically resolves some other type symbols. Adjusting this to also add my new types did not solve the problem unfortunately (and I'm not getting a linker error anyway...)Does anyone know where the table is that I am not updating properly?