Repository navigation
esp: any way to save code objects to .mpy file ? #4187
Description
Activity
The C-code for this is already there in
persistentcode.cbut as of the latest commits, it's not exposed to Python.What does repr do ?
Seems easier to provide add buffer protocol to the function object type so it can be used with file write.
I am evaluating to expose these functions via a CPython module
A CPython module (named "mpy", for example, or something similar) with some wrapping function for: mp_lexer, mp_parse, mp_compile_to_raw_code, mp_raw_code_save*, mp_raw_code_load_*
These functions can be very handy for a tool-chain that manages mpy code
The only issue is to build an .so (for the CPython) with libmicropython inside ...This sounds more like “how do I do this” rather than “this is a bug” and is more appropriate for the forum.
Reply to this thread with the form URL.
@depaolim the cpython counter part module is called "marshal" https://docs.python.org/3/library/marshal.html, so maybe "umarshal" would be the name, but why would you want to link from cpython as you can just popen mpy-cross ?
Anyway adding the buffer protocol to code object seems the way, though i can't find (yet) where the "code" object class members are defined.
Anyway adding the buffer protocol to code object seems the way, though i can't find (yet) where the "code" object class members are defined.
const mp_obj_type_t mp_type_fun_bc = { { &mp_type_type }, .name = MP_QSTR_function, #if MICROPY_CPYTHON_COMPAT .print = fun_bc_print, #endif .call = fun_bc_call, .unary_op = mp_generic_unary_op, #if MICROPY_PY_FUNCTION_ATTRS .attr = fun_bc_attr, #endif };
thx again, i get it now and i've found the relevant place in builtinevex.c. after where comes how ;)
edit/ code->bytecode does not look to have the same prelude as raw_code , is there function to convert or some pointers about the differences ?
Thanks for your answers, I will reconsider my use case
please consider my question as closedHere's how I implemented
os.compile(), a method to compile a.pyfile and save it to the file system as a.mpyfile. You'll have to come up with your ownraw_code_save_file()andfile_write_strn()to match the file system APIs on your target. And you could pass anymp_raw_code_tstructure intoraw_code_save_file()to store it as a.mpy.Feel free to incorporate this into your program, or to integrate it into upstream MicroPython.
// os.compile(source_filename, mpy_filename=None) // Compile source_filename and store as mpy_filename. If mpy_filename set // to None, uses source_filename with .py extension removed (if present) and // then appending .mpy. STATIC mp_obj_t os_compile(size_t n_args, const mp_obj_t *args) { const char *source_file = mp_obj_str_get_str(args[0]); vstr_t vstr; const char *mpy_file; if (n_args == 2 && args[1] != mp_const_none) { mpy_file = mp_obj_str_get_str(args[1]); } else { vstr_init(&vstr, strlen(source_file) + 5); // +5 for NUL and .mpy vstr_add_str(&vstr, source_file); if (vstr.len > 3 && memcmp(&vstr.buf[vstr.len - 3], ".py", 3) == 0) { // remove .py extension to replace with .mpy vstr_cut_tail_bytes(&vstr, 3); } vstr_add_str(&vstr, ".mpy"); mpy_file = vstr_null_terminated_str(&vstr); } gc_collect(); mp_micropython_mem_info(0, NULL); mp_printf(MP_PYTHON_PRINTER, "Parsing %s...\n", source_file); mp_parse_tree_t parse_tree = mp_parse(mp_lexer_new_from_file(source_file), MP_PARSE_FILE_INPUT); gc_collect(); mp_micropython_mem_info(0, NULL); mp_printf(MP_PYTHON_PRINTER, "Compiling...\n"); mp_raw_code_t *rc = mp_compile_to_raw_code(&parse_tree, qstr_from_str(source_file), MP_EMIT_OPT_NONE, false); gc_collect(); mp_micropython_mem_info(0, NULL); mp_printf(MP_PYTHON_PRINTER, "Saving %s...\n", mpy_file); raw_code_save_file(rc, mpy_file); return mp_const_none; } MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(mod_os_compile_obj, 1, 2, os_compile); // Implementation of mp_print_strn_t API, with the fh_t cast to (void *). void file_write_strn(void *data, const char *str, size_t len) { fh_t file_handle = (fh_t)(intptr_t)data; RTOS_ERR rtos_err; CPU_SIZE_T bytes_written; bytes_written = file_write(file_handle, str, len, &rtos_err); (void) bytes_written; // should we check for a short write? check_rtos_err_raise(file_handle, RTOS_ERR_CODE_GET(rtos_err)); } // Save .mpy file to file system int raw_code_save_file(mp_raw_code_t *rc, const char *filename) { RTOS_ERR rtos_err; fh_t file_handle = file_open(wrk_dir_handle, filename, ACCESS_MODE_WR | ACCESS_MODE_CREATE | ACCESS_MODE_TRUNCATE, &rtos_err); check_rtos_err_raise(file_handle, RTOS_ERR_CODE_GET(rtos_err)); mp_print_t code_print = { (void*)(intptr_t)file_handle, file_write_strn }; mp_raw_code_save(rc, &code_print); int ret = file_seek(file_handle, 0, ORIGIN_CUR, &rtos_err); file_close(file_handle, &rtos_err); return ret; }
Hi, would be usefull when mpy-cross is not available ( eg: esp connected to some mobile web-ide )
how hard is it to have onboard something like ?
which actually raise : TypeError: object with buffer protocol required