Skip to content

Improve sys.settrace to help support debugpy / pdb debugging - #8767

Open
andrewleech wants to merge 2 commits into
micropython:masterfrom
andrewleech:pdb_support
Open

andrewleech wants to merge 2 commits into
micropython:masterfrom
andrewleech:pdb_support

Conversation

@andrewleech

@andrewleech andrewleech commented Jun 16, 2022 •

Copy link
Copy Markdown
Contributor

Companion support for micropython/micropython-lib#1022 & micropython/micropython-lib#499

Requires micropython be built with MICROPY_PY_SYS_SETTRACE eg. unix standard variant

Includes:

@andrewleech
andrewleech force-pushed the pdb_support branch 2 times, most recently from 16bd204 to e027113 Compare June 19, 2022 23:51
@codecov-commenter

codecov-commenter commented Jun 20, 2022 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.42857% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.39%. Comparing base (2dc2e30) to head (3280c2a).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
py/profile.c 44.82% 16 Missing ⚠️
py/modsys.c 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8767      +/-   ##
==========================================
- Coverage   98.46%   98.39%   -0.08%     
==========================================
  Files         176      176              
  Lines       22784    22816      +32     
==========================================
+ Hits        22435    22450      +15     
- Misses        349      366      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 20, 2022
andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 20, 2022
andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 22, 2022
Requires micropython to be compiled with MICROPY_PY_SYS_SETTRACE.
Also requires micropython/micropython#8767
andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 22, 2022
Requires micropython to be compiled with MICROPY_PY_SYS_SETTRACE.
Also requires micropython/micropython#8767
andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 22, 2022
Requires micropython to be compiled with MICROPY_PY_SYS_SETTRACE.
Also requires micropython/micropython#8767
Comment thread py/modsys.c Outdated
MP_DEFINE_CONST_FUN_OBJ_0(mp_sys_gettrace_obj, mp_sys_gettrace);

// _getframe(): Return current frame object.
STATIC mp_obj_t mp_sys_getframe(size_t n_args, const mp_obj_t *args) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please call it mp_sys__getframe

Comment thread py/objfun.c Outdated
if (attr == MP_QSTR___code__) {
mp_obj_fun_bc_t *self = MP_OBJ_TO_PTR(self_in);
mp_obj_code_t *code = MP_OBJ_TO_PTR(mp_obj_new_code(self->context, self->rc));
if (code != NULL) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can code be null?

Comment thread py/profile.c Outdated
switch (attr) {
case MP_QSTR_f_back:
dest[0] = mp_const_none;
if (o->code_state->prev_state) {
if (!o->code_state->prev_state->frame) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you find a case where this could be NULL? maybe it needs a test...

Comment thread py/profile.c Outdated
dest[0] = o->trace_obj;
break;
case MP_QSTR_f_locals:
dest[0] = MP_OBJ_FROM_PTR(o->code->dict_locals);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this needed? locals in uPy doesn't work well

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just return an empty dict instead?

Comment thread py/profile.c Outdated
@@ -274,6 +292,7 @@ mp_obj_t mp_obj_new_frame(const mp_code_state_t *code_state) {
o->lineno = mp_prof_bytecode_lineno(rc, o->lasti);
o->trace_opcodes = false;
o->callback = MP_OBJ_NULL;
o->trace_obj = MP_OBJ_NULL;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe call it f_trace

Comment thread py/profile.h Outdated
@@ -60,7 +61,9 @@ mp_obj_t mp_obj_new_frame(const mp_code_state_t *code_state);

// This is the implementation for the sys.settrace
mp_obj_t mp_prof_settrace(mp_obj_t callback);
mp_obj_t mp_prof_gettrace();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please put void in arg list

Comment thread py/profile.c Outdated
@@ -313,6 +332,33 @@ mp_obj_t mp_prof_settrace(mp_obj_t callback) {
return mp_const_none;
}

mp_obj_t mp_prof_gettrace() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void in arg list

Comment thread py/profile.c Outdated
return prof_trace_cb;
}

mp_obj_t mp_prof_get_frame(int depth) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we change int to size_t to indicate it can't be negative?

@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jun 29, 2022
@dpgeorge dpgeorge changed the title Draft: Working towards pdb debugging support. Improve sys.settrace support to help support bdb/pdb debugging Jun 29, 2022
@dpgeorge dpgeorge changed the title Improve sys.settrace support to help support bdb/pdb debugging Improve sys.settrace to help support bdb/pdb debugging Jun 29, 2022
andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Aug 24, 2023
Requires micropython to be compiled with MICROPY_PY_SYS_SETTRACE.
Also requires micropython/micropython#8767
@github-actions

github-actions Bot commented Aug 24, 2023 •

Copy link
Copy Markdown

Code size report:

Reference:  extmod: Add USB Network (NCM) driver implementation. [1c3c201]
Comparison: py/profile: Keep tracing usable after a trace callback raises. [merge of a764b6c]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +39512 +4.536% standard[incl +14208(data)]
      stm32:    +0 +0.000% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

RetiredWizard pushed a commit to RetiredWizard/micropython that referenced this pull request Dec 30, 2023
@projectgus

Copy link
Copy Markdown
Contributor

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.

andrewleech pushed a commit to andrewleech/micropython-lib that referenced this pull request Jun 11, 2025
Requires micropython to be compiled with MICROPY_PY_SYS_SETTRACE.
Also requires micropython/micropython#8767
@andrewleech andrewleech changed the title Improve sys.settrace to help support bdb/pdb debugging Improve sys.settrace to help support debugpy / pdb debugging Jun 11, 2025
@andrewleech
andrewleech force-pushed the pdb_support branch 4 times, most recently from 0bbcb6e to e0237a8 Compare March 26, 2026 05:12
Add sys.gettrace() to mirror settrace, add sys._getframe() for stack
frame inspection, and enable sys.settrace() by default on unix
standard variant. These additions provide the baseline infrastructure
needed for pdb and debugpy to function on MicroPython.

Signed-off-by: Andrew Leech <[email protected]>
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Jul 5, 2026
…#8767)

Improve sys.settrace to support debugpy/pdb debugging: adds
sys.settrace, sys.gettrace, sys._getframe, frame.f_trace, and enables
settrace on the unix standard variant.
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Jul 14, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Jul 14, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Jul 14, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 4, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 5, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 6, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Aug 6, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
A raising callback leaves mp_prof_callback_invoke through an nlr jump,
so mp_prof_is_executing stayed set for the rest of the process. Every
trace hook in the VM tests that flag, so no callback was ever invoked
again while sys.settrace() and sys.gettrace() went on reporting success:
tracing was silently and permanently off, and a fresh sys.settrace()
could not recover it. With assertions enabled the same state trips
assert(!mp_prof_is_executing) in mp_prof_frame_enter.

The call is now wrapped in nlr_push/nlr_pop; the exception path clears
the guard, unsets the trace callback and re-raises. Unsetting matches
CPython, and it is what makes the state after a raise recoverable rather
than merely different: a callback that raised once is on an error path
and would raise again on every subsequent line, which on a device is
unrecoverable. A debugger that means to keep tracing re-installs
explicitly.

TRACE_TICK now also requires the global callback to still be set.
Without it "unset" does not unset: frames already on the stack hold the
callback the tracer returned for them and go on calling it, so
sys.settrace(None) did not stop tracing the frames that were running.

A deliberate raise from a trace callback is how a debugger unwinds a
target out of a frame it is stopped in - with no second thread and a
message pump that runs inside the trace function, it is the only
mechanism available.

tests/misc/sys_settrace_raise.py covers propagation into the traced
program, the unset, that a following sys.settrace() traces exactly as
much as it did before the raise, and settrace(None) called from inside a
traced frame. It has no .exp file deliberately: the output is identical
under CPython, so the test runner compares the two.

Signed-off-by: Andrew Leech <[email protected]>
@octoprobe-bot

octoprobe-bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Octoprobe PR report

Test Tests
passed
Tests
skipped
Tests
xfailed
Tests
failed
format flash 9 8
run-tests.py 30396 4389 15 50
run-tests.py --via-mpy --emit native 32756 5899 54 70
run-tests.py --via-mpy 32828 4678 14 59
run-perfbench.py 388 15 2
run-natmodtests.py 732 146 11
run-mpremote-tests.sh 308 84
run-tests.py --test-dirs=extmod_hardware 154 148 64 39
run-tests.py --test-dirs=extmod_hardware --emit-native 160 158 66 39
Total 97731 15441 226 341
Failures

Group: run-mpremote-tests.sh

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
esp8266
7921-
LOLIN_D1_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
6038-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
7b21-
PYBV11
stm32
3a21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
test_recursive_cp.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass
test_errno.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass
test_resume.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass
test_fs_tree.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass
test_filesystem.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass
test_mip_local_install.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass
test_eval_exec_run.sh pass pass pass pass pass pass pass FAIL FAIL pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass
test_mount.sh FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL

Group: run-tests.py --test-dirs=extmod_hardware

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
esp8266
7921-
LOLIN_D1_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
5f2c-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
7b21-
PYBV11
stm32
3a21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
extmod_hardware/machine_can2.py FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL skip skip skip skip skip skip skip [skip](https://reports.octoprobe.org/github\_selfhosted\_testrun\_737/RUN-TESTS\_EXTMOD\_HARDWARE@5334-...*[Comment body truncated]

andrewleech added a commit to andrewleech/micropython that referenced this pull request Aug 10, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
Signed-off-by: Andrew Leech <[email protected]>
andrewleech added a commit to andrewleech/micropython that referenced this pull request Sep 24, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
Signed-off-by: Andrew Leech <[email protected]>
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Sep 27, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Sep 27, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
andrewleech pushed a commit to andrewleech/micropython that referenced this pull request Sep 27, 2026
sys.settrace: gettrace, _getframe, f_trace support (PR micropython#8767)

MBM-PR: 8767
MBM-URL: micropython#8767
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

py-core Relates to py/ directory in source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants