tests: various improvements for Octoprobe (WIP) by dpgeorge · Pull Request #17782 · micropython/micropython

dpgeorge · 2025-07-29T10:16:47Z

Summary

This is a collection of various small improvements to the test suite to get Octoprobe to pass more of the tests.

Once fully working, these changes will be made into separate PRs.

Testing

To be tested by Octoprobe: https://reports.octoprobe.org/

Test report prior to this PR: https://reports.octoprobe.org/github_selfhosted_testrun_194/octoprobe_summary_report.html

Test report with this PR: https://reports.octoprobe.org/github_selfhosted_testrun_199/octoprobe_summary_report.html

codecov · 2025-07-29T10:20:12Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.58%. Comparing base (0fd6c57) to head (82bfdc7).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #17782   +/-   ##
=======================================
  Coverage   98.58%   98.58%           
=======================================
  Files         182      182           
  Lines       23322    23323    +1     
  Branches        5        5           
=======================================
+ Hits        22993    22994    +1     
  Misses        328      328           
  Partials        1        1

Flag	Coverage Δ
unix-coverage-32bit	`98.59% <100.00%> (+<0.01%)`	⬆️
unix-coverage-64bit	`98.52% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-07-31T13:02:04Z

Code size report:

Reference:  esp32/modmachine: Use popcount to count wake pins. [8cf130d]
Comparison: rp2/boards: Make some boards use float-abi=hard. [merge of 82bfdc7]
  mpy-cross:   +16 +0.004% 
   bare-arm:    +8 +0.014% 
minimal x86:   +14 +0.007% 
   unix x64:  +224 +0.026% standard
      stm32:   +80 +0.020% PYBV10
      esp32:  +184 +0.010% ESP32_GENERIC
     mimxrt:   +96 +0.024% TEENSY40
        rp2:   +96 +0.010% RPI_PICO_W
       samd:   +84 +0.030% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:  +134 +0.029% VIRT_RV32

hmaerki · 2025-07-31T16:50:18Z

@dpgeorge: Two easy to reproduce bugs which could be added to ths PR

In --via-mpy --emit native mode. Signed-off-by: Damien George <[email protected]>

Tested on ADAFRUIT_ITSYBITSY_M0_EXPRESS. Signed-off-by: Damien George <[email protected]>

Signed-off-by: Damien George <[email protected]>

To reduce heap memory use. Signed-off-by: Damien George <[email protected]>

Not sure if this is actually better, may lead to more fragmentation. Probably needs some tests. See micropython#4087 Really need to retry alloc with smaller amount if it fails, but then really we need a lazy malloc, per the TODO in py/misc.h. Signed-off-by: Damien George <[email protected]>

On SAMD21, 2400 baud is not accurate enough (it's peripheral freq is 48MHz and the baud divisor is 16-bits (with 13-bit integer + 3-bit fraction). And add 38400 to cover more cases. TODO: maybe fix this another way, maybe skip 2400 baud on SAMD, or have a list of baud rates to test in target_wiring.py, or fix SAMD. Signed-off-by: Damien George <[email protected]>

works on PYBD_SF6! time ./run-tests.py -t a0 --via-mpy --emit native -d basics takes 58.7s prior to this change, 38.5s with this change TODO: - test on esp8266, especially timing - test with prelude shared/runtime/pyexec: Enable REPL helpers without compiler. Signed-off-by: Damien George <[email protected]>

Signed-off-by: Damien George <[email protected]>

TODO: - feature check this on the board first - if target cannot compile at all, also mpy the feature checks Signed-off-by: Damien George <[email protected]>

Can now run .mpy files without the compiler enabled! Signed-off-by: Damien George <[email protected]>

Signed-off-by: Damien George <[email protected]>

Can now use "mpremote run" on targets without a compiler! Signed-off-by: Damien George <[email protected]>

Signed-off-by: Hans Maerki <[email protected]>

Signed-off-by: Damien George <[email protected]>

Eg on ADAFRUIT_ITSYBITSY_M0_EXPRESS. Signed-off-by: Damien George <[email protected]>

Not sure how to configure it for all RP2350 in ARM mode... we only know the MCU after initialising pico-sdk, but by then it's too late to set PICO_HARD_FLOAT_ABI. Signed-off-by: Damien George <[email protected]>

dpgeorge added the tests Relates to tests/ directory in source label Jul 29, 2025

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 18ad8c0 to 5958518 Compare July 31, 2025 02:27

hmaerki mentioned this pull request Jul 31, 2025

LOLIN_C3_MINI / RUN-TESTS_STANDARD / invalid pin octoprobe/testbed_micropython#34

Open

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from c399b93 to ca374fa Compare July 31, 2025 13:16

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from aee37bb to 8920560 Compare August 1, 2025 04:05

hmaerki mentioned this pull request Aug 1, 2025

LOLIN_D1_MINI: net_hosted/ssl_verify_callback.py octoprobe/testbed_micropython#49

Open

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 3 times, most recently from 66db214 to 3993063 Compare August 10, 2025 02:13

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from ec94811 to f7dc897 Compare August 13, 2025 09:22

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from 941611d to 4d60e1c Compare August 16, 2025 14:53

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from 96642d5 to 87300a8 Compare August 26, 2025 07:17

This was referenced Aug 26, 2025

nrf/modules/machine/spi: Print SPI baudrate, polarity and phase. #17984

Merged

esp8266/main: Decrease usable stack by 64 bytes. #17986

Merged

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 87300a8 to 35f3163 Compare September 8, 2025 03:01

hmaerki mentioned this pull request Sep 20, 2025

Use new --test-instance argument consistently in all test runners octoprobe/testbed_micropython#68

Merged

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 35f3163 to a82b761 Compare September 22, 2025 00:33

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 7aed4bf to fdaf899 Compare June 12, 2026 04:19

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 5 times, most recently from 2874214 to c166c50 Compare July 15, 2026 06:48

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from c166c50 to 55ae425 Compare July 17, 2026 13:47

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from 5761812 to 04faee5 Compare July 29, 2026 13:28

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 04faee5 to 8c641bf Compare August 3, 2026 04:01

dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from 5c97ce1 to 8067070 Compare August 19, 2026 07:13

dpgeorge and others added 16 commits August 25, 2026 13:22

tests/float/string_format_modulo2_intbig.py: Get working on esp8266.

477cfed

In --via-mpy --emit native mode. Signed-off-by: Damien George <[email protected]>

tests/run-tests.py: Skip various tests on small SAMD targets.

48a6f9d

Tested on ADAFRUIT_ITSYBITSY_M0_EXPRESS. Signed-off-by: Damien George <[email protected]>

tests/ports/stm32/adcall.py: Print values if test ADC fails.

2817709

Signed-off-by: Damien George <[email protected]>

tests/misc/rge_sm.py: Use list.append instead of list += [...].

cf24223

To reduce heap memory use. Signed-off-by: Damien George <[email protected]>

tools/pyboard.py: Support sending mpy files directly.

14c00bc

Signed-off-by: Damien George <[email protected]>

tests/test_utils.py: Send mpy files directly to raw REPL.

5bb6e81

TODO: - feature check this on the board first - if target cannot compile at all, also mpy the feature checks Signed-off-by: Damien George <[email protected]>

tools/pyboard.py: Allow running .mpy directly.

17ea0c4

Can now run .mpy files without the compiler enabled! Signed-off-by: Damien George <[email protected]>

tools/mpremote: Support raw REPL mpy mode.

2674248

Signed-off-by: Damien George <[email protected]>

tools/mpremote: Allow executing a .mpy directly.

8ed6832

Can now use "mpremote run" on targets without a compiler! Signed-off-by: Damien George <[email protected]>

tools/mpremote: Fix errors while running run-mpremote-tests.sh.

7caf1a2

Signed-off-by: Hans Maerki <[email protected]>

tests/ports/webassembly/heap_expand.mjs.exp: Update.

a017a22

Signed-off-by: Damien George <[email protected]>

tests/test_utils.py: Automatically skip if unittest does not import.

fd4202b

Signed-off-by: Damien George <[email protected]>

tests/stress/dict_copy.py: Further skip if OOM.

b7f85b0

Eg on ADAFRUIT_ITSYBITSY_M0_EXPRESS. Signed-off-by: Damien George <[email protected]>

dpgeorge force-pushed the tests-improvements-for-octoprobe branch from 8067070 to b7f85b0 Compare August 25, 2026 03:23

rp2/boards: Make some boards use float-abi=hard.

82bfdc7

Not sure how to configure it for all RP2350 in ARM mode... we only know the MCU after initialising pico-sdk, but by then it's too late to set PICO_HARD_FLOAT_ABI. Signed-off-by: Damien George <[email protected]>

Navigation Menu

Repository navigation

tests: various improvements for Octoprobe (WIP) - #17782

tests: various improvements for Octoprobe (WIP)#17782
dpgeorge wants to merge 17 commits into
micropython:masterfrom
dpgeorge:tests-improvements-for-octoprobe

dpgeorge commented Jul 29, 2025 •

edited

Loading

Uh oh!

codecov Bot commented Jul 29, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 31, 2025 •

edited

Loading

Uh oh!

hmaerki commented Jul 31, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

dpgeorge commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

codecov Bot commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hmaerki commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dpgeorge commented Jul 29, 2025 •

edited

Loading

codecov Bot commented Jul 29, 2025 •

edited

Loading

github-actions Bot commented Jul 31, 2025 •

edited

Loading

hmaerki commented Jul 31, 2025 •

edited

Loading