mpremote: on ESPxxx all mpremote commands are followed by a hard reboot

@jimmo

according to @jimmo mpremote should be able to resume a repl session on a connected mcu without a reboot by using the following command pattern:

mpremote connect exec "start_something()"
mpremote connect resume exec "continue_here()"

where the resume is the important part.
see Discord discussion

however when I try this :

I see that any variables set in the the first mpremote commands are lost
I see that the esp32 performs a soft reset after exiting the raw REPL
(observed through logic analyzer)

So even though the 2nd mpremote command does not force a reset, by that time a soft reboot has already occurred.
I also see a soft reset after:

exiting the repl
eval "42"
exec "print(42)"

Actual result on windows :

C:\develop\usb>mpremote exec "foo=bar=42;print(foo)"
42

C:\develop\usb>mpremote resume exec "print(bar)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'bar' isn't defined

Expected

the MCU not to reset between two mpremote commands similar to the behavour on linux

$ mpr exec "foo=bar=42; print(foo)"
42
$ mpr exec "print(bar)"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'bar' isn't defined
$ mpr exec "foo=bar=42; print(foo)"
42
$ mpr resume exec "print(bar)"
42

results as seen by a logic analyzer hooked up to the RX/TX pins of the esp32

\x01\x03
>>> 
>>> 
>>> 
>>> 
raw REPL; CTRL-B to exit
\x02OKx05A\x01foo=bar=43;print(foo)\x04
MPY: soft reboot
raw REPL; CTRL-B to exit
>R\x01\x80\0\x01\x0443
\x04\x04>ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4540
ho 0 tail 12 room 4
load:0x40078000,len:12448
load:0x40080400,len:4124
entry 0x40080680
MicroPython v1.19.1 on 2022-06-18; 4MB/OTA module with ESP32
Type "help()" for more information.
>>> 
>>> 
>>> 
>>> 
>>> 
raw REPL; CTRL-B to exit
\x02R\x01\x80\0\x01\x04\x04Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'foo' isn't defined
\x04>ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:4540
ho 0 tail 12 room 4
load:0x40078000,len:12448
load:0x40080400,len:4124
entry 0x40080680
MicroPython v1.19.1 on 2022-06-18; 4MB/OTA module with ESP32
Type "help()" for more information.
>>>

other observations

same behavior is observed with multiple firmware versions, and on different boards
esp8266 devices do NOT soft-reset in the same way ( note: all my esp8266 devices use a different serialport driver)

Versions

mpremote : 0.4.0
pyserial: 3.5
Python 3.9.13
Version: 1.72.2 (user setup)
OS: Windows_NT x64 10.0.22621
Micropython - v1.19.1
MCU - all esp32
serial port driver esp32 : Silicon Laboratories Inc. 11.1.0.53
serial port driver esp8266 : wch.cn - No version or other details

> mpremote devs
COM9 01665F52 10c4:ea60 Silicon Labs None # esp32
COM11 0001 10c4:ea60 Silicon Labs None # esp32 
COM20 00000000A 0403:6001 FTDI None # UART debug 
COM21  1a86:7523 wch.cn None # ESP8266

firmwares

mpremote connect com9 get-info
{'ver': 'v1.19.1', 'build': '', 'platform': 'esp32', 'port': 'esp32', 'machine': 'ESP32 module (spiram) with ESP32', 'release': '1.19.1', 'nodename': 'esp32', 'name': 'micropython', 'family': 'micropython', 'sysname': 'esp32', 'version': '1.19.1'}
mpremote connect com11 get-info
{'ver': 'v1.19.1', 'build': '', 'platform': 'esp32', 'port': 'esp32', 'machine': '4MB/OTA module with ESP32', 'release': '1.19.1', 'nodename': 'esp32', 'name': 'micropython', 'family': 'micropython', 'sysname': 'esp32', 'version': '1.19.1'}
mpremote connect com21 get-info
{'ver': 'v1.19.1', 'build': '', 'platform': 'esp8266', 'port': 'esp8266', 'machine': 'ESP module (1M) with ESP8266', 'release': '1.19.1', 'nodename': 'esp8266', 'name': 'micropython', 'family': 'micropython', 'sysname': 'esp8266', 'version': '1.19.1'}

oldest firmware tested with same behavior:

mpremote connect com9 get-info
{'ver': 'v1.10', 'build': '', 'platform': 'esp32', 'port': 'esp32', 'machine': 'ESP32 module with ESP32', 'release': '1.10.0', 'nodename': 'esp32', 'name': 'micropython', 'family': 'micropython', 'sysname': 'esp32', 'version': '1.10.0'}

if you want to take a look at a trace :
three_device_session.zip

open with : https://www.saleae.com/downloads/

@Josverl

@Josverl This is a hard reset, not a soft reset. A soft reset would not print out the esp boot sequence.

The only hint I have is that on many ESP32 boards, they wire up RTS/CTS from the on-board cp210x / ftdi in a way such that esptool.py can trigger the board to reset into the bootloader. I wonder if somehow the way the serial port is being opened on Windows is also triggering this?

you are correct , there is both a soft and a hard reset. I was blinded by the first one.
but im sure that I and students in bootcamps have been able to connect to and disconnect many times to a board without it rebooting.

my esp32s all use a CP210x ,
while my esp8266 all have an CH340

but that is about the most common UART/USB interface used,

Could it be that the state of the DTR/RTS signals have an effect ? ( duh , yes - you were also thinking that 🤦‍♂️)
One fix that I have needed to add to a few terminal tools is to compensate for the fact that that windows and linux actually handle the DTR/RTS signals differently. (neither is wrong, just different)
(I've not looked into how pyserial handles this yet)

I encountered this recently with a new-to-me board, SparkFun MicroMod ESP32. The schematic documents the DTR/RTS behavior for resetting:

If DTR is LOW, toggling RTS from HIGH to LOW resets to run mode
If RTS is HIGH, toggling DTR from LOW to HIGH resets to bootloader

As a test, I changed the Pyboard constructor in pyboard.py to assert the DTR and RTS pins:

@@ -275,7 +275,11 @@ class Pyboard:
             delayed = False
             for attempt in range(wait + 1):
                 try:
-                    self.serial = serial.Serial(device, **serial_kwargs)
+                    self.serial = serial.Serial(**serial_kwargs)
+                    self.serial.dtr = True
+                    self.serial.rts = False
+                    self.serial.port = device
+                    self.serial.open()
                     break
                 except (OSError, IOError):  # Py2 and Py3 have different errors
                     if wait == 0:

This fixed the problem on Linux and Windows, but not macOS (Big Sur, Monterrey).

On Linux, using

stty -F /dev/ttyUSB0 -hup

also prevented the hard reset.

@mgsb

Thanks @mgsb -- From what I've read elsewhere this seems like the right approach, although frustrating about macOS.

I haven't had much chance to look into this in detail but one thing I'd like to understand -- is this a new issue, or has it always been a problem? i.e. was this always causing DTR/RTS to bounce quickly, but there was enough capacitance and/or pull strength that it was never noticed? Are we seeing it now because just a handful of boards have less capacitance on these lines?

@Josverl does this solve it for your board?

Following long debugging session on Discord today (22Dec2022) I can confirm I had this issue on an ESP32-C3FH4 (ESP32-C3 with RBG matrix on board) and that the fix above fixed my issue

Who do I see about getting it incorporated into standard code base? :)

I'll give this a test, but the .py fix is like what I've been doing in .js for pymakr, but with the added condition/constraint to only manipulate dtr rts on win32/64

Following long debugging session on Discord today (22Dec2022) I can confirm I had this issue on an ESP32-C3FH4 (ESP32-C3 with RBG matrix on board) and that the fix above fixed my issue

Some more details -- this was Windows, and the behaviour was that there was a reset on disconnect.

The C3 is interesting because it's on-chip USB/UART, rather than external, but clearly it's internally wired up the same way with the DTR/RTS pins.

Who do I see about getting it incorporated into standard code base? :)

I will send a PR. I think it sounds like this change doesn't solve everything, but overall only improves things and doesn't make anything break. But will need some more testing.

tools/pyboard.py: Set DTR on Windows to avoid ESPxx hard reset.

I will send a PR. I think it sounds like this change doesn't solve everything, but overall only improves things and doesn't make anything break. But will need some more testing.

Lucky me found the case where it does break :-D on windows and stm32 with "custom" driver installed (with zadig) to give the ports a personalised name! So yeah not a common situation, though common to every device I make at work...

I'm not sure how to handle it better here though really, other than adding a extra bit of extra injected code on connection to query os.uname() to enable stuff like this per-port. Of course this would fail if the device isn't at repl on connection (though much of mpremote fails in this circumstance).

In theory I might be able to update my driver creation / installation to keep the manufacturer as "Microsoft" to bypass this bit of mpremote code too, though that doesn't help devices / drivers already in the field...

@andrewleech

@andrewleech,
did you find the root cause for your problem ?
if it was indeed the RTS held high then perhaps its also possible to prevent that if there is a common VID for STM devices ?

VID 303A, PID 1001.

These VID:PID are same as for the ESP32-C3 board I have.

and for a ESP32-C2 - single Connector board:

  idVendor                 : 0x303A (Espressif Incorporated)
  idProduct                : 0x4001
  bcdDevice                : 0x0100

So checking for the VID should work, unless board manufacturers replace it with their own VID's.
then we would need to add a config option / file.

@crackwitz

I did not try other DTR/RTS combinations since the first one I tried worked already

@crackwitz, the challenge is not only fixing this for your board,
but doing that without breaking mpremote + other boards across a number use different usecases.

I am aware of that, which is why I strive to make perfectly clear what I didn't try.

@andrewleech

@andrewleech,
did you find the root cause for your problem ?
if it was indeed the RTS held high then perhaps its also possible to prevent that if there is a common VID for STM devices ?

My particular issue was from having a custom vid pid and separately loaded driver that did not have Microsoft as the listed manufacturer so it didn't match the checks in code.

I tried making changes to this block of code to resolve the problems better but every option I tried had other subtle negative outcomes too, they're in a PR here somewhere I think.

did not have Microsoft as the listed manufacturer so it didn't match the checks in code.

Thanks for clarifying. The only way that we can avoid that is by having a list of regexes to match the known driver names/vendors for the uart drivers. That is the approach we ended up using for Pymakr. Simpler than gathering all possible VIDs for the serial UART drivers.

[ 		"Pycom.*", 		"FTDI", 		"Microsoft", 		"Microchip Technology, Inc.", 		"Silicon Labs",  "Espressif.*", ....	]

I think that should be a separate PR though

This is probably a silly question as I'm not across the whole thing, but in the limit is it reasonable to patch the opposite case? (i.e. don't clear DTR/RTS for drivers/configs where we know that will cause a problem, do it for all other boards which should include most/all ESP32 boards plus a bunch of boards which don't care either way.

Alternatively, if it's too hard to get all of the custom cases then maybe this needs to be an option on mpremote resume (i.e. mpremote resume --esp ... or similar.) - in addition to detecting the most obvious cases.

not silly , but hard to parse your inverse logic, but I think we may see the same way ahead.

let me try to recap my experience, and intents

pyboard/pyserial defaults are OK on linux / macOS, and were mostly broken on Windows. ( different world visions between the OS ecosystems)
this mostly surfaces as spikes in the DTR/CTSlines at either connection , but more often at disconnection
this is most problematic in boards that use these to do special things, such as the ESPxxx boards that connect these lines to some logic and !EN and other strapping Pins
it is not always simple to see / trace these signals - but on the ESP32 / ESP8266 chips it often is, so I used there for my testing.
most of these boards have UART-USB convertors on board - with multiple different drivers and driver versions for each of these.
Naming of these drivers is - not quite consistent, and new drivers are released .

To avoid ESPxx board to reset on disconnect , or hang in bootloader on connect we need the set DTR/CTS to False/False Only for these boards, both for UART/USB as well as for USB/CDC , but there is no authoritative list of VIDs / vendor names to do so.

With Pycom I was able to iterate quickly on user feedback - and updates were done in weeks, so we were able to add regexes for vendor strings that were reported quickly.
with mpremote some of the same updates have been stalled for months, or longer. That is the main reason I have chosen for the if not "Microsoft" driver approach.

Now with the modern ESPxxx boards switching to native USB , and less reliance on 3rd party UART drivers I think it is a good time to switch tactics

The default mpremote config should have a list of VIDs-PIDs that require "ESP-Handling" on Windows
this config should be extendible/overridable though the mpremote configuration file (currently still broken on windows )
when connecting to a port - mpremote can check if it needs ESP-Handling - and then act accordingly

if / when needed new vids/pids can be added to the default list via PRs, and included in the next release
personal / interim changes can be done by editing the mpremote config file.

personal / interim changes can be done by editing the mpremote config file.

I think that would be good, to be able to customise/configure/override the default behaviour based on user configuration.

Ok , I can create a draft PR for that, so we can iterate over the details.

then we can move the conversation there as well.

tools/mpremote: Update ESPxxx detection for USB-CDC ports.

tools/pyboard.py: Set DTR on Windows to avoid ESPxx hard reset.

tools/mpremote: Update ESPxxx detection for USB-CDC ports.

Navigation Menu

Repository navigation

mpremote: on ESPxxx all mpremote commands are followed by a hard reboot #9659

Actual result on windows :

Expected

other observations

Versions

firmwares

8 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

mpremote: on ESPxxx all mpremote commands are followed by a hard reboot #9659

Description

Actual result on windows :

Expected

other observations

Versions

firmwares

Activity

Josverl commented on Oct 17, 2022

jimmo commented on Oct 18, 2022

Josverl commented on Oct 18, 2022

mgsb commented on Nov 13, 2022

jimmo commented on Nov 14, 2022

cymplecy commented on Dec 22, 2022

Josverl commented on Dec 22, 2022

jimmo commented on Dec 22, 2022

andrewleech commented on Jul 31, 2023

8 remaining items

Josverl commented on Jul 28, 2025

Josverl commented on Jul 28, 2025

Josverl commented on Jul 28, 2025

crackwitz commented on Jul 29, 2025

andrewleech commented on Jul 29, 2025

Josverl commented on Jul 29, 2025

projectgus commented on Jul 31, 2025

Josverl commented on Jul 31, 2025

dpgeorge commented on Jul 31, 2025

Josverl commented on Jul 31, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions