Skip to content

py/gc: Support multiple heaps. - #3533

Closed
aykevl wants to merge 1 commit into
micropython:masterfrom
aykevl:multiheap
Closed

aykevl wants to merge 1 commit into
micropython:masterfrom
aykevl:multiheap

Conversation

@aykevl

@aykevl aykevl commented Jan 2, 2018

Copy link
Copy Markdown
Contributor

Enable the addition of heap space at runtime. Advantages:

  • The ESP32 has a fragmented heap so to use all of it the heap must be split.
  • Support a dynamic heap while running on an OS, adding more heap when necessary.

With this code, I managed to extend the MicroPython heap to ~200kB on the ESP32:

MicroPython v1.9.3-239-gd16caf776-dirty on 2018-01-02; ESP32 module with ESP32
Type "help()" for more information.
>>> import micropython
>>> micropython.mem_info(True)
stack: 752 out of 15360
GC: total: 206976, used: 5200, free: 201776
 No. of 1-blocks: 30, 2-blocks: 7, max blk sz: 264, max free sz: 6936
GC area #0 (size 96000), from 3ffb30a0:
00000: h=AhhBMh=DhhhDBBBBAhh===h===Ahh==h==============================
00400: ================================================================
00800: ================================================================
00c00: ================================================================
01000: =========================================h==Bh=ShShhThAh=h=Bh==B
01400: ..h.h=......h=..................................................
       (87 lines all free)
17400: ................................................
GC area #1 (size 110976), from 3ffe4de0:
       (108 lines all free)
1b000: ........................
>>> import gc
>>> gc.mem_free()
201616
>>> 

There appears no way to have one large heap on the ESP32, as the ESP-IDF itself fragments the heap:

I (344) heap_init: Initializing. RAM available for dynamic allocation:
I (347) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (353) heap_init: At 3FFDCE50 len 000031B0 (12 KiB): DRAM
I (360) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (366) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (372) heap_init: At 4008FC7C len 00010384 (64 KiB): IRAM

I haven't included the code for the ESP32 yet, but it is very simple.

I have tested these changes using #3532 and haven't seen a regression.

Comment thread py/mpstate.h Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one semicolon too much

Enable the addition of heap space at runtime. Advantages:
  - The ESP32 has a fragmented heap so to use all of it the heap must be
    split.
  - Support a dynamic heap while running on an OS, adding more heap when
    necessary.
@aykevl

aykevl commented Jan 24, 2018

Copy link
Copy Markdown
Contributor Author

Replaced by a better PR that supports disabling of multiheap support: #3580.

@aykevl aykevl closed this Jan 24, 2018
@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Jul 22, 2022
UKTailwind added a commit to UKTailwind/micropython that referenced this pull request Sep 2, 2026
The PicoMite tree ran its TinyUSB 0.21 host work on this same hardware -
two keyboards (one marginal), a touch panel and a flash drive behind one
hub - across software, hardware and power-on resets, and wrote up what
actually held (docs/usb-host-hardening).  Root cause: the RP2 SIE keeps
ONE handshake-result latch shared between EPX and the interrupt-endpoint
poller (upstream micropython#3533), so an interrupt poll can overwrite a control
transfer's ACK before the IRQ handler reads it, and the transfer is
misread as RX_TIMEOUT.  The validated answer is to tolerate the race
above the silicon, not to adopt the strict micropython#3533 driver rewrite - which
enumerated FEWER devices on the marginal rig.  Applied here:

  hcd_rp2040.patch  Grace period for EP0 RX timeouts: within a 1 s
                    window the transfer is left armed and the clobbered
                    result outlasted; the window closes on any EP0
                    completion.  Expiry (a genuinely dead device) takes
                    the original fail path, which keeps our buffer clear
                    (micropython#3874).  Bulk/interrupt keep the fast-fail: their
                    flow control is NAK, which never raises RX_TIMEOUT.

  usbh.patch        Enumeration-exclusive control dispatch: while a
                    device enumerates, only address 0, the enumerating
                    address and the hub in use may claim the control
                    slot; LED writes and touch handshakes wait in the
                    pending FIFO (gated at the claim, the two drain
                    checks and the dispatcher, by peeking the head).
                    And a failed enumeration now disables its hub port
                    (CLEAR_FEATURE(PORT_ENABLE), async no-op callback) -
                    0.21 left the abandoned device enabled at address 0,
                    where it answers in parallel with the next device's
                    bring-up.  The 100 ms reset recovery (micropython#3876) stays.

  mp_usbh.c         Mount-callback prints deferred: the callbacks now
  usb_msc.c         format into a small static ring and mp_usbh_task()
                    prints it after tuh_task() returns.  A print in the
                    callback runs the VM via dupterm mid-enumeration -
                    the measured cost of "one connect chime" on the
                    PicoMite rig was the marginal device.  The existing
                    reentrancy guard treated the symptom; this removes
                    the stall itself.

  tusb_config.h     CFG_TUH_CONTROL_PENDING_QUEUE_SZ 4 -> 8: the gate
                    parks more traffic in the FIFO during enumeration.

Not adopted, by the doc's own measurement: the micropython#3533 reference hcd
(fewer devices on marginal hardware; revisit when merged upstream).
Already carried, now cross-validated: MULTI_HUB_FIX and the 64-entry
event queue.  Builds clean; the board leg - all reset kinds, hot
attach, MSC beside HID, pulling the marginal device - is owed before
this is believed.

Co-Authored-By: Claude Fable 5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

py-core Relates to py/ directory in source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants