Repository navigation
Too little DMA memory (or free memory block) is available for the eps32s3 (or other esp32s) #12075
Description
Hi, I am having trouble to get the maximum 256KB DMA block using the latest release. After some hardcoded debug print in a C-Module, I found out that the maximum allocatable memory block is roughly at 247KB. This is the code I've used:
STATIC void frame_buffer_alloc(mp_lcd_rm67162_obj_t *self, int len) {
self->frame_buffer_size = len;
size_t free_size = heap_caps_get_largest_free_block(MALLOC_CAP_DMA);
self->frame_buffer = heap_caps_malloc(self->frame_buffer_size, MALLOC_CAP_DMA);
int threshold = free_size / 1024;
if (self->frame_buffer == NULL) {
if (threshold <= 247) {
mp_raise_msg(&mp_type_OSError, MP_ERROR_TEXT("Failed to allocate DMA'able framebuffer,
Maiximum available DMA size: less than 247KB"));
}
if (threshold <= 248) {
mp_raise_msg(&mp_type_OSError, MP_ERROR_TEXT("Failed to allocate DMA'able framebuffer,
Maiximum available DMA size: less than 248KB"));
}
if (threshold <= 249) {
mp_raise_msg(&mp_type_OSError, MP_ERROR_TEXT("Failed to allocate DMA'able framebuffer,
Maiximum available DMA size: less than 249KB"));
} else {
mp_raise_msg(&mp_type_OSError, MP_ERROR_TEXT("Failed to allocate DMA'able framebuffer,
Maiximum available DMA size: more than 249KB"));
}
}
memset(self->frame_buffer, 0, self->frame_buffer_size);
}Now I don't know if this is a bug... But, if uPy is using roughly 16KB of RAM as it advertised (which I highly doubt on the esp32 port, because of wifi and stuff), and freeRTOS taking at most 10KB according to this (that being all the ROM has been loaded into the RAM), why isn't it possible to allocate 256KB of continous block? Is this caused by some bug or have I underestimate the memory usage of uPy by a huge margine?
Activity
mattytrentini commentedon Jul 24, 2023
Related to #11853? (That issue is high on the priority list and is a consequence of the recent move to IDF v5.0.2)
nspsck commentedon Jul 24, 2023
I highly doubt that would be the case. I also tried to modify the
main.cfile but it had no effect on this issue. To be precise, I have modified this line to/ 3/ 4/ 6, It had no effect on this problem.Further I tracked down the change log of the main.c file (I searched for main.c on that site to locate the file) and it seems like the micropython heap is allocated the same way. In the older version, line 140, 146 and 147 are effectively excuted for the esp32s3 and in the newer version, these are the same 3 lines. The main difference between newer and older version is, that the newer Version uses
heap_caps_malloc()and the older version just usesmalloc().So I took my next approach. First of all, I changed the
heap_caps_malloc()tomalloc(), then I modified the following lines insdkconfigusingmenuconfig, noteCONFIG_SPIRAM_USE_MALLOC=ymost likely impliesCONFIG_SPIRAM_USE_CAPS_ALLOC=y, since it is called in themenuconfig: "make memory also allocatable using malloc()."It built and booted, but the same problem is still present. So I assume that this problem is not the new idf version related... But there is a fairly chance that I am wrong since I am a beginner and know very little about this whole project.
EDIT: of cause I assumed that there are no significant changes to the
heap_caps_malloc()related function in the esp-idf. And because there are indeed roughly 247KB internal RAM block available no matter of the divider, I assume that divider only limits what uPy can see and use, not the C-Modules. Which leads me to believe that this issue has something to do with freeRTOS or how uPy handles C-Modules or RAM in general. But again, those are just my speculations.shariltumin commentedon Jul 24, 2023
set the MP heap hardcoded to 2000*1024=2048000. We should get about 2000KB MP heap.
Still gc.mem_free() shows about half of the 8MB PSRAM free.
But esp32.idf_heap_info() reports 'correctly' - (8388608, 6338304, 6291456, 6338304) the 6338304 is about 6MB since MP was set to 2000KB.
gc.mem_free() uses py/gc.c gc_info(), maybe gc.c is broken?
mem_info() also reports a total of 4274752, but it should be 2048000.
dpgeorge commentedon Jul 25, 2023
@shariltumin you need to change the
size_t mp_task_heap_sizevariable.shariltumin commentedon Jul 25, 2023
@dpgeorge Thanks! My mistake.
With 8MB PSRAM I managed to build two firmwares v1.20.0-283-g7d66ae603 esp-idf 5.0.2 on ESP32S3-N16R8 board: 1) 2/6 MP/IDF and 2) 6/2 MP/IDF
Below is a somewhat lengthy report of the results of my tests.
The msgpipe is a C module that allocates/deallocates memory to/from a ring buffer structure using malloc()/free().
#Firmware 2/6 test.
Source modification:
REPL run:
The idf heap is not deallocated during a soft reboot.
#Firmware 6/2 test.
Source modification:
REPL run:
ESP-WARNING "ESP32 has a few hundred kilobytes of internal RAM, residing on the same die as the rest of the chip components. It can be insufficient for some purposes, so ESP32 has the ability to use up to 4 MB of virtual addresses for external PSRAM (Psuedostatic RAM) memory. The external memory is incorporated in the memory map and, with certain restrictions, is usable in the same way as internal data RAM."
Despite the above warning, I manage to allocate more than 4MB of memory block. I do not know the impact on system stability, perhaps more knowledgeable can explain.
This is my SPIRAM configuration file in the sdkconfig.board file.
Sorry for taking up so much space, but I hope it is useful.