ports: add new PSOC Edge port (proof of concept that doesn't use ModusToolbox) - #18843

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.42%. Comparing base (fdb7c0f) to head (a4868f5).
⚠️ Report is 3 commits behind head on master.

@@            Coverage Diff             @@
##           master   #18843      +/-   ##
==========================================
- Coverage   98.42%   98.42%   -0.01%     
==========================================
  Files         174      174              
  Lines       22334    22328       -6     
==========================================
- Hits        21983    21977       -6     
  Misses        351      351              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Code size report:

Reference:  tests/extmod/os_urandom.py: Add test for os.urandom. [6dbabc9]
Comparison: tools/ci.sh: Remove packages. [merge of a4868f5]
  mpy-cross:    +0 +0.000% 
   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64:    +0 +0.000% standard
      stm32:    +0 +0.000% PYBV10
      esp32:    +0 +0.000% ESP32_GENERIC
     mimxrt:    +0 +0.000% TEENSY40
        rp2:    +0 +0.000% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

Hi @dpgeorge,

I had the chance to check the PR and all is working smoothly😊 !

Only changes:

• I needed to add the GIT_MODULES = ... to the makefile. This was missing.
• Adjust the paths to the tools.

Still I am using the gcc-arm Infineon version, but I believe the official one should work as well. Honestly, I am unaware of the actual differences 😅

I would say the bsp folder should be under boards/KIT_PSE84_AI, as this is board specific. Each kit/board will have its own cfg sources. How do you see it?

Regarding all the includes, flags, etc. How did you choose them? Manually copied from the MTB build output? Or did you use any other reference?

I also see you added manually some of the cmsis header to the bsp, but did not include the Infineon cmsis repo. There is already a lib/cmsis 5.9.0. in the project and in our case we were pulling https://github.com/Infineon/cmsis 6.1.0.
What would be the approach here?

Thanks!

@jaenrig-ifx thanks for checking out this PR!

Only changes:

* I needed to add the` GIT_MODULES = ...` to the makefile. This was missing.

You are right, I missed that bit. I've now added it.

* Adjust the paths to the tools.

Yes that's necessary, depending on where you have them installed. But it was hopefully easy to do, just make OPENOCD=<path> (and also a Python venv for edgeprotecttools).

Still I am using the gcc-arm Infineon version, but I believe the official one should work as well. Honestly, I am unaware of the actual differences 😅

Right. It shouldn't matter which gcc you use, as long as it's modern enough to support Cortex-M33 it should work.

I would say the bsp folder should be under boards/KIT_PSE84_AI, as this is board specific. Each kit/board will have its own cfg sources. How do you see it?

OK, I've now changed that, moved bsp into the board folder.

Regarding all the includes, flags, etc. How did you choose them? Manually copied from the MTB build output? Or did you use any other reference?

I just added the minimal set of flags needed to get it building and running, through trial and error adding files and flags until it would compile. I did not look at MTB.

Well, I also knew from experience what a Cortex-M33 needs (eg -mcmse) and for things like -DCOMPONENT_SECURE_DEVICE it would not build without them and I found out by looking at the source files that needed such things defined.

I also see you added manually some of the cmsis header to the bsp, but did not include the Infineon cmsis repo. There is already a lib/cmsis 5.9.0. in the project and in our case we were pulling https://github.com/Infineon/cmsis 6.1.0. What would be the approach here?

Right. There are a few approaches we can use here:

1. continue to use 5.9.0 with the psoc-edge port (it works... but maybe something is missing?)
2. update our local copy in lib/cmsis to the latest 6.1.0 (we should probably do that anyway, for other ports to use)
3. add a copy of https://github.com/Infineon/cmsis as a submodule in lib/psoc-edge/cmsis and use that just for psoc-edge port

All options are pretty easy to get working. I would prefer option (2).

Let me emphasize that while I do like this PR for its simplicity and similarity with other ports, we do not have to go with this approach if it doesn't work for you. We need to find something that works for both sides (for us maintainers and for you/Infineon).

If we can get this PR into a state where you are happy to work on it and extend it, that would be great! That's also my preferred option. But if that doesn't work, at the very least we can hopefully use some ideas from here to improve your PR #18554.

I also added CI in this PR to build the new psoc-edge port. You can see that it's rather simple, just install the ARM toolchain (the same as all the other ARM ports), install edgeprotecttools, and run make.

Hi @dpgeorge,

From the preliminary exploration, and after checking with the team, it seems reasonable for us to go with this approach.

We understand that for the general MicroPython perspective, the harmonization of port integration eases the maintenance and support.
In our case, we are trading some pros and cons with this approach compared to the ModusToolbox one. And that should be fine.

I will take this PR and refactor our current fork's main branch (which enables a few more features already) based on it, and confirm that there isn't any major unforeseen issue.

Give me some days to do that, and I will come back and we can discuss how to proceed with this PR, and eventually the upcoming ones.

Thanks!

Let me also reply here to comments and points you brought up in this draft PR and in the other PR #18554:

Yes, that's definitely a valid reason to use ModusToolbox, to easily support multiple boards and libraries.

As two counterpoints to that:

1. We also need to consider keeping the port up to date with the rest of MicroPython. When things in MicroPython change or are added (eg a new wireless driver, improved IPv6 support, etc) the closer all ports are aligned in their implementation, the easier it is to change everything at once and keep the ports consistent.
2. I would hope that most of the middleware libraries can be provided by MicroPython, rather than by ModusToolbox. Eg filesystem drivers like FAT and littlefs, networking integration, Bluetooth support, USB via TinyUSB, among many other things.

I think we should discuss in more detail point (2). For example TCP/IP: we use lwIP as the TCP/IP library and have a relatively mature (non-OS) integration of that with the bare-metal ports. It works very well, is pretty efficient, and we have spent a lot of time to make sure the socket module in Python acts like it does on a desktop PC. Then on top of that sit all the other networking libraries, like requests, iperf3, microdot etc. It would be ideal if the PSOC Edge port could also use the lwIP integration provided by MicroPython, because then all the networking things would "just work".

Similarly with BLE, all the port needs to provide is a HCI UART/stream interface, and then MicroPython does the rest: using NimBLE or BTstack, and then our bluetooth module bindings, and then the aioble library on top of that. We really need to leverage all that existing infrastructure.

Hopefully that doesn't sound restrictive to you. After all, you didn't want to use the Zephyr port because you wanted more freedom with adding features. But I think there's still a lot of freedom to be had in a bare-metal PSOC Edge port, even if things like networking and BLE are provided by MicroPython.

In general, we have based our development of features on reference examples provided by Infineon in ModusToolbox. With the provided working reference and pre-configured stack middleware libraries.

In the case of the connectivity stacks, for example for PSOC6, this saved us quite some work to glue together LwIP, the transceiver driver, RTOS, etc. We could focus on enabling the network.WLAN module using the high level API provided by ModusToolbox.
Integrating some of these might not be trivial, and might require deeper knowledge of all these components.

I fully agree that we should strive for maximum reusability of existing MicroPython middleware. Still, in some cases, it might be more feasible from an effort and knowledge perspective to use the ModusToolbox-provided middleware, at least as a starting point. We can then later on refactor it to use the existing libs. I am sure we can discuss this on a case-by-case basis.

I just added the minimal set of flags needed to get it building and running, through trial and error adding files and flags until it would compile. I did not look at MTB.

Well, I also knew from experience what a Cortex-M33 needs (eg -mcmse) and for things like -DCOMPONENT_SECURE_DEVICE it would not build without them and I found out by looking at the source files that needed such things defined.

😊 Nice. Here we are not so experienced, and we find it handy to take the reference SDK examples to find out the right flags, includes, etc.

Right. There are a few approaches we can use here:

1. continue to use 5.9.0 with the psoc-edge port (it works... but maybe something is missing?)
2. update our local copy in lib/cmsis to the latest 6.1.0 (we should probably do that anyway, for other ports to use)
3. add a copy of https://github.com/Infineon/cmsis as a submodule in lib/psoc-edge/cmsis and use that just for psoc-edge port

All options are pretty easy to get working. I would prefer option (2).

All good for me, I would also favor option (2) to have the latest CMSIS version available for all ports. (3) as an intermediate step if we break a lot of things with such a bump.

Ideally, we can have a one-to-one mapping of the bsp folder with the "generated sources" dir, and with sufficient traceability to ease future updates.
There is information in the file banner about the toolchain version: https://github.com/micropython/micropython/pull/18843/changes#diff-d181533928c481d0caf54ed5d6525b0d17f0a0e22aefdea3d8dc9375945a907fR1-R12
Together with the BSP as a submodule, that should be sufficient to track the changes and update these BSP sources when needed.

Therefore, preferably any other files such as cy_cmsis_utils.h and armv8m_mpu.h should be included from their original repos.

I also added CI in this PR to build the new psoc-edge port. You can see that it's rather simple, just install the ARM toolchain (the same as all the other ARM ports), install edgeprotecttools, and run make.

That already looks great, thanks! 😊

From the preliminary exploration, and after checking with the team, it seems reasonable for us to go with this approach.

Excellent!

I will take this PR and refactor our current fork's main branch (which enables a few more features already) based on it, and confirm that there isn't any major unforeseen issue.

OK, I look forward to seeing the result.

(I did try myself to enable UART interrupts to have a larger input ringbuffer (using the standard MicroPython approach with py/ringbuf.h) but I could not get that to work correctly. Maybe you will have better success than me doing that.)

I fully agree that we should strive for maximum reusability of existing MicroPython middleware. Still, in some cases, it might be more feasible from an effort and knowledge perspective to use the ModusToolbox-provided middleware, at least as a starting point. We can then later on refactor it to use the existing libs. I am sure we can discuss this on a case-by-case basis.

Good that you agree we should try to reuse the existing MicroPython middleware code. I think we have good agreement as to the overall approach. Let's be pragmatic on a case-by-case basis, we can use ModusToolbox components if necessary.

Ideally, we can have a one-to-one mapping of the bsp folder with the "generated sources" dir, and with sufficient traceability to ease future updates.

Yes, I fully agree with that.

Might be good to add a makefile target to regenerate these bsp files from ModusToolbox (assuming the latter is installed somewhere on the system).

Therefore, preferably any other files such as cy_cmsis_utils.h and armv8m_mpu.h should be included from their original repos.

Agreed. At least armv8m_mpu.h should be available in an updated CMSIS version.

For now I will leave this PR as-is, and wait for your update.

Hi @dpgeorge,

Just a heads-up. As you can see in PR69, it was not so straightforward (at least to me) to handle the TrustZone and non-secure CM33 hex generation.

The MicroPython application is hosted in the non-secure section of the CM33, but the boot for the non-secure part needs to be triggered always from the secure enclave.
You will find a secboot/ directory which creates that binary and a nsc_veneer.o file.
This veneer is required during linking for the non-secure binary (the MicroPython one).

We are taking as reference the SDK projects and examples, in which a separate .hex is built for each of the cm33 secure, cm33 non-secure, and cm55 (which isn´t used/enabled here).
Maybe there are simpler and more elegant ways to handle this... but for now this I hope this is neat enough 😊

We are wrapping up the HIL testing enablement, and making this our main fork branch. And from there we will create a new PR.

Thanks!

The MicroPython application is hosted in the non-secure section of the CM33, but the boot for the non-secure part needs to be triggered always from the secure enclave.
You will find a secboot/ directory which creates that binary and a nsc_veneer.o file.
This veneer is required during linking for the non-secure binary (the MicroPython one).

Ah yes, I thought this might be an issue. I saw your original port #18554 created a secure "bootloader" that jumped into a non-secure MicroPython application. But then I also saw zephyr do everything in secure mode, ie the zephyr application boots into secure mode and stays in secure mode for the entire application. So that's how I did it in this PR, and it seemed to work OK.

I don't know the exact details and pros/cons of running in secure vs non-secure mode for the PSOC Edge. I'm keen to see your new PR and learn how you approached this problem.

I don't know the exact details and pros/cons of running in secure vs non-secure mode for the PSOC Edge. I'm keen to see your new PR and learn how you approached this problem.

Yes, I am not well versed either. My current understanding is basically that there are limitations and/or additional considerations when handling memory regions and peripherals from the secure mode. The benefits would be the secure features: data protection, firmware integrity, encrypted storage, protected communications...

I took your implementation to start with, but soon when I added some additional enablement it was not working anymore. The linker script was not the same provided by the BSP repo so the GC would just worked, the initial version was implemented for the non-secure part, and my zero experience with the secure mode. Thus, I decided to conservatively get things running as they were: minimal secure boot + micropython in non-secure😊

While we don´t need to leverage the actual secure features, not sure if we will benefit from running everything in secure mode other than simplifying the build with a single hex.

Anyhow, this not an approach that can't be changed in the future as we learn further 😊

Closing, superseded by #18910.