Repository navigation
New machine.CAN driver #12337
Description
Activity
I am ready to correct port/esp32: Add CAN(TWAI) driver #12331 according to this 'machine.CAN' API.
Reacted by Alexey Zhukovsky and peruihkxtReacted by Angus Gratton, winnie xiong, jornamon, peruihkxt and Sveinung Kval BakkenAwesome!
Some comments on Transmit and Receive:
I previously worked at Embark Trucks doing a lot of their embedded system design. I have seen how to use CAN effectively and have thoughts on what works well and what does not.
CAN technically is a message buffer interface system. Generally, most embedded systems do not actually use FIFO based approaches when sending or receiving CAN data. Typically, the message buffer allocation for RX/TX buffers is fixed and pre-allocated by your code. The processor then determines when message buffers have new data and it reads that and processes it.
However, this kind of interface is very foreign when compared to Ethernet/WiFi/UART which all have stream based approaches. Given this, most users don't know what to do with message buffers and will be confused on how to use them correctly if they have never used CAN before.
So, the FIFO based stream interface makes CAN easier to use for beginners. Given this, I support that we should use a fifo based interface over a RAW message buffer based interface.
Moving, send/receive are easy for users to understand. However, it's important that any CAN driver that's implemented respect message ordering with FIFOs. E.g. on my IMXRT port recv() messages that pass through the CAN filters will be in order given what was sent on the bus. For transmit, I explicitly re-use the same message buffers for transmit if the CAN id matches in-order to keep transmit message ordering too. I am pointing out this requirement as if transmit order is not kept it makes it impossible to perform any CAN message protocols (which exist). The STM32 PYB.CAN port does not implement TX message ordering for it's basic CAN (non-fd) transmit method (Message ordering between different CAN ids is generally not required).
Anyway, TLDR, message ordering on ID should be a requirement.
Reacted by Angus Gratton, Damien George and AndrewDAs for masks on filters.
This is very rarely used. Typically, you are just looking for a few IDs that you have filters setup for. I would drop support for masking message filters. We never used this in practice on any of our MCUs. You either had a message you were looking for or not. Same for a range of IDs.
Reacted by Angus Gratton and Damien GeorgeCould we consider how the API might support external CAN controllers as well ? I currently maintain my own MP driver for the ubiquitous MCP2515 SPI chip but I'd be amenable to changing the interface to meet the new standard, particularly if it meant my application code were more portable to architectures with on-chip CAN. I also maintain a number of Arduino libraries for various CAN implementations. The lowest common denominator abstract interface is init, available, receive and send. The CircuitPython canio interface/base class seems similar.
@obdevel external controllers seem relevant to me too, surely a common CAN interface should be able to support them too!
Could you share some details about the public API you currently have as a point of comparison?
I'm personally invested in promoting an api that matches desktop
python-cansimply to make supporting other existing higher level can libraries, but I'm not experienced with CAN so looping people who do actively use it into the conversation makes sense to me.I'd recommend to never use the MCP2515. It actively randomly corrupts CAN packets. If you are building any application in the world using that part will lead to intermittent failures happening. It may be possible to write a driver such that this doesn't happen... but, I repeatedly ran into that chip failing to register packets sent on the CAN bus to the host MCU and also saw events where it would corrupt nominal data on the bus during transmit.
@obdevel external controllers seem relevant to me too, surely a common CAN interface should be able to support them too!
Could you share some details about the public API you currently have as a point of comparison?
I'm personally invested in promoting an api that matches desktop
python-cansimply to make supporting other existing higher level can libraries, but I'm not experienced with CAN so looping people who do actively use it into the conversation makes sense to me.From my experience maintaining Arduino libraries for the past few years, the underlying implementations - whether on-chip or external - are so different that any API is going to be pretty abstract. As I said, it's at the level of init(), send(), checkavailable() and receive(). I currently support ESP32, Pico PIO, SAMC and SAME, as well as the external MCP2515.
The Adafruit CircuitPython canio API is documented here: https://docs.circuitpython.org/en/latest/shared-bindings/canio/index.html, although I don't particularly like it ;)
I'm happy to work with whatever API the project comes up with. For an external SPI controller, it would probably be a a pure-python implementation, rather than baked into the interpreter, but I'm happy to be challenged on that.
I'd recommend to never use the MCP2515. It actively randomly corrupts CAN packets. If you are building any application in the world using that part will lead to intermittent failures happening. It may be possible to write a driver such that this doesn't happen... but, I repeatedly ran into that chip failing to register packets sent on the CAN bus to the host MCU and also saw events where it would corrupt nominal data on the bus during transmit.
That is not my experience over the past 6 or 7 years, although many of the existing driver implementations are fairly poor. You need to make sure you service interrupts in a timely fashion as it only has two hardware reception buffers.
The MCP2515 does have one big advantage when considering the hobbyist and maker community ... it's the only external CAN transceiver chip available in a through-hole package. All others are small surface-mount packages which are beyond the capability of most people.
Could be that. I always grabbed a driver from online.
I forgot to mention before that there is a RP2040 PIO implementation of a CAN controller, written in C (https://github.com/KevinOConnor/can2040). I created a thin Arduino wrapper library last year and it works very well. There are only very minor deviations from the CAN spec. Our application project has felt comfortable enough to standardise on this.
Reacted by Angus GrattonThanks everyone for the input here, it's really helpful to see the scope of what people are thinking about Re: Micropython and CAN and to learn from collective experience. It will be a little while before there's much more movement from me on this, but I'm very keen!
I'm happy to work with whatever API the project comes up with. For an external SPI controller, it would probably be a a pure-python implementation, rather than baked into the interpreter, but I'm happy to be challenged on that.
Yes, I agree this is a good goal - to make it possible to implement a driver using an external controller whose API matches the
machine.CANone (i.e. can instantiate the driver object and then pass it in place of amachine.CANinstance, subject to whatever restrictions the controller has).It would be great if these are eventually contributed as drivers in micropython-lib (so people can
mip installthem), written in pure Python. This all seems to me like it should be viable.Reacted by obdevel6 remaining items
Hi can we get some progress on this? The OpenMV Cam RT1060 is coming out soon and CAN support is blocked right now.
Hey @kwagyeman,
I understand this is a hassle for the OpenMV product release and that it's be very desirable for you folks to have full upstream support including a CAN driver.
For now, the situation in the first post of the issue remains true - this is blocked behind the USB device driver work. It will be a while until there's visible progress, and I suspect even once a draft API is available then will remain a draft for a while longer as trade-offs are being talked through, details checked, early versions of actual drivers are implemented and tested, etc. I also don't want the process to take too long, but getting a good long-term result will require some of that back and forth. I don't foresee any way around that, unfortunately.
In terms of your product schedule, is there a way to work around missing upstream CAN support (and/or defined
machine.CANAPI) in the interim? Does maintaining a fork (for now) or an out of tree driver work as a stop-gap?We can, I finished the driver already, it's just that it will be the wrong long-term API. Only a few people will probably need it so we can just give me what I have right now... and then switch later.
Reacted by Angus GrattonSounds good. Hopefully switching will be an option before too long.
It's really important that upstream support is done properly, and avoids the common CAN bear traps (eg priority inversion). I'd prefer to wait for that so that at the end I don't need to maintain a fork and can put future efforts into enhancing the upstream.
Any updates on this?
Looking forward to it very much.
Thanks everyone for your patience. An initial draft machine.CAN API is up, please take a look and leave any comments.
Where is the link to the draft?
Ah, I see it above. Was hidden on GitHub mobile app.
As a typical "python user" with basic hw-knownledge, that has the interest to use CAN on micropython, I was captivated by reading where this implementation started, #5087 and how it formed over the past years, only to be finally merged soon! I really take my hat off to your work and look forward to being a user of your fruits soon. I hope my words motivate you a little!
I'm planning to build a tiny dual-CAN-device using off-the-shelf parts and programming in micropython. :)
Is there a plan to restart the effort on this? It would be a great feature for a project I have planned with an imxrt.
Reacted by tigattack, peruihkxt, Sveinung Kval Bakken, Andrew Wingate and TomblaromReacted by peruihkxt, Andrew Wingate and TomblaromI was excited seeing @Tomblarom's comment from February, but seems like it was wrong February. What is remaining to get this merged? In particular for esp32.
@sveinungkb
I corrected the basic parameters port/esp32: Add CAN(TWAI) driver #12331 according to this 'machine.CAN' API.I'm also very interested. Is there anything I can do to help this along?
Reacted by derpston
Summary
Planned cross-platform
machine.CANdriver for Micropython.Background
MicroPython currently only has one driver for a CAN bus controller, the pyb.CAN driver on STM32. Various folks have contributed functional and useful pull requests with CAN drivers for other ports. Many implement a
machine.CANinterface. Quite sensibly, these drivers have all based their new APIs on thepyb.CANdriver. These efforts are hugely appreciated 🙏.@jimmo, @dpgeorge, and I discussed this recently. We all agree MicroPython should have a cross-platform
machine.CANinterface. None of us want to use thepyb.CANAPI for this:pyb.CANdon't match the best conventions ofmachineAPIs.Goals
MicroPython should have a
machine.CANinterface which is:machineAPIs.asyncio.Also:
Plan
We plan to do the following:
machine.CANAPI as a docs PR, discuss with other maintainers and the community. See docs: Add machine.CAN low-level API docs. #13149extmod/machine_can.c(similar toextmod/machine_pwm.c, etc).machine.CANimplementation on STM32 (this will be a thin wrapper aroundpyb.CAN, which will remain for legacy compatibility.)machine.CANdrivers for new ports.GitHub sponsor funds have been allocated for the first three steps, and I plan to work on it. Currently other work (USB device driver functionality) is the priority, but this is the next priority after that.
This issue post will be updated with overall status of this effort.