Simpler mechanisms for asynchronous processing (thoughts)

These are some strawman thoughts about how to provide handling of asynchronous events in a simple way in CircuitPython. This was also discussed at some length in our weekly audio chat on Nov 12, 2018, starting at 1:05:36: https://youtu.be/FPqeLzMAFvA?t=3936.

Every time I look at the existing solutions I despair:

asyncio: it's complicated, has confusing syntax, and pretty low level. Event loops are not inherent in the syntax, but are part of the API.
interrupt handlers: MicroPython has them, but they have severe restrictions: should be quick, can't create objects.
callbacks: A generalization of interrupt handlers, and would have similar restrictions.
threads: Really hard to reason about.

I don't think any of these are simple enough to expose to our target customers.

But I think there's a higher-level mechanism that would suit our needs and could be easily comprehensible to most users, and that's

Message Queues
A message queue is just a sequence of objects, usually first-in-first-out. (There could be fancier variations, like priority queues.)

When an asynchronous event happens, the event handler (written in C) adds a message to a message queue when. The Python main program, which could be an event loop, processes these as it has time. It can check one or more queues for new messages, and pop messages off to process them. NO Python code ever runs asynchronously.

Examples:

Pin interrupt handler: Add a timestamp to a queue of timestamps, recording when the interrupt happened.
Button presses: Add a bitmask of currently pressed buttons to the queue.
UART input: Add a byte to the queue.
I2CSlave: Post an I2C message to a queue of messages.
Ethernet interface: Adds a received packet to a queue of packets.

When you want to process asynchronous events from some builtin object, you attach it to a message queue. That's all you have to do.

There are even already some Queue classes in regular Python that could serve as models: https://docs.python.org/3/library/queue.html

Some example strawman code is below. The method names are descriptive -- we'd have to do more thinking about the API and its names.

timestamp_queue = MessageQueue()        # This is actually too simple: see below.
d_in = digitalio.DigitalIn(board.D0)
d_in.send_interrupts_to_queue(timestamp_queue, trigger=RISE)

while True:
    timestamp = timestamp_queue.get(block=False, timeout=None) # Or could check for empty (see UART below)
     if timestamp:    # Strawman API: regular Python Queues actually throw an exception if nothing is read.
        # Got an interrupt, do something.
        continue
        # Do something else.

Or, for network packets:

packet_queue = MessageQueue()
eth = network.Ethernet()
eth.send_packets_to_queue(packet_queue)
...

For UART input:

uart_queue = MessageQueue()
uart = busio.UART(...)
uart.send_bytes_to_queue(uart_queue)
while True:
    if not uart_queue.is_empty:
        char = uart_queue.pop()

Unpleasant details about queues and storage allocation:

It would be great if queues could just be potentially unbounded queues of arbitrary objects. But right now the MicroPython heap allocator is not re-entrant, so an interrupt handler or packet receiver, or some other async thing can't allocate the object it want to push on the queue. (That's why MicroPython has those restrictions on interrupt handlers.) The way around that is pre-allocate the queue storage, which also makes it bounded. Making it bounded also prevents queue overflow: if too many events happen before they're processed, events just get dropped (say either oldest or newest). So the queue creation would really be something like:

# Use a list as a queue (or an array.array?)
timestamp_queue = MessageQueue([0, 0, 0, 0])
# Use a bytearray as a queue
uart_queue = MessageQueue(bytearray(64))

# Queue up to three network packets.
packet_queue = MessageQueue([bytearray(1500) for _ in range(3)], discard_policy=DISCARD_NEWEST)

The whole idea here is that event processing takes place synchronously, in regular Python code, probably in some kind of event loop. But the queues take care of a lot of the event-loop bookkeeping.

If and when we have some kind of multiprocessing (threads or whatever), then we can have multiple event loops.

@BBoSer

For a different and interesting approach to asynchronous processing, see @BBoSer's https://github.com/bboser/eventio for a highly constrained way of using async / await, especially the README and https://github.com/bboser/eventio/tree/master/doc. Perhaps some combination of these makes sense.

I'm unqualified at this point to talk about implementation, but from an end user perspective I like the idea of this abstraction quite a bit. It feels both like a way to shortcut some ad hoc polling loop logic that I suspect people duplicate a lot (and often badly), and also something that could be relatively friendly to people who came up on high-level languages in other contexts.

People aren't going to stop wanting interrupts / parallelism, but this answers a lot of practical use cases.

I like event queues and I agree they are quite easy to understand and use, however, I'd like to point out a couple of down sides for them, so that we have more to discuss.

Queues need memory to store the events. Depending on how large the event objects are, how often they get added and how often you check them, this can be a lot of memory. Since events are getting created and added from a callback, that has to be pre-allocated memory. And I don't know of a good way of signalling and handling overflows in this case — depending on use case, you might want to have an error, drop old events, drop new events, etc. To save memory you might want to have an elaborate filtering scheme, and that gets complex really fast.
Queues encourage a way of writing code that introduces unpredictable latency. The way your code usually would flow, you would do your work for the given frame, then you would go through the content of all the event queues and act on them, then you would wait for the next frame. In many cases that is perfectly fine, but in some you would rather want to react to the event as soon as possible.
Every new kind of queue will need a new class and its own C code for the callback and handling of the data. So if you have a sensor that signals availability of new data with an interrupt pin, you will need custom C code that will get called, read the sensor readings and put them on the queue. That means that all async drivers would need to be built-in.
Sometimes a decision needs to be done while the event is being created, and can't wait. For example, in the case of the I2C slave, you need to ACK or NACK the data, and you have to hold the bus in a clock-stretch until you do.

That's all I can think of at the moment.

Hm, I do not fully understand, why such a MessageQueue model should be easier to understand than callbacks. Maybe it's, because I am used to callbacks ;-)
What is so special with your target customers, that you think, they do not understand callbacks?

I think, you have to invest much more brain in managing a couple of MessageQueues for different types of events (ethernet, i2c, timer, exceptions, spi, .....) or one MessageQueue, where you have to distinguish between different types of events, than in implementing one callback for each type of event ant pass it to a built in callback-handler.

def byte_reader(byte):
      deal_with_the(byte)

uart = busio.UART(board.TX, board.RX, baudrate=115200, byte_reader)

@deshipu

I like the idea of message queues but I'm not convinced that they're any easier to understand than interrupt handers. Rather I think that conceptually interrupt handlers/callbacks are relatively easy to understand but understanding how to work with their constraints is where it gets a bit more challenging. Message queues are a good way of implementing the "get the operable data out of the hander and work on it in the main loop" solution to the constraints of interrupt handlers but as @deshipu pointed out, there are still good reasons to need to put some logic in the handler. Maybe both?

Similarly I like how eventio works but I think it's even more confusing than understanding and learning to work with the constraints of interrupt handlers. That in mind, it's tackling concurrency in a way that I think might be more relatable to someone who came to concurrency from the "why can't I blink two leds at once" angle.

One thing I was wondering about is what a bouncy button would do to a message queue. Ironically I think overflow might actually be somewhat useful in this case as if the queue was short enough you'd possibly lose the events for a number of bounces (but not all of them unless your queue was len=1. I'll have to ponder this one further). With a longer queue you could easily write a debouncer by looking for a time delta between events above a threshold.

No matter how you slice it, concurrency is a step beyond the basics of programming and I don't think any particular approach is going to allow us to avoid that. It seems to me that we're being a bit focused choosing a solution to a set of requirements that we don't have a firm grasp on yet. I think it's worth taking the time to understand who the users of this solution are and what their requirements are.

See #1415 for an async/await example.

What is so special with your target customers, that you think, they do not understand callbacks?

The problem is not with the callback mechanism itself, but in the constraint that MicroPython has that you can't allocate memory inside a callback. This is made much more complex than necessary by the fact that Python is a high level language with automatic memory management, that lets you forget about memory allocation most of the time, so it's not really obvious what operations can be used in a callback, and how to work around the ones that can't.

One solution would be to enable MICROPY_ENABLE_SCHEDULER and only allow soft IRQ's, running the callback inline with the VM. This would prevent people from shooting themselves in the foot.

Refs:

Thank you all for your thoughts and trials on this. I'll follow up in the near future but am deep in Bluetooth at the moment. The soft interrupts idea and the simplified event loop / async / await stuff is very interesting. I think we can make some progress on this.

@siddacious

From my experience with what I think is the CircuitPython audience (I teach technology to designers), I don't think message queues are easier to understand than other approaches, and are probably harder in many cases. As @siddacious says, concurrency takes a while for newcomers to wrap their heads around no matter what the method.

I also think it's important to distinguish between event driven needs and parallelism. In my experience, the most common need amongst my students is doing multiple things at once, e.g. fading two LEDs at different rates, and perhaps doing this while polling a distance sensor. This requirement is different from the straw man example above.

Some possible directions:

MicroPython supports _threads. To me, this would be a fine solution, especially since it is already documented and there are examples out there. Mine for example: https://github.com/pvanallen/esp32-getstarted/blob/master/docs/multitasking.md
Implement something along the lines of the TaskScheduler Arduino library, which is great: https://github.com/arkhipenko/TaskScheduler

Hi, I am sorry but do not want to read all comments on this issue. I study and work on real-time systems. Circuit python is for microcontrollers. They are not PCs - real PC-multitasking on a single CPU is not needed at all.

Everything that CircuitPython programmer needs is scheduler :) If three tasks (A, B, C) with two sections (example: AA) runs this way:
ABACBC or BBAACC is the same. If you need cooperation between tasks, you can use shared variables, queues, etc. and split tasks into multiple interlaced tasks.

In real-time applications, what is needed are priorities, a way to set order or schedule, and a tool that tells you when tasks run.

The essential is scheduling. Users can plan everything, write it down into a timetable, and then implement it as the schedule for the scheduling mechanism. The user only needs to know when the task starts, its deadline, and if it is continuous and period of task repetition. There are required only two things: a way to create a schedule and inform the user if a task misses its deadline.
Priorities are harder to implement - you need a mechanism to stop a running task and run a task with the highest priority if required. Much more important is way to temporarily elevate the priority of low priority task if it access to resources needed by higher priority task to prevent deadlocks.

If someone implements scheduling framework into CircuitPython, someone else can implement its scheduler, and an inexperienced user can use it. There are multiple scheduling strategies. Every single one is good for a different problem, and no single strategy is right for everything.

Multitasking implemented with the scheduler is the way to go. It is predictive and straightforward. If every task informs CircuitPython about its start and end, tasks can easily be visualized for an inexperienced user to debug deadlines, schedules, or priorities easily.

I would be pleased if I found an EDF scheduler in CircuitPython or at least a way to set up static scheduling. I don't think it's going to be possible to implement advanced scheduler only in Python, and it's probably going to take help from inside of CircuitPython implementation.

@electriczity

@electriczity that is exactly what async/await does.

@deshipu

@deshipu Async&await is for PCs or Mobile phones when you do not need information on how asynchronous functions are executed in the background and only required is the result and or smooth GUI response. A small delay or more significant overhead is easily overlooked and usually does not bother anyone. ...and!.. usually good async scheduler needs another thread or some level of cooperative multitasking. When the async&await mechanism is implemented using some sort of synchronous polling it is not asynchronous, but it is a weird synchronous scheduler with wrong syntax sugar.

So, the short answer:
Only type async and await keyword is not enough and gives little control over things and can be dangerous in a limited microcontroller universe.

Long one:
Maybe you do not actually understand what microcontroller programming is. Yes, microcontrollers can be programmed using standard paradigms like on a PC, but with a relatively unsuccessful or unsatisfying outcome. Once you start using microcontrollers, you intentionally or unintentionally create a realtime system. They will be hard realtime systems or soft realtime systems. Hard RT systems are these when the tasks have deadlines, and these must be fulfilled at all costs (a nice example is automatic espresso machine). Soft RT systems have more laxity, and most people make them and some of them are crying out for something better. There are tasks without critical deadlines or with soft deadlines and the programmer has little control over the schedule or does not need it. Literally, there is sufficient that tasks are executed even with the delay or in different order.

On microcontrollers, you need to manage tasks in an entirely different way than multithreading or async calling (asynchronous functions besides interrupts exist even in realtime systems, but they are much less common and they are scheduled differently). Task have time when it can start, and time when it must end. You need to control when the task is executed by yourself (static scheduling) or need a predictive scheduling algorithm because everything has side effects. Side effects are there intended or hidden. Intended are relay switching, led blinking, etc. Hidden side effects like memory allocation can be dangerous. Tasks or async routines have memory usage, bus usage, and peripheries can be power-hungry, etc.

@electriczity

Hi, I am sorry but do not want to read all comments on this issue.

Hi @electriczity, this issue has moved well past the theoretical and into practical territory. Please feel free to avail yourself of the concrete code listings in this issue and both the Circuitpython and upstream Micropython projects if you'd like to share insights from a position of context and understanding.

In any case, async/await is the Pythonic way to talk about cooperative multitasking. Its overhead is implementation dependent and could be anything, including zero.
How to actually schedule these tasks when you need soft realtime (hint: you often don't) is an orthogonal problem. Same for the tasks' side effects, same for the method(s) to wake up a task that's waiting for an external event (= interrupt).

Please don't conflate these issues.

@smurfix Hi, you are right. It is my fault. The scheduler is different problem and it is actually separable from an asynchronous calling implementation. Scheduler must support asynchronous tasks or they can be executed in a specific periodic task.

@dhalbert

async/await keyword support has been accepted into CircuitPython as of #3540

There is currently no native scheduler implementation or interrupt-scheduled coroutines (yet!) but as @dhalbert mentioned back in August simple time-based pure-python async scheduling has been demonstrated on CircuitPython; also it's easy to reproduce from scratch (and completely understand) with not much more than following along with the good Mister Beazley https://www.youtube.com/watch?v=Y4Gt3Xjd7G8

I think it's wonderful that async/await keywords are not tied to a specific implementation of concurrency - you can use a global event loop if you like, or you can keep track of all your tasks and call send(None) on them in your own loop() method to move them along, or you can use a scoped / context manager event loop to do trio style concurrency... or you can literally do all 3 at once if you want to. The language is unopinionated about how you use coroutines and that's just dandy =)

There's one missing piece here: we need to be able to schedule a task when something happens, i.e. an interrupt routine must be able to wake up the core scheduler. This requires an atomic "check whether this attribute of that object is None; sleep until the next interrupt / for time X only if it is" library function as a building block.

MicroPython has machine.lightsleep(); I didn't find a CircuitPython equivalent. time.sleep() doesn't work for this because it's not terminated by an interrupt AFAIK.

yes, i mentioned that interrupt support is not there.

But no, it is not necessary. If nobody can interrupt and there are no currently active tasks, then nobody can make a new task until the next scheduled task. See the code in the tasko library if you are unsure of how that works. Just like in CPython, you need to call sleep on the event loop if you want the event loop to do the sleeping.

I'm not talking about the application, I'm talking about the event loop itself. In CPython the event loop doesn't call "sleep", it calls "select" or "epoll" or whatever, which is a sleep that can be stopped by what's essentially an interrupt.

What do you mean, it's not necessary? Of course it's necessary. People want to put the MCU into some sleep state when it has nothing to do, that saves a ton of battery power. There are lots of situations where you want the sleep to end when an interrupt arrives. Input pin change, serial character arrives, I2C slave gets selected … why should I wake up my MCU every 10ms to check for that? that' what interrupts are for. I want to use them.

An interruptible scheduler works by

while there are any runnable tasks: run them.
disable interrupts
if there are any runnable tasks: race condition averted! re-enable interrupts, start at the top
if there's a timeout (i.e. a task that's scheduled to run in the future), tell the hardware to trigger an interrupt when the time is up
PUT_CPU_TO_SLEEP hardware instruction (which implicitly re-enables interrupts)

Without steps 2 and 3 you'd get a race condition. You'd also get one if you re-enable interrupts before SLEEPing, the MCU must do that atomically as part of its SLEEP instruction.

If time.sleep() was guaranteed to terminate when an interrupt arrives, great, we could use it for the last two steps of the above algorithm – but I just looked at the code, and mp_hal_delay_ms() obviously doesn't do that. It also runs background tasks while sleeping, thus we can't even disable interrupts before calling it.

To build a "real" event loop, we'd need to be able to call port_interrupt_after_ticks() and port_sleep_until_interrupt() from Python, in addition to disabling and re-enabling interrupts (which is already possible). Simple enough to do IMHO.

friend be my guest and implement it. I look forward to your pull request.

I'm not sure if I'm communicating badly but literally all I'm trying to say is that async/await keywords exist and they do sane things (sane as defined by their CPython behavior).

Scheduling is an utterly different problem altogether and yeah, if you are designing a scheduler of course you'd want interrupts but no you definitely do not need them to schedule tasks in an ecosystem that does not have interrupts. I've provided an existence proof. If you want them, or if you need them for some reason, please feel welcome to contribute. I also want them and will add support at some point when I have free time unless someone else has free time and motivation first.

In the meantime, barebones async/await noun/verb pair should work great for people to learn on circuitpython, library support notwithstanding.

Will do.

I'm closing this issue because async and await have been enabled thanks to @WarriorOfWire. For further work we can open new issues. Thanks all!

@smurfix See #2796 for deep sleep and #2795 for light sleep APIs.

Navigation Menu

Repository navigation

Simpler mechanisms for asynchronous processing (thoughts) #1380

98 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Simpler mechanisms for asynchronous processing (thoughts) #1380

Description

Activity

dhalbert commented on Dec 6, 2018

brennen commented on Dec 6, 2018

deshipu commented on Dec 6, 2018

framlin commented on Dec 19, 2018

siddacious commented on Dec 19, 2018

notro commented on Dec 20, 2018

deshipu commented on Dec 21, 2018

notro commented on Dec 22, 2018

dhalbert commented on Dec 26, 2018

pvanallen commented on Apr 3, 2019

98 remaining items

electriczity commented on Sep 6, 2020

deshipu commented on Sep 6, 2020

electriczity commented on Sep 7, 2020

kvc0 commented on Sep 7, 2020

m-u-xyz commented on Sep 7, 2020

electriczity commented on Sep 7, 2020

kvc0 commented on Oct 13, 2020

m-u-xyz commented on Oct 17, 2020

kvc0 commented on Oct 17, 2020

m-u-xyz commented on Oct 17, 2020

kvc0 commented on Oct 17, 2020

m-u-xyz commented on Oct 17, 2020

tannewt commented on Oct 18, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions