Ideas on optimizing uasyncio

With #2219 landed and uasyncio updated to it (in a branch), discuss further ideas for improvement (many would require changes to uPy for "optimizations"), so ticket is here:

A thought: millisecond-perfect tasks scheduling requires working with times on microsecond level, and in a smart way. E.g. there's no warranty that after sleep_ms(20) ticks_ms() will be 20 late (can be 21, on funny systems can be 19).
The aim should be achieving memalloc-free running of non-recursive coroutines. The current culprit is actually packing task entries into a queue, which requires creating a tuple. Thus, uasyncio queue should be optimized, and now that RFC: Optimized heap queue implementation for uasyncio #2219 has landed, the next obvious is step is to revert it and redo as an adhoc queue. That all was expected long ago, but seeing's believing.
yield from's aka await's from cosubroutines still will pose a problem, each such cocall will cause allocation. Here, we should try to exploit paradigmatic duality of closures and objects - they both approach storing internal state from different sides. And indeed, yield from works not with just coros, but with objects, on which it will call __next__(). However, we still don't want to allocate object either, what I have in mind is having and async method of an object with a special call semantics, where it's passed its object and can yield (unlike __next__??). This is not achievable in Python, but we'll definitely need ability to code coroutines in C. So, the idea is to have:

line = await stream.readline()

to have following call semantics:

StreamClass.readline__next__(stream)

I.e. there should be special methods in an object, await'ing which just calls them as a normal method, but allowing it to indicate that it yields in addition to just returning a value. The problem comes with handling arguments, e.g.

line = await stream.readline(512)

That either will require special compiler treatment to transform that to:

StreamClass.readline__next__(stream, 512)

Or, given that coroutine semantics is strictly sequential, i.e. it's an error to call .readline() coroutine on the same object while another call is active, i.e. there's nor reenterability problem, we can do it like:

stream.last_args = (512,)
StreamClass.readline__next__(stream)

This can be completely implemented on object level, i.e. __call__ on special comethod stores passed args and returns self, then __next__ on it executes it.

A bit off topic, but an async library at the lower, socket level like curio seems a lot more intuitive and better suited to embedded.

So instead of having to use streams, transports, etc, there'd also be the option to simply await socket.recv(100).

but an async library at the lower, socket level

Either that, or vice-versa, at the higher and C-optimized HTTP (etc.) level.

like curio seems

Recent discussion on python-tulip was indeed insightful. First of all, even well-known in CPython community people criticize asyncio and dare to keep writing alternative coroutine libraries (just remember CPython2 times, there were couple of dozens of such libs, that was fun), people who want native Python coroutine library, not Twisted frankenstein, complain more and more, and upstream asyncio developers keep ignoring feedback and give hilarious answers to any pin-points at where asyncio is deficient. All that no wonder, because the main requirements behind asyncio were political, not technical (to get that zoo of a couple of dozens async libs from CPy2 era together into CPy3 and love each other).

uasyncio was on that way long before curio, but I exactly "didn't dare" to stray away too much from the upstream. Now that the upstream vividly fails at unifying async usage, I do wonder if going independent way have been, or still is, a viable way. But I don't have any specific ideas how to rewrite it specifically to be totally incompatible with asyncio in a way that it was actually useful. Any incompatible, but useful divergences were made, and more will be made as "baremetal" support is added, but there is still familiar asyncio methods, basic knowledge can be reused, etc. If you have specific ideas - feel free to open RFC thread on the forum.

So instead of having to use streams

Sorry, but it's impossible not to use streams. All I/O is done via streams. In CPython, socket is hack-up of, but still a stream. In uPy, a socket is subclass of stream. There can be no sockets, but can't be no streams.

transports

Transports are political idiosyncrasy in asyncio, specifically picked up from Twisted to get its crowd "onboard". Nobody uses them IRL. There never have been transports in uasyncio.

await socket.recv(100).

Sorry, can't imagine anything more useless than that. That gives you random number of bytes in range 1-100, and what will you do with that? Mix and match it on your application side? In uasyncio, it's stream.read(100), and gives (will give after refactoring complete) guaranteed amount of bytes from any stream, be it socket, UART, whatever.

Sorry, can't imagine anything more useless than that.

Works for most people.

My point is that people know sockets and tend to be confused by asyncio (I "learned" asyncio a year ago but now I'd struggle to recall how to write a UDP server if you asked). Making async work seamlessly by basically dropping in a await would make it easy. Of course having a file-like stream API is good too, but they don't have to be mutually exclusive (just like in curio).

FYI this is a good read: https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/

@peterhinch

Works for most people.

Most people write incorrect or excessive/supeflous/bloated/inefficient code. None is particular good for approach for "embedded" with which you started your comment.

But I gave examples above to show that there never has been agreement on how to do it in one way, and curio example seals the final stamp on ability to have it for Python3. Async stuff was also one of the earliest examples of duplication of effort (and thus hindered overall progress) in MicroPython (@peterhinch can confirm). That was 2 years ago though, and now we're at the stage where we definitely not getting more agreement, but at the stage where pluralism of solutions is what's going to move MicroPython forward. So, if you have some idea, I whole-heartedly recommend: go and implement it (don't wait for someone else to do that).

they don't have to be mutually exclusive (just like in curio).

Of course they don't have to be mutually exclusive - one of them is simply superfluous, subsumed by the other. And curio just keeps dragging two of them along, in the tradition of "desktop" CPython. At 140K and 19 files, I find that I have little to learn from curio beyond what I already shared above: more and more people start to more and more seriously ignore asyncio and develop alternatives. Otherwise, curio looks like a typical insightful David Beazley's presentation, just went beyond the "presentation" level into a "practical" "for desktop/server usage" system. That's how seemingly simple ideas transformed into 140K of code of the average level of ugliness. I encourage you to try and build elegant, lightweight, and still practical system on the same ideas and prove me wrong. I however going to work from what I learnt of my practice of working with an "embedded" implementation (and from watching actual "embedded" competition) - and I have to admit a lot of that are still experimental ideas.

Ok, so this has gone into an RFC of discussing possible alternative route than just staying "somewhat" compatible with asyncio. So, references:

Curio discussion on python-tulip list: https://groups.google.com/forum/#!topic/python-tulip/7iOVVM6eQjg
Armin Ronacher "I don't understand Python's Asyncio" http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/ (nice write-up which can be reduced to: "if you print asyncio documentation, shred into pieces, and try to read, you won't understand anything", but of course the point is shown well - asyncio is overly complex)
Random doc from the blog above: https://docs.google.com/document/d/1tlQ0R6wQFGqCS5KeIw0ddoLbaSYx6aU7vyXOkv-wvlM/edit# ,

The doc above discusses suffering of JavaScript community with their "full stack, same tool on client and server" approach, except that the tool is rickety hammer, so everyone around just knocks about. I especially enjoy comments like "Go is young enough and opinionated enough that an edict can work. Passing the context explicitly ensures that it cannot be lost in transit." So, if some adopts the most obvious solution, it's because they "young and opinionated". And if JS community can't - just can't - find a good solution, it's not because their tools aren't up to task, half of the community doesn't understand the problem, and overall nobody really cares (it's JavaScript after all, it should be easy), it's because of some other reasons.

Anyway, either I'm dumb, or that doc discusses just wanted we (or maybe just I) have done above: when doing async programming, it's really useful to have a good way to access context across various parts of the code - well, this part is just the same as for syn code, but with async it may be a bit less straightforward than with sync programming. That's exactly the reason why I talk about going for higher-level approach (like, optimized HTTP clients with the object to store state), and express concerns of practicality of usage of "raw sockets" in async code. And either curio "sockets" aren't really sockets (and 140K of code kinda hint of that), or beyond simple usage, curio may be not that practical.

I don't want to appear overly pedantic, but can I just point out that asyncio is actually a Kickstarter stretch goal. Personally, all I want is a good async mechanism, but around 1400 people did buy into the deliverables, and where reasonable that should be respected.

I also have difficulty in understanding the full glory of asyncio, but a subset which is appropriate for embedded would seem to be the way to me. The stream abstraction, if it doesn't cost too much, also is very attractive to me.

Or, given that coroutine semantics is strictly sequential, i.e. it's an error to call .readline() coroutine on the same object while another call is active,

I don't follow that, but then I don't use asyncio much. In general terms it isn't necessarily an error it just results in a spinlock until the resource mutex is released. Still sequential, but no error - just a wait state.

stream.last_args = (512,) StreamClass.readline__next__(stream)

Ugly. I don't know how much re-engineering it takes to handle the other mechanism though. In fact I can't see why it is materially different to passing parameters to any other class method than needs an implicit instance reference. Which source file best illustrates the problem?

@pfalcon

@pfalcon I'm baffled how you reconcile these views: over two years on you evidently still think my work on a scheduler

thus hindered overall progress

yet in the same post you say to someone else

So, if you have some idea, I whole-heartedly recommend: go and implement it

I agree with your second statement and think there's room for more than one solution; I'm genuinely baffled by your evident continuing disapproval of my simple Python effort.

That aside, I'm agnostic on the subject of how suited asyncio will be for hardware related projects rather than networking (for which it seems well suited) and look forward to trying your solution. On a practical note, while very fast allocation free context switching is awesome, in real MicroPython applications latencies on the order of low tens of milliseconds are more the norm because threads have to get some work done before yielding control to the scheduler. This is controllable in the code design, but I suspect that practical experience will generally bear out the results of my testing.

@pfalcon

@pfalcon I'm baffled how you reconcile these views: over two years on you evidently still think my work on a scheduler

thus hindered overall progress

Yes, because if we worked together, asynchronous programming for MicroPython would have been much better supported by now.

yet in the same post you say to someone else

So, if you have some idea, I whole-heartedly recommend: go and implement it

Yes, because I gave up and now take for granted that we (as a community in general) won't work in some specific direction to achieve some goal faster, but rather each party will pull in its own direction. (*) That's ok, because at least with Damien we worked in enough concord and over all this time, there's now a solid foundation and indeed it's time for multi-directional community to have formed around uPy.

(*) I mean, in "our own free time". Community-funded and maintainer-implemented Kickstarter shows how to reconcile that, and indeed, that works pretty well, but not an "everyday" solution.

If you find "baffling" the concept of "things (and views on them) change over time", try to read random matching post on the matter http://lucumr.pocoo.org/2016/11/5/be-careful-about-what-you-dislike (though I'm buffled at the need to explains such simple concepts at "things change over time").

I'm genuinely baffled by your evident continuing disapproval of my simple Python effort.

Same here, I'm baffled by your being buffled by your suspected my disapproval. On my side, I'm just reciting the facts. If anything you could find in my words, then an affirmation then you were right all this time (that's the entailment one can make from me saying that now everyone should do how you did it from the beginning, right?).

This can be completely implemented on object level, i.e. call on special comethod stores passed args and returns self, then next on it executes it.

I now have prototype of this implemented, in Python. To make this possible in Python, few issues in the core had to be tackled (yet need to be brushed up and pushed/submitted for review, as some are indeed call for deeper refactors, not workarounds).

But yep, I now have a concept which runs yield from uasyncio.sleep_ms(100) with zero memory allocation. And yep, those Python folks must be smart to have designed a framework which allows all that, except, as Armin Ronacher points put, that took a few iterations (which aren't finished yet!), and there's long trace of debris (i.e. effectively deprecated features) which can cloud the view of someone who starts to learn it.

In the meantime, propagated #2219 changes have been merged: in micropython-lib: uasyncio.core which should be able run on baremetal systems, uasyncio updated for it (still unix only), the reference example: https://github.com/micropython/micropython/blob/master/examples/hwapi/soft_pwm_uasyncio.py (tested on Linux machine module so far (Dragonboard))

bf31880 . This has visual artifacts on DragonBoard/Linux, but almost certainly not because Linux is not real-time enough for this, but because of:

A thought: millisecond-perfect tasks scheduling requires working with times on microsecond level, and in a smart way. E.g. there's no warranty that after sleep_ms(20) ticks_ms() will be 20 late (can be 21, on funny systems can be 19).

I now have prototype of this implemented, in Python.

Ok, here's implementation to serve as a reference for commits/PRs:

-def sleep_ms(ms):
-    yield ms
+#def sleep_ms(ms):
+#    yield ms
+
+class SleepMs(SysCall1):
+
+    def __init__(self):
+        self.v = None
+        self.arg = None
+
+    def __call__(self, arg):
+        self.v = arg
+        print("__call__")
+        return self
+
+    def __iter__(self):
+        print("__iter__")
+        return self
+
+    def __next__(self):
+        if self.v is not None:
+            print("__next__ syscall enter")
+            self.arg = self.v
+            self.v = None
+            return self
+        print("__next__ syscall exit")
+        _stop_iter.__traceback__ = None
+        raise _stop_iter
+
+_stop_iter = StopIteration()
+sleep_ms = SleepMs()

First optimization for above is went as c3d96d3 . It allows trick like:

_stop_iter = StopIteration()

_stop_iter.__traceback__ = None
raise _stop_iter

To raise exceptions without memory allocation by reusing pre-allocated exc instance. The only thing which needs to be done is clearing traceback before each re-raise.

@pfalcon

@pfalcon I have absolutely no interest in being considered "right", nor any desire to pursue this discussion. I'd have considered it closed two years ago were it not for your tendency periodically to disinter it in public forums such as this.

The facts from my point of view are as follows. My first contribution to MicroPython was an implementation of a conventional coroutine based scheduler in Python. It never occurred to me that this would be remotely contentious. For that I received criticism from yourself. Given that this was after I'd written it, it wasn't clear to me what I was expected actually to do. It still isn't. You mentioned collaborating on uasyncio which would have appealed were it not for your attitude to my effort which I found distinctly off-putting.

In point of fact, were it not for encouragement I received from project contributors, I'd have abandoned MicroPython. I came very close. If I'd realised at the time that you were one of the two maintainers I would without doubt have gone away. Perhaps I'm over sensitive but I prefer communities with a constructive rather than critical ethos.

Please can we agree to drop it? I make clear in my references to scheduling that uasyncio is the official, preferred solution. I'm not sure what more I can do to avoid causing provocation, which is the last thing I intend. I'll consider any reasonable requests from you if it would enable us to draw a line under this.

@chrismas9

You mentioned collaborating on uasyncio which would have appealed were it not for your attitude to my effort which I found distinctly off-putting.

I'm sorry.

I'd have abandoned MicroPython. I came very close.

Same here. End of last year, I was "wrapping up" things I was doing for MicroPython, prepared to strictly ration time I put into it going forward. Then suddenly Damien said then we finally can consider ESP8266 Kickstarter - I discussed it with him and also with @chrismas9 long before that, and came to conclusion it's not going anywhere and there's not enough momentum to keep it going forward. Since then, I'm just doing my job. Which also includes responding to people. I probably just should have shut up, but then people would say they're ignored, so it's catch 22 situation.

Please can we agree to drop it? I make clear in my references to scheduling that uasyncio is the official, preferred solution. I'm not sure what more I can do to avoid causing provocation, which is the last thing I intend. I'll consider any reasonable requests from you if it would enable us to draw a line under this.

But we did and there's no provocation, I don't understand why you treat it this way. And I don't have "requests for you" exactly for to not be suspected of causing any "provocation". I'd say that I'd appreciate your close attention to current development stride on uasyncio (to avoid "echo" effect like with umqtt, when you discovered some issues, but by that time I moved to next tasks, and can't address them immediately), in particular would appreciate your think out how to implement button handling in it (but in general manner). But I don't do that exactly by the reasons above.

Anyway, this is technical discussion. Some code has been posted already. Any interested parties are welcome to look into it.

uasyncio.core 1.1, implementing optimizations discussed above (alloc-free scheduling and delays) was released (ahead of micropython 1.8.7 which is required for it).

Ok, with basic event loop memoryalloc optimizations done, and polling memoryalloc-optimized API being finalized (#1550 (comment)), time to think about next things to optimize, and that's buffer handling. Proposed:

io.BytesIO(n) - preallocates buffer of given len
io.BytesIO.stealvalue() - Recasts buffer value as bytes, sets buffer to empty
io.copy(dst_stream, dst_stream, num_bytes) - copy number of bytes from one stream to another
io.copyuntil(dst_stream, dst_stream, sep="\n") - copy until separator

Alternatives: I once thought about optimizing BytesIO.getvalue() so "stealing" a value worked transparently: on .getvalue(), buffer contents are converted to a string, and ref to it is stored in io.BytesIO. Consecutive calls to .getvalue() will return this stored string. However, if a buffer-changing operation is called, the buffer is reinitialized, and string content is copied back to buffer first. Maybe it's worth implementing this instead of adding non-standard method.

io.copy/io.copyuntil are proposed in such way for generality. Their intended usecase is for io.BytesIO though, and they can be implemented as its methods. Such implementation will be simpler.

io.BytesIO(n) - preallocates buffer of given len

IMO this is a no-brainer, similar to bytes and bytearray constructor. Probably even upstream would agree to this change.

Alternatives: I once thought about optimizing BytesIO.getvalue() so "stealing" a value worked transparently ... Maybe it's worth implementing this instead of adding non-standard method.

Yes I would suggest to implement it like this. It would be a little bit more code, but then all uses of io.BytesIO that call getvalue() without modifying it afterwards would benefit. And doing it this way means there is less to document :)

io.copy/io.copyuntil are proposed in such way for generality. Their intended usecase is for io.BytesIO though, and they can be implemented as its methods.

io.BytesIO has a .getbuffer() method which returns a r/w memoryview of the underlying buffer. If this is implemented (very easy) then we can also implement general memcpy/memset/memcpyuntil/etc which have lots of use in other places. And memcpy/memset don't need a new function, they can be optimisations of the syntax dest[:n] = src[:n], or something like that.

io.BytesIO(n) - preallocates buffer of given len

Implemented in 50d3a9d

io.BytesIO has a .getbuffer()

This is slightly different (another) thing, not related to what I currently have in mind. The current task is to allow to efficiently (fragmentation-free) accumulate needed amount of information from non-blocking reads. Accumulate == grow, which memoryview as returned by .getbuffer() can't do. It may be useful at later time, when it comes to extending asyncio's API in uasyncio to return such a buffer, but that's one of the later stages.

Accumulate == grow

To do that without fragmentation means you'll need to preallocate the buffer. Then you can just do something like this to copy the streams:

buf = bytearray(100)
dest = io.BytesIO(1000)
while keep_going:
    n = src.readinto(buf)
    dest.write(memoryview(buf)[:n])

Ok, the current outcome of optimizing uasyncio during last half-year:

With picoweb built as frozen bytecode, while the picoweb reference application, https://github.com/pfalcon/notes-pico, run as non-frozen bytecode (it can't be frozen, because we don't support accessing resources in frozen modules, #1346), all of that on 32-bit unix, is able to run within 30KB of heap, with notes-pico code taking 7KB of it. This shows that the technology should be viable for platforms like esp8266, at least for simple apps, but requires further optimization and implementation of missing features (assuming the ultimate target is indeed writing webapps).

Alternatives: I once thought about optimizing BytesIO.getvalue() so "stealing" a value worked transparently: on .getvalue(), buffer contents are converted to a string, and ref to it is stored in io.BytesIO. Consecutive calls to .getvalue() will return this stored string.

This implementation was approached from different side in #3130 (so it paves the road to implement the above feature too).

@dpgeorge

Ok, optimizations done over these months, even if not ideas were fully implemented (yet), gave its fruit, notes-pico now runs on ESP8266. @dpgeorge, feel free to give it a try when you have a chance: pfalcon/notes-pico@5a76449

(Also posted on forum: https://forum.micropython.org/viewtopic.php?f=16&t=2966#p20150)

notes-pico now runs on ESP8266

I tried it and it does work but is quite flaky. It gave a TypeError: 'int' object is not callable error on first load (after loading static/js/notes.js) but continued and seemed ok. Then it gave a hard crash (Fatal exception 28(LoadProhibitedCause)) after I added 2 notes. Then next time I started it (after a hard reset) it wouldn't run and gave MemoryError. Then the third attempt it worked and seemed a bit more stable.

I saw all that during optimization work, when the app didn't run. When it finally started to run, it was "flawless" for me (thru whole 3 reflashing attempts with ~3 interactions per run).

TypeError: 'int' object is not callable

The likely cause is GC issues. E.g. such error happened when uasyncio on unix used epoll() and objects passed to it no longer hard references in upy heap.

Then it gave a hard crash (Fatal exception 28(LoadProhibitedCause))

Yep, we both have reports of this happening, and I rarely see such too. Anything can be involved - bugs somewhere, out assumption of too much stack (likely. there're less than 8k available), etc.

I would like to get back to this older comment: #2622 (comment)

Accumulate == grow

To do that without fragmentation means you'll need to preallocate the buffer. Then you can just do something like this to copy the streams:

buf = bytearray(100)
dest = io.BytesIO(1000)
while keep_going:
    n = src.readinto(buf)
    dest.write(memoryview(buf)[:n])

So, let's look what happens here, line by line. We'd like to get our data accumulated in dest, so let's start with it.

dest = io.BytesIO(1000)

Here, we allocate 1000 bytes.

buf = bytearray(100)

The we allocate 100 bytes more. Why? We already have 1000 allocated.

n = src.readinto(buf)

Here we read stuff into buf ...

dest.write(memoryview(buf)[:n])

... just to copy it to dest right away.

The whole idea of new methods proposed in #2622 (comment) is to avoid allocating and extra copying to "intermediate" buffers, if we already have final buffer allocated and ready to be written to.

However, in the latest iteration of thinking on it, instead of uio.copy(dst_stream, dst_stream, num_bytes), I consider to extend .readinto() and friends to accept BytesIO as the first argument. Doing so with the right semantics mostly fits with the existing buffer protocol, but requires a slight extension to it (specifically, to "finalize" a write operation to such a buffer).

uasyncio has moved on a lot since this was opened, and lots of optimisations made, so I'll close this issue.

Navigation Menu

Repository navigation

Ideas on optimizing uasyncio #2622

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ideas on optimizing uasyncio #2622

Description

Activity

roger- commented on Nov 12, 2016

pfalcon commented on Nov 12, 2016

roger- commented on Nov 12, 2016

pfalcon commented on Nov 12, 2016

pfalcon commented on Nov 12, 2016

mkarliner commented on Nov 12, 2016

Lysenkoism commented on Nov 12, 2016

peterhinch commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

pfalcon commented on Nov 13, 2016

peterhinch commented on Nov 14, 2016

2 remaining items

pfalcon commented on Nov 14, 2016

pfalcon commented on Dec 30, 2016

pfalcon commented on Jan 26, 2017

dpgeorge commented on Jan 31, 2017

pfalcon commented on Feb 1, 2017

pfalcon commented on Feb 1, 2017

dpgeorge commented on Feb 3, 2017

pfalcon commented on Apr 30, 2017

pfalcon commented on Jun 9, 2017

pfalcon commented on Jun 11, 2017

dpgeorge commented on Jun 13, 2017

pfalcon commented on Jun 13, 2017

pfalcon commented on Sep 30, 2018

dpgeorge commented on Jun 9, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions