Add .offset()/.seek() method for memoryview for inplace "slicing"

Here's an idiom to send a large data buffer thru a short-write stream (e.g., non-blocking socket):

        buffer = memoryview(data).cast('b')
        while buffer:
            try:
                nsent = self._socket_send(buffer, flags)
                buffer = buffer[nsent:]

The last statement still produces bunch of object garbage. If we could do

buffer.offset(nsent)

it would be cool. Suggestions for better name are welcome.

I had for a long time the desire for this kind of thing (see eg this comment which is 2 years old: https://github.com/micropython/micropython/blob/master/py/objarray.c#L217-L218)

The idea (eg for a sensor/lcd driver) would be that you create a buffer (bytearray) and a memoryview into this buffer, and then "slide" the memoryview offset/length around so that you can access any part of the buffer with no extra heap. You should be able to adjust the offset (forward and backward) and the length of the memoryview.

From a bigger-picture point of view, we really need to have a plan for reducing heap allocations / making things inplace. For example, reading/writing buffers to streams has 2 orthogonal paths:

Make stream methods take extra args for offset/length of the buffer. Eg we already have stream.readinto(buf, maxlen).
Make memoryview's cheap/free so they can be used to specify the offset and length. Eg stream.readinto(inpl_memoryview(buf, offset, len)), where the compiler allocates the inpl_memoryview object on the Python value stack.

Perhaps it would be possible to add to the slice notation similar magic as there is for the range in the for loop -- when the slicing is used directly as an argument to the stream functions, do it in place?

I'm not very familiar with how the vm is constructed, so this may be a ridiculous suggestion, but I will put it here anyways. A more general solution would be to introduce a special "ephemeral" kind of object, that would always get allocated on the value stack, and possibly re-allocated when assigned to a non-local variable or returned. The potential candidates for such objects are memoryview, range, slice, reversed, enumerate iterator, and maybe even things like the iterator returned by the dict.items().

You should be able to adjust the offset (forward and backward) and the length of the memoryview.

Makes sense to be able to adjust both offset and length. It would be still called .offset() I guess (like, offset from the end), because e.g. .slice() would give impression that a new object is allocated (and the sound confusing as there's native slicing []).

From a bigger-picture point of view, we really need to have a plan for reducing heap allocations / making things inplace.

Well, that work is happening for a while, since reviving of esp8266 port. But I don't think we need a big committee here, like for HW API, and can just do it step by step. This ticket in particular came from thinking how to make uasyncio viable for esp8266.

Make stream methods take extra args for offset/length of the buffer. Eg we already have stream.readinto(buf, maxlen).

We even have stream.write(data, offset=0, len=ALL), Which means .readinto should have offset added for consistency.

Make memoryview's cheap/free so they can be used to specify the offset and length. Eg stream.readinto(inpl_memoryview(buf, offset, len)), where the compiler allocates the inpl_memoryview object on the Python value stack.

That sounds far too magic for me, and I have no idea how that would work.

A more general solution would be to introduce a special "ephemeral" kind of object, that would always get allocated on the value stack, and possibly re-allocated when assigned to a non-local variable or returned.

Congratulations, you invented https://en.wikipedia.org/wiki/Escape_analysis. Except for "re-allocated when assigned to a non-local variable or returned", it also should be "when passed as an argument", which means in any real code it will almost never gives any opportunity when looking at a single function, and indeed, escape analysis is known to be global analysis.

From Wikipedia:
Escape analysis is implemented in Java Standard Edition 6.

In a project with a million budget and dozens if not hundreds people working on it, which crashes on gigabyte servers due to out of memory issues.

it also should be "when passed as an argument"

Well, that would defeat the whole purpose we are discussing here, wouldn't it?

From Wikipedia:
Escape analysis is implemented in Java Standard Edition 6.

In a project with a million budget and dozens if not hundreds people working on it, which crashes on gigabyte servers due to out of memory issues.

This is a non-sequitur. They also use a garbage collector. Does that mean that we shouldn't?

@dpgeorge

@dpgeorge , btw, with the advent of mpy-cross, which is guaranteed to work on the host, do you want to look at more heavy optimizations? ;-) Function inlining might be the lowest-hanging and biggest-boon one.

(No, you don't want, we should work on esp8266 port!)

This is a non-sequitur. They also use a garbage collector. Does that mean that we shouldn't?

No, that means that we also should have a million budget, like they.

No, that means that we also should have a million budget, like they.

That's an excellent idea! Let's do it!

btw, with the advent of mpy-cross, which is guaranteed to work on the host, do you want to look at more heavy optimizations? ;-) Function inlining might be the lowest-hanging and biggest-boon one.

Yes I thought about it a lot. Also things like new VM opcodes such as CALL_SLEEP_US which is emitted for code X.sleep_us(Y) and has a fast-path to check that X is indeed the time module and Y is an integer; you get the picture :) [This particular case is relevant because VM opcode dispatch is O(us) so for accurate timing would be nice to be optimised.] But I have no time for such things right now.

I admit that I have very little knowledge of the inner workings of the parser and VM (working on it now), but perhaps we could think about a way to make this work efficiently using standard Python constructs, before introducing completely incompatible new things and effectively inventing a new language?

Following up on this, I'm finding the need to also change the underlying buffer of a memoryview. The use-case is pretty much the same as in the OP here, and the idea is to be able to write a function that processes a buffer in chunks without allocating anything on the heap.

The idea is you would pre-create a memoryview object, cache it, then use it to slice up the buffer that you're processing. Eg:

mv = memoryview(b'')

# this function uses no heap at all
def process(buf):
    offset = 0
    while offset < len(buf):
        mv.reset(buf, offset) # reset the memoryview to point to buf[offset:]
        offset += stream.write(mv)

Could it be implemented as a function in the micropython module, so that the same code could run on standard Python, by providing an implementation emulating this (even if not very efficient)?

Could it be implemented as a function in the micropython module, so that the same code could run on standard Python,

If we did that it'd need to be something like mv = micropython.reset_memoryview(mv, buf, offset). You'd need to reassign to the "mv" variable for the case that the function creates a new memoryview object.

I think it would be neater to provide a custom memoryview object if you want to use CPython, eg:

class CustomMemoryview:
    ...

__builtins__.memoryview = CustomMemoryview

This is maybe slightly off-topic, but what is preferred way to pre-allocate an empty bytearray with given capacity? This works:

b = bytearray(100)
b[:] = bytes()

it seems a bit hackish and dependent on allocation policy that doesn't truncate the buffer after subscr
i'm not sure if it is really allocation free
maybe its more suited for something like BytesIO?

Thank you!

To assign data to a subset of a pre-allocated buffer I'd suggest a memoryview:

>>> a = bytearray(100)
>>> b = memoryview(a)
>>> b[0:4] = bytes((1,2,3,4))
>>> a[0:10}
bytearray(b'\x01\x02\x03\x04\x00\x00\x00\x00\x00\x00')
>>>

This assigns without truncating or (as I understand it) allocating.
This type of general query really belongs in the forum.

memoryview is really useful most of the time, but I was thinking about something with a more stream-like interface, with extend() and append() methods. For example, in trezor-core, we pass pre-allocated bytearrays to serialization functions when we need to buffer the response, and a bytearray-like wrapper around sha256 when we're just computing a hash. Thats the reason for my remark about BytesIO -- this really seems like a stream concept, not a bytearray/buffer concept.

Thats the reason for my remark about BytesIO -- this really seems like a stream concept, not a bytearray/buffer concept.

And that's indeed how it evolves, uio.BytesIO is a generic buffer concept, a goto for various needs. (I from time to time get back-thinking about need for different, optimized buffer types, but of course that would require strong reasons to actually introduce them). For example, #3382 adds readbin()/writebin() method exactly to BytesIO, because that's where it's most useful (and not to other stream types so far, due to #1159).

Following up on this, I'm finding the need to also change the underlying buffer of a memoryview.

Generalization of this idea is posted as #4331

Merge pull request micropython#2552 from jepler/samd-stereo-single-dma

Navigation Menu

Repository navigation

Add .offset()/.seek() method for memoryview for inplace "slicing" #2552

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add .offset()/.seek() method for memoryview for inplace "slicing" #2552

Description

Activity

dpgeorge commented on Oct 24, 2016

deshipu commented on Oct 24, 2016

deshipu commented on Oct 24, 2016

pfalcon commented on Oct 24, 2016

pfalcon commented on Oct 24, 2016

deshipu commented on Oct 24, 2016

pfalcon commented on Oct 24, 2016

pfalcon commented on Oct 24, 2016

deshipu commented on Oct 24, 2016

dpgeorge commented on Oct 25, 2016

deshipu commented on Oct 29, 2016

dpgeorge commented on Nov 24, 2016

deshipu commented on Nov 24, 2016

dpgeorge commented on Nov 25, 2016

jpochyla commented on Nov 25, 2016

peterhinch commented on Nov 25, 2016

jpochyla commented on Nov 25, 2016

pfalcon commented on Dec 1, 2018

pfalcon commented on Dec 1, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions