Repository navigation
RFC: Implementing generic generator protocol #402
Description
So, we now have complete implementation of generator protocol - but for generators themselves. Here's proposal how to extend it to other compatible objects.
So, let there be
mp_vm_return_kind_t mp_resume(mp_obj_t self_in,
mp_obj_t send_value, mp_obj_t throw_value, mp_obj_t *ret_val)
function which dispatch to mp_obj_gen_resume() if object is generator, to mp_iternext() if object supports C-level iterator protocol, and otherwise assume that it's ducktype generator and use dynamic method lookup.
Note that generator protocol is superset of iterator protocol, so the 2nd part of the proposal is to move ducktyping implementation from mp_iternext() to mp_resume(), and use the latter consistently in the interpreter.
Activity
pfalcon commentedon Mar 30, 2014
Ok, maybe "use mp_resume() consistently" is too much. mp_iternext() should just dispatch to it for ducktypes.
pfalcon commentedon Mar 30, 2014
First part is pushed in f39d3b9 . Didn't touch mp_iternext yet. MP_OBJ_NULL optimization keeps confusing me...
dpgeorge commentedon Mar 30, 2014
We really need this optimisation, as we don't want to raise StopIteration just to end a simple loop.
On the other hand, the end of a loop is not a frequently occuring thing (not as much as the actual iterations of the loop), and maybe it won't hurt performance that much if we do throw an exception...
Well, at least the API for mp_iternext should return MP_OBJ_NULL for those callers that don't want to have to wrap the call explicitly in an nlr block. Whether mp_iternext does a full optimisation, or just catches StopIteration shouldn't concern the caller.
dpgeorge commentedon Mar 30, 2014
MP_OBJ_NULL is just a shorthand for StopIteration(). I think there is now full support for throwing an actualy StopIteration exception, whether it has arguments or not.
pfalcon commentedon Mar 30, 2014
The current problem is that it's handled in many places and on many layers.
Nope, we surely need it. Partly confusion is still due to MP_OBJ_NULL - didn't adopt to it yet from mp_const_stop_iteration.
We now have mp_resume, and individual objects' iternext and resume, and interaction among all them is not much clear for me (I patched stuff intuitively to get current tests work). We now have 3 ways to return exception: nlr_push/nlr_jump, MP_VM_RETURN_EXCEPTION, and MP_VM_RETURN_NORMAL/MP_OBJ_NULL. I'm glad if you know which case to expect in particular place, I, again, dunno ;-). And we still miss ability to pass StopIteration value while optimizing. Anyway, I specifically wanted to land mp_resume before you would start implementing it. Now we at least entire functionality in place. So, if you're still interested in it, there the code ;-).
dpgeorge commentedon Mar 30, 2014
I'll need to spend some quality time understanding all your new code. Until then, I'll trust it :)
Merge pull request micropython#402 from tannewt/update_deps
dpgeorge commentedon Dec 18, 2019
Generic iterators for user types have been available for some time now. See eg
tests/basics/gen_yield_from_ducktype.py