Repository navigation
Allow yielding result of function that returns None #1933
Description
Activity
To clarify: I do not think that
tcshould beAny. I had previously thought thattrshould beforall T. T, (which we can write asAny, thoughobjectwould probably be better), but after reading PEP 380 I need to think more about it.My thoughts on
tc:
None of the superclasses ofGeneratorhave asendmethod, sotccan never matter in an external-facing way (i.e. to anything outside of the generator function). Inside the generator function, we know that in all statements of the formc = yield ...,cwill have the valueNone(because no values can be sent into the generator). Therefore,tcshould have the typeNoneType. OrVoid, if you want to parallel the way None-returning functions work (which I think should also be changed, but that's a whole different can of worms). I thinktcworking in this way is uncontentious, modulo the choice ofNoneTypevsVoid.My thoughts on
tr:
After reading PEP 380, I realized you can writer = yield from some_iterator, which will leaverwith the valueNone. This means thatGenerators which are used asIterators had better also returnNone. This would mean that if I had aGenerator[int, None, str]I can't use it as anIterator[int], which perhaps is reasonable. I'm starting to feel a bit addled by my cold, so I'll finish thinking this through later...I haven't thought about this very carefully, but I'd rather generate errors on questionable code even if doesn't fail at runtime, as in this case the rejected code seems pointless and it's likely almost always an error. Some specifics below.
About
tc:Voidseems reasonable to me, assuming this results in a reasonable error message. If not,Nonesounds okay. If the return type isIteratororIterable, we can probably outright reject yield expressions.About
tr: If a function returnsIterableorIterator, I'd probably reject return statements with a value. AgainVoidorNoneseems reasonable, but special casing an error message would be reasonable if the error message doesn't look helpful otherwise.About
r = yield from some_iterator: Perhaps we can reject this if the expression is anIterableorIterator, and thus we'd sidestep the issue.About being able to use
Generator[int, None, str]as anIterator[int]: For consistency it seems like it should be allowed, and real code expectingyield fromto returnNonedoesn't seem like a common thing.Currently, the error for
x = yield yas well as that fora = yield from bisFunction does not return a valueif theyieldoryield fromis
seen as having no value (foryield ythis is when the containing
generator is declared as an Iterator or asGenerator[..., None, ...]; for
yield from bthis is when the called iterator (i.e.b) is declared as
Generator[..., ..., None](but not if it's an Iterator, because the
current code defaults to Any for rv).We can keep this issue open as a reminder to improve the error messages. I
can change the default return type to Void/None in phase 2 of my
async/await work (PR #1946).- added this to the milestone
on Aug 4, 2016 - removed this from the milestone
on Mar 29, 2017 Here is an update on the issue:
def f() -> None: pass def g() -> Iterator: yield from 1 # "yield from" can't be applied to "int", which is a good error message x = f() # "f" does not return a value, which is a reasonable error yield f() # "f" does not return a value, which is a false positive, since we may just want to give control to "f"
I am updating labels and title accordingly.
- changed the title
[-]Discussion about default generator parameter types[/-][+]Allow yielding result of function that returns None[/+]on May 17, 2018 I don't think the error on
yield f()is a false positive; it's basically the same asreturn f()wheref()returns None.I don't think the error on yield f() is a false positive; it's basically the same as return f() where f() returns None.
I think both are legitimate statements,
x = f()is prohibited because it's most likely an error, but we don't want users to write for example:def g(): ... f() yield None
also this error is confusing in case of
yield from f(), we should use the same as inyield from 1.- added a commit that references this issue
on Oct 30, 2024 - added a commit that references this issue
on Nov 14, 2024
UPDATE: see below for current status #1933 (comment)
This is a spin-off from PR #1808. @ddfisher and I cannot agree on what the default type should be in the three methods
get_generator_yield_type(),get_generator_receive_type(),get_generator_return_type().The setting: when the user defines a generator function, they can specify a return type. The return type is required to be a supertype of
Generator; this constrains it toIterator,Iterable,object(andAny, which is special in other ways).The
Generatortype has three type parameters:ty, the yield type: the type ofyinyield ytc, the receive type: the type ofcinc = yield ...tr, the return type: the type ofrinreturn rNote that
tris fairly new -- it was introduced in Python 3.3 by PEP 380 for the benefit ofyield from.tcis older, it dates back to Python 2.5 and PEP 342 (it is the value sent to the generator using the.send()method).tyis the oldest; whenyieldwas originally introduced in Python 2.2 by PEP 255, it was a statement. Before PEP 380,returnin a generator function was not allowed to return a value.When a generator function is just intended to be used as an iterator, there's no need to specify
tcandtr, and the recommended type isIterator[ty]. (Iterable[ty]is also allowed and means roughly the same thing.) In this case the user of the iterator has no access to the.send()method. Hence,yieldmight as well be considered a statement, since it's never going to return anything other thanNone. Likewise, there's no point is returning a value with areturnstatement since the user of the iterator in all likelihood is just going to iterate over it until it raisesStopIteration(since that's all that regular iterators do).This is where @ddfisher and I disagree -- I find it useful if the defaults for
tcandtrare None (or perhaps Void), so that mypy will flag an error in the generator body if you either try to useyield ...as an expression, or try to return a value usingreturn <expr>. David (IIUC) believes that both should beAny, giving the reason that "it should be allow to pointless but harmless things." (The code currently hastcdefault toVoidandtrdefault toAny. Go figure.)My problem with David's position is that I want mypy to be strict about pointless things that are likely the result of either a misunderstanding or a refactoring gone bad. I have seen enough people make the mistake of accidentally writing
return vinstead ofyield vin a generator that I want this to be flagged as an error. There's also the somewhat analogous case of a regular function declared to returnNone-- it would be pointless but harmless to return some value here, so maybe David believes that we shouldn't flag that as an error either?I'm not sure why the existing code (written by David) has
Voidas the default fortc, but I think it may be because this parameter ofGeneratoris contravariant. So hopefully we at least agree here. :-)David, please feel free to explain your position more fully (I think you were calling on the formal type system but we ran out of time in our in-person discussion at that point).