Repository navigation
Infer optional type for list with None values #1863
Description
Activity
Maybe, but in most cases mypy just sees
Optional[x]asUnion[x, None], and unification is loath to create unions. So we'd have to make an exception for unification with NoneType. (That's fine, I'm just not surprised that it doesn't already work this way.)Yeah, I think it's reasonable to make an exception here.
- added this to the milestone
on Jul 14, 2016 Instead, I think we should be less loath to infer unions. If someone has
[1, "foo", 3]I think they're more likely to expect it to haveList[Union[int, str]]thanList[object].Inferring a union type for something like
[1, 'foo']is not very controversial, but it gets less clear if we consider all the possible cases where this might happen. Some examples below.First, sometimes mypy can already infer a non-
objectjoin. It's not clear if this should take precedence over a union or not:class Employee: pass class Manager(Employee): pass class Engineer(Employee): pass class Designer(Employee): pass workers = [Manager(), Engineer()] # List[Employee] or List[Union[Manager, Engineer]]? workers.append(Designer()) # Should this be okay?Second, if some of the types involved aren't simple instance types, the union may become tricky to work with, and it can be difficult to predict what the union actually looks like. Some examples below:
def f(y: MyClass = None) -> int: return 0 def g(x: int, z: int) -> int: return x + z a = [f, g] # Should the item type be a union of callables? b = [f, int] # Callable[[], int] or Union[something]? c = [(1, 2), [1, 2]] # List[Sequence[int]] or List[Union[Tuple[int, int], List[int]]]? d = [MyClass, MyClass()] # List[object] or Union[MyClass, Type[MyClass]]? e = [ClassA, ClassB] # Union[Type[ClassA], Type[ClassB]], Union[Type[ClassA, ClassB]], # some callable, or 'type'? x = [1] y = [''] f = [x, y] # List[Union[List[int], List[str]]?Third, it's not uncommon to have complex data structures involving nested lists, dictionaries, tuples and types/callables initialized at the top level of modules. If we'd infer unions aggressively, it's likely that for some of these we'd infer something silly looking like
Dict[str, List[Union[Dict[str, Union[....]...]. This might even pass type checking, but when there is an error, mypy would print out this monster type and it would be difficult to figure out what it actually means. Complex error messages are one of the more common complaints about languages like Haskell and C++, and I wouldn't like to mypy to join this group :-)Finally, an additional reason why mypy likes to fall back to
List[object]etc. is that these are pretty 'safe' fallback types, even if they aren't what the programmer expects, since almost any operation onobjectwill be rejected, so the bad types won't propagate very far and will be easily caught. If we are very aggressive about type inference, we can infer types that are technically correct but don't make much sense intuitively.One idea would be to only infer "simple" union types such as
Union[int, str]orOptional[int]but not union types involving callables, tuple types or generic types that only differ in values of type arguments. Also, perhaps instead of always falling back toList[object], require a type annotation as it's usually not the desired type (if we are inferring a variable definition). The exact rules would have to be much more involved, since there are various edge cases to consider. Some examples:a = [1, ''] # List[Union[int, str]] a = [1, None] # List[Optional[int]] b = [int, str] # Require annotation, as this is tricky x = [1] y = [''] d = [x, y] # Require annotation
This is unexpected:
I'd expect the list item type to to be
Optional[int]. A join of a non-optional type withNoneshould probably produce an optional type.