Skip to content

Type substitutions in Callable ignore TypeVar bounds #8922

Description

@mthuurne

When I run mypy on the following code:

from typing import Callable, TypeVar

class Base:
    pass

class Sub(Base):
    pass

class Other:
    pass

BT = TypeVar('BT', bound=Base)

def check(obj: BT) -> None:
    pass

check(Base())
check(Sub())
check(Other())

Factory = Callable[[], BT]

def f() -> Other:
    pass

fAny:   Factory        = f
fBase:  Factory[Base]  = f
fSub:   Factory[Sub]   = f
fOther: Factory[Other] = f

It outputs:

19: error: Value of type variable "BT" of "check" cannot be "Other"  [type-var]
    check(Other())
    ^
27: error: Incompatible types in assignment (expression has type "Callable[[], Other]",
variable has type "Callable[[], Base]")  [assignment]
    fBase:  Factory[Base]  = f
                             ^
28: error: Incompatible types in assignment (expression has type "Callable[[], Other]",
variable has type "Callable[[], Sub]")  [assignment]
    fSub:   Factory[Sub]   = f
                             ^

All of the things mypy warns about are valid. But there are also statements it doesn't warn about that are incorrect code.

The most clear one is the type annotation of fOther (last line of the test case): while the function f does indeed return an object of type Other, that type gets substituted for BT in the definition of Factory and BT has Base as its bound, which is a type unrelated to Other.

A similar but more complex case can be made for fAny, which doesn't provide a type argument. However, there cannot exist a type argument that would satisfy both the bound of BT and the signature of f.

I have limited knowledge of the internals of mypy, but it seems feasible that the fOther case could be reported. If the fAny case can be reported as well that would be great, but I can imagine that determining that no valid type argument can exist is a lot harder than validating a given type argument.

I'm using mypy 0.770 on Python 3.8.2.

Activity

  1. JukkaL commented on Jun 8, 2020

    @JukkaL
    Collaborator

    When substituting values for type variables in generic type aliases, mypy doesn't check that the value is compatible with the bound. Here's a simplified repro:

    from typing import Callable, TypeVar, List
    
    N = TypeVar('N', bound='int')
    
    F = Callable[[], N]
    L = List[N]
    
    x: F[str]  # No error, but should be error
    y: L[str]  # No error, but should be error
  2. JukkaL commented on Jun 8, 2020

    @JukkaL
    Collaborator

    The fAny case is working as designed. If a type variable is omitted, it defaults to Any. It's pretty confusing, and I'd suggest using --disallow-any-generics.

  3. mthuurne commented on Jun 8, 2020

    @mthuurne
    ContributorAuthor

    The fAny case is working as designed. If a type variable is omitted, it defaults to Any.

    What does Any mean exactly? From the name, I would assume it means "any type": a unique anonymous TypeVar. But in how it is implemented, it seems to mean "we don't know anything about this type".

    In your simplified repro, y: L[T] implies:

    • isinstance(a, T) holds for every element a in y (property of List)
    • issubclass(T, int) holds (bound of N)

    Combining both implications, this means that for any type T that can be substituted for N, isinstance(a, int) holds for every element a in y.

    So even if we don't know which concrete type is passed to L, we do know that the elements of the list must all be integers. But this information is lost if L[Any] is evaluated as List[Any].

    Note that the fAny case isn't important to me as a programmer, but I think it's interesting from a theoretical point of view.

  4. mthuurne commented on Jun 9, 2020

    @mthuurne
    ContributorAuthor

    I opened a separate issue about the handling of Any: #8981, since that is not about checking the substitution, but about remembering the bound in the substitution result.

  5. BvB93 commented on Aug 18, 2020

    @BvB93
    Contributor

    The fAny case is working as designed. If a type variable is omitted, it defaults to Any.

    Should this be the case though?
    The fact that the bound parameter has been specified implies that the BT typevar cannot adopt any arbitrary value;
    yet defaulting to Any suggests the opposite.

    Would it not make more sense to default to whichever value is specified as bound?

  6. EdwardBlair commented on Aug 15, 2021

    @EdwardBlair

    When substituting values for type variables in generic type aliases, mypy doesn't check that the value is compatible with the bound. Here's a simplified repro

    Deserves separate issue.

  7. posita commented on Mar 6, 2022

    @posita
    Contributor

    Still happens in 0.931.

  8. ilevkivskyi commented on Feb 5, 2025

    @ilevkivskyi
    Member

    Mypy now checks for type variable bounds in type aliases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions