functools.partial support

I haven't been able to find another issue for this, weird that this hasn't been reported already.

from typing import *
from functools import partial

def inc(a:int) -> int:
    return a+1

def foo(f: Callable[[], int]) -> int:
    return f()

foo(partial(inc, 1))

This will fail with

error: Argument 1 to "foo" has incompatible type partial[int]; expected Callable[[], int]

a similar code by using a dyadic add instead of inc will yield:

error: Argument 1 to "foo" has incompatible type partial[int]; expected Callable[[int], int]

(so, partial apparently doesn't currently retain any type information on the non-applied inputs... I guess this'll make the fix non trivial)

The error message is generated because a class with a __call__ method (in this case, partial) isn't considered a subtype of Callable, even though it should be. See #797.

Also, partial only retains information about the return type, since the type system can't represent a more precise type.

the type system can't represent a more precise type.

You mean that the partial class doesn't represent it at runtime, right?
I think mypy could special case partial, and coerce it to the appropriate Callable type transparently

I meant that mypy would have to special case partial, since we can't write a good enough stub using PEP 484 features only. Mypy already does some special casing, but for this I'd rather have a more general mypy plugin/extension system instead of even more ad-hoc special case logic in the type checker.

Maybe this could be supported by #3299

Yes, this looks like a good candidate for function plugins. Increasing priority since this is may be pretty easy to implement now. Just supporting positional arguments for partial should cover the majority of uses and would probably be good enough.

This particular case and probably other basic use cases are already supported by protocols (see tests in PR #3132), simply because partial has __call__ and therefore is a structural subtype of Callable.

Protocols won't help with inferring the argument types of the result, though?

Protocols won't help with inferring the argument types of the result, though?

Yes, I didn't do any special-casing, partial in typeshed is only generic in one type _T which is the original return type. Maybe with the new support for flexible Callable this might be improved, but probably a plugin will still be required if we want precise types.

@AlexWaygood

Another issue caused by this is that overloads don't work. It seems that the return type of partial(func, ...) is always the return type of the last @overload of func, regardless of arguments.

Not sure how TypeVars are supposed to work here, but it'd probably be safer to infer the type as Any or a Union of all @overloads, rather than taking an arbitrary return type.

My recent tests show that mypy is currently using the first @overload signature, rather than the last.

@AlexWaygood has reasonably judged #12675 to be a duplicate of the problem described in @Dreamsorcerer's comment. For anyone working on that problem: the now-closed issue (which I assume will continue to be available) contains some code to exercise the undesired behaviour.

I stumbled across this:

from typing import reveal_type
from functools import partial

class A:
    pass

p = partial(A)

reveal_type(p)  # Revealed type is "functools.partial[A]"
reveal_type(p.func)  # Revealed type is "def (*Any, **Any) -> A"
print(issubclass(p.func, A))
# error: Argument 1 to "issubclass" has incompatible type "Callable[..., A]"; expected "type"  [arg-type]

I don't know if this is considered a mypy bug or even related to partials, but the code runs fine, and issubclass returns True as expected.

I don't know if this is considered a mypy bug or even related to partials, but the code runs fine, and issubclass returns True as expected.

It's because in the typeshed, you have

class partial(Generic[_T]):
    func: Callable[..., _T]

So, the class is transformed into a callable on assignment. MyPy is doing the right thing here.

I see.
Is this the desired behavior? The typeshed does not seem to reflect what partial does at runtime in this case.

Is this the desired behavior?

Yes, assigning to a member of a class can lose type information. Just like tuple(list(('a', 'b'))) gives you tuple[str, ...] and not tuple[str, str].

This issue is about partial matching callable.

Should this be a separate issue?

A partial of a type is not recognised as a type:

def foo(a: Type[int], **kwargs):
    x = a(1)

foo(functools.partial(int, base=10))  # Argument 1 to "foo" has incompatible type "partial[int]"; expected "Type[int]"

Admittedly, this could also be not type-safe:

foo(functools.partial(int, 5))

Not sure if there's a better way to handle this. My actual use case is to accept any subclass of an abstract type, but some implementations may need positional arguments and some not, so I use partial to include the positional arguments needed before passing into the function.

@Dreamsorcerer

@Dreamsorcerer, I don't think your example above should type check. In other words, I think mypy is doing the right thing because partial[int] is not type compatible with type[int]. For example, if you attempt to pass a base keyword argument to partial[int], you will receive an error because base was already provided. You can make your code type safe by changing the definition of foo to the following, which type checks without error:

def foo(a: Callable[..., int], **kwargs): ...

Right, but then we lose all typing for arguments.

My actual use case is:

def init_app(session_storage: Type[aiohttp_session.AbstractStorage]):
    ...
    storage = session_storage(cookie_name="SESSION", max_age=3600, secure=True)

# First use case
init_app(aiohttp_session.SimpleCookieStorage)
# Second use case, which needs a positional key parameter
init_app(functools.partial(EncryptedCookieStorage, Fernet(config["fernet_key"]))

Would be nice not to lose typing on the kwargs (and I get an explicit Any error too).

Although, that's interesting that I can do init_app(EncryptedCookieStorage), which then type checks, but would fail at runtime...

So, actually, maybe the problem is that it allows a subclass of the abstract type to require an argument which the abstract class does not require...
https://github.com/aio-libs/aiohttp-session/blob/master/aiohttp_session/cookie_storage.py#L18

So, actually, maybe the problem is that it allows a subclass of the abstract type to require an argument which the abstract class does not require...

In Python, constructors don't obey LSP. That's why you shouldn't annotate the function with type[...]: there's no way to know how with which parameters to construct it. You should annotate with Callable.

Right, but then we lose all typing for arguments.

You can type the arguments by providing them to Callable or by creating a protocol.

Add support for functools.partial

Add support for functools.partial (#16939)

chore: since python/mypy#1484 has been addressed we can now use parti…

Navigation Menu

Repository navigation

functools.partial support #1484

19 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

functools.partial support #1484

Description

Activity

JukkaL commented on May 5, 2016

berdario commented on May 5, 2016

JukkaL commented on May 5, 2016

gvanrossum commented on May 5, 2017

JukkaL commented on Jun 13, 2017

ilevkivskyi commented on Jun 13, 2017

JukkaL commented on Jun 13, 2017

ilevkivskyi commented on Jun 13, 2017

19 remaining items

finite-state-machine commented on Apr 26, 2022

raphCode commented on Feb 15, 2023

NeilGirdhar commented on Feb 15, 2023

raphCode commented on Feb 16, 2023

NeilGirdhar commented on Feb 16, 2023

Dreamsorcerer commented on Feb 25, 2023

erictraut commented on Feb 25, 2023

Dreamsorcerer commented on Feb 25, 2023

Dreamsorcerer commented on Feb 25, 2023

NeilGirdhar commented on Feb 25, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions