mypy not inferring that variable cannot be None after assignment from dict.get

Code

from typing import Dict, Optional

d: Dict[str, str] = {}

def foo(arg: Optional[str] = None) -> None:
    if arg is None:
        arg = d.get("a", "foo bar")
    print(arg.split())

Expected output

None, since the code is correctly typed (afaict, at least)

Actual output

with --strict-optional:

mvce.py:10: error: Item "None" of "Optional[str]" has no attribute "split"

Inserting reveal_type(arg) before the print gives Union[builtins.str, builtins.None].

The following typechecks fine, so mypy definitely understands the return type of dict.get with two arguments:

d: Dict[str, str] = {}

a: str = d.get("a", "a")

Similarly, mypy can infer that arg cannot be None after an is None and simple assignment:

d: Dict[str, str] = {}

def foo(arg: Optional[str] = None) -> None:
    if arg is None:
        arg = "foo bar"
    reveal_type(arg)  # str
    print(arg.split())

but combining the two fails.

Python 3.6.3
mypy 0.580

Good research! I can also make the error go away through a temporary variable:

a = d.get("a", "foo bar")
arg = a

I've got a feeling it's an bug in the type solver.

In my project, I have code like:

import typing


class Task:
    def __init__(self, name: typing.Union[None, str] = None) -> None:
        self.name = name
        if not self.name:
            self.name = "Komu"

    def caps(self) -> str:
        return self.name.upper()

when I run mypy on it I get:
error: Item "None" of "Optional[str]" has no attribute "upper"

I'm been forced to now right my code as:

class Task:

    def caps(self) -> str:
        # this assert is here to make mypy happy
        assert isinstance(self.name, str)
        return self.name.upper()

@komuw

@komuw That looks like a different problem. Can you open a new issue for it?

@komuw

@komuw This is unrelated to this issue, and not actually a bug, mypy can't guess your intention for the ("long living" and externally visible) .name. You should rewrite your code to make your intention explicit. For example:

class Task:
    def __init__(self, name: Union[None, str] = None) -> None:
        if name is not None:
            self.name = name
        else:
            self.name = "Komu"

Or even simpler (assuming this is the actual code you wanted to type):

class Task:
    def __init__(self, name: str = "Komu") -> None:
        self.name = name

@ilevkivskyi

@ilevkivskyi

thanks for the pointer, re-writing as suggested makes mypy happy and is what I'll result to in my project.
I'm just surprised that mypy does not recognise that this two snippets are equivalent, and it should apply the same rules to both;

self.name = name
if not self.name:
    self.name = "Komu"

if name is not None:
    self.name = name
else:
    self.name = "Komu"

Mypy fix (#28)

and it should apply the same rules to both

No, it shouldn't.

Within the method body they result in same types for self.name (you can check this with reveal_type()). But outside the method (and outside the class) mypy uses the first inferred type for instance variables. In this case it is Optional[str] in first snippet and str in the second one.

If we would infer str for instance variables this would cause errors in other places:

task = Task()
name: Optional[str]
task.name = name  # Should this be valid?

Since this is a backwards incompatible change with high potential for breaking currently passing code, there is no way this will be changed.

I'm having a similar issue, but with Callables.

def outer(func: Optional[Callable] = None) -> Callable:
    if func is None:
        func = lambda: None
    def inner(*args, **kwargs) -> Any:
        return func(*args, **kwargs)
    return inner

error: "None" not callable

As gvanrossum mentioned, the workaround of assigning it to a new variable makes the error go away.

def outer(
    func: Optional[Callable] = None,
) -> Callable:
    if func is None:
        func = lambda: None
    my_func = func
    def inner(*args, **kwargs) -> Any:
        '''Inner function'''
        return my_func(*args, **kwargs)
    return inner

No error

@leahein

@leahein Your issue is actually different, see #2608 (the workaround is however the same).

Add Type Ignore To Workaround The Bug Of Mypy

I've come across this issue (I believe).

Sharing a similar piece of code in case it helps in troubleshooting.

from typing import Optional

KUBERNETES_REGISTRIES_ALIASES = {
    'production': 'melchior-1.myorg.com',
    'staging': 'balthasar-2.myorg.com',
}

kubernetes_registry: Optional[str] = 'casper-3.myorg.com'  # from user input, may be None, an alias, or a full FQDN

if kubernetes_registry:
    reveal_type(kubernetes_registry)  # Revealed type is "builtins.str"
    kubernetes_registry = KUBERNETES_REGISTRIES_ALIASES.get(kubernetes_registry, kubernetes_registry)  # resolve alias (if it is an alias), otherwise use it as is
    reveal_type(kubernetes_registry)  # Revealed type is "Union[builtins.str, None]"

With mypy 0.910

Hello, any update on this? I've encountered a very similar problem with v0.910

The same thing happens if you guard against Nones with an exception:

def __init__(self, my_var_id: str, my_var: Optional[MyVar] = None):
    self._my_var = my_var
    if self._my_var is None:
        self._my_var = self._lookup_my_var(my_var_id) # Function that may retrieve a result i.e. from a DB
    if self._my_var is None:
        raise Exception("My var does not exist")

self._my_var will be considered Optional in the rest of the code and produce the Item "None" of "Optional[MyVar]" has no attribute "..." error.

In my case, I was certain that the instance is not None (although the library that I use defined it as Optional[TYPE]).
My solution was to use typing.cast to the underlying type.

# logger.parent is defined as Optional[logging.Logger]
prefect_logger = cast(logging.Logger, logger.parent)

One can argue if this is elegant or not, but this is a solution I'm happy with (it silenced the FA).

@shaybensasson

@shaybensasson this solution worked for me as well, when I knew that if I didn't return by a certain point my variable was guaranteed not to be None

@shaybensasson

@shaybensasson, rather than using a cast, you might consider using assert logger.parent is not None. This not only eliminates the type checking error, it also clearly documents your assumption in the code and provides runtime verification when running your program in debug mode.

Try empty context when assigning to union typed variables (#14151)

Navigation Menu

Repository navigation

mypy not inferring that variable cannot be None after assignment from dict.get #4805

Code

Expected output

Actual output

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

mypy not inferring that variable cannot be None after assignment from dict.get #4805

Description

Code

Expected output

Actual output

Activity

gvanrossum commented on Mar 27, 2018

komuw commented on Mar 11, 2019

JukkaL commented on Mar 11, 2019

ilevkivskyi commented on Mar 11, 2019

komuw commented on Mar 12, 2019

ilevkivskyi commented on Mar 12, 2019

leahein commented on Jul 30, 2019

ilevkivskyi commented on Aug 7, 2019

andresdelfino commented on Aug 12, 2021

xtuchyna commented on Oct 12, 2021

alextriaca commented on Dec 2, 2021

shaybensasson commented on Jan 7, 2022

shawalli commented on Jan 21, 2022

erictraut commented on Jan 21, 2022

pkit commented on May 7, 2023

pkit commented on May 7, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions