Dynamic base classes are not handled

repro:

set some $BASEDIR for files
git clone https://github.com/python/typeshed.git $BASEDIR/typeshed
cd $BASEDIR/typeshed; git checkout f36d7a3; rm -rf third_party/2/sqlalchemy/ (note: sqlalchemy stubs are incomplete, so ignore them)
put the following code at $BASEDIR/failure.py

from sqlalchemy import Column, String
Base = declarative_base()

class Foo(Base):
  __tablename__ = 'foo'
  key = Column(String, primary_key=True)

install mypy 0.4.6
run mypy --py2 -v -v -v --fast-parser -s --custom-typeshed-dir $BASEDIR/typeshed $BASEDIR/failure.py

Output is something like

failure.py:33: error: Invalid type "failure.Base"
failure.py:33: error: Invalid base class

when it should be clean

Simpler repro (without the typeshed/sqlalchemy stuff):

def func(): pass
Base = func()
class C(Base): pass  # errors on this line

Errors (mypy -s x.py):

x.py:3: error: Invalid type "x.Base"
x.py:3: error: Invalid base class

(Update: used def func... instead if import.)

The reason this doesn't work is that the type of Base is not yet inferred during semantic analysis, and that's when we process base classes. You can add an annotation as a workaround:

from typing import Any

def func(): pass
Base = func()  # type: Any
class C(Base): pass  # now works

Should we close this then or update the docs somewhere? Maybe the error message could also be better.

I think that we should at least see if we can make the error message better.

improved error message would be helpful. I did discover # type: Any as a workaround but it is good to be aware this is a known issue.

Is there a github issue I should be tracking for the semantic analysis improvement, or just watch the release notes?

Let's use this issue to track both the error message and the underlying semantic analysis issue for now.

I seem to have hit a very similar bug that's a bit more convoluted (involving Optional and importing), but probably has a related base cause. It happened to me originally with sqlalchemy's declarative_base() as well, but this was my attempt to find a simple reproduction (Python 3.6):

file1.py:

from typing import Any

Base: Any

file2.py:

from typing import Optional
from .file1 import Base

class SubClass(Base):
    pass

def optional_return_func() -> Optional[SubClass]:
    return SubClass()

def should_fail() -> int:
    test_var = optional_return_func()
    reveal_type(test_var)

    if not test_var:
        return 1

    return 'not an int'

When I run mypy on this, the result is error: Revealed type is 'builtins.None'

The expected result should be Revealed type is 'Union[file2.SubClass, builtins.None]' along with an Incompatible return value type error, but since mypy is certain that test_var is None, it considers the final line that returns a string unreachable.

Hope that makes sense, let me know if any more information would be helpful.

@JukkaL

@JukkaL sorry for the ping, but I just wanted to check and see if you think the issue I described above is similar enough that leaving it as part of this one is good, or if I should open a new, separate issue for it.

@Deimos

@Deimos The builtins.None issue seems to be fixed on github master -- can you verify that it works for you? It's unrelated to the original issue, but no need to create an issue if it already works.

Oh, yes, it does seem to be fixed using current github master. That's great, thanks (and sorry for bothering you about something that was already fixed).

But if I add a Any annotation the problem is that it thinks the class can be anything, so doing stuff like

import typing as t
base: t.Any = make_base()
class A(base): pass

def make_list() -> t.Sequence[A]:
    return A()

does not yield an error which makes anything using this class (A in this case) not typesafe anymore.

But what would you expect? Except for a few trivial edge cases, when the base class is computed dynamically, mypy is unable to understand the structure of your class. That's why it gives an error when you use a dynamic base class -- if you shut it up using Any, literally "Any"thing can happen. :-)

[I changed the issue topic because it has nothing to do with sqlalchemy.]

Correct Mypy "invalid base class" error

If mypy can't parse it, it is unhelpful to say it's a type error

In case others are interested in the Django type checking issues I mentioned here and they find this thread, I'd like to point to this plugin for mypy: https://github.com/mkurnikov/django-stubs I haven't started using it yet, but it should solve a lot of Django type checking issues.

@dms-cat
Is this inside of a function?

Note the workaround from Guido above does not work when wrapped in a function in general:

def foo():
    from typing import TYPE_CHECKING
    def make_base(): pass
    if TYPE_CHECKING:
      base = object
    else:
      base = make_base()
    class C(base):  # !! Variable "base" is not valid as a type
      def foo(self): pass
    c = C()
    c.foo()
    c.bar()  # Error again

Simpler:

if True:  # works
    base = object

    class C(base):
        pass


def f():
    base = object

    class C(base):  # Variable "base" is not valid as a type / Invalid base class "base"
        pass

(0.750+dev.a94e649de794ecd169a5bcc42274e5205ffbb1fb)

bot: Split into multiple compartmentalized ABC mixins

For anyone who comes across this via the original report, a simple way to address sqlalchemy/mypy issues is to also import DeclarativeMeta.

from sqalalchemy.ext.declarative import declarative_base, DeclarativeMeta
Base: DeclarativeMeta = declarative_base()
...

@micahjsmith

Thanks @micahjsmith, just ran into this for the first time (on a 2-yo project, no clue why it was never reporting before). Base: Declarative = ... is way better than Base = ... # type: ignore

abap: fix mypy typing error

Mypy workaround

Navigation Menu

Repository navigation

Dynamic base classes are not handled #2477

20 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Dynamic base classes are not handled #2477

Description

Activity

gvanrossum commented on Nov 24, 2016

JukkaL commented on Nov 24, 2016

gvanrossum commented on Nov 24, 2016

JukkaL commented on Nov 24, 2016

snakescott commented on Nov 24, 2016

JukkaL commented on Jan 16, 2017

Deimos commented on Jan 19, 2017

Deimos commented on Jan 30, 2017

JukkaL commented on Jan 31, 2017

Deimos commented on Jan 31, 2017

libre-man commented on Jul 9, 2017

gvanrossum commented on Jul 10, 2017

20 remaining items

revmischa commented on Jun 18, 2019

w0rp commented on Jun 18, 2019

blueyed commented on Nov 7, 2019

micahjsmith commented on Oct 3, 2020

kevlarr commented on Dec 9, 2020

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions