Skip to content

Plugin development: Use of get_dynamic_class_hook causes deserialization of types to fail from .mypy_cache #17410

Description

@jackson-at-arista

Hello folks. I'm working on a plugin that uses get_dynamic_class_hook to construct type constructors (TypeInfo) and class definitions (ClassDef) that don't exist in a module. I add these definitions to a MypyFile. Finally, I add this MypyFile to the modules dictionary provided by the SemanticAnalyzerPluginInterface API to convince mypy of my "fake" module.

This works wonderfully until a second run of my plugin when the .mypy_cache has been constructed. Before semantic analysis, mypy seems to be trying to read my fake module, because in the cache names in real modules have a target that refers to the fake module. For instance,

{
  ".class": "MypyFile",
  "_fullname": "RealModule",
   ...
  "names": {
    ".class": "SymbolTable",
    ...
    "SomeType": {
      ".class": "SymbolTableNode",
      "kind": "Gdef",
      "node": {
        ".class": "TypeAlias",
        ...
        "fullname": "RealModule.SomeType",
        ...
        "target": "FakeModule.SomeType"
      }
    },
    ...

The stacktrace is below. mypy fails to resolve FakeModule.SomeType because FakeModule doesn't exist in its modules dictionary. I don't think my use of get_dynamic_class_hook is unusual, so I think there's a hole in my understanding of this hook.

...
-> process_graph(graph, manager)
  /usr/lib/python3.9/site-packages/mypy/build.py(3333)process_graph()
-> process_fresh_modules(graph, prev_scc, manager)
  /usr/lib/python3.9/site-packages/mypy/build.py(3414)process_fresh_modules()
-> graph[id].fix_cross_refs()
  /usr/lib/python3.9/site-packages/mypy/build.py(2110)fix_cross_refs()
-> fixup_module(self.tree, self.manager.modules, self.options.use_fine_grained_cache)
  /usr/lib/python3.9/site-packages/mypy/fixup.py(52)fixup_module()
-> node_fixer.visit_symbol_table(tree.names, tree.fullname)
  /usr/lib/python3.9/site-packages/mypy/fixup.py(156)visit_symbol_table()
-> self.visit_type_info(value.node)
  /usr/lib/python3.9/site-packages/mypy/fixup.py(115)visit_type_info()
-> self.current_info = save_info
  /usr/lib/python3.9/site-packages/mypy/fixup.py(158)visit_symbol_table()
-> value.node.accept(self)
  /usr/lib/python3.9/site-packages/mypy/nodes.py(1037)accept()
-> return visitor.visit_var(self)
  /usr/lib/python3.9/site-packages/mypy/fixup.py(211)visit_var()
-> v.type.accept(self.type_fixer)
  /usr/lib/python3.9/site-packages/mypy/types.py(1444)accept()
-> return visitor.visit_instance(self)
  /usr/lib/python3.9/site-packages/mypy/fixup.py(230)visit_instance()
-> inst.type = lookup_fully_qualified_typeinfo(
  /usr/lib/python3.9/site-packages/mypy/fixup.py(367)lookup_fully_qualified_typeinfo()
-> stnode = lookup_fully_qualified(name, modules, raise_on_missing=not allow_missing)
  /usr/lib/python3.9/site-packages/mypy/lookup.py(31)lookup_fully_qualified()
-> assert "." in head, f"Cannot find module for {name}"

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions