Skip to content

Special-case value-expression inference of special form subscriptions - #16877

Merged
AlexWaygood merged 1 commit into
mainfrom
alex/subscript-inference
Mar 20, 2025
Merged

Special-case value-expression inference of special form subscriptions#16877
AlexWaygood merged 1 commit into
mainfrom
alex/subscript-inference

Conversation

@AlexWaygood

@AlexWaygood AlexWaygood commented Mar 20, 2025

Copy link
Copy Markdown
Member

Summary

Currently for something like X = typing.Tuple[str, str], we infer the value of X as object. That's because Tuple (like many of the symbols in the typing module) is annotated as a _SpecialForm instance in typeshed's stubs:

ruff/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

Line 215 in 23382f5

Tuple: _SpecialForm

and we don't understand implicit type aliases yet, and the stub for _SpecialForm.__getitem__ says it always returns object:

ruff/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

Lines 198 to 200 in 23382f5

@final
class _SpecialForm(_Final):
def __getitem__(self, parameters: Any) -> object: ...

We have existing false positives in our test suite due to this:

ruff/crates/red_knot_python_semantic/resources/mdtest/annotations/annotated.md

Lines 76 to 78 in 23382f5

# TODO: False positive
# error: [invalid-base]
class C(Annotated[int, "foo"]): ...

and it's causing many new false positives in #16872, which tries to make our annotation-expression parsing stricter in some ways.

This PR therefore adds some small special casing for KnownInstanceType variants that fallback to _SpecialForm, so that these false positives can be avoided.

Test Plan

Existing mdtest altered.

Cc. @MatthewMckee4

@AlexWaygood AlexWaygood added the ty The ty type checker label Mar 20, 2025
@AlexWaygood
AlexWaygood force-pushed the alex/subscript-inference branch from f2b6891 to 37084f0 Compare March 20, 2025 17:41
@github-actions

github-actions Bot commented Mar 20, 2025

Copy link
Copy Markdown
Contributor

mypy_primer results

Changes were detected when running on open source projects 
black (https://github.com/psf/black)
- error[lint:non-subscriptable] /tmp/mypy_primer/projects/black/src/black/trans.py:54:11: Cannot subscript object of type `object` with no `__getitem__` method
- Found 289 diagnostics
+ Found 288 diagnostics

@AlexWaygood

AlexWaygood commented Mar 20, 2025 •
edited
Loading

Copy link
Copy Markdown
Member Author

mypy_primer results

Changes were detected when running on open source projects

black (https://github.com/psf/black)
- error[lint:non-subscriptable] /tmp/mypy_primer/projects/black/src/black/trans.py:54:11: Cannot subscript object of type `object` with no `__getitem__` method
- Found 289 diagnostics
+ Found 288 diagnostics

Here's the link to the line in question: https://github.com/psf/black/blob/dd278cb316d75868716a0478c35b1fcd600a5249/src/black/trans.py#L54

Result is a type alias to a Union[] subscription, and Union is stubbed as an instance of _SpecialForm in typeshed:

https://github.com/psf/black/blob/dd278cb316d75868716a0478c35b1fcd600a5249/src/black/rusty.py#L28

So it's good that we no longer emit this error!

@AlexWaygood
AlexWaygood merged commit 296d67a into main Mar 20, 2025
@AlexWaygood
AlexWaygood deleted the alex/subscript-inference branch March 20, 2025 21:46
dcreager added a commit that referenced this pull request Mar 21, 2025
@dcreager
Merge branch 'main' into dcreager/two-phase-binding
01a5b3a 
* main: (26 commits)
  Use the common `OperatorPrecedence` for the parser (#16747)
  [red-knot] Check subtype relation between callable types (#16804)
  [red-knot] Check whether two callable types are equivalent (#16698)
  [red-knot] Ban most `Type::Instance` types in type expressions (#16872)
  Special-case value-expression inference of special form subscriptions (#16877)
  [syntax-errors] Fix star annotation before Python 3.11 (#16878)
  Recognize `SyntaxError:` as an error code for ecosystem checks (#16879)
  [red-knot] add test cases result in false positive errors (#16856)
  Bump 0.11.1 (#16871)
  Allow discovery of venv in VIRTUAL_ENV env variable (#16853)
  Split git pathspecs in change determination onto separate lines (#16869)
  Use the correct base commit for change determination (#16857)
  Separate `BitXorOr` into `BitXor` and `BitOr` precedence (#16844)
  Server: Allow `FixAll` action in presence of version-specific syntax errors (#16848)
  [`refurb`] Fix starred expressions fix (`FURB161`) (#16550)
  [`flake8-executable`] Add pytest and uv run to help message for `shebang-missing-python` (`EXE003`) (#16855)
  Show more precise messages in invalid type expressions (#16850)
  [`flake8-executables`] Allow `uv run` in shebang line for `shebang-missing-python` (`EXE003`) (#16849)
  Add `--exit-non-zero-on-format` (#16009)
  [red-knot] Ban list literals in most contexts in type expressions (#16847)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm approved these changes
@sharkdp sharkdp Awaiting requested review from sharkdp
@dcreager dcreager Awaiting requested review from dcreager

Assignees

No one assigned

Labels

ty The ty type checker

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexWaygood @carljm