Skip to content

Add support for common collections.abc types - #25798

Merged
jrmccluskey merged 5 commits into
apache:masterfrom
jrmccluskey:collectionsAgency
Mar 14, 2023
Merged

Add support for common collections.abc types#25798
jrmccluskey merged 5 commits into
apache:masterfrom
jrmccluskey:collectionsAgency

Conversation

@jrmccluskey

@jrmccluskey jrmccluskey commented Mar 10, 2023

Copy link
Copy Markdown
Contributor

Adds support for the collections.abc module's Iterable, Iterator, and Generator containers as type hints. The typing module equivalents had previously been supported but are deprecated as of Python 3.9, so support for the new recommended container types was necessary.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI.

@jrmccluskey

jrmccluskey commented Mar 10, 2023

Copy link
Copy Markdown
Contributor Author

R: @tvalentyn

@github-actions

github-actions Bot commented Mar 10, 2023

Copy link
Copy Markdown
Contributor

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

@tvalentyn

tvalentyn commented Mar 10, 2023

Copy link
Copy Markdown
Contributor

PTAL at test failures.

>         self.assertEqual(converted_beam_type, expected_beam_type, description)
E         AssertionError: collections.abc.Iterable[int] != Iterable[<class 'int'>] : collection iterable

@jrmccluskey

jrmccluskey commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

Looks like the mechanic I was leveraging to check if the type was an iterable was new behavior in 3.11, not older versions. The new routing of checking for an __iter__ attribute should fix that.

@codecov

codecov Bot commented Mar 13, 2023 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #25798 (1d1a36c) into master (f44c553) will decrease coverage by 0.10%.
The diff coverage is 20.00%.

@@            Coverage Diff             @@
##           master   #25798      +/-   ##
==========================================
- Coverage   72.81%   72.71%   -0.10%     
==========================================
  Files         775      777       +2     
  Lines      102928   103078     +150     
==========================================
+ Hits        74946    74958      +12     
- Misses      26527    26665     +138     
  Partials     1455     1455
Flag Coverage Δ
python 81.80% <20.00%> (-0.16%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...apache_beam/typehints/native_type_compatibility.py 79.19% <20.00%> (-3.64%) ⬇️

... and 11 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@jrmccluskey

jrmccluskey commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

TIL our python codecov is not running on 3.9 or 3.10. Interesting.

@jrmccluskey

jrmccluskey commented Mar 13, 2023

Copy link
Copy Markdown
Contributor Author

Run Python_Integration PreCommit

@tvalentyn

tvalentyn commented Mar 13, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

TIL our python codecov is not running on 3.9 or 3.10. Interesting.

Yes, some tests only run on one of the Python versions, typically the lowest supported by Beam.

tvalentyn
tvalentyn approved these changes Mar 13, 2023
View reviewed changes
Comment thread sdks/python/apache_beam/typehints/native_type_compatibility.py Outdated
@tvalentyn

tvalentyn commented Mar 13, 2023

Copy link
Copy Markdown
Contributor

Seeing some segfaults in Python 3.10... is this related to this PR?

@jrmccluskey

jrmccluskey commented Mar 14, 2023

Copy link
Copy Markdown
Contributor Author

The MacOS 3.10 unit test run appears to be segfaulting at head, not this PR

@jrmccluskey
jrmccluskey merged commit 81517a7 into apache:master Mar 14, 2023
@jrmccluskey
jrmccluskey deleted the collectionsAgency branch March 14, 2023 15:37
@jrmccluskey jrmccluskey mentioned this pull request Mar 16, 2023
Closed
@jrmccluskey jrmccluskey mentioned this pull request Aug 30, 2023
Closed
1 of 15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tvalentyn tvalentyn tvalentyn approved these changes

Assignees

No one assigned

Labels

python

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jrmccluskey @tvalentyn