Skip to content

Move Python formatting to a supported formatter #158384

Open
Open
Move Python formatting to a supported formatter#158384
Labels
P2Important issues not at the top of the work listc: contributor-productivityTeam-specific productivity, code health, technical debt.c: tech-debtTechnical debt, code quality, testing, etc.engineflutter/engine related. See also e: labels.team-engineOwned by Engine teamtriaged-engineTriaged by Engine team

Description

@cbracken
cbracken
opened on Nov 8, 2024

We currently format all Python code using yapf. This is done via the tools/yapf.sh and tools/yapf.bat wrapper scripts.

Unfortunately, yapf is abandoned and only works with Python versions up to and including Python 3.10. As OS and local Python installations are upgraded, this will become more and more problematic. On some Linux distributions side-by-side python installations are unsupported, for example.

Workarounds

Those on macOS can work around this by installing homebrew then:

brew install [email protected]

Related

Activity

  1. cbracken
    added
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    engineflutter/engine related. See also e: labels.
    P2Important issues not at the top of the work list
    c: tech-debtTechnical debt, code quality, testing, etc.
    on Nov 8, 2024
  2. auto-submit
    added a commit that references this issue on Nov 8, 2024

    yapf: Add more detailed error message and TODO (#56458)

    6fd55a6
  3. huycozy
    added
    team-engineOwned by Engine team
    on Nov 11, 2024
  4. jonahwilliams
    added
    triaged-engineTriaged by Engine team
    on Nov 11, 2024

  5. zanderso commented on Nov 11, 2024

    @zanderso
    zanderso
    on Nov 11, 2024
    Member

    yapf was only chosen because it was already used for LUCI recipes, and we already had a mirror set up for it. One thing to investigate would be whether pylint has options to also enforce formatting, which we might currently have disabled so that yapf and pylint don't fight with eachother.

  6. matanlurey commented on Nov 12, 2024

    @matanlurey
    matanlurey
    on Nov 12, 2024
    Contributor

    It looks like the Python Software Foundation currently supports black: The uncompromising Python code formatter.

    We would need to (a) import it via a mirror, (b) change our internal tooling to support it, and (c) potentially reformat the repo.

    If this is something we want to do, I can look into getting started with (a).

  7. cbracken commented on Nov 13, 2024

    @cbracken
    cbracken
    on Nov 13, 2024
    MemberAuthor

    I think we should do something about this.

    Working on Linux is frustrating since the latest Ubuntu/Debian LTS releases no longer offer the option to install older python3 versions compatible with our version check.

    Not to kick the can further down the road, but I was able to successfully run the check using python3.13 with the 2to3 package installed separately. It seems like we might be able to widen the version check further if we needed to, but this isn't sustainable long-term.

    @matanlurey I can do a trial run later this week sometime to see how black looks locally on my machine. If it looks like it'd be a reasonable fit, I can report back then and look at wiring up the mirroring unless anyone has any objection.

  8. cbracken
    self-assigned this
    on Nov 13, 2024

  9. matanlurey commented on Nov 13, 2024

    @matanlurey
    matanlurey
    on Nov 13, 2024
    Contributor

    That would amazing, thanks Chris!

  10. jmagman commented on Nov 13, 2024

    @jmagman
    jmagman
    on Nov 13, 2024
    Member
    cipd ls -r | grep black

    There are some existing black cipd pacakges.

  11. gaaclarke commented on Nov 18, 2024

    @gaaclarke
    gaaclarke
    on Nov 18, 2024
    Member

    If we move to black what are we going to do about its dependencies? It's using pyproject.toml to specify them. Are they baked into the cipd packages?

    edit: for example, hatchling. This seems to be a build dependency, so maybe it doesn't matter?

  12. nick9822
    added a commit that references this issue on Dec 18, 2024

    yapf: Add more detailed error message and TODO (flutter-team-archive/…

    cff2f44
  13. cbracken
    removed their assignment
    on Feb 26, 2025
  14. cbracken
    added a commit that references this issue on Apr 4, 2025

    yapf: Allow python 3.13 in yapf

    8d2b5f4
  15. github-merge-queue
    added a commit that references this issue on Jun 10, 2025

    Verify old version of Python has the `lib2to3` import available (#170187

    db8a64f

  16. cbracken commented on Jun 20, 2025

    @cbracken
    cbracken
    on Jun 20, 2025
    MemberAuthor

    There have been a few patches to get us as far as Python 3.12, which still ships with the python 2to3 library that yapf relies on. 2to3 no longer ships with Python 3.13 so we need to get on this now.

  17. jtmcdole commented on Jan 8, 2026

    @jtmcdole
    jtmcdole
    on Jan 8, 2026
    Member

    Its 2026. We're still using yapf. The best way to work around this for now (in my opinion) is pyenv:

    pyenv install 3.12.12
cd engine/src
pyenv local 3.12.12
  18. harryterkelsen
    added a commit that references this issue on Feb 26, 2026

    Migrate Python formatter from yapf to black (flutter#158384)

    a7679cb
  19. Nakiha
    added a commit that references this issue on Sep 28, 2026

    Verify old version of Python has the `lib2to3` import available (#170…

    69372cf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work listc: contributor-productivityTeam-specific productivity, code health, technical debt.c: tech-debtTechnical debt, code quality, testing, etc.engineflutter/engine related. See also e: labels.team-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions