Skip to content

Test performance has regressed #31511

Closed
Closed

Description

@Hixie

Caused by #31207

cc @chunhtai

Activity

  1. added this to the Scheduled customer work milestone on Apr 23, 2019
  2. jonahwilliams commented on Apr 23, 2019

    @jonahwilliams
    Contributor

    We can try applying a Fingerprinter to the assets and skip the build if that passes.

    As a last resort (though I ad suggested not to do this in part because of the complexity/duplication), we can teach the flutter_tester how to look up each asset from disk directly. The flutter_tool and flutter_test cannot share code however, so the logic would need to be duplicated and tested heavily so that it was kept in sync.

  3. Hixie commented on Apr 23, 2019

    @Hixie
    ContributorAuthor

    We could also default to off, since most tests don't need this.

  4. jonahwilliams commented on Apr 23, 2019

    @jonahwilliams
    Contributor

    since most tests don't need this.

    I think there is some survivorship bias there, since assets haven't ever been supported in tests.

  5. chunhtai commented on Apr 23, 2019

    @chunhtai
    Contributor

    A lot of test pass because the assets never load. The test will proceed with half rendered widget

    It might cause flakiness without this patch. I will try to come up with a better solution

  6. chunhtai commented on Apr 26, 2019

    @chunhtai
    Contributor

    Merged, let's see if this fixes it

  7. Hixie commented on Apr 27, 2019

    @Hixie
    ContributorAuthor

    Doesn't seem to have helped, but it's hard to tell for sure.

  8. jonahwilliams commented on Apr 27, 2019

    @jonahwilliams
    Contributor

    We should disable the flag by default to make the work opt-in. I'm concerned this is going to hurt discoverability of the feature - which is incredibly useful for real applications.

    Also If we can switch back to the app-jit snapshots we'll get back the lost performance and then some.

  9. chunhtai commented on Apr 29, 2019

    @chunhtai
    Contributor

    One thing that might worth point out is, the real assets loading also take time in the test. Before this change, the assets loading is stuck at waiting future to finish. so the asset does not really load. Now, the assets does get loaded synchronously. which will actually take times to load. I am not sure whether the time increase is in building the assets or actually assets loading during test.
    if it is the later, one other thing I can try will be making the assets load asynchronically. It will not spending time loading during the test unless user use tester.runAsync if they really want to test the assets load. What do you think?

  10. jonahwilliams commented on Apr 29, 2019

    @jonahwilliams
    Contributor

    The hello_world example that the benchmark runs on does not use assets, so ideally this change would not regress that time. Maybe we need one more short circuit for an empty asset manifest?

  11. Hixie commented on Apr 30, 2019

    @Hixie
    ContributorAuthor

    (I've rebaselined the relevant tests, so that we can see further regressions; that doesn't impact the validity of this bug.)

  12. chunhtai commented on Apr 30, 2019

    @chunhtai
    Contributor

    Merged, let's see if that improve

  13. chunhtai commented on May 1, 2019

    @chunhtai
    Contributor

    I have seen the test performance drop down to normal since yesterday afternoon. Looks like we are good now. Closing this issue

  14. github-actions commented on Aug 30, 2021

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  15. locked as resolved and limited conversation to collaborators on Aug 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

a: tests"flutter test", flutter_test, or one of our testsc: performanceRelates to speed or footprint issues (see "perf:" labels)c: regressionIt was better in the past than it is nowcustomer: product

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions