Skip to content

Linux web_long_running_tests_2_5 is 2.04% flaky #143834

Closed
Closed
Linux web_long_running_tests_2_5 is 2.04% flaky#143834
Assignees
harryterkelsen
Labels
P0Critical issues such as a build break or regressionc: flakeTests that sometimes, but not always, incorrectly passteam-webOwned by Web platform team

Description

@fluttergithubbot
fluttergithubbot
opened on Feb 21, 2024

The post-submit test builder Linux web_long_running_tests_2_5 had a flaky ratio 2.04% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/14957
Commit: fe4ab13

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/14957
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/14953

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_2_5

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

Activity

  1. fluttergithubbot
    added
    c: flakeTests that sometimes, but not always, incorrectly pass
    P0Critical issues such as a build break or regression
    team-webOwned by Web platform team
    on Feb 21, 2024
  2. fluttergithubbot
    assigned
    yjbanov
    on Feb 21, 2024

  3. yjbanov commented on Feb 27, 2024

    @yjbanov
    yjbanov
    on Feb 27, 2024
    Contributor

    No update yet. Just started tracking this yesterday.

  4. yjbanov commented on Mar 6, 2024

    @yjbanov
    yjbanov
    on Mar 6, 2024
    Contributor

    Still no update.

  5. fluttergithubbot commented on Mar 6, 2024

    @fluttergithubbot
    fluttergithubbot
    on Mar 6, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-02-28 and 2024-03-05 is 0.00%. Flaky number: 0; total number: 100.

  6. yjbanov commented on Mar 12, 2024

    @yjbanov
    yjbanov
    on Mar 12, 2024
    Contributor

    I haven't made any progress on this.

  7. harryterkelsen
    assigned
    harryterkelsen
    and unassigned
    yjbanov
    on Mar 12, 2024

  8. fluttergithubbot commented on Mar 13, 2024

    @fluttergithubbot
    fluttergithubbot
    on Mar 13, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-05 and 2024-03-12 is 2.02%. Flaky number: 2; total number: 99.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15324
    Commit: 187ec75
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15324
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15278

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_2_5

  9. harryterkelsen commented on Mar 16, 2024

    @harryterkelsen
    harryterkelsen
    on Mar 16, 2024
    Contributor

    Possibly due to #109474 ?

    The test just times out while trying to connect

  10. fluttergithubbot commented on Mar 20, 2024

    @fluttergithubbot
    fluttergithubbot
    on Mar 20, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-13 and 2024-03-19 is 1.01%. Flaky number: 1; total number: 99.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15456
    Commit: ca864c0
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15456

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_2_5

  11. eyebrowsoffire commented on Mar 22, 2024

    @eyebrowsoffire
    eyebrowsoffire
    on Mar 22, 2024
    Contributor

    This has gotten even more flaky lately. It seems to always flake on text_editing_integration.dart, so I'm just going to disable it for now. I spent some time reproducing locally and with the browser open, clearly a lot of wacky stuff is going on but I didn't figure out the exact root cause. We should dig deeper on this and get this re-enabled.

  12. fluttergithubbot commented on Mar 27, 2024

    @fluttergithubbot
    fluttergithubbot
    on Mar 27, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-19 and 2024-03-26 is 3.06%. Flaky number: 3; total number: 98.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15530
    Commit: afe0150
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15530
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15486
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15482

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_2_5

  13. harryterkelsen commented on Apr 2, 2024

    @harryterkelsen
    harryterkelsen
    on Apr 2, 2024
    Contributor

    Continuing to investigate this issue. The cause is still unknown.

  14. fluttergithubbot commented on Apr 3, 2024

    @fluttergithubbot
    fluttergithubbot
    on Apr 3, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-03-27 and 2024-04-02 is 1.01%. Flaky number: 1; total number: 99.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15635,%2015636,%2015634
    Commit: 1a2f34a
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15636
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15635
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_2_5/15634

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_2_5

  15. eyebrowsoffire commented on Apr 8, 2024

    @eyebrowsoffire
    eyebrowsoffire
    on Apr 8, 2024
    Contributor

    I seem to be reproducing this now locally, so I have a bit more info to share. For one thing, I believe these flakes seem to only happen when running flutter drive in --debug mode. Doing this in --release, which uses dart2js, doesn't seem to have this same type of flaky hanging.

    My repro steps are as follows:

    flutter drive --driver=test_driver/integration_test.dart --target=integration_test/example_test.dart --browser-name=chrome -d web-server --debug --web-renderer=canvaskit --no-headless

    This doesn't reproduce 100% of the time, but it doesn't usually take more than a couple of tries and the browser will stall. You can look in the JS console when this happens, and there is usually some sort of error related to module loading. Here is one such example:

    dart_sdk.js:56 Uncaught TypeError: Cannot read properties of undefined (reading 'Symbol(_privateNames)')
    at dart.privateName (dart_sdk.js:56:24)
    at load__packages__flutter_test__src__test_text_input_key_handler_dart (test_text_input_key_handler.dart.js:1057:23)
    at Object.execCb (require.js:1696:33)
    at Module.check (require.js:883:51)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.<anonymous> (require.js:1139:34)
    at require.js:134:23
    at require.js:1189:21
    at each (require.js:59:31)
    at Module.emit (require.js:1188:17)
    at Module.check (require.js:938:30)
    at Module.enable (require.js:1176:22)
    at Module.init (require.js:788:26)
    at callGetModule (require.js:1203:63)
    at Object.completeLoad (require.js:1590:21)
    at HTMLScriptElement.onScriptLoad (require.js:1717:29)

    The stacks seem to vary across different repros, but it always appears to be some sort of issue with module loading. It usually appears that the JavaScript assumes that some other script has run and has set some global state.

  16. fluttergithubbot commented on Apr 10, 2024

    @fluttergithubbot
    fluttergithubbot
    on Apr 10, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-03 and 2024-04-09 is 0.00%. Flaky number: 0; total number: 94.

  17. fluttergithubbot commented on Apr 17, 2024

    @fluttergithubbot
    fluttergithubbot
    on Apr 17, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-04-08 and 2024-04-16 is 0.00%. Flaky number: 0; total number: 98.

  18. yjbanov commented on Apr 22, 2024

    @yjbanov
    yjbanov
    on Apr 22, 2024
    Contributor

    It seems these flakes migrated into #146877. Deduping.

  19. yjbanov
    closed this as completedon Apr 22, 2024

  20. github-actions commented on May 6, 2024

    @github-actions
    github-actionsbot
    on May 6, 2024 – with GitHub Actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  21. github-actions
    locked as resolved and limited conversation to collaborators on May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P0Critical issues such as a build break or regressionc: flakeTests that sometimes, but not always, incorrectly passteam-webOwned by Web platform team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions