Skip to content

Linux web_long_running_tests_5_5 is 5.10% flaky #149238

Closed
Closed
Linux web_long_running_tests_5_5 is 5.10% flaky#149238
Assignees
yjbanov
Labels
P0Critical issues such as a build break or regressionc: flakeTests that sometimes, but not always, incorrectly passteam-webOwned by Web platform team

Description

@fluttergithubbot
fluttergithubbot
opened on May 29, 2024

The post-submit test builder Linux web_long_running_tests_5_5 had a flaky ratio 5.10% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16861
Commit: 65e3007

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16861
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16835
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16829
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16798
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20web_long_running_tests_5_5/16783

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20web_long_running_tests_5_5

Please follow https://github.com/flutter/flutter/wiki/Reducing-Test-Flakiness#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

Activity

  1. fluttergithubbot
    added
    c: flakeTests that sometimes, but not always, incorrectly pass
    P0Critical issues such as a build break or regression
    team-webOwned by Web platform team
    on May 29, 2024
  2. fluttergithubbot
    assigned
    yjbanov
    on May 29, 2024

  3. loic-sharma commented on May 29, 2024

    @loic-sharma
    loic-sharma
    on May 29, 2024
    Member

    Example error:

    Bad state: StreamSink is closed
#0      _StreamSinkImpl.add (dart:_http/http_impl.dart:912:7)
#1      _WebSocketImpl.add (dart:_http/websocket_impl.dart:1222:11)
#2      VmService._call.<anonymous closure> (package:vm_service/src/vm_service.dart:1785:22)
#3      VmService._call (package:vm_service/src/vm_service.dart:1787:8)
#4      VmService.streamListen (package:vm_service/src/vm_service.dart:1710:7)
#5      FlutterVmService.findExtensionIsolate (package:flutter_tools/src/vmservice.dart:982:21)
#6      FlutterResidentDevtoolsHandler._waitForExtensionsForDevice (package:flutter_tools/src/resident_devtools_handler.dart:234:38)
#7      FlutterResidentDevtoolsHandler._devicesWithExtensions (package:flutter_tools/src/resident_devtools_handler.dart:226:59)
#8      FlutterResidentDevtoolsHandler.serveAndAnnounceDevTools.callServiceExtensions (package:flutter_tools/src/resident_devtools_handler.dart:126:63)
#9      FlutterResidentDevtoolsHandler.serveAndAnnounceDevTools (package:flutter_tools/src/resident_devtools_handler.dart:140:13)
<asynchronous suspension>

╔═╡ERROR #1╞════════════════════════════════════════════════════════════════════
║ Command: ../../../bin/flutter run --debug -d chrome --web-run-headless --dart-define=FLUTTER_WEB_USE_SKIA=false --dart-define=FLUTTER_WEB_AUTO_DETECT=false --dart-define=test.valueA=Example,A --dart-define=test.valueB=Value -t lib/web_define_loading.dart
║ Command exited with exit code 1 but expected zero exit code.
║ Working directory: /b/s/w/ir/x/w/flutter/dev/integration_tests/web
╚═══════════════════════════════════════════════════════════════════════════════

    This same error also causes this other flake: #148811

  4. loic-sharma commented on May 29, 2024

    @loic-sharma
    loic-sharma
    on May 29, 2024
    Member

    @eyebrowsoffire has a potential fix here: https://dart-review.googlesource.com/c/sdk/+/367821

  5. yjbanov commented on Jun 5, 2024

    @yjbanov
    yjbanov
    on Jun 5, 2024
    Contributor

    After some investigation, strongly suspecting a race in the tool. After "q" is sent to the tool, the tool attempts to talk to the app through the VmService, but by then the browser is closed and the VM service connection is dropped.

    Also noticed a bug in the test itself, where the "q" keystroke was sent repeatedly after the test was finished, which perhaps exacerbated the race. Fixed that one in #149767.

  6. auto-submit
    added a commit that references this issue on Jun 6, 2024

    Attempt to fix some web test flakiness. (#149828)

    5fb34b7

  7. fluttergithubbot commented on Jun 12, 2024

    @fluttergithubbot
    fluttergithubbot
    on Jun 12, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-06-05 and 2024-06-11 is 0.00%. Flaky number: 0; total number: 95.

  8. yjbanov commented on Jun 12, 2024

    @yjbanov
    yjbanov
    on Jun 12, 2024
    Contributor

    #149828 by @eyebrowsoffire fixed this one.

  9. yjbanov
    closed this as completedon Jun 12, 2024

  10. github-actions commented on Jun 26, 2024

    @github-actions
    github-actionsbot
    on Jun 26, 2024 – with GitHub Actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  11. github-actions
    locked as resolved and limited conversation to collaborators on Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P0Critical issues such as a build break or regressionc: flakeTests that sometimes, but not always, incorrectly passteam-webOwned by Web platform team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions