Skip to content

Linux analyzer_benchmark is 6.00% flaky #158243

Closed as not planned
Closed as not planned
Linux analyzer_benchmark is 6.00% flaky#158243
Assignees
andrewkolos
Labels
P1High-priority issues at the top of the work listc: flakeTests that sometimes, but not always, incorrectly passdependency: dartDart team may need to help usteam-toolOwned by Flutter Tool team

Description

@fluttergithubbot
fluttergithubbot
opened on Nov 6, 2024

The post-submit test builder Linux analyzer_benchmark had a flaky ratio 6.00% for the past (up to) 100 commits, which is above our 2.00% threshold.

One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
Commit: 4a33136

Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11118
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11104
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11101
https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11072

Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

Please follow https://github.com/flutter/flutter/blob/master/docs/infra/Reducing-Test-Flakiness.md#fixing-flaky-tests to fix the flakiness and enable the test back after validating the fix (internal dashboard to validate: go/flutter_test_flakiness).

Activity

  1. fluttergithubbot
    added
    c: flakeTests that sometimes, but not always, incorrectly pass
    P0Critical issues such as a build break or regression
    team-toolOwned by Flutter Tool team
    on Nov 6, 2024
  2. fluttergithubbot
    assigned
    andrewkolos
    on Nov 6, 2024
  3. auto-submit
    added a commit that references this issue on Nov 10, 2024

    Marks Linux analyzer_benchmark to be flaky (#158244)

    f04b0f4

  4. andrewkolos commented on Nov 11, 2024

    @andrewkolos
    andrewkolos
    on Nov 11, 2024
    Contributor

    Looks like the analysis server is exiting unexpectedly with code -9. Is this a SIGKILL?

    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137/overview:

    log excerpt 
    [2024-11-04 09:40:51.102727] [STDOUT] ═════════════╡ ••• Analyze Flutter repo with watcher - 1 / 3 ••• ╞══════════════
[2024-11-04 09:40:51.102749] [STDOUT] 
[2024-11-04 09:40:51.105266] [STDOUT] Executing "/opt/s/w/ir/x/w/rc/tmpsuwcz424/flutter sdk/bin/flutter analyze --benchmark --watch --flutter-repo --ci --debug-logs-dir=/opt/s/w/ir/x/w/rc/flutter_logs_dir" in "/opt/s/w/ir/x/w/rc/tmpsuwcz424/flutter sdk" with environment {BOT: true, LANG: en_US.UTF-8}
[2024-11-04 09:49:31.206231] [STDOUT] stdout: Analyzing Flutter repository...                                 
[2024-11-04 09:49:31.220947] [STDOUT] stderr: Analysis server exited with code -9.
[2024-11-04 09:49:31.334361] [STDERR] Task failed: Executable "/opt/s/w/ir/x/w/rc/tmpsuwcz424/flutter sdk/bin/flutter" failed with exit code 247.

    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134/overview:

    log excerpt 
    [2024-11-04 05:28:19.135930] [STDOUT] ═══════════════════╡ ••• Analyze Flutter repo  - 1 / 3 ••• ╞════════════════════
[2024-11-04 05:28:19.136055] [STDOUT] 
[2024-11-04 05:28:19.136144] [STDOUT] Executing "/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/bin/flutter analyze --benchmark --flutter-repo --ci --debug-logs-dir=/opt/s/w/ir/x/w/rc/flutter_logs_dir" in "/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk" with environment {BOT: true, LANG: en_US.UTF-8}
[2024-11-04 05:31:32.051616] [STDOUT] stdout: Analyzing flutter sdk...                                        
[2024-11-04 05:31:32.086913] [STDOUT] stderr: 
[2024-11-04 05:31:32.086971] [STDOUT] stderr: Exception: analysis server exited with code -9 and output:
[2024-11-04 05:31:32.086995] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/binding_fail_test.dart","errors":[]}}
[2024-11-04 05:31:32.087097] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/response_serialization_test.dart","errors":[]}}
[2024-11-04 05:31:32.087146] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/web_extension_test.dart","errors":[]}}
[2024-11-04 05:31:32.087185] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/data/pass_test_script.dart","errors":[]}}
[2024-11-04 05:31:32.087269] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/data/fail_then_pass_test_script.dart","errors":[]}}
[2024-11-04 05:31:32.087310] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/data/fail_test_script.dart","errors":[]}}
[2024-11-04 05:31:32.087342] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/data/pass_then_fail_test_script.dart","errors":[]}}
[2024-11-04 05:31:32.087379] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/binding_test.dart","errors":[{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/test/binding_test.dart","offset":3957,"length":84,"startLine":127,"startColumn":8,"endLine":127,"endColumn":92},"message":"TODO(jiahaog): Remove when https://github.com/flutter/flutter/issues/66006 is fixed.","code":"todo","hasFix":false}]}}
[2024-11-04 05:31:32.087425] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test_driver.dart","errors":[]}}
[2024-11-04 05:31:32.087451] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/extension.dart","errors":[]}}
[2024-11-04 05:31:32.087479] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/callback.dart","errors":[]}}
[2024-11-04 05:31:32.087508] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/_callback_web.dart","errors":[]}}
[2024-11-04 05:31:32.087533] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/_extension_io.dart","errors":[]}}
[2024-11-04 05:31:32.087716] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/_extension_web.dart","errors":[]}}
[2024-11-04 05:31:32.087748] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/channel.dart","errors":[]}}
[2024-11-04 05:31:32.087780] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/src/_callback_io.dart","errors":[]}}
[2024-11-04 05:31:32.087811] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/common.dart","errors":[]}}
[2024-11-04 05:31:32.087998] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test_driver_extended.dart","errors":[]}}
[2024-11-04 05:31:32.088061] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test.dart","errors":[{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test.dart","offset":1978,"length":60,"startLine":53,"startColumn":10,"endLine":53,"endColumn":70},"message":"TODO(jiahaog): Print the message directing users to run with","code":"todo","hasFix":false},{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test.dart","offset":14627,"length":69,"startLine":414,"startColumn":8,"endLine":414,"endColumn":77},"message":"TODO(CareF): remove this when flush FrameTiming is readily in engine.","code":"todo","hasFix":false},{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test.dart","offset":16068,"length":84,"startLine":453,"startColumn":8,"endLine":453,"endColumn":92},"message":"TODO(jiahaog): Remove when https://github.com/flutter/flutter/issues/66006 is fixed.","code":"todo","hasFix":false},{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/lib/integration_test.dart","offset":16860,"length":63,"startLine":468,"startColumn":8,"endLine":468,"endColumn":71},"message":"TODO(jiahaog): Integration test binding should not inherit from","code":"todo","hasFix":false}]}}
[2024-11-04 05:31:32.088155] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/packages/integration_test/example/lib/main.dart","errors":[]}}
[2024-11-04 05:31:32.088177] [STDOUT] stderr: 
[2024-11-04 05:31:32.157494] [STDERR] Task failed: Executable "/opt/s/w/ir/x/w/rc/tmpzr1p32l2/flutter sdk/bin/flutter" failed with exit code 1.
[2024-11-04 05:31:32.157779] [STDERR] 
[2024-11-04 05:31:32.157795] [STDERR] Stack trace:
[2024-11-04 05:31:32.201474] [STDERR] package:flutter_devicelab/framework/utils.dart 110:3       fail
[2024-11-04 05:31:32.201549] [STDERR] package:flutter_devicelab/framework/utils.dart 383:5       _execute
[2024-11-04 05:31:32.201565] [STDERR] ===== asynchronous gap ===========================
[2024-11-04 05:31:32.201580] [STDERR] dart:async                                                 _CustomZone.registerBinaryCallback
[2024-11-04 05:31:32.201596] [STDERR] package:flutter_devicelab/framework/framework.dart 299:26  _TaskRunner._performTask.<fn>
[2024-11-04 05:31:32.201609] [STDERR] package:stack_trace                                        Chain.capture
[2024-11-04 05:31:32.201621] [STDERR] package:flutter_devicelab/framework/framework.dart 298:11  _TaskRunner._performTask
[2024-11-04 05:31:32.201636] [STDERR] package:flutter_devicelab/framework/framework.dart 191:43  _TaskRunner.run

    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11118/overview:

    log except 
    [2024-11-01 11:34:22.376373] [STDOUT] ═══════════════════╡ ••• Analyze Flutter repo  - 1 / 3 ••• ╞════════════════════
[2024-11-01 11:34:22.376477] [STDOUT] 
[2024-11-01 11:34:22.376567] [STDOUT] Executing "/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/bin/flutter analyze --benchmark --flutter-repo --ci --debug-logs-dir=/opt/s/w/ir/x/w/rc/flutter_logs_dir" in "/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk" with environment {BOT: true, LANG: en_US.UTF-8}
[2024-11-01 11:36:42.162499] [STDOUT] stdout: Analyzing flutter sdk...                                        
[2024-11-01 11:36:42.162924] [STDOUT] stderr: 
[2024-11-01 11:36:42.162943] [STDOUT] stderr: Exception: analysis server exited with code -9 and output:
[2024-11-01 11:36:42.162962] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/l10n/generated_widgets_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163327] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/l10n/generated_material_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163355] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/l10n/generated_cupertino_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163380] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/l10n/generated_date_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163413] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/cupertino_localizations.dart","errors":[{"severity":"INFO","type":"TODO","location":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/cupertino_localizations.dart","offset":20650,"length":79,"startLine":509,"startColumn":12,"endLine":509,"endColumn":91},"message":"TODO(xster): fix when https://github.com/dart-lang/intl/issues/207 is resolved.","code":"todo","hasFix":false}]}}
[2024-11-01 11:36:42.163455] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/utils/date_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163480] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/material_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163504] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/src/widgets_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163528] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_localizations/lib/flutter_localizations.dart","errors":[]}}
[2024-11-01 11:36:42.163552] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/web_tests/web_driver_test.dart","errors":[]}}
[2024-11-01 11:36:42.163576] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/web_tests/web_extension_test.dart","errors":[]}}
[2024-11-01 11:36:42.163600] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/gpu_memory_summarizer_test.dart","errors":[]}}
[2024-11-01 11:36:42.163786] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/gpu_summarizer_test.dart","errors":[]}}
[2024-11-01 11:36:42.163811] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/find_test.dart","errors":[]}}
[2024-11-01 11:36:42.163835] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/io_extension_test.dart","errors":[]}}
[2024-11-01 11:36:42.163860] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/request_data_test.dart","errors":[]}}
[2024-11-01 11:36:42.163884] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/stubs/stub_finder_extension.dart","errors":[]}}
[2024-11-01 11:36:42.163909] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/stubs/stub_finder.dart","errors":[]}}
[2024-11-01 11:36:42.163933] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/stubs/stub_command.dart","errors":[]}}
[2024-11-01 11:36:42.163957] [STDOUT] stderr: [stdout] {"event":"analysis.errors","params":{"file":"/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/packages/flutter_driver/test/src/real_tests/stubs/stub_command_extension.dart","errors":[]}}
[2024-11-01 11:36:42.163978] [STDOUT] stderr: 
[2024-11-01 11:36:42.163990] [STDOUT] Checking for reboot
[2024-11-01 11:36:42.183993] [STDERR] Task failed: Executable "/opt/s/w/ir/x/w/rc/tmpc5mphe1c/flutter sdk/bin/flutter" failed with exit code 1.
[2024-11-01 11:36:42.184284] [STDERR]

  5. andrewkolos commented on Nov 11, 2024

    @andrewkolos
    andrewkolos
    on Nov 11, 2024
    Contributor

    Here is the source of this exception:

    flutter/packages/flutter_tools/lib/src/commands/analyze_once.dart

    Lines 94 to 105 in 95cd3c0

    await server.start();
    // Completing the future in the callback can't fail.
    unawaited(server.onExit.then<void>((int? exitCode) {
    if (!analysisCompleter.isCompleted) {
    analysisCompleter.completeError(
    // Include the last 20 lines of server output in exception message
    Exception(
    'analysis server exited with code $exitCode and output:\n${server.getLogs(20)}',
    ),
    );
    }
    }));

  6. matanlurey
    added
    dependency: dartDart team may need to help us
    on Nov 12, 2024

  7. andrewkolos commented on Nov 12, 2024

    @andrewkolos
    andrewkolos
    on Nov 12, 2024
    Contributor

    IIRC 9 is SIGKILL. From chatting with @brianquinlan, I believe a process exited using SIGKILL will have an exit code of -9. My guess here, then, is that something is killing the dart process for the analyzer (either the tool itself or the OS, I assume).

  8. fluttergithubbot commented on Nov 13, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 13, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-03 and 2024-11-10 is 6.45%. Flaky number: 6; total number: 93.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    Commit: 8461cee
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11187
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11183
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11172
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  9. fluttergithubbot commented on Nov 13, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 13, 2024
    ContributorAuthor

    [staging pool] flaky ratio for the past (up to) 100 commits between 2024-11-10 and 2024-11-12 is 0.00%. Flaky number: 0; total number: 15.

  10. fluttergithubbot commented on Nov 20, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 20, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-04 and 2024-11-10 is 6.82%. Flaky number: 6; total number: 88.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    Commit: 8461cee
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11187
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11183
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11172
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  11. fluttergithubbot commented on Nov 20, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 20, 2024
    ContributorAuthor

    [staging pool] flaky ratio for the past (up to) 100 commits between 2024-11-12 and 2024-11-19 is 1.00%. Flaky number: 1; total number: 100.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/125
    Commit: 8f62a99
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/125

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  12. a-siva commented on Nov 20, 2024

    @a-siva
    a-siva
    on Nov 20, 2024
    Contributor

    //cc @bwilkerson , it appears that the analyzer is timing out and getting killed running this benchmark, any ideas why it would time out flakily ?

  13. brianquinlan commented on Nov 20, 2024

    @brianquinlan
    brianquinlan
    on Nov 20, 2024
    Contributor

    The test infra itself sends sigkill in a few places e.g.

    flutter/packages/flutter_tools/test/integration.shard/test_driver.dart

    Line 234 in 53bd643

    Future<int> _killForcefully() {

  14. bwilkerson commented on Nov 20, 2024

    @bwilkerson
    bwilkerson
    on Nov 20, 2024
    Contributor

    ... any ideas why it would time out flakily ?

    Not off the top of my head, no.

  15. a-siva commented on Nov 20, 2024

    @a-siva
    a-siva
    on Nov 20, 2024
    Contributor

    ... any ideas why it would time out flakily ?

    Not off the top of my head, no.

    Would be great if somebody from the analyzer team could investigate this, the issue has been marked as P0 as the flake rate is pretty high.

  16. andrewkolos
    added
    P1High-priority issues at the top of the work list
    and removed
    P0Critical issues such as a build break or regression
    on Nov 21, 2024

  17. fluttergithubbot commented on Nov 27, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 27, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-04 and 2024-11-10 is 6.82%. Flaky number: 6; total number: 88.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    Commit: 8461cee
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11187
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11183
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11172
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  18. fluttergithubbot commented on Nov 27, 2024

    @fluttergithubbot
    fluttergithubbot
    on Nov 27, 2024
    ContributorAuthor

    [staging pool] flaky ratio for the past (up to) 100 commits between 2024-11-18 and 2024-11-26 is 1.00%. Flaky number: 1; total number: 100.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/233
    Commit: fc2254c
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/233

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  19. fluttergithubbot commented on Dec 4, 2024

    @fluttergithubbot
    fluttergithubbot
    on Dec 4, 2024
    ContributorAuthor

    [prod pool] flaky ratio for the past (up to) 100 commits between 2024-11-04 and 2024-11-10 is 6.82%. Flaky number: 6; total number: 88.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    Commit: 8461cee
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11211
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11187
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11183
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11172
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11137
    https://ci.chromium.org/ui/p/flutter/builders/prod/Linux%20analyzer_benchmark/11134

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  20. fluttergithubbot commented on Dec 4, 2024

    @fluttergithubbot
    fluttergithubbot
    on Dec 4, 2024
    ContributorAuthor

    [staging pool] flaky ratio for the past (up to) 100 commits between 2024-11-21 and 2024-12-03 is 1.00%. Flaky number: 1; total number: 100.
    One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/233
    Commit: fc2254c
    Flaky builds:
    https://ci.chromium.org/ui/p/flutter/builders/staging/Linux%20analyzer_benchmark/233

    Recent test runs:
    https://flutter-dashboard.appspot.com/#/build?taskFilter=Linux%20analyzer_benchmark

  21. a-siva commented on Dec 4, 2024

    @a-siva
    a-siva
    on Dec 4, 2024
    Contributor

    Looks like the flaky number in the periods 2024-11-18 and 2024-11-26, 2024-11-21 and 2024-12-03 has dropped to 1.00%,
    Can the issue be closed ?

  22. a-siva commented on Dec 4, 2024

    @a-siva
    a-siva
    on Dec 4, 2024
    Contributor

    @andrewkolos please see comment above.

  23. andrewkolos
    closed this as not plannedon Dec 4, 2024

  24. github-actions commented on Dec 18, 2024

    @github-actions
    github-actionsbot
    on Dec 18, 2024 – with GitHub Actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  25. github-actions
    locked as resolved and limited conversation to collaborators on Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P1High-priority issues at the top of the work listc: flakeTests that sometimes, but not always, incorrectly passdependency: dartDart team may need to help usteam-toolOwned by Flutter Tool team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions