Skip to content

flutter_tools\test\asset_bundle_test.dart is flaky on windows #34446

Closed
#42597
Closed
flutter_tools\test\asset_bundle_test.dart is flaky on windows#34446
#42597
Assignees
jonahwilliams
Labels
c: flakeTests that sometimes, but not always, incorrectly pass

Description

@jonahwilliams
jonahwilliams
opened on Jun 13, 2019

test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories

Activity

  1. jonahwilliams
    added
    ⚠ TODAY
    on Jun 13, 2019

  2. Hixie commented on Jun 13, 2019

    @Hixie
    Hixie
    on Jun 13, 2019
    Contributor

    When did it start being flaky?

  3. tvolkert commented on Jun 13, 2019

    @tvolkert
    tvolkert
    on Jun 13, 2019
    Contributor

    Do you have an example Cirrus task that failed, and what the output was?

  4. tvolkert
    added
    team: flakes
    on Jun 13, 2019

  5. jonahwilliams commented on Jun 13, 2019

    @jonahwilliams
    jonahwilliams
    on Jun 13, 2019
    ContributorAuthor

    It was on my latest commit but someone clicked rerun, or cirrus re-ran it

  6. tvolkert commented on Jun 13, 2019

    @tvolkert
    tvolkert
    on Jun 13, 2019
    Contributor

    https://cirrus-ci.com/task/6705135426732032

    00:19 +144: test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories                                                                                                      Expected: <true>
  Actual: <false>

#0      fail (package:test_api/src/frontend/expect.dart:152:30)
#1      _expect (package:test_api/src/frontend/expect.dart:146:3)
#2      expect (package:test_api/src/frontend/expect.dart:59:3)
#3      main.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/asset_bundle_test.dart:131:7)
<asynchronous suspension>
#4      testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:108:42)
<asynchronous suspension>
#5      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#6      _rootRun (dart:async/zone.dart:1124:13)
#7      _CustomZone.run (dart:async/zone.dart:1021:19)
#8      _runZoned (dart:async/zone.dart:1516:10)
#9      runZoned (dart:async/zone.dart:1463:12)
#10     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#11     testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:96:30)
#12     _rootRun (dart:async/zone.dart:1124:13)
#13     _CustomZone.run (dart:async/zone.dart:1021:19)
#14     _runZoned (dart:async/zone.dart:1516:10)
#15     runZoned (dart:async/zone.dart:1500:12)
#16     testUsingContext.<anonymous closure>.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:94:18)
#17     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#18     _rootRun (dart:async/zone.dart:1124:13)
#19     _CustomZone.run (dart:async/zone.dart:1021:19)
#20     _runZoned (dart:async/zone.dart:1516:10)
#21     runZoned (dart:async/zone.dart:1463:12)
#22     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#23     testUsingContext.<anonymous closure>.<anonymous closure> (org-dartlang-app:///test/src/context.dart:69:22)
#24     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:153:29)
<asynchronous suspension>
#25     _rootRun (dart:async/zone.dart:1124:13)
#26     _CustomZone.run (dart:async/zone.dart:1021:19)
#27     _runZoned (dart:async/zone.dart:1516:10)
#28     runZoned (dart:async/zone.dart:1463:12)
#29     AppContext.run (package:flutter_tools/src/base/context.dart:152:18)
<asynchronous suspension>
#30     runInContext (package:flutter_tools/src/context_runner.dart:56:24)
<asynchronous suspension>
#31     testUsingContext.<anonymous closure> (org-dartlang-app:///test/src/context.dart:68:11)
<asynchronous suspension>
#32     Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:168:27)
<asynchronous suspension>
#33     Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test_api/src/backend/invoker.dart:250:15)
<asynchronous suspension>
#34     _rootRun (dart:async/zone.dart:1124:13)
#35     _CustomZone.run (dart:async/zone.dart:1021:19)
#36     _runZoned (dart:async/zone.dart:1516:10)
#37     runZoned (dart:async/zone.dart:1463:12)
#38     Invoker.waitForOutstandingCallbacks (package:test_api/src/backend/invoker.dart:247:5)
#39     Declarer.test.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/declarer.dart:166:33)
#40     _rootRun (dart:async/zone.dart:1124:13)
#41     _CustomZone.run (dart:async/zone.dart:1021:19)
#42     _runZoned (dart:async/zone.dart:1516:10)
#43     runZoned (dart:async/zone.dart:1463:12)
#44     Declarer.test.<anonymous closure> (package:test_api/src/backend/declarer.dart:165:13)
<asynchronous suspension>
#45     Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test_api/src/backend/invoker.dart:400:25)
<asynchronous suspension>
#46     new Future.<anonymous closure> (dart:async/future.dart:176:37)
#47     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#48     StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
#49     _rootRun (dart:async/zone.dart:1120:38)
#50     _CustomZone.run (dart:async/zone.dart:1021:19)
#51     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
#52     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
#53     StackZoneSpecification._run (package:stack_trace/src/stack_zone_specification.dart:209:15)
#54     StackZoneSpecification._registerCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart:119:48)
#55     _rootRun (dart:async/zone.dart:1124:13)
#56     _CustomZone.run (dart:async/zone.dart:1021:19)
#57     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:947:23)
#58     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:21:15)
#59     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:382:19)
#60     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:416:5)
#61     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)


00:19 +144 -1: test\asset_bundle_test.dart: AssetBundle.build handle removal of wildcard directories [E]                                                                                               
  Expected: <true>
    Actual: <false>
  
  package:test_api                                       expect
  org-dartlang-app:///test/asset_bundle_test.dart 131:7  main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                             _AsyncAwaitCompleter.completeError
  org-dartlang-app:///test/asset_bundle_test.dart        main.<fn>.<fn>
  ===== asynchronous gap ===========================
  dart:async                                             _asyncThenWrapperHelper
  org-dartlang-app:///test/asset_bundle_test.dart        main.<fn>.<fn>
  org-dartlang-app:///test/src/context.dart 108:42       testUsingContext.<fn>.<fn>.<fn>.<fn>.<fn>

  7. tvolkert commented on Jun 13, 2019

    @tvolkert
    tvolkert
    on Jun 13, 2019
    Contributor

    I looked at the last 100 Cirrus runs, and this failed on 4 of them - all indications are that it's been flaky before that. So we're looking at ~4% flakiness potentially since the test was written.

  8. tvolkert commented on Jun 14, 2019

    @tvolkert
    tvolkert
    on Jun 14, 2019
    Contributor

    Skipping the test on Windows, and removing the TODAY label.

  9. tvolkert
    removed
    ⚠ TODAY
    on Jun 14, 2019
  10. tvolkert
    added a commit that references this issue on Jun 14, 2019

    Skip flaky test on Windows (#34464)

    530824f

  11. gspencergoog commented on Oct 11, 2019

    @gspencergoog
    gspencergoog
    on Oct 11, 2019
    Contributor

    Hmm. this started failing very consistently again today. Failed three times in a row, and reverting the first PR didn't stop the failing:

    https://cirrus-ci.com/task/6014762248306688?command=test_all#L570

  12. Hixie
    added this to the milestone on Oct 11, 2019
  13. Hixie
    added
    ⚠ TODAY
    on Oct 11, 2019

  14. dnfield commented on Oct 11, 2019

    @dnfield
    dnfield
    on Oct 11, 2019
    Contributor

    I'm pretty sure this is flaky because it's trying to compare timestamps that end up not being what's expected.

    https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/asset.dart#L103 in particular is looking at the stat of the file in a MemoryFileSystem. It's possible that the file writes are happening quickly enough where it's resolving to the same time (instead of some time later).

  15. tvolkert commented on Oct 11, 2019

    @tvolkert
    tvolkert
    on Oct 11, 2019
    Contributor

    dart-archive/file.dart#129 was merged yesterday to allow this to be fixed - published in package:file version 5.1.0.

  16. dnfield commented on Oct 11, 2019

    @dnfield
    dnfield
    on Oct 11, 2019
    Contributor

    @tvolkert - that actually just makes the failure 100% instead of flakily dependent on the clock. We'd still end up with an implementation that wants to compare a timestamp of when a method got called with the stat of a file, which .. probably needs to change. I'm going to mark the test as skipped again to unblock the tree. @jonahwilliams says he's going to work on this some more on Monday.

  17. jonahwilliams
    self-assigned this
    on Oct 11, 2019
  18. dnfield
    removed
    ⚠ TODAY
    on Oct 11, 2019
  19. jonahwilliams
    closed this as completedin #42597on Oct 14, 2019

  20. github-actions commented on Aug 13, 2021

    @github-actions
    github-actionsbot
    on Aug 13, 2021

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  21. github-actions
    locked as resolved and limited conversation to collaborators on Aug 13, 2021
  22. flutter-triage-bot
    added
    c: flakeTests that sometimes, but not always, incorrectly pass
    on Jul 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

c: flakeTests that sometimes, but not always, incorrectly pass

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions