Flutter hangs in xcodebuild -showBuildSettings when building for iOS (v1.7.8+hotfix.3)

@eseidelGoogle

Building our Flutter app will hang about 20% of the time using the following command:

flutter build ios --release --no-codesign -t lib/main.dart

Android builds do not exhibit this behavior.

The last output we see is something like:

Warning: Building for device with codesigning disabled. You will have to manually codesign before deploying to device.
Building ws.flea.Fleaflicker for device (ios-release)...

Each build is executed from an identical clean environment.

Steps to Reproduce

Behavior is not reproducible every time. The current workaround is to retry the build (in a clean environment) which eventually succeeds.

Our build script is not much more complex than this (adapted from here):

git clone https://github.com/flutter/flutter.git
export PATH=`pwd`/flutter/bin:$PATH

flutter channel stable
flutter doctor

flutter build ios --release --no-codesign

I'd be happy to share the codebase and access to our continuous deployment tool (Microsoft AppCenter) with Flutter devs.

+ flutter doctor
Downloading Material fonts...                                       0.5s
Downloading android-arm-profile/darwin-x64 tools...                 0.3s
Downloading android-arm-release/darwin-x64 tools...                 0.2s
Downloading android-arm64-profile/darwin-x64 tools...               0.3s
Downloading android-arm64-release/darwin-x64 tools...               0.3s
Downloading android-x86 tools...                                    1.0s
Downloading android-x64 tools...                                    1.1s
Downloading android-arm tools...                                    0.5s
Downloading android-arm-profile tools...                            0.3s
Downloading android-arm-release tools...                            0.3s
Downloading android-arm64 tools...                                  0.5s
Downloading android-arm64-profile tools...                          0.4s
Downloading android-arm64-release tools...                          0.3s
Downloading ios tools...                                            2.5s
Downloading ios-profile tools...                                    1.5s
Downloading ios-release tools...                                    1.1s
Downloading Gradle Wrapper...                                       0.0s
Downloading package sky_engine...                                   0.1s
Downloading common tools...                                         0.5s
Downloading common tools...                                         0.5s
Downloading darwin-x64 tools...                                     1.6s
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, v1.7.8+hotfix.3, on Mac OS X 10.14.5 18F132, locale en)
[!] Android toolchain - develop for Android devices (Android SDK version 29.0.0)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses
[✓] Xcode - develop for iOS and macOS (Xcode 10.2.1)
[✗] iOS tools - develop for iOS devices
    ✗ libimobiledevice and ideviceinstaller are not installed. To install with Brew, run:
        brew update
        brew install --HEAD usbmuxd
        brew link usbmuxd
        brew install --HEAD libimobiledevice
        brew install ideviceinstaller
    ✗ ios-deploy not installed. To install:
        brew install ios-deploy
[!] Android Studio (not installed)
[!] Connected device
    ! No devices available

! Doctor found issues in 4 categories.

We've observed this behavior since first using Flutter with v1.2.1. We only use the stable channel.

Dart package dependencies we've used since observing this behavior:

  shared_preferences
  protobuf
  http
  intl
  dynamic_theme
  timeago
  event_bus
  firebase_admob

(Possibly the same issue as described and closed in #28415. cc @eseidelGoogle)

@kristiansagi

@kristiansagi may know if nevercode is seeing this too?

@xster

@xster or @zanderso may have context. Thanks for the report!

I'm not aware of any changes recently. Can you try

flutter build ios --release --no-codesign --verbose -t lib/main.dart and show us the output?

I am now unable to reproduce the issue with the verbose flag.

Going to close until I can gather more details.

@eseidelGoogle

@eseidelGoogle I can confirm on behalf of @NevercodeHQ / @kristiansagi that we've seen a spike in iOS builds getting stuck since July 9. flutter build ios ...args does not exit and the last bit of log is

Xcode build done.                                           123.4s

So far we've seen it happening on versions 1.7.8+hotfix.2, 1.7.8+hotfix.3, 1.8.0, 1.8.1-pre.2, 1.8.1-pre.22.

@priitlatt

Reopening based on @priitlatt's comments.

@jonahwilliams @zanderso does this show up in our analytics?

I will add a single data point from a build that got stuck with the verbose flag:

[   +1 ms] Copying service frameworks to '/Users/vsts/agent/2.154.1/work/1/s/ios/Frameworks'.
[   +1 ms] Creating service definitions manifest at '/Users/vsts/agent/2.154.1/work/1/s/ios/ServiceDefinitions.json'
[  +34 ms] Found plugin firebase_admob at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/firebase_admob-0.9.0+1/
[        ] Found plugin firebase_analytics at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/firebase_analytics-3.0.3/
[        ] Found plugin firebase_messaging at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/firebase_messaging-5.1.0/
[   +1 ms] Found plugin flutter_image_compress at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/flutter_image_compress-0.6.3/
[   +7 ms] Found plugin image_picker at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/image_picker-0.6.0+15/
[   +5 ms] Found plugin package_info at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/package_info-0.4.0+3/
[   +4 ms] Found plugin shared_preferences at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/shared_preferences-0.4.3/
[   +6 ms] Found plugin url_launcher at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/url_launcher-5.0.5/
[   +8 ms] executing: [/Users/vsts/agent/2.154.1/work/1/s/ios/Runner.xcodeproj/] /usr/bin/xcodebuild -project /Users/vsts/agent/2.154.1/work/1/s/ios/Runner.xcodeproj -target Runner -showBuildSettings

[Build got stuck here]

The next line should have been something like:

[+1449 ms] Exit code 0 from: /usr/bin/xcodebuild -project /Users/vsts/agent/2.154.1/work/1/s/ios/Runner.xcodeproj -target Runner -showBuildSettings

I will also add that we're seeing these less frequently than before the 1.7 stable release so I don't believe this is the same issue as #28415

@stuartmorgan

Looks like it's the invocation here that is hanging: https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/macos/cocoapods.dart#L197. @stuartmorgan could you take a look?

@orischwartz

I'm a bit confused; @orischwartz has log output that's stuck in a different place than @priitlatt. The bug was re-opened based on the former, but the last comment is focusing on the latter.

Do we have reason to believe the two cases are related even though the location where the build stopped is different?

I've scraped some usage statistics from @codemagic-ci-cd and still see that small fraction of builds do not get past the xcodebuild -showBuildSettings on the first run, even when built with Flutter 1.9.7+.

What we see is something like

Builders-Mac-2:capybara builder$ flutter clean && flutter build ios -v
Cleaning Xcode workspace...                                         5.3s
Deleting build...                                                    2ms
[  +51 ms] executing: [/Users/builder/programs/flutter/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[  +79 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[        ] af9f210443971a86484c4772f54003838e64b388
[        ] executing: [/Users/builder/programs/flutter/] git describe --match v*.*.* --first-parent --long --tags
[  +49 ms] Exit code 0 from: git describe --match v*.*.* --first-parent --long --tags
[        ] v1.9.7-80-gaf9f21044
[  +17 ms] executing: [/Users/builder/programs/flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +23 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] origin/master
[        ] executing: [/Users/builder/programs/flutter/] git ls-remote --get-url origin
[  +19 ms] Exit code 0 from: git ls-remote --get-url origin
[   +1 ms] https://github.com/flutter/flutter.git
[ +115 ms] executing: [/Users/builder/programs/flutter/] git rev-parse --abbrev-ref HEAD
[  +31 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] master
[  +52 ms] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[   +8 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[ +706 ms] executing: /usr/bin/plutil -convert json -o - /Users/builder/capybara/ios/Runner/Info.plist
[  +15 ms] Exit code 0 from: /usr/bin/plutil -convert json -o - /Users/builder/capybara/ios/Runner/Info.plist
[        ]
{"CFBundleName":"capybara","CFBundleIdentifier":"$(PRODUCT_BUNDLE_IDENTIFIER)","CFBundleInfoDictionaryVersion":"6.0","UIMainStoryboardFile":"Main","CFBundleVersi
on":"$(FLUTTER_BUILD_NUMBER)","UILaunchStoryboardName":"LaunchScreen","CFBundleExecutable":"$(EXECUTABLE_NAME)","LSRequiresIPhoneOS":true,"UISupportedInterfaceOr
ientations~ipad":["UIInterfaceOrientationPortrait","UIInterfaceOrientationPortraitUpsideDown","UIInterfaceOrientationLandscapeLeft","UIInterfaceOrientationLandsc
apeRight"],"UISupportedInterfaceOrientations":["UIInterfaceOrientationPortrait","UIInterfaceOrientationLandscapeLeft","UIInterfaceOrientationLandscapeRight"],"UI
ViewControllerBasedStatusBarAppearance":false,"CFBundleSignature":"????","CFBundlePackageType":"APPL","CFBundleDevelopmentRegion":"en","ITSAppUsesNonExemptEncryp
tion":false,"CFBundleShortVersionString":"$(FLUTTER_BUILD_NAME)"}
[  +30 ms] Building io.codemagic.capybara for device (ios-release)...
[ +196 ms] executing: [/Users/builder/capybara/ios/] /usr/bin/xcodebuild -list
[+2314 ms] Information about project "Runner":
                        Targets:
                            Runner

                        Build Configurations:
                            Debug
                            Release

                        If no build configuration is specified and -scheme is not passed then "Release" is used.

                        Schemes:
                            Runner
[   +7 ms] executing: [/Users/builder/capybara/ios/Runner.xcodeproj/] /usr/bin/xcodebuild -project /Users/builder/capybara/ios/Runner.xcodeproj -target Runner
-showBuildSettings

At this point the build hangs, and at the same time

Builders-Mac-2:~ builder$ ps aux | grep flutter
builder          3326 100.0  2.3  4531192 195128 s000  R+    7:15AM   0:56.09 /Users/builder/programs/flutter/bin/cache/dart-sdk/bin/dart --packages=/Users/builder/programs/flutter/packages/flutter_tools/.packages /Users/builder/programs/flutter/bin/cache/flutter_tools.snapshot build ios -v
builder          3310   0.0  0.0  4269644   1216 s000  S+    7:15AM   0:00.01 bash /usr/local/bin/flutter build ios -v
builder          3392   0.0  0.0  4267932    440 s002  R+    7:16AM   0:00.00 grep flutter
Builders-Mac-2:~ builder$ pstree 3310
-+= 03310 builder bash /usr/local/bin/flutter build ios -v
 \--- 03326 builder /Users/builder/programs/flutter/bin/cache/dart-sdk/bin/dart --packages=/Users/builder/programs/flutter/packages/flutter_tools/.packages /Users/builder/pro

Flutter doctor output on this environment was

Running flutter doctor...
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, v1.9.8-pre.80, on Mac OS X 10.14.6 18G95, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 10.3)
[✓] Android Studio (version 3.3)
[!] Connected device
    ! No devices available

! Doctor found issues in 1 category.

@priitlatt

Hi @priitlatt, #39579 is in review, which will add a timeout/retry to the -showBuildSettings invocation that is still hanging.

#39579 has landed on master.

@zanderso

@zanderso does v1.10.1-pre.41 have the fix?

Had a build hang today from master channel

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, v1.10.1-pre.41, on Mac OS X 10.14.6 18G95, locale en)
...
[   +9 ms] Found plugin url_launcher at /Users/vsts/.pub-cache/hosted/pub.dartlang.org/url_launcher-5.1.2/
[   +9 ms] executing: [/Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj/] /usr/bin/xcodebuild -project /Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj -target Runner -showBuildSettings
##[error]The operation was canceled.

^^ build killed after hanging for ~25 minutes.

Just for anyone coming to this thread to know if it was fixed by the stable update yesterday, it was not. Still seeing this on stable v1.9.1+hotfix.2.

@orischwartz

@orischwartz Apologies for the churn and delay. The previous change was included in v1.10.1-pre.41, but I introduced a bug in the process timeout logic by using a mock for testing that didn't properly simulate a timing out process. I've sent #40159 to fix that issue. Thanks again for your patience.

#40159 has landed on master.

👍 Haven't had a build hang yet, 9 consecutive attempts. We've never had more than 4 consecutive builds succeed. Not sure if that's statistically significant but as far as I'm concerned this fixes the original issue I reported. Thanks!

Glad to hear it! I will close this issue, but please feel free to re-open if you see the same thing again.

@zanderso

@zanderso Any chance that #39899 introduced a regression that reverted the fix?

I've had two builds hang since then. All of the successful builds yesterday were before that commit.

Also possible that I spoke too soon but it really seemed like it was fixed on yesterday's master.

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, v1.10.2-pre.62, on Mac OS X 10.14.6 18G95, locale en)
...
[   +9 ms] executing: [/Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj/] /usr/bin/xcodebuild -project /Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj -target Runner -showBuildSettings
##[error]The operation was canceled.

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel master, v1.10.2-pre.63, on Mac OS X 10.14.6 18G95, locale en)
[   +8 ms] executing: [/Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj/] /usr/bin/xcodebuild -project /Users/vsts/agent/2.155.1/work/1/s/ios/Runner.xcodeproj -target Runner -showBuildSettings
##[error]The operation was canceled.

There was a remaining synchronous (blocking) call without a timeout. I've removed it in #40435.

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

Flutter hangs in xcodebuild -showBuildSettings when building for iOS (v1.7.8+hotfix.3) #35988

Steps to Reproduce

29 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Flutter hangs in xcodebuild -showBuildSettings when building for iOS (v1.7.8+hotfix.3) #35988

Description

Steps to Reproduce

Activity

eseidelGoogle commented on Jul 11, 2019

eseidelGoogle commented on Jul 11, 2019

xster commented on Jul 11, 2019

orischwartz commented on Jul 11, 2019

priitlatt commented on Jul 15, 2019

eseidelGoogle commented on Jul 15, 2019

orischwartz commented on Jul 15, 2019

zanderso commented on Jul 15, 2019

stuartmorgan-g commented on Jul 15, 2019

29 remaining items

priitlatt commented on Sep 4, 2019

zanderso commented on Sep 4, 2019

zanderso commented on Sep 5, 2019

orischwartz commented on Sep 9, 2019

MisterJimson commented on Sep 10, 2019

zanderso commented on Sep 10, 2019

zanderso commented on Sep 11, 2019

orischwartz commented on Sep 12, 2019

zanderso commented on Sep 12, 2019

orischwartz commented on Sep 13, 2019

zanderso commented on Sep 16, 2019

github-actions commented on Aug 28, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions