Repository navigation
Unable to complete any iOS UI tests that lasts longer than ~6 minutes #145143
Description
Steps to reproduce
Add the following to pubspec.yaml:
integration_test:
sdk: flutter
Setup project for UI testing:
https://github.com/flutter/flutter/tree/main/packages/integration_test#ios-device-testing
Add a UI test that takes longer than 6 minutes to run. I provided a code sample for this.
TEST CASE:
Use Xcode to run the UI tests on a physical device.
The test will run for a while but exit after a certain time before completion. In my test cases it always happened around 5 minutes and 43 seconds.
My tests consist of a simple loop of pressing a button and increasing the count with await Future.delayed(const Duration(seconds: 1)); within the loop.
But our production tests show the same result. Unrelated to what tests run and in what order it always interrupts around that time mark.
Notes:
- It works on older Xcode versions (14.2 for example)
- It works when running through Flutter with debug or run from vscode
- It works on a simulator from Xcode or Flutter in vscode
- It works on Xcode 15 (or later) while using Swift for a simple UI test with a similar setup.
- It does not work on Xcode 15 (or later) from a Flutter project on a physical device
That's why I think it's Flutter related.
Expected results
Test completes after 600 prints.
Actual results
Test fails after ~330 prints (somewhere before 6 minutes)
Code sample
Code sample
main.dart (default generated file, comments removed)
import 'package:flutter/material.dart';
void main() {
runApp(const MyApp());
}
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Flutter Demo',
theme: ThemeData(
colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
useMaterial3: true,
),
home: const MyHomePage(title: 'Flutter Demo Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
const MyHomePage({super.key, required this.title});
final String title;
@override
State<MyHomePage> createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
int _counter = 0;
void _incrementCounter() {
setState(() {
_counter++;
});
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
backgroundColor: Theme.of(context).colorScheme.inversePrimary,
title: Text(widget.title),
),
body: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
const Text(
'You have pushed the button this many times:',
),
Text(
'$_counter',
style: Theme.of(context).textTheme.headlineMedium,
),
],
),
),
floatingActionButton: FloatingActionButton(
onPressed: _incrementCounter,
tooltip: 'Increment',
child: const Icon(Icons.add),
),
);
}
}
Add custom UI test file widget_test.dart in /integration_test with the following content:
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:bug/main.dart';
void main() {
testWidgets('Counter increments smoke test', (WidgetTester tester) async {
// Build our app and trigger a frame.
await tester.pumpWidget(const MyApp());
// Verify that our counter starts at 0.
expect(find.text('0'), findsOneWidget);
expect(find.text('1'), findsNothing);
// Verify that our counter has incremented.
for (var i = 0; i < 600; i++) {
await tester.tap(find.byIcon(Icons.add));
await tester.pump();
debugPrint("${i + 1}");
await Future.delayed(const Duration(seconds: 1));
expect(find.text((i + 1).toString()), findsOneWidget);
}
});
}Screenshots or Video
Screenshots / Video demonstration
[Upload media here]
Logs
Logs
This is the output in text from .xcresult
Runner (3702) encountered an error
Test runner never began executing tests after launchingFlutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.19.3, on macOS 14.2.1 23C71 darwin-arm64, locale en-NL)
• Flutter version 3.19.3 on channel stable at /Users/myname/Developer/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision ba39319843 (7 days ago), 2024-03-07 15:22:21 -0600
• Engine revision 2e4ba9c6fb
• Dart version 3.3.1
• DevTools version 2.31.1
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
• Android SDK at /Users/myname/Library/Android/sdk
• Platform android-34, build-tools 33.0.2
• ANDROID_HOME = /Users/myname/Library/Android/sdk
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
• All Android licenses accepted.
[✓] Xcode - develop for iOS and macOS (Xcode 15.3)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 15E204a
• CocoaPods version 1.15.2
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 2022.1)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 11.0.15+0-b2043.56-8887301)
[✓] VS Code (version 1.87.2)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.84.0
[✓] Connected device (4 available)
• MartiniPhone Pro (mobile) • _hidden_ • ios • iOS 17.4 21E219
• iPhone 15 Pro (mobile) • 97C25645-A5DD-4BAC-8646-2AFED3D23A94 • ios • com.apple.CoreSimulator.SimRuntime.iOS-17-4 (simulator)
• macOS (desktop) • macos • darwin-arm64 • macOS 14.2.1 23C71 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 122.0.6261.129
[✓] Network resources
• All expected network resources are available.
• No issues found!
Activity
huycozy commentedon Mar 15, 2024
I can also reproduce this issue. The test runs on my physical device
iPhone 7, iOS 15.8, but it unexpectedly quits when the counter (from sample code) is around 300.Also, there is an output file named:
Hang (Spindump).txtfrom XCode; its content is only a textNo samples:Hang (Spindump).txt
The complete sample project: reproduce_ios_integration_test.zip
flutter doctor -v (stable and master)
[✓] Flutter (Channel stable, 3.19.3, on macOS 14.1 23B74 darwin-x64, locale en-VN) • Flutter version 3.19.3 on channel stable at /Users/huynq/Documents/GitHub/flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision ba39319843 (6 hours ago), 2024-03-07 15:22:21 -0600 • Engine revision 2e4ba9c6fb • Dart version 3.3.1 • DevTools version 2.31.1 [✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0) • Android SDK at /Users/huynq/Library/Android/sdk • Platform android-34, build-tools 34.0.0 • ANDROID_HOME = /Users/huynq/Library/Android/sdk • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 15.2) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 15C500b • CocoaPods version 1.15.2 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2023.2) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • android-studio-dir = /Applications/Android Studio.app/ • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874) [✓] VS Code (version 1.87.0) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.84.0 [✓] Connected device (3 available) • iPhone (mobile) • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios • iOS 15.8 19H370 • macOS (desktop) • macos • darwin-x64 • macOS 14.1 23B74 darwin-x64 • Chrome (web) • chrome • web-javascript • Google Chrome 122.0.6261.112 [✓] Network resources • All expected network resources are available. • No issues found!Yokoi-K commentedon Mar 21, 2024
I'm running into the exact same problem with my app.
It crashes around the 6 minute mark.
flutter doctor
Keeeeen commentedon Mar 21, 2024
I encountered the same issue with Xcode 15.2.
It consistently crashes within 4 to 8 minutes.
Is there any solution available?
flutter doctor -v
ostk0069 commentedon Mar 21, 2024
this reproduces also in my environment.
Maybe any updates in xctest for xcode 15 cause the issue?
7 remaining items
ivard commentedon Oct 22, 2024
I experience the same problem. I tested it using Flutter 3.24.3. It doesn't make a difference whether I use XCode 15.4 or XCode 16.0. After ~6 minutes the tests stop without any warning or error message.
ivard commentedon Oct 25, 2024
When the tests are pre-empted the last error message is
Runner (528) encountered an error (Test runner never began executing tests after launching). It seems like the test framework thinks the test never started.In the XCode 15 changelog the following is mentioned:
My best guess would be that the bug is related to this change.
Keeeeen commentedon Nov 6, 2024
I'm having trouble updating the version of Xcode. Do you have any suggestions or workarounds?
FujiKinaga commentedon Nov 6, 2024
If you are using flutterfire, XCode 15 or above is starting to become mandatory.
This is a pretty serious state of affairs.
khanhandpad commentedon Nov 27, 2024
same here, should someone help us to resolve this, thanks
Grodien commentedon Mar 4, 2025
Same issue here, anyone found a working solution yet?
bahrimootaz commentedon Apr 17, 2025
Same here.
➜ flutter doctor -v [✓] Flutter (Channel stable, 3.22.2, on macOS 15.3.2 24D81 darwin-arm64, locale en-DE) • Flutter version 3.22.2 on channel stable at ... • Upstream repository https://github.com/flutter/flutter.git • Framework revision 761747bfc5 (11 months ago), 2024-06-05 22:15:13 +0200 • Engine revision edd8546116 • Dart version 3.4.3 • DevTools version 2.34.3 [✓] Android toolchain - develop for Android devices (Android SDK version 32.1.0-rc1) • Android SDK at .... • Platform android-34, build-tools 32.1.0-rc1 • ANDROID_HOME = ..... • Java binary at: ...../java • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 16.2) • Xcode at /Applications/Xcode.app/Contents/Developer • Build 16C5032a • CocoaPods version 1.15.2 [✓] Chrome - develop for the web • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome [✓] Android Studio (version 2023.2) • Android Studio at /Applications/Android Studio.app/Contents • Flutter plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: 🔨 https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874) [✗] Cannot determine if IntelliJ is installed ✗ Directory listing failed [✓] VS Code (version 1.99.2) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.102.0 [✓] Connected device (4 available) • .. [✓] Network resources • All expected network resources are available. ! Doctor found issues in 1 category.Test-Runner-2025.04.17_14-33-39-+0200.xcresult.zip
MSRPRO commentedon Jun 27, 2025
Same here with Xcode version
16.4and Flutter version3.29.3. Who should we tag to get this issue the attention it needs? Our iOS integration tests have been on hold for quite some time now, and we need to re-enable them ASAP.userException commentedon Nov 19, 2025
Making this thread alive again. Still having the same issue with even with Xcode 16.3/26 + Flutter 3.35.7.
Breaking down the tests into smaller chunks is not an option especially for a test cases where we do happy path of a full user journey, like signup, login, do some stuff, logout. It definitely is longer than 5 minutes and breaking this into smaller test cases beats the purpose of testing a happy path in one go without relaunches/re-logins.
Please consider this high priority for production use cases using Flutter.
abhishekbedi1432 commentedon Dec 4, 2025
We are facing similar issue in flutter release 3.34.0
JKennedy24 commentedon Jan 6, 2026
Has anyone tried with Xcode 26?
smallTrogdor commentedon Mar 19, 2026
With the information from @ivard and by looking at the source code, I suspect the issue to be that during XCTestCase construction, the thread is blocked by the runLoop started here until all dart tests have been reported. Before Xcode 15, this was never a problem.
Can the Flutter team confirm this and
maybe move this to P1consider this in the upcoming Swift migration? This issue preventing any integration tests lasting longer than 6 minutes on iOS has been open for > two years - @jmagman ? Thank you for all the previous work done!Can reproduce this on Flutter 3.41.4 and Xcode 26.3.
EDIT
After looking a bit more into past and current issues concerning this plugin, especially these ones #88013 and #71873 the picture became a bit clearer.
I therefore updated my above statement and hope for a consideration of this issue when it is migrated to a Swift implementation.
Fix Xcode 15+ watchdog killing integration tests after ~6 minutes
Grodien commentedon Jun 8, 2026
For everyone still looking for a workaround. You can replace the RunnerTests.m implementation with the following code:
Joaccko0 commentedon Oct 1, 2026
Workaround: survive the construction-phase watchdog without losing per-test XCTest reporting
Tested against a real, production CI test suite (6 modules, 35 testWidgets() tests, ~20 minutes of actual device run time) on BrowserStack App Automate.
Confirming the root cause at the source level
Reading packages/integration_test/ios/integration_test/Sources/integration_test/FLTIntegrationTestRunner.m:
Two things this confirms:
So the fix has to move the blocking wait out of +testInvocations — which is also exactly what @Grodien's workaround does. The only reason it collapses to one result is that +testInvocations has no way to know in advance how many tests exist, or what to call them, without running them first.
The idea: know the test names ahead of time
IntegrationTestPlugin.testResults is a plain NSDictionary<NSString*, NSString*> keyed by the test's description string. That description is knowable statically — it's just the literal first argument to testWidgets(...) in your .dart source. Two things worth calling out:
Given that, the fix is:
Generator script
Ruby, run at build time against your test source files, before the Xcode build. Needs only the standard library (kept it compatible back to the Ruby ~2.6 that ships on macOS, since that's commonly what's available on a bare CI runner without rbenv/rvm — Array#tally, 2.7+, bit me once here; group_by/select work everywhere).
Run it wherever you currently write RunnerTests.m (I generate it in the same CI step that previously did echo 'INTEGRATION_TEST_IOS_RUNNER(RunnerTests)' > RunnerTests.m):
What it produces - Illustrative shape:
Each of those becomes its own XCTest invocation, so BrowserStack (or plain xcresult) shows them as separate rows with real names, same as it already does for Android.
Validation
Ran this against our actual CI suite — 6 feature modules combined into a single app session, 35 testWidgets() tests, via BrowserStack App Automate.
I added two canary tests (one designed to always pass, one designed to always fail) to rule out the possibility that the mechanism only looks correct because nothing had failed yet:
Confirmed visually on the BrowserStack dashboard for this exact run:
test35_CANARY_PASS_... shows green, test36_CANARY_FAIL_...a_proposito shows red and distinct from it, both with their full readable names next to every other real test in the suite (test26_WAL_991... through test34_WAL_1258_..., all green). So both the per-test naming and the pass/fail distinction hold end to end, from the Dart expect() down to the BrowserStack UI.
Limitations