Tool Failures: Instrumentation did not complete

Steps to reproduce

I am unable to find the output of the tests. After I have ran the tests, it just says the tests have been executed successfully regardless of the results of the individual tests even that I know that some have failed and expect to see something like this.

[PASS] Test 1
[FAIL] Test 2
[PASS] Test 3
[PASS] Test 4

I just started the test by patrol test or patrol test --flavor development and waited for the the result of the test such as

group('Dummy tests =>', ()
{
    patrolWidgetTest(
      'counter state is the same after going to home and switching apps',
      config: const PatrolTesterConfig(visibleTimeout: Duration(seconds: 10)),
          ($) async {
        await $.pumpWidgetAndSettle(
          MaterialApp(
            home: Scaffold(
              appBar: AppBar(title: const Text('app')),
              backgroundColor: Colors.blue,
            ),
          ),
        );
        expect($('app'), findsOneWidget);
      },
    );
});

Here goes my MainActivityTest class placed in \android\app\src\androidTest\kotlin\<applicationId> while the MainActivity class is placed in \android\app\src\main\kotlin\<applicationId>. Both of the classes are Kotlin. Secondly, I am unable to use a Java implementation of the MainActivityTest class as provided in your sample app hence it cannot find the Kotlin implementation of the MainActivity class.

package <applicationId>

import androidx.test.rule.ActivityTestRule
import dev.flutter.plugins.integration_test.FlutterTestRunner
import org.junit.Rule
import org.junit.runner.RunWith

@RunWith(FlutterTestRunner::class)
class MainActivityTest {
    @get:Rule
    val rule = ActivityTestRule(MainActivity::class.java, true, false)
}

I am expecting one or more of the following output options

build\app\reports\androidTests\connected\index.html
No tests added to the file even that the file is re-written?
Console
No info about the individual tests listed in the console after execution?
File given by argument to the test execution
No parameter that says where the test results should be listed?

It is also related to: #2066, flutter/flutter#62237, #1170

Note: I am not sure whether it is because it does not produce any output, or because no tests are found even tests actually exists.

Actual results

Patrol Test Output

No device specified, using the first one (emulator-5554)
• Building apk with entrypoint test_bundle.dart...
✓ Completed building apk with entrypoint test_bundle.dart (40.2s)
• Executing tests of apk with entrypoint test_bundle.dart on emulator-5554...
✓ Completed executing apk with entrypoint test_bundle.dart on emulator-5554 (8.8s)

Logs

LogCat

----- begin exception -----
java.lang.RuntimeException: Unable to run tests due to missing activity rule
	at dev.flutter.plugins.integration_test.FlutterTestRunner.run(FlutterTestRunner.java:57)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:67)
	at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:58)
	at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:446)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2402)
----- end exception -----

Patrol version

patrol: ^3.5.0
patrol_finders: ^2.0.1+1
Patrol CLI version: 2.6.1

Patrol Doctor output

Patrol doctor:
Patrol CLI version: 2.6.1
Android:
• Program adb found in AppData\Local\Android\Sdk\platform-tools\adb.exe
• Env var $ANDROID_HOME set to AppData\Local\Android\Sdk

Flutter Doctor output

[√] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
[√] Chrome - develop for the web
[!] Visual Studio - develop Windows apps (Visual Studio Build Tools 2022 17.6.2)
    X Visual Studio is missing necessary components. Please re-run the Visual Studio installer for the "Desktop development with C++" workload, and include these components:
        MSVC v142 - VS 2019 C++ x64/x86 build tools
         - If there are multiple build tool versions available, install the latest
        C++ CMake tools for Windows
        Windows 10 SDK
[√] Android Studio (version 2022.2)
[√] IntelliJ IDEA Community Edition (version 2022.3)
[√] VS Code (version 1.77.3)
[√] Connected device (4 available)
[√] Network resources

Please note that the issue from flutter doctor (see above) can be ignored hence I use Android Studio and not Visual Studio.

UPDATE

I tried to change the runner

// testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

Plus adding the dependencies

androidTestImplementation  'com.android.support.test:rules:0.5'
androidTestImplementation  'com.android.support:support-annotations:25.2.0'
androidTestImplementation  'com.android.support.test:runner:0.5'

This gives the following error that boils down to stack trace with roots in Patrol

StackTrace from LogCat

✗ Failed to execute tests of apk with entrypoint test_bundle.dart on emulator-5554 (Gradle test execution failed with code 1) (15.9s)
Exception: Gradle test execution failed with code 1
#0      AndroidTestBackend.execute.<anonymous closure> (package:patrol_cli/src/android/android_test_backend.dart:140:9)
<asynchronous suspension>
#1      DisposeScope.run (package:dispose_scope/src/dispose_scope.dart:46:7)
<asynchronous suspension>
#2      AndroidTestBackend.execute (package:patrol_cli/src/android/android_test_backend.dart:106:5)
<asynchronous suspension>
#3      TestCommand._execute (package:patrol_cli/src/commands/test.dart:309:7)
<asynchronous suspension>
#4      TestCommand.run (package:patrol_cli/src/commands/test.dart:201:23)
<asynchronous suspension>
#5      CommandRunner.runCommand (package:args/command_runner.dart:212:13)
<asynchronous suspension>
#6      PatrolCommandRunner.runCommand (package:patrol_cli/src/runner/patrol_command_runner.dart:341:18)
<asynchronous suspension>
#7      PatrolCommandRunner.run (package:patrol_cli/src/runner/patrol_command_runner.dart:285:18)
<asynchronous suspension>
#8      patrolCommandRunner (package:patrol_cli/src/runner/patrol_command_runner.dart:70:20)
<asynchronous suspension>
#9      main (file:///C:/Users/jen/AppData/Local/Pub/Cache/hosted/pub.dev/patrol_cli-2.6.2/bin/main.dart:6:20)
<asynchronous suspension>

The reason for changing the runner away from PatrolJUnitRunner was that it was unable to run tests due to missing activity rule.

testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"

StackTrace when using PatrolJUnitRunner

----- begin exception -----
java.lang.RuntimeException: Unable to run tests due to missing activity rule
	at dev.flutter.plugins.integration_test.FlutterTestRunner.run(FlutterTestRunner.java:57)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:67)
	at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:58)
	at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:446)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2402)
----- end exception -----

Sticking to the PatrolJUnitRunner by using the Java version of the MainActivityTest and porting the MainActivity to Java gives the following StackTrace and hangs forever. However, the binding exception was solved by removing IntegrationTestWidgetsFlutterBinding.ensureInitialized(); in top of the main() method in the integration test.

StackTrace when MainActivity and MainActivityTest is Java and PatrolJUnitRunner is used

E  [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: 'package:flutter/src/foundation/binding.dart': Failed assertion: line 153 pos 12: '_debugInitializedType == null': Binding is already initialized to PatrolBinding
#0      _AssertionError._doThrowNew (dart:core-patch/errors_patch.dart:51:61)
#1      _AssertionError._throwNew (dart:core-patch/errors_patch.dart:40:5)
#2      new BindingBase (package:flutter/src/foundation/binding.dart:153:12)
#3      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding (package:flutter_test/src/binding.dart)
#4      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding (package:flutter_test/src/binding.dart)
#5      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding (package:flutter_test/src/binding.dart)
#6      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding (package:flutter_test/src/binding.dart)
#7      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding (package:flutter_test/src/binding.dart)
#8      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding (package:flutter_test/src/binding.dart)
#9      new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding&WidgetsBinding (package:flutter_test/src/binding.dart)
#10     new _TestWidgetsFlutterBinding&BindingBase&SchedulerBinding&ServicesBinding&GestureBinding&SemanticsBinding&RendererBinding&PaintingBinding&WidgetsBinding&TestDefaultBinaryMessengerBinding (package:flutter_test/src/binding.dart)
#11     new TestWidgetsFlutterBinding (package:flutter_test/src/binding.dart)
#12     new LiveTestWidgetsFlutterBinding (package:flutter_test/src/binding.dart)
#13     new IntegrationTestWidgetsFlutterBinding (package:integration_test/integration_test.dart)
#14     IntegrationTestWidgetsFlutterBinding.ensureInitialized (package:integration_test/integration_test.dart:154:7)
#15     main (file:///.../integration_test/app_test.dart:48:40)
#16     Declarer.group.<anonymous closure> (package:test_api/src/backend/declarer.dart:267:37)
#17     _rootRun (dart:async/zone.dart:1399:13)
#18     _CustomZone.run (dart:async/zone.dart:1301:19)
#19     _runZoned (dart:async/zone.dart:1804:10)
#20     runZoned (dart:async/zone.dart:1747:10)
#21     Declarer.declare (package:test_api/src/backend/declarer.dart:169:7)
#22     Declarer.group (package:test_api/src/backend/declarer.dart:264:14)
#23     group (package:flutter_test/src/test_compat.dart:189:13)
#24     main (file:///.../integration_test/test_bundle.dart:69:3)
<asynchronous suspension>

SOLVED

Keep MainActivityTest in Java at android\app\src\androidTest\java\<applicationId>

package <applicationId>;

import androidx.test.platform.app.InstrumentationRegistry;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import pl.leancode.patrol.PatrolJUnitRunner;

@RunWith(Parameterized.class)
public class MainActivityTest {
    @Parameters(name = "{0}")
    public static Object[] testCases() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        instrumentation.setUp(MainActivity.class);
        instrumentation.waitForPatrolAppService();
        return instrumentation.listDartTests();
    }

    public MainActivityTest(String dartTestName) {
        this.dartTestName = dartTestName;
    }

    private final String dartTestName;

    @Test
    public void runDartTest() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        instrumentation.runDartTest(dartTestName);
    }
}

Rewrite MainActivity in Java at android\app\src\main\java\<applicationId>
Removes MainActivity in Kotlin that is placed at android\app\src\main\kotlin\<applicationId>

Kotlin approach does not compute!

Adds the following to the dependencies section in build.gradle

dependencies {
    // ...
    androidTestUtil 'androidx.test:orchestrator:1.4.2'
}

Adds the following to the defaultConfig section in build.gradle

defaultConfig {
    // ...
    testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
    testInstrumentationRunnerArguments clearPackageData: "true"
}

Avoid calling IntegrationTestWidgetsFlutterBinding.ensureInitialized(); in main()
Find the result at build\app\reports\androidTests\connected\index.html as expected, but not in console when all tests pass. However, when failures it is shown with a link to the test overview, but when all tests are passed it will be appreciated if it is visible upfront that they have been found and executed with success.

Note: A better guide that more strictly describes how to setup, and clearly describes what does not work is appreciated. Especially, the Java vs. Kotlin approach as it is pretty confusing hence your sample app mixes those even that it does work. Feel free to use the steps I have outlined above for inspiration to an improved guide.

@nc-jeni

Hi @nc-jeni. Sorry for the late reply. I'm glad that you managed to fix your issue.
I see two action points based on what you've written:

Improve the output of test results in the console (we already have an issue for this),
Makes it more clear in docs how to setup the patrol in project when main android file is MainActivity.kt.
So let's use this issue to track the second one.

Makes it more clear in docs how to setup the patrol in project when main android file is MainActivity.kt. So let's use this issue to track the second one.

Does this require additional change?

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.

Navigation Menu

Repository navigation

Tool Failures: Instrumentation did not complete #2082

Steps to reproduce

Actual results

Logs

Patrol version

Patrol Doctor output

Flutter Doctor output

UPDATE

SOLVED

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tool Failures: Instrumentation did not complete #2082

Description

Steps to reproduce

Actual results

Logs

Patrol version

Patrol Doctor output

Flutter Doctor output

UPDATE

SOLVED

Activity

piotruela commented on Feb 25, 2024

SantoshGunashekar-Freo commented on Aug 6, 2024

github-actions commented on Aug 15, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions