Re-evaluate on-device Android native_assets tests

@jtmcdole

Unless I'm missing something, all of these tests could be replaced/would catch the same issues as tests targeting simulators or emulators.

Simulators and emulators can run on presubmit, while device-based tests cannot (due to capacity).

If true, we should turn these down and just use emulator tests (there are none today) and simulator (there is already one for iOS).

flutter/.ci.yaml

Lines 3093 to 3100 in a33b81c

    
             - name: Linux_pixel_7pro native_assets_android 
        
               recipe: devicelab/devicelab_drone 
        
               presubmit: false 
        
               timeout: 60 
        
               properties: 
        
                 tags: > 
        
                   ["devicelab", "android", "linux", "pixel", "7pro"] 
        
                 task_name: native_assets_android

flutter/.ci.yaml

Lines 4848 to 4855 in a33b81c

    
             - name: Mac_arm64_mokey native_assets_android 
        
               recipe: devicelab/devicelab_drone 
        
               presubmit: false 
        
               timeout: 60 
        
               properties: 
        
                 tags: > 
        
                   ["devicelab", "android", "mac", "arm64", "mokey"] 
        
                 task_name: native_assets_android

flutter/.ci.yaml

Lines 5279 to 5286 in a33b81c

    
             - name: Mac_ios native_assets_ios 
        
               recipe: devicelab/devicelab_drone 
        
               presubmit: false 
        
               timeout: 60 
        
               properties: 
        
                 tags: > 
        
                   ["devicelab", "ios", "mac"] 
        
                 task_name: native_assets_ios

flutter/.ci.yaml

Lines 6893 to 6900 in a33b81c

    
             - name: Windows_mokey native_assets_android 
        
               recipe: devicelab/devicelab_drone 
        
               presubmit: false 
        
               timeout: 60 
        
               properties: 
        
                 tags: > 
        
                   ["devicelab", "android", "windows", "mokey"] 
        
                 task_name: native_assets_android

/cc @jtmcdole

@reidbaker

For iOS, we run both device and simulator since correct signing behavior isn't checked in the simulator.

For Android, the emulators run x64 while the devices run arm64. I believe it should be enough to only run on a physical device for a single host OS. Does it matter capacity-wise which host OS runs on a real device?

FYI @reidbaker

Unless a test is trying to test exercising tool code it does not matter if the host os is Linux/Mac/Windows. One issue for moving to emulators is that the tests for emulators are currently disabled because of a KVM issue in luci.

Let's assume KVM will be fixed (it's currently a P0), we won't have emulators turned off forever.

I am interested in having this fail in presubmit, instead of post-submit. See #169798 as an example.

Edited to just be Android - thanks for clarifying.

@matanlurey

I am interested in having this fail in presubmit, instead of post-submit. See #169798 as an example.

Yes, I agree we should have at least one running on the emulator to get presubmit! Thanks for bringing this up @matanlurey!

I do want coverage of at least one physical device due known and unknown differences. I'd like to cover al host OSes. (It wouldn't be the first time your truly does something that doesn't work on Windows. 😄) But I'm fine with having the host OSes covered with either emulator or device. 👍

Should we keep the largest pool hostOS device. Is that Linux? And then switch MacOS and Windows to emulators?

Edit: Or do we have so much emulator capacity that we should make all 3 OSes emulator, and have Linux as device. (So Linux is both device and emulator.)

Emulator capacity on Linux is the cheapest thing we have, there is no special machine pool.

Reland "[native assets] Build dev dependencies in `flutter test integ…

Reverts "Reland "[native assets] Build dev dependencies in `flutter t…

[native assets] Switch device lab Android with Linux host to emulator (…

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

[native assets] Switch device lab Android with Linux host to emulator (…

[native assets] Switch device lab Android with Linux host to emulator…

Navigation Menu

Repository navigation

Re-evaluate on-device Android `native_assets` tests #170682

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	- name: Linux_pixel_7pro native_assets_android
	recipe: devicelab/devicelab_drone
	presubmit: false
	timeout: 60
	properties:
	tags: >
	["devicelab", "android", "linux", "pixel", "7pro"]
	task_name: native_assets_android

	- name: Mac_arm64_mokey native_assets_android
	recipe: devicelab/devicelab_drone
	presubmit: false
	timeout: 60
	properties:
	tags: >
	["devicelab", "android", "mac", "arm64", "mokey"]
	task_name: native_assets_android

	- name: Mac_ios native_assets_ios
	recipe: devicelab/devicelab_drone
	presubmit: false
	timeout: 60
	properties:
	tags: >
	["devicelab", "ios", "mac"]
	task_name: native_assets_ios

	- name: Windows_mokey native_assets_android
	recipe: devicelab/devicelab_drone
	presubmit: false
	timeout: 60
	properties:
	tags: >
	["devicelab", "android", "windows", "mokey"]
	task_name: native_assets_android

Re-evaluate on-device Android native_assets tests #170682

Description

Activity

dcharkes commented on Jun 16, 2025

reidbaker commented on Jun 16, 2025

matanlurey commented on Jun 16, 2025

matanlurey commented on Jun 16, 2025

dcharkes commented on Jun 16, 2025

matanlurey commented on Jun 16, 2025

github-actions commented on Jul 1, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Re-evaluate on-device Android `native_assets` tests #170682