Repository navigation
Support native view screenshot in e2e integration tests #83856
Description
Integration tests currently returns a Skia raster image.
This is fine for Flutter widgets, but it misses platform views.
As a result, customer: money or dream cannot test platform views.
Plugins like google_mobile_ads cannot have their own driver tests
that ensure that the native view is sized properly, etc...
We also happen to have the scenario app in the engine, which has
a workaround for this limitation. This workaround breaks occasionally,
and doesn't work with Skia Gold.
For all of these issues, I propose adding a new API to Flutter drive that takes the
device's screenshot and crops the status or action bars if needed
(This will be required on Android if adb is used).
cc @dnfield, @xster, @jmagman for iOS too.
Related bug: http://b/180715355
Activity
dnfield commentedon Jun 3, 2021
How will we deal with synchronization issues?
blasten commentedon Jun 3, 2021
what is an example of a "synchronization issue" ?
dnfield commentedon Jun 3, 2021
Making sure the platform view has finished loading/animating before taking a screenshot
blasten commentedon Jun 3, 2021
oh I see. If
SurfaceViewis usedandroid.view.Choreographercannot be used. IfFlutterImageViewis used, we could schedule the screenshot the next Android frame. Luckily, this is already the case with platform views that use HC.blasten commentedon Jun 3, 2021
Also, at this time we have merged the threads, so the raster and the platform threads are the same.
I was looking at the 2s delay described in the public docs. I think if we get the next frame callback from the platform, and the raster and platform threads are merged, we should be able to remove the 2s if wanted to.
On Android at least, the only issue is that SurfaceView/SurfaceTextureView aren't synchronized to the Android view hierarchy.
jiahaog commentedon Jun 3, 2021
Is this going to be an API on the Flutter Driver host tools or within the Flutter Driver extension? The latter would be preferable so that it can be reused for
package:integration_testas well.jmagman commentedon Jun 3, 2021
I added
flutter drive --screenshotto the tool in #78822 but it only fires when drive fails. They are uploaded in our LUCI tests.For example: https://ci.chromium.org/p/flutter/builders/prod/Mac_ios%20integration_ui_ios_textfield/1060
Shows the device looked like this when it failed:
jmagman commentedon Jun 4, 2021
@blasten in light of this, what additional functionality is this feature tracking?
blasten commentedon Jun 4, 2021
oh
flutter drive --screenshotis more for debugging, right? This is tracking an API that can generate images from a driver test to use as baseline for future invocations of the test.jmagman commentedon Jun 4, 2021
If you want the platform views/keyboards, etc it should work.
--screenshotis using the tool's platform-specificflutter screenshotlogic, so usesadb,idevicescreenshot, fuchsia'sscreencap, custom device registered hooks, etc.It doesn't have your "crops the status or action bars if needed" feature, though (not sure how that would work). As long as the same simulator version is used every time to compare goldens you probably don't need to crop it (that's what the scenario app does). I may be missing some additional features though.
One thing that would be really nice is for the test itself to be able to trigger screenshots or screen recordings (tap button, screenshot, start screen recording, scroll, stop screen recording, tap button, screenshot), which would maybe would involve a VM service hook the tool responds to? Not sure...
8 remaining items
blasten commentedon Jun 7, 2021
Sounds good.
On a separate note, flutter-team-archive/engine#26602 could use this feature. Once we increase API level coverage, we will be able to ensure that input focus and keyboard interactions work as expected.
blasten commentedon Jun 12, 2021
#56591 appears to be a blocker for this. Currently, Gradle test-only dependencies end up in the release APK. cc @jonahwilliams
jonahwilliams commentedon Jun 12, 2021
I don't see why #56591 would be a blocker - assuming you're already using the integrtion_test package then you already have that problem?
blasten commentedon Jun 14, 2021
Right. The release APK contains test-only dependencies as seen below.
AGP has a configuration (
androidTestImplementation) that generates different dependency graphs depending on the task, but it requires a different directory layout than the one used by plugins public APIs. e.g. androidTest/java, etc...Sunbreak commentedon Aug 6, 2021
A lot of related issues are locked: #25306
Do we have any progress here?
tvolkert commentedon Aug 6, 2021
@Sunbreak this is actively being worked on.
Pulkit07 commentedon Mar 11, 2022
Hey, we are looking to take screenshots of camera view using screenshot plugin but are unable to do so because of this issue. Are there any updates on this?
Thanks for all the work and amazing framework!
jmagman commentedon Mar 18, 2022
https://github.com/flutter/flutter/tree/master/packages/integration_test#screenshots
@blasten do you know what is still needed for this feature?
blasten commentedon Mar 19, 2022
This is functional on Android. As far as Android support is concerned, this is done.
Unless there's anything pending from iOS, this issue can be closed.
We are tracking another issue #97853 about adding tests that need this e2e screenshot functionality.
@Pulkit07 what issue did you run into?
github-actions commentedon Apr 2, 2022
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.