Repository navigation
Make E2E use a custom test reporter so that invoking test() will work. #50816
Description
Invoking test() won't result in failures being reported.
We could address this with either
- Improving documentation
- Do something clever with with imports (Dart provides mechanisms for hiding, showing, and aliasing of symbols) to get an app-facing test method that wraps testWidgets without causing compile errors.
- Hook into package:test instead of package:flutter_test at a different layer.
Activity
build: Android support embedding v2
dnfield commentedon Jul 24, 2020
We can fix this by calling
registerReporterfrom package:test_core in the e2e initialization logic. This way we can avoid the need to overriderunTest, and we can catch anypackage:testtests that are run.dnfield commentedon Jul 24, 2020
/cc @natebosch @jiahaog
CareF commentedon Jul 24, 2020
Does it make sense to have a
testWidgetsequivalence for E2E itself?testWidgetsandWidgetTesterlooks more like optimized for unit test, but I think E2E is more like for integration and performance test.#61511 is probably also relevant.
/cc @liyuqian
natebosch commentedon Jul 24, 2020
Performance testing is not a use case targeted by
package:testand you should not trust that it won't subtly impact the way your code performs.dnfield commentedon Jul 24, 2020
We should be able to measure the impact of performance tests based on the difference between package:e2e and package:flutter_driver (which doesn't use package:test on the device itself).
More generally speaking though, e2e is for integration tests, which is broader than just performance tests. If we want to create a new way for e2e to do performance tests specifically that's fine, but we should support running
testas well astestWidgetsin an e2e test.CareF commentedon Jul 24, 2020
So it makes more sense to have something like
perfWidgetstogether withtestWidgetsfor e2e for performance test purposes.My current measurement result is e2e +
testWidgetsis NOT lower thanflutter_driver, if not faster. And in fact I do see cases where e2e is slightly faster thanflutter_driver. https://github.com/CareF/measure_overhead for example.dnfield commentedon Jul 24, 2020
I think we should probably just not worry too much about it until we have a compelling case to worry about it :) If we notice that performance based testing is not stable enough using testWidgets/test then we can address it.
[-]More clearly document that testWidgets is required when using e2e[/-][+]Make E2E use a custom test reporter so that invoking test() will work.[/+]18 remaining items
flutter-triage-bot commentedon Nov 27, 2023
This issue is marked P1 but has had no recent status updates.
The P1 label indicates high-priority issues that are at the top of the work list. This is the highest priority level a bug can have if it isn't affecting a top-tier customer or breaking the build. Bugs marked P1 are generally actively being worked on unless the assignee is dealing with a P0 bug (or another P1 bug). Issues at this level should be resolved in a matter of months and should have monthly updates on GitHub.
Please consider where this bug really falls in our current priorities, and label it or assign it accordingly. This allows people to have a clearer picture of what work is actually planned. Thanks!