Repository navigation
Enable writing reference tests #3227
Description
It's difficult to verify that complex trees widgets have been rendered correctly. For example we need to verify that complex app bars - like the gallery's home page - render correctly as the app is scrolled. One way to approach this is with a "reference test". Per a recent discussion:
[from @abarth] In other projects, I've had some success with "reference tests," which work by having the test author provide two ways of drawing the same content. Generally, one that is simple and reliable and another that uses the code being tested. The test framework then checks that you get exactly the same pixels out of both code paths. That approach tends to be a bit easier to maintain as the underlying implementation of how the pixels are drawn evolves.
[from @Hixie] Reference tests would be ok if we're good about building them, but there's definitely an art to making sure you're not testing the same thing twice (and thus not testing it at all). We do definitely need to test paint behaviour, we have nothing testing that today.
Activity
eseidelGoogle commentedon Aug 5, 2016
We have on our quarterly goals to provide a solution for verifying visual correctness with flutter testing. Is this the correct bug for that?
Hixie commentedon Aug 5, 2016
yes
eseidelGoogle commentedon May 4, 2017
We've had trouble with visual regressions not being caught until we release into Google recently. I think we should consider wiring up something like this in May to start enforcing visual stability in a similar way to how we are starting to have api stability.
zoechi commentedon Jun 27, 2018
Is this about what golden tests provide now?
Hixie commentedon Jun 27, 2018
Reference tests are a different kind of test than golden tests, but golden tests are filling a similar need and I don't think we're planning on doing reference tests any time soon so I'll close this.
github-actions commentedon Aug 8, 2021
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.