Repository navigation
"Attempted to insert a duplicate test item ID" when modifying tests after running them using testGoldens from golden_toolkit #3776
Description
import 'package:golden_toolkit/golden_toolkit.dart';
import 'package:test/test.dart';
void main() {
group('foo', () {
testGoldens('test 1', (tester) async {});
});
}Then Test: Run All Tests, wait for results, then try to modify the file (eg. add a new linebreak):
Activity
[-]"Attempted to insert a duplicate test item ID" when modifying tests after running them[/-][+]"Attempted to insert a duplicate test item ID" when modifying tests after running them using testGoldens from golden_toolkit[/+]DanTup commentedon Jan 11, 2022
This looks like it's caused by
@isTestGroupon thetestGoldensfunction. We get a test group from the discovered that uses the outline, and a test with the same name from the results.The package should be updated, but we should not crash in this case. We could use ID formats for groups/tests to avoid colliding like this.
DanTup commentedon Jan 11, 2022
Looks like this method used to create a group, but was changed in eBay/flutter_glove_box@f1d486c.
I've filed a PR for the annotation at eBay/flutter_glove_box#140.
up-doc commentedon Jan 11, 2022
@DanTup - even without the golden tests being inside the group, we get some console errors.
Perhaps the overall number of
testGoldensI have in our work project is what's causing the testing status, code lens and test count to be flaky or for the most part wrong?I've just tested my work code after commenting out all the
testGoldencalls. All 229 of them. And ran the tests a number of times. The count appears to pick up the tests. No errors are coming up on the console. All seems a lot better.So I think this might be promising. :)
DanTup commentedon Jan 11, 2022
@moonytoes29
Sorry, the group wasn't necessary - it's just the code I started with when I triggered it. The important of groups is that the
testGoldensfunction is annotated with@isTestGroupand that causes us to believe there is a group with the name you supplied here at analysis time. But when you run the tests, we now get a set of test results that show a test with that name. We have to merge data from both analysis and runtime to build that tree, so we end up creating two nodes with the same ID (one is the group from the analyzer data, and one the test from the results).The fix I made in VS Code says that groups and tests always have separate IDs (eg. they are prefixed with
GROUP:andTEST:) which ensures we can never create collisions like this that will cause things to fail. However, it means that (until thetestGoldensfunction is fixed), you'll see two nodes for the tests in the tree (one from analysis, one from test results) which is ofcourse a little odd.If you switch to the pre-release channel noted above, it will resolve the errors. Another option is to modify your source code for
golden_toolkitlocally to be@isTestinstead of@isTestGroup, although modifying things in your pubcache is a little weird :-)DanTup commentedon Jan 11, 2022
@moonytoes29 a new version of golden_toolkit was published with the annotation fix. If you can upgrade to that, everything should be good now :-)
up-doc commentedon Jan 12, 2022
Wish I could. But alongside your simple fix, they've now made that latest version dependent on having SDK >=2.15. Which my project is not!.
So I'll have to do a fork to apply just your fix for now. Then look to migrate sooner rather than later.
DanTup commentedon Jan 12, 2022
Ah, that's a shame. Although if it's just you and not a team, I'd probably just modify the file in the pub cache to have
@isTest. I don't think that's likely to change anything except fix this (otherwise, if you use the pre-release extension, it at least won't stop you running tests, you'll just have some repeated nodes in the tree).