Enhance CI with separate buckets for Scala 2 and 3 - #5145

I'm a little nervous about this much conditional logic in the build. Lots of if: checks in CI like this are sometimes a code smell and could be better handled differently. That said, sometimes this is avoidable.

I agree and it definitely is a code smell. Part of the issue is that we treat test.yml as something that can be build matrixed which is useful to some extent but means there may be a lot of points in the resulting matrix we don't actually care about (we can remove almost all of the if checks but it will result in a lot of redundant/pointless runs). I think if we can change test.yml to handle the build matrix itself, maybe have its inputs be lists instead of individual version, then we can do matrices at a per job level...

Given that this is essentially temporary until we're on Scala 3, I think this is acceptable.

I'm actually of two minds if this should be temporary. On the one hand, we will drop 2.13, but on the other, once on Scala 3 we might want to consider testing against multiple version, e.g. latest LTS and latest. We also might want to test against minimum Java and latest LTS, but of course only some tests should be run this way, not all, thus the weird matrix.

All this being said, despite the code smell I think this is better for the time being, it's certainly a lot faster.