Major pytest 7.3.0 performance regression on Windows #10896

I noticed that after upgrading from pytest 7.2.2 to 7.3.0, the qutebrowser Windows CI got slower by a factor of almost two:

• 28-29 minutes with pytest 7.2.2
• 49-58 minutes with pytest 7.3.0

Running only the tests/unit part locally in a Windows VM:

• 9 minutes with pytest 7.2.2
• 14 minutes with pytest 7.3.0

I'll be trying to reduce this further so I can actually bisect things. Opening this already so that we have a place to discuss if anyone else has seen this already (and so that I have a place for notes, will probably edit this later).