The rerunfailures plugin can be used to detect flaky tests and ignore them by trying to rerun previous failed tests. If a repeated run succeeds that is then considered "good enough".
I am interested in the "opposite" feature and thought it might fit into this plugin. To ensure that a test is not flaky I want to run it multiple times. This plugin already supports that use case. In the case where a test fails I don't want the remaining invocations to happen (in order to save time). Using the pytest option -x is not feasible for that case since I am still interested in the result of all different tests and don't want to abort testing all together.
Would an additional option like --stop-repeating-same-test-if-it-failed-once (just a name to describe the semantic) fit into this plugin? If yes, with a little pointer I might be able to provide a pull request if that is helpful.
The rerunfailures plugin can be used to detect flaky tests and ignore them by trying to rerun previous failed tests. If a repeated run succeeds that is then considered "good enough".
I am interested in the "opposite" feature and thought it might fit into this plugin. To ensure that a test is not flaky I want to run it multiple times. This plugin already supports that use case. In the case where a test fails I don't want the remaining invocations to happen (in order to save time). Using the
pytestoption-xis not feasible for that case since I am still interested in the result of all different tests and don't want to abort testing all together.Would an additional option like
--stop-repeating-same-test-if-it-failed-once(just a name to describe the semantic) fit into this plugin? If yes, with a little pointer I might be able to provide a pull request if that is helpful.