Repository navigation
tests: Don't specify planned number of tests up-front - #6794
Merged
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Collaborator
Author
|
I think it would also be good to split at least the worst offenders - |
Collaborator
|
Yup, sounds all good to me. I would prefer to be consistent and just never use plan_tests then. |
smcv
marked this pull request as draft
August 21, 2026 14:19
smcv
changed the title
Collaborator
Author
Updated accordingly |
smcv
marked this pull request as ready for review
August 21, 2026 14:20
This will help us to emit correct TAP syntax, without having to always declare up-front how many tests we are going to run. Signed-off-by: Simon McVittie <[email protected]>
This is approximately the equivalent of 'plan' in Perl Test::More: it announces how many tests we plan to run. Signed-off-by: Simon McVittie <[email protected]>
This is the equivalent of the function of the same name in Perl's Test::More. If we already emitted a test plan, it asserts that the number of tests we planned to do equals the number we actually did. If not, it assumes that however many tests we have done, that's all of the tests that we intend to do - this can be useful in test scripts that routinely increase in length, like test-run.sh which is becoming rather long (and has frequent conflicts for the "plan" line when we cherry-pick new test coverage to older branches). Signed-off-by: Simon McVittie <[email protected]>
Instead of announcing ahead of time how many tests will be run, just log 1..32 or similar at the end, from the done_testing function. This should go some way towards preventing unnecessary cherry-pick conflicts when tests added to this script get backported. Resolves: flatpak#6793 Signed-off-by: Simon McVittie <[email protected]>
Signed-off-by: Simon McVittie <[email protected]>
swick
approved these changes
Aug 21, 2026
Collaborator
Author
|
Oops, this wasn't quite right. Followup in #6796 |
Merged
smcv
added a commit
to smcv/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
smcv
added a commit
to smcv/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
bbhtt
pushed a commit
to flatpak/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
bbhtt
pushed a commit
to flatpak/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
bbhtt
pushed a commit
to flatpak/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
bbhtt
pushed a commit
to flatpak/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
bbhtt
pushed a commit
to flatpak/flatpak-builder
that referenced
this pull request
Sep 15, 2026
Similar to flatpak/flatpak@b1fdf6c this makes it more obvious where the boundary between test-cases is, and similar to flatpak/flatpak#6794 it avoids needing to declare up-front how many tests we plan to run (which makes conflicts inevitable if we add more than one test to the same script). Signed-off-by: Simon McVittie <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests: Track and emit TAP test numbers in shell script tests
This will help us to emit correct TAP syntax, without having to always
declare up-front how many tests we are going to run.
tests: Add a function to declare the "plan", instead of direct echo
This is approximately the equivalent of 'plan' in Perl Test::More:
it announces how many tests we plan to run.
tests: At the end of each test, explicitly log that we have finished
This is the equivalent of the function of the same name in Perl's
Test::More. If we already emitted a test plan, it asserts that the
number of tests we planned to do equals the number we actually did.
If not, it assumes that however many tests we have done, that's all of
the tests that we intend to do - this can be useful in test scripts
that routinely increase in length, like test-run.sh which is becoming
rather long (and has frequent conflicts for the "plan" line when we
cherry-pick new test coverage to older branches).
tests: Consistently Use the "no plan" style for TAP tests
Instead of announcing ahead of time how many tests will be run, just
log 1..32 or similar at the end, from the done_testing function.
This should go some way towards preventing unnecessary cherry-pick
conflicts when tests added to this script get backported.
Resolves: Cherry-picking new test coverage usually causes conflicts #6793
tests: Drop plan_tests function, no longer used
This is going to be a bit annoying for backports to stable-branches at first, but no worse than the current situation. We could backport these tests improvements (they only touch tests and not production code, and are not too intrusive), or we could accept that 1.18.x is going to continue to be a bit annoying but at least backporting from 1.21.x to 1.20.x will be easier.