Skip to content

e2e: an empty TIA selection concludes "E2E Tests" green with no suite run #3392

Description

@myasnikovdaniil

hack/select-e2e.sh emits an empty selection for a changed path it does not recognise outside packages/. Both e2e lanes read an empty selection as "nothing to run": the Chainsaw step is skipped and the required E2E Tests status is posted success with zero suites executed. This affects pull-requests.yaml and e2e-fork.yaml equally.

Skipping is the intended behaviour for genuinely inert paths — a PR touching only .gitignore should not burn a full e2e run. The defect is that the silent-ignore branch currently swallows paths that are not inert. full_suite_pattern covers hack/[^/]+\.sh$ and hack/[^/]+\.bats$ but not hack/*.mk, so a change to hack/common-envs.mk — which supplies the tag, push and output flags of every image in the tree — selects nothing and skips e2e entirely. .github/workflows/e2e-fork.yaml is in the same position: pull-requests.yaml is in the pattern, the fork lane's own workflow is not.

A blanket "empty selection means full suite" fallback is the wrong fix, because it escalates exactly the inert paths the design deliberately skips. What this needs is an audit of the currently-ignored paths and an explicit classification:

  • affects every build (hack/*.mk, shared Makefile fragments) → full suite
  • affects an e2e lane itself (.github/workflows/e2e-fork.yaml) → full suite
  • no runtime impact (.gitignore, editor config, renovate.json, *.md outside docs) → keep skipping, but by an explicit rule

Turning the third group into an explicit skip-list rather than a silent default is the part that matters: anything matching neither list would then escalate instead of quietly selecting nothing, so the next path someone adds fails safe rather than fails open.

This is pre-existing behaviour and not a regression from #3262 — it was found while reviewing that PR, at #3262 (comment).

Activity

  1. added
    kind/bugCategorizes issue or PR as related to a bug
    area/ciIssues or PRs related to CI workflows, GitHub Actions, automation
    on Jul 21, 2026
  2. lexfrei commented on Aug 7, 2026

    @lexfrei
    Contributor

    One addition from review work on the fork e2e lane: there are now two consumers of an empty TIA selection, not one. The in-tree e2e job and the fork lane's resolve/e2e pair both read an empty selection as "skip Chainsaw" and still conclude green, so the fix should cover both call sites. The escalate-unclassified change (#3511) closes the in-tree half and, once the fork lane lands (#3262), applies to it as well, since both feed the same hack/select-e2e.sh.

  3. added a commit that references this issue on Aug 8, 2026
    ae4e9e5
  4. added a commit that references this issue on Aug 8, 2026
    68e1e41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/ciIssues or PRs related to CI workflows, GitHub Actions, automationkind/bugCategorizes issue or PR as related to a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions