Skip to content

testing: 89 of 165 packages have no helm test target, including eight user-facing apps #3631

Open

Description

Summary

89 of the 165 packages carrying a Makefile have no test target at all, so make helm-unit-tests never visits them. A green run therefore means "the opted-in half of the catalog passed", not "the catalog passed". The suite runner is honest about its boundary (skipping targetless packages is deliberate and pinned by its tests), but the boundary itself is wider than most readers of a green check assume.

Eight of the 89 are user-facing apps in packages/apps/: foundationdb, http-cache, mongodb, rabbitmq, redis, tcp-balancer, vpc, vpn. These ship to tenants with zero helm-unittest coverage: no render pin, no values-contract test, nothing that would redden on a template regression.

Counted with the runner's own detection gate over the current tree: 165 Makefiles, 76 with a working test target, 89 without.

Fix shape

Per chart, not per script: add a tests/ suite and a test: target to each chart, starting with the eight user-facing apps. Each one is a self-contained first contribution: render the chart with default values, pin the objects a tenant relies on, and wire the standard test: target the sibling charts already use. The detection work that makes missing suites loud (a chart with a test target must actually report suites) already guards the opted-in half, so every chart added here stays protected once it opts in.

Activity

  1. added
    communityCommunity contributions are welcome in this issue
    kind/cleanupCategorizes issue or PR as related to cleanup of code, process, or technical debt
    area/testingIssues or PRs related to testing (e2e, bats, unit tests)
    on Aug 7, 2026
  2. lexfrei commented on Sep 10, 2026

    @lexfrei
    ContributorAuthor

    Three of the 89 carry an extra hazard worth using to order the work. Their update recipe clears their own templates/ before refetching, so a first-party file added there is deleted by the next make update, which exits 0. With no suite in the package, nothing downstream reddens and the loss reaches a cluster.

    Measured at origin/main, f7cb9e785:

    REF=origin/main
    for mk in $(git ls-tree -r --name-only "$REF" -- packages | grep -E '^packages/[^/]+/[^/]+/Makefile$'); do
      body=$(git show "$REF:$mk")
      w=$(printf '%s\n' "$body" | awk '/^update:/ {r=1; next}
           r && /^\t[[:space:]]*@?#/ {next}
           r && /^\t/ {print; next}
           r && /^[[:space:]]*$/ {next}
           r && /^[[:space:]]*#/ {next}
           r {exit}' | grep -cE 'rm[[:space:]]+-(rf|fr|r)[[:space:]]+(\./)?templates(/\*?)?([[:space:]]|;|$)')
      [ "$w" -gt 0 ] || continue
      printf '%s\n' "$body" | grep -qE '^test:' || echo "$mk"
    done

    Nine packages clear their own templates/ at that ref. Three of them have no test target: kubevirt-cdi-operator, kubevirt-instancetypes, objectstorage-controller.

    The ref is in the command because the count moves with the branch you run it on. kubevirt-cdi is the ninth wiper at origin/main and stops being one when #4097 lands, so a run inside a checkout of that branch gives eight.

    Not a separate issue, and not the same as #4206 or #4119: those are about a guard that cannot see a loss, this is about a package that has no suite to report one. It just says which three of the 89 to do first.

  3. added
    triage/acceptedIndicates an issue is ready to be actively worked on
    priority/backlogGeneral backlog priority. Lower than priority/important-longterm
    and removed
    triage/needs-triageIndicates an issue needs triage by a maintainer
    on Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/testingIssues or PRs related to testing (e2e, bats, unit tests)communityCommunity contributions are welcome in this issuegood first issueGood for newcomerskind/cleanupCategorizes issue or PR as related to cleanup of code, process, or technical debtpriority/backlogGeneral backlog priority. Lower than priority/important-longtermtriage/acceptedIndicates an issue is ready to be actively worked on

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions