Skip to content

bucket: the ACME cleanup hook passes --request-timeout, so it never reaches the apiserver and reports success anyway #4288

Description

@IvanHunters

Describe the bug

packages/apps/bucket/templates/hooks/cleanup-acme.yaml passes --request-timeout=30s on all four of its kubectl calls. A non-zero value for that flag makes kubectl skip its in-cluster credentials: client-go consults them only while the assembled config still equals the default, and the flag lands in that config, so the branch that loads the ServiceAccount token never runs. kubectl talks to its built-in default cluster, localhost:8080, and is refused. This is the same defect as #4276, in a second package.

It fails more quietly here. Every call is wrapped in || echo "WARNING: ..." and the script sets no -e, so the hook exits 0, prints ACME cleanup complete., and the uninstall proceeds. The Job then carries helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded, so Helm deletes it on that success and the warnings go with it. The hook reports the outcome it was written to prevent, and takes its own evidence away.

Environment

  • Cozystack: main
  • Affects packages/apps/bucket

To Reproduce

The flag's effect is measurable directly on the pinned image, with a token and ca.crt mounted and KUBERNETES_SERVICE_HOST set:

flag                     where kubectl went
none                     in-cluster
--request-timeout=30s    localhost:8080
--request-timeout=0      in-cluster

Expected behaviour

The hook lists and deletes the release's ACME Challenges, solver Ingresses and Certificate through its ServiceAccount, and says so.

Actual behaviour

Not observed on a cluster: no e2e suite covers this hook, which is why it is silent rather than caught. What is established is the mechanism above and that all four call sites carry the flag. The expected consequence is that ACME solver objects for a deleted bucket are never cleaned up, with the only diagnostic being warnings inside a Job that is deleted moments later.

Additional context

packages/apps/bucket/tests/cleanup_acme_test.yaml:87 and :94 match the rendered command including --request-timeout=30s, so the suite currently holds the broken form in place: removing the flag turns those two assertions red. They need updating with the fix, not around it.

None of the in-tree hooks that are known to work pass the flag. apps/kubernetes names it in a comment only, apps/harbor and apps/mariadb never mention it, and apps/tenant/templates/cleanup-job.yaml records that its DELETE calls "carry kubectl's default --request-timeout of 0". Bounding the call, which is what the flag was reaching for, is what activeDeadlineSeconds: 120 on this Job already does.

Worth deciding once rather than per package: nothing stops the next hook from reaching for this flag, since it reads as ordinary defensive practice and the manifest renders cleanly either way. A shared assertion, or a lint over rendered hook commands, would cover the class instead of the two instances.

Checklist

  • I have checked the documentation
  • I have searched for similar issues
  • I have included all required information
  • I have provided clear steps to reproduce
  • I have included relevant logs

Activity

  1. added
    kind/bugCategorizes issue or PR as related to a bug
    triage/needs-triageIndicates an issue needs triage by a maintainer
    on Sep 15, 2026
  2. added a commit that references this issue on Sep 17, 2026
    8a6fe02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bugtriage/needs-triageIndicates an issue needs triage by a maintainer

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions