Feat/webhook workload monitors - #1448

packages/extra/monitoring/images/grafana.tag (1)

1-1: Avoid using the latest tag; prefer explicit version tag or digest-only

Replace the reference in packages/extra/monitoring/images/grafana.tag (uses :latest with a pinned digest) with a digest-only or explicit version reference:

-ghcr.io/cozystack/cozystack/grafana:latest@sha256:cc1843297f2dadb1740ebc4dfdc3dbe5b4ac840d1da37ed6fe93549d3000196c
+ghcr.io/cozystack/cozystack/grafana@sha256:cc1843297f2dadb1740ebc4dfdc3dbe5b4ac840d1da37ed6fe93549d3000196c

packages/system/cozystack-api/values.yaml (1)

2-2: Avoid :latest tag; keep digest-only to prevent AlwaysPull and ambiguous rollouts.

Using :latest forces imagePullPolicy: Always by default and can surprise rollouts/rate‑limit pulls. You already pin by digest—drop the tag for reproducible, predictable behavior.

-  image: ghcr.io/cozystack/cozystack/cozystack-api:latest@sha256:be6aca4df4b769538454d6c65b7045b2bef81b2a15d70eeddbfdf55d6bd17d6c
+  image: ghcr.io/cozystack/cozystack/cozystack-api@sha256:be6aca4df4b769538454d6c65b7045b2bef81b2a15d70eeddbfdf55d6bd17d6c

packages/system/objectstorage-controller/values.yaml (1)

3-3: Avoid ":latest" tags in production charts; keep semver tag + digest for traceability.

"latest" obscures the running version in labels/telemetry. Prefer a specific version tag pinned by digest.

Confirm the digest corresponds to the intended controller release, and consider restoring a versioned tag.

packages/apps/clickhouse/images/clickhouse-backup.tag (1)

1-1: Use versioned tag rather than "latest" for backup tooling images.

Backups/restores are sensitive to version drift. Keep a semver tag with the digest for deterministic rollbacks and audits.

packages/system/seaweedfs/values.yaml (1)

121-121: Sidecar image should use explicit versioned tag + digest, not "latest".

Prevents ambiguous versioning in labels and improves SBOM/compliance tracking.

packages/system/cilium/values.yaml (1)

17-18: Cilium image tag set to "latest" while digest is pinned — prefer semver tag for clarity and compatibility checks.

Many components/readiness checks and labels surface the tag; "latest" can mask real version, hindering debugging and drift detection. Keep a concrete version tag with the digest.

Make sure chart/cluster features configured here are compatible with the actual digest’s Cilium version.

packages/core/installer/values.yaml (1)

2-2: Installer image should remain on a versioned tag for reproducibility.

Switching to "latest" reduces ability to bisect/rollback installs reliably. Recommend semver tag + digest.

packages/system/dashboard/charts/kubeapps/templates/dashboard/configmap.yaml (1)

79-79: Don’t hardcode appVersion to "latest" in user‑facing config; derive from chart/image.

This degrades UX and debug value. Use the chart/image version templating already available in this chart.

Apply this change to keep it consistent with labels:

-      "appVersion": "latest",
+      "appVersion": {{ include "common.images.version" ( dict "imageRoot" .Values.dashboard.image "chart" .Chart ) | toJson }},

packages/system/kubevirt-csi-node/values.yaml (1)

3-3: Pin kubevirt-csi-driver to a specific version tag with digest; avoid "latest".

CSI components are critical; explicit versions help with compatibility and incident response.

packages/system/kubeovn/values.yaml (1)

67-67: Replace 'latest' tags (even with digest) with semver tags + digest

Using latest@sha256:... pins content but keeps the "latest" tag in labels/tooling — use a versioned tag (e.g., vX.Y.Z@sha256:0891335d...) for provenance, rollbacks and SBOMs.
Change packages/system/kubeovn/values.yaml (line 67)
tag: latest@sha256:0891335d3938dc604b47ae310f9bea815c352c15bea34dee111526c6aa6a48de
to a semver tag with the same digest; repo-wide grep returned many other latest occurrences — update them or gate latest behind a dev overlay.

packages/apps/http-cache/images/nginx-cache.tag (1)

1-1: Avoid using the "latest" tag even with a pinned digest for traceability and pull behavior.

• "latest" defaults imagePullPolicy to Always, which can cause unnecessary pulls even though the digest is immutable.
• Prefer an explicit version tag matching the digest, or use repo@sha256 without a tag if supported by your tooling.

Can you confirm the digest corresponds to the intended release (and that you want Always pulls here)?

packages/core/testing/values.yaml (1)

2-2: e2e image set to "latest": confirm intent and stability.

Pinned digest ensures reproducibility, but "latest" will still default pullPolicy to Always. For CI stability and speed, consider an explicit version tag or explicitly setting pullPolicy in the chart.

Is Always pull desired in your CI runners for this image?

packages/extra/seaweedfs/images/objectstorage-sidecar.tag (1)

1-1: Prefer versioned tag or digest-only to avoid "latest".

• Keeps provenance clear in audits and release notes.
• Avoids implicit Always pulls from using "latest".

Please confirm the digest maps to the target release.

packages/extra/bootbox/images/matchbox.tag (1)

1-1: Use an explicit version tag (or digest-only) rather than "latest".

Improves auditability and avoids implicit Always pulls.

Confirm the digest is the intended version.

packages/apps/kubernetes/images/ubuntu-container-disk.tag (1)

1-1: For VM base images, avoid "latest" to prevent unexpected pull behavior and ease provenance.

• Prefer a semver tag aligned with the digest or digest-only.
• If consumers run this with "latest", Kubernetes will default to Always pulling unless overridden.

Is the consuming chart setting pullPolicy explicitly for this image?

packages/system/dashboard/values.yaml (1)

51-52: kubeapps-apis: avoid latest — set pullPolicy or use a versioned tag.

packages/system/dashboard/values.yaml sets kubeappsapis.image.tag: latest (digest present). Prefer a fixed version tag or add an explicit pullPolicy so imagePullPolicy is deterministic (templates read .Values.kubeappsapis.image.pullPolicy).

Suggested change:
image:
registry: ghcr.io/cozystack/cozystack
repository: kubeapps-apis
tag: latest
digest: "sha256:7f6a822c80dd84ee77276183b59ffe073ececff056bddd71e686b2d542f6fde8"
pullPolicy: IfNotPresent

cmd/cozystack-controller/main.go (1)

218-231: Registering the webhook + controller: LGTM; add explicit webhook server settings and a success log.

• Minor: log successful setup to ease troubleshooting.
• Recommended: set explicit Port and CertDir for the webhook server (defaults work but being explicit avoids chart/template drift).

Apply this within the selected block to add a success log:

 if err := lineageControllerWebhook.SetupWithManagerAsWebhook(mgr); err != nil {
   setupLog.Error(err, "unable to setup webhook", "webhook", "LineageWebhook")
   os.Exit(1)
 }
+setupLog.Info("Lineage webhook and controller registered")

Additionally, set explicit webhook server options (outside this block, around Lines 128-131):

webhookServer := webhook.NewServer(webhook.Options{
    TLSOpts: tlsOpts,
    Port:    9443,
    CertDir: "/tmp/k8s-webhook-server/serving-certs",
})

Ensure your Helm chart mounts certs to the same path or update CertDir accordingly.

Can you confirm the chart/template mounts serving certs at /tmp/k8s-webhook-server/serving-certs (or update CertDir to match)?

packages/apps/kubernetes/images/kubevirt-csi-driver.tag (1)

1-1: Avoid :latest tag; use digest-only or a versioned tag + digest.

Using latest@sha256:... is confusing and can mislead humans/tools. Prefer either digest-only or a specific version tag plus the digest.

Apply:

-ghcr.io/cozystack/cozystack/kubevirt-csi-driver:latest@sha256:35684db82ef7d5ad0cb39fa2568aa09cf60c50f3307099dbb89521aefad5baf6
+ghcr.io/cozystack/cozystack/kubevirt-csi-driver@sha256:35684db82ef7d5ad0cb39fa2568aa09cf60c50f3307099dbb89521aefad5baf6

pkg/lineage/lineage_test.go (1)

29-42: Avoid init-time real cluster dependencies in unit tests.

init() with GetConfigOrDie() and real discovery makes this a manual/integration test. Consider moving setup into the test and using envtest or the dynamic fake client with a static RESTMapper.

packages/system/cozystack-controller/templates/deployment.yaml (1)

31-42: Add basic probes and hardening for the webhook container.

Readiness/liveness on 9443 (tcp) improves rollout safety; minimal securityContext/resources help harden defaults.

Apply:

         - name: webhook
           containerPort: 9443
+        readinessProbe:
+          tcpSocket:
+            port: 9443
+          initialDelaySeconds: 2
+          periodSeconds: 10
+        livenessProbe:
+          tcpSocket:
+            port: 9443
+          initialDelaySeconds: 10
+          periodSeconds: 20
         volumeMounts:
           - name: webhook-certs
             mountPath: /tmp/k8s-webhook-server/serving-certs
             readOnly: true
+        securityContext:
+          runAsNonRoot: true
+          allowPrivilegeEscalation: false
+          readOnlyRootFilesystem: true
+        resources:
+          requests:
+            cpu: 50m
+            memory: 64Mi
+          limits:
+            cpu: 500m
+            memory: 256Mi
       volumes:
         - name: webhook-certs
           secret:
             secretName: cozystack-controller-webhook-cert
             defaultMode: 0400

internal/lineagecontrollerwebhook/controller.go (2)

12-12: RBAC: include get to be safe.

Controller cache typically needs list/watch; adding get is a low-cost hedge if future code uses direct Get.

Apply:

-// +kubebuilder:rbac:groups=cozystack.io,resources=cozystackresourcedefinitions,verbs=list;watch
+// +kubebuilder:rbac:groups=cozystack.io,resources=cozystackresourcedefinitions,verbs=get;list;watch

---

23-23: Make the namespace configurable with a default.

Hard-coding "cozy-system" limits portability.

Apply:

- if err := c.List(ctx, crds, &client.ListOptions{Namespace: "cozy-system"}); err != nil {
+ ns := "cozy-system"
+ if v := ctrl.GetConfigOrDie(); v != nil {
+   // optionally: pick from env var if provided
+   if n := os.Getenv("COZYSTACK_SYSTEM_NAMESPACE"); n != "" {
+     ns = n
+   }
+ }
+ if err := c.List(ctx, crds, &client.ListOptions{Namespace: ns}); err != nil {

Add os to imports.

internal/lineagecontrollerwebhook/matcher.go (1)

8-15: Handle nil selectors and log invalid LabelSelectors.

Avoid treating nil as match-all and emit a warning on parse errors to aid CRD debugging.

Apply:

 func matchLabelsToSelector(l map[string]string, s *metav1.LabelSelector) bool {
-	// TODO: emit warning if error
-	sel, err := metav1.LabelSelectorAsSelector(s)
-	if err != nil {
-		return false
-	}
-	return sel.Matches(labels.Set(l))
+	if s == nil {
+		return false
+	}
+	sel, err := metav1.LabelSelectorAsSelector(s)
+	if err != nil {
+		log.Log.Error(err, "invalid LabelSelector in CozystackResourceDefinition", "selector", s)
+		return false
+	}
+	return sel.Matches(labels.Set(l))
 }

And add import:

 import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/labels"
+	"sigs.k8s.io/controller-runtime/pkg/log"
 )

packages/system/cozystack-controller/templates/mutatingwebhookconfiguration.yaml (2)

22-22: Style: add spaces after commas in the list.

Resolves yamllint warning; improves readability.

-        resources: ["pods","secrets", "services", "persistentvolumeclaims"]
+        resources: ["pods", "secrets", "services", "persistentvolumeclaims"]

---

18-27: Consider adding timeoutSeconds and objectSelector guardrails.

• Set timeoutSeconds (e.g., 10) explicitly.
• If your webhook skips SA token secrets internally, fine; otherwise consider an objectSelector to reduce blast radius (labels only; SA token detection likely needs in-handler checks).

Can you confirm the webhook handler explicitly ignores service account token secrets (type kubernetes.io/service-account-token) and other control-plane managed objects?

internal/lineagecontrollerwebhook/config.go (1)

39-43: Key may collide across SourceRef namespaces.

chartRef uses only SourceRef.Name and Chart. If multiple HelmRepository/OCIRepository objects share the same Name in different namespaces, mappings will collide.

Consider including SourceRef.Namespace in chartRef (and the controller that populates the map) to avoid collisions.

packages/system/cozystack-controller/templates/certmanager.yaml (1)

43-45: Add FQDN to DNS names for webhook TLS.

Including the cluster-local FQDN improves compatibility with resolvers that use FQDNs.

   dnsNames:
     - cozystack-controller
     - cozystack-controller.{{ .Release.Namespace }}.svc
+    - cozystack-controller.{{ .Release.Namespace }}.svc.cluster.local

api/v1alpha1/cozystackresourcedefinitions_types.go (1)

93-103: Validation hints for label selectors (optional).

If you want to enforce non-empty items: add kubebuilder validation (MinItems) to Include/Exclude arrays. Optional.

internal/lineagecontrollerwebhook/types.go (1)

14-14: Align webhook name with deployed MWC (optional).

Marker uses name=mlineage.cozystack.io while Helm MWC uses lineage.cozystack.io. If you rely on generated manifests elsewhere, consider aligning to avoid duplicate MWCs.

-// +kubebuilder:webhook:path=/mutate-lineage,mutating=true,failurePolicy=Fail,sideEffects=None,groups="",resources=pods,secrets,services,persistentvolumeclaims,verbs=create;update,versions=v1,name=mlineage.cozystack.io,admissionReviewVersions={v1}
+// +kubebuilder:webhook:path=/mutate-lineage,mutating=true,failurePolicy=Fail,sideEffects=None,groups="",resources=pods,secrets,services,persistentvolumeclaims,verbs=create;update,versions=v1,name=lineage.cozystack.io,admissionReviewVersions={v1}

pkg/lineage/lineage.go (2)

88-91: Use structured logger instead of stderr.

Keeps logs consistent with controller-runtime and avoids noisy stderr writes.

-			fmt.Fprintf(os.Stderr, "Could not fetch owner %s/%s (%s): %v\n", obj.GetNamespace(), owner.Name, owner.Kind, err)
+			l.Error(err, "could not fetch owner",
+				"namespace", obj.GetNamespace(), "name", owner.Name, "kind", owner.Kind)

Also remove the unused os import after this change.

---

117-121: Optional: log when prefix trimming is ineffective.

HelmRelease name not starting with prefix results in a likely miss; a debug log helps troubleshooting.

-		ownerObj, err := getUnstructuredObject(ctx, client, mapper, a, k, obj.GetNamespace(), strings.TrimPrefix(obj.GetName(), p))
+		targetName := strings.TrimPrefix(obj.GetName(), p)
+		if targetName == obj.GetName() {
+			l.V(1).Info("helmrelease name did not have expected prefix", "name", obj.GetName(), "prefix", p)
+		}
+		ownerObj, err := getUnstructuredObject(ctx, client, mapper, a, k, obj.GetNamespace(), targetName)

internal/lineagecontrollerwebhook/webhook.go (3)

73-89: Simplify error handling; remove redundant for-loop.

Logic is single-pass; a straight if/switch is clearer.

-	labels, err := h.computeLabels(ctx, obj)
-	for {
-		if err != nil && errors.Is(err, NoAncestors) {
-			return admission.Allowed("object not managed by app")
-		}
-		if err != nil && errors.Is(err, AncestryAmbiguous) {
-			warn = append(warn, "object ancestry ambiguous, using first ancestor found")
-			break
-		}
-		if err != nil {
-			logger.Error(err, "error computing lineage labels")
-			return admission.Errored(500, fmt.Errorf("error computing lineage labels: %w", err))
-		}
-		if err == nil {
-			break
-		}
-	}
+	labels, err := h.computeLabels(ctx, obj)
+	if err != nil {
+		if errors.Is(err, NoAncestors) {
+			return admission.Allowed("object not managed by app")
+		}
+		if errors.Is(err, AncestryAmbiguous) {
+			warn = append(warn, "object ancestry ambiguous, using first ancestor found")
+		} else {
+			logger.Error(err, "error computing lineage labels")
+			return admission.Errored(500, fmt.Errorf("error computing lineage labels: %w", err))
+		}
+	}

---

145-148: Handle potential error from SetNestedStringMap (best-effort).

Consider capturing/logging the error to aid debugging; current code silently ignores failures.

Example:

if err := unstructured.SetNestedStringMap(o.Object, labels, "spec", "managedMetadata", "labels"); err != nil {
	// TODO: add logging if a logger is available here
}

---

27-27: Comment/name mismatch.

Update the comment to match the function name.

-// SetupWithManager registers the handler with the webhook server.
+// SetupWithManagerAsWebhook registers the handler with the webhook server.

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Learnt from: lllamnyp
PR: cozystack/cozystack#1400
File: packages/system/cozystack-controller/templates/certmanager.yaml:39-45
Timestamp: 2025-09-11T16:05:52.112Z
Learning: In Cozystack, `cluster.local` is not the cluster domain, and the Kubernetes API server doesn't use the cluster domain when contacting webhooks. DNS names for webhook certificates should match the actual service names without assuming `cluster.local` suffix.