feat(db-autoscaler): Database Horizontal Autoscaler for managed databases #3394

Problem / motivation

Managed databases in Cozystack (postgres, mariadb, redis, mongodb) can only be scaled by hand today: an operator edits the replicas value of the application and waits for the underlying engine to converge. There is no automated way to add or remove read replicas under load — a database saturated with read traffic during business hours and idle at night has to be noticed, hand-tuned, and undone by a human. Reusing a stock HorizontalPodAutoscaler does not fit: it only writes a replica count (for CloudNativePG, Cluster.spec.instances) and is blind to database topology — nothing encodes the synchronous-commit quorum floor (maxSyncReplicas + 1), so it can drive the count below it, and there is no gate on replication lag. Which instance to add or remove, and in what order, is the engine operator's decision (CNPG removes the highest-ordinal standby, never the primary), not something HPA can express.

Proposed solution

Ship an optional, opt-in operator db-autoscaler and a new namespaced CR DatabaseHorizontalAutoscaler (DHA, group autoscaling.cozystack.io/v1alpha1). It automatically adjusts the number of read replicas of a managed database in response to load and applies every decision by patching the Application's replicas value — the Flux-compatible, tenant-facing write path — never the engine CR directly. It reuses existing telemetry (VictoriaMetrics + WorkloadMonitor) and is safe for stateful workloads: it respects the synchronous-replica quorum floor, brakes on replication lag (write-activity gated), uses separate scale-up/scale-down stabilization windows, hands scale-down to the engine operator's graceful instance removal, honours tenant quotas, and offers a dryRun mode with HPA-like status conditions and events. Off by default; enabled via cozystack.db-autoscaler in bundles.enabledPackages.

The architecture is already agreed in the accepted design proposal, so this issue tracks the implementation rather than reopening the design: https://github.com/cozystack/community/tree/main/design-proposals/database-horizontal-autoscaling

Scope check

• This affects a single component / package (if it spans many, consider a design proposal)
• I can describe the expected behavior concretely

Note: this change is deliberately cross-cutting (a new API group, a standalone operator, an admission webhook, and platform wiring), so it does not fit "single component". That is exactly why it went through a design proposal first (linked above); this issue tracks landing the agreed design, not settling an open design question.

Alternatives considered

A controller inside cozystack-controller instead of a standalone operator (rejected for lifecycle isolation and independent release cadence); patching the engine CR / HelmRelease directly (reverted by Flux / bypasses the supported surface); stock HPA + KEDA (topology-unaware: no synchronous-commit quorum floor, no replication-lag gate, cannot express which instance to remove). See the "Alternatives considered" section of the design proposal for the full discussion.

Additional context

Implemented by #3381 (draft). MVP ships the Postgres/CloudNativePG topology adapter with the full guardrail set, a Helm platform package, and a validating admission webhook on the backing Flux HelmRelease (kube-apiserver does not run admission webhooks for the aggregated apps.cozystack.io API). The PR is kept as a draft pending the live-cluster ownership spike and PromQL calibration required by the design proposal's Testing section. Follow-up adapters (mariadb, redis, mongodb) are tracked separately.