[Feature Request] Support the scale sub-resource for Clusters

We would like to utilize the vertical pod autoscaler to automatically scale the PG pods in the Cluster. The VPA can support managing pods of custom resources as long as they implement the scale sub-resource. It would be great to have that for the Cluster CRD.

Example VPA Manifest

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  creationTimestamp: "2023-08-11T19:53:06Z"
  generation: 2
  name: primary-api-pg
  namespace: primary-api
  resourceVersion: "5833222"
  uid: 307a843d-9891-4a0c-8a4c-b6c05827697e
spec:
  targetRef:
    apiVersion: postgresql.cnpg.io/v1
    kind: Cluster
    name: primary-api-pg
  updatePolicy:
    updateMode: Auto

Current Error

Cannot get target selector from VPA's targetRef. Reason: Unhandled targetRef postgresql.cnpg.io/v1 / Cluster / primary-api-pg, last error Resource primary-api/primary-api-pg has an empty selector for scale sub-resource

More info on the implementation spec here.

Good point! Looks like the only thing we are missing is the selectorpath

Need this also. Can I propose a PR ?

Here is my workaround:

disable cnpg webhooks, here are the values passed to the official cnpg chart

webhook:
    mutating:
      create: false
    validating:
      create: false

add a kyverno rule to mutate the cluster crd and the clusters

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: cnpg-cluster-scale-selector
spec:
  background: true
  generateExisting: true
  mutateExistingOnPolicyUpdate: true
  useServerSideApply: true
  rules:
    - name: add-spec-selector-to-crd
      match:
        resources:
          kinds:
            - CustomResourceDefinition
          names:
            - clusters.postgresql.cnpg.io
      mutate:
        targets:
          - apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            name: clusters.postgresql.cnpg.io
        patchStrategicMerge:
          spec:
            versions:
              - (name): v1
                schema:
                  openAPIV3Schema:
                    type: object
                    served: true
                    storage: true
                    properties:
                      spec:
                        properties:
                          selector: 
                            type: string
                subresources:
                  scale:
                    labelSelectorPath: .spec.selector
                    specReplicasPath: .spec.instances
                    statusReplicasPath: .status.instances
    - name: add-spec-selector-to-clusters
      match:
        resources:
          kinds:
            - postgresql.cnpg.io/v1/Cluster
      mutate:
        targets:
          - apiVersion: postgresql.cnpg.io/v1
            kind: Cluster
            name: {{ request.object.metadata.name }}
        patchStrategicMerge:
          spec:
            selector: "cnpg.io/cluster={{ request.object.metadata.name }}"

and my vpa:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-vpa
spec:
  targetRef:
    apiVersion: postgresql.cnpg.io/v1
    kind: Cluster
    name: my-pg-cluster
  updatePolicy:
    updateMode: "Off"

and I can see the provided recommendations from the vpa

tada !!!

I finally had problem caused by this issue: #3753 so I reactived webhook and removed a part using the following manual patch
and after upgraded cnpg to latest version the kyverno policy part that update the crd didn't work anymore, so
here is my tricky solution for now

kubectl -n cnpg patch crd clusters.postgresql.cnpg.io --type json --patch-file patches/cluster-crd.patch.yaml
kubectl patch mutatingwebhookconfiguration cnpg-mutating-webhook-configuration --type=json  --patch-file patches/webhook.patch.yaml

patches/cluster-crd.patch.yaml:

[
  {
    "op": "add",
    "path": "/spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/selector",
    "value": { "type": "string" },
  },
  {
    "op": "add",
    "path": "/spec/versions/0/subresources/scale/labelSelectorPath",
    "value": ".spec.selector",
  },
]

patches/webhook.patch.yaml:

[{ "op": "remove", "path": "/webhooks/1" }]

and in argocd applicationset set.templatate.spec:

ignoreDifferences:
        - group: admissionregistration.k8s.io
          kind: MutatingWebhookConfiguration
          jsonPointers:
            - /webhooks
        - group: apiextensions.k8s.io
          kind: CustomResourceDefinition
          jsonPointers:
            - /spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/selector
            - /spec/versions/0/subresources/scale/labelSelectorPath

and the new kyverno policy:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: cnpg-cluster-scale-selector
spec:
  background: true
  generateExisting: true
  mutateExistingOnPolicyUpdate: true
  useServerSideApply: true
  rules:
    - name: add-spec-selector-to-clusters
      match:
        resources:
          kinds:
            - postgresql.cnpg.io/v1/Cluster
      mutate:
        targets:
          - apiVersion: postgresql.cnpg.io/v1
            kind: Cluster
            name: {{ request.object.metadata.name }}
        patchStrategicMerge:
          spec:
            selector: "cnpg.io/cluster={{`{{ request.object.metadata.name }}`}}"

@gbartolini

@gbartolini Are you open to a PR here? I think above demonstrates this is just a small config change.

@devthejo

To follow up on @devthejo 's solution, you can use patchesJson6902 in the kyverno policy instead of using kubectl to patch. I wasn't able to get patchStrategicMerge to work.

Additionally, as @devthejo has called out, removing the second webhook from cnpg-mutating-webhook-configuration is required. Otherwise, the selector will not show up on the Cluster spec (the CNPG webhook overwrites it).

Finally, using "cnpg.io/cluster={{{{ request.object.metadata.name }}}}" as the selector is bad as this will also select any Pooler pods you are running. The role of the pod should be added also.

I have included our Kyverno policy to work around this limitation below:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"kyverno.io/v1","kind":"ClusterPolicy","metadata":{"annotations":{},"labels":{"panfactum.com/environment":"production","panfactum.com/local":"false","panfactum.com/module":"kube_cloudnative_pg","panfactum.com/region":"us-east-2","panfactum.com/root-module":"kube_cloudnative_pg","panfactum.com/stack-commit":"local","panfactum.com/stack-version":"local","test.1/2.3.4.5":"test.1.2.3.4.5","test1":"foo","test2":"bar","test3":"baz","test4":"42"},"name":"cnpg-scale-selector"},"spec":{"generateExisting":true,"mutateExistingOnPolicyUpdate":true,"rules":[{"match":{"resources":{"kinds":["CustomResourceDefinition"],"names":["clusters.postgresql.cnpg.io"]}},"mutate":{"patchesJson6902":"- \"op\": \"replace\"\n  \"path\": \"/spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/selector\"\n  \"value\":\n    \"type\": \"string\"\n- \"op\": \"replace\"\n  \"path\": \"/spec/versions/0/subresources/scale/labelSelectorPath\"\n  \"value\": \".spec.selector\"\n","targets":[{"apiVersion":"apiextensions.k8s.io/v1","kind":"CustomResourceDefinition","name":"clusters.postgresql.cnpg.io"}]},"name":"add-spec-selector-to-crd"},{"match":{"resources":{"kinds":["MutatingWebhookConfiguration"],"names":["cnpg-mutating-webhook-configuration"]}},"mutate":{"patchesJson6902":"- \"op\": \"remove\"\n  \"path\": \"/webhooks/1\"\n","targets":[{"apiVersion":"admissionregistration.k8s.io/v1","kind":"MutatingWebhookConfiguration","name":"cnpg-mutating-webhook-configuration"}]},"name":"remove-cnpg-mutating-webhook"},{"match":{"resources":{"kinds":["postgresql.cnpg.io/v1/Cluster"]}},"mutate":{"patchStrategicMerge":{"spec":{"selector":"cnpg.io/cluster={{ request.object.metadata.name }},cnpg.io/podRole=instance"}},"targets":[{"apiVersion":"postgresql.cnpg.io/v1","kind":"Cluster","name":"{{ request.object.metadata.name }}"}]},"name":"add-spec-selector-to-clusters"}],"useServerSideApply":true}}
  creationTimestamp: "2024-11-13T15:41:04Z"
  generation: 18
  labels:
    panfactum.com/environment: production
    panfactum.com/local: "false"
    panfactum.com/module: kube_cloudnative_pg
    panfactum.com/region: us-east-2
    panfactum.com/root-module: kube_cloudnative_pg
    panfactum.com/stack-commit: local
    panfactum.com/stack-version: local
    test.1/2.3.4.5: test.1.2.3.4.5
    test1: foo
    test2: bar
    test3: baz
    test4: "42"
  name: cnpg-scale-selector
  resourceVersion: "288754720"
  uid: 020d360d-16f6-40c4-b781-ead13871b980
spec:
  admission: true
  background: true
  emitWarning: false
  generateExisting: true
  mutateExistingOnPolicyUpdate: true
  rules:
  - match:
      resources:
        kinds:
        - CustomResourceDefinition
        names:
        - clusters.postgresql.cnpg.io
    mutate:
      patchesJson6902: |
        - "op": "replace"
          "path": "/spec/versions/0/schema/openAPIV3Schema/properties/spec/properties/selector"
          "value":
            "type": "string"
        - "op": "replace"
          "path": "/spec/versions/0/subresources/scale/labelSelectorPath"
          "value": ".spec.selector"
      targets:
      - apiVersion: apiextensions.k8s.io/v1
        kind: CustomResourceDefinition
        name: clusters.postgresql.cnpg.io
    name: add-spec-selector-to-crd
    skipBackgroundRequests: true
  - match:
      resources:
        kinds:
        - MutatingWebhookConfiguration
        names:
        - cnpg-mutating-webhook-configuration
    mutate:
      patchesJson6902: |
        - "op": "remove"
          "path": "/webhooks/1"
      targets:
      - apiVersion: admissionregistration.k8s.io/v1
        kind: MutatingWebhookConfiguration
        name: cnpg-mutating-webhook-configuration
    name: remove-cnpg-mutating-webhook
    skipBackgroundRequests: true
  - match:
      resources:
        kinds:
        - postgresql.cnpg.io/v1/Cluster
    mutate:
      patchStrategicMerge:
        spec:
          selector: cnpg.io/cluster={{ request.object.metadata.name }},cnpg.io/podRole=instance
      targets:
      - apiVersion: postgresql.cnpg.io/v1
        kind: Cluster
        name: '{{ request.object.metadata.name }}'
    name: add-spec-selector-to-clusters
    skipBackgroundRequests: true
  useServerSideApply: true
  validationFailureAction: Audit
status:
  autogen: {}
  conditions:
  - lastTransitionTime: "2024-11-13T15:41:04Z"
    message: Ready
    reason: Succeeded
    status: "True"
    type: Ready
  rulecount:
    generate: 0
    mutate: 3
    validate: 0
    verifyimages: 0
  validatingadmissionpolicy:
    generated: false
    message: ""

@gbartolini

@gbartolini I want to present a case for getting this into the roadmap for 1.25:

Vertical autoscaling is a really powerful feature, especially for databases where horizontal scaling isn't possible / practical. This would help CNPG reach parity with other operators and managed postgres services. This issue is in the top 10 most requested features (by 👍🏻 ) for CNPG currently.
The level of effort for the change should be low. This really only impacts the CRDs (though some logic will need to be added to keep the .spec.selector in sync).
This update is 100% backwards compatible.
Both @devthejo and I have verified that with the changes outlined above, the VPA integrates well.

Happy to take a crack at it but also want to get directional sign-off from a maintainer first.

@gbartolini

@gbartolini Wanted to circle back here to see if there were any thoughts here from the project maintainers.

I am happy to implement this if given the OK.

@fullykubed

@fullykubed I'd say just open a PR and reference this issue. I'm pretty sure the maintainers will be happy with the contribution 😊 And if they require any change you can always address it in the PR.

This issue is stale because it has been open for 60 days with no activity.

Not stale

This issue is stale because it has been open for 60 days with no activity.

@Dosub

@Dosub
it's still revelant in v1.29.1.

feat: add label selector to the Cluster scale subresource (#8996)

Navigation Menu

Repository navigation

[Feature Request] Support the scale sub-resource for Clusters #2574

Example VPA Manifest

Current Error

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Feature Request] Support the scale sub-resource for Clusters #2574

Description

Example VPA Manifest

Current Error

Activity

fullykubed commented on Aug 12, 2023

phisco commented on Aug 12, 2023

devthejo commented on Jun 7, 2024

devthejo commented on Jun 7, 2024

devthejo commented on Jun 10, 2024

fullykubed commented on Oct 26, 2024

fullykubed commented on Nov 13, 2024

fullykubed commented on Nov 29, 2024

fullykubed commented on Dec 21, 2024

kaisoz commented on Jan 7, 2025

github-actions commented on Mar 26, 2025

fullykubed commented on Mar 26, 2025

github-actions commented on May 29, 2025

26 remaining items

sebv004 commented on Jun 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions