Repository navigation
feat(redis): add TLS support via nginx stream sidecar - #2540
Arsolitt (Arsolitt) wants to merge 33 commits into
Conversation
Add helm-unittest tests for the upcoming cozy-lib.tls.certificate helper. Tests cover: required fields, dnsNames, issuerRef, labels, default and custom duration/renewBefore, default and custom usages, secretName defaulting, and both ClusterIssuer and Issuer kinds. Signed-off-by: Arsolitt <[email protected]> Signed-off-by: Arsolitt <[email protected]>
Add cozy-lib.tls.certificate template that renders a cert-manager Certificate CR for services needing TLS. This provides a shared foundation so each chart does not duplicate cert-manager boilerplate. Also adds cozy-lib.tls.secretName helper for resolving a TLS secret name from user input or generating one from the release name. Signed-off-by: Arsolitt <[email protected]> Signed-off-by: Arsolitt <[email protected]>
…faults and add missing tests Signed-off-by: Arsolitt <[email protected]>
…lidation Signed-off-by: Arsolitt <[email protected]>
…name Signed-off-by: Arsolitt <[email protected]>
… and improve docs Signed-off-by: Arsolitt <[email protected]>
… validation Add missing space in renewBefore parameter description and add fail guards for missing Release and suffix in the cozy-lib.tls.secretName helper. Signed-off-by: Arsolitt <[email protected]>
Signed-off-by: Arsolitt <[email protected]>
Signed-off-by: Arsolitt <[email protected]>
Add tls section to values.yaml and values.schema.json so users can enable TLS termination on the client-facing port. Defaults preserve current behavior (disabled), and the schema documents the relationship with cert-manager. Signed-off-by: Arsolitt <[email protected]>
Define local helpers redis.tls.secretName (wraps the cozy-lib helper with the redis-tls suffix) and redis.tls.dnsNames (covers the external LB service and the operator-managed master service rfs-<release>). Signed-off-by: Arsolitt <[email protected]>
Render a cert-manager Certificate via the cozy-lib helper (only when tls.enabled and no user-provided secret) and a ConfigMap with the nginx stream config that terminates TLS on 6380 and proxies to Redis on 6379. Signed-off-by: Arsolitt <[email protected]>
Use the operator's spec.redis.extraContainers / extraVolumes hooks to attach an nginx-unprivileged stream sidecar that terminates TLS on 6380. The Redis container itself is untouched, so operator probes, sentinel, and replication continue talking plaintext on 6379. Signed-off-by: Arsolitt <[email protected]>
When tls.enabled is true the LoadBalancer service exposes 6380 / redis-tls (targeting the sidecar) instead of 6379. Plaintext is intentionally not exposed externally in TLS mode; intra-cluster apps keep using the operator's rfs-* ClusterIP service on 6379. Signed-off-by: Arsolitt <[email protected]>
Add the TLS secret (resolved via redis.tls.secretName, which respects user-provided tls.secretName) to the Role's secrets resourceNames so tenants can fetch ca.crt from the dashboard when TLS is enabled. Signed-off-by: Arsolitt <[email protected]>
Regenerate Go types, README parameter table, normalized JSON schema, and
the redis-rd openAPISchema/keysOrder so generator output stays in sync
with the values.yaml additions for TLS.
Switch the inline `@param {struct} tls` annotation to a pair of
`@typedef`s (`TLS` and `TLSIssuerRef`) so cozyvalues-gen can resolve
the nested issuerRef object.
Signed-off-by: Arsolitt <[email protected]>
Allow the dashboard to read the auto-generated `redis-<name>-redis-tls` secret so tenants can download the CA certificate and connect with TLS clients. The pattern matches the secret produced by cozy-lib's TLS helper when no user-provided `tls.secretName` is set. Signed-off-by: Arsolitt <[email protected]>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds client-facing TLS for Redis: new Go API TLS types and deepcopy methods, Helm values/schema/docs, cozy-lib TLS helpers and cert-manager Certificate template, nginx TLS proxy sidecar + ConfigMap, conditional Services/RBAC, ApplicationDefinition schema updates, and comprehensive Helm unit tests. Changes
Sequence DiagramsequenceDiagram
participant Client as Client
participant NGINX as NGINX TLS Proxy
participant Redis as Redis Server
participant CertMgr as cert-manager
participant Secret as Kubernetes Secret
CertMgr->>Secret: Issue Certificate -> Secret
Secret->>NGINX: Mount tls.crt / tls.key
Client->>NGINX: Connect to :6380 (TLS)
NGINX->>NGINX: Terminate TLS
NGINX->>Redis: Proxy plaintext to :6379
Redis->>NGINX: Respond plaintext
NGINX->>Client: Return encrypted response
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Per-package make generate runs cozyvalues-gen and update-crd.sh, but deepcopy code is generated only by root-level make generate (which calls hack/update-codegen.sh). Add the missing DeepCopy methods for the new TLS and TLSIssuerRef structs. Signed-off-by: Arsolitt <[email protected]>
Spell out which endpoints serve TLS (master/replicas ClusterIP on 6380), why Sentinel-aware clients cannot use TLS today, and how applications should connect to keep automatic master failover under TLS. Signed-off-by: Arsolitt <[email protected]>
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/apps/redis/tests/service-tls_test.yaml (1)
86-94: Duplicate test case with existing scenario at Line 9.
Line 86-94 repeats the same setup and assertion as Line 9-17 (external: false,tls.enabled: false,count: 0). Consider removing one to keep the suite lean.Suggested cleanup
- - it: no services when TLS disabled and external false - set: - external: false - tls: - enabled: false - asserts: - - hasDocuments: - count: 0🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/apps/redis/tests/service-tls_test.yaml` around lines 86 - 94, The test case titled "no services when TLS disabled and external false" is duplicated (one at the top and one later); remove the redundant block (the entry with set: external: false tls: enabled: false and asserts: hasDocuments count: 0) so only a single instance of this scenario remains, leaving the original test and other unique cases intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@packages/apps/redis/tests/service-tls_test.yaml`:
- Around line 86-94: The test case titled "no services when TLS disabled and
external false" is duplicated (one at the top and one later); remove the
redundant block (the entry with set: external: false tls: enabled: false and
asserts: hasDocuments count: 0) so only a single instance of this scenario
remains, leaving the original test and other unique cases intact.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 75783378-41b0-487f-a030-e9aec0834b15
📒 Files selected for processing (6)
packages/apps/redis/templates/_tls.tplpackages/apps/redis/templates/service.yamlpackages/apps/redis/templates/tls-certificate.yamlpackages/apps/redis/tests/service-tls_test.yamlpackages/apps/redis/tests/tls-certificate_test.yamlpackages/apps/redis/values.yaml
✅ Files skipped from review due to trivial changes (1)
- packages/apps/redis/tests/tls-certificate_test.yaml
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/apps/redis/templates/_tls.tpl
- packages/apps/redis/values.yaml
Address review feedback from coderabbitai on api/apps/v1alpha1/redis/types.go:75: Add kubebuilder Enum validation marker to restrict Kind to "Issuer" or "ClusterIssuer", catching typos at admission time rather than deferring to cert-manager reconciliation. Assisted-By: Claude <[email protected]> Signed-off-by: Arsolitt <[email protected]>
Address review feedback from coderabbitai on tests/tls-certificate_test.yaml:48: Switch the fixture from kind: ClusterIssuer (the default) to kind: Issuer and assert both name and kind fields, so the test catches regressions where the chart ignores user-supplied issuerRef.kind. Assisted-By: Claude <[email protected]> Signed-off-by: Arsolitt <[email protected]>
…ML safety Add optional extraDnsNames parameter to the TLS certificate helper so consumers can append user-supplied custom domains to the SAN list. Pipe all dnsNames (and extraDnsNames) through | quote to ensure wildcard entries like *.example.com are properly YAML-quoted. Assisted-By: Claude <[email protected]> Signed-off-by: Arsolitt <[email protected]>
…ML safety Add optional extraDnsNames parameter to tls.certificate helper, allowing callers to append user-supplied custom domains to the Certificate SAN list. Apply quote filter to all dnsNames entries for safe YAML rendering of values containing special characters (e.g. wildcards). Assisted-By: Claude <[email protected]> Signed-off-by: Arsolitt <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
api/apps/v1alpha1/redis/types.go (1)
1-10:⚠️ Potential issue | 🟠 Major | ⚡ Quick winGenerated file is drifting from
make generateoutputLine 1 marks this as generated, and CI already shows regeneration rewrites these edits (including import/marker changes). Please move these changes to the values-gen source and commit regenerated artifacts; otherwise the enum/default behavior can be lost on the next generate run.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@api/apps/v1alpha1/redis/types.go` around lines 1 - 10, The file api/apps/v1alpha1/redis/types.go is a generated artifact that has been modified directly (header, imports/markers like resource and v1), causing CI to overwrite it; revert any manual edits here and instead make the required changes in the values-gen source used to produce this file, then run the generation step (make generate) and commit the regenerated artifacts so types.go (package redis and its imports) matches the canonical generated output.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@api/apps/v1alpha1/redis/types.go`:
- Around line 68-72: The Helm values schema currently allows any string for
tls.issuerRef.kind while the API type TLSIssuerRef (field Kind) enforces an enum
("Issuer" or "ClusterIssuer") and a default; update the Helm schema
(packages/apps/redis/values.schema.json) to add an enum constraint for
tls.issuerRef.kind with allowed values "Issuer" and "ClusterIssuer" and set the
same default ("ClusterIssuer") so Helm validation matches the TLSIssuerRef Kind
validation in the API.
---
Outside diff comments:
In `@api/apps/v1alpha1/redis/types.go`:
- Around line 1-10: The file api/apps/v1alpha1/redis/types.go is a generated
artifact that has been modified directly (header, imports/markers like resource
and v1), causing CI to overwrite it; revert any manual edits here and instead
make the required changes in the values-gen source used to produce this file,
then run the generation step (make generate) and commit the regenerated
artifacts so types.go (package redis and its imports) matches the canonical
generated output.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 978f98c1-8bb2-4226-a5aa-367feece31af
📒 Files selected for processing (2)
api/apps/v1alpha1/redis/types.gopackages/apps/redis/tests/tls-certificate_test.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/apps/redis/tests/tls-certificate_test.yaml
Allow users to specify additional DNS Subject Alternative Names for the TLS certificate, enabling custom domain support. The field is passed through to the cozy-lib tls.certificate helper. Assisted-By: Claude <[email protected]> Signed-off-by: Arsolitt <[email protected]>
Incorporate the latest cozy-lib extraDnsNames changes from the upstream branch so that PR #2485 can be merged before PR #2540 without conflicts. Signed-off-by: Arsolitt <[email protected]>
Signed-off-by: Arsolitt <[email protected]>
Signed-off-by: Arsolitt <[email protected]>
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
NOT LGTM — the nginx-sidecar TLS approach is sensible (Spotahome operator gap is real, sidecar avoids forking the operator) and the test coverage on both cozy-lib._tls.tpl and the redis chart is solid. But two combinations are broken in default paths: dashboard RBAC doesn't cover the new TLS-only services, so tenants cannot discover their master/replicas endpoints; and the auto-issued certificate carries only cluster-local SANs, so external clients fail TLS verification when external=true && tls.enabled=true.
Business context: Add client-facing TLS for Redis without forking the unmaintained Spotahome operator — an nginx stream sidecar terminates TLS on port 6380 and proxies to localhost:6379, where the operator probes/sentinel/replication continue plaintext.
Blockers
B1: dashboard RBAC missing TLS-only services
File: packages/apps/redis/templates/dashboard-resourcemap.yaml:10-15
Issue: When tls.enabled=true, the chart introduces new ClusterIP services {{ .Release.Name }}-master and {{ .Release.Name }}-replicas (service.yaml:6, 27) carrying the actual TLS endpoints. The dashboard Role only allows reading rfs-*, rfrm-*, rfrs-*, and *-external-lb. The new TLS services are not in resourceNames, so any dashboard/tenant view using this RBAC cannot fetch them.
Evidence: service.yaml:1-44 renders the two new services under if .Values.tls.enabled. dashboard-resourcemap.yaml:10-15 lists exactly the four legacy service names. With tls.enabled=true && external=false, those four services either don't exist or aren't where the TLS terminates, so the connection point is invisible from the tenant API.
Impact: Tenants enabling internal TLS see the secret in dashboard but no service to connect to. The whole non-external TLS path is unreachable from the dashboard.
Fix: Append {{ .Release.Name }}-master and {{ .Release.Name }}-replicas to the resourceNames list, gated on if .Values.tls.enabled (mirror the secrets block at lines 22-24).
B2: auto-issued external TLS certificate has only cluster-local SANs
File: packages/apps/redis/templates/tls-certificate.yaml:11-14
Issue: When tls.enabled=true && external=true, the chart exposes Redis through a LoadBalancer service and clients connect via the LB IP or provider hostname. The auto-generated Certificate only contains {{ .Release.Name }}-external-lb, {{ .Release.Name }}-external-lb.{{ .Release.Namespace }}, etc. — purely in-cluster names. TLS hostname verification fails for any external client unless they disable it or supply their own secret via tls.secretName.
Evidence: tls-certificate.yaml:1-22 builds $dnsNames only from cluster-internal patterns. service.yaml:50-79 creates the LB but the cert's SANs don't include any external-routable hostname. PR description claims external=true clients use redis-cli --tls --cacert ca.crt — but --cacert only fixes CA trust, not hostname mismatch.
Impact: The combination external=true && tls.enabled=true && tls.secretName="" (the documented default issuance path) is broken from the external connection perspective. Users who follow the README path get TLS errors and have to either disable verification or roll their own secret.
Fix: One of:
(a) Add a tls.dnsNames value letting the user inject the external hostname (or a list of hostnames) into the SAN list before rendering — minimum viable.
(b) Document explicitly that for external=true, users must provide tls.secretName with a cert containing the external hostname; fail-loud or warn if external=true && tls.enabled=true && tls.secretName=="".
(c) Discover the LB hostname/IP from the Service status and fold it into the SAN list — clean but requires post-provisioning regeneration.
Non-blocking follow-ups
-
api/apps/v1alpha1/redis/types.go:75—TLSIssuerRef.kindhas no+kubebuilder:validation:Enum=Issuer;ClusterIssuertag, so a typo (ClusterIssue,cluseterIssuer, etc.) only fails late when cert-manager reconciles. The cozy-lib helper does template-time validation, but API-layer validation is expected for cozystack CRs and matches what other resources do. -
packages/library/cozy-lib/templates/_tls.tpl:78—- {{ . }}inside the dnsNames range is unquoted. Today's callers don't pass wildcard SANs, but*.example.comwould be parsed as a YAML alias reference and break rendering. Quote it:- {{ . | quote }}. Trivial defense-in-depth for a helper meant to be reused (per PR description's reference to #473 phases). -
packages/apps/redis/tests/tls-certificate_test.yaml:48— the test fixture for a customissuerRef.kindstill usesClusterIssuer(which is the default), so a regression where the chart hardcodedClusterIssuerinstead of consumingissuerRef.kindwould not be caught. Switch the fixture toIssuer(and add the missingIssuer-kind resource if the test setup needs it) so the assertion is meaningful. -
packages/system/redis-rd/cozyrds/redis.yamlopenAPI schema markstls.required: [enabled, issuerRef, secretName], with each field defaulted. Standard JSON Schema validators apply defaults before checking required, so this should work for partial submissions liketls: {enabled: true}. Worth confirming with cozystack-api's actual validation path that the dashboard's "toggle TLS" round-trip doesn't get rejected.
Restrict issuerRef.kind to "Issuer" or "ClusterIssuer" via @enum annotation in values.yaml. This propagates through cozyvalues-gen to the Helm schema, Go API types, and ApplicationDefinition CRD, catching typos at validation time rather than at apply time. Signed-off-by: Arsolitt <[email protected]>
When tls.enabled=true, the chart creates dedicated ClusterIP services
({{ .Release.Name }}-master and {{ .Release.Name }}-replicas) for TLS
endpoints. These were missing from the dashboard Role's resourceNames,
making them invisible to tenants via the dashboard API.
Address review feedback from lexfrei on dashboard-resourcemap.yaml.
Signed-off-by: Arsolitt <[email protected]>
|
Thanks for the thorough review — addressing each point: B1: Dashboard RBAC missing TLS-only services — Valid, fixed in 45e88cb. Added B2: Auto-issued external TLS certificate has only cluster-local SANs — The certificate already includes all four FQDN forms for Non-blocking follow-ups:
|
Aleksei Sviridkin (lexfrei)
left a comment
There was a problem hiding this comment.
LGTM. Both blockers from the prior review are addressed:
- B1 (dashboard RBAC missing TLS services):
45e88cb84adds{{ .Release.Name }}-masterand{{ .Release.Name }}-replicastodashboard-resourcemap.yaml:15-18gated on.Values.tls.enabled, mirroring the secrets-section pattern at lines 26-28 (which already added the TLS secret to resourceNames). The non-externaltls.enabled=truepath is now reachable from the dashboard. - B2 (auto-issued cert has only cluster-local SANs):
7dbfe8062introduces thetls.extraDnsNamesvalue (option (a) from the prior review — minimum-viable user-injected SAN list) andtls-certificate.yaml:20passes it through theextraDnsNamesparameter ofcozy-lib.tls.certificate. Documented invalues.yaml(@field {[]string} [extraDnsNames]) and the README parameter table. Users withexternal=true && tls.enabled=truenow have a clean path to inject the LB hostname into the cert SANs without having to roll their own secret.
Non-blocking follow-ups from the prior review:
- Non-blocking #1 (kubebuilder enum on
TLSIssuerRef.Kind): addressed in6a0294ca9/7eb7296ef—IssuerKindtype now carries+kubebuilder:validation:Enum="Issuer";"ClusterIssuer", so a typo fails at API admission rather than late at cert-manager reconcile time. CI'sVerify generated code is up to dateis green, so the deepcopy regeneration is in lockstep. - Non-blocking #2 (cozy-lib helper unquoted
- {{ . }}for dnsNames): addressed via thefeat/cozy-lib-tlslineage that this branch merges; thednsNamesandextraDnsNamesranges now render as- {{ . | quote }}. The remaining unquotedissuerRef.{name,kind,group}scalars in this branch's copy of_tls.tplare quoted on the canonicalfeat/cozy-lib-tlsHEAD (#2485) and will land in main once #2485 merges; this PR is right to defer rather than carry conflicting copies. - Non-blocking #3 (test fixture for
custom issuerRef.kindreused the default): addressed ine0272dab7— theIssuer-kind branch is now exercised by a non-default fixture, so a regression where the chart hardcodedClusterIssuerwould actually fail the test.
Verification:
helm unittest packages/apps/redis→ 30/30 pass locally across 5 suites (Dashboard RBAC, RedisFailover sidecar injection, Service architecture, TLS Certificate, TLS nginx ConfigMap).- CI green including E2E and
Verify generated code is up to date— the deepcopy regeneration onIssuerKindenum +extraDnsNamesdid not drift. - Architectural decision worth flagging for future readers: the chart accepts the operator's plaintext-on-6379 contract as load-bearing (intra-cluster traffic from operator probes, sentinel, and replication stays plaintext on 6379; the nginx stream sidecar terminates TLS on 6380 and proxies to localhost:6379). This means TLS for cluster-internal connections is intentionally out of scope here and can only be addressed by either forking Spotahome or migrating to a different operator. Sentinel-aware clients still operate over plaintext, and the README is honest about that — clients that need automatic master failover with TLS are pointed at
<release>-master:6380(where the K8s Service selector follows the master role label and auto-routes after failover). Reasonable scope for an unmaintained-operator workaround.
What this PR does
Add TLS termination for client-facing Redis connections via an nginx stream sidecar.
The Spotahome Redis Operator (#268, #418) does not support TLS natively and is effectively unmaintained. This PR layers TLS on top of the operator without modifying it: an nginx sidecar listens on port 6380 with cert-manager-issued (or user-provided) certificates and proxies decrypted traffic to
localhost:6379. Internal traffic (operator probes, sentinel, replication) continues over plaintext on 6379, so the operator stays happy.When
tls.enabledandexternalare both true, the LoadBalancer routes to the TLS port. The TLS secret is exposed to tenants through the dashboard viaredis-rdsecrets.includeso they can retrieve the CA cert and connect withredis-cli --tls --cacert ca.crt. The TLS-specific services (-masterand-replicas) are included in the dashboard RBAC so tenants can discover their connection endpoints.This PR depends on #2485 (shared
cozy-lib.tls.certificatehelper). Implements #2157.New values
tls.enabled(bool, default false)tls.secretName(string, default empty — when set, cert-manager Certificate is not rendered and user manages the secret)tls.issuerRef.name(string, defaultselfsigned-cluster-issuer) — cert-manager Issuer resource nametls.issuerRef.kind(enum:Issuer|ClusterIssuer, defaultClusterIssuer)tls.extraDnsNames(list of strings, optional) — extra DNS SANs for the TLS certificate (e.g. external hostnames)Backward compatible: with
tls.enabled: false(default) nothing changes.Out of scope
Screenshots
N/A — no UI changes.
Release note
Summary by CodeRabbit
New Features
Documentation
Tests
Other