Skip to content

fix(harbor-rd): carry the release prefix in the resource selectors - #4170

Merged
Timofei Larkin (lllamnyp) merged 5 commits into
mainfrom
fix/rd-release-prefix-selectors
Sep 14, 2026
Merged

Timofei Larkin (lllamnyp) merged 5 commits into
mainfrom
fix/rd-release-prefix-selectors

Conversation

@kvaps

@kvaps Andrei Kvapil (kvaps) commented Sep 8, 2026 •

Copy link
Copy Markdown
Member

What this PR does

harbor declares release.prefix: "harbor-" and then selects its Secret, Service and Ingress without it, so all three resourceNames selectors match nothing. This corrects the three names and adds a guard so the next omission fails a test instead of passing silently.

Why they never matched

Release names are <prefix><app name>, but the lineage webhook evaluates selectors against {kind, name, namespace}, where name is the custom resource's name and carries no prefix. Every definition therefore spells the prefix out itself, and the others do — postgres-{{ .name }}-credentials, bucket-{{ .name }}-credentials, and so on. harbor is the only one that does not.

A selector that matches nothing raises no error: the webhook labels the object internal.cozystack.io/tenantresource=false and continues, which is indistinguishable from a selector that legitimately has nothing to match. That is why this survived.

What actually changes at runtime

Close to nothing, and that is the point — the manifest was wrong, not the behaviour:

  • the Secret already reaches tenants, because fix(harbor): mark credentials Secret with user-secret label #2541 worked around this exact mismatch by stamping apps.cozystack.io/user-secret: "true" on it and matching by label. That workaround is deliberately left in place;
  • the Service and Ingress labels flip from false to true, which nothing consumes: spec.services and spec.ingresses are read only by the webhook that computes the label, and the label's only consumer is the TenantSecret registry, which handles Secrets. The dashboard reaches both through the per-release Role, which names them correctly.

So this closes a latent defect before it bites, rather than fixing an outage.

The guard

hack/check-rd-release-prefix.bats asserts that every resourceNames entry in packages/system/*-rd/cozyrds/*.yaml contains <prefix>{{ .name }}. hack/*.bats is auto-discovered by BATS_UNIT_FILES, so it runs under make unit-tests with no registration.

The assertion is "contains", not "starts with", because operator-created objects legitimately carry leading words of their own: mongodb excludes internal-mongodb-{{ .name }}-users, which is correct and stays green. Definitions with an empty prefix are skipped, since hack/update-crd.sh generates the definitions for packages/extra charts with PREFIX="".

A selector that is correct and still fails the check can opt out on its own with a trailing # cozy-lint:ignore rd-release-prefix -- <reason> comment. A directive without a reason does not count.

Verified by mutation: the guard fails on the three harbor selectors before the fix and passes after. Across the rest of the tree harbor was the only violation.

What this PR does not do

It does not settle whether the prefix belongs in the template context instead. That would remove the duplication, but it requires migrating the definitions that spell the prefix out today, or they would double it. The guard is the cheaper half and closes the class either way.

Screenshots

Downstream repositories

Trigger map walked against the diff, which touches packages/system/harbor-rd/cozyrds/harbor.yaml and adds hack/check-rd-release-prefix.bats. The two nearest entries were checked and neither fires:

  • terraform-provider-cozystack lists "change release.prefix, or rename an output Secret or Service". Neither happens here: release.prefix is untouched and no object is renamed. The provider concatenates the prefix itself to derive names, so it was already computing the correct ones — it is this definition's selectors that were wrong.
  • ccp lists "rename packages/system/<name>-rd/cozyrds/<name>.yaml" and "move or rename anything under hack/". The file is edited, not renamed, and the new bats file is an addition; no make target changes behaviour.

website would fire on a change to ApplicationDefinition semantics. Semantics are unchanged — one definition's data is corrected — and content/en/docs/next/cozystack-api/application-definitions.md documents neither resourceNames nor the prefix rule, so there is nothing there to bring in step.

Release note

fix(harbor-rd): the ApplicationDefinition selectors for Harbor's Secret, Service and Ingress now carry the release prefix, so they match the objects the chart creates. No user-visible change — the Secret already reached tenants through a label-based workaround, and nothing consumes the Service and Ingress labels.

Summary by CodeRabbit

  • Bug Fixes

    • Updated Harbor resource naming patterns to consistently include the harbor- prefix for credentials, services, and ingress resources.
    • Improved alignment between Harbor resource names and their configured release prefixes.
  • Tests

    • Added validation to detect missing release prefixes in secret, service, and ingress resource selectors across system application definitions.
    • Validation reports all affected selectors and supports documented exceptions where necessary.

The definition declares a "harbor-" release prefix and then selects its
Secret, Service and Ingress without it, so all three resourceNames
selectors match nothing. The lineage webhook evaluates them against
{kind, name, namespace}, where name is the custom resource's name, so
the prefix has to be spelled out — as the other definitions do.

Signed-off-by: Andrei Kvapil <[email protected]>
…refix

A selector that omits the prefix matches nothing and reports nothing,
which is indistinguishable from a selector that legitimately has nothing
to match. Assert instead that every entry contains the declared prefix,
so the next omission fails a test rather than passing silently.

Signed-off-by: Andrei Kvapil <[email protected]>
@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 9822189e-dfed-48db-84f0-6d35fc86b080

📥 Commits

Reviewing files that changed from the base of the PR and between 9e577a5 and 9111202.

📒 Files selected for processing (2)
  • hack/check-rd-release-prefix.bats
  • packages/system/harbor-rd/cozyrds/harbor.yaml

Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

The pull request adds a Bats check for RD resource name prefixes and updates Harbor secret, service, and ingress selectors to include the harbor- prefix.

Changes

RD release prefix validation

Layer / File(s) Summary
Add RD prefix validation
hack/check-rd-release-prefix.bats
The new test scans RD definitions, validates resourceNames selectors, supports justified ignore comments, rejects empty inspection results, and reports violations.
Update Harbor selectors
packages/system/harbor-rd/cozyrds/harbor.yaml
Secret, service, and ingress selectors now include harbor- before {{ .name }}.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 91112

The Harbor selectors satisfy the intended prefix contract, and the validation guard has no confirmed merge-blocking defect.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the primary change: adding the Harbor release prefix to resource selectors. It is concise and specific, although it does not mention the new validation guard.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/rd-release-prefix-selectors

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size/M This PR changes 30-99 lines, ignoring generated files area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug labels Sep 8, 2026

@lllamnyp Timofei Larkin (lllamnyp) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix in packages/system/harbor-rd/cozyrds/harbor.yaml is right, and the commit message states the reason well: the lineage webhook evaluates these selectors against a context carrying only {kind, name, namespace}, where name is the custom resource's name and not the Helm release name, so a selector that omits the harbor- prefix names an object that does not exist. Worth guarding too, for exactly the reason the second commit gives — the omission reports nothing, so it is indistinguishable from a selector that legitimately has nothing to match.

Four things on hack/check-rd-release-prefix.bats before this comes out of draft.

1. The test does not run under the repo's BATS runner

make bats-unit-tests executes every hack/*.bats through hack/cozytest.sh, a pure-POSIX-shell reimplementation of a BATS subset rather than the bats binary. It implements no run helper and sets no $status or $output. The result today:

$ hack/cozytest.sh hack/check-rd-release-prefix.bats
hack/cozytest.sh: 33: run: not found
❌ Test failed: every resourceNames selector carries its release prefix (exit 1)

This is the only hack/*.bats file using run. Eight existing files state the constraint in their own headers — hack/clickhouse-update-versions_test.bats puts it as "written in the portable subset hack/cozytest.sh understands (no run/$status/$lines, no setup/teardown, no standalone } inside a test body)", and hack/promote-retag_test.bats, hack/promote-rewrite-tags_test.bats, hack/common-envs_test.bats, hack/migration-56-adopt-path.bats, hack/kubernetes-nodes-unpin-hook.bats, hack/capture-dataplane.bats and hack/cilium-leak-healer_test.bats say the same. The house pattern is a plain $(...) capture asserted with [ ... ], relying on set -e.

2. Please drop the inlined Python for bash + yq

The heredoc'd python3 script also brings a dependency the project does not declare. make build-deps asserts find docker skopeo jq gh helm, mikefarah yq, GNU tar and GNU sed; python3 is not in that list, and the unit-test job in .github/workflows/pull-requests.yaml installs only the helm-unittest plugin. import yaml is a further step beyond that, since PyYAML is not in the standard library, so on a runner with python3 but no PyYAML this fails with a traceback attributed to a harbor selector.

For contrast, 36 of the 105 files in hack/ reach for yq, including the nearest neighbours — check-mongodb-rd-secrets.bats, check-redis-rd-secrets.bats, check-qdrant-rd-secrets.bats, check-nats-rd-secrets.bats, check-mariadb-rd-secrets.bats, check-rabbitmq-rd-secrets.bats — and check-applicationdefinition-crd-ordering.bats, which is structurally the same job as this one: a repo-wide invariant over a glob of YAML, done in yq plus jq with a vacuity guard. Only two files invoke python3 at all, and both do so because the thing under test is itself Python: md-no-hardwrap.bats exercises .claude/hooks/md-no-hardwrap.py and says so in a header that notes "other hack/*.bats deliberately avoid a python3 dependency"; talos-image-cache_test.bats compiles a generated serve.py. There is also hack/check-rd-presets.sh, which already walks packages/system/*-rd/cozyrds/*.yaml in bash with yq and jq and is wired into make unit-tests as rd-presets-check — the closest thing to a template for this.

The rewrite is short. This version runs clean on the tree as of this PR (84 selectors inspected across the 24 prefixed definitions) and reports all three harbor selectors when the fix is reverted:

@test "every resourceNames selector carries its release prefix" {
  command -v yq >/dev/null || { echo "yq (mikefarah v4+) is required" >&2; exit 1; }

  violations=""
  total=0
  for f in "$REPO_ROOT"/packages/system/*-rd/cozyrds/*.yaml; do
    prefix="$(yq -r '.spec.release.prefix // ""' "$f")"
    [ -n "$prefix" ] || continue

    names="$(yq -r '
      (.spec.secrets, .spec.services, .spec.ingresses) | select(. != null)
      | (.include, .exclude) | select(. != null) | .[]
      | .resourceNames | select(. != null) | .[]
    ' "$f")"
    [ -n "$names" ] || continue
    total=$(( total + $(printf '%s\n' "$names" | grep -c .) ))

    bad="$(printf '%s\n' "$names" | grep -vF "${prefix}{{ .name }}" || true)"
    [ -z "$bad" ] || violations="$violations$(printf '%s\n' "$bad" | sed "s|^|${f#"$REPO_ROOT"/}: |")
"
  done

  # A glob that matched nothing, or a yq that stopped returning names, would
  # otherwise pass silently.
  [ "$total" -ge 1 ] || { echo "inspected zero selectors; this test is broken" >&2; exit 1; }

  if [ -n "$violations" ]; then
    echo "resourceNames selectors that can never match the object their chart creates:" >&2
    printf '%s' "$violations" >&2
    echo "Fix: spell the release prefix out, e.g. \"harbor-{{ .name }}-credentials\"." >&2
    exit 1
  fi
}

3. Cut the header comment down to what the code cannot say

The 25-line header is longer than both commit messages put together — 227 words against 67 and 56 — and most of it is those messages retold. The bar in docs/agents/contributing.md (Review Blockers: Messages, Trailers, Comments) is that "a comment says what the code cannot: the reason for this way over the obvious one, a constraint or unit the types do not carry, an external anchor, a trap for the next refactorer". Measured against that, and against what the commits already carry:

  • The opening line restates the @test name one line below it.
  • The paragraph deriving the prefix from the release name and the webhook's template context is the fix commit's body at three times the length. Someone who wants that reaches for git log or git blame, and there it stays correct.
  • The paragraph on the omission being silent is the test commit's body, likewise. The harbor incident that follows it — that the definition shipped this way and the symptom was worked around by stamping a label on the Secret — is the one genuinely new thing in the header, but it is motivation for the change rather than something a future editor of this file needs, and as a comment it goes stale the day that label workaround is removed. Both commit messages are already terse and sufficient as they stand; this is a request to cut, not to move anything into them.

Two things in there are worth keeping, because nothing else in the repo records them:

  • Why the assertion is contains rather than a prefix match. Nine live selectors across clickhouse, mongodb, redis and valkey carry their own leading words, so someone tightening this to startsWith would break all nine, and this is the only place that warns them.
  • Why an empty prefix is skipped, which otherwise reads as a hole in the check rather than a deliberate exclusion.

Both fit in five lines:

#!/usr/bin/env bats
# "contains <prefix>{{ .name }}" rather than a prefix match: operator-created
# objects carry their own leading words — redis's "rfs-redis-{{ .name }}",
# mongodb's excluded "internal-mongodb-{{ .name }}-users".
#
# An empty spec.release.prefix is skipped: hack/update-crd.sh generates the
# definitions for packages/extra charts with PREFIX="".

One correction while that paragraph is being rewritten: the empty-prefix skip is currently described as covering packages/extra/*, but the glob only reads packages/system/*-rd/cozyrds/*.yaml and there are no definitions under packages/extra at all. What happens is that eight definitions under packages/system — bootbox, computeplane, etcd, external-dns, info, ingress, monitoring, seaweedfs — are generated from charts in packages/extra, and hack/update-crd.sh gives those PREFIX="".

4. Add a per-selector opt-out

A repo-wide name check will eventually meet a selector that is correct and still fails it — an object an operator names on its own terms, or one deliberately matched across releases. Right now the only escape is the blanket empty-prefix skip, which is all-or-nothing per file. Something closer to Go's //nolint, scoped to the one selector and carrying a reason, would keep the guard from turning into a thing people route around.

A YAML line comment is the natural home, since the selector line is the thing being exempted, and yq can read it directly. These files already carry comments inside resourceNames lists — mongodb-rd/cozyrds/mongodb.yaml annotates several of its exclude entries that way — and a comment survives hack/update-crd.sh, which edits the file with yq -i on named fields and round-trips everything else:

  secrets:
    include:
      - resourceNames:
          - "{{ .name }}-credentials" # cozy-lint:ignore rd-release-prefix -- named by the operator, not the release

Honouring it costs one more expression and one more filter. Emit the comment alongside each name and drop the annotated rows before the assertion:

    names="$(yq -r '
      (.spec.secrets, .spec.services, .spec.ingresses) | select(. != null)
      | (.include, .exclude) | select(. != null) | .[]
      | .resourceNames | select(. != null) | .[]
      | [., (line_comment // "")] | @tsv
    ' "$f")"
    ...
    bad="$(printf '%s\n' "$names" \
      | grep -v 'cozy-lint:ignore rd-release-prefix' \
      | grep -vF "${prefix}{{ .name }}" || true)"

Verified against yq v4.45.1: line_comment returns the trailing comment on a sequence entry, and adding the marker to one of the three pre-fix harbor selectors suppresses exactly that one while the other two still fail. Requiring the -- <reason> tail, the way a //nolint directive is expected to explain itself, keeps a bare suppression from being the path of least resistance.

make unit-tests executes hack/*.bats through hack/cozytest.sh, which
has no `run` helper, so the guard aborted before inspecting a single
selector. It also depended on python3 and PyYAML, neither of which the
build dependencies or the unit-test job provide. Walk the definitions
with yq instead, as the neighbouring rd checks do, and fail when zero
selectors were inspected so a broken glob or query cannot pass.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <[email protected]>
…ader

The header retold both commit messages at several times their length.
What stays is what nothing else records: why the check is "contains"
rather than a prefix match, so tightening it does not break the nine
selectors with their own leading words, and why an empty prefix is
skipped. The latter now names the actual source of those definitions,
since they live under packages/system and are generated from charts in
packages/extra.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <[email protected]>
A repo-wide name check will eventually meet a selector that is correct
and still fails it, such as an object named on the operator's terms.
The only escape so far was an empty prefix, which exempts a whole
definition, and a guard with no narrower way out is one people learn to
route around. A trailing "cozy-lint:ignore rd-release-prefix -- <reason>"
comment now exempts that one entry, and a directive without a reason
does not count, so a bare suppression is never the easy path.

Assisted-by: LLM
Signed-off-by: Andrei Kvapil <[email protected]>
@kvaps

Copy link
Copy Markdown
Member Author

Thanks, fixed.

Two things differ from your snippets:

  • grep -v dropped a directive row even with no reason after it, so now a directive without -- <reason> is reported as a violation. The check also looks only at the name column, so a comment that contains harbor-{{ .name }} can't make a bad name pass.
  • On a flow sequence ([a, b] # ...) yq attaches the comment to the whole list, so the directive works only on block-style entries. There is a comment about it in the test. No definition uses flow style for resourceNames today.

Checked by mutation on the pre-fix harbor file, under sh and dash. Description updated too.

@lllamnyp Timofei Larkin (lllamnyp) left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four points are addressed, and the opt-out went further than requested — enforcing the -- <reason> tail rather than merely conventioning it closes the hole where the directive quietly becomes a mute button.

Checked the rewritten guard against the tree rather than only reading it. It runs under hack/cozytest.sh and inspects 84 selectors across the 24 prefixed definitions, so the port to yq did not narrow what the check sees. Reverting the three harbor selectors reports all three, labelled by section and branch. A marker carrying a reason exempts that one entry and leaves the others failing; a marker with no reason, or with an empty one, is reported as (exempted without a reason) rather than honoured.

The flow-sequence caveat in the header is worth the two lines it takes. Putting the marker on resourceNames: ["{{ .name }}-credentials"] # cozy-lint:ignore ... leaves the entry still reported, since the comment binds to the list rather than to the item — so a directive that does not take makes the test fail rather than pass. That is the right direction for it to break in, and the comment is the only place a reader would learn it.

index($2, needle) is a nicer answer than a fixed-string grep for the containment test, and the [ -f "$f" ] guard covers an unmatched glob expanding to a literal path.

One cosmetic thing, not worth another round: the failure header reads "selectors that can never match the object their chart creates", which does not quite describe a reason-less exemption — though the (exempted without a reason) suffix on the row disambiguates it where it counts.

LGTM.

@lllamnyp
Timofei Larkin (lllamnyp) merged commit ae7a711 into main Sep 14, 2026
20 checks passed
@lllamnyp
Timofei Larkin (lllamnyp) deleted the fix/rd-release-prefix-selectors branch September 14, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/uncategorized PR auto-labeler could not map title scope to a known area/*; please review kind/bug Categorizes issue or PR as related to a bug size/M This PR changes 30-99 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants