fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 by yrahman82 · Pull Request #4834 · cozystack/cozystack

yrahman82 · 2026-10-09T20:21:19Z

What this PR does

Bumps the velero-plugin-for-aws init container in packages/system/velero/values.yaml from v1.14.1 to v1.14.4, and updates the pin in tests/velero_test.yaml.

v1.14.1 sets x-amz-tagging on every PutObject, even when the BackupStorageLocation configures no tags, in which case the header is empty. Cloudflare R2 rejects that with 501 NotImplemented: Header 'x-amz-tagging' with value '' not implemented. Velero then cannot upload backup metadata, so every Velero backup to an R2-backed location fails. That includes Cozystack's own cozy-default BackupStorageLocation when the platform backup bucket is on R2.

Upstream fixed this in v1.14.3: velero-plugin-for-aws#309 (a backport of #299, "Only set PutObject Tagging when tags are configured"). v1.14.4 is the current patch release of the 1.14 line. Its only change on top of v1.14.3 is CVE dependency bumps (#336). v1.14.3 itself also carries #310 (SSE-C presigned URLs), #313 (no amz-checksum encapsulation when checksumAlgorithm="") and CVE/Go bumps. The plugin's compatibility table lists v1.14.x for Velero v1.18.x, which is what the chart ships (velero:v1.18.1), so this stays inside the supported pairing.

Testing done:

make test in packages/system/velero (helm-unittest v1.1.2): 7 passed, with the existing init-container pin assertion moved to v1.14.4.
helm template renders velero/velero-plugin-for-aws:v1.14.4 as initContainers[0] of the velero Deployment.
On a Cozystack v1.6.4 staging cluster with the backup bucket on Cloudflare R2: before the bump, Velero backups failed on the empty x-amz-tagging header. After pinning the same plugin image through a Package values override (docker.io/velero/velero-plugin-for-aws:v1.14.4@sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa), all of our volume backup and restore drills pass. That covers CSI snapshots with the Kopia data mover to R2, and incremental backups confirmed.

Backwards compatibility: same plugin minor line and the same Velero version. The init container image changes, so the velero Deployment rolls once on upgrade. Users who never set tagging on a location now get no tagging header at all; users who do set it get the same header as before.

The reference stays in the existing shape (tag only, no digest), like the current v1.14.1 line. If you'd rather pin third-party init containers by digest, the v1.14.4 multi-arch index digest is sha256:c85a7fa803569e3128c33e4f5766f84ab156c8d171c9f1dd2715581d45ce98fa.

Screenshots

Not a UI change.

Downstream repositories

Trigger map walked against the diff: a third-party image tag in a system package values file and its test; no app schema, platform values, names or tooling.

Fixes #4852

Release note

fix(velero): velero-plugin-for-aws is bumped from v1.14.1 to v1.14.4, which no longer sends an empty x-amz-tagging header, so Velero backups work against S3-compatible stores that reject it, such as Cloudflare R2.

Summary by CodeRabbit

Updates
- Updated the AWS backup plugin used by Velero to version 1.14.4.

velero-plugin-for-aws v1.14.1 sets x-amz-tagging on every PutObject, empty when the BackupStorageLocation configures no tags. Cloudflare R2 answers that with 501 NotImplemented ("Header 'x-amz-tagging' with value '' not implemented"), so Velero cannot write backup metadata and every backup to an R2 location fails, the cozy-default location included when the platform bucket is on R2. The plugin sends the header only when tags are configured since v1.14.3 (velero-plugin-for-aws#309, a backport of cozystack#299). v1.14.4 is the current patch release of the 1.14 line, which upstream lists as the one compatible with Velero 1.18, and adds only CVE dependency bumps on top of v1.14.3. Signed-off-by: Yasar Rahman <[email protected]> Assisted-by: LLM

coderabbitai · 2026-10-09T20:21:59Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: cozystack/cozystack/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: 6490d9a1-4cbe-44a8-a43e-3aff8c4467a6

📥 Commits

Reviewing files that changed from the base of the PR and between f61e13f and 83aaa65.

📒 Files selected for processing (2)

packages/system/velero/tests/velero_test.yaml
packages/system/velero/values.yaml

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Walkthrough

Walkthrough

The Velero AWS plugin image tag changes from v1.14.1 to v1.14.4. The init-container rendering test now expects the updated tag.

Changes

Velero AWS plugin image

Layer / File(s)	Summary
Update plugin image tag `packages/system/velero/values.yaml`, `packages/system/velero/tests/velero_test.yaml`	The configured AWS plugin image tag and the init-container test expectation change to v1.14.4.

Priority: ➖ Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix

Suggested reviewers: lexfrei

Merge Risk: ⚪ Minimal · up to 83aaa

The plugin image pin and its test expectation are aligned; no actionable merge-blocking risk is evident in the reviewed changes.

Pre-merge checks | 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage		No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the AWS plugin version bump and its purpose: resolving backup failures with R2.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Aleksei Sviridkin (lexfrei)

yrahman82 NOT LGTM, only because one line in the PR description has to go before this merges. The change itself is correct, and I'll approve once that line is fixed.

The description ends with Fixes #NNNN (the issue in ISSUE-velero-r2-empty-tagging.md, if filed first). This repo merges with merge commits, and the PR description becomes the merge commit message. So a placeholder and the name of a local draft file would land in the history of main. Please drop the line, or put a real issue number there if you file one.

I checked the upstream side. The tagging fix (#309 in the plugin repo) shipped in v1.14.3, and v1.14.4 adds only CVE bumps on top. The plugin README pairs v1.14.x with Velero v1.18.x, which matches the vendored chart. The cozy-default BackupStorageLocation sets no tagging, so on v1.14.1 every upload to it sends the empty header. helm unittest in packages/system/velero passes on the PR head, and the branch merges cleanly into current main.

A backport to release-1.6 makes sense, since it ships the same v1.14.1.

yrahman82 · 2026-10-10T19:23:35Z

Aleksei Sviridkin (@lexfrei) thanks for checking the upstream side. Filed #4852 and replaced the placeholder: the description now ends with Fixes #4852, with no draft file name. #4852 also notes the release-1.6 backport, since that line ships the same v1.14.1.

Aleksei Sviridkin (lexfrei)

yrahman82 LGTM. The placeholder is gone and the description now ends with Fixes #4852. helm unittest in packages/system/velero passes on a trial merge with current main.

github-actions · 2026-10-10T20:38:10Z

Created backport PR for release-1.6:

[Backport release-1.6] fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 #4854 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-4834-to-release-1.6
git worktree add --checkout .worktree/backport-4834-to-release-1.6 backport-4834-to-release-1.6
cd .worktree/backport-4834-to-release-1.6
git reset --hard HEAD^
git cherry-pick -x 83aaa65ed1985544a1a8881dd001579c3098428d
git push --force-with-lease

Aleksei Sviridkin (lexfrei) · 2026-10-11T15:39:42Z

yrahman82 the backport to release-1.6 is open as #4854. That branch ships the same Velero and plugin versions, so the bump applies there too.

github-actions Bot added area/storage Issues or PRs related to storage (linstor, seaweedfs, bucket, velero, harbor) kind/bug Categorizes issue or PR as related to a bug size/XS This PR changes 0-9 lines, ignoring generated files labels Oct 9, 2026

yrahman82 marked this pull request as ready for review October 9, 2026 20:25

yrahman82 requested review from IvanHunters, Andrey Kolkov (androndo), Andrei Kvapil (kvaps), Aleksei Sviridkin (lexfrei), Timofei Larkin (lllamnyp), mattia-eleuteri and myasnikovdaniil as code owners October 9, 2026 20:25

Aleksei Sviridkin (lexfrei) requested changes Oct 10, 2026

View reviewed changes

yrahman82 mentioned this pull request Oct 10, 2026

velero: velero-plugin-for-aws v1.14.1 sends an empty x-amz-tagging header, so no Velero backup works against Cloudflare R2 #4852

Closed

yrahman82 requested a review from Aleksei Sviridkin (lexfrei) October 10, 2026 19:39

Aleksei Sviridkin (lexfrei) approved these changes Oct 10, 2026

View reviewed changes

Aleksei Sviridkin (lexfrei) enabled auto-merge October 10, 2026 20:08

Aleksei Sviridkin (lexfrei) added the kind/backport Categorizes issue or PR as requiring a backport to the current release line label Oct 10, 2026

Aleksei Sviridkin (lexfrei) merged commit 1b0a5a2 into cozystack:main Oct 10, 2026
21 checks passed

github-actions Bot mentioned this pull request Oct 10, 2026

[Backport release-1.6] fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 #4854

Draft

Navigation Menu

Repository navigation

fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2 - #4834

fix(velero): bump velero-plugin-for-aws to v1.14.4 so backups work on R2#4834
Aleksei Sviridkin (lexfrei) merged 1 commit into
cozystack:mainfrom
yrahman82:fix/velero-aws-plugin-r2

yrahman82 commented Oct 9, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 9, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Uh oh!

yrahman82 commented Oct 10, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) commented Oct 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

yrahman82 commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does

Screenshots

Downstream repositories

Release note

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Choose a reason for hiding this comment

Uh oh!

yrahman82 commented Oct 10, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026

Uh oh!

Aleksei Sviridkin (lexfrei) commented Oct 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yrahman82 commented Oct 9, 2026 •

edited

Loading

coderabbitai Bot commented Oct 9, 2026 •

edited

Loading