velero: velero-plugin-for-aws v1.14.1 sends an empty x-amz-tagging header, so no Velero backup works against Cloudflare R2 #4852

Summary

packages/system/velero ships velero/velero-plugin-for-aws:v1.14.1. That version sets the x-amz-tagging header on every PutObject, even when the BackupStorageLocation has no tagging configured, in which case the header is empty. Cloudflare R2 rejects such requests, so Velero cannot upload backup metadata and every backup to an R2-backed location fails. This also hits Cozystack's own cozy-default BackupStorageLocation when the platform backup bucket is on R2.

Environment

• Cozystack v1.6.4 (same plugin version on main and release-1.6), Velero v1.18.1 from the bundled chart
• Backup storage: Cloudflare R2 (S3 API), configured as the platform backup bucket
• Volume backups through Velero with CSI snapshots and the Kopia data mover

Steps to reproduce

1. Point a Velero BackupStorageLocation (or the platform cozy-default location) at a Cloudflare R2 bucket, without tagging in its config.
2. Create any Velero backup that targets that location.

Expected

The backup uploads its metadata and completes.

Actual

The upload of backup metadata fails with:

501 NotImplemented: Header 'x-amz-tagging' with value '' not implemented

and the backup does not complete.

Cause and fix

The plugin's PutObject always set Tagging. velero-plugin-for-aws#299 (on main) and its 1.14 backport #309 ("Only set PutObject Tagging when tags are configured") send the header only when tags are configured. The first 1.14 release with it is v1.14.3; the current one is v1.14.4, which adds only CVE dependency bumps. The plugin's compatibility table pairs v1.14.x with Velero v1.18.x, so the bump stays inside the supported pairing.

We have run velero-plugin-for-aws:v1.14.4 through a Package values override on a v1.6.4 staging cluster backed by R2. With it, all of our volume backup and restore drills pass. PR: #NNNN (bump to v1.14.4). Since release-1.6 ships the same v1.14.1, a backport would help R2 users on 1.6.

Workaround

Override the init container image of the velero Package to velero/velero-plugin-for-aws:v1.14.3 or later.

The fix is in #4834 (bump to v1.14.4; the tagging fix itself shipped upstream in velero-plugin-for-aws v1.14.3, velero-io/velero-plugin-for-aws#309). release-1.6 ships the same v1.14.1, so a backport would help users on v1.6.x.