monitoring: platform VictoriaLogs (vlinsert-generic) silently unavailable — logs dropped while HelmRelease is Ready #3181

Describe the bug

On a management cluster the platform monitoring stack (system/monitoring, released as monitoring in cozy-monitoring) reports Ready and its VMClusters (shortterm/longterm) run fine, but VictoriaLogs ingestion is completely broken and all logs are silently dropped.

The observed cause is a vlinsert-generic Service in cozy-monitoring that is a dangling ExternalName pointing at a VictoriaLogs instance that does not exist:

• packages/system/cozystack-basics/templates/monitoring-external-services.yaml unconditionally creates three ExternalName Services in cozy-monitoring — vlinsert-generic, vminsert-shortterm, vminsert-longterm — all targeting *.tenant-root.svc.cluster.local.
• packages/core/platform/sources/monitoring.yaml installs system/monitoring into cozy-monitoring with no logsStorages override, so the chart default (logsStorages: [{name: generic}]) applies and the release is expected to create a local VLCluster/generic. The VictoriaMetrics operator then owns an insert Service named vlinsert-generic — the same name and namespace as the ExternalName above.
• For metrics this resolves cleanly: the operator's real vminsert-shortterm/vminsert-longterm ClusterIP Services take over the ExternalName aliases.
• For logs, the VLCluster/generic object was absent (no VLCluster / VLogs present in the cluster at all), so the operator never created the real vlinsert-generic Service. The cozystack-basics ExternalName therefore stays and points to vlinsert-generic.tenant-root.svc, which does not resolve because the root Tenant has monitoring: false (the CRD default) and hosts no VictoriaLogs.

Net effect: fluent-bit ships to vlinsert-generic.cozy-monitoring.svc:9481, the ExternalName CNAME dead-ends in tenant-root, DNS returns no data, chunks fail and are dropped. The monitoring HelmRelease still reports Ready, so the failure is completely silent.

Environment

• Cozystack version: v1.4.2 (code paths also present on current main, v1.5.0)
• Provider: on-prem, bare-metal Talos

To Reproduce

1. Install Cozystack with the platform monitoring stack; leave the root Tenant at the default monitoring: false.
2. Reach a state where cozy-monitoring/monitoring is Ready but VLCluster/generic is not present (in the observed cluster the release had gone through the VLogs -> VLCluster migration).
3. kubectl -n cozy-monitoring get vlcluster -> none; kubectl -n cozy-monitoring get svc vlinsert-generic -> ExternalName -> vlinsert-generic.tenant-root.svc.
4. Inspect fluent-bit logs (below).

Expected behaviour
Platform log ingestion must not fail silently. Either the local VLCluster/generic is reliably present whenever the monitoring release is Ready, or a missing log store surfaces as a not-ready/failing condition instead of logs being dropped. A vlinsert-generic ExternalName pointing at a non-existent tenant-root VictoriaLogs (default monitoring: false) should not be a silent fallback.

Actual behaviour

• monitoring HelmRelease: Ready; VMClusters running.
• No VLCluster / VLogs anywhere; no real vlinsert-generic Service.
• vlinsert-generic in cozy-monitoring is an ExternalName -> vlinsert-generic.tenant-root.svc (owned by the cozystack-basics release); tenant-root has no VictoriaLogs.
• All logs dropped.

Logs

[net] getaddrinfo(host='vlinsert-generic.cozy-monitoring.svc', err=1): DNS server returned answer with no data
[output:http:http.0] no upstream connections available to vlinsert-generic.cozy-monitoring.svc:9481
[engine] chunk '...' cannot be retried

Additional context
Two things combine into a silent failure:

1. cozystack-basics unconditionally injects a vlinsert-generic ExternalName into cozy-monitoring that shadows the platform monitoring's own log-insert Service and points at tenant-root, which by default has no VictoriaLogs (monitoring: false).
2. The monitoring HelmRelease reports Ready even though its VLCluster/generic is not present, so nothing surfaces the broken log pipeline.

Question for maintainers: is platform VictoriaLogs intended to live locally in cozy-monitoring, or to be centralized in tenant-root (which would require the root Tenant to have monitoring: true)? The unconditional ExternalName -> tenant-root and the local logsStorages: [generic] default appear to pull in opposite directions.

Checklist

• I have checked the documentation
• I have searched for similar issues
• I have included all required information
• I have provided clear steps to reproduce
• I have included relevant logs