Skip to content

[Bug]: Typed helper: proxmox.lxc_filesystems always fails on PVE because PrivateNetwork=true blocks pmxcfs IPC #2511

Open

Description

@ddkedr

Environment

  • Proxmox VE 9.2.20 (Debian 13), standalone node, 16 running LXC
  • Pulse server and agent v6.5.0-rc.1 (same behaviour on v6.4.3-rc.1)
  • Agent installed with --least-privilege --enable-privileged-helper --enable-proxmox --proxmox-type pve

What happens

Every proxmox.lxc_filesystems call through pulse-agent-helper fails, while smart.snapshot works:

operation="proxmox.lxc_filesystems" operation_version=1 ... success=false error_code="provider_unavailable" duration_ms=579
operation="smart.snapshot" operation_version=1 ... success=true error_code=""

About 1,600 failures per day, from the first helper start onwards. The failure takes ~0.6 s, so it is not the collection timeout.

Cause

pct reads guest configuration from pmxcfs (/etc/pve) through the libqb IPC socket, which lives in the abstract Unix socket namespace. Abstract sockets are scoped to the network namespace, so the helper unit's PrivateNetwork=true cuts pct off from pmxcfs. pct list then fails, collectProxmoxLXCFilesystemsResult marks the result Degraded, and localProxmoxProvider.LXCFilesystems returns provider_unavailable.

Reproduction on the node, as root, isolating just that one unit property:

# systemd-run --wait --pipe --quiet -p PrivateNetwork=yes /usr/sbin/pct list
ipcc_send_rec[1] failed: Connection refused
ipcc_send_rec[2] failed: Connection refused
ipcc_send_rec[3] failed: Connection refused

# systemd-run --wait --pipe --quiet /usr/sbin/pct list
VMID       Status     Lock         Name
101        running                 jellyfin
102        running                 searxng
...

Relevant code and docs

  • cmd/pulse-agent-helper/providers.go, localProxmoxProvider.LXCFilesystems
  • internal/hostagent/proxmox_lxc_filesystems.go, collectProxmoxLXCFilesystemsResult (pct list, then pct df <vmid>)
  • docs/AGENT_SECURITY.md lists this capability as "Implemented, unqualified on a representative PVE node" and the helper unit as PrivateNetwork=true, RestrictAddressFamilies=AF_UNIX

Related: #1784 fixed the per-container timeout budget for this collector and added the statfs fast path via lxc-info. That path is not reached here, because pct list already fails. lxc-info also talks to the LXC monitor over an abstract socket, so the fast path may hit the same namespace boundary inside the helper.

Possible directions

  • Keep the helper network-isolated but let the LXC provider reach pmxcfs, for example by running only this provider outside the private network namespace, or by reading the guest list without IPCC (/proc plus lxc-info, which the collector already uses for its statfs fast path).
  • Separately: one failed container currently discards the whole snapshot. Returning the containers that did succeed would degrade more gracefully.

Data is not lost in practice, since LXC disk usage still arrives through the Proxmox API connection. The visible effect is the constant failure log and missing per-mount detail from the host agent.

Activity

  1. pulse-triage commented on Oct 5, 2026

    @pulse-triage
    Contributor

    The local collector keeps successful rows, but the typed helper rejects any incomplete inventory; that separate completeness case is now #2513. Keep the helper's network isolation and least-privilege profile intact and use the Proxmox API disk-usage readings you already receive: overriding PrivateNetwork or switching the collector to root bypasses the supported profile rather than repairing this operation.

    How Pulse handles triage

  2. added a commit that references this issue on Oct 6, 2026
    4d8bede
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions