docker volume plugin: state restore after reboot loses volume path (mounts at remote root); stale mounts state also blocks plugin start #9853

What is the problem you are having with rclone?

rclone serve docker (the Docker volume plugin) mishandles state restoration after a host reboot or docker plugin disable/docker plugin enable. Two related defects, both in Volume.restoreState() (cmd/serve/docker/volume.go):

1. The path option is lost — volumes remount at the remote root.

For volumes created with the documented type + path driver options (e.g. type: s3, path: bucket/subdir), the persisted state file is correct:

{"name": "calibre-web-automated_calibre-library", "mountpoint": "/mnt/calibre-web-automated_calibre-library", "fs": "", "type": "s3", "path": "calibre-library", ...}

But restoreState() only feeds fs and type back into applyOptions():

volOpt := vol.Options
volOpt["fs"] = vol.Fs
volOpt["type"] = vol.Type
if err := vol.applyOptions(volOpt); err != nil {

vol.Path is persisted but never restored, and applyOptions() deliberately excludes "path" from vol.Options (it consumes it into explicitPath). The rebuilt fsString therefore becomes the bare :s3,provider=...,...: connection string with no path, and the volume mounts at the remote root instead of bucket/subdir.

Observed in production (S3/Cloudflare R2 with bucket-scoped tokens): after reboot, mount shows :s3{...}:calibre-library → :s3{...}: (no path); every root-level directory operation becomes ListBuckets, which R2 denies with 403, so all I/O through the mount returns EIO. Containers that autostart at boot then fail (e.g. a SQLite app gets unable to open database file). This reproduces on every reboot.

2. Stale mounts entries in the saved state block plugin startup entirely (v1.74.3).

v1.74.3 restoreState() re-mounts synchronously for every saved mount ID:

for _, id := range vol.Mounts {
    if err := vol.mount(id); err != nil {
        return err
    }
}

If the plugin was killed while containers held mounts (unclean shutdown, docker plugin disable --force), the saved mounts arrays still contain container IDs. On the next docker plugin enable, rclone restores the volumes and synchronously re-mounts all of them before serving the plugin socket, which can exceed dockerd's enable timeout — the socket never appears, and every subsequent docker plugin enable fails with:

Error response from daemon: dial unix /run/docker/plugins/<plugin-id>/rclone.sock: connect: no such file or directory

...while the rclone process is actually alive and busy restoring. The plugin becomes un-startable until the mounts arrays are manually cleared in the state file (/var/lib/docker-plugins/rclone/cache/docker-plugin.state) or dockerd's timeout luck runs out.

I see master has moved to deferred mounting (pendingMounts / getPendingMounts), which addresses defect 2 — but defect 1 (the dropped path) is still present in master (restoreState() is unchanged in this respect).

Reproduction (no cloud credentials needed)

$ docker volume create --driver rclone --opt type=local --opt path=/tmp/rt vol-restore-test
$ docker run --rm -v vol-restore-test:/t alpine sh -c 'echo hello > /t/marker.txt; ls /t'
marker.txt                      # path honored: mount is rooted at /tmp/rt
$ docker plugin disable rclone && docker plugin enable rclone
$ docker run --rm -v vol-restore-test:/t alpine ls /t
bin  data  dev  home  ...      # BUG: mount is now the remote root, not /tmp/rt

With a local backend this is harmless-looking; with s3 + a bucket-scoped token it makes every path resolution fail with 403/EIO, since the mount root now requires ListBuckets.

Suggested fix

In restoreState(), restore the persisted path before calling applyOptions():

volOpt := vol.Options
volOpt["fs"] = vol.Fs
volOpt["type"] = vol.Type
if vol.Path != "" && vol.Fs == "" {   // type+path volumes
    volOpt["path"] = vol.Path
}

(and equivalently keep path working for connection-string volumes, where it is already embedded in vol.Fs).

Workaround for users until fixed: create volumes with a remote connection-string driver option (remote: ':s3,provider=...,...:bucket/subdir') instead of type + path — vol.Fs is persisted and restored intact, and reboots then work correctly. Verified on a production VPS: after converting four volumes this way, a plugin restart re-established the correct bucket-scoped mounts.

What is your rclone version?

Plugin bundles rclone v1.74.3 (plugin rclone/docker-volume-rclone:amd64, installed June 2026). The restoreState() path-loss is also present in current master (cmd/serve/docker/volume.go).

Which OS you are using

Ubuntu 26.04 (kernel 7.0.0-30-generic), Docker Engine 29.7.2, managed plugin.

Which cloud storage system are you using?

S3 (Cloudflare R2, bucket-scoped API tokens) — four volumes; also reproduced with type=local.