Skip to content

vfs: add RC endpoint to insert virtual entries into the dir cache #9310

Open

Description

@christianromeni

The associated forum post URL https://forum.rclone.org/t/add-an-entry-to-vfs-via-rc-api/51242

What is your current rclone version (output from rclone version)?

Latest stable (v1.69.1)

What problem are you trying to solve?

vfs/refresh replaces the cached directory listing with whatever the backend returns. On S3-compatible backends (Garage in our case) this causes two issues:

  1. Race condition - if called shortly after an upload, the backend may not have committed the object yet, so the new entry is missing from the refreshed listing.

  2. Temporary disappearance of existing entries - on large directories, an incomplete or eventually-consistent listing response replaces the cache, causing entries to vanish until the next refresh.
    Applications watching the mount interpret this as deletions.

There is currently no way to add a single entry to the dir cache without re-listing the entire directory.

How do you think rclone should be changed to solve that?

rclone already has Dir.AddVirtual(leaf, size, isDir) internally (used by the VFS write-back cache) which inserts entries without backend re-listing, survives subsequent vfs/refresh calls, and auto-promotes
to real entries once the backend confirms them.

I propose exposing this via two new RC endpoints:

rclone rc vfs/adddir dir=data/projects/new-project
rclone rc vfs/addfile file=data/projects/report.pdf size=4521984

Use case: multiple machines mounting the same S3 backend via rclone. Machine A uploads content and notifies machine B to update its cache. vfs/refresh on large directories risks losing existing entries,
while vfs/adddir / vfs/addfile would surgically insert just the new entry.

I also noticed a bug while reading the code: VFS.AddVirtual() in vfs/vfs.go:848 hardcodes false for isDir:

dir.AddVirtual(leaf, size, false) // should be: dir.AddVirtual(leaf, size, isDir)

Happy to submit a PR for both.

Activity

  1. ncw commented on Apr 9, 2026

    @ncw
    Member

    I don't particularly like the idea of external programs adding things to the vfs cache - I'm worried about integrity of the cache. What if the thing added didn't exist?

    What do you think about a UI to refresh a directory entry (could be a file or a directory) instead? This leaves rclone in charge of updating its own cache which I would prefer. Say vfs/refresh-entry which is given a path to refresh? It would then find the directory that the path is in an refresh the single directory entry pointed to.

    I also noticed a bug while reading the code: VFS.AddVirtual() in vfs/vfs.go:848 hardcodes false for isDir:

    dir.AddVirtual(leaf, size, false) // should be: dir.AddVirtual(leaf, size, isDir)

    That does look like a bug. I traced the calls to it and I think it is only ever called with false though but, yes, please send a PR!

  2. added a commit that references this issue on Sep 21, 2026
  3. added a commit that references this issue on Oct 6, 2026
  4. added a commit that references this issue on Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions