Vault

A private file manager that uploads into your Google Drive. People you allow can sign in, upload folders in bulk, and organise files, but every byte lands in your Drive quota instead of theirs.

Built for one job: hand a photographer a link, let him push 1 TB of wedding photos and video over a flaky connection, and have it survive dropped wifi, closed laptops, browser crashes and server restarts without losing or duplicating anything.

The trick is where the bytes go.

browser  ------ control plane, small json calls ------>  FastAPI on your VM
   |                                                            |
   |                                                    OAuth as your account
   |                                                            v
   +------ data plane, file chunks straight to Google ---->  Google Drive

The server never touches file content. It authenticates as you, asks Drive to open a resumable upload session, and hands the session URL back to the browser. The browser then PUTs chunks directly to Google.

Three things fall out of that:

1. Files are created by your account, so they consume your storage, not the uploader's.
2. A terabyte never passes through your VM. No bandwidth bill, no request timeouts, and a 2 GB machine is plenty.
3. Resumable sessions are Google's own protocol, so resuming after a failure is exact rather than a guess.

Uploading a folder creates the same folder tree in Drive. Folder creation is deduplicated with a lock plus a local cache, so a thousand files in one directory do not create a thousand folders.

• A Google account with the storage you want to fill (a personal Gmail with Google One works)
• A Google Cloud project (free, the Drive API costs nothing)
• A domain name pointed at your VM, needed for HTTPS
• Docker on the machine you deploy to

1. Go to https://console.cloud.google.com and create a project, or reuse one.

2. Enable the Drive API: APIs and services, Library, search "Google Drive API", Enable.

3. Open APIs and services, OAuth consent screen. Choose External. Fill in app name, your email, developer email. Save.

4. On the Data access (or Scopes) page, add one scope only:

   https://www.googleapis.com/auth/drive.file
   

5. Go back to the Audience page and click Publish app, so publishing status reads "In production".

   This step matters more than it looks. drive.file is a non sensitive scope, so publishing needs no verification and no security assessment. If you leave the app in Testing mode instead, Google expires your refresh token after 7 days and uploads stop working halfway through the job.

6. Credentials, Create credentials, OAuth client ID, application type Desktop app. Copy the client ID and client secret.

drive.file means the app can only see files it created itself. It cannot read the rest of your Drive, which is the point. It also means you should create the root folder through this app rather than in the Drive web UI, otherwise the app will not be able to see it.

Run this on a machine with a browser (your laptop, not the server):

export GOOGLE_CLIENT_ID=xxxx.apps.googleusercontent.com
export GOOGLE_CLIENT_SECRET=xxxx
python3 scripts/get_refresh_token.py

It opens a consent page, you approve with the account that owns the storage, and it prints a GOOGLE_REFRESH_TOKEN=... line. Keep it secret, it is full access to files this app creates.

If it says no refresh token was returned, revoke the app at https://myaccount.google.com/permissions and run it again.

cp .env.example .env
openssl rand -hex 32   # paste as SECRET_KEY

Fill in .env:

GOOGLE_CLIENT_ID=xxxx.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=xxxx
GOOGLE_REFRESH_TOKEN=1//xxxx

APP_USERS=nazmus:a-long-password,photographer:another-long-password
SECRET_KEY=the-openssl-output

PUBLIC_ORIGIN=https://vault.example.com
DOMAIN=vault.example.com

APP_USERS is a comma separated list of name:password. Add or remove people by editing this line and restarting. Login is rate limited to 8 attempts per IP per 5 minutes.

PUBLIC_ORIGIN must be the exact URL people will visit, scheme included. It is sent as the Origin header when opening upload sessions, and a mismatch shows up as CORS errors on the chunk PUTs.

Optional knobs:

Two terminals.

# terminal 1
cd server
pip install -r requirements.txt
set -a && source ../.env && set +a
PUBLIC_ORIGIN=http://localhost:5173 uvicorn app.main:app --reload --port 8000

# terminal 2
cd web
npm install
npm run dev

Open http://localhost:5173. Vite proxies /api to port 8000.

To run the production build in one process:

cd web && npm run build && cd ..
cd server && STATIC_DIR=../web/dist uvicorn app.main:app --port 8000

A small VM is enough because file data bypasses it.

Reserve an IP and create the VM.

gcloud config set project YOUR_PROJECT_ID

gcloud compute addresses create vault-ip --region=asia-southeast1

gcloud compute instances create vault \
  --zone=asia-southeast1-b \
  --machine-type=e2-small \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --boot-disk-size=20GB \
  --boot-disk-type=pd-balanced \
  --address=vault-ip \
  --tags=http-server,https-server

asia-southeast1 (Singapore) is the closest region to Dhaka. Any region works.

Open the firewall.

gcloud compute firewall-rules create vault-allow-http \
  --allow=tcp:80 --target-tags=http-server --direction=INGRESS

gcloud compute firewall-rules create vault-allow-https \
  --allow=tcp:443 --target-tags=https-server --direction=INGRESS

Skip either rule if your project already has it. Port 80 must stay open, Let's Encrypt uses it to issue the certificate.

Point DNS at the VM. Get the IP:

gcloud compute addresses describe vault-ip --region=asia-southeast1 --format='value(address)'

Create an A record for vault.example.com pointing at it. Wait until dig +short vault.example.com returns that IP before continuing, otherwise the certificate request fails.

Install Docker.

gcloud compute ssh vault --zone=asia-southeast1-b

sudo apt-get update && sudo apt-get install -y ca-certificates curl git
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo tee /etc/apt/keyrings/docker.asc > /dev/null
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
exit

Log back in so the group membership takes effect.

Deploy.

gcloud compute ssh vault --zone=asia-southeast1-b
git clone YOUR_REPO_URL vault && cd vault
nano .env          # paste the config from step 3
docker compose up -d --build
docker compose logs -f

Wait for drive root folder ready: <id> in the logs and certificate obtained successfully from Caddy. Open https://vault.example.com and sign in.

Caddy handles TLS certificates and renewal by itself. There is nothing to configure beyond DOMAIN.

If the build runs out of memory on e2-small, add swap once:

sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile
sudo mkswap /swapfile && sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Send the photographer the URL and his username and password. Tell him:

• Drag a whole folder onto the page, or use New, Folder upload
• The folder structure is recreated in Drive
• He can close the laptop or lose wifi, the queue continues when he comes back
• If the browser was force closed and the dock shows "Reselect folder", he picks the same folder again and it resumes from the exact byte, it does not start over

Chrome or Edge is the best experience for folder uploads. Firefox works. Safari is fine for single files but its folder handling is weaker.

Update after a code change

git pull && docker compose up -d --build

Back up state

docker run --rm -v vault_state:/data -v $PWD:/backup alpine \
  tar czf /backup/vault-state.tar.gz -C /data .

The volume name is your project directory name plus _state, check with docker volume ls.

This database holds the upload registry and folder cache, not your files. If you lose it, nothing in Drive is lost. In flight uploads restart from zero, finished ones are detected as duplicates and skipped.

Change passwords

Edit APP_USERS in .env, then docker compose up -d. Existing sessions keep working until they expire, so also change SECRET_KEY if you want to kick everyone out immediately.

Logs

docker compose logs -f app
docker compose logs -f caddy

Every login, rename, move, delete and checksum failure is also recorded in the events table in the state database.

Health check

GET /healthz            liveness
GET /api/uploads/stats  counts and bytes by upload status, needs a session

Why the browser uploads directly to Google. Proxying 1 TB through the VM would mean egress charges, request timeouts, and a much larger machine. It would also make resuming harder, because the server would need to track partial state that Google already tracks perfectly.

Why drive.file instead of full Drive access. It is a non sensitive scope, so no Google verification, no security assessment, and no 7 day token expiry. It also limits the blast radius: a leaked token cannot read your personal Drive, only files this app created.

How resume actually works. The client tracks its own offset optimistically. On any failure it stops trusting that and calls GET /api/uploads/{key}/status, which makes the server ask Google Content-Range: bytes */size on the session. Google replies with exactly how many bytes it has. The client continues from there. The reason this goes through the server rather than the browser is that Google does not always expose the Range response header to cross origin JavaScript, so reading it client side is not dependable.

How deduplication works. Every file gets a key derived from its relative path, size and modification time. That key is written to the Drive file as an appProperty, which is queryable. Before opening a session the server checks for a file carrying that key. If it exists with a matching size, the upload is skipped entirely. This survives renames and works across browsers and machines. Name plus size in the target folder is checked as a second line of defence, and repeats of a file that changed are uploaded as a new revision of the same file rather than a second copy.

Chunk sizing. Starts at 16 MB. Halves on any failure down to 1 MB, doubles after three consecutive clean chunks up to 64 MB. Every size stays a multiple of 256 KB, which Google requires for all but the final chunk.

Integrity. MD5 is computed incrementally as chunks are read, so verification costs one pass, not two. If an upload resumes from the middle, the running hash is discarded and the file is hashed in a separate pass at the end instead. A mismatch trashes the Drive file and resets the item so it uploads again.

Backend, against a mock Drive API:

cd server && python -m pytest tests/ -q

Covers auth, duplicate skipping, resume offsets, checksum mismatch handling, retry on transient 5xx, and idempotent nested folder creation.

Upload engine, with a simulated hostile Drive:

cd web && npm test

Covers multi chunk transfers, network drops mid transfer, session expiry and restart, concurrency limits, and duplicate skipping. It asserts on every chunk that the size is 256 KB aligned and that the body length matches the Content-Range header, and it checks that the MD5 is still correct after a forced restart.

server/app/
  main.py            app wiring, SPA serving, background sweeper
  config.py          env parsing
  auth.py            cookie sessions, login throttling
  drive.py           Drive REST client, token refresh, retries, resumable sessions
  folders.py         root folder bootstrap, concurrency safe path creation
  routes_files.py    browse, rename, move, trash, download, thumbnails
  routes_uploads.py  session create, status probe, complete, dedup
  db.py              SQLite upload registry and folder cache

web/src/
  App.jsx            shell, browsing, dialogs, drag and drop
  api.js             fetch wrapper with retry and 401 handling
  upload/manager.js  the queue, chunking, retries, resume, verification
  upload/store.js    IndexedDB persistence
  upload/entries.js  directory traversal for drag and drop
  components/        UI

• Passwords sit in plain text in .env because that is what the deployment model calls for. Keep the file at mode 600 and do not commit it. Anyone with server access can read them.
• Resumable sessions expire after roughly a week. The sweeper marks stale ones expired every hour, and a stale upload transparently opens a new session.
• Because of drive.file, folders you create by hand in the Drive web UI are invisible to this app. Create them in the app.
• Downloads stream through the server, so large downloads do use VM bandwidth. Uploads, the part that matters here, do not.