Today I'm releasing LlamaStash 0.0.2: a zero-overhead, terminal-native launcher for llama.cpp.
One Rust binary that's a TUI, a CLI, a daemon, and an OpenAI-compatible proxy.
Demo below 🧵
Just published pi-automode-classifier, an auto mode plugin for the Pi coding agent that uses Jev or Kev/Laya (running locally) to classify commands.
Risky commands need my approval before they run.
pi install npm:pi-automode-classifier
I gave the same complex Rust feature to Opus 5.5 and Qwen3.8-Flash-Next running locally. Opus did it 7 times faster but Qwen made the better PR which eventually got merged
Full writeup
LlamaStash v0.6.0 is out 🦙
Highlights:
- a request that doesn't fit now unloads idle models to make room, and 'presets save --idle-ttl 0 --preload' keeps a model warm.
- Claude Code's /effort now reaches llama.cpp models.
I benchmarked engines for Qwen 3.8-Flash-Next on Strix Halo (Flow Z13, 128 GB). Halogen with its native weights is fastest at 39-40 t/s decode, then gufo and CIRU. But Halogen is closed source. gufo is open source and loads 4x faster from cold.