Overmind

@OvermindLab

Agent optimisation & model training

London

overmindlab.ai

Joined January 2026

Following

Followers

Replies Reposts Media Articles

Pinned
Overmind
@OvermindLab
Aug 26
Frontier models are overkill for agent tasks. Specialised models beat them on cost, latency, and accuracy. Overmind maps your agent, turns traces into training and eval data, and trains an open-weight model benchmarked against the model you run today. Ship the model as a PR.
10
Overmind
@OvermindLab
1h
An LLM judge can change its verdict on an edge case. Three checks before trusting the score: 1. Score the same answer against the same rubric repeatedly, using fresh judge calls. 2. Inspect disagreements. 3. Compare with held-back human labels.
2
Overmind
@OvermindLab
Sep 10
"Ask someone to grab me a latte and they respond by opening a coffee plantation. Eventually it arrives, the output is correct, but the process was absurd." The signal is in how the agent got there. Mark the behaviours, not the result in isolation.
Tyler Edwards
@TylerEdwas
Sep 10
Most agent evals just mark the final output which misses the point. The interesting signal is in how the agent got there. Ask someone to grab me a latte and they respond by opening a coffee plantation. Eventually it arrives, the output is correct, but the process was absurd.
3
Overmind
@OvermindLab
Sep 10
Before the next prompt change: 1. Save representative failures. 2. Separate tuning cases from held-out tests. 3. Record the baseline. 4. Change one thing. 5. Inspect which cases improved or regressed. A higher average can still hide a broken task.
2
Overmind
@OvermindLab
Sep 9
Own the model your product runs on. Train it from your own agent traces. @TylerEdwas spent six years building ML systems for British Intelligence before founding Overmind. He’ll walk through the full path from traces to a shipped open-weights model. DevGuild, SF, 24 September.
Heavybit
@heavybit
Sep 8
An autonomous org has to improve itself without anyone stepping in. Six years building ML systems for British Intelligence, and now at @[OVERMIND], @[TYLER] automates the path from agent traces to a shipped model. DevGuild, Sept 24 → luma.com/long-horizon-s…
3