Stephen Roller

@stephenroller

NYC

Stephen Roller
@stephenroller
Apr 24
Didn't have hash layers on my bingo card
DeepSeek
@deepseek_ai
Apr 24
🚀 DeepSeek-V4 Preview is officially live & open-sourced! Welcome to the era of cost-effective 1M context length. 🔹 DeepSeek-V4-Pro: 1.6T total / 49B active params. Performance rivaling the world's top closed-source models. 🔹 DeepSeek-V4-Flash: 284B total / 13B active params.
3
Stephen Roller
@stephenroller
Feb 20
A fully-loaded nvl72 rack weighs 3,300lbs. Meaning the per-gpu weight is ~45lb, or one standard olympic plate.
1
Stephen Roller
@stephenroller
Aug 12, 2025
There’s a line of critique/reviewer feedback in quantization literature that the OPT models are too easy to quantize bc they’re undertrained; but all scales are trained for the same 300B tokens, making the 6.7B and smaller overtrained by Chinchilla estimates.
2
Stephen Roller
@stephenroller
Jul 15, 2025
We are moving incredibly fast. Come light up GPUs with us.
Mira Murati
@miramurati
Jul 15, 2025
Thinking Machines Lab exists to empower humanity through advancing collaborative general intelligence. We're building multimodal AI that works with how you naturally interact with the world - through conversation, through sight, through the messy way we collaborate. We're
12
Stephen Roller
@stephenroller
May 13, 2025
I once trained hyperbolic (Poincaré) networks with Riemannian SGD and HogWild. Your optimization stack does not scare me.
6