Penghui Qi

@QPHutu

Researcher @SeaAIL PhD student @NUSingapore Working on RL, LLM Reasoning, and MLSys.

Joined August 2022

295

Following

1,549

Followers

Replies Reposts Media

Pinned
Penghui Qi
@QPHutu
Feb 5
This time we should say goodbye to PPO/GRPO for real 👋 PPO is a great algorithm in classical RL settings. However, it is fundamentally flawed in LLM regime due to the large, long-tailed vocabulary.💔 Checkout our paper for more details👇
13
Penghui Qi
@QPHutu
Sep 10
Amazing work!
Minseon Kim
@kim__minseon
Sep 8
4B parameters. ZERO distillation. 61.5% on SWE-bench Verified 🤯 Meet FrogNano 🐸: Qwen3.5-4B post-trained purely with RL on synthetic tasks from TaskPilot. Just 5 iterations × 300 tasks. Who said coding agents have to be huge? 🐸
Penghui Qi
@QPHutu
Sep 9
10000 agents🤣 so the next scaling law is the number of agents?
OpenAI
@OpenAI
Sep 8
Replying to @OpenAI
This model represents a step-function improvement on many benchmarks, and its training is ongoing. Our internal model group arrived at the Navier–Stokes solution in 88 hours, using around 10,000 coordinating AI agents. Throughout the effort, we maintained the strict
4
Penghui Qi
@QPHutu
Sep 2
super cool 😎
Yiming Shi
@yyyyyyyiming
Sep 2
Replying to @NickZhou523786 and @QPHutu
Can confirm this! Tested DPPO at ~300B scale RL Post-Training: • Prevents entropy collapse • Much faster & more stable convergence • Great length extrapolation (e.g., trained on 256k➔ keeps scaling nicely at longer context during inference) Awesome work! 🚀
Penghui Qi
@QPHutu
Sep 2
So happy to see more and more models tried DPPO and found it useful 😀
Edward Hu
@edwardjhu
Sep 1
How does one RL post-train a 397B model for long-horizon knowledge work? 👩‍💼 We share every step we took to bring Qwen 3.5 397B from 16.1% Pass@1 to 27.3% on APEX-Agents using DPPO, including final models weights and the full training script🚀 This is the first of many works from
2