Akarsh Kumar

@akarshkumar0101

PhD Student @MIT_CSAIL RS Intern @SakanaAILabs RL, Meta-Learning, Emergence, Open-Endedness, ALife

akarshkumar.com

Joined February 2018

Replies Reposts Media

Pinned
Akarsh Kumar
@akarshkumar0101
Jun 7
We never really knew how to train nonlinear RNNs well… BPTT struggled with vanishing grads (no long-range memory) and sequential rollout (hard to parallelizable). What if instead an oracle told us the optimal memory state m_t at each step? Then the RNN could do one-step
18
Akarsh Kumar
@akarshkumar0101
Aug 26
Second edition of "The Surprising Creativity of Digital Evolution" but with general AI systems now! This is one of those papers that everyone should read, even nontechnical people. Great work @_aadharna and team! Awesome title too
Jeff Clune
@jeffclune
Aug 26
Excited to share “AI Finds a Way.” 🦖 🦕✨ 🤖 AI can be surprisingly creative, outsmarting the researchers who use it. That can lead to scientific breakthroughs, superhuman capabilities, and generating new knowledge. Such creativity can also be mischievous, raising safety
2
Akarsh Kumar
@akarshkumar0101
Aug 23
Recurrence was always destined to come back. The question now is what is the right format for recurrent memory? Agentic markdown files or something more latent? Our paper is a bet on latent memory.
Christopher Manning
@chrmanning
Aug 23
When teaching CS224N, I thought it important to introduce students to a broad neural toolbox – not just transformers but FFNs, CNNs, LSTMs, tree-recursive NNs, BiDAF QA nets, highway nets, …. I think the resurgence of work using recurrence shows the importance of this approach.
3
Akarsh Kumar
@akarshkumar0101
Jul 31
Very elegant idea! The best kind of work has the simplest pseudocode. Also ties into ideas from evolution about maintaining diversity via exploration (i.e. you want enough exploration such that at least one of your mutations is good).
Alexi Gladstone
@AlexiGlad
Jul 31
We discovered a third pretraining axis beyond parameters and data: exploration. Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation. In the simplest case, it's just a for loop. Introducing Explorative
5
Akarsh Kumar
@akarshkumar0101
Jul 27
If you don't know much about ALife, this is a great video to start! Explanations like these are why I got into research in the first place. Great video @max_romana!
Max | Emergent Garden
@max_romana
Jul 27
New video on Artificial Life🦠 I play with some amazing programs that begin to simulate life-like behaviors and evolutionary systems. (Full video below👇) What are we missing for truly open-ended evolution?
00:00