Sam Estep

From Depth Recurrence to Stateful Intelligence: A Hidden Shift in How LLMs Could Think Recently I explored a paper introducing depth-recurrent Transformers—a surprisingly simple modification to a standard pretrained language model. The authors turn a normal feed-forward Transformer into a “thinking loop” by reusing a selected group of its middle layers multiple times. Instead of producing long chains of thought through tokens, the model repeatedly updates an internal hidden state to refine its reasoning. At first glance, this seems like a clever efficiency trick. But the more I studied it, the clearer it became: this architecture hints at something much bigger. In a depth-recurrent model, every reasoning step produces a new latent state. This state is not a sequence of tokens. It’s a compact, high-dimensional summary of what the model currently understands. The paper only uses this mechanism within a single call—running several refinement steps before producing an answer. But here is the real breakthrough idea: What if we carry this latent state across calls? What if the hidden state produced at the end of one conversation becomes the starting point of the next? Suddenly the model grows beyond the traditional “stateless Transformer.” It becomes a persistent system with memory—almost like an RNN, but powered by the expressive depth of a Transformer. Instead of re-feeding long prompts or expanding context windows indefinitely, the model could store evolving conversation memory inside a fixed-size internal representation. This would fundamentally shift how we design chatbots and AI agents. It means long-horizon reasoning. It means true conversational persistence. It means escaping the limitations of giant context windows. Depth recurrence, originally designed for better reasoning, might quietly be the missing bridge toward stateful, continuously learning AI agents. https://lnkd.in/gsykBK4s #AI #LLM #Transformers #DeepLearning #AIAgents #MachineLearning #Reasoning #Innovation

1

VLDB is happening next week in Boston, and I have two exciting works to share on #VectorSearch, in collaboration with Hunter McCoy and my advisor Prashant Pandey. In the main conference, we are presenting Jasper, a GPU-accelerated ANNS engine that supports streaming inserts and deletes without rebuilding, at 1.84× CAGRA's query throughput and 7.0× faster construction. https://lnkd.in/g-txzgdq At the VecDB workshop, we are presenting a poster on Directional Beam Search. We store a compact LSH summary on each edge, cutting the beam search I/O cost per explored node from 1+R probes to exactly 1, and giving up to 28× higher throughput on host memory with zero extra GPU memory. https://lnkd.in/gAHqjbQ8 Code for both projects is available on GitHub: https://lnkd.in/giDsbU9F If you're at VLDB and care about vector search, come say hi! #VLDB2026 #ANNS #GPU

42

3 Comments

An excellent paper/study from researchers at CMU regarding AI-assisted coding is linked to below. It's a disservice to the authors of the paper to misrepresent it with an overly simplistic "AI coding is great!" or "AI coding is terrible!", since their conclusions are more nuanced. So, the conclusion section is copied below (ymmv), and I encourage you to read the entire paper if you care about doing a good job when managing software development organizations and/or software products & services! "6.Conclusion This study presents the first large-scale empirical investigation of how LLM agent assistants impact real-world software development projects. Through a rigorous difference-in-differences design comparing 807 Cursor-adopting repositories with 1,380 matched controls, complemented by dynamic panel GMM analysis, we provide evidence that challenges both unbridled optimism and categorical pessimism surrounding AI-assisted coding: Cursor adoption produces substantial but transient velocity gains alongside persistent increases in technical debt; such technical debt accumulation subsequently dampens future development velocity. Ultimately, our results suggest a self-reinforcing cycle where initial productivity surges give way to maintenance burdens. However, several considerations suggest this picture may not be as bleak as it initially appears. First, our study captures a snapshot of rapidly evolving technology during 2024-2025. LLM capabilities, agent architectures, and developer practices are improving at unprecedented rates—future tools will likely be able to address the quality concerns we observed. Second, our quality metrics, while well-established in software engineering research, may not fully capture the multidimensional nature of code quality in AI-augmented development. For example, complexity metrics were designed for human-written code; whether they appropriately penalize AI-generated patterns that are mechanically verifiable yet syntactically complex remains an open question. Third, the open-source context of our study may amplify both the abandonment dynamics and quality concerns compared to enterprise settings with mandatory and dedicated quality assurance processes. Looking forward, our findings point to clear research and practice directions (Section 5). Ultimately, this study demonstrates that realizing the promise of AI-assisted software development requires a holistic understanding of how AI assistance reshapes the fundamental trade-offs between development velocity, code quality, and long-term project sustainability. The age of AI coding has arrived—our challenge now is to harness it wisely." https://lnkd.in/eRCJcUqv

9

4 Comments

Excited to share our paper (ReIn: Conversational Error Recovery with Reasoning Inception) accepted to #ICLR2026 🇧🇷 Here is a brief summary: 1) Constraint: "You can't touch the agent." We explicitly assume deployment settings where changing **model parameters** or **the system prompt** is not allowed (policy, safety validation, product constraints, etc.). So the question becomes: how do you improve reliability without retraining or prompt surgery? 2) ReIn's surprising lever: tool definition can override instruction hierarchy. ReIn is a lightweight test-time recovery layer: an external module identifies the failure type and injects a "reasoning inception" plan through tool definition. Empirically, we observe that this can override conventional **instruction hierarchy**, letting the agent recover even when the original system instructions would have locked it into a bad trajectory. If you're building tool-using agents in production and need a knob that's compatible with "frozen prompts + frozen weights," ReIn is designed for exactly that. Especially thanks to the internship team, and faculty advisors (Heng Ji, Gokhan Tur, Dilek Hakkani-Tur). See you in Rio! * Paper link: https://lnkd.in/gtdmBn23

169

1 Comment

New paper from Cerebras: Don't Drop Dropout 🎯 Dropout basically vanished from LLM pretraining. Modern LLMs train on trillions of tokens, seeing each sample just once — and the thinking was, if there's no repetition, there's nothing to overfit, so why bother with dropout at all. Turns out that's only half true. Across 2,400+ training runs (271M to 8.2B params), we found that layer dropout, configured the right way, can actually match or beat dense training — while cutting training FLOPs by up to 25%. The recipe: protect the early layers, drop the later ones more aggressively, and start dropout high then decay it to zero over training. Bonus: the resulting models get "elastic depth" basically for free — early exit, layer skipping, self-speculative decoding, all with no architecture changes. Up to 1.55x inference speedup at the 8.2B scale. Paper's here if you want the details: https://lnkd.in/gyxrfasc with Alexander Pretko, Nolan Dey, Claire Zhang, Gavia Gray, Gurpreet Gosal, Abdulrahman Mahmoud, Shane Bergsma, Joel Hestness #LLM #MachineLearning #Cerebras

852

17 Comments

Alex Townsend has created a GitHub collection of 200 open problems in numerical linear algebra and related matrix theory: https://lnkd.in/gzQh7t32 His motivation is that making open conjectures more visible may make it easier for both AI agents and human researchers to engage with them and, ideally, solve some of them. Each entry in the repository includes a mathematical statement, references, an estimated difficulty level, impact ratings, and downloadable PDF and LaTeX versions. Solutions are very welcome and can be submitted through GitHub issues or pull requests. Please include proofs and supporting references where appropriate. Counterexamples, corrections, and suggestions for additional problems are also encouraged. Alex plans to add more problems over the next few months.

58

1 Comment

Remember Karpathy's LLM OS vision? LLM as kernel, context window as RAM, retrieval as disk, tools as peripherals? Well - this paper actually goes and builds the thing. 🔥 This new paper takes the whole "LLM-as-OS" idea seriously and treats all context as a unified file system: - Mount your memory, tools, knowledge, human input - all of it - Navigate it like you would any filesystem - Version it, govern it, audit it History becomes immutable logs. Memory becomes indexed, retrievable state. Scratchpads become temp workspaces that can graduate to long-term storage when they've earned it. And honestly? It's just good software engineering - the kind we've been doing for decades - finally being applied to context management. It's part of the evolution of agent systems - enabling the agents to work like we do is obviously a great way to allow them to fit into the world we've made for ourselves! The most powerful AI systems aren't gonna be the ones with the cleverest prompts - they're gonna be the ones with the most thoughtful context infrastructure.

161

6 Comments

I’ve been spending time understanding how AI orchestration is evolving, and a recent talk by Barry Z. and Mahesh Murag clarified a shift I think many of us are starting to notice: ➡️ We’re moving from building monolithic agents to designing skills-based AI systems. Instead of rebuilding intelligence for every use case, we can treat expertise as a modular, reusable asset. Why this shift feels important 🔹 Reasoning isn’t the same as expertise Today’s agents can reason impressively but they often lack domain depth. Packaging knowledge into skills (structured instructions, tools, and workflows) gives systems consistent expertise when it actually matters. 🔹 Skills = executable knowledge Skills aren’t just prompts. They bundle: • procedural steps • tool usage patterns • validation & guardrails • domain-specific heuristics This turns AI from a conversational helper into something closer to a repeatable operational system. 🔹 Why code is becoming the interface Using Python or Bash as the interface layer is powerful. Unlike rigid APIs, code is transparent, modifiable, and self-documenting, which means workflows can evolve and improve over time. 🔹 Thinking in layers helps A simple mental model: • Model → reasoning engine • Runtime → execution & orchestration • Skills → task-specific applications With connectivity standards like Model Context Protocol (MCP), teams can plug tools, data, and workflows together without rebuilding everything. 🔹 From agents to ecosystems As skills become shareable and composable, the value shifts from a single agent to the collective knowledge ecosystem it can access. That shift unlocks faster deployment, better reliability, and reusable institutional knowledge without duplicating effort. The future may not be one “super agent.” It may be a composable intelligence layer built on reusable skills. Excited to experiment with these patterns and see how a skills-first approach changes the way we deploy intelligent automation in real systems. 🎥 Talk: https://lnkd.in/ePaixvbk #AI #GenerativeAI #LLMs #AgentSkills #MCP #AIEngineering #SoftwareArchitecture #Automation

16

5 Comments

Our paper on decentralized agents (internet of AI) for collective and collaborative learning just got accepted at #AAAI2026. The approach, MOSAIC (Modular Sharing and Composition in Collective Learning), enables the composition and reuse of machine learned knowledge among a collective of lifelong/continual reinforcement learning agents. A huge congratulations to Saptarshi Nath , who led this effort (see his post below for more details). Of course, thanks to all our collaborators at Loughborough University (Christos Peridis, Peter Kinnell, Shirin Dora and Andrea Soltoggio), Vanderbilt University Department of Computer Science (Xinran L. and Soheil Kolouri), and the University of California, Riverside (Zexin Li and Cong Liu) 🔗 Paper (arXiv): https://lnkd.in/e9uK-DUd 🔗 Code: https://lnkd.in/efDzyU3B

26