Nested Learning: Ali Behrouz on the Quest for Continual Learning & Illusion of AI Architectures
Ali Behrouz, grad student at Cornell and Google researcher, discusses his potentially transformative work on new architectures for continual learning in AI. His paper "Nested Learning," praised by Jeff Dean as a possible paradigm shift, enables models to adapt to new context while preserving core knowledge by updating different layers at different frequencies, inspired by human memory systems. The conversation also covers his latest work on AI "sleep" for memory consolidation, why he sees all deep learning as associative memory, and the profound implications of continual learning for privacy, alignment, and the path to AGI.
Mercury: The fintech trusted by ambitious companies and individuals to run their finances, with virtual cards, spending limits, merchant/category locks, and AI-friendly tools like API keys, MCP, and CLI. Check out Mercury at mercury.com
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
AI 2025 → 2026 Live Show | Part 1
This year-end live show features nine rapid-fire conversations to make sense of AI’s 2025 and what might define 2026. PSA for AI builders: Interested in alignment, governance, or AI safety? Learn more about the MATS Summer 2026 Fellowship and submit your name to be notified when applications open: https://matsprogram.org/s26-tcr. Zvi Moshowitz maps the OpenAI–Anthropic–Google race, the denialism gap, and why his PDoom is still ~60–70%. Greg (ARC-AGI Prize), Eugenia Kuyda, Ali Behrouz, Logan Kirkpatrick, and Jungwon Hwang cover sample-efficient benchmarks and ARC-AGI 3, companions and human-flourishing metrics, continual-learning memory, Gemini 3 Flash for developers, and AI for scientific decisions.
Sponsors:
Gemini 3 in Google AI Studio:
Gemini 3 in Google AI Studio lets you build fully functional apps from a simple description—no coding required. Start vibe coding your idea today at https://ai.studio/build
MATS:
MATS is a fully funded 12-week research program pairing rising talent with top mentors in AI alignment, interpretability, security, and governance. Apply for the next cohort at https://matsprogram.org/s26-tcr
Framer:
Framer is the all-in-one tool to design, iterate, and publish stunning websites with powerful AI features. Start creating for free and use code COGNITIVE to get one free month of Framer Pro at https://framer.com/design
Shopify:
Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
Tasklet:
Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
CHAPTERS:
(00:00) Sponsor: Gemini 3 in Google AI Studio
(00:31) Live show experiment
(02:26) Zvi: discourse and denial
(13:28) Continual learning and doom
(22:05) ArcAGI: what's missing (Part 1)
(22:09) Sponsors: MATS | Framer
(25:28) ArcAGI: what's missing (Part 2)
(31:58) Scaffolds and tiny models
(38:58) ArcAGI 3 game worlds
(45:13) AI companions landscape (Part 1)
(45:21) Sponsors: Shopify | Tasklet
(48:29) AI companions landscape (Part 2)
(58:14) Wabi apps and caution
(01:08:16) Nested learning, layered memory
(01:23:14) Gemini 3 Flash launch
(01:34:09) RAG, agents, dev advice
(01:42:20) Elicit speeds evidence synthesis
(01:57:57) Outro
PRODUCED BY:
https://aipodcast.ing
Titans: Neural Long-Term Memory for LLMs, with author Ali Behrouz
In this episode of The Cognitive Revolution, Ali Behrouz, a PhD student at Cornell University, delves into his research on enhancing memory mechanisms in large language models through his latest paper titled Titans. Behrouz discusses the limitations of current models in maintaining long-term coherence and introduces the concept of a neural network as a memory module. Highlighting architectures such as memory as context and memory as gate, he explains how these innovative approaches can significantly improve long-term memory retention in AI systems. The discussion also touches upon challenges such as catastrophic forgetting and the need for more effective models in reinforcement learning and decision-making tasks. This insightful conversation sheds light on the future directions and potential applications of advanced memory mechanisms in AI.
SPONSORS:
ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution
Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive
The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/?utm_campaign=fy25q4_agntcy_amer_paid-media_agntcy-cognitiverevolution_podcast&utm_channel=podcast&utm_source=podcast
Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive
PRODUCED BY:
https://aipodcast.ing