Sakana AI - Chris Lu, Robert Tjarko Lange, Cong Lu
We speak with Sakana AI, who are building nature-inspired methods that could fundamentally transform how we develop AI systems.
The guests include Chris Lu, a researcher who recently completed his DPhil at Oxford University under Prof. Jakob Foerster's supervision, where he focused on meta-learning and multi-agent systems. Chris is the first author of the DiscoPOP paper, which demonstrates how language models can discover and design better training algorithms. Also joining is Robert Tjarko Lange, a founding member of Sakana AI who specializes in evolutionary algorithms and large language models. Robert leads research at the intersection of evolutionary computation and foundation models, and is completing his PhD at TU Berlin on evolutionary meta-learning. The discussion also features Cong Lu, currently a Research Scientist at Google DeepMind's Open-Endedness team, who previously helped develop The AI Scientist and Intelligent Go-Explore.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting!
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Goto https://tufalabs.ai/
***
* DiscoPOP - A framework where language models discover their own optimization algorithms
* EvoLLM - Using language models as evolution strategies for optimization
The AI Scientist - A fully automated system that conducts scientific research end-to-end
* Neural Attention Memory Models (NAMMs) - Evolved memory systems that make transformers both faster and more accurate
TRANSCRIPT + REFS:
https://www.dropbox.com/scl/fi/gflcyvnujp8cl7zlv3v9d/Sakana.pdf?rlkey=woaoo82943170jd4yyi2he71c&dl=0
Robert Tjarko Lange
https://roberttlange.com/
Chris Lu
https://chrislu.page/
Cong Lu
https://www.conglu.co.uk/
Sakana
https://sakana.ai/blog/
TOC:
1. LLMs for Algorithm Generation and Optimization
[00:00:00] 1.1 LLMs generating algorithms for training other LLMs
[00:04:00] 1.2 Evolutionary black-box optim using neural network loss parameterization
[00:11:50] 1.3 DiscoPOP: Non-convex loss function for noisy data
[00:20:45] 1.4 External entropy Injection for preventing Model collapse
[00:26:25] 1.5 LLMs for black-box optimization using abstract numerical sequences
2. Model Learning and Generalization
[00:31:05] 2.1 Fine-tuning on teacher algorithm trajectories
[00:31:30] 2.2 Transformers learning gradient descent
[00:33:00] 2.3 LLM tokenization biases towards specific numbers
[00:34:50] 2.4 LLMs as evolution strategies for black box optimization
[00:38:05] 2.5 DiscoPOP: LLMs discovering novel optimization algorithms
3. AI Agents and System Architectures
[00:51:30] 3.1 ARC challenge: Induction vs. transformer approaches
[00:54:35] 3.2 LangChain / modular agent components
[00:57:50] 3.3 Debate improves LLM truthfulness
[01:00:55] 3.4 Time limits controlling AI agent systems
[01:03:00] 3.5 Gemini: Million-token context enables flatter hierarchies
[01:04:05] 3.6 Agents follow own interest gradients
[01:09:50] 3.7 Go-Explore algorithm: archive-based exploration
[01:11:05] 3.8 Foundation models for interesting state discovery
[01:13:00] 3.9 LLMs leverage prior game knowledge
4. AI for Scientific Discovery and Human Alignment
[01:17:45] 4.1 Encoding Alignment & Aesthetics via Reward Functions
[01:20:00] 4.2 AI Scientist: Automated Open-Ended Scientific Discovery
[01:24:15] 4.3 DiscoPOP: LLM for Preference Optimization Algorithms
[01:28:30] 4.4 Balancing AI Knowledge with Human Understanding
[01:33:55] 4.5 AI-Driven Conferences and Paper Review
Prof. Jakob Foerster - ImageNet Moment for Reinforcement Learning?
Prof. Jakob Foerster, a leading AI researcher at Oxford University and Meta, and Chris Lu, a researcher at OpenAI -- they explain how AI is moving beyond just mimicking human behaviour to creating truly intelligent agents that can learn and solve problems on their own. Foerster champions open-source AI for responsible, decentralised development. He addresses AI scaling, goal misalignment (Goodhart's Law), and the need for holistic alignment, offering a quick look at the future of AI and how to guide it.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting!
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich.
Goto https://tufalabs.ai/
***
TRANSCRIPT/REFS:
https://www.dropbox.com/scl/fi/yqjszhntfr00bhjh6t565/JAKOB.pdf?rlkey=scvny4bnwj8th42fjv8zsfu2y&dl=0
Prof. Jakob Foerster
https://x.com/j_foerst
https://www.jakobfoerster.com/
University of Oxford Profile:
https://eng.ox.ac.uk/people/jakob-foerster/
Chris Lu:
https://chrislu.page/
TOC
1. GPU Acceleration and Training Infrastructure
[00:00:00] 1.1 ARC Challenge Criticism and FLAIR Lab Overview
[00:01:25] 1.2 GPU Acceleration and Hardware Lottery in RL
[00:05:50] 1.3 Data Wall Challenges and Simulation-Based Solutions
[00:08:40] 1.4 JAX Implementation and Technical Acceleration
2. Learning Frameworks and Policy Optimization
[00:14:18] 2.1 Evolution of RL Algorithms and Mirror Learning Framework
[00:15:25] 2.2 Meta-Learning and Policy Optimization Algorithms
[00:21:47] 2.3 Language Models and Benchmark Challenges
[00:28:15] 2.4 Creativity and Meta-Learning in AI Systems
3. Multi-Agent Systems and Decentralization
[00:31:24] 3.1 Multi-Agent Systems and Emergent Intelligence
[00:38:35] 3.2 Swarm Intelligence vs Monolithic AGI Systems
[00:42:44] 3.3 Democratic Control and Decentralization of AI Development
[00:46:14] 3.4 Open Source AI and Alignment Challenges
[00:49:31] 3.5 Collaborative Models for AI Development
REFS
[[00:00:05] ARC Benchmark, Chollet
https://github.com/fchollet/ARC-AGI
[00:03:05] DRL Doesn't Work, Irpan
https://www.alexirpan.com/2018/02/14/rl-hard.html
[00:05:55] AI Training Data, Data Provenance Initiative
https://www.nytimes.com/2024/07/19/technology/ai-data-restrictions.html
[00:06:10] JaxMARL, Foerster et al.
https://arxiv.org/html/2311.10090v5
[00:08:50] M-FOS, Lu et al.
https://arxiv.org/abs/2205.01447
[00:09:45] JAX Library, Google Research
https://github.com/jax-ml/jax
[00:12:10] Kinetix, Mike and Michael
https://arxiv.org/abs/2410.23208
[00:12:45] Genie 2, DeepMind
https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
[00:14:42] Mirror Learning, Grudzien, Kuba et al.
https://arxiv.org/abs/2208.01682
[00:16:30] Discovered Policy Optimisation, Lu et al.
https://arxiv.org/abs/2210.05639
[00:24:10] Goodhart's Law, Goodhart
https://en.wikipedia.org/wiki/Goodhart%27s_law
[00:25:15] LLM ARChitect, Franzen et al.
https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf
[00:28:55] AlphaGo, Silver et al.
https://arxiv.org/pdf/1712.01815.pdf
[00:30:10] Meta-learning, Lu, Towers, Foerster
https://direct.mit.edu/isal/proceedings-pdf/isal2023/35/67/2354943/isal_a_00674.pdf
[00:31:30] Emergence of Pragmatics, Yuan et al.
https://arxiv.org/abs/2001.07752
[00:34:30] AI Safety, Amodei et al.
https://arxiv.org/abs/1606.06565
[00:35:45] Intentional Stance, Dennett
https://plato.stanford.edu/entries/ethics-ai/
[00:39:25] Multi-Agent RL, Zhou et al.
https://arxiv.org/pdf/2305.10091
[00:41:00] Open Source Generative AI, Foerster et al.
https://arxiv.org/abs/2405.08597
<trunc, see PDF/YT>