Jeff Clune - Agent AI Needs Darwin
AI professor Jeff Clune ruminates on open-ended evolutionary algorithms—systems designed to generate novel and interesting outcomes forever. Drawing inspiration from nature’s boundless creativity, Clune and his collaborators aim to build “Darwin Complete” search spaces, where any computable environment can be simulated. By harnessing the power of large language models and reinforcement learning, these AI agents continuously develop new skills, explore uncharted domains, and even cooperate with one another in complex tasks.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events?
They are hosting an event in Zurich on January 9th with the ARChitects, join if you can.
Goto https://tufalabs.ai/
***
A central theme throughout Clune’s work is “interestingness”: an elusive quality that nudges AI agents toward genuinely original discoveries. Rather than rely on narrowly defined metrics—which often fail due to Goodhart’s Law—Clune employs language models to serve as proxies for human judgment. In doing so, he ensures that “interesting” always reflects authentic novelty, opening the door to unending innovation.
Yet with these extraordinary possibilities come equally significant risks. Clune says we need AI safety measures—particularly as the technology matures into powerful, open-ended forms. Potential pitfalls include agents inadvertently causing harm or malicious actors subverting AI’s capabilities for destructive ends. To mitigate this, Clune advocates for prudent governance involving democratic coalitions, regulation of cutting-edge models, and global alignment protocols.
Jeff Clune:
https://x.com/jeffclune
http://jeffclune.com/
(Interviewer: Tim Scarfe)
TOC:
1. Introduction
[00:00:00] 1.1 Overview and Opening Thoughts
2. Sponsorship
[00:03:00] 2.1 TufaAI Labs and CentML
3. Evolutionary AI Foundations
[00:04:12] 3.1 Open-Ended Algorithm Development and Abstraction Approaches
[00:07:56] 3.2 Novel Intelligence Forms and Serendipitous Discovery
[00:11:46] 3.3 Frontier Models and the 'Interestingness' Problem
[00:30:36] 3.4 Darwin Complete Systems and Evolutionary Search Spaces
4. System Architecture and Learning
[00:37:35] 4.1 Code Generation vs Neural Networks Comparison
[00:41:04] 4.2 Thought Cloning and Behavioral Learning Systems
[00:47:00] 4.3 Language Emergence in AI Systems
[00:50:23] 4.4 AI Interpretability and Safety Monitoring Techniques
5. AI Safety and Governance
[00:53:56] 5.1 Language Model Consistency and Belief Systems
[00:57:00] 5.2 AI Safety Challenges and Alignment Limitations
[01:02:07] 5.3 Open Source AI Development and Value Alignment
[01:08:19] 5.4 Global AI Governance and Development Control
6. Advanced AI Systems and Evolution
[01:16:55] 6.1 Agent Systems and Performance Evaluation
[01:22:45] 6.2 Continuous Learning Challenges and In-Context Solutions
[01:26:46] 6.3 Evolution Algorithms and Environment Generation
[01:35:36] 6.4 Evolutionary Biology Insights and Experiments
[01:48:08] 6.5 Personal Journey from Philosophy to AI Research
Shownotes:
We craft detailed show notes for each episode with high quality transcript and references and best parts bolded.
https://www.dropbox.com/scl/fi/fz43pdoc5wq5jh7vsnujl/JEFFCLUNE.pdf?rlkey=uu0e70ix9zo6g5xn6amykffpm&st=k2scxteu&dl=0
Automated Design of Agentic Systems with Shengran Hu - #700
Today, we're joined by Shengran Hu, a PhD student at the University of British Columbia, to discuss Automated Design of Agentic Systems (ADAS), an approach focused on automatically creating agentic system designs. We explore the spectrum of agentic behaviors, the motivation for learning all aspects of agentic system design, the key components of the ADAS approach, and how it uses LLMs to design novel agent architectures in code. We also cover the iterative process of ADAS, its potential to shed light on the behavior of foundation models, the higher-level meta-behaviors that emerge in agentic systems, and how ADAS uncovers novel design patterns through emergent behaviors, particularly in complex tasks like the ARC challenge. Finally, we touch on the practical applications of ADAS and its potential use in system optimization for real-world tasks.
The complete show notes for this episode can be found at https://twimlai.com/go/700.
Accelerating Intelligence with AI-Generating Algorithms with Jeff Clune - #602
Are AI-generating algorithms the path to artificial general intelligence(AGI)?
Today we’re joined by Jeff Clune, an associate professor of computer science at the University of British Columbia, and faculty member at the Vector Institute. In our conversation with Jeff, we discuss the broad ambitious goal of the AI field, artificial general intelligence, where we are on the path to achieving it, and his opinion on what we should be doing to get there, specifically, focusing on AI generating algorithms. With the goal of creating open-ended algorithms that can learn forever, Jeff shares his three pillars to an AI-GA, meta-learning architectures, meta-learning algorithms, and auto-generating learning environments. Finally, we discuss the inherent safety issues with these learning algorithms and Jeff’s thoughts on how to combat them, and what the not-so-distant future holds for this area of research.
The complete show notes for this episode can be found at twimlai.com/go/602.
Artem Cherkasov and Olexandr Isayev on Democratizing Drug Discovery with Deep Learning - Ep. 172
It may seem intuitive that AI and deep learning can speed up workflows — including novel drug discovery, a typically years-long and several-billion-dollar endeavor.
But professors Artem Cherkasov and Olexandr Isayev were surprised to find that no recent academic papers provided a comprehensive, global research review of how deep learning and GPU-accelerated computing impact drug discovery.
In March, they published a paper in Nature to fill this gap, presenting an up-to-date review of the state of the art for GPU-accelerated drug discovery techniques.
Cherkasov, a professor in the department of urologic sciences at the University of British Columbia, and Isayev, an assistant professor of chemistry at Carnegie Mellon University, join NVIDIA AI Podcast host Noah Kravitz this week to discuss how GPUs can help democratize drug discovery.
In addition, the guests cover their inspiration and process for writing the paper, talk about NVIDIA technologies that are transforming the role of AI in drug discovery, and give tips for adopting new approaches to research.
Information foraging: the tricks great developers use to find solutions
You can check out some more of Henley's work on his blog here. Recent pieces include:
A theory of how developers seek information
All my career rejections
Navigate your code like it's 2021
Why is it so hard to see code from 5 minutes ago?
An inquisitive code editor: Overcome bugs before you know you have them
How much time does the average developer spend typing in their editor versus researching, exploring, and pondering? Henley believes half an hour of inputting actual code a day is realistic, despite what you've heard about the 10X developer in your area.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
#163: Python in Geoscience
See the full show notes for this episode on the website at talkpython.fm/163