The Fractured Entangled Representation Hypothesis (Kenneth Stanley, Akarsh Kumar)
Are the AI models you use today imposters?
Please watch the intro video we did before this: https://www.youtube.com/watch?v=o1q6Hhz0MAg
In this episode, hosts Dr. Tim Scarfe and Dr. Duggar are joined by AI researcher Prof. Kenneth Stanley and MIT PhD student Akash Kumar to discuss their fascinating paper, "Questioning Representational Optimism in Deep Learning."
Imagine you ask two people to draw a perfect skull. One is a brilliant artist who understands anatomy, the other is a machine that just traces the image. Both drawings look identical, but the artist understands what a skull is—they know where the mouth is, how the jaw works, and that it's symmetrical. The machine just has a tangled mess of lines that happens to form the right picture.
An AI with an elegant representation, has the building blocks to generate truly new ideas.
The Path Is the Goal: As Kenneth Stanley puts it, "it matters not just where you get, but how you got there". Two students can ace a math test, but the one who truly understands the concepts—instead of just memorizing formulas—is the one who will go on to make new discoveries.
The show is a mixture of 3 separate recordings we have done, the original Patreon warmup with Tim/Kenneth, the Tim/Keith "Steakhouse" recorded after the main interview, then the main interview with Kenneth/Akarsh/Keith/Tim. Feel free to skip around. We had to edit this in a rush as we are travelling next week but it's reasonably cleaned up.
TOC:
00:00:00 Intro: Garbage vs. Amazing Representations
00:05:42 How Good Representations Form
00:11:14 Challenging the "Bitter Lesson"
00:18:04 AI Creativity & Representation Types
00:22:13 Steakhouse: Critiques & Alternatives
00:28:30 Steakhouse: Key Concepts & Goldilocks Zone
00:39:42 Steakhouse: A Sober View on AI Risk
00:43:46 Steakhouse: The Paradox of Open-Ended Search
00:47:58 Main Interview: Paper Intro & Core Concepts
00:56:44 Main Interview: Deception and Evolvability
01:36:30 Main Interview: Reinterpreting Evolution
01:56:16 Main Interview: Impostor Intelligence
02:11:15 Main Interview: Recommendations for AI Research
REFS:
Questioning Representational Optimism in Deep Learning:
The Fractured Entangled Representation Hypothesis
Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley
https://arxiv.org/pdf/2505.11581
Kenneth O. Stanley, Joel Lehman
Why Greatness Cannot Be Planned: The Myth of the Objective
https://amzn.to/44xLaXK
Original show with Kenneth from 4 years ago:
https://www.youtube.com/watch?v=lhYGXYeMq_E
Kenneth Stanley is SVP Open Endedness at Lila Sciences
https://x.com/kenneth0stanley
Akarsh Kumar (MIT)
https://akarshkumar.com/
AND... Kenneth is HIRING (this is an OPPORTUNITY OF A LIFETIME!)
Research Engineer: https://job-boards.greenhouse.io/lila/jobs/7890007002
Research Scientist: https://job-boards.greenhouse.io/lila/jobs/8012245002
TRANSCRIPT:
https://app.rescript.info/public/share/W_T7E1OC2Wj49ccqlIOOztg2MJWaaVbovTeyxcFEQdU
The Fractured Entangled Representation Hypothesis (Intro)
What if today's incredible AI is just a brilliant "impostor"? This episode features host Dr. Tim Scarfe in conversation with guests Prof. Kenneth Stanley (ex-OpenAI), Dr. Keith Duggar (MIT), and Arkash Kumar (MIT).While AI today produces amazing results on the surface, its internal understanding is a complete mess, described as "total spaghetti" [00:00:49]. This is because it's trained with a brute-force method (SGD) that’s like building a sandcastle: it looks right from a distance, but has no real structure holding it together [00:01:45].To explain the difference, Keith Duggar shares a great analogy about his high school physics classes [00:03:18]. One class was about memorizing lots of formulas for specific situations (like the "impostor" AI). The other used calculus to derive the answers from a deeper understanding, which was much easier and more powerful. This is the core difference: one method memorizes, the other truly understands.The episode then introduces a different, more powerful way to build AI, based on Kenneth Stanley's old experiment, "Picbreeder" [00:04:45]. This method creates AI with a shockingly clean and intuitive internal model of the world. For example, it might develop a model of a skull where it understands the "mouth" as a separate component it can open and close, without ever being explicitly trained on that action [00:06:15]. This deep understanding emerges bottom-up, without massive datasets.The secret is to abandon a fixed goal and embrace "deception" [00:08:42]—the idea that the stepping stones to a great discovery often don't look anything like the final result. Instead of optimizing for a target, the AI is built through an open-ended process of exploring what's "interesting" [00:09:15]. This creates a more flexible and adaptable foundation, a bit like how evolvability wins out in nature [00:10:30].The show concludes by arguing that this choice matters immensely. The "impostor" path may be hitting a wall, requiring insane amounts of money and energy for progress and failing to deliver true creativity or continual learning [00:13:00]. The ultimate message is a call to not put all our eggs in one basket [00:14:25]. We should explore these open-ended, creative paths to discover a more genuine form of intelligence, which may be found where we least expect it.REFS:Questioning Representational Optimism in Deep Learning:The Fractured Entangled Representation HypothesisAkarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanleyhttps://arxiv.org/pdf/2505.11581Kenneth O. Stanley, Joel LehmanWhy Greatness Cannot Be Planned: The Myth of the Objectivehttps://amzn.to/44xLaXKOriginal show with Kenneth from 4 years ago:https://www.youtube.com/watch?v=lhYGXYeMq_EKenneth Stanley is SVP Open Endedness at Lila Scienceshttps://x.com/kenneth0stanleyAkarsh Kumar (MIT)https://akarshkumar.com/AND... Kenneth is HIRING (this is an OPPORTUNITY OF A LIFETIME!)Research Engineer: https://job-boards.greenhouse.io/lila/jobs/7890007002Research Scientist: https://job-boards.greenhouse.io/lila/jobs/8012245002Tim's Code visualisation of FER based on Akarsh repo: https://github.com/ecsplendid/ferTRANSCRIPT: https://app.rescript.info/public/share/YKAZzZ6lwZkjTLRpVJreOOxGhLI8y4m3fAyU8NSavx0
Kenneth Stanley created a new social network based on serendipity and divergence
See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network". The idea is that you follow interests, NOT people. It's a social network without the popularity contest. We discuss the phgilosophy and technology behind the venture in great detail. The main ideas of which came from Kenneth's famous book "Why greatness cannot be planned".
See what Sam Altman advised Kenneth when he left OpenAI! Professor Kenneth Stanley has just launched a brand new type of social network, which he calls a "Serendipity network".The idea is that you follow interests, NOT people. It's a social network without the popularity contest.
YT version: https://www.youtube.com/watch?v=pWIrXN-yy8g
Chapters should be baked into the MP3 file now
MLST public Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778
Please support our work on Patreon - get access to interviews months early, private Patreon, networking, exclusive content and regular calls with Tim and Keith.
https://patreon.com/mlst
Get Maven here:
https://www.heymaven.com/
Kenneth:
https://twitter.com/kenneth0stanley
https://www.kenstanley.net/home
Host - Tim Scarfe:
https://www.linkedin.com/in/ecsquizor/
https://www.mlst.ai/
Original MLST show with Kenneth:
https://www.youtube.com/watch?v=lhYGXYeMq_E
Tim explains the book more here:
https://www.youtube.com/watch?v=wNhaz81OOqw
#81 JULIAN TOGELIUS, Prof. KEN STANLEY - AGI, Games, Diversity & Creativity [UNPLUGGED]
Support us (and please rate on podcast app)
https://www.patreon.com/mlst
In this show tonight with Prof. Julian Togelius (NYU) and Prof. Ken Stanley we discuss open-endedness, AGI, game AI and reinforcement learning.
[Prof Julian Togelius]
https://engineering.nyu.edu/faculty/julian-togelius
https://twitter.com/togelius
[Prof Ken Stanley]
https://www.cs.ucf.edu/~kstanley/
https://twitter.com/kenneth0stanley
TOC:
[00:00:00] Introduction
[00:01:07] AI and computer games
[00:12:23] Intelligence
[00:21:27] Intelligence Explosion
[00:25:37] What should we be aspiring towards?
[00:29:14] Should AI contribute to culture?
[00:32:12] On creativity and open-endedness
[00:36:11] RL overfitting
[00:44:02] Diversity preservation
[00:51:18] Empiricism vs rationalism , in gradient descent the data pushes you around
[00:55:49] Creativity and interestingness (does complexity / information increase)
[01:03:20] What does a population give us?
[01:05:58] Emergence / generalisation snobbery
References;
[Hutter/Legg] Universal Intelligence: A Definition of Machine Intelligence
https://arxiv.org/abs/0712.3329
https://en.wikipedia.org/wiki/Artificial_general_intelligence
https://en.wikipedia.org/wiki/I._J._Good
https://en.wikipedia.org/wiki/G%C3%B6del_machine
[Chollet] Impossibility of intelligence explosion
https://medium.com/@francois.chollet/the-impossibility-of-intelligence-explosion-5be4a9eda6ec
[Alex Irpan] - RL is hard
https://www.alexirpan.com/2018/02/14/rl-hard.html
https://nethackchallenge.com/
Map elites
https://arxiv.org/abs/1504.04909
Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
https://arxiv.org/abs/1912.02400
[Stanley] - Why greatness cannot be planned
https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237
[Lehman/Stanley] Abandoning Objectives: Evolution through the Search for Novelty Alone
https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf
Kenneth Stanley - Greatness Without Goals - [Invest Like the Best, EP.283]
My guest today is Ken Stanley. Ken is a Professor in Computer Science and a pioneer in the field of neuroevolution. He is also the co-author of a book called, Why Greatness Cannot Be Planned, which details a provocative idea that setting big, audacious goals can reduce the odds of achieving something great. We discuss that revelation in detail and how to apply it in our day-to-day lives. Please enjoy this great discussion with Ken Stanley.
For the full show notes, transcript, and links to mentioned content, check out the episode page here.
-----
This episode is brought to you by Tegus. Tegus streamlines the investment research process so you can get up to speed and find answers to critical questions on companies faster and more efficiently. The Tegus platform surfaces the hard-to-get qualitative insights, gives instant access to critical public financial data through BamSEC, and helps you set up customized expert calls. It’s all done on a single, modern SaaS platform that offers 360-degree insight into any public or private company. As a listener, you can take Tegus for a free test drive by visiting tegus.co/patrick. And until 2023 every Tegus license comes with complimentary access to BamSec by Tegus.
-----
Today's episode is brought to you by Brex, the integrated financial platform trusted by the world's most innovative entrepreneurs and fastest-growing companies. With Brex, you can move money fast for instant impact with high-limit corporate cards, payments, venture debt, and spend management software all in one place. Ready to accelerate your business? Learn more at brex.com/best.
-----
Invest Like the Best is a property of Colossus, LLC. For more episodes of Invest Like the Best, visit joincolossus.com/episodes.
Past guests include Tobi Lutke, Kevin Systrom, Mike Krieger, John Collison, Kat Cole, Marc Andreessen, Matthew Ball, Bill Gurley, Anu Hariharan, Ben Thompson, and many more.
Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here.
Follow us on Twitter: @patrick_oshag | @JoinColossus
Show Notes
[00:02:36] - [First question] - The best way to change the world is to stop trying to change it
[00:06:26] - The kinds of goals his work addresses and the ones it doesn’t
[00:08:46] - Almost no prerequisite to any major invention was invented with that major invention in mind
[00:14:04] - Picbreeder
[00:17:21] - How looking for specific results often makes arriving at them a longer process
[00:24:00] - The importance of the individual in a web of invention and disruption
[00:28:30] - How generations progressed in Picbreeder when consensus mechanisms were inserted into the process
[00:31:24] - Examples of stepping stones that were invented that became something even greater
[00:36:02] - What his research means for how we should conduct ourselves writ large
[00:44:17] - Thoughts on necessity being the mother of all invention
[00:50:08] - The ways that society is arranged is psychologically toxic
[00:55:14] - The role that constraints play in creative output and outcomes in general; Brett Victor - Inventing on Principle
[01:01:10] - What the constraints are that he sets for himself in AI development
[01:04:44] - To know what’s new you need to know what’s not new
[01:06:47] - The kindest thing anyone has ever done for him
[01:08:28] - How he would allocate resources to create more innovation in the world
#72 Prof. KEN STANLEY 2.0 - On Art and Subjectivity [UNPLUGGED]
YT version: https://youtu.be/DxBZORM9F-8
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Prof. Ken Stanley argued in his book that our world has become saturated with objectives. The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture. He’s not saying that objectives are bad per se, especially if they’re modest, but he thinks that when goals are ambitious then the search space becomes deceptive.
Is the key to artificial intelligence really related to intelligence? Does taking a job with a higher salary really bring you closer to being a millionaire? The problem is that the stepping stones which lead to ambitious objectives tend to be pretty strange, they don't resemble the final end state at all. Vaccum tubes led to computers for example and Youtube started as a dating website.
What fascinated us about this conversation with Ken is that we got a much deeper understanding of his philosophy. He lead by saying that he thought it's worth questioning whether artificial intelligence is even a science or not. Ken thinks that the secret to future progress is for us to embrace more subjectivity.
[00:00:00] Tim Intro
[00:12:54] Intro
[00:17:08] Seeing ideas everywhere - AI and art are highly connected
[00:28:40] Creativity in Mathematics
[00:30:14] Where is the intelligence in art?
[00:38:49] Is AI disappointingly simple to mechanise?
[00:42:48] Slightly conscious
[00:46:27] Do we have subjective experience?
[00:50:23] Fear of the unknown
[00:51:48] Free Will
[00:54:22] Chalmers
[00:55:08] What's happening now in open-endedness
[00:58:31] Generalisation
[01:06:34] Representation primitives and what it means to understand
[01:12:37] Appeal to definitions, knowledge itself blocks discovery
Make sure you buy Kenneth's book!
Why Greatness Cannot Be Planned: The Myth of the Objective [Stanley, Lehman]
https://www.amazon.co.uk/Why-Greatness-Cannot-Planned-Objective/dp/3319155237
Abandoning Objectives: Evolution through the
Search for Novelty Alone [Lehman, Stanley]
https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/lehman_ecj11.pdf
Twitter
https://twitter.com/kenneth0stanley
#038 - Professor Kenneth Stanley - Why Greatness Cannot Be Planned
Professor Kenneth Stanley is currently a research science manager at OpenAI in San Fransisco. We've Been dreaming about getting Kenneth on the show since the very begininning of Machine Learning Street Talk. Some of you might recall that our first ever show was on the enhanced POET paper, of course Kenneth had his hands all over it. He's been cited over 16000 times, his most popular paper with over 3K citations was the NEAT algorithm. His interests are neuroevolution, open-endedness, NNs, artificial life, and AI. He invented the concept of novelty search with no clearly defined objective. His key idea is that there is a tyranny of objectives prevailing in every aspect of our lives, society and indeed our algorithms. Crucially, these objectives produce convergent behaviour and thinking and distract us from discovering stepping stones which will lead to greatness. He thinks that this monotonic objective obsession, this idea that we need to continue to improve benchmarks every year is dangerous. He wrote about this in detail in his recent book "greatness can not be planned" which will be the main topic of discussion in the show. We also cover his ideas on open endedness in machine learning.
00:00:00 Intro to Kenneth
00:01:16 Show structure disclaimer
00:04:16 Passionate discussion
00:06:26 WHy greatness cant be planned and the tyranny of objectives
00:14:40 Chinese Finger Trap
00:16:28 Perverse Incentives and feedback loops
00:18:17 Deception
00:23:29 Maze example
00:24:44 How can we define curiosity or interestingness
00:26:59 Open endedness
00:33:01 ICML 2019 and Yannic, POET, first MSLST
00:36:17 evolutionary algorithms++
00:43:18 POET, the first MLST
00:45:39 A lesson to GOFAI people
00:48:46 Machine Learning -- the great stagnation
00:54:34 Actual scientific successes are usually luck, and against the odds -- Biontech
00:56:21 Picbreeder and NEAT
01:10:47 How Tim applies these ideas to his life and why he runs MLST
01:14:58 Keith Skit about UCF
01:15:13 Main show kick off
01:18:02 Why does Kenneth value serindipitous exploration so much
01:24:10 Scientific support for Keneths ideas in normal life
01:27:12 We should drop objectives to achieve them. An oxymoron?
01:33:13 Isnt this just resource allocation between exploration and exploitation?
01:39:06 Are objectives merely a matter of degree?
01:42:38 How do we allocate funds for treasure hunting in society
01:47:34 A keen nose for what is interesting, and voting can be dangerous
01:53:00 Committees are the antithesis of innovation
01:56:21 Does Kenneth apply these ideas to his real life?
01:59:48 Divergence vs interestingness vs novelty vs complexity
02:08:13 Picbreeder
02:12:39 Isnt everything novel in some sense?
02:16:35 Imagine if there was no selection pressure?
02:18:31 Is innovation == environment exploitation?
02:20:37 Is it possible to take shortcuts if you already knew what the innovations were?
02:21:11 Go Explore -- does the algorithm encode the stepping stones?
02:24:41 What does it mean for things to be interestingly different?
02:26:11 behavioral characterization / diversity measure to your broad interests
02:30:54 Shaping objectives
02:32:49 Why do all ambitious objectives have deception? Picbreeder analogy
02:35:59 Exploration vs Exploitation, Science vs Engineering
02:43:18 Schools of thought in ML and could search lead to AGI
02:45:49 Official ending
Neuroevolution: Evolving Novel Neural Network Architectures with Kenneth Stanley - TWiML Talk #94
Today, I'm joined by Kenneth Stanley, Professor in the Department of Computer Science at the University of Central Florida and senior research scientist at Uber AI Labs. Kenneth studied under TWiML Talk #47 guest Risto Miikkulainen at UT Austin, and joined Uber AI Labs after Geometric Intelligence, the company he co-founded with Gary Marcus and others, was acquired in late 2016. Kenneth’s research focus is what he calls Neuroevolution, applies the idea of genetic algorithms to the challenge of evolving neural network architectures. In this conversation, we discuss the Neuroevolution of Augmenting Topologies (or NEAT) paper that Kenneth authored along with Risto, which won the 2017 International Society for Artificial Life’s Award for Outstanding Paper of the Decade 2002 - 2012. We also cover some of the extensions to that approach he’s created since, including, HyperNEAT, which can efficiently evolve very large networks with connectivity patterns that look more like those of the human and that are generally much larger than what prior approaches to neural learning could produce, and novelty search, an approach which unlike most evolutionary algorithms has no defined objective, but rather simply searches for novel behaviors. We also cover concepts like “Complexification” and “Deception”, biology vs computation including differences and similarities, and some of his other work including his book, and NERO, a video game complete with Real-time Neuroevolution. This is a meaty “Nerd Alert” interview that I think you’ll really enjoy.