The Universal Hierarchy of Life - Prof. Chris Kempes [SFI]
"What is life?" - asks Chris Kempes, a professor at the Santa Fe Institute.
Chris explains that scientists are moving beyond a purely Earth-based, biological view and are searching for a universal theory of life that could apply to anything, anywhere in the universe. He proposes that things we don't normally consider "alive"—like human culture, language, or even artificial intelligence; could be seen as life forms existing on different "substrates".
To understand this, Chris presents a fascinating three-level framework:
- Materials: The physical stuff life is made of. He argues this could be incredibly diverse across the universe, and we shouldn't expect alien life to share our biochemistry.
- Constraints: The universal laws of physics (like gravity or diffusion) that all life must obey, regardless of what it's made of. This is where different life forms start to look more similar.
- Principles: At the highest level are abstract principles like evolution and learning. Chris suggests these computational or "optimization" rules are what truly define a living system.
A key idea is "convergence" – using the example of the eye. It's such a complex organ that you'd think it evolved only once. However, eyes evolved many separate times across different species. This is because the physics of light provides a clear "target", and evolution found similar solutions to the problem of seeing, even with different starting materials.
**SPONSOR MESSAGES**
—
Prolific - Quality data. From real people. For faster breakthroughs.
https://www.prolific.com/?utm_source=mlst
—
Check out NotebookLM from Google here - https://notebooklm.google.com/ - it’s really good for doing research directly from authoritative source material, minimising hallucinations.
—
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
—
Prof. Chris Kempes:
https://www.santafe.edu/people/profile/chris-kempes
TRANSCRIPT:
https://app.rescript.info/public/share/Y2cI1i0nX_-iuZitvlguHvaVLQTwPX1Y_E1EHxV0i9I
TOC:
00:00:00 - Introduction to Chris Kempes and the Santa Fe Institute
00:02:28 - The Three Cultures of Science
00:05:08 - What Makes a Good Scientific Theory?
00:06:50 - The Universal Theory of Life
00:09:40 - The Role of Material in Life
00:12:50 - A Hierarchy for Understanding Life
00:13:55 - How Life Diversifies and Converges
00:17:53 - Adaptive Processes and Defining Life
00:19:28 - Functionalism, Memes, and Phylogenies
00:22:58 - Convergence at Multiple Levels
00:25:45 - The Possibility of Simulating Life
00:28:16 - Intelligence, Parasitism, and Spectrums of Life
00:32:39 - Phase Changes in Evolution
00:36:16 - The Separation of Matter and Logic
00:37:21 - Assembly Theory and Quantifying Complexity
REFS:
Developing a predictive science of the biosphere requires the integration of scientific cultures [Kempes et al]
https://www.pnas.org/doi/10.1073/pnas.2209196121
Seeing with an extra sense (“Dangerous prediction”) [Rob Phillips]
https://www.sciencedirect.com/science/article/pii/S0960982224009035
The Multiple Paths to Multiple Life [Christopher P. Kempes & David C. Krakauer]
https://link.springer.com/article/10.1007/s00239-021-10016-2
The Information Theory of Individuality [David Krakauer et al]
https://arxiv.org/abs/1412.2447
Minds, Brains and Programs [Searle]
https://home.csulb.edu/~cwallis/382/readings/482/searle.minds.brains.programs.bbs.1980.pdf
The error threshold
https://www.sciencedirect.com/science/article/abs/pii/S0168170204003843
Assembly theory and its relationship with computational complexity [Kempes et al]
https://arxiv.org/abs/2406.12176
The Day AI Solves My Puzzles Is The Day I Worry (Prof. Cristopher Moore)
We are joined by Cristopher Moore, a professor at the Santa Fe Institute with a diverse background in physics, computer science, and machine learning.
The conversation begins with Cristopher, who calls himself a "frog" explaining that he prefers to dive deep into specific, concrete problems rather than taking a high-level "bird's-eye view".
They explore why current AI models, like transformers, are so surprisingly effective. Cristopher argues it's because the real world isn't random; it's full of rich structures, patterns, and hierarchies that these models can learn to exploit, even if we don't fully understand how.
**SPONSORS**
Take the Prolific human data survey - https://www.prolific.com/humandatasurvey?utm_source=mlst and be the first to see the results and benchmark their practices against the wider community!
---
cyber•Fund https://cyber.fund/?utm_source=mlst is a founder-led investment firm accelerating the cybernetic economy.
Oct SF conference - https://dagihouse.com/?utm_source=mlst - Joscha Bach keynoting(!) + OAI, Anthropic, NVDA,++
Hiring a SF VC Principal: https://talent.cyber.fund/companies/cyber-fund-2/jobs/57674170-ai-investment-principal#content?utm_source=mlst
Submit investment deck: https://cyber.fund/contact?utm_source=mlst
***
Cristopher Moore:
https://sites.santafe.edu/~moore/
TOC:
00:00:00 - Introduction
00:02:05 - Meet Christopher Moore: A Frog in the World of Science
00:05:14 - The Limits of Transformers and Real-World Data
00:11:19 - Intelligence as Creative Problem-Solving
00:23:30 - Grounding, Meaning, and Shared Reality
00:31:09 - The Nature of Creativity and Aesthetics
00:44:31 - Computational Irreducibility and Universality
00:53:06 - Turing Completeness, Recursion, and Intelligence
01:11:26 - The Universe Through a Computational Lens
01:26:45 - Algorithmic Justice and the Need for Transparency
TRANSCRIPT: https://app.rescript.info/public/share/VRe2uQSvKZOm0oIBoDsrNwt46OMCqRnShVnUF3qyoFk
Filmed at DISI (Diverse Intelligences Summer Institute)
https://disi.org/
REFS:
The Nature of computation [Chris Moore]
https://nature-of-computation.org/
Birds and Frogs [Freeman Dyson]
https://www.ams.org/notices/200902/rtx090200212p.pdf
Replica Theory [Parisi et al]
https://arxiv.org/pdf/1409.2722
Janossy pooling [Fabian Fuchs]
https://fabianfuchsml.github.io/equilibriumaggregation/
Cracking the cryptic [YT channel]
https://www.youtube.com/c/CrackingTheCryptic
Sudoko Bench [Sakana]
https://sakana.ai/sudoku-bench/
Fractured entangled representations “phylogenetic locking in comment” [Kumar/Stanley]
https://arxiv.org/pdf/2505.11581 (see our shows on this)
The War Against Cliché: [Martin Amis]
https://www.amazon.com/War-Against-Cliche-Reviews-1971-2000/dp/0375727167
Rule 110 (CA)
https://mathworld.wolfram.com/Rule150.html
Universality in Elementary Cellular Automata [Matt Cooke]
https://wpmedia.wolfram.com/sites/13/2018/02/15-1-1.pdf
Small Semi-Weakly Universal Turing Machines [Damien Woods]
https://tilde.ini.uzh.ch/users/tneary/public_html/WoodsNeary-FI09.pdf
COMPUTING MACHINERY AND INTELLIGENCE [Turing, 1950]
https://courses.cs.umbc.edu/471/papers/turing.pdf
Comment on Space Time as a causal set [Moore, 88]
https://sites.santafe.edu/~moore/comment.pdf
Recursion Theory on the Reals and Continuous-time Computation [Moore, 96]
Large Language Models and Emergence: A Complex Systems Perspective (Prof. David C. Krakauer)
Prof. David Krakauer, President of the Santa Fe Institute argues that we are fundamentally confusing knowledge with intelligence, especially when it comes to AI.
He defines true intelligence as the ability to do more with less—to solve novel problems with limited information. This is contrasted with current AI models, which he describes as doing less with more; they require astounding amounts of data to perform tasks that don't necessarily demonstrate true understanding or adaptation. He humorously calls this "really shit programming".
David challenges the popular notion of "emergence" in Large Language Models (LLMs). He explains that the tech community's definition—seeing a sudden jump in a model's ability to perform a task like three-digit math—is superficial. True emergence, from a complex systems perspective, involves a fundamental change in the system's internal organization, allowing for a new, simpler, and more powerful level of description. He gives the example of moving from tracking individual water molecules to using the elegant laws of fluid dynamics. For LLMs to be truly emergent, we'd need to see them develop new, efficient internal representations, not just get better at memorizing patterns as they scale.
Drawing on his background in evolutionary theory, David explains that systems like brains, and later, culture, evolved to process information that changes too quickly for genetic evolution to keep up. He calls culture "evolution at light speed" because it allows us to store our accumulated knowledge externally (in books, tools, etc.) and build upon it without corrupting the original.
This leads to his concept of "exbodiment," where we outsource our cognitive load to the world through things like maps, abacuses, or even language itself.
We create these external tools, internalize the skills they teach us, improve them, and create a feedback loop that enhances our collective intelligence.
However, he ends with a warning. While technology has historically complemented our deficient abilities, modern AI presents a new danger. Because we have an evolutionary drive to conserve energy, we will inevitably outsource our thinking to AI if we can. He fears this is already leading to a "diminution and dilution" of human thought and creativity. Just as our muscles atrophy without use, he argues our brains will too, and we risk becoming mentally dependent on these systems.
TOC:
[00:00:00] Intelligence: Doing more with less
[00:02:10] Why brains evolved: The limits of evolution
[00:05:18] Culture as evolution at light speed
[00:08:11] True meaning of emergence: "More is Different"
[00:10:41] Why LLM capabilities are not true emergence
[00:15:10] What real emergence would look like in AI
[00:19:24] Symmetry breaking: Physics vs. Life
[00:23:30] Two types of emergence: Knowledge In vs. Out
[00:26:46] Causality, agency, and coarse-graining
[00:32:24] "Exbodiment": Outsourcing thought to objects
[00:35:05] Collective intelligence & the boundary of the mind
[00:39:45] Mortal vs. Immortal forms of computation
[00:42:13] The risk of AI: Atrophy of human thought
David Krakauer
President and William H. Miller Professor of Complex Systems
https://www.santafe.edu/people/profile/david-krakauer
REFS:
Large Language Models and Emergence: A Complex Systems Perspective
David C. Krakauer, John W. Krakauer, Melanie Mitchell
https://arxiv.org/abs/2506.11135
Filmed at the Diverse Intelligences Summer Institute:
https://disi.org/
Decompiling Dreams: A New Approach to ARC? - Alessandro Palmarini
Alessandro Palmarini is a post-baccalaureate researcher at the Santa Fe Institute working under the supervision of Melanie Mitchell. He completed his undergraduate degree in Artificial Intelligence and Computer Science at the University of Edinburgh. Palmarini's current research focuses on developing AI systems that can efficiently acquire new skills from limited data, inspired by François Chollet's work on measuring intelligence. His work builds upon the DreamCoder program synthesis system, introducing a novel approach called "dream decompiling" to improve library learning in inductive program synthesis. Palmarini is particularly interested in addressing the Abstraction and Reasoning Corpus (ARC) challenge, aiming to create AI systems that can perform abstract reasoning tasks more efficiently than current approaches. His research explores the balance between computational efficiency and data efficiency in AI learning processes.
DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?
MLST is sponsored by Tufa Labs:
Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.
Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.
Interested? Apply for an ML research position: benjamin@tufa.ai
TOC:
1. Intelligence Measurement in AI Systems
[00:00:00] 1.1 Defining Intelligence in AI Systems
[00:02:00] 1.2 Research at Santa Fe Institute
[00:04:35] 1.3 Impact of Gaming on AI Development
[00:05:10] 1.4 Comparing AI and Human Learning Efficiency
2. Efficient Skill Acquisition in AI
[00:06:40] 2.1 Intelligence as Skill Acquisition Efficiency
[00:08:25] 2.2 Limitations of Current AI Systems in Generalization
[00:09:45] 2.3 Human vs. AI Cognitive Processes
[00:10:40] 2.4 Measuring AI Intelligence: Chollet's ARC Challenge
3. Program Synthesis and ARC Challenge
[00:12:55] 3.1 Philosophical Foundations of Program Synthesis
[00:17:14] 3.2 Introduction to Program Induction and ARC Tasks
[00:18:49] 3.3 DreamCoder: Principles and Techniques
[00:27:55] 3.4 Trade-offs in Program Synthesis Search Strategies
[00:31:52] 3.5 Neural Networks and Bayesian Program Learning
4. Advanced Program Synthesis Techniques
[00:32:30] 4.1 DreamCoder and Dream Decompiling Approach
[00:39:00] 4.2 Beta Distribution and Caching in Program Synthesis
[00:45:10] 4.3 Performance and Limitations of Dream Decompiling
[00:47:45] 4.4 Alessandro's Approach to ARC Challenge
[00:51:12] 4.5 Conclusion and Future Discussions
Refs:
Full reflist on YT VD, Show Notes and MP3 metadata
Show Notes: https://www.dropbox.com/scl/fi/x50201tgqucj5ba2q4typ/Ale.pdf?rlkey=0ubvk7p5gtyx1gpownpdadim8&st=5pniu3nq&dl=0
Prof. Melanie Mitchell 2.0 - AI Benchmarks are Broken!
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/ESrGqhf5CB
Prof. Melanie Mitchell argues that the concept of "understanding" in AI is ill-defined and multidimensional - we can't simply say an AI system does or doesn't understand. She advocates for rigorously testing AI systems' capabilities using proper experimental methods from cognitive science. Popular benchmarks for intelligence often rely on the assumption that if a human can perform a task, an AI that performs the task must have human-like general intelligence. But benchmarks should evolve as capabilities improve.
Large language models show surprising skill on many human tasks but lack common sense and fail at simple things young children can do. Their knowledge comes from statistical relationships in text, not grounded concepts about the world. We don't know if their internal representations actually align with human-like concepts. More granular testing focused on generalization is needed.
There are open questions around whether large models' abilities constitute a fundamentally different non-human form of intelligence based on vast statistical correlations across text. Mitchell argues intelligence is situated, domain-specific and grounded in physical experience and evolution. The brain computes but in a specialized way honed by evolution for controlling the body. Extracting "pure" intelligence may not work.
Other key points:
- Need more focus on proper experimental method in AI research. Developmental psychology offers examples for rigorous testing of cognition.
- Reporting instance-level failures rather than just aggregate accuracy can provide insights.
- Scaling laws and complex systems science are an interesting area of complexity theory, with applications to understanding cities.
- Concepts like "understanding" and "intelligence" in AI force refinement of fuzzy definitions.
- Human intelligence may be more collective and social than we realize. AI forces us to rethink concepts we apply anthropomorphically.
The overall emphasis is on rigorously building the science of machine cognition through proper experimentation and benchmarking as we assess emerging capabilities.
TOC:
[00:00:00] Introduction and Munk AI Risk Debate Highlights
[05:00:00] Douglas Hofstadter on AI Risk
[00:06:56] The Complexity of Defining Intelligence
[00:11:20] Examining Understanding in AI Models
[00:16:48] Melanie's Insights on AI Understanding Debate
[00:22:23] Unveiling the Concept Arc
[00:27:57] AI Goals: A Human vs Machine Perspective
[00:31:10] Addressing the Extrapolation Challenge in AI
[00:36:05] Brain Computation: The Human-AI Parallel
[00:38:20] The Arc Challenge: Implications and Insights
[00:43:20] The Need for Detailed AI Performance Reporting
[00:44:31] Exploring Scaling in Complexity Theory
Eratta:
Note Tim said around 39 mins that a recent Stanford/DM paper modelling ARC “on GPT-4 got around 60%”. This is not correct and he misremembered. It was actually davinci3, and around 10%, which is still extremely good for a blank slate approach with an LLM and no ARC specific knowledge. Folks on our forum couldn’t reproduce the result. See paper linked below.
Books (MUST READ):
Artificial Intelligence: A Guide for Thinking Humans (Melanie Mitchell)
https://www.amazon.co.uk/Artificial-Intelligence-Guide-Thinking-Humans/dp/B07YBHNM1C/?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=44ccac78973f47e59d745e94967c0f30&camp=1634&creative=6738
Complexity: A Guided Tour (Melanie Mitchell)
https://www.amazon.co.uk/Audible-Complexity-A-Guided-Tour?&_encoding=UTF8&tag=mlst00-21&linkCode=ur2&linkId=3f8bd505d86865c50c02dd7f10b27c05&camp=1634&creative=6738
Show notes (transcript, full references etc)
https://atlantic-papyrus-d68.notion.site/Melanie-Mitchell-2-0-15e212560e8e445d8b0131712bad3000?pvs=25
YT version: https://youtu.be/29gkDpR2orc
#57 - Prof. Melanie Mitchell - Why AI is harder than we think
Since its beginning in the 1950s, the field of artificial intelligence has vacillated between periods of optimistic predictions and massive investment and periods of disappointment, loss of confidence, and reduced funding. Even with today’s seemingly fast pace of AI breakthroughs, the development of long-promised technologies such as self-driving cars, housekeeping robots, and conversational companions has turned out to be much harder than many people expected. Professor Melanie Mitchell thinks one reason for these repeating cycles is our limited understanding of the nature and complexity of intelligence itself.
YT vid- https://www.youtube.com/watch?v=A8m1Oqz2HKc
Main show kick off [00:26:51]
Panel: Dr. Tim Scarfe, Dr. Keith Duggar, Letitia Parcalabescu (https://www.youtube.com/c/AICoffeeBreak/)
Complexity and Intelligence with Melanie Mitchell - #464
Today we’re joined by Melanie Mitchell, Davis Professor at the Santa Fe Institute and author of Artificial Intelligence: A Guide for Thinking Humans.
While Melanie has had a long career with a myriad of research interests, we focus on a few, complex systems and the understanding of intelligence, complexity, and her recent work on getting AI systems to make analogies. We explore examples of social learning, and how it applies to AI contextually, and defining intelligence.
We discuss potential frameworks that would help machines understand analogies, established benchmarks for analogy, and if there is a social learning solution to help machines figure out analogy. Finally we talk through the overall state of AI systems, the progress we’ve made amid the limited concept of social learning, if we’re able to achieve intelligence with current approaches to AI, and much more!
The complete show notes for this episode can be found at twimlai.com/go/464.
Melanie Mitchell: Concepts, Analogies, Common Sense & Future of AI
Melanie Mitchell is a professor of computer science at Portland State University and an external professor at Santa Fe Institute. She has worked on and written about artificial intelligence from fascinating perspectives including adaptive complex systems, genetic algorithms, and the Copycat cognitive architecture which places the process of analogy making at the core of human cognition. From her doctoral work with her advisors Douglas Hofstadter and John Holland to today, she has contributed a lot of important ideas to the field of AI, including her recent book, simply called Artificial Intelligence: A Guide for Thinking Humans.
This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon.
This episode is presented by Cash App. Download it (App Store, Google Play), use code “LexPodcast”.
Episode Links:
AI: A Guide for Thinking Humans (book)
Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time.
00:00 – Introduction
02:33 – The term “artificial intelligence”
06:30 – Line between weak and strong AI
12:46 – Why have people dreamed of creating AI?
15:24 – Complex systems and intelligence
18:38 – Why are we bad at predicting the future with regard to AI?
22:05 – Are fundamental breakthroughs in AI needed?
25:13 – Different AI communities
31:28 – Copycat cognitive architecture
36:51 – Concepts and analogies
55:33 – Deep learning and the formation of concepts
1:09:07 – Autonomous vehicles
1:20:21 – Embodied AI and emotion
1:25:01 – Fear of superintelligent AI
1:36:14 – Good test for intelligence
1:38:09 – What is complexity?
1:43:09 – Santa Fe Institute
1:47:34 – Douglas Hofstadter
1:49:42 – Proudest moment
257: AI: How Far We Haven’t Actually Come
In this episode of the SuperDataScience Podcast, I chat with Melanie Mitchell, one of the leading researchers in the field of AI. You will learn about complexity, what it is and how it works, and how it can be seen in different areas of life. You will hear about common sense, meta-cognition, explainable AI, and you will also hear Melanie's ideas and thoughts on the future of AI, which break down into two areas which you'll find out in this podcast.
If you enjoyed this episode, check out show notes, resources, and more at www.superdatascience.com/257
a16z Podcast: Network Effects, Origin Stories, and the Evolution of Tech
“The rules of the game are different in tech,” argues — and has long argued, despite his views not being accepted at first — W. Brian Arthur, technologist-turned-economist who first truly described the phenomenon of “positive feedbacks” in the economy or “increasing returns” (vs. diminishing returns) in the new world of business… a.k.a. network effects. A longtime observer of Silicon Valley and the tech industry, he’s seen how a few early entrepreneurs first got it, fewer investors embrace it, entire companies be built around it, and still yet others miss it… even today.
If an inferior product/technology/way of doing things can sometimes “lock in” the market, does that make network effects more about luck, or strategy? It’s not really locked in though, since over and over again the next big thing comes along. So what does that mean for companies and industries that want to make the new technology shift? And where does competitive advantage even come from when everyone has access to the same building blocks (open source, APIs, etc.) of innovation? Because Arthur — former Stanford professor, visiting researcher at PARC, and external professor at Santa Fe Institute who is also known as one of the fathers of complexity theory in economics — has written about the nature of technology and how it evolves, observing that new technology doesn’t come out of nowhere, but instead, is the result of “combinatorial” innovation. Does this then mean there’s no such thing as a dramatic breakthrough?!
In this hour-long episode of the a16z Podcast, we (Sonal Chokshi with Marc Andreessen) explore many of these questions with Arthur. His answers take us from “the halls of production” to the “casino of technology”; from the “prehistory” to the history of tech; from the invisible underground autonomy economy to the “internet of conversations”; from externally available information to externalized intelligence; and finally, from Silicon Valley to Singapore to China to India and back to Silicon Valley again. Who’s going to win; what are the chances of winning? We don’t know, because it’s a very different game… Do you still want to play?
Stay Updated:
Find a16z on YouTube: YouTube
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.