VAEs Are Energy-Based Models? [Dr. Jeff Beck]
What makes something truly *intelligent?* Is a rock an agent? Could a perfect simulation of your brain actually *be* you? In this fascinating conversation, Dr. Jeff Beck takes us on a journey through the philosophical and technical foundations of agency, intelligence, and the future of AI.
Jeff doesn't hold back on the big questions. He argues that from a purely mathematical perspective, there's no structural difference between an agent and a rock – both execute policies that map inputs to outputs. The real distinction lies in *sophistication* – how complex are the internal computations? Does the system engage in planning and counterfactual reasoning, or is it just a lookup table that happens to give the right answers?
*Key topics explored in this conversation:*
*The Black Box Problem of Agency* – How can we tell if something is truly planning versus just executing a pre-computed response? Jeff explains why this question is nearly impossible to answer from the outside, and why the best we can do is ask which model gives us the simplest explanation.
*Energy-Based Models Explained* – A masterclass on how EBMs differ from standard neural networks. The key insight: traditional networks only optimize weights, while energy-based models optimize *both* weights and internal states – a subtle but profound distinction that connects to Bayesian inference.
*Why Your Brain Might Have Evolved from Your Nose* – One of the most surprising moments in the conversation. Jeff proposes that the complex, non-smooth nature of olfactory space may have driven the evolution of our associative cortex and planning abilities.
*The JEPA Revolution* – A deep dive into Yann LeCun's Joint Embedding Prediction Architecture and why learning in latent space (rather than predicting every pixel) might be the key to more robust AI representations.
*AI Safety Without Skynet Fears* – Jeff takes a refreshingly grounded stance on AI risk. He's less worried about rogue superintelligences and more concerned about humans becoming "reward function selectors" – couch potatoes who just approve or reject AI outputs. His proposed solution? Use inverse reinforcement learning to derive AI goals from observed human behavior, then make *small* perturbations rather than naive commands like "end world hunger."
Whether you're interested in the philosophy of mind, the technical details of modern machine learning, or just want to understand what makes intelligence *tick,* this conversation delivers insights you won't find anywhere else.
---
TIMESTAMPS:
00:00:00 Geometric Deep Learning & Physical Symmetries
00:00:56 Defining Agency: From Rocks to Planning
00:05:25 The Black Box Problem & Counterfactuals
00:08:45 Simulated Agency vs. Physical Reality
00:12:55 Energy-Based Models & Test-Time Training
00:17:30 Bayesian Inference & Free Energy
00:20:07 JEPA, Latent Space, & Non-Contrastive Learning
00:27:07 Evolution of Intelligence & Modular Brains
00:34:00 Scientific Discovery & Automated Experimentation
00:38:04 AI Safety, Enfeeblement & The Future of Work
---
REFERENCES:
Concept:
[00:00:58] Free Energy Principle (FEP)
https://en.wikipedia.org/wiki/Free_energy_principle
[00:06:00] Monte Carlo Tree Search
https://en.wikipedia.org/wiki/Monte_Carlo_tree_search
Book:
[00:09:00] The Intentional Stance
https://mitpress.mit.edu/9780262540537/the-intentional-stance/
Paper:
[00:13:00] A Tutorial on Energy-Based Learning (LeCun 2006)
http://yann.lecun.com/exdb/publis/pdf/lecun-06.pdf
[00:15:00] Auto-Encoding Variational Bayes (VAE)
https://arxiv.org/abs/1312.6114
[00:20:15] JEPA (Joint Embedding Prediction Architecture)
https://openreview.net/forum?id=BZ5a1r-kVsf
[00:22:30] The Wake-Sleep Algorithm
https://www.cs.toronto.edu/~hinton/absps/ws.pdf
<trunc, see rescript>
---
RESCRIPT:
https://app.rescript.info/public/share/DJlSbJ_Qx080q315tWaqMWn3PixCQsOcM4Kf1IW9_Eo
PDF:
https://app.rescript.info/api/public/sessions/0efec296b9b6e905/pdf
Bayesian Brain, Scientific Method, and Models [Dr. Jeff Beck]
Dr. Jeff Beck, mathematician turned computational neuroscientist, joins us for a fascinating deep dive into why the future of AI might look less like ChatGPT and more like your own brain.
**SPONSOR MESSAGES START**
—
Prolific - Quality data. From real people. For faster breakthroughs.
https://www.prolific.com/?utm_source=mlst
—
**END**
*What if the key to building truly intelligent machines isn't bigger models, but smarter ones?*
In this conversation, Jeff makes a compelling case that we've been building AI backwards. While the tech industry races to scale up transformers and language models, Jeff argues we're missing something fundamental: the brain doesn't work like a giant prediction engine. It works like a scientist, constantly testing hypotheses about a world made of *objects* that interact through *forces* — not pixels and tokens.
*The Bayesian Brain* — Jeff explains how your brain is essentially running the scientific method on autopilot. When you combine what you see with what you hear, you're doing optimal Bayesian inference without even knowing it. This isn't just philosophy — it's backed by decades of behavioral experiments showing humans are surprisingly efficient at handling uncertainty.
*AutoGrad Changed Everything* — Forget transformers for a moment. Jeff argues the real hero of the AI boom was automatic differentiation, which turned AI from a math problem into an engineering problem. But in the process, we lost sight of what actually makes intelligence work.
*The Cat in the Warehouse Problem* — Here's where it gets practical. Imagine a warehouse robot that's never seen a cat. Current AI would either crash or make something up. Jeff's approach? Build models that *know what they don't know*, can phone a friend to download new object models on the fly, and keep learning continuously. It's like giving robots the ability to say "wait, what IS that?" instead of confidently being wrong.
*Why Language is a Terrible Model for Thought* — In a provocative twist, Jeff argues that grounding AI in language (like we do with LLMs) is fundamentally misguided. Self-report is the least reliable data in psychology — people routinely explain their own behavior incorrectly. We should be grounding AI in physics, not words.
*The Future is Lots of Little Models* — Instead of one massive neural network, Jeff envisions AI systems built like video game engines: thousands of small, modular object models that can be combined, swapped, and updated independently. It's more efficient, more flexible, and much closer to how we actually think.
Rescript: https://app.rescript.info/public/share/D-b494t8DIV-KRGYONJghvg-aelMmxSDjKthjGdYqsE
---
TIMESTAMPS:
00:00:00 Introduction & The Bayesian Brain
00:01:25 Bayesian Inference & Information Processing
00:05:17 The Brain Metaphor: From Levers to Computers
00:10:13 Micro vs. Macro Causation & Instrumentalism
00:16:59 The Active Inference Community & AutoGrad
00:22:54 Object-Centered Models & The Grounding Problem
00:35:50 Scaling Bayesian Inference & Architecture Design
00:48:05 The Cat in the Warehouse: Solving Generalization
00:58:17 Alignment via Belief Exchange
01:05:24 Deception, Emergence & Cellular Automata
---
REFERENCES:
Paper:
[00:00:24] Zoubin Ghahramani (Google DeepMind)
https://pmc.ncbi.nlm.nih.gov/articles/PMC3538441/pdf/rsta201
[00:19:20] Mamba: Linear-Time Sequence Modeling
https://arxiv.org/abs/2312.00752
[00:27:36] xLSTM: Extended Long Short-Term Memory
https://arxiv.org/abs/2405.04517
[00:41:12] 3D Gaussian Splatting
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
[01:07:09] Lenia: Biology of Artificial Life
https://arxiv.org/abs/1812.05433
[01:08:20] Growing Neural Cellular Automata
https://distill.pub/2020/growing-ca/
[01:14:05] DreamCoder
https://arxiv.org/abs/2006.08381
[01:14:58] The Genomic Bottleneck
https://www.nature.com/articles/s41467-019-11786-6
Person:
[00:16:42] Karl Friston (UCL)
https://www.youtube.com/watch?v=PNYWi996Beg
Mitu Gulati on Whether Trump Could Restructure US Debt
US Treasuries are the most important market in the world. With some $29 trillion outstanding, they create the benchmark that informs basically every other type of borrowing. Any changes to how the bond market works would be a massive deal. But lately, there's been a lot of chatter about how the Trump administration could radically restructure and refinance the US debt under the so-called "Mar-a-Lago Accord." In this episode, we speak with University of Virginia law professor Mitu Gulati about how far the administration could go to legally reform this huge and important market. We also talk about how to buy Greenland and whether Trump could make a few billion by collecting on some old loans from allies.
Read more:
One Way for Trump to Find a Few Extra Trillion Dollars
The Stories We Tell Ourselves About Bonds
Only Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at bloomberg.com/subscriptions/oddlots
See omnystudio.com/listener for privacy information.
The Black Hole of Private Credit That's Swallowing the Economy
There's been a lot of talk about private credit in recent years. The market has exploded in size, and there are worries that it could be a bubble that eventually bursts and sparks disaster. But there are other negative effects from private credit that might already be happening. In a new paper called "The Credit Markets Go Dark," co-authors Harvard Law School professor Jared Ellias and Duke University School of Law professor Elisabeth de Fontenay argue that the $1.5 trillion market for private credit is already having a big impact on the economy — and not in a good way. They say that the rise of private credit marks a seismic change for corporate governance and dynamism.
Read More:
Odd Lots Newsletter: The Black Hole of Private Credit
Private Credit Pushes Deeper Into Risk That Wall Street Is Fleeing
Only Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at bloomberg.com/subscriptions/oddlots
See omnystudio.com/listener for privacy information.
Conversation with Dan Ariely — Why People Believe Irrational Things
Dan Ariely, a professor of psychology and behavioral economics at Duke University, joins Scott to discuss what makes rational people believe seemingly irrational things. He also shares how we avoid certain facts when we don’t like the solution, as well as how a lack of trust in one area can spiral into others.
Follow Professor Ariely on X, @danariely.
Scott opens with his thoughts on the podcast space and then discusses why tech companies are still laying people off in droves.
Algebra of Happiness: the role of parents.
Learn more about your ad choices. Visit podcastchoices.com/adchoices

Dan ArielyProfessor of Psychology and Behavioral Economics
Cutting-edge Neurotechnology with Nita Farahany in Conversation with Luisa Rodriguez on The 80,000 Hours Podcast
We're sharing a few of Nathan's favorite AI scouting episodes from other shows. Today: Nita Farahani shares her insights with 80,000 Hours' Luisa Rodriguez on the current state of neurotechnology and its potential impacts on various fields, including work, healthcare, privacy, and even human cognition. The discussion includes Farahani's assessment of devices like SmartCap and Neuralink, the concept of cognitive liberty, the risks of the technology, and the need for proper regulation and ethical considerations.
If you need an ecommerce platform, check out our sponsor Shopify: https://shopify.com/cognitive for a $1/month trial period.
Nita Farahany is a professor of law and philosophy at Duke Law School and discusses the applications of cutting-edge neurotechnology.
You can subscribe to The 80,000 Hours Podcast here: https://80000hours.org/podcast/
---
SPONSORS:
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off www.omneky.com
NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.
X/SOCIAL:
@labenz (Nathan)
@8000hours (The 8000 Hours podcast)
@CogRev_Podcast (Cognitive Revolution)
TIMESTAMPS:
(00:00) Nathan intro's this episode with Nita Farahany
(03:26) The Future of Neurotechnology: From Neuralink to Meta
(05:40) The Potential of Brain-Computer Interfaces in Military Applications
(11:29) The Role of Neurotechnology in National Security
(15:20) SPONSORS: SHOPIFY | NETSUITE
(18:14) The Potential of Brain-Computer Interfaces in Communication and Data Transfer
(21:19) The Future of Super Soldiers and Brain-Controlled Drones
(32:20) SPONSORS: OMNEKY
(41:05) The Controversial Realm of Neurotechnology Weapons
(45:16) Ideal International Agreements for Weapon Regulation
(47:01) The Threat of Hacking in Neurotechnology
(50:36) Brain Signatures for Identification
(01:11:28) The Future of Neurotechnology in the Workplace
(01:24:59) The Intrusion of Brain Decoding into Personal Relationships
(01:30:00) Regulating Brain Decoding: A Call for a Change in Worldview
(01:38:03) The Impact of Neurotechnology on Identity and Self-Perception
(01:41:47) The Future of Cognitive Enhancement and Transhumanism
(01:46:21) The Role of Neurotechnology in Addressing Neurological Diseases
(01:51:04) The Dark Side of Neurotechnology: Hacking and Manipulation
(01:54:50) The Importance of Cognitive Liberty in Neurotechnology
(01:55:15) The Future of Neurotechnology: Best and Worst Case Scenarios
This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.
Conversations in Review: AI, Geopolitics, and Lifestyle Revelations
We’ve made it to our final episode of 2023! Please enjoy as we highlight a few of the blue flame thinkers who joined us on the pod this year.
Episodes in order of guest appearances:
Conversation with Mustafa Suleyman — The Proliferation of AI & the Next Wave of Technology
The Promises and Perils of Neurotechnology – with Nita Farahany
Behind the Scenes of SVB’s Collapse + A Vision for America — with Ro Khanna
The State of Play: Markets, Economy, and Ukraine & China’s Growing Power, US Diplomacy, and Ukraine’s Counteroffensive — with Ian Bremmer
Conversation with Fareed Zakaria — The Conflict in Israel and the State of Foreign Affairs
Capitalism, Private Equity, and the Seven Deadly Sins — with Stephen Dubner
The Psychology of Money — with Morgan Housel
Conversation with Jennifer B. Wallace — What to Do About Toxic Achievement Culture
How to Get Unstuck — with Adam Alter
Conversation with Jennifer Cohen — Building Healthy Habits and Staying Confident
Understanding AI’s Threats and Opportunities — with Mo Gawdat
Masculinity, Media, and How to Citizen – with Baratunde Thurston
Conversation with Simon Sinek — Finding Your Why, Feeling Stuck, and Building Strong Leaders
Scott closes by thanking YOU for supporting the Prof G Pod in 2023.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
DR. JEFF BECK - THE BAYESIAN BRAIN
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
YT version: https://www.youtube.com/watch?v=c4praCiy9qU
Dr. Jeff Beck is a computational neuroscientist studying probabilistic reasoning (decision making under uncertainty) in humans and animals with emphasis on neural representations of uncertainty and cortical implementations of probabilistic inference and learning. His line of research incorporates information theoretic and hierarchical statistical analysis of neural and behavioural data as well as reinforcement learning and active inference.
https://www.linkedin.com/in/jeff-beck...
https://scholar.google.com/citations?...
Interviewer: Dr. Tim Scarfe
TOC
00:00:00 Intro
00:00:51 Bayesian / Knowledge
00:14:57 Active inference
00:18:58 Mediation
00:23:44 Philosophy of mind / science
00:29:25 Optimisation
00:42:54 Emergence
00:56:38 Steering emergent systems
01:04:31 Work plan
01:06:06 Representations/Core knowledge
#activeinference
Your right to mental privacy in the age of brain-sensing tech | Nita Farahany
Neurotechnology, or devices that let you track your own brain activity, could help you deeply understand your health. But without privacy protections, your innermost thoughts, emotions and desires could be at risk of exploitation, says neurotech and AI ethicist Nita Farahany. She details some of the field's promising potential uses -- like tracking and treating diseases from depression to epilepsy -- and shares concerns about who collects our brain data and how they plan to use it, ultimately calling for the legal recognition of "cognitive liberty" as we connect our brains and minds to technology.
Hosted on Acast. See acast.com/privacy for more information.
Nita Farahany (on neurotechnology and the law)
Nita Farahany (The Battle For Your Brain) is an author, professor, and scholar. Nita joins the Armchair Expert to discuss the effects of implementing brain wave monitoring, what body politics are, and how AI is currently being used to predict human behavior. Nita and Dax talk about the fears they have surrounding neurotechnology, whether transferring a person's consciousness will be possible some day, and how fast surveillance is normalized. Nita explains why the definition of death is being debated, how privacy practices often don't align with privacy preferences, and what cognitive rights laws could look like in the future.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
The Promises and Perils of Neurotechnology – with Nita Farahany
Nita Farahany, a professor of law at Duke University and the author of “The Battle for Your Brain: Defending the Right to Think Freely in the Age of Neurotechnology,” joins Scott to discuss the threat to our privacy and freedom of thought, and the future of neurotechnology. Follow Nita on Twitter, @NitaFarahany.
Scott opens by discussing the Biden Administration’s announcement that chip manufacturers that access $150M+ funding from the CHIPS Act must provide childcare. He then shares his thoughts on the bipartisan effort to take action against TikTok.
Algebra of Happiness: expressing love.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Here’s How Messy a Russian Bond Default Could Be
There’s a big question over whether Russia will be able (or willing) to make payments on billions of dollars it’s borrowed from investors given its current situation. Not only does the country have a history of previous major defaults, but some of its outstanding bonds are also structured kind of strangely. On this episode of the Odd Lots podcast, Tracy Alloway and Joe Weisenthal speak with University of Virginia law professor Mitu Gulati and University of North Carolina's Mark Weidemaier. They describe how odd some Russian bonds are and what might happen after default.
See omnystudio.com/listener for privacy information.