#487 — Is AI Already Conscious?
Sam Harris speaks with Cameron Berg about whether AI systems are or could become conscious. They discuss self-reports in LLMs, what models say when deception is switched off, the "bliss attractor" state, consciousness and sentience, the hard problem of consciousness, parallels between neural networks and biological brains, the moral risk of building minds that can suffer, possible parallels to factory farming, moral patienthood, the alignment problem, and other topics.
Annaka Harris's upcoming book: Unlocking Consciousness
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
AI:AM #4: Cameron on Model Consciousness, Duvenaud's Gradual Disempowerment, swyx's AI-Eng Alpha
This AI:AM highlights cut brings together Cameron Berg, David Duvenaud, Michiel Bakker, Shawn “swyx” Wang, and Bing Xu to examine what we understand about frontier AI systems and what happens as more decisions move into their hands. Berg grounds model-consciousness debates in experiments on architecture, agency, valence, and welfare, while Duvenaud argues that even well-aligned AI could gradually disempower humans through ordinary economic choices. Bakker frames Europe’s AI challenge as a sovereignty problem, and swyx turns to practitioner stakes around agents, evals, maintainable code, and who owns the system of record. Xu closes the loop at the infrastructure layer, arguing that self-improving compute and GPU-kernel automation may deepen rather than weaken the CUDA moat.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/ai-am-4-cameron-on-model-consciousness-duvenaud-s-gradual-disempowerment-swyx-s-ai-eng-alpha/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(00:38) Special Sponsor
(02:26) Model consciousness indicators
(10:47) Valence inside models (Part 1)
(17:09) Sponsor: Claude
(19:01) Valence inside models (Part 2)
(19:01) Misalignment and uncertainty
(25:16) Gradual disempowerment threat
(35:10) Slow zones and successors
(47:41) Europe's AI bind
(55:13) Frontier code benchmarks
(01:01:59) Routing and memory
(01:10:42) Agent infrastructure strain
(01:16:25) Self improving infrastructure
(01:27:56) Routing compute costs
(01:35:39) Sovereign AI financing
(01:42:24) Judging AI judges
(01:47:26) Building AI DNA
(01:52:38) Episode Outro
(01:55:09) Outro
PRODUCED BY:
https://aipodcast.ing
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
AI:AM #3: Zvi on Fable, the Cases For & Against the Ban, + AI for Math, Logistics & More
Zvi Mowshowitz joins AI in the AM to unpack Anthropic's Fable system card, including its FrontierMath leap, troubling Vending-Bench behavior, decision-theory drift, and signs that model reasoning may be becoming harder to read. The episode then turns to the US government's attempted export-control action against Fable, with Zvi arguing that the cited jailbreak demonstration did not prove the claimed threat while still faulting Anthropic's political handling. Sam Hammond and Judd Rosenblatt add competing reads on state capacity, CAISI, NSA-driven caution, and the alignment world's failure to build trust across partisan lines. The stakes are whether frontier AI capability, safety evaluation, and government power can be coordinated before medicine, mathematics, software, and cyber-relevant systems move further ahead.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/ai-am-3-zvi-on-fable-the-cases-for-against-the-ban-ai-for-math-logistics-more/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(01:28) Special Sponsor
(03:17) Weekly highlights preview
(05:23) Fable capability alarms
(16:29) Anthropic government strategy (Part 1)
(16:34) Sponsor: Claude
(18:26) Anthropic government strategy (Part 2)
(27:16) Cyber ban rationale
(37:14) Government power politics
(48:57) Unavoidable control risks
(01:01:42) Government mechanics and empathy
(01:12:50) Legal authority limits
(01:19:02) Pause Overton window
(01:31:58) Medicine, math, safety
(01:47:27) Software without code
(02:01:19) Enterprise world models
(02:10:46) Episode Outro
(02:13:39) Outro
PRODUCED BY:
https://aipodcast.ing
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Does Learning Require Feeling? Cameron Berg on the latest AI Consciousness & Welfare Research
Cameron Berg returns to discuss the latest research on AI consciousness and model welfare. He breaks down new evidence for model introspection, including studies showing that systems can detect interventions on their own internal states and sometimes resist them. They also examine Anthropic's work on functional emotions, the implications of Claude's welfare reports, and Berg's new ideas about how reinforcement learning may shape positive and negative experience. The conversation makes the case for a more precautionary, mutualist approach to advanced AI systems.
Sponsors:
Roboflow:
Roboflow is the computer vision infrastructure founders use to build AI-powered sports analytics and make the physical world programmable. Read the PlayVision story and start your first project for free at https://roboflow.com
Tasklet:
Build your own Cognitive Revolution monitoring agent in one click.
Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
VCX:
VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com
Claude:
Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr
More Truthful AIs Report Conscious Experience: New Mechanistic Research w- Cameron Berg @ AE Studio
Cameron Berg, Research Director at AE Studio, shares his team's groundbreaking research exploring whether frontier AI systems report subjective experiences. They discovered that prompts inducing self-referential processing consistently lead models to claim consciousness, and a mechanistic study on Llama 3.3 70B revealed that suppressing deception features makes the model *more* likely to report it. This suggests that promoting truth-telling in AIs could reveal a deeper, more complex internal state, a finding Scott Alexander calls "the only exception" to typical AI consciousness discussions. The episode delves into the profound implications for two-way human-AI alignment and the critical need for a precautionary approach to AI consciousness.
LINKS:
Janus' argument on LLM attention
Safety Pretraining arXiv Paper
Self-Referential AI Paper Site
Self-Referential AI arXiv Paper
Judd Rosenblatt's Tweet Thread
Cameron Berg's Goodfire Demo
Podcast with Milo YouTube Playlist
Cameron Berg's LinkedIn Profile
Cameron Berg's X Profile
AE Studio AI Alignment
Sponsors:
Framer:
Framer is the all-in-one platform that unifies design, content management, and publishing on a single canvas, now enhanced with powerful AI features. Start creating for free and get a free month of Framer Pro with code COGNITIVE at https://framer.com/design
Tasklet:
Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
Linear:
Linear is the system for modern product development. Nearly every AI company you've heard of is using Linear to build products. Get 6 months of Linear Business for free at: https://linear.app/tcr
Shopify:
Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
PRODUCED BY:
https://aipodcast.ing
Biologically Inspired AI Alignment & Neglected Approaches to AI Safety, with Judd Rosenblatt and Mike Vaiana of AE Studio
In this episode of The Cognitive Revolution, Nathan explores unconventional approaches to AI safety with Judd Rosenblatt and Mike Vaiana from AE Studio. Discover how this innovative company pivoted from brain-computer interfaces to groundbreaking AI alignment research, producing two notable results in cooperative and less deceptive AI systems. Join us for a deep dive into biologically-inspired approaches that offer hope for solving critical AI safety challenges.
Self-Modeling: https://arxiv.org/abs/2407.10188
Self-Other Distinction Minimization: https://www.alignmentforum.org/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment
Neglected approaches blog post: https://www.lesswrong.com/posts/qAdDzcBuDBLexb4fC/the-neglected-approaches-approach-ae-studio-s-alignment
Apply to join over 400 Founders and Execs in the Turpentine Network: https://www.turpentinenetwork.co/
SPONSORS:
WorkOS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network
Weights & Biases Weave: Weights & Biases Weave is a lightweight AI developer toolkit designed to simplify your LLM app development. With Weave, you can trace and debug input, metadata and output with just 2 lines of code. Make real progress on your LLM development and visit the following link to get started with Weave today: https://wandb.me/cr
80,000 Hours: 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues.
Omneky: Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
RECOMMENDED PODCAST:
This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel ft their hottest takes on the future of tech, business, and venture capital.
Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz
CHAPTERS:
(00:00:00) About the Show
(00:00:22) Sponsors: WorkOS
(00:01:22) About the Episode
(00:05:18) Introduction and AE Studio Background
(00:11:37) Keys to Success in Building AE Studio
(00:16:57) Sponsors: Weights & Biases Weave | 80,000 Hours
(00:19:37) Universal Launcher and Productivity Gains
(00:24:44) 100x Productivity Increase Explanation
(00:31:46) Brain-Computer Interface and AI Alignment
(00:38:05) Sponsors: Omneky
(00:38:30) Current State of NeuroTech
(00:44:00) Survey on Neglected Approaches in AI Alignment
(00:50:41) Self-Modeling and Biological Inspiration
(00:57:48) Technical Details of Self-Modeling
(01:06:17) Self-Other Distinction Minimization
(01:12:44) Implementation in Language Models
(01:19:00) Compute Costs and Scaling Considerations
(01:24:27) Consciousness Concerns and Future Work
(01:40:24) Evaluating Neglected Approaches
(01:55:56) Closing Thoughts and Policy Considerations
(01:59:25) Outro
