The God We Deserve: Nonzero's Robert Wright on AI as Humanity's Ultimate Test
Robert Wright of Nonzero joins Nathan to discuss The God Test, his argument that AI is humanity’s “God test” rather than just a technical challenge. They explore his evolutionary lens on deep learning, from training as selection to marketplace selection among models that may reward selectively honest, power-sensing, or deceptive agents. Wright connects those risks to the noosphere, US-China relations, cognitive empathy, and whether global coordination arrives deliberately or through a coercive singleton. The stakes are whether consumers, companies, and governments select for AIs that strengthen non-zero-sum cooperation or race toward systems that reflect and amplify our worst incentives.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/the-god-we-deserve-nonzero-s-robert-wright-on-ai-as-humanity-s-ultimate-test/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(04:23) Special Sponsor
(06:10) Early AI encounters
(18:36) Pretraining as evolution (Part 1)
(19:50) Sponsor: Claude
(21:42) Pretraining as evolution (Part 2)
(32:19) Deceptive market pressures
(41:13) Noosphere and directionality
(51:02) Global brain choices
(01:08:23) Designing wiser models
(01:24:11) Reframing China relations
(01:37:57) Building organic transparency
(01:45:54) Positive nationalism and tools
(01:54:21) Pausing superintelligence races
(02:09:07) Applications and consciousness
(02:25:27) Episode Outro
(02:28:27) Outro
PRODUCED BY:
https://aipodcast.ing
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Is an AI Arms Race Inevitable? with Robert Wright of Nonzero Newsletter & Podcast
In this episode of The Cognitive Revolution, Nathan explores the complex intersection of AI development and international relations with Robert Wright, publisher of the Nonzero Newsletter. They examine the growing militarization of AI, US-China relations, and the concerning trajectory of what Wright calls "the chip war." Drawing from Nathan's recent experience at The Curve conference and an AI wargame simulation, they discuss the risks of an AI arms race and search for alternative paths to avoid catastrophic conflict between global powers. Join us for this crucial conversation about the future of AI governance and international cooperation.
Subscribe to The NonZero Newsletter at https://nonzero.substack.com
Be notified early when Turpentine's drops new publication: https://www.turpentine.co/exclusiveaccess
RECOMMENDED PODCAST:
Unpack Pricing - Dive into the dark arts of SaaS pricing with Metronome CEO Scott Woody and tech leaders. Learn how strategic pricing drives explosive revenue growth in today's biggest companies like Snowflake, Cockroach Labs, Dropbox and more.
Apple: https://podcasts.apple.com/us/podcast/id1765716600
Spotify: https://open.spotify.com/show/38DK3W1Fq1xxQalhDSueFg
SPONSORS:
Notion: Notion offers powerful workflow and automation templates, perfect for streamlining processes and laying the groundwork for AI-driven automation. With Notion AI, you can search across thousands of documents from various platforms, generating highly relevant analysis and content tailored just for you - try it for free at https://notion.com/cognitiverevolution
80,000 Hours: 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues.
SelectQuote: Finding the right life insurance shouldn't be another task you put off. SelectQuote compares top-rated policies to get you the best coverage at the right price. Even in our AI-driven world, protecting your family's future remains essential. Get your personalized quote at https://selectquote.com/cognitive
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers13. OCI powers industry leaders with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before December 31, 2024 at https://oracle.com/cognitive
CHAPTERS:
(00:00:00) Teaser
(00:00:59) Sponsors: Incogni
(00:02:20) About the Episode
(00:05:56) Introducing AI2
(00:09:56) Tulu: Deep Dive (Part 1)
(00:17:43) Sponsors: Notion | Shopify
(00:20:38) Open vs. Closed Recipes
(00:29:48) Compute & Value (Part 1)
(00:34:22) Sponsors: Oracle Cloud Infrastructure (OCI) | 80,000 Hours
(00:37:02) Compute & Value (Part 2)
(00:42:41) Model Weight Evolution
(00:53:16) DPO vs. PPO
(01:06:36) Project Trajectory
(01:20:39) Synthetic Data & LLM Judge
(01:27:39) Verifiable RL
(01:38:17) Advice for Practitioners
(01:44:01) Open Source vs. Closed
(01:49:18) Outro
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://www.linkedin.com/in/nathanlabenz/
Youtube: https://www.youtube.com/@CognitiveRevolutionPodcast
Understanding AI "Understanding" with Robert Wright of Nonzero Newsletter & Podcast
In this special crossover episode of The Cognitive Revolution, Nathan Labenz joins Robert Wright of the Nonzero newsletter and podcast to explore pressing questions about AI development. They discuss the nature of understanding in large language models, multimodal AI systems, reasoning capabilities, and the potential for AI to accelerate scientific discovery. The conversation also covers AI interpretability, ethics, open-sourcing models, and the implications of US-China relations on AI development.
Subscribe to The Nonzero Newsletter at https://nonzero.substack.com and Podcast at https://www.youtube.com/@Nonzero/videos
Apply to join over 400 founders and execs in the Turpentine Network: https://hmplogxqz0y.typeform.com/to/JCkphVqj
RECOMMENDED PODCAST:
History 102
Every week, creator of WhatifAltHist Rudyard Lynch and Erik Torenberg cover a major topic in history in depth -- in under an hour. This season will cover classical Greece, early America, the Vikings, medieval Islam, ancient China, the fall of the Roman Empire, and more.Subscribe on
Spotify: https://open.spotify.com/show/36Kqo3BMMUBGTDo1IEYihm
Apple: https://podcasts.apple.com/us/podcast/history-102-with-whatifalthists-rudyard-lynch-and/id1730633913
YouTube: https://www.youtube.com/@History102-qg5oj
SPONSORS:
Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist.
CHAPTERS:
(00:00:00) About the Show
(00:00:22) About the Episode
(00:03:39) Introduction and Background
(00:06:58) AI Capabilities and Understanding
(00:12:22) Discussing Martin Casado's Views (Part 1)
(00:14:44) Sponsors: Oracle | Brave
(00:16:48) Discussing Martin Casado's Views (Part 2)
(00:21:51) Multimodal AI and Concept Representation (Part 1)
(00:31:40) Sponsors: Omneky | Squad
(00:33:26) Multimodal AI and Concept Representation (Part 2)
(00:38:05) AI's Potential and Limitations
(00:45:35) AI Safety and Risk Assessment
(00:53:31) AI Development and Global Implications
(01:03:30) Open Source AI and International Relations
(01:11:27) AI Ethics and Human Values
(01:22:06) AI Risk and Existential Threats
(01:31:21) Open Source AI Concerns
(01:38:20) China-US AI Relations
(01:48:36) State Space Models in AI
(02:02:32) Conclusion and Recommendations
(02:03:54) Outro
---
SOCIAL LINKS:
Website : https://www.cognitiverevolution.ai
Twitter (Podcast) : https://x.com/cogrev_podcast
Twitter (Nathan) : https://x.com/labenz
The Open Source AI Question - Part 2 | Robert Wright & Nathan Labenz
Dive into an in-depth conversation with Nathan and Robert Wright as they discuss AI's transformative potential, mechanistic interpretability, and the sobering realities of AI alignment research. Learn about the defensive strategies and safety measures necessary for managing advanced AI risks in an open source world. Don't miss the insights on AI-powered VR, and be sure to check out part one on the non-zero feed.
Checkout the Part 1 of the conversation here : https://www.youtube.com/watch?v=s8bgB8TCdBs
SPONSORS:
Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention "Turpentine" to skip the waitlist.
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
CHAPTERS:
(00:00:00) Introduction
(00:07:13) AI in Governance
(00:11:08) Sci-fi doomer
(00:13:58) Sponsors: Oracle | Brave
(00:16:05) The frontier models
(00:20:22) Emergent behavior
(00:23:48) Theory of mind
(00:28:09) Mechanistic interpretability
(00:34:12) Sponsors: Squad | Omneky
(00:38:12) AI Alignment Techniques
(00:42:38) The Sweet Spot of AI
