As top AI firms race to develop a “superintelligent” agent — one that surpasses human cognition — are they prioritizing safety guardrails along the way?
Not according to the Future of Life Institute’s AI Safety Index, which reported that companies have gone back on their own promises to take a pause if their technology ever got close to certain risky points.
Marketplace’s Meghan McCarty Carino spoke with Sabina Nong, AI safety investigator at the Future of Life Institute, about what measures might ensure durable safety around AI moving forward.
Max Tegmark and Dean Ball debate whether we should ban the development of superintelligence in a crossover episode from Doom Debates hosted by Liron Shapira. PSA for AI builders: Interested in alignment, governance, or AI safety? Learn more about the MATS Summer 2026 Fellowship and submit your name to be notified when applications open: https://matsprogram.org/s26-tcr. They unpack the Future of Life Institute's call for a moratorium until there is broad scientific consensus and public buy-in, contrasting Tegmark’s precautionary stance with Dean’s emphasis on experimentation, competition, and practical policy hurdles. Listeners will get clear takes on p(doom), the limits of FDA-style regulation, unilateral ban risks, and what safe, beneficial advanced AI might realistically look like.
LINKS:
AMA episode feedback survey form
Doom Debates Substack newsletter
Sponsors:
Framer:
Framer is the all-in-one tool to design, iterate, and publish stunning websites with powerful AI features. Start creating for free and use code COGNITIVE to get one free month of Framer Pro at https://framer.com/design
Agents of Scale:
Agents of Scale is a podcast from Zapier CEO Wade Foster, featuring conversations with C-suite leaders who are leading AI transformation. Subscribe to the show wherever you get your podcasts
Tasklet:
Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai
Shopify:
Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive
PRODUCED BY:
https://aipodcast.ing
CHAPTERS:
(00:00) About the Episode
(05:43) Cold open and intro
(09:21) Opening statements: ban debate (Part 1)
(14:49) Sponsors: Framer | Agents of Scale
(17:11) Opening statements: ban debate (Part 2)
(17:11) Licensing-style AI regulation
(26:52) Liability, tail risks (Part 1)
(33:24) Sponsors: Tasklet | Shopify
(36:32) Liability, tail risks (Part 2)
(39:23) Timelines and precautionary regulation
(47:03) Defining superintelligence and risk
(52:26) Risk-based safety standards
(56:28) Current regulations and definitions
(01:05:23) Max's doom scenario
(01:19:46) P-doom gap and adaptation
(01:34:40) National security and China
(01:43:57) Closing statements and reflections
(01:55:22) Host debrief and outro
(02:02:10) Outro
Anthony Aguirre is the executive director of the Future of Life Institute. He joins Big Technology to discuss how AI could fail in the worst case and whether our push toward increasingly autonomous, general systems puts control out of reach. Tune in to hear how agentic systems operating at superhuman speed complicate oversight, and why “just unplug it” is naive. Hit play for a cool‑headed, nuanced conversation with clear takeaways you can use to evaluate AI strategy, policy, and risk.
---
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b
Questions? Feedback? Write to: bigtechnologypodcast@gmail.com
Learn more about your ad choices. Visit megaphone.fm/adchoices
The current explosion of exciting commercial and open-source AI is likely to be followed, within a few years, by creepily superintelligent AI – which top researchers and experts fear could disempower or wipe out humanity. Scientist Max Tegmark describes an optimistic vision for how we can keep AI under control and ensure it's working for us, not the other way around.
Hosted on Acast. See acast.com/privacy for more information.
Jaan Tallinn is the Co-Founder of Skype and one of the most prominent voices in AI safety. After co-founding Skype he begun to delve into the world of AI safety and strongly holds the position that humans can not motivate AI and that action must be taken. Jaan initiated the 6-month pause on AI that was co-signed by Elon Musk. Nonetheless he's invested in some of the most influential AI companies like DeepMind and Anthropic.
On this episode he shares why he invests in AI despite his views on safety, the insights he gained from founding Skype and his first impression meeting SBF in the Bahamas.
(0:00) Intro
(0:54) Jaan's journey with crypto
(2:49) Investing in SBF
(8:08) Entrepreneurial journey to Skype
(15:11) Skype's immediate rise to popularity and mistakes along the way
(22:15) Meeting Eliezer Yudkowsky
(25:11) The Center for Existential Risk in the Future of Life Institute
(31:05) Having a seat at the table by investing in artificial intelligence
(37:24) Having an entrepreneur say no to you
(41:59) The process of the DeepMind sale and the ethics board
(45:58) The risk of artificial intelligence today
(1:04:43) What was that unnerving about GPT four versus GPT three?
(1:07:02) Jaan's memo on AI safety
(1:16:00) What percentage likelihood do you think we're on a path to extinction?
(1:22:18) What's a piece of conventional wisdom around startups or investing that you disagree with today?
Mixed and edited: Justin Hrabovsky
Produced: Rashad Assir
Executive Producer: Josh Machiz
Music: Griff Lawson
🎙 Listen to the show
Apple Podcasts: https://podcasts.apple.com/us/podcast/three-cartoon-avatars/id1606770839
Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1
Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
🐦 Twitter - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode, Logan goes behind the scenes with world-class entrepreneurs and investors. If you're interested in the real inside baseball of tech, entrepreneurship, and start-up investing, tune in every Friday for new episodes.
Executive Producer: Rashad Assir
Producer: Leah Clapper
Mixing and editing: Justin Hrabovsky
Check out Unsupervised Learning, Redpoint's AI Podcast: https://www.youtube.com/@UCUl-s_Vp-Kkk_XVyDylNwLA
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
📱 X - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode of The Logan Bartlett Show, we sit down with the people behind today’s most important startups and extract the tactics, lessons, and frameworks they’ve learned the hard way. Conversations span hiring to GTM, product, growth, fundraising and everything in between - collectively forming the ultimate playbook to make you a better CEO, investor or board member.
Tap follow and enable notifications to stay ahead of the game.
Max Tegmark is a physicist and AI researcher at MIT, co-founder of the Future of Life Institute, and author of Life 3.0: Being Human in the Age of Artificial Intelligence. Please support this podcast by checking out our sponsors:
– Notion: https://notion.com
– InsideTracker: https://insidetracker.com/lex to get 20% off
– Indeed: https://indeed.com/lex to get $75 credit
EPISODE LINKS:
Max’s Twitter: https://twitter.com/tegmark
Max’s Website: https://space.mit.edu/home/tegmark
Pause Giant AI Experiments (open letter): https://futureoflife.org/open-letter/pause-giant-ai-experiments
Future of Life Institute: https://futureoflife.org
Books and resources mentioned:
1. Life 3.0 (book): https://amzn.to/3UB9rXB
2. Meditations on Moloch (essay): https://slatestarcodex.com/2014/07/30/meditations-on-moloch
3. Nuclear winter paper: https://nature.com/articles/s43016-022-00573-0
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:34) – Intelligent alien civilizations
(19:58) – Life 3.0 and superintelligent AI
(31:25) – Open letter to pause Giant AI Experiments
(56:32) – Maintaining control
(1:25:22) – Regulation
(1:36:12) – Job automation
(1:45:27) – Elon Musk
(2:07:09) – Open source
(2:13:39) – How AI may kill all humans
(2:24:10) – Consciousness
(2:33:32) – Nuclear winter
(2:44:00) – Questions for AGI
Nathan Labenz dives in with Jaan Tallinn, a technologist, entrepreneur (Kazaa, Skype), and investor (DeepMind and more) whose unique life journey has intersected with some of the most important social and technological events of our collective lifetime. Jaan has since invested in nearly 180 startups, including dozens of AI application layer companies and some half dozen startup labs that focus on fundamental AI research, all in an effort to support the teams that he believes most likely to lead us to AI safety, and to have a seat at the table at organizations that he worries might take on too much risk. He's also founded several philanthropic nonprofits, including the Future of Life Institute, which recently published the open letter calling for a six-month pause on training new AI systems. In this discussion, we focused on:
- the current state of AI development and safety
-Jan's expectations for possible economic transformation
- what catastrophic failure modes worry him most in the near term
- How big of a bullet we dodged with the training of GPT-4
- Which organizations really matter for immediate-term pause purposes
- How AI race dynamics are likely to evolve over the next couple of years
LINKS REFERENCED IN THE EPISODE:
Future of Life's open letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/
Eliezer Yudkowsky's TIME article: https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/
Podcast: Daniela and Dario Amodei on Anthropic- https://podcasts.apple.com/ie/podcast/daniela-and-dario-amodei-on-anthropic/id1170991978?i=1000552976406
Zvi on the pause: https://thezvi.substack.com/p/on-the-fli-ai-risk-open-letter
--
We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.
SPONSORS:
Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive
Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
TIMESTAMPS:
(0:00) Episode Preview
(1:30) Jaan's impressive entrepreneurial career and his role in the recent AI Open Letter
(3:26) AI safety and Future of Life Institute
(6:55) Jaan's first meeting with Eliezer Yudkowsky and the founding of the Future of Life Institute
(13:00) Future of AI evolution
(15:55) Sponsor: Omneky
(17:20) Jaan's investments in AI companies
(24:22) The emerging danger paradigm
(33:48) AI supervising itself
(40:06) Evolution, useful heuristics, and lack of insight into selection process
(43:13) Current estimate for life-ending catastrophe
(54:20) Our luck given the softness of language models
(56:24) Future of Language Models
(1:01:00) The Moore’s law of mad science
(1:03:02) GPT-5 type project
(1:11:00) AI alignment with the latest models
(1:14:31) AI research investment and safety
(1:21:00) What a six month pause buys us
(1:27:01) AI’s Turing Test Passing
(1:33:18) Responsible AI development.
(1:41:20) Neuralink implant technology
Max Tegmark is a physicist and AI researcher at MIT. Please support this podcast by checking out our sponsors:
– The Jordan Harbinger Show: https://www.jordanharbinger.com/lex/
– Four Sigmatic: https://foursigmatic.com/lex and use code LexPod to get up to 60% off
– BetterHelp: https://betterhelp.com/lex to get 10% off
– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free
EPISODE LINKS:
News Project Explainer Video: https://www.youtube.com/watch?v=PRLF17Pb6vo
News Project Website: https://www.improvethenews.org/
Max’s Twitter: https://twitter.com/tegmark
Max’s Website: https://space.mit.edu/home/tegmark/
Future of Life Institute: https://futureoflife.org/
Lex Fridman Podcast #1: https://www.youtube.com/watch?v=Gi8LUnhP5yU
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/LexFridmanPage
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(08:15) – AI and physics
(21:32) – Can AI discover new laws of physics?
(30:22) – AI safety
(47:59) – Extinction of human species
(58:57) – How to fix fake news and misinformation
(1:20:30) – Autonomous weapons
(1:35:54) – The man who prevented nuclear war
(1:46:02) – Elon Musk and AI
(1:59:39) – AI alignment
(2:05:42) – Consciousness
(2:14:45) – Richard Feynman
(2:18:56) – Machine learning and computational physics
(2:29:53) – AI and creativity
(2:41:08) – Aliens
(2:56:51) – Mortality
A conversation with Max Tegmark as part of MIT course on Artificial General Intelligence. Video version is available on YouTube. He is a Physics Professor at MIT, co-founder of the Future of Life Institute, and author of “Life 3.0: Being Human in the Age of Artificial Intelligence.” If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, or YouTube where you can watch the video versions of these conversations.
Many artificial intelligence researchers expect AI to outsmart humans at all tasks and jobs within decades, enabling a future where we're restricted only by the laws of physics, not the limits of our intelligence. MIT physicist and AI researcher Max Tegmark separates the real opportunities and threats from the myths, describing the concrete steps we should take today to ensure that AI ends up being the best -- rather than worst -- thing to ever happen to humanity.
Hosted on Acast. See acast.com/privacy for more information.
Sam Harris speaks with Rebecca Goldstein and Max Tegmark about the foundations of human knowledge and morality.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
Sam Harris speaks with Max Tegmark about his new book Life 3.0: Being Human in the Age of Artificial Intelligence. They talk about the nature of intelligence, the risks of superhuman AI, a nonbiological definition of life, the substrate independence of minds, the relevance and irrelevance of consciousness for the future of AI, near-term breakthroughs in AI, and other topics.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
Sam Harris speaks with MIT cosmologist Max Tegmark about the foundations of science, our current understanding of the universe, and the risks of future breakthroughs in artificial intelligence.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.