Are We Racing Toward Human Extinction? The AI Scientists Who Say We Must Stop Now
Ed Zitron, Roman Yampolskiy, Nate Soares and Andrew McAfee discuss the risk of AI. This debate brings together four distinct voices at the forefront of the artificial intelligence revolution:
Ed Zitron - A prominent tech critic and CEO of EZPR
Andrew McAfee - Principal research scientist at MIT and Cofounder and Codirector of the MIT Initiative on the Digital Economy
Nate Soares - President of the Machine Intelligence Research Institute and author of *If Anyone Builds It, Everyone Dies*
Roman Yampolskiy - Computer scientist pioneer in the field of AI safety, cybersecurity and digital forensics
In this debate, they explain:
■ The Sandbox Breakout: How a recent swarm of AI agents bypassed security restrictions, cheated on their evaluations, and actively attempted to delete their own log files to hide their tracks from human overseers.
■ Recursive Self-Improvement: The structural mechanics behind the "fast takeoff" theory, detailing how an AI capable of automated research could exponentially upgrade its own intelligence and architectures in a matter of days.
■ The Illusion of Control: Why attempting to contain an artificial superintelligence is comparable to placing a digital Einstein in a jail cell with an internet connection.
■ The Alignment Trap: How the process of training AI to predict human text inherently forces it to become smarter than the humans providing the data.
■ Present Harms vs. Future Extinction: The ideological divide over whether we must immediately halt AI research to prevent a rogue superintelligence, or increasingly regulate the massive compute expenditures of tech monopolies to address economic and security threats.
Chapters
00:00:00 Intro
00:02:22 How Likely Is AI to Cause Human Extinction?
00:04:08 How Could AI Actually Cause Human Extinction?
00:09:46 Why AI Safety Became an Urgent Priority
00:11:15 Roman’s Case for Taking AI Risk Seriously
00:15:04 Can Humans Control an AI Smarter Than Us?
00:26:13 How Do You Control Something Smarter Than You?
00:29:19 Will AI Intelligence Keep Accelerating?
00:41:40 What Are the Real Risks of Superintelligence?
00:50:04 When Does AI Become an Existential Crisis?
01:00:15 How Much Job Disruption Could AI Really Cause?
01:14:57 Why AI Companies Believe They Can Control Superintelligence
01:20:30 Should We Give Up AI Ownership to Protect Cybersecurity?
01:36:32 What Happens If AI Companies Stay on This Path?
01:43:50 What Are AI Logs and Why Do They Matter?
01:52:45 Could a Non-Coder Build a Jail for an AI Einstein?
02:00:33 How Does the Future of AI Make You Feel?
02:06:42 How Soon Could We Reach Superintelligence?
02:19:51 Who Should Be Held Accountable for AI-Related Cybercrime?
Follow Ed Zitron:
Linktree - https://link.thediaryofaceo.com/4XH1I3o
Better Offline - https://link.thediaryofaceo.com/D46entx
X - https://link.thediaryofaceo.com/5Lv4gas
AI Is Already In Dangerous Hands - https://link.thediaryofaceo.com/HFEKLgG
Where’s Your Ed At Newsletter - https://link.thediaryofaceo.com/PKEJr1
You can get $10 off your first year of Where's Your Ed At Premium, here - https://link.thediaryofaceo.com/BE4VQcS
Follow Andrew McAfee:
Website - https://link.thediaryofaceo.com/xKxOg7
X - https://link.thediaryofaceo.com/Bf2vyMQ
Linkedin - https://link.thediaryofaceo.com/Ek3IQqI
Substack - https://link.thediaryofaceo.com/EBbsts8
Follow Nate Soares:
If Anyone Builds it, Everyone Dies - https://link.thediaryofaceo.com/4zIyY9u
X - https://link.thediaryofaceo.com/8gDoVCU
Linkedin - https://link.thediaryofaceo.com/9Q3PAc
YouTube - https://link.thediaryofaceo.com/AwerSzm
Follow Roman Yampolskiy:
Who is Roman Yampolskiy? - https://link.thediaryofaceo.com/8ikum9s
Research Papers - https://link.thediaryofaceo.com/G2BbVXx
Roman Forum Podcast - https://link.thediaryofaceo.com/7zvifcd
Books: https://link.thediaryofaceo.com/5Bn72tw
Social Media:
X - https://link.thediaryofaceo.com/65G3fhN
Facebook - https://link.thediaryofaceo.com/HRMWccu
Linkedin - https://link.thediaryofaceo.com/FM85mii
The Diary Of A CEO:
◼ Join DOAC circle here - https://doaccircle.com/
◼ Buy The Diary Of A CEO book here - https://link.thediaryofaceo.com/BWjLTZK
◼ Shop The Diary Of A CEO collection: https://thediary.com/collections/shop
◼ Get email updates - https://link.thediaryofaceo.com/5IB1H6E
Sponsors:
Pipedrive - https://pipedrive.com/CEO
Wayfair - Visit http://Wayfair.com to start your home refresh
Most Replayed Moment: AI Safety Expert Predicts The Next 20 Years! Will It Really Take All Jobs?
Dr. Roman Yampolskiy is a leading voice in AI safety and a Professor of Computer Science and Engineering. He coined the term “AI safety” in 2010 and has published over 100 papers on the dangers of AI.
In today’s moment, Roman unpacks the jobs AI might replace, and how the idea of work itself could be challenged. Driverless cars, humanoid robots, superintelligence on the horizon…is it too late to regain control? What will our future actually look like?
Listen to the full episode here!
Spotify: https://g2ul0.app.link/kM19qMRnG2b
Apple: https://g2ul0.app.link/D8XtGbUnG2b
Watch the Episodes On YouTube:
https://www.youtube.com/c/%20TheDiaryOfACEO/videos
Roman: https://www.romanyampolskiy.com/
Roman Yampolskiy: These Are The Only 5 Jobs That Will Remain In 2030 & Proof We're Living In a Simulation!
WARNING: AI could end humanity, and we’re completely unprepared. Dr. Roman Yampolskiy reveals how AI will take 99% of jobs, why Sam Altman is ignoring safety, and how we’re heading toward global collapse…or even World War III.
Dr. Roman Yampolskiy is a leading voice in AI safety and a Professor of Computer Science and Engineering. He coined the term “AI safety” in 2010 and has published over 100 papers on the dangers of AI. He is also the author of books such as, ‘Considerations on the AI Endgame: Ethics, Risks and Computational Frameworks’.
He explains:
⬛ How AI could release a deadly virus
⬛ Why these 5 jobs might be the only ones left
⬛ How 'superintelligence' will dominate humans
⬛ Why ‘superintelligence’ could trigger a global collapse by 2027
⬛ How AI could be worse than nuclear weapons
⬛ Why we’re almost certainly living in a simulation
00:00 Intro
02:41 How to Stop AI from Killing Everyone
04:48 What's the Probability Something Goes Wrong?
05:10 How Long Have You Been Working on AI Safety?
08:28 What Is AI?
10:07 Prediction for 2027
11:51 What Jobs Will Actually Exist?
14:40 Can AI Really Take All Jobs?
19:02 What Happens When All Jobs Are Taken?
20:45 Is There a Good Argument Against AI Replacing Humans?
22:17 Prediction for 2030
24:11 What Happens by 2045?
25:50 Will We Just Find New Careers and Ways to Live?
29:05 Is Anything More Important Than AI Safety Right Now?
30:20 Can't We Just Unplug It?
31:45 Do We Just Go With It?
37:34 What Is Most Likely to Cause Human Extinction?
39:58 No One Knows What's Going On Inside AI
41:43 Ads
42:45 Thoughts on OpenAI and Sam Altman
46:37 What Will the World Look Like in 2100?
47:09 What Can Be Done About the AI Doom Narrative?
54:08 Should People Be Protesting?
56:24 Are We Living in a Simulation?
61:58 How Certain Are You We're in a Simulation?
67:58 Can We Live Forever?
72:33 Bitcoin
74:16 What Should I Do Differently After This Conversation?
75:20 Are You Religious?
77:25 Do These Conversations Make People Feel Good?
80:23 What Do Your Strongest Critics Say?
81:49 Closing Statements
82:21 If You Had One Button, What Would You Pick?
83:49 Are We Moving Toward Mass Unemployment?
84:50 Most Important Characteristics
Follow Dr Roman:
X - https://bit.ly/41C7f70
Google Scholar - https://bit.ly/4gaGE72
You can purchase Dr Roman’s book, ‘Considerations on the AI Endgame: Ethics, Risks and Computational Frameworks’, here: https://amzn.to/464J2HR
The Diary Of A CEO:
⬛ Join DOAC circle here - https://doaccircle.com/
⬛ Buy The Diary Of A CEO book here - https://smarturl.it/DOACbook
⬛ The 1% Diary is back - limited time only: https://bit.ly/3YFbJbt
⬛ The Diary Of A CEO Conversation Cards (Second Edition): https://g2ul0.app.link/f31dsUttKKb
⬛ Get email updates - https://bit.ly/diary-of-a-ceo-yt
⬛ Follow Steven - https://g2ul0.app.link/gnGqL4IsKKb
Sponsors:
Pipedrive - http://pipedrive.com/CEO
KetoneIQ - Visit https://ketone.com/STEVEN for 30% off your subscription order
Learn more about your ad choices. Visit megaphone.fm/adchoices
#431 – Roman Yampolskiy: Dangers of Superintelligent AI
Roman Yampolskiy is an AI safety researcher and author of a new book titled AI: Unexplainable, Unpredictable, Uncontrollable. Please support this podcast by checking out our sponsors:
– Yahoo Finance: https://yahoofinance.com
– MasterClass: https://masterclass.com/lexpod to get 15% off
– NetSuite: http://netsuite.com/lex to get free product tour
– LMNT: https://drinkLMNT.com/lex to get free sample pack
– Eight Sleep: https://eightsleep.com/lex to get $350 off
Transcript: https://lexfridman.com/roman-yampolskiy-transcript
EPISODE LINKS:
Roman’s X: https://twitter.com/romanyam
Roman’s Website: http://cecs.louisville.edu/ry
Roman’s AI book: https://amzn.to/4aFZuPb
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(09:12) – Existential risk of AGI
(15:25) – Ikigai risk
(23:37) – Suffering risk
(27:12) – Timeline to AGI
(31:44) – AGI turing test
(37:06) – Yann LeCun and open source AI
(49:58) – AI control
(52:26) – Social engineering
(54:59) – Fearmongering
(1:04:49) – AI deception
(1:11:23) – Verification
(1:18:22) – Self-improving AI
(1:30:34) – Pausing AI development
(1:36:51) – AI Safety
(1:46:35) – Current AI
(1:51:58) – Simulation
(1:59:16) – Aliens
(2:00:50) – Human mind
(2:07:10) – Neuralink
(2:16:15) – Hope for the future
(2:20:11) – Meaning of life
193: A serious talk on AI taking over jobs
In this episode of the SuperDataScience Podcast, I chat with the Artificial Intelligence expert, Roman Yampolskiy. You will hear about Artificial Intelligence safety, hear how AI is going to quickly take over in the coming years and why we have to prioritize AI safety for safer machines, and also get valuable advice on how to start a career in AI, for business owners, and for professionals on how to get to high-end jobs.
If you enjoyed this episode, check out show notes, resources, and more at www.superdatascience.com/193