Why Superhuman AI Would Kill Us All - Eliezer Yudkowsky - #1011
Eliezer Yudkowsky is an AI researcher, decision theorist, and founder of the Machine Intelligence Research Institute.
Is AI our greatest hope or our final mistake? For all its promise to revolutionize human life, there’s a growing fear that artificial intelligence could end it altogether. How grounded are these fears, how close are we to losing control, and is there still time to change course before it’s too late
Expect to learn the problem with building superhuman AI, why AI would have goals we haven’t programmed into it, if there is such a thing as Ai benevolence, what the actual goals of super-intelligent AI are and how far away it is, if LLMs are actually dangerous and their ability to become a super AI, how god we are at predicting the future of AI, if extinction if possible with the development of AI, and much more…
Timestamps:
(0:00) Superhuman AI Could Kill Us All
(10:25) How AI is Quietly Destroying Marriages
(15:22) AI is an Enemy, Not an Ally
(26:11) The Terrifying Truth About AI Alignment
(31:52) What Does Superintelligence Advancement Look Like?
(45:04) Are LLMs the Architect for Superhuman AI?
(52:18) How Close are We to the Point of No Return?
(01:01:07) Experts Need to be More Concerned
(01:15:01) How Can We Stop Superintelligence Killing Us?
(01:23:53) The Bleak Future of Superhuman AI
(01:31:55) Could Eliezer Be Wrong?
Sponsors:
See discounts for all the products I use and recommend: https://chriswillx.com/deals
Get 15% off your first order of Intake’s magnetic nasal strips at https://intakebreathing.com/modernwisdom
Get 10% discount on all Gymshark’s products at https://gym.sh/modernwisdom (use code MODERNWISDOM10)
Get 4 extra months of Surfshark VPN at https://surfshark.com/modernwisdom
Extra Stuff:
Get my free reading list of 100 books to read before you die: https://chriswillx.com/books
Try my productivity energy drink Neutonic: https://neutonic.com/modernwisdom
Episodes You Might Enjoy:
#577 - David Goggins - This Is How To Master Your Life: lnkfi.re/SN-Goggins
#712 - Dr Jordan Peterson - How To Destroy Your Negative Beliefs: lnkfi.re/SN-Peterson
#700 - Dr Andrew Huberman - The Secret Tools To Hack Your Brain: lnkfi.re/SN-Huberman
-
Get In Touch:
Instagram: https://www.instagram.com/chriswillx
Twitter: https://www.twitter.com/chriswillx
YouTube: https://www.youtube.com/modernwisdompodcast
Email: https://chriswillx.com/contact
-
Learn more about your ad choices. Visit megaphone.fm/adchoices
#434 — Can We Survive AI?
Sam Harris speaks with Eliezer Yudkowsky and Nate Soares about their new book, If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI. They discuss the alignment problem, ChatGPT and recent advances in AI, the Turing Test, the possibility of AI developing survival instincts, hallucinations and deception in LLMs, why many prominent voices in tech remain skeptical of the dangers of superintelligent AI, the timeline for superintelligence, real-world consequences of current AI systems, the imaginary line between the internet and reality, why Eliezer and Nate believe superintelligent AI would necessarily end humanity, how we might avoid an AI-driven catastrophe, the Fermi paradox, and other topics.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
Are We Past Peak iPhone? + Eliezer Yudkowsky on A.I. Doom
Apple’s yearly iPhone event took place this week, and it left us asking, Is Apple losing the juice? We break down all the new products the company announced and discuss where it goes from here. Then, Eliezer Yudkowsky, one of the most fascinating people in A.I., has a new book coming out: “If Anyone Builds It, Everyone Dies.” He joins us to make the case for why A.I. development should be shut down now, long before we reach superintelligence, and how he thinks that could happen.
Guests:
Eliezer Yudkowsky, founder of Machine Intelligence Research Institute and a co-author of “If Anyone Builds It, Everyone Dies”
Additional Reading:
A.I.’s Prophet of Doom Wants to Shut It All Down
AI as Normal Technology, revisited
Apple’s misunderstood crossbody iPhone strap might be the best I’ve seen
We want to hear from you. Email us at hardfork@nytimes.com. Find “Hard Fork” on YouTube and TikTok.
Subscribe today at nytimes.com/podcasts or on Apple Podcasts and Spotify. You can also subscribe via your favorite podcast app here https://www.nytimes.com/activate-access/audio?source=podcatcher. For more podcasts and narrated articles, download The New York Times app at nytimes.com/app.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Eliezer Yudkowsky and Stephen Wolfram on AI X-risk
Eliezer Yudkowsky and Stephen Wolfram discuss artificial intelligence and its potential existen‑
tial risks. They traversed fundamental questions about AI safety, consciousness, computational irreducibility, and the nature of intelligence.
The discourse centered on Yudkowsky’s argument that advanced AI systems pose an existential threat to humanity, primarily due to the challenge of alignment and the potential for emergent goals that diverge from human values. Wolfram, while acknowledging potential risks, approached the topic from a his signature measured perspective, emphasizing the importance of understanding computational systems’ fundamental nature and questioning whether AI systems would necessarily develop the kind of goal‑directed behavior Yudkowsky fears.
***
MLST IS SPONSORED BY TUFA AI LABS!
The current winners of the ARC challenge, MindsAI are part of Tufa AI Labs. They are hiring ML engineers. Are you interested?! Please goto https://tufalabs.ai/
***
TOC:
1. Foundational AI Concepts and Risks
[00:00:01] 1.1 AI Optimization and System Capabilities Debate
[00:06:46] 1.2 Computational Irreducibility and Intelligence Limitations
[00:20:09] 1.3 Existential Risk and Species Succession
[00:23:28] 1.4 Consciousness and Value Preservation in AI Systems
2. Ethics and Philosophy in AI
[00:33:24] 2.1 Moral Value of Human Consciousness vs. Computation
[00:36:30] 2.2 Ethics and Moral Philosophy Debate
[00:39:58] 2.3 Existential Risks and Digital Immortality
[00:43:30] 2.4 Consciousness and Personal Identity in Brain Emulation
3. Truth and Logic in AI Systems
[00:54:39] 3.1 AI Persuasion Ethics and Truth
[01:01:48] 3.2 Mathematical Truth and Logic in AI Systems
[01:11:29] 3.3 Universal Truth vs Personal Interpretation in Ethics and Mathematics
[01:14:43] 3.4 Quantum Mechanics and Fundamental Reality Debate
4. AI Capabilities and Constraints
[01:21:21] 4.1 AI Perception and Physical Laws
[01:28:33] 4.2 AI Capabilities and Computational Constraints
[01:34:59] 4.3 AI Motivation and Anthropomorphization Debate
[01:38:09] 4.4 Prediction vs Agency in AI Systems
5. AI System Architecture and Behavior
[01:44:47] 5.1 Computational Irreducibility and Probabilistic Prediction
[01:48:10] 5.2 Teleological vs Mechanistic Explanations of AI Behavior
[02:09:41] 5.3 Machine Learning as Assembly of Computational Components
[02:29:52] 5.4 AI Safety and Predictability in Complex Systems
6. Goal Optimization and Alignment
[02:50:30] 6.1 Goal Specification and Optimization Challenges in AI Systems
[02:58:31] 6.2 Intelligence, Computation, and Goal-Directed Behavior
[03:02:18] 6.3 Optimization Goals and Human Existential Risk
[03:08:49] 6.4 Emergent Goals and AI Alignment Challenges
7. AI Evolution and Risk Assessment
[03:19:44] 7.1 Inner Optimization and Mesa-Optimization Theory
[03:34:00] 7.2 Dynamic AI Goals and Extinction Risk Debate
[03:56:05] 7.3 AI Risk and Biological System Analogies
[04:09:37] 7.4 Expert Risk Assessments and Optimism vs Reality
8. Future Implications and Economics
[04:13:01] 8.1 Economic and Proliferation Considerations
SHOWNOTES (transcription, references, summary, best quotes etc):
https://www.dropbox.com/scl/fi/3st8dts2ba7yob161dchd/EliezerWolfram.pdf?rlkey=b6va5j8upgqwl9s2muc924vtt&st=vemwqx7a&dl=0
Revolutionizing AI: Tackling the Alignment Problem | Zuzalu #3
In this episode, we delve into the frontier of AI and the challenges surrounding AI alignment. The AI / Crypto overlap at Zuzalu sparked discussions on topics like ZKML, MEV bots, and the integration of AI agents into the Ethereum landscape.
However, the focal point was the alignment conversation, which showcased both pessimistic and resigned optimistic perspectives. We hear from Nate Sores of MIRI, who offers a downstream view on AI risk, and Deger Turan, who emphasizes the importance of human alignment as a prerequisite for aligning AI. Their discussions touch on epistemology, individual preferences, and the potential of AI to assist in personal and societal growth.
------ 🚀 Join Ryan & David at Permissionless in September. Bankless Citizens get 30% off. 🚀 https://bankless.cc/GoToPermissionless
------ BANKLESS SPONSOR TOOLS:
🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://k.xyz/bankless-pod-q2
🦊METAMASK PORTFOLIO | TRACK & MANAGE YOUR WEB3 EVERYTHING https://bankless.cc/MetaMask
⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum
🛞MANTLE | MODULAR LAYER 2 NETWORK https://bankless.cc/Mantle
👾POLYGON | VALUE LAYER OF THE INTERNET https://polygon.technology/roadmap
------
Timestamps
0:00 Intro 1:50 Guests
5:30 NATE SOARES 7:25 MIRI 13:30 Human Coordination 17:00 Dangers of Superintelligence 21:00 AI’s Big Moment 24:45 Chances of Doom 35:35 A Serious Threat 42:45 Talent is Scarce 48:20 Solving the Alignment Problem 59:35 Dealing with Pessimism 1:03:45 The Sliver of Utopia
1:14:00 DEGER TURAN 1:17:00 Solving Human Alignment 1:22:40 Using AI to Solve Problems 1:26:30 AI Objectives Institute 1:31:30 Epistemic Security 1:36:18 Curating AI Content 1:41:00 Scalable Coordination 1:47:15 Building Evolving Systems 1:54:00 Independent Flexible Systems 1:58:30 The Problem is the Solution 2:03:30 A Better Future
----- Resources
Nate Soares https://twitter.com/So8res?s=20
Deger Turan https://twitter.com/degerturann?s=20
MIRI https://intelligence.org/
Less Wrong AI Alignment xhttps://www.lesswrong.com/tag/ai-alignment-intro-materials
AI Objectives Institute https://aiobjectives.org/
------
Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.
Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures
Will superintelligent AI end the world? | Eliezer Yudkowsky
Decision theorist Eliezer Yudkowsky has a simple message: superintelligent AI could probably kill us all. So the question becomes: Is it possible to build powerful artificial minds that are obedient, even benevolent? In a fiery talk, Yudkowsky explores why we need to act immediately to ensure smarter-than-human AI systems don't lead to our extinction.
Hosted on Acast. See acast.com/privacy for more information.
EP 63: Eliezer Yudkowsky (AI Safety Expert) Explains How AI Could Destroy Humanity
Eliezer Yudkowsky is a researcher, writer, and advocate for artificial intelligence safety. He is best known for his writings on rationality, cognitive biases, and the development of superintelligence. Yudkowsky has written extensively on the topic of AI safety and has advocated for the development of AI systems that are aligned with human values and interests. Yudkowsky is the co-founder of the Machine Intelligence Research Institute (MIRI), a non-profit organization dedicated to researching the development of safe and beneficial artificial intelligence. He is also a co-founder of the Center for Applied Rationality (CFAR), a non-profit organization focused on teaching rational thinking skills. He is also a frequent author at LessWrong.com as well as Rationality: From AI to Zombies.
In this episode, we discuss Eliezer’s concerns with artificial intelligence and his recent conclusion that it will inevitably lead to our demise. He’s a brilliant mind, an interesting person, and genuinely believes all of the stuff he says. So I wanted to have a conversation with him to hear where he is coming from, how he got there, understand AI better, and hopefully help us bridge the divide between the people who think we’re headed off a cliff and the people who think it’s not a big deal.
(0:00) Intro
(1:18) Welcome Eliezer
(6:27) How would you define artificial intelligence?
(15:50) What is the purpose of a firm alarm?
(19:29) Eliezer’s background
(29:28) The Singularity Institute for Artificial Intelligence
(33:38) Maybe AI doesn’t end up automatically doing the right thing
(45:42) AI Safety Conference
(51:15) Disaster Monkeys
(1:02:15) Fast takeoff
(1:10:29) Loss function
(1:15:48) Protein folding
(1:24:55) The deadly stuff
(1:46:41) Why is it inevitable?
(1:54:27) Can’t we let tech develop AI and then fix the problems?
(2:02:56) What were the big jumps between GPT3 and GPT4?
(2:07:15) “The trajectory of AI is inevitable”
(2:28:05) Elon Musk and OpenAI
(2:37:41) Sam Altman Interview
(2:50:38) The most optimistic path to us surviving
(3:04:46) Why would anything super intelligent pursue ending humanity?
(3:14:08) What role do VCs play in this?
Show Notes:
https://twitter.com/liron/status/1647443778524037121?s=20
https://futureoflife.org/event/ai-safety-conference-in-puerto-rico/
https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy
https://www.youtube.com/watch?v=q9Figerh89g
https://www.vox.com/the-highlight/23447596/artificial-intelligence-agi-openai-gpt3-existential-risk-human-extinction
Eliezer Yudkowsky – AI Alignment: Why It's Hard, and Where to Start
Mixed and edited: Justin Hrabovsky
Produced: Rashad Assir
Executive Producer: Josh Machiz
Music: Griff Lawson
🎙 Listen to the show
Apple Podcasts: https://podcasts.apple.com/us/podcast/three-cartoon-avatars/id1606770839
Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1
Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
🐦 Twitter - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode, Logan goes behind the scenes with world-class entrepreneurs and investors. If you're interested in the real inside baseball of tech, entrepreneurship, and start-up investing, tune in every Friday for new episodes.
Executive Producer: Rashad Assir
Producer: Leah Clapper
Mixing and editing: Justin Hrabovsky
Check out Unsupervised Learning, Redpoint's AI Podcast: https://www.youtube.com/@UCUl-s_Vp-Kkk_XVyDylNwLA
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
📱 X - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode of The Logan Bartlett Show, we sit down with the people behind today’s most important startups and extract the tactics, lessons, and frameworks they’ve learned the hard way. Conversations span hiring to GTM, product, growth, fundraising and everything in between - collectively forming the ultimate playbook to make you a better CEO, investor or board member.
Tap follow and enable notifications to stay ahead of the game.
Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality
For 4 hours, I tried to come up reasons for why AI might not kill us all, and Eliezer Yudkowsky explained why I was wrong.
We also discuss his call to halt AI, why LLMs make alignment harder, what it would take to save humanity, his millions of words of sci-fi, and much more.
If you want to get to the crux of the conversation, fast forward to 2:35:00 through 3:43:54. Here we go through and debate the main reasons I still think doom is unlikely.
Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.
Timestamps
(0:00:00) - TIME article
(0:09:06) - Are humans aligned?
(0:37:35) - Large language models
(1:07:15) - Can AIs help with alignment?
(1:30:17) - Society’s response to AI
(1:44:42) - Predictions (or lack thereof)
(1:56:55) - Being Eliezer
(2:13:06) - Othogonality
(2:35:00) - Could alignment be easier than we think?
(3:02:15) - What will AIs want?
(3:43:54) - Writing fiction & whether rationality helps you win
Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
#368 – Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization
Eliezer Yudkowsky is a researcher, writer, and philosopher on the topic of superintelligent AI. Please support this podcast by checking out our sponsors:
– Linode: https://linode.com/lex to get $100 free credit
– House of Macadamias: https://houseofmacadamias.com/lex and use code LEX to get 20% off your first order
– InsideTracker: https://insidetracker.com/lex to get 20% off
EPISODE LINKS:
Eliezer’s Twitter: https://twitter.com/ESYudkowsky
LessWrong Blog: https://lesswrong.com
Eliezer’s Blog page: https://www.lesswrong.com/users/eliezer_yudkowsky
Books and resources mentioned:
1. AGI Ruin (blog post): https://lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities
2. Adaptation and Natural Selection: https://amzn.to/40F5gfa
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(05:19) – GPT-4
(28:00) – Open sourcing GPT-4
(44:18) – Defining AGI
(52:14) – AGI alignment
(1:35:06) – How AGI may kill us
(2:27:27) – Superintelligence
(2:34:39) – Evolution
(2:41:09) – Consciousness
(2:51:41) – Aliens
(2:57:12) – AGI Timeline
(3:05:11) – Ego
(3:11:03) – Advice for young people
(3:16:21) – Mortality
(3:18:02) – Love
DEBRIEF - We're All Gonna Die
Debriefing the episode with Eliezer Yudkowsky. This one was so good, we had to share. The fate of humanity might depend on it.
The Debrief Episode goes out EVERY MONDAY for Bankless Citizens. Want the Debrief Episode? Get the Premium RSS feed by Subscribing to Bankless!
WATCH THE FULL EPISODE HERE: https://youtu.be/gA1sNLL6yg4
------ 🚀 SUBSCRIBE TO NEWSLETTER: https://newsletter.banklesshq.com/
----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.
Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures
159 - We’re All Gonna Die with Eliezer Yudkowsky
Eliezer Yudkowsky is an author, founder, and leading thinker in the AI space.
------ ✨ DEBRIEF | Unpacking the episode: https://shows.banklesshq.com/p/debrief-eliezer ------ ✨ COLLECTIBLES | Collect this episode: https://collectibles.bankless.com/mint
------ We wanted to do an episode on AI… and we went deep down the rabbit hole. As we went down, we discussed ChatGPT and the new generation of AI, digital superintelligence, the end of humanity, and if there’s anything we can do to survive.
This conversation with Eliezer Yudkowsky sent us into an existential crisis, with the primary claim that we are on the cusp of developing AI that will destroy humanity.
Be warned before diving into this episode, dear listener. Once you dive in, there’s no going back.
------ 📣 MetaMask Learn | Learn Web3 with the Leading Web3 Wallet https://bankless.cc/
------ 🚀 JOIN BANKLESS PREMIUM: https://newsletter.banklesshq.com/subscribe
------ BANKLESS SPONSOR TOOLS:
🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://bankless.cc/kraken
🦄UNISWAP | ON-CHAIN MARKETPLACE https://bankless.cc/uniswap
⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum
👻 PHANTOM | #1 SOLANA WALLET https://bankless.cc/phantom-waitlist
------ Topics Covered
0:00 Intro 10:00 ChatGPT 16:30 AGI 21:00 More Efficient than You 24:45 Modeling Intelligence 32:50 AI Alignment 36:55 Benevolent AI 46:00 AI Goals 49:10 Consensus 55:45 God Mode and Aliens 1:03:15 Good Outcomes 1:08:00 Ryan’s Childhood Questions 1:18:00 Orders of Magnitude 1:23:15 Trying to Resist 1:30:45 Miri and Education 1:34:00 How Long Do We Have? 1:38:15 Bearish Hope 1:43:50 The End Goal
------ Resources:
Eliezer Yudkowsky https://twitter.com/ESYudkowsky
MIRI https://intelligence.org/
Reply to Francois Chollet https://intelligence.org/2017/12/06/chollet/
Grabby Aliens https://grabbyaliens.com/
----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.
Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures
#116 — AI: Racing Toward the Brink
Sam Harris speaks with Eliezer Yudkowsky about the nature of intelligence, different types of AI, the "alignment problem," IS vs OUGHT, the possibility that future AI might deceive us, the AI arms race, conscious AI, coordination problems, and other topics.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
