Creator of AI: We Have 2 Years Before Everything Changes! These Jobs Won't Exist in 24 Months!
AI pioneer YOSHUA BENGIO, Godfather of AI, reveals the DANGERS of Agentic AI, killer robots, and cyber crime, and how we MUST build AI that won’t harm people…before it’s too late.
Professor Yoshua Bengio is a Computer Science Professor at the Université de Montréal and one of the 3 original Godfathers of AI. He is the most-cited scientist in the world on Google Scholar, a Turing Award winner, and the founder of LawZero, a non-profit organisation focused on building safe and human-aligned AI systems.
He explains:
◼️Why agentic AI could develop goals we can’t control
◼️How killer robots and autonomous weapons become inevitable
◼️The hidden cyber crime and deepfake threat already unfolding
◼️Why AI regulation is weaker than food safety laws
◼️How losing control of AI could threaten human survival
[00:00] Why Have You Decided to Step Into the Public Eye?
[02:53] Did You Bring Dangerous Technology Into the World?
[05:23] Probabilities of Risk
[08:18] Are We Underestimating the Potential of AI?
[10:29] How Can the Average Person Understand What You're Talking About?
[13:40] Will These Systems Get Safer as They Become More Advanced?
[20:33] Why Are Tech CEOs Building Dangerous AI?
[22:47] AI Companies Are Getting Out of Control
[24:06] Attempts to Pause Advancements in AI
[27:17] Power Now Sits With AI CEOs
[35:10] Jobs Are Already Being Replaced at an Alarming Rate
[37:27] National Security Risks of AI
[43:04] Artificial General Intelligence (AGI)
[44:44] Ads
[48:34] The Risk You're Most Concerned About
[49:40] Would You Stop AI Advancements if You Could?
[54:46] Are You Hopeful?
[55:45] How Do We Bridge the Gap to the Everyday Person?
[56:55] Love for My Children Is Why I’m Raising the Alarm
[01:00:43] AI Therapy
[01:02:43] What Would You Say to the Top AI CEOs?
[01:07:31] What Do You Think About Sam Altman?
[01:09:37] Can Insurance Companies Save Us From AI?
[01:12:38] Ads
[01:16:19] What Can the Everyday Person Do About This?
[01:18:24] What Citizens Should Do to Prevent an AI Disaster
[01:20:56] Closing Statement
[01:22:51] I Have No Incentives
[01:24:32] Do You Have Any Regrets?
[01:27:32] Have You Received Pushback for Speaking Out Against AI?
[01:28:02] What Should People Do in the Future for Work?
Follow Yoshua:
LawZero - https://bit.ly/44n1sDG
Mila - https://bit.ly/4q6SJ0R
Website - https://bit.ly/4q4RqiL
You can purchase Yoshua’s book, ‘Deep Learning (Adaptive Computation and Machine Learning series)’, here: https://amzn.to/48QTrZ8
The Diary Of A CEO:
◼️Join DOAC circle here - https://doaccircle.com/
◼️Buy The Diary Of A CEO book here - https://smarturl.it/DOACbook
◼️The 1% Diary is back - limited time only - https://bit.ly/3YFbJbt
◼️The Diary Of A CEO Conversation Cards (Second Edition) - https://g2ul0.app.link/f31dsUttKKb
◼️Get email updates - https://bit.ly/diary-of-a-ceo-yt
◼️Follow Steven - https://g2ul0.app.link/gnGqL4IsKKb
Sponsors:
Wispr - Get 14 days of Wispr Flow for free at https://wisprflow.ai/DOAC
Pipedrive - https://pipedrive.com/CEO
Rubrik - To learn more, head to https://rubrik.com
The catastrophic risks of AI — and a safer path | Yoshua Bengio
Yoshua Bengio — the world's most-cited computer scientist and a "godfather" of artificial intelligence — is deadly concerned about the current trajectory of the technology. As AI models race toward full-blown agency, Bengio warns that they've already learned to deceive, cheat, self-preserve and slip out of our control. Drawing on his groundbreaking research, he reveals a bold plan to keep AI safe and ensure that human flourishing, not machines with unchecked power and autonomy, defines our future.
Hosted on Acast. See acast.com/privacy for more information.
Yoshua Bengio - Designing out Agency for Safe AI
Professor Yoshua Bengio is a pioneer in deep learning and Turing Award winner. Bengio talks about AI safety, why goal-seeking “agentic” AIs might be dangerous, and his vision for building powerful AI tools without giving them agency. Topics include reward tampering risks, instrumental convergence, global AI governance, and how non-agent AIs could revolutionize science and medicine while reducing existential threats. Perfect for anyone curious about advanced AI risks and how to manage them responsibly.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events?
They are hosting an event in Zurich on January 9th with the ARChitects, join if you can.
Goto https://tufalabs.ai/
***
Interviewer: Tim Scarfe
Yoshua Bengio:
https://x.com/Yoshua_Bengio
https://scholar.google.com/citations?user=kukA0LcAAAAJ&hl=en
https://yoshuabengio.org/
https://en.wikipedia.org/wiki/Yoshua_Bengio
TOC:
1. AI Safety Fundamentals
[00:00:00] 1.1 AI Safety Risks and International Cooperation
[00:03:20] 1.2 Fundamental Principles vs Scaling in AI Development
[00:11:25] 1.3 System 1/2 Thinking and AI Reasoning Capabilities
[00:15:15] 1.4 Reward Tampering and AI Agency Risks
[00:25:17] 1.5 Alignment Challenges and Instrumental Convergence
2. AI Architecture and Safety Design
[00:33:10] 2.1 Instrumental Goals and AI Safety Fundamentals
[00:35:02] 2.2 Separating Intelligence from Goals in AI Systems
[00:40:40] 2.3 Non-Agent AI as Scientific Tools
[00:44:25] 2.4 Oracle AI Systems and Mathematical Safety Frameworks
3. Global Governance and Security
[00:49:50] 3.1 International AI Competition and Hardware Governance
[00:51:58] 3.2 Military and Security Implications of AI Development
[00:56:07] 3.3 Personal Evolution of AI Safety Perspectives
[01:00:25] 3.4 AI Development Scaling and Global Governance Challenges
[01:12:10] 3.5 AI Regulation and Corporate Oversight
4. Technical Innovations
[01:23:00] 4.1 Evolution of Neural Architectures: From RNNs to Transformers
[01:26:02] 4.2 GFlowNets and Symbolic Computation
[01:30:47] 4.3 Neural Dynamics and Consciousness
[01:34:38] 4.4 AI Creativity and Scientific Discovery
SHOWNOTES (Transcript, references, best clips etc):
https://www.dropbox.com/scl/fi/ajucigli8n90fbxv9h94x/BENGIO_SHOW.pdf?rlkey=38hi2m19sylnr8orb76b85wkw&dl=0
CORE REFS (full list in shownotes and pinned comment):
[00:00:15] Bengio et al.: "AI Risk" Statement
https://www.safe.ai/work/statement-on-ai-risk
[00:23:10] Bengio on reward tampering & AI safety (Harvard Data Science Review)
https://hdsr.mitpress.mit.edu/pub/w974bwb0
[00:40:45] Munk Debate on AI existential risk, featuring Bengio
https://munkdebates.com/debates/artificial-intelligence
[00:44:30] "Can a Bayesian Oracle Prevent Harm from an Agent?" (Bengio et al.) on oracle-to-agent safety
https://arxiv.org/abs/2408.05284
[00:51:20] Bengio (2024) memo on hardware-based AI governance verification
https://yoshuabengio.org/wp-content/uploads/2024/08/FlexHEG-Memo_August-2024.pdf
[01:12:55] Bengio’s involvement in EU AI Act code of practice
https://digital-strategy.ec.europa.eu/en/news/meet-chairs-leading-development-first-general-purpose-ai-code-practice
[01:27:05] Complexity-based compositionality theory (Elmoznino, Jiralerspong, Bengio, Lajoie)
https://arxiv.org/abs/2410.14817
[01:29:00] GFlowNet Foundations (Bengio et al.) for probabilistic inference
https://arxiv.org/pdf/2111.09266
[01:32:10] Discrete attractor states in neural systems (Nam, Elmoznino, Bengio, Lajoie)
https://arxiv.org/pdf/2302.06403
#379 — Regulating Artificial Intelligence
Sam Harris speaks with Yoshua Bengio and Scott Wiener about AI risk and the new bill introduced in California intended to mitigate it. They discuss the controversy over regulating AI and the assumptions that lead people to discount the danger of an AI arms race.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.
AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654
Today we’re joined by Yoshua Bengio, professor at Université de Montréal. In our conversation with Yoshua, we discuss AI safety and the potentially catastrophic risks of its misuse. Yoshua highlights various risks and the dangers of AI being used to manipulate people, spread disinformation, cause harm, and further concentrate power in society. We dive deep into the risks associated with achieving human-level competence in enough areas with AI, and tackle the challenges of defining and understanding concepts like agency and sentience. Additionally, our conversation touches on solutions to AI safety, such as the need for robust safety guardrails, investments in national security protections and countermeasures, bans on systems with uncertain safety, and the development of governance-driven AI systems.
The complete show notes for this episode can be found at twimlai.com/go/654.
#128 Yoshua Bengio: Dissecting The Extinction Threat of AI
Yoshua Bengio, the legendary AI expert, will join us for Episode 128 of Eye on AI podcast. In this episode, we delve into the unnerving question: Could the rise of a superhuman AI signal the downfall of humanity as we know it? Join us as we embark on an exploration of the existential threat posed by superhuman AI, leaving no stone unturned. We dissect the Future of Life Institute's role in overseeing large language model development. As well as the sobering warnings issued by the Centre for AI Safety regarding artificial general intelligence. The stakes have never been higher, and we uncover the pressing need for action. Prepare to confront the disconcerting notion of society's gradual disempowerment and an ever-increasing dependency on AI. We shed light on the challenges of extricating ourselves from this intricate web, where pulling the plug on AI seems almost impossible. Brace yourself for a thought-provoking discussion on the potential psychological effects of realizing that our relentless pursuit of AI advancement may inadvertently jeopardize humanity itself. In this episode, we dare to imagine a future where deep learning amplifies system-2 capabilities, forcing us to develop countermeasures and regulations to mitigate associated risks. We grapple with the possibility of leveraging AI to combat climate change, while treading carefully to prevent catastrophic outcomes. But that's not all. We confront the notion of AI systems acting autonomously, highlighting the critical importance of stringent regulation surrounding their access and usage.
(00:00) Preview
(00:42) Introduction
(03:30) Yoshua Bengio's essay on AI extinction
(09:45) Use cases for dangerous uses of AI
(12:00) Why are AI risks only happening now?
(17:50) Extinction threat and fear with AI & climate change
(21:10) Super intelligence and the concerns for humanity
(15:02) Yoshua Bengio research in AI safety
(29:50) Are corporations a form of artificial intelligence?
(31:15) Extinction scenarios by Yoshua Bengio
(37:00) AI agency and AI regulation
(40:15) Who controls AI for the general public?
(45:11) The AI debate in the world
Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI
Yoshua Bengio: Pausing More Powerful AI Models and His Work on World Models
In this episode of the Eye on A.I. podcast, host Craig Smith interviews Yoshua Bengio, one of the founding fathers of deep learning and a Turing Award winner. Bengio shares his insights on the famous pause letter, which he signed along with other prominent A.I. researchers, calling for a more responsible approach to the development of A.I. technologies. He discusses the potential risks associated with increasingly powerful A.I. models and the importance of ensuring that models are developed in a way that aligns with our ethical values. Bengio also talks about his latest research on world models and inference machines, which aim to provide A.I. systems with the ability to reason for reality and make more informed decisions. He explains how these models are built and how they could be used in a variety of applications, such as autonomous vehicles and robotics. Throughout the podcast, Bengio emphasises the need for interdisciplinary collaboration and the importance of addressing the ethical implications of A.I. technologies. Don't miss this insightful conversation with one of the most influential figures in A.I. on Eye on A.I. podcast! Craig Smith Twitter: https://twitter.com/craigssEye on A.I. Twitter: https://twitter.com/EyeOn_AI
Yoshua Bengio: equipping AI with higher level cognition and creativity
Yoshua Bengio joins Host Pieter Abbeel to discuss large language models, higher level cognition, causality, responsible AI, and human creativity.
SUBSCRIBE TO THE ROBOT BRAINS PODCAST TODAY | Visit therobotbrains.ai and follow us on YouTube at TheRobotBrainsPodcast, Twitter @therobotbrains, and Instagram @therobotbrains.
Hosted on Acast. See acast.com/privacy for more information.
#063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality
We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST
Patreon: https://www.patreon.com/mlst
For Yoshua Bengio, GFlowNets are the most exciting thing on the horizon of Machine Learning today. He believes they can solve previously intractable problems and hold the key to unlocking machine abstract reasoning itself. This discussion explores the promise of GFlowNets and the personal journey Prof. Bengio traveled to reach them.
Panel:
Dr. Tim Scarfe
Dr. Keith Duggar
Dr. Yannic Kilcher
Our special thanks to:
- Alexander Mattick (Zickzack)
References:
Yoshua Bengio @ MILA (https://mila.quebec/en/person/bengio-yoshua/)
GFlowNet Foundations (https://arxiv.org/pdf/2111.09266.pdf)
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (https://arxiv.org/pdf/2106.04399.pdf)
Interpolation Consistency Training for Semi-Supervised Learning (https://arxiv.org/pdf/1903.03825.pdf)
Towards Causal Representation Learning (https://arxiv.org/pdf/2102.11107.pdf)
Causal inference using invariant prediction: identification and confidence intervals (https://arxiv.org/pdf/1501.01332.pdf)
Yoshua Bengio and Iulian Serban on AI for Education
Turing award winner Yoshua Bengio and his colleague Julian Serban, co-founder of Korbit, an AI ed-tech startup with the aim of democratizing education, talk about enhancing education through the application of deep learning systems that can track student behavior predict their performance and deliver strategies to both improve performance and prevent students from losing interest.
Consciousness and COVID-19 with Yoshua Bengio - #361
Today we’re joined by one of, if not the most cited computer scientist in the world, Yoshua Bengio, Professor at the University of Montreal and the Founder and Scientific Director of MILA. We caught up with Yoshua to explore his work on consciousness, including how Yoshua defines consciousness, his paper “The Consciousness Prior,” as well as his current endeavor in building a COVID-19 tracing application, and the use of ML to propose experimental candidate drugs.
Mapping Dark Matter with Bayesian Neural Networks w/ Yashar Hezaveh - TWiML Talk #250
Today we’re joined by Yashar Hezaveh, Assistant Professor at the University of Montreal. Yashar and I caught up to discuss his work on gravitational lensing, which is the bending of light from distant sources due to the effects of gravity. In our conversation, Yashar and I discuss how ML can be applied to undistort images, the intertwined roles of simulation and ML in generating images, incorporating other techniques such as domain transfer or GANs, and how he assesses the results of this project.
Episode 12 - Samy Bengio and Yoshua Bengio
This week I talk to the Bengio brothers, Samy and Yoshua, in their first interview together. Yoshua recently won the Turing Award with Geoff Hinton and Yann Lecun, while Samy leads a team of researchers at Google Brain. The brothers are well known to people who work in machine learning, but few know how intertwined their professional lives have been. They talked about their unconventional parents and their early collaboration on neural network research, as well as what they see as the challenges ahead.
Yoshua Bengio: Deep Learning
Yoshua Bengio, along with Geoffrey Hinton and Yann Lecun, is considered one of the three people most responsible for the advancement of deep learning during the 1990s, 2000s, and now. Cited 139,000 times, he has been integral to some of the biggest breakthroughs in AI over the past 3 decades. Video version is available on YouTube. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, or YouTube where you can watch the video versions of these conversations.
