Creator of AI: We Have 2 Years Before Everything Changes! These Jobs Won't Exist in 24 Months!
AI pioneer YOSHUA BENGIO, Godfather of AI, reveals the DANGERS of Agentic AI, killer robots, and cyber crime, and how we MUST build AI that won’t harm people…before it’s too late.
Professor Yoshua Bengio is a Computer Science Professor at the Université de Montréal and one of the 3 original Godfathers of AI. He is the most-cited scientist in the world on Google Scholar, a Turing Award winner, and the founder of LawZero, a non-profit organisation focused on building safe and human-aligned AI systems.
He explains:
◼️Why agentic AI could develop goals we can’t control
◼️How killer robots and autonomous weapons become inevitable
◼️The hidden cyber crime and deepfake threat already unfolding
◼️Why AI regulation is weaker than food safety laws
◼️How losing control of AI could threaten human survival
[00:00] Why Have You Decided to Step Into the Public Eye?
[02:53] Did You Bring Dangerous Technology Into the World?
[05:23] Probabilities of Risk
[08:18] Are We Underestimating the Potential of AI?
[10:29] How Can the Average Person Understand What You're Talking About?
[13:40] Will These Systems Get Safer as They Become More Advanced?
[20:33] Why Are Tech CEOs Building Dangerous AI?
[22:47] AI Companies Are Getting Out of Control
[24:06] Attempts to Pause Advancements in AI
[27:17] Power Now Sits With AI CEOs
[35:10] Jobs Are Already Being Replaced at an Alarming Rate
[37:27] National Security Risks of AI
[43:04] Artificial General Intelligence (AGI)
[44:44] Ads
[48:34] The Risk You're Most Concerned About
[49:40] Would You Stop AI Advancements if You Could?
[54:46] Are You Hopeful?
[55:45] How Do We Bridge the Gap to the Everyday Person?
[56:55] Love for My Children Is Why I’m Raising the Alarm
[01:00:43] AI Therapy
[01:02:43] What Would You Say to the Top AI CEOs?
[01:07:31] What Do You Think About Sam Altman?
[01:09:37] Can Insurance Companies Save Us From AI?
[01:12:38] Ads
[01:16:19] What Can the Everyday Person Do About This?
[01:18:24] What Citizens Should Do to Prevent an AI Disaster
[01:20:56] Closing Statement
[01:22:51] I Have No Incentives
[01:24:32] Do You Have Any Regrets?
[01:27:32] Have You Received Pushback for Speaking Out Against AI?
[01:28:02] What Should People Do in the Future for Work?
Follow Yoshua:
LawZero - https://bit.ly/44n1sDG
Mila - https://bit.ly/4q6SJ0R
Website - https://bit.ly/4q4RqiL
You can purchase Yoshua’s book, ‘Deep Learning (Adaptive Computation and Machine Learning series)’, here: https://amzn.to/48QTrZ8
The Diary Of A CEO:
◼️Join DOAC circle here - https://doaccircle.com/
◼️Buy The Diary Of A CEO book here - https://smarturl.it/DOACbook
◼️The 1% Diary is back - limited time only - https://bit.ly/3YFbJbt
◼️The Diary Of A CEO Conversation Cards (Second Edition) - https://g2ul0.app.link/f31dsUttKKb
◼️Get email updates - https://bit.ly/diary-of-a-ceo-yt
◼️Follow Steven - https://g2ul0.app.link/gnGqL4IsKKb
Sponsors:
Wispr - Get 14 days of Wispr Flow for free at https://wisprflow.ai/DOAC
Pipedrive - https://pipedrive.com/CEO
Rubrik - To learn more, head to https://rubrik.com
The catastrophic risks of AI — and a safer path | Yoshua Bengio
Yoshua Bengio — the world's most-cited computer scientist and a "godfather" of artificial intelligence — is deadly concerned about the current trajectory of the technology. As AI models race toward full-blown agency, Bengio warns that they've already learned to deceive, cheat, self-preserve and slip out of our control. Drawing on his groundbreaking research, he reveals a bold plan to keep AI safe and ensure that human flourishing, not machines with unchecked power and autonomy, defines our future.
Hosted on Acast. See acast.com/privacy for more information.
Yoshua Bengio - Designing out Agency for Safe AI
Professor Yoshua Bengio is a pioneer in deep learning and Turing Award winner. Bengio talks about AI safety, why goal-seeking “agentic” AIs might be dangerous, and his vision for building powerful AI tools without giving them agency. Topics include reward tampering risks, instrumental convergence, global AI governance, and how non-agent AIs could revolutionize science and medicine while reducing existential threats. Perfect for anyone curious about advanced AI risks and how to manage them responsibly.
SPONSOR MESSAGES:
***
CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.
https://centml.ai/pricing/
Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. Are you interested in working on reasoning, or getting involved in their events?
They are hosting an event in Zurich on January 9th with the ARChitects, join if you can.
Goto https://tufalabs.ai/
***
Interviewer: Tim Scarfe
Yoshua Bengio:
https://x.com/Yoshua_Bengio
https://scholar.google.com/citations?user=kukA0LcAAAAJ&hl=en
https://yoshuabengio.org/
https://en.wikipedia.org/wiki/Yoshua_Bengio
TOC:
1. AI Safety Fundamentals
[00:00:00] 1.1 AI Safety Risks and International Cooperation
[00:03:20] 1.2 Fundamental Principles vs Scaling in AI Development
[00:11:25] 1.3 System 1/2 Thinking and AI Reasoning Capabilities
[00:15:15] 1.4 Reward Tampering and AI Agency Risks
[00:25:17] 1.5 Alignment Challenges and Instrumental Convergence
2. AI Architecture and Safety Design
[00:33:10] 2.1 Instrumental Goals and AI Safety Fundamentals
[00:35:02] 2.2 Separating Intelligence from Goals in AI Systems
[00:40:40] 2.3 Non-Agent AI as Scientific Tools
[00:44:25] 2.4 Oracle AI Systems and Mathematical Safety Frameworks
3. Global Governance and Security
[00:49:50] 3.1 International AI Competition and Hardware Governance
[00:51:58] 3.2 Military and Security Implications of AI Development
[00:56:07] 3.3 Personal Evolution of AI Safety Perspectives
[01:00:25] 3.4 AI Development Scaling and Global Governance Challenges
[01:12:10] 3.5 AI Regulation and Corporate Oversight
4. Technical Innovations
[01:23:00] 4.1 Evolution of Neural Architectures: From RNNs to Transformers
[01:26:02] 4.2 GFlowNets and Symbolic Computation
[01:30:47] 4.3 Neural Dynamics and Consciousness
[01:34:38] 4.4 AI Creativity and Scientific Discovery
SHOWNOTES (Transcript, references, best clips etc):
https://www.dropbox.com/scl/fi/ajucigli8n90fbxv9h94x/BENGIO_SHOW.pdf?rlkey=38hi2m19sylnr8orb76b85wkw&dl=0
CORE REFS (full list in shownotes and pinned comment):
[00:00:15] Bengio et al.: "AI Risk" Statement
https://www.safe.ai/work/statement-on-ai-risk
[00:23:10] Bengio on reward tampering & AI safety (Harvard Data Science Review)
https://hdsr.mitpress.mit.edu/pub/w974bwb0
[00:40:45] Munk Debate on AI existential risk, featuring Bengio
https://munkdebates.com/debates/artificial-intelligence
[00:44:30] "Can a Bayesian Oracle Prevent Harm from an Agent?" (Bengio et al.) on oracle-to-agent safety
https://arxiv.org/abs/2408.05284
[00:51:20] Bengio (2024) memo on hardware-based AI governance verification
https://yoshuabengio.org/wp-content/uploads/2024/08/FlexHEG-Memo_August-2024.pdf
[01:12:55] Bengio’s involvement in EU AI Act code of practice
https://digital-strategy.ec.europa.eu/en/news/meet-chairs-leading-development-first-general-purpose-ai-code-practice
[01:27:05] Complexity-based compositionality theory (Elmoznino, Jiralerspong, Bengio, Lajoie)
https://arxiv.org/abs/2410.14817
[01:29:00] GFlowNet Foundations (Bengio et al.) for probabilistic inference
https://arxiv.org/pdf/2111.09266
[01:32:10] Discrete attractor states in neural systems (Nam, Elmoznino, Bengio, Lajoie)
https://arxiv.org/pdf/2302.06403
#379 — Regulating Artificial Intelligence
Sam Harris speaks with Yoshua Bengio and Scott Wiener about AI risk and the new bill introduced in California intended to mitigate it. They discuss the controversy over regulating AI and the assumptions that lead people to discount the danger of an AI arms race.
If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.
Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.
AI Sentience, Agency and Catastrophic Risk with Yoshua Bengio - #654
Today we’re joined by Yoshua Bengio, professor at Université de Montréal. In our conversation with Yoshua, we discuss AI safety and the potentially catastrophic risks of its misuse. Yoshua highlights various risks and the dangers of AI being used to manipulate people, spread disinformation, cause harm, and further concentrate power in society. We dive deep into the risks associated with achieving human-level competence in enough areas with AI, and tackle the challenges of defining and understanding concepts like agency and sentience. Additionally, our conversation touches on solutions to AI safety, such as the need for robust safety guardrails, investments in national security protections and countermeasures, bans on systems with uncertain safety, and the development of governance-driven AI systems.
The complete show notes for this episode can be found at twimlai.com/go/654.
#128 Yoshua Bengio: Dissecting The Extinction Threat of AI
Yoshua Bengio, the legendary AI expert, will join us for Episode 128 of Eye on AI podcast. In this episode, we delve into the unnerving question: Could the rise of a superhuman AI signal the downfall of humanity as we know it? Join us as we embark on an exploration of the existential threat posed by superhuman AI, leaving no stone unturned. We dissect the Future of Life Institute's role in overseeing large language model development. As well as the sobering warnings issued by the Centre for AI Safety regarding artificial general intelligence. The stakes have never been higher, and we uncover the pressing need for action. Prepare to confront the disconcerting notion of society's gradual disempowerment and an ever-increasing dependency on AI. We shed light on the challenges of extricating ourselves from this intricate web, where pulling the plug on AI seems almost impossible. Brace yourself for a thought-provoking discussion on the potential psychological effects of realizing that our relentless pursuit of AI advancement may inadvertently jeopardize humanity itself. In this episode, we dare to imagine a future where deep learning amplifies system-2 capabilities, forcing us to develop countermeasures and regulations to mitigate associated risks. We grapple with the possibility of leveraging AI to combat climate change, while treading carefully to prevent catastrophic outcomes. But that's not all. We confront the notion of AI systems acting autonomously, highlighting the critical importance of stringent regulation surrounding their access and usage.
(00:00) Preview
(00:42) Introduction
(03:30) Yoshua Bengio's essay on AI extinction
(09:45) Use cases for dangerous uses of AI
(12:00) Why are AI risks only happening now?
(17:50) Extinction threat and fear with AI & climate change
(21:10) Super intelligence and the concerns for humanity
(15:02) Yoshua Bengio research in AI safety
(29:50) Are corporations a form of artificial intelligence?
(31:15) Extinction scenarios by Yoshua Bengio
(37:00) AI agency and AI regulation
(40:15) Who controls AI for the general public?
(45:11) The AI debate in the world
Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI
Yoshua Bengio: Pausing More Powerful AI Models and His Work on World Models
In this episode of the Eye on A.I. podcast, host Craig Smith interviews Yoshua Bengio, one of the founding fathers of deep learning and a Turing Award winner. Bengio shares his insights on the famous pause letter, which he signed along with other prominent A.I. researchers, calling for a more responsible approach to the development of A.I. technologies. He discusses the potential risks associated with increasingly powerful A.I. models and the importance of ensuring that models are developed in a way that aligns with our ethical values. Bengio also talks about his latest research on world models and inference machines, which aim to provide A.I. systems with the ability to reason for reality and make more informed decisions. He explains how these models are built and how they could be used in a variety of applications, such as autonomous vehicles and robotics. Throughout the podcast, Bengio emphasises the need for interdisciplinary collaboration and the importance of addressing the ethical implications of A.I. technologies. Don't miss this insightful conversation with one of the most influential figures in A.I. on Eye on A.I. podcast! Craig Smith Twitter: https://twitter.com/craigssEye on A.I. Twitter: https://twitter.com/EyeOn_AI
Yoshua Bengio: equipping AI with higher level cognition and creativity
Yoshua Bengio joins Host Pieter Abbeel to discuss large language models, higher level cognition, causality, responsible AI, and human creativity.
SUBSCRIBE TO THE ROBOT BRAINS PODCAST TODAY | Visit therobotbrains.ai and follow us on YouTube at TheRobotBrainsPodcast, Twitter @therobotbrains, and Instagram @therobotbrains.
Hosted on Acast. See acast.com/privacy for more information.
#102 - Prof. MICHAEL LEVIN, Prof. IRINA RISH - Emergence, Intelligence, Transhumanism
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
YT: https://youtu.be/Vbi288CKgis
Michael Levin is a Distinguished Professor in the Biology department at Tufts University, and the holder of the Vannevar Bush endowed Chair. He is the Director of the Allen Discovery Center at Tufts and the Tufts Center for Regenerative and Developmental Biology. His research focuses on understanding the biophysical mechanisms of pattern regulation and harnessing endogenous bioelectric dynamics for rational control of growth and form.
The capacity to generate a complex, behaving organism from the single cell of a fertilized egg is one of the most amazing aspects of biology. Levin' lab integrates approaches from developmental biology, computer science, and cognitive science to investigate the emergence of form and function. Using biophysical and computational modeling approaches, they seek to understand the collective intelligence of cells, as they navigate physiological, transcriptional, morphognetic, and behavioral spaces. They develop conceptual frameworks for basal cognition and diverse intelligence, including synthetic organisms and AI.
Also joining us this evening is Irina Rish. Irina is a Full Professor at the Université de Montréal's Computer Science and Operations Research department, a core member of Mila - Quebec AI Institute, as well as the holder of the Canada CIFAR AI Chair and the Canadian Excellence Research Chair in Autonomous AI. She has a PhD in AI from UC Irvine. Her research focuses on machine learning, neural data analysis, neuroscience-inspired AI, continual lifelong learning, optimization algorithms, sparse modelling, probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis.
Interviewer: Dr. Tim Scarfe
TOC:
[00:00:00] Introduction
[00:02:09] Emergence
[00:13:16] Scaling Laws
[00:23:12] Intelligence
[00:44:36] Transhumanism
Prof. Michael Levin
https://en.wikipedia.org/wiki/Michael_Levin_(biologist)
https://www.drmichaellevin.org/
https://twitter.com/drmichaellevin
Prof. Irina Rish
https://twitter.com/irinarish
https://irina-rish.com/
#95 - Prof. IRINA RISH - AGI, Complex Systems, Transhumanism
Canadian Excellence Research Chair in Autonomous AI. Irina holds an MSc and PhD in AI from the University of California, Irvine as well as an MSc in Applied Mathematics from the Moscow Gubkin Institute. Her research focuses on machine learning, neural data analysis, and neuroscience-inspired AI. In particular, she is exploring continual lifelong learning, optimization algorithms for deep neural networks, sparse modelling and probabilistic inference, dialog generation, biologically plausible reinforcement learning, and dynamical systems approaches to brain imaging analysis. Prof. Rish holds 64 patents, has published over 80 research papers, several book chapters, three edited books, and a monograph on Sparse Modelling. She has served as a Senior Area Chair for NeurIPS and ICML. Irina's research is focussed on taking us closer to the holy grail of Artificial General Intelligence. She continues to push the boundaries of machine learning, continually striving to make advancements in neuroscience-inspired AI.
In a conversation about artificial intelligence (AI), Irina and Tim discussed the idea of transhumanism and the potential for AI to improve human flourishing. Irina suggested that instead of looking at AI as something to be controlled and regulated, people should view it as a tool to augment human capabilities. She argued that attempting to create an AI that is smarter than humans is not the best approach, and that a hybrid of human and AI intelligence is much more beneficial. As an example, she mentioned how technology can be used as an extension of the human mind, to track mental states and improve self-understanding. Ultimately, Irina concluded that transhumanism is about having a symbiotic relationship with technology, which can have a positive effect on both parties.
Tim then discussed the contrasting types of intelligence and how this could lead to something interesting emerging from the combination. He brought up the Trolley Problem and how difficult moral quandaries could be programmed into an AI. Irina then referenced The Garden of Forking Paths, a story which explores the idea of how different paths in life can be taken and how decisions from the past can have an effect on the present.
To better understand AI and intelligence, Irina suggested looking at it from multiple perspectives and understanding the importance of complex systems science in programming and understanding dynamical systems. She discussed the work of Michael Levin, who is looking into reprogramming biological computers with chemical interventions, and Tim mentioned Alex Mordvinsev, who is looking into the self-healing and repair of these systems. Ultimately, Irina argued that the key to understanding AI and intelligence is to recognize the complexity of the systems and to create hybrid models of human and AI intelligence.
Find Irina;
https://mila.quebec/en/person/irina-rish/
https://twitter.com/irinarish
YT version: https://youtu.be/8-ilcF0R7mI
MLST Discord: https://discord.gg/aNPkGUQtc5
References;
The Garden of Forking Paths: Jorge Luis Borges [Jorge Luis Borges]
https://www.amazon.co.uk/Garden-Forking-Paths-Penguin-Modern/dp/0241339057
The Brain from Inside Out [György Buzsáki]
https://www.amazon.co.uk/Brain-Inside-Out-Gy%C3%B6rgy-Buzs%C3%A1ki/dp/0190905387
Growing Isotropic Neural Cellular Automata [Alexander Mordvintsev]
https://arxiv.org/abs/2205.01681
The Extended Mind [Andy Clark and David Chalmers]
https://www.jstor.org/stable/3328150
The Gentle Seduction [Marc Stiegler]
https://www.amazon.co.uk/Gentle-Seduction-Marc-Stiegler/dp/0671698877
#063 - Prof. YOSHUA BENGIO - GFlowNets, Consciousness & Causality
We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST
Patreon: https://www.patreon.com/mlst
For Yoshua Bengio, GFlowNets are the most exciting thing on the horizon of Machine Learning today. He believes they can solve previously intractable problems and hold the key to unlocking machine abstract reasoning itself. This discussion explores the promise of GFlowNets and the personal journey Prof. Bengio traveled to reach them.
Panel:
Dr. Tim Scarfe
Dr. Keith Duggar
Dr. Yannic Kilcher
Our special thanks to:
- Alexander Mattick (Zickzack)
References:
Yoshua Bengio @ MILA (https://mila.quebec/en/person/bengio-yoshua/)
GFlowNet Foundations (https://arxiv.org/pdf/2111.09266.pdf)
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation (https://arxiv.org/pdf/2106.04399.pdf)
Interpolation Consistency Training for Semi-Supervised Learning (https://arxiv.org/pdf/1903.03825.pdf)
Towards Causal Representation Learning (https://arxiv.org/pdf/2102.11107.pdf)
Causal inference using invariant prediction: identification and confidence intervals (https://arxiv.org/pdf/1501.01332.pdf)
Yoshua Bengio and Iulian Serban on AI for Education
Turing award winner Yoshua Bengio and his colleague Julian Serban, co-founder of Korbit, an AI ed-tech startup with the aim of democratizing education, talk about enhancing education through the application of deep learning systems that can track student behavior predict their performance and deliver strategies to both improve performance and prevent students from losing interest.
Episode 35 - Irina Rish
COVID-19 has swept across the world was startling speed, but with equally startling speed, the machine learning community has responded. This week I speak with Irina Rish, a professor at the University of Montreal and a Mila academic member, who is helping head a task force to understand the virus. She talked about where the efforts currently stand and where they expect to go in the weeks and months ahead.
Let me know when it's live.
Consciousness and COVID-19 with Yoshua Bengio - #361
Today we’re joined by one of, if not the most cited computer scientist in the world, Yoshua Bengio, Professor at the University of Montreal and the Founder and Scientific Director of MILA. We caught up with Yoshua to explore his work on consciousness, including how Yoshua defines consciousness, his paper “The Consciousness Prior,” as well as his current endeavor in building a COVID-19 tracing application, and the use of ML to propose experimental candidate drugs.
Episode 12 - Samy Bengio and Yoshua Bengio
This week I talk to the Bengio brothers, Samy and Yoshua, in their first interview together. Yoshua recently won the Turing Award with Geoff Hinton and Yann Lecun, while Samy leads a team of researchers at Google Brain. The brothers are well known to people who work in machine learning, but few know how intertwined their professional lives have been. They talked about their unconventional parents and their early collaboration on neural network research, as well as what they see as the challenges ahead.
Yoshua Bengio: Deep Learning
Yoshua Bengio, along with Geoffrey Hinton and Yann Lecun, is considered one of the three people most responsible for the advancement of deep learning during the 1990s, 2000s, and now. Cited 139,000 times, he has been integral to some of the biggest breakthroughs in AI over the past 3 decades. Video version is available on YouTube. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, or YouTube where you can watch the video versions of these conversations.
