Why Can't AI Make Its Own Discoveries? — With Yann LeCun
Yann LeCun is the chief AI scientist at Meta. He joins Big Technology Podcast to discuss the strengths and limitations of current AI models, weighing in on why they've been unable to invent new things despite possessing almost all the world's written knowledge. LeCun digs deep into AI science, explaining why AI systems must build an abstract knowledge of the way the world operates to truly advance. We also cover whether AI research will hit a wall, whether investors in AI will be disappointed, and the value of open source after DeepSeek. Tune in for a fascinating conversation with one of the world's leading AI pioneers.
---
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/
Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology
Questions? Feedback? Write to: bigtechnologypodcast@gmail.com
Learn more about your ad choices. Visit megaphone.fm/adchoices
Meta's Chief AI Scientist Yann LeCun Makes the Case for Open Source | On With Kara Swisher
We're bringing you a special episode of On With Kara Swisher! Kara sits down for a live interview with Meta's Yann LeCun, an “early AI prophet” and the brains behind the largest open-source large language model in the world. The two discuss the potential dangers that come with open-source models, the massive amounts of money pouring into AI research, and the pros and cons of AI regulation. They also dive into LeCun’s surprisingly spicy social media feeds — unlike a lot of tech employees who toe the HR line, LeCun isn’t afraid to say what he thinks of Elon Musk or President-elect Donald Trump.
This interview was recorded live at the Johns Hopkins University Bloomberg Center in Washington, DC as part of their Discovery Series.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Meta's Chief AI Scientist Yann LeCun Makes the Case for Open Source
Kara sits down for a live interview with Yann LeCun, an “early AI prophet” and the brains behind the largest open-source large language model in the world. The two discuss the potential dangers that come with open-source models, the massive amounts of money pouring into AI research, and the pros and cons of AI regulation. They also dive into LeCun’s surprisingly spicy social media feeds — unlike a lot of tech employees who toe the HR line, Yann isn’t afraid to say what he thinks of Elon Musk or President-elect Donald Trump.
This interview was recorded live at the Johns Hopkins University Bloomberg Center in Washington, DC as part of their Discovery Series.
Questions? Comments? Email us at on@voxmedia.com or find us on Instagram and TikTok @onwithkaraswisher
Learn more about your ad choices. Visit podcastchoices.com/adchoices
#416 – Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI
Yann LeCun is the Chief AI Scientist at Meta, professor at NYU, Turing Award winner, and one of the most influential researchers in the history of AI. Please support this podcast by checking out our sponsors:
– HiddenLayer: https://hiddenlayer.com/lex
– LMNT: https://drinkLMNT.com/lex to get free sample pack
– Shopify: https://shopify.com/lex to get $1 per month trial
– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil
Transcript: https://lexfridman.com/yann-lecun-3-transcript
EPISODE LINKS:
Yann’s Twitter: https://twitter.com/ylecun
Yann’s Facebook: https://facebook.com/yann.lecun
Meta AI: https://ai.meta.com/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(09:10) – Limits of LLMs
(20:47) – Bilingualism and thinking
(24:39) – Video prediction
(31:59) – JEPA (Joint-Embedding Predictive Architecture)
(35:08) – JEPA vs LLMs
(44:24) – DINO and I-JEPA
(45:44) – V-JEPA
(51:15) – Hierarchical planning
(57:33) – Autoregressive LLMs
(1:12:59) – AI hallucination
(1:18:23) – Reasoning in AI
(1:35:55) – Reinforcement learning
(1:41:02) – Woke AI
(1:50:41) – Open source
(1:54:19) – AI and ideology
(1:56:50) – Marc Andreesen
(2:04:49) – Llama 3
(2:11:13) – AGI
(2:15:41) – AI doomers
(2:31:31) – Joscha Bach
(2:35:44) – Humanoid robots
(2:44:52) – Hope for the future
20VC: AI's Biggest Questions: The Commoditisation of LLMs, Open vs Closed: Who Wins, Model Size vs Data Quality, Why Google are Vulnerable and Apple are the Dark Horse
Des Traynor is a Co-Founder of Intercom, and has built and led many teams within the company, including Product, Marketing, and Customer Support.
Yann LeCun is VP & Chief AI Scientist at Meta and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of FAIR and of the NYU Center for Data Science.
Emad Mostaque is the Co-Founder and CEO @ StabilityAI, the parent company of Stable Diffusion. Stability are building the foundation to activate humanity's potential.
Jeff Seibert is the Founder & CEO @ Digits, building the future of AI-powered accounting. Digits have raised funding from the likes of Peter Fenton @ Benchmark and 20VC.
Tomasz Tunguz is the Founder and General Partner @ Theory Ventures, just announced last week, Theory is a $230M fund that invests $1-25m in early-stage companies that leverage technology discontinuities into go-to-market advantages.
Douwe Kiela is the CEO of Contextual AI, building the contextual language model to power the future of businesses.
Cris Valenzuela is the CEO and co-founder of Runway, the company that trains and builds generative AI models for content creation.
Richard Socher is the founder and CEO of You.com. Richard previously served as the Chief Scientist and EVP at Salesforce. Before that, Richard was the CEO/CTO of AI startup MetaMind, acquired by Salesforce in 2016.
In Today's Episode We Discuss:
Foundational Models: Analysis
Will foundational models become commoditized?
Who are the major players? What are their different strengths?
Who will win? Who will lose?
How important is the size of the model vs the quality of the data?
2. Open vs Closed:
What are the biggest pros and cons of an open ecosystem for LLMs?
Why is it naive to think that open-source LLMs will prevail?
What will determine which method wins?
3. An Analysis of the Incumbents:
Why is Google the most vulnerable? What can they do to regain ground?
Why is Apple the sleeping giant? How could they win the next wave of AI?
What should Amazon do today to compete with Microsoft?
4. The Future: Doom and Gloom?
Why is it ridiculous to assume AI systems want to dominate?
Why will AI create a renaissance of creativity and human freedom?
What role should regulation play in the advancement and progression of AI?
#150: Yann LeCun on World Models, AI Threats and Open-Sourcing
This episode is sponsored by Oracle. AI is revolutionizing industries, but needs power without breaking the bank. Enter Oracle Cloud Infrastructure (OCI): the one-stop platform for all your AI needs, with 4-8x the bandwidth of other clouds. Train AI models faster and at half the cost. Be ahead like Uber and Cohere.
If you want to do more and spend less like Uber, 8x8, and Databricks Mosaic - take a free test drive of OCI at https://oracle.com/eyeonai
Welcome to episode 150 of the 'Eye on AI' podcast. In this episode, host Craig Smith sits down with Yann LeCun, a Turing Award winner who has been instrumental in advancing convolutional neural networks and whose work spans machine learning, computer vision, and more.
Tune is as Craig and Yann explore the intricacies of AI, world models, and the challenges of continuous learning.
In this episode, Yann delves deep into the concept of a "world model" - systems that can predict the world's future states, allowing agents to make informed decisions. The discussion transitions to the challenges of training these models, particularly when dealing with diverse data like text and images. We then discuss the computational demands of modern AI models, with Yann highlighting the nuances between generative models for videos and language.
He also touches upon the idea of the "Embodied Turing Tests" and how augmented language models can bridge the gap between human-like behavior and computational efficiency.The spotlight then shifts to pressing concerns surrounding the open-source nature of AI models, with Yann articulating the legal ramifications and the future of open-source AI. Drawing from global perspectives, including China's stance on open-source, Yann underscores the imperative for a collaborative approach in the AI space, ensuring it's reflective of diverse global needs.
Craig Smith Twitter: https://twitter.com/craigss
Eye on A.I. Twitter: https://twitter.com/EyeOn_AI
(00:00) Preview, Oracle and Introduction
(02:42) Decoding The World Model and Gaia 1
(07:43) Energy and Computational Demands of AI
(08:06) Video vs. Text Processing & True AI Capabilities
(11:17) Embodied Turing Test & Augmented LLMs
(15:38) Is AI a Threat To Society?
(25:04) Where is AI Development Headed?
(31:06) Interplay of Neuroscience and AI**
(33:33) Yann's Vision, JEPA, and Learning Challenges
(39:05) Yann's Career, AI Progress, and Challenges
(44:47) The Open Source Debate in AI
(55:30) Oracle Cloud Infrastructure
20VC: Who Wins the AI Race; Startups or Incumbents & Does Having Proprietary Data Really Matter For Startups Today?
One of the core questions in AI and investing today; who wins, startups or incumbents? Startups have speed and innovation but incumbents have scale, resources, and distribution?
Today we hear from 6 leading investors and founders discussing where they place their bets who has the advantage; startups or incumbents?
Emad Mostaque is CEO @ StabilityAI, the parent company of Stable Diffusion. To date, Emad has raised over $110M with Stability with the latest round reportedly pricing the company at $4BN.
Yann LeCun is VP & Chief AI Scientist at Meta and Professor at NYU. He was the founding Director of FAIR and of the NYU Center for Data Science.
Clem Delangue is the Co-Founder and CEO @ Hugging Face, the AI community building the future. Clem has raised over $160M from the likes of Sequoia, Coatue, Addition and Lux Capital to name a few.
Sarah Guo is the Founding Partner @ Conviction Capital, a $100M first fund purpose-built to serve "Software 3.0" companies. Prior to founding Conviction, Sarah was a General Partner at Greylock.
Vince Hankes is a Partner @ Thrive Capital where he has led the firm's investments in OpenAI, Melio, and Airplane.dev. Prior to Thrive, Vince learned the craft of venture from Lee Fixel @ Tiger.
Tomasz Tunguz is the Founder and General Partner @ Theory Ventures, a $230M fund that invests $1-25m in companies that leverage technology discontinuities into go-to-market advantages.
The Question of the Day:
Who wins? Startups or Incumbents?
20VC: Yann LeCun on Why Artificial Intelligence Will Not Dominate Humanity, Why No Economists Believe All Jobs Will Be Replaced by AI, Why the Size of Models Matters Less and Less & Why Open Models Beat Closed Models
Yann LeCun is VP & Chief AI Scientist at Meta and Silver Professor at NYU affiliated with the Courant Institute of Mathematical Sciences & the Center for Data Science. He was the founding Director of FAIR and of the NYU Center for Data Science. After a postdoc in Toronto he joined AT&T Bell Labs in 1988, and AT&T Labs in 1996 as Head of Image Processing Research. He joined NYU as a professor in 2003 and Meta/Facebook in 2013. He is the recipient of the 2018 ACM Turing Award for "conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing". Huge thanks to David Marcus for helping to make this happen.
In Today's Episode with Yann LeCun:
1.) The Road to AI OG:
How did Yann first hear about machine learning and make his foray into the world of AI?
For 10 years plus, machine learning was in the shadows, how did Yan not get discouraged when the world did not appreciate the power of AI and ML?
What does Yann know now that he wishes he had known when he started his career in machine learning?
2.) The Next Five Years of AI: Hope or Horror:
Why does Yann believe it is nonsense that AI is dangerous?
Why does Yann think it is crazy to assume that AI will even want to dominate humans?
Why does Yann believe digital assistants will rule the world?
If digital assistants do rule the world, what interface wins? Search? Chat? What happens to Google when digital assistants rule the world?
3.) Will Anyone Have Jobs in a World of AI:
From speaking to many economists, why does Yann state "no economist thinks AI will replace jobs"?
What jobs does Yann expect to be created in the next generation of the AI economy?
What jobs does Yann believe are under more immediate threat/impact?
Why does Yann expect the speed of transition to be much slower than people anticipate?
Why does Yann believe Elon Musk is wrong to ask for the pausing of AI developments?
4.) Open or Closed: Who Wins:
Why does Yann know that the open model will beat the closed model?
Why is it superior for knowledge gathering and idea generation?
What are some core historical precedents that have proved this to be true?
What did Yann make of the leaked Google Memo last week?
5.) Startup vs Incumbent: Who Wins:
Who does Yann believe will win the next 5 years of AI; startups or incumbents?
How important are large models to winning in the next 12 months?
In what ways does regulation and legal stop incumbents? How has he seen this at Meta?
Has his role at Meta ever stopped him from being impartial? How does Yan deal with that?
Yann LeCun: Filling the Gap in Large Language Models
In this episode, Yann LeCun, a renowned computer scientist and AI researcher, shares his insights on the limitations of large language models and how his new joint embedding predictive architecture could help bridge the gap. While large language models have made remarkable strides in natural language processing and understanding, they are still far from perfect. Yann LeCun points out that these models often cannot capture the nuances and complexities of language, leading to inaccuracies and errors. To address this gap, Yann LeCun introduces his new joint embedding predictive architecture - a novel approach to language modelling that combines techniques from computer vision and natural language processing. This approach involves jointly embedding text and images, allowing for more accurate predictions and a better understanding of the relationships between original concepts and objects. Craig Smith Twitter: https://twitter.com/craigssEye on A.I. Twitter: https://twitter.com/EyeOn_AI
Is ChatGPT A Step Toward Human-Level AI? — With Yann LeCun
Yann LeCun is the chief AI scientist at Meta, a professor of computer science at NYU, and a pioneer of deep learning. He joins Big Technology Podcast to put Generative AI in context, discussing whether ChatGPT and the like are a step toward human-level artificial intelligence, or something completely different. Join us for a fun, substantive discussion about this technology, the makeup of OpenAI, and where the field heads next. Stay tuned for the second half, where we discuss the ethics of using others' work to train AI models.
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/
Questions? Feedback? Write to: bigtechnologypodcast@gmail.com
Learn more about your ad choices. Visit megaphone.fm/adchoices
#86 - Prof. YANN LECUN and Dr. RANDALL BALESTRIERO - SSL, Data Augmentation, Reward isn't enough [NEURIPS2022]
Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning".
Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications.
Episode recorded live at NeurIPS.
YT: https://youtu.be/9dLd6n9yT8U (references are there)
Support us! https://www.patreon.com/mlst
Host: Dr. Tim Scarfe
TOC:
[00:00:00] LeCun interview
[00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)
#258 – Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning
Yann LeCun is the Chief AI Scientist at Meta, professor at NYU, Turing Award winner, and one of the seminal researchers in the history of machine learning. Please support this podcast by checking out our sponsors:
– Public Goods: https://publicgoods.com/lex and use code LEX to get $15 off
– Indeed: https://indeed.com/lex to get $75 credit
– ROKA: https://roka.com/ and use code LEX to get 20% off your first order
– NetSuite: http://netsuite.com/lex to get free product tour
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
EPISODE LINKS:
Yann’s Twitter: https://twitter.com/ylecun
Yann’s Facebook: https://www.facebook.com/yann.lecun
Yann’s Website: http://yann.lecun.com/
Books and resources mentioned:
Self-supervised learning (article): https://bit.ly/3Aau1DQ
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(06:58) – Self-supervised learning
(17:17) – Vision vs language
(23:08) – Statistics
(28:55) – Three challenges of machine learning
(34:45) – Chess
(42:47) – Animals and intelligence
(52:31) – Data augmentation
(1:13:51) – Multimodal learning
(1:25:40) – Consciousness
(1:30:25) – Intrinsic vs learned ideas
(1:34:37) – Fear of death
(1:42:29) – Artificial Intelligence
(1:56:18) – Facebook AI Research
(2:12:56) – NeurIPS
(2:29:08) – Complexity
(2:37:33) – Music
(2:42:28) – Advice for young people
061: Interpolation, Extrapolation and Linearisation (Prof. Yann LeCun, Dr. Randall Balestriero)
We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST
Patreon: https://www.patreon.com/mlst
Yann LeCun thinks that it's specious to say neural network models are interpolating because in high dimensions, everything is extrapolation. Recently Dr. Randall Balestriero, Dr. Jerome Pesente and prof. Yann LeCun released their paper learning in high dimensions always amounts to extrapolation. This discussion has completely changed how we think about neural networks and their behaviour.
[00:00:00] Pre-intro
[00:11:58] Intro Part 1: On linearisation in NNs
[00:28:17] Intro Part 2: On interpolation in NNs
[00:47:45] Intro Part 3: On the curse
[00:48:19] LeCun
[01:40:51] Randall B
YouTube version: https://youtu.be/86ib0sfdFtw
Getting AI To Think And Learn Like Humans — With Daniel Kahneman and Yann LeCun
Daniel Kahneman is a Nobel prize-winning psychologist and economist and author of Thinking, Fast and Slow, a landmark book that decodes human decision-making. Yann LeCun is the chief AI scientist at Meta (Facebook) and a pioneer in the field of deep learning, which the cutting edge of AI is based on today. The two come together on Big Technology Podcast this week to discuss how machines and humans learn, whether there are parallels, and what each field can learn from each other.
Learn more about your ad choices. Visit megaphone.fm/adchoices
Yann LeCun explains why Facebook would crumble without AI
On episode three of The Robot Brains Podcast we have deep learning pioneer: Yann LeCun. Yann is a winner of the Turing Award (often called the Nobel Prize of Computer Science) who in 2013 was handpicked by Mark Zuckerberg to bring AI to Facebook. Yann also offers his predictions for the future of artificial general intelligence, talks about his life straddling the worlds of academia and business and explains why he likes to picture AI as a chocolate layer cake with a cherry on top. Host: Pieter Abbeel Executive Producers: Ricardo Reyes & Henry Tobias Jones Audio Production: Kieron Matthew Banerj Hosted on Acast. See acast.com/privacy for more information.
Yann LeCun: Deep Learning, Convolutional Neural Networks, and Self-Supervised Learning
Yann LeCun is one of the fathers of deep learning, the recent revolution in AI that has captivated the world with the possibility of what machines can learn from data. He is a professor at New York University, a Vice President & Chief AI Scientist at Facebook, co-recipient of the Turing Award for his work on deep learning. He is probably best known as the founder of convolutional neural networks, in particular their early application to optical character recognition. This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on iTunes or support it on Patreon.
Episode 17 - Yann Lecun
This week I talk to Yann Lecun, one of the brightest minds in machine learning today. Yann's work lies behind some of the most critical AI applications, most notably computer vision systems that power everything from face recognition software to self-driving cars. He recently won the Turing Award, the highest prize in computer science. We talked about Yann's first computer, about how music led him into computer science, and about his work on self-supervised learning, which he believes will take us to human-level intelligence in machines.
.jpg?width=400){kind=small}
.jpg?width=400){kind=small}
