Google DeepMind's Vision for AI, Search and Gemini with Oriol Vinyals from Google DeepMind
In this episode of No Priors, hosts Sarah and Elad are joined by Oriol Vinyals, VP of Research, Deep Learning Team Lead, at Google DeepMind and Technical Co-lead of the Gemini project. Oriol shares insights from his career in machine learning, including leading the AlphaStar team and building competitive StarCraft agents. We talk about Google DeepMind, forming the Gemini project, and integrating AI technology throughout Google products. Oriol also discusses the advancements and challenges in long context LLMs, reasoning capabilities of models, and the future direction of AI research and applications. The episode concludes with a reflection on AGI timelines, the importance of specialized research, and advice for future generations in navigating the evolving landscape of AI.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @oriolvinyalsml
Show Notes:
(00:00) Introduction to Oriol Vinyals
(00:55) The Gemini Project and Its Impact
(02:04) AI in Google Search and Chat Models
(08:29) Infinite Context Length and Its Applications
(14:42) Scaling AI and Reward Functions
(31:55) The Future of General Models and Specialization
(38:14) Reflections on AGI and Personal Insights
(43:09) Will the Next Generation Study Computer Science?
(45:37) Closing thoughts
#306 – Oriol Vinyals: Deep Learning and Artificial General Intelligence
Oriol Vinyals is the Research Director and Deep Learning Lead at DeepMind. Please support this podcast by checking out our sponsors:
– Shopify: https://shopify.com/lex to get 14-day free trial
– Weights & Biases: https://lexfridman.com/wnb
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium
EPISODE LINKS:
Oriol’s Twitter: https://twitter.com/oriolvinyalsml
Oriol’s publications: https://scholar.google.com/citations?user=NkzyCvUAAAAJ
DeepMind’s Twitter: https://twitter.com/DeepMind
DeepMind’s Instagram: https://instagram.com/deepmind
DeepMind’s Website: https://deepmind.com
Papers:
1. Gato: https://deepmind.com/publications/a-generalist-agent
2. Flamingo: https://deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model
3. Language Models are Few-Shot Learners: https://arxiv.org/abs/2005.14165
4. Emergent Abilities of Large Language Models: https://arxiv.org/abs/2206.07682
5. Attention Is All You Need: https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(05:18) – AI
(20:14) – Weights
(26:33) – Gato
(1:01:22) – Meta learning
(1:15:21) – Neural networks
(1:37:46) – Emergence
(1:44:30) – AI sentience
(2:08:27) – AGI
Oriol Vinyals
Oriol Vinyals, who leads DeepMind's deep learning team, talks about AlphaCode, his group's code-writing language model, and DeepMind's winding road toward artificial general intelligence.
Deep Learning, Transformers, and the Consequences of Scale with Oriol Vinyals - #546
Today we’re excited to kick off our annual NeurIPS, joined by Oriol Vinyals, the lead of the deep learning team at Deepmind. We cover a lot of ground in our conversation with Oriol, beginning with a look at his research agenda and why the scope has remained wide even through the maturity of the field, his thoughts on transformer models and if they will get us beyond the current state of DL, or if some other model architecture would be more advantageous. We also touch on his thoughts on the large language models craze, before jumping into his recent paper StarCraft II Unplugged: Large Scale Offline Reinforcement Learning, a follow up to their popular AlphaStar work from a few years ago. Finally, we discuss the degree to which the work that Deepmind and others are doing around games actually translates into real-world, non-game scenarios, recent work on multimodal few-shot learning, and we close with a discussion of the consequences of the level of scale that we’ve achieved thus far.
The complete show notes for this episode can be found at twimlai.com/go/546
Oriol Vinyals: DeepMind AlphaStar, StarCraft, Language, and Sequences
Oriol Vinyals is a senior research scientist at Google DeepMind. Before that he was at Google Brain and Berkeley. His research has been cited over 39,000 times. He is one of the most brilliant and impactful minds in the field of deep learning. He is behind some of the biggest papers and ideas in AI, including sequence to sequence learning, audio generation, image captioning, neural machine translation, and reinforcement learning. He is a co-lead (with David Silver) of the AlphaStar project, creating an agent that defeated a top professional at the game of StarCraft. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations.