“While language models may help generate new ideas, they cannot attack the hard part of science, which is simulating the necessary physics,” says AI professor Anima Anandkumar. She explains how her team developed neural operators — AI trained on the finest details of the real world — to bridge this gap, sharing recent projects ranging from improved weather forecasting to cutting-edge medical device design that demonstrate the power of AI with universal physical understanding.
Hosted on Acast. See acast.com/privacy for more information.
These two scientists have mapped out the insides or “reachable space” of a language model using control theory, what they discovered was extremely surprising.
Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening.
https://patreon.com/mlst
YT version: https://youtu.be/Bpgloy1dDn0
Aman Bhargava from Caltech and Cameron Witkowski from the University of Toronto to discuss their groundbreaking paper, “What’s the Magic Word? A Control Theory of LLM Prompting.” (the main theorem on self-attention controllability was developed in collaboration with Dr. Shi-Zhuo Looi from Caltech).
They frame LLM systems as discrete stochastic dynamical systems. This means they look at LLMs in a structured way, similar to how we analyze control systems in engineering. They explore the “reachable set” of outputs for an LLM. Essentially, this is the range of possible outputs the model can generate from a given starting point when influenced by different prompts. The research highlights that prompt engineering, or optimizing the input tokens, can significantly influence LLM outputs. They show that even short prompts can drastically alter the likelihood of specific outputs. Aman and Cameron’s work might be a boon for understanding and improving LLMs. They suggest that a deeper exploration of control theory concepts could lead to more reliable and capable language models.
We dropped an additional, more technical video on the research on our Twitter account here: https://x.com/MLStreetTalk/status/1795093759471890606
Additional 20 minutes of unreleased footage on our Patreon here: https://www.patreon.com/posts/whats-magic-word-104922629
What's the Magic Word? A Control Theory of LLM Prompting (Aman Bhargava, Cameron Witkowski, Manav Shah, Matt Thomson)
https://arxiv.org/abs/2310.04444
LLM Control Theory Seminar (April 2024)
https://www.youtube.com/watch?v=9QtS9sVBFM0
Society for the pursuit of AGI (Cameron founded it)
https://agisociety.mydurable.com/
Roger Federer demo
http://conway.languagegame.io/inference
Neural Cellular Automata, Active Inference, and the Mystery of Biological Computation (Aman)
https://aman-bhargava.com/ai/neuro/neuromorphic/2024/03/25/nca-do-active-inference.html
Aman and Cameron also want to thank Dr. Shi-Zhuo Looi and Prof. Matt Thomson from from Caltech for help and advice on their research. (https://thomsonlab.caltech.edu/ and https://pma.caltech.edu/people/looi-shi-zhuo)
https://x.com/ABhargava2000
https://x.com/witkowski_cam
Generative AI-based models can not only learn and understand natural languages — they can learn the very language of nature itself, presenting new possibilities for scientific research.
Anima Anandkumar, Bren Professor at Caltech and senior director of AI research at NVIDIA, was recently invited to speak at the President’s Council of Advisors on Science and Technology.
At the talk, Anandkumar says that generative AI was described as “an inflection point in our lives,” with discussions swirling around how to “harness it to benefit society and humanity through scientific applications.”
On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with Anandkumar on generative AI’s potential to make splashes in the scientific community.
It can, for example, be fed DNA, RNA, viral and bacterial data to craft a model that understands the language of genomes. That model can help predict dangerous coronavirus variants to accelerate drug and vaccine research.
Generative AI can also predict extreme weather events like hurricanes or heat waves. Even with an AI boost, trying to predict natural events is challenging because of the sheer number of variables and unknowns.
However, Anandkumar explains that it’s not just a matter of upsizing language models or adding compute power — it’s also about fine-tuning and setting the right parameters.
“Those are the aspects we’re working on at NVIDIA and Caltech, in collaboration with many other organizations, to say, ‘How do we capture the multitude of scales present in the natural world?’” she said. “With the limited data we have, can we hope to extrapolate to finer scales? Can we hope to embed the right constraints and come up with physically valid predictions that make a big impact?”
Anandkumar adds that to ensure AI models are responsibly and safely used, existing laws must be strengthened to prevent dangerous downstream applications.
She also talks about the AI boom, which is transforming the role of humans across industries, and problems yet to be solved.
“This is the research advice I give to everyone: the most important thing is the question, not the answer,” she said.
Modern life runs on wireless technology. What if the energy powering our devices could also be transmitted without wires? Electrical engineer Ali Hajimiri explains the principles behind wireless energy transfer and shares his far-out vision for launching flexible solar panels into space in order to collect sunlight, convert it to electrical power and then beam it down to Earth. Learn how this technology could power everything -- and light up our world from space.
Hosted on Acast. See acast.com/privacy for more information.
Today we’re joined by Anima Anandkumar, Bren Professor of Computing And Mathematical Sciences at Caltech and Sr Director of AI Research at NVIDIA. In our conversation, we take a broad look at the emerging field of AI for Science, focusing on both practical applications and longer-term research areas. We discuss the latest developments in the area of protein folding, and how much it has evolved since we first discussed it on the podcast in 2018, the impact of generative models and stable diffusion on the space, and the application of neural operators. We also explore the ways in which prediction models like weather models could be improved, how foundation models are helping to drive innovation, and finally, we dig into MineDojo, a new framework built on the popular Minecraft game for embodied agent research, which won a 2022 Outstanding Paper Award at NeurIPS.
The complete show notes for this episode can be found at twimlai.com/go/614
Jed Buchwald is a historian and philosopher of science at Caltech. Please support this podcast by checking out our sponsors:
– GiveWell: https://www.givewell.org/ and use code LEX to get donation matched up to $1k
– Theragun: https://therabody.com/lex to get 30 day trial
– LMNT: https://drinkLMNT.com/lex to get free sample pack
– Fundrise: https://fundrise.com/lex
– BetterHelp: https://betterhelp.com/lex to get 10% off
EPISODE LINKS:
Jed’s Caltech page: https://bit.ly/38eLLRF
Jed’s Books: https://amzn.to/2WoxGPi
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(06:34) – How does science progress?
(22:48) – Theory of Everything
(34:40) – Consciousness
(38:15) – Most Beautiful Moments in Science
(46:00) – Isaac Newton
(1:12:13) – Competition in Science
(1:22:47) – Newton’s Career
(1:35:58) – Importance of Data
(1:42:17) – Alchemy
(1:46:31) – Newton and Religion
(1:49:44) – Showing Newton the future
(1:54:28) – Newton and Einstein
Barry Barish is a theoretical physicist at Caltech and the winner of the Nobel Prize in Physics. Please support this podcast by checking out our sponsors:
– MUD\WTR: https://mudwtr.com/lex and use code LEX to get 5% off
– GiveDirectly: https://givedirectly.org/lex to get gift matched up to $300
– BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off
– Four Sigmatic: https://foursigmatic.com/lex and use code LexPod to get up to 60% off
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
EPISODE LINKS:
Barry’s Nobel Prize entry: https://www.nobelprize.org/prizes/physics/2017/barish/facts/
Barry’s Caltech profile: https://pma.caltech.edu/people/barry-c-barish
LIGO’s Website: https://www.ligo.caltech.edu/
LIGO’s Twitter: https://twitter.com/LIGO
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(08:27) – Early Math and Physics questions
(18:02) – Enrico Fermi
(24:34) – Birth of the Nuclear Age
(29:42) – The Fermi Paradox
(34:45) – Gravity
(51:28) – Philosophical Implications of General Relativity
(58:34) – Detecting Gravitational Waves
(1:01:47) – LIGO
(1:34:45) – Nobel Prize
(1:49:34) – Black Holes
(2:01:53) – Space Exploration
(2:09:48) – Books
(2:18:37) – Advice for young people
(2:24:33) – Meaning of life
Konstantin Batygin is a planetary astrophysicist at Caltech. Please support this podcast by checking out our sponsors:
– Squarespace: https://lexfridman.com/squarespace and use code LEX to get 10% off
– Literati: https://literati.com/lex
– Onnit: https://lexfridman.com/onnit to get up to 10% off
– National Instruments (NI): https://www.ni.com/perspectives
EPISODE LINKS:
Konstantin’s Twitter https://twitter.com/kbatygin
Konstantin’s Website https://www.konstantinbatygin.com/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:17) – Overview of our Solar System
(22:14) – What is the Oort Cloud?
(27:10) – Life in the interstellar medium
(28:42) – Are there aliens out there?
(31:22) – How unique is Earth?
(34:02) – Did Jupiter destroy early planets?
(40:16) – How hard is it to simulate the Universe?
(44:49) – Quantum mechanics in evolution of objects in the Solar system
(49:15) – Simulating the first formations around the Sun
(55:02) – Will it be possible to simulate the full history of the Solar System?
(57:23) – How far should we go with the simulation?
(59:43) – Increasing immersion in video games
(1:06:09) – What is Planet Nine?
(1:12:37) – The origin of life
(1:15:02) – Evidence of Planet Nine
(1:17:32) – Discovery of Neptune
(1:18:42) – When will we find Planet Nine?
(1:21:21) – Planet Nine throws rocks into the Kuiper Belt
(1:25:15) – Could Planet Nine be a primordial black hole?
(1:35:20) – Commercial space revolution boosts science and the human condition
(1:42:46) – Solving sex in space
(1:43:24) – Would humans evolve if we couldn’t see the stars?
(1:49:08) – Military funding and science
(1:53:11) – Is Oumuamua space junk from a distant alien civilization?
(2:06:33) – Wild ideas create the future
(2:14:22) – The perfect place to die
(2:16:03) – Greatest song of all time
(2:22:34) – Music enables science for Konstantin
(2:24:51) – Music practice tips for busy people
(2:28:41) – Memories of 1990s Russia
(2:35:14) – Advice for young people
(2:41:10) – Meaning of life
Anima Anandkumar joins us to discuss her work as a researcher in machine learning at NVIDIA and a professor at CalTech, and how they often go hand-in-hand and inform each other.
In this episode you will learn:
Anima’s recent discovery of yoga [5:20]
How does Anima balance her work? [12:25]
Applications of Anima’s work [14:45]
Tensors [22:55]
Anima’s favorite NVIDIA projects [35:35]
What tools does NVIDIA use? [41:55]
CalTech interdisciplinary science [47:41]
The path to generalized artificial intelligence [57:19]
The skills to have to get into this field [1:00:27]
LinkedIn questions for Anima [1:07:03]
Additional materials: www.superdatascience.com/473
Katherine de Kleer is a professor of Planetary Science and Astronomy at Caltech. Please support this podcast by checking out our sponsors:
– Fundrise: https://fundrise.com/lex
– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium
– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
EPISODE LINKS:
Katherine’s Website: http://web.gps.caltech.edu/~dekleer/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:07) – Pluto
(12:14) – Kuiper belt
(16:12) – How to study planets and moons
(19:54) – Volcanoes on Io – moon of Jupiter
(32:25) – Is there life in the oceans of Europa?
(41:46) – How unlikely is life on Earth?
(52:15) – Life on Venus
(54:30) – Mars
(1:01:17) – What is interesting about Earth as a planet?
(1:11:55) – Weather patterns
(1:17:04) – Asteroids
(1:26:06) – Will an asteroid hit Earth soon?
(1:34:50) – Oumuamua
(1:50:00) – Book recommendations
(1:56:37) – Advice for young people
Anima Anandkumar is setting a personal record this week with seven of her team’s research papers accepted to NeurIPS 2020.
The 34th annual Neural Information Processing Systems conference is taking place virtually from Dec. 6-12. The premier event on neural networks, NeurIPS draws thousands of the world’s best researchers every year.
Anandkumar, NVIDIA’s director of machine learning research and Bren professor at CalTech’s CMS Department, joined AI Podcast host Noah Kravitz to talk about what to expect at the conference, and to explain what she sees as the future of AI.
https://blogs.nvidia.com/blog/2020/12/09/neurips-nvidia-caltech-anima-anandkumar/
Pietro Perona, a professor at the California Institute of Technology, is one of the brains behind a pair of smartphone apps that help you do just that: eBird and iNaturalist. But simple as the apps are to use in identifying tens of thousands of species, the science behind them is complex. Gathering enough examples for each species to train a neural network is impossible, so much of Prof. Perona's work has been focused on making machines more efficient learners, requiring less training data.
While at the NVIDIA GPU Technology Conference 2019 in Silicon Valley, Chris enjoyed an inspiring conversation with Anima Anandkumar. Clearly a role model - not only for women - but for anyone in the world of AI, Anima relayed how her lifelong passion for mathematics and engineering started when she was only 3 years old in India, and ultimately led to her pioneering deep learning research at Amazon Web Services, CalTech, and NVIDIA.
Sponsors:
DigitalOcean – Check out DigitalOcean’s dedicated vCPU Droplets with dedicated vCPU threads. Get started for free with a $100 credit. Learn more at do.co/changelog.
DataEngPodcast – A podcast about data engineering and modern data infrastructure.
Rollbar – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog.
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Featuring:
Anima Anandkumar – Website, LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Show Notes:
Anima Anandkumar on Wikipedia
NVIDIA GTC Speaker
NVIDIA GTC Presentation: Role of Tensors in Machine Learning
NVIDIA GTC Presentation: Infusing Physics into Deep Learning Algorithms with Applications to Stable Landing of Drones
CalTech CMS TensorLab
Google Scholar
Upcoming Events:
Register for upcoming webinars here!
In the inaugural TWiML Live, Sam Charrington is joined by Amanda Askell (OpenAI), Anima Anandkumar (NVIDIA/CalTech), Miles Brundage (OpenAI), Robert Munro (Lilt), and Stephen Merity to discuss the controversial recent release of the OpenAI GPT-2 Language Model.
We cover the basics like what language models are and why they’re important, and why this announcement caused such a stir, and dig deep into why the lack of a full release of the model raised concerns for so many.
In this episode of our AI Rewind series, we’re back with Anima Anandkumar, Bren Professor at Caltech and now Director of Machine Learning Research at NVIDIA.
Anima joins us to discuss her take on trends in the broader Machine Learning field in 2018 and beyond. In our conversation, we cover not only technical breakthroughs in the field but also those around inclusivity and diversity.
For this episode's complete show notes, visit twimlai.com/talk/215.
In this episode of our TrainAI series, I sit down with Anima Anandkumar, Bren Professor at Caltech and Principal Scientist with Amazon Web Services. Anima joined me to discuss the research coming out of her “Tensorlab” at CalTech. In our conversation, we review the application of tensor operations to machine learning and discuss how an example problem–document categorization–might be approached using 3 dimensional tensors to discover topics and relationships between topics. We touch on multidimensionality, expectation maximization, and Amazon products Sagemaker and Comprehend. Anima also goes into how to tensorize neural networks and apply our understanding of tensor algebra to do perform better architecture searches. The notes for this show can be found at twimlai.com/talk/142. For series info, visit twimlai.com/trainai2018
This week on the podcast we’re featuring a series of conversations from the AWS re:Invent conference in Las Vegas. I had a great time at this event getting caught up on the latest and greatest machine learning and AI products and services announced by AWS and its partners. Today we’re joined by Aaron Ames, Professor of Mechanical & Civil Engineering at Caltech. Aaron joined me before his talk at the Deep Learning Summit “Eye, Robot: Computer Vision and Autonomous Robotics” and I had a ton of questions for him. While he considers himself a “hardware guy”, we got into a great discussion centered around the intersection of Robotics and ML Inference. We cover a range of topics, including Boston Dynamics backflipping robot (If you haven't seen it, check out the show notes), Humanoid Robotics, His work on motion primitives and transitions and he even gives us a few predictions on the future of robotics.
