🔬 “We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing
<p>A few years ago, Caltech Prof. and co-founder of Accelerated Understanding, Anima Anandkumar set out to develop the first open-source weather model with AI. Talking to experts in the field, she was met with skepticism. Weather is chaotic, physics simul...</p><p><a href="https://lspace.beehiiv.com/podcast/s/latent_space_the_ai_engineer_podcast/we_have_foundation_models_for_language_not_for_physics_anima_anandkumar_bren_professor_of_computing">Full show notes and links</a></p>

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research
AI that connects the digital and physical worlds | Anima Anandkumar
“While language models may help generate new ideas, they cannot attack the hard part of science, which is simulating the necessary physics,” says AI professor Anima Anandkumar. She explains how her team developed neural operators — AI trained on the finest details of the real world — to bridge this gap, sharing recent projects ranging from improved weather forecasting to cutting-edge medical device design that demonstrate the power of AI with universal physical understanding.
Hosted on Acast. See acast.com/privacy for more information.

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research
What’s the Magic Word? A Control Theory of LLM Prompting.
These two scientists have mapped out the insides or “reachable space” of a language model using control theory, what they discovered was extremely surprising.
Please support us on Patreon to get access to the private Discord server, bi-weekly calls, early access and ad-free listening.
https://patreon.com/mlst
YT version: https://youtu.be/Bpgloy1dDn0
Aman Bhargava from Caltech and Cameron Witkowski from the University of Toronto to discuss their groundbreaking paper, “What’s the Magic Word? A Control Theory of LLM Prompting.” (the main theorem on self-attention controllability was developed in collaboration with Dr. Shi-Zhuo Looi from Caltech).
They frame LLM systems as discrete stochastic dynamical systems. This means they look at LLMs in a structured way, similar to how we analyze control systems in engineering. They explore the “reachable set” of outputs for an LLM. Essentially, this is the range of possible outputs the model can generate from a given starting point when influenced by different prompts. The research highlights that prompt engineering, or optimizing the input tokens, can significantly influence LLM outputs. They show that even short prompts can drastically alter the likelihood of specific outputs. Aman and Cameron’s work might be a boon for understanding and improving LLMs. They suggest that a deeper exploration of control theory concepts could lead to more reliable and capable language models.
We dropped an additional, more technical video on the research on our Twitter account here: https://x.com/MLStreetTalk/status/1795093759471890606
Additional 20 minutes of unreleased footage on our Patreon here: https://www.patreon.com/posts/whats-magic-word-104922629
What's the Magic Word? A Control Theory of LLM Prompting (Aman Bhargava, Cameron Witkowski, Manav Shah, Matt Thomson)
https://arxiv.org/abs/2310.04444
LLM Control Theory Seminar (April 2024)
https://www.youtube.com/watch?v=9QtS9sVBFM0
Society for the pursuit of AGI (Cameron founded it)
https://agisociety.mydurable.com/
Roger Federer demo
http://conway.languagegame.io/inference
Neural Cellular Automata, Active Inference, and the Mystery of Biological Computation (Aman)
https://aman-bhargava.com/ai/neuro/neuromorphic/2024/03/25/nca-do-active-inference.html
Aman and Cameron also want to thank Dr. Shi-Zhuo Looi and Prof. Matt Thomson from from Caltech for help and advice on their research. (https://thomsonlab.caltech.edu/ and https://pma.caltech.edu/people/looi-shi-zhuo)
https://x.com/ABhargava2000
https://x.com/witkowski_cam
Anima Anandkumar on Using Generative AI to Tackle Global Challenges - Ep. 203
Generative AI-based models can not only learn and understand natural languages — they can learn the very language of nature itself, presenting new possibilities for scientific research.
Anima Anandkumar, Bren Professor at Caltech and senior director of AI research at NVIDIA, was recently invited to speak at the President’s Council of Advisors on Science and Technology.
At the talk, Anandkumar says that generative AI was described as “an inflection point in our lives,” with discussions swirling around how to “harness it to benefit society and humanity through scientific applications.”
On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with Anandkumar on generative AI’s potential to make splashes in the scientific community.
It can, for example, be fed DNA, RNA, viral and bacterial data to craft a model that understands the language of genomes. That model can help predict dangerous coronavirus variants to accelerate drug and vaccine research.
Generative AI can also predict extreme weather events like hurricanes or heat waves. Even with an AI boost, trying to predict natural events is challenging because of the sheer number of variables and unknowns.
However, Anandkumar explains that it’s not just a matter of upsizing language models or adding compute power — it’s also about fine-tuning and setting the right parameters.
“Those are the aspects we’re working on at NVIDIA and Caltech, in collaboration with many other organizations, to say, ‘How do we capture the multitude of scales present in the natural world?’” she said. “With the limited data we have, can we hope to extrapolate to finer scales? Can we hope to embed the right constraints and come up with physically valid predictions that make a big impact?”
Anandkumar adds that to ensure AI models are responsibly and safely used, existing laws must be strengthened to prevent dangerous downstream applications.
She also talks about the AI boom, which is transforming the role of humans across industries, and problems yet to be solved.
“This is the research advice I give to everyone: the most important thing is the question, not the answer,” she said.

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research
How wireless energy from space could power everything | Ali Hajimiri
Modern life runs on wireless technology. What if the energy powering our devices could also be transmitted without wires? Electrical engineer Ali Hajimiri explains the principles behind wireless energy transfer and shares his far-out vision for launching flexible solar panels into space in order to collect sunlight, convert it to electrical power and then beam it down to Earth. Learn how this technology could power everything -- and light up our world from space.
Hosted on Acast. See acast.com/privacy for more information.
How LLMs and Generative AI are Revolutionizing AI for Science with Anima Anandkumar - #614
Today we’re joined by Anima Anandkumar, Bren Professor of Computing And Mathematical Sciences at Caltech and Sr Director of AI Research at NVIDIA. In our conversation, we take a broad look at the emerging field of AI for Science, focusing on both practical applications and longer-term research areas. We discuss the latest developments in the area of protein folding, and how much it has evolved since we first discussed it on the podcast in 2018, the impact of generative models and stable diffusion on the space, and the application of neural operators. We also explore the ways in which prediction models like weather models could be improved, how foundation models are helping to drive innovation, and finally, we dig into MineDojo, a new framework built on the popular Minecraft game for embodied agent research, which won a 2022 Outstanding Paper Award at NeurIPS.
The complete show notes for this episode can be found at twimlai.com/go/614

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research
#214 – Jed Buchwald: Isaac Newton and the Philosophy of Science
Jed Buchwald is a historian and philosopher of science at Caltech. Please support this podcast by checking out our sponsors:
– GiveWell: https://www.givewell.org/ and use code LEX to get donation matched up to $1k
– Theragun: https://therabody.com/lex to get 30 day trial
– LMNT: https://drinkLMNT.com/lex to get free sample pack
– Fundrise: https://fundrise.com/lex
– BetterHelp: https://betterhelp.com/lex to get 10% off
EPISODE LINKS:
Jed’s Caltech page: https://bit.ly/38eLLRF
Jed’s Books: https://amzn.to/2WoxGPi
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(06:34) – How does science progress?
(22:48) – Theory of Everything
(34:40) – Consciousness
(38:15) – Most Beautiful Moments in Science
(46:00) – Isaac Newton
(1:12:13) – Competition in Science
(1:22:47) – Newton’s Career
(1:35:58) – Importance of Data
(1:42:17) – Alchemy
(1:46:31) – Newton and Religion
(1:49:44) – Showing Newton the future
(1:54:28) – Newton and Einstein
#213 – Barry Barish: Gravitational Waves and the Most Precise Device Ever Built
Barry Barish is a theoretical physicist at Caltech and the winner of the Nobel Prize in Physics. Please support this podcast by checking out our sponsors:
– MUD\WTR: https://mudwtr.com/lex and use code LEX to get 5% off
– GiveDirectly: https://givedirectly.org/lex to get gift matched up to $300
– BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off
– Four Sigmatic: https://foursigmatic.com/lex and use code LexPod to get up to 60% off
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
EPISODE LINKS:
Barry’s Nobel Prize entry: https://www.nobelprize.org/prizes/physics/2017/barish/facts/
Barry’s Caltech profile: https://pma.caltech.edu/people/barry-c-barish
LIGO’s Website: https://www.ligo.caltech.edu/
LIGO’s Twitter: https://twitter.com/LIGO
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(08:27) – Early Math and Physics questions
(18:02) – Enrico Fermi
(24:34) – Birth of the Nuclear Age
(29:42) – The Fermi Paradox
(34:45) – Gravity
(51:28) – Philosophical Implications of General Relativity
(58:34) – Detecting Gravitational Waves
(1:01:47) – LIGO
(1:34:45) – Nobel Prize
(1:49:34) – Black Holes
(2:01:53) – Space Exploration
(2:09:48) – Books
(2:18:37) – Advice for young people
(2:24:33) – Meaning of life
#201 – Konstantin Batygin: Planet 9 and the Edge of Our Solar System
Konstantin Batygin is a planetary astrophysicist at Caltech. Please support this podcast by checking out our sponsors:
– Squarespace: https://lexfridman.com/squarespace and use code LEX to get 10% off
– Literati: https://literati.com/lex
– Onnit: https://lexfridman.com/onnit to get up to 10% off
– National Instruments (NI): https://www.ni.com/perspectives
EPISODE LINKS:
Konstantin’s Twitter https://twitter.com/kbatygin
Konstantin’s Website https://www.konstantinbatygin.com/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:17) – Overview of our Solar System
(22:14) – What is the Oort Cloud?
(27:10) – Life in the interstellar medium
(28:42) – Are there aliens out there?
(31:22) – How unique is Earth?
(34:02) – Did Jupiter destroy early planets?
(40:16) – How hard is it to simulate the Universe?
(44:49) – Quantum mechanics in evolution of objects in the Solar system
(49:15) – Simulating the first formations around the Sun
(55:02) – Will it be possible to simulate the full history of the Solar System?
(57:23) – How far should we go with the simulation?
(59:43) – Increasing immersion in video games
(1:06:09) – What is Planet Nine?
(1:12:37) – The origin of life
(1:15:02) – Evidence of Planet Nine
(1:17:32) – Discovery of Neptune
(1:18:42) – When will we find Planet Nine?
(1:21:21) – Planet Nine throws rocks into the Kuiper Belt
(1:25:15) – Could Planet Nine be a primordial black hole?
(1:35:20) – Commercial space revolution boosts science and the human condition
(1:42:46) – Solving sex in space
(1:43:24) – Would humans evolve if we couldn’t see the stars?
(1:49:08) – Military funding and science
(1:53:11) – Is Oumuamua space junk from a distant alien civilization?
(2:06:33) – Wild ideas create the future
(2:14:22) – The perfect place to die
(2:16:03) – Greatest song of all time
(2:22:34) – Music enables science for Konstantin
(2:24:51) – Music practice tips for busy people
(2:28:41) – Memories of 1990s Russia
(2:35:14) – Advice for young people
(2:41:10) – Meaning of life
473: Machine Learning at NVIDIA
Anima Anandkumar joins us to discuss her work as a researcher in machine learning at NVIDIA and a professor at CalTech, and how they often go hand-in-hand and inform each other.
In this episode you will learn:
Anima’s recent discovery of yoga [5:20]
How does Anima balance her work? [12:25]
Applications of Anima’s work [14:45]
Tensors [22:55]
Anima’s favorite NVIDIA projects [35:35]
What tools does NVIDIA use? [41:55]
CalTech interdisciplinary science [47:41]
The path to generalized artificial intelligence [57:19]
The skills to have to get into this field [1:00:27]
LinkedIn questions for Anima [1:07:03]
Additional materials: www.superdatascience.com/473

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research
#184 – Katherine de Kleer: Planets, Moons, and Asteroids in Our Solar System
Katherine de Kleer is a professor of Planetary Science and Astronomy at Caltech. Please support this podcast by checking out our sponsors:
– Fundrise: https://fundrise.com/lex
– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium
– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off
EPISODE LINKS:
Katherine’s Website: http://web.gps.caltech.edu/~dekleer/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:07) – Pluto
(12:14) – Kuiper belt
(16:12) – How to study planets and moons
(19:54) – Volcanoes on Io – moon of Jupiter
(32:25) – Is there life in the oceans of Europa?
(41:46) – How unlikely is life on Earth?
(52:15) – Life on Venus
(54:30) – Mars
(1:01:17) – What is interesting about Earth as a planet?
(1:11:55) – Weather patterns
(1:17:04) – Asteroids
(1:26:06) – Will an asteroid hit Earth soon?
(1:34:50) – Oumuamua
(1:50:00) – Book recommendations
(1:56:37) – Advice for young people
Behind the Scenes at NeurIPS with NVIDIA and CalTech’s Anima Anandkumar - Ep. 131
Anima Anandkumar is setting a personal record this week with seven of her team’s research papers accepted to NeurIPS 2020.
The 34th annual Neural Information Processing Systems conference is taking place virtually from Dec. 6-12. The premier event on neural networks, NeurIPS draws thousands of the world’s best researchers every year.
Anandkumar, NVIDIA’s director of machine learning research and Bren professor at CalTech’s CMS Department, joined AI Podcast host Noah Kravitz to talk about what to expect at the conference, and to explain what she sees as the future of AI.
https://blogs.nvidia.com/blog/2020/12/09/neurips-nvidia-caltech-anima-anandkumar/

Anima AnandkumarBren Professor of Computing and Mathematical Sciences and Senior Director of AI Research