“While language models may help generate new ideas, they cannot attack the hard part of science, which is simulating the necessary physics,” says AI professor Anima Anandkumar. She explains how her team developed neural operators — AI trained on the finest details of the real world — to bridge this gap, sharing recent projects ranging from improved weather forecasting to cutting-edge medical device design that demonstrate the power of AI with universal physical understanding.
Hosted on Acast. See acast.com/privacy for more information.
Generative AI-based models can not only learn and understand natural languages — they can learn the very language of nature itself, presenting new possibilities for scientific research.
Anima Anandkumar, Bren Professor at Caltech and senior director of AI research at NVIDIA, was recently invited to speak at the President’s Council of Advisors on Science and Technology.
At the talk, Anandkumar says that generative AI was described as “an inflection point in our lives,” with discussions swirling around how to “harness it to benefit society and humanity through scientific applications.”
On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with Anandkumar on generative AI’s potential to make splashes in the scientific community.
It can, for example, be fed DNA, RNA, viral and bacterial data to craft a model that understands the language of genomes. That model can help predict dangerous coronavirus variants to accelerate drug and vaccine research.
Generative AI can also predict extreme weather events like hurricanes or heat waves. Even with an AI boost, trying to predict natural events is challenging because of the sheer number of variables and unknowns.
However, Anandkumar explains that it’s not just a matter of upsizing language models or adding compute power — it’s also about fine-tuning and setting the right parameters.
“Those are the aspects we’re working on at NVIDIA and Caltech, in collaboration with many other organizations, to say, ‘How do we capture the multitude of scales present in the natural world?’” she said. “With the limited data we have, can we hope to extrapolate to finer scales? Can we hope to embed the right constraints and come up with physically valid predictions that make a big impact?”
Anandkumar adds that to ensure AI models are responsibly and safely used, existing laws must be strengthened to prevent dangerous downstream applications.
She also talks about the AI boom, which is transforming the role of humans across industries, and problems yet to be solved.
“This is the research advice I give to everyone: the most important thing is the question, not the answer,” she said.
Today we’re joined by Anima Anandkumar, Bren Professor of Computing And Mathematical Sciences at Caltech and Sr Director of AI Research at NVIDIA. In our conversation, we take a broad look at the emerging field of AI for Science, focusing on both practical applications and longer-term research areas. We discuss the latest developments in the area of protein folding, and how much it has evolved since we first discussed it on the podcast in 2018, the impact of generative models and stable diffusion on the space, and the application of neural operators. We also explore the ways in which prediction models like weather models could be improved, how foundation models are helping to drive innovation, and finally, we dig into MineDojo, a new framework built on the popular Minecraft game for embodied agent research, which won a 2022 Outstanding Paper Award at NeurIPS.
The complete show notes for this episode can be found at twimlai.com/go/614
Anima Anandkumar joins us to discuss her work as a researcher in machine learning at NVIDIA and a professor at CalTech, and how they often go hand-in-hand and inform each other.
In this episode you will learn:
Anima’s recent discovery of yoga [5:20]
How does Anima balance her work? [12:25]
Applications of Anima’s work [14:45]
Tensors [22:55]
Anima’s favorite NVIDIA projects [35:35]
What tools does NVIDIA use? [41:55]
CalTech interdisciplinary science [47:41]
The path to generalized artificial intelligence [57:19]
The skills to have to get into this field [1:00:27]
LinkedIn questions for Anima [1:07:03]
Additional materials: www.superdatascience.com/473
Anima Anandkumar is setting a personal record this week with seven of her team’s research papers accepted to NeurIPS 2020.
The 34th annual Neural Information Processing Systems conference is taking place virtually from Dec. 6-12. The premier event on neural networks, NeurIPS draws thousands of the world’s best researchers every year.
Anandkumar, NVIDIA’s director of machine learning research and Bren professor at CalTech’s CMS Department, joined AI Podcast host Noah Kravitz to talk about what to expect at the conference, and to explain what she sees as the future of AI.
https://blogs.nvidia.com/blog/2020/12/09/neurips-nvidia-caltech-anima-anandkumar/
Sean Carroll is a theoretical physicist at Caltech and Santa Fe Institute specializing in quantum mechanics, arrow of time, cosmology, and gravitation. He is the author of Something Deeply Hidden and several popular books and he is the host of a great podcast called Mindscape. This is the second time Sean has been on the podcast. You can watch the first time on YouTube or listen to the first time on its episode page. This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts or support it on Patreon. Here’s the outline with timestamps for this episode (on some players you can click on the timestamp to jump to that point in the episode):
00:00 – Introduction
01:23 – Capacity of human mind to understand physics
10:49 – Perception vs reality
12:29 – Conservation of momentum
17:20 – Difference between math and physics
20:10 – Why is our world so compressable
22:53 – What would Newton think of quantum mechanics
25:44 – What is quantum mechanics?
27:54 – What is an atom?
30:34 – What is the wave function?
32:30 – What is quantum entanglement?
35:19 – What is Hilbert space?
37:32 – What is entropy?
39:31 – Infinity
42:43 – Many-worlds interpretation of quantum mechanics
1:01:13 – Quantum gravity and the emergence of spacetime
1:08:34 – Our branch of reality in many-worlds interpretation
1:10:40 – Time travel
1:12:54 – Arrow of time
1:16:18 – What is fundamental in physics
1:16:58 – Quantum computers
1:17:42 – Experimental validation of many-worlds and emergent spacetime
1:19:53 – Quantum mechanics and the human mind
1:21:51 – Mindscape podcast
Sean Carroll is a theoretical physicist at Caltech, specializing in quantum mechanics, gravity, and cosmology. He is the author of several popular books: one on the arrow of time called From Eternity to Here, one on the Higgs boson called The Particle at the End of the Universe, and one on science and philosophy called The Big Picture: On the Origins of Life, Meaning, and the Universe Itself. He has an upcoming book on Quantum Mechanics that you can preorder now called Something Deeply Hidden. Finally, and perhaps most famously, he is the host of a podcast called Mindscape that you should subscribe to and support on Patreon. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations.
While at the NVIDIA GPU Technology Conference 2019 in Silicon Valley, Chris enjoyed an inspiring conversation with Anima Anandkumar. Clearly a role model - not only for women - but for anyone in the world of AI, Anima relayed how her lifelong passion for mathematics and engineering started when she was only 3 years old in India, and ultimately led to her pioneering deep learning research at Amazon Web Services, CalTech, and NVIDIA.
Sponsors:
DigitalOcean – Check out DigitalOcean’s dedicated vCPU Droplets with dedicated vCPU threads. Get started for free with a $100 credit. Learn more at do.co/changelog.
DataEngPodcast – A podcast about data engineering and modern data infrastructure.
Rollbar – We move fast and fix things because of Rollbar. Resolve errors in minutes. Deploy with confidence. Learn more at rollbar.com/changelog.
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Featuring:
Anima Anandkumar – Website, LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Show Notes:
Anima Anandkumar on Wikipedia
NVIDIA GTC Speaker
NVIDIA GTC Presentation: Role of Tensors in Machine Learning
NVIDIA GTC Presentation: Infusing Physics into Deep Learning Algorithms with Applications to Stable Landing of Drones
CalTech CMS TensorLab
Google Scholar
Upcoming Events:
Register for upcoming webinars here!
In the inaugural TWiML Live, Sam Charrington is joined by Amanda Askell (OpenAI), Anima Anandkumar (NVIDIA/CalTech), Miles Brundage (OpenAI), Robert Munro (Lilt), and Stephen Merity to discuss the controversial recent release of the OpenAI GPT-2 Language Model.
We cover the basics like what language models are and why they’re important, and why this announcement caused such a stir, and dig deep into why the lack of a full release of the model raised concerns for so many.
In this episode of our AI Rewind series, we’re back with Anima Anandkumar, Bren Professor at Caltech and now Director of Machine Learning Research at NVIDIA.
Anima joins us to discuss her take on trends in the broader Machine Learning field in 2018 and beyond. In our conversation, we cover not only technical breakthroughs in the field but also those around inclusivity and diversity.
For this episode's complete show notes, visit twimlai.com/talk/215.
In this episode of our TrainAI series, I sit down with Anima Anandkumar, Bren Professor at Caltech and Principal Scientist with Amazon Web Services. Anima joined me to discuss the research coming out of her “Tensorlab” at CalTech. In our conversation, we review the application of tensor operations to machine learning and discuss how an example problem–document categorization–might be approached using 3 dimensional tensors to discover topics and relationships between topics. We touch on multidimensionality, expectation maximization, and Amazon products Sagemaker and Comprehend. Anima also goes into how to tensorize neural networks and apply our understanding of tensor algebra to do perform better architecture searches. The notes for this show can be found at twimlai.com/talk/142. For series info, visit twimlai.com/trainai2018
In today’s show, I sit down with David Van Valen, assistant professor of Bioengineering & Biology at Caltech. David joined me after his talk at the Figure Eight TrainAI conference to chat about his research using image recognition and segmentation techniques in biological settings. In particular, we discuss his use of deep learning to automate the analysis of individual cells in live-cell imaging experiments. We had a really interesting discussion around the various practicalities he’s learned about training deep neural networks for image analysis, and he shares some great insights into which of the techniques from the deep learning research have worked for him and which haven’t. If you’re a fan of our Nerd Alert shows, you’ll really like this one. Enjoy! The notes for this show can be found at twimlai.com/talk/141. For more information on this series, visit twimlai.com/trainai2018.
