Today, we're joined by Devi Parikh, co-founder and co-CEO of Yutori, to discuss browser use models and a future where we interact with the web through proactive, autonomous agents. We explore the technical challenges of creating reliable web agents, the advantages of visually-grounded models that operate on screenshots rather than the browser’s more brittle document object model, or DOM, and why this counterintuitive choice has proven far more robust and generalizable for handling complex web interfaces. Devi also shares insights into Yutori’s training pipeline, which has evolved from supervised fine-tuning to include rejection sampling and reinforcement learning. Finally, we discuss how Yutori’s “Scouts” agents orchestrate multiple tools and sub-agents to handle complex queries, the importance of background, "ambient" operation for these systems, and what the path looks like from simple monitoring to full task automation on the web.
The complete show notes for this episode can be found at https://twimlai.com/go/756.
Tech firms are racing to develop robot assistants that can take over our dreaded household chores. But teaching machines to perform these deceptively simple tasks is tedious. They need to observe the actions thousands, sometimes millions of times. And there's a cottage industry springing up to provide this training.
Marketplace’s Meghan McCarty Carino spoke with Ayanna Howard, roboticist and dean of Ohio State University’s college of engineering, to learn more.
Today we're joined by Alex Havrilla, a PhD student at Georgia Tech, to discuss "Teaching Large Language Models to Reason with Reinforcement Learning." Alex discusses the role of creativity and exploration in problem solving and explores the opportunities presented by applying reinforcement learning algorithms to the challenge of improving reasoning in large language models. Alex also shares his research on the effect of noise on language model training, highlighting the robustness of LLM architecture. Finally, we delve into the future of RL, and the potential of combining language models with traditional methods to achieve more robust AI reasoning.
The complete show notes for this episode can be found at twimlai.com/go/680.
Video dominates modern media consumption, but video creation is still expensive and difficult. AI-generated and edited video is a holy grail of democratized creative expression. This week on No Priors, Sarah Guo and Elad Gil sit down with Devi Parikh. She is a Research Director in Generative AI at Meta and an Associate Professor in the School of Interactive Computing at Georgia Tech. Her work focuses on multimodality and AI for images, audio and video. Recently, she worked on Make a Video 3D, also called MAV3D, which creates animations from text prompts. She is also a talented AI-generated and analog artist herself.
Elad, Sarah and Devi talk about what’s exciting in computer vision, what’s blocking researchers from fully immersive Generative 4-D, and AI controllability.
No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.
Show Links:
Devi Parikh - Google Scholar
Text-To-4D Dynamic Scene Generation named MAV3D (Make-A-Video3D)
Full Research Paper
Website with examples of image to 4 D generation
Devi’s Substack
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @DeviParikh
Show Notes:
(0:00:06) - Democratizing Creative Expression With AI-Generated Video
(0:08:31) - Challenges in Video Generation Research
(0:15:57) - Challenges and Implications of Video Processing
(0:20:43) - Control and Multi-Modal Inputs in Video
(0:25:50) - Audio's Role in Visual Content
(0:39:00) - Don't Self-Select & Devi’s tips for young researchers
For Episode 13, we're joined by industry pioneer, Dean Ayanna Howard. She began working at NASA's JPL at 18-years old to help build the Mars Rover and never slowed down from there. She is a successful roboticist, entrepreneur, educator, and is the author of the recent book: Sex, Race, and Robots: How to be Human in the Age of AI.
Dr. Ayanna Howard is the current dean of The Ohio State University College of Engineering. Before joining Ohio State, she was Professor at Georgia Tech, where she was the founder and director of the Human-Automation Systems Lab (HumAnS). She led a range of projects including applying her knowledge of the Rover's SmartNav abilities to create robots that could explore remote areas of Antartica and Alaska to collect data for studying climate change. In addition to her leadership at Ohio State, she also started Zyrobotics, a non-profit dedicated to helping children that require extra development support.
The conversation spans Ayanna's childhood, her inspirations like the Bionic Woman, and her views on building a more equitable AI ecosystem for all individuals.
SUBSCRIBE TO THE ROBOT BRAINS PODCAST TODAY | Visit therobotbrains.ai and follow us on YouTube at TheRobotBrainsPodcast, Twitter @therobotbrains, and Instagram @therobotbrains.
Hosted on Acast. See acast.com/privacy for more information.
In episode twelve of The Robot Brains Podcast we are joined by Charles Isbell Jr, professor and Dean of the College of Computing at the Georgia Institute of Technology. After starting his career as an industrial researcher at the legendary Bell Labs, and a long research career in Interactive and Human-Centric AI, Charles has more recently turned his attention to the major issues of ethics, fairness and diversity that are becoming ever more important as AI is being deployed in the real world. Speaking with Pieter Abbeel, Charles explains why researchers can find making ethical AI challenging, his fascinating keynote speech at NeurIPS, and how more diversity in academic admissions can help to improve AI research. Host: Pieter Abbeel | Executive Producers: Ricardo Reyes & Henry Tobias Jones | Audio Production: Kieron Matthew Banerji | Title Music: Alejandro Del Pozo Hosted on Acast. See acast.com/privacy for more information.
Today we’re joined by Mark Riedl, a Professor in the School of Interactive Computing at Georgia Tech. In our conversation with Mark, we explore his work building AI storytelling systems, mainly those that try and predict what listeners think will happen next in a story and how he brings together many different threads of ML/AI together to solve these problems. We discuss how the theory of mind is layered into his research, the use of large language models like GPT-3, and his push towards being able to generate suspenseful stories with these systems.
We also discuss the concept of intentional creativity and the lack of good theory on the subject, the adjacent areas in ML that he’s most excited about for their potential contribution to his research, his recent focus on model explainability, how he approaches problems of common sense, and much more!
The complete show notes for this episode can be found at https://twimlai.com/go/478.
Today we’re joined by returning guest and newly appointed Dean of the College of Engineering at The Ohio State University, Ayanna Howard.
Our conversation with Dr. Howard focuses on her recently released book, Sex, Race, and Robots: How to Be Human in the Age of AI, which is an extension of her research on the relationships between humans and robots. We continue to explore this relationship through the themes of socialization introduced in the book, like associating genders to AI and robotic systems and the “self-fulfilling prophecy” that has become search engines.
We also discuss a recurring conversation in the community around AI being biased because of data versus models and data, and the choices and responsibilities that come with the ethical aspects of building AI systems. Finally, we discuss Dr. Howard’s new role at OSU, how it will affect her research, and what the future holds for the applied AI field.
The complete show notes for this episode can be found at https://twimlai.com/go/460.
Today we’re joined by returning guest and newly appointed Dean of the College of Engineering at The Ohio State University, Ayanna Howard.
Our conversation with Dr. Howard focuses on her recently released book, Sex, Race, and Robots: How to Be Human in the Age of AI, which is an extension of her research on the relationships between humans and robots. We continue to explore this relationship through the themes of socialization introduced in the book, like associating genders to AI and robotic systems and the “self-fulfilling prophecy” that has become search engines.
We also discuss a recurring conversation in the community around AI being biased because of data versus models and data, and the choices and responsibilities that come with the ethical aspects of building AI systems. Finally, we discuss Dr. Howard’s new role at OSU, how it will affect her research, and what the future holds for the applied AI field.
The complete show notes for this episode can be found at https://twimlai.com/go/460.
Charles Isbell is the Dean of the College of Computing at Georgia Tech. Michael Littman is a computer scientist at Brown University. Please support this podcast by checking out our sponsors:
– Athletic Greens: https://athleticgreens.com/lex and use code LEX to get 1 month of fish oil
– Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savings
– MasterClass: https://masterclass.com/lex to get 2 for price of 1
– Cash App: https://cash.app/ and use code LexPodcast to get $10
EPISODE LINKS:
Charles’s Twitter: https://twitter.com/isbellHFh
Charles’s Website: https://www.cc.gatech.edu/~isbell/
Michael’s Twitter: https://twitter.com/mlittmancs
Michael’s Website: https://www.littmania.com/
Michael’s YouTube: https://www.youtube.com/user/mlittman
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/LexFridmanPage
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:51) – Is machine learning just statistics?
(12:14) – NeurIPS vs ICML
(14:30) – Data is more important than algorithm
(20:14) – The role of hardship in education
(28:57) – How Charles and Michael met
(33:30) – Key to success: never be satisfied
(36:47) – Bell Labs
(48:15) – Teaching machine learning
(58:25) – Westworld and Ex Machina
(1:06:24) – Simulation
(1:13:14) – The college experience in the times of COVID
(1:41:52) – Advice for young people
(1:48:44) – How to learn to program
(2:00:07) – Friendship
As we continue our NeurIPS 2020 series, we’re joined by friend-of-the-show Charles Isbell, Dean, John P. Imlay, Jr. Chair, and professor at the Georgia Tech College of Computing.
This year Charles gave an Invited Talk at this year’s conference, You Can’t Escape Hyperparameters and Latent Variables: Machine Learning as a Software Engineering Enterprise. In our conversation, we explore the success of the Georgia Tech Online Masters program in CS, which now has over 11k students enrolled, and the importance of making the education accessible to as many people as possible. We spend quite a bit speaking about the impact machine learning is beginning to have on the world, and how we should move from thinking of ourselves as compiler hackers, and begin to see the possibilities and opportunities that have been ignored.
We also touch on the fallout from Timnit Gebru being “resignated” and the importance of having diverse voices and different perspectives “in the room,” and what the future holds for machine learning as a discipline.
The complete show notes for this episode can be found at twimlai.com/go/441.
Charles Isbell is the Dean of the College of Computing at Georgia Tech. Please support this podcast by checking out our sponsors:
– Neuro: https://www.getneuro.com and use code LEX to get 15% off
– Decoding Digital: https://appdirect.com/decoding-digital
– MasterClass: https://masterclass.com/lex to get 15% off annual sub
– Cash App: https://cash.app/ and use code LexPodcast to get $10
EPISODE LINKS:
Charles’s Twitter: https://twitter.com/isbellHFh
Charles’s Website: https://www.cc.gatech.edu/~isbell/
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/LexFridmanPage
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:16) – Top 3 movies of all time
(13:26) – People are easily predictable
(19:08) – Breaking out of our bubbles
(30:54) – Interactive AI
(37:26) – Lifelong machine learning
(45:53) – Faculty hiring
(53:27) – University rankings
(1:00:55) – Science communicators
(1:10:20) – Hip hop
(1:19:20) – Funk
(1:20:44) – Computing
(1:36:35) – Race
(1:52:40) – Cop story
(2:01:01) – Racial tensions
(2:10:23) – MLK vs Malcolm X
(2:13:44) – Will human civilization destroy itself?
(2:18:14) – Fear of death and the passing of time
Today we’re joined by Devi Parikh, Associate Professor at the School of Interactive Computing at Georgia Tech, and research scientist at Facebook AI Research (FAIR). In our conversation, we touch on Devi’s definition of creativity, explore multiple ways that AI could impact the creative process for artists, and help humans become more creative. We investigate tools like casual creator for preference prediction, neuro-symbolic generative art, and visual journaling.
Ayanna Howard is a roboticist and professor at Georgia Tech, director of Human-Automation Systems lab, with research interests in human-robot interaction, assistive robots in the home, therapy gaming apps, and remote robotic exploration of extreme environments.
This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon.
This episode is presented by Cash App. Download it (App Store, Google Play), use code “LexPodcast”.
Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time.
00:00 – Introduction
02:09 – Favorite robot
05:05 – Autonomous vehicles
08:43 – Tesla Autopilot
20:03 – Ethical responsibility of safety-critical algorithms
28:11 – Bias in robotics
38:20 – AI in politics and law
40:35 – Solutions to bias in algorithms
47:44 – HAL 9000
49:57 – Memories from working at NASA
51:53 – SpotMini and Bionic Woman
54:27 – Future of robots in space
57:11 – Human-robot interaction
1:02:38 – Trust
1:09:26 – AI in education
1:15:06 – Andrew Yang, automation, and job loss
1:17:17 – Love, AI, and the movie Her
1:25:01 – Why do so many robotics companies fail?
1:32:22 – Fear of robots
1:34:17 – Existential threats of AI
1:35:57 – Matrix
1:37:37 – Hang out for a day with a robot
In this episode, the third in our Black in AI series, I speak with Ayanna Howard, Chair of the Interactive School of Computing at Georgia Tech. Ayanna joined me for a lively discussion about her work in the field of human-robot interaction. We dig deep into a couple of major areas she’s active in that have significant implications for the way we design and use artificial intelligence, namly pediatric robotics and human-robot trust. That latter bit is particularly interesting, and Ayanna provides a really interesting overview of a few of her experiments, including a simulation of an emergency situation, where, well, I don’t want to spoil it, but let’s just say as the actual intelligent beings, we need to make some better decisions. Enjoy! Are you looking forward to the role AI will play in your life, or in your children’s lives? Or, are you afraid of what’s to come, and the changes AI will bring? Or, maybe you’re skeptical, and don’t think we’ll ever really achieve enough with AI to make a difference? As a TWiML listener, you probably have an opinion on the role AI will play in our lives, and we want to hear your take. Sharing your thoughts takes two minutes, can be done from anywhere, and qualifies you to win some great prizes. So hit pause, and jump on over twimlai.com/myai right now to share or learn more. Be sure to check out some of the great names that will be at the AI Conference in New York, Apr 29–May 2, where you'll join the leading minds in AI, Peter Norvig, George Church, Olga Russakovsky, Manuela Veloso, and Zoubin Ghahramani. Explore AI's latest developments, separate what's hype and what's really game-changing, and learn how to apply AI in your organization right now. Save 20% on most passes with discount code PCTWIML at twimlai.com/ainy2018. The notes for this show can be found at twimlai.com/talk/110. For complete contest details, visit twimlai.com/myai. For complete series details, visit twimlai.com/blackinai2018.
My guest this time is Charles Isbell, Jr., Professor and Senior Associate Dean in the College of Computing at Georgia Institute of Technology. Charles and I go back a bit… in fact he’s the first AI researcher I ever met. His research focus is what he calls “interactive artificial intelligence,” a discipline of AI focused specifically on the interactions between AIs and humans. We explore what this means and some of the interesting research results in this field. One part of this discussion I found particularly interesting was the intersection between his AI research and marketing and behavioral economics. Beyond his research, Charles is well known in the ML and AI worlds for his popular Machine Learning course sequence on Udacity, which he teaches with Brown University professor Michael Littman, and for the Online Master’s of Science in Computer Science program that he helped launch at Georgia Tech. We also spend quite a bit of time talking about what’s really missing in machine learning education and how to make it more accessible. The notes for this show can be found at twimlai.com/talk/4.
