In this episode of Gradient Dissent, Together AI co-founder and Stanford Associate Professor Percy Liang joins host, Lukas Biewald, to discuss advancements in AI benchmarking and the pivotal role that open-source plays in AI development.
He shares his development of HELM—a robust framework for evaluating language models. The discussion highlights how this framework improves transparency and effectiveness in AI benchmarks. Additionally, Percy shares insights on the pivotal role of open-source models in democratizing AI development and addresses the challenges of English language bias in global AI applications. This episode offers in-depth insights into how benchmarks are shaping the future of AI, highlighting both technological advancements and the push for more equitable and inclusive technologies.
✅ Subscribe to Weights & Biases → http://wandb.me/yt_subscribe
Connect with Percy Liang:
https://www.linkedin.com/in/percy-liang-717b8a/
https://twitter.com/percyliang
Anticipatory Music Composer:
https://stanford.io/3y5VycN
Blog Post:
https://crfm.stanford.edu/2024/02/18/helm-instruct.html
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
When AI research is evolving at warp speed and takes significant capital and compute power, what is the role of academia? Dr. Percy Liang – Stanford computer science professor and director of the Stanford Center for Research on Foundation Models talks about training costs, distributed infrastructure, model evaluation, alignment, and societal impact.
Sarah Guo and Elad Gil join Percy at his office to discuss the evolution of research in NLP, why AI developers should aim for superhuman levels of performance, the goals of the Center for Research on Foundation Models, and Together, a decentralized cloud for artificial intelligence.
No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.
Show Links:
See Percy’s Research on Google Scholar
See Percy’s bio on Stanford’s website
Percy on Stanford’s Blog: What to Expect in 2023 in AI
Together, a decentralized cloud for artificial intelligence
Foundation AI models GPT-3 and DALL-E need release standards - Protocol
The Time Is Now to Develop Community Norms for the Release of Foundation Models - Stanford
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @PercyLiang
Show Notes:
[1:44] - How Percy got into machine learning research and started the Center for Research and Foundation Models at Stanford
[7:23] - The role of academia and academia’s competitive advantages
[13:30] - Research on natural language processing and computational semantics
[27:20] - Smaller scale architectures that are competitive with transformers
[35:08] - Helm, holistic evaluation of language models, a project with the the goal is to evaluate language models
[42:13] - Together, a decentralized cloud for artificial intelligence