D. Sculley — Technical Debt, Trade-offs, and Kaggle
D. Sculley is CEO of Kaggle, the beloved and well-known data science and machine learning community.
D. discusses his influential 2015 paper "Machine Learning: The High Interest Credit Card of Technical Debt" and what the current challenges of deploying models in the real world are now, in 2022. Then, D. and Lukas chat about why Kaggle is like a rain forest, and about Kaggle's historic, current, and potential future roles in the broader machine learning community.
Show notes (transcript and links): http://wandb.me/gd-d-sculley
---
⏳ Timestamps:
0:00 Intro
1:02 Machine learning and technical debt
11:18 MLOps, increased stakes, and realistic expectations
19:12 Evaluating models methodically
25:32 Kaggle's role in the ML world
33:34 Kaggle competitions, datasets, and notebooks
38:49 Why Kaggle is like a rain forest
44:25 Possible future directions for Kaggle
46:50 Healthy competitions and self-growth
48:44 Kaggle's relevance in a compute-heavy future
53:49 AutoML vs. human judgment
56:06 After a model goes into production
1:00:00 Outro
---
Connect with D. and Kaggle:
📍 D. on LinkedIn: https://www.linkedin.com/in/d-sculley-90467310/
📍 Kaggle on Twitter: https://twitter.com/kaggle
---
Links:
📍 "Machine Learning: The High Interest Credit Card of Technical Debt" (Sculley et al. 2014): https://research.google/pubs/pub43146/
---
💬 Host: Lukas Biewald
📹 Producers: Riley Fields, Angelica Pan, Anish Shah, Lavanya Shukla
---
Subscribe and listen to our podcast today!
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 Spotify: http://wandb.me/spotify
Data Debt in Machine Learning with D. Sculley - #574
Today we kick things off with a conversation with D. Sculley, a director on the Google Brain team. Many listeners of today’s show will know D. from his work on the paper, The Hidden Technical Debt in Machine Learning Systems, and of course, the infamous diagram. D. has recently translated the idea of technical debt into data debt, something we spend a bit of time on in the interview.
We discuss his view of the concept of DCAI, where debt fits into the conversation of data quality, and what a shift towards data-centrism looks like in a world of increasingly larger models i.e. GPT-3 and the recent PALM models. We also explore common sources of data debt, what are things that the community can and have done to mitigate these issues, the usefulness of causal inference graphs in this work, and much more! If you enjoyed this interview or want to hear more on this topic, check back on the DCAI series page weekly at https://twimlai.com/podcast/twimlai/series/data-centric-ai.
The complete show notes for this episode can be found at twimlai.com/go/574
Anthony Goldbloom — How to Win Kaggle Competitions
Anthony Goldbloom is the founder and CEO of Kaggle. In 2011 & 2012, Forbes Magazine named Anthony as one of the 30 under 30 in technology. In 2011, Fast Company featured him as one of the innovative thinkers who are changing the future of business.
He and Lukas discuss the differences in strategies that do well in Kaggle competitions vs academia vs in production. They discuss his 2016 Ted talk through the lens of 2020, frameworks, and languages.
Topics Discussed:
0:00 Sneak Peek
0:20 Introduction
0:45 methods used in kaggle competitions vs mainstream academia
2:30 Feature engineering
3:55 Kaggle Competitions now vs 10 years ago
8:35 Data augmentation strategies
10:06 Overfitting in Kaggle Competitions
12:53 How to not overfit
14:11 Kaggle competitions vs the real world
18:15 Getting into ML through Kaggle
22:03 Other Kaggle products
25:48 Favorite under appreciated kernel or dataset
28:27 Python & R
32:03 Frameworks
35:15 2016 Ted talk though the lens of 2020
37:54 Reinforcement Learning
38:43 What’s the topic in ML that people don’t talk about enough?
42:02 Where are the biggest bottlenecks in deploying ML software?
Check out Kaggle: https://www.kaggle.com/
Follow Anthony on Twitter: https://twitter.com/antgoldbloom
Watch his 2016 Ted Talk: https://www.ted.com/talks/anthony_goldbloom_the_jobs_we_ll_lose_to_machines_and_the_ones_we_won_t
Visit our podcasts homepage for transcripts and more episodes!
www.wandb.com/podcast
Get our podcast on Soundcloud, Apple, and Spotify!
Soundcloud: https://bit.ly/2YnGjIq
Apple Podcasts: https://bit.ly/2WdrUvI
Spotify: https://bit.ly/2SqtadF
We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it!
Weights and Biases:
We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions.
* Blog: https://www.wandb.com/articles
* Gallery: See what you can create with W&B - https://app.wandb.ai/gallery
* Join our community of ML practitioners working on interesting problems - https://www.wandb.com/ml-community
Host: Lukas Biewald - https://twitter.com/l2k
Producer: Lavanya Shukla - https://twitter.com/lavanyaai
Editor: Cayla Sharp - http://caylasharp.com/