1006: In Case You Missed It in June 2026
In this month's episode of ICYMI, hear from Chip Huyen, Andrey Kurenkov, Frank Basso and Gilbert Eijkelenboom, discussing why moats are shifting toward physical systems and accumulated product intuition, how Astrocade built vibe coding before the term existed, what it's really like inside a deafeningly loud AI data center, why only 15% of people are technically self-aware and whether AGI requires anything like consciousness.
Additional materials: www.superdatascience.com/1006
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
In this episode you will learn:
(00:00) The Cost of Building Software Is Going to Zero — Now What?
(10:18) We Built Vibe Coding Before Anyone Called It That
(21:08) AI Data Centers Are Louder Than a Rock Concert
(28:39) Why 85% of Data Scientists Can't Communicate Their Work
(33:46) Are Humans Also Just Predicting the Next Token?
999: What's Left to Build When Software Is Free, with Chip Huyen
Chip Huyen joins host Jon Krohn for this milestone episode 999 to talk about her record-breaking book "AI Engineering" the most-read title on the O'Reilly platform last year and how the AI landscape has shifted since her last appearance. Chip breaks down what separates AI engineering from machine learning engineering, makes the case for a "start simple" workflow, gets candid about the real costs of running LLMs in production, and shares why she's now fascinated by physical AI, robotics, and world models and why the durable problems worth solving are increasingly human ones. Jon Krohn guides the conversation from the practical content of the book through to where the field is heading next.
Additional materials: https://www.superdatascience.com/999
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
In this episode you will learn:
(06:48) What separates AI engineering from machine learning engineering
(14:44) The “start simple” approach: prompting, then RAG, then fine-tuning
(18:19) Why web search is so painfully expensive in production
(35:11) Is the “ChatGPT moment” for physical AI really here?
(52:21) Why the durable problems left to solve are people problems
Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)
Chip Huyen is a core developer on Nvidia’s Nemo platform, a former AI researcher at Netflix, and taught machine learning at Stanford. She’s a two-time founder and the author of two widely read books on AI, including AI Engineering, which has been the most-read book on the O’Reilly platform since its launch. Unlike many AI commentators, Chip has built multiple successful AI products and platforms and works directly with enterprises on their AI strategies, giving her unique visibility into what’s actually happening inside companies building AI products.
We discuss:
1. What people think makes AI apps better vs. what actually makes AI apps better
2. What pre-training vs. post-training is, and why fine-tuning should be your last resort
3. How RLHF (reinforcement learning from human feedback) actually works
4. Why data quality matters more than which vector database you choose
5. Why high performers are seeing the most gains from AI coding tools
6. Why most AI problems are actually UX issues
—
Brought to you by:
Dscout—The UX platform to capture insights at every stage: from ideation to production: https://www.dscout.com/
Justworks—The all-in-one HR solution for managing your small business with confidence: https://www.justworks.com
Persona—A global leader in digital identity verification: https://withpersona.com/lenny
—
Where to find Chip Huyen:
• X: https://x.com/chipro
• LinkedIn: https://www.linkedin.com/in/chiphuyen/
• Website: https://huyenchip.com/
• Substack: https://substack.com/@chiphuyen
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Introduction to Chip Huyen
(04:28) Chip’s viral LinkedIn post
(07:05) Understanding AI training: pre-training vs. post-training
(08:50) Language modeling explained
(13:55) The importance of post-training
(15:20) Reinforcement learning and human feedback
(22:23) The importance of evals in AI development
(31:55) Retrieval augmented generation (RAG) explained
(38:50) Challenges in AI tool adoption
(43:19) Challenges in measuring productivity
(45:20) The three-bucket test
(49:10) The future of engineering roles
(55:31) ML Engineers vs. AI engineers
(57:12) Looking forward: the impact of AI
(01:05:48) Model capabilities vs. perceived performance
(01:08:23) Lightning round and final thoughts
—
Referenced:
• Chip’s LinkedIn post on what actually improves AI apps: https://www.linkedin.com/posts/chiphuyen_aiapplications-aiengineering-activity-7358971409227792384-y0mf/
• Prediction and Entropy of Printed English: https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf
• Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor): https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody
•Inside the expert network training every frontier AI model | Garrett Lord (Handshake CEO): https://www.lennysnewsletter.com/p/inside-handshake-garrett-lord
• First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason Droege: https://www.lennysnewsletter.com/p/first-interview-with-scale-ais-ceo-jason-droege
• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next
• Why AI evals are the hottest new skill for product builders | Hamel Husain & Shreya Shankar (creators of the #1 eval course): https://www.lennysnewsletter.com/p/why-ai-evals-are-the-hottest-new-skill
• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell
• Stanford webinar—How AI Is Changing Coding and Education, Andrew Ng & Mehran Sahami: https://www.youtube.com/watch?v=J91_npj0Nfw
• He saved OpenAI, invented the “Like” button, and built Google Maps: Bret Taylor on the future of careers, coding, agents, and more: https://www.lennysnewsletter.com/p/he-saved-openai-bret-taylor
• Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann: https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann
• Lenny’s vibe-coded app made on Lovable: https://gdoc-images-grab.lovable.app/
• Story of Yanxi Palace: https://www.imdb.com/title/tt8865016/
• Steve Jobs’s quote: https://www.goodreads.com/quotes/427317-remembering-that-i-ll-be-dead-soon-is-the-most-important
—
Recommended books:
• The Complete Sherlock Holmes: https://www.amazon.com/Complete-Sherlock-Holmes-Volumes/dp/0553328255
• AI Engineering: Building Applications with Foundation Models: https://www.amazon.com/AI-Engineering-Building-Applications-Foundation/dp/1098166302
• The Selfish Gene: https://www.amazon.com/Selfish-Gene-Anniversary-Introduction/dp/0199291152
• From Third World to First: The Singapore Story: 1965-2000: https://www.amazon.com/Third-World-First-Singapore-1965-2000/dp/0060197765
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
Lenny may be an investor in the companies discussed.
To hear more, visit www.lennysnewsletter.com
AI Engineering with Chip Huyen
Supported by Our Partners
• Swarmia — The engineering intelligence platform for modern software organizations.
• Graphite — The AI developer productivity platform.
• Vanta — Automate compliance and simplify security with Vanta.
—
On today’s episode of The Pragmatic Engineer, I’m joined by Chip Huyen, a computer scientist, author of the freshly published O’Reilly book AI Engineering, and an expert in applied machine learning. Chip has worked as a researcher at Netflix, was a core developer at NVIDIA (building NeMo, NVIDIA’s GenAI framework), and co-founded Claypot AI. She also taught Machine Learning at Stanford University.
In this conversation, we dive into the evolving field of AI Engineering and explore key insights from Chip’s book, including:
• How AI Engineering differs from Machine Learning Engineering
• Why fine-tuning is usually not a tactic you’ll want (or need) to use
• The spectrum of solutions to customer support problems – some not even involving AI!
• The challenges of LLM evals (evaluations)
• Why project-based learning is valuable—but even better when paired with structured learning
• Exciting potential use cases for AI in education and entertainment
• And more!
—
Timestamps
(00:00) Intro
(01:31) A quick overview of AI Engineering
(05:00) How Chip ensured her book stays current amidst the rapid advancements in AI
(09:50) A definition of AI Engineering and how it differs from Machine Learning Engineering
(16:30) Simple first steps in building AI applications
(22:53) An explanation of BM25 (retrieval system)
(23:43) The problems associated with fine-tuning
(27:55) Simple customer support solutions for rolling out AI thoughtfully
(33:44) Chip’s thoughts on staying focused on the problem
(35:19) The challenge in evaluating AI systems
(38:18) Use cases in evaluating AI
(41:24) The importance of prioritizing users’ needs and experience
(46:24) Common mistakes made with Gen AI
(52:12) A case for systematic problem solving
(53:13) Project-based learning vs. structured learning
(58:32) Why AI is not the end of engineering
(1:03:11) How AI is helping education and the future use cases we might see
(1:07:13) Rapid fire round
—
The Pragmatic Engineer deepdives relevant for this episode:
• Applied AI Software Engineering: RAG https://newsletter.pragmaticengineer.com/p/rag
• How do AI software engineering agents work? https://newsletter.pragmaticengineer.com/p/ai-coding-agents
• AI Tooling for Software Engineers in 2024: Reality Check https://newsletter.pragmaticengineer.com/p/ai-tooling-2024
• IDEs with GenAI features that Software Engineers love https://newsletter.pragmaticengineer.com/p/ide-that-software-engineers-love
—
See the transcript and other references from the episode at https://newsletter.pragmaticengineer.com/podcast
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com.
Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe
AI Engineering Pitfalls with Chip Huyen - #715
Today, we're joined by Chip Huyen, independent researcher and writer to discuss her new book, “AI Engineering.” We dig into the definition of AI engineering, its key differences from traditional machine learning engineering, the common pitfalls encountered in engineering AI systems, and strategies to overcome them. We also explore how Chip defines AI agents, their current limitations and capabilities, and the critical role of effective planning and tool utilization in these systems. Additionally, Chip shares insights on the importance of evaluation in AI systems, highlighting the need for systematic processes, human oversight, and rigorous metrics and benchmarks. Finally, we touch on the impact of open-source models, the potential of synthetic data, and Chip’s predictions for the year ahead.
The complete show notes for this episode can be found at https://twimlai.com/go/715.
661: Designing Machine Learning Systems
Chip Huyen, co-founder of Claypot AI and author of O'Reilly's best-selling "Designing Machine Learning Systems" is here to share her expertise on designing production-ready machine learning applications, the importance of iteration in real-world deployment, and the critical role of real-time machine learning in various applications. Technical listeners like data scientists and machine learning engineers will definitely enjoy this one!
This episode is brought to you by Pathway, the reactive data processing framework (pathway.com), and by epic LinkedIn Learning instructor Keith McCormick (linkedin.com/learning/instructors/keith-mccormick). Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.
In this episode you will learn:
• Why Chip wrote 'Designing Machine Learning Systems' [08:58]
• How Chip ended up teaching at Stanford [13:18]
• About Chip's book 'Designing Machine Learning Systems' [21:12]
• What makes ML feel like magic [30:53]
• How to align business intent, context, and metrics with ML [37:55]
• The lessons Chip learned about training data [42:03]
• Chip's secrets to engineering good features [53:19]
• How Chip optimizes her productivity [1:07:48]
Additional materials: www.superdatascience.com/661
Chip Huyen — ML Research and Production Pipelines
Chip Huyen is a writer and computer scientist currently working at a startup that focuses on machine learning production pipelines. Previously, she’s worked at NVIDIA, Netflix, and Primer. She helped launch Coc Coc - Vietnam’s second most popular web browser with 20+ million monthly active users. Before all of that, she was a best selling author and traveled the world.
Chip graduated from Stanford, where she created and taught the course on TensorFlow for Deep Learning Research.
Check out Chip's recent article on ML Tools: https://huyenchip.com/2020/06/22/mlops.html
Follow Chip on Twitter: https://twitter.com/chipro
And on her Website: https://huyenchip.com/
Visit our podcasts homepage for transcripts and more episodes!
www.wandb.com/podcast
🔊 Get our podcast on Apple and Spotify!
Apple Podcasts: https://bit.ly/2WdrUvI
Spotify: https://bit.ly/2SqtadF
We started Weights and Biases to build tools for Machine Learning practitioners because we care a lot about the impact that Machine Learning can have in the world and we love working in the trenches with the people building these models. One of the most fun things about these building tools has been the conversations with these ML practitioners and learning about the interesting things they’re working on. This process has been so fun that we wanted to open it up to the world in the form of our new podcast called Gradient Dissent. We hope you have as much fun listening to it as we had making it!
👩🏼🚀Weights and Biases:
We’re always free for academics and open source projects. Email carey@wandb.com with any questions or feature suggestions.
- Blog: https://www.wandb.com/articles
- Gallery: See what you can create with W&B - https://app.wandb.ai/gallery
- Continue the conversation on our slack community - http://bit.ly/wandb-forum
🎙Host: Lukas Biewald - https://twitter.com/l2k
👩🏼💻Producer: Lavanya Shukla - https://twitter.com/lavanyaai
📹Editor: Cayla Sharp - http://caylasharp.com/