MLOps is NOT Real
We all hear a lot about MLOps these days, but where does MLOps end and DevOps begin? Our friend Luis from OctoML joins us in this episode to discuss treating AI/ML models as regular software components (once they are trained and ready for deployment). We get into topics including optimization on various kinds of hardware and deployment of models at the edge.
Sponsors:
Changelog++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with extended episodes, make the ads disappear, and increment your audio quality with higher bitrate mp3s. Let’s do this!
Ship It! – A podcast about getting your best ideas into the world and seeing what happens. Listen to an episode that interests you and subscribe today.
Featuring:
Luis Ceze – LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X
Show Notes:
OctoML
Upcoming Events:
Register for upcoming webinars here!
Luis Ceze — Accelerating Machine Learning Systems
From Apache TVM to OctoML, Luis gives direct insight into the world of ML hardware optimization, and where systems optimization is heading.
---
Luis Ceze is co-founder and CEO of OctoML, co-author of the Apache TVM Project, and Professor of Computer Science and Engineering at the University of Washington. His research focuses on the intersection of computer architecture, programming languages, machine learning, and molecular biology.
Connect with Luis:
📍 Twitter: https://twitter.com/luisceze
📍 University of Washington profile: https://homes.cs.washington.edu/~luisceze/
---
⏳ Timestamps:
0:00 Intro and sneak peek
0:59 What is TVM?
8:57 Freedom of choice in software and hardware stacks
15:53 How new libraries can improve system performance
20:10 Trade-offs between efficiency and complexity
24:35 Specialized instructions
26:34 The future of hardware design and research
30:03 Where does architecture and research go from here?
30:56 The environmental impact of efficiency
32:49 Optimizing and trade-offs
37:54 What is OctoML and the Octomizer?
42:31 Automating systems design with and for ML
44:18 ML and molecular biology
46:09 The challenges of deployment and post-deployment
🌟 Transcript: http://wandb.me/gd-luis-ceze 🌟
Links:
1. OctoML: https://octoml.ai/
2. Apache TVM: https://tvm.apache.org/
3. "Scalable and Intelligent Learning Systems" (Chen, 2019): https://digital.lib.washington.edu/researchworks/handle/1773/44766
4. "Principled Optimization Of Dynamic Neural Networks" (Roesch, 2020): https://digital.lib.washington.edu/researchworks/handle/1773/46765
5. "Cross-Stack Co-Design for Efficient and Adaptable Hardware Acceleration" (Moreau, 2018): https://digital.lib.washington.edu/researchworks/handle/1773/43349
6. "TVM: An Automated End-to-End Optimizing Compiler for Deep Learning" (Chen et al., 2018): https://www.usenix.org/system/files/osdi18-chen.pdf
7. Porcupine is a molecular tagging system introduced in "Rapid and robust assembly and decoding of molecular tags with DNA-based nanopore signatures" (Doroschak et al., 2020): https://www.nature.com/articles/s41467-020-19151-8
---
Get our podcast on these platforms:
👉 Apple Podcasts: http://wandb.me/apple-podcasts
👉 Spotify: http://wandb.me/spotify
👉 Google Podcasts: http://wandb.me/google-podcasts
👉 YouTube: http://wandb.me/youtube
👉 Soundcloud: http://wandb.me/soundcloud
Join our community of ML practitioners where we host AMAs, share interesting projects and meet other people working in Deep Learning:
http://wandb.me/slack
Check out Fully Connected, which features curated machine learning reports by researchers exploring deep learning techniques, Kagglers showcasing winning models, industry leaders sharing best practices, and more:
https://wandb.ai/fully-connected
Apache TVM and OctoML
90% of AI / ML applications never make it to market, because fine tuning models for maximum performance across disparate ML software solutions and hardware backends requires a ton of manual labor and is cost-prohibitive. Luis Ceze and his team created Apache TVM at the University of Washington, then left founded OctoML to bring the project to market.
Sponsors:
O'Reilly Media – Learn by doing — Python, data, AI, machine learning, Kubernetes, Docker, and more. Just open your browser and dive in. Learn more and keep your teams’ skills sharp at oreilly.com/changelog
Snowplow Analytics – The behavioral data management platform powering your data journey. Capture and process high-quality behavioral data from all your platforms and products and deliver that data to your cloud destination of choice. Get started and experience Snowplow data for yourself at snowplowanalytics.com
Changelog++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this!
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Featuring:
Luis Ceze – LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X
Show Notes:
Apache TVM
OctoML
OctoML: Automated Deep Learning Engineering with Jason Knight and Luis Ceze
Upcoming Events:
Register for upcoming webinars here!