Baseten CEO Tuhin Srivastava on the AI Inference Crunch, Custom Models, and Building the Inference Cloud
Baseten CEO and co-founder Tuhin Srivastava sits down with Sarah Guo and Elad Gil to discuss the rapid growth of AI inference demand, Baseten’s 30x growth, and why inference is becoming the strategic “last market.” Tuhin Srivastava argues the application layer will persist because companies with unique user signals can encode value into workflows and post-train specialized models, citing examples like Abridge and support workflows. The conversation covers GPU capacity constraints, Baseten’s multi-cloud fabric across 18 clouds and 90 clusters, long-term contracting dynamics, the importance of the software layer for stickiness, evolving workloads, multichip possibilities, and operational lessons at scale.
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Tuhinone
Chapters:
00:31 Baseten growth
01:55 Why the app layer wins
05:57 Serving frontier customers
07:55 Open source model mix
09:21 Chinese models and geopolitics
13:07 Custom inference dominates
14:22 Post training acquisition
17:10 When to invest in custom models
18:35 Supply crunch and data centerse
22:25 Longer GPU Contracts
24:09 What Makes a Winner
26:07 Multi Chip Future
28:19 Runtime Roadmap
31:08 Scaling Edge Cases
33:48 Hiring and Leadership
36:44 Operations Pager Culture
38:19 Efficiency Drives Demand
40:41 Concierge Everything Future
42:34 Conclusion
The CEO Behind the Fastest-Growing AI Inference Company | Tuhin Srivastava
In this episode of Gradient Dissent, Lukas Biewald talks with Tuhin Srivastava, CEO and founder of Baseten, one of the fastest-growing companies in the AI inference ecosystem. Tuhin shares the real story behind Baseten’s rise and how the market finally aligned with the infrastructure they’d spent years building.
They get into the core challenges of modern inference, including why dedicated deployments matter, how runtime and infrastructure bottlenecks stack up, and what makes serving large models fundamentally different from smaller ones.
Tuhin also explains how vLLM, TensorRT-LLM, and SGLang differ in practice, what it takes to tune workloads for new chips like the B200, and why reliability becomes harder as systems scale.
The conversation dives into company-building, from killing product lines to avoiding premature scaling while navigating a market that shifts every few weeks.
Connect with us here:
Tuhin Srivastva: https://www.linkedin.com/in/tuhin-srivastava/
Lukas Biewald: https://www.linkedin.com/in/lbiewald/
Weights & Biases: https://www.linkedin.com/company/wandb/
The server-side rendering equivalent for LLM inference workloads
Ryan is joined by Tuhin Srivastava, CEO and co-founder of Baseten, to explore the evolving landscape of AI infrastructure and inference workloads, how the shift from traditional machine learning models to large-scale neural networks has made GPU usage challenging, and the potential future of hardware-specific optimizations in AI.
Episode notes:
Baseten is an AI infrastructure platform giving you the tooling, expertise, and hardware needed to bring AI products to market fast.
Connect with Tuhin on LinkedIn or reach him at his email tuhin@baseten.co.
Shoutout to user Hitesh for winning a Populist badge for their answer to Cannot drop database because it is currently in use.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Speed will win the AI computing battle with Tuhin Srivastava from Baseten
At a time when users are being asked to wait unthinkable seconds for AI products to generate art and answers, speed is what will win the battle heating up in AI computing. At least according to today’s guest, Tuhin Srivastava, the CEO and co-founder of Baseten which gives customers scalable AI infrastructures starting with interference. In this episode of No Priors, Sarah, Elad, and Tuhin discuss why efficient code solutions are more desirable than no code, the most surprising use cases for Baseten, and why all of their jobs are very defensible from AI.
Show Links:
Baseten
Benchmarking fast Mistral 7B inference
Sign up for new podcasts every week. Email feedback to show@no-priors.com
Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @tuhinone
Show Notes:
(0:00) Introduction
(1:19) Capabilities of efficient code enabled development
(4:11) Difference in training inference workloads
(6:12) AI product acceleration
(8:48) Leading on inference benchmarks at Baseten
(12:08) Optimizations for different types of models
(16:11) Internal vs open source models
(19:01) timeline for enterprise scale
(21:53) Rethinking investment in compute spend
(27:50) Defensibility in AI industries
(31:30) Hardware and the chip shortage
(35:47) Speed is the way to win in this industry
(38:26) Wrap
Self-hosting & scaling models
We’re excited to have Tuhin join us on the show once again to talk about self-hosting open access models. Tuhin’s company Baseten specializes in model deployment and monitoring at any scale, and it was a privilege to talk with him about the trends he is seeing in both tooling and usage of open access models. We were able to touch on the common use cases for integrating self-hosted models and how the boom in generative AI has influenced that ecosystem.
Sponsors:
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
Typesense – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster!
Featuring:
Tuhin Srivastava – GitHub, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X
Show Notes:
Baseten
Upcoming Events:
Register for upcoming webinars here!
The fastest way to build ML-powered apps
Tuhin Srivastava tells Daniel and Chris why BaseTen is the application development toolkit for data scientists. BaseTen’s goal is to make it simple to serve machine learning models, write custom business logic around them, and expose those through API endpoints without configuring any infrastructure.
Sponsors:
O'Reilly Media – Learn by doing — Python, data, AI, machine learning, Kubernetes, Docker, and more. Just open your browser and dive in. Learn more and keep your teams’ skills sharp at oreilly.com/changelog
Snowplow Analytics – The behavioral data management platform powering your data journey. Capture and process high-quality behavioral data from all your platforms and products and deliver that data to your cloud destination of choice. Get started and experience Snowplow data for yourself at snowplowanalytics.com
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.
Featuring:
Tuhin Srivastava – GitHub, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X
Show Notes:
BaseTen
BaseTen | LinkedIn
BaseTen | Twitter
BaseTen | Blog
Upcoming Events:
Register for upcoming webinars here!