40 Trillion Tokens a Day (Yes, More Than OpenAI) | Lin Qiao, CEO of Fireworks
Should American companies be worried about Chinese open-source AI models?
Lin Qiao, CEO of Fireworks, doesn't think so.
In this episode, she joins Lukas Biewald to talk about why she believes the industry is at a turning point, one that calls for more open intelligence, not less.
They cover how Fireworks now processes more tokens a day than OpenAI's API, why she thinks the future belongs to specialized models built on private company data rather than general-purpose ones, and why she believes OpenAI and Anthropic should be open-sourcing their own models too.
Connect with us here:
Lin Qiao
Fireworks
Lukas Biewald
Weights and Biases
20VC: Are OpenAI and Anthropic Overvalued? The Open-Source AI Reality | How Token Costs Will Fall 10x And Usage Will Explode 100x | The Future Is Not One AGI; It's Millions of Specialised Models with Lin Qiao, Founder and CEO @ Fireworks
Lin Qiao is the Co-Founder and CEO of Fireworks AI, the leading specialized intelligence and AI inference platform that last week raised $1.5BN at a whopping $17BN valuation. With just 200 people, the company has hit $1BN in ARR and expects to hit $2BN before the end of the year. Prior to Fireworks, Lin spent several years at Meta including on the founding team of PyTorch.
AGENDA:
00:07 — Why Did Fireworks Bet on Inference When Everyone Else Was Chasing Training?
00:13 — Can Open-Source Models Turn AI Infrastructure into a Commodity?
00:19 — Should Enterprises Trust Chinese Open Models With Their Most Sensitive Data?
00:25 — Will Model Progress Keep Moving This Fast—or Are We Nearing a Plateau?
00:28 — Will the Multi-Model World Create a $100BN Routing Layer?
00:37 — How Much Will AI Token Usage Explode Over the Next Two Years?
00:43 — Will Token Costs Fall 10x—and Unleash 100x More Demand?
00:49 — Does Fireworks Eventually Have to Build Its Own Data Centres?
01:02 — What Is the Real Bottleneck Holding Back the AI Economy?
The good, the bad, and the AI apps
Ryan welcomes Benny Chen, co-founder of Fireworks AI, to the show to explore what actually makes an AI application good or not, how to balance qualitative signals with quantitative metrics when evaluating AI, and how open-source eval protocols and community efforts are setting the standard for AI evaluation.
Episode notes:
Fireworks AI is a cloud platform designed for developers and enterprises to run, customize, and scale open-source generative AI models.
Connect with Benny on LinkedIn.
Congrats to user techtabu for winning a Stellar Answer badge for answering How can I delete all local Docker images?.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
How Cursor Trained Composer on Fireworks: Distributed Infrastructure for High-Performance RL
Cursor's Federico Cassano and Fireworks' Dmytro Dzhulgakov explain how they collaborated to build Composer as a specialized foundation model. The core insight: models have finite capacity in their weights, and allocating all those bits to the singular task of software engineering in Cursor frees the model to be both better at the task and far more efficient at inference. Rather than start from pre-training and work up, they took an unconventional top-down approach — mid-training and RL on top of an open-source base to get a useful model into users' hands fast, then specializing the model around real Cursor usage. With Fireworks providing distributed infrastructure, Composer delivers frontier-class coding performance with the speed of a much smaller model.
Hosted by Sonya Huang, Sequoia Capital
982: In Case You Missed It in March 2026
Jon Krohn rounds up March’s interviews in this ICYMI episode. Hear from AI and data science experts across the fields of education and business in this wide-ranging series of clips that take listeners from the Renaissance to the near future. Guests include Lin Quiao (Episode 971), Chris Fregly (Episode 973), Zack Kass (Episode 975), Kyunghyun Cho (Episode 977), and Rohit Choudhary (Episode 979).
Additional materials: www.superdatascience.com/982
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
971: 90% of The World’s Data is Private; Lin Qiao’s Fireworks AI is Unlocking It
Lin Qiao, CEO of Fireworks AI, talks to Jon Krohn about how she builds effective models quickly, why coding agents can perform at the level of a junior engineer, and what she attributes to the success of Fireworks AI: True to its name, the company exploded into the AI industry with over $300 million secured in venture capital, as well as netting a further $250 million Series C funding. For Lin, many enterprises miss out by not being familiar with open models. Open models give a lot of control to the user, offering customizability and at a much lower price point. Listen to hear how Fireworks AI helps companies continue to save money through AI.
This episode is brought to you by the Dell, by Intel, by Cisco and by Acceldata.
Additional materials: www.superdatascience.com/971
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
In this episode you will learn:
(01:19) All about Fireworks AI
(24:16) Why companies need to take notice of open models
(33:05) The commercial viability of slow-reasoning models
(38:51) Fireworks AI’s approach to model performance evaluations
Closing the Loop Between AI Training and Inference with Lin Qiao - #742
In this episode, we're joined by Lin Qiao, CEO and co-founder of Fireworks AI. Drawing on key lessons from her time building PyTorch, Lin shares her perspective on the modern generative AI development lifecycle. She explains why aligning training and inference systems is essential for creating a seamless, fast-moving production pipeline, preventing the friction that often stalls deployment. We explore the strategic shift from treating models as commodities to viewing them as core product assets. Lin details how post-training methods, like reinforcement fine-tuning (RFT), allow teams to leverage their own proprietary data to continuously improve these assets. Lin also breaks down the complex challenge of what she calls "3D optimization"—balancing cost, latency, and quality—and emphasizes the role of clear evaluation criteria to guide this process, moving beyond unreliable methods like "vibe checking." Finally, we discuss the path toward the future of AI development: designing a closed-loop system for automated model improvement, a vision made more attainable by the exciting convergence of open and closed-source model capabilities.
The complete show notes for this episode can be found at https://twimlai.com/go/742.
Fireworks Founder Lin Qiao on How Fast Inference and Small Models Will Benefit Businesses
In the first wave of the generative AI revolution, startups and enterprises built on top of the best closed-source models available, mostly from OpenAI. The AI customer journey moves from training to inference, and as these first products find PMF, many are hitting a wall on latency and cost.
Fireworks Founder and CEO Lin Qiao led the PyTorch team at Meta that rebuilt the whole stack to meet the complex needs of the world’s largest B2C company. Meta moved PyTorch to its own non-profit foundation in 2022 and Lin started Fireworks with the mission to compress the timeframe of training and inference and democratize access to GenAI beyond the hyperscalers to let a diversity of AI applications thrive.
Lin predicts when open and closed source models will converge and reveals her goal to build simple API access to the totality of knowledge.
Hosted by: Sonya Huang and Pat Grady, Sequoia Capital
Mentioned in this episode:
Pytorch: the leading framework for building deep learning models, originated at Meta and now part of the Linux Foundation umbrella
Caffe2 and ONNX: ML frameworks Meta used that PyTorch eventually replaced
Conservation of complexity: the idea that that every computer application has inherent complexity that cannot be reduced but merely moved between the backend and frontend, originated by Xerox PARC researcher Larry Tesler
Mixture of Experts: a class of transformer models that route requests between different subsets of a model based on use case
Fathom: a product the Fireworks team uses for video conference summarization
LMSYS Chatbot Arena: crowdsourced open platform for LLM evals hosted on Hugging Face
00:00 - Introduction
02:01 - What is Fireworks?
02:48 - Leading Pytorch
05:01 - What do researchers like about PyTorch?
07:50 - How Fireworks compares to open source
10:38 - Simplicity scales
12:51 - From training to inference
17:46 - Will open and closed source converge?
22:18 - Can you match OpenAI on the Fireworks stack?
26:53 - What is your vision for the Fireworks platform?
31:17 - Competition for Nvidia?
32:47 - Are returns to scale starting to slow down?
34:28 - Competition
36:32 - Lightning round
AI Inference: Good, Fast, and Cheap, with Lin Qiao & Dmytro Ivchenko of Fireworks AI
In this episode, we delve into the intricate world of AI inference with cofounders of Firework AI. Discover the strategies behind optimizing AI performance, the importance of balancing latency and throughput, and the nuances of different AI architectures from GPT-3 to Stable Diffusion. Learn about their partnership with Stability AI, their unique focus on reducing total cost of ownership, and their vision for a seamless developer experience.
RECOMMENDED PODCAST:
How Do You Use ChatGPT with Dan Shipper.
Dan Shipper talks to programmers, writers, founders, academics, tech executives, and others to walk through all of their ChatGPT use cases (including Nathan!). They even use ChatGPT together, live on the show. Listen to How Do You Use ChatGPT? from Dan Shipper and the team at Every, wherever you get your podcasts: https://link.chtbl.com/hdyuchatgpt
SPONSORS:
Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR
Plumb is a no-code AI app builder designed for product teams who care about quality and speed. What is taking you weeks to hand-code today can be done confidently in hours. Check out https://bit.ly/PlumbTCR for early access.
Head to Squad to access global engineering without the headache and at a fraction of the cost: head to https://choosesquad.com/ and mention “Turpentine” to skip the waitlist.
CHAPTERS :
(00:00:00) Introduction
(00:08:34) Compute Stack
(00:19:23) Fireworks Product Philosophy
(00:24:11) Sponsors : Brave / Omneky
(00:25:40) Fine-tuning Strategy
(00:38:40) Sponsors : Plumb / Squad
(00:40:33) NVIDIA Stack Overview
(00:47:14) TensorFlow Triton Service
(00:55:25) Reduced Precision Advantages
(01:03:57) Different Deployment Scenarios
(01:08:27) Seeking Intuition on Sharding
(01:19:28) Announcing Stability AI Partnership
(01:32:00) Closing Remarks
