The RL Fine-Tuning Playbook: CoreWeave's Kyle Corbitt on GRPO, Rubrics, Environments, Reward Hacking
Kyle Corbitt, founder of OpenPipe, breaks down reinforcement learning and custom fine-tuning for modern AI models. He explains how RL differs from supervised fine-tuning, why GRPO and LLM-as-judge post-training matter, and how these techniques can improve performance, latency, and cost on open source models. The conversation also covers reward hacking, evaluation design, LoRA adapters, and how Chinese labs are using distillation to fast-follow frontier models.
Sponsors:
Sequence:
Sequence handles the full revenue workflow for complex pricing, from quoting and metering to invoicing, revenue recognition, and collections. Book a public demo at https://sequencehq.com and use code Cognizant in the source field to save 20% off year one
AvePoint:
AvePoint is building the control layer for AI agents so you can securely govern, audit, and recover every action at scale. Design trusted agentic outcomes from day one at https://avpt.co/tcr
VCX:
VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
Why RL Won — Kyle Corbitt, OpenPipe (acq. CoreWeave)
In this deep dive with Kyle Corbitt, co-founder and CEO of OpenPipe (recently acquired by CoreWeave), we explore the evolution of fine-tuning in the age of AI agents and the critical shift from supervised fine-tuning to reinforcement learning. Kyle shares his journey from leading YC’s Startup School to building OpenPipe, initially focused on distilling expensive GPT-4 workflows into smaller, cheaper models before pivoting to RL-based agent training as frontier model prices plummeted. The conversation reveals why 90% of AI projects remain stuck in proof-of-concept purgatory - not due to capability limitations, but reliability issues that Kyle believes can be solved through continuous learning from real-world experience. He discusses the breakthrough of RULER (Relative Universal Reinforcement Learning Elicited Rewards), which uses LLMs as judges to rank agent behaviors relatively rather than absolutely, making RL training accessible without complex reward engineering. Kyle candidly assesses the challenges of building realistic training environments for agents, explaining why GRPO (despite its advantages) may be a dead end due to its requirement for perfectly reproducible parallel rollouts. He shares insights on why LoRAs remain underrated for production deployments, why GEPA and prompt optimization haven’t lived up to the hype in his testing, and why the hardest part of deploying agents isn’t the AI - it’s sandboxing real-world systems with all their bugs and edge cases intact. The discussion also covers OpenPipe’s acquisition by CoreWeave, the launch of their serverless reinforcement learning platform, and Kyle’s vision for a future where every deployed agent continuously learns from production experience. He predicts that solving the reliability problem through continuous RL could unlock 10x more AI inference demand from projects currently stuck in development, fundamentally changing how we think about agent deployment and maintenance.
Key Topics:
* The rise and fall of fine-tuning as a business model
* Why 90% of AI projects never reach production
* RULER: Making RL accessible through relative ranking
* The environment problem: Why sandboxing is harder than training
* GRPO vs PPO and the future of RL algorithms
* LoRAs: The underrated deployment optimization
* Why GEPA and prompt optimization disappointed in practice
* Building world models as synthetic training environments
* The $500B Stargate bet and OpenAI’s potential crypto play
* Continuous learning as the path to reliable agents
References
https://www.linkedin.com/in/kcorbitt/
* Aug 2023 https://openpipe.ai/blog/from-prompts-to-models
* DEC 2023 https://openpipe.ai/blog/mistral-7b-fine-tune-optimized
* JAN 2024 https://openpipe.ai/blog/s-lora
* MAY 2024 https://openpipe.ai/blog/the-ten-commandments-of-fine-tuning-in-prod
* Oct 2024 https://openpipe.ai/blog/announcing-dpo-support
* AIE NYC 2025 Finetuning 500m agents
* AIEWF 2025 How to train your agent (ART-E)
* SEPT 2025 ACQUISTION https://openpipe.ai/blog/openpipe-coreweave
* W&B Serverless RL https://openpipe.ai/blog/serverless-rl?refresh=1760042248153
Full Video Episode
Timestamps
00:00 Introductions
03:15 The Evolution of OpenPipe: From SFT to RL
07:49 The Mistral Era and LoRA Adapters
11:40 When You Actually Need Fine-Tuning
14:43 The Pivot to Reinforcement Learning
21:29 GRPO vs PPO: The Technical Trade-offs
24:02 The Environment Problem in RL
35:52 JAPA and Automated Prompt Optimization
44:35 Open vs Closed Models: The Token Economics
50:38 Ruler: Self-Supervised RL Rewards
57:09 World Models as Environment Solutions
1:00:15 CoreWeave Acquisition and Future Vision
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe