Building Voice AI Agents That Don’t Suck with Kwindla Kramer - #739
In this episode, Kwindla Kramer, co-founder and CEO of Daily and creator of the open source Pipecat framework, joins us to discuss the architecture and challenges of building real-time, production-ready conversational voice AI. Kwin breaks down the full stack for voice agents—from the models and APIs to the critical orchestration layer that manages the complexities of multi-turn conversations. We explore why many production systems favor a modular, multi-model approach over the end-to-end models demonstrated by large AI labs, and how this impacts everything from latency and cost to observability and evaluation. Kwin also digs into the core challenges of interruption handling, turn-taking, and creating truly natural conversational dynamics, and how to overcome them. We discuss use cases, thoughts on where the technology is headed, the move toward hybrid edge-cloud pipelines, and the exciting future of real-time video avatars, and much more.
The complete show notes for this episode can be found at https://twimlai.com/go/739.
[AIEWF Preview] Gemini in 2025 and Realtime Voice AI
As part of our AI Engineer World’s Fair preview, we’re releasing a special cross podcast recorded with Sam Charrington of TWiML AI at last week’s Google I/O!
TUESDAY: Shrestha and Kwindla’s workshop: https://www.ai.engineer/schedule#milliseconds-to-magic-real-time-workflows-using-the-gemini-live-api-and-pipecat
TUESDAY: Kwindla’s workshop: https://www.ai.engineer/schedule#building-voice-agents-with-gemini-and-pipecat
WEDNESDAY: Shrestha and Kwindla’s talk: https://www.ai.engineer/schedule#milliseconds-to-magic-real-time-workflows-using-the-gemini-live-api-and-pipecat
WEDNESDAY: Kwindla’s keynote: https://www.ai.engineer/schedule#-voice-keynote-your-realtime-ai-is-ngmi
THURSDAY: Logan’s keynote: https://www.ai.engineer/schedule#a-year-of-gemini-progress-what-comes-next
Catch all the speakers at AIE (both workshops and talks):
Logan Kilpatrick: https://www.latent.space/p/chatgpt-gpt4-hype-and-building-llm
Shrestha Basu Mallick: https://www.linkedin.com/in/shresthabm/
Kwindla Hultman Kramer: https://www.linkedin.com/in/kwkramer
Full Video Episode
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe
Google I/O 2025 Special Edition - #733
Today, I’m excited to share a special crossover edition of the podcast recorded live from Google I/O 2025! In this episode, I join Shawn Wang aka Swyx from the Latent Space Podcast, to interview Logan Kilpatrick and Shrestha Basu Mallick, PMs at Google DeepMind working on AI Studio and the Gemini API, along with Kwindla Kramer, CEO of Daily and creator of the Pipecat open source project. We cover all the highlights from the event, including enhancements to the Gemini models like thinking budgets and thought summaries, native audio output for expressive voice AI, and the new URL Context tool for research agents. The discussion also digs into the Gemini Live API, covering its architecture, the challenges of building real-time voice applications (such as latency and voice activity detection), and new features like proactive audio and asynchronous function calling. Finally, don’t miss our guests’ wish lists for next year’s I/O!
The complete show notes for this episode can be found at https://twimlai.com/go/733.
At the Head of Community Development with Wesley Faulkner
About Wesley
Wesley Faulkner is a first-generation American, public speaker, and podcaster. He is a founding member of the government transparency group Open Austin and a staunch supporter of racial justice, workplace equity, and neurodiversity. His professional experience spans technology from AMD, Atlassian, Dell, IBM, and MongoDB. Wesley currently works as a Developer Advocate, and in addition, co-hosts the developer relations focused podcast Community Pulse and serves on the board for SXSW.
Links Referenced:
Twitter: https://twitter.com/wesley83
Polywork: https://polywork.com/wesley83
Personal Website: https://www.wesleyfaulkner.com/
Neurodiversity as an Advantage with Wesley Faulkner
About Wesley
Wesley Faulkner is a first-generation American. He is a founding member of the government transparency group Open Austin and ran for Austin City Council in 2016. His professional experience also includes work as a social media and community manager for the software company Atlassian, and various roles for the computer processor company AMD, Dell, and IBM. Wesley Faulkner serves as a board member for South by Southwest Interactive (SXSWi) and is a Developer Advocate for Daily.
Links:
Daily website: https://www.daily.co/
Twitter: https://twitter.com/wesley83