The fastest agent in the race has the best evals
Ryan welcomes Benjamin Klieger, lead engineer at Groq, to explore the infrastructure behind AI agents, how you can turn a one-minute agent into a ten-second agent, and how they used fast inference and effective evals to build their efficient and reliable Compound agent.
Episode notes:
Groq delivers fast, low-cost inference using their custom-designed LPU, the first chip built for inference. Check out their agent, Compound, which can search the web and run code.
Connect with Benjamin on LinkedIn and X.
Congrats to user Bart Kiers for winning a Stellar Answer badge on their response to Regular expression to match a line that doesn't contain a word.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
20VC: OpenAI and Anthropic Will Build Their Own Chips | NVIDIA Will Be Worth $10TRN | How to Solve the Energy Required for AI... Nuclear | Why China is Behind the US in the Race for AGI with Jonathan Ross, Groq Founder
Jonathan Ross is the Founder & CEO of Groq, the AI chip company redefining inference at scale. Under his leadership, Groq has raised over $3B from top investors. The company has reached a valuation of nearly $7B, positioning itself as one of NVIDIA's most formidable challengers. Previously at Google, Jonathan led the team that built the first Tensor Processing Unit (TPU), making him one of the leading architects of modern AI hardware.
AGENDA:
00:00 The Future of AI and Compute
05:07 Why the Hyperscalers Have to Keep Spending Recklessly on AI
12:49 Why OpenAI and Anthropic Will Have to Build Their Own Chips
19:47 OpenAI and Anthropic Will be $5BN Companies: The Bull Case
29:50 Why China is Behind the US in AI and Deepseek is More Expensive to Run
33:55 How Europe Could Compete in AI and Why the US is More Risk Averse Than Europe
42:51 Why AI Will Lead to Too Many Not Too Few Jobs
43:19 Deflationary Pressures and New Job Markets
45:51 The Future of Vibe Coding
47:14 Why AI Companies Should Strive to Have Low Margins
49:31 Why We Have to Have Nuclear Energy and How to Bring it Back
56:55 How Permits are Ruining the Potential of AI
01:03:58 Why OpenAI and Anthropic are so Undervalued
01:16:53 Quickfire: Biggest Fear, Nvidia: $10TRN, Zuck Buying AI: Work or Not
Software and hardware acceleration with Groq
How do you enable AI acceleration (at both the hardware and software layers) that stays ahead of rapid industry shifts? In this episode, Dhananjay Singh from Groq dives into the evolving landscape of AI inference and acceleration. We explore how Groq optimizes the serving layer, adapts to industry shifts, and supports emerging model architectures.
Featuring:
Dhananjay Singh – LinkedIn, X
Chris Benson – Website, GitHub, LinkedIn, X
Daniel Whitenack – Website, GitHub, X
Links:
Groq
Sponsors:
Augment Code - Developer AI that uses deep understanding of your large codebase and how you build software to deliver personalized code suggestions and insights. Augment provides relevant, contextualized code right in your IDE or Slack. It transforms scattered knowledge into code or answers, eliminating time spent searching docs or interrupting teammates.
20VC: NVIDIA vs Groq: The Future of Training vs Inference | Meta, Google, and Microsoft's Data Center Investments: Who Wins | Data, Compute, Models: The Core Bottlenecks in AI & Where Value Will Distribute with Jonathan Ross, Founder @ Groq
Jonathan Ross is the Founder & CEO of Groq, the creator of the world's first Language Processing Unit (LPUTM). Prior to Groq, Jonathan began what became Google's Tensor Processing Unit (TPU) as a 20% project where he designed and implemented the core elements of the first-generation TPU chip. Jonathan next joined Google X's Rapid Eval Team, the initial stage of the famed "Moonshots Factory", where he devised and incubated new Bets (Units) for Google's parent company, Alphabet.
In Today's Episode We Discuss:
04:20 Interview with Jonathan Ross Begins
04:59 Scaling Laws and AI Model Training
06:22 Synthetic Data and Model Efficiency
12:01 Inference vs. Training Costs: Why NVIDIA Loses Inference
17:06 The Future of AI Inference: Efficiency and Cost
18:15 Chip Supply and Scaling Concerns
20:57 Energy Efficiency in AI Computation
25:40 Why Most Dollars Into Datacenters Will Be Lost
31:05 Meta, Google, and Microsoft's Data Center Investments
41:11 Distribution of Value in the AI Economy
42:10 Stages of Startup Success
43:17 The AI Investment Bubble
45:00 The Keynesian Beauty Contest in VC
48:40 NVIDIA's Role in the AI Ecosystem
53:39 China's AI Strategy and Global Implications
57:51 Europe's Potential in the AI Revolution
01:10:14 Future Predictions and AI's Impact on Society
20VC: Deepseek Special: Is Deepseek a Weapon of the CCP | How Should OpenAI and the US Government Respond | Why $500BN for Stargate is Not Enough | The Future of Inference, NVIDIA and Foundation Models with Jonathan Ross @ Groq
Jonathan Ross is the Co-Founder and CEO of Groq, providing fast AI inference. Prior to founding Groq, Jonathan started Google's TPU effort where he designed and implemented the core elements of the original chip. Jonathan then joined Google X's Rapid Eval Team, the initial stage of the famed "Moonshots factory," where he devised and incubated new Bets (Units) for Alphabet.
The 10 Most Important Questions on Deepseek:
How did Deepseek innovate in a way that no other model provider has done?
Do we believe that they only spent $6M to train R1?
Should we doubt their claims on limited H100 usage? Is Josh Kushner right that this is a potential violation of US export laws?
Is Deepseek an instrument used by the CCP to acquire US consumer data?
How does Deepseek being open-source change the nature of this discussion?
What should OpenAI do now? What should they not do?
Does Deepseek hurt or help Meta who already have their open-source efforts with Lama?
Will this market follow Satya Nadella's suggestion of Jevon's Paradox?
How much more efficient will foundation models become?
What does this mean for the $500BN Stargate project announced last week?
Sora Disappoints, ChatGPT Pro Tested, Inference Time Reasoning & More with Sunny Madra | E2062
Timestamps:
(0:00) Jason and Sunny kick off the show
(1:26) Discussing Groq's recent developments and Middle East tech inspiration
(3:51) Global business challenges and Jason's 2025 announcements
(7:53) Gemini app demo and ChatGPT 4 comparison
(9:57) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
(11:24) Evaluating ChatGPT 4.0 Pro limitations and robo-taxi fleet costs
(12:46) Differences between ChatGPT models and future improvements
(19:57) OpenPhone - Get 20% off your first six months at https://www.openphone.com/twist
(20:58) LLM focus shift and robo taxi cost model experimentation
(23:01) Importance of prompt engineering in AI tools adoption
(25:31) Workplace AI adoption challenges and generational tech differences
(27:22) New productivity hacks and Gemini app's growth
(30:27) Zendesk - Get six months free at https://www.zendesk.com/twist
(34:06) Startup support programs and Google Gemini's research capabilities
(37:08) AI model performance evaluation and simplification
(45:02) Meta's Llama 3.370b launch and AI industry impact
(46:37) Infrastructure costs, competitive landscape, and AI-generated content evolution
(55:44) Trust and bias in language models, news analysis startup ideas
*
Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com
Check out the TWIST500: https://www.twist500.com
Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp
*
Mentioned on the show:
Check out Sora here: https://sora.com/
Check out Kling here: https://klingai.com/
Check out Groq: https://groq.com/
*
Follow Sunny:
X: https://x.com/sundeep
LinkedIn: https://www.linkedin.com/in/sundeepm
*
Follow Jason:
X: https://twitter.com/Jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Thank you to our partners:
(9:57) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
(19:57) OpenPhone - Get 20% off your first six months at https://www.openphone.com/twist
(30:27) Zendesk - Get six months free at https://www.zendesk.com/twist
*
Great TWIST interviews: Will Guidara, Eoghan McCabe, Steve Huffman, Brian Chesky, Bob Moesta, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
*
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow TWiST:
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
Instagram: https://www.instagram.com/thisweekinstartups
TikTok: https://www.tiktok.com/@thisweekinstartups
Substack: https://twistartups.substack.com
*
Subscribe to the Founder University Podcast: https://www.youtube.com/@founderuniversity1916
AI Demos: ChatGPT-o4 Canvas, Notebook LM & Meta's Movie Gen | E2026
This Week in Startups is brought to you by…
Lemon.io - Hire pre-vetted remote developers, get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
Fidelity Private Shares℠. Manage your cap table and data room, get faster, more accurate 409A valuation and fully automate your next financing round. Visit https://www.fidelityprivateshares.com! Mention our podcast and receive 20% off your first-year paid subscription.
CLA. Innovation takes balance. CLA's CPAs, consultants, and wealth advisors can help you get from startup to where you want to end up. Get started now at https://claconnect.com/tech
*
Sunny Madra joins Jason to discuss Tesla’s Robotaxis (2:17), demo ChatGPT-4o new Canvas interface (25:23), break down Meta’s Movie Gen model (41:11), Notebook LM (54:41), and much more!
*
Timestamps:
(0:00) Sunny Madra and Jason kick off the show
(2:17) Tesla's Robotaxi reveal and challenges with regulatory approval
(10:00) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
(11:22) Self-driving architectures and potential applications for Tesla's Cybervan
(18:15) Update on Tesla's Optimus robot and the future of autonomous vehicles
(22:04) Interest rates and startup funding challenges
(22:13) Fidelity Private Shares℠ - Visit https://www.fidelityprivateshares.com! Mention our podcast and receive 20% off your first-year paid subscription.
(23:14) Traffic congestion, urban planning, and future players in autonomous vehicles
(25:23) ChatGPT-4o with Canvas demo
(31:28) CLA - Get started with CLA's CPAs, consultants, and wealth advisors now at https://www.claconnect.com/tech
(41:11) Meta vision models and Groq demo
(42:35) Apple AirPods updates and AR glasses privacy concerns
(47:46) Meta's new vision models, open-source impact, and OpenAI's business model
(54:41) Notebook LM's tools
(1:00:13) Capturing family history with AI and Meta Movie Gen's potential
*
Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp
*
LINKS:
Check out OpenAI’s Canvas: https://openai.com/index/introducing-canvas/
Check out Notebook LM: https://notebooklm.google.com/
Check out Meta Movie Gen: https://ai.meta.com/research/movie-gen/
Check out Meta Llama 3.2: https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/
*
Follow Sunny:
X: https://twitter.com/sundeep
LinkedIn: https://www.linkedin.com/in/sundeepm
*
Follow Jason:
X: https://twitter.com/Jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Thank you to our partners:
(10:00) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
(22:04) Fidelity Private Shares℠ - Visit https://www.fidelityprivateshares.com! Mention our podcast and receive 20% off your first-year paid subscription.
(31:32) CLA - Get started with CLA's CPAs, consultants, and wealth advisors now at https://www.claconnect.com/tech
*
Subscribe to the TWiST500 newsletter: https://ticker.thisweekinstartups.com/
Check out the TWIST500: twist500.com
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
*
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow TWiST:
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
Instagram: https://www.instagram.com/thisweekinstartups
TikTok: https://www.tiktok.com/@thisweekinstartups
Substack: https://twistartups.substack.com
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast
Ep18. Jensen Recap - Competitive Moat, X.AI, Smart Assistant | BG2 w/ Bill Gurley & Brad Gerstner
Open Source bi-weekly convo w/ Bill Gurley and Brad Gerstner on all things tech, markets, investing & capitalism. This week, joined by Sunny Madra (Groq) they discuss Jensen Huang’s recent appearance on the podcast, including scaling intelligence towards AGI, NVIDIA's strategic positioning, the role of CUDA in the developer ecosystem, the future of inference workloads, systems-level thinking, Elon Musk's influence on AI development, the future of AI assistants, open vs closed AI models, safety and security in AI development, & more. Enjoy another episode of BG2.
Chapters
(00:00) Introduction and Initial Reactions to Jensen
(04:32) NVIDIA's Position in Accelerated Compute
(05:11) CUDA and NVIDIA’s Competitive Moat
(12:53) Challenges to NVIDIA’s Competitive Advantage
(18:22) Future Outlook on Inference
(24:46) The Insatiable Demand for AI and Hardware
(27:12) Elon Musk' and X.ai
(31:47) Scaling AI Models and Clusters
(34:17) Economic Models and Funding in AI
(39:08) The Future of AI Pricing and Consumption Models
(42:25) Memory, Actions, and Intelligent Agents
(47:08) The Role of AI in Business Productivity
(51:03) Open vs Closed Models in AI Development
Produced by Benny Beausoleil
Music by Yung Spielberg
#jensenhuang #nvidia #bradgerstner #billgurley #clarktang #xai #memphiscluster #elonmusk #noambrown #openai #gptstrawberry
Ep8. AI Models, Data Scaling, Enterprise & Personal AI | BG2 with Bill Gurley & Brad Gerstner
Open Source bi-weekly convo w/ Bill Gurley and Brad Gerstner on all things tech, markets, investing & capitalism. This week, joined by Sunny Madra (Head of Groq Cloud) they discuss AI Models, Data Scaling, Enterpise AI, Personal AI, the tech markets, and more. Enjoy another episode of Bg2.
Timestamps:
(00:00) Intro
(01:10) $META Llama 3
(20:40) Enterprise AI
(36:50) Personal AI
(50:07) $TSLA Earnings Call & Rideshare
(52:10) Tech Check
Available on Apple, Spotify, www.bg2pod.com
Follow:
Brad Gerstner @altcap https://x.com/altcap
Bill Gurley @bgurley https://x.com/bgurley
Sunny Madra @sundeep https://x.com/sundeep
BG2 Pod @bg2pod https://x.com/BG2Pod
AI demos: Figma’s FigJam, AI drawing, dancing avatars, and more! | E1857
This Week in Startups is brought to you by…
Squarespace. Turn your idea into a new website! Go to Squarespace.com/TWIST for a free trial. When you’re ready to launch, use offer code TWIST to save 10% off your first purchase of a website or domain.
Brave. If you’re building AI and search-based applications, train your models with the Brave Search API. Get started for free at brave.com/jason
Lemon.io - Hire pre-vetted remote developers, get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
Today’s show:
Sunny Madra joins Jason to demo Figma FigJam (2:08), a product that generates images from real-time drawing (31:05), and more while discussing the implications of IP theft writ large.
*
Timestamps:
(0:00) Sunny Madra joins Jason
(2:08) Sunny demos Figma’s FigJam
(9:43) Squarespace - Use offer code TWIST to save 10% off your first purchase of a website or domain at https://Squarespace.com/twist
(10:40) Figma FigJam demo continued
(16:02) Jason shares his AI generated images and discusses how companies navigate copyright issues
(24:58) Brave - Try the Brave Search API at http://brave.com/jason
(26:12) Feedback and thoughts for FigJam
(31:05) Sunny demos Fal, an AI that gives generated images from real time drawing
(40:28) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist
(41:49) Sunny demos the Real-Time Latent Consistency Model
(47:01) Sunny demos Alibaba’s Animate Anyone
*
Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp
*
Check out FigJam: https://help.figma.com/hc/en-us/articles/1500004362321-Guide-to-FigJam
Check out Fal: https://www.fal.ai/dynamic
Check out the hugging face video demo: https://huggingface.co/spaces/radames/Real-Time-Latent-Consistency-Model
Check out Animate Anyone: https://www.youtube.com/watch?v=8PCn5hLKNu4
*
Follow Sunny: https://twitter.com/sundeep
*
Follow Jason:
Twitter: https://twitter.com/jason
Instagram: https://www.instagram.com/jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
*
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow TWiST:
Substack: https://twistartups.substack.com
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast
AI demos: Bard's YouTube summarizer, Stability's image editing tools, and more with Sunny Madra | E1853
This Week in Startups is brought to you by…
Squarespace. Turn your idea into a new website! Go to Squarespace.com/TWIST for a free trial. When you’re ready to launch, use offer code TWIST to save 10% off your first purchase of a website or domain.
Vanta. Compliance and security shouldn't be a deal-breaker for startups to win new business. Vanta makes it easy for companies to get a SOC 2 report fast. TWiST listeners can get $1,000 off for a limited time at vanta.com/twist
Northwest Registered Agent. When starting your business, it's important to use a service that will actually help you. Northwest Registered Agent is that service. They'll form your company fast, give you the documents you need to open a business bank account, and even provide you with mail scanning and a business address to keep your personal privacy intact. Visit http://northwestregisteredagent.com/twist to get a 60% discount on your next LLC.
Today’s show:
Sunny Madra joins Jason to demo Bard’s YouTube summarizer (2:27), an iPhone app that draws apps (41:14), Stability’s Clipdrop (59:20), and much more!
*
Time stamps:
(0:00) Sunny Madra joins Jason
(2:27) Sunny demos Bard’s YouTube summarizer
(7:00) Jason's revamping his Substack! First topic; Jason's new ADD management framework: Automate, Deprecate, Delegate
(13:58) Squarespace - Use offer code TWIST to save 10% off your first purchase of a website or domain at https://Squarespace.com/twist
(14:54) Bard’s YouTube summarizer demo continued
(27:09) Vanta - Get $1000 off your SOC 2 at https://vanta.com/twist
(29:09) Sunny demos tlDraw and discusses the evolution of low-code environments
(40:08) Northwest Registered Agent - Get a 60% discount on your next LLC at http://northwestregisteredagent.com/twist
(41:14) Sunny showcases UIDraw, an app to draw and build a web app on your phone with AI
(46:57) Sunny demos Trace, a tool to design a UI via AI
(53:38) Sunny demos Runway TV
(59:20) Sunny demos Stability’s image editing tools, including Clipdrop
(1:05:18) Jason and Sunny’s AI predictions and bet
*
Links:
https://www.wired.com/story/inside-story-of-pong-excerpt/https://en.wikipedia.org/wiki/Vin_Scelsastabilityhttps://twitter.com/jsngr/status/1728848624048853442?s=42&t=79J_alIF_jrz1wn1s5pboQclipdrop
*
Check out:
https://www.trace.zip/https://www.runway.tv/https://makereal.tldraw.com/
*
Follow Sunny: https://twitter.com/sundeep
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow Jason:
Twitter: https://twitter.com/jason
Instagram: https://www.instagram.com/jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Follow TWiST:
Substack: https://twistartups.substack.com
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast
OpenAI Chaos Continues with Sunny Madra and David Sacks | E1852
This Week in Startups is brought to you by…
NetSuite. Once your business gets to a certain size the cracks start to emerge. Things you used to do in a day take a week. You deserve a customized solution - and that's NetSuite. Learn more when you download NetSuite’s popular KPI Checklist - absolutely free, at NetSuite.com/twist
LinkedIn Marketing. To redeem a $100 LinkedIn ad credit and launch your first campaign, go to linkedin.com/thisweekinstartups
Brave. If you’re building AI and search-based applications, train your models with the Brave Search API. Get started for free at brave.com/jason
Today’s show:
Sunny Madra and David Sacks join Jason to break down all the chaos of the past few days at OpenAI including why the situation is so unprecedented (17:46), the board’s failure (30:18), the impact on startups (43:34), and much more!
*
Time stamps:
(0:00) Recapping the chaotic past few days at OpenAI plus some immediate impacts from the fallout
(16:43) NetSuite - Download your free KPI Checklist at http://netsuite.com/twist
(17:46) David Sacks joins the show and explains why this situation is so unprecedented in Silicon Valley
(28:51) LinkedIn Marketing - Get a $100 LinkedIn ad credit at https://linkedin.com/thisweekinstartups
(30:18) How the board could have handled this better, Sam Altman's misaligned incentives packages (or lack thereof)
(37:14) Silicon Valley's battle-tested standard structure, and why you shouldn't innovate on corporate structure
(42:20) Brave - Try the Brave Search API at http://brave.com/jason
(43:34) Impact on startups reliant on OpenAI's models, a brutal momentum killer for OpenAI
(50:39) Predictions: Return of the King? Mass Exodus? Something else?
*
Follow Sunny: https://twitter.com/sundeep
Follow Sacks: https://twitter.com/DavidSacks
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow Jason:
Twitter: https://twitter.com/jason
Instagram: https://www.instagram.com/jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Follow TWiST:
Substack: https://twistartups.substack.com
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast