Snap’s Secret to Processing 10 Petabytes a Day: GPU-Accelerated Spark | NVIDIA AI Podcast Ep. 298
Snap processes more than 10 petabytes of experimentation data every single morning—and with NVIDIA GPU-accelerated Apache Spark on Google Cloud, Snap cut job costs by 76%, reduced memory usage by 80%, and eliminated 120 terabytes of disk spill from its pipelines.
Prudhvi Vatala, head of engineering platforms at Snap, joins the NVIDIA AI Podcast to break down how he and his team completely modernized data infrastructure for a social platform serving nearly a billion monthly active users—using NVIDIA cuDF plugin (formerly referred to as NVIDIA RAPIDS plugin) for Apache Spark on Google Kubernetes Engine, with zero application code changes.
🔬Topics covered:
How Snap runs A/B tests at planetary scale using rigorous statistical methods like heterogeneous treatment effect detection and variance reduction
Why Snap reuses idle inference GPUs between 1–5 a.m. for batch data processing—and how it built a Kubernetes-based platform to do it
How NVIDIA cuDF delivered 3x+ speedups on join-heavy Spark jobs with no code rewrites
The full business impact: 76% cost reduction, 62% fewer cores, 80% less memory, 120 TB of spill eliminated
How a three-way partnership between Snap, NVIDIA, and Google Cloud made it possible in just 8–9 months
Chapters:
0:00 Introduction and Snap overview
3:35 What is Snap’s experimentation platform?
4:05 Why experimentation, safety, and privacy are core at Snap
4:52 How A/B testing works at billion-user scale
8:14 Discovering NVIDIA cuDF plugin
9:06 Benchmarking results: join, union, and aggregation jobs
12:00 Reusing idle GPUs overnight via GKE
13:24 Building a bottom-up GPU data platform at Snap
17:48 Results: 76% cost reduction and partnership impact
20:56 Snap’s evolution and what’s next
Learn more:
NVIDIA cuDF: https://developer.nvidia.com/topics/ai/data-science/cuda-x-data-science-libraries/cudf#accel-apache
Snapchat CEO: Why distribution has become the most important moat | Evan Spiegel
Evan Spiegel, the co-founder and CEO of Snap, is one of the very few people in the world who has successfully built and scaled a lasting consumer social product. Snapchat has nearly 1 billion MAUs, and Evan and his team invented some of the most important consumer products and features, including Stories, AR glasses, swipe-based navigation, the camera as the primary UX, and a lot more.
In our in-depth conversation, we discuss:
1. Why distribution is now the biggest challenge for creating a consumer technology business
2. How Snap innovates at scale with a 9-to-12-person design team: no titles, no hierarchy, hundreds of ideas reviewed weekly with the CEO
3. Why a pure software business is no longer a moat, and what actually creates durable competitive advantages today
4. How AI is changing the way designers work and why they’re now shipping code
5. Why every major Snap feature was copied and how that forced the company to work differently
6. Evan’s prediction that humanity’s comfort with AI will be a bigger bottleneck than the technology itself
7. This year’s crucible moment for Snap
—
Brought to you by:
WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lenny
Vanta—Automate compliance, manage risk, and accelerate trust with AI: https://vanta.com/lenny
—
Episode transcript: https://www.lennysnewsletter.com/p/snapchat-ceo-why-distribution-is
—
Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0
—
Where to find Evan Spiegel:
• X: https://x.com/evanspiegel
• Snapchat: https://www.snapchat.com/@evan
• LinkedIn: https://www.linkedin.com/in/evan-spiegel
• Website: https://www.spiegelfamilyfund.com
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Introduction to Evan Spiegel
(02:28) Why consumer social products are so hard to build
(04:31) How Snapchat cracked distribution with close friends, not network size
(05:50) Why distribution is the new moat in the AI era
(08:39) Snapchat’s innovation track record (and why software isn’t a moat)
(11:39) Why Snap is betting on two of the hardest businesses: consumer social and hardware
(16:00) Specs use cases
(17:56) The innovation process
(21:34) The velocity of design work at Snapchat
(25:07) Why Evan says you must talk to customers
(26:06) The origin story of Stories
(28:25) How screenshot detection saved early Snapchat
(31:03) Why they waited to hire PMs—and what role they play now
(34:41) How AI is shifting the designer-PM-engineer triad
(36:10) Design as an intentional bottleneck for product cohesion
(37:24) Why staying close to customers matters for any leader
(39:39) What Evan looks for when hiring designers
(41:57) How to develop young design talent
(44:16) Designers shipping code with AI—and the guardrails needed at scale
(47:20) Using jobs-to-be-done to organize AI transformation
(48:50) How the CEO job has changed over 15 years
(51:30) Learning to communicate
(54:08) Why this year is Snapchat’s “crucible moment”
(56:22) Being the “middle child” in tech
(57:51) Screen-time philosophy with four kids (ages 2 to 15)
(1:01:08) AI Corner
(1:04:02) Contrarian Corner
(1:06:04) Lightning round and final thoughts
—
References: https://www.lennysnewsletter.com/p/snapchat-ceo-why-distribution-is
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
—
Lenny may be an investor in the companies discussed.
To hear more, visit www.lennysnewsletter.com
The Grittiest Conversations of 2025: AI, Business & Beyond
In this recap episode, we highlight the best moments from our 2025 interviews and reflect on the ideas that defined the year.
Featuring:
David Rubenstein (co-founder of Carlyle)
Yamini Rangan (CEO of HubSpot)
Ben Chestnut (co-founder of Mailchimp)
Winston Weinberg (co-founder and CEO of Harvey)
Garrett Lord (co-founder of Handshake)
Aidan Gomez (co-founder and CEO of Cohere)
Michelle Zatlyn (co-founder of Cloudflare)
Evan Spiegel (co-founder and CEO of Snap)
Connect with Joubin
X: https://x.com/Joubinmir
LinkedIn: https://www.linkedin.com/in/joubin-mirzadegan-66186854/
Email: grit@kleinerperkins.com
Follow on LinkedIn:
https://www.linkedin.com/company/kpgrit
Follow on X:
https://x.com/KPGrit
Learn more about Kleiner Perkins: https://www.kleinerperkins.com/
How Snap Plans to Win the AR Race | Evan Spiegel on Spectacles
Turning down a $3B offer from Facebook is a bold move for any young CEO.
Evan Spiegel shares how Snap’s early dream was to stay independent and give its community an authentic voice, a bet that proved right.
He also explains why they are now doubling down on AR glasses and why the anxiety around AI deserves far more attention from tech leaders.
Guest: Evan Spiegel, co-founder and CEO of Snap Inc. and Bing Gordon, Advisor at Kleiner Perkins
Connect with Evan Spiegel
X:https://x.com/evanspiegel?lang=en
LinkedIn:https://www.linkedin.com/in/evan-spiegel/
Connect with Bing Gordon
X: https://x.com/bingfish
LinkedIn: https://www.linkedin.com/in/binggordon/
Connect with Joubin
X: https://x.com/Joubinmir
LinkedIn: https://www.linkedin.com/in/joubin-mirzadegan-66186854/
Email: grit@kleinerperkins.com
Learn more about Kleiner Perkins:https://www.kleinerperkins.com/
CEO Diaries: The Brutal Mistake That Almost Crippled Snapchat!
In this episode of CEO Diaries, Snapchat CEO Evan Spiegel reveals the biggest hiring mistakes that nearly cost him everything. Overvaluing experience, ignoring culture, and avoiding tough feedback can quietly kill a company, and this brutally honest insight will change how you build teams forever.
Visit - www.linkedin.com/DOAC
Listen to the full episode here -
Spotify - https://g2ul0.app.link/GSm1crv4TTb
Apple - https://g2ul0.app.link/6VjMycy4TTb
Watch the Episodes On YouTube -
https://www.youtube.com/c/%20TheDiaryOfACEO/videos
Learn more about your ad choices. Visit megaphone.fm/adchoices
Snap: Evan Spiegel
What started as a design project for Stanford student Evan Spiegel quickly flourished into one of the most-used social media platforms in the world: Snapchat. It only took two years for Mark Zuckerberg to make a multi-billion offer for the company. But Evan turned it down — convinced of Snap’s potential to disrupt human communication in an even bigger way. And while Evan’s path has been anything but smooth, today Snap is valued at more than $13 billion, with ambitions beyond its hero mobile app.
This episode was produced by Alex Cheng with music composed by Ramtin Arablouei. It was edited by John Isabella with research help from Katherine Sypher. Our engineers were Patrick Murray and Gilly Moon.
You can follow HIBT on X & Instagram, and email us at hibt@id.wondery.com. Sign up for Guy’s free newsletter at guyraz.com or on Substack.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Snapchat CEO: Exact Formula Used To Build A $130 Billion Company! I Said No To $3 Billion From Mark Zuckerberg! It’s Time To Quit Your Job When You Feel This!
From turning down $3 billion from Facebook to building a $100 billion empire, Evan Spiegel reveals the blueprint behind Snapchat.
Evan Spiegel, co-founder & CEO of Snapchat (now Snap Inc.), founded the app while at Stanford and became the youngest billionaire at 25.
In this conversation, Evan and Steven cover how Snapchat almost didn’t exist, Evan’s ‘T-Shaped’ leadership style, getting bullied before founding Snapchat, and the harshest day of his CEO journey.
Chapters:
00:00 Intro
02:29 The Dots That Got You Here
03:33 Did You Feel Like You Fitted In?
03:46 When Did Computers First Come In?
05:02 Things Aren’t as Complicated as They Seem
05:28 You Got Bullied
06:09 What Were You Like as a Kid
07:33 Why CEOs Don’t Do Many Podcasts
08:55 Why Did You Choose Product Design?
10:30 Your Class in Entrepreneurship
11:00 Key Lesson From Entrepreneurship Class
12:02 Big Ambitions
13:26 Entrepreneurship in Europe
16:00 Your First Failure
18:00 How to Know When to Quit
18:50 Why Love & Passion Matter
19:19 Launching Early & Getting Feedback Fast
20:39 How Initial Ideas Can Be Wrong
21:37 How You Started Snapchat
27:03 Customer Feedback to Implement
28:36 Raising Capital
29:46 Investor Feedback
30:17 Building a Social Network Sounds Delusional
31:51 Doubting Snapchat's Success
36:19 Quitting University for Snapchat
37:33 Advice for Young Entrepreneurs
39:27 Are Job Titles Limiting Creativity?
40:34 Hierarchy Issues in Companies
42:22 Innovating at Snapchat
47:59 Importance of Hiring
49:00 Hiring Mistakes to Avoid
51:06 Leadership Traits of a Perfect Hire
52:25 Being Nice vs. Being Kind
53:52 T-Shaped Leadership
56:44 Advice to Younger Evan
59:39 Embedding Company Culture Early
01:01:51 When Company Culture Dilutes
01:03:35 Company Incentives
01:04:25 Worst Early Advice
01:05:47 How Mark Zuckerberg Approached You
01:10:04 Saying No to That Offer
01:13:13 Youngest Billionaire at 25
01:14:19 Managing Romantic Relationships
01:19:23 Ads
01:20:21 Your LinkedIn Bio Joke
01:22:45 Messaging Zuckerberg When They Copied Features
01:26:57 Should Big Tech Monopolies Be Stopped?
01:29:25 Hardest Day When Copied
01:30:30 Leading in Tough Times
01:31:48 Content Moderation Challenges
01:36:28 Why Meta Rolled Back Moderation Policies
01:39:20 Optimism About America
01:40:08 Social Media & Your Kids
01:42:51 Is TikTok's Ban Good for Snapchat?
01:46:07 Snapchat Going Public
01:48:28 Killing Projects You Loved
01:50:15 How Do You Prioritize?
01:52:23 Ray-Ban Spectacles Launch Reaction
01:53:37 Will Kids Learn From AI?
01:56:34 Tradeoffs With AI
01:58:22 Ads
02:00:35 Snapchat in 2025
02:02:14 Importance of Counsels at Work
02:05:34 When to Listen to Your Team
02:06:22 Work-From-Home Policies
02:08:42 Principles of a Successful Entrepreneur
02:10:18 Managing Stress
02:13:10 Worst Days of Snapchat
02:13:50 Do You Have Imposter Syndrome?
02:15:08 Would You Start Another Tech Company?
02:17:23 Hardest Thing You’ve Overcome
02:18:17 Self-Awareness
02:20:45 Do You Feel Impatient as a Leader?
02:21:35 What Would Your Team Say About You?
02:22:12 What Are You Really Good At?
02:26:55 Biggest Question Entrepreneurs Should Ask
Follow Evan:
Twitter - https://g2ul0.app.link/hnvyN8X8SRb
More about Snap's new glasses - https://g2ul0.app.link/GDzqC6XeTRb
Watch on YouTube:
https://g2ul0.app.link/DOACEpisodes
Get My Book & Cards:
'The 33 Laws Of Business & Life' - https://g2ul0.app.link/DOACBook
Conversation Cards: https://g2ul0.app.link/f31dsUttKKb
Sign up to receive email updates about Diary Of A CEO here: https://bit.ly/diary-of-a-ceo-yt
Ready to think like a CEO? Gain access to the 100 CEOs newsletter here: https://bit.ly/100-ceos-newsletter
Follow Me:
https://g2ul0.app.link/gnGqL4IsKKb
Sponsors:
LinkedIn Ads - https://www.linkedin.com/DIARY
Perfect Ted - https://www.perfectted.com (Code: DIARY40)
Learn more about your ad choices. Visit megaphone.fm/adchoices
Snap’s Evan Spiegel on AI, AR, and – yes – TikTok
The Snap CEO talks about the challenges facing the company’s AI chatbot, how augmented reality glasses could change how we compute the real world and why a TikTok ban would “help” Snap.
You can follow Kara and Nayeema on Instagram. Search for @karaswisher and @nayeemaraza.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
20Product: Snap's VP Product on How Snap Hires 10x Product People, What Makes Evan Spiegel So Special at Product, Three Ways to Prioritise Product Ideas in Teams, The Future of AR, Why Snap Glasses Will be Huge and Snap Will Be Massive in Japan with Jack
Jack Brody is the VP Product @ Snap. Jack joined Snap in 2014 as a Product Designer, and ultimately helped build out the design organization as the Head of Design before taking on his current role overseeing all of Product for the Snapchat application and Hardware. In his 9 years at Snap, he helped create Memories, the Snap Map, and AR Lenses like Face Swap.
In Today's Episode with Jack Brody We Discuss:
The Shortest Internship in Tech:
How did Jack get an internship with Evan Spiegel and Snap while he was still at college?
How did it turn into the shortest internship in tech history?
What are the single biggest product lessons Jack has from working with Evan Spiegel?
2. Product 101: Art vs Science:
Does Jack believe product is more art or science? If he were to assign numbers to them, what would they be?
How does Jack define creativity? What can founders and product leaders do to ensure their teams are as creative as possible?
What is the 3 step framework through which product leaders should prioritize product ideas?
Does Jack believe that when the CEO is no longer the Head of Product, the company is dead?
Does Jack agree with Gustav Soderstrom, "talk is cheap, so we should do more of it"?
3. The SNAP Hiring Process: What Works and What Does Not:
What is the hiring process for the product team at SNAP?
What questions are most revealing of 10x product people in the interview process?
What case studies and tests does Jack use in the interview process?
What other roles and functions does Jack bring into the interview process as part of the decision?
What are the single biggest mistakes founders make in the hiring process for product?
4. SNAP, The Future, and The World Around Us:
What do Jack and SNAP believe will be the future for augmented reality?
What country is SNAP not big in today but will be in the next 5 years? Why that one?
Why did SNAP tear down its android app and start again? What has been the impact?
Were the SNAP glasses a success? What is their future?
Disinflation Begins, Snap's Cultural Roadblocks, Meta's 'Year Of Efficiency'
Ex-Snap and Instagram executive Meghana Dhar joins Ranjan Roy and Alex Kantrowitz for our weekly news recap show. We cover 1) The implications of the Fed's talk of "deflation." 2) Snap's disappointing earnings and operational challenges with an insider's perspective. 3) Meta's wooing of Wall Street and whether its pivot is real or just talk. 4) What the combination of layoffs and buybacks really mean. 5) Downturn as an excuse to kill ESG? 6) AI Hands!
----
Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice.
For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/
Questions? Feedback? Write to: bigtechnologypodcast@gmail.com
Learn more about your ad choices. Visit megaphone.fm/adchoices
Snap Inc.'s Evan Spiegel at Code 2022
2022 has been a difficult year for Snap, Inc. Shortly after announcing layoffs, Snap CEO Evan Spiegel sat down with Kara and Scott to discuss the challenges facing his company, and why he thinks they can be overcome. Recorded on September 7 at Code 2022 in Los Angeles.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Recode Decode: Snap CEO Evan Spiegel (Live at Code 2018)
Evan Spiegel, the co-founder and CEO of Snapchat maker Snap, talks with Recode’s Kara Swisher at the 2018 Code Conference. Spiegel talks about why Snapchat underwent a controversial redesign, why it partially reversed that decision and what he learned from the backlash. He also discusses how he’s evolving as a CEO since Snap’s IPO in 2017, having learned that leading a public company “requires a bit more grit.” He also addresses the departure of a female engineer who accused the company of having a toxic male-driven culture, calling her objections a “wake-up call.” Plus: How Spiegel thinks about “traditional social media,” i.e. Facebook, lifting features from the Snapchat app, and why it will be “harder” to copy those features in the future.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
.jpg?width=400){kind=small}
.jpg?width=400){kind=small}
