The apps and websites we use every day depend on systems most of us never see.
Jay Kreps joins Joubin Mirzadegan to share how Confluent became the ‘central nervous system’ for companies like Expedia and eBay, letting them respond to business operations instantly.
They also break down why the myth of AI-driven efficiency falls short, and why building truly transformative companies takes far longer than most people expect.
Guest: Jay Kreps, Co-Founder & CEO of Confluent
Connect with Jay
X
LinkedIn
Connect with Joubin
X
LinkedIn
Email: grit@kleinerperkins.com
Learn more about Kleiner Perkins
Confluent is an all-in-one, real-time platform that allows you to stream, connect, process, and govern your data.
Connect with Sean on Linkedin, and listen to the Software Engineering Daily podcast, where he co-hosts.
Do you know the answer to Loadrunner lr_get_attrib_string always returns null? If so, you can help out Trncvs and receive the bounty offered on the question.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Shaun Clowes is the chief product officer at Confluent and former CPO at Salesforce’s MuleSoft and at Metromile. He was also the first head of growth at Atlassian, where he led product for Jira Agile and built the first-ever B2B growth team. In our conversation, we discuss:
• Why most PMs are bad, and how to fix this
• Why great AI products are all about the data
• Why he changed his mind about being data-driven
• How to build your B2B growth team
• How to choose your next career stop
• Much more
—
Brought to you by:
• Enterpret—Transform customer feedback into product growth
• BuildBetter—AI for product teams
• Wix Studio—The web creation platform built for agencies
—
Find the transcript at: https://www.lennysnewsletter.com/p/why-great-ai-products-are-all-about-the-data-shaun-clowes
—
Where to find Shaun Clowes:
• X: https://x.com/ShaunMClowes
• LinkedIn: https://www.linkedin.com/in/shaun-clowes-80795014/
• Website: https://shaunclowes.com/about-shaun
• Reforge: https://www.reforge.com/profiles/shaun-clowes
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Shaun’s background
(05:08) The state of product management
(09:33) Becoming a 10x product manager
(13:23) Specific ways to leverage AI in product management
(17:15) Feedback rivers
(19:20) AI's impact on data management
(24:35) The future of enterprise businesses with AI
(35:41) Data-driven decision-making
(45:50) Building effective growth teams
(50:18) The evolution of product-led growth
(56:16) Career insights and decision-making
(01:07:45) Failure corner
(01:12:32) Final thoughts and lightning round
—
Referenced:
• Steve Blank’s website: https://steveblank.com/
• Getting Out of the Building. 2 Minutes to See Why: https://www.youtube.com/watch?v=TbMgWr1YVfs
• OpenAI: https://openai.com/
• Claude: https://claude.ai/
• Sachin Rekhi on LinkedIn: https://www.linkedin.com/in/sachinrekhi/
• Video: Building Your Product Intuition with Feedback Rivers: https://www.sachinrekhi.com/video-building-your-product-intuition-with-feedback-rivers
• Confluent: https://www.confluent.io
• Workday: https://www.workday.com/
• Lenny and Friends Summit: https://lennyssummit.com/
• A conversation with OpenAI’s CPO Kevin Weil, Anthropic’s CPO Mike Krieger, and Sarah Guo: https://www.youtube.com/watch?v=IxkvVZua28k
• Anthropic: https://www.anthropic.com/
• Salesforce: https://www.salesforce.com/
• Atlassian: https://www.atlassian.com/
• Jira: https://www.atlassian.com/software/jira
• Ashby: https://www.ashbyhq.com/
• Occam’s razor: https://en.wikipedia.org/wiki/Occam%27s_razor
• Breaking the rules of growth: Why Shopify bans KPIs, optimizes for churn, prioritizes intuition, and builds toward a 100-year vision | Archie Abrams (VP Product, Head of Growth at Shopify): https://www.lennysnewsletter.com/p/shopifys-growth-archie-abrams
• Charlie Munger quote: https://www.goodreads.com/quotes/11903426-show-me-the-incentive-and-i-ll-show-you-the-outcome
• Elena Verna on how B2B growth is changing, product-led growth, product-led sales, why you should go freemium not trial, what features to make free, and much more: https://www.lennysnewsletter.com/p/elena-verna-on-why-every-company
• The ultimate guide to product-led sales | Elena Verna: https://www.lennysnewsletter.com/p/the-ultimate-guide-to-product-led
• Metromile: https://www.metromile.com/
• Tom Kennedy on LinkedIn: https://www.linkedin.com/in/tom-kennedy-37356b2b/
• Building Wiz: the fastest-growing startup in history | Raaz Herzberg (CMO and VP Product Strategy): https://www.lennysnewsletter.com/p/building-wiz-raaz-herzberg
• Wiz: https://www.wiz.io
• Colin Powell’s 40-70 rule: https://www.42courses.com/blog/home/2019/12/10/colin-powells-40-70-rule
• Detroiters on Netflix: https://www.netflix.com/title/80165019
• Glean: https://www.glean.com/
• Radical Candor: Be a Kick-Ass Boss Without Losing Your Humanity: https://www.amazon.com/Radical-Candor-Kick-Ass-Without-Humanity/dp/1250103509
• Listen: Five Simple Tools to Meet Your Everyday Parenting Challenges: https://www.amazon.com/Listen-Simple-Everyday-Parenting-Challenges/dp/0997459301
• Empress Falls Canyon and abseiling: https://bmac.com.au/blue-mountains-canyoning/empress-falls-canyon-and-abseiling
—
Recommended books:
• The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses: https://www.amazon.com/Lean-Startup-Entrepreneurs-Continuous-Innovation/dp/0307887898
• Inspired: How to Create Products Customers Love: https://www.amazon.com/Inspired-Create-Products-Customers-Love/dp/0981690408
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
—
Lenny may be an investor in the companies discussed.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe
In the New Stack Makers episode, Adi Polak, Director, Advocacy and Developer Experience Engineering at Confluent discusses the operational and analytical estates in data infrastructure. The operational estate focuses on fast, low-latency event-driven applications, while the analytical estate handles long-running data crunching tasks. Challenges arise due to the "schema evolution" from upstream operational changes impacting downstream analytics, creating complexity for developers.
Apache Iceberg and Flink help mitigate these issues. Iceberg, a table format developed by Netflix, optimizes querying by managing file relationships within a data lake, reducing processing time and errors. It has been widely adopted by major companies like Airbnb and LinkedIn.
Apache Flink, a versatile data processing framework, is driving two key trends: shifting some batch processing tasks into stream processing and transitioning microservices into Flink streaming applications. This approach enhances system reliability, lowers latency, and meets customer demands for real-time data, like instant flight status updates. Together, Iceberg and Flink streamline data infrastructure, addressing developer pain points and improving efficiency.
Learn more from The New Stack about Apache Iceberg and Flink:
Unfreeze Apache Iceberg to Thaw Your Data Lakehouse
Apache Flink: 2023 Retrospective and Glimpse into the Future
4 Reasons Why Developers Should Use Apache Flink
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
This Week in Startups is brought to you by…
DevSquad. Most dev agencies only offer developers. Why? Because product management is hard. Get an entire product team for the cost of one US developer plus 10% off at http://devsquad.com/twist.
Hubspot for Podcast Networks. Looking to up your marketing game?
Check out the podcast: Marketing Against the Grain
Hosted by: Hubspot CMO Kipp Bodnar and Zapier CMO Kieran Flannigan
They bring you the latest in marketing trends, growth tactics and innovation.
Available on all your favorite podcast apps.
Gelt. It’s time to take control over your taxes. Discover how Gelt can help you to manage and optimize both your personal and business taxes. Visit https://joingelt.com/twist now.
*
Todays show:
David Weisburd hosts Apurva Mehta, Jack Altman, and Jason Calacanis to discuss Sam Altman’s huge investment wins (2:34), the role of SPVs (13:19), CalPERS increasing their exposure to venture (26:19), and liquidation preferences (45:56).
*
Timestamps:
(0:00) David Weisburd intros Apurva Mehta, Jack Altman, and Jason Calacanis
(2:34) Reddit's IPO and Sam Altman's investment success
(5:48) Apurva's investment strategy and thoughts on fund size and portfolio strategy
(11:50) DevSquad - Get an entire product team for the cost of one US developer plus 10% off at http://devsquad.com/twist
(13:19) The role of SPVs and the importance of trust in the investment ecosystem
(24:53) Marketing Against the Grain https://www.youtube.com/watch?v=xHrjktuM1Dc
https://lnk.to/h3vKHnTW
(26:19) CalPERS and pension funds increasing their exposure to venture capital and private equity
(28:11) Is it a good time to invest in venture?
(41:48) Gelt. It’s time to take control over your taxes. Visit https://joingelt.com/twist now
(43:02) Fundraising and decision-making processes
(45:56) Higher liquidation preferences at the later stage
(55:49) Rapid fire segment on recent investments
*
Mentioned on the show:
https://www.retellai.com
https://www.heygen.com
https://www.owner.com
https://www.foundationhealth.com
https://peregrine.io
https://www.marvl.io
https://getprops.ai
https://www.arkitask.ai
*
Follow Apurva:
X: https://twitter.com/mehtaaapurva
LinkedIn: https://www.linkedin.com/in/apurvaamehta/
*
Follow Jack:
X: https://twitter.com/jaltma
LinkedIn: https://www.linkedin.com/in/jackealtman/
*
Follow David:
X: https://twitter.com/DWeisburd
LinkedIn: https://www.linkedin.com/in/dweisburd
Check out: https://10xcapital.com
*
Follow Jason:
X: https://twitter.com/Jason
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp
*
Thank you to our partners:
(11:50) DevSquad - Get an entire product team for the cost of one US developer plus 10% off at http://devsquad.com/twist
(24:53) Marketing Against the Grain https://www.youtube.com/watch?v=xHrjktuM1Dc
https://lnk.to/h3vKHnTW
(41:48) Gelt. It’s time to take control over your taxes. Visit https://joingelt.com/twist now
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
*
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow TWiST:
Substack: https://twistartups.substack.com
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
Instagram: https://www.instagram.com/thisweekinstartups
TikTok: https://www.tiktok.com/@thisweekinstartups
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast
In the episode, Confluent CEO Jay Kreps dives into Confluent's transformation from a LinkedIn spin-out to a publicly traded company. He discusses the challenges of turning an open-source project into a thriving commercial venture and the bold decision to launch a second product while the first was thriving. Additionally, Jay opens up about his transition into the role of CEO, how he only went to one year of high school, and his unique philosophies about steering startups to success.
(0:00) Intro
(1:26) How being a writer benefited Jay as a CEO
(3:05) Building a management team
(13:28) The Role of Titles in a Company
(17:54) Only Going To One Year of High School
(23:02) The Decision to Pursue Computer Science
(31:05) The Birth of Project Kafka
(34:32) Reflections on the Success of Kafka
(36:47) Launching an Open Source Project
(37:43) The Power of Product Marketing
(39:35) Should You Be A Founder?
(42:00) The Transition from Individual Contributor to CEO
(47:51) Navigating the Public Markets
(1:09:46) What's Wrong With Hybrid Work
(1:14:27) Navigating Politics in the Workplace
(1:17:36) Why Fairness Matters
(1:26:51) The Evolution of Open Source
(1:35:22) The Future of Artificial Intelligence
(1:43:41) The Shift from Using Software to Becoming Software
Produced: Rashad Assir & Leah Clapper
Mixed and edited: Justin Hrabovsky
Executive Producer: Josh Machiz
🎙 Listen to the show
Apple Podcasts: https://podcasts.apple.com/us/podcast/the-logan-bartlett-show/id1606770839
Spotify: https://open.spotify.com/show/5WqBqDb4br3LlyVrdqOYYb?si=3076e6c1b5c94d63&nd=1
Google Podcasts: https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zaW1wbGVjYXN0LmNvbS9zb0hJZkhWbg
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
🐦 Twitter - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode, Logan goes behind the scenes with world-class entrepreneurs and investors. If you're interested in the real inside baseball of tech, entrepreneurship, and start-up investing, tune in every Friday for new episodes.
Executive Producer: Rashad Assir
Producer: Leah Clapper
Mixing and editing: Justin Hrabovsky
Check out Unsupervised Learning, Redpoint's AI Podcast: https://www.youtube.com/@UCUl-s_Vp-Kkk_XVyDylNwLA
🎥 Subscribe on YouTube: https://www.youtube.com/channel/UCugS0jD5IAdoqzjaNYzns7w?sub_confirmation=1
Follow on Socials
📸 Instagram - https://www.instagram.com/theloganbartlettshow
📱 X - https://twitter.com/loganbartshow
🎬 Clips on TikTok - https://www.tiktok.com/@theloganbartlettshow
About the Show
Logan Bartlett is a Software Investor at Redpoint Ventures - a Silicon Valley-based VC with $6B AUM and investments in Snowflake, DraftKings, Twilio, and Netflix. In each episode of The Logan Bartlett Show, we sit down with the people behind today’s most important startups and extract the tactics, lessons, and frameworks they’ve learned the hard way. Conversations span hiring to GTM, product, growth, fundraising and everything in between - collectively forming the ultimate playbook to make you a better CEO, investor or board member.
Tap follow and enable notifications to stay ahead of the game.
Sean hosts Partially Redacted, a podcast about data privacy, security, and compliance.
He also hosts the podcast Software Engineering Daily, which features technical interviews on everything from the ethics of GPTs to cloud-native search and WebAssembly. Start with the recent episode Surviving ChatGPT with Christian Hubicki (of Survivor fame).
You can also read about how he crowdsourced a behavioral model for Survivor.
Sean spent four years working in developer relations (DevRel) at Google. Here’s a Software Engineering Daily episode about the role DevRel plays at Google.
Connect with Sean on LinkedIn or Twitter (I mean, X), or check out his website.
Kudos to Great Question badge winner Kai Sellgren for asking How to remove an element from a vector given the element?.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Amidst the volatility of today's economic climate and market, GTM leaders need to be especially savvy when it comes to their company growth strategies and earning their customers' love. A renowned enterprise software leader that has led revenue organizations and spearheaded the cloud GTM strategies at companies such as Oracle, New Relic and now Confluent, Erica Schultz is Confluent's President of Field Operations and will share what she has seen evolve in enterprise GTM and tips on how to thrive in the current environment. From PLG and consumption-based pricing, to value-based selling and driving efficient growth, Erica will share veteran insights that will help you develop your own successful GTM strategy.
Full video: https://youtu.be/j293NArWW38
Want to join the SaaStr community? We're the 🌎largest community for B2B software.
Subscribe for weekly updates: https://www.saastr.com/subscribeform
Twitter: https://twitter.com/saastr
LinkedIn: https://www.linkedin.com/company/2724976
Quora Group: https://www.quora.com/q/cloud
Facebook: https://www.facebook.com/SaaStr/
Instagram: https://www.instagram.com/saastr/
Our North American Event: https://bit.ly/2OXeAYh
Our European Event: https://bit.ly/2OZTad8
Summary
The best way to make sure that you don’t leak sensitive data is to never have it in the first place. The team at Skyflow decided that the second best way is to build a storage system dedicated to securely managing your sensitive information and making it easy to integrate with your applications and data systems. In this episode Sean Falconer explains the idea of a data privacy vault and how this new architectural element can drastically reduce the potential for making a mistake with how you manage regulated or personally identifiable information.
Announcements
Hello and welcome to the Data Engineering Podcast, the show about modern data management
When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show!
Atlan is the metadata hub for your data ecosystem. Instead of locking all of that information into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Go to dataengineeringpodcast.com/atlan today to learn more about how you can take advantage of active metadata and escape the chaos.
Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold.
Data teams are increasingly under pressure to deliver. According to a recent survey by Ascend.io, 95% in fact reported being at or over capacity. With 72% of data experts reporting demands on their team going up faster than they can hire, it’s no surprise they are increasingly turning to automation. In fact, while only 3.5% report having current investments in automation, 85% of data teams plan on investing in automation in the next 12 months. 85%!!! That’s where our friends at Ascend.io come in. The Ascend Data Automation Cloud provides a unified platform for data ingestion, transformation, orchestration, and observability. Ascend users love its declarative pipelines, powerful SDK, elegant UI, and extensible plug-in architecture, as well as its support for Python, SQL, Scala, and Java. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP. Go to dataengineeringpodcast.com/ascend and sign up for a free trial. If you’re a data engineering podcast listener, you get credits worth $5,000 when you become a customer.
Your host is Tobias Macey and today I’m interviewing Sean Falconer about the idea of a data privacy vault and how the Skyflow team are working to make it turn-key
Interview
Introduction
How did you get involved in the area of data management?
Can you describe what Skyflow is and the story behind it?
What is a "data privacy vault" and how does it differ from strategies such as privacy engineering or existing data governance patterns?
What are the primary use cases and capabilities that you are focused on solving for with Skyflow?
Who is the target customer for Skyflow (e.g. how does it enter an organization)?
How is the Skyflow platform architected?
How have the design and goals of the system changed or evolved over time?
Can you describe the process of integrating with Skyflow at the application level?
For organizations that are building analytical capabilities on top of the data managed in their applications, what are the interactions with Skyflow at each of the stages in the data lifecycle?
One of the perennial problems with distributed systems is the challenge of joining data across machine boundaries. How do you mitigate that problem?
On your website there are different "vaults" advertised in the form of healthcare, fintech, and PII. What are the different requirements across each of those problem domains?
What are the commonalities?
As a relatively new company in an emerging product category, what are some of the customer education challenges that you are facing?
What are the most interesting, innovative, or unexpected ways that you have seen Skyflow used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Skyflow?
When is Skyflow the wrong choice?
What do you have planned for the future of Skyflow?
Contact Info
LinkedIn
@seanfalconer on Twitter
Website
Parting Question
From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
Thank you for listening! Don’t forget to check out our other show, Podcast.__init__ to learn about the Python language, its community, and the innovative ways it is being used.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers
Links
Skyflow
Privacy Engineering
Data Governance
Homomorphic Encryption
Polymorphic Encryption
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Support Data Engineering Podcast
When Confluent’s President of Field Ops Erica Schultz was 23, she was working at Oracle and cold-emailed the manager of the Argentina office, asking to work for him. This experience would open the door to opportunities in Buenos Aires and Miami, a time in Erica’s life she does not take for granted. As a leader today, she hopes to pass on this sentiment, constantly looking for individuals worth taking a chance on: “As I look around my organization, I think, OK, who’s the undiscovered not-yet-fully-realized talent that we should think about for this role?”
In this episode, Erica and Joubin talk about why Buenos Aires, Argentina is the best city in the world; the lessons she learned from her father and what changed for her after he died of a rare form of cancer at age 54; her stints at Oracle, LivePerson, and New Relic; the importance of earning responsibility as you advance in your career; staying both humble and paranoid; and the importance of what Confluent is doing in the ever-changing digital infrastructure business.
In this episode, we cover:
The incredible influence of Erica’s namesake, her father, who passed away as her career was taking off (09:28)
“The impact we leave is the impact we have on people” (15:21)
How Erica became the captain of the Dartmouth rowing team after being cut from the swim team (18:03)
Developing leaders from within a high-growth organization, and earning responsibility (31:36)
Why Erica left a CRO role at LivePerson to work for the CRO of New Relic (37:03)
Why she had her team at New Relic read “The Boys in the Boat” by Daniel James Brown, and loves the story of runner Roger Bannister (41:34)
Being humbled by a changing competitive landscape, and the transformation of the digital infrastructure world (44:17)
Real-time data and why both businesses and consumers increasingly need companies like Confluent (49:19)
What Erica thought when she first met Confluent’s founder CEO Jay Kreps (56:03)
How to transition from operator to executive to board member (59:14)
Links:
Connect with EricaLinkedIn
Connect with JoubinTwitter
LinkedIn
Email: grit@kleinerperkins.com
Learn more about Kleiner Perkins
This episode was edited by Eric Johnson from LightningPod.fm
Apurva Mehta is the Managing Partner @ Summit Peak Investments, investing in early stage venture capital funds and making direct co-investments. To date they have backed the likes of Raymond Tonsing, Lachy Groom and Josh Buckley to name a few on the fund side and then on the direct side, invested in Airtable, Virta Health and Sourcegraph. Prior to founding Summit Peak, Apurva spent 7 years as the Deputy Chief Investment Officer at Cook's Children's Hospital and before that spent 3 years as Director of Portfolio Investments at The Juilliard School.
In Today's Episode You Will Learn:
1.) How did Apurva make his way into the world of fund investing? How did that lead to his founding Summit Peak and also becoming a GP?
2.) How does Apurva think about how much importance to place on references when diligencing managers? What reference types mean a lot? Which mean less? Why does Apurva still believe early-stage is the most inefficient segment of the venture landscape?
3.) How does Apurva think about GP commits? Is it fair to have a required benchmark? How does Apurva advise founders on LP concentration limits? When is one LP too much of a fund? How does Apurva advise managers on selling a stake in the management company?
4.) As a fund of funds, how does Apurva approach fund portfolio construction today? How does this differ between the fund portfolio vs the direct portfolio? How does Apurva think about the compression of fundraising timelines both with GPs and Founders? Why does Apurva believe founders at the early-stage care less about firm brand today?
5.) How does Apurva feel about investing in managers he has not met in person? How does the GP/LP fundraising process need to change? How does COVID change the fundraising process for venture funds? How will LPs react to these changes?
Items Mentioned In Today's Show:
Apurva's Fave Book: Principles: Life and Work by Ray Dalio
Apurva's Most Recent Investment: Sourcegraph
As always you can follow Harry and The Twenty Minute VC on Twitter here!
Likewise, you can follow Harry on Instagram here for mojito madness and all things 20VC.
Summary
Building applications on top of unbounded event streams is a complex endeavor, requiring careful integration of multiple disparate systems that were engineered in isolation. The ksqlDB project was created to address this state of affairs by building a unified layer on top of the Kafka ecosystem for stream processing. Developers can work with the SQL constructs that they are familiar with while automatically getting the durability and reliability that Kafka offers. In this episode Michael Drogalis, product manager for ksqlDB at Confluent, explains how the system is implemented, how you can use it for building your own stream processing applications, and how it fits into the lifecycle of your data infrastructure. If you have been struggling with building services on low level streaming interfaces then give this episode a listen and try it out for yourself.
Announcements
Hello and welcome to the Data Engineering Podcast, the show about modern data management
When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With 200Gbit private networking, scalable shared block storage, a 40Gbit public network, fast object storage, and a brand new managed Kubernetes platform, you’ve got everything you need to run a fast, reliable, and bullet-proof data platform. And for your machine learning workloads, they’ve got dedicated CPU and GPU instances. Go to dataengineeringpodcast.com/linode today to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
Are you spending too much time maintaining your data pipeline? Snowplow empowers your business with a real-time event data pipeline running in your own cloud account without the hassle of maintenance. Snowplow takes care of everything from installing your pipeline in a couple of hours to upgrading and autoscaling so you can focus on your exciting data projects. Your team will get the most complete, accurate and ready-to-use behavioral web and mobile data, delivered into your data warehouse, data lake and real-time streams. Go to dataengineeringpodcast.com/snowplow today to find out why more than 600,000 websites run Snowplow. Set up a demo and mention you’re a listener for a special offer!
You listen to this show to learn and stay up to date with what’s happening in databases, streaming platforms, big data, and everything else you need to know about modern data management. For even more opportunities to meet, listen, and learn from your peers you don’t want to miss out on this year’s conference season. We have partnered with organizations such as O’Reilly Media, Corinium Global Intelligence, ODSC, and Data Council. Upcoming events include the Software Architecture Conference in NYC, Strata Data in San Jose, and PyCon US in Pittsburgh. Go to dataengineeringpodcast.com/conferences to learn more about these and other events, and take advantage of our partner discounts to save money when you register today.
Your host is Tobias Macey and today I’m interviewing Michael Drogalis about ksqlDB, the open source streaming database layer for Kafka
Interview
Introduction
How did you get involved in the area of data management?
Can you start by describing what ksqlDB is?
What are some of the use cases that it is designed for?
How do the capabilities and design of ksqlDB compare to other solutions for querying streaming data with SQL such as Pulsar SQL, PipelineDB, or Materialize?
What was the motivation for building a unified project for providing a database interface on the data stored in Kafka?
How is ksqlDB architected?
If you were to rebuild the entire platform and its components from scratch today, what would you do differently?
What is the workflow for an analyst or engineer to design and build an application on top of ksqlDB?
What dialect of SQL is supported?
What kinds of extensions or built in functions have been added to aid in the creation of streaming queries?
How are table schemas defined and enforced?
How do you handle schema migrations on active streams?
Typically a database is considered a long term storage location for data, whereas Kafka is a streaming layer with a bounded amount of durable storage. What is a typical lifecycle of information in ksqlDB?
Can you talk through an example architecture that might incorporate ksqlDB including the source systems, applications that might interact with the data in transit, and any destinations sytems for long term persistence?
What are some of the less obvious features of ksqlDB or capabilities that you think should be more widely publicized?
What are some of the edge cases or potential pitfalls that users should be aware of as they are designing their streaming applications?
What is involved in deploying and maintaining an installation of ksqlDB?
What are some of the operational characteristics of the system that should be considered while planning an installation such as scaling factors, high availability, or potential bottlenecks in the architecture?
When is ksqlDB the wrong choice?
What are some of the most interesting/unexpected/innovative projects that you have seen built with ksqlDB?
What are some of the most interesting/unexpected/challenging lessons that you have learned while working on ksqlDB?
What is in store for the future of the project?
Contact Info
@michaeldrogalis on Twitter
michaeldrogalis on GitHub
LinkedIn
Parting Question
From your perspective, what is the biggest gap in the tooling or technology for data management today?
Links
ksqlDB
Confluent
Erlang
Onyx
Apache Storm
Stream Processing
Kafka
ksql
Kafka Streams
Pulsar
Podcast Episode
Pulsar SQL
PipelineDB
Podcast Episode
Materialize
Podcast Episode
Kafka Connect
RocksDB
Java Jar
CLI == Command Line Interface
PrestoDB
Podcast Episode
ANSI SQL
Pravega
Podcast Episode
Eventual Consistency
Confluent Cloud
MySQL
PostgreSQL
GraphQL
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Support Data Engineering Podcast
Today we deep dive into what startups can learn from the large SaaS incumbents of today.
Sara Varni: CMO @ Twilio on her biggest takeaways from her time at Salesforce.
Erica Schultz: President of Field Operations @ Confluent on her biggest takeaways from her time at Oracle.
Whitney Bouck: COO @ Hellosign on her biggest takeaways from her time at Box.
Leyla Seka: Partner @ Operator Collective on her biggest takeaways from her time at Salesforce.
Ryan Bonnici: CMO @ G2 on his biggest takeaways from his time at Salesforce.
Ryan Barretto: SVP @ Sprout Social on his biggest takeaways from his time at Salesforce
Tien Tzuo: Founder & CEO @ Zuora on his biggest takeaways from his time at Salesforce.
Paul Albright: Board member @ Clarizen on his biggest takeaways from his time at SuccessFactors.
Jaleh Rezaei: Founder & CEO @ Mutiny on her biggest takeaways from her time at Gusto.
Eugenio Pace: Founder & CEO @ Auth0 on his biggest takeaways from his time at Microsoft.
Liat Bycel: VP @ Airtable on her biggest takeaways from her time at Twitter.
Mark Goldberg: Partner @ Index on his biggest takeaways from his time at Dropbox.
Read the full transcript on our blog.
If you would like to find out more about the show and the guests presented, you can follow us on Twitter here:
Jason Lemkin
Harry Stebbings
SaaStr
Erica Schultz is Chief Revenue Officer @ New Relic, the company that gives you the real time insights your software driven business needs to innovate faster. Prior to their IPO, New Relic raised over $214m in funding from some of the best in the business including Benchmark, Insight Venture Partners and Blackrock, to name a few. As for Erica, under her CRO role, she leads all go-to-market functions including Marketing, Sales, Operations, Customer Success, Services, and Support. Prior to New Relic, Erica served as Executive Vice President of Global Sales and Customer Success at LivePerson and before that, Erica had a 16-year tenure with Oracle Corporation, where she founded and led numerous teams within the sales organization, including pioneering the company's cloud business, and leading teams for North American and Latin American markets.
In Today's Episode We Discuss:
How Erica made her way into the world of SaaS and came to be Chief Revenue Officer @ New Relic? What were some of her biggest takeaways from her incredible 16 year journey with Oracle?
Why does Erica believe that enterprise is a "company sport"? Why does each department need to re-platform when making the move to enterprise? How can founders know when is the right time to make the move from SMB to enterprise? Where does Erica often see founders make mistakes with this scaling?
How does the move to enterprise fundamentally impact the sales team? How does the structure of the sales team change with the move? How does the role of marketing change with the move to enterprise? How does this move impact the relationship between sales and marketing? How should compensation plans be altered with the move?
With the scaling of departments and teams, what has Erica seen work really well when it comes to making cross-functional teams communicate really well? What are the inflection points where Erica often see communication or process begin to breakdown? How does Erica ensure the team are still in the trenches with the clients despite the scaling?
From Erica's experience, how do the very best sales reps build relationships with their prospects? Where do many go wrong? How much time does Erica believe reps should be given when it comes to translating relationships to dollars? What is the right way to think about payback period today?
Erica's 60 Second SaaStr:
What does Erica know now that he wishes he had known at the beginning?
The optimal relationship between CRO and CEO?
The hardest element of being CRO @ New Relic?
Read the full transcript on our blog.
If you would like to find out more about the show and the guests presented, you can follow us on Twitter here:
Jason Lemkin
Harry Stebbings
SaaStr
Summary
To process your data you need to know what shape it has, which is why schemas are important. When you are processing that data in multiple systems it can be difficult to ensure that they all have an accurate representation of that schema, which is why Confluent has built a schema registry that plugs into Kafka. In this episode Ewen Cheslack-Postava explains what the schema registry is, how it can be used, and how they built it. He also discusses how it can be extended for other deployment targets and use cases, and additional features that are planned for future releases.
Preamble
Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at dataengineeringpodcast.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your data pipelines or trying out the tools you hear about on the show.
Continuous delivery lets you get new features in front of your users as fast as possible without introducing bugs or breaking production and GoCD is the open source platform made by the people at Thoughtworks who wrote the book about it. Go to dataengineeringpodcast.com/gocd to download and launch it today. Enterprise add-ons and professional support are available for added peace of mind.
Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
You can help support the show by checking out the Patreon page which is linked from the site.
To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
Your host is Tobias Macey and today I’m interviewing Ewen Cheslack-Postava about the Confluent Schema Registry
Interview
Introduction
How did you get involved in the area of data engineering?
What is the schema registry and what was the motivating factor for building it?
If you are using Avro, what benefits does the schema registry provide over and above the capabilities of Avro’s built in schemas?
How did you settle on Avro as the format to support and what would be involved in expanding that support to other serialization options?
Conversely, what would be involved in using a storage backend other than Kafka?
What are some of the alternative technologies available for people who aren’t using Kafka in their infrastructure?
What are some of the biggest challenges that you faced while designing and building the schema registry?
What is the tipping point in terms of system scale or complexity when it makes sense to invest in a shared schema registry and what are the alternatives for smaller organizations?
What are some of the features or enhancements that you have in mind for future work?
Contact Info
ewencp on GitHub
Website
@ewencp on Twitter
Parting Question
From your perspective, what is the biggest gap in the tooling or technology for data management today?
Links
Kafka
Confluent
Schema Registry
Second Life
Eve Online
Yes, Virginia, You Really Do Need a Schema Registry
JSON-Schema
Parquet
Avro
Thrift
Protocol Buffers
Zookeeper
Kafka Connect
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Support Data Engineering Podcast
.jpg?width=400){kind=small}
