PlanetScale

Sugu Sougoumarane, creator of Vitess, comes off sabbatical to bring Vitess to Postgres. We discuss what motivated Sugu to come off sabbatical, why now is the time, the technical challenges of doing so, the implementation details of Multigres (Vitess for Postgres). We also discuss the state of Postgres at scale. Join the discussion Changelog++ members save 6 minutes on this episode because they made the ads disappear. Join today! Sponsors: Depot – 10x faster builds? Yes please. Build faster. Waste less time. Accelerate Docker image builds, and GitHub Actions workflows. Easily integrate with your existing CI provider and dev workflows to save hours of build time. CodeRabbit – AI-native code reviews, built for the modern dev stack. — CodeRabbit is your always-on code reviewer—flagging hallucinations, surfacing smells, and enforcing standards, all without leaving your IDE or GitHub PRs. Trusted by top teams to ship better code, faster. Start free at CodeRabbit.ai Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. Featuring: Sugu Sougoumarane – GitHub, LinkedIn, X Adam Stacoviak – Website, GitHub, LinkedIn, Mastodon, X Jerod Santo – Website, GitHub, LinkedIn, Mastodon, X Show Notes: Multigres Vitess Supabase Neon PlanetScale Something missing or broken? PRs welcome!

Sugu Sougoumarane

Creator of Vitess

SupabasePlanetScale

About Taylor Taylor Barnett is a Staff Developer Advocate at PlanetScale. She is passionate about building great developer experiences emphasizing empathy within product, documentation, and other developer-facing projects. For the past decade, Taylor has worked at various data and API-focused startups in software development and developer relations. In her free time, as a firm believer in "touching grass," she's either gardening, taking long walks, climbing rocks with friends, trying to find the funkiest sour beers, or hanging out with her corgi, Yoda, and spouse in Austin, Texas. Links Referenced: PlanetScale: https://planetscale.com/ Twitter: https://twitter.com/taylor_atx Personal website: https://taylorbar.net

Taylor Barnett

Staff Developer Advocate

PlanetScale

Summary One of the most critical aspects of software projects is managing its data. Managing the operational concerns for your database can be complex and expensive, especially if you need to scale to large volumes of data, high traffic, or geographically distributed usage. Planetscale is a serverless option for your MySQL workloads that lets you focus on your applications without having to worry about managing the database or fight with differences between development and production. In this episode Nick van Wiggeren explains how the Planetscale platform is implemented, their strategies for balancing maintenance and improvements of the underlying Vitess project with their business goals, and how you can start using it today to free up the time you spend on database administration. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudder Build Data Pipelines. Not DAGs. That’s the spirit behind Upsolver SQLake, a new self-service data pipeline platform that lets you build batch and streaming pipelines without falling into the black hole of DAG-based orchestration. All you do is write a query in SQL to declare your transformation, and SQLake will turn it into a continuous pipeline that scales to petabytes and delivers up to the minute fresh data. SQLake supports a broad set of transformations, including high-cardinality joins, aggregations, upserts and window operations. Output data can be streamed into a data lake for query engines like Presto, Trino or Spark SQL, a data warehouse like Snowflake or Redshift., or any other destination you choose. Pricing for SQLake is simple. You pay $99 per terabyte ingested into your data lake using SQLake, and run unlimited transformation pipelines for free. That way data engineers and data users can process to their heart’s content without worrying about their cloud bill. For data engineering podcast listeners, we’re offering a 30 day trial with unlimited data, so go to dataengineeringpodcast.com/upsolver today and see for yourself how to avoid DAG hell. Your host is Tobias Macey and today I’m interviewing Nick van Wiggeren about Planetscale, a serverless and globally distributed MySQL database as a service Interview Introduction How did you get involved in the area of data management? Can you describe what Planetscale is and the story behind it? What are the core problems that you are solving with the Planetscale platform? How might an engineering team address those challenges in the absence of Planetscale/Vitess? Can you describe how Planetscale is implemented? What are some of the addons that you have had to build on top of Vitess to make Planetscale What are the impacts that a serverless database has on the way teams approach their application/platform design and development? metrics exposed to help users optimize their usage What is your policy/philosophy for determining what capabilities to include in Vitess and what belongs in the Planetscale platform? What are the most interesting, innovative, or unexpected ways that you have seen Planetscale/Vitess used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Planetscale? When is Planetscale the wrong choice? What do you have planned for the future of Planetscale? Contact Info @nickvanwig on Twitter LinkedIn nickvanw on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story. To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Planetscale Vitess CNCF == Cloud Native Computing Foundation Hadoop OLTP == Online Transactional Processing Galera Yugabyte DB Podcast Episode CitusDB MariaDB SkySQL Podcast Episode CockroachDB Podcast Episode NewSQL AWS PrivateLink Planetscale Connect Segment Podcast Episode BigQuery The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast

Nick van Wiggeren

PlanetScale

This week we’re joined by Deepthi Sigireddi, Vitess Maintainer and engineer at PlanetScale — of course we’re talking about all things Vitess. We talk about its origin inside YouTube, how Vitess handles sharding, Deepthi’s journey to Vitess maintainer, when you should begin using it, and how it fits into cloud native infra. Join the discussion Changelog++ members get a bonus 10 minutes at the end of this episode and zero ads. Join today! Sponsors: InfluxData – The time series platform for building and operating time series applications — InfluxDB empowers developers to build IoT, analytics, and monitoring software. It’s purpose-built to handle massive volumes and countless sources of time-stamped data produced by sensors, applications, and infrastructure. Learn more at influxdata.com/changelog FireHydrant – The reliability platform for every developer. Incidents impact everyone, not just SREs. FireHydrant gives teams the tools to maintain service catalogs, respond to incidents, communicate through status pages, and learn with retrospectives. Try FireHydrant free for 14 days at firehydrant.io WorkOS – A platform that gives developers a set of building blocks for quickly adding enterprise-ready features to their application. Add Single Sign-On (Okta, Azure, Google, Microsoft OAuth), sync users from any SCIM directory, HRIS integration, audit trails (SIEM), free magic link sign-in. WorkOS is designed for developers and offers a single, elegant interface that abstracts dozens of enterprise integrations. Learn more and get started at WorkOS.com Sourcegraph – Move fast, even in big codebases. Sourcegraph is universal code search for every developer and team. Easily search across all the code that matters to you and your organization: find example code, explore and read code, debug issues, and more. Head to info.sourcegraph.com/changelog and click the button “Try Sourcegraph now” to get started. Featuring: Deepthi Sigireddi – GitHub, X Adam Stacoviak – Website, GitHub, LinkedIn, Mastodon, X Jerod Santo – Website, GitHub, LinkedIn, Mastodon, X Show Notes: vitess.io Vitess Architecture github.com/vitessio/vitess Vitess Benchmarks (Are we fast yet?) New in MySQL 8.0 CNCF Graduation Criteria v1.3 Something missing or broken? PRs welcome!

Deepthi Sigireddi

Engineer & Vitess Maintainer

PlanetScale

Vercel is a developer-first, frontend-focused platform. Together with Google and Meta, Vercel built Next.js, an open-source React framework that helps developers build high-performance web experiences with ease. PlanetScale is a MySQL-compatible serverless database platform that enables infinite SQL horizontal scale. Tools like Webflow and Squarespace have made web development accessible for casual programmers, but what does this mean for professional developers? This week’s Lifeboat badge goes to user Michael Thelin for their answer to How can I play a Spotify audio track with Python?. Find Guillermo on LinkedIn here. Find Sam on LinkedIn here. See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Guillermo Rauch

Founder & CEO

VercelZEITSocket.IO

Sam Lambert

CEO

PlanetScaleGitHub

PlanetScale is built on Vitess, the open-source database clustering system that runs at colossal scale hosting YouTube, Slack, and GitHub. A familiar theme: Big cloud companies aren’t set up for independent developers. Sam and Ceora discuss how serverless can get projects—even businesses—up and running quickly. Choosing the stack for a new business? Tools like Netlify can scale with your product, so you don’t have to change your architecture as you evolve. Staging environments should be a thing of the past. That’s why PlanetScale enables database branching. And finally, a question from Law Stack Exchange: Can satellite images be copyrighted? See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Sam Lambert

CEO

PlanetScaleGitHub

Vitess is a cloud native database clustering system for horizontal scaling of MySQL. It was built for YouTube, open sourced, and has recently graduated from the CNCF. Two members of the team who wrote and ran Vitess at YouTube, Jiten Vaidya and Sugu Sougoumarane, are CEO and CTO of PlanetScale; a company they founded to support Vitess commercially. They join Craig and Adam to talk databases. Do you have something cool to share? Some questions? Let us know: web: kubernetespodcast.com mail: kubernetespodcast@google.com twitter: @kubernetespod Chatter of the week Our meetup at KubeCon The WeWork booth at KubeCon You Must Build A Boat You Have To Win The Game News of the week Anthos news from Next UK Multi-cluster management with Anthos GitLab Serverless with Cloud Run for Anthos Project Antrea from VMware Managed Istio GA on IBM Kubernetes Service IBM open sources developer tools Kui and iter8 Episode 47 with Kim Lewandowski Solo.io introduces Autopilot Hubble, from Cilium ByteBuilders introduces Kubeform Cloudbees adds Jenkins X UI to their community distribution Juniper updates Contrail Slack Vitess case study Debugging network stalls on Kubernetes by Theo Julienne at GitHub Volterra's control plane for distributed PaaS Gravitational takes $25m investment Datadog's 2019 container report Aqua Security acquires Cloudsploit CNCF 2019 award winners Episode 72 with Lachlan Evenson Episode 77 with Katharine Berry Links from the interview Vitess About Jiten and Sugu Graduated from the CNCF Database shards Vitess history YouTube acquired by Google in 2006 Go; 10 years old Google storage systems: Bigtable Colossus Scaling MySQL in the cloud with Vitess and Kubernetes and Cloud Native MySQL Sharding with Vitess and Kubernetes by Anthony Yeh, Google Cloud Case studies: Stich Labs, Hubspot, JD.com Vitess at KubeCon: Vitess: Stateless Storage in the Cloud by Sugu Sougoumarane Geo-partitioning with Vitess by Deepthi Sigireddi and Jiten Vaidya How to Migrate a MySQL Database to Vitess by Sugu Sougoumarane & Morgan Tocker Gone in 60 Minutes: Migrating 20 TB from AKS to GKE in an Hour with Vitess by Derek Perkins from Nozzle Postgres support PlanetScale Announcing PlanetScale's CNDb The name Voltron Strong Bad's advice on naming things Jiten Vaidya and Sugu Sougoumarane on Twitter

Jiten Vaidya

CEO

PlanetScale

Sugu Sougoumarane

Creator of Vitess

PlanetScaleSupabase

In today's episode, I interview Sam Lambert, Director of Systems at GitHub. Mentioned or relevant to today's episode GitHub @isamlambert Hubot Slack (chat) Hubot plugins ChatOps: How GitHub Manages MySQL Hubot: The Hubber that never sleeps. Whiteboard GitHub Pull Request "DBA" - DataBase Administrator Today's episode is sponsored by Rollbar. With Rollbar, you get the context, insights and control you need to find and fix bugs faster. Rollbar is offering Developer Tea listeners the Bootstrap Plan, free for 90 days (300,000 errors tracked for free)! And lastly... Please take a moment and subscribe and review the show! Click here to review Developer Tea in iTunes.

Sam Lambert

CEO

PlanetScaleGitHub

In today's episode, I interview Sam Lambert, Director of Systems at GitHub. Mentioned or relevant to today's episode GitHub.com Sam Lambert Example of a Systems Engineer job at GitHub (they're hiring!) Jonathan on GitHub Git (not GitHub) Subversion Mercurial (another SCM) git-svn Telemetry GitHub pull requests GitHub issues "Human Development" (Interview with Ernie Miller) Today's episode is sponsored by Hired.com! If you are looking for a job as a developer or a designer and don't know where to start, head over to Hired now! If you get a job through this special link, you'll receive a $2,000 bonus - that's twice the normal bonus provided by Hired. Thanks again to Hired for sponsoring the show! And lastly... Please take a moment and subscribe and review the show! Click here to review Developer Tea in iTunes.

Sam Lambert

CEO

PlanetScaleGitHub