When AI Starts Writing the Pull Requests with Madelyn Olson
AI-generated code is no longer just producing low-quality pull requests. According to AWS Principal Engineer and Valkey core maintainer Madelyn Olson, the quality of AI-assisted contributions has improved dramatically in just the last few months.
In this episode of Screaming in the Cloud, Corey Quinn and Madelyn discuss how AI is changing open source development, the growing burden on maintainers, and how projects like Valkey are using AI to find bugs, improve security, and harden production systems. They also explore Valkey's continued growth, the future of software development, and why experience, operational knowledge, and community still matter in an age where code is becoming cheaper to create.
*Since this episode was recorded without video, we thought it was the perfect opportunity to get creative with AI!
Show Highlights:
(00:00) Open Source Stability Push
(00:32) Reinvent Afterglow Banter
(01:40) AI PRs Get Better
(04:36) Whimsy Versus AI Slop
(06:14) AI Security Hunting Reality
(09:12) Maintainers Adapt to AI
(11:28) Valkey Fork Wins Adoption
(14:43) Fighting the AI Tidal Wave
(23:45) Next Five Years and Roadmap
(27:02) Release Woes and Where to Follow
About Madelyn:
Madelyn Olson is a co-creator and core maintainer of Valkey, a high-performance key-value datastore, and a Principal Engineer at Amazon Web Services (AWS). She specializes in building secure, highly reliable systems and is passionate about collaborating with open-source communities. In her role at Amazon, Madelyn serves as a Principal Software Development Engineer for Amazon ElastiCache and Amazon MemoryDB, where she focuses on advancing distributed data technologies and contributing to the growth and success of the Valkey project.
Sponsored by:
duckbillhq.com
Steering Through Open Source Waters with Madelyn Olson
This episode features Madelyn Olson, maintainer for the open-source project Valkey, to discuss the growth and impact of open-source projects in the tech industry. Corey and Madelyn explore the transformations within these projects, particularly the challenges and shifts in governance and licensing practices that affect how companies like AWS contribute to and utilize open-source software. Furthermore, Madelyn shares insights into the motivations behind Valkey, its differentiation from Redis, and the broader implications for open-source sustainability and corporate involvement.
Show Highlights:
(00:00) Introduction and discussion on AWS's approach to open-source
(01:41) Recap of the Redis controversy and licensing changes
(02:35) Madelyn's role at AWS and her work on ElastiCache and MemoryDB
(04:11) The enduring relevance and importance of open source in solving global technology problems
(06:15) The freedoms of open source and the broad implications for software development
(08:19) The evolution of governance and project management in the Valkey project
(09:53) The full transition of Madelyn's efforts from Redis to Valkey
(17:27) Why Valkey was created and its future direction
(24:57) The separation of duties between Madelyn's roles at AWS and the Valkey project
(32:34) Closing thoughts and where to find more information on Valkey
About Madelyn:
Madelyn Olson is a co-creator and maintainer of Valkey, a high-performance kev-value data store and Principal Engineer at Amazon Web Services (AWS). She focuses on building secure and highly reliable features, with a passion for working with open-source communities.
Links Referenced:
Website: https://valkey.io/
Linkedin: https://www.linkedin.com/in/madelyn-olson-valkey/
GitHub: https://github.com/madolson
Twitter: https://x.com/reconditerose
*Sponsor
Panoptica: https://www.panoptica.app/
Valkey: A Redis Fork with a Future
Valkey, a Redis fork supported by the Linux Foundation, challenges Redis' new license. In this episode, Madelyn Olson, a lead contributor to the Valkey project and former Redis core contributor, along with Ping Xie, Staff Software Engineer at Google and Dmitry Polyakovsky, Consulting Member of Technical Staff at Oracle highlights concerns about the shift to a more restrictive license at Open Source Summit 2024 in Seattle.
Despite Redis' free license for end users, many contributors may not support it. Valkey, with significant industry backing, prioritizes continuity and a smooth transition for Redis users. AWS, along with Google and Oracle maintainers, emphasizes the importance of open, permissive licenses for large tech companies. Valkey plans incremental updates and module development in Rust to enhance functionality and attract more engineers. The focus remains on compatibility, continuity, and consolidating client behaviors for a robust ecosystem.
Learn more from The New Stack about the Valkey Project and changes to Open Source licensing
Linux Foundation Backs 'Valkey' Open Source Fork of Redis
Redis Pulls Back on Open Source Licensing, Citing Stingy Cloud Services
HashiCorp's Licensing Change is only the Latest Challenge to Open Source
Join our community of newsletter subscribers to stay on top of the news and at the top of your game.
Redis is not just a Cache
Redis is not just a cache. It is used in the broader cloud native ecosystem, fits into many service-oriented architectures, and simplifies the deployment and development of modern applications, according to Madelyn Olson, a principal engineer at AWS, during an interview on the New Stack Makers at KubeCon North America in Detroit.
Olson said that people have a primary backend database or some other workflow that takes a long time to run. They store the intermediate results in Redis, which provides lower latency and higher throughput.
"But there are plenty of other ways you can use Redis," Olson said. "One common way is what I like to call it a data projection API. So you basically take a bunch of different sources of data, maybe a Postgres database, or some other type of Cassandra database, and you project that data into Redis. And then you just pull from the Redis instance. This is a really great, great use case for low latency applications."
Redis creator Salvatore Sanfilippo's approach provides a lesson in how to contribute to open source, which Olson recounted in our interview.
Olson said he was the only maintainer with write permissions for the project. That meant contributors would have to engage quite a bit to get a response from Sanfilippo. So Olson did what open source contributors do when they want to get noticed. She "chopped wood and carried water," a term that in open source reflects on working to take care of tasks that need attention. That helped Sanfilippo scale himself a bit and helped Olson get involved in the project.
It is daunting to get into open source development work, Olson said. A new contributor will face people with a lot more experience and get afraid to open issues. But if a contributor has a use case and helps with documentation or a bug, then most open source maintainers are willing to help.
"One big problem throughout open source is, they're usually resource constrained, right?," Olson said. "Open source is oftentimes a lot of volunteers. So they're usually very willing to get more people to help with the project."
What's it like now working at AWS on open source projects?
Things have changed a lot since Olson joined AWS in 2015, Olson said. APIs were proprietary back in those days. Today, it's almost the opposite of how it used to be.
To keep something internal now requires approval, Olson said. Internal differentiation is not needed. For example, open source Redis is most important, with AWS on top as the managed service.
