How to measure AI developer productivity in 2025 | Nicole Forsgren
Nicole Forsgren created the most widely used frameworks for measuring developer productivity—DORA and SPACE. She wrote the foundational book Accelerate and is about to release her newest book, Frictionless, a practical guide for helping teams move faster in the AI era. She’s currently Senior Director of Developer Intelligence at Google.
We discuss:
1. Why most productivity metrics are a lie
2. Signs that your engineering team could be moving much faster
3. Why AI accelerates coding but developers aren’t speeding up as much as you think
4. AI’s impact on engineers getting into “flow”
5. Her framework for building and scaling a developer experience team
6. The three components of developer experience: flow state, cognitive load, and feedback loops
—
Brought to you by:
Mercury—The art of simplified finances: https://mercury.com/
WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs: https://workos.com/lenny
Coda—The all-in-one collaborative workspace: https://coda.io/lenny
—
Where to find Nicole Forsgren:
• Twitter: https://twitter.com/nicolefv
• LinkedIn: https://www.linkedin.com/in/nicolefv/
• Website: https://nicolefv.com/
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Introduction to Nicole Forsgren
(05:09) The concept of developer experience (DevEx)
(08:33) Flow state and cognitive load in the age of AI
(12:02) Challenges in measuring productivity with AI
(21:19) The importance of developer experience for business value
(22:20) Common issues and solutions in developer experience
(26:49) Signs your eng team is moving too slow
(29:52) How AI is improving productivity
(33:32) Real examples of productivity improvements
(36:35) Introducing her new book, Frictionless
(43:40) How to get started building a DevEx team
(45:15) The impact of forming developer experience teams
(46:15) How to measure the impact of DevEx teams
(48:53) Measuring the impact of AI tools on productivity
(55:16) Survey design for developer experience
(57:59) Popular AI tools for developers
(59:08) Bringing a product mindset to DevEx improvements
(01:00:40) AI corner
(01:02:33) Lightning round and final thoughts
—
Referenced:
• How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google): https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer
• DORA: https://dora.dev/
• The SPACE framework: A comprehensive guide to developer productivity: https://getdx.com/blog/space-metrics/
• Measuring developer productivity with the DX Core 4: https://getdx.com/research/measuring-developer-productivity-with-the-dx-core-4/
• Gloria Mark’s website: https://gloriamark.com/
• Taking Flight with Copilot: https://dl.acm.org/doi/10.1145/3589996
• DevEx in Action: https://spawn-queue.acm.org/doi/10.1145/3639443
• CodeX: https://openai.com/codex/
• Devin: https://devin.ai/
• Abi Noda on LinkedIn: https://www.linkedin.com/in/abinoda/
• DX is joining Atlassian: https://getdx.com/blog/dx-is-joining-atlassian/
• GitHub Copilot: https://github.com/features/copilot
• Cursor: https://cursor.com/
• The rise of Cursor: The $300M ARR AI tool that engineers can’t stop using | Michael Truell (co-founder and CEO): https://www.lennysnewsletter.com/p/the-rise-of-cursor-michael-truell
• Gemini Code Assist: https://codeassist.google/
• Claude Code: https://www.claude.com/product/claude-code
• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper
• Love Is Blind on Netflix: https://www.netflix.com/title/80996601
• Shrinking on AppleTV+: https://tv.apple.com/us/show/shrinking/umc.cmc.apzybj6eqf6pzccd97kev7bs
• Ninja Creami: https://www.amazon.com/Ninja-NC301-CREAMi-Containers-Bundle/dp/B0BLGR5JPV/
• Jura coffee maker: https://www.amazon.com/Jura-Nordic-Automatic-Coffee-Machine/dp/B0CF65BFZ1/
—
Recommended books:
• Frictionless: https://developerexperiencebook.com/
• DevEx Workbook: https://developerexperiencebook.com/#workbook
• Outlive: The Science and Art of Longevity: https://www.amazon.com/Outlive-Longevity-Peter-Attia-MD/dp/0593236599
• Back Mechanic: https://www.amazon.com/Back-Mechanic-Stuart-McGill-2015-09-30/dp/B01FKSGJYC
• How Big Things Get Done: The Surprising Factors That Determine the Fate of Every Project, from Home Renovations to Space Exploration and Everything in Between: https://www.amazon.com/How-Big-Things-Get-Done/dp/0593239512/
• The Undoing Project: A Friendship That Changed Our Minds: https://www.amazon.com/dp/B01KBM82M4/
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
Lenny may be an investor in the companies discussed.
To hear more, visit www.lennysnewsletter.com
Developer productivity with Dr. Nicole Forsgren (creator of DORA, co-creator of SPACE)
Supported by Our Partners
• WorkOS — The modern identity platform for B2B SaaS
• CodeRabbit — Cut code review time and bugs in half
• Augment Code — AI coding assistant that pro engineering teams love
—
How do you architect a live streaming system to deal with more load than it’s ever been done before? Today, we hear from an architect of such a system: Ashutosh Agrawal, formerly Chief Architect of JioCinema (and currently Staff Software Engineer at Google DeepMind.)
We take a deep dive into video streaming architecture, tackling the complexities of live streaming at scale (at tens of millions of parallel streams) and the challenges engineers face in delivering seamless experiences. We talk about the following topics:
• How large-scale live streaming architectures are designed
• Tradeoffs in optimizing performance
• Early warning signs of streaming failures and how to detect them
• Why capacity planning for streaming is SO difficult
• The technical hurdles of streaming in APAC regions
• Why Ashutosh hates APMs (Application Performance Management systems)
• Ashutosh’s advice for those looking to improve their systems design expertise
• And much more!
—
Timestamps
(00:00) Intro
(01:28) The world record-breaking live stream and how support works with live events
(05:57) An overview of streaming architecture
(21:48) The differences between internet streaming and traditional television.l
(22:26) How adaptive bitrate streaming works
(25:30) How throttling works on the mobile tower side
(27:46) Leading indicators of streaming problems and the data visualization needed
(31:03) How metrics are set
(33:38) Best practices for capacity planning
(35:50) Which resources are planned for in capacity planning
(37:10) How streaming services plan for future live events with vendors
(41:01) APAC specific challenges
(44:48) Horizontal scaling vs. vertical scaling
(46:10) Why auto-scaling doesn’t work
(47:30) Concurrency: the golden metric to scale against
(48:17) User journeys that cause problems
(49:59) Recommendations for learning more about video streaming
(51:11) How Ashutosh learned on the job
(55:21) Advice for engineers who would like to get better at systems
(1:00:10) Rapid fire round
—
The Pragmatic Engineer deepdives relevant for this episode:
• Software architect archetypes https://newsletter.pragmaticengineer.com/p/software-architect-archetypes
• Engineering leadership skill set overlaps https://newsletter.pragmaticengineer.com/p/engineering-leadership-skillset-overlaps
• Software architecture with Grady Booch https://newsletter.pragmaticengineer.com/p/software-architecture-with-grady-booch
—
See the transcript and other references from the episode at https://newsletter.pragmaticengineer.com/podcast
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@pragmaticengineer.com.
Get full access to The Pragmatic Engineer at newsletter.pragmaticengineer.com/subscribe
How to measure and improve developer productivity | Nicole Forsgren (Microsoft Research, GitHub, Google)
This episode is brought to you by DX—a platform for measuring and improving developer productivity.
—
Dr. Nicole Forsgren is a developer productivity and DevOps expert who works with engineering organizations to make work better. Best known as co-author of the Shingo Publication Award-winning book Accelerate and the DevOps Handbook, 2nd edition and author of the State of DevOps Reports, she has helped some of the biggest companies in the world transform their culture, processes, tech, and architecture. Nicole is currently a Partner at Microsoft Research, leading developer productivity research and strategy, and a technical founder/CEO with a successful exit to Google. In a previous life, she was a software engineer, sysadmin, hardware performance engineer, and professor. She has published several peer-reviewed journal papers, has been awarded public and private research grants (funders include NASA and the NSF), and has been featured in the Wall Street Journal, Forbes, Computerworld, and InformationWeek. In today’s podcast, we discuss:
• Two frameworks for measuring developer productivity: DORA and SPACE
• Benchmarks for what good and great look like
• Common mistakes to avoid when measuring developer productivity
• Resources and tools for improving your metrics
• Signs your developer experience needs attention
• How to improve your developer experience
• Nicole’s Four-Box framework for thinking about data and relationships
—
Find the full transcript at: https://www.lennysnewsletter.com/p/how-to-measure-and-improve-developer
—
Where to find Nicole Forsgren:
• Twitter: https://twitter.com/nicolefv
• LinkedIn: https://www.linkedin.com/in/nicolefv/
• Website: https://nicolefv.com/
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• Twitter: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Nicole’s background
(07:55) Unpacking the terms “developer productivity,” “developer experience,” and “DevOps”
(10:06) How to move faster and improve practices across the board
(13:43) The DORA framework
(18:54) Benchmarks for success
(22:33) Why company size doesn’t matter
(24:54) How to improve DevOps capabilities by working backward
(29:23) The SPACE framework and choosing metrics
(32:51) How SPACE and DORA work together
(35:39) Measuring satisfaction
(37:52) Resources and tools for optimizing metrics
(41:29) Nicole’s current book project
(45:43) Common pitfalls companies run into when rolling out developer productivity/optimizations
(47:42) How the DevOps space has progressed
(50:07) The impact of AI on the developer experience and productivity
(54:04) First steps to take if you’re trying to improve the developer experience
(55:15) Why Google is an example of a company implementing DevOps solutions well
(56:11) The importance of clear communication
(57:32) Nicole’s Four-Box framework
(1:05:15) Advice on making decisions
(1:08:56) Lightning round
—
Referenced:
• Chef: https://www.chef.io/
• DORA: https://dora.dev/
• GitHub: https://github.com/
• Microsoft Research: https://www.microsoft.com/en-us/research/
• What is DORA?: https://devops.com/what-is-dora-and-why-you-should-care/
• Dustin Smith on LinkedIn: https://www.linkedin.com/in/dustin-smith-b0525458/
• Nathen Harvey on LinkedIn: https://www.linkedin.com/in/nathen/
• What is CI/CD?: https://about.gitlab.com/topics/ci-cd/
• Trunk-based development: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development
• DORA DevOps Quick Check: https://dora.dev/quickcheck/
• Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations: https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339
• The SPACE of Developer Productivity: https://queue.acm.org/detail.cfm?id=3454124
• DevOps Metrics: Nicole Forsgren and Mik Kersten: https://queue.acm.org/detail.cfm?id=3182626
• How to Measure Anything: Finding the Value of Intangibles in Business: https://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/
• GitHub Copilot: https://github.com/features/copilot
• Tabnine: https://www.tabnine.com/the-leading-ai-assistant-for-software-development
• Nicole’s Decision-Making Spreadsheet: https://docs.google.com/spreadsheets/d/1wItAODkhZ-zKnnFbyDERCd8Hq2NQ03WPvCfigBQ5vpc/edit?usp=sharing
• How to do linear regression and correlation analysis: https://www.lennysnewsletter.com/p/linear-regression-and-correlation-analysis
• Good Strategy/Bad Strategy: The difference and why it matters: https://www.amazon.com/Good-Strategy-Bad-difference-matters/dp/1781256179/
• Designing Your Life: How to Build a Well-Lived, Joyful Life: https://www.amazon.com/Designing-Your-Life-Well-Lived-Joyful/dp/1101875321
• Ender’s Game: https://www.amazon.com/Enders-Game-Ender-Quintet-1/dp/1250773024/ref=tmm_pap_swatch_0
• Suits on Netflix: https://www.netflix.com/title/70195800
• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso
• Never Have I Ever on Netflix: https://www.netflix.com/title/80179190
• Eight Sleep: https://www.eightsleep.com/
• COSRX face masks: https://www.amazon.com/COSRX-Advanced-Secretion-Hydrating-Moisturizing/dp/B08JSL9W6K/
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
—
Lenny may be an investor in the companies discussed.
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.lennysnewsletter.com/subscribe
Innovation Through Software Development and IT
One of the recurring themes we talk about a lot on the a16z Podcast is how software changes organizations, and vice versa... More broadly: it’s really about how companies of all kinds innovate with the org structures and tools that they have.
But we've come a long way from the question of "does IT matter" to answering the question of what org structures, processes, architectures, and roles DO matter when it comes to companies -- of all sizes -- innovating through software and more.
So in this episode (a re-run of a popular episode from a couple years ago), two of the authors of the book Accelerate: The Science of Lean Software and DevOps, by Nicole Forsgren, Jez Humble, and Jean Kim join Sonal Chokshi to share best practices and large-scale findings about high performing companies (including those who may not even think they’re tech companies). Nicole was co-founder and CEO of Dora, which was acquired by Google in December 2018; she will soon be joining GitHub as VP of Research & Strategy. Jez was CTO at DORA; is currently in Developer Relations at Google Cloud; and is the co-author of the books The DevOps Handbook, Lean Enterprise, and Continuous Delivery.
Stay Updated:
Find a16z on YouTube: YouTube
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
Five Characteristics That Define the Cloud with Nicole Forsgren, PhD
About Nicole Forsgren, PhD
Dr. Nicole Forsgren does research and strategy at Google Cloud following the acquisition of her startup DevOps Research and Assessment (DORA) by Google. She is co-author of the Shingo Publication Award winning book Accelerate: The Science of Lean Software and DevOps, and is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been an entrepreneur, professor, sysadmin, and performance engineer. Nicole’s work has been published in several peer-reviewed journals. Nicole earned her PhD in Management Information Systems from the University of Arizona, and is a Research Affiliate at Clemson University and Florida International University.
Links Referenced
Twitter Username: @nicolefv
LinkedIn URL: https://www.linkedin.com/in/nicolefv/
Personal site: nicolefv.com
Company site: cloud.google.com/devops
Manifold: https://www.manifold.co/
How to Grade DevOps Teams with Nicole Forsgren, PhD
About Nicole Forsgren, PhD
Dr. Nicole Forsgren does research and strategy at Google Cloud following the acquisition of her startup DevOps Research and Assessment (DORA) by Google. She is co-author of the Shingo Publication Award winning book Accelerate: The Science of Lean Software and DevOps, and is best known for her work measuring the technology process and as the lead investigator on the largest DevOps studies to date. She has been an entrepreneur, professor, sysadmin, and performance engineer. Nicole’s work has been published in several peer-reviewed journals. Nicole earned her PhD in Management Information Systems from the University of Arizona, and is a Research Affiliate at Clemson University and Florida International University.
Links Referenced:
Twitter Username: @nicolefv
LinkedIn URL: https://www.linkedin.com/in/nicolefv/
Personal site: nicolefv.com
Company site: cloud.google.com/devops
X-Team: x-team.com/cloud
a16z Podcast: Feedback Loops -- Company Culture, Change, and DevOps
with Nicole Forsgren (@nicolefv), Jez Humble (@jezhumble) and Sonal Chokshi (@smc90)
From the old claim that "IT doesn't matter" and question of whether tech truly drives organizational performance, we've been consumed with figuring out how to measure -- and predict -- the output and outcomes, the performance and productivity of software. It's not useful to talk about what happens in one isolated team or successful company; we need to be able to make it happen at any company -- of any size, industry vertical, or architecture/tech stack. But can we break the false dichotomy of performance vs. speed; is it possible to have it all?
This episode of the a16z Podcast boldly goes where no man has gone before -- trying to answer those elusive questions -- by drawing on one of the largest, large-scale studies of software and organizational performance out there, as presented in the new book, Accelerate: The Science of Lean Software and DevOps -- Building and Scaling High Performing Technology Organizations by Nicole Forsgren, Jez Humble, and Gene Kim. Forsgren (co-founder and CEO at DevOps Research and Assessment - DORA; PhD in Management Information Systems; formerly at IBM) and Humble (co-founder and CTO at DORA; formerly at 18F; and co-author of The DevOps Handbook, Lean Enterprise, and Continuous Delivery) share the latest findings about what drives performance in companies of all kinds.
But what is DevOps, really? And beyond the definitions and history, where does DevOps fit into the broader history and landscape of other tech movements (such as lean manufacturing, agile development, lean startups, microservices)? Finally, what kinds of companies are truly receptive to change, beyond so-called organizational "maturity" scores? And for pete's sake, can we figure out how to measure software productivity already?? All this and more in this episode!
Stay Updated:
Find a16z on YouTube: YouTube
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.