Search a title or topic

Over 20 million podcasts, powered by 

Player FM logo
Artwork

Content provided by Brook Perry. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brook Perry or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.
Player FM - Podcast App
Go offline with the Player FM app!

Measuring AI code assistants and agents with the AI Measurement Framework

41:14
 
Share
 

Manage episode 500434740 series 3338504
Content provided by Brook Perry. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brook Perry or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

In this episode of Engineering Enablement, DX CTO Laura Tacho and CEO Abi Noda break down how to measure developer productivity in the age of AI using DX’s AI Measurement Framework. Drawing on research with industry leaders, vendors, and hundreds of organizations, they explain how to move beyond vendor hype and headlines to make data-driven decisions about AI adoption.

They cover why some fundamentals of productivity measurement remain constant, the pitfalls of over-relying on flawed metrics like acceptance rate, and how to track AI’s real impact across utilization, quality, and cost. The conversation also explores measuring agentic workflows, expanding the definition of “developer” to include new AI-enabled contributors, and avoiding second-order effects like technical debt and slowed PR throughput.

Whether you’re rolling out AI coding tools, experimenting with autonomous agents, or just trying to separate signal from noise, this episode offers a practical roadmap for understanding AI’s role in your organization—and ensuring it delivers sustainable, long-term gains.

Where to find Laura Tacho:

• X: https://x.com/rhein_wein

• LinkedIn: https://www.linkedin.com/in/lauratacho/

• Website: https://lauratacho.com/

Where to find Abi Noda:

• LinkedIn: https://www.linkedin.com/in/abinoda

• Substack: ​​https://substack.com/@abinoda

In this episode, we cover:

(00:00) Intro

(01:26) The challenge of measuring developer productivity in the AI age

(04:17) Measuring productivity in the AI era — what stays the same and what changes

(07:25) How to use DX’s AI Measurement Framework

(13:10) Measuring AI’s true impact from adoption rates to long-term quality and maintainability

(16:31) Why acceptance rate is flawed — and DX’s approach to tracking AI-authored code

(18:25) Three ways to gather measurement data

(21:55) How Google measures time savings and why self-reported data is misleading

(24:25) How to measure agentic workflows and a case for expanding the definition of developer

(28:50) A case for not overemphasizing AI’s role

(30:31) Measuring second-order effects

(32:26) Audience Q&A: applying metrics in practice

(36:45) Wrap up: best practices for rollout and communication

Referenced:

  continue reading

84 episodes

Artwork
iconShare
 
Manage episode 500434740 series 3338504
Content provided by Brook Perry. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Brook Perry or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

In this episode of Engineering Enablement, DX CTO Laura Tacho and CEO Abi Noda break down how to measure developer productivity in the age of AI using DX’s AI Measurement Framework. Drawing on research with industry leaders, vendors, and hundreds of organizations, they explain how to move beyond vendor hype and headlines to make data-driven decisions about AI adoption.

They cover why some fundamentals of productivity measurement remain constant, the pitfalls of over-relying on flawed metrics like acceptance rate, and how to track AI’s real impact across utilization, quality, and cost. The conversation also explores measuring agentic workflows, expanding the definition of “developer” to include new AI-enabled contributors, and avoiding second-order effects like technical debt and slowed PR throughput.

Whether you’re rolling out AI coding tools, experimenting with autonomous agents, or just trying to separate signal from noise, this episode offers a practical roadmap for understanding AI’s role in your organization—and ensuring it delivers sustainable, long-term gains.

Where to find Laura Tacho:

• X: https://x.com/rhein_wein

• LinkedIn: https://www.linkedin.com/in/lauratacho/

• Website: https://lauratacho.com/

Where to find Abi Noda:

• LinkedIn: https://www.linkedin.com/in/abinoda

• Substack: ​​https://substack.com/@abinoda

In this episode, we cover:

(00:00) Intro

(01:26) The challenge of measuring developer productivity in the AI age

(04:17) Measuring productivity in the AI era — what stays the same and what changes

(07:25) How to use DX’s AI Measurement Framework

(13:10) Measuring AI’s true impact from adoption rates to long-term quality and maintainability

(16:31) Why acceptance rate is flawed — and DX’s approach to tracking AI-authored code

(18:25) Three ways to gather measurement data

(21:55) How Google measures time savings and why self-reported data is misleading

(24:25) How to measure agentic workflows and a case for expanding the definition of developer

(28:50) A case for not overemphasizing AI’s role

(30:31) Measuring second-order effects

(32:26) Audience Q&A: applying metrics in practice

(36:45) Wrap up: best practices for rollout and communication

Referenced:

  continue reading

84 episodes

All episodes

×
 
Loading …

Welcome to Player FM!

Player FM is scanning the web for high-quality podcasts for you to enjoy right now. It's the best podcast app and works on Android, iPhone, and the web. Signup to sync subscriptions across devices.

 

Copyright 2025 | Privacy Policy | Terms of Service | | Copyright
Listen to this show while you explore
Play