How To Measure Productivity Gain From AI

How can product and engineering leaders actually prove the ROI of AI adoption when everyone is being asked to justify the investment but few teams have a clear measurement playbook? In this podcast hosted by Hoda Mehr, Co-founder and CEO of Up My Mojo and a Board Member at Products That Count, Pensero AI Co-Founder Bernardo Hernandez will be speaking on how to measure productivity gain from AI in a way that is rigorous, practical, and useful for annual planning cycles. As tools like Cursor and Claude Code promise 3 to 10× productivity gains in engineering, leadership teams are moving quickly to allocate budgets and set expectations, while product leaders are being asked to benchmark progress and quantify impact without clear precedents. This conversation explores how experienced operators are approaching that challenge and what it takes to turn AI productivity claims into measurable results that teams can confidently use for planning and decision-making.

Join us for new conversations with leading product executives every week. Roll through the highlights of this week’s event below, then head on over to our Events page to see which product leaders will be joining us next week.

Show Notes:

1. Measuring productivity in engineering has historically been difficult because the data needed to evaluate performance is fragmented across tools like code repositories, tickets, documents, and conversations.
2. AI enables organizations to aggregate and structure engineering activity data into a coherent productivity model for the first time at scale.
3. Productivity gains from AI adoption vary widely across companies depending on how AI is used, ranging from incremental improvements to several-fold increases in delivery speed.
4. One of the biggest productivity unlocks is not individual output increases but reducing gaps between high- and low-performing engineers.
5. Transparency into engineering workflows allows organizations to identify hidden bottlenecks that traditional metrics like velocity or story points fail to reveal.
6. In one case example, identifying definition problems, quality issues, and uneven performance across teams enabled a company to ship four times faster within three months.
7. Early collaboration between product and engineering teams is one of the strongest predictors of higher engineering productivity.
8. When engineers work in isolation before aligning with product teams, features often require multiple iterations and rework cycles.
9. Many productivity problems in engineering organizations originate from poorly defined features rather than weak execution.
10. Quality—not just speed—is a critical dimension of productivity that traditional engineering metrics rarely capture effectively.
11. AI-generated code can increase output volume but may introduce quality issues if used without structured workflows and oversight.
12. Measuring how engineers actually use AI tools is essential for understanding whether AI adoption improves or harms delivery outcomes.
13. Engineering organizations often contain “pockets of unproductive resources,” especially once teams exceed roughly 50 engineers.
14. Some companies discover entire teams contributing minimal output after acquisitions or organizational changes once transparency tools are introduced.
15. Benchmarking engineering performance across companies helps leaders understand whether productivity issues are individual, team-level, or systemic.
16. High-performing engineers typically demonstrate stronger collaboration patterns, especially with product teams during early definition stages.
17. Identifying and scaling best practices from top performers is one of the fastest ways to improve overall team productivity.
18. Real-time performance insight enables managers to address issues early instead of waiting for quarterly or annual review cycles.
19. Product teams benefit from engineering intelligence platforms because they can validate estimation accuracy and understand roadmap deviation sources.
20. The future of engineering management is likely to resemble modern marketing analytics: decisions increasingly grounded in measurable ROI, benchmarking, and continuous performance visibility rather than assumptions.