Video
April 12, 2018
Convoy, Chief Product Officer
Member
The starting point for product development is figuring out what you’re trying to accomplish. In other words, we all want to make an impact with our products. From there, you can create your product vision and define the metrics that you will use to measure progress.
In addition, you need to have more than quality ideas to drive your vision forward. Product culture is equally as important as your ideas. This defines the rules for how you operate and get things done. As a result, the success of your product team is the combination of amazing ideas and a great working environment.
One way to measure product impact is to evaluate the results of experiments. This method comes from academic literature and data science. However, depending on the structure of your business, there are a number of ways that you can conduct these experiments.
First, I’m going to start with a simple gamble. If you roll dice and get 1/2/3, you win $1 million from me. If you roll 4/5/6, then you pay me $1 million. These are pretty fair odds, but most people would say no because of “loss aversion.” Said differently, people value what they have more than the potential upside of an outcome.
Here’s another gamble. If you get 100 rolls and hit 1/2/3/4, you win $10,000 from me. If you roll 5/6, then you would pay me $10,000. With these odds, you’re guaranteed to win money 99.98 percent of the time.
When you apply these two experiments to product development, you’re able to get a sense of the impact that your product will have. If you follow the $1 million example – the frequency of testing is low and the cost of being wrong is high. Conversely, the $10,000 example gives you a higher frequency of testing and a lower cost of being wrong.
In summary, your impact will be highest when you have low confidence and more false positives. Simply put, you are able to experiment with more scenarios to ensure that your product is ready for long-term success.
Ziad Ismail is the Chief Product Officer of Convoy, responsible for data science, design, growth and product teams. Before joining Convoy, Ziad was the Chief Product Officer of Marchex, Chief Executive Officer of CitiKey, and served in other product leadership roles at Microsoft.
Prior to that, he spent two winters at the Arctic Circle in the Swedish military. Ziad earned an MSc in Computer Science from the Royal Institute of Technology in Sweden and an MBA from Harvard Business School.
Member
Anne Retterer is the Founder and CEO of Mindspand – an online community that enables organizations to list course offerings and provide access to services for local customers. Prior to starting Mindspand, Anne managed the product portfolio at Expedia and established an investment fund for tech companies in Chile with Hambrecht & Quest (now JPMorgan Chase & Co.). Anne holds an MBA from UC Davis and currently lives in Seattle.
HostIn this episode hosted by Sonjoy Ganguly, Emrecan Dogan will be speaking on why shipping more features doesn’t mean creating more value.
Explore the journeys of 2026’s top product leaders in our latest eBook. The Product Leader Awards spotlight leaders who are building learning organizations, making AI...
In this episode of the CPO Rising Series hosted by Jay Patel, Jonathan Shottan speaks on building the world’s largest kinetic dataset.