This post has been so popular we have created a downloadable template for you to use to track your product success and failure criteria. You can download the template here.
“How do you know if your brilliant new feature has worked out? How do you know if it has failed and you need to kill it?” I asked.
“The best teams set exit criteria” Nick Coster replied. Nick, the co-founder of Brainmates and arguably Australia’s top product manager trainer went on to explain that the best teams he has worked with set their criteria upfront as to what success or failure for their new product or new product feature looks like.
Let’s unpack this because there is so much gold to be had, let’s look at why doing this is essential and then how you can do it.
Why set success criteria for product features upfront?
Setting your success criteria at the outset has a range of benefits.
Focuses everyone on what matters. Having a clearly defined criteria for what will lead to success will focus everyone on the most important activities that are needed in order to make the feature a success. It will help cut out all those activities that seem like they “should” be done but really aren’t necessary or aren’t necessary right now in order to gauge the success of your new feature.
Avoids emotion perpetuating a product/feature. One of the greatest drivers of poor decisions is emotion (just ask leading investors like Buffett or Howard Marks). You fall in love with the feature, you don’t want the feature to fail because you fear job loss, you desperately want it to succeed to prove something to the world, you’re greedy because some financial incentive is perceived to be there. Setting clear criteria as early as possible removes the emotion.
Ensures what you’re doing is achievable and useful and valuable. Taking the time to determine your success or failure criteria at the outset usually, by its very nature, causes you to consider how the feature will be valuable to the business, be useful to customers and be achievable.
Kill bad ideas ahead of time. Thinking about the criteria might lead you to kill the idea before you waste time and money on it. You may realise that, in order to be successful, the activities you need or the success criteria is unachievable or provides no value to the business.
Alignment on expectations. You’re most likely operating with a team, whether you’re in a company with thousands of people working on product or you’re in a startup. Setting expectations at the outset for new products has huge benefits that will help avoid arguments and difficulties down the track when things have or haven’t gone to plan.
Lastly, you wouldn’t make an investment in stock or property without some kind of expectation of the return you want. So don’t invest in your product without an idea of the return.
How to set success criteria
Now that you understand the benefits of setting success criteria for new features and new products, you’ll be looking at how to apply this. Here are some useful guidelines to use in order to get the most out of setting success criteria:
- Choose one metric only. This makes it easier for teams to rally around, easier to understand and easier to focus on the activities that are important (and not so important). The one caveat to choosing one metric is that you may need a counter-metric or two.
- Choose a metric as closely aligned as possible to both customer success and business success. It’s OK to spend more time here than you think is needed to really ensure the metric you are choosing is as close to measuring whether a customer is getting value and the business is/will receive value. If you aren’t addressing both then be careful, it may be that you’re measuring a vanity metric or setting yourself up for a product feature that isn’t going to move the needle.
- Make the metric specific to the feature/product you are creating. Personally, most of my mistakes have been from using general metrics where a more specific metric, specific to my exact product would have better allowed me to set success criteria. For instance, active users is often touted as a way to set success criteria but if you haven’t previously had clarity on what an active user means to your product then active users is just a vanity number.
- Change the metric if you need. If you discover that the metric you have chosen isn’t the best one you could be using to judge success or failure then change it. Just don’t change it to make it easier to pass so you feel good.
- Make it a measure of people and their behaviour. Usually products are about changing behaviour in some way or taking action in some way for people. So the closer your metric is to people and their behaviour the more likely it is a better measure of success.
- Write out what you will do for different results. For example, if 20 messages are sent with this new feature by each customer then it is a success. If only 5 messages are sent each week then it is a failure.
- Use a hypothesis if you don’t have the data. Using a hypothesis driven approach will let you keep moving and keep measurable. There are also more benefits to a hypothesis-driven approach.
Now you know how and why to set success criteria for your new feature or product, so when you and your team are celebrating the success, you’ll know it’s for the right reasons.