Release Now or Later? How to Assess Risk

Many organisations worry about the risk of releasing more than the risks of not releasing. People focus on what will go wrong if you release a product instead of what opportunities you will miss if you don’t release. If you want to ship great products, you need to have a more balanced and constructive conversation. 

People and organisations that worry more about the risk of releasing have this concern for valid reasons. Usually, they’ve been burnt in the past by a release full of bugs, or they’ve released a solution that customers couldn’t use or didn’t want. There is also the human tendency to always try making a process problem-free and controllable, as well as the tendency to avoid pain rather than move towards gains.

However, these worries shouldn’t be the reason for not releasing it. You can try to reduce problems, but counter-intuitively releasing, in fact, reduces the risk of releases. When I discuss this concept with people their heads nod but then releases still get delayed. 

I’ve found in risk-averse companies that you need to focus on the risks of not releasing. The upside lens (e.g. if we release faster, we’ll get more market share) just doesn’t get the same cut through.

So, in order to make a positive change, we’re going to draw upon negatives. You will see a comparison of the risks of not releasing versus the risks of releasing. Then I’ll refute reasons people often use for not releasing.

Risks of releasing your product

First, let’s cover a few risks of releasing. I’ll use paraphrased quotes to bring it to life:

1. “If we release with an error it could hurt someone”. Software releases that deal with health and safety need some special consideration that is a bit beyond the scope of this post, even though you can take some of the principles here. For many of you, you aren’t in a situation this serious. I’ve listed it here first to give some (much needed) perspective. 
2. “If we release with an error, we could lose a lot of money for [us|our customers]”. An error in a software product that involves money always carries the risk of financial loss, either to the customer or the business. This could also be by losing financial data that leads to financial loss. It’s important to quantify the total possible exposure of an individual release and understand the non-software mitigation mechanisms that are available (e.g. insurance).
3. “If we release, then what if there is some other major error?” This is usually followed up with statements like “we’ll lose customers” and “the big boss will be angry”. This is a likely scenario because most software is released with errors of some form or another. On a human level, people are concerned with avoiding pain; no one likes others being angry at them and no one likes losing customers.
4. “We did a release before and customers wouldn’t use it, we had to rebuild it. It was a disaster.” This too is a likely scenario for software products. Especially for organisations that aren’t releasing frequently or regularly spending time with customers. Even for companies that are on the leading edge of multiple releases per day and customer development practices, it is still a likely probability. It’s hard to know what customers want (if it wasn’t we’d all be Steve Jobs).

Risks of not releasing your product

Now let’s take a look at the risks of not releasing:

1. Your product or feature might not be exactly what customers want. The only way to really know what customers want is to release your product to them and hear some feedback. By not releasing, you increase the risk of releasing something that your customers will reject. The sooner you release, the sooner you can know how well your product fits the customer’s needs. It is highly likely that what you release won’t fit 100% of your client criteria and you’ll need to do subsequent releases to improve it. Not even trying can make some – or all – of your investment worthless.
2. You miss the opportunity. By not being able to release, you may miss the opportunity, often to a currently unknown or unseen competitor. You might miss the opportunity to win or impress a customer. Sometimes it’s that the opportunity is tied to a specific event. If you miss the date, you miss the opportunity. This can often play out slowly and isn’t always obvious to act before it’s too late. 
3. You come under pressure. When money is invested, but product releases aren’t made and results aren’t posted, people get nervous and apply more pressure. This risk becomes higher and higher with each week of delay, and pressure can lead to bad outcomes.
4. You release with major errors. By not releasing, you may be compounding or introducing bugs. Nothing is a better test than the real thing. Often major errors aren’t detected until real usage occurs. It might be that production systems are different, the real-world scale is different or real user experience is different. The longer you leave off releasing the more errors can accumulate, the more embedded within the software they become and higher the costs are to resolve. 

To release or not to release: a decision-making approach

Now that you understand the high-level risks, let’s go through an approach to making a decision on whether to release your product or not.

The steps are:

1. Create a table of risks for release/no release using the risks listed above. Keep it high level. We’ve created a template to help you do this which you can access at the bottom of this page.
2. Quantify and assess the impact and probability of the risks.
3. Compare the risks. 
4. Readjust the release scope/frame.
5. Repeat.

If you’re lucky, then this will give you a clear picture. If you’re in a more common situation, there will be grey areas and hard-to-quantify risks.

Comparing the risks

Now after doing the above, what you’ll find is that the risk of releasing is easier to quantify. You have a good sense of current errors/bugs. You’ll have a sense of the possible negative impact of the features. But it’s often harder to quantify the “not releasing risks”. For example, it is usually harder to quantify the likelihood of a customer going to a competitor. It’s also often harder to quantify the unforeseen production error that comes from deploying to production for the first time and takes the system down.

The interesting point here is that the “not releasing risks” are often significant. If the customer doesn’t want what you release, then your investment could be worth $0. If you miss the date of an event for an opportunity, then your investment is also worth $0. If you come under pressure because you haven’t released your product, then you might also not have a job (this happens more frequently than you think).

Readjust the release scope or frame

What you may need to do is reframe or rescope the release. Some ideas:

1. Reframe the customer – pick a subset of your user/customer base that will be more receptive to your broader product vision. This will allow you to release a subset.
2. Drop scope – remove non-essential requirements from the release. If you know your customer well and their Jobs-to-be-Done, then this is an easier conversation to have.
3. Launch a beta instead of a v1 – set expectations with your customers and users about the nature of the release. Getting something into their hands earlier may help reduce the pressure from them and internal stakeholders. This makes people nervous because they worry about losing a customer, more often than not though this doesn’t happen or it’s a good thing (you learn they didn’t actually care about your product).
4. Release part of what you have – release the parts of what you have that are working and solve a problem or that you want to get real feedback on. Sometimes this involves some manual setup or manual concierge that doesn’t scale, but that’s OK. The gains from learning and releasing might outweigh the short-term cost of “concierging”.

Handling common reasons for not releasing

There are some common reasons people give for not releasing. Here are a few lines of discussion I like to take to probe whether there is legitimately a good reason not to release or whether we’re better off releasing.

The following somewhat restates what I’ve shared above, but in a more actionable form with specific ideas for resolving the situation.

1. We can’t release it now because feature XYZ isn’t ready. Rather than change the product scope, introduce an interim release that targets a subset of customers or users that can get their job done without this feature.
2. We only have one chance for a first impression, we want it to be perfect. This is a valid statement. The flip side is that opportunities usually have a finite time window. Take too long and it disappears. If what you’re doing is genuinely solving a problem, then people will forgive imperfection. Just be sure they can still get their job done or some part of it.
3. We can’t release something that isn’t perfect because it will damage our brand reputation. This one is tough to overcome, so here are a few thoughts:
   1. Many of the companies with great technology brands release early and often. Some do so in controlled ways, such as to subsets of customers. 
   2. The only way to reach perfection is through iteration with customers.
   3. Consider if it’s more damaging to the brand to not show customers they are trying to do new things to help them. Unfortunately, it is easier to tangibly explain the short term risk to the brand of releasing versus the long-term risk of not regularly releasing.
4. We can’t release in case we lose money or our customer’s money. Evaluate whether an “error fund” is worthwhile, a fund that you set up to reimburse customers for errors. It may actually be cheaper and better for your customers to simply wear the cost of errors. This has a side benefit of increasing customer loyalty, compare “they didn’t argue with me, I got my money back straight away” with “I’m so annoyed they lost my money/data and they said they’ll fix it, but won’t do anything for me because they say their contract says they don’t have to”.

Final Thoughts

Here are a few final thoughts to share:

1. A common phrase I hear is “we can’t release it now because XYZ feature isn’t ready yet”. Remember that perfectionistic behavior is not always a good thing.
2. Imagine if the time and energy that went into avoiding risks with the releases and doing finger-pointing style post-mortems went into getting your technology right to release more often and spending time with customers.
3. You can never completely remove risk from a project. The art and science are between choosing which risks you’d prefer to take.
4. When it comes to releasing, people seem to take a pessimistic view asking, “what will go wrong?” instead of asking “what could go right?”. Plan as much as you can for both scenarios.

---