Don’t Confuse Product Validation With Product Research

In product, I see validation used all too often to describe what is actually research. This misuse misleads thinking about whether a product or feature is likely to succeed. I’m going to unpack the difference between product validation versus research and share what each is best for in this article.

The misuse has come about through the rise of the Lean Startup movement. Everyone who read the book, or absorbed the concepts, has become obsessed with the need to validate. The focus on product validation is great, it has pushed people to focus on understanding their customers better and ship product sooner. It also helped people question their assumptions about what will or won’t work.

In a bid to be seen, to be following best practice, or due to a misunderstanding of the concept, the words “validate” and “validation” now get overused – and often, used in places they don’t belong. 

What is Product Validation

The only form of validation that matters is observing your target user or customer interacting or failing to interact with your product or service in the way you expected.

Everything else is research to help you form a better hypothesis about what will be successfully validated.

Validation is observing your target user or customer interacting or failing to interact with your product or service in the way you expected.

Understanding this definition is crucial, so let’s unpack it further.

Observing interaction is about observing behaviour. You need to be able to track and record data on what happened (here are some examples of behavioural data). It might be you watch a user over their shoulder, or you might have analytics tracking what your customer clicks in your software. You might watch how someone browses a physical space in a retail setting. It just needs to be behaviour, not the intent. Not the opinion. Actual behaviour.

It also needs to be the actual behaviour of your target user or customer. If you’re targeting single dog owners, you can only validate or invalidate if you observe their behaviour. Observing a married lady without a dog doesn’t validate or invalidate your hypothesis. Similarly, in B2B, if you’re targeting an HR department with at least 50 people, then positive usage from a single person HR department in a smaller organisation doesn’t count. I’ve personally fallen afoul of this one a few more times than I’d like to admit, taking feedback or reinforcing my faulty assumptions based on usage from non-target customers. 

The last key element of unpacking product validation is understanding that, in order to validate your hypothesis, the interaction needs to be with your product or service. You need to ship something. If you haven’t put something in front of people to interact with as intended, you cannot observe behaviour. So you can’t validate or invalidate.

You see, there is always nuance – in the product, the context, the user – that is missed, unless it’s a genuine interaction: the little things and the big things.

I’ve failed here as well. I’ve taken multiple workshops with multiple target customers using detailed wireframes and clickable prototypes as validation only to ship a major feature that flopped. The workshops were a raging success, with lots of head nodding. Lot’s of “this would be great”, “this is amazing”, “we need this”. But, when push came to shove other problems that neither me, my team or our users had considered, it kept ruining our amazing solution. The data wasn’t right. The user didn’t have the time they thought they would to use the feature due to other problems.

Putting wireframes in front of people, giving them a clickable prototype, running a landing page experiment, talking them through how it works or giving them a sales pitch are all great research activities that might save you time or bring you success faster, but they aren’t validation.

Now, a close proxy might be ok. A close proxy might get you 80% there for 20% of the effort and let you iteratively improve. But you just need to recognise that a close proxy isn’t interaction with the actual product or service, so it can only inform your thinking. Manage your investment closely and be aware of the assumptions you’re making if you proceed.

What is Product Research

In the context of product validation, the simple definition of research is everything that isn’t validation itself, but leads you to either validating more successfully faster or deciding not to pursue something (and thus, saving time and cost).

This is a much bigger topic than that, but this article’s purpose was to clarify the confusion between research and validation.

Not Product Validation

To finish up with and make life easy for you here is a quick list of what is not product validation:

1. Talking to a partner about their views on your product
2. Talking to another business about what worked for them or didn’t 
3. Talking to a customer about what they think of your product
4. Surveys
5. (Sometimes) making a sale (possibly controversial but it’s happened to me!)
6. Doing spreadsheets or powerpoints
7. Your boss saying it’s a good idea (heh!)
8. Focus groups
9. Your colleagues saying it’s a good idea
10. Customer interviews

---