Here is a framework for knowing what types of tools you need in your product analytics stack.
It’s useful in a variety of situations: improving your existing product analytics stack, setting up a new stack for a new product or reviewing what you currently have. It can also help get everyone on the same page with what product analytics is.
The framework defines the different types of product analytics tools you need in your stack, discusses them briefly and then provides some examples.
Discussing the merits of each tool is out of scope for this article because it is a secondary consideration to determining the types of tools you need in your stack. The types of tools needs to come first, then you can evaluate better within each category.
Essentials
At a bare minimum you need to track individual users, their actions and your actions towards them. This is essential.
The industry calls this “event-driven product analytics” where individual users are tracked and the events (their actions, sort of) associated with them are recorded.
Broadly this is usually done two ways:
- Using a general purpose event-driven product analytics tool
- Using your product’s database
It isn’t mutually exclusive, you could do both.
Using a general purpose event-driven product analytics tool
An event-driven analytics tool has some out-of-the-box power around tracking individual users, building funnels, cohort analysis, graphs, custom reporting and more. Usually for a reasonable price.
Examples are: Amplitude, Mixpanel, Heap and even Intercom. Google Analytics sort of fits here with the caveat that it doesn’t track individual users.
Using your product’s database
This has been the traditional go-to. Your product, especially if it is digital, is most likely recording the actions of users as a course of functioning correctly. So, you can build reports using SQL yourself or have engineers build reporting screens to understand users and user behaviour.
This is an essential tool at your disposal because of how easy (and cheap) it is to run a SQL query. However, once you get into building your own reports you really need to think twice because the product analytics tools out there will give you much greater bang for buck and your engineers are better focused on solving the needs of your customers.
We’ll come back to complex and adhoc reporting later in this article. For now, just make a mental note that engineers building reports is not the default position for the essential element of your product analytics stack, use the general purpose, event-driven product analytics tools outlined above for that.
Almost Essential: Enablers (a.k.a Plumbing)
Getting your product analytics plumbing right can give you a significant productivity increase. The type of tool in this category is a tool that you plugin and it then enables you to plugin new tools with minimal effort and hopefully minimal technical assistance.
The best examples of an enabler is Segment.
Pro tip: I don’t ever put together an analytics stack without segment. I’ve heard many people grumble about extra cost, but once you see the gains you can’t turn back.
Journey Research (a.k.a Debugging)
At various points in the product lifecycle you will run into challenges where your user journey is failing or not performing as well as you think it could. In these instances, getting a deeper understanding is needed, often it requires more than your general purpose product analytics stack can provide. Additionally, useful interviews and feedback might be hard to come by.
Recently, for example, we could not understand why users were not able to get through to completing the first Job-to-be-Done no matter how long we stared at our funnels in our analytics tool. Given the nature of the product, feedback by email or phone from those that dropped out was hard to get.
In situations like this you need to look to tools that will provide you with a better understanding of your user’s journey as it is unfolding.
Pro tip: Shift your mind into using the tools in this category on an as-needs-basis and then unsubscribing/discarding them. Suddenly that $600/month tool is a no-brainer to better understand and hopefully fix your problem.
The types of tools that are useful here are screen recording tools, heat map tools, onboarding insights and even A/B analytics/test tools.
Examples of tools in this category are: Appcues, Pointzi, Optimizely.
Platform and Domain Specific Tools
This is about the types of product analytics tools that are purpose built for your technology or domain. This could be analytics tools specific to mobile, ecommerce, IoT or chat.
Examples of this are: Fullstory, RetentionGrid, Metrilo, Kumolos, Localytics
Performance Analytics
You may also need to understand the performance of your product and the underlying technology. Amazon’s research shows how important performance can be; a 1 second slow down in response time can result in a $1.6 billion loss in revenue for Amazon. There is some overlap here with the tools your engineers and DevOps team might use. If you think performance is a significant driver of user experience and customer satisfaction then you’ll want to include tools of this type in your stack.
Examples of product performance analytics tools are: New Relic, Sentry
This category also includes error reporting because errors are, in many ways, an almost binary signal of non-performance.
Complex and Ad-hoc Reporting
Once you’re maturing as a team or if you have some complexity that the out-of-the-box tools can’t handle then you will need to consider tools that can assist you with adhoc reporting and doing complex reports across multiple data sources.
Tools in this category are easy to connect to the tools above, link them together and help you with reporting.
Examples of tools in this category are: Tableau, Mode, (the humble) Excel. Of course you can also get significantly more advanced with Redshift, BigQuery, Apache Hadoop, Periscope and more. But that’s a topic for a different day.