Product Development Health Check Playbook

A Product Development Health Check is like a fitness test for your product development practice. Just like the best Olympic athletes benefit from taking stock of their health, so can your organisation. Like any health check, some of the benefits come from doing these regularly and assessing how/if things change.

An important consideration is to not limit the health check to only one area or aspect of your product development practice. Taking a holistic view of product development across product, delivery, and technology can help a health check identify otherwise hidden issues. To help get multiple perspectives, we typically do a Product Development Health Check with a product manager, a delivery manager and a lead engineer. Yet, you may want to involve anyone that understands how you work and whose job involves multiple parts of the product development practice.

This playbook focuses on the essential elements for running a Product Development Health Check, including the key concepts, five key elements to check, what healthy product development looks like, and next steps. It is essential you consider and assess against great frameworks, use more than one as each may highlight different issues or areas for growth. The ones mentioned here are, from our experience, the most effective.

Why is a Product Development Health Check Necessary?

Whether you are a large enterprise with vast resources or a startup with a limited budget, it’s common to find that you have product development problems that are either not identified or not properly addressed.

Some of the problems that signal a health check might be needed are:

Issues with prioritisation
Lack of alignment in the product team
Lack of customer intimacy (B2B) or market knowledge (B2C) and no mechanism or process for user feedback
Siloed communications and working between product, engineering, sales, support and service teams
Executives frustrated by the lack of results and size of their product development investment
Poor morale in the product development teams
Confusing titles and roles (e.g. project managers or SMEs guiding roadmaps)

A Product Development Health Check can help identify issues in your product development practice, find the underlying causes and give you ideas about how to address them.

5 Key Elements to Check

A single product issue may reflect multiple underlying causes. A team failing to deliver successful outcomes in time could have flaws in the delivery process, or lack of product vision, or a need to improve the efficiency of the technology stack. To ensure you take a holistic approach, review these five key elements of your product development practice:

Mindsets
Organisation
Practices
Tools and Technologies
Financials

1. Mindsets

Are the right mindsets in place for successful product development? Are the right mindsets promoted by the frameworks used and organisational structure?

Mindsets are fundamental to how we operate at work, and so they are first in our list of key elements. Without a shared understanding of what you are doing and without mechanisms reinforcing the right mindsets, you are likely to end up with unhelpful mindsets getting in the way of product development excellence.

While unhelpful mindsets can be obvious, this is the most subtle problem to diagnose the root cause of. People can have assumptions about what is right, what success means, and what is appropriate – misaligned understandings and contentions between key stakeholders can set project outcomes up for differing expectations and undermine how success can be measured.

One technique that often helps is to get shared clarity about assumptions, preconceptions and attitudes by using argument mapping, which allows you to make thinking and assumptions tangible and objectively assessable.

David Marquet’s book ‘Leadership is Language‘ is a great source of ideas for helping promote the right mindsets.

2. Organisation

Has the organisation and its management structure been set up to support effective delivery? Is the team the right size with the right stakeholders engaged? Are all the right people with the right skills involved? Is the team composition fit for efficient delivery? Is there clear responsibility and accountability?

Conway’s Law is a great example of how organisational structure will affect the products we create:

“Any organisation that designs a system (defined broadly) will produce a design whose structure is a copy of the organisation’s communication structure.”

Steven Sinofsky coined the term as “don’t ship the org chart”, but then admitted you can’t help shipping it in your products, so you better get it right.

Beyond how our structure affects our product design, there is also whether it helps or works against our product development goals. Teams that have mixed reporting lines can find themselves pulled apart by competing instructions and priorities.

Sometimes you know your structure has problems, but not how bad the effect of that is on your product development outcomes. A Product Development Health Check can not just identify there is a problem, but can also look at what effect that has.

The Pragmatic Framework from the Pragmatic Institute is a great tool to use to assess how product management responsibilities are allocated across your organisation. Conflicts and items that are not being managed can be readily identified by mapping roles to the framework.

The Product Excellence Maturity Model from ProductBoard is another great resource for checking whether your organisation is truly excelling at product development, especially assessing how the organisation understands and relates to product strategy and objectives.

Marty Cagan’s book Empowered is a great source for ideas on how to organise product development to get better results through building trustworthy product teams and then trusting them.

3. Practices

Do the practices that are in use support product development and adhere to the frameworks that should ideally be used? Are they fit for the outcomes that your organisation is striving for? Are people adhering to the practices set out and/or monitoring them?

These can be the most obvious source of problems, and there are many great resources on how to do product development well, especially when scaling beyond a single team. They are also sometimes the least effective place to make changes because they will tend to get trumped by poor mindsets and your organisation’s structural problems.

The important part is to identify what are the best practices for your business goals and product development scenarios.

We believe that Agile software development practices are some of the best at helping reduce waste and support great product development. These should be the basis of your product development technical practice. Like any framework, it helps to ensure you understand the principles behind it.

Do your practices support iterative development where the product team solves business problems, or is the product team left simply implementing a plan incrementally?

The Product Excellence Maturity Model mentioned above is also useful for evaluating how prioritisation is done and whether it properly considers and addresses strategic priorities as well as product quality issues.

In the technical area, we need to make sure we follow best practices for great product development. Things like code reviews, source code control, automated testing and shared code style guides are all important basic practices.

Other practices like continuous integration/continuous deployment (CI/CD) are likely to be appropriate or not based on your product development context. However, you should seek out the best practices for these technical areas as well.

4. Tools and Technologies

Are the tools in use ones that support the frameworks selected? How well do product technology choices support the product strategy and current roadmap? Are the tools or technologies being used enabling the team or creating friction?

If the tools used by the product team get in the way of their processes, create manual time-sinks or hide important details, then you might need to change them out for ones that are a better fit. Sometimes, too many tools are being used, and the team is lost doing data entry or choosing which tool to place information in over actually making product decisions.

If the wrong technology platform has been selected, then future product development can be hampered or entirely derailed by problems with that platform. Even a good platform, or a well-architected solution, may not be the right way for that product. It may bring extra costs or create maintenance headaches that are not justified by what it brings to the product.

5. Financials

Are the right metrics and rewards in place to support good product outcomes? Are budget processes supportive of the needs of product development?

An example of a potential problem is funding product development by treating each release as its own project. Projects usually need to prove their return on investment (ROI), and when a release is treated like this, it can prevent tech debt from being addressed in any given product release. Similar limitations arise with a product budget that pays for a certain number of releases; again, there is a limited amount of forward planning done. The best model is an operating budget that is simply renewed over time and is regularly reviewed based on product strategy and business needs.

Another problem can arise from bonus schemes and KPI programs that financially reward people for achieving goals that are not aligned to their product work. Sometimes this leads to negative consequences as people can act in their own best interest, despite it not being what is best for the product or organisation as a whole.

What Does Healthy Product Development Look Like?

There are two aspects to getting product development right. The first is building the right things, the domain of product management. The second is building the things right, the domain of delivery management.

From a product management perspective, when product management is being done in a healthy way, we find that the following are true:

Strategic alignment is aimed for and achieved, both to business strategy and architecture strategy
Responsibilities and accountability for results is well understood
Decisions are made transparently
Everyone involved in the end-to-end product sales/development/delivery/support process is included
Clear communications (roadmap, go-to-market, risks) from product management
Customer (users and choosers), staff and partner feedback is sought, valued and digested
Metrics/KPIs/OKRs are setup, tracked and regularly re-evaluated
Prioritisation is focused on real outcomes for customers and the business
Minimal waste is allowed in the product development process (queues, inventory, etc.)

From a delivery management perspective, when delivery management is being done in a healthy way, we find that the following are true:

The organisation and its processes are set up to support effective delivery
The team understands, agrees to and practices the delivery process, and tweaks/adaptations are regularly made
Clear business goals are defined, communicated, understood, tracked, and are aligned with business value
Success metrics are defined, communicated, and agreed upon
The budget is well planned, approved, communicated, and reported regularly
Timeline and scope are known, communicated, and tracked
Ownership, responsibilities, and accountabilities are well defined, communicated, and understood
Business stakeholders and sponsors are actively engaged
Effective communications among all the delivery parties
Risk and issues are documented, assigned, tracked, and reported regularly
Changes and decisions are made transparently, documented, and tracked
The team has the right skill set, the right size, the right composition and is supported to work effectively and efficiently
The team is highly collaborative, empowered, and encouraged to develop/learn
Proper tools are provided to set up the team for success (including facilitating communications, enabling remote work, promoting transparency, enabling knowledge sharing, etc.)

How to do a Health Check

There are three main activities in a Product Development Health Check:

Plan
Research and identify issues
Root cause analysis

1. Plan

The first step is to plan and decide the timing of the health check. Broad cooperation is required for a health check to go smoothly. This step is also an opportunity to set up the team, the goals and their tools. Meet and greet the key stakeholders and schedule interviews and workshops for the next step.

2. Research and identify issues

Gather and digest relevant documents, then interview people involved in the processes under review. This step is trying to observe and empathise with the current status without judgement. Take the problem as it is and gather as much information as possible.

The research and observations can be done iteratively; sometimes you may need to interview people again when actual practices in use don’t match what they claim the process is. The audit step is optional and varies from case to case. However, the goal of this step is always to discover potential issues within the key elements: mindset, organisation, practices, and technology & tools.

With a possible list of issues identified, we can decide what to focus on for the next step.

3. Root cause analysis

Decide which issues need further analysis (some may be obvious or immaterial). With those issues, run workshops to review the research outcomes and validate findings with key stakeholders.

During this step, there might be more issues found that may be worth another round of research and observation until there is a decision on which areas to focus on for root cause analysis and recommendations. Evaluate the impact of the problems so that the key issues will be analysed and addressed with priority.

The root cause analysis may be done per problem or per a group of problems, as several inefficient practices may actually result from the same problem in mindset or organisation. For example, poor execution of process and practice may come from the team’s lack of understanding and alignment of the business goal.

Next Steps

Once you have identified the root causes of the most important issues, you can then bring your product teams together to develop hypotheses and recommendations for addressing each issue.

Our Practice Uplift playbook (coming soon) will give you some ideas for implementing these sorts of experiments in order to find the best approach to resolve product development issues.

Last but not least, a Product Development Health Check is not a once-off process and should be done periodically. You should schedule regular health checks to review progress and improve continuously.