We’ve developed an API-as-Product Assessment Framework that we’re using to assess public APIs. We’re sharing this framework because you will likely find it to be a useful tool for understanding your own API as a product or set of products.
The API-as-Product Assessment Framework has evolved through a combination of years of experience working with others’ APIs and developing APIs ourselves as well as a recent high-level market scan, and more in-depth analysis of a handful of APIs (e.g. Xero and the bank NAB). It looks at an API from multiple perspectives, like through the lens of product management, and not just the technical implementation.
This framework is purposefully trying to move beyond slideware from vendors to a practical understanding of what makes a great API, grounded in real-world data. It’s also trying to bridge the gap between technical best practice and product best practice to create a holistic view of your API.
The framework has been designed so that you can take it and assess how great your organisation’s API is (or not) in a more thoughtful and considered manner.
The API-as-Product Assessment Framework is broken down into the following key areas:
- Onboarding – the ease and speed with which you can get started using the API
- Documentation – the information available to help you understand the API
- API Design – the technical design of the API itself
- API as Product – how the API fits with product disciplines
- Community – how users are engaged together around the API and documentation
- Other – anything that doesn’t fit into the above, or its own key area, but is deemed important.
Key Area 1: Onboarding
Onboarding is usually the first area that any user of your API will touch. They will need to get onboarded to access documentation, run test queries and begin developing against your services.
At best, a great onboarding process can be a source of growth for an API and a business. In many cases, it isn’t this extreme, but a smooth onboarding will assist with productivity and reduce support costs. At worst, a poor onboarding experience will mean developers choose other providers or create friction in your relationship with them.
This is the part of onboarding where you get an account and/or access.
Getting Started Help
The guidance and assistance provided to make first time use straightforward.
Getting to Your First Query
This is a measure of how quickly a developer can run their first query.
Time from registration has been included because for some APIs it is the registration process that requires background checks and other activities.
Getting to Production
This is a measure of how quickly a developer can get a query to the API running against the APIs production data/systems.
Key Area 2: Documentation
After someone has onboarded, they are going to need to start working with your API, which is where the documentation comes in. Strong documentation will mean little to no human interaction is required. Poor documentation will result in high support costs and developer frustration.
Key Area 3: API Design
This key area looks at the technical implementation and design of the API itself.
Key Area 4: As Product
Great APIs are developed as products.
Key Area 5: Community
Great APIs have engaged communities. Engaged communities assist each other with solutions and help drive the API forward in the best way possible. Community members will often become evangelists that help promote the API to others.
Key Area 6: Other
This area is reserved for capturing any other information that is relevant while assessing that hasn’t been captured elsewhere.
Over time, as we develop this assessment framework, we expect to merge common-themes from this area into the other key areas or new key areas.
In doing the assessments we have done so far, this section has been a useful placeholder.
Notes on Using the Framework
Why are some evaluations able to be left as blank? When doing the assessment of some criteria (like time to register) the evaluator can leave it as blank. Leaving as blank might be necessary if the criteria is too hard to evaluate in a commercially reasonable timeframe.
Download the Template
CEO & Founder
Scott is the CEO and founder of Terem, Australia’s leading tech product development firm. Terem has featured on the Financial Review’s Fast 100 for two years running. Scott has been involved in the launch and growth of 61+ products.