Success Metrics are a thing that product manager usually struggled on creating. In my earlier days as a product manager, I just focused on pleasing the stakeholder by agreeing on developing any feature based on their request. At that time I didn’t even consider success metrics as a part of my requirement component.

Until performance review came and I was asked what is the impact of the project that I conducted, and I panically started to ask the stakeholder what is my project impact for them.

So until this day I always consider the success metrics as a part of my requirement / things to be stated in the product requirement document.

The thing is, I struggled to calculate the expected number of impact that my project brings. It is weird / extremely wrong to just create the number of thin air without any justification. For example:

“this project will increase conversion rate by 1%”

how it is based?

So this is the typical flow of pre-kickoff project that I have handled in the past.

As you can see that success metrics weren’t in any of the flow process. Usually stakeholders are the one who reported to me about any problem that they faced. I didn’t even have success metrics at that time, and just jumped into the requirement gathering. Back then, the only thing that matters to me are only the speed of project completion.

While this is my latest flow for pre-kick off project:

as you can see that there is always a north start metrics for our team to believe in. Any kind of request / project idea has to go through success metrics validation & valuation process first.

So I want to share to you how I hyphotize, build, and calculate my success metrics:

Formulating Problem Statement

the problem statement usually comes when something that are unfolds differently than our expectation.

For example:

• Ecommerce business in a similar industry standard stated that average conversion rate from product detail page to add-to-card are usually 10%
• But our product only have about 5% conversion rate from product detail page to add-to-cart

So the after research, data said that most of user didn’t click add-to-cart because the product image is not attractive enough.

so the problem statement we want to focus on is: How can we make product image more attractive so that it could increase conversion rate of product detail page to add-to-cart

of, after getting the right problem statement, the next thing we should consider is…

Calculating Impact

In this part, we want to estimate on how much of an improvement that we could made by improving the image quality. From my experience, this impact calculation could be done with several method:

• Market sizing — this is to estimate how many of user could be impacted by this project. For example: from 100% of our customer that clicked on product detail page let’s say 10,000 people, there are about 42% that abandon the cart because the images are not attractive. So it is expected that 4,200 people will be attracted with this change.
• Forecast / Projection — we usually use this mainly to predict the future. For example since our initial user are 4,200 people, and expected user base growth are about 10% per year, the project of making the image attractive could be 4,620 people by the end of next year.
• Behaviour Changes — our internal data could be affected by this. The clearest example is: if more user proceed to add-to-cart, then number of click on add-to-cart button would be increased as well.
• User Interview / testing — we take several sample of our customer and ask them qualitative question, such as “what makes you didn’t proceed to add-to-cart?”.

Let’s say that it turns out that from customer survey said that roughly 42% are not interested in the product because of the image. you still can drive deeper to the problem, for unattractive image there may be several causes:

• Bad Image Quality — maybe because the resolution are to small and makes the image stretched too much
• Image didn’t describe the product accurate enough — poor representation of product
• Image are to small — too small to be seen
• Etc

after the user interview, 50% said that bad image quality made the product unattractive to them.

High Level Problem Statement & Success Metrics

So project that you would want to tackle is

“Increase Image resolution so that the image would be attractive to customer, which would increase the conversion rate from product detail page to add-to-cart”

So back on the problem statement. If we have 42% user said that they didn’t add-to-cart because the image are unattractive, and 50% of them said that it is because the resolution is too small, if we improve the image resolution, then given the initial conversion rate from product detail page to add-to-cart are 5%, then the calculation would be :

Increase in conversion rate : 50% (Bad image resolution) * 42% (user unattracted with the image) * 5% (Product detail page to add-to-cart conversion rate) = 1.05%

So the high level success metrics would be :

Increase conversion rate from product detail page to add-to-cart by 1.05% from 5% to 6.05%

Exploratory Success Metrics

This metrics usually supports high level metrics, usually gives an affirmation and create confidence for our project. For example, for this bad image quality project, it would be:

• # of click in image — this might tell as that customer are interested in the picture presented.
• Customer Survey — an affirmation from a customer that the image are already good, maybe give customer specific question about the image.

Non-Success Metrics

This is from my own perspective and experience that usually your project might not be as successful as you envisioned, so you must build some data that might suggest you on what to build next/possibly deprecate if things go south. For bad image quality, I might suggest:

• # of Bad Review on Image — This could mean that your project are not improving anything on customer sight
• Feedback from customer about image — this would help you on deciding whether your enhancement are good enough or not, and if it’s not good enough, what are the customer expectations
• Gap Between Expected & Reality Success Metrics — let’s say that you expect that there are increase in conversion rate from product detail page to add-to-cart for 1.05%, but in reality only 0.5% increase, so there must be something wrong in your project, either it was miscalculation, or maybe the solution are not optimal enough to fulfil your customer’s expectations

Summary

There are several key point that I want to emphasize:

• Always consider success metrics in every project and make sure that it supports your team’s metrics and in turn, support your company’s OKR.
• Formulate Problem statement from any gap in your current metrics vs industry standard.
• Choose on what to tackle on (in a form of project) and calculate the impact for your metrics via Market sizing, forecast / projection, behaviour changes, and user interview / testing.
• Several Type of Success Metrics: High level success Metrics — support your team metrics, Exploratory Success Metrics — addition success metrics to give affirmation and create confidence, and Non Success Metrics — which can give you a hint on things to do next.