Good and bad metrics for product improvements
No product manager, experience designer or software engineer would have a job if all they did is deliver products on time and on budget. Because, of course, anyone’s job is only as valuable as the value they create and any company only survives if it generates enough profit (unless you work for an NGO, the government or Uber). The problem, however, is that value is not only hard to define for each role, department or product, but it’s equally hard to measure.
In most cases value is ill-defined as money earned. In some cases it’s more clearly defined as income, profit or revenue. But it’s nearly impossible to measure anyone’s performance purely on these terms (unless you work in sales), because they are an outcome of many activities. Some of these activities I’ve found extremely useful for measuring success and failure.
Don’t just value what’s measurable (like click-through rates), but measure what’s valuable.
Let’s start:
Revenue
Revenue — all the money a company receives — is the most elusive level to measure any individual activity. Whilst profit is dependent on income, revenue is not dependent on income and profit. Process changes (eg improved after sales care), design improvements (eg enhanced product perception), and better strategic alignment (eg improved company focus on core growth areas) are some activities that can lead to measurable revenue growth.
Increased income
Income — the earnings of a company during a defined period — is a bad metric. Good metrics are the activities leading to increased income, such as increased conversion rates, wider customer reach and better pricing to name but a few.
A great example is Grom, a delicious (not too sweet) ice cream only specific Italian grocery stores in London or Waitrose sell.
Waitrose sells these products and at low volume and at a loss. In conventional business logic, this loss making and low sale item would be removed from shelves quickly.
Yet, Waitrose knows their customers, like myself. I only go to Waitrose for this ice cream, which they sell for half the price than the smaller Italian stores. But once I’m already there, I also add a bottle of wine, some cheese and bread. All profitable items, Waitrose otherwise wouldn’t have sold to me. Customer research and data analysis, can uncover these dependencies and hint to pricing strategies that go beyond direct sales to increase income. Here’s a brief article on pricing strategy and ‘loss leaders’.
Reduced costs
Profit — the surplus a company makes after operating costs and expenses — is an easier metric than income, because more than just increased sales can lead to higher income: Reduced costs, increased efficiencies, lower customer churn etc.
Customer churn, or loosing customers, is expensive. The cost of retaining an existing customer is 5 times lower than acquiring a new one.
At Vodafone, where I worked in the past, customers have regularly complained about unexpectedly high phone bills. This is a common problem in the telco industry and known as ‘bill shock’. Customers are more likely to leave a provider when they have a negative experience, like a bill shock. Vodafone therefore developed a feature into their main app that discovers when customers are likely occurring an unexpectedly high bill and proactively sends notification messages to inform customers about the incurring costs. These activities have decreased churn rates… by a lot.
Improved efficiencies
Dover Street Market is a good example, where design improvements to its store layout lead to increased brand perception and sales. The high-fashion store successfully sells extremely expensive clothes. Part of this success can be attributed to how items are positioned on the floors. Most clothes hang on rails in seemingly random positions. Customers typically go through the items from the edge towards the middle.
On the edge of each rail, the most expensive and eccentric items can be found. As a customer approaches the middle of the rail, items become cheaper and more ‘wearable’. First, a customer would see a Renli Su dress for £2,399 on the edge and a few swipes later a more affordable and wearable dress for ‘only’ £778. In comparison, the cheaper dress appears like a bargain.
Dover Street Market has no intention to sell the items on the edges of each rail. Its intention is to ‘frame’ the perception of the price of the items in the middle, the actual items it wants to sell. This well known method in retail is referred to as anchoring. There can be many examples found of how this works in digital retail too. Amazon regularly displays items alongside more expensive ones to frame an item’s value.
For any activity product managers, designers or engineers are engaged with, there should always be a measurable outcome. It’s also important that there is a multitude of metrics for success. No single metric can paint a good picture. For example it’s quite pointless to measure economic health of a country by merely looking at GPD as it would be to measure human health by calories eaten.
The above is by no means a finite list of metrics. Other more popular metrics are daily active users, customer engagement, time on site, Customer Lifetime Value (CLV), Average Revenue per User (ARPU), bounce rate, Net Promoter Score (NPS) etc.
Metrics are invaluable! They tell us what has happened. Yet, they rarely tell us why.
Qualitative research should always follow quantitative analysis.
Angel investor Dave McClure has greatly influenced this topic ever since his awesome talk from 2007.
Follow me on Twitter, check out my blog and have a fantastic day!
