Metrics Hierarchy and Metrics Pyramid: Aligning Product and Business Goals
Metrics Pyramid, Metrics Hierarchy, Unit Economy, Funnels — there are many ways to connect and organize metrics and dashboards.
Choosing the right approach is 80% of success.
As a consultant on advanced analytics, I often hear, “Our business metrics were rising. But as we were cracking a bottle to celebrate, the metrics suddenly dropped”. In most cases, such happens because the metrics hierarchy rule* was not respected. Top-level business goal metrics should never drop, even if the lower-level ones are growing.
Let’s dive into understanding metrics hierarchy and the easiest way to build it — metrics pyramids. Buckle up — it will be a bumpy ride!
Metrics hierarchy
When you should think about the hierarchy of metrics
At 9–14 months, a baby decides to walk on two legs. And so he does. At approximately the same age of a product, its leader understands his data analytics is a mess and wants to align the metrics. And so he should.
But the truth is, you don’t have to wait nine months to start building the hierarchy of metrics. The earlier, the better. Even if you are still searching for a product-market fit but already have enough data about your users — start building the hierarchy.
On average, product leaders start building the hierarchy of metrics after a year of trying to tame the user data. Prevent the difficulties by doing so earlier!
Why you should think about the hierarchy of metrics
Nobody cares if a small e-comm project’s conversion rate drops from 0.12% to 0.11%. The revenue and the audience (MAU, WAU, DAU) will unlikely change much because of this drop. But let’s imagine an e-comm project is Amazon: then the 0.01% fluctuation will mean losing millions of dollars on digital advertising alone. Millions of users are the reason why mass products need to care about even tiny fluctuations in conversion rates and other product metrics.
The reasons for such fluctuations are not always obvious. Nor is it obvious which problems they cause to the business goals. Luckily, the hierarchy of metrics can help define the reasons and foresee the problems. And this is why we need it.
How to build the hierarchy of metrics
People often see a hierarchy simply as a tree structure that helps organize the metrics. Typical reasoning can sound like this:
Look, the lifetime value of a customer (LTV) depends on retention. It seems like LTV and retention rate are connected in a tree structure.
A hierarchy built this way is just a mental exercise. And for a small e-comm project, it is a good starting point. But if you hope to become as big as Amazon, you have to build the hierarchy of metrics according to science. It means you must:
- make metrics sensitive, logically related to product hypotheses, and actionable;
- analyze retrospective dynamics of metrics;
- research user-defined metric distribution;
- investigate metrics connection using the data.
Without a scientific approach and data analysis, the hierarchy of metrics is a mere imagination.
What are the pitfalls
A reliable hierarchy is hard to build and thus takes time. Also, two complicated questions need answers:
- What should be at the top of a hierarchy?
- How to decompose the top metrics into lower-level ones?
Some approaches suggest building the hierarchy of metrics from the most important one, also known as NSM — North Star Metric. But it is not easy to define a proper NSM. And more importantly, NSM is a product metric: most likely, it will measure customer behavior, not business goals.
Let’s go back to the story of a bottle cracked to celebrate the growth of the product metrics while the main business metrics unexpectedly dropped. The moral of this story is this: it is all about the right balance of business goals and meaningful product metrics, including NSM. And respecting the main product metrics hierarchy rule, of course.
The metrics hierarchy rule
As we said, a metrics hierarchy is more than just a tree structure. It must help you avoid bad product decisions that can lower such meaningful values as Market share, Revenue, Margin, and ROMI. And a rule of thumb is:
Top-level metrics should not drop, even if the lower-level ones keep skyrocketing.
It means one should always care about the top-level metrics of a hierarchy:
- when considering any changes to a product or in a marketing strategy, be that change due to mere intuition, a math model, or deep customer interviews;
- when planning and executing the A/B-tests;
- when tracking rolling releases.
Be that a minor change in a web-button color or a global pivot of a product model, — it’s always wise to apply the metrics hierarchy rule.
Metrics Pyramid
A metrics pyramid is more than a hierarchy
The product metrics pyramid is almost like the product hierarchy of metrics. But it is easier to build and more powerful in use.
The metrics pyramid connects business goals with customers’ needs by unit economics (LTV, OPEX, CAC) and loyalty metrics (Retention rate, Churn rate, NPS, and Stickiness).
Also, the product metrics pyramid simplifies the building of a hierarchy!
How to make metrics pyramid work
Classification is a magic word when it comes to an effective metrics system. Let’s see how it works.
Phase 1. Brainstorm
Ask your team:
What are the key metrics on our dashboards, reports, and ad-hoc calculations? What are the key metrics in our minds? Why do the business goal metrics of the same company differ from team to team? (Spoiler: they must be similar, if not the same).
Be ready for a deep discussion with your product team (2–6 hours). I call it “Metrics brainstorming”. The better everyone prepares for it, the more effective it will be.
Phase 2. Reduce the mess
When the host stops the discussion, you should reduce the mess (similar to a mapping process in data engineering).
Group all your metrics according to the pyramid layers: Business, Margin, Loyalty, Product value, Product quality, and Traffic structure. Each layer is already predesigned and based on the mathematically proven relationship between the metrics. Pyramid inherits the business hierarchy logic: business goals are on the top, and customer needs are at the bottom.
You can build a balanced metrics hierarchy in 2–6 hours simply by grouping all of your metrics according to the pyramid layers. Isn’t it cool?
Building a pyramid takes less than creating a hierarchy. No need to think through a structure — the pyramid is already divided into the key layers. No need to decompose the metrics — simply place them into correct layers. No need to check the math connections between the metrics — they are already built-in.
Building a pyramid takes 20% the effort compared to building a metrics hierarchy and brings 80% the result.
For most businesses, it is more than okay. But if you are big or plan to be bigger than Amazon, go for a data-driven, well-balanced metrics hierarchy first, and then create a powerful pyramid off of it.
In my next article, I will reveal a secret superpower of the data-driven metrics pyramid.
Please let me know if you have any questions!
Conclusions
- Create a metrics hierarchy or pyramid for your product as soon as you have enough data.
- The hierarchy of metrics can show the exact reasons why metrics change and how these changes can cause problems for business goals.
- Top-level metrics of the hierarchy should not drop, even if the target release metric grows.
- The scientific method helps create an actionable data-driven metrics hierarchy.
- A metrics pyramid method simplifies the building of a well-balanced hierarchy of metrics.
- Use the scientific method to improve your metrics pyramid and unlock its superpowers.
