Any good goal is measurable, so organizations with properly defined objectives will establish metrics and KPIs to track their progress and momentum. KPIs attach measurability to what matters most and provide a North Star for the entire organization. A quick glance at the stats tells you if things are getting better, worse or standing pat, which is far more efficient and consistent than other, more subjective methods.
For product managers, KPIs also provide an easy filter for feature prioritization—if it isn’t expected to impact a KPI, then it shouldn’t be prioritized over something else that will.
But, in my experience, down in the trenches where product managers make feature-specific decisions on a daily basis, it is not always so easy to connect the dots between a particular choice and the big-picture numbers driving the entire company. And what happens if those high-level KPIs appear to run counter to customer feedback?
4 Reasons Why You Must Use Metrics to Determine Your Feature Decisions
Here are the four reasons why feature usage metrics should determine your feature direction.
1. Relying on your instincts isn’t sustainable
Seasoned product managers and newbies alike are continually tempted to make calls on the fly and use their knowledge and experience to inform impromptu decisions. It’s a pretty natural reaction—we’re intelligent folks with a decent track record wanting to appear decisive and wise.
In our modern era, however, it’s our responsibility to perform due diligence for every decision point we face. But despite the fact that 81% of people believe data should drive decision making, it’s actually only the case less than one-quarter of the time.
While your gut feeling may seem like the obvious answer, the numbers often prove otherwise and it’s foolish to ignore them in the name of expediency or ego.
“Relying wholly on your gut instinct may give you a decent start in business. But it’s unlikely to sustain you for the long haul, especially if you have ambitions beyond what you are already doing,” says David Molian of the Cranfield School of Management. “The message is not to ignore your gut instinct but – particularly if you are entering into new territory – to validate it with data and the advice of others.”
When you have a track record of success, it’s easy to fall into the trap of believing what’s worked before will work in the future.
“What makes expert intuition dangerous is that it has a magnetic force that draws us towards repeating positive experiences that have worked in the past. If a blue button was clicked more times than a red button, expert intuition would tell us to choose the blue one next time we’re faced with a similar problem,” says Geoffrey Keating of Intercom. “We’re usually forced to be reactive, rather than reflective. But in reality, if you’re too quick to find a solution, you’re probably not thinking hard enough about the problem.”
2. Overcome inherent biases and use metrics in your product decisions
We don’t always realize we’re relying on our instincts because, well, they’re instinctual! Just like we don’t think about our breathing, we’re constantly assessing multiple variables. Then, we make decisions based on the information at our disposal without even realizing it.
But I implore you, to make data-driven decisions, we’ve got to resist our instincts. Instead, try to intentionally short-circuit your default behavior and inject an additional step in the process. That’s where I’ve found processes and checklists can come in handy.
If part of your decision-making process is to validate against existing data, collect new data, or run experiments it’s far more likely to happen.
Another advantage of relying on data to corroborate our assumptions about features is that we’ll stand a far better chance of overcoming the inherent biases we all carry with us.
“Bias is the culprit behind the numerous faulty ways we perceive information and can lead us into making irrational decisions. They are pitfalls of our self-delusion where we judge situations through the filters of our personal experience and preferences while being completely ignorant of the fact that we are being affected,” says Hayden Hsu of GreenOrbit. “By going with our intuition, we don’t stop to consider all the details so we save time and energy and therefore are more efficient at decision making, but are we effective?
Learn how to prioritize the right initiatives that will make the largest impact:
3. Metrics help to avoid making snap product decisions
Beyond your own decision-making, leverage data to the feedback that you receive from stakeholders and customers. Just because one vocal individual (or even a handful) feels a certain way doesn’t mean an idea should be quashed or prioritized.
Instead, the suggestions and opinions can be validated with data analysis. When an executive declares “No one uses that!” you can examine the data and tell them exactly how often something in your product is being used. It’s helpful to have something that removes the emotions and politics from the decision and lets the facts speak for themselves.
4. Usage metrics will quantify what happens when people use your product
Data is only valuable if it’s useful, and it’s only useful if it can help you make decisions. What I’m getting at is, kick out those vanity metrics. Usage metrics are one of the best ways to quantify what’s actually happening when people use your product.
Feature Metrics You Should Track
So which metrics are most helpful when it comes to making feature-level decisions? Here are some of my favorites, many of which can also be found in our SaaS Product Metrics Pyramid:
- Active Users—This tells you how many people are using something (by day, month, or year). While this is often applied to the product as a whole, there’s nothing stopping you from zooming in on active users of a particular feature, a particular part of a feature, or even a particular button. When you see movement in these numbers, you know something’s going on. Tie it back to an intentional change and you can begin quantifying its impact.
- Session Duration—Generally, if people are spending more time using your product, it’s a good thing. But if you have a transaction or task-based feature, it might actually be a negative if it’s taking them longer than before to get something done.
- Bounce Rate—While this metric is more often associated with the conversion funnel, it can also be valuable at the feature level. If users start using part of your product and then head for the hills, it’s an indicator that they’re not having a great experience.
- Sessions Per User—If they like it, they’re likely to come back for more. If they don’t, they won’t.
- Actions Per Session—This is a great measure of engagement and usability. If users are doing more things when utilizing a specific aspect of your product, they’re likely happy with the results and realize the value.
- Sustained Feature Adoption—While usually reserved for the product as a whole, this can also be examined at the feature level. Simply calculate growth plus retention for the specific feature.
- Time to Value—This looks at how long it takes a user to complete a task you’ve already associated with other success metrics. For example, completing an order, setting up an account, or importing their contact lists. The shorter the better for this one, so if you can better track the changes made to shrink this stat.
With these metrics in tow, you can identify issues that require attention. Despite their usefulness in determining whether your feature-level decision-making is on the right track, metrics shouldn’t be the only thing powering your thought process. Data’s great, but it only tells part of the story.
“The promise of usage metrics is offset by a large limitation: Telemetric data is strictly behavioral and does not illuminate user intent, expectations, or satisfaction,” says Jerrod Larson and Daan Lindhout of Qualtrics. “Thus, usage metrics should not be considered in isolation; instead, they should be considered the starting point for additional research (or A/B tests) or a means of triangulating insights gleaned from surveys, usability studies, heuristic analysis, and so forth.”
How do Product Metrics Support Customers?
The pressure to universally apply data-driven decision-making can sometimes cause organizations to swing the pendulum a little too far and disregard customers’ feedback because it “disagrees” with the product metrics’ numbers. But data doesn’t mean you chuck your common sense out the window. Revenue and good references are important as long as the data isn’t indicating not to do something because of negative repercussions for the rest of your business.
Ideally, you communicate with sales, customer service, and actual customers. And then collect and synthesize that feedback and turning it into hypotheses. Then you can validate against existing data or test in an experiment.
Customer feedback is also often the canary in the coal mine. When a squeaky wheel lets you know they’re having trouble or are unsatisfied, you can focus your data analysis efforts on quantifying just how big a deal it really is. You’ll have the opportunity to react instead of waiting for the data to tell you months later that you’ve got some major issues.
Leverage Data Visualization
OK, so we’re all in agreement that data should play a major role in decision making, but when you’re on the hook for dozens of decisions, who has the time to scroll through endless streams of it? The lifesaver is data visualization.
A well-designed chart is a format people are wired to interpret and respond to. In fact, we process images 60,000 times faster than text. There’s a biological reason we’re such a fan of dashboards and presentations that skip dense text in favor of compelling visuals.
You shouldn’t entirely ignore your instincts in favor of product metrics, either
I’m not saying that millennia of evolution and your own years of experience are worthless in comparison to the latest analytics. Data is only as good as the analysis that’s applied to it, and someone has to figure out what exactly they’re trying to prove.
Rare is the breakthrough that came from simply looking at the numbers; it instead appears when you’re applying a particular lens to the available data. Listening to our gut as an impetus to test the hypothesis is the ideal point where instinct and data combine for decision making. It leverages the unique value human insight brings to the table but backs it up with cold, hard facts.
“Much of what I do as a product manager is probing. I look at a lot of data and talk to a lot of people, and watch for patterns,” says Alex Pukinskis of Native Instruments. “As I see patterns, I make intuitive leaps: If this pattern really exists, this action should produce this response. I then design experiments to test my intuition, and I use the data from the experiments to steer what we build.”
Make Metrics Work for You
As we wrap things up, it’s important to remember that data is your friend. It might seem endless and intimidating, but when you have a question, data can typically help provide the answer. And once you’ve found a way to wrangle it to your tastes, it becomes a powerful weapon in your arsenal.
De-politicize contentious disagreements, justify your decisions and boost your profile with gorgeous charts illustrating the sterling results of your strategic product leadership. With the right metrics, you can put opinions out to pasture and focus on fact-based execution to achieve the goals your company has created.
Ready to dive deeper into metrics and data? Sign up for our email course.
