Better Product Management through Science

Build models and test them with data

Aaron Berdanier
Product Coalition

--

Photo by CDC on Unsplash

You can become a better product manager by adapting a scientific mindset.

That sentence might seem sensational, but bear with me. I’ve been a scientist for 13 years and a product manager for three and I think there is something to the connection. I see a lot of opportunity in the overlap. Let me start by explaining what I mean by science and products.

Science is an approach for understanding the world that is based on empirical evidence. The “evidence” is data from observations or experiments. And the “understanding” is a model to explain or predict those observations. So scientists make models and test them with data.

Products are goods that people consume. A successful product must meet the needs of its users and add value for the business. To help ensure this success, product managers prioritize problems and discover solutions that are both valuable for customers and viable for the company.

How does a scientific approach apply to product management?

Let’s mash them up.

If managing a product were a scientific practice, what would we be trying to understand? Product managers need to understand what problems are worth solving. With a list of problems that a development team could solve, there are many potential ways to prioritize and solve them. These roadmaps and backlogs are different ideas about what will add value for customers and what will be viable for the business. In scientific terms, they are different models for understanding “what the team will build” to add value.

Given that product managers need to understand their priorities to support the team, how is using evidence helpful? Product managers also need to build alignment and develop a shared understanding of “what the team will build” across multiple stakeholders: users, customers, executives, and engineers. A product manager could outline a strategy or a roadmap or a backlog that is not backed by data. But, once trade-offs come up, the conversation will turn to questions about “why”:

Why is this our model of the world? Why was X prioritized before Y? Why wasn’t my request considered?

Without evidence, that conversation devolves pretty quickly into different people’s opinions. As Jim Barksdale famously said as a CEO, “If all we have are opinions, let’s go with mine”. Evidence is a great way to build influence without authority.

So, I think that viewing roadmaps and backlogs as models that we can test with data (the scientific approach) is a powerful way to make decisions and build support. This approach fits the ways that we often talk about product management responsibilities, from “discovering problems” to “measuring outcomes”. Of course, this is sometimes easier said than done.

How can you adapt some scientific habits in your product management practice?

Luckily, people have been practicing science for centuries, so there are a lot of examples for what works (and what doesn’t — like being led astray by cognitive biases). Also luckily, there are some mindset shifts that can help you think like a scientist. Scientific thinking is a skill that can be cultivated, even if the only thing you remember about science right now is what you learned in high school and even if you are unsure about what kinds of data you could have at your disposal. You can take small steps on your journey to becoming a scientific product manager!

I’m starting a list of some of these scientific thought patterns that are relevant for product management:

  • All models are wrong
  • Discovery is nonlinear
  • Multiple hypotheses prevent biases
  • Correlation does not imply causation
  • Assumption tests reduce risks
  • Falsification is stronger than validation
  • Model building and testing are continuous activities

For example, let’s start with the first one — all models are wrong, which is usually followed by noting that some models are useful. If you view your roadmap as a model, you can recognize that it is simply an hypothesis about how the product could add value. There is no perfect roadmap because we cannot perfectly see the future. This idea has helped me make more useful roadmaps by forcing me to question the plan and be more comfortable changing the plan if necessary.

There is no perfect roadmap because we cannot perfectly see the future.

When I remember that all roadmaps are wrong, I immediately start to ask myself “What are we trying to demonstrate?”, which pushes me to clarify the desired outcomes instead of just stating the output. This mindset leaves the roadmap open to revision as we learn more about how we might want to achieve our goals. For example, I say we’re going to help customers do Capability X with Feature Z because we think it will increase product adoption. If I acknowledge that this might be wrong, then it is easier to adjust when a sales teammate tells me that none of their prospects talk about Capability X. That situation could either lead to a cool brainstorming conversation about how we could figure out if Capability X would be valuable or a decision to pivot and re-prioritize if there is enough evidence.

Overall, an all roadmaps are wrong mindset takes some of the pressure off to get it perfect, which could help you start talking about it sooner, which will ultimately benefit the product.

Are you interested in learning how to apply these scientific habits to product management? Give this post a clap if you want me to write more about it. Do you have any ideas to contribute? Feel free to add them in comments.

--

--