Data — Is it the product manager’s best friend?
Data:
information, especially facts or numbers, collected to be examined and considered and used to help decision-making, or information in an electronic form that can be stored and used by a computer
In my previous article, I’ve talked about the similarities between the product manager and the intelligence officer. The first commonality was data. In this article I’ll cover whether and why data is important for the product manager.
In today’s world, we are surrounded with data, to a point it is overwhelming. But, I like to say — there’s no such thing as too much data :).
Data — What’s it all about?
I grew up on the 80`s (from this data — you can now figure out my age), and as a curious kid, I had to collect my data from books, magazines and TV shows. My access to data was limited and very time consuming. I didn’t have a data collection engine with keywords collecting the data, nor Google search. I had to work hard to collect my data. When you wanted to learn something or gather data — you had to physically search it in the books — I still have the hard copy of Encyclopedia Britannica. I also remember the joy of finding what I was searching for.
Today, our ability to access more data faster has transformed us from collectors of fine data to hoarders of raw data. No longer limited by physical limitations, we create huge Data Lakes which we repeatedly explore on our analysis journey.
A data lake is a vast pool of raw data, the purpose for which is not yet defined.
Think about being able to take only 36 pictures on your phone at any given time as opposed to almost unlimited. I know, some of the readers of this article will go WHAAAT? but that was the reality not that far back.
So, we have collected all this data, simply because we can, what’s next? How the hell can we extract what we need? What do we need? Why did I take 30 pictures of my kid in the same pose?
“Without data, you’re just another person with an opinion “ — W. Edwards Deming
A valid question would be — Why did we collect all this data?
Before we began collecting the data, we started with a question / hypothesis. The data is the aftermath of this question, what we use in order to validate our questions.
The intelligence officer
Let’s go back to the intelligence world for a second:
As the intelligence officer we have a question — For an example:
Where is a specific terrorist is hiding. The data to be collected is widespread, but, a good data collection is based on segmentation. Based on this question we start collecting a widespread of data from different segments or sources:
- His family / friends / colleagues / enemies.
- Ideology
- Social media activity
- Verified seen places
So, now we have a vast collection of information. We need to analyze the data, it’s relevancy, and start locating the pieces of the puzzle. The pieces we are connecting, are results of the data we collected in each segment. What happens next is our Aha moment.
This happens when we understand that we have something interesting, as we’ve all seen it in the movies, when the intelligence analyst / detective finds the big clue. I can tell you from my own army experience, that this is not only a Hollywood thing :).
Now, as we understand the direction we took is correct, we lay our next question or hypothesis. Once we have the next hypothesis — we will concentrate the data we need to collect, refine our segmentation in order to validate it, and so forth.
The product manager
Let’s take it to the product management world:
It always starts with a question or an idea:
- An aha moment in the shower / toilet / dream — choose your own creative spot
- What do our users do/need/want/like/don’t like
- Is the new feature being used?
- The users acquisition flow / funnel
- Our branding
- What our competitors are doing / not doing / going to do
Same as the intelligence officer — we need the data to do a better job.
Collecting the data
So, we start collecting the data. The more the question is clear, so is the accuracy of the data we collect and the target of our analysis.
We incorporate data collection methods:
- Analytics tools
- User interviews
- Support tickets
- Customer success
The collected data might be wide at first, so we segment it. The segments can be based on sources of the data, geospatial, personas and whatever we see fit for the question we asked.
As the data is gathered, and although you’ve applied your segmentation — you need to prepare yourself for one thing:
The data is what it is, and you need to understand that the data you collect might suck!
And as much as it might contradict everything we thought — we need to stand behind it. Remember that the whole reason behind the collection of the data is the validation of our hypothesis or idea. And we rather fail fast and deliver the right thing, rather than enforce our vision.
Another pitfall to be aware of, is when all the data indicates you are correct, and there is not even a shred of data contradicting your hypothesis. This might be an indicator that there is some data you are missing. Be aware of this!
As product managers, we constantly exploring the current question / hypothesis / aha moment — This is why we always need to analyze the data, validate the question and create the next one.
Another aspect of the data that we need to consider is the nature of our system or product. Although we all have users, a B2C product will differ in many traits from a B2B or even a B2B2C product.
For example, in a B2B product, sometimes the data collection is not that trivial, especially when you have to deal with on premise systems and no connection to the outer world. In this case you have a few options:
- User interviews and usability testing on site
- Customer success team
- Support team — open tickets and feature requests.
In a B2C product the data collection is usually much easier, as we have direct interaction with our users and they have connectivity, so we can apply multiple methods of data collection.
Nevertheless, whenever you go over the data, DO NOT forget — the data is what it is, and there are times the data will suck.
Not always the answer you were looking for will be in the data you collected.
“Many of the truths that we cling to depend on our point of view.” Yoda
For example, I remember we’ve added a simple feature — Three buttons which enables you to apply a filter quickly, instead of going through the filters list. When we did a UI uplift to the system, our hypothesis was that we can simply remove these buttons.
As soon as we released this version, among the first data collected was: “Where the heck are my buttons”. We truly didn’t expect this, it completely contradicted our hypothesis, and we had to revert this change in the next version.
The data is what it is, and there are times the data might suck. Not always the answer you were looking for will be in the data you collected.
I’ll finish on a different note and I’d love to hear from your experience:
After this being said (and I’ll contradict myself a bit) — I know there are times where the data will show something clearly, but something in our gut says it’s wrong. Whether to go with the data or your gut, is a question which is hard to answer, and can be a whole post on its own. The answer is subjective and based on experience, as there are no clear rules.
Anyone inexperienced puts faith in every word, but the shrewd one considers his steps
The experience is what enables you to decide when to do go with the data and when to go with your guts. What do you do?
