A closer look at how a popular mental bias, survivorship bias, might be affecting our decision making in our day-to-day as product leaders. And why we should care!

I love entrepreneurship, storytelling and design. So it’s no wonder that for the past few months, during my morning commute, I have been methodically listening through hundreds of podcasts on some of the most successful startup stories out there.

Here’s a some of the inspiring stories I got to listen to:

• Jennifer Hyman and Jennifer Fleiss founding Rent the Runway at the age of 20 whilst studying in Harvard Business School. Today, Rent the Runway is valued at over $770 Million.
• John Mackey dropping out of College to work in a local co-op and later, starting Whole Foods Market with his girlfriend, Renee Lawson. John is now an incredible entrepreneur with an estimated net worth of $100 million.
• Miguel McKelvey starting WeWork from a small office space in Brooklyn and now employing 5000 people in almost 100 cities.
• Stewart Butterfield struggling to fund the development of a video game and thus switching to building the leading enterprise conversational platform, Slack.
• And so many more…

On another note, I recently read the book Good to Great where Jim Collins describes how companies transition from being good companies to great companies, and how most companies fail to make the transition. By doing so, Jim Collins tries to highlight traits of great companies that must be responsible for their success, advising others to follow the example.

As I was collating all this information, I felt eager to make it part of my own personal development. Yet, a weird feeling struck me... A feeling that maybe something is off. Are we all only looking at one side of the stories we are given, or in a more formal way, are we suffering from survivorship bias?

What is survivorship bias?

Let’s start with an understanding of what survivorship bias is. Wikipedia defines it as:

“The logical error of concentrating on the people or things that survived some process and inadvertently overlooking those that didn’t because of their lack of visibility.”

In a nutshell, it is the tendency to focus on people, products or companies that have succeeded and trying to replicate the exceptional elements that brought them there within your own space.

Imagine yourself as a fresh graduate with the world at your fingertips. You just heard this cool podcast about how Jeff Bezos built Amazon from his garage. Inspiring, right? Well, guess what… For every successful startup founder, there are tons of students with brilliant ideas who end up in a dirty garage.

Why is it important?

There is in essence nothing wrong with learning from the survivors, they can teach you a lot about what worked for them. However, they might be depicting only a small part of what you are trying to understand: what not to do.

Let’s take an example: a lottery winner. This person is probably very passionate about sharing what got him there and his strategy is straight-forward: “never stop buying lottery tickets, no matter what”.

Would you follow the advice? This is meant to be a simple illustration of how we tend to focus on traits of the winners (in this case, the lottery winner) and ignore the losers (millions of people buying tickets every day).

Understanding the impact of survivorship bias can (probably) lead to making better decisions.

Why, as a product leader, should we care?

As Product Managers, Startup Founders, Visionaries, Tech Enthusiasts and anyone else with an appetite for solving problems; we are constantly bombarded by stories on what we should do better, making it sound like it’s as simple as ticking tasks off a list.

Reason #1: “Our competitor managed to double its revenues by doing this. Let’s to do it too!”

Whatever works for others, may not work for you and the uniqueness of your business. It’s great to get inspiration from the market and competitors, I do it all the time, but the most important is the deep understanding of the reasons why it worked for those businesses whilst controlling for external influences or differences with your own business.

It’s important to always keep in mind what your customers are trying to achieve when using your specific product. A useful framework to do this is called the Jobs-to-be-done framework with the Value Proposition canvas.

You can’t expect consumers to design the finished product… But that doesn’t mean they don’t know what they want! With my teams, we spend a lot of time talking to users and asking the right questions, in the right way. Users will tell you exactly what are their current problems, what they like and don’t like about their current process. It’s our job to design the right products or services to meet their needs.

Understand what it is people want to achieve, what their big frustrations and problem are, what benefits and value they’re looking for, and you’ll have a very clear idea about what your solution should look like. It’s very likely this solution will be unique to your business and customers.

Reason #2: “Our top customers are asking for these features, let’s start building them!”

In product development, “non survivors” tend to be customers who started interacting with your product and then gave up at some stage of the funnel for whatever reason. Those people may not have reached out for help simply because they didn’t go far enough within the funnel to be able to. You lose the opportunity to get their feedback!

On the other end, you have people who are very happy with your product (the survivors) and they will often give you feedback about things that they’d like to see in the product for which they may currently have found a work around for. Now the problem is that these top users, those who you have already sold to, those who have flown all the way down your marketing or sales funnel and now use your product regularly, are the ones from whom it is most easy to gather feedback.

The most interesting is understanding whether the survivors’ opinions about how to improve the product are the same as that of the non survivors — survivorship bias suggests this probably isn’t the case so we need to find a way to learn what’s hurting the non survivors.

To conclude…

Whilst I will not stop listening to podcasts, reading competitors’ success stories or listening to our most loyal customers, the idea as a product leader is to balance the information overload you are getting and focus on the most important problems to solve for your existing and future customers today!