Three Principles for Making Better Decisions
Product Management, Meet Behavioural Economics
Of the countless decisions you have made throughout your life, how many of them would you say were correct?
I don’t actually expect you to come up with a number, but I think it’s fair to say we all want to maximise the number of correct decisions. Put another way, we want to avoid making errors.
Daniel Kahneman is a Nobel Prize-winning Psychologist and perhaps the world’s leading expert on judgement and decision-making. In his latest book “Noise: A Flaw in Human Judgement” (co-authored by Olivier Sibony and Cass R. Sunstein), Kahneman describes how decisions gone awry can have devastating impacts, and outlines strategies for avoiding these errors.
According to Kahneman, errors in decision-making are comprised of two things: bias and noise. But when people (or organisations) want to improve their decision-making, they tend to focus only on reducing biases. This is because biases are much more visible when reflecting on an incorrect decision in hindsight — humans crave causal explanations, and biases have an “explanatory charisma” which noise lacks.
“Confirmation bias is the reason they made the wrong decision”.
“They hired the wrong person because they fell for the halo effect”.
“No one spoke up beforehand because of social desirability bias”.
Biases, once identified, are predictable. If a set of bathroom scales gave a reading that was consistently higher than your true weight, then you would say the scales are biased. Because the error is predictable, the process of eliminating it is more obvious.
In contrast, a noisy bathroom scale would be inconsistent, giving a different reading every time you stood on it. This is why noise is so often ignored — the errors it produces are less visible because they are unpredictable. Though less visible, the consequences of these errors are no-less costly.
The Cost of a Bad Decision Can Be Enormous
The manufacturers of the Concorde supersonic airplane famously fell victim to a common bias: the sunk-cost fallacy, when they made the erroneous decision to continue the project even after it was clear that breaking-even was impossible. Their aversion to “losing” the significant investment already made ultimately caused them to lose more than they otherwise would have — in the end approximately £3.4 billion pounds (adjusted for inflation).
Decision errors are more common than you think, and because they accumulate over time, the financial impacts can be staggering. Therefore, it’s not surprising that lot of work has been done to minimise the number of errors made in organisational decisions.
Many organisations now actively use debiasing techniques to reduce the business risks associated with biased decision-making. The methods they employ are based on learnings from the fields of psychology and behavioural economics, to which Kahneman’s work has contributed significantly. In his latest book he builds upon this work by highlighting the contribution of noise to the error equation:
Kahneman outlines a set of principles for making better decisions, which he calls decision hygiene. Just as practicing good personal hygiene can prevent illness, practicing good decision hygiene can prevent errors in decision-making.
In this article I’ve summarised these principles below, through the lens of a product manager. However I’ve kept the scenario somewhat vague, since I believe these principles are universally valuable and can be applied in a variety of contexts. I’ve also included a short summary at the end for those who are already somewhat familiar with these principles — think of it as a “decision hygiene cheat-sheet”.
“Wherever there is judgement, there is noise — and more of it than you think.”
— from “Noise: A Flaw in Human Judgement” (p.12)
Three Principles of Decision Hygiene
Let’s imagine the following scenario:
You’re a product manager and you want to build a feature which will decrease the time to value for your customers. After some discovery work, you’ve narrowed down your options to five possible features — all of which seem desirable, usable, feasible and viable — but you can only build one.
How do you decide which one is the right choice? In an ideal world you would choose the one that will produce the greatest benefit with the lowest cost, but how can you tell which that is without being able to see the future?
Principle One — “Resist premature intuitions.”
Structure your judgements using several independent criteria.
When you look at the five options, you might find that one immediately appears more attractive than the others — this is your intuition talking, and you must ignore it (for now, it will come in handy at the end!).
Don’t get me wrong, your intuition is a great time-saver but it is quick to jump to conclusions, and it often makes mistakes (If you’ve read Kahneman’s previous book “Thinking, Fast and Slow”, this is System 1). We can reign in our intuitions by giving our judgements some structure.
Instead of evaluating each option as a whole, define 3–5 specific criteria which are relevant to the decision being made. Then give each option a rating along each of these specific dimensions. For example, you might want to compare options based on technically complexity, Or maybe how much of an impact each would have on a specific organisational metric. The important point is to consider each criteria in isolation, to avoid letting a good score on one criteria blind you to a bad score on another.
By focusing on specific criteria, you force yourself to compare the options on a level playing field. This principle is particularly powerful in the hiring process (Google use it!) as it can help organisations avoid common biases, such as the halo effect — where a positive first impression casts a “halo” that can lead a hiring manager to overestimate a candidate’s ability, or even disregard a clear red flag.
“We have a structured process to make hiring decisions. Why don’t we have one for strategic decisions? After all, options are like candidates.”
— from “Noise: A Flaw in Human Judgement” (p.324)
Principle Two — “Think like a statistician.”
Take the “outside view”, because everything regresses to the mean. Try to use relative judgements rather than absolute ones.
You don’t need to have formally studied statistics to think like a statistician (though bonus points if you have!), the key here is to take the outside view when evaluating your options.
In our example, building Option B might be highly popular within the organisation, and so you might be tempted to choose it based solely on that fact. However this is taking the inside view — you have only considered the information from within your current situation.
Imagine instead that you looked more broadly at what happened when other organisations chose to build something similar to Option B. It would not be surprising to find that within your own organisation, the expected benefits of Option B are overly optimistic when put in context.
In short, taking the outside view means taking into account the statistics of a class of cases similar to the present one. Anchor your expectations to the average — a good rule-of-thumb given the inevitable regression to the mean.
Another important thing to consider is the scale on which you are evaluating your options. Where possible, use relative scales as opposed to absolute ones. It has been shown that people are much better at ranking options than they are at scoring them on an abstract scale.
A relative scale is less noisy than an absolute one because pairwise comparisons are more reliable both between individuals, and within an individual at different points in time.
For example, if you were asked to assign Option B a number between 0 and 10 today, and then again in a few weeks time, you’d likely choose a different number. However your pairwise comparisons (“B is better than A, but not as good as E”) will be much more consistent over time.
(Author’s Note: I am not suggesting to avoid absolute scales entirely! However, when you are comparing the relative values of a number of different options, ratings along relative scales are more statistically reliable).
“We effortlessly invoke causes for the events we observe, but thinking statistically about them must be learned and remains effortful. Causes are natural; statistics are difficult.”
— from “Noise: A Flaw in Human Judgement” (p.219)
Principle Three — “Accuracy, not individual expression.”
Aggregate multiple independent judgements. Use the wisdom of the crowd to anchor decisions to a reliable estimate.
The goal of judgement is to be accurate, not to express the opinion or intuition of a single individual. Therefore, it is almost always a good idea to leverage the combined experience/knowledge of a group of stakeholders.
If you’ve ever watched the game show “Who Wants to be a Millionaire”, you’ve witnessed this principle in action whenever a contestant chooses to “ask the audience”. Every audience member votes for the answer they believe is correct, and the contestant then uses the aggregate results to anchor their own decision. This is a powerful lifeline to use, in fact the US host has claimed the aggregated votes of the audience are correct 95% of the time.
This principle works because of the phenomenon known as the “wisdom of the crowd”. A group of people will naturally hold a variety of viewpoints, and therefore a variety of biases or preconceptions. From a statistical perspective, aggregating multiple viewpoints “averages out” the error within the group — leading to a more accurate estimate of the true value.
It is important to note this can backfire spectacularly if the crowd you choose is too homogenous (i.e. not enough diversity of knowledge/expertise). If everyone in the group shares the same biases, averaging their judgements will not average out the bias.
The key to choosing a wise crowd
Find a diverse group of stakeholders with experiences and knowledge that vary. Make sure each person has the time and space to make their judgements independently, and early on (using the first two principles!), before the group comes together for a round-table discussion. Too much cross-talk will cause the group to become more homogenous in their thinking — because people will naturally influence each others decisions.
Finally, aggregate the independent ratings so that you have a single score for each item. There are many ways to do this, but the easiest method is a simple average (For the maths nerds reading this, follow me on Medium if you’re interested in reading a follow-up article describing more specific methods!)
Aggregating the ratings has the added benefit of anonymising the individuals judgements. If people know that their judgements will be visible, they may alter their judgements to avoid scrutiny — this is called social desirability bias. This observer effect is a key reason why we use voting booths in elections and anonymous surveys when gathering employee feedback — because otherwise we cannot expect the responses to be honest.
“Unfortunately, most roundtables are run in a way that makes it much too easy to achieve agreement, because participants quickly converge on the opinions stated first or most confidently.”
— from Harvard Business Review
Making the Final Decision
At the beginning of this article, I told you to temporarily ignore your intuitions. It is at this point that you can unleash it.
If you followed the principles of decision hygiene described above, you will have a reliable rating for each of the five options, and these ratings will reflect the structured, independent judgements of a wise crowd.
It is important that your process does not prescribe the final decision! The goal of decision hygiene is to reduce decision errors by delaying intuition, not banning it. If you simply let the highest aggregate score determine which option is chosen, stakeholders may feel their hands have been tied and reject the process entirely. Instead, take the wisdom of the crowd to the table and use it as an anchor in the final discussion.
Without an anchor it is more likely that a single loud or prominent voice will unconsciously steer a group off-course. But someone would need a particularly strong and reasoned argument to dispute a decision reliably anchored by structured considerations, reliable statistics, and a wise crowd.
A Short Guide to Practicing Decision Hygiene
- Structure your judgements into several independent criteria. This will help you consider the different facets of each option more clearly, and therefore more objectively.
- Avoid using absolute rating scales. Instead, rank your options from best to worst, considering each criteria in isolation. That means if you have three different criteria in your evaluation, you will have three separate rankings.
- Consider the statistics of similar situations when making your judgements. Don’t fall into the trap of thinking “because X happened once, it is likely to happen again”— anchor your estimates to the mean.
- Use the wisdom of the crowd. Gather a diverse group of trusted stakeholders, and avoid discussing the decision before every person has evaluated all options independently.
- Aggregate the scores into a single metric for each option. Use this aggregated score as an anchor when making the final decision.
With time, practicing decision hygiene becomes second-nature, and it will feel as simple (and necessary!) as washing your hands. Now go forth and take these principles with you as you navigate the many decisions that lie ahead!
Choose wisely!
If you’re curious…
I’m a computational neuroscientist working in fintech, and I use my background in quantitative research, statistics and behavioural science to help guide product development.
This article was heavily inspired by the book “Noise: A Flaw in Human Judgement” by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein (2021). I believe the principles and practices it outlines have immense value, and I want to share the things that I have found helpful in my own line of work.
We all make multiple decisions every day. Granted most of them aren’t high-stakes organisational bets, but regardless — decision hygiene isn’t a process, it’s a set of principles I strive to apply to every judgement I make.
I hope I’ve inspired you to do the same.
