Join us for our upcoming user group & networking event in Orange County on April 16th - register now for free!
Product Development

How to Avoid Psychological Biases in Your Research

October 8, 2019

In a perfect world, all the data you collect from your surveys would be 100% unbiased. But I don’t have to tell you that’s not the world we live in. In reality, bias is common in research. These psychological biases — the tendency to make decisions or take action in an illogical way — are introduced both in the way questions are asked and the way they’re answered.

Fortunately, addressing these biases does wonders to help you notice when they creep into your survey questions — and inadvertently skew your results. This blog will dive deeper into the most common psychological biases found in survey research and how to combat them to get more reliable data.

Social Desirability Bias (Conformity)

This bias occurs when people report attitudes or behaviors that they feel are socially desirable (or underreport what they feel is undesirable). Deep down, people want other people to think the best of them. This is especially true when you’re asking something personal or sensitive in nature (such as details about personality, knowledge, or performance).

Example:

How easy/difficult was it for you to install your router?

Very difficult

Difficult

Moderate

Easy

Very easy

Asking someone to admit to a potential shortcoming (i.e., it was difficult for them personally to install the router) might discourage a truthful response. In the suggestion below, the emphasis falls on the installation process, not on the person performing it. It’s a subtle but powerful difference.

Try this instead:

How easy/difficult was the router installation process?

Very difficult

Difficult

Moderate

Easy

Very easy

Other ways to discourage social desirability bias include: ensuring anonymity, randomizing options (and possibly questions), and cross-referencing data to identify outliers.

Acquiescence Bias (Yea-Saying)

It might surprise you, but people generally want to be agreeable. This causes another type of bias: acquiescence bias, or the tendency to report attitudes and experiences in a more positive light. Acquiescence bias goes hand in hand with social desirability bias because people want you to think they’re agreeable and polite. As you’re writing your survey, make sure you avoid framing your questions in a way will that unfairly bias the results in your favor.

Example:

Did our services meet all of your needs?

Yes

No

Because this question is a yes/no, people are more likely to respond “yes” — even if their needs weren’t completely met. By offering a scale, survey-takers are able to respond in a way that reflects their experience more accurately.

Try this instead:

Please indicate the extent to which you agree/disagree with the following statement: Our services met your needs.

Strongly disagree

Disagree

Neutral

Agree

Strongly Agree

Order Effects Bias (Question Order)

In psychology, “priming” is an effect that occurs when one stimulus influences the response to another stimulus. For example, people who see or hear the word yellow will recognize the word banana faster than those who don’t!

The order of your survey questions can change the results in a way that is very similar to priming. By asking particular questions first, you are potentially influencing the response to later questions. If you’re not careful, this will bias your overall responses and affect the accuracy of your data.

Example:

1) Describe any problems you faced while using this product. [insert answer choice]

2) How dissatisfied/satisfied are you with this product?

Very dissatisfied

Dissatisfied

Neither satisfied nor dissatisfied

Satisfied

Very satisfied

Priming survey-takers to think of the negatives before they assess your product overall could falsely weigh down your satisfaction scores. By asking respondents to rate their satisfaction then explain why they felt that way, they’re able to report their overall perception of the product experience with less bias.

Try this instead:

1) How dissatisfied/satisfied are you with this product?

Very dissatisfied

Dissatisfied

Neither satisfied nor dissatisfied

Satisfied

Very satisfied

2) Please explain why you were [insert answer choice].

Think about the mindset you are putting your respondents in. You can also combat question order bias by randomizing question categories. And, of course, by testing your survey.

Confirmation Bias

Biased data doesn’t always come from the people taking the survey. When you start building your project objectives, having a hypothesis in mind for how that research data will look is a given. Confirmation bias occurs when a researcher is so focused on proving their hypothesis that they ignore any data that could disprove it.

It’s important to recognize when you’re projecting your thoughts and attitudes onto your data, rather than staying true to your audience. Here are three steps to combating confirmation bias in your research.

  1. Acknowledge confirmation bias
    If you start to see results that confirm your hypothesis, it’s important to take a step back and acknowledge any personal biases.
  2. Ask yourself, “How does this data back up my hypothesis?”
    Take a moment to check in and see what’s supporting your hypothesis. Are there many indicators pointing to your hypothesis or just one?
  3. Have someone else review your results
    It’s always good practice to have someone check your work. A second pair of eyes (especially when those eyes belong to a trained researcher) exposes biases that you are too close to your research to see.

Ready to put these techniques into practice? Learn how to pull in squeaky clean data from your CV surveys with the structure and question writing tactics in our free ebook, Building Surveys for Customer Validation.

Dig into the Survey Ebook

No items found.