Writing Next-Level Surveys: How to Get "Story Data" from Static Questions

Surveys are a controversial user research methodology.

On one hand, they give a large amount of quantitative data, which makes stakeholders feel more comfortable making decisions.

On the other, they turn users into numbers and don't allow us to dig deep in understanding the "why."

But there is a time and a place for surveys. The important thing is recognizing when that is—and implementing solid best practices for creating them.

Why surveys aren’t the best UXR tool

I know companies that rely solely on survey data to make data-driven decisions. They include comment boxes with the hope users will leave a message to explain their survey response— but the majority of users don’t.

The truth is surveys don’t drive better decisions on their own. At times, too many companies use surveys as a way to convince themselves they are customer-centric. But when you only use one method of user research or use only quantitative or qualitative data, you are missing other pieces of the puzzle that give you a more robust and empathetic understanding of your users.

However, there is a time and a place for surveys. And when that time and place surfaces, you need to be prepared with how to create a useful survey. My best piece of advice is to think of a survey as a conversation, not a list of questions. This approach will get you out of a survey-question rut, and into a more creative headspace.

Surveys can be a great complement to qualitative research, and give you big numbers, which helps with gaining momentum and buy-in.

This self-reported data gives you essential information about the user's perceptions. It is necessary to follow-up with qualitative data to understand why users are experiencing these perceptions and reactions.

How could we use surveys effectively?

We want our users to tell us stories about their experiences, even in our surveys. Stories are compelling because they tell us what a person did, instead of what they think they might do in a given situation.

What people say they would do versus what they do can lead to two very different scenarios. Getting these stories out of people can be very difficult through a survey.

However, there are a few ways to optimize your surveys to achieve this potential. Let's look at how you can write better surveys by making sure they are delightful for users and allow them to tell you their stories.

Ask about past behavior

The best predictor of future behavior is past behavior, so it is vital to use this method regardless of the type of survey question.

Asking users about the past forces them to answer with what they did, instead of projecting potential future behavior in a given scenario or opinions.

For example:

• Future-based question (that I see way too often): "Would you purchase this particular gym membership package?"
  • Follow-up question: "Why would you?" OR "Why wouldn't you?"
  • Follow-up question: "What would make you?"
• Past-based question: "In the past, have you ever purchased a gym membership package?
  • Follow-up question: "Why did you purchase?" OR "Why did you not purchase?"
  • Follow-up question: "If you did purchase, did you use the membership package?"
  • Follow-up question: "Why did you?" OR "Why didn't you?"
  • Follow-up question: "What was missing from this membership package?"
  • Follow-up question: "What was a frustrating moment you had with the membership package?" AND "What was a positive experience you had with the membership package?"

The past-based items allow you to understand how a person responded in a scenario. You are giving them the chance to recall particular memories and explain real-life examples. You are not asking them to extrapolate or simulate future behavior.

People cannot accurately describe their future behavior, and this includes whether or not they will use or buy a product/service.

By seeing how they responded in the past, you can construct a story of how they might act in the future. You can also ask them what was missing from their previous experience to fill in those gaps.

First-person questions

First-person questions are super cool and always make me smile when I see them on a survey. These replace the standard third-person survey questions with a more personal touch.

For example:

• Third-person question: "What is the hardest part of our check-out process?"
• First-person question: "The hardest part I encountered during the check-out process was…"

By using the first-person question writing technique, you can increase the chances of gathering authentic and personal responses through a survey.

First-person questions are a great technique to turn a survey from an impersonal form to something more comfortable and more personable for a user to fill out. I now use this technique in most of my survey design.

Open-ended questions

Open-ended questions on surveys might make people slightly skeptical, and for a good reason.

How many times have you zoomed past a, "Please let us know if you have any other comments" box on a survey? It’s hard to get qualitative data in a survey, but I highly recommend including these types of questions.

Why are these so important? They can give you a small insight into why people are responding in a particular way to your "closed" survey questions. There are three ways you can incorporate open-ended questions into your surveys:

• A stand-alone question
• Follow-up questions
• An "other" option for each (relevant) question

Then there’s the most difficult open-ended question: the stand-alone. This question is completely free from other survey questions and requires the user to think and write down their thoughts.

There are a few examples of this in the above section, such as, "What is missing from your current gym membership package?"

A way to call more attention to, and make these questions more appealing, is to use the first-person questioning approach. You would turn the above into, "What I am missing from the current gym membership package is..." This approach won't guarantee more answers, but it helps catch people's eyes and make the survey feel more personal.

Follow-up questions

Follow-up questions can also be difficult to place correctly in a survey. These tend to come in the form of logic-based questions.

For example: "On a scale of 1-5 (1 = bad, 5 = excellent), how do you feel about your current gym membership package?"

If someone answers with a 1 or 2, we can ask, "why do you feel that way about your current gym membership package?" You could also do this if someone responds very positively with a 5.

Follow-up questions give the respondent the chance to explain their answer, which is especially helpful for a rating scale.

Finally, always include an "other" option with an open-text field. I have seen many surveys with precise answers to questions, which can be great, but they didn't have an option that was relevant to me.

By including an "other" option with an open-text field, you allow respondents to answer in a way most relevant to them. This also gives you insight into the types of answers you are not aware that you could include in future surveys.

Give your surveys a mix of questions

The best surveys I have seen include multiple types of questions. They achieve the delicate balance of closed and open-ended questions.

It is essential not to write every survey question as an open-ended question, and also to include the opportunity for respondents to express themselves. I typically use the rule of 60/40.

For example, your survey might have sixty percent closed questions, and forty percent a mix of the different types of open-ended questions. Forty percent may seem like a lot of open-ended questions, but not each one is a stand-alone.

Always practice your survey, or send it out internally, to make sure it isn't overburdening your participants.

Conclusion

Although it can seem complicated, survey design can be fun and effective. Use these tips above to add some spice to your survey questions, and don't be afraid to make people smile.

I just recently released a candid survey with some fun language, such as "cool" or "sweet" or asking for "total honesty." Mix it up, and don't be afraid to break the mold of typical survey writing!