How to ask better user research questions: A guide for startup Product Managers
We do research to bridge the knowledge gap between reality and our model of it. Without writing a single line of code, user research sessions rapidly generate a lot of insights. However, running a research session is hard. We are distilling insights from anecdotes shared by imperfect historians. To get meaningful data in a short amount of time, we need two things in our arsenal:
- Questions to uncover the problem
- Tactics to keep the conversation focused
Questions to uncover the problem
There are two types of research sessions that I frequently run:
- Early-stage research
- Getting feedback on ideas
Early stage research
The goal of early-stage research is to uncover what’s holding people back from doing what they want. For this, I like to use Chuck Liu’s approach:
- What are you trying to get done, and why?
- How do you do it today?
- What could be better about how you do this?
What are you trying to get done, and why? Use this question to gain context about their goal. To get a full picture of the environment in which they are looking to solve problems, ask a few follow-up questions:
- Why is this goal important?
- Who are the other stakeholders?
- What happens next after this goal is achieved?
How do you do it today? Use this question to develop an understanding of their current workflow. Additional follow-up questions to understand the details:
- When was the last time you did this task? Was it representative of what you usually do?
- What are the key steps?
- Could you walk me through it?
- What are the documents that accompany these steps?
- Who is responsible for each step?
What could be better about how you do this? Listen to answers to find opportunities and ideas. Questions to get people to reflect:
- What is harder than it should be?
- What takes the most time?
- What unofficial workarounds have you had to build?
I find it helpful to conduct research by keeping in mind the summary I would share with the team in the end. The research summary looks like:
In order to _____ (goal), users _____ (action).
Today, they achieve this goal by using ____ (current approach).
The challenges with this approach are:
1. ____
2. ____
3. ____Ideally, they would prefer a solution where ____ (ideal solution).
An example of a completed summary looks like:
*In order to* know when to run an analysis,
*scientists* need to know the status of their request.*Today, they achieve this goal by* emailing people to ask for updates.
*The challenges with this approach are:*
1. Tedious back-and-forth between scientists to get requisite data
2. Hard to keep track of all the requests being fulfilled
3. Difficult to split by a request between assignees*Ideally*, they would prefer a solution where* they can see the status of all their requests, broken out by sample.
Getting feedback on ideas
The best way I have found to get input on a proposal is from a walkthrough where people think out loud. The key objective is to get people to share how the new solution will impact them.
Typically, I frame these sessions by talking through the key features in the mock, and then asking users to walk through their own task.
Let’s take the last time you did this task. Could you walk through how you would do that task using this design?
As they walk through the design, I look for moments where they pause or look confused. These become segues to ask for more details and tease apart their mental model. Helpful questions to guide the conversation are:
- How would you start? What are you looking for?
- What do you think should happen?
- It seems like you had some concerns about this. Could you tell me more?
- What could be better?
Keeping the conversation focused
Research sessions easily veer off-track. It’s challenging to communicate effectively with someone you just met, and harder still to get the insights you need. Some tactics I have found helpful to keep the conversation focused:
Be concrete
Have people walk you through the last time they ran into an issue, instead of asking about how they usually do a task. Responses to “How do you generally do this task?” are often indicative of how they think they do the task, rather than how they actually do it.
Asking people to walk through the last time they did the task brings out how they executed the task, as well as the people and systems they needed to interact with to get the job done. Instead of “How do you generally do this task?”, ask specifically for how they last did the task.
The last time you did this task, how did you do it? Is that representative of how you typically do it?
Get into the weeds by asking for documents such as the spreadsheets they used, or the files they created. Even better, do a screen-share or in-person visit where they can walk you through their flow.
Focus on the problem, thank people for their suggestions
As a researcher, your goal is to uncover problems affecting your users. The sign of a valuable problem is that it’s common across different teams. Solutions suggested by users are often unique to their circumstance and rarely get to the root of the problem. Don’t expect people to tell you exactly what they need — instead, observe and reflect with them to tease out the challenges.
Echo back
Paraphrase new insights and echo them back to people so that they can confirm or clarify. This ensures that you have the correct takeaway. As an added bonus, paraphrasing forces you to evaluate whether you truly understand their needs.
Cherish “no”, and moments of hesitation
When people hesitate or look confused, you have a rare moment to unpack their concerns. Instead of explaining the product further, ask them to describe the issues they see.
It seems like you had some concerns about this. Could you tell me more?
For questions on how the product behaves, focus on what’s prompting their questions. Gently reflect back by asking questions such as, “How do you think that would work? What would you like to happen?”
Don’t sell the product
Explain what the product does, but don’t try to convince people of its value. When people see you being defensive about the product, they stop giving honest feedback.
Don’t be shy about asking follow-up questions when you don’t understand
When you don’t understand something that a technical user is saying, it’s not always clear, whether it’s because you lack domain knowledge you’re expected to have, or just don’t know an organization’s internal jargon. A question that helps get more context without worrying about sounding ignorant is:
I know different teams approach this differently; how does your team do it?
Explain what you are trying to understand
When people don’t understand the purpose of a question, it’s easy for them to get frustrated. Ask for their patience when you get in trouble and explain your motivation. For example, I often explain that I’m trying to understand how common a problem is, or how frequently they will need to use the feature they requested.
What research questions and tactics have you found the most illuminating?
