Why is usability testing important? It helps to understand how people interact with your product. What does “usable” mean? Steve Krug, the author of “Don’t Make Me Think,” summed it up: 

“It means making sure that something works well: that a person of average ability and experience can use the thing—whether it’s a website, a toaster, or a revolving door — for its intended purpose without getting hopelessly frustrated.” 

This article will help you understand the purpose of usability tests when to do usability testing, the types of usability testing, and why it’s beneficial to do usability testing during the early design phase.

What is usability testing?

Our usability testing definition is the following: it helps to validate product ease-of-use and its functionality.

Usability testing is one of the user experience research methods, usually a qualitative one, and it is the core of UX research. 

A user should interact with a digital product without frustration and easily accomplish set goals. 

There is no strictly defined usability testing process. You’ll have to fine-tune and customize it to your needs. However, the most critical first step is to learn the basics of usability testing, start doing UX testing and make it an essential part of your UX process. 

What is the purpose of usability testing?

What is usability testing in web design? It not only serves as a method for discovering usability issues but also indicates what works. 

You have to pay close attention because intuitive, self-explanatory steps and attributions might not get noticed by participants during a usability test.

Typical usability testing examples: 

• Identify the product’s main pain points. 
• Check if users understand the navigation.
• Observe how easily and quickly people accomplish tasks.
• Validate the value proposition of an app or website. Do your potential customers understand it?
• Test competitors’ solutions. (You can test competitors’ solutions with the target group to gain insights on what could be changed or improved in your product.)

As a top ui/ux design agency, we’ve helped over 250 companies across various industries do user experience research and make data-driven business decisions. If you, too, need help running usability testing, contact us. We offer various usability testing services for websites, mobile, and web platforms and we are ready to help you with any challenges.

When you shouldn’t do usability testing 

Usability testing won’t help you gain insights in case you would like to:

• Test your product design associations and visuals.  
• Get quantitative data about product usage. 
• Identify preferences between different versions of visuals or copy. 
• Validate desirability.
• Estimate market demand and value.

👉 Pro tip:  Does your product meet users’ needs? Do you target the right market? If you want to find answers to these questions, don’t expect to get them from usability testing. Why not? Because it’s not a proper method to find answers to these questions. Ideally, the product team has already passed that. If not, take a step back and conduct market research. 

When to do usability testing

Conduct usability testing during different phases of the product design process. 

• When forming a concept, you can test low-effort paper prototypes or competitors’ sites. 
• At the beginning of a project – test the current design solution that you want to improve.
• UX design or website redesign phase.
• During the development phase.

📌 Example:   82% of your users drop off on the checkout page of a food supplement webshop. You know the numbers, but you don’t know the whys. Running a few usability tests focused on the checkout page can reveal the underlying reasons. It might turn out that the page asks for credit card information too early.

Aim to have usability testing as a recurring activity in your product design process. Practice it during every phase of your product development process. It can help you improve your product’s usability. So, don’t forget the lean UX cycle: think, make, check.

How many usability tests you should do

When it comes to one-on-one usability tests, you might wonder: what makes a representative sample size? 

According to Nielsen Norman Group, testing five people in one cycle reveals 85% of all usability issues. After a round, prioritize the problems and work on them, iterate, and test again.

Another popular framework, RITE (Rapid Iterative Testing and Evaluation), promotes faster processes and solutions. It favors running one usability-test and correcting the discovered problems immediately afterward.

Applying this framework allows you to test faster and more frequently. The image below illustrates the number of problems arising from the usability-tests over time. The vertical dashed lines signify the iterations. Six rapid iterations allow creating an error-free prototype.

Types of usability testing

First, adjust the scale, rigidity, and frequency of your projects’ usability studies. our approach and methodology might change when working with an agency on a short-term project or as part of a product team in the longer run.

Always consider multiple factors, and be mindful of the resources before choosing a certain approach or method. In-person or remote? Monitored or unmonitored? 

One approach does not work better than the other. You learn the most when you go where your target audience hangs out – a mall, an airport, or a new café – and do quick usability testing.

Some products require a large-scale controlled study. Some even build their own usability labs with two-way mirrors and other fancy equipment. If the sheer thought of investing in the construction of a usability lab alone discourages you, aim for a budget lab.

It might seem tedious at first: all the recruitment, preparation, set-up, analysis, and report for one short test, but it will save effort and money in the long run. Just go for it! Even if you can only do informal, run-in guerrilla tests, it is better than nothing. Your effort will reap actionable insights from potential users.

Usability testing will put you on the right track of giving users the functionality they want and can use. It will also help you avoid costly mid- and post-development changes.
With live products, testing out ideas lets you know what function or interaction is worth your effort and what to abandon, as it no longer provides any value.

At UX studio, as a top ui/ux design agency, our research professionals did usability testing at football venues, tried to catch participants at the airports and on crowded streets, conducted lots of interviews with participants from different time zones to help our partners improve their product’s performance.

If you are looking for reliable website usability testing services, we are ready to discuss your challenges and goals.

Four steps of the usability testing process

Step 1: Plan your study

To properly plan your usability study, first, clarify: 

The product to test: Are you onboarding a mobile app, a website filtering system, or a kiosk interface prototype?

The platform: When testing a mobile app, determine if the OS matters and if it might bias the study. If so, let participants choose the OS. 

Research objectives:  If users understand passwordless registration and login. If they can easily navigate the product detail page. Turn your high-level objectives into concrete usability testing questions. Concentrate on a few tasks, pages, areas, and assumptions to test at once.

Target audience:

• Who are they?
• How many participants do you need?
• How are you going to reach them?
• What will you give as an inventive? (money, gift card, discount, tickets, lifetime access to your product, etc.)
• How do you make sure the participants are your target audience?

If you want HR representatives to test a remote hiring platform, check their experience, current position, and qualifications and make a screener with specific questions. 

Remote or in-person: Base your decision on the type of product, project scope, objectives, and target audience. Choose the more feasible one (technically and financially).

📌 Example:  If you target people with high-demand jobs and salaries, either go to them or schedule a remote test. A neurosurgeon won’t have time to come to you. Adapt to their lifestyle. 

Write a usability study plan:

• Goals
• Participants
• Setting (where, when, device type)
• Roles (moderator, observer, note-taker)
• Tasks

👉 Pro tip:  Aim to have no more than two people with you during a usability test. Having too many in the room can lead to frustration and intimidation. 

Write a usability testing script:  

UX professionals always test products, but never users. You have to emphasize users do not cause failures, and they cannot do anything wrong.

Inform users about the purpose of the usability test and how their feedback can help create powerful digital products.

Adjust the usability test script according to participants’ prior knowledge. Some explanation is required on what a prototype is if they have never heard of the concept. 

Come up with a realistic scenario. You want to hear users’ personal reactions to the product. Help them engage in the testing.

Come up with the exact usability testing questions and tasks you want your participants to perform. They should find the scenarios and tasks as realistic as possible.

Include intro questions and mini-interviews. It builds rapport between you (the moderator) and the test participant. Moreover, it provides an opportunity to validate user personas. Having enough data on real users (your participants), you can revisit the personas and make them more accurate.

📌  Example:  “As a homeowner with a garden, you want to install an automated irrigation system, so you don’t have to schedule your daily routine around watering. You google the term with your city name and stumble upon this website.”

Step 2: User test participants

Usability testing recruitment can be challenging. It can make you feel like a TV commercial trying to reach out and “sell” the usability testing to as many people as possible.

What makes recruitment for usability testing challenging

First, many people have never heard of UX research activity, and the whole thing might sound suspicious and cumbersome. Find what works best as a recruitment script – on the phone or in email – to evoke trust rather than suspicion or confusion.

Finding the right person

When deciding who to target, keep a clear persona or a particular group of people in mind. The main rule of thumb would be to find someone who has not tried your application yet. 

Your developers will not give you the required feedback. You’ll need someone with a certain level of expertise for a more specialized product. 

To make sure your potential user test participant fits the criteria, make a screener. You can have an online form or pose the screening questions directly over the phone. The work is almost done. However, you’ll still have to keep track of everyone, handle your inbox, and schedule test sessions.

👉 Pro tip:  Always schedule extra user interviews as 10-20% of scheduled tests are canceled by participants. 

If you have no time to do usability testing in-house, you can ask a dedicated user experience design agency to conduct usability testing for you. At UX studio, we help multiple startups and established brands do user experience research and audit web and mobile platforms. Contact us to book a free consultation, and let’s discuss how we can help you.

Step 3: Lead usability tests

Pilot test

Make sure to test the usability test! Take it for a dry run or do a pilot. The pilot participant doesn’t have to come from the target audience. Run a session with a colleague or ask your friends for help. Try out your plan from A to Z. You will find some glitches in the prototype, vagueness in the task instructions, or your 30-minute usability test will end up taking an hour.

Here is a checklist for you that will help you run a successful usability test and get as many valuable insights from your participants as possible.

• Have a quiet room booked for the session.
• Remind the user about the usability-test.
• Print the usability testing scripts.
• Set up the equipment and the recording. Double-check if everything works properly.
• Welcome the participant.
• Introduce yourself.
• Summarize the procedure (“We’re going to test a food delivery application specializing in Asian food”).
• Emphasize that you are testing the app, not the participant.
• Implement the “think out loud” protocol.
• Remind the participant that you want to know how they would use the product elsewhere without help.
• Present the consent form and NDA if needed.
• Share some practical information and allow the participant to ask questions: “The test is going to take one hour.”, “If you would like to stop the test, please let me know.”, “Do you have any questions about the test? Can we start?”

During the test

Scenario: Make the scenario-specific and realistic so your participants could perform it elsewhere. Help thoroughly paint the picture for participants so they can easily imagine the scenario.

📌 Example: “Imagine you are a marketing manager at a company. Your task is to choose a newsletter service provider.”

Warm-up questions: 

• Needs: Where does the problem arise? What are they looking for?
• Experience: How do they use similar products?
• Intentions: Why do they use those products?
• Knowledge: What do they know about the topic?

Tasks: At UX studio, we always give participants tasks to focus their attention on some area or aspects of a user interface, whether it is an overall impression of the landing page or the process of buying shoes on an e-commerce platform.

There are three types of tasks:

• Broadly interpreted: These instructions show how users start using the application and figure out what it does. “Discover the ways you could use this application to enhance your performance.” “Explore this landing page and see what this company offers.”
• Tasks related to a particular goal: These tasks aim to test specific processes. “Buy a TV!” “Sign up!”
• Tasks related to specific interface elements: “Where can you subscribe to a newsletter here?” “How can you put an ebook in the basket on this screen?”

Handy psychology tips

• Be non-judgemental.
• Be non-directive (“Where do you find the chat function?” Instead: “What would you do if you need developers’ help?”)
• Don’t show reactions indicating whether participants do the “right thing.” Remain neutral.
• Make your users feel relaxed. 
• Probe participants to say everything on their minds. Specify you don’t want to hear ALL thoughts, just those connected to the product and usability experience.
• Pick up non-verbal cues (frowning, fidgeting, biting lips, etc.)
• Don’t be afraid to stand the silence.
• Paraphrase (“So, you are saying you expected this button to take you to the checkout page, right?”)
• Ask back (“Will clicking this button pause or exit the game?” – “What do you think will it do?”)

Steve Krug’s “Things a Therapist Would Say” highlights many situations that often occur in usability testing, offering possible verbal reactions from the moderator’s side. In our experience, it regularly happens, so it merits preparing standard questions and responses to these situations.

After the test

Feeling like a test subject (in place of the product) can frustrate participants. It might happen even if you try your best to emphasize they are not the ones being tested. Participants might remain tense until the very end of a session. 

To help your participants relax, ask them how they felt and perceived their experience. A bit of post-test chit-chat indicates the “serious” part of the test has finished, and they can share their honest opinion. It helps open up some people, and they might give a lot of feedback during the post-test questions.

Post-test question examples

• Did I forget to ask you about anything? / Would you like to mention anything?
• On a scale from 1 to 10, how easy was it to use? (You might break it down according to different tasks)
• What three main things did you like the most about the app or this experience?
• What are the three things that confused you or you didn’t like?
• Did it lack anything?

Don’t forget to thank participants afterward and assure them their feedback proved valuable.

What if things go wrong?

Nothing. Glitches happen. They happen to everyone. It happens because researchers work with different prototypes, recording devices, and software. Speak honestly. Tell your participants you need to fix something or deal with an issue, but not at the expense of their time. Always stick to the pre-agreed time. If any glitch causes a major setback, politely ask participants if they have more time.

Don’t panic!

Brace yourself. These typical examples show what can go wrong in a user test and how to set them right. 

The recording is cut off or not working. Continue the test even if the recording is off. Take notes later.

The prototype does not work properly. Steer it in the right direction by coming up with different tasks and guide users back to a familiar place. If the prototype doesn’t work, use the time on your hands to conduct an interview. You might discover something important.

The user looks pretty frustrated and does not follow the “think out loud” protocol. Ask probing questions. Emphasize once again they cannot do anything wrong and help them imagine themselves naturally interacting with the product.

Comfort the user

Never let the participants feel frustrated. Treat participants like MVPs (most valuable person) even if they may prove difficult. Thank them. Make them feel comfortable. Emphasize you are not testing them, but a product.

Step 4: Analyze & Report

A successful testing session doesn’t mark the finish line. The following part matters as much. You need to make sense of it, group the feedback, prioritize it, and report back to your team or your client in a straightforward manner. Analysis and reporting usually go hand-in-hand as you typically don’t have time to rewatch everything twice. 

What type of usability testing report do you need?

Lots of usability research projects don’t require extensive reports. After a round of tests, you can share the test notes and provide a more detailed report when a bigger cycle ends. 

The usability testing might require five or six iterations and three to four design sprints. It always depends on your product roadmap, product strategy, and how much research you set out to do. 

You conduct usability testing to iterate on your product and improve its usability. If you don’t ask how users feel, then the observations might be highly subjective, and the interpretations will be 100% hypothetical. Getting actionable insights from the test comes foremost.

Research system or simple test reports?

For more extensive projects with weekly usability tests, it might merit building up a research system to synthesize your endless number of observations and insights into a searchable, consistent system. 

👉 Pro tip: For a leaner approach, try to include the essentials only, such as task, observation, location (page, step), issue severity.

Prioritize usability issues

To avoid ending up with an enormous mass of findings, make sure to prioritize the usability issues.

If possible, set up a meeting after a round of tests, so you can help stakeholders understand the results and answer emerging questions. Provide snippets of test recordings, include screenshots and quotes from the participants. It will shift the nature of the usability tests from very abstract and mysterious research activity to a tangible method contributing to informed product decisions. It will also help organize the vast amount of feedback, information, and insights from usability testing sessions.

Develop your own standards and a system. Distinguishing between usability issues, positive findings, feature requests, and general observations makes a good starting point. There’s no “one size fits all” approach to reporting. It depends on your client or team, how you work together, and the type of usability tests you’re running.

Key takeaways

No one has a perfect usability testing process, and we all make mistakes. Remember, every usability test has a point and meaning; you always learn something that can help you improve the product.

Do usability tests often and include other stakeholders. It will help them see the value of your method and make their mindset more user-centered. 

Usability testing can serve as a firm basis for the research you do on a digital product. It works best complemented with other UX research methods and quantitative data. 

Searching for the right UX agency?

UX studio has successfully worked with over 250 companies worldwide. 

Is there anything we can do for you at this moment? Get in touch with us, and let’s discuss your current challenges. 

Our experts would be happy to assist with the UX strategy, product and user research, or UX/UI design.