Remote Usability Testing 101: Definition, Process, and Tools

Usability testing is an essential part of the product design process. However, face-to-face testing isn’t always practical, so UX teams turn to remote usability testing as an alternative.

In this 101 guide, we give you an overview of:

• What remote usability testing is
• Its benefits and challenges
• Main types
• How to conduct a remote usability test, and
• Best tools for the job

Let’s dive in, shall we?

TL;DR

• Remote usability testing is a UX research method that doesn’t require meeting the participants face-to-face. Instead, the sessions take place online with the aid of specialist tools.
• Such usability tests are quicker to arrange and run at scale and considerably less expensive than tests carried out in person, even if no lab hire is involved.
• Most importantly, remote testing enables companies to recruit representative participant samples even when their user populations are geographically dispersed.
• However, external factors, like technical issues or background distractions, can affect the reliability of the results. Moreover, it’s more difficult for researchers to establish rapport with participants and notice non-verbal clues online.
• Moderated usability testing is when the facilitator leads the participants through the tasks and interviews them as they work through them.
• Unmoderated usability testing, on the other hand, involves no facilitation – users complete tasks independently at their own pace.
• Start remote usability testing by defining your goal, like “improve the adoption of feature X by 11%.” Based on the goal, choose the test type and prepare relevant tasks and questions.
• Next, select test subjects. You can use product analytics to pick them up from your current users or recruit them from the tester pool offered by your testing tool.
• Before running the live test with real participants, pilot it with your colleagues to ensure there are no mistakes and the instructions are clear. For moderated tests, make sure your equipment and software are reliable.
• During the test, create a relaxing atmosphere. Don’t interfere unless testers get stuck. Even then, try to understand why they’re experiencing difficulties first before helping them.
• Maze and UX Army are both powerful remote usability testing tools that cover the majority of testing scenarios and needs.
• Userpilot is a digital adoption platform that you can use to select test participants, analyze their product interactions in heatmaps, and collect feedback. Book the demo to find out more!

What is remote usability testing?

Usability testing is a UX research method used to evaluate how intuitive and easy to use the product is.

Unlike traditional usability testing, remote usability testing doesn’t involve meeting test participants face-to-face. So no lab sessions, face-to-face interviews, or asking people to take your product for a spin in public places.

Instead, you test your product design using a video conferencing solution or a dedicated usability testing tool that allows you to record the screen and voice of the participant as they engage with the product in their natural environment.

Benefits of remote usability testing

The fact that test subjects engage with the product in their natural contexts is one of the main advantages of remote usability testing as it makes it more realistic.

However, there are more.

Easier to execute and scale

Remote usability tests are easier to run at scale.

That’s because it’s quicker to recruit your participants remotely and even run multiple tests at the same time. Thanks to that, you can collect more data, making the testing result more reliable.

Such large-scale usability testing would be very difficult to pull off face-to-face as it would require lots of people meeting specific criteria to be physically present at a particular time and place.

Larger test participants pool

The fact that you’re not limited by physical constraints means you get access to a larger participant pool.

This is particularly important for SaaS products because their user populations are often spread all over the globe.

Consequently, recruiting a representative user sample for your experiments in your location wouldn’t be possible. Neither would you be able to test localized versions of your product with their target users.

Cost-effective compared to other user research methods

Finally, remote usability testing is more cost-effective than regular testing.

The cost of user research software is considerably lower than hiring a lab. And even if they were the same, you’d still have to cover the travel expenses and sustenance and compensate participants for more of their time.

Challenges of the remote testing process

While remote usability testing solves several problems, it also comes with multiple challenges.

For starters, you have limited control over the testing environment. This means, there could be other factors, like background distractions or technical issues, affecting user interactions with the product and skewing the results.

The main challenge, however, is the lack of presence of the physical observer.

It makes it more difficult to establish rapport with participants and researchers can miss subtle body language and non-verbal clues that would provide additional insights. And in unmoderated tests, there’s nobody who can clarify the test to users if they don’t understand it the first time.

Two types of remote usability testing

There are main types of usability tests, including remote ones: moderated and unmoderated.

Remote moderated usability testing

In moderated usability tests, the facilitator works directly with the participants, guides them through the research task, and helps them overcome challenges that arise.

Its main advantage is that it allows the researcher to establish a good rapport with the subject and follow up on ideas they bring up.

However, objective moderation can be challenging, especially if the researcher is invested in the product design. They can unintentionally lead the participants to biased questions.

Remote unmoderated usability testing

In unmoderated usability testing, there’s no facilitator present. It’s just the participant and the product.

The participant works their way through the tasks and questions at their own pace, which makes it less resource-intensive.

However, if users run into problems, misunderstand the task, or simply don’t take it seriously enough, there’s no one to step in and help them through. This means they require more thorough preparation.

How to conduct a remote usability test?

Whether you choose moderated or unmoderated usability testing, the testing process is the same. Here’s how to run a remote usability test, step by step.

Define remote user testing objectives and test parameters

The first question to answer is why you’re running usability testing.

Clarifying your goals is essential because your choice of participants and testing methods depends on them.

For example, you may want to improve the adoption of a particular feature. To achieve this, you test how easy it is for users to find and use the feature to complete a specific task.

It’s key to make your goal specific and measurable. In our example, this could “improve the adoption of feature X by 11%.”

Based on your goal, you determine:

• Whether to run a moderated or unmoderated test
• What questions to ask and in what order, and
• What context information to share with the participants before the test.

Identify and recruit participants for the remote research

Based on your goals, choose the participants for the usability testing session.

For example, if you want to improve the adoption of a feature, choose users with relevant use cases.

If you have an existing product with an active user base, you may want to recruit participants from among them. You can do it by targeting a specific user segment with an in-app message inviting them to take part.

If you’re still working on your MVP or would like to test the design with users who are unfamiliar with your product, you can recruit participants through your testing tool. Solutions like Hotjar or UserTesting all give you access to a pool of testers from different backgrounds.

Launch a pilot test and iterate

Before you run the test, first pilot it.

This is basically to ensure that the test is well-designed and has no mistakes because you won’t be able to rectify them once it goes live.

The easiest way to do it is by sending the test to your colleagues from other teams and letting them run through it. It’s good practice to do it in batches so that you can iterate on the feedback and test it out again with another group.

For moderated tests, you can arrange a session with one or two users to test out all the equipment, testing software, and tasks, and only then schedule the test with the rest of the cohort.

Execute the remote usability tests and analyze the results

After the pilot, all that is left is to run the usability test and analyze the results.

If it’s a moderated test, follow the script to ensure consistency. However, be prepared to adapt the session based on unexpected issues or participant behavior.

Try to create a friendly atmosphere to put the participants at ease so that their behavior is as natural as possible. Encourage users to be direct and honest in their feedback.

As users are performing a task, try not to interfere unless they come across an obstacle they cannot overcome. However, even then, avoid telling them what to do straight away. Instead, ask them probing questions, like ‘What do you think you should do?’ or ‘Why is this challenging?’

Record the results in a spreadsheet or table. Focus on whether users completed a task, how long it took them, and the comments and feedback they give.

If you’re conducting unmoderated usability tests, your testing tool will do all the recording for you. It will also analyze and summarize the results in a dashboard for easy sharing with other team members and stakeholders.

The best remote usability testing tools to explore

To carry out remote usability testing, you need the right tools.

For a simple moderated test, video conferencing tools with screen-sharing will do, but you will still need a way to select and recruit users. Unmoderated usability testing requires specialist testing software.

Let’s have a look at a few tools that you may want to consider.

Userpilot

Userpilot is not a testing tool but a product adoption platform. Still, it offers features that you can leverage in your usability tests.

For starters, you can use its analytics and segmentation tools to select the right participants for your tests. This could be your power users or the opposite – unsuccessful users who have been failing to complete a specific task or NPS detractors.

Once you single out prospective testers, you can reach out to them inside the product by triggering an in-app message.

Moreover, Userpilot offers heatmap functionality so you can analyze user interactions with different features and UI elements.

Finally, Userpilot enables teams to run in-app surveys to collect user feedback.

Maze

Maze is a dedicated platform for usability testing and user research known for its user-centric design.

It provides several tools and features to help designers and researchers create and analyze remote usability tests, both moderated and unmoderated.

The main ones include:

• Drag-and-drop test editor allowing you to create tests out of blocks
• Live website testing
• In-product prompts (surveys and tasks embedded in the product UI for ongoing testing)
• Session recordings
• Closed and open card sorting – to improve your information architecture
• Tree testing – to optimize your menu structure.
• Interview studies
• Participant database for easy recruitment

UXArmy

UX Army is a cloud-based remote user research platform that allows teams to streamline their usability testing processes.

Just like Maze, it offers a range of features that allow you to conduct most industry-standard tests.

With the tool, you can create tests and recruit testers from its panel. As they’re working through the tasks, UX Army records them so you can rewatch the sessions later or create short clips to illustrate specific points.

It also offers interactive heatmaps, navigation paths, and mouse paths for granular analysis as well as a dashboard with usability metrics, including the System Usability Scale (SUS).

On top of that, you can use UX Army for moderated live interviews, card sorting, and tree testing.

Conclusion

Remote usability testing is a perfect alternative for SaaS products with geographically dispersed and diverse user populations. It’s also more cost-effective, quicker, and logistically easier to arrange.

If you want to see how Userpilot can help you run remote usability testing sessions, book the demo!