Remote unmoderated usability testing (sometimes referred to as URUT) is the process of conducting user tests without real-time communication between the test participant and the researcher and in separate locations. This type of testing allows you to do a significant amount of tests in a short time and with few resources. Therefore, it is an easy and cheap way to put your digital products in front of real users before your team starts writing expensive code.

In its nature of being unmoderated, this type of testing requires fewer resources than moderated testing. Instead, it sets high-demands for the preparation phase of the test, as there is no moderator to guide the test participant throughout the session.

By being remote means merely that the testing doesn’t take place in a specific geographic location, but is typically done online through different user testing tools.

In this article we will aim to include as much as possible to get you entirely started with URUT on your own:

  • Definition
  • Types of data
  • What to test
  • Benefits
  • Step by step guide

What is unmoderated and remote UX testing exactly?

When doing remote user testing, there are two methods available for the UX researcher: the unmoderated or moderated version.

In a moderated user testing session the researcher is communicating directly with the test participant and is able to see what the test person is doing in real time. Typically, the moderator dictates the tasks and observers the test participant complete them. On the other side of the table, it is the test participant job to reflect and think-out-loud while completing the tasks.

In an unmoderated user testing setting, there are no real-time interactions between the test person and the researcher. Instead, the tasks are created before the test and usually completed by the test participant through an online testing tool.

To sum up, we define URUT as:

“Remote unmoderated usability testing is the process of conducting user tests without real-time communication between the test participant and the researcher.”

Types of feedback and data

There is a significant difference in the kind of data you can get from unmoderated user testing, which is related to what you can test with this approach. Before you settle for URUT, you need to consider what it is you want to test and what kind of data you’ll need.

Many tools for remote testing includes the option to record the user (audibly and visually) while completing the tasks, which will give you loads of qualitative data. Other tools will focus solely on quantifiable metrics such as heat- and click maps, task duration, task completion rate, and other screen statistics.

There are pros and cons to all types of unmoderated data. Therefore you should decide which kind of data is right for you considering time and resources available. Naturally, it takes longer to analyze qualitative data such as video recordings, which means you’ll probably have a smaller sample size, but more in depth-knowledge a specific usability subject. On the other hand, focusing on quantifiable data will give you access to a more substantial amount of data. Additionally, many online tools will handle data processing and -analysis, and present your results in a way that makes decision-making fast and straightforward.

The way in which data is recorded and presented is dependent on your user testing tool of choice. Choose carefully, the right tool might turn out to be a great time saver.

When should you choose the unmoderated approach?

All the time! Unmoderated tests are excellent for iterative processes as they allow for quick and effortless testing of your designs, which is why it should be a process you continuously return to. However, there are specific times when this approach is especially relevant:

  • If you can check any of these boxes, this type of testing is probably for you:
  • You need quick results
  • You need to do repetitive testing for an iterative process
  • You need a large sample
  • Your budget is low
  • You hate data processing
  • You need to test in a different geographic location

The question of prototype fidelity

The pressing issue is often the level of fidelity, and when you can start testing your ideas.

Level of prototype fidelity is dependent on three things content, functionality, and visuals. With most online tools you can quickly make both interactive and static prototypes.

Level of prototype fidelity


In this article, we would like to argue that the level of fidelity should be related to what you are testing. Are you testing layouts, colors, and fonts you’ll probably need good visuals, are you testing general UX concepts or validating ideas you’ll need a high level of functionality, finally, are you testing messages, you’ll probably need to include almost complete content.

Learn more: Testing low fidelity prototypes: Why you should do it!

What are the benefits of unmoderated usability testing?

  • Saving time and money!
    Conducting unmoderated usability testing is significantly cheaper and less time consuming than moderated - simply because it doesn’t require a researcher to be present. Also, tests can be conducted at any time which means there is no need for setting up a specific appointment. If you select a test tool which not only collects but also analyses your data you will save additional time in the post-testing phase.

  • Test earlier and find problems earlier.
    By being cheaper and fast, you can test more, and you can test earlier. This saves a lot of time and energy since it’s a hundred times easier changing your designs before the actual development phase.

  • Fast and actionable results.
    By getting your test results faster, you can start acting on your knowledge sooner. This means less data processing and more time to create remarkable usability experiences.

  • A large and diverse sample.
    The simple fact that you don’t have to be present for the test eliminates the challenges of time and geography. Unmoderated tests are often shared through an URL, which gives you endless possibilities regarding sample size and audience diversity. Thank you internet.

  • Recruitment is easy (easier).
    Recruiting testers for remote tests are easier than recruiting for non-remote tests. Your test participants can complete your test from the comforts of their home, and they can do it when they feel like it. With test tools such as Preely, you can recruit testers via email and URL, and manage your own test panel to ensure you never run out of testers.

  • The right environment / and across the globe.
    Your test participants can do the test from everywhere - their home, office, in the supermarket, etc. - this means it is easy to simulate a real environment and context in which the actual product is intended to be used. It also means you easily can reach user segments around the world.

  • It’s excellent for iterative processes.
    Iterative processes require frequent testing and feedback. With unmoderated, you can do more testing for fewer resources, which makes it great for cyclic approaches to design and product development.

Step by step guide


Step 1: Preparing for the test

When doing remote unmoderated usability testing, most of the work lies in the preparation phase. As you are not there to assist the test participant while completing the test, it is essential that you carefully consider and plan every detail of the test to avoid confusion and mistakes during the actual testing.

Here is what you need:

  • A testable prototype (consider what level of fidelity you need to get the right answers)

  • Test participants (see step 2)

  • The right testing tool (You could try Preely)

  • Questions and tasks for the user (Consider what you need to test)

Finally, always remember to check, try, and test everything before you send it out to actual users.

Step 2: Share your test with the right audience

The typical user testing tool will give you a ton of opportunities to share with- and recruit users for your test. Some of the most effective ways are:

  • URL sharing and recruitment
    It all starts with a unique URL that you can share or implement everywhere to recruit testers. A great place to start is in your website, in a pop-up, or even a sponsored ad on Facebook, where you ask users to contribute either for a fee or for free (it’s often surprisingly easy to get existing customer to participate if you ask them nicely).

Learn more about recruitment: How to recruit testers remotely - like a pro

  • Email Sharing. Share your test directly with your dedicated email list. An easy way to share your test if you already have pre-built a panel of testers.

  • Recruit through agencies. Expensive, but a very convenient and effective way of recruiting the right users.

  • Build your own test panel. Having a panel of testers who are always ready to test is great, and not that hard to get. A great place to start building is in your existing group of customers. Often they are happy to contribute and have an influence on future digital products.

Step 3: Analyse and act on your results

As stated earlier, the types of data and analytics you will get from remote and unmoderated user testing varies from your approach. Compared to analyzing moderated tests it fast and easy to get actionable feedback from moderated tests. Most online tools will actually provide with a full report and analytics in real time as answers are coming in.

That’s it. Now implement this into your development process, and consider to do frequent user tests, that can support your iterative methods. It will not only help you discover UX problems earlier, but it will also save you money while making your products more user-driven.

If you need three key takeaways from this article, I believe they should be:

  1. Test testing as early and often as possible
  2. Level of prototype fidelity for user testing is contextual
  3. Getting actionable feedback does not have to be time consuming or expensive



Previous post Next post

Get actionable user feedback today!

Faster. Better. Cheaper.
Free forever

GET STARTED



TestLab is now Preely!
As we strengthen and expand our platform, we have changed our name to Preely. Testing remains at the heart of what we do, and we're happy to continue providing our service in 2018.
Happy testing!
Team Preely
👍