During Covid-19 the importance of being able to conduct remote user tests became evident. Add to this that UX researchers and -designers often are a scarce resource, who would benefit from being able to conduct user tests unmoderated to save time.

Not to mention, that from a sustainability point of view, remote user testing is a great way to test with a diverse user group without traveling, having time zone issues, etc. The participant can be located anywhere since the entire test is conducted online. Hence, the prototype needs to be digital and preferably also interactive. Remote user tests are normally task-based, where the participant receives either in-person or written instructions, depending on it being moderated or unmoderated. Remote user tests are therefore quite similar to in-person testing, but without the facilitator being psychical together with the participant.

Moderated, remote user testing

In moderated, remote user testing the participant and the facilitator are present in the same virtual space at the same time. The only difference between this and an in-person user test is their physical location. The facilitator takes the traditional role, watching the test and communicating with the participant as within an in-person test. The communication is normally direct via an online meeting space, however, it can also be via phone, chat, etc. A drawback with moderated, remote testing is knowing when to ask questions and it can be difficult to find the balance between letting users know you are listening and interrupting them.

Unmoderated, remote user testing

In unmoderated, remote user testing the participants are on their own without a facilitator and they complete the test when the time suits them. A drawback is that it might be quite tough to get the participants to think aloud since this is something people often need encouragement to do. However, there are a lot of benefits when it comes to keeping the test consistent, since a facilitator will not interfere during the test, and you can still collect behavioral data, it all depends on the tool being used. Preely for instance, collects and tracks all clicks, scrolls, paths, etc. Leaving you with a lot of behavioral information or performance metrics of where the participant got confused (e.g. used a lot of time, clicked many times, returned to again and again, etc.). Furthermore, you have the opportunity of asking predefined follow-up questions when you find it necessary, giving you the option of collecting more qualitative data as well.

When not to do user tests remote

It’s important to emphasize that sometimes remote testing is not the right choice. Remote testing should be seen as an addition to more traditional user research and -testing, and as a great way to collect more user feedback and insights.

Before choosing to test remote or not, you should consider where you are in the development process, which information you need, and you should also consider which stakeholders you need to either convince or who need the data to make decisions.

In the early stages of a design process and when needing to identify complex design issues, it’s often necessary to have in-person contact both to understand the area, but also to provide directions and ask behavior-driven follow-up questions to fully understand the participant’s behavior. And remember that some hypothesis and research questions are better answered with another testing approach e.g. an AB test, interview, or Contextual Inquiry.

Furthermore, you should consider your target segment – are they willing to use the tools needed? Will they understand the scenario if not explained verbally? Would they participate without having the security of a facilitator? Are they at all able to conduct such a test (e.g. children)?

Benefits of remote, unmoderated testing

Remote, unmoderated user testing is an easy and fast way to set up a test and get fast and actionable results at a low cost. With this approach, you do not waste time moderating. This leaves time to do all the other UX work you need to do. Imagine setting up a test at the end of a Sprint, use the next Sprint for something else on the backlog, and then come back and have data and feedback, you can make informed decisions from. Last, but not least, by testing this way you have a great potential of reaching participants globally, without spending time and money on traveling.