How to save DKK 250,000 with 1.5 days of work
From post launch evaluations to remote, unmoderated tests as integrated part of design sprints. How Sif la Cour was a main driving force in Fullrate’s UX transformation.
Background
When Sif la Cour started as UX lead at Fullrate (previously part of Nuuday) in 2019, she joined a team of designers and front-end developers who had very little experience with research and user involvement. They made post-launch evaluations and could pull data from Google Analytics to feed their analysis, but they did not have access to their actual users as such. As a strong believer in testing early to build right from the start, which in time also saves resources, Sif was determined to change the team’s approach to include user involvement in the early stages of product development, while still working with prototypes. They needed to see it before they could believe it.
Challenge
The team did not yet realize the full value potential of user testing. Around the organization people thought that user involvement had to be qualitative and moderated, meaning that it would be expensive and time consuming to conduct user research.
Solution
One day, an opportunity occurred. A manager approached the team, asking them to create a feature for Fullrate’s website. The manager had seen similar features on competitors’ websites and deduced that they had to have it too. The feature was a survey with a few questions that would apply the user’s answers to suggest a recommended Fullrate service. Sif put herself in a consumer’s position, and asked herself what would be important from this point of view: What kind of internet connection do I need? What matters to me if I game, stream or use multiple devices? What am I (not) willing to do to get the information I need?
Desk research
Based on desk research, she formed a few hypotheses, where some went against developing the feature. An example of a user hypothesis could be: “They will just recommend whatever service they want to sell, and not what I need.”
Now, Sif had a chance to prove to the team that testing the concept on users would bring value to the project. She approached the marketing team who gave her access to a list of customer email addresses who had given permission to being contacted. She used Preely to set up two separate tests with two different prototypes. One based on the prototype of the proposed feature. The other one with screenshots of the existing web pages where users could find the same information by reading through it.
She decided to split the test into two groups and would send one test to half of the people on the list, and the other test to the rest of them. The tests were created and sent out within a day.
Results
The next day she had more than 85 responses for each test. The feedback contained some criticism towards the proposed feature. The users questioned the authenticity and purpose behind the feature. They lacked transparency in how they found the results, and the overall opinion was negative.
Sif could take the results and show them to her manager. It turned out that users by far preferred to find the service they need by reading through pages, rather than by answering questions to get a recommendation. Although the manager was surprised by the results, he decided to drop the project. Later the development team estimated that the project would have taken DKK 250,000 in development costs.
Our team saved DKK 250,000
“Our team saved DKK 250,000 with a day and a half of work, and could prove the value of remote, unmoderated user testing to the rest of the organization. This marked the start of a new era of user involvement at Fullrate where we used Preely for remote, unmoderated testing through a dedicated customer panel.”
– Sif la Cour
They went on and requested a budget for user involvement and started recruiting panelists for their own user panel. The word spread internally, and helped to create a common understanding that user testing is a good idea. This marked a shift where user testing became an integrated part of design sprints, where remote (unmoderated) tests are conducted simultaneously with moderated tests.
Preely Meetup: Hybrid user testing & research processes
Try Preely and see for yourself
Stay connected to your users no matter how far away they are.
- 14-day trial
- No credit card required
- Cancel anytime.