ChatGPT and Preely Insights

Preely uses ChatGPT to generate summaries, ratings, and follow-up questions in user test analyses.

Benefits

Efficiency, scalability, speed!
Manual analysis of open-ended questions can be time-consuming and feel tedious and repetitive. ChatGPT can quickly process many responses and generate summaries, ratings, and follow-up questions, saving valuable time for the researcher.

Analytics

Together with Session you will find Insights. Click and see summaries for open-ended questions, Rating and follow-up questions. Preely has tested various prompts and refined them in order to deliver useful and actionable insights for the researcher to easily apply in their further product development.

Limitations

Preely’s Insights uses OpenAI’s servers which means that should the researcher choose to use the function, the data will not be kept in a closed environment. However, it is solely the responses to the given question that will be analyzed and condensed to Insights. No additional data on test participants will be shared with OpenAI in order to maintain the privacy of participants, but it is ultimately the researcher’s responsibility to make sure responses do not include personal data that could compromise the privacy of a participant. It is not until the button ‘Insights’ is clicked that the data is shared with OpenAI’s servers, which means that if a researcher does not click the button ‘Insights’, all data will be kept within Preely’s closed environment.

Bias

As with any qualitative analysis done by researchers, there is a risk of bias and different interpretations between different people. This goes for Insights as well. However, research indicates that with properly refined prompts, chatGPT can offer nearly as consistent and reliable results as experienced researchers (Schiavone, 2023).

Note!

However, it’s important to note that ChatGPT’s analysis should not replace human judgment entirely. It can be a powerful tool for enhancing analysis. However, it’s still essential for researchers to review the generated summaries, ratings, and follow-up questions to ensure accuracy, context, and relevance.

References

Can ChatGPT Replace UX Researchers? An Empirical Analysis of Comment Classifications, Will Schiavone, PhD • Christopher Roberts, PhD • David Du, MS • Jeff Sauro, PhD • Jim Lewis, PhD. May 23, 2023