We often talk about formative and summative evaluation or tests, when talking about user and usability tests. In this post we’ll try to explain the difference between the two and when to use which one.

Let’s start by defining what we mean when talking about formative and summative evaluation. The terms originate from education theory.

Formative evaluation

Formative evaluation in education theory is the tests being performed e.g. during a semester to evaluate whether the content is understood or needs to be revisited. Often this is done ongoing via e.g. a quiz to assess the students’ understanding of the study topics. Then the teacher uses the results to shape the lectures to support the students in the best possible manner.

Formative testing in user testing is performed to improve the design. Here you assess what works and what does not work in the solutions, and why it does not work. This helps you identify what needs to be changed or improved. This is the type of evaluation or test you’ll often see at the beginning of a design process due to its exploratory nature. It is often done iterative and if development is done incrementally, it can be a part of the iterative development process.

Summative evaluation

Summative evaluation in education theory is used to determine if the content taught was well learned and assess the student’s mastery of the material. This could for instance be by final exams, standardized tests, etc. This means that the information is fixed and seen as an evaluation, not something to act on. Summative evaluation evaluates the end result, not the process and often it normalizes scores for comparison against a pre-decided standard (e.g. a grading scale).

Summative evaluation in user testing is performed to validate a design, hence it is performed less frequently. Often you will validate your design or prototype against an older design, industry standard or competitor benchmark, etc. It is also with a summative test, you can validate your design or product against company-specific UX metrics and KPIs. Even though summative testing focuses on a normative result, you can collect valuable data here as well. Normally, this is done via questions and observations.

Formative and summative evaluation in iterative development environments

During the development process, you’ll normally go from formative testing and turn more and more to summative testing the further you go. The same happens if you design and develop in a stage-gate model, where the development needs to be assessed via a summative test before passing the gate.

We believe that usability and UX work and testing should be part of the overall development process and we believe that user testing is most effective and powerful when it’s a part of an iterative development process.