Usability tests in a nutshell

I believe that design should be done in a manner which we can prove our professional intuition. Testing the product brings it to the tracks as well as the great feeling of been designing for other people.

We are going to approach the thinking aloud method.

THE THINKING ALOUD METHOD

Definition

In a thinking aloud test, you ask test participants to use the system while continuously thinking out loud — that is, simply verbalizing their thoughts as they move through the user interface.

Jakob Nielsen. 1993.

Why we chose this method

• Flexible: Websites, software applications, intranets, consumer products, enterprise software, mobile design: does not matter — thinking aloud addresses them all, because we rely on the users doing the thinking;
• Cheap: No special equipment is needed; you simply sit next to a user and take notes as he or she talks;
• Convincing: From the intern to executives, they usually soften up when they get direct exposure to how customers think about their work. Getting the rest of your team to sit in on a few thinking-aloud sessions does not take a lot of their time and is the best way to motivate them to pay attention to usability;

WHEN TO APPLY

You can use the method at any stage in the development life-cycle, from early paper prototypes to fully implemented, running systems.

Even in early stages, the test can result in good findings that will help to measure and get future challenges in the spotlight.

Early stage

Any possible solution to a problem can be tested with paper prototypes or navigable wireframe built upon images.

In-development stage

If it is possible to apply any new idea to an existing environment that will consider past decisions. Maybe all design is not applied and all features are not in place, but it is already considering business’ aspects or technology limitations.

Mature stage

There are many business’ aspects defined, many technologies choices are set, the project is running with data from real use cases.

OUTCOMES

The overall aim of the usability test is to find usability problems with any product been evaluated, so they can be seen by the team and prioritised.

As a tool for evaluation, the team will get a list of usability problems that were found during the test. This list can come together with the audio and/or video file, and each problem will be connected to a time on this file.

There will be one list per participant and the performed tasks will be based on the test goals, however, there is nothing against adding new and relevant findings in the report.

TEST ROLES

• Participant: She is going to be the one performing the usability test. Choosing this person is key for collecting good quality data, a biased person can add noise to the test results.
• Moderator: This person should be a member of the UX team;
• Responsibilities: Build and conduct the usability test session;
• Observer: This person should be someone in the team with enough background of the software piece being evaluated;
• Responsibilities: Build and data log the usability test session;

BUILDING THE TEST

A usability test is composed by goals, red routes, scenarios and tasks.

GOALS

The test should have goals and reasons for each of them clarified.

Example:

• Goal: Identify pain points on the booking flow;
• Reason: We got feedback that users are struggling to finish their booking, but we are not sure of which parts of the system are causing such issues;

The scenarios as well as tasks, which we will go through soon, should aim to bring up information that will help to achieve each test goal.

RED ROUTES

The moderator and observer should, whether it is not yet, define the main tasks on product that has been evaluated.

These main tasks are named as red routes and they play the functional part of the system, whether they fail, the product may not achieve its goals.

The importance of defining them, even whether they are not the test focus, is to detect problems on them, since they are crucial, it is a good practice to be observed whenever it is possible.

SCENARIOS

A scenario is a set of contextual information that the participant will consider while performing the tasks, during any action they do, it is mandatory to take the scenario into account.

Example:

1. Booking a business trip to New York;
2. Booking a romantic trip to Paris.

On different contexts, the participants have different priorities, so they may perform the task in a different way.

The importance of defining them is to qualify the collected data, since the participants will be receiving their environment from the scenario made.

TASKS

Once the red routes and scenarios are defined, we can begin to set the test tasks.

A task is something that the participants should try to perform, they can or can not be successful doing it.

The task should be connected to the test goals and stimulate the user to use the system’s part which the team believes that might contains usability problems.

A good task does not show users how to do something, it should just give them an objective so the moderator and observer can evaluate it.

TASK SESSIONS

In order to accommodate the participant, moderator and observer during the sessions, get these following things in place:

• Print each task in a paper card. It is going to be given to participants in the sessions so day can recall the task with no shame and interrupting the moderator and observer’s note taking.
• Print the test script [link to this document], so the moderator will have guidance, there is no shame on reading the script during the session in order to make sure the moderator is not forgetting anything;
• Whether the session is going to be recorded, make sure the software of your choice is working properly.

THE USABILITY TEST

PILOT TESTING

The purpose of the pilot test is to identify and resolve any technical issues with the recording equipment or with the product itself that might cause delays to the actual test.

We expect the pilot test to take a short period of time, however we can not risk to find out serious issues right before the session, that is why the moderator and observer must get this done at least one day before.

TEST SESSIONS

Each session will be organised in the same way to facilitate consistency.

The session is composed by:

1. Introduction: Onboard the participant in the usability test;
2. Pre-task: The moderator will explain how a thinking aloud usability test works by giving the participant a simple task. During the pre-task, it is crucial that the moderator makes sure the participants practice this process so they can actually understand how to perform the real tasks;
3. Task: The moderator will ask participants to do some tasks using the product. The moderator’s role is to keep up a good flow of comments as the participant works, so the observer can collect notes and data log it for an afterwards analyse.
4. Participant’s questions: Save some time for a free talk with the participant, so she can ask questions and the moderator and observer get some insight from it;
5. Port-test questionnaire: At the end of the test session, participants will complete a satisfaction questionnaire.