Predicting User Interfaces: The KLM

You just finished the mock-ups of the app for your startup, or you have to decide between which design is better, or maybe you can understand that there is a problem on a design and you have to prove that your suggestion is better. But you don’t have the users to test… Keystroke-Level Model (aka KLM) is here to help you!

Photo by Jen Theodore

What is KLM?

KLM is a tool that can predict accurately (enough) how much time it will take for an expert (a user that doesn’t make errors and knows the system by heart) to complete a goal. Let’s get into it, and we can discuss the pros and cons later.

Prefer a visual approach? Skip the tutorial by watching Paul Green’s Video on Youtube.

HOWTO

Step 1:

Find the goal of the user and describe it as a task.

Step 2:

Make the appropriate assumptions.

Step 3:

Break the goal into very small fundamental actions (called operators) that are the same for all applications and can be measured once. Some of these actions can be a mouse pointing to an element on the screen, a button press, a swipe etc. Here is a list of some widely used:

Some KLM operators

This is a combination of standard operators, Jeff Sauro’s and Batran’s et al. research.

Step 4:

Calculate the total time of the task by adding all the individual parts.

Using Spotify to understand the KLM

We all love Spotify, but most importantly we all understand it. That’s why we will use it to demonstrate the use of KLM.

Predicting a goal duration

Let’s start out simple by examining how much time it will take for a user to search for a song to listen to.

User goal: Search for a song and listen to it.

Although this is a simple task we have to take into account the user start position, his mental state, how the system will behave, etc. This is done using assumptions.

Assumptions:

• The user just opened the (android) application.
• He knows the song name and how it is spelled.
• The song exists on Spotify (dah).
• Spotify suggests the song after 5 characters are typed and the user expects that behaviour.

To identify the fundamental actions we need to find the path that the user will follow and describe it step by step. In parenthesis we are including the corresponding operators for each step.

The path:

1. Press the “Search” button (M I)
2. Press the search bar (M I)
3. Type the first 5 letters (M 5K)
4. Check if the song is suggested (S)
5. Press the song (I)

The path for searching a song.

So the total time it will take for this specific user to search and listen to a song is 3*M + 3*I + 5*K + S = 7.58 sec.

Why this specific user? Because there are many ways to search and listen to a song, different starting location, mental models etc. that affect the duration of the task.

Making sure that a *design improvement* is better than the original

Getting into more real stuff, let’s now imagine that a designer has come up with an idea of adding a button that undoes the action of adding a song to playlist.

The current (left) and the suggested (right) versions.

First let’s predict the duration of adding a song to a playlist.

User goal: Add song to a playlist.

Assumptions:

• The user is already listening to the song he wants to add.
• He is also add the “song playing” page.
• He correctly chooses the playlist.
• The playlist is one swipe down. (At the moment the playlist sorting of Spotify is somewhat unpredictable, there is no obvious way to the user to correctly guess the position of the playlist.)

The path:

1. Press the three dots (M I)
2. Press the “add to playlist” button (M I)
3. Check if the playlist is visible (S)
4. Scroll down (U)
5. Check if the playlist is visible (S)
6. Press on the playlist (I)
7. Automatically return to the “song playing” page (-)

The path for adding a song to a playlist.

The total time for the specific user to add song to a playlist is 2*M + 3*I + 2*S + U = 7.59 sec.

What happens if the user miss-clicks, for example he presses the playlist that is exactly bellow the desired playlist?

User goal: Remove a song from a playlist.

Assumptions:

• User is already listening to the song he wants to add.
• He is also add the “song playing” page.
• He selects the incorrect playlist.
• The playlist is one swipe down on both the “add to playlist” screen and on the “your library” page.
• The user adds the song to the correct playlist before removing it from the incorrect.

For the current version:

1. Add song to playlist goal — but select the incorrect playlist (7.59 sec.)
2. Add song to playlist goal — but select the correct playlist (7.59 sec.)
3. Press the down arrow (M I)
4. Press the “your library” button (M I)
5. Check if the playlist is visible (S)
6. Scroll down (U)
7. Check if the playlist is visible (S)
8. Press the playlist (I)
9. Scroll to the bottom (U)
10. Press the dots (M I)
11. Press the “remove from playlist” button (M I)

The path for removing a song from a playlist.

Total time for the current version 2*7.59 + 4*M + 5*I + 2*S+ 2*U = 25.63 sec.

For the suggested version:

1. Add song to playlist goal — but select the incorrect playlist (7.59 sec.)
2. Press the “undo” button (M I)
3. Add song to playlist goal — but select the correct playlist (7.59 sec.)

Total time for the suggested version: 2*7.59 + M + I = 16.61

Based on our predictions the suggested improvement saves 35% of time.

Additional info

KLM was originally proposed by Card, Moran, & Newell back in 1983, it is part of what is called heuristics in interfaces, metrics to predict user behaviour based on statistics. Originally it was used for PC applications but is still relevant today and extended for mobile devices by Batran, & Dunlop.

This model gives a relatively easy (and engineer-friendly) way of predicting the duration of a task, it is accurate enough, simple to understand and can save a lot of money and time from user interviews. What is the trade-off?

Restrictions

Keystroke-Level Model was designed to address the prediction of the duration of a task to be completed, from only one aspect. Initially it was proposed as a lower bound on the effectiveness of new proposals. The restrictions of the model goes as follow:

• The user must be an expert.
• The task must be a routine unit task.
• The method must be specified in detail.
• The process must be error-free.

Despite these restrictions, it is still a pretty neat tool.

Further reading

More examples of KLM applications can be found in this article from David Kieras.

Want to dive deeper in KLM? Take a look at this article by Peter Zalman.

KLM sounds it could be automated, is there any tool to do the work for me?

— Yes it exists.