Measuring and manipulating usability

Priya Toshniwal
UX Collective
Published in
6 min readOct 12, 2020
A person holding a measuring tape as if to measure something
Source: Aha blinds

On a relaxed Friday afternoon, we (the development team) received an email from the Executive Director of R&D at our company. On a mission to improve the team’s productivity, he wanted to understand our biggest frustrations at work that impacts productivity. One of the developers replied, “some of the tools (software and hardware tools) we use and the processes we follow have inefficiencies that hamper our output significantly”. If you were a product manager for these tools, what would be the first thought running in your mind? Which aspects of the tools was this developer complaining about? If you guessed ‘usability’, good job!

Usability relates to how well a product is able to solve a user’s problem, thereby affecting user experience. Simply put, it determines how user-friendly your product is. To understand usability better, refer to this.

According to Wikipedia’s definition, Usability can be described as the capacity of a system to provide a condition for its users to perform the tasks safely, effectively, and efficiently while enjoying the experience. In software engineering, usability is the degree to which a software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use.”

Three main elements that determine a product’s usability (given the context in which the user is interacting with the product) are Effectiveness, Efficiency, and Satisfaction.

Effectiveness determines how effective your solution/product is in helping the user achieve their goal or perform a task using your product. Example: If your product provides a taxi booking service and the goal of the user is to book a taxi and they are able to book a taxi to travel from place A to B, the effectiveness, in this case, is 100%, since the user’s goal was accomplished.

Efficiency determines how efficiently was the user able to complete the task. Example: In the example of the taxi service, did the user reach her destination B? Yes. Was it done efficiently? If the taxi took path 1, then yes, it was done in the most efficient manner, given the constraints. For paths 2 and 3, the efficiency would be lower.

An image showing 3 different routes to reach from point A to B, taking 30 mins, 1 hour and 2 hour respectively.
Source: Drawn on my iPad

Satisfaction relates to user satisfaction after performing the task.

Set of 5 smileys depicting different user emotions like happiness, disappointment, indifference, appreciation, frustration
Source: Psychom.net

Now, going back to the developer’s complain, it was clear that the tools had a usability issue for the developer, given the context in which he was using them. As a product manager, if you want to improve the product’s usability, you must measure your current usability and metrics that focus on effectiveness, efficiency and satisfaction can be deployed here.

Metrics for Effectiveness

1. Completion rate of the task

Let’s say you have 20 users performing the same task of booking a taxi on your app. 15 booked a taxi. The task completion rate here is (15/20) = 0.75 or 75%. With this metric, we know that 25% of our users were unsuccessful in completing the task and this gives you a starting point to dig deeper into the underlying cause. You can also monitor and measure the completion rate of sub-tasks (all the series of small tasks that are part of that 1 big task) to understand the sub-tasks which users could not complete and then try to optimise those.

2. Error rate(when performing the task)

Out of the 15 users who booked a taxi, 6 users made a few errors during the steps. Example: they booked the wrong type of taxi, they put the wrong destination, etc. Here, 6/15 = 40% users made errors. You can also calculate the number of errors/user, the average number of error/user, number of errors/sub-task, etc. That will help you identify sub-tasks that prone to user errors and you can target those for improvement.

Metrics for Efficiency

1. Time taken to complete a task

It is the time elapsed from the time the user starts the task to the point of its completion. This metric should be reasonable and should be as low as possible. Nobody wants to spend more time accomplishing the same goal/task if it can be made efficient and can be done in less amount of time. Let’s say the average time for booking = 3 minutes.

2. Total number of steps taken to complete a task

Example: Your users booked a taxi in just 3 steps, compared to 6 steps in the previous version of your app. The goal is to get the work done with as little user effort as possible for better user experience (instant noodles require only 1 step of pouring hot water as compared to normal noodles which need to be boiled first before adding the condiments.)

3. Number of times assistance was required (to complete a task)

Example: 3 out of the 15 users referred to the HELP page in order to book a taxi. If your product design is more intuitive, users will require less assistance and will make fewer errors.

So you see, just measuring Effectiveness is not very insightful because it will only describe how many users were able to complete a task and will give you no details on the user’s journey in completing the task

Metrics for Satisfaction

1. User satisfaction (after performing the task)

Example: 2 out of the 15 (who completed the task) and 7 out of 20 users, overall, felt dissatisfied with your app. Satisfaction rate (for users who completed the task) was 13/15 = 86.6%. Overall satisfaction rate was 65%.

A satisfaction meter with 4 levels of user satisfaction rating: poor, fair , good and excellent
source: Dave Schneider

2. User emotion after performing the task

Your users can have positive, negative or neutral emotions after using your app. Your goal is to have as many users as possible feeling positive emotions after using your product.

An image of Plutchik’s wheel of emotions displaying a wide spectrum of user emotions
Source: saintnicks.uk.com

For overall usability, you could use the formula:

Usability score = (Effectiveness*Efficiency*Satisfaction)

If you want a higher emphasis on efficiency, you could always manipulate the equation, for example,

Usability score = [Effectiveness*(2*Efficiency)*Satisfaction]

How you craft the equation is really up to you. Often, there are dependencies amongst the metrics. Higher effectiveness and efficiency will result in higher user satisfaction. Therefore, in most cases, you will need to focus on improving the effectiveness and efficiency, and user satisfaction will be taken care of. In such a case, you can remove ‘Satisfaction’ from the equation and focus on the other 2 variables, if ‘Satisfaction’ is largely dictated by the other 2 variables.

There you go! You can add such metrics to your usability studies to understand the various usability issues that need improvements. These metrics, when used wisely, will provide you with a good picture of your product’s usability. Don’t just measure one metric and leave out the rest unless they don’t add any insights to our understanding. They will provide you with more visibility and objective data to make sounds decisions.

So, go be a good product manager and start using usability metrics to your advantage to understand and optimise the usability of your products, thereby improving your customers’ experience and loyalty for your products!

For questions and feedback, please leave your comments.

The UX Collective donates US$1 for each article published in our platform. This story contributed to Bay Area Black Designers: a professional development community for Black people who are digital designers and researchers in the San Francisco Bay Area. By joining together in community, members share inspiration, connection, peer mentorship, professional development, resources, feedback, support, and resilience. Silence against systemic racism is not an option. Build the design community you believe in.

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

Written by Priya Toshniwal

Discovering the world beyond the realms of salt and pepper. Mostly day dreaming about food and other people’s dogs.

No responses yet

Write a response