What I learned from User Testing
If you work in product, you might have had the chance to lead a user test, or at least observe one. User testing can be one of the most intimidating and valuable steps of the design process because that’s where assumptions may either be proven true or fall flat.
Even though it is considered a no-brainer for designers, there are few challenges to overcome before embarking on a user test.
One is to justify the time invested in testing. Another challenge can be finding the time to do it. Finally, when conducting your user test it’s important to remember (and easy to forget) to lead a rigorous test so you can gather maximum feedback for your team.
Here are a few things that I learned through my experience with user testing.
User testing is never a waste of time
I often find myself relying too much on expertise via heuristics. I sometimes only ask my design peers, or worse, rely on my own assertions. The problem is, it’s nearly impossible to avoid our own biases. What’s more, it’s a very indirect process for the person who is trying to find the problem to not be the one who has it.
This article from usability shows the difference of results between Heuristics evaluation (HE) and Usability Testing (UT). Some interesting findings: only 36% of actual mistakes are spotted when relying on your design expertise, while a third of what you suspected to be problematic was actually not. Worse, without Usability Testing, you might miss 1 problem out of 2.
Relying on design expertise can result in missing a lot of problems, which will result in you having to fix them as you find them, or rely on your developers to fix them later. So next time, think about it if you hear “we don’t have enough time for testing”.
The 5 tests rule is good, but frequency is key
If you say 5 to UX Designers, chances are that they’ll immediately think of a User Test. This number has been proven to be the best benefit-expense ratio when it comes to spotting usability problems. More than an arbitrary number Jakob Nielsen and Thomas Landauer found the math behind Usability testing and proved that 5 users are enough to spot more than 80% of the problems.
But summing up User test to the sole number 5 would actually be too simple. I was really strict about this number when I started my first user tests. In reality, it really depends on the complexity of the project you’re leading.
At AB Tasty we do Iterative Design all the time, which made me understand that frequency is really what matters. It’s okay not to test with 5 users for each prototype. Do the test with 3 users, uncover the problems, fix them, then move on to the next iteration and repeat. You don’t always have to stick to 5 users as long as you have this approach.
In order to lead user tests the right way, clearly state what you’re trying to achieve. What are the top tasks or flows that you want your target users to complete? What’s most important? Always keep this in mind and learn frequently.
User testing shouldn’t be a designer-only task
As designers, empathy towards your other is a key principle. But this applies also to your coworkers. You should also picture your developers, sales, consultant as users of your project specifications and deliverables.
If you don’t step into their shoes and throw them your deliverables, they will have a hard time translating it the correct way and, by consequence, negatively impact the user experience in the end. Tell them why it’s done this way, whose problem it is and how you will address it together. User Experience is a mindset that brings empathy to your colleagues as well as yourself.
In that way, make sure you work collaboratively with your peers. Show your designs, but also let your team participate in user testing. That’s a great way to make them realize who gets to use the product and what impact their work has on it, be it features, product performance or copy. Everyone who has an impact on the end user experience should be able to participate in user testing.
Don’t get too passionate
When conducting a user test, it can be very easy to influence the participant. I always introduce the test saying that I’m making the test for someone else, so the tester should feel confident expressing his opinion.
Be rigorous and ask no leading questions to avoid spoiling your results. If the participant is stuck and asks for help, one of my favorite techniques is what’s called “echo”, I simply repeat what the user said in an interrogative way to make them answer and think aloud.
Try to always stay neutral and don’t express any feelings — negative or positive. It can be frustrating to see that your design doesn’t meet up the user needs or understanding, but see it as an opportunity for improvement. The same applies if everything goes smoothly during the test–don’t get too excited and jump to conclusions, or you might distract the participant and miss relevant insights.
User testing is a cycle
Even if you do a lot of testing and uncover most of the problems, it’s likely that new issues will show up along the way. This can be explained by many reasons.
As I said before, sufficiently rigorous testing, as well as impartiality, can help to draw out more insights out of your users. However, some reasons for future problems springing up can be complicated to control, like the Hawthorne effect. This psychological behavior is known as a response when users know they’re being watched, which change their behaviors compared to what they would normally do in a normal situation.
As product designers, we don’t want to make things to display–we want them to be used and useful. Never stop tracking and discovering what people do with what you make. It can be done in so many ways, like talking with your support or clients, using a heatmap, qualitative surveys and so on. Stay alert and curious. The journey is never over.
Many things can get in the way of user testing, depending on the complexity of your project and environment. In any case, just do it and be rigorous. It only takes to do it frequently to learn a lot and improve your work and yourself.
NB: Thanks to Lucas for the proofreading
