Member-only story

What a quantitative user study that nearly failed taught me about metrics

The benefits to collecting metrics while user testing, even with a few users

Kai Wong
UX Collective
A hand holding a bunched up tape measure together. The 14 Foot marker is visible in the foreground, with several bunches of previous measurements behind it.
Photo by Elisa Michelet on Unsplash

Our team almost had to consider throwing out a portion of our quantitative user test because some of our users canceled on us.

We needed to look at Time-on-Task to answer one of our research questions. This meant we needed to have enough users to provide statistically significant comparisons.

We were able to scrape together 20 participants, the minimum recommended number for quantitative testing, but two of the participants fell through last minute. That meant we had to scramble to get more.

But during that process, I began to wonder: what if we had only managed to get 18 participants instead? Would we have had to throw out that metric entirely? Why wouldn’t that be useful?

I dug deeper into the subject as a result and learned three crucial things:

Create an account to read the full story.

The author made this story available to Medium members only.
If you’re new to Medium, create a new account to read this story on us.

Or, continue in mobile web

Already have an account? Sign in

No responses yet

What are your thoughts?