UX Measurement #2

So, how can we measure UX?

It’s less straightforward than one might think.

Created with DALL·E.

This is the second in a series of articles on user experience measurement:

1. Conversion rate & average order value are not UX metrics
2. ☞ So, how can we measure UX?
3. Seven heuristics for identifying proper UX instruments and metrics

The precise, quantitative measurement of user experience (UX) based on one or more metrics is invaluable for design, research, and product teams in assessing the impact of UX designs and identifying opportunities. Yet these teams often employ supposed UX metrics like conversion rate (CR) and average order value (AOV), which can’t provide that measurement [1]. In fact, I believe this can be extended to an even more general statement: In themselves, none of the metrics that are usually readily and easily available from Web analytics data can reliably measure UX. I understand that this is frustrating news to many, since resources are always limited, attention spans short, and Web analytics so very, very convenient.

Whenever I discuss this, I encounter objections like, “But we have to do something,” or “It’s easy to just state what one shouldn’t do, but that doesn’t help much.” And while it’s a perfectly fine start to know what not to do (cf. Nicholas Taleb’s The Black Swan <— this is an affiliate link and I generate revenue out of purchases), in the case of UX, we must not despair, because there are ways to reliably measure it, albeit ones that are not as simple as pulling some number out of Google Analytics (as nice as that would be).

A simple outcome of measuring UX could be, “The last release improved checkout UX from 75/100 to 80/100,” but there could be more-nuanced measurements for different aspects of UX (e.g., usability, aesthetics, joy of use) and user groups. Before diving deeper into how we can do this, let’s first get familiar with three concepts:

• Latent variables (like UX) “are variables that are not directly observed but are rather inferred through a mathematical model from other variables that are observed” [2]. Take, for example, the Big Five personality assessment [2]. You can’t just ask someone, “What’s your personality?” and…