Measuring user trust
Tell me if this sounds familiar: You’re shopping for a hotel or a pair of earrings online, and suddenly you are bombarded with warnings to act fast. Rooms are running out. Hundreds of other shoppers have looked at this item. Or even worse, the earrings are already in someone else’s cart! These warnings are annoying at best and panic-inducing at worst, so why do so many sites use them? It turns out that the product managers behind these sites apply tips and tricks from social psychology 101– like scarcity — to get people to buy their products, or “convert.” And it works.
Traditionally, transactional metrics like conversion are the key metrics that businesses like Expedia use to measure success. But taking this transactional view doesn’t always foster a great experience for users. Tactics like urgency messaging may increase conversion, but they may also cause customers to lose trust in the brand. As a result, customers may never shop with the brand again. This is not only bad for customers, it’s bad for business. For each customer a company loses, it has to invest in attracting and converting a new customer. It’s much more valuable to foster relationships with loyal customers who trust the brand and come back again and again.
Companies understand the inherent value of trust, but typically rely on measuring unreliable proxies of trust — like transactions — rather than trust itself. For trust to inform product decisions, we need a way to measure trust itself, not proxies, at scale. As a trust researcher on the CX Measurement team at Expedia Group, my job was to tackle this conundrum.
The problem was: trust is psychological. It lives inside our users’ heads. Unlike an easily observable behavior like conversion, reliably measuring something inside people’s heads is really hard. After combing the literature, I found a widely-used trust scale that seemed promising. The scale measured brand trust with four questions and showed robust relationships with business metrics like market share. But, for the business to adopt this trust measure as a key metric of product success, it needed to meet a high bar.
First, the trust measure had to be valid. In other words, it had to actually measure what it was supposed to measure — trust! Second, it needed to be easy to implement across many contexts, including in A/B tests on the live site and apps. Finally, it had to predict meaningful business outcomes, like repeat purchases and direct traffic.
With these goals in mind, I ran a series of experiments with users of Expedia Group’s vacation rental brand, Vrbo, that evaluated the validity of the trust measure. Drawing on my background as a social psychologist, I designed experiments where users imagined experiences with Vrbo where either everything went as expected, or everything went terribly wrong. After imagining themselves in these scenarios, users completed the trust measure along with other measures like brand loyalty.
These experiments showed that the measure was valid. Specifically, the studies showed that:
1. The measure was sensitive to differences across experiences: Users reported lower trust after bad experiences than good experiences.
2. The measure predicted business outcomes: Users who trusted Vrbo more also said they’d be more likely to book with Vrbo again.
At this point, the CX Measurement team had a valid measure of trust. What we lacked was a means to measure trust at scale. To achieve scale, we first needed the right technology. We needed to measure trust in real time on our live sites and apps so we could use the trust metric in A/B testing and in daily monitoring. To get this real-time insight, we used a tool called site intercept, which injects a pop-up survey invitation into the site or app experience.
Second, we needed a strategy to ensure that we were measuring trust at the right place and at the right time. We also wanted to ensure that we were capturing trust at key moments in our travelers’ journey without disrupting that journey. To roll out site intercept, we partnered with UX design and content to ensure that the surveys were minimally disruptive and occurred at sensible moments in our travelers’ interactions with our products. Over time, we built partnerships across organizations like insights and product analytics to establish a rigorous, human-centered approach to CX measurement. This broader CX measurement strategy captured other key experience metrics like ease and usability across travelers’ end-to-end journey, including after travelers had left our sites.
With a solid strategy in place, the final piece of the puzzle was to convince product teams to adopt trust and other CX measures as key metrics of success. We had to prove the value of these metrics to the business. Through A/B tests and causally modeling impacts on traveler behavior over time, we showed that trust and other CX measures tied directly to behaviors like conversion and repeat bookings.
Today at Expedia Group, traveler trust is one of the key indicators of a successful product, shaping how our products are built and measured. When products are designed to optimize for relationships rather than transactions, our travelers win.
Learn More
How Design Thinking and Emerging Technology Will Enhance Travel Experiences
