Should product designers rely on intuition or data?

The intuition vs. data premise is common but has always seemed flawed to me. It’s a false dilemma.

Intuition (in the colloquial sense) is tricky, but it’s not magic. It amounts to having a feeling of knowing the answer without being able to explain how you know. It’s automatic, and a good intuition is often based on past experiences. In true Scooby-Doo style, it’s easy to pull back the mask on this and reveal that past experiences are data; First-hand observations, things you have read or been told, or feedback you’ve been given are all data points of varying quality.

Under the hood of every good intuition is data!

Can designers rely on intuition?

Well, it depends… In the paper Conditions for intuitive expertise: a failure to disagree psychologists Daniel Kahneman and Gary Klein have this to say on intuition:

[…] reliably skilled intuitions are likely to develop when the individual operates in a high-validity environment and has an opportunity to learn the rules of that environment

So, according to Kahneman and Klein, it’s possible to develop “reliably skilled intuitions” but we need two ingredients:

1. A high-validity environment, which means; There are objective and stable clues about the reality of a situation. For example, when you interact with the same person repeatedly over a long period (say a spouse or colleague) you will get to observe stable patterns in their body language and behaviour.
2. The opportunity to learn the rules; An ability to practice and get immediate feedback. For example, you might guess your spouse’s mood based on her behaviour. With repeated practice (guessing her mood) and immediate feedback (perhaps from directly asking her) you will eventually develop a good intuition for her mood and no longer need to ask.

For me as a product designer, this makes sense; If I work on a product with a stable user base (high-validity), and spend a bunch of time observing how my design decisions affect user behaviour (learning the rules) then my intuition is likely to become reliable over time. Awesome!

But, not so fast gut-feelers! Here’s more from the paper:

These conditions often remain unmet in professional contexts, either because the environment is insufficiently predictable (as in the long-term forecasting of political events) or because of the absence of opportunities to learn its rules (as in the case of firefighters exposed to a fire in a skyscraper with unexpected damage to the heat shielding of its structural support).

If I instead work on a product that changes frequently or has a large and ever-changing user base, validity might be lacking. If I don’t have access to direct and specific feedback on my design decisions then good intuitions won’t develop. This is not to say that my intuitions would always be wrong, but they may be unreliable; Any successes could be due to luck or factors outside of my control. A data-informed approach sounds pretty useful in this scenario.

The messy reality

In reality it‘s not always easy to tell when intuition can be trusted.

I’ve designed what seems like basic, tried and true interactions, and felt very confident, only to later find that those interactions weren’t familiar at all to a subset of users.

On other occasions I’ve sunk so much time into doing research only to learn that my first intuition was totally fine.

More recently I’ve found it helpful to think back upon the 2 ingredients and ask myself:

• High or low validity? Am I designing something for a population that is stable and mostly alike (i.e. same goals, same capabilities, culture, language) or is the population rapidly changing and diverse?
• Have I learned the rules? Have I shipped similar interactions or experiences to my target audience before and learned what works, or is this completely new territory?

Wrapping it up:

• Reliable intuition is based on data under the hood — framing discussions as data vs. intuition doesn’t make a lot of sense.
• Developing a reliable intuition requires a high-validity (stable) environment, and a period of practice to learn the rules.
• If the ingredients required for a reliable intuition aren’t in place, take care and consider being more data-informed.
• The point is not to always rely on data — if conditions are right, then intuition can be both reliable and efficient.