How Confirmation Bias can affect user research
Biases are part of us and we can’t get rid of them. One of the ways to reduce their effect on us is to simply be aware they exist. Everybody experiences them, including you.
When doing user research, this knowledge is especially useful, because it can prevent you from offering solutions that are not helpful for your users. As designers, we like to find ways to improve people’s lives by trying to understand what they need. We use design thinking to help us solve creative problems and try our best not to let our assumptions about what users need guide our design decisions. That’s why we do user research and user testing to ensure we solve the right problems. Sometimes though, even with the best of intentions in mind, we can let our own biases creep into the process. By learning more about the confirmation bias and how it works, you will be better equipped to avoid some of its undesirable effects when conducting user research.
Biases: the result of how our brain works intuitively
Biases are not the symptoms of a low IQ. They are simply the result of how our cognition naturally works. Even though we prefer to think of ourselves as less biased and more objective than the rest of the crowd, the truth is we are all vulnerable to biases simply because we are all human. In fact, even when we try to be perfectly objective, deep inside of us, we all have certain preferences. These sets of personal preferences can sometimes sneak into our objective selves without us realizing it.
Most biases are the result of our brain trying to make quick connections with what’s going on around us. To be more precise, the processes inside our brain that reach these quick conclusions are called heuristics. These heuristics are cognitive shortcuts we use to help us make decisions. Most of the time, they serve us fairly well, because they allow us to react quickly without having to rely on a huge amount of data and analysis. For example, you won’t take into account all the possible alternatives before deciding what to wear in the morning, you’ll take a heuristic: what shirt would go best with those jeans? Our brain will attempt to solve this problem with an easier way to make our lives simpler. That is where biases can sometimes creep in and distort your thinking. These cognitive shortcuts create good enough solutions most of the time, but can sometimes lead to bad decision making and result in biased judgments.
Seeking out information that only confirms what we believe
We sometimes forget that the world isn’t exactly like we see it. Confirmation bias can be dangerous because it can falsely distort your perception of the world. When seeking information, we tend to look for information that confirms our own set of beliefs. We unconsciously ignore information that disagrees with our hypotheses or our convictions. More drastically, it can even make you think you are right most of the time because there’s always a way to seek information that seems to agree with what you believe. For example, you want to find out if the president is making good decisions about the environment? Just search the name of a journalist who always agrees with him on this issue. Bingo!
Note to everybody: I’m not saying we do this all the time, but we do these things without being fully aware. Since it is a natural way to reason, it takes us double the effort to do the contrary, that is to disconfirm our beliefs.
But there is good news: research shows that being aware of the confirmation bias can help you be more careful. Our intuition is always part of us. Even when we process information deliberately, our intuition is still part of the game. That’s because we reason with both intuitive and deliberate processes. You can’t unplug your intuitive self, that is why it is a challenge. And intuition is where most of our biases take place. If you want to minimize the confirmation bias, use your deliberate self. Analyze the situation carefully and adopt a more open mind by trying to seek all kinds of evidence. It can be hard because confirming we are right makes us feel good and confident. Plus, disconfirming your beliefs puts you into a negative state, which you like to prevent most of the time.
Confirmation bias when doing user research
When researching to understand who your users are and what they need, it can get tricky to take into account a diverse pool of users. Since we are only aware of the users we think we know are using our product, we can be tempted to only ask those users. While it can work, this way of doing research can only help you solve the problems of the users who confirmed your assumptions. But the rest of the group who you never asked their opinion in the first place won’t get to be part of the game. Sometimes, it can also make you focus on the wrong target of users because you are not aware of the full range of users. That is why it is important to keep an open mind and try to find as many diverse users are possible. With that being said, you can’t solve every problem for every user, it would be impossible to know what to prioritize. But knowing the diverse pool of users who utilize your product will help you assess which features need to be prioritized for a group of users and help you solve the right problems.
User feedback can be good to confirm that what we know is correct, but it should also help us identify problems that we didn’t even know existed before. In any way, user research needs to help you expand your mind and help you see with a new set of eyes. It needs to help you discover new information that you didn’t even know was important to consider. When the user talks about his experience, make sure you don’t ignore the information that you think might not be important to you. Take it into account anyway and wait until the end of all your interviews to make that decision. This might be a recurrent pattern that could help you design a better feature. Leave your own set of assumptions behind and keep asking your questions.
How you ask questions matters
When doing user research, how you phrase your questions plays an important role. Why? Because you can suggest an answer based on your preconceptions. For example, if you want to know if the interviewee likes using the search bar, you could be tempted to ask a question like: what do you like about this search bar? The structure of the question is erroneous because you already assume in the first place that the user likes the feature. This type of question can influence the user to answer in a particular way and make him forget mentioning what he also doesn’t like about it.
To make sure you understand his whole experience with the search bar, let the user speak for himself. Ask something like: describe to me your experience when using this search bar. Then if you notice he doesn’t explain himself much, you can ask a follow-up question like: what do you like about it? Then don’t forget to also ask: And what don’t you like? Is there anything you think that is missing that would help you accomplish x task? In this way, you have less chance of influencing the interviewee and you actually get to hear the whole story.
Conclusion
Confirmation bias can make you feel good because it protects you from the information that doesn’t agree with you. This shield can be dangerous though because it can prevent you from seeing the whole picture. To make sure this bias doesn’t get the best of you, try keeping an open mind when you have to make important design decisions. Even if disconfirming your assumptions can be painful at times, this will help you see the world as it really is, and not how you would like it to be instead. As designers, we are called to question our assumptions and ask the right questions to find out the truth. That is, even if it isn’t what we wanted to hear in the first place. After all, we are not the users who we are designing for.
Bibliography
Hallihan, Gregory M., and L.h. Shu. “Considering Confirmation Bias in Design and Design Research.” Journal of Integrated Design and Process Science, vol. 17, no. 4, 2013, pp. 19–35.
Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2015.
Cherry, Kendra. “Why Do We Favor Information That Confirms Our Existing Beliefs?” Verywell Mind, 19 Feb. 2020, www.verywellmind.com/what-is-a-confirmation-bias-2795024.
