Types of cognitive biases you need to be aware of as a researcher

Cognitive biases are errors in reasoning, evaluating, remembering, often occurring as a result of holding onto one’s preferences and beliefs regardless of contrary information.

“Bias is any process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth” — Sackett (1979)

According to some social psychologists, cognitive biases can make us process information faster, especially during dangerous circumstances. However, they also can lead us to make consequential mistakes.

In UX research, biases can skew the findings and ultimately affect the decision making of a project at any stage. Cognitive bias can also affect both researchers and participants, so it is important for the researcher to be aware and familiar with these biases in order for them to conduct screening, usability testing and synthesizing more carefully.

One of the talks by Ruth Ellison @ UXSG 2016 Conference

According to Ruth Ellison, the following are some common cognitive biases that are important for researchers to be aware of when they are carrying out their tasks.

Confirmation Bias ☹️

The often unconscious act of referencing only those perspectives that fuel our pre-existing views, while at the same time ignoring or dismissing opinions — no matter how valid — that threaten our world view.

During research, we tend to unconsciously filter out feedback from users that does not help in supporting our assumptions.

How to avoid it during research process?

1. You must treat all data(findings) equally.
2. Play the devil advocate and look at your hypothesis from the opposite side.
3. Be skeptical, especially if everyone agrees with you 🤔
4. Leave your ego by the door. You are not the users and you cannot predict the future. (also read Hindsight bias, also known as the knew-it-all-along effect)
5. List all the assumptions down before begin your study.
6. Be open! We need to feel comfortable having our assumptions or hypothesis challenged with open mind in order to uncover valuable insights that will help the Design team in their design process.

Groupthink and the Bandwagon Effect ☹️

The phenomenon of groupthink is closely related with the Bandwagon Effect.

People working in a group tend to maintain harmony between members of the group. To attain harmony, the members may agree upon a decision that deviates from the correct decision. Thus, for the sake of avoiding conflict, members agree upon a point without critical evaluation.

How to avoid it during research process?

1. Triangulate with other research methods, like observational.
2. Avoid stating your preferences upfront to your participants. If you feel the sign up process will be a couple of steps, do not say it in front of the participants!
3. Avoid stating your expectation upfront to your participants. Again, don’t tell them that completing this task is very easy. For example: Don’t say this: “Okay, so the next task is very simple…” 😱
4. Allow someone in your team to question/challenge your assumptions. Be open and listen to their thoughts.

Anchoring Bias ☹️

During decision making, anchoring occurs when individuals use an initial piece of information to make subsequent judgments. Once an anchor is set, other judgments are made by adjusting away from that anchor, and there is a bias toward interpreting other information around the anchor. A/B Testing

Anchoring bias is the human tendency to “rely too heavily on the first piece of information offered when making decisions”.

In short, we tend to prefer the first thing we are shown when asked to make a decision, such as choose between a set of options.

How to avoid it during research process?

1. During A/B Testing, make sure to have a different set of orders of the variants. Alternate the design concepts for your participants. For example, let participants 1, 3 and 5 will see Option A first, while participants 2, 4 & 6 will see Option B first.
2. Again, use open-ended questions
3. Consider the order of your questions carefully.

A/B Testing with different set of variants to avoid Anchoring Bias results.

Selection Bias ☹️

This bias occurs when the researcher decides which type individuals or the number of individuals to participate in the study. Because the selection of participants isn’t random, as the result, the validity of the studies may be undermined.

How to avoid it during research process?

1. Selection bias usually occurs at the stage of recruitment. Use multiple channels to screen participants. For example, if you are working on a research about how people perform food ordering, go to both foodcourts and online stores. Don’t just tick to one!
2. Avoid recruit professional respondents. Period.

Clustering Illusion & Reporting Bias ☹️

Clustering Illusion occurs when we tend to look for patterns in a pool of random data. In other words, we are ‘pattern machines’ and we recognise people and things from their overall pattern rather than the bigger picture.

Reporting bias occurs when the direction or statistical significance of results influences whether and how research is reported

How to avoid it during research process?

1. Sample size — It’s about the WHY?
2. Consider evidence equally, not just the ones that confirm your belief/assumptions.
3. Collaborative analysis sessions — When analysing the data, try to get another partner to work together.

Clustering Illusion.

Observer Expectancy Effect ☹️

The observer-expectancy effect (also called the observer effect) is a form of reactivity in which a researcher’s cognitive bias causes them to subconsciously influence the participants of a study.

Quiet often, Observer Effect occurs during the Usability Testing session where the researcher subconsciously make ‘sigh’ sounds or shook their heads when participants were performing their tasks.

How to avoid it during research process?

1. Keep a neutral body language all the time. For example: Do not shook your heads when they failed to perform a task. Watch out for the tone of your voice too, keep calm. Don’t say: ‘Oh that’s interesting!…” Just say “I see.”
2. Avoid leading questions. Keep it open ended. Do not ask: “What do you dislike about Viki?”.
3. Apply more observational research method. Talk less. 😬

Thanks again to Ruth Ellison for her interesting talk at the UXSG 2016 Conference. Here is the recap of the biases that I learned, I am ware that are more which you can always google them up.

1. Confirmation Bias
2. Groupthink and the Bandwagon Effect
3. Anchoring Bias
4. Selection Bias
5. Clustering Illusion & Reporting Bias
6. Observer Expectancy Bias