5 tips for producing quality user research
The Association for Qualitative Research defines research bias as:
“A systematic error, where a particular research finding deviates from a ‘true’ finding. This might come about through errors in the manner of interviewing, or by errors in sampling”.
To simplify, research bias is when ‘false’ findings are produced from research due to errors made by the way research was conducted or how the sample (participants) were chosen. Common types of research bias include:
- Confirmation bias: when a researcher forms a hypothesis or belief and uses respondents’ information to confirm that belief.
- Wording bias: Phrasing questions in a way which will get a particular answer from a user.
- Sampling bias: choosing your sample of users in a way that excludes a variety of user types.
Research bias can occur consciously or unconsciously but it is our job to ensure we do the best we can to limit bias and produce reliable results. So here are the key 5 things you need to consider in order to ensure you are producing quality research:
1. ALWAYS refer back to your research objective(s)
A research objective is the “why” of the research and quite frankly there should be no research conducted without it because it should determine the research method you use and must be referred back to when analysing and presenting the results.
For example, let’s say I own a fashion brand and I want to find out whether I should expand my range to include petite and tall sizes. In such a case, I should have at least 2 research objectives:
- To determine whether the business will benefit from introducing a petite range and
- To determine whether the business will benefit from introducing a tall range
I should then determine which research methods to use based on what will best answer my objectives, which could be a survey, interviews or observations if I own a store. Then once I have conducted the research and analysed the results, I should refer back to the objectives and confirm whether expanding into Petite and Tall is a good idea or not.
2. Use multiple research methods
Using more than one research method can help you to produce more robust results as you see the data from different angles. I particularly like to use a quantitive research method along with a qualitative one as the former confirms how much something is true while the latter gives insight as to why this is so. Using multiple research methods can also provide validation to results or an insight to where anomalies or incorrect assumptions occur.
For example, in a research project I conducted with a Travel Company, we interviewed 5 customers and found that they were all professionals who frequently travelled and mostly valued loyalty points. However, when we conducted a survey which collected 561 responses, we found that the customers we interviewed only represented 1% of their customer base. Had we not conducted the survey too, their business strategy could have been revised to fit a customer profile which only accounted for 1% of their business which would have been a total disaster.
3. Recruit as many participants as possible
Science teaches us that the bigger the sample (participant) population, the better, as it increases the reliability of results. So when conducting research, aim for the maximum of users you can find.
If we go back to the example I gave of the travel project, if we had been able to interview more than 5 customers then we may have been able to build a better representation of their customer’s profiles and find out more about them beyond the generic questions we asked in the survey. (In case you are wondering why we only interviewed 5 customers this was because they were the only customers who we were available within the 2 days we had to conduct interviews).
4. NEVER use leading questions
Leading questions are questions that influence a participant to a particular answer e.g. asking a user ‘did you have any problems with our new app?’ instead of asking ‘ tell me about your experience with our new app”. The former implies that the user encountered problems while the latter is neutral and lets the user determine whether they experienced any issues or not.
5. Test your results
As we all know, humans love to say one thing but do another (I’m particularly guilty of this when it comes to exercise), so conducting tests are a great way to determine if the findings are true. This could range from mocking up a new feature and seeing how users react to it, to creating a sample fashion range and checking if customers actually buy it (like in the earlier fashion brand example). Doing so also helps to save time and money and prevents you from assigning resources to a project which will not create a great return on investment.
To conclude, research is very valuable but only when it is done with best practices. In the real world we can not always treat our research like academic projects but by incorporating these key research principles then we can still produce quality results. So the next time you start a research project, don’t forget to always refer to your research objectives, use multiple research methods, get as many participants as you can, never use leading questions and test your results if possible!
Here are some great resources for further reading:
