Running a remote user research study
Whilst in-person interviews continue to be the gold standard in conducting user research interviews, simply because human beings are naturally wired to communicate in-person rather than through screens, the current climate in the world (Coronavirus) has required us to adapt and pivot to doing user research studies remotely.
There are some great advantages that come with doing user research interviews remotely. These include opening your pool of potential participants, saving time and cost on travel, and increasing chances of participation since the participant can be anywhere.
Although a lot of the following tips can be applied to in-person user interviews as well, this guide will give you some things to look out for and help you be more prepared for your next remote user research study.
Plan ahead
Like that saying goes “If you fail to plan, you are planning to fail” and it’s no different when it comes to running remote user research sessions. While it may require a bit more work, preparation is vital to ensure your user research studies are as successful as possible.
So then, how do you best plan ahead? Here are some tips that are worth considering when planning your next session.
Create a research plan
Writing up a test plan will help you get your thoughts onto paper and make sure you have a clear idea of what it is you’re looking to achieve with the user research. It will also help you decide on what type of user research you need to do.
Some high level things you may want to cover off in your research plan are:
Context
Provide a brief overview of why this research needs to be done and how it will help with the work you’re currently doing
Research goal
What is the purpose of this research and what are you hoping to validate, uncover or learn
Assumptions
What are some assumptions the team currently has that you’re looking to validate
Who, what, when, how
Who are the right type of users you want to interview
What type of research are you hoping to do
When do you need to do it
And how are you planning on doing it
Questions
What questions do you want to ask, how should you phrase these questions (super important), are there any follow up questions you should consider
Tasks
If you’re doing usability testing, you want to know what tasks you’ll be asking participants to do and how to best phrase those tasks to avoid giving too many hints away which may bias the results you get from the test.
Share with your team
Once you’ve got your plan in a good place, it’s wise to share this with the wider team. Sharing with your team at this stage ensures that everyone is aligned on what the purpose of the research is before any interviews are started.
This will save you time in the long run, avoiding the potential disruption of mixed opinions once you’ve already started with user interviews.
The team may also be able to help you recruit the right participants that you want to reach out to for your user research.
Test your interview setup
If you’ve ever tried to demo something to your colleagues, you’ll know how easily things can go wrong at the worst possible time. Test your setup with a colleague, preferably someone who doesn’t know the project well so as to remove any potential bias. Testing your setup will help you identify any issues that might arise and will reduce the potential for future embarrassment when it finally comes time to doing the session with a customer.
Organise the interview
On the surface, organising the interview might come across as self explanatory. There are however a few small things to remember which improve the chances of a successful user interview and a positive experience for everyone involved.
Recruit participants
One of my favourite methods of recruiting participants is to simply email them. Emailing your potential participants is a great way to find out who is interested in participating in the study, while also giving you the opportunity to give them some context.
In your email, try using common language by avoiding the use of terminology that may only be known internally or that could be considered industry jargon.
Also, consider including some of the following:
- Brief introduction of who you are and what you do
- Short description of why you’re reaching out to them and how it will benefit them
- An idea of how long the session will be
- A link to your calendar (can use tools such as calend.ly)
The above ensures that the participants will have all the vital information before making their decision to participate in the study. The link to your calendar is optional, but it will save you and the participants a lot of time on scheduling ping pong, trying to find a time that works for everyone.
Invite a teammate
Ask a colleague to join the interview as an observer and note taker. This leaves you fully focused on listening to the participant and asking good follow up questions.
Send a follow up email
You’ve sent your emails out and the customers have accepted your meeting invitations. You can choose to do a couple things, wait until the interview and hope for the best, or send a follow up email as a confirmation.
A follow up email the day before or the morning of the interview can act as a good reminder for the following:
- Session start time and how long the interview will be
- Apps or tools that will be used like Zoom have been installed
- Devices or equipment needed for the interview are ready
This saves you from jumping into an interview, spending interview time on setting up or waiting around for 15 minutes, only to find out the participant won’t be joining.
Run the interview
Finally, you’re ready to conduct the interview. Although we have a clear goal in mind for our interview, it’s also a great opportunity to get to know the end users better.
Here are some tips that will help you run a smooth interview.
Welcome participants
When the participant joins the call, welcome them, thank them for their time and introduce yourself and your teammates. Ask them how their day is going before jumping straight into the interview process.
Give the participant a brief overview of what you’ll be covering in the interview. If you’re doing a usability test, run them through the process. A lot of people have never heard of or been through a usability test. It’s good practice to remind participants that there are no right or wrong answers and that we’re here to listen to all feedback.
If you’re thinking of recording the interview, ask the participants for permission before hitting the record button. For a usability test, you’ll want the participant to share their screen before you start the test.
Ask better Interview questions
Start with warm up questions that are easy to answer before you delve into the more specific questions. People tend to be more open about answering questions in more detail once you’ve established a rapport with them.
Some examples of warm up questions are:
- Could you please describe your role?
- What does your business do?
- Could you walk us through your average work day?
- What other apps or software do you use on a regular basis?
Once you think the participant is comfortable, you can start with some of the more involved questions. Remember to phrase these as open ended questions to encourage participants to give you more detail than you would otherwise get with closed-ended (yes/no) questions.
Here are some examples of how a closed question can be turned into an open question:
Closed questions → Open Questions
Do you use this feature? → How do you use this feature?
Do you like this feature? → What do you think about this feature?
Have you done this before? → Describe your experience with this
Was that difficult? → How was your experience with that?
Usability tasks
Being remote means there might be plenty of distractions for the participants as they go through your usability test. This sometimes makes it hard for the participant to stay focused on the task at hand.
Reading the task out loud to the participants is a good way to get them actively thinking about what it is that you would like them to do. It’s also useful to give participants the ability to refer back to the task if they need to be reminded of what they need to do. An easy way of providing that is sending a written version of the task via the chat feature in zoom.
If a participant gets stuck or confused midway through the task, try using facilitation methods such as echo, boomerang and Columbo to avoid giving away hints or asking leading questions.
Here’s a brief example of each of these methods:
- Echo
When a participant says something like “Interesting…” you might follow up with “interesting?” inviting the participant to elaborate further - Boomerang
When a participant asks something like “Do I have to login first?” you can deflect with “How would you normally do it?” so as to not give away the answer - Columbo
In an inquisitive manner you might say to the participant “you were hovering your mouse in this area…?” waiting and hoping that they might fill in the blanks
Finally, when writing your tasks, try to avoid using any terms that will appear in the user interface. This ensures that you’ve removed any hints that the participant might not get in a real life scenario if they were to attempt the task by themselves.
Finish the interview
If you’ve asked all your questions and or gone through all your tasks with some spare time leftover, it could be a great opportunity to ask the participant if they have any general questions or feedback. You’d be surprised at how much useful information you can gather with this broad question. It’s also a good opportunity to ask if the participant would be interested in any future studies.
Once you’re all done, thank the participant again for their time and inform them that their feedback is critical to shaping how the feature or product turns out.
Ask your teammates to stay behind for a debrief to share and discuss notes. It’s best to do this right after the interview while it’s still fresh in everyone’s minds.
Reflect & analyse
You’re done with the interview and you’ve had a chance to debrief with your team to collect their notes. Now you need a way to share it with the relevant stakeholders in a digestible format.
Find a home for your research
It’s a good idea to keep all your research in one place. I personally love using Dovetail as my home for all things user research. Dovetail makes it super easy to analyse, synthesise and extract insights from your user research interviews.
Share with stakeholders
Once you’ve synthesised all the notes into common themes and takeaways, collate all of it into a digestible format that can be shared with the team. A short report with a brief summary of key takeaways and next steps is usually popular with stakeholders.
