Fast & Simple User Research

Published in

UX Collective

11 min readOct 10, 2019

There are millions of apps on the app stores that are available to us and while some of them have been successful and users love them, most of them experience a drop of interest just a week after launch and they never become profitable. According to Forrester Research’s study, 70% of projects fail due to lack of user acceptance. [1] This is mainly caused by the incorrect assumption of the need that the app is supposed to fill for users and a poor user experience. Bain & Company’s says that even though 80% of companies feel like they delivered a great experience to their customers, only 8% of their customers agree. [2]

The importance of user research and its impact on UX quality and overall business performance is undoubtable. On the other hand, if you’re working for a small company or if you’re new to this field, it might be hard to get started. It feels complex, overwhelming, and time-consuming – academic studies and panel surveys takes months or even years. Also the bad excuses like “we already know our users” or “we have no time and budget” tend to come to mind easier than supportive arguments.

The truth is that user research might be super easy and with just a little effort, you can gain valuable insights. In this article, I will share with you several cheap and undemanding research methods that I use.

When To Do User Research

The short answer would be anytime. In every stage of your product development cycle, user research might be beneficial and bring new understandings to light that will further help your development. Even though your product development cycle might look little bit different, it usually consists of a subset or all of the following steps:

As you might have noticed, the user research and hence, the users themselves, enter the development process in three stages. First, you’re discovering what problem your users are facing and what the overall market situation looks like. Then, you’re validating your designed solution with real users and typically iterate a few times on it. At the end, after the product release, you evaluate whether it’s used as expected, whether it meets people’s expectations, and how it’s performing from a KPI perspective.

Let’s take a look at each stage in greater detail, as well as some examples of methods which can be used to get most out of it for as cheap as possible, and in as little time as possible.

I. Discovery Research

The key goal of this phase is to do an exploratory research study in your project’s context. Don’t rely on your stakeholder’s assumptions and proposed solution. They might have found an interesting niche market, but their vision might be inappropriate for the users that you’re planning to target. Before your team starts with any implementation, you should learn about:

Customers (understand their motivation and problems they’re facing, map their customer journey, see how they solve the problem now…)
Market (review competitors, and figure out the market situation and its potential,…)
Business (what’s the long term goal, where does the company want to be in 1 or even 5 years, how we’re going to measure the success…)

All of this will help you to design a better solution for your customers, and eventually, your solution will find its place in the market. Consider the discovery phase to be a standalone project and dedicate enough time to it. If done right, it will save you a lot of time later. All product team members should participate in the research process and the findings should be stored in a clear form documentation that is easily accessible by the entire team.

“No matter how good the team… if we’re not solving the right problem, the project fails.” – Woody Williams

While some research agencies are performing complex panels or focus groups that are often expensive and demanding, I believe that in startup environment, discovery research can be done in much cheaper and easier way. Here are my 3 favorite methods:

Secondary Research

There are two kinds of research: Primary and Secondary. By Primary Research, we mean organizing our own data collecting and processing it by ourselves. By Secondary Research, we mean utilizing already existing research.

Any analysis usually starts with Secondary Research because it’s fast and inexpensive. Look at your internal resources like analytical tools, reports, already executed surveys, and interviews, as well as external resources like statistical office reports, professional databases, scientific papers and studies, or public information about competitors and go from there. In a few hours, you might get all the data you need.

In-depth Interviews

Don’t get scared by fancy title. In-depth interview is just a noble name for a human conversation. With an interview, you can gain a deeper understanding of the motivation and attitude of your interviewee, but also their hopes, beliefs, and experiences as well. It is a qualitative form of data-collection, so it will help you to answer the “why” questions. Don’t use it for validation of statistical or other quantity hypotheses. To validate your hypothesis, speak with at least 3–5 people. Sometimes, you’ll need even more (10+) to get robust insight. Here are some good resources on organizing interviews:

In the case of a new product, I’d start with customers from your expected target audience, your competitor’s users, and possibly experts in the field. You should also interview your stakeholders and technical people to get all-encompassing and exhaustive insight.

Survey

I bet everyone has done at least one survey throughout their lifetime, right? In fact, the questionnaire is one of the most popular data collection tools. It’s mainly because their flexibility, inexpensiveness, and their easy evaluation. Surveys are typically used for collecting quantitative data, but they can be also utilized to obtain qualitative data as well.

It will help you find out who your users are, what they want to accomplish, and how crucial the problem is that you’re solving. Don’t expect open opinions and a deep understanding of their motivation and reasons. For this type of insight, combine your survey with the interview before (as an input for survey construction) and after (as an additional source of insights). Here are good resourcesto help you prepare a survey:

II. (In)validation Research

The second phase of your development process, where the user’s input is absolutely crucial, is the validation phase. After you ideate, prototype, or design some possible solutions you should always test it with your users. Notice, that word “or” in the previous sentence. It’s not necessary to wait till the design is fully functional and visually beautiful. You can start testing as soon as you have a first sketch or a simple screen wireframe.

The validation phase should be about challenging the design team, finding issues and weaknesses, and receiving needed input for additional improvement. After this phase, you should finally have an answer to these questions:

Did your design solve the original issue?
Which aspects of your design work?
Which aspects of your design do not work?
Is the information architecture right?
Is the functionality understandable and working?
Does it have pleasant look and feel?

Heuristics Evaluation

Don’t get me wrong. This method can’t be used as the one and only method, and you should always confront your design with real people, but using heuristics can help you further finetune your design and get rid of obvious nonsense.

Heuristics is a strategy for solving the problem using previous experiences with a similar problem, and this experience doesn’t need to be necessarily yours. Many smart people have already spent years with human-computer interaction, usability testing, and system design, so there is no need to reinvent the wheel. Instead, build on top of their experience. The simplest thing you can do to check basic aspects of your design is to compare them with the basic usability heuristics.

Usability Testing

After you spend a few weeks with designing your app, you can’t see it without bias anymore because you’ve already invested too much into it. The only way to find out if it really works is to test it with people out there in the world. With usability testing, you have an opportunity to observe your customers as they perform a real task with your prototype, see their reactions, emotions, and better understand their workflow.

The user testing can be conducted as soon as you have a low fidelity paper prototype. And can be very cheap,since you don’t need any special facilities or high-tech tools to do it. I’d paraphrase Garrett Hoffman: If you’re not embarrassed by your prototype, you’re testing it too late. In my experience, when the design is too finalized, people are more focused on colours and shape of buttons because it’s easiest part of the design to comment on.

Your first tests can be done in the spirit of guerilla: simply grab what you have, go to nearest coffee shop and exchange a coffee for a 10 minute talk about your prototype. For validation ideas, I recommend conducting several 10 minute talks. For investigating more robust prototypes, I recommend a few 30–60 minute sessions in the private meeting room or user-testing lab. For advanced testing, there are tools like invision or axure, which can help you to make your prototype look almost like a real app.

Airbnb Guerilla Usability Testing by Raghav Haran
The Mom Test Book by Rob Fitzpatrick

Remote Unmoderated Testing

Sometimes you need to quickly validate a little adjustment or an immediate idea and it’s not feasible to organize a full-featured user testing process in your office– especially if you have an international client base. Services like usabilityhub.com, optimalworkshop.com, or usertesting.com can help you with that.

First impressions matter, and if you need to test the clarity of your design on first sight, the 5 second test might be a good choice. The overall visual hierarchy and attractiveness of your call to actions may be tested by using things called first click tests. And a similar way you can test different visual approachesisby conducting a preference test. It helps you choose the best variation of a design by getting real people to tell you which one they like best. And the best part is that for each of these tests, you can set them up in 15 minutes and you can have valuable results in a couple of hours.

III. Post-Launch Research

After months of hard work,you’ve finally released your product. There is always a great excitement around a launch, and it feels like the end of journey. But the release of your product is actually a beginning. Now you have a chance to see real people interact with everything you’ve created and get their feedback.

Post-launch research is an essential part of a product development cycle which helps you to evaluate how successful you were and collect insights for further adjustments. It should help you to find answers on following questions:

Does it meet the users’ expectations?
What works and what doesn’t?
Is the product pleasant to use?
What features do your users miss?
Where are the stumbling blocks?

Customer Reviews Analysis

Read your App Store and Play Store reviews. They’re a great source of knowledge and feedback. If your product/feature launch was successful, you’ll have dozens of them just after a few days.

After you have a collection of reviews, categorize them based on sentiment and review type. Learn from negative reviews — these often complain about issues with elementary features that don’t meet the basic expectations or others ask for new features. Also, don’t forget about positive reviews — what made their author fall in love with your app and what features do they like to use? Perhaps, you could deliver this experience also to the other users. More than 20% of 4-star reviews include suggestion from a customers who want to turn 4-star review to a 5-star review.

How to analyze customer feedback and make it actionable

Data Analysis

When releasing a new product, many startups tend to make one of following mistakes: they either don’t measure anything or they measure everything. While the second approach might seem right, it’s equally bad. With measuring every irrelevant and unimportant interaction, you can easily drown in the sea of data which are often wrongly measured because developers didn’t have enough time to think it through.

“What gets measured gets managed.”

Focus only on measuring important conversion flows (registration and onboarding, key app actions, subscription,…) and metrics which directly influence your KPIs. You should be carefully following your app’s engagement and day 1 retention, since it directly influences your long-term retention. Some good tools for measuring your app’s performance are Firebase, Analytics, Mixpanel,or AppFigures — you can find more in this Ultimate Analytics List.

Exit Surveys

There is no better source of learning about you product’s issues than the minds of customers who stopped using it. You can easily reach them via email after they stop using your app for several days or when they cancel their subscriptions. In Spendee, we’ve also got specific surveys that are sent to users when they’re about to convert, but for some reason, didn’t finish.

“Your most unhappy customers are your greatest source of learning.” — Bill Gates, Microsoft

Keep your questionnaire short and be humble. Maybe they stopped using your service after a serious bug and now they’re mad at you. Also, keep the questions open-ended, so your customers can pinch what’s bothering them. Your assumptions might be wrong.

Other Resources

For more content like this, follow me on Twitter: @michallangmajer