Part 1
Creating better research plans, part 1
Research is all about learning. Your goals should reflect that.
As a full-time design/UX researcher, I’m often asked for advice from others wanting to conduct their own research. The most common suggestion I give is to not rush the planning phase.
In the enthusiasm to speak with customers, too often I’ve seen plans become an afterthought… a hastily thrown together doc with poorly articulated goals; or a doc that focuses solely on logistics; or (worst of all) not documented at all.
No matter if you've got 2 days for research, 2 weeks, or 2 months, you should stop, pause, and plan. A good research plan clearly articulates and aligns your team’s priorities. From it, everyone should clearly know what you're hoping to learn, why, how, and what's out of scope.
The most important part of your research plan are your goals and questions: what you need to learn. This is where the majority of your planning time should be spent. You should define goals and questions before you even think about methodology.
As good old Albert Einstein said: “If I had an hour to solve a problem I’d spend 55 minutes thinking about the problem and five minutes thinking about solutions.”
You must focus on the what, before you can focus on the how.
A quick note on the difference between research goals and questions
In my plans, I include both research questions and a goal statement.
- Research Questions: these are the heart of your plan. They are what you need to learn; what you want to answer. They’re written in question form. When crafting my plan, I start with these first. (They are not the same as interview questions.)
- Research Goal(s): this is the TL;DR of your questions. It summarizes what you aim to learn in a few sentences. Goal statements typically start with a verb like Assess, Discover, Document, etc. and are declarative.
example
Research Goal: Assess the current experience for new users trialing our product. Document friction, confusion, and opportunities to improve the experience.
Research Questions:
- What questions do new users have before signing up? Concerns?
- How does the current onboarding experience fit users’ needs and expectations?
- What emotions do users experience during the trial? Anxiety, curiosity, excitement?
- How do users learn to use our product? What aspects are easy to learn? What’s difficult, and why?
- [etc]
Other researchers may leave out the goal. I prefer to have both for different stakeholder needs. In my experience, some stakeholders are quite invested and want to get in the weeds; others may just want the high-level details.
Yet crafting research goals and questions can be deceivingly complex. In Part 1, I’ll share tips on writing goals. I also discuss the overall mindset you should adopt as you plan.
In Part 2, I’ll share tips on crafting research questions and getting team alignment.
Part 1. The difference between goals, research goals and business goals
When teams hastily throw together research plans, they often state goals like:
- Conduct 10 interviews with runners
- Validate our decision to build <feature>
- See if users prefer version A or B
Unfortunately, these aren’t very good research goals at all. Let’s unpack why.
Research goals should center around learning
If you’ve ever made a New Year’s Resolution, you’ve made what I call a ‘basic goal.’ Basic goals are great. They get us motivated; they challenge us; they throw down the gauntlet.
Here’s a few basic goals:
- Basic goal: Save up $150 to buy new Nikes.
- Basic goal: Run 5 times a week.
- Basic goal: Train and run the Boston marathon in April.
Basic goals are all about accomplishing something specific.
However research, isn’t just about completing a task. It’s about learning, discovering, identifying, uncovering. There’s information that you don’t know, and you’re trying to find it. There’s something you need to learn, and you’re going to go learn it. That’s what your research goal should articulate.
“Just as you need a clearly articulated problem to create a solid design solution, a useful research study depends on a clear problem statement. In design, you’re solving for user needs and business goals. In research, you’re solving for a lack of information. ”
— Erika Hall, Just Enough Research
Here’s a better research goal:
- Research goal: Identify motivating factors for novice runners training and running in their first marathon.
This goal clearly spells out what you need to learn. It also includes specifics such as “novice runners” and “first marathon.” That helps narrow your scope and keep you tightly focused.
A common mistake is including your methodology in your goals, for example: “Conduct 10 interviews with runners”.
Here’s why this is problematic: what if you conduct 10 interviews, and you still haven’t learned what you need to? You still don’t feel confident making design or strategy decisions. Are you done? Should you hang up your hat and move on?
No. You should probably pause, and assess if your methodology, participant criteria, or discussion guide should be changed. Or maybe you need to conduct more interviews than you initially expected.
Research should always be a quest for information, knowledge, confidence — rather than ticking a box. By decoupling method from purpose, you ensure you and your team are focused on learning rather than simply executing.
🚧 How to fix it: If your goal includes a specific methodology, think back to why you proposed it in the first place. What did you want to learn, and why? Focus on the information and confidence you seek — are there are other methodologies you should be considering to help answer your questions? Maybe even a mixed methods approach would be best. Perhaps there’s existing data you can incorporate, like previous surveys or analytics.
Research goals aren’t business goals
Another common mistake is writing your desired impact of the research as the goal itself.
For example, your team might have a goal of increasing monthly active users by 50% by the end of the year. You write:
- Learn how to increase monthly active users. (Poorly written goal)
Another common example: say your team’s spent a lot of time brainstorming a new feature. You’re really hoping this research proves it’s a good idea. So you write:
- Validate our idea to build <XYZ feature>. (Poorly written goal)
Your research goal must be about what you want to learn from from your participants and data. Not what decisions you want to make afterwards.
If you’re not used to thinking this way (ahem, product managers), this might feel a bit like re-wiring your brain.
One tip: Take with your business goal and then backtrack. Envision yourself speaking with a participant, or reviewing all the session data afterwards. What kind what kind of unprompted behavior, actions, or reactions would you need to see from your participants?
For example, perhaps you realize you need to hear that users are experiencing a lot of pain around a specific task. It’s taking them a lot of time to do it, and (unprompted) they express a desire for alternate solutions.
So then, your goal becomes something like
- Document users’ current workflows with <related task>, including workarounds. Assess what aspects are most painful /time-consuming/ frustrating, and why.
This might sound pedantic to you. You might say, I know the difference between what we want to learn and what we want to do with the information. But does everyone on your team? Do all your stakeholders?
And here’s the bigger issue: If you’ve crafted your goals around business outcomes, it’s all too easy to slip into the bad habit of asking participants leading questions, or asking them to predict their future behavior.
Then suddenly you wind up asking a participant something like “How valuable did you find that new feature? Would you use it if we shipped it? ” Shock. Horror. This breaks research rules on so many levels. Please keep business goals and research goals separate. Focus on learning, rather than validating.
“[Research] will uncover potential areas of improvement or unmet needs, but it will not answer the question “What exact product should we build to beat the competition?” It is not a customer’s job to innovate products.”
— Sam Ladner, Practical Ethnography
Going into research sessions, everyone on your team should know you’re not making decisions… yet. That way, you can observe and soak in the details of the participants, their worlds and their current experience — rather than seeking validation or direction. That part comes after analysis!
🚧 How to fix it: Create a separate ‘Impact’ section in your research plan, which specifies what will be done with the findings. Work with your team to backtrack: what do we need to learn, see, observe in order for us to make this decision? How can we design the study to get unbiased responses from participants? If the business stakes are high, it may be worth asking a neutral third-party to review your plan and/or moderate sessions.
Think open, not closed
When conducting an interview, the golden rule is to ask open, not closed questions.
- Closed interview question: “Do you think our product is too expensive?”
- Open interview question: “What are your thoughts on the price?”
In my opinion, the same goes for research goals.
- Closed, not so good goal: Learn if users can successfully buy items using our new check-out flow.
The problem here is it sets up a “pass / fail” scenario. The implicit options are:
a) yes, participants can buy items
b) no, participants can’t buy items
Not great. Because research is all about learning, you need to be prepared to find out things you weren’t expecting. 99% of the time, findings come in shades of grey. Let’s open this up a bit.
- Open, better goal: Evaluate what aspects of our website help users successfully buy things. Assess what aspects slow them down, or prevent them from purchasing.
Now you’re open to finding out things like perhaps your participants could purchase items, but they misunderstood the shipping options. Or one step was quite painful, but they soldiered through it. Or several participants commented that they wanted longer descriptions, which was unexpected but quite valuable.
Research isn’t QA. You should be trying to learn, not to prove or validate. I know, I know this is easier said than done. You may be a designer or product manager who’s spent months on a feature. Naturally, you’re heavily invested in seeing it shipped. But putting on your blinders and ignoring “shades of grey” findings only hurts your users. So go in with a beginner’s mind, ready to find anything and everything that increases your understanding of the user experience. (Even if you didn’t change that part of the flow, or another team ‘owns’ it.)
Writing your research goals in an open way puts you on the right mindset from the beginning.
✅ DO use words like:
- How. Explore how users plan…
- Why. Document the reasons why …
- What. Assess what the biggest… What aspects…
- Where. Evaluate where…
- When / at what point. Evaluate when users encounter…
- Compare / contrast. Compare version A with B to understand…
⚠️ Watch out for:
- Like / prefer. Do users like A or B? Do they prefer us or our competitors?
- Or. … Do users do this or that?
- Starting with “do”. Do users …?
- Can. See if users can …?
- Will. Will users be able to … ?
- If. Find out if, See if..
🚧 How to fix it: Stress-test your research goal. Does it set up a pass/fail scenario? Envision future “shades of grey” research findings and what they might look like. Where might you see some variability? How can you re-word your goal in a way that captures that variance? If you’re stuck, try using words from the list above.
By now, you should understand research is all about learning.
Your goals should focus on what you’ll learn from your participants: not your methodology, or what you’ll do with your findings afterwards. And you know to plan for “shades of grey” findings by crafting open goals.
Your research plan is already much stronger!
Thanks for reading Part 1. Part 2 will cover defining your research questions.
