USER RESEARCH

12 simple techniques for user interview questions

Formulating questions that foster more reliable answers

Slava Shestopalov đŸ‡ș🇩
Design Bridges

Aseasoned interviewer takes care of many things: builds hypotheses, chooses interviewees, schedules appointments, sets the stage, writes the script, and so on. Any of these preparations can go wrong, but a bad script undermines all efforts.

Originally published in Smashing Magazine.

The source of good questions is recognizing what you don’t know yet

If you are a designer who hasn’t interviewed users before and there is no dedicated UX researcher nearby, aim at crafting robust questions. Then, there is a chance they’ll smooth out possible shortcomings. Right questions don’t simply roll off the tongue, but it’s a handy skill one can train. Today we’ll focus on problem interviews that help capture a real-life problem your digital product is going to solve. You might already have a solution or its prototype, or you may want to build one.

Pitfall 1. Hypothetical questions

“I don’t care if people will use our new features,” said no budget owner ever. Investing in design and development, people want to make sure money will return. And direct asking, unfortunately, is not an effective way to check it, although it may intuitively seem a great idea.

Hypothetical questions breed speculated answers

In my practice, there were a lot of cases when people said they liked something but were reluctant to pay for it or utilize it later on. So, is there any method to make sure that a product or separate feature will be of high demand when implemented?

The interviewer asks a hypothetical question and opens the door to subjectivity

I cannot recall anything more relevant than referring to people’s experiences and examples of similar behavior from the past. For example, if your users don’t have a habit of saving articles for later in all their news apps, what is the chance they’ll start doing it in your app? As Jakob Nielsen said, “Users spend most of their time on other sites.”

Let’s recap:

  • Don’t overuse the questions like “would you
” “how probable is that
” and “what if there were
”
  • Try to find out if a person has relevant experience.

Pitfall 2. Closed questions

Closed questions appear from a natural human wish to be approved and gain support. However, in the interviews, they aren’t that useful. A yes-or-no question doesn’t provoke reserved people to talk and doesn’t help much to reveal their motives and way of thinking.

Closed questions don’t encourage detailed answers

To be fair, closed questions are not evil. For example, they can serve a handy facilitation technique to make a talkative interviewee stop and turn back to the point. Also, they can help to double-check the information previously received through open questions. But if your goal is to gather as much data as possible, open questions will work better.

The interviewer kills storytelling with a sword of closed questions

Let’s recap:

  • Aim at making your interviewee tell stories, not approve/reject variants.
  • Hot-fix closed questions with “how.” Instead of “Do you do X often?” say, “How often do you do X?”

Pitfall 3. Leading questions

The things considered polite in everyday conversations may be harmful to the efficiency of a user interview. Trying to help an interviewee with the options can guide them in saying what they don’t really think.

Leading questions disguise the truth

A user interview is not the most comfortable situation for the majority of people, and they try to pass it as quickly as possible and at minimum effort. As a result, people tend to agree with anything more or less close to the truth or with a socially-expected choice instead of composing their answer from scratch. That’s why it’s better to move one step at a time and build the next question upon the answer to the previous one.

The interviewer deceives her idea behind a question

Let’s recap:

  • Don’t try to help your interviewee — it’ll discard answer reliability.
  • Better rephrase a question than suggest answer options.

Pitfall 4. Selfish questions

Idea authors sometimes act like proud parents — they want everyone to admire their child. The downside of such an attitude in user interviews is the unconscious usage of the pronouns “we” or “our.” As a result, users feel as if they are taking an exam and should either adore what they see or maintain neutrality, disguising real complaints.

Selfish questions make interviewees conceal negative feedback

In your interview script, replace possessive pronouns with neutral words like “this site” and “that application” or just call a subject of conversation by the name. Pro tip: as an interviewer, you can try hiding or understating your title and relation to the topic. For instance, before discussing digital products, don’t emphasize you are a designer.

A polite user lies to the selfish interviewer

Let’s recap:

  • An interview is not a demo. Don’t show your relation to the topic.
  • Edit “I,” “we,” and “our” out of the question list and as much as possible from other materials shared with interviewees.

Pitfall 5. Stacked questions

There are many reasons why we ask stacked questions. It can be a human desire to be heard, the fear of being interrupted, or worrying that you might forget the next question while listening to the current answer. However, for interview efficiency, stacked questions are not an option. Interviewees often select the one they are more comfortable to answer to or the one they managed to memorize from the stack.

Stacked questions lead to confused and incomplete answers

Remembering questions shouldn’t become the interviewee’s burden, so it’s better to ask them one by one. And maybe the answers are so comprehensive that you won’t need some of the planned questions anymore.

The interviewer tears a user apart by too many questions

Let’s recap:

  • Ask one question (sentence with a question mark) at a time.
  • If you feel you need to ask about several things at once, better compose and ask one broader question.

Pitfall 6. Explanation instead of a question

Teams that work together for some time often establish own language and tend to bring it into the product they are building. But will users understand such words as “dashboard,” “module,” “inclusion,” or “trigger”? Explanatory questions put an interviewee into the position of a lexicographer and check what sense (if any) they put into brand concepts and expert terminology. For a designer, it gives insights into how the future product — a website, app, or self-service terminal — should speak to people.

Explanations don’t allow to learn about interviewees’ way of thinking

The opposite side of this approach is explaining it yourself and leading people before they have a chance to share their opinions. Think about this: in the interview, you are superior and can put pressure on users making them get your point. But will you always be there for thousands of users to explain how the product works? Probably no. So, it’s more efficient to discover people’s thinking styles and then create self-explanatory solutions rather than create something and push it in interviews.

The interviewer tries to persuade instead of asking

Let’s recap:

  • An interview is not a pitch: interview “sales” are fake sales.
  • If you feel an interviewee might be out of context, you either invited the wrong people or want to defend an unclear solution.

Pitfall 7. Question clutter

Open questions are great until you realize there are too many details to figure out. The best method in such a situation is storytelling — describing a recent or the most prominent experience. As a result, an interviewee talks about a real situation and is less inclined to compose a socially desired answer or summarize various cases.

Asking to tell a story might work better than an open question

Besides, storytelling gives the freedom to speak about aspects a person considers necessary. Usually, people start with or talk longer about the most crucial experiences.

The interviewer encourages a user to tell about his recent experience

Let’s recap:

  • When in doubt which aspect to start with, ask to tell the whole story.
  • Sometimes an optimal interview question is not a question.

Pitfall 8. Too theoretical questions

When you’ve figured out regularity or general attitude, it’s the right moment to ask the interviewee about an example. Recent-experience questions can fill in the gaps, which might have appeared while answering general questions.

One can check an interviewee’s answer by asking about a recent case

For an interviewer, it’s another powerful method to check if users aren’t accidentally exaggerating or dropping significant details.

The interviewer checks a general statement from a user

Let’s recap:

  • Encourage your interviewee to illustrate all significant thoughts with recent or most remarkable examples.
  • Try to find the real-life equivalents of the metrics you want to ask about (for example, interest — reading time).

Pitfall 9. Talking about what you can observe

If you are lucky enough and interview people in their “natural habitat,” it’s a perfect chance to see their work process with your own eyes. So, if there is an opportunity to ask a user to demonstrate typical actions — offline or online — you’ll gather tons of insights.

Screen-sharing might work better than a thousand words

It’s a chance to learn about users’ habits (including shortcuts and favorite programs), level of computer skills, software environment, and the way of thinking (mental model).

A user shows his work environment to the interviewer

Let’s recap:

  • Whenever possible, ask interviewees to show it rather than describe it.
  • Prepare a compelling explanation of why you need to see something with your own eyes; otherwise, you may get rejected.

Pitfall 10. Tolerating vagueness

Abstract nouns and adjectives, for example, “comfort,” “accessibility,” “support,” “smart,” or “user-friendly,” are probably the trickiest words in the language because everyone interprets them differently. When you hear abstract names, that’s not enough to document them as they are. These words require “unboxing” and only then can support design decision-making.

Abstract concepts shouldn’t be left unclarified

Want to hear my interview mantra? “Nothing is clear enough.” It’s my second favorite slogan after the classical UX phrase “It depends.” It means that you cannot be certain about the meaning of interviewee’s words if you hardly visualize a scenario behind them. The best way to unbox abstract concepts is by turning them into verbs.

The interviewer collects user’s insights as they go

Let’s recap:

  • Abstract nouns and adjectives are not helpful and actionable insights.
  • Don’t hesitate to unbox abstract words until you are able to imagine the scenario of what was happening.

Pitfall 11. Overlooking exaggerations

Generalizations like “all,” “never,” “always,” “nobody,” “often,” or “frequently” are as unclear as abstract nouns and adjectives. But the way to “unbox” generalizations is different — through quantifying. Basically, you ask questions about approximate numbers or proportions.

Exaggerations and abstract adverbs shouldn’t be left unclarified

An interviewee, of course, might not provide you statistics, but at least you’ll understand whether the user’s “very frequent” is about “more than a half” or “nearly 20%.” Another example: the same phrase “a lot” can mean “50 per day” for emails but only “5 per year” for critical cybersecurity alerts.

The interviewer unboxes vagueness

Let’s recap:

  • When you hear vague characteristics of frequency, probability, or quantity, don’t forget to figure out what they mean to a person.
  • To unbox a vague adverb, ask about an approximate number behind it.

Pitfall 12. Undervalued WH-questions

As a non-native speaker, I remember these questions from the English classes at school. The teacher often asked us to make WH-questions (What? Where? When? Who? How?) so that we could start a conversation and break the awkward silence. Nothing had changed from school times. Now, as a designer, I often use WH-questions as the main interviewing instrument.

WH-questions are, probably, the most useful interview questions

My favorite question is “why.” For the sake of politeness and a more friendly atmosphere, I conceal it behind the following phrases, “What are you trying to achieve when you
?” or “Can you please explain the reason/value of
?” This is how in pursuit of a root cause you can ask several “whys” in a row without annoying your interviewee.

The interviewer writes a lot of WH-questions

Let’s recap:

  • WH-questions help to understand all circumstances: who, what, where, when, why, and how.
  • A good interview script usually includes a lot of WH-questions.

Summary

The question techniques above are pretty straightforward and might not take into account the nuances of a particular conversation or interviewee. Of course, even the best questions won’t make all the answers automatically objective, but they can make information more reliable and actionable. All in all, it’s always on an interviewer to adjust according to the situation. Here are the three core principles if you are in doubt about particular questions.

Experience holds more truth than a hypothesis

That’s why it’s recommended to ask about cases from the past and similar examples from other areas of a user’s life.

Let them tell their story; your ideas can wait

The goal of an interview is to explore the truth. If you force one interviewee to support you, it doesn’t mean you’ll cope with persuading a thousand. Also, give preference to clarifying the unknown versus checking hypotheses — for hypotheses, a better method is prototyping and testing.

If you cannot imagine it, you don’t get it

In a series of 1–2-hour user interviews, it’s so easy to get lazy and pretend you understand what you hear. Try challenging the interviewee’s statements in your mind, “Did he say the truth? Do I know why she says that? What exactly do they mean telling me about it?”

Irrelevant questions kill the reliability of insights

Bonus: self-check exercise

Take a sheet of paper and write numbers from 1 to 15 in a column. Read the following question examples. If you consider a question formulated properly for an interview, put a “plus” (+) in front of the number. If you think it won’t work, put a “minus” (−). Correct answers are after “Recommended reading.”

  1. What are your work responsibilities?
  2. What are your main challenges with our web application?
  3. How do you start your working day?
  4. Do you usually publish just text or text with photos?
  5. What do you feel when the button is not working?
  6. Please tell me about your company: its business model, primary markets, KPIs, and organizational structure.
  7. Do you like the autosave feature?
  8. How do you use the “Smart Publishing” module?
  9. What are you trying to achieve by archiving drafts?
  10. Would you click on the “Get started” button if we add the note that the trial doesn’t require a credit card number?
  11. What do you think this button does?
  12. You’ve just said “user-friendly.” Please tell me more about the apps you use and consider to be user-friendly?
  13. How much will you pay for the ability to watch and restore previous versions of a document?
  14. What do you take into account to mark a project “Approved”?
  15. On which of the stages that you’ve just mentioned do you have the most critical issues?

Recommended reading

Self-check: correct answers

Interviewing is not a mathematically precise topic, so please don’t take the following answers as dogma. However, they can serve as a benchmark and help you to enhance your next interview script.

  1. + (Open question.)
  2. − (Has the word “our,” which is selfish.)
  3. + (Open question, a good conversation starter.)
  4. − (Provides options, a guiding question.)
  5. +/− (Open question; however, feelings are subjective and might not help to discover what a person tries to accomplish.)
  6. − (Stacked questions.)
  7. − (Closed question; has the word “feature,” which belongs to IT terminology.)
  8. − (Includes the branded name of a feature, a selfish question; but if users are familiar with this feature, it might not be a problem.)
  9. + (Open question about the way of thinking.)
  10. − (Hypothetic question.)
  11. + (Open question about the way of thinking.)
  12. + (Past-experience and unpacking question aimed at understanding what an abstract word means to an interviewee.)
  13. − (Hypothetic question.)
  14. + (Open question about the decision-making style.)
  15. + (On which stage = when; WH-question, although doesn’t sound like one.)
  • If you found this article useful, feel free to buy me a coffee ☕
  • Here is a reading list with all my design articles on Medium.
  • Let’s connect on Linkedin and Twitter.
  • Drop me a note, if you want me to present this or another design-related topic at a conference or meetup.
  • Bored by “how-to” design articles? Check my other blog where I talk about overlooked architecture.
Design Bridges
Design Bridges

Published in Design Bridges

Practical articles on all aspects of software design + dank memes

Slava Shestopalov đŸ‡ș🇩
Slava Shestopalov đŸ‡ș🇩

Written by Slava Shestopalov đŸ‡ș🇩

Design leader and somewhat of a travel blogger. Author of “Design Bridges” and “5 a.m. Magazine” · savelife.in.ua/en/donate-en

Responses (6)

What are your thoughts?