How to create product design hypotheses: a step-by-step guide
(or, How to take down a rampaging HiPPO in one move)
So, you’ve decided to get your product team running Lean. You might even have a degree in behavioural science and understand the scientific method as if you were suckled on Karl Popper’s breast, but your team… not so much. Worse still, the minute you start talking about induction and deduction or null and alternate hypotheses everyone’s eyes glaze over and you realise that if this was an experiment you’d be rejecting the hypothesis that ‘Lean will help me make better products’ (and imagine trying to explain that you’d actually be accepting the null hypothesis). Let’s not do that. I’m going to simply walk you through the what, why and how of hypothesis-driven design with a step-by-step guide.
(Cheat code: There’s a one-minute guide at the bottom of this page).
What is a product design hypothesis?
Well, the first thing to accept is that no matter how much research you do, your product is just a theoretical solution to a human need or want that you hope will result in business success. The hypothesis is your guess at why a particular solution will succeed. Once you succeed, and people are buying and using your product it’s no longer a hypothesis. It’s a fact.
Where do hypotheses come from?
I’m lazy, so when it comes to why we need good hypotheses the answer is, crap in means crap out because alchemy is not a thing. We won’t learn anything useful if our hypotheses are not insightful and well-formed. A lot of people that take on the Lean approach seem to think that they should just get in a room ‘ideate’, then throw it out to customers and see what sticks. Do that and chances are we’re going to waste a lot of time and money. Good hypotheses come from good observations. If you want to be more than a one-hit-wonder and develop an ongoing process of product development and innovation you need to find significant problems to solve. This is what observation is for and (and that can include the observation based on experience that we often call intuition) and why research is essential. Once you understand the space you’re working in ideation can begin and it can be systematic and rigorous.
Starting vision
Imagine, for example, that you have discovered that a lot of people suffer back pain as a result of sitting at an office desk and you want to make a product that solves this issue. Once you have formed a good customer problem statement (turning your intuitions or observations into a rationale) and prioritised your project by mapping its risks, your observations and business goals need to be summed up in a clearly articulated vision of the future. Once you know the general direction you’re going, you can move into solving problems.
It’s okay to use your imagination
Don’t let being scientific mean that your team turns into a group of soulless empiricists who may have learned how to apply a Vulcan nerve pinch but couldn’t design their way out of a paper bag.
Imagination and intuition are essential and have a very important role to play. Use them to diverge, and create as many possible solution ideas as possible. These are your hypotheses.
The actual Step-by-Step Guide starts here…
Step 1: Imagine the change you want, and write it down
What will the world be like for users of your product or service once they have it? This is your outcome, it’s your grand design, your vision for the future in which your product or service is a huge success and peoples lives are transformed. For example:
People who use our product no longer suffer back pain. They regard the product as a necessity and a delight. In fact, significant numbers of customers order more than one and frequently request the expansion of our product range. They love the product so much they basically sell them for us, and in the first year of release, we have sold 300,000 units and have over 30 distributors nationally.
This is your outcome statement. It provides a point of focus moving forward and should direct your investigation in general as well as directing your choice of success metrics for each and every experiment. A well-formed vision will cover the product’s desirability, viability and feasibility.
Step 2: Why is the status quo is the status quo
Now that you know what the world will be like after you succeed you need to ask, ‘What’s preventing the outcome being achieved?’ That is, ‘why isn’t it already how you want it?’
If these causes were removed your outcome would already be a reality. These are the root of your hypotheses. There should be a few of these and they are often multi-layered. There are two parts to this, one is causes and the other is blockers. A cause might be that muscles seize when not moved, a blocker however can be behavioural or situational, like, ‘I’m too lazy to do regular exercise’ ;) This is important because we have to work on things we can affect. The problem we are solving might be that the poorest people in our community have no savings, one cause is low income, and that could easily reduce the ideation to increasing income. However, it turns out that often this segment spends a meaningful amount on lottery tickets. By going beyond simple causes and looking at blockers we will broaden our potential for solutions. And remember,
Behavior is the medium of design — Robert Fabricant
So, back to our back pain problem:
- People have bad posture when they sit at a computer which causes strain on spinal discs, muscles and ligaments
a) They sit too close to the screen
b) They sit too far from the screen
c) They sit at the wrong height relative to the screen - People sit for too long causing ligaments and muscles to tighten
a) They don’t move their muscles enough
i. They lose track of time
ii. They don’t have a clock nearby
iii. They‘re too focussed to check the time
iv. They are too busy to get up and walk around
v. They aren’t sufficiently motivated to move until it’s too late - People don’t use ergonomic chairs because they are expensive and think they are ugly
You get the idea. Write as many as you can think of, obviously it’s preferable if these are based on actual research, but not essential.
Step 3: Dream like a scientist, ideate like a lunatic
Using your imagination effectively doesn’t mean smoking weed and waiting for the muse to magically infer upon you the perfect solution. Conversely, you don’t have to try to be a genius, experimentation will do that work for us and make us look like Einsteins.
Consider each of the obstacles listed above one at a time. What new ways of doing things might we try in order to remove these obstacles? What are the alternatives to the way things are currently done?
Let’s take cause number 2. People sit for too long causing ligaments and muscles to tighten
a) They don’t move their muscles enough
b) They aren’t sufficiently motivated to move until it’s too late
c) They forget to move regularly
… as many as you can find
and ideate some solutions…
- Make the chair remind them to move. After sitting in the seat for 30 minutes make the chair vibrate enough to irritate them until they get out of the seat
- Make the chair massage the muscles that cause back pain. Add massage pads to the chair that activate on an appropriate schedule and provide the muscle activation equivalent to getting out of the chair
It’s very easy to start ideating when the problems you’re solving for are specific and granular enough to engage with directly. Broad and abstract goals are very hard to ideate on because the parameters are too vague. For me it’s like being confronted with a blank canvas, it’s all a bit overwhelming and it’s very easy to freeze up creatively. More importantly, you’re now a hair’s breadth from having a rock-solid hypothesis to test.
Step 4: Writing hypotheses
I’m sure all your ideas are amazing, just like mine ;). However, it is statistically possible that some are or less amazing than others. So at this stage let’s agree that it’s still all hypothetical whether or not our ideas will successfully remove the obstacles to our desired outcome. Which brings us to the next step, writing hypotheses.
Take all your ideas and turn them into testable hypotheses. Do this by rewriting each idea as a prediction that claims the causes proposed in Step 2 will be overcome, and furthermore that a change will occur to the metrics you outlined in Step 1 (your outcome).
For example:
Massage pads built into a chair that trigger on a schedule and provide the muscle activation equivalent to getting out of the chair will prevent muscle spasm that causes back pain.
which would be translated as
I believe that by adding massage pads to an office chair which provide the muscle activation equivalent to getting out of a chair every 30 minutes we will reduce back pain among office workers because the user’s muscles will be sufficiently activated.
(technically this is a prediction. It’ll help you design your test. Note also that it is specific, ‘every 30 minutes’, as frequency is obviously a variable and you don’t want to give up because it turns out the core idea is good but the frequency needed was actually 29 minutes).
The structure of your hypothesis
I BELIEVE THAT <my feature/product/solution>
WILL <direction of change><thing that will change>
FOR <target user>
BECAUSE<reason for change>
Technically the first two lines are a prediction and only the last part (reason for change) is the underlying hypothesis, but it’s practical for us to combine them. In product development, it’s fair to say you could leave out the <reason for change> because you only want to know that it worked and may not care how or why. However, if you approach it in this way and discover from your first experiment that it didn’t succeed, you won’t know why it didn’t work and will be far less effective in getting to the product that succeeds. It also helps to improve your experiment design because we need to isolate the variables quite explicitly.
Our high level product design hypothesis might read,
I believe that office chairs with massage pads on the lumbar and thorasic spine that activate every 30 minutes will significantly reduce back pain for people whose injury is caused by excessive immobility because it will provide sufficient oxygen and nutrients (blood flow) to the muscles most vulnerable to injury.
Great work, we’ve got our first testable hypothesis. Do this for the rest of the ideas and when you’re done, don’t try to prove these to be true. Do the opposite.
Step 5: Testing the right thing and testing the thing right
By now you’re probably a bit excited about my massage chair, I know I am. I can already see towering office buildings full of people moaning with pleasure as my massaging office chair edifies their working life. I know my experiment is going to prove me right. This is the best chair ever!
But, you’re probably going to need to convince a bunch of stakeholders that they’re opinions are wrong and you are right. This approach is key to transforming the mindset of everyone in your organisation to one that understands the difference between opinion and justifed belief. The HIPPO (Highest Paid Person’s Opinion) won’t know what hit him or her.
I’m not sure if I trust myself to design a fair test. Experimentation is not only about finding out if people agree with me, even if they’re customers, it’s also about understanding why something works. That way it becomes repeatable, it becomes valuable knowledge. Knowledge helps us avoid bad decisions, like investing millions of dollars in a chair that promises to fix back pain and make people more relaxed only to discover that it doesn’t actually fix back pain any better than what is currently being used. So we really do need to be rigorous in our testing, and one of the most important steps on that road is to make sure you know what you think you’re changing, and this means disproving the current belief or the status quo. To do that you need to know what the status quo is and make sure it’s measurable.
Our hypothesis is wonderful and promises to be a great alternative to the status quo of back pain, stress and general unhappiness. Let’s prove we can do better than whatever’s happening now. If you’re so confident your hypothesis is true then this shouldn’t be a problem at all.
Rewrite the hypothesis as follows:
Activating the muscles and ligaments with massage pads in an office chair will not decrease the prevalence of back pain in the target group.
or if there is a competitor that people believe works your null hypothesis might read,
Activating the muscles and ligaments with massage pads in an office chair will not decrease the prevalence of back pain in the target group more than the current method of informing people they need to stand up every 30 minutes.
The reason is that this will direct the design of the experiment to focus on whether or not the solution has a measurable impact or not. It means you’ll need to have a measurement for the status quo, and this is the metric that you are hoping will change when you test your new chair. If it doesn’t change then you need to go back and alter your original hypothesis and try again. It’s as simple as that. If you can’t measure the status quo your hypothesis is technically invalid because it’s not testable. If this is the case, and the impact of a bad investment is large, you will need to go back to either step 3 or 4 and write a new hypothesis.
Warning: Avoid the trap of the interesting but not impactful
For product innovators, we usually just care that it works, not neccessarily that it’s more effective than our competitors so we usually just test against not having our product. More importantly though, don’t get trapped into trying to discover things that are interesting but not impactful on the design decisions themselves, wasting time trying to get incremental improvements in accuracy.
The role of all research (including experimentation) is to reduce the risk of bad decisions. Remember, speed-to-market is a critical factor in product success. If we waste time answering irrellevant questions or trying to get perfect answers we are likely to increase the risk to our product’s success by being too slow.
Amazing work, we’re ready to start designing our experiments.
Damn, that looks like a lot of experiments
Remember when I told you I was lazy? It’s still true. We don’t have to test every single idea, and considering resources have limits for most of us, it’s important to reduce the risk of failure by prioritising. I created Uncertainty Mapping to prioritise my experiments.
The 1-minute guide
- OUTCOME: Describe the ideal end state. What is the perfect world you’re aiming to create by bringing your product or service into existence? How will you know (what will you literally see) when this is achieved?
- OBSTACLES: What are the things that research shows, or you believe, are preventing the outcome being achieved? That is, why isn’t the world already how you want it?
- ALTERNATIVES: What new ways of doing things might we try in order to overcome these obstacles?
- HYPOTHESES: Make a prediction based on each alternative under #3 that claims a change will occur to the metrics you outlined in it and in #1 (your outcome).
- NULL HYPOTHESIS: Write the counter-argument to your first hypothesis. If your hypothesis claimed that something will happen, replace the word ‘will’ with ‘won’t’. Design your experiment to disprove this statement.
- PRIORITISE: Apply the uncertainty mapping tool to your hypotheses to prioritise them.
7. EXPERIMENT DESIGN: Let’s talk about this another time, I’m sure I’ve pushed my luck getting you to read this far (cheatcode users excepted).
Thanks for reading, hope it’s useful.
(after several years and thousands of reads I’ve made some small edits to fix some grammar and hopefully improve clarity here and there. Thanks to all the readers, clappers and followers, it really means a lot)
