The lean research loop for digital products
At PwC Digital, iterative design research with humans is an integral part of our process to create products and services that make a human difference.
However, we’re seeing increasing demand for products to be delivered faster. This need for speed has been affecting our ability to think creatively and evaluate problems in fast-paced and often technology driven programs. So, as the speed to market accelerates, and the demand for working within agile teams becomes the normality…
How might we incorporate design research for digital product design in a more iterative way, so we continuously innovate to inform the outcome before it’s shipped?
Here is a 30 second snapshot of what this article covers:
- We’ve created The Lean Research Loop: An iterative, hypothesis and experiment-driven approach to validated learning, so we continuously innovate to create game-changing digital products and services
- It has four stages; Question and Hypothesise, Prototype and Experiment, Synthesise and Iterate, Release and Measure
- This article explores the 4 stages in depth and covers the key steps needed to get from hypotheses to new product features in market
- It’s a guide for Digital Product Designers to embed iterative, effective and repeatable design research processes into the fast-paced environments of agile teams, so they can deliver a continuous innovation cycle while sprinting.
If this is of interest, read on!
Introducing The Lean Research Loop
An iterative, hypothesis and experiment-driven approach to validated learning, so we continuously innovate to create game-changing digital products and services.
The Lean Research Loop helps solve two needs for Product Teams:
- How might we uncover new ways to solve existing/underserved customer needs, so we can deliver a digital product to market that creates a differentiated value proposition?
- How might we better understand the existing products ability to support customer needs, so that we can optimise its features to make a more successful experience?
What do we mean by a Product Team? For us, Product Teams are cross-functional (product, design and engineering); focused on outcomes (rather than output); and empowered to figure out the best way to solve the problems they’ve been asked to solve.
The Lean Research Loop is an approach we use for helping Product Teams solve the above statements through 4 stages.
The 4 Stages
We defined 4 stages to The Lean Research Loop for digital products which allows us (and hopefully you) to rapidly iterate, improve and innovate on your digital products and services.
Stage 1: Question and Hypothesise
During this stage, focus on uncovering the challenges and pain points customers are experiencing with your existing product or service. Use those challenges and any existing knowledge of customer needs you have to hypothesise how your customers might react to adjustments you make to your digital product.
Let’s look at an example…
Imagine we have a website that allows customers to ‘get a quote’ for a new mobile phone and data plan. We know:
- Website analytics show that only 50% of customers finish the existing ‘get a quote’ process once they start
- Previous customer research shows that the existing ‘get a quote’ flow has too many steps and requires too much effort.
With this in mind, we can explore a new ‘get a quote’ experience to resolve the challenges at hand. In order to ensure the new experience is better, we create the following hypothesis to support our approach…
“We believe that… the new ‘get a quote’ experience for customers will increase the number of successful quotes being completed through the website.
We will make a prototype that allows customers to complete a quote, then ask questions to understand how they found the experience, what needs they have when completing this task and why.
We will have confidence that the hypothesis is supported when more than 80%* of customers we test with are satisfied using the new ‘get a quote’ experience.”
*This isn’t 80% of the set of customers you tested with in one loop. This should be 80% across several cohorts of customers. This is important to understand so you don’t bias your learnings with just a few customer perspectives.
Hypothesising how you solve challenges is the most important part of The Lean Research Loop. Be comfortable with ambiguity and the fact that you will not have all the answers upfront to make all the right decision for your product. But fear not, the answers will come by experimenting with customers and learning how they interact with your products features.
With hypothesis in hand and solutions to support them, we can now begin creating our new prototype to experiment with customers.
Stage 2: Prototype and Experiment
We have our hypothesis completed and potential solutions defined. Now it’s time to design and conduct research experiments with customers to rapidly evaluate solutions, so we can prove out hypotheses (don’t forget you need to recruit customers!).
Next, create a scenario, stories that give customers context as to why they’re going to go through the experiments you’ve prepared. These stories will help them get in the right frame of mind as they progress through the experiments you created, so you the best possible learnings from your research.
Also, think about the space in which you’re running your experiments compared to how it would happen in the wild. Does it make more sense to run the sessions in the customers home? in a branch? at their place of work? outdoors? Think about how your customers would use your product, then try to emulate the experience as much as possible to give you the best possible research insights.
With the story created, you can then get into the nitty gritty of creating tasks your looking for customers to complete, so you can prove out the hypotheses you defined earlier.
Once you’ve identified the scenarios and tasks you’re experimenting, create a prototype that allows the team to run experiments with customers that will help them yield the right learnings. This would be your solution in the form of some kind of clickable interface that you can put in front of customers. This allows the team to get eye-witness evidence on how customers are reacting to the solutions they’ve proposed.
Lastly, create an Observation Framework that enables the team to rapidly capture and synthesise research into actionable recommendations.
We’ve used the wall of an observation room to place the experience we’re testing. Using different coloured post-it notes, we capture pain points, quotes, opportunities and blockers (urgent stuff that stops customers from completing the task).
With that done, you’re ready to run your experiments with customers, then begin your synthesis and iteration.
Stage 3: Synthesise and Iterate
If you’ve set up an observation framework, synthesising your findings shouldn’t be too difficult. What is important here is not the findings themselves, but the patterns you observed as you watched customers go through your experiments, whether your hypotheses were supported or refuted and why the outcome of your experiments came about.
Use the learnings to define a reporting framework that is easily repeatable and allows you to quickly summarise the and share back learnings. It shouldn’t take you more than a couple of days to report out findings. In the past, we’ve focused on:
- Confirming whether the hypothesis was proven to be successful or not and why
- What behavioural patterns we observed that affected the way customers interacted with a product
- What aspects of the products interface did customers struggle with when completing tasks
Creating Actionable Recommendations
No doubt you will have gathered lots of great insight and learnings from your research experiments. Because we’re working in a team that is continuously sprinting, it’s important that we describe the recommendations in a way that they can be embedded into your product backlog and tracked as work to complete.
We’ve used Job Stories (an extension on the Jobs To Be Done Framework) to convert learnings from research experiments into actionable recommendations that can be entered into the product backlog of work.
We’ve used a design kanban to capture these. The kanban helps us document, prioritise then tackle the design changes in collaboration with the rest of the Product Team.
Continuing with the example I mentioned earlier…
Let’s say during our research experiments, we learnt two things:
- Customers prefer to customise the amount of mobile data on their plans
- They want to see the price progressively communicated, rather than receiving sticker shock at the end of the process
Because the two customer needs are intertwined, we turned them into a single Job Story that says…
When a customer is getting a quote for a new mobile phone and data plan, they want to customise how much data they can have and the cost implications, so they can adjust the quote to suit their budget.
This allows us to convert our learnings into potential features, or the inception of further design exploration if the Job Story is a more challenging one.
With your learnings created in a way that can be easily embedded into your products backlog of work, all that is left to do is prioritise each potential feature to determine whether the business is going to accept the new features, or whether more consideration is required. We’ve typically used desirability, feasibility and viability as lenses to prioritise new feature requests.
With a prioritised digital product ready to hit the streets, it’s time to release and learn.
Stage 4: Release and Measure
Will your team pilot the new features, or take a big bang approach and release them to everyone that uses your product at the same time?
As a Digital Product Designer, sometimes these decisions go outside of your remit and you’re left to focus on the next challenge you’re looking to solve for your digital product.
What you should know is that the way updates to your product are released into the wild plays a critical role in how it will be perceived by your customers. When we design new experiences or evolve existing ones, we do it with the best of intentions for how we best support our customers. In the fast pace of digital product delivery, we sometimes forget that the experiences we make can result in unintended consequences.
If we used the example we went through in this article, we could assume the solution to improving the ‘get a quote’ experience might be to launch a progressive disclosure pricing feature. This would provide transparency to customers on pricing as they refine their product selection. But if we do this, we’re also creating transparency in the way the business prices its products. This could result in customers getting confused by the way pricing is modelled, deterring them from continuing.
There are lots of different approaches for how to minimise the negative impact new features or experiences could have to your customers. From A/B testing through to proof of value tests, find the right method to suit your products context and use it to ensure the releases you push to the wild are providing the outcome you desired.
Prove the Value of the Experience
As a final piece of the puzzle (but the most important one), we need to make sure we’re continuously measuring the impact of the digital products we’re making, so they’re delivering on the customer and business value we expected.
In the example we had earlier, the obvious metric to track would be whether we’re seeing an increase in the number of customers completing the new ‘get a quote’ experience. But in addition to that, we’d also want to understand the impact the experience is having further down the funnel (with purchases) and further up the funnel (in product discovery).
Having quantifiable data to back your qualitative research learnings will strengthen your ability to stand by the product decisions your team makes. If it’s providing strong business and customer value, you will be given space you need to be more innovative, more creative and ultimately have more freedom to deliver an amazing experience to your customers.
We did call this article The Lean Research Loop though. So with all that hard work done and dusted, it’s time to question and hypothesise again!
Conclusion
Setting up and running The Lean Research Loop for your digital products can be immensely useful for your team and broader stakeholders. Done well, you can deliver enormous amounts of value in very short amount of time by using lean methods to move from strategy into action.
I’ll leave you with a great quote from Jake Knapp, author of Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days who wrote:
The Wright brothers didn’t use sprints to invent the airplane. Forming a question, building a prototype, and running a test became a way of life.
Remember, every loop is an opportunity to learn. Just because your findings support a hypothesis in one sprint, it doesn’t mean it will be the same outcome in the next one. Human needs evolve, so our digital products need to continuously evolve with them. We created The Lean Research Loop to do just that.
References
A lot of the concepts, ideas and learnings to help put this thinking together were inspired by amazing knowledge from around globe, so please check out all these great references for deeper reading and context:
5 steps to a hypothesis-driven design process
https://www.invisionapp.com/inside-design/hypothesis-driven-design-process/
Time to talk about Research-Ops
https://uxdesign.cc/time-to-talk-about-research-ops-cbacd8446dbe
Lean UX: Applying Lean Principles to Improve User Experience
https://www.amazon.com/Lean-UX-Applying-Principles-Experience/dp/1449311652
Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days
https://www.amazon.com/Sprint-Solve-Problems-Test-Ideas/dp/1442397683
The Lean Startup
https://www.amazon.com/Lean-Startup-Entrepreneurs-Continuous-Innovation/dp/0307887898
How to Jira for Designers
https://community.atlassian.com/t5/Jira-articles/How-to-Jira-for-designers/ba-p/976054
Incorporating UX Work into Your Agile Backlog
https://www.nngroup.com/articles/ux-agile-backlog/
4 Types of User Research
http://ueberproduct.de/en/4-types-of-research/
5 Tips for Writing a Job Story
https://jtbd.info/5-tips-for-writing-a-job-story-7c9092911fc9
