BEHAVIOURAL DESIGN FUNDAMENTALS: PART 3 OF 3
The Behavioural Design framework of test and learn in 4 steps
Using a Behavioural Design process for testing and measuring ideas for making a positive impact on the user experience.
Whether you’re a startup, an agency, or a specialist team within an organisation, knowing if your ideas will have an impact on your customers is the big question. Even when you think you’ve designed a revolutionary solution that will change or shape behaviours, you’re going to need proof that your idea will work! Without any testing nor learning, you’re going to risk hurting your customer experience and business value.
In the third and final chapter of Behavioural Design Fundamentals, we’ll observe how we can test and measure our ideas for building the right long-term behaviours. And by doing so, we can work towards creating customer-centric products or services that will have people hooked.
Test and learn in 4 steps
In part one, we observed Behavioural Design by definition, benefits and ethics. Then in part two, we unpacked a toolbox of ideas and techniques which can help us better understand and shape human behaviours in products or services. Now we’re going to focus on how we can test and measure solutions by using a four-step framework.
What do we mean by test and learn?
Designing for behavioural change involves an agile and iterative process of test and learn. This means testing with potential solutions and measuring for success. Whether the outcome of an experiment is successful or not, evidence-based findings will help validate ideas, inform decisions, and guide the project team.
Avoiding a process of test and learn, however, can result in building products or services which are unfit for user needs. Consequently, more time, money, and precious resource will be spent on fixing problems.
Using a structure
To help validate ideas and measure for success, we can rely on the framework of test and learn. Inspired by Design Thinking and other iterative UX processes, the structure helps teams understand users, define problems, and test and measure innovative solutions for creating long-term behaviours.
The framework is an agile and iterative learning and development strategy that can be used for creating products or services with long-term behaviours
The framework of test and learn is also collaborative and can help the project team make informed decisions to achieve customer and commercial objectives. Now let’s take a peek at how the framework is structured…
At a glance, the framework includes the following steps:
- Empathise: develop a deeper understanding of your customer
- Define: specify the problem or opportunity
- Test: experiment with potential solutions for behavioural change
- Measure: analyse the experimental data
The sequence of steps starts with gaining an empathetic understanding of our customer’s needs, wants, frustrations, and motivations; these insights will help define the objectives and overall strategies for behavioural change. Then, the phases in test and measure involve experimenting with ideas and learning from our observations.
Once we reach the end of the sequence, we can refine our ideas or explore new ones by revisiting the steps as part of an iterative cycle.
Sound good? Let’s breakdown the framework…
1. Empathise
The first step is about understanding everything we can about the individual or group of people we are designing for. Who are they? What are their current behaviours? What is the painkiller we are trying to solve for them?
To gain a deeper insight into our users, there are various empathy tools and techniques we can adopt:
- MI and data: quantified information on segments (groups of users), and behavioural trends can provide us with a holistic view of where the opportunities and struggles exist.
- Interviews: help us gain qualitative information on the target audience, including motivations and frustrations.
- Surveys: traditionally used as a method for obtaining feedback on customer satisfaction, but they can also be used for learning about opinions and trends.
- Empathy maps: capture a deeper insight into our customers, including what they think, feel, say, and do. They also serve well as collaborative tools.
- Personas: a representative of our customer. They provide a quick reference for all teams including product and development.
Scenario
To put empathy into context, let’s imagine we’ve interviewed some working professionals to learn more about their everyday activities. Insight reveals that Lucy, a particular subject, is often distracted by push notifications, messages, and apps on her smartphone while at work. It’s incredibly hard for her to not get distracted, and it’s hard not to turn off completely.
Let’s keep this insight in mind as we’ll be revisiting the scenario throughout the next steps…
2. Define
Whether we’re looking to influence new behaviours or shape existing ones, we must first understand the reasons for doing so. Are we trying to solve a known problem? Or is there is a new opportunity for change?
Define the customer and commercial objectives
Rich customer insights can provide us with guidance on the type of ‘painkiller’ that’s missing in peoples’ lives, while key stakeholder interviews can inform us of the desired commercial objectives.
When defining our objectives, we must consider if they are specific, measurable, attainable, relevant, and bound to a deadline. Once we have our goals, we need to identify some success measures to let us know if we’re achieving our objectives or not.
Define the success measures
We can use Behavioural KPIs (key performance indicators) as values to measure success. They are the essential steppings stones that bridge our learnings to our objectives.
Behavioural KPIs will play an important role in the process of test and learn. They are the crucial levers we can pull to generate behavioural change. When combined with the customer and commercial objectives, we’ll have a strategy to test.
Scenario
Now let’s reflect on our insights gained from the empathise step. We know that our persona Lucy struggles to focus on key work-related activities due to habits of unconscious smartphone use. This discovery tells us that there is both a problem to solve and an opportunity to meet.
We could define the problem and opportunity as:
- Problem: reduce smartphone usage while at work.
- Opportunity: introduce a product that can help increase work productivity.
What Lucy needs is a way to replace existing habits with new behaviours. We don’t explicitly have the solution yet, but we can define a Behavioural KPI that can read something like this…
Reduce smartphone usage during working activities by at least 20%
Now we need to test some ideas to meet our Behavioural KPI…
3. Test
Testing for behavioural change is at the nucleus of the framework. This involves experimenting with different ideas to learn if they work to meet our Behavioural KPIs.
Knowing what to test requires a creative process of generating ideas (also known as ideation). Once we identify some potential solutions, we can drive them forward by forming a hypothesis.
A hypothesis is simply a structured idea we want to test. It can be written as a statement for the theory behind our proposed change for behaviour. Take the following example:
By adding soft incentives to our signup form, we can increase the number of subscribers
Form a hypothesis
Dr Nick Fine, a lead user experience researcher, describes a good hypothesis as something that supports creativity, informs innovation, and feeds ideation. And what’s more, we don’t need a PhD to create a one.
To form a hypothesis, we can follow Dr Nick Fine’s simple framework comprised of three parts: action, change, and measure.
- Action (X): the synthetic adjustment
- Change (Y): the target behaviour
- Measure (Z): the metric we track to prove the hypothesis true
A written structured hypothesis may look something like this:
By adding or altering the action (X) will influence change (Y). We will know if this is true when we achieve the measure (Z)
Determine the right testing method
Choosing the appropriate testing method depends on the hypothesis we want to prove with the behaviours we want to measure. Here are some possible qualitative and quantitative research techniques we can adopt:
- Usability tests: test for natural user behaviours in interactive prototypes or live environments.
- Click tests: measure the time and density of clicks to understand how discoverable content is.
- Tree tests: determine the navigation paths users are likely to take to understand how findable information is.
- Timeout tests: gauge how discoverable and memorable your design is.
- Card sorts: test for mental models to understand how people perceive and expect their environment to work.
- Live tests: A/B and Multi-Variant Tests (MVTs) are great for experimenting with large sample sizes. Compare one design against another to measure the changes.
When building a test, we’ll need to figure out an appropriate sample size and segment to experiment with. Testing our ideas with a small group of people using qualitative methods, such as usability studies, will give us quality insight into behaviours. However, scaling up the numbers using quantitative techniques, such as an MVT, will enable us to compare and measure changes with real customers.
Scenario
Let’s imagine we’ve generated a potential solution aimed to meet the Behavioural KPI we defined from the previous step.
Our solution is to enable our target user, Lucy, to access a personalised workspace on her smartphone, which contains only the chosen applications she wants to interact with during work. To activate the environment, Lucy just needs to tap her device on a totem — a physical object that sits on her desk.
This target behaviour essentially activates a distraction-free environment, which enables Lucy to work more efficiently yet without missing anything important, such as an important message from her loved ones.
Now let’s write a hypothesis to go with it…
By providing users with a trigger to activate a personalised workspace, will influence them to decrease their smartphone usage during working hours. We will know if this is true when device usage is reduced by 20%
With a hypothesis formed, we can experiment with some solutions using the appropriate testing methods. Once the results are in, we’ll need to find a way to measure them…
4. Measure
We’ve now arrived at the final step of the framework: measure. This involves analysing the outputs from our experiments and measuring for success. Have our changes worked? Are users performing the intended behaviours? Are we meeting our Behavioural KPIs? And what actions do we take next?
Measure for success
When we observe the results from our experiments, we’ll want to understand if our changes have influenced people to perform the target behaviours. We’ll also want to check for attitudes, such as perception on ease of use, appearance, and if the product or service is regarded trustworthy.
If our Behavioural KPIs are working and prove the hypothesis correct, then we can consider scaling the sample size or experimenting with a whole userbase. Factor in an A/B or MVT if we haven’t already tested with live customers; this will provide us with quantitative data to help back up our ideas.
If our changes are not working or even cause detriment to the customer experience, then we should consider rolling back and revisit the hypothesis. This isn’t all bad news though, as we’ll still learn something valuable that we can use to develop our ideas. The more we learn, the more we’ll succeed.
Use a benchmark metric
When we experiment with different variations, we’ll need a way to measure changes consistently and quantitatively. One effective way of doing that is by using a single benchmarking metric to track our progress.
A benchmark metric can be a combination of measurements in behavioural and attitudinal behaviour from our experiments. With each iteration tested, we can refer to the point of reference to gauge if our changes are working through our research efforts.
For an example of a benchmarking metric, take a look at remote research specialist, UserZoom’s QXScore.
Use learnings to develop ideas
We can use our learnings as a fundamental tool for developing our ideas and growing an overall strategy. Any information we obtain from previous experiments can be stored as insights and shared with the project team.
Even when a hypothesis is proved correct, we can continue to develop our ideas by undergoing the same steps within the framework of test and learn. The more we can enhance our products and services, then the more satisfaction, delight, and meaning we can add to our customer experiences.
Scenario
Now let’s see how this all comes together by visiting our scenario once more. We’ve tested our smartphone workspace idea and now we want to prove the hypothesis correct by comparing the results with our Behavioural KPI.
If users like Lucy are reducing their smartphone usage during working hours then we can consider the experiment a success. We can use any feedback (positive or negative) to determine the areas that can be improved, which can lead to further iterations and testing.
We can also choose to revisit any of the steps within the framework to develop our ideas for behavioural change. The more we empathise, define, test, and measure, then the more we can learn to enhance the customer experience.
The Behavioural Design framework of test and learn can guide us in developing and refining our ideas. It’s agile, iterative, and extremely beneficial for learning.
Whether a hypothesis is proved successful or not, we can iterate our ideas by revisiting the four steps within the framework. And by doing so, we can continue to innovate new ways of building better and more meaningful customer experiences.
Takeaways from the series
We’ve now reached the end of the Behavioural Design Fundamentals series. To wrap it up, here are some key learnings we can take away…
- Behavioural Design is a creative and scientific approach to building and attaining long-term behaviours for products or services.
- Ethics must be at the forefront when designing for behavioural change.
- Ideas and techniques can be adopted as ‘thinking tools’ for generating potential solutions.
- A process of test and learn can help validate different ideas and guide product teams. By adopting a framework, we can make sure we’ve covered the right steps.
Special mentions
- Behavioural Design framework for test and learn inspired by Boundless Mind, Digital Behavioural Design
- Hypothesis testing by Dr Nick Fine
- Scenario idea based on Let Me Think
- As always, invaluable help and insight from Paul Humphrey
- Kawaii icons (used in all illustrations) from Freepik
