Shadowing & dogfooding… or how we research for Monzo’s internal product

I’m a product designer at Monzo, working on the internal product that supports Monzo’s customer support (or as we prefer “COps” 👮‍♀️) team. In my last article I wrote about designing for Monzo’s internal product.

In this article I thought I’d share more about how we research for the internal product. Specifically, how we go about gathering feedback from our colleagues without overly distracting them from their work!

Research for internal products

One of the best things about researching for internal products is that you have your users at your fingertips in their ‘natural environment’. Gone are the days struggling with recruitment, incentives and recreating realistic conditions.

But then again, you don’t want to disturb your colleagues too much with requests for feedback and testing sessions that might overly distract them from their work. Especially somewhere like Monzo where the team is incredibly busy and change is the only constant!

So gathering research and feedback in a light touch way is essential. In this article I’m going to talk about some of the methods we use at Monzo to get research insight, with minimal interference.

Shadowing

The first method we use is “shadowing”, a type of contextual observation where you sit with users and see how they’re using a product in their natural environment. We use shadowing to gather any general painpoints (and spot opportunities!), and observe how COps are using specific features that we’re currently working on.

We either sit with COps (in our London and Cardiff offices) or set up a hangout where a COp shares their screen with us so we can see what they’re working on. We ask them to try to “think out loud” so we can follow their work process, and note anything that they find particularly good or frustrating.

The sessions usually last an hour to give a fair idea of how someone is working. The first 5-10 minutes are an “ice breaker” — to put the person at ease and explain why we are watching them work (the key is to explain we’re testing the product NOT them). Most of the rest of the session is for observation and we focus on taking notes of what the COp is doing.

Our aim is that our users forget they are being watched so we generally stay quiet apart from some encouraging noises! As we make notes we flag any questions in bold so that we can come back to them in the last 5-10 minutes of our session — rather than jumping in and distracting their flow.

What’s great about shadowing is that it’s really easy to do and everyone can get involved. Most of the engineers (and all the product managers) in our team have jumped in to help, with just a little bit of guidance on setting up and running sessions. Not only does this help with research efforts, but it means that the whole team gets to know the product better and build empathy with our users. And COps are also less shy about approaching the team with other feedback or ideas 🙌

Dogfooding

The second method is “dogfooding” or “eating your own dog food”, which means using the product that you are designing/building, to understand what it’s like for your users. It’s a great initiative that one of our Product Managers has always pushed the team to do— to stay in touch with COps and our customers.

We have an hour session once a week where the whole team jumps on our product to answer customer queries. We usually sit with some COps so that they can guide us if we get stuck (it’s not an easy job!). Again, these sessions help us see where potential problems and opportunities lie. We also understand our users better, by getting an idea of what challenges they face on a daily basis.

Card sorting

The third method we use is “card sorting”, where participants organise cards (with information or images on them) into topics based on their mental model of a product or topic. It’s often used to organise the navigation of an app or website.

At Monzo we’ve started using it to solve issues around information density. One of the main challenges our team faces is that most product teams across Monzo contribute to the internal product (e.g. a team may build a lending feature that then needs mirroring support features that COps will use).

Because other product teams are focused on the customer experience they don’t have as much awareness of how COps work (that’s where our team comes in). This means they sometimes provide too much information to COps, or display things in a way that’s not always intuitive. We’ve been using card sorting to rationalise and refine information so that it’s more helpful to COps, and build a clearer information architecture that will help everyone contribute to and use the product.

We run very simple card sorting exercises, using pieces of paper (revision cards also work well) with information on which we ask COps to organise into groupings and prioritise. We also encourage participants to think out loud as they do the exercise as their comments are just as helpful as the results. This is an “open” card sort because we allow COps to organise them as they choose rather than into topics that we have set (a closed card sort has defined headings or topics).

There are loads of great tools to do card sorting digitally (e.g. Optimal sort) which we’ve started to use, which takes the leg work out of recording the results and allows remote sessions without you needing to be present. Although there’s something about the throwaway nature of paper that really helps participants be honest!

Feedback channels

Lastly, we encourage COps to report back to our team through Slack if they spot any bugs or problems, or have any ideas for the internal product. This means we hear very quickly if there are any big issues we need to fix, and we have yet another way of gathering insight into any painpoints COps are facing, and the opportunities they spot.

Avoiding creeping biases

Of course the risk with all of us being so closely involved in research is that we may influence the results. We all have biases, and it’s interesting to see how team members pick up on different feedback because of their own experiences and expectations, myself included!

However, we have a variety of different feedback sources that input into our decision making. As well as processes in place that help us select work that will have the most impact, rather than what we personally believe may be most important. We’re also very aware of our biases, and the fact that we are not our users. Our research team runs training for everyone contributing to research, which focuses on being aware of our biases and how to avoid them influencing our research.

—

We’ve recently hired a fantastic user researcher and I’m excited to see how our approach evolves now that we can dedicate more time to research.

Thank you for reading! If you enjoyed this article, please show me some love by clapping 👏