8 unexpected benefits of conducting user tests with colleagues

User tests with colleagues can provide useful insights while increasing transparency and building relationships that are essential for product development.

Forgive me UX gods, for I have sinned. I have started doing user tests with colleagues. I know what you’re thinking: this leads to no good. Employees are biased, they know the product, have a vested interest in it, and testing with colleagues inevitably leads to incestuous thinking patterns that can catastrophically influence product development.

I am not trying to argue with that; in fact, I fully agree. While I am strongly against only testing internally within the company, I am convinced that there is a time and place to test with colleagues and it comes along with some underappreciated perks.

We ran the first round of user tests with colleagues a few months ago. This came after we did a round of user tests with users, in which we tested a prototype. We had convincing indications that the design was easy to understand, but we did have a few open questions. We needed input on how to go forward quickly and given that we already had a promising first round of tests with users, I suggested running some user tests with colleagues in the office. I went over to our colleagues in sales and human resources and spent a few minutes going through the prototype with them, focusing especially on the open questions following user tests, and aspects of the design that might be different on mobile.

Given that we already had a round of insights from users, and directions about areas that might cause the most difficulty, the tests with colleagues were very efficient, each lasting only a few minutes. These tests with colleagues served as sanity check to ensure findings from the first round of user tests with users held up. Aside from serving as a quick sanity check, conducting the tests with colleagues had additional unexpected positive consequences.

Increased transparency and interest in the project

• The tests raised awareness for both, the activities and designs of the product team, but also for the type of work that we as UX Researchers do within these product teams. Colleagues got an inside look into both my daily work, and the work of the product team, leading to increased transparency and interest in the project. Seeing the work in progress during user tests reduced the feeling that product teams work in silos and that the rest of the company only sees the work when it is live. Running user tests with colleagues creates a culture of collaborative contribution to the quality of the product instead of broad critique when seeing a product that is already live.

Lighting up the design process

• The design process can sometimes come across as a black box. Running user tests with colleagues can help make it clear to others that designs are being tested and are not just the outcome of gut feelings. Making the process more transparent helps increase understanding of how decisions are made and takes the veil off of the design process.

Enhanced communication with colleagues

• The internal user tests also gave us an opportunity to talk to colleagues we rarely see. Most of our user tests ended with invitations to come sit in our team areas from time to time. Whether we get taken up on that invitation or not, we now recognize one more face in the office, and that recognition alone can break down barriers that stand in the way of giving feedback spontaneously at a future time point.

Giving employees an opportunity to voice ideas and influence product

• Seeing a prototype or work-in-progress provides colleagues with an additional opportunity to voice their opinions and concerns. People outside of the product teams often have thoughtful feedback and great ideas but limited paths to get those ideas into the product teams. This can lead to frustration and a feeling of me versus you. Seeing work in progress and collecting ideas from colleagues during product development can have the consequence of uniting the company behind what we are doing. Running user tests with colleagues can break down barriers that might prevent colleagues from offering valuable feedback during the design process.

Refining the protocol to maximize output of tests with real users

• Since we have started conducting user tests with colleagues, we have also experimented with trying out tests with colleagues before taking them to users. This has the added advantage that you, as a researcher, can refine your protocol and make sure you are asking the questions you are interested in, in line with best practice before talking to users. Regardless of how much you prepare, there are always behaviors and unforeseen situations that only come to light during user tests. Testing with colleagues can ensure that you are as prepared as possible for more of these situations and adjust the way you react in order to minimize bias and ask the right questions. This not only helps you make the most out of your time with recruited users, it is also respectful of users’ time. Running user tests with colleagues before running them with real users also has the added advantage of increased confidence and feeling of competence when you are with users. Of course, it is also a great opportunity to get familiar with the technical set up and ensure everything works smoothly.

Providing a playground to foster research capabilities in others

• Testing with colleagues is not only a way to include colleagues from other departments in the design process, it also serves as a good opportunity to include designers or Product Managers in the research process. Testing internally with colleagues provides a low cost and low risk alternative to allow other colleagues to conduct research and expand their research capacities.

Heightened empathy for users and appreciation for usability

• Conducting user tests with colleagues increases awareness for how important usability is in our product development and refinement. Running into usability issues during a user test also helps colleagues be more empathetic towards the issues and frustrations our users encounter. As colleagues encounter more and more usability issues during user tests, they learn the important of user experience and grow their appreciation for good design.

Cost, speed, sample size and confidentiality

• Additional benefits can relate to lower recruitment costs, and quicker response times. Following a first round of user tests with users, we were able to run a second round of tests with 10 employees (focusing on the main issues identified in the user tests) within two hours. Lastly, if there are topics that are extremely confidential, testing with internal employees provides an avenue to collect some data despite confidentiality constraints.

Initially, we started user tests with colleagues in a very impromptu and informal way. However, along the way, we got such useful and insightful feedback we thought it might be worthwhile to start better documentation and record the tests. Our colleagues are often full of great ideas and feedback, and some interactions are important to note down. We are also starting to integrate user tests as part of the onboarding process, as new colleagues are less familiar with the website, and also less influenced by internal politics. This is still a work in progress, but it’s a great extension of conducting internal user tests with colleagues.

There is a time and a place for when testing with colleagues might be a better or worse fit, and it is important to continue emphasizing that this is not a stand-alone solution or get-out-of-jail-free card to not test with actual users. Instead, testing with colleagues can supplement user tests with real users, while increasing communication, interest and understanding within the design and product development process.

In part two of this article, I outline a list of considerations that might help you figure out if this is the right method to supplement your research efforts.