“Failure” in the Design Process

A reflection about improving.

A year ago, I was taking a senior product design class in which teams of about 20 students were tasked to create meaningful products. The first few weeks were dedicated towards brainstorming and preliminary research, and my team ultimately decided to build a handheld device for helping visually impaired users interpret 2D images. Our target users were mainly visually impaired or blind students who needed to interpret images, such as graphs and simple illustrations, in educational contexts.

Research

We determined target users and product goals by doing market research and interviewing subject matter experts. Current solutions for aiding visually impaired students in schools involved expensive technology usually in the form of large machines. My team took this as a positive sign, believing our product’s competitive edge could be affordability and mobility. We also conducted in-person interviews with a blind elderly man as well as phone interviews with staff members of blind centers. The feedback we gained were generally positive and open to our product idea. We saw a green light in the direction we were headed and were excited for an opportunity to contribute to the field of assistive technology.

Prototyping

Within the next few weeks, we took our idea from paper to initial prototype. It had a basic form with a camera sensor and small vibrating motor attached to a Raspberry Pi. Whenever the camera sensed a black area, the motor would vibrate to inform the user.

(Left) illustration of concept; (Right) first prototype

We demonstrated the prototype with both sighted people and a blind elderly man, but the feedback wasn’t conclusive — we needed to test with accurate users of our target audience. Consequentially, our next step was contacting nearby blind centers and scheduling appointments for several of our team members to engage with visually impaired students.

User Testing & Interviews

We made several preparations going into user testing and interviews:

• Create several more prototypes testing different forms and interfaces (although the underlying mechanism of camera sensor to vibrating motor was still the same)
• Prepare tasks for participants to complete, mainly using the prototype to identify different 2D images
• Prepare interview questions, not just with using the prototype but also their general experiences and current methods identifying images

User testing at a blind center

I wasn’t one of the members who visited the centers, but my understanding from my teammates’ reports was that the process went according to plan and we were able to gain valuable feedback from accurate users.

Feedback

The feedback was either observational data through user testing or qualitative feedback from conversational interviews. We found that the latter was the most useful for guiding our next actionable and could be summarized with several key points:

• Visually impaired individuals have existing solutions for the problem space that are well integrated into their lives.
• Schools and centers are obligated to aid visually impaired students, which includes providing them with the necessary tools for interpreting images, so affordability isn’t an issue.
• Given current solutions and existing technology, it’d probably make more sense for another solution to be a software product, which wasn’t an option for this given class.

Result

It was evident from the feedback that our user need likely didn’t exist. Of course, we inevitably decided to change directions and achieve a user-centered final product, but that’s not what I wanted to talk about here today.

Reflection

We had already spent half our timeline in a product direction we just proved to have no user need. As college students in a class that’s only a semester long, time was our most valuable asset. It was easy for my team to see this situation as a failure — our morale was down, we had no clear next direction, and our past efforts seemed a waste. However, looking back on it now, I realize that this “failure” is an integral part of the design process. Of course, user testing is supposed to inform design decisions. There is always the possibility of user research proving your hypotheses wrong. That’s why research and testing exist. However, it’s important to identify the assumptions and hypotheses that led to those outcomes in order to learn from these experiences.

What went wrong?

Let’s analyze the situation chronologically, listing the assumptions as we go:

1. Brainstorming — Assumption: Visually impaired and blind people need a better way to interpret 2D images.
2. Market Research — Assumption: Existing solutions are mostly expensive machines. There is an opportunity for an affordable, handheld device.
3. Subject Matter Expert Interviews — Assumption: A blind elderly man or staff members of blind centers are representative of our target user group, visually impaired students

There are many more I could’ve listed as the design process can get pretty complicated, but I saw these as the most influential ones. As you can see they’re also the most early assumptions we made, framing the rest of our decisions and actions from the get-go. Now I’ll list some questions we should’ve of emphasized in correspondence to these assumptions:

1. How do we know that this user need exists? Do we understand this target user group? How can we empathize with accurate users as soon as possible?
2. Why aren’t there more affordable and mobile solutions? How reliable and updated are these online resources?
3. Can subject matter experts replace target users? How accurate to the target user group are they?

How do I improve?

I don’t think these assumptions were “wrong” because it makes sense for the design process to start off with initial hypotheses and uninformed assumptions. Yes, we can do market research and SME interviews for preliminary research, but the conclusiveness of the results should be questioned constantly. With this is mind, here are some main points of improvement that I have learned from the experience:

• Make assumptions clear from the start — Making initial hypotheses and assumptions is natural for initiating the design process, but it’s easy to unconsciously start accepting assumptions as truths. Clarify and share assumptions with the whole team, emphasizing the fact that they are based on personal beliefs and require real data. Use uncertainties to form testable hypotheses.
• Engage with target users ASAP — Any source of information is useful, but nothing beats getting feedback from accurate target users. Use research to learn more about the problem space and form hypotheses, but the sooner you test with real users, the sooner you can be confident about moving forward.
• Challenge beliefs constantly — Sometimes we might not even realize our beliefs are assumptions. However, it’s important to question why we believe certain statements, especially if they pertain to user need and product direction. Always ask yourself why you believe something, what data supports your belief, how objective/subjective is this data, etc. Don’t be afraid to bring up these questions in team meetings as well.

This reflection has helped me improve my approach to user-centered design, and I hope it can help you learn something as well. Thanks for reading!