Deep UX flaws: what, why, and how to deal with them
User Experience (UX) professionals are trained in industry-standard ways of finding and tracking defects. These frequently include:
- Usability studies, which can begin even before the first prototype. The earlier the better!
- Automated unit testing, especially when your UX depends on Javascript and other front-end code.
- Verification against user stories or use cases. The UX is there to help people get things done, so let’s be clear what those people need to accomplish.
- Bug tracking when we discover that the UX or UI was not implemented as designed or doesn’t use the required design system in some way.
When it’s “right” but still fails
Despite this, users still end up serving as unpaid testers.
I’m sure you’ve found problems in apps, websites, and programs you use daily. These problems are what I call deep defects and they happen despite best-practice application of our tools and methods.
A deep defect comes from a gap between our design and its use in real-world practice. These defects are inevitable, and should be anticipated.
- We make design assumptions, and our assumptions are not always validated or correct. Check out this deep dive by Rick Dzekman on UX research and assumption validation.
- We have biases and encode our biases into our designs. See for example this piece on unconscious biases in design from the Interaction Design Foundation.
- Peoples’ needs change, and when we introduce new tools we inevitably alter the situations in which they are used, a typical aspect of wicked problems.
Finding and addressing deep defects requires a holistic integration of UX methods with your development lifecycle.
Deep defects and the pushback against them
People who work in development or project management are used to making judgments against written standards and test plans. They often don’t understand why, if the design has been implemented as we created it, we claim there are still defects.
The gap between “I did what you told me to do” and “but people still can’t do what we want” is large and not well covered by most software development lifecycle methods.
From our partners’ perspective, all the UI elements function properly, the system has been implemented as we requested, and therefore it should be “done”. But from our perspective as the voice of the customer, we recognize that people still have problems getting things done with the implemented design. Those problems point to defects!
Perhaps the screens are not clear enough, and people get stumped by how to operate what we’ve given them. Perhaps our design doesn’t offer the right information scent — the cues that help people move from page to page or step to step in a flow (see this primer on information scent from NN/G). Or it may be that people’s actual workflow is different from what we’ve allowed in our design; we thought it was A-B-C-D but it turns out that often people want to go A-D-B-C.
Regardless of the cause, there are gaps between our design and what people need in order to use it — deep defects.
Getting these UX defects accepted
To make matters more complicated, fixing deep design defects almost always requires design changes, and reimplementation. From our designer perspective this is both right and natural — we iterate on our designs and improve them as we get feedback. See for example Design iteration brings powerful results from the Interaction Design Foundation.
But from our partners’ point of view it often appears that we’re not dealing with a defect. Instead, our request for design updates appears like new functionality, which is often disallowed once we’ve passed a certain point in our development lifecycle. I’ve lost count of the number of meetings where we debated over whether requests for design updates were “defect fixes” or “new features”. Sadly, I’m here to tell you there’s no silver bullet for this situation.
Dealing with deep defects while avoiding debates
In the absence of such a silver bullet, here are a handful of techniques that have smoothed out the process and produced positive impact in the products I’ve designed. Try these five and see what difference they can make in your situation.
- Include whoever will be involved in the defect/feature debate in the testing. I try to include at least development and quality people as test observers. If you can get product managers and documentation involved that’s even better. Seeing the impact of the design failure first-hand helps our partners understand why we judge things to be deep defects.
- Include fit-for-purpose testing such as User Acceptance Testing (UAT, see this introduction to UAT) in the overall development process. Then participate as a UAT tester yourself. Deep defects should be included with UAT-generated defects. After all, if people can’t complete tasks, it’s hard to argue that the system is fit for purpose!
- Advocate in planning stages for allocating design defect-fix capacity. For example, one company I worked at would designate specific sprints for “bug burndown” and we began including design-defect remediation in these sprints, which were always part of our increment planning. The “Scrum FTW” blog has a nice discussion of different strategies for this.
- Learn your organization’s QA system and use it when reporting usability results. Likely there are criteria for classifying defects in terms of severity and impact. By speaking in the quality language your organization already uses, you will get greater understanding of the issues you are raising. There are good resources online to guide you in assessing the severity and impact of deep design defects, such as this overview of assessment from MeasuringU.
- Develop UX metrics and educate your partners on them. Sites like UX Matters have examples of how to create business-relevant UX metrics. Our business partners should understand the cost implications of design defects because we are asking the organization to incur a cost to fix them.
Design defects — deep defects — should be treated with other defects and allocated resources for fixing along with the other problems that our products, apps, and sites invariably have. That can’t happen unless we integrate design defects with the work of the organizations that drive fixing them.
