Beyond “done”— prioritizing quality in product experiences

When quantity of features is the default for many organizations, how can UX-minded individuals bring about, and protect quality? Let’s talk about a simple starting point that you can integrate into your teams operations.

First, a reality check.

Let’s be honest, unless you’re in one of a very small number of markets, your organization is not going to be best served by trying to platinum-plate every touchpoint, in every user experience in every product, every time.

Often, when someone tries to do so, depending on their available resources, they end up with a jack of all trades product that ‘should’ win on paper every time but fails to… or for those with more pragmatic budgets, the impact of quality-oriented people like designers can end up spreading their influence so thin, that it’s barely measurable on the end result.

Instead, we as product and design leaders, must plan intentionally where, and when we apply our highly limited resources to make the biggest positive impact for our users and customers.

Let’s talk about one simple and lightweight way to get started.

Pick and choose where to spend the time to fill in the gaps. Credit: Created with the help of Dalle-3

The challenge: Meeting expectations

In the world of delivering software, the challenge of consistently delivering high-quality user experiences is perennial. Leading to many a designer to despair that their beautiful “mockups weren’t implemented right” and to ask the questions the we’ve all heard such as “how can I get my devs to build quality”. As an unnamed development leader candidly put it once to me after discussing his organizations next ‘big’ release:

“If we were to do it properly, we’d do it this way, but we just didn’t have the time because we have to get all this other stuff done too so the budget was stretched already.”

Every team across all disciplines shares a common goal: to deliver products that not only meet but exceed user expectations. However, the reality of limited time and resources means that not every part of every user experience can be perfected.

A common current state: Quantity over quality

Currently, release planning often prioritizes the number of epics that can fit into a release cycle, with a focus on quantity over quality. This approach typically overlooks the importance of defining and aiming for a targeted experience quality, leading to products that technically meet requirements but fail to delight users.

The proposal: Experience quality acceptance criteria

To address this, I propose the introduction of Experience Quality Acceptance Criteria for each epic under consideration for a release. This system would help teams prioritize epics based on their potential impact on user adoption and satisfaction, rather than merely their feasibility.

Operationalizing quality:

1. Quality tagging: Begin by tagging each epic with a ‘quality’ level based on its strategic importance and the expected user impact.
2. Quality-driven estimation: Use the quality tag to influence the sizing estimates, ensuring that epics crucial for user satisfaction are given the resources they need to excel.
3. Execution and review: Implement the designs and continuously review whether the delivery is meeting the scoped quality levels.

Quality levels defined:

While I encourage you to establish your own definitions, calibrated to the maturity of your organization, these are a few easy points to start from.

With a basic experience goal, you want most folks to not even really know they exist. Those that do find them, might have questions and challenges however. Image: Me

⭐ Basic (1 Star)

We need to get this thing done and the box checked. It needs to be generally usable, but the quality of the experience isn’t going to win or lose the business.

• Goal: Ensure general usability without aiming for delight. Suitable for features that need to be functional but aren’t differentiators.
• Risk tolerance: Moderate
  Speed is important and we accept we might need to come back later. As a result it may be ok to deliver first without rigorous user testing, and get feedback from the live product.
• Sizing modifier guideline: 1x

With a satisfactory experience goal, everyone might use it, but they will hardly notice it and won’t feel particularly strongly one way or the other. Image: Me

⭐⭐ Satisfactory (2 Stars)

This should be a decent experience on par with what we normally deliver. We need this to be usable by all users, and to meet their expectations.

• Goal: Aim for a good user experience that aligns with what we typically deliver, ensuring usability and reliability.
• Risk tolerance: Low.
  Some level of internal and/or external testing has been completed against this epic
• Sizing modifier guideline: 1.25x

With a delightful experience goal, users interacting with this capability should want to outwardly exclaim in well… delight! Image: Me

⭐⭐⭐ Delightful (3 Stars)

This is going to be one of the moments that make our users go “aha!” and exceeds a customer’s expectations. Its a key differentiator and likely to be used in marketing, drive growth in product, and feature in demonstrations.

• Goal: Exceed expectations with standout features that can be used in marketing and are likely to influence buying decisions.
• Risk tolerance: Very low
  We have confirmed at the design stage that if we deliver as designed, we will meet our metric targets through rigorous testing with actual users.
• Sizing modifier guidline: 1.5x

There are of course measured experiences of lower quality than those outlined above (eg: Burdensome), but from some form of UX hippocratic oath to do no harm to users, I have omitted these as things below acceptability and therefore not a logical target.

On sizing modifiers.

These are simple guides to development & design leaders estimating epic sizes, when compared to current estimation process.

If the initial estimate ‘to just make it possible’ at a basic level is 50points, then we call that 1x, and make the assumption to ‘make it delightful’ should budget 1.5x (75points). This is not necessarily the truth of course, but it function serves as a good rule of thumb when making high-level estimates.

Selecting the right quality target

Choosing the appropriate quality level should be a strategic decision influenced by:

• User expectations: Understanding what users expect and desire from the product. If you can get research with users — this is where it connects directly into the $, allowing the team to use those expectations in decided how much ‘spend’ is needed to actually make a difference vs let them down. (Did anyone say Kano study?)
• Market analysis: Evaluating how competitors are satisfying similar needs. Sure your users might not be shouting about the inherent value of quality in a given area… but be careful not to overlook the risk that its because they consider it table-stakes based on their experiences with your competitors.
• Product strategy: Aligning feature development with long-term product goals. Sometimes we choose to be good at some things, and bad at others intentionally to hit particular niche to win in a red ocean. (Take Southwest Airlines bus of the sky as a great example).

Image credit: Created with the help of Dalle-3

In conclusion: Create a strategy for success

By integrating Experience Quality Acceptance Criteria into the design process, we can shift from a culture of ‘fitting it in’ to one of ‘making it fit.’ This approach not only enhances the user experience but also aligns product development with strategic business outcomes, ensuring that every feature released moves us closer to delivering a winning product experience.