It’s time to rethink UX design challenges

Design challenges have become more and more popular for interviews for UX roles. I think we need to fundamentally rethink how we use them.

Photo by Joanjo Pavon on Unsplash.

I’ve been working in UX and design for a decade now. During this time, I’ve seen, participated in, and requested dozens of UX design challenges — both as the candidate and as part of the hiring team.

Design challenges are typically requested as part of the interview process for UX roles, with varying levels of complexity between companies, roles, and teams, and may or may not be tailored towards a specific skillset.

My sense is that design challenges are reaching a critical point of popularity as a tool during the interview process. My first couple of design jobs did not ask for a design challenge: my work and experience spoke for itself. However, in the past few years, I’ve been asked to complete a challenge in all but one of my interviews. I’ve heard the same from my peers — along with rising frustration and exasperation.

I think that we, as an industry, need to rethink how we approach design challenges as part of the interview process for UX.

In this article, I’m going to share my theory on how we got here, the problems inherent to the process and the consequences we face as a result, and my ideas for how we can rethink how we use design challenges in hiring for UX roles.

How we got here

My theory is that UX design challenges are rooted in the interview process for software engineers and developers.

Interviewing for a developer has a specific set of challenges:

• Companies need to understand a candidate’s capabilities, particularly when they can’t show their work due to non-disclosure agreements or other limitations.
• It’s often hard to tell what code a candidate wrote on their own and whether that code reflects their ability — because the vast majority of code belongs within a larger context.
• Companies need to find candidates who can take and interpret a problem statement, plan their approach, and craft effective solutions on their own.

In response, teams began giving candidates take-home coding projects. These take-home projects usually have:

• A defined problem and specific requirements (or the ability to request specific requirements as part of the project).
• A limited scope and timeline.
• Clearly defined deliverables.

These take-home coding projects offer development teams a number of benefits¹:

• By and large, they let companies effectively evaluate candidates’ ability to interpret requirements, create a plan, execute the work, and communicate their ideas and approach.
• They create fairly realistic working conditions: a candidate is usually free to use whichever tools and resources (such as StackOverflow) they need to solve their problems — they aren’t expected to remember what they’ve memorized about a language, algorithm, or framework on the spot.
• The complete solution will either work or it won’t, and the code can be reviewed based on very clearly defined guidelines, or even the same way that working code would be reviewed.
• It helps get around a candidate’s inability to share work that’s under an NDA or proprietary.
• It helps as a check before investing both your and the candidate’s time in an on-site interview.

When we’re hiring for UX roles, we have almost identical objectives that UX design challenges can help us reach:

• To understand a candidate’s capabilities and fit for a role, particularly when a candidate doesn’t have permission to share their work.
• To understand a candidate’s own capabilities when their individual contributions to a project may not be clear.
• To understand how a candidate thinks about and approaches their work, and to see if they’re able to communicate their process, ideas, and solutions to others.

As UX has grown as a field, I suspect we (meaning primarily product teams) looked at the take-home projects we were asking developers to complete, and adapted them for UX. As soon as a few companies started to do this, others followed, and it snowballed until it we reached today, where they’re a default part of the UX interview process.

However, by the very nature of the work, take-home UX design challenges are not parallel to take-home coding projects.

The problems

Despite the benefits that UX design challenges offer, there are several problems with using them to evaluate candidates:

• “UX design” is a very wide discipline. A UX designer’s role covers a very wide range of the process of bringing a product to life, spanning research, problem definition, ideation, wireframing and prototyping, user testing, high-fidelity UI design, UI animation, and even copywriting. Most UX designers will work across many or each of these areas in every project.
• UX design is intensely collaborative. UX designers rarely work or produce their best work in isolation. A significant measure of how successful a UX designer is in their role comes down to how well they communicate across a company to uncover and build shared understanding of objectives, goals, and requirements. The best design is often a result of ongoing creative collaboration, experimentation, and feedback loops.
• UX design outcomes are inherently subjective. There’s very rarely an explicitly correct or an explicitly incorrect answer to any given UX design problem — and often, the only way to evaluate the success of a design is to put it in front of a real user and see what happens. Until then, when we evaluate the outcomes of a design challenge, it’s inherently subjective.

The consequences

These problems with trying to evaluate candidates for UX roles through UX design challenges lead to some real consequences for both candidates and hiring teams:

• Design challenges ask for significant headspace, mental energy, and time from candidates. While UX design challenges usually tell candidates to limit their time or how long something “should” take, this doesn’t prevent a candidate from putting more time into it than expected.
• Design challenges disproportionately reward those who can spend more mental energy and time on the challenge. Though design challenges will often specify that only low-fidelity wireframes are required, the subjective visual nature of design means that the candidates will still be evaluated by the quality of the wireframe. Should a candidate choose to create a high-fidelity design, they’ll further influence the subjective interpretation of their work.
• Design challenges ask disproportionately more of the candidates than they ask of the company. Design challenges are “easy” to request, and have become the default first-step after — or even before (!) — an initial screen. This imbalance means that each individual candidate is asked to invest far more mental energy and time than the company itself invests in finding good candidates.
• Design challenges can’t accurately evaluate UX designers across the design process. When a field is as wide as UX —anything from research and strategy to UI design and animation — each candidate will have strengths and weaknesses in different areas. It’s simply not possible to evaluate how effective a candidate might be in and across each of these areas within any reasonable time limit. While many design challenges try to get around this by focusing on only a few parts of the design process, this hand-waving means that work is created without true context or understanding, and doesn’t producing anything of meaningful value — for the company or the candidate.

• Design challenges are close to, or are, spec work. More often than not, design challenges are related to a company’s own product or industry, because that’s the industry that hiring teams feel comfortable evaluating. This means the work a candidate creates in response to the design challenge can influence and inspire those reviewing the work, directly producing value for the company on the behalf of the company without providing compensation unless you’re one of the lucky few who receives a job offer later on.
• Design challenges don’t save much time for hiring teams. When a company asks candidates to do a design challenge, they still have to review and evaluate the work. If a candidate is promising, they almost certainly still need to progress through a number of skill and cultural interviews before receiving an offer.
• Design challenges make your company a less appealing place to work. From numerous conversations I’ve had with others in the industry, I’ve heard a common theme: by the time most candidates complete their design challenge, they experience (and express to others!) a sense of resentment for the amount of mental energy and time they’re asked to invest for slim odds of a payoff (a job offer) — which is exacerbated as more companies ask for design challenges earlier and earlier in the interview process.
• Design challenges mean that your company is going to miss out on amazing candidates. When design challenges ask for significant mental energy and time, they hurt those those who don’t have the mental energy or time to invest in the process. Every candidate asked to complete a design challenge may have an active personal or professional life, may be dealing with burnout in their current role, may be dealing with major life events, may (almost certainly) be interviewing with several companies at the same time (and being asked to complete a design challenge for each one) or may simply be fed up with design challenges. As amazing as they might be, the design challenge is likely to drop them out of the hiring funnel.

I’m not saying we should stop using design challenges altogether. They solve real problems for hiring teams. But we can’t deny the problems and consequences that result from design challenges as we use them today.

That’s why we need to rethink how we use design challenges.

Improving the interview process for UX roles

After years of participating in and conducting UX interviews and countless conversations with other designers, I have a few ideas for how we can improve the interview process for UX roles to try and solve these problems.

First, let’s revisit the goals we’re trying to achieve:

• To understand a candidate’s capabilities and fit for a role, particularly when a candidate doesn’t have permission to share their work.
• To understand a candidate’s own capabilities when a their individual contributions to a project may not be clear.
• To understand how a candidate thinks about and approaches their work, and to see if they’re able to communicate their process, ideas, and solutions to others.

Then, we can look at some different ways to achieve these goals.

1. Strengthen the screening and initial interview process.

First, make sure you’ve clearly defined what skillset you’re looking for in a role, and carefully evaluate candidate’s portfolios and previous experience to see whether their skillset aligns with what you team needs. A competency map can help understand existing strengths and gaps, and consider how well a candidate would fit in.

Be hands-on in the interview process, if you can: involve people who have experience evaluating the skillsets you’re looking for. If you don’t have people with those skillsets within your team or company, see if you can get help from others who do².

Ask more up front. Spend an hour — rather than 15 minutes — having a focused conversation with a promising candidate. Done with purpose, it can reveal just as much about their knowledge, skillset, and experience as a design challenge.

Be deliberate about who you move forward in the interview process. Don’t use the design challenge to answer ambiguity.

2. Consider what you’re trying to learn with a design challenge.

When you come across a compelling candidate and want to move forward in the interview process, consider whether a design challenge is the right next step. What are you trying to learn by having them do a design challenge? Is a design challenge the right way to learn it? Will a one-size-fits-all challenge give you the answers you’re looking for?

Don’t use design challenges because that’s what everyone else does, or because that’s what you or your team had to do. Designers ought “pay their dues” by learning and growing through experience, not through ritual hardship — and a design challenge rarely offers those opportunities.

3. Offer the choice of presenting a case study, or doing a design challenge.

Looking back on our actual goals, we’re trying to understand a candidate’s capabilities and fit for a role, their approach to their work, and whether they’re strong storytellers.

In the end, asking a candidate to share and tell the story about a case study isn’t much different from asking them to share and tell the story about a design challenge.

Most UX candidates will have case studies and stories they can share — and would be excited to talk to you about their work. You can dig in and ask questions and learn a lot more about their process and role than in a theoretical design challenge.

However, not all candidates will be able to share a case study, perhaps due to a nondisclosure agreement. That’s why offering a choice can be the right move.

4. Do paid design challenges.

If you’re struggling to evaluate a specific skill or ability that you can’t uncover from a candidate’s previous work, a design challenge can be a valuable tool.

However, treat it like it is: work produced for your company at the request of your company. It’s an investment of mental energy and time by the candidate on your behalf. Compensate the candidate along the industry average hourly rate, for 4 or 8 hours of time (or more, if you choose).

Clearly define the challenge and make sure you have specific measurable outcomes tied to the skillset you’re evaluating. If you want to learn more about their UI design sensitivity, ask them to design a UI element with certain constraints. If you want to learn more about their ability to plan a research project, ask them to put together a project plan. Make it crystal clear what you expect as deliverables.

Make sure that the challenge can actually be completed within the time you specify, perhaps by doing it yourself or asking another team member to do it as part of their day-to-day work.

Create a scorecard or framework for evaluating the outcome to remove some of the subjectivity from the evaluation process.

At the point you’re asking for a paid design challenge, the team ought to be quite confident about the candidate: the goal at this stage is to verify specific capabilities in line with the needs of the role. Do they have the chops to deliver? If you’re not willing to invest a few hundred dollars into the candidate at this point of the process, they probably aren’t a good fit in the first place.

What are you doing?

I’m always looking for new ideas: how does your team approach UX design challenges?

Footnotes

[1]: To be certain, the hiring process for developers has serious flaws, too.
[2]: I’ve often wondered if there’s space for UX consultants brought in specifically to evaluate UX candidates.

Thanks for hitting the 👏 if you enjoyed this article! It’ll tell me to write more like this.

Quinn Keast is a UX Designer + Partner at Caribou, a user experience strategy and design consultancy.