Highlights from UXRConf 2024
What’s stirring up the UX Research industry this year
UXRConf 2024 — a key UX Research conference, took place in-person last month in San Francisco, with a live telecast on YouTube. Throughout the single day conference, there was a wide range of talks led by industry experts, from using AI in the UX research process to deep dives into new roles in the industry. Here’s a quick summary of some of the sessions from the event.
Moneyball UXR
The first session of the day was presented by Joris Drayer, a staff UX researcher at Robinhood, who previously worked on the analytics for Oakland Athletics. Fun fact, the book Moneyball and subsequent film is based on the work his team at Oakland Athletics had done.
Joris’s talk focused on how his team had used a mixed-methods approach in the work they did and showed us why a successful research approach should depend on both qualitative and quantitative methods. Here are some of the key highlights from his talk:
- Think of research in terms of probabilities.
Use Bayes theorem, where the prior probability is based on past research where the insights stem from qualitative research. The probability of a new event is based on current research. And with this theorem, you can predict the confidence of how your suggested solutions can be beneficial. - Keep track of the research.
Note down 2–3 key research questions for each study or user group and write and refine answers to these based on your research insights. Use these as your prior probability as you move forward. - Pursue new research questions.
While it’s important to refine your understanding of key research questions on your user group, it’s also important to ensure that you ask new questions and refine your understanding of them. Set aside 10% of your research resources for this as it can help approach the area in new ways.
The Rise of the UX Research Scientist
Madeline Huberth, a UX Research Scientist at Meta, walked us through what it means to be a UX Research Scientist (UXRS) — an emerging role in the industry — showcasing its benefit, both from the perspective of hiring managers and individuals interested in the role.
The UX Research Scientist role is a specialized one that utilizes deep expertise in areas like perception, cognition, motor control and movement to define, evaluate & improve hardware and software to ensure a good user experience. At present it’s used in high risk, future research or in 0–1 product development that is heavily reliant on hardware and software, while also leaning on human care heuristics.
Other key points that came up during this talk were:
- This role focuses on the human’s capabilities and constraints
The UXRS role stands out amongst others in the field as it is concerned with how a human being’s constraints and capabilities affect how a product should be designed, over other aspects like the human being’s needs. - You need to have a deep area expertise to qualify for this role.
In addition, you also need to have a good product sense, a deep understanding of the human/technology interaction, and, great communication skills. - The roles use experimental methods in research
The role uses experimental methods and lab-based quantitative methods for research, and, the artifacts that emerge can be frameworks that explain the tradespace between technological and human variables.
If this resonates with you, then this role might be your calling. And if you are a hiring manager who wants quantitative research in an area without a scaled user base, is in search of specialized expertise, or would like input on the human capabilities of your product that has a heavy hardware/software component, then, you might want to hire someone for this role.
Leading with Strategic Foresight
In her talk titled “How Researchers can lead with strategic foresight”, Sam Ladner spoke about keeping an eye on the future and how one should scan the margins for change in the industry to be able to update the research strategy being used at your company to stay ahead of the curve. Sam has been involved with strategic work at companies like Workday, Amazon and Microsoft. Here are some key highlights from her talk:
- To build strategic foresight, it is important to scan for changes in multiple areas and to imagine multiple futures. It’s also important to have a holistic view of the ecosystem/industry you are in.
- The process of gaining foresight has the following key steps:
1. Inputs: Scan for changes that affect your industry
2. Analyze: Categorize these inputs and track them through some form of a database
3. Interpret: Obtain insights from these inputs and draw conclusions
4. Prospection: Create sets of scenarios that act as multiple futures based on these insights
5. Outputs: Describe the actions that can be taken to maximize the benefits and minimize the harms of these potential futures
6. Strategize: Choose a set of these actions and plan for them - Improve your process by putting in work daily, such as scanning your sources and categorizing inputs for a short duration each day, and by incorporating feedback, such as, are there a section of stakeholders that contact your more frequently for insights. Tailor the process to serve your stakeholders in a better way.
AI for UXRs
Christie McAllister, Director of Experience Research at Autodesk, walked us through how her team leverages Artificial Intelligence (AI) in their UX process.
- AI Agents need to have a clear objective when trained. They require a lot of hand-holding, such as the data you feed them needs to be clustered, they don’t create meaning on their own, and they don’t understand body language, subtext or humor. They also report back with confidence, which is why the results need to be reviewed by a human.
- Humans are critical to research processes involving AI as they need to make meaning of the data, design and lead research studies, and manage stakeholder expectations, which an AI is incapable of. Hence AI can enhance our research, but a human-in-the-loop is still necessary.
- Autodesk built upon AI models from Relevance AI by identifying 3 key areas it would help them in: avoiding human bias, optimizing resources for coding unstructured data, and, sharing insights at scale. They ran small human-led and AI-led studies in parallel and tweaked their AI models based on their results till a certain level of performance was reached.
The benefits of this approach is that it enables the human to focus on doing the research, while the AI handles the ability to share the insights at scale and provides an interface for the rest of the organization to query these insights to find answers to any questions they have. It also doubles up to make note of potential areas of research for questions it cannot answer. Have a working example of an AI-baked research process, does serve as some food for thought.
Insights to Income
In this talk Michael Lombardo, reflected on a key project he completed during his time as a UXR Manager at Supernal. He walked us through the user-centered design process of building one of their crafts and how his team took into account accessibility and other key factors. It was also interesting to note that even on such a unique project aspects of a UX career such as advocating for your users and stakeholder management still played a role. Here are the highlights of his talk:
- A unique constraint that his team had to factor in was that they were designing their product to work against gravity. As a result, the amount of weight that could be contained on a craft was a constraint and had to be optimized. Only things worth the weight could be included.
- To be able to understand what the essential elements for a positive flying experience were (things which had to be worth the weight) his team sought to understand what items people packed for their travel.
- They prototyped a craft using foam boards to understand to what extent could people be comfortably constrained in a craft, and it also helped them understand the jobs to be done, which they then mapped to use cases and personas
- But to get stakeholder buy-in and, most importantly, to convert these insights into income, they also did a competitive differentiation and clearly defined their value proposition. They came to realize that their value proposition was that Supernal provided the fastest way for a passenger to get in and out. What this meant for the business was that for the same cost of fuel, more number of trips were possible.
A key point from this talk was that it is good to have quantitative data handy, which in this case amounted to data like flying time, as it could help cater to business goals, track insights and help prove how UX brings in revenue.
I hope that this article helped shed some light on key themes in the industry at this time and provided you with some useful ideas to help improve the work being done at your company or on the work you do as a researcher.
