Synthetic users: the next revolution in UX Research?
Is it possible to conduct research with users… without users? Going beyond the obvious to think about possibilities.
Is it possible to conduct research with users without actual users? Could we obtain valid insights by having artificial intelligence simulate human behaviors and respond to our questions? In this text, we go beyond the obvious to think about these possibilities.
As researchers, we generally believe that the quality of the answers we receive is linked to our ability to ask appropriate questions to the right people. But what would happen if the answers no longer came from humans, but from artificial intelligence?
This new scenario is made possible by the advancement of research with synthetic users, who are at the forefront of applying Artificial Intelligence in design processes. With this revolution, various methodological and even philosophical questions arise.
But don’t worry; this is not just another text about Artificial Intelligence and its capacity (or lack thereof) to simulate empathy.
Here, we go beyond the obvious, without the pretense of offering definitive answers. The intention is to share valuable insights to stimulate reflection.
By the end of this reading, you will have explored the following topics:
- Understanding what a synthetic user is
- Limitations of this type of research
- Potential for relevant application
Shall we?
What is a Synthetic User?
First is important to note that it is a new and contested concept, resulting in various possible interpretations.
That said, in this text, we understand synthetic users as clusters of behaviors generated by the combination and analysis of a large dataset from an LLM (Large Language Model). To simplify understanding, we can imagine intelligent avatars simulating interactions with real users.
Synthetic users act as digital explorers in desk research, or conversational personas in the data visualtization after the research with real users. They can perform the role of virtual guides that provide valuable insights into user behavior, even in a simulated environment.
Their construction process is similar to the creation of a persona. Typically, researchers collect data from different users to understand an average behavior. From there, they build a narrative to operationalize a mass of information, creating simplifications that facilitate team communication, analysis, and ultimately, value generation.
However, unlike traditional personas developed by researchers or designers, a synthetic user is an artificially generated entity, often through artificial intelligence or computational simulation. They are programmed using public blogs, social media, forums, scientific research, and other sources to mirror behaviors, preferences, and usage patterns similar to those of real users.
Imagine something similar to interviewing ChatGPT — but a specific version that acts according to the created persona, respecting defined age, gender, life history, economic class, and cultural aspects.
Maria Rosala and Kate Moran explain in more detail the differences between an user interview and a synthetic user interview, highlighting that synthetic users synthesize vast amounts of available data about a specific user group and present it in a digestible way.
This allows the synthetic user to be, in essence, a narrative created from a vast dataset. One of the significant values they deliver is that once created, it is possible to have an infinite conversation with them.
Qualitative or Quantitative?
When discussing the nature of research with synthetic users, we are breaking barriers. It is a hybrid approach: neither purely qualitative nor purely quantitative. We can feed the model with hard data but also explore the nuances and quality of experiences on a more qualitative level.
Traditionally, qualitative research, characterized by interviews, focus groups, and case studies, was commonly considered labor-intensive, costly, and time-consuming. It requires a significant investment of time and skilled professionals to ensure we find the right people to talk to and then conduct the conversations in a way that minimizes biases and extracts maximum value.
On the other hand, quantitative research, such as that conducted through questionnaires, is seen as more scalable but sometimes lacking in depth. With it, we can identify the “what” people do, think, and desire. But it is very challenging to map the “why,” the motivation behind behaviors.
Therefore, these two approaches often go hand in hand.
With advances in Artificial Intelligence, however, maybe we are blurring the lines between qualitative and quantitative research, opening new possibilities for data collection and interpretation. Using synthetic users, conducting 10 interviews or sending out 5,000 questionnaires to assess a product’s attractiveness now can require approximately the same effort.
With synthetic users, it seems we are entering a third margin of the river — there is still much to discover and reflect upon.
Limitations Related to Bias
Yes, just like in any research, biases here are inevitable.
As researchers, we are always concerned with biases. We try to minimize them as much as possible, but we know that, no matter how much we strive to create a neutral environment, achieving a completely bias-free result is impossible. More important than avoiding biases is knowing that they exist and identifying what they are.
The very presence of the researcher, by itself, influences the environment — there is no human interaction without this kind of exchange.
For example, a participant might provide answers they believe the researcher wants to hear, what is known in psychology studies as the Social Desirability Bias. Sometimes, the researcher’s expectations can subtly influence participant behavior, leading to results that align with the researcher’s beliefs. Beyond the Social Desirability, there are a lot of different biases — in her book about Leadership Decision Making, Yinying Wang explores 52 different types of biases.
Synthetic users, on the other hand, do not possess this type of human bias but can reflect the biases present in the training data or in how they are used. It is crucial to clarify that language models trained on available data from the entire internet are not representative of real people. They have a geographic distribution, gender distribution (the internet tends to be male-biased), and a different socioeconomic status, which many refer to as digital bias.
Its important to note that many insightful UX researchers have critically examined the downsides of using AI for research. Feifei Liu and Kate Moran, for example, outline key challenges associated with using AI in UX research. It highlights how AI tools, while useful, often struggle with capturing the depth of user emotions and context, potentially leading to biased or incomplete insights. The article emphasizes the need for human oversight to ensure that AI tools complement rather than replace traditional research methods.
Thats why I believe that any tool, no matter how advanced, needs regular evaluation and refinement to ensure it meets its intended purpose without introducing unintended biases.
If your research focuses on countries, economic classes, or age groups with less internet access, for example, you might struggle to obtain sufficiently comprehensive and reliable data sets to create synthetic users safely.
Remember: synthetic users are created based on the combination of available data, and thus, are reliable as long as there is confidence in that data and the way it was analyzed.
Limitations Related to Language
Conducting research with synthetic users presents visual limitations, as we are restricted to linguistic models — meaning, a set of textual data.
If you’ve ever conducted a card sorting exercise or in-person interviews, you’ve probably noticed that it is possible to map aspects that go beyond text. We are capable of interpreting micro facial expressions and intonations, which are crucial elements that make a significant difference in understanding and improving the user experience.
Human communication goes beyond words.
We lose emotional nuances and subtle details that would be perceptible in human interactions — the richness of subtle details, facial expressions, and intonations, for example. It’s like trying to paint a picture with only words, losing a bit of the magic of the image.
Interpretation becomes limited when written language cannot fully capture the range of human emotions and reactions.
Speaking of details, a small note on the nature of interviews: they are definitely not like interviews with real people. You will notice that responses are denser, there is less small talk and hesitation, and less waiting time.
It’s a conversation with fewer onomatopoeias. Synthetic users don’t pause to think, take a sip of water, or say “uhms” and “uhs”. Interviews with synthetic users are like an espresso coffee — concentrated and efficient.
Potential of Synthetic User Research
Synthetic users can accelerate the understanding of desirability and usability, providing quick insights for product development.
We can identify the obvious more quickly, which can translate into faster time-to-market for ideas. At the current stage of development, with some reservations, synthetic users might particularly contribute to mapping desirability and usability in a more agile way.
The ability of synthetic users to rapidly process large volumes of data and provide feedback means companies can iterate on designs, campaigns, or strategies more quickly, responding more agilely to market demands.
In this sense, offering synthetic users a seat at the ideation table could be an interesting way to conduct a brainstorming session, for example. It’s a faster method to create a narrative based on data collected by the company or publicly available data, which is more accessible and aids in the idea generation process.
It’s worth noting Nielsen’s perspective, which observed that, regarding usability tests, with approximately five users, data collected in research begins to show significant repetition.
Following this line of reasoning, Kwame Ferreira highlights the ability of synthetic users to efficiently provide the 80% obvious insights that commonly arise in the early stages of research. By addressing these predictable and already known aspects, they could allow designers to focus their efforts on the remaining 20%, where disruptive innovation opportunities lie.
The idea is that by moving more quickly through the redundancy phase in the early stages of research, designers can direct their creativity and resources towards exploring new perspectives, ideas, and impactful solutions.
By optimizing the data collection process, synthetic users could be valuable allies in driving innovation and advancing products and services. However, they should never completely replace real users. It is important to evolve and refine synthetic users to make them as close to organic research as possible, but always comparing both.
In smart research, synthetic users are not seen as a replacement, but as an additional tool and resource.
Possible Futures
Regardless of the technology, the most important factor remains the ability to ask the right questions. The key is to explore the adjacent possibilities. The questions you ask are directly correlated to the answers you receive. This is true for both synthetic and organic users.
From my perspective, the area where Synthetic Users offer the most value in research is data visualization. They can act as conversational personas after the research is completed. Instead of a slide or printed persona page, we can have a synthetic user to interact with and help generate ideas. Additionally, they can assist in desk research during the initial phase, mapping out the obvious and allowing us to reach more interesting insights when talking to real users.
If you are curious and want to get started exploring the possibilities of Synthetic Users, you can begin by creating your own flow or maybe testing specialized platforms, such as SyntheticUsers.com. Or deep dive into the contents shared here to gather more information about it as well, afterall it is a very new topic and there is a lot of divergent opinions.
I hope this summary of our discussion has sparked your curiosity about the future of design research. Keep exploring and questioning, as it is in this process that we find the most valuable answers.
If you have any doubts or want to discuss innovation further, feel free to send me a message!
References Mentioned in This Text
- AI Innovator’s Podcast: Kwame Ferreira discusses synthetic users.
- Nielsen Article: Explains the theory of why it is necessary to conduct tests with five users.
- Synthetic Users: If, When, and How to Use article that explains in more detail the differences between synthetic interviews and user interviews.
Additional Content Worth Checking Out
- Cameron Hanson’s Talk: Discusses tools that use artificial intelligence to optimize design processes, providing interesting counterpoints.
- Synthetic Users Article: Describes how the team of SyntheticUsers.com is applying intentional research and diverse data sources to improve the performance of synthetic users.
Check out more articles from this series
- How AI is enabling designers to go beyond personas: Personas has brought us this far, but it will no longer be enough for the approaching future.
- Synthetic data for designers: what you need to know: A $2.1 billion market is emerging, unlocking new design roles — and it’s already underway.
