What is the effect of personalization on the UX of Chatbots?
For the past years, you’ve probably noticed the increased popularity of chatbots, also known as conversational interfaces. Some of these chatbots have a very personal approach, some of them don’t. This made me wonder if we can improve the User Experience of Chatbots by using Personalization. In this article, I will give you the insights I gained from my research about the effect of personalization on the UX of Chatbots in the banking sector.
This article is based on my Thesis for the Master Information Studies (Human-Centered Multimedia at the University of Amsterdam).
Insights
I will start with the main insights, then the learnings on chatbots, AI for chatbots, the type of interactions, goals and tasks, the user experience of chatbots and personalization of chatbots.
Main insights
When designing chatbots, make sure that the goal of the chatbot is communicated beforehand and as clear as possible. When the goal is clear, and the interaction of the chatbot user-friendly, then:
- Users perceive the chatbot most useful.
- Users’ expectations are met and therefore they are more satisfied with the chatbot.
- Users make fewer errors and a frustrating user experience is being prevented.
Other aspects that should be taken into account when designing chatbots are:
- The maturity of your artificial intelligence (which has a big effect on the UX of chatbots).
- The amount of information you send to the user at once (e.g. when you send a long message or multiple messages, it can overload the user or the interaction can be perceived as less natural).
- The accessibility (e.g. some users had difficulty reading the small font).
- Communicate about the security and privacy of your chatbot, especially when it contains sensitive data (e.g. financial or personal data).
Current State: Chatbots, AI, Interaction, UX and Personalization
Chatbots
When we are talking about chatbots, we mean an interface that the user can interact with by means of a conversation. But why is it a good idea to start with chatbots anyways? It appeared from research, that when specific study material is presented to the user in a conversational style, the users learn (or remember) more deeply and are more cognitively engaged than a non-conversational approach (Mayer, 2003). Also, big players (e.g. Facebook) launched bot frameworks which makes it easier to experiment with chatbots. In May 2018, Facebook announced that more than 300.000 bots have been created for Facebook Messenger compared to 33.000 in September 2016.
Use-cases for chatbots are sprouting within a wide variety of domains, such as: e-commerce, financial, travel, productivity, healthcare, insurance, entertainment and more. An example of a chatbot (or conversational interface) I like from the insurance industry is Lemonade.
Artificial Intelligence
Most chatbots are using artificial intelligence in some form these days, but the maturity of AI has not yet proven effective in most implementations. One of the limitations of the current AI is that the user’s input is not always recognized or interpreted correctly. An example of AI within chatbots going wrong is Microsoft’s chatbot “Zo”, which was programmed to avoid offensive topics, but in practice actually conversed about those topics.
If your chatbot’s AI is not mature enough, let the user interact with the chatbot by using pre-defined answers.
Types of interaction, goals, and tasks
A user can interact in different ways with chatbots. For example, by free text or by selecting pre-defined answers. The chatbot responds to these answers by using certain rules or by natural language processing. Besides the differences in the way of interacting, chatbots can also differ in the type of goal they can fulfill. Chatbots can be used for customer support, navigation, or for assisting the user in things, like a personal coach. Also, chatbots can differ in the type of task:
- Simple or complex tasks
- Problem or opportunity based tasks
In the table below you can see an overview of the types of tasks. I created a chatbot to block a bank card (simple and problem-based) and to get financial advice (complex and opportunity-based).
User Experience
User experience is a broad concept, covering “the users’ emotions, beliefs, preferences, perceptions, physical and psychological responses, behaviors and accomplishments that occur before, during and after use” (ISO 9241–210:2010).
A recommended method to measure user experience is the HEART framework (designed by Kerry Rodden, Hilary Hutchinson and Xin Fu, from Google’s research team).
The HEART framework could be used to measure and monitor the UX of your chatbot for the long-term.
For my research I needed a UX framework to measure short-term usage. Therefore I created a framework in which I focused on the aspects ‘usefulness’, ‘usability’ and ‘satisfaction’ to measure the user experience of chatbots. This framework exists of 12 questions that measure the UX aspects described before.
The Chatbot UX framework can be used to set up an online survey, to categorize the data of (semi) structured interviews or to measure the UX of a chatbot.
Personalization
Personalization means that an information system is changed in such a way that it becomes more relevant to the user. Implementations of personalization can be that the chatbot responds to the input of the user, creates an answer based on user data or the chatbot’s personality. In my research I implemented personalization in the following way:
- Writing in a friendly/personal tone-of-voice
- Conveying emotions by using emoticons.
- Thinking along with the customer (giving advice).
- Using user data (e.g. using facebook name to welcome the person).
Results of the study
To create these chatbots, I used Chatfuel so that these chatbots could be shared and tested easily and fast with Facebook Messenger. For both the tasks, we created a general and personalized chatbot. This means we ended up with these four chatbots:
As we read in the main insights, it appeared that there was no significant effect of personalization on the user experience of chatbots. However, there was a significant difference found in the effect on the type of task on the user experience of chatbots.
It appeared that a simple and problem-based task was perceived as more useful, more satisfying, and more user-friendly.
From the qualitative research, we found a couple of interesting insights that influence the usability, usefulness and satisfaction of chatbots.
- Usability: some users found the quick replies within the chatbot unclear and started to type instead of using these pre-defined answers. We saw that people also tend to type quicker (instead of using quick replies) when the chatbot approaches them in a personal way. If you want to use quick replies only, communicate this also to the user to meet their expectations.
Use pre-defined answers in the chatbot when the AI of your chatbot is insufficient.
- Usefulness: the usefulness of a chatbot was perceived higher when the goal or flow was clear and when the chatbots met the expectations of the users. Also, the user wants to know what is possible within the chatbot to use the chatbot to its full potential.
Before the user will interact with your bot, clearly communicate the chatbot’s goal and functionalities.
- Satisfaction: some users who don’t like calling, loved the way you can just achieve your goal by a quick chat. Also, some users found it pleasant that the chatbot was thinking with them and using emoticons. Some users were confused about a formal tone of voice (because of the financial use-case) combined with emoticons. This means that the interaction could lead to confusion or even users being insulted by the manner they are approached.
Personalization (e.g. showing empathy, use of emoji’s or friendly tone-of-voice) of chatbots can be best applied in informal domains
From this research, we can conclude that personalization does not have an added value to the perceived user experience. However, it could be that other implementations of personalization do have added value for the UX of chatbots.
We found out that simple problem-based tasks, within a chatbot, receive a higher UX score than a complex opportunity-based task. This UX score is constructed by measuring the perceived usability, usefulness, and satisfaction of the chatbot. This framework can be used to create a survey, categorize the data of (semi) structured interviews or as a checklist to evaluate the UX of a chatbot.
Lastly, I would recommend everyone to keep the goal of your chatbot as clear as possible, listen carefully to your user and experiment with different implementations of personalization to see what works best in certain use-cases.
Thanks for reading and if you have recommendations or advice, connect with me on Linkedin. Also, you can send me a message for more info or to chat about UX, chatbots and innovative technology.
Also special thanks to Wouter Slager and Jacobijn Sandberg.
