Emotions in video game UX
Designing experiences to leverage subjectivity in perception
Video games are classified as a digital experience, and all digital experiences are facilitated by digital technology. But as a community of creatives, we sometimes fail to realise that digital experiences are only a part of the real player experience. The most important part of any experience is the perception by the individual consumer, of which the digital experience is a very small part.
Emotions are psychological states that are changed by stimuli, and the stimuli is largely dependent on the context of the state of mind. This means that emotions are largely dependent on how something is perceived by the person experiencing, which may or not may not be the design’s true intent. This opens a complex discussion about design decisions which do not only deal with how to achieve an emotional response, but how to achieve the intended emotional response.
The premier literature regarding emotional design is Emotional Design by Don Norman. It states three chief cognitive responses that a design must elicit to achieve a strong emotional perception.
Here is an article that summarises how each of these three aspects contribute towards achieving a positive emotion from the user.
The other lens to analyse this understanding is applying our understanding of the two subsystems of thought as defined by Daniel Kahneman (system 1 is fast, intuitive, and unconscious; system 2 is slow, calculating, and conscious), which can show that the experience must be analysed by both these systems of thought before an emotional reaction can be formed. The visceral aspect is quick to stimulate system 1 whereas system 2 will reflect on the experience at a much later stage.
Emotions in digital service applications
Digital service applications are simply interfaces for a largely physical service. While the digital experience is an important part of the overall experience, only the visceral and behavioural aspects can be tackled via the digital application. To reflect upon the experience, the user must have experienced the complete application.
While the digital interface may be geared towards appealing to the user’s visceral and behavioural needs, the user will not reflect upon the digital interface but the rather the physical and tangible outcome of the whole experience.
Which means that perception of the digital experience will always be lower influencing factor than the hot and convenient food, the instantaneous doctor’s appointment, or getting one’s groceries delivered towards achieving an emotional response. The objective goal of the user is the most important factor that determines their emotional response.
Emotions in video games
While video games are a digital experience, there is no service behind the scenes that they are interfacing. The experience of digital entertainment is wholly atomic, and therefore the reflection is purely dependent on the perceiver. This means that there is no goal to achieve at the end of the experience. The experience is the goal.
The lack of goal fulfilment will mean that a player will reflect on the various emotions they felt throughout the experience. The emotional response during the stage of reflection will be a recollection of the smaller incidences that elicited momentary emotions by in-game events. This makes the context of a player’s mind very important, for it will dictate how these in-game events are perceived that will contribute to the experience as a whole.
With no tangible outcomes, the outcome solely depends on the player experiencing this. This creates an extreme dependence on a player’s emotional state and how far away it is from the intended emotions of the design.
Why is this helpful to know?
The very nature of an experience without tangible outcomes makes it more susceptible to be valued with accordance of the perceiver as opposed to a service, where objective outcome can prove that perception of a negatively reacting consumer is not a true measure of the quality. On the other side, given the fickle and volatile nature of human mental states, it becomes easy to elicit an emotional response with minimal effort if a designer knows how to target intended emotions. For example, the result screen of a casual game (not competitively ranked) in Halo MCC shows a player all their stats, and pits them against teammates and opposition players. This verbose presentation of quantities has no objective, the performance does not build objectively towards any tangible outcome. Despite the intrinsic motivations of succeeding at Halo, a casual game does not build towards anything. Yet this presentation will no doubt set goals with a comparison, and goal setting is an internalised activity associated with either of confidence, resolve, frustration, or even disappointment.
Yet in Overwatch, the result screen doesn’t simply present all the numbers. The game cherry-picks the best bits of a performance from both teams and presents them in separate categories that does not allow direct comparison. And allowing for the social interaction of voting changes the perception of the experience, transforming it from the sports analyst’s report to a casual match highlight for the every person.
It does have a screen that compares a player’s performance, but only with themselves. It is quite evident that positive aspects are highlighted and the comparison is only with the player’s average self, making sure that there is something positive to see half of the times on at least one of the stats every single time.
While both games track these numbers, the emotional response for each of the experiences is different. Either game may succeed towards achieving a strong visceral stimuli and offering the best behavioural alignment, but without any tangible outcome, the true emotional stimuli depend on what the player will reflect upon. And at the end of the experience, these two different games are presenting the end of their experience to get the player to reflect on different aspects of the game, and therefore will elicit different emotional responses.
This is but one example, which highlights one aspect that will influence the emotional response.
Understanding Emotions
Understanding perception
Perception in the way in which something is regarded, understood, or interpreted by an individual. Which makes perception subjective and inconsistent.
One of the most popular examples of the subjectivity of perception is the Kuleshov Effect.
While the effect itself was studied to explain the importance of editing in film, it can easily translate into any media. The phenomenon of an individual projecting their perception of an entity on another personality is very easily transferable to outcomes of experiences. The experiment demonstrates that viewers will project their understanding of the object on the blank face, and therefore it is not a genuine reaction but the viewers’ perception of what they see. Projecting the personal emotion of sadness or melancholy on a seemingly blank face makes viewers’ perceive the face with an expression of sadness or melancholy. The involuntary action of projecting perception is why the current emotional state of the player is so much important.
Individual players can project their negative or positive emotional response towards an outcome to the experience preceding it, which can make them forget the emotions they felt throughout the experience while it was going on. Or the other way round, albeit rarely, where the collection of emotions felt throughout the experience can overcome the emotions of the outcome. This is known as emotional misattribution. It does not come through retrospection but through an instant labelling of current or the latest, strongest emotion felt during the experience on the entire experience, which is a form a involuntary action. Associating emotion to something that was not the source of the emotion is known as misattribution, and is a common phenomenon of why humans think that their tools, or instruments, are at fault rather than their own skill.
Apophenia
Apophenia is the tendency to perceive meaningful connections between unrelated things. Humans enjoy patterns and associations, especially those which can be interpreted as a series of coherent events. For a practical sense, any series of random events can be interpreted as a story, and if designed with deliberate vagueness, can elicit an emotional response when participated or reflected upon.
In the example above, the outcome was the fox AI agent leading the players to a point of interest. Even though Skyrim is a dense world, not every point of interest has a treasure and therefore on an average, half the incidences would have no reward. Not only that, but the AI agent would lead players to a general vicinity with no superficial intention, and it was up to the player to truly search for and discover the treasure. Yet the pattern was easy to interpret as “a wild fox led me to some treasure”.
This proves that while game mechanics can be intentionally designed to achieve outcomes objectively (Fox AI in Ghost of Tshushima leading to shrines), the irrationality of human mind can easily suppose and reflect upon the same outcome through a non-existing or unintended experience.
Let’s analyse examples of how games utilise to draw emotional responses, and derive actionable guides from them
Emotional design for visceral reaction
Set-ups
Visceral feeling is defined as the non-intellectual perception of stimulus, or what we would call as “gut feel”. There is very little time between stimulus, perception, and reaction. It can be considered as the perception and reaction at the face value of an experience, which is mostly when it is going on.
Visceral reaction play the part of eliciting reactionary emotions, and therefore need to be set up. Let’s take the example of the launch screen of the Outer World.
From a purely behavioural view, the part of the experience offers nothing. It’s not even a transition to mask loading times, it simply exists as a buffer. But the imagery and the music sets up a mood and atmosphere, an expectation of what’s to come. The buffer is for the player’s mind to assume things about the world and immerse themselves into the experience to come, by simply letting their imagination run wild. There are no external nudges, promotions, or inspirations. It is merely a vague and ambiguous setup for the player’s imagination and mental state to transition to derive maximum the upcoming experiences, and also to derive seemingly coherent stories from unintended experiences (what is popularly known as emergent gameplay). This is a superb example of creating a buffer and also a set-up that allows players to detach from the current emotional state and embrace the intended emotional state which the experience hopes to achieve. These techniques are often implemented in menus, cut-scenes, and gameplay before sections of the experience that hope to achieve a different emotional response and influence the players emotional state beforehand to make it easy.
Another example of a set-up during real time gameplay can be seen in level design. FromSoftware games are famous for employing real time gameplay tricks to set-up emotions of wonder and curiosity, or dread and foreboding with the almost painting-like composition of the level design. Seeing a castle bathed in sunlight may awaken a sense of exploration, which is a set-up for many positive emotions such as wonder and awe, or very memorable subversive experiences that elicit emotions such as horror or fear if the interiors of the castle are contrasting to the initial setup. It is easy for a player to feel these emotions viscerally if they have been set-up properly.
(It is common for level designer to employ similar psychological sciences as used by UX designer, and hence it is advised for UX designers to learn the basics of level design to gain useful insight. Even if not professionally useful, it’s fun)
Ambience
Ambience, or the character and atmosphere of a place, is the sandbox in which players will experience emotions. And players are already aware of these norms.
What kind of an experience does one expect to have in these two places?
Each of these settings has an expectation of a different experience, and is a set up for different emotions. At a surface level, this is what we call “look and feel”.
The look and feel of the game, or rather the ambience, is a great technique to elicit a visceral emotional response.
While both of these games involved team based combat and shooting, to a first time consumer they will present completely contrasting ambiences. Brawl Stars will allow a player to feel safe, informal, and even take their initial play session casually, disregarding consequences. Call of Duty mobile might make a player feel serious, determined, and even sincere from the onset.
Body language
Taken in the sense of what we know as human body language, this would be the nonverbal communication in which physical behaviours, as opposed to words or imagery, are used to express or convey an emotion. Motion to which a player can attach positive emotion to is always inviting or appealing, and motion to which a negative emotion is attached is always repelling.
The motion design incorporated into the UI design of Tennis Clash is associated with a positive body language, with affirming and cheerful gestures. It’s inviting and every moving element on the screen is hoping to delight the player. The body language of the UI (and the general art direction) is also in alignment with the sporty nature of the game.
Meanwhile the UI of Injustice 2 reflects the utilitarian, mechanical, and functional nature of the theme the game represents. While the player may still be delighted by the outcome of the lootbox, the UI itself is not designed to elicit any emotions. The body language of the UI is as cold and functional as the themes of the game.
The body language is another tool for eliciting emotions as humans can easily derive stimulus from their subjective perception to which there can be an emotional response.
Looking at the example of the Magnum in the Halo franchise, it is evident how much the body language (a summation of art style, animation, SFX, VFX) can change the perception of an item in the game. Each of these could be doing a static amount of damage to an enemy, but the impact of the sound and animation can add to or subtract from the perception of power in a player’s mind, and in turn change their emotional state from calm, due to safety assured by a powerful weapon, to anxiety, by being equipped with a weak weapon. This would then become a powerful tool to influence the emotional state of a player during a gameplay sequence hoping to make a player feel assured, or terrified.
Emotional design for behavioural appeal
Evolutionary and psychological triggers
Over the course of thousands of years, humans have developed affinities and reluctance towards certain common natural events. These evolved due to our biological characteristics which put us at a natural disadvantage or advantage in specific situations. These natural tendencies are involuntarily governed by emotional impulses, such as fear to keep humans away from the dark (where humans are at a disadvantage) or delight at finding a tall, vantage point (where humans are at an advantage). Using these in interactive experiences is extremely impactful, and quite easy to pull off.
Taking an example from Hellblade : Senua’s Sacrifice, the general world of the game is well lit and provides adequate spatial awareness. Then to achieve a fearful emotional state, the game takes away the comfort of light and awareness in an aptly named section known as the Blindness Trial.
From a purely technical implementation point of view, this is easy to achieve. Removing light sources, adding a post-processing filter to take away vision clarity, and enhancing the audio, yet the impact of the sequence is incredibly high for players. The same trick is excellently employed in the game A Hat in Time.
When it comes to natural phenomenon, it is important to note the absence of any external agent seeking to purposefully elicit these emotions. The darkness, closed spaces, long corridors, serene waters, or any other natural occurring phenomenon is simply that — a characteristic of the environment. It is the perception of these events that elicit the emotional response. It is a combination of multiple factors that should be employed together, as night sky can be perceived as a calm and beautiful event triggering positive emotions whereas dark, poorly lit corridors with lurking predators can be terrifying events triggering negative emotional responses.
Interaction
It was earlier mentioned that “the experience is the goal”, and interactions will make up the bulk of it. The interaction can be designed to mimic the real world delights and pains, which can be one of the most impactful way to elicit strong emotional responses.
The selection menu is extremely archaic in the Last of Us franchise, requiring a player to manually use the D-pad to navigate around it while the character is immobile. While this may appear as bad UX design when compared to the sleek and convenient weapon wheels of Doom or any other shooter game, it’s a deliberate decision to mimic the infuriating act of stopping and opening a messy backpack to find something in the real world. This, in turn, enhances the sense of discomfort or panic when a player needs to select a very specific item during a tense situation, which all humans experience normally when having to retrieve something from their luggage when in a queue or some equally pressing situation. (You can read more about weapon wheels here)
It seems like the antithesis to “eliminating friction of the interaction” philosophy of service design, which is always aimed at speed and reliability. But game UX designers must remember that players are here to experience the struggles and joys of the experience offered by the game as there are no objective goals to meet.
The Behemoth(s) in Battlefield 1 are meant to be operated by multiple players. It has inherent flaws that make sure it can’t be operated or performs well due to the actions of only one player, and is therefore no the best at any singular function. Driving the tank is a terrible experience in itself, as it would be during WW1 with the dated technology, yet it arouses feelings of safety, efficiency, assuredness (for a good team of operators) or feelings of panic, frustration, at risk (for a poorly coordinated team of operators). These feelings are catalysts for multitude of positive and negative emotions.
Matching interactions and function to player expectations
Choosing the correct interaction to match the action on function is very important. In service design, the goal is to increase efficacy by allowing users to take the right action at the right moment.
If the player is expecting to be making critical and profound decisions about tactics and strategy, is it best to incorporate the difficulty of moment-to-moment execution of the action? Perhaps not, because it brings down the efficiency of the experience to fulfil player needs.
There are numerous complaints about when the gameplay of Helltaker transforms into the equivalent of the bullet storm on the final level, whereas the rest of the game was a puzzle. There are similar frustration when experiences become something else without accommodating or making sense to the players, thus eliciting many negative emotions.
A good example of changing interactions and functions but preserving the intended emotional state is It Takes Two, which keeps the interaction and functions relevant to the player expectations. They’re concise and coherent, with a natural and rational transition between them, which makes it a delight to experience the breadth of the game rather than the frustration and annoyance of having to learn or do something brand new all the time (even though everything introduced is brand new in the game).
Emotional design for reflective appreciation
People reflect upon an experience at different stages, sometimes immediately after it has concluded, sometimes much later, sometimes on a second or third occurrence, and sometimes while it is going on. Reflecting on something is not an immediate or visceral reaction, it is deliberate and uses the part of brain that is much more equipped for slow thinking process, to evaluate the experience.
“…considers the rationalization and intellectualization of a product. Can I tell a story about it? Does it appeal to my self-image, to my pride?”
Don Norman on Reflective Design
(You can know more about the 3 levels of design for emotional designer here)
To truly design for reflection, designers have to understand what appeals to players who have intrinsic motivations to pick up the game in the first place. For simplicity, that can be boiled down to story and challenge.
Humans love stories, they tell them all the time. The easiest way to remember something, coherent or not, is to turn it into a story. The storyteller is always a centre of attention, and if the storyteller is a participant of the story then there is always some leftover respect from the audience after the end. The outcome of the story may leave the audience feeling many things, from sadness to awe, but stories that touch the audience are always remembered and propagate.
The second aspect is the challenge overcome to achieve the outcome, which is why the narrator is remembered strongly if they are also the participant of the story.
Story
While we hold stories to high regard of transferring emotions, the truth is that they are simply perception of events, connected or otherwise. This article has earlier stated that it is the subjectivity in perception that adds or subtracts value from a nexus of events, and can represent more than intended by the creators. To design stories for reflection, designers have to leave room for interpretation. This comes in multiple forms.
In character driven static stories, players often reflect upon the actions they took as the character, while playing, and use it to rationalise the outcome of the pre-determined outcome.
In Last of Us 2, the players are left reflecting on the moment-to-moment story beats presented in the cutscene, and at the end they reflect on the outcome of Ellie’s (the player character) life. “Were the decision worth it? Was the violence worth it? What did we learn after investing 60 hours of our time in this?”, are the common ways to reflect upon this story. The ambiguous sections of the plot will be reflected upon with a “what if” mindset, or the various interpretations of vague outcomes will crop up on Reddit and YouTube.
But it critical to remember that players want to be participants in stories, not spectators. And video games are an interactive medium which makes it easy.
Last of Us 2 has multiple, system driven, ambiguous, and open ended stories that are completely player interaction driven, which gives rise to stories where the players are active actors. Allowing players to be actors in the stories make it much more personally embedded, and are easily remembered when the experience is reflected upon.
This is a great example of why video games can exceed beyond static plots to become experiences that are easy to reflect on. There are many examples where the subjectivity of perception enhances stories beyond the intended impact.
There are games that are completely designed with only this one intention in mind. Dwarf Fortress, a game that is neither viscerally nor behaviorally pleasing, has stood the test of time simply because it allows players to reflect upon their experience easily.
Some games find a middle route, where the plot is static but the stories can be subjectively interpreted. This is a common ideology employed in literature, such as Lovecraftian fiction. While the plot is straightforward, it is the ambiguous characters, locations, creatures, lore, or situations that capture the reader’s imagination. And a captured imagination will always lead to questions, and questions will lead to the consumer to ruminate on the experience and the ‘existence’ of things within it.
Bloodborne (which does borrow heavily from Lovecraftian style of fiction), leaves much to the players imagination. Not just in the form of objects and entities, but also through interactive gameplay mechanics.
Not just objects of interest, but complete mechanics are left up to players imagination (read an example here). The steps taken in the example can clearly point to a leftover development artefact, a discarded feature, or even a bug. But to a captured imagination, none of that matters. We can see the subjectivity of perception in action where meaning and value is attached to something incoherent.
To design games to be reflected upon
- Leave ambiguous elements in the experience, either by design or as a mistake (enough examples exist to show that both work)
- Allow players to be participants of stories
It even works for games with no “stories”, for actors will justify what they participated in. (The lesson is repeated again — stories are a collection of events). The atomic decisions that a player takes in a game of Hearthstone will be narrated as a story by the player when revisiting the experience. The “Play of the Game” in Overwatch is nothing but a self contained story, with it’s own set of events that make coherent sense even though they were neither planned nor do they have characters or settings, they are just interactions and decisions.
(Designing Games : A Guide to Engineering Experiences by Tynan Sylvestre is an amazing read on this subject)
Challenge
An experience is only as good as the ups and downs in it, either for the consumer or the audience. The effort that the player goes through is always used to rationalise the outcome of the experience, and must be considered carefully.
Regardless what they thought about the story of Dark Souls or Doom : Eternal, there is no doubt that players who completed it reflect on the challenge they faced along the way. Yet challenge does not only stem from the systems or the game’s intention, it can also stem from ambition. In the game Chivalry 2, a medieval combat simulator, players can choose to conquer an objective in multiple ways. Yet there are some actions a team can take that result in extravagant outcomes. It is not the game that offers a challenge, but the player’s ambitions that can provide the challenge within the experience. Not all challenges will be reflected upon as “it was difficult to do” but also as “it was hard to pull off”, where the proactivity of the player is the defining feature. When told as a story, the tales of how a player built something amazing in Minecraft is just as awe inspiring as someone beating a boss in Dark Souls. And when the stories can be recounted again and again to inspire awe, the player will reflect upon them often.
Conclusion
There is no implementation in UX design that cannot be objectively researched before designing, or cannot be evaluated after designing. There are a number of emergent techniques to research emotions in an experience. A few resources are linked here
- https://uxpamagazine.org/the-future-of-ux-research/
- https://uxdesign.cc/how-to-research-emotion-3f85b3afb1c
Due to subjectivity of perception and the intimate nature of emotions, it is very difficult to target them in an experience. But looking at successful experiences that do elicit emotion, intended or otherwise, can allow us to analyse what works and derive some actionable insight.
Note from the author
These articles are not definitive statements but rather conversation starters. If you’d like to converse, contact me. If you feel like supporting my efforts, buy me a coffee.
