Umwelt and virtual experience models

The haptic device in front of John Latham’s painting “Full Stop” uses ultrasound to create the sensation of rain falling from the bottom up in the hand. The Guardian.

Why do I insist so much that the UX (human-machine relationship) is a function so closely related to a social/biological condition, even more than simple system design? Because the human interface — which every effort of UX makes use of to provide humanly appropriate applications and processes — is a cultural interface. We could easily trace the specific cultural origin of all the technologies used in creating any systems that surround us, however, exploring this theme would transcend the intentions of this essay.

UX is not a hard science; it is true though that processes used in its constitution as AI (with steps such as mapping, data mining, datasets and so forth) are based on hard sciences, but this threshold is sketchy. This is because the logical constitution of programming used in artificial intelligence, with algorithms that use inductive propositions (I shall talk about Inductive Learning Algorithm (ILA) soon), is extremely more human than the straight cold calculation of pure mathematics.

So UX par excellence cannot be carried away by the calculation solely, because after all, this would go against the core of what its title proposes: ‘User’ as human and ‘Experience’ as a human concept.

Umwhat?

Bee in its habitat on the left, and the Umwelt with elements of its perception on the right, conceptualized by Uexküll in 1934.

I usually propose a framework for the perception of the phenomenon of the human-machine relationship by the study of Umwelt. Umwelt is a term used by Jakob von Uexküll on early studies of theoretical biology, dating from 1906, which indicates a perceptual sphere around individuals. Terms such as ‘Externality’ or ‘Sphere of Perception’ would be good adaptations too. This idea had a very profound implication in the perception of the relations between consciousness and space, because, under this notion, it sustains that the individual carries the world with him, as possible registers, that is, the world is not an encounter but a constant re-encounter. It has a huge impact on how “perception” is studied:

Uexküll puts forward, according to Scheler, “more appropriate ideas” than the “mechanical theory of life” endorsed by Charles Darwin and Herbert Spencer. (More)

In practical terms, a bee’s Umwelt (world around) would be an area of approximately 4 meters in all directions in the visual dimension, a few kilometers in the perception of odors, a few centimeters possibly in the auditory dimension, and so on. Its projections occur within these spheres, the set of which is configured as its objective reality.

The question of ‘perception spheres’

LIDAR for autonomous vehicles

Uexküll provides several interesting examples of these records to support his theory; later appropriated by none other than Heidegger himself. The concept ended up spreading up from biology to anthropology, ethology, but also phenomenology and existentialism, and more recently to neurophysiology and studies of Machine Learning. Now, the term is appropriated by Artificial Intelligence (AI) studies in an effort to signify the perceptual ‘bubble’ that elements such as a LIDAR provide to the machine as data from the outside world. In both, the biological and technological fields, the world only actualizes (becomes real) during the journey of the individual (or machine) in this world, and developing research based on this assumption is an important point to move away from the Cartesian dualist premises that have been resisting in the scientific world for so long.

But what about UX?

Well, since it is the practice of applying tests whose result focus on a perfect leveling between information and action, from the perspective that prioritizes experience, it is enough that we consider the human being as a being endowed with a perceptual sphere, as any other complex animal such as higher mammals, and that its bio-cognitive response derives from the stimuli provided by the system, that this person will receive in his Umwelt.

1. Within the Umwelten of people, we will find perceptual spheres limited by physiological characteristics, such as the constitution of the senses of one’s body, whether tactile, visual or auditory. Thus, physical limitations inherently define the experience.
2. The spheres are contained in the individual, that is, they are ‘modeling spheres’, that depart from culture and project possibilities of use or action, as well as recognition of elements of the world.
3. In the wake of Uexküll, it is worth reminding that they are projective spheres, that is, the individual’s experience refers to a reconciliation with concepts or possibilities previously established in their culture.

The question of Error vs. Creativity

These spheres are, objectively, spaces for the perception of the physical world that individuals have around them, as stated before. Yet the first lesson provided by the observation of these relations of registering a ‘reality’ — noticed since the time of Uexküll’s studies — was that they were not direct records of “physical reality”, but rather ‘projections’ — many of which creative — of a set of elements that surround the individual.

Passing briefly over the principles of phenomenology, in the mind of individuals immersed in social organizations, we do not perceive objects as pure matter and shape, but as objects, that is, as elements with an objective, function and relationship. A chair is an inculcated image of the sitting or resting functions or even a social status (i.e. The origin of the term Cathedra). We teach an infant to understand that chair is everything shaped in such a way, that provides a capacity of sitability (the ability to sit on). Until their actions are not hindered or castrated, the child will climb on the chair, jump, bite, turn upside down. The lesson here is: the senses grasp the shape differently depending on the cultural appropriation over that shape as a function.

Starting from the primordial experience, the modeled image is projected as a possibility to other situations of daily life. Uexküll, while observing the behavior of animals like frogs and dogs, noticed that many times, the animals “mistakenly” registered the elements that entered their Umwelt, but here comes the tricky part: regardless of the error, biochemical responses occur in the animals’ organism, generating real sensations and actions. This is the case of the dog that gets confused with the movement of a leaf pushed by a breeze, and barks at it treating it as a possible animal, given the record of the movement of the leaf and the projective model of characteristic behaviors of other animals, such as creeping. During this process, the image-effect caused real release of adrenalin in the dog’s brain. Another example is the relationship between cats and cucumbers, once cats are famously frightened by the shape of the vegetable, given that the perceptual category of this shape probably relates to harmful animals.

VOID is an interactive sensory art installation created by New York-based artists Sergio Mora-Diaz

In humans this is — of course — more complex, and these exchanged records or mistaken perceptions are easily identified in situations where cognitive-organic responses are typically fear, or more interestingly, laughter among other responses. Creativity occurs by this overlap of registers, conjoining two or more distinct elements ultimately rendering something ‘new’. And why is this so important for UX? Because this is exactly how a system integrates a person’s daily life as a simulation. Apps are registers that resemble prior experiences, and just like the leaf for the dog, sometimes it does it right, sometimes not. How? A bank application is a simulation of the concept of bank already contained by people’s Umwelten. Its possibilities as an application are such-and-such according to the way it structurally resembles a form of service known and registered as ‘banking’ in real life.

An application, whatever it may be, as long as a simulation, comprises a model of representation that appeals to the Umwelt (perceptual sphere) of a human being, and conducts a series of mediated associations to make a certain demand viable. We must keep in mind that a bank application is a simulation of the concept of bank that people already carry with them. The more it provides the right design and cues to the User’s Umwelt, the better are the projection of possibilities over the app, and more adequate the system gets towards fitting into a role.

Example: The Bank

Ikea’s Augmented Reality app. They are good examples of integrating online services that take advantage of the human Umwelt in real life.

Let’s consider that you know how a bank operates, and the sections and divisions that comprise its works in real life; say, receptionist, identification, clerk, and areas as loans, credit card, investments and so on. Consider that you have experienced being physically inside a bank, navigating through sections and looking for solutions. Your Umwelt captured the systemic relations in this space, in order to provide a register of what a bank is, and what are the possibilities it holds. Our senses (Umwelt) carry on all these registers, and social inculcation defines the meanings of them. Once you find an app that closely relates to that experience as a conceptual metaphor of a bank, all these sets of rules will automatically surface and a bell will ring: “Banking”, but without waiting in lines.

The app will evoke the semantic universe of “baking” as you know it, with its possibilities and limitations, and in doing so, it will require a much smaller cognitive load to be grasped as a whole.

You know you can look for a section called Credit Cards, a section for loans, investments and so on, and a great flux-designing of these elements will amount to a great experience. How can an UX developer know the registers of the users to provide a plainly recognizable service? 1. Qualitative research (interview), 2. Quantitative research (data) 3. A/B tests (experimentation).

From the UX perspective, the possibilities for creating recognizable content are endless; a digital bank that understands the dynamics and fluxes of a real bank. A navigating app with fast recognition, understanding the wills behind the use of real-world maps. A delivery app that understands what is a good menu experience to a target group. The ways in which the experience in the app relate to the dimensions of a person’s Umwelt, with tactical choices of interfaces and the ordering of each part of the process, corroborate for a reunion with a composite of concepts of purposes and uses that this person already has in mind concerning a given service.

The limits of these implementations are yet to be tested, but we could claim that a good experience of an app has proven to be the ability to facilitate a service that was done before in real life, and the recognition of this service by the user is a great start.

Be it Uber as a simplified version of “using taxi services”, Fintechs as simplified banks, Waze as simplified navigation and prediction; all applications invoke models of previous functions already contained in people’s minds, and therefore, determine this recognition.

UX would then create, guide and control how this experience becomes a good recollection, through the orchestration of register in people’s Umwelt (sometimes visual experiences, sometimes auditory, sometimes tactile) considering online and offline supports, delivering a general compound called a product, service, or simply, experience.

Understanding Umwelt as a perceptual sphere, as well as its physical and intellectual characteristics, is fundamental not only for the new artificial intelligence sciences, but for designing applications that integrate ever more naturally in the world.