A room for understanding
The dawn of a new workspace
Before computers existed, carpenters, blacksmiths and stonemasons slowly adapted their workshops, benches and tools to perfection. The capabilities of their bodies, the qualities of the material and the desire to create new objects shaped their workspaces.
Over the last twenty years computers have infiltrated almost every profession. Designers, typesetters, doctors, farmers and even car mechanics now use computers in their jobs. Although this has brought us many benefits it did come at a cost. Where previously we could use our body and the space around us to understand the challenges we dealt with, these days computers have limited our physical possibilities to our head, eyes and the tips of our fingers.
Can we take the best from both worlds? Can we re-introduce our body and the space around us in our work, while still keeping the benefits of digitisation? Could we work with information as a carpenter works with wood?
I. The intelligent use of space
Imagine walking around in a workshop of a master carpenter. In the middle, surrounded by equipment and materials we see her quietly working on a new wardrobe. She doesn’t just use her hands, she uses her whole body to transform the wood into her ideas. Never sitting down for very long, she uses the room as an enormous tool to craft what was once a tree into an intricate object.
In his paper The Intelligent Use of Space David Kirsh explores how experts such as chefs and bakers use their work space to make complicated tasks doable.
‘How we manage the spatial arrangement of items around us, is not an afterthought; it is an integral part of the way we think, plan and behave.’
David Kirsh
Use space to amplify mental capabilities
Experts can go to great lengths to organise the environment around them. Not only do they use space and tools to amplify their physical abilities, they also use it to amplify mental capabilities.
Extend memory
We can use space to extend our memory and to simplify remembering. Examples are all around us: a list with items to buy on the fridge or an envelop placed next to the door to remind you to take it with you when you go out.
Sense making
We also use space to do tasks that could be done by mental effort alone, but are remarkably easier by using the space around us. You could for example think through a complicated user flow in your head, but drawing it on paper makes the task a lot easier.
New ways of seeing
Research also suggests that the memories of artefacts such as drawings and objects are very hard to reinterpret by just thinking about them. Only by redrawing or revisiting the artefact can we come up with other interpretations. Expressions such as ‘taking a step back’, ‘seeing the bigger picture’ or ‘looking at it from a different angle’ can therefore be applied quite literally.
Organising space for complicated activities
Chefs, carpenters and other experts use the space around them to help them think. Kirsh observed that experts ‘constantly rearrange items to make it easy to track the state of the task or notice the properties signalling what to do next.’ By doing this experts have to do less hard conscious thinking and can stay longer in a mode where they can rely on learned rules.
Simplify choice
The first thing they do is simplify choice. When making a chair for example, a carpenter does not use all the tools she owns. She will pick the tools she needs from the wall and place them on the bench in the order she’s planning to use them.
We can see this as a trade-off. She limits her options by leaving some tools out of reach, but the smaller selection makes it easier to focus on the task at hand. She doesn’t have to worry about picking the wrong chisel, or spend a long time looking for the right one.
‘Once a context of action has been triggered, the local affordances make clear what can and must be done [and] prevent us from considering irrelevant alternatives.’
David Kirsh
Furthermore, she can apply the principles of gestalt to organise her workbench. By grouping all the chisels, it becomes easier to ‘read’ the environment, and to effortlessly tell hammers apart from chisels.
Simplify what is next
Space can also help us prepare the order in which the tasks needs to be done by making it easy to understand what to do next. By placing the right width chisel on the workbench before she starts to draw the lines on the timber, she doesn’t have to think again about what to do next when she’s finished drawing.
‘If I can arrange items to display the sequence they are to be used in, then I don’t have to remember that order.’
David Kirsh
Simplify how next
Not only does the environment show what she should do next, it can also indicate how she should do her next action. By placing the chisel after the hammer and with the sharp side pointing away from her, she can read from the environment that she should first pick up the hammer and second that she should pick up both of them at the side of the handle.
Experts create little assembly lines of tasks, switching between short bursts of high cognitive environmental preparation tasks and longer lower cognitive execution tasks.
II. Understanding through interaction
One of the reasons experts use space to amplify their mental capabilities without much effort is because our body is deeply integrated in the world we live in. Our bodies are optimised for living in a world where space, weight and distance all play a role, and they are capable of advanced spatial-temporal problem solving by combining our mind, limbs and senses in remarkable ways. Contemporary computer design, however, is optimised for tasks that barely use our physical abilities (besides keeping our eyes open and our hands on the keyboard) and limits us in using the spatial organisation possibilities that craftsmen exploit so effortlessly.
How might we redesign computing to make use of our body and the world around us? Can we create a computing system where we can turn, twist and tweak information with our hands? Where we can be free to walk, build, model, compare and invite others to work with us? Can we invent what computing pioneer Bret Victor called: ‘knowledge work that incorporates the body’?
In his book Where the Action is, the Foundations of Embodied Interaction, Paul Dourish aims to set out the foundations for this new computing paradigm. In his view computing should be centred around ‘the creation, manipulation, and sharing of meaning through engaged interaction with artifacts.’
Using the insights in Dourish’s book I’ve formulated seven ‘rules’ for the future of interaction design.
1. Observable-and-reportable
One of the reasons that it is enjoyable to watch a master carpenter at work is that we understand what she’s doing based on our general knowledge of how physical interaction with objects works in the world. Watching a computer programmer at work is a lot less enjoyable because, unless you are familiar with all of the layers of abstraction, just observing the actions won’t provide you with much understanding.
Dourish calls the way we can make sense of a carpenters actions a system that is ‘observable-and-reportable’.
The design of the system should be done in such a way that interaction ‘reveals the purposes for which it was designed and the ways in which the designer intended it to be used.’ The user should be able to ‘develop an understanding of the consequences of objects and actions in the system’ through observation and interaction.
We can make a system easier to understand by making sure that the behaviour we see:
- ‘strongly connects to the system’s actual behaviour’
- ‘strongly ties to the actions that caused it’
- ‘emerges with our interactions’
- ‘is precise in communicating how it was caused by the exact piece of work that we triggered in exactly its current configuration.’
It’s a good idea to build systems that tell you what they’re doing
— Paul Dourish
2. Representations should relate to the world
The best way to create systems that tell what they are doing is by having its components refer to how people categorise and understand the world. This can be quite a challenge since computing has been designed as many layers of abstractions on top of each other. However, because most users do not care about the internal workings of the computer and aim to achieve something in the outside world, almost all elements in digital systems can be represented as related to elements in the physical world. The thumbnails used in e-commerce to represent the product are a good example of this. We should design our systems in such a way that this relation between the world and its representations is clear to the user.
Not only can we use representations to make things easier, we can also use it to make things harder or impossible. Like a saw that is pointed only on one side, we can make data objects that only allows certain kinds of activities. For example if we want to make it impossible to connect two spatial digital objects we can use spheres. If, on the other hand, we want to encourage connections we can use cubes.
Objects can be designed so that they fit together only in certain ways, making it impossible for users to connect them in ways that might make sense physically, but not computationally.
— Paul Dourish
3. Physical representation
Many of our digital systems follow these ideas. Moving towards tangible and spatial computing can, however, provide us with a large leap forwards. Spatial computing systems can provide actionable, observable artefacts from the start. This allows us to create interfaces which exploit, in Dourish words: ‘the skills we already have: skills of exploring, sensing, assessing, manipulating, and navigating’.
Dourish continues that instead of relying on multiple layers of ‘abstract, symbolic styles of representation’ we can make use of the fact that we have a body and are in a world and ‘transition from symbolic representations to physical ones’. This is not simply about mapping symbolic representations on physical objects, but about exploring ‘how physical interaction models can “hold on” to the symbolic’.
Dourish suggests that we can do this by designing in two directions. First, we can manipulate ‘digital information and functionality through the manipulation of physical objects.’ Second we can transform the physical environment through computing. ‘Information can be “displayed” as changes in light patterns, audio signals, movement of physical objects, and so forth’.
We can imagine an information workshop that is filled with applications for storing documents, for acting on numerical data, for recording conversations or processing email.
4. Direct manipulation
We act in the world by exploring the opportunities for action that it provides to us — whether through its physical configuration, or through socially constructed meanings.
—Paul Dourish
One of Dourish’s core arguments is that we find meaning in the world through acting in it (in contrast to reflecting on it). It therefore makes sense for computing to be built around direct manipulation. In their paper Interaction and the Epistemic Potential of Digital Libraries Karl Fast and Kamran Sedig distinguish three important qualities for direct manipulation that build on the ideas of observable-and-reportable:
- We can make rapid and reversible actions that provide immediate visible feedback
- We can interact through physical movements (instead of using an intermediary language)
- We can effectively understand the object’s affordances and quickly learn which actions are possible and which outcomes can be expected
5. From space to place
‘Spatial models provide a natural metaphor for collaborative systems design. [Space can be used] as a way for people to manage their accessibility, orient toward shared artifacts, and provide a “setting” for particular forms of interaction.’
—Paul Dourish
Once we bring computing and physicality together it makes sense for these spatial models to be somewhere. To make the most of space we should design for the ability to grow familiar with it. It’s what Dourish describes as the transformation from a space that has only physical value to a place that also has social value.
Like carpenters who carefully build up their workshop over the years, the best way to redesign computing is to focus on the relationship between activities and the space in which they take place.
For example, we can find ourselves in many positions in relation to our work, we can be very close, take a few steps back or lower ourselves to get a different view. All these actions and positions should alter how the system behaves.
As seen earlier in the work of the carpenter, a large part of what experts do is configure their space to suit what they are doing. A high level of customisability is therefore required to make sure a space doesn’t draw attention to itself and allows users to focus on the tasks at hand.
The challenge and opportunity for knowledge workers of the future is to design ways in which physical and digital spaces can be adapted fluently and interdependently to meet the users’ immediate needs.
6. Design for collaboration
Since our daily experience is both physical and social, it makes sense to construct our system as a social system from the ground up. Multiple users in multiple locations should always be aware of each other’s work.
If we think about this system as the manipulating and transforming artefacts, then it becomes much simpler to design for shared feedback. As Dourish writes: ‘All users will see the results of an action because they all see the same artifact.’
7. Design for dynamic exploration
Computing is at its most powerful through its capability of creating interactive visual models that help people understand the world better. This dream of interactive models that can be created in the moment is what Bret Victor called ‘the dynamic spatial representation of thought’.
By using our body and the world around us we can lower the abstraction of the information we deal with, without doing away with the complexity. As a result we can better deal with complex issues without being overwhelmed.
III. A room for understanding
To explore what could happen in a room for understanding, I’ll build upon the work of Karl Fast and Kamran Sedig. In their paper Interaction and the Epistemic Potential of Digital Libraries they bring interaction and understanding together by defining interactions as the ‘low-level mechanisms by which people explore, break down, analyze, and recombine information-bearing components.’ These ‘interactions are helping people create knowledge, develop understanding, and acquire insight from digital [data]’. Furthermore they distinguish fourteen types of core interactions such as fragment, filter, annotate and group (shown in italics in the text below). I’ll use a selection of these concepts to walk through a potential room for understanding.
A workshop visit
Let’s visit a design studio in the near future, where a small team works on a model to understand how a busy crossroad can be made safer for cyclists to use.
When you enter, you see that the room is not just a space for desks and chairs, it’s a space where you are free to walk, collect, compare, compose, model, and interact with information. On the desks there are magical objects such as spatial models, lenses and filters. Old projects are stored on shelves, large tools are used to perform special actions on data sets, and multiple people work together on creating interactive models. One of the designers volunteers to tell you how they work.
Building a collection of digital objects
The team starts by collecting the relevant contextual information. They interview several commuters to understand the choices they make. Furthermore they find several earlier models of modes of transport, travel times, speeds and costs.
They fragment the data sets to create separate tables for cycles, cars, busses, trains and pedestrians. They split the interviews so it becomes easier to the compare the answers to the questions they asked.
Whilst working on this, they annotate the things they are collecting. The system has automatic annotating build in: heat maps show which part of the models have been used most often, and auto-tracking remembers who made the latest changes.
Making sense of the collected information
To make sense of the data they use several techniques. They filter the streams of traffic and try to understand how they relate.
One way of working is stringing a range of filters together, one discrete filter allows them to focus only on the current cycle usage and another range-based filter makes it possible to see how usage differs between the different hours of the day.
Someone created a magic lens which makes it easy to switch between several modes of transports, and find out how they relate to the group. A second magic lens allows them to immediately see the areas where people got stuck during testing.
After a bit of work they are able to start probing an early prototype of the model. Probing can be triggered automatically based on distance and how you position yourself. When you move closer and bend over to inspect the model you can see small individual stories of people on bikes. When you lower yourself you get a different point of view, and with your hands you can actually interact with the model and cause the data to change.
Stepping back activates a semantic zoom and changes the small cyclist into dots and with another step backwards, the dots transform into a streaming line.
Adding focus
Next, it’s time to create some focus. They cut some of the introduction parts of the interviews and decide to focus on two of the interviewees experiences. For clarity they also decide to only create a model of one specific busy crossing to serve as an example of the effects that can be expected elsewhere.
They rearrange the model several times, sometimes exploring the relation of the cyclist to the car drivers, others times focussing more on the effects on the current cycle hire scheme.
The model also makes it possible to switch between open and closed states to explore the underlying data. When closed, a data object takes up little space and can be moved around as one big unit, but when opened, the underlying items that make up the objects can been seen and interacted with individually.
Creating new knowledge
The final step is composing a new information objects by putting together individual pieces into a cognitive model. The aim of composing is to create a new unified model.
One powerful technique that computers make possible is the ability to animate almost anything. If we deal with dynamic systems of information flows, we can use animation to make the flows of information and the relations between the various parts of the system easier to understand. Doing this fluently and freely is what Bret Victor describes as the dynamic spatial representation of thought.
Summary
By bringing computing, bodies and space together we can start exploring how spatial interaction design can create the workspaces of the future, enabling new ways of creatively understanding the world.
- We use space and artefacts to amplify our physical and our cognitive abilities
- We use space to create assembly lines
- We can redesign computing systems to make it easier to interact, observe and share what we do with others
- Interaction is the key connector between us, the world and others
Perhaps interactive model making can become the design job of the future.
Interested in learning more? Have a look at these two talks:
- Karl Fast — Deep Interaction
- Bret Victor — The Humane Representation of Thought
