UI Evolution: From Computers To AR and VR Headsets
In this article we’ll learn a bit of the UI’s history, how it was born and how it has become what it is today. Then we’ll learn a little about how UI has been adapted to other digital products such as smart watches. Finally, we’ll take a look at the state of UI in VR today and how it is evolving in terms of technology and Design Systems.
The first UI in history
The first UI, as we know UI nowadays, was created in 1973 by Xerox Palo Alto Research Center, aka Xerox PARC. They developed Xerox Alto, which was the first personal computer to use windows, icons and menus. However it wasn’t a commercial product. Later on, in 1981, they launched Xerox Star, which was a commercial product. Both influenced future developments like the first Apple personal computers, Lisa and Macintosh. To put a little context, January 1st, 1983 is considered the birth of the internet.
UI evolution
Technologies
After these first UI inventions, brilliant brains started to work on this and standards started to come up. As it is usual in tech, the first to arrive is engineering, it owns the tools to create functional things. Design, cannot expect engineering to make these things work and be delightful at the first attempt. So, HTML (Tim Berners-Lee, 1993), JavaScript (Brendan Eich for Netscape, 1995) or CSS (Håkon Wium Lie, 1996) are the basis for what later on will be our beloved Design Systems.
First Design Systems
Before Design Systems, it was about pattern libraries. The need for patterns was not something new. So different libraries were out there to use, but the process was something like this: This set of inputs from this library, I will create the buttons, I do like these sets of icons and so on. Designers and engineers struggled to maintain consistency. Upon the need of achieving more consistency, we went from designing interfaces to designing systems.
In 2006, Yahoo launched YUI, it was a whole system of components so it went a step forward in terms of creating a system instead of creating just independent components or patterns. Learn more about YUI here
In 2010, Mark Otto and Jacob Thornton, Twitter employees, created Bootstrap for Twitter. Learn more about its history here.
The next one was Material Design in 2014. Learn more about its history here.
Designers join the game, product design tools
Obviously, pioneer designers took part in the design of the mentioned Design Systems. However, in this section I’ll talk about the designers who use these Design Systems to create products, not the ones who designed the actual Design System.
So, in order to create easy to use and delightful products, designers need a tool to work with. It started with Adobe Photoshop or Illustrator, which had features to re-create the components visually and work from there, but they lack features to maintain consistency.
After that, in 2010, Sketch was launched, and it was revolutionary since it was built especially for product design and it included several features intended to maintain consistency across the components and styles. Learn more about Sketch history here
Later on, in 2015, Adobe launched Adobe XD. A tool with the same purpose as Sketch, helping designers on creating and maintaining digital products. Learn more about Adobe XD history here
A year later, in 2016, Figma was launched. A tool with the same purposes but cloud-based. Learn more about Figma history here
At this point, designers already have the tools to work properly on digital products. These tools provide designers with design, prototype and testing features.
Beyond mobile and desktop
Sketch, Adobe XD and Figma have the capacity to adapt to more than mobile and desktop apps, they also can be used to design UI for other devices such as smart watches for instance.
As for the design systems for smart watches, due to the small size of the screens, and to the unique way of navigating, different watch-focused design systems came up. The most relevant are Wear OS by Android and WatchOS by Apple.
Wear OS by Android is the equivalent to Material Design for smart watch apps
WatchOS by Apple is the equivalent to Human Interface for smart watch apps
UI in VR
It is obvious that the industry has understood that a design system is required for a product to become mainstream. Designers need an organized set of components to create.
Technologies and devices
As I mentioned before, engineering comes first. VR apps are developed using 3D engines based on C#. These are Unity and Unreal. Worldwide known as the main tools to create video games. This is the first interesting point, how the world of apps starts to mix with the gaming world. This fact brings on the table 3D modeling tools such as Cinema 4D and Blender, just to mention two popular ones.
As for the devices, there are several players out there, similar to the smartphone market. Each one with its own resolution and properties. Here you can check out the most populars.
Design Systems and frameworks in VR
As for Design Systems as such, the market in VR is not, by far, as mature as it is for mobile and desktop. However there are some solutions that can be used as a starting point and inspire what is coming.
Unity UI Toolkit
UI Toolkit is the newest UI system in Unity. It is based on standard web technologies so it allows you to define reusable components and styles. However it is far from being a Design System as such, it just gives you the tools to define your design system and maintain it.
MRTK
Mixed Reality Toolkit is a framework from Microsoft that has a predesigned set of components and interactions. It has a View Only Figma file too.
MRTK just launched (2022) MRTK 3, which introduces a big improvement when it comes to UI.
Oculus Integration
Oculus Integration is an SDK from Meta that provides pre-designed sets of components and interactions.
As we can see, the market is still engineering focused. For now, as a designer, it is difficult to jump into the opportunities that VR offers to UI without Unity or Unreal knowledge. However it is not impossible, there is information about how to design UI for VR out there waiting for you, but, be ready to break your 2D mindset.
FloatGrids VR and AR Design System
In order to make this transition easier, I designed and developed a Design System for VR back in 2021. It is available in Figma and Unity. The idea was closing the gap between engineering and design and at the same time provide some tips for designers when it comes to designing UI for VR. You can check it out and try the mroe than 50 components, layouts and templates here for Figma or here if you want the Unity package, it’s free!
UI tooling for VR
As for the tooling, the history repeats itself, but with some learnings. So far, neither Sketch, Adobe XD nor Figma are ready to provide the new features that UI in VR projects require. However, as it used to happen with Adobe Photoshop and Illustrator back then, we still can use them to design UI for VR.
However, there are some XR dedicated prototyping tools such as ShapesXR, which has MRTK and Oculus Design Systems available to use and prototype with them.
Conclusions
Imagine being there when the first design systems were being conceived. As I see it, this is where we are now in terms of UI for VR.
Engineers and big players such as Meta, Google, Microsoft, HTC and Apple are working on delivering easy-to-use and delightful experiences for both VR and AR. In terms of VR Meta is clearly taking the lead. I wonder what Apple is working on, rumors say that they will launch its headset in 2023. What is for sure is that its device will come with a VR/AR tailored Design System. However, I wonder if it will actually be a headset, they are known as launching products for day-to-day use, will they launch actual AR glasses? Just normal glasses meant to see better or protect from the sun but enriched with AR features? I don’t know, but what I foresee is that the XR world will become wild in the following years.
