How to design Virtual Reality menus that do not suck
New paradigms for menu design in Virtual Reality.
Have you dreaded over the menus in Virtual Reality whenever you have played a VR game or tested a VR application? Did you get tired of the same copy-pasted 2D menu with boxes to click using a handheld controller over 360° of space? Though it may seem like what is the need to rethink menus in VR, it is quite clear from the user research that the 2D menu interface is not an efficient way to handle inventory in the virtual world. The virtual world is still a reality that needs inventory management systems just like in the real world.
VR should be as close to real-world as possible.
Recently, I got an exercise for a company to design menus in VR. I explored new ways that have been tested or under research for menus in VR.
My thought process for approaching the exercise
- Starting with existing user guidelines for the web and mobile: Even though it may seem counterintuitive, a lot of user experience practices still apply to virtual reality design. For eg., Fitt’s law (which describes that it takes more time to interact with things far away) applies equally strongly in Virtual Reality or perhaps to a bigger extent. It is because real body movements are involved in VR that can tire people very quickly if people have to move over long distances to interact with things in the virtual world.
- Ask people about their experiences with VR and the real world: This may come as a surprise but I asked people how do they find things in their day to day life. For eg., I asked a friend that if you entered a room in a library and have to find a book, what would you do? The answer was to go through the bookshelf, read the labels and pick the desired book. (Hint: The bookshelf and the books arranged in it are a menu)
- Academic research in VR/AR is very handy: Academic research is a perfect resource for finding out what works in VR and what not works. It is especially useful when you do not have a large user base to test things out. Academic research could be applied to any emerging technology where direct users are not available.
- VR is a replication of reality: This is related to point 2 above. Each experience and scenario may need a different approach to handle the inventory of that world.
Live objects as inventory holders
The menu systems in a virtual world can be integrated into the interactions with an object. Fantastic Contraptions is a great example of this technique.
The original 2D game has the menu items on the top. A person can select any item and place it on the objects below.
The guys at Fantastic Contraptions found that the 2D version sucks in VR. The guys over Fantastic Contraptions came with an amazing idea to utilize a live cat as a medium to carry items required in the game. It follows the user everywhere and provides the user with a “bag” from which the user can draw out items to make their contraptions.
This works because we as humans have evolved to use tools. Playful building experience is much more engaging and enjoyable than picking items through clicks.
Models as Menu Items
Models are a beautiful concept to design menus in the virtual world. Every object in the real world is a 3D model that we can interact with physically.
Take an example of this table with books kept on it. Imagine that there is a VR application that allows you to enter a classroom for a lecture. You may need some reference material for it. A traditional application will have the same old and boring 2D menu to select the paper or book. Then you select the door to enter the lecture hall.
But the new VR application could have a table outside the lecture hall with the relevant material (with the teacup or your favorite drink). The user picks up the relevant teaching material and enters the hall as if it is a game.
Menu as shortcuts
This is a very innovative method of inventory handling. We do keep a lot of things close to our own bodies every time. For eg., the purse or wallet, phone, bag and much more. If there is a fighting game, it can have a shortcut of drawing the weapon by a gesture on the back. It would be as if you are drawing the sword from the back just like in the real world. Other utilities could be drawn out from a utility belt just like Batman does. This could turn out to be way more fun than scrolling around a 2D menu.
Radial Menus
Radial menus are definitely an improvement upon horizontal or vertical menus in the sense that it perfectly incorporates Fitt’s law. A lot of items could be fit in a circular space. The menu can appear anytime the user wants it and can be accessed easily by the touchpads.
TULIP Menu
This isan interesting idea that I read in a paper by Doug A. Bowman and Chadwick A. Wingrave [1].
The first level menu items are shown on the left hand. The second level items are shown on the right hand and they can be scrolled together.
I thought that this can be modified to be even more interactive. If the menu items are five or less then one can show the items on the left hand. To select the item, the user plucks the item from the finger.
In the end, there could be many more ways to design menus in VR. VR is a blank slate to start designing for even obvious interfaces. The below mindset would be the most appropriate approach for VR.
VR should be as close to real-world as possible.
References
[1] Bowman, Doug A., and Chadwick A. Wingrave. “Design and evaluation of menu systems for immersive virtual environments.” Proceedings IEEE Virtual Reality 2001. IEEE, 2001.
Other papers to read on menus in VR:
- Sheng, C., Jiang, L., Tang, B. and Tang, X., 2017, May. A novel menu interaction method using head-mounted display for smartphone-based virtual reality. In 2017 Progress In Electromagnetics Research Symposium-Spring (PIERS) (pp. 2384–2388). IEEE.
- Davis, M.M., Gabbard, J.L., Bowman, D.A. and Gracanin, D., 2016, March. Depth-based 3D gesture multi-level radial menu for virtual object manipulation. In 2016 IEEE Virtual Reality (VR) (pp. 169–170). IEEE.
- Kim, N., Kim, G.J., Park, C.M., Lee, I. and Lim, S.H., 2000, March. Multimodal Menu Presentation and Selection in Immersive Virtual Environments. In VR(p. 281).
I also made a short presentation for the same idea which you can use for presenting these ideas in case you design for VR and AR. Have fun!
