Research at the HIT Lab

Reality is what we experience in our daily lives. It’s the world we are familiar with. Our experiences in the real world, collected in our memory, are what sets most of our expectations for new experiences. Examples include simple things like walking down the street, having a coffee, arguing with a friend, or surfing the Internet. The real world, however, is just one of many realities we can experience.

In their 1994 seminal paper, Paul Milgram and Fumio Kishino formulated the notion of the “Reality-Virtuality Continuum” [1, 2]. This work clarified the relationship between reality, virtual reality, and augmented reality: immersion in the virtual world increases as connection to the real world decreases.

Our work at the HIT Lab NZ is focused on providing people with technological support for experiencing various realities to enhance work and daily life. We put people first by looking at the tasks they are trying to accomplish. We then use an appropriate mix of technology to support these tasks within a given environment. Most of our work involves some combination of real and virtual content.

Human-Robot Interaction (HRI) involves people interacting with physical entities that have the ability to sense, ponder and act on the real world. While there are many aspects to HRI, we can view this interaction as being just to the right of the Real Environment in the figure above. Examples include interacting with robots face-to-face (such as a hotel concierge) or operating remote search-and-rescue robots.

Augmented Reality (AR) is the merging of digital content with an otherwise real world. AR adds generated information to our perception of the real world. For example, smart glasses can overlay contextual graphics with real-world objects, such as call-outs indicating points of interest in the direction the user is looking. Another example is AR apps for phone-based historical tour guiding.

Augmented Virtuality (AV) is the merging of real-world content with an otherwise virtual world. AV adds information captured from the real world to our perception of the virtual world. For example, the “weatherman effect” blends real-world actors into virtual sets.

The main difference between AR and AV is the notion of the “primary world.” In AR, the real world is the primary world, while in AV the virtual world is the primary world.

Virtual Reality (VR) is an immersive digital experience in which everything the user sees, hears and touches is virtual. Like a person in a swimming pool who is completely immersed in water, the user in a virtual world is completely immersed in the VR experience. All of the senses are stimulated through the medium of water, or in this case, the virtual environment.

Overall, the HIT Lab NZ has deep expertise in all aspects of technology application along the Reality-Virtuality Continuum. We draw on this human-centered deep understanding to design effective systems for work and play.

Citations

[1] Milgram, F. Kishino, “A taxonomy of mixed reality visual displays”, IEICE (Institute of Electronics, Information and Communication Engineers) Transactions on Information and Systems, Special issue on Networked Reality, Dec. 1994.

[2] Milgram, H. Takemura, A. Utsumi, F. Kishino, Proc. SPIE 2351, Telemanipulator and Telepresence Technologies, 282 (December 21, 1995), doi:10.1117/12.197321