Augmented Virtuality Enhanced Visualization in an Immersive Cinematic Environment



The rapid development of affordable head-mounted displays (HMDs) has led to inclusion of Virtual Reality (VR) in a home entertainment system which in turn has created a niche for 360 degree panoramic movies. Previous research blended a user and real-world objects into a virtual movie scene seamlessly, making the user feel part of the virtual environment as if they were in the movie space. This thesis further developed the concept by overlaying context-aware virutal costumes on the user’s real body. A prototype was developed by combining Microsoft Kinect, SoftKinetic depth camera, and an HMD. The prototype captured user’s real body and embedded it in a virtual 360 movie scene; hence, augmenting the virtual scene with reality resulting in augmented virtuality (AV). Furthermore, virtual costumes related to the movie scene were overlaid on user’s real body to enhance user experience. The virtual content, captured real body, and 360 degree movie were combined in Unity and visualized in the Oculus DK2 HMD. With the created prototype, a user experiment was conducted to investigate how context-aware virtual costumes on user’s real body affected the user’s sense of presence and preference in a 360◦ movie scene. Results showed augmenting user’s real body with context-aware virtual costumes was most preferred by users, compared to only watching a movie and just augmenting user’s real body. The results offer a future direction to generate more enhanced 360 movie watching experience in an HMD.


Senior Supervisor: Rob Lindeman

Co-Supervisor: Gun Lee

Associate Supervisor: Mark Billinghurst