From HIT Lab NZ
Comics: Alive on the Inside!! (3D Augmented Reality MagicBook)
There is a new 3D AR MagicBook on display at UC central library from 10 - 16 Aug.
This exhibit demonstrates how a normal comic book can be transformed into a MagicBook: a new type of reading experience made possible through the use of Augmented Reality (AR) technology. Augmented Reality enhances our view of the world by adding 3D computer graphics into the real environment around us, so that the real and virtual images are blended together.
In the AR MagicBook, 3D virtual scenes from the story appear to pop out of the real pages of the comic book, bringing the characters to life. This is achieved through a combination of creative illustration, 3D modelling, software programming and computer hardware.
Illustrator Katy Bang has produced a comic book, “Jack the Time Traveller”. The story follows Jack’s work day, as he travels through time helping to ensure past events run smoothly. The events in the story are enhanced with 3D virtual content using the HIT Lab NZ’s Augmented Reality and 3D graphics technology. Katy designed and illustrated each page with a style and colours suitable for the software that adds the graphics onto the pages.
The 3D scenes were modelled by Shunsuke Fukuden, 3D Designer and Animator. Shunsuke took Katy’s storyboard and character designs and worked with her to make 3D versions of her illustrations. The 3D models are animated to show the story scenes. To keep the software running smoothly Shunsuke needed to keep the 3D designs simple, while still making them look impressive!
Special software is required to combine Katy’s illustrations and Shunsuke’s 3D models into the Magic Book experience. HIT Lab NZ developers Julian Looser and Adrian Clark created the software for this exhibit. Using the application they wrote, a person can look at the pages of the book through a web camera and see the 3D animated scenes leap from the pages. Even if the reader moves around and looks at the book from different positions, the 3D objects still appear to be attached to the pages. This is possible because computer vision software is used to interpret what the webcam is seeing, and insert the 3D graphics into the right place in the video frame. This process, known as “Natural Feature Tracking”, uses visually distinct parts in the pictures themselves to track the page. The software identifies the page being looked at, and so displays the correct 3D model on it.
To further enhance the Magic Book, the HIT Lab NZ has developed a special handheld unit that conveniently incorporates the camera with small LCD displays. The reader looks through the handheld display to see the augmented reality Magic Book in action.
