Alexander Shearer, Lei Guo, Junshu Liu, Megumi Satkowski, Robert LiKamWa
School of Arts, Media & Engineering | Arizona State University
Holostage is a prototype framework to investigate the hybrid integration of head-mounted and handheld mixed-reality devices with immersive screen-based environments. Our platform integrates three devices:
- A Microsoft HoloLens head-mounted mixed-reality device, using depth cameras and other sensors to position virtual objects in a real environment. The HoloLens employs an Intel Atom processor and a specialized holographic processing unit to render visualizations.
- A NVIDIA Shield Tablet K1 , running Android 7.0 on a Tegra K1 System-on-Chip. The Shield Tablet features powerful graphics performance on top of a mobile sensor package, including an inertial motion unit, a front-facing camera, and a rear-facing camera.
- The iStage (Intelligence Stage) at Arizona State University, consisting of motion capture, controllable lighting, and immersive projection covering a 10m x 10m floor and a 10m x 8m screen. Data management and projection runs on Max 7 software on Mac Pro computers.
We develop software to render particle visualizations for each device, using the Unity Game Engine for the HoloLens and C++ bindings of the Vulkan Graphics API for the NVIDIA Shield Tablet. Our software uses the PTC Vuforia SDK to geometrically register the devices, creating a uniform coordinate system for the virtual environments. In addition to providing the ability to visualize scientific and creative data, our multi-device platform serves as a testbed to explore system limitations.
Source code:HoloLens and Unity implementation: [Github Repo]
Android/Vulkan/Vuforia implementation: [Github Repo]