Wake: Co-located, Co-Present Mixed Reality Experience

Overview

Wake is a mixed reality performance and research project — a guided experience for one participant in-headset, one dancer, and one facilitator. The participant is immersed in a virtual environment and uses spoken instructions and movement to navigate the space, interact with virtual and tangible objects, and encounter a co-located, virtually present dancer who invites them into simple collaborative movement.

The experience is built in Unity 3D for the HTC Vive, Vive trackers, and an Intel RealSense Depth Camera placed at the position of the participant’s eyes in the headset. The project uses a custom algorithm to stream and render in real-time a high resolution volumetric video (synced depth + RGB feeds) of a physically and virtually co-present dancer.

Created in collaboration with the multi-disciplinary performance duo slowdanger (Anna Thompson and Taylor Knight), Wake investigates embodied interaction, kinesthetic awareness in mediated experiences, trust and emotion within co-present head-mounted XR, and the shifting boundaries between physical and digital. The project formed the basis of Anna Henson’s Masters of Science thesis in Computational Design at Carnegie Mellon University.

Research & Impact

A user study (N = 25) was conducted with Wake, examining embodied interaction, affect, and social presence. The study combined semi-structured phenomenological interviews, standardized self-reported emotion metrics, and proxemics data measured in real-world units. Findings are published in the Masters of Science thesis ”We’re in this together: Embodied interaction, affect, and design methods in asymmetric, co-located, co-present mixed reality” (Carnegie Mellon University, School of Architecture, Computational Design).

Wake was featured by Intel on the RealSense blog, performed live at Thrival Festival and the Carnegie Museum of Art, and was the subject of a performance lecture delivered at the Association for Computers and the Humanities Conference 2019 at the University of Pittsburgh.