Exploring Tangible Interfaces and Relational Experiences in Virtual Reality
A Multiplayer Collaborative Project with Yujin Ariza
Tech : Unity, Oculus, Kinect for Windows, Makey Makey Sensor, Kinetic Sand, Custom Game Board
Funded in Part By the Frank-Ratchye Fund For Art @ the Frontier
Virtual reality can be a dissociative and disembodying experience. How can we instead create a shared VR experience to augment our physical senses, and accentuate the relationship of participants’ bodies with each other and with their environment?
Sand•VR is an infinite game with no winners or losers, but rather a playful, exploratory environment at the intersection of shared and solo play, and virtual and physical space.
In Sand•VR, we used a Kinect to visualize in realtime a streaming depth mesh of the sand as it is molded by the participants. Small glowing virtual spheres crawl and slide around on this shifting landscape, knitting the virtual and physical worlds together. When players touch, they complete a circuit created with a Makey Makey, which illuminates a part of the virtual landscape, highlighting this moment of connection between the two players.
We explored many projects within the tangible interface space for our inspirations — including work at the Tangible Media Lab in MIT, and interaction demos by Leap Motion — but our biggest source of inspiration came from Brazilian artist Lygia Clark. Clark designed situations for people to engage directly and tangibly in her artworks. Through interactive pieces such as Air and Stone and Abyss Mask, Clark creates sculptures and interactive scenarios which exist through and are enlivened by direct relationship with the participants’ body. She called these pieces ‘Relational Objects.’ In our virtual experience, we were inspired to create a ‘relational’ system made of tangible, virtual, and human elements, made manifest through interaction.
• We sought out to create a multiplayer VR experience using tangible materials as interface, without traditional controllers.
• We wanted to cultivate a sense of embodiment within a mixed reality experience, and promote connection between the players through touch.
• We wanted to create an experience that encouraged collaboration and communication between the two players, through primarily non-verbal means.
In the virtual experience, little virtual spheres (nicknamed “Guppies”) are physically affected in real time by the changing sand terrain. Participants can move the guppies around by cupping them in their hands, or by making rivers and pathways with the sand. The darkness of the environment was utilized to interrogate VR as a blindfold, and heighten the sense of touch as a key to immersion within the environment, rather than rely on our privileged visual sense.