Blurring the lines between virtual and reality

When you don this next-generation virtual reality headset, you will be able to manipulate objects that you see in the headset – objects that don’t exist, of course, but also objects that do, like a table or chair. You’ll be able to touch and move real-world objects just like their virtual counterparts. And if you look down, you’ll see your fingers; your virtual hands will reproduce even the slightest movement. This kind of unparalleled immersive experience is exactly what Hugo Hueber, a Master’s student in computer science at EPFL, is aiming to create through an enhanced VR video game he is working on as part of his Master’s project.

“My goal was to develop a video game that combines the latest technology with the 3D interactive research we’re carrying out at the lab. That will let video gamers interact physically with a virtual environment that can be transferred to any location – a living room, office or even classroom – instantly,” says Hueber.

That requires first measuring real-world objects like a chair and then modeling and calibrating them in the video game, which can be done in just a few clicks. Once this is done, players can use the object in the game at the same time as they physically touch it with their fingers. Achieving this was a formidable technical challenge for the Master’s student, but it seriously enhances the immersive experience for users.

Seeing your fingers in 3D

Another feature of Hueber’s technology is that it improves users’ perception of their fingers, further blurring the line between virtual and reality. His system doesn’t require a joystick – a piece of equipment that lessens the sense of reality. Rather, users can see each of their fingers and move them with a great deal of precision, for instance tapping them together or picking up an object with ease. And that’s not all – by placing sensors on their bodies, users can see themselves move inside a video game. “The system tracks the sensors’ movements and generates real-time images of those movements as they affect different parts of the body. That means users will have the impression of being fully immersed in a video game, since their physical movements will be synchronized directly in the game,” adds Hueber.

Video games are useful for research, too

Hueber’s project is just one application of the research carried out over the past few years by the Immersive Interaction Group, headed by Ronan Boulic. This Group aims to develop technology that transposes physical interactions into virtual environments. Hueber would like to go further in combining the playful, creative aspect of video-game development with research. “People don’t realize how much video games can contribute to other types of R&D,” he says.

The game is available here: https://astache.itch.io/mum-proof (you will need HTC Vive and Leap Motion.)