[ad_1]
Facebook Reality Labs wants to help people see your eyes when you’re in virtual reality – even if the results are somewhere between a slightly confusing and a nightmare. Earlier this week, the FRL released a paper “reverse pass VR”, recipe to physically reduce VR headphones. The researchers came up with a method for turning the face on the front of the headphones, although they stress that it is still strictly experimental.
“Passthrough VR” refers to a feature that displays a live video feed from the headset’s cameras and allows users to see the real world while the device is on. For example, Facebook’s Oculus Quest platform shows users throughput as they step outside the confines of their VR mode. It is useful for quickly shutting down VR, and can also take the form of augmented reality by adding virtual objects to the camera input. But as the FRL notes, people around the headset user cannot make eye contact even if the user sees them perfectly. It’s tricky if bystanders are used to seeing the face of a friend or co-worker.
FRL researcher Nathan Matsuda decided to change this. Blog post explains that Matsuda started in 2019 when he attached a 3D display to the Oculus Rift S headset. The screen showed a virtual representation of his top and custom eye-tracking cameras that took Matsuda’s gaze, so his Avatar’s eyes can point in the same direction. The result was basically Matsuda dressed in a telepresence tablet with a copy of his own face – which is undoubtedly just as awkward, but with a more interesting postmodern twist.
According to the blog, FRL chief researcher Michael Abrash – quite understandably – didn’t find the idea very practical. “My first reaction was that it was a silly idea, at best a novelty,” he notes. “But I’m not telling researchers what to do because you don’t get to innovate without the freedom to try new things.”
Matsuda ran with the concept, and over the next two years, he led the team to develop a lighter design. The team’s prototype headphones – which it unveiled ahead of next week’s SIGGRAPH conference – add a stack of lenses and cameras to the standard VR headset display. Stereo cameras take a picture of the face and eyes inside the headphones, and their movement is mapped to a digital model of the face. The image is then projected onto an outward-facing light field display. This screen creates the illusion of looking through thick goggle lenses and seeing pairs of eyes, when in reality you can still see a real-time animated copy. If the user moves back to full virtual reality, the screen may clear to indicate that they are no longer in contact with the outside world.
The result is octagonal goggles that look at home in the movie Terry Gilliam. FRL used a simple virtual face presentation, but it also introduced the system with its more realistic Codec Avatars, as shown below.
FRL acknowledges that not all of the individual components of the system are revolutionary. HTC already has face tracking add-on For Vive Pro headphones; it describes the movement to the avatar inside the VR, not to the outward screen, but the principle is similar. This week’s magazine focuses on the potential of light field displays and the potential of the system to enhance personal social interaction.
HoloLens-style projection glasses theoretically leave your face much brighter than transparent screens — even though many of these glasses have tinted lenses, and like The road to VR notes, reflected light from bright lenses can also block your gaze. But like companies like Apple reportedly try A new study on Facebook shows, according to models that have gone through, that a fixed screen isn’t necessarily a barrier to eye contact … sort of.