VR presence depends on accurate hands and spatial audio

Transcript

Transcribed by machine (Whisper) from the episode’s audio, so a word may be off here and there.

1. 1:52:38

   important to deliver so i guess taking a step back a lot of how we think about the goal here is delivering like a realistic sense of presence right no technology today gives you the feeling as if you're like physically there with another person right you're you're like interacting with them through a phone you have this like little window it's kind of taking you away from everything um that's like the magic of augmented and virtual reality is like you actually feel this like presence like you're there with another person right so the question is okay how do you do that and it's like there's like a million things that that contribute to that i mean obviously first just being able to look around and have the the room stay um getting good spatial audio right if someone speaks then it should do the audio it needs to be 3d and come from the place where

2. 1:53:24

   they're speaking um it's actually it's very interesting which things end up being important for the this kind of creating the sense of presence and which don't so having hands obviously if you're just looking around but you can't actually like move things that that that breaks the illusion but having hands um like hand tracking that you can do stuff is important one thing that we found that's kind of funny is it's actually not that important that you see your arms you just need to see your hands obviously seeing your arms is a bonus unless we incorrectly interpolate where your elbows are or something so if we have if we're looking at your hand or we if we have a controller we can know okay your hand is here but that doesn't necessarily tell us where your elbow is your elbow could be like this it could be like this so you can kind of guess from the but if we

3. 1:54:12

   get that wrong and you like see in vr it's like you see the hand there and your elbow is like looks like it's here when it's actually out there you're like ah what's going on like that's messed stuff. Um, so it's a lot of these things, like you just don't want to get these details wrong. So haptics, the most important first thing for haptics is on the hand, right? I mean, we have so many more, um, the neurons basically, or, uh, not neurons, but just like the, like sensation. It's, it's like such higher resolution, um, on your, on your fingertips than anywhere else in the body. So, you know, when you grab something, you know, making it so that you feel some pushback right when you there's a lot of gaming systems at this point where if you like pull a trigger you get like a little bit of a rumble or something um we built this one thing

4. 1:55:00

   where it's like a ping pong paddle with a sensor in it and it you you feel the ball hit like the virtual ball hitting the ping pong paddle and it feels like like when you're actually playing ping pong it doesn't it's not like a generic thing where just like you feel it hit the paddle you feel where it hits the paddle and we basically built a system where now with this like physical paddle you can kind of it that the haptics make it so you can feel where the ball hits the paddle so it's like all these things like are just going towards delivering a more realistic experience so um full body haptics so there are some things that i think it could do like if you get if you're playing a boxing game you get punched in the stomach um you can probably simulate something like that a little um it's not gonna be able to deliver that much

5. 1:56:00

   Force. So, I mean, I guess that's maybe a good thing because no one wants to get punched in the stomach that hard. But like, it's not going to be able to deliver enough force for you to, for example, let's say you're not just boxing, you're kickboxing. Like, I don't know, you need something on the other side to be able to complete it, right? Because it's like when you kick, when you're just practicing, it's like you spin, right? Because you don't want to just like stop.

6. 1:56:25

   and it's um that's like like the shadowing a kick like there's not going to be anything that you can do as like a you know single person playing vr with a haptic suit that like makes it so that you're going to be able to kick someone who's not there physically and actually be able to do that right um so like grappling it's like i think that that like jujitsu is going to be the last thing that we're able to do in in vr because you like need the momentum of the other person and to be able to move them the boxing thing is actually good boxing works yeah yeah boxing works even and you don't really need the haptics um i think it would be better with it um that's probably one of the better cases i think it's that and getting shot or like sword fighting type stuff um so you can like just feel feel it on your body but i don't know i i think what's basically

7. 1:57:15

   gonna end up happening is you're gonna have like a home setup for these things and then you're going to have there are these like location-based services where like people um it's almost like a theme park where you can go into and it can and you can have like a a really immersive vr experience where it's not just that you get like a a vest that can simulate some haptics it's that you're also like in a real physical environment so they can like have smoke come out or something and you can smell that and feel that or like spray some water and it feels humid and um i think that it It still is going to be a while before you can just, like, virtually create all those sensations.

8. 1:57:53

   So I think a lot of those really rich experiences are going to be in these very constructed environments. Is the bridge when they figure out some sort of a neural interface?