User Inputs Part 3 – VR Controllers

Time to start touching the WebXR API a little more closely

Step 1

In this tutorial, we'll be extending our code from Part 2 to listen for user input via VR Controllers

Our VR Controllers provide more than just gamepad information, they also provide the controller's position and rotation. You can find the controllers by first accessing the XR Session via renderer.xr.getSession() and then looking at its input sources

I recommend taking some time to read through the WebXR API if you want a better understanding of how to access important XR information, but it won't be necessary for this tutorial given how I spoon feed everything to you

three.js also provides a way to keep track of controllers based on an index, but this leads to a weird situation. Say we determine that index 0 gives us the right hand controller and index 1 gives us the left controller. If the battery of our right controller dies, index 0 will then give us the left controller. So we'd have to constantly be checking which index is mapped to what controller which I don't like. I'd rather write my own solution based off of the code they have, but just keeping track of the left controller and right controller as that's all I care about

Let's start by only worrying about the gamepad information of the controllers. Controllers can be accessed via the inputSources attribute of our XR Session, and we can also be notified if new ones get added or old ones get deleted from the session's inputsourcechange event

So in InputHandler we should:

1. Accept renderer as a parameter in the constructor
2. Declare this._leftXRInputSource and this._rightXRInputSource in the constructor
3. Store controller information on sessionstart events
4. Clear controller information on sessionend events
5. Keep track of controller changes based off of the XR Sessions inputsourcechange event
6. Have a function getXRGamepad(hand) that returns the correct input source's gamepad based off of the hand parameter

Doing all of those things gives us our updated InputHandler.js below

Make sure to pass the renderer in when creating InputHandler in Main.js

Great, now we can add some code to test it in our Spaceship.js. Let's make the spaceship move forward when the main trigger of our right hand is pressed down. The xr-standard mapping for gamepads have the primary trigger mapped to the button at index 0. Let's once again alter the conditional for moving forward in the update(timeDelta) function of Spaceship.js to also use the gamepad like so

Running this on our VR Headset we see that the ship moves based on our right trigger without a problem. Awesome!

Step 2

Now we need to handle tracking the position and orientation details of our controllers. If you skimmed over the WebXR API Documentation you may have noticed that there is both a target ray space and a grip space for the controllers. The target ray space is oriented so that forward is in the direction of where the controller's selection line, "target ray", extends to. The grip space is oriented so that forward is in the direction of the thumb-ish. If you imagine holding a sword upright in place of the controller, forward is up

Remember when I say forward I mean the local negative Z direction

The WebXRManager section of three.js has code to handle tracking controller position and orientation which we'll base our solution off of

We'll need:

1. Object3Ds to represent the grip space and target ray space in both the left and right hands
2. To add the relevant hand's target ray space and grip space Object3Ds to the scene when that hand's controller becomes available
3. To remove the relevant hand's target ray space and grip space Object3Ds from the scene when that hand's controller becomes unavailable
4. An update(frame) function that updates the Object3Ds of both right and left controllers if possible
5. A function getXRController(hand, type) that returns the appropriate Object3D representation of the controller

Based off of these needs and how controller tracking data is updated in three.js, we get these updates to InputHandler.js

Notice how we now accept controllerParent in the constructor!

If any of this feels a little bit confusing, again I recommend reading through the WebXR API to get a butter understanding of what's happening. If you're still confused after that, feel free to drop a comment

Now we need to call the update function in InputHandler whenever we're using an XR Device. Rather than have a conditional in our animation loop (including InputHandler), let's change our animation loop slightly depending on our device type. While we're at it we can also make it so our SessionHandler is only updated if we're on a Mobile Device

By moving the conditional to outside the animation loop, we remove the needless repetitive checking on every frame and just check the device type once on startup

Here's how we can do that at the bottom of our constructor in Main.js

Great, now InputHandler keeps track of the position and orientation of VR Controllers too and is fully functional. All that's left to do is to set our Spaceship's orientation to match the right controller's orientation in the update(timeDelta) function like this

And it works! Phew this was a lot of code and a quite a bit of work. But we've set up a good foundation for handling any sort of User Input in the future (except for mouse clicks, but that should be easy for you to figure out after this 😉)

Finished code in action

Git Repository

Glossary

• Set – A data structure implemented in Javascript as a Hash Set thereby giving it a O(1) time complexity for retrieval

Feeling generous/want to support me in writing tutorials? Then buy me a coffee, which I'll probably use for boba or almonds (raw and unsalted you heathens) because I don't like coffee