Chapeau, Chaperone!

So, CES news are flooding in!

Oculus to win the race being first…

Oculus has revealed their a bit too pricey Rift yesterday and switched pre-order website on. It’s available for USD 600 or € 699 plus shipping, starting end of March. Expected shipping date is June if you pre-order today. Unfortunately they won’t ship their tracked touch controllers along, which kills a lot of the immersive fun (try to play the climb with a xbox controller – most probably less fun)… But, anyway: congratz, Oculus team! Great to see it finally hit the market – and lucky kickstarter supporters getting it for free! Nice!

…but HTC Vive beating them on features?

HTC changed their plans last minute not releasing their HMD in December 2015 but rather wait due to a major technical break-through. Now the new Vive Pre was presented at CES and we can find the first videos and hands-on reports in the web to check out the updated system:

The new version has a slimmer HMD design and the wireless controllers look less bulky and seem to balance better in your hands. Overall a good consumer-version-look – also on the lighthouse laser trackers you need to install in your room, that now work wireless, too. But two more things get the attention: the mura-correction for the displays and the front-facing camera to improve the chaperone system:

The mura correction (for a common problem for displays) seems to enhance the visual experience pretty heavily. Users confirm that the dirty window feeling is gone and the immersion gets way deeper through the updated version. So, is this the break-through (as discussed in the web here)?

… or is it the front-facing camera of the Vive Pre? (As claimed e.g. by theverge or Tom’s Hardware.) A camera was discussed before and we don’t know for sure how it works. Is it only RGB or a depth-sensing camera (time of flight or structured light) to help reconstruct the physical environment? Since HTC’s HMD can be used whilst standing and moving around in a real room, it is pretty useful to have a warning in VR when you reach the end of your volume – before you bump into a wall. The earlier version of this system called chaperone only showed a grid in your virtual world when getting too close to the end of the usable space. The new version overlays outlines of your real world in a blue-ish, tron-ish, wireframe-y look as people report. A short snippet of one demo’s chaperone can be seen in the video from boredatwork (at 4:40). The Verge describes it as follows:

The original Vive’s tracking space was bounded by a blue grid, letting wearers know they’d gone too far. While effective, it could give the misleading impression that you’d hit a flat wall. Now, instead of a wall, that grid acts as a membrane. Step or reach beyond its limits, and a black-and-teal rendering of the real world swims into view outside. It’s detailed enough to make out objects and patterns, but not photorealistic enough to feel like a grainy imitation of reality, the way the Gear VR does.

This mode can also be activated by double-clicking a controller button and lets the user see real world objects around, e.g. to find a chair to sit down onto without tripping over it (hence the featured photo). No technical specs were given yet, but SDK users will be able to access the camera data for their own purposes…

Is it RGB only with some clever 3D mapping of edge-detected features? Or does the camera work in full combination with the lighthouse trackers to create fully-fletched 3D reconstruction of real world objects or your own complete room? Valve has been interested in photogrammetry before, so ideally we can hope for this.

We have seen demos last summer showing those approaches of including real world pieces into VR to interact (or just to drink your tea). I’ve blogged about Dassault’s system in June. This is a pretty damn cool feature to have it included in the first consumer release of a VR headset! On more… we can only speculate until we see further tech details – could the camera data at least be used to render one’s own body (showing arms and feet for better immersive feeling)? What can we as developers use it for?

I was hoping for a setup with integrated stereo cameras for full augmented experiences on a click of a button. But this mixed reality approach bringing the real world (somehow) into VR is the next best thing we could hope for. So whatever the marketeers of HTC wanted to advertise with their break-through… both features are pretty promising! HTC made a smart move (willingly or by accident) to ship second and to bring in new features that Oculus lacks. With the higher price of the Rift some might wait for further news from HTC to get a full immersive experience – even with a promising mixed reality mode on-top! Woohoo!