Hololens Hands-On #1 with RoboRaid
Hi everbody! It´s Holo-Time!
Finally, I have my own Hololens on my desk (well, okay, … my company´s) to try the HMD now full-time. Before I could try the augmented reality headset from Microsoft a couple of times on conferences – but never play around with it for a longer period or even upload my own apps and demos. So, from now on expect some more hands-on reports on the tech and upcoming applications or games!
Today I want to give my first short impressions of a longer use and give an example of a good demo game: the Robo Raid!
First impression
My first impression was that it´s quite comfortable to wear. The weight felt well balanced and I did not get much pressure on the nose. The material feels velvety soft but high class and the overall shipping and transport box looks already product-ready (no, I did not make an unboxing video, sorry).
This impression continued when booting the Hololens up for the first time and setting the device up. Besides a bit annoying air-tapping a WiFi password (don´t use too many special characters…) it went smoothly and was online straight afterwards. The room was scannend and the initial menu using the flower showed up promptly without further after-sales-installations or updates. It was ready within a few minutes and I could start placing holograms inside my living room. No hassle, it just worked. Well done for a dev kit!
Second impression: the overall tracking is surprisingly stable. You do notice a slight drift and jitter, but depending on the scale of your room and the distance to the virtual object it felt rather solid than shaky. With a few exceptions I have a good feeling most of the time that does not smash my immersion or the augmented feeling of the objects really “being there”.
Third impression: the brightness is great! If you crank up the brightness settings you won´t see real objects behind (non-black) virtual objects anymore and I could even top my running TV screen. Everything has the touch of a translucency to it (obviously this fits the name of the product – promoting a flaw to a feature), but does not bother me.
Draw-Backs to live with
Obviously there are draw-backs in the current hardware that are well-known and I`m sure confirming these, too.
First off, the field of view is very limited. Period. You will notice it right away and depending on the content it might be no issue or get on your nerves totally. All virtual objects are just cut off top and bottom dramatically. I don´t have issues in the peripheral left/right, though. Sometimes it´s enough to resize a hologram a bit (make it smaller) to fit into the field of view or to pin a 2D Windows 10 window (like the shop or settings windows) farther away from you to have it not cut off. When starting different applications you notice quickly who thought about the limited viewport during design and who did not (or who just ignored it waiting for better hardware in the future).
For example, the HoloTours demo switches between AR (see an easel standing in your room) to VR (complete Rome appears around the easel from the street artist). Here the illusion breaks and you scream: give me my Vive to continue!
But other demos just work fine. Sometimes you can´t scale objects up to be cut off or objects disappear when you get too close, etc. I´ll go into the details with the Robo Raid demo below later!
So, now I´ve downloaded and tried all demos and apps and clicked my way through them. How does it feel?
Air-tapping through the day
After hours wearing the glasses I can already say: we will all be living with AR sooner than we think! You get used to spatial computing so quickly and love it right away, though 2016 it´s not there yet. But order! Order in the court! Step by step:
The weight and bulkyness sure is an issue, but during initial tests it did not get on my nerves yet. Obviously this needs to go down dramatically to have an all-day-device. Now it´s limited to specific sessions for a special task (repear an engine, do a design review, make a skype hologram call) and also battery time does not allow more than 3-4 hours of full usage.
As said, overall room scanning, object placing, re-recognition of a previously trained space, spatial audio – all works as expected. Sometimes the WiFi failed or took longer to connect (but that might just be my crappy provider – damn, I´m happy that I`m switching).
Flower Power for the holographic user?
Okay, okay, clickbaity-headline. Of course, I´m talking about the user interface. Calling the home screen or Hololens menu is triggered by the flower gesture “bloom” (opening your closed hand with five fingers spreading to all directions (facing upwards)). This works very stable – as long as you hold your hand within the camera´s volumes. Air-tapping (bringing the index finger to your thumb to select objects or menu items) works stable too, although I´ve seen many first time users who get it wrong (holding up more than the index finger, holding the hand outside the volume). Here I`d be happy about more space to move my hands. I get easily tired of air-clicking too often (for this you get your bluetooth clicker along).
So, the simple gestures work well, tap-dragging objects around, too. What´s missing? I`d say the overall user interface is not ready yet for AR. We are still in the learning phase and here you notice it the most for the Hololens (or other devices, e.g. when you use your LeapMotion for the Rift). To gaze at a button with your head and then air-tapping can fail easily on longer distances and is not really that intuitive. It drags the old mouse metaphor over into AR. That´s certainly nothing we want in the end! (But hey, we need to start somehow…)
I´d love to have more ready-made gestures to use or even accurate Leap-style finger tracking via the SDK to have more power for natural interaction trials. Many people I´ve seen trying it on first time directly wanted to use their hands naturally to grab things, but failed. Abstract gestures like the flower need to be learned and can be a problem, too. So, time to learn and let´s try out different ways to get closer to natural spatial computing.
Two last things I want to mention here are multi tasking and registration before moving to the game.
It seems like most demos don´t really support application / context switching and will just close when you go back to the home screen space with the flower. When you go back in they will reboot. That said, there is no real context switching of apps (Alt+Tab) on the go. But maybe it´s just my RAM being crammed up with apps or demonstrator limitations. I will dive deeper into this another time.
Second is the missing registration. The tracking works quite stable as said and I´m very happy for a DK1! But you cannot really align your holograms with the real world by a centimer or millimeter. You can just place it somewhere… But what if I want to overlay a specific piece with augmentations? Like the new spoiler for my car, the digital window (replacing my dirty city landscape), etc.? No demo does it today and I`ve seen nothing within the SDK to achieve this out of the box. Well, time to code it myself, I guess. :-) I will cover this extra another time, as it is one of the biggest issues! We need full object recognition and 3D registration by the millimeter to reach the full potential of AR. Right now, it feels a bit wasted on the tech, if you cannot overlay things 1:1.
So, but as I said, it will only be a short first recap and I will go into some tech details during the next posts. So let´s take a look at one demo as an example now! They´ve done their job quite well – using the DK1 wisely.
Evil Robo Aliens to destroy your living room
The RoboRaid was done by Microsoft as a demonstrator and was shown before at one of their presentations. The demo nicely shows what can be done today. You briefly need to scan your room, select a wall and the game is ready to start. Holes will get punched into your walls and alien robots will break through. You need to shoot them air-tapping before they hit you too often (dodge them). Take a look:
In reality it looks better as your world is not recorded through the camera (no pixelation), you have little motion blur and the stereo effect helps a lot. The thing I like about the demo is, that they integrate all features. First off, your real room plays a crucial role in the game! This is often forgotten and many demos could happen anywhere in your physical space. They just show a holographic experience that floats in your room, but do not take into account your actual environment! Chapeau, Microsoft! I want more in this style!
Only if we really integrate the virtual objects into our space and let them interact with it, augmented reality makes sense!
Next, they use the air-taps to shoot, spatial audio to orientate and the dodge of enemy bullets so that you really have to move in physical space. The cracks in the walls are a lot of fun and give a good augmented depth in stereo when you watch your appartment go bye-bye!
The last piece, I mentioned above is: they use the field of view wisely. The enemies are small enough to fit into the screen at a typical room distance and the cracks in the wall also don´t fill up the whole wall from top to bottom. Only one crack per wall appears so that you don´t notice the limted FOV too often. You can really dive into the game. Well done for a demonstrator, I`d say!
A good start for the spatial future
So overall, I`m quite happy with a good selection of the demos I´ve seen. But even more impressive is how quickly I get used to having my holograms around me in my daily life at home! I go to the living room to “shop” a new app – because I pinned the window there. I have my pictures, holographic skype and calendar pinned next to my classic computer… It feels natural and I can now, after a longer trial period, really sign the prophecy that we will all replace our smartphones and basically all screens by glasses in a few years. We only need to work on standards and sugar-cube-sized tech.
So, spatial computing is the future!
I will write more about my Hololens experiences from now on (after all: with glasses we can really step into an AR world as I always wanted). But to not become a fan-boy of one system, I`d be happy to try out different systems as soon as possible, of course. Hey, Meta! Hey, MagicLeap! I´m happy to review your devices! :-)
Cheers,
TOBY.
Update from Oct, 12th
Finally! Microsoft moved a step forward and you can now pre-order the Hololens outside of the US without hassles. Germany is among the 2nd wave countries! So we can expect many more demos to pop up soon. I will definitely cover the best here.
Personally, I`m relocating to Spain now to lead my own mixed reality team. Therefore I might be offline for the next 3-4 weeks or so. But I will be right back with more on AR!! Keep it up!