My MWC Hololens³ Wishlist

I’m sure you are following Alex “father of HoloLens” Kipman on twitter and saw his teaser video of “something to be revealed” next Sunday during Mobile World Congress in Barcelona. Well, it could be the teaser for a new terminator movie, brining back mimetic poly-alloy or some freaky colorful alien-styled worm race crawling along ropes to frighten us. But most probably we are talking HOLOLENS TWO here. Or HoloLens 3 if you will… leaving out the 2nd iteration that never saw the light of day. If you haven’t seen the teaser, here you go:

Now we are all speculating what we can expect during MWC. I’d thought I’ll create a short wish list with comments, just to pass the time:

Form Factor & Comfort

The current HoloLens is a bit fragile and bulky to wear. On the other hand it’s great to NOT have a cable going down to your belt. A new form factor with a pocket PC plus cable to slim glasses could be the big change. Will the new HoloLens opt for the Magic Leap approach? A lot of assisted reality HUD systems do it this way and we are used to “cables going down from our head to our pocket device” (unless you are wearing ugly bluetooth Apple headphones, sorry). It just gives more freedom for device exchange (e.g. for CPU upgrades), personalized glasses and less weight on your nose. With new standards like the fresh USB connector or virtual link, etc. this could become a common setup. I consider it rather probable and would vote “yes” on it. Especially with the teaser video, that might be showing the cable connecting the glasses with some brick. The light worms crawling along: a hint towards the same awesome lightfields ML was teasing us with? (Compare small image.)

Visual Quality

Talking about visuals. It’s definitely time for an upgrade here. More recent chips should do the trick for way more computing power to push the visual quality… and to fill a broader field of view with more pixels. Will it be doubled field of view and quadrupled pixel numbers? I will say: yes, for sure! The biggest critic on HL1 was always the field of view. Still today, I have to explain to first-time users that there is nothing broken or wrong with the glasses. “It’s just the way it is.” But can we expect mutiple focal planes? With the focal plane setup in the Magic Leap is was disappointed (too slow switch), the Avegant glasses were awesome but still bulky. Not sure if Microsoft was yet able to squeeze it all into a device at an interesting price point. Honestly, I doubt it. I’d guess they will keep it “as is” and upgrade on the fied of view, pixels, etc. and do some clever software fakes to make it comfortable.

Interaction

Looking at “making it comfortable”. They could integrate eye tracking to close up to other devices and to use it for improved interaction, foveated rendering and stereoscopic comfort. It’s definitely on my wish list, but I have my doubts if Microsoft will include it. Maybe it’s even a privacy issue and could prevent them from doing so by means of corporate privacy compliance rules?

But regarding further interaction I’m sure they will upgrade on their gesture set and possibly include an upgraded 6DOF-clicker. A clicker, that can be attached magnetically to the belt device, please! Gestures are of course the must-have sci-fi feature all try out out on a HoloLens today (and fail). For me, a 6-DOF clicker would be even better. I’m too lazy to raise my hand and a physical tool in my hand would be preferred… But maybe Microsoft wants to keep things clean and tidy and rather focuses on extended gestures and AI-supported speech control to make life easier. Some touch pad on the brick could be enough to leave out a clicker at all… Still, for the sake of sci-fi AR I still vote “yes” on full gesture support and hand tracking. Like the Project North Star and Leap Motion demos I finally want someone to bring this fully fledged gesture and hand interaction into an AR-glasses product I can buy! Be the first, Microsoft, please!

Smarter Software

Talking about software. We saw the upgraded Kinect-style scans with way higher resolutions form Microsoft. We all expect a huge upgrade on accuracy and speed of real-time meshing. Maybe we won’t go much further than the typical 3-4 meters distance for scanning, but a quicker update of the world would be my dream and is rather probable. This is top priority and will def. come. Once we have improved world knowledge in (rather) real-time there should be more included from my wish list. I want some ready-made (classic) marker tracking (think QR codes or fiducial markers Hiro-style!).

But even more important and I think this could be Microsoft’s biggest move: object recognition, learning and tracking. Their dedicated AI chip could deliver a speedy world recognition that identifies object types and matches these with the physical space. The SDK needs a big update here to give us tagged objects in return. Now the HoloToolkit gives us some “surface to sit on”, etc. info. This could be extended heavily to react faster to an environment. The toolkit could give us lots of tools to faster develop and e.g. include AI avatar companions that (or who) automatically search a good position to stand or rest, etc. In combination with gesture and hand tracking this could be cleverly used to identify tools and objects-to-interact with. How to combine virtual and categorized real objects.
If we had moving real elements in a physical space being tracked and given to me with auto-updates in my Unity code: this would be heaven! Maybe I’m getting carried away here. But we are talking Microsoft and HoloLens 3 here… we SHOULD expect something big!

Connectivity

Talking about tracking of spaces and objects. What about the AR cloud and shared experiences? Will there be a city-scale AR cloud by Microsoft just yet? Will they present some move into this direction? I doubt it. For me, Microsoft is too B2B focused and there is not yet any purpose in creating such a mega mesh. Big company “cities” exist, too. But still, use case scenarios are rather well defined and limited in spatial usage. I’d say it’s low prio. If we get a better object recognition that should be fine.

What I’d love to see in the updated version would be an auto-sync between local HoloLenses. Shared anchors, meshes, etc. Multiple HoloLens users in the same room should automatically see what the others see. Or have the option. Opt-in for private apps of course. But offer a platform (maybe the team using the HoloLens needs to set up a small dedicated server for this with a defined WiFi) for a out-of-the-box shared experience would be such a great deal! The HoloToolkit could give us some toys to accelerate this without re-coding it all ourselves.

Demos & Target Group

Talking about platforms. Will we see lots of news on the HoloLens store? What demos will Microsoft show us? Store-wise I don’t have high hopes. But in the demo realm I’d expect the typical remote assistance, telepresence pieces B2B. The above feature wish list includes my hopes: an easily shared experience could help a lot designers and trainers. A finer mesh could help planners and previs guys. HoloLens is well established in the B2B sector – behind closed doors. I don’t expect Microsoft to open up the doors and flood the consumer space with Angry Birds or finally a real Minecraft HoloLens 2019 style! If most of tech specs come true the price tag won’t go below 2k. Consumers are still too far off this way… So, but hey, please Alex! Prove me wrong, please! In any case… expect a review here as soon as possible!

For now… guess we’ll see on Suday. Do you agree? What would you expect? What are you waiting for? Let me know!