Think beyond Screens – Creating Worlds in AR/VR @ Digility

Day 1, Part 2

Hi guys, as said I wanted to continue my report on the Digility event in Cologne last week. It´s not my job to summarize everything (the hosts would sue me giving away all info for free), but I`d love to continue giving an overview of what happened on day 1 during the development track. It got a bit more technical at first – talking about e.g. performance by nvidia, hardware (GoPro), indoor navigation with Project Tango (“Google Tango will be a gamechanger for indoor navigation, google wants to scan the whole world … and they want (our) Data”) or even how to orchestrate 360 sound for VR (Berlin Phil Media). But then it jumped more towards content, use cases and storytelling and the meta level of all things.

Boot Camp for Reality

As a bridge to it, I want to briefly mention the talk from Clemens Conrad from Vectorform. He started by pointing out to a classic quote from Benjamin Franklin, but if you haven´t heard it, it´s a good jump into VR and the question why VR can help us out. Why should I learn within VR?

“Tell me and I forget, teach me and I may remember, involve me and I learn.”

They build a virtual training software and ecosystem to let people learn better and faster. They want to increase efficiency and satisfaction with less errors or casualties (e.g. for jobs with high consequences – being very dangerous). Here it makes total sense to spend some time in a (safe) place. E.g. another company SoftVR showed a multi-user training scenario in VR where you needed to work in a radioactive power plant. Training the right steps can be very critical. It´s better to train it deeply in a boot camp first for sure.

Doing user tracking and analytics was shown as a great tool to optimize the training results. Did the user bend over too much? Can he or she reach the handle above the head? VR matches here perfectly since you must move naturally through space. Training can be customized personally – if a user performs badly the system could give more hints – and less hints for the pro`s.

Close The Gap

John Peeters from Holition showed more specific AR examples out on the street and talked about their experiences so far. He sees digital tech as a good tool to connect people, but at the same time it happens that people don´t want to use tech in public. People are afraid of looking stupid.

Small working examples are digital try-ons with an augmented mirror (jewelry, watches, …), but there are also big fails in this field. If the digital prop jumps around or looks crappy you should have rather not used AR to show it. The digital add-on needs to be relevant for the store and come with great quality. Otherwise it`s better to stick with this:

Don´t always force yourself to use the latest technology. Try old technology with a digital twist! Never start thinking about the technology. Choose the right tech for the right job!

E.g. they did not augment a full jacket to the person in front of the mirror but rather used one physical jacket that could only be adjusted in color (making it easier to try on different versions with a single gesture).

He proclaims that AR will be big in the beauty sector. Digital make-up, make-up transfer previews from friends or celebrity photos onto your own face – or (awesome idea, I`d say) digitally augmented make-up tutorials (Hey, youtube-lady Dagi Bee, go AR make-up now)! Go beyond the glass frame of your analogue mirror!

“Remove the glass ceiling”

So, time to go meta. Enough technology! What can and should we really do with all the tech? How should we do it? Ola Björling from the MediaMonks gives us an idea how to approach VR by thinking about human perception – and how to hack it. Typically, a story in, for example, a book will only come to life in your brain. You visualize it with your imagination. It´s not that direct. Even TV shows or movies need to bridge the gap between the rectangle (frame of the TV set) and us on the couch:

VR and AR now hold the potential of taking out the translation and interpretation of the presented information. It becomes a direct sensory experience! VR will not be at its peak when we figure out how to film 360 – we must look beyond. We must start all over again to reach its full potential!

Ola states that we must raise the quality bar dramatically for mixed reality experiences to keep people convinced: “The uncanny valley is extremely deep in VR”. To reach it, everything matters. If haptics are involved they must work flawlessly (touch a reel steering wheel of a car in VR must work by the millimetre). If music and sound is there (they should be) it can become the deal breaker or the one feature that sells it. Music can be the “hotline to our emotions” and can create moods easily. Optimized spatial sound will help out in a subtle way and must play a key role! With all, we must “aim higher to reach deeper”.

The 4th wall is down

Astrid Kahmke from the Bavarian Film Center shares this approach and talked about the design of the content and the stories: Presence is key for VR magic!

If quality is high enough (visually, sensory inputs – but also content) we can get immersed and convinced. The 4th wall is down and there is no distance between the narration and the consumer (=actor) anymore. You are active in the story.
The classical Campbell hero´s journey with it´s arc of suspense in story telling becomes the user´s journey. New spatial narratives will push storytelling forward once again. Storytelling will and must change. People can interact even further (than in “normal” games for instance) and the designers need to design their worlds and stories in a way that we can connect with the protagonist (= ourselves) to believe the story and to get emotionally attached. Also, people tend to enjoy their freedom in VR – let them run free. Don´t manipulate them. Leave space to relax and enjoy. Storytelling needs to relearn its toolset!

Though Astrid loves VR and AR storytelling, she reminds us, to not blindly fall for it. Does my story make sense in VR? Does it really fit? It´s the moment of truth. We should use the right media for the right story. Rather use a screen or pages for your story, if the result can be better. Your story counts, not the technology!

I`m afraid of Augmenticans

Speaking about what counts: Alysha Naples entered the stage again and reminded us once more what matters most. (Again, some moments just don´t work in written summary or even in a virtual reality. But let´s try.) Will we go beyond screens? Probably yes. What happens along the way? Will we get it right? Will we adopt?

Alysha repeats once more that humans don´t change. Technology around us does. But humans still live a physical life, have a family, relations, they are eager to learn, to share, to play and to explore the world.

Mankind is (unfortunately hopefully) to stay. Technology changes over the years and should work to our best and meet our needs. Looking at today´s approaches we always see a screen in front of us. Let it be a rectangle or a stereoscopic panel. Let it be video-see-through or an optical see-through overlay – we still observe and experience the world through a mediator. Today the mediator can be easily identified – a TV, phone or computer screen – and can be easily turned off (see illustrations picked by me).

But once we have the filtered reality through glasses or contact lenses it might become hard to escape. We have big potential to get it wrong! – I would also add that it becomes very dangerous and we are likely to turn into zombies that are even more dependent on the big corporate input from the glasses/eyephone manufacturer. The closed-system mixed realities will steer us into their ecosystem to consume – if no open metaverse evolves. (Well, let´s keep that for another day.)

Alysha warns us to not only think about the wearer or user of AR glasses. We must not think hard about the cool and crazy content for him or her – but we need to think about the non-wearers, the have-not´s. What´s with the other people in the room? How does AR and spatial computing affect society? Will we turn again into glassholes? Will we be running into a digital divide with one half being afraid of the augmented people? Will they have a good reason to be?

We don´t want a hyper-reality as Keiichi described it multiple times in his videos. We are smarter than that! We will be clever enough to make it right! We are awesome! Oh, really? A good example is the image below to rethink our arrogance:

The inventors of the automobile surely didn´t plan for this to happen. They wanted to create save, cheap and fast transportation and democratic access to it. Not so sure anymore, that we will make it, right? So let´s think about AR and VR for a bit:

What we create today will affect the future long-term! We must slow down and think about our decisions!

So, how to get it right? We need to ask the right questions. Simple example: don´t ask “How can I type in VR in mid-air?”, but rather ask “Does typing in the air make sense?” Alysha wanted to throw the Hololens out of the window when she was forced to enter a WiFi password in mid air (I can totally relate). We need to focus on the essence of things. When I want to text in AR/VR – what do I really want? I want to communicate asynchronously. I don´t necessarily want to enter text in mid-air. We must go abstract from known approaches. We must go meta. Let´s make room for the new and use the tremendous opportunity that lies before us today. Let´s not do a twitter client for our eyeballs, please!

Important aspects need to get addressed by us today. Will people fall into the virtual world? Will they fall into the screen (well, without the screen boarders)? The more you belong into the screen space communities – the less you belong into your physical space real life community. So, where do we want to belong? Probably both worlds. So, Alysha:

To belong, you must be who you truly are!

Let´s keep this in mind, let´s not get wild about technology. Let´s take a step back and use it to our advantage. To realize our fantasies, to reach our true goals, to create great tools, to connect.

Today we are at the crossroads of things. Let´s not make the wrong choice. Let´s not rush into the prettier looking green forest. Maybe we need to take the detour – the hard way – to get things right for mankind.

… To be continued next days…