Interview with Audi – AR, VR and a Hackathon at Digility

Today, I’m happy to publish my interview I had with Audi. I was interested in learning more about their current AR/VR activities and their plans for the Digility conference next week. Since they have a talk there and organize the before-mentioned hackathon, I wanted to give my readers some insights ahead of time. Find out about their mixed reality plans today.

Jan Pflüger, Audi

I had the chance to talk to Jan Pflüger (pictured aside) and Jens Angerer (pictured in the featured image). Jan is coordinator of the “Center of Competence” at Audi to control the activities and roll-outs of AR/VR solutions. He is long-term involved in this field and was lecturer at the Universities of Applied Sciences Pforzheim and Northwestern Switzerland before he switched to the industry. I know him since his days at RTT AG / Dassault Systemes / 3DExcite, before he switched to Audi. Jens Angerer is Project Lead for Human Machine Interfaces at the Audi Production Lab, which he co-founded in 2012. He specialized in VR User Interfaces longer ago. Currently, Jens is also teaching VR/AR to UX Design and Computer Science students as an adjunct professor at the University of Applied Sciences Ingolstadt. The interview became a little larger, but I didn’t want to cut down on it. So, let’s jump right into it!

Let’s talk about AR/VR

augmented.org: Hi Jens, thanks for taking your time. This year’s Digility conference is right around the corner and you will be talking about “The true meaning of Mixed Reality”. Can you explain what you mean by that?

Jens: Sure! Essentially Augmented and Virtual Reality are just different forms of Mixed Reality, which I conveniently abbreviate with xR. But if you ask someone else he might use different terms and abbreviations for the same things. How can that be? I think we are missing a common understanding as well as common terminology and standards for these technologies. Even more, the xR community – especially in Europe – is missing a joint effort to advance this field together. There won’t come a better time to collaborate and create meaningful xR experiences – both in B2B and B2C. At Digility I will talk about my approach to this and show some examples how Audi is using xR today. And thanks to Jan, Audi is also shaping the future of xR by sponsoring the Digility Hackathon.

Hi Jan, great to chat again! I’m looking forward to meet up in person soon again. Typically, we meet at conferences, where Audi is present with an R&D booth or a center stage demo. How come that you organize and support a hackathon at this year’s Digility conference?

Jan: It is always exciting to see the outcome and what is happening during the limited time of a hackathon. We take this as an opportunity to support and establish the European xR scene. Digility provides an excellent framework for this and we look forward to come in touch with the creative minds to advance topics such as virtual collaboration, universal interaction and intuitive UI / UX designs.

Hackathon tasks are often revealed only shortly before the event. Can you already give an idea what the developers can expect? What do you expect as Audi, as a take-away or result for you?

Jan: I do not want to spoil anything just yet, but the participants can expect some exciting use cases that are relevant not only from an automotive perspective. Since we have a wide range of application areas, there will surely be something for everyone. Working in interdisciplinary teams together with the mentors of the Hackathon will be a unique experience for every participant. Not to mention the very cool prizes for the winners! Of course, we are also interested in talents and hope for some new approaches, which we will pursue together in the future.

Cool. Before we check the future, let´s talk a little bit about the beginning. VW/Audi has been long-time involved in AR/VR activities, I’m thinking of research projects like ARVIDA, ARVIKA, etc. Can you give us an idea when and why research started in this field?

Jan: Personally, I had the first contacts to the big research projects in my role as Research Associate at the Institute for Interface Design in Switzerland. The foundation for research was laid way before I started working for Audi. As far as I know they are carrying out research in this field for almost 20 years. We saw the potential of Augmented and Virtual Reality at an early stage and worked hard to bring those research projects into application at brands like Audi and VW. But the results and insights of this work have spread far beyond this. Think for example of metaio, close partner of us for years, whose technology still today at Apple drives the industry.


Example for AR at Audi for the consumer: eKurzinfo from 2014; More going on behind closed doors…

So, how do you currently support this field today?

Jan: Virtual techniques have been established as an important part of the digital transformation. This was obvious for us long time ago and as one of the outcomes we founded the Center of Competence AR&VR. This serves as a central network node for xR and provides advice for our internal customers regarding the use of AR/VR technologies. At the same time, we are collecting requirements and discuss them with our external partners to improve the solutions in this field continuously. This is where Jens and I are closely collaborating.

Jens: Absolutely. Jan and I are both driven to bring new technologies into application. I’m often describing my work at the Audi Production Lab as “Science Fiction”. I am inspired by the Fiction part about xR and use Science to make it come to life. The prototypes that we build in our lab then come into application at our production sites and logistics at Audi worldwide.

Can you give us an example of where AR is being used in a productive or pilot environment today at Audi? What technology are you using? Where is the biggest practical advantage in an industrial environment?

Jens: There is a broad range of use cases we explore today. In logistics for example we train people with VR. In production, we use wearables like Google Glass to guide our employees in very complex engine assemblies. Our studies have shown that this reduces assembly time and improves Human Machine Interaction as well as flexibility. AR however is still very challenging to implement, but training is definitely a promising area.

Jan: AR has a great potential. For the launch of HoloLens we developed a service and aftersales scenario together with Microsoft, which we are now implementing successively into our productive processes. The biggest advantage, of course, is that you can use AR to set information into a real context. Our teams and specialists are spread all over the world – working collaboratively together on projects is a key factor to enhance the product process and solve issues when they appear. Using technology helps us to connect the right people with the correspondent information – additional preparation work and specialization on every side can thus be reduced. In general, I see the advantages of AR across all areas – some can be realized in a timely manner, for others processes must be adjusted appropriately.

Talking about adjustments… 3D-Rendering gets better and more performant, tracking quality rises and practical AR glasses are (hopefully) close to release. Still, hardware and software is sometimes a bit complicated to use or error-prone. What do you see as the remaining main challenges?

Jan: In my eyes the most important question is: What is the main part? I do not want to wear AR hardware for the sake of wearing strange goggles. I saw a lot of demos and “use cases”, but to be honest – most of them are manually generated demonstrators without connection to the “real” environment. The challenge will be to deliver the right information in the right context. If this could be fulfilled in an automated manner the future of AR will be ready.

OK, tech is one thing. The other are the users and the integration into their workflow. People are used to their IT landscape and day job routine. Did you already solve to integrate these technologies into productive workflows, from a technical standpoint but also from a user perspective? Is to say: is it mature? What’s missing?

Jens: As a User Interface guy, I still see huge challenges before we have ubiquitous AR. Just as an example: I am avoiding to wear my prescription glasses whenever I can. The weight of this relatively light frame is unacceptable for me, although it gives me eyesight. That’s a high barrier for AR glasses to take.

Jan: We are working on the solutions to ensure the integration into our future IT landscape and process chains. We also help to reduce the barriers to the use of technology by showing the advantage of such technologies in various events.
As a company, you have to take responsibility for the upcoming changes. This leads to many questions. How does the solution affect the daily work of the employees? What qualifications will be required in the future? Also, new technologies do not usually have robust studies on health aspects and work safety. There must be appropriate answers for topics such as hygiene and so on… With this not yet complete list, I would like to point out that behind a truly productive integration into the corporate context lies much more than to implement technological solutions. For us, therefore, hardware or technological issues are not in the main focus. The solution lies rather in the integration into processes or rather – how must the processes be designed to take advantage of the technology! We have evolved suitable fields of application to take advantage of AR, but I still see a way ahead of us, until it will naturally be part of the workplace and our daily environment.

Thanks for the insights, maybe let’s take a step back and check the current AR/VR landscape in general. You do AR/VR for a long time now and we see the advantages in specific industrial scenarios. Now, in 2016 and 2017 AR has gained a bigger popularity outside of research labs. Everybody talks about it and recently Apple released their ARKit. How do you recognize the current hype around AR? Is it justified? A bubble or for real?

Jan: For me it is for real – I always believed in the power of AR! ;-) By now we have the first time an easy access to the technology. Platforms enabling artists and developers to create stunning content and pushing the technology to its limits. AR can be experienced by everyone. The hype around Pokemon was a precursor. It’s a question of expectation – in my opinion, it is unrealistic to wait for the one killer app that carries AR at once in all households. This will be done successively and along with platforms that provide the relevant content. At some point, AR technology will be integrated into our everyday life as naturally as the use of the smartphone is now.

Jens: I agree. Smartphone based AR will be the main platform for the years to come. Everything after that will need to make the smartphone obsolete.

Comparing your industrial view and specific AR there. The industry has other requirements and more money to set up big tracking volumes or render farms. John Doe on the street might not have all this available, but is willing to use AR, too. How do you see the maturity level of AR for consumers today? As you say, will it be the smartphone for consumer AR for a longer period? Will e.g. ARKit cause a big bang? Or what is missing?

Jan: With regard to operability and scaling, we must also take other paths and challenge classical infrastructures. Solutions we want to bring to the customer must ultimately also work on a generally accessible basis. Everything that helps to simplify the access to and the use of AR content will promote the penetration into our everyday life. ARKit makes the platform discussion exciting again and, of course, also the field of applications we can expect in near future. If we talk about AR HMDs, which in our context are usually more interesting, the consumer market is much more limited.

Jens: I believe that without John Doe using AR, it will also stay to a niche for the industry. So I’m hoping for a big bang in AR – regardless of what causes it.

Speaking about missing pieces. What are you hoping for to come next in this field of tech? What do you wish for? Or maybe, … you can even give us a short sneak peek into Audi’s activities here? What can the consumer expect in the future?

Jan: For me it would be great to overcome some limitations such as low field of view, bad tracking experience and especially latency of the system to speak about tech. But the challenge is another one: we need to transform our processes to be ready for the future. Only then can we not only support our current development in a better way, but also enter new areas of product development and product experience. We are combining areas such as machine learning and AI with current data analytics and sensor / IoT topics and are trying to make the findings come true in reality. Audi is working in a lot of areas today to push not only technology, but also customer experience to a new level. So I am confident that you will see more and more in the future.

Jens: I will show some of our activities in my presentation and talk about THE Killer App for xR: being together. I believe there is nothing more important for the future of xR than the possibility to communicate and collaborate over great distances. And of course, Audi will have a booth at the Expo and the Hackathon. So really good reasons to attend Digility 2017 I believe.

augmented.org: Well, thank you so much. Very glad for the insights you shared. Looking forward to your presentation then for more to be revealed! Good luck with the Hackathon during Digility!

Conclusion

So, there you have it. Another voice supporting the smartphone access next to AR. For me, it’s a tough call to hear from Jens that regular glasses already annoy him during work – even if they are superslim (without AR). But let’s see how things turn out, I’d be happy to put on slim AR glasses if they helped me with my task. No, I’m not willing to put in AR contact lenses – ever! Anyway, a good chat and happy to share some opion from industry perspective. There are still some hackathon spots left, so feel free to jump over to Digility page and sign up.

Cheers!