Interview with Fyusion´s CEO Radu Rusu on AR content generation

AR, Interviews, Mobile & Wearable

Last week Fyusion went live with their video showing off their new app that lets you generate 3D models of any real world object quickly and robustly. The video showed a man walking around a woman while holding up his phone towards her. Later on we see him wearing a Hololens and presenting the holographic version of the lady floating in mid-air.

Fyusion claims to have a “patented 3D spatio-temporal platform [that] uses advanced sensor fusion, machine learning and computer vision algorithms” to achieve their results. But does it work out of the box? How can we use it? Time to check for some answers.

Today I had the chance to briefly chat with Radu B. Rusu, theo CEO of Fyusion, Inc. Though there was not much time, I could ask a few questions about Fyusion and the future of AR content generation. So, let´s take a look at the video and jump right in afterwards:

augmented.org: Hi Radu. Thanks for taking your time. We´ve all seen the cool video and are wondering who is behind it. Could you please describe briefly who your team is? Since when do you work on it?

Radu Rusu: We are a team of PhDs and engineers with backgrounds in robotics, machine learning, and 3D computer vision, who have been working on a novel, advanced spatial photography visual format since 2013.

Great. So, how did you come up with the idea for your product? What drove you?

We believe that in this new digital age, with all the new sets of challenges and opportunities we are facing, we cannot rely on legacy visual formats created 100 years ago (such as 2D photography and video), but rather require a new way of thinking about the world in terms of 3D, in terms of space.

Our products include both a comprehensive platform that is currently being deployed on more than half a billion devices with many commercial partners, and our own suite of applications (including Fyuse) built on top of this platform.

Speaking about devices: the app works on iOS and Android today. But do you also already offer a ready-made Hololens app to record and show content for example? If it not available today: when will the app be available for those devices?

We are waiting for the Hololens device to be available for the public first, before committing to releasing a consumer product. In the meantime, any Fyuse representation created using our consumer application is AR ready and will be able to be shown in Hololens in the future.

Great. So, let´s talk a bit about the technology. Do you create full 3D models of the scanned objects? Or is it currently only a number of 2D images that change when moving around or tilting your device? From the video we cannot really tell.

We generate a 3D representation of the captured subject and display that to the viewer.

So, when converting it to 3D you key out the background like shown for the hololens demo?

Yes, our processing pipelines involve the creation and modeling of holograms from our own 3D spatial photography format (i.e., fyuse) with high accuracy, thus removing background elements from the subject hologram.

How is the created content stored? Can the data be reused and shared in other applications?

Yes, as long as these applications use our SDK.

OK. So let´s take a look at the SDK asap, please! Will your software and SDK be available for other devices like Tango, Vive, Oculus or for example the Meta?

We are currently focusing on AR devices only, but are investigating the potential marketplace for VR as well.

Cool. So, can you give us your thoughts on what needs to happen for AR to be successful for the broad public?

We believe that for consumer AR to be successful, anyone should be able to create content for it. Currently, there is a small number of content creators and an even smaller number of hardware players. Each is seemingly dependent upon the other to make the first move. With Fyusion’s technology, all content that is created on the platform today (including with the Fyuse app) is currently ready for AR and we’ll be announcing major partnerships with top tier AR hardware producers soon.

So, what other plans do you have for the near future?

There are going to be several announcements in the next few months regarding other technological advancements from our company that we unfortunately cannot disclose at the moment. All we can say is that we’re just getting warmed up ;-)

Sounds great! Can´t wait for the news. I`ll be happy to see more from you guys and to be able to generate 3D AR content with it. So all the best and thanks a lot for your time, Radu!

So, it seems we can expect more coming up from fyusion. Let´s wait and see! It is definitely a great next thing to have 3D memories that can be created by the masses. But how long will it take to have a proper 3D scan of your wedding guests? Guess it´s back to square one of photography… where exposure times are minute long and people have to stand still again. But, what the heck! Count me in! :-)