ISMAR – Tracking Thriller

Keeping the visions from the keynotes in mind, we today go to the other extreme of the ISMAR conference: hardcore algorithms in the tracking competition. This is far away from visionary outlooks, but rather a necessity to create good visual overlays. We go way down to the bottom of the code, tweaking the algorithms and implementing all into the tiny mobile boxes, we carry around today.

So, what is it all about? you might ask, if you haven’t been there or even if you have been there. Let me introduce you briefly to this hardcore IT tech topic, not going to algorithmic level though! :-)

The tracking competition at ISMAR already has quite a history and was this year set up by the nice guys from TU Munich ResearchAR’s. Obviously the best team gets the prize, while each team tries to complete as many tracking tasks as possible.

But how does it work? Basically you as a team get a bunch of 3-D coordinates and you have to tell the judges to which real world object these coordinates belongs.

Let’s make an example: one coordinate lies somewhere around the mirror, i.e. one of the marbles is the target to be identified. You have to have an accurate tracking and visualization to pinpoint the correct item. Then, tell the judges which one it is and you score! Another task was to tell which candy is the chosen one, lying on a colorful table full of tiny candy bars. If your tracking is not well calibrated or accurate enough, you can only pick the right sugarcane by accident…

To achieve this goal of identifying the 14 items during each team’s 30 minutes period, they had the chance of setting up their tracking environment before the tournament (given 80 min. each): small fiducial markers on the walls of the room with known coordinates where there to help you, defining the coordinate system of your world. All markers define a single system and once you register within this system and your tracking is stable you should be able to fly to the unknown coordinates…

Now let’s introduce (a bit more techy) the teams and their approaches:

Team 1: FAR, Technische Universität München, Germany
Frieder Pankratz, et.al.

The TU Munich – as the organizers – took part non-competitively. They set up a fusion of an optical SLAM (Simultaneous Localization and Mapping) based approach using a camera on a mobile device and a smart-phone based optical outside-in tracking system.

Team 2: Millennium Three Engineering, Canada

Mark Fiala’s team used fiducial marker tracking throughout the tracking environment using a partially offline 3D registration process and online pose filtering and fusion with an inertial measurement unit. You have possibly seen their work yielding in the ARTAG library.

Team 3: VTT Technical Research Centre of Finland and Aalto University – School of Science, Finland with Alain Boyer, Petri Honkamaa, Tuomas Kantonen, Otto Korkalo, Timo Tossavainen

A client-server framework based tracking setup using iterative feature-based tracking and target visualization on the mobile client side and pose initialization and recovery on the server side. The only team using a server-based approach. All teams used smartphones nevertheless. (Though it was not mandatory.)

Team 4: Dept. of Virtual and Augmented Reality, Fraunhofer IGD, Germany
Harald Wuest, Mario Becker, Folker Wientapper

Streaming tracking system with mobile phone (image acquisition and visualization) and laptop (tracking and rendering). Optical flow based (KLT) feature tracking (frame-by-frame) using a SLAM approach for geometry acquisition.

Team 5: metaio GmbH, Germany, Thomas Olszamowski

Junaio framework featuring visual 3D markerless tracking running exclusively on a mobile device with support by built-in inertial sensors. Offline learning of an environment map with a marker-based approach which is extended online using a SLAM framework.

Competition

During the competition itself the teams were not allowed to keep any markers around inside the room (markers are still set up in the first picture and were removed during official run). All tracking had to be done visually through a RGB mono camera. Given this fact, you can imagine how tricky things can get trying to get a hold on 2D features if there is no textured floor, just reflections (due to the mirror) or way too many tiny features (candy). Low light condidition tasks, transparent plexi-glass and reflective, shiny, flowing car surfaces without edges made it even worse for the teams!

While the teams were competing live one after another, you could cut the air with a knife where the crowd was observing the final match! IGD – the winner of 2009 – didn’t handle a few tasks and VTT ran into problems losing valuable time for most targets… metaio did a good job using their very own junaio channel for the competition. Take a closer look at the tensed situation in the audience and at the battle between the two leaders metaio and IGD on flickr:

And the winner is…

Team	Correct	False	Not handled	Score
metaio	10	4	0	12
IGD	8	2	4	8
VTT	2	0	12	2
M3 Eng.	1	13	0	0
TUM (no rank)	1	0	13	1

… metaio! :-)

The Future

A great and exiting competition! Congratulations to the winners and lets expect some even more complicated tracking tasks next year! E.g. a rotating Volkswagen scale model…

So, enjoy the tracking and I’ll get back to you with a report on ISMAR-demos next days!