Augmented Reality for Movie Productions (I)
“Movie productions and AR – does that match?”, you might ask. Movies are produced offline and not real-time! Well, that’s true and I don’t want to get into the field of interactive “movie” experiences or games like in theme parks, either. But I want to focus on one special topic: helping movie productions with previsualization (previs) and real-time – but this time augmented!
For that reason I want to recap shortly the process and in the second part during next week I am very happy to be able to publish an interview I had with the CEO of a leading industry company in this field of live previs and “live post-production”!
Combining computer images with real actors and real sets for a movie production is a complex task. We all know the bad results from B movies and outdated movies, that used to look awesome, but fail to convince us today. When it’s done well, we will forget that it is actually a fake & combined image we see on the big screen. Visually great movies like Lord of the Rings, Avatar or (going back) Terminator or Tron are great examples of this combination of real and virtual. Especially since CGI blockbusters rule the cinema we are more sensitive to visual effects and defects: we won’t immerse until it’s done 99%+ right. We’ve reached a level of visual perfection, that is so scary, the images can make any dream convincingly true…
…but there remains one problem: the production process is a sequential more-step-process, that only integrates computer generated content after shooting the real actors and real sets. “What’s the biggy?” you want to ask. The huge problem with this approach is that neither actors nor directors nor the photographers/cameramen can actually see and experience the monsters, dinosaurs, flood waves, etc. This leads to blind acting and demands a high level of imagination, intuition, a lot of repetitions and a long post-processing period for integrating the CGI material right. Once the real actor was shot, there might be no chance to reshoot this actor later on… The cameraman won’t know how to plan the shot and where to point the camera (possibly cutting off parts, that would only appear later on during the post processing). This again leads to numbers of repetitions and makes it more costly in production. Acting blind leads to worse performances and to get the timing right between a not-yet-existing virtual character or prop and the real actor is very hard.
You see these problems every day during the news, when the weatherman stands in front of a green screen (see tech video below) pointing nowhere, while possibly checking his acting on an extra screen aside the stage. This usually leads to funny poses, when the weatherman tries to get a glimpse on that screen without doing it too obvious. Sorry, it always fails. ;-) Even though, it already is a live integration with a fixed camera!
But getting back to movie productions: as it is a complex task with a lot of variables and high costs, the producers started visualizing the story boards more accurate. They not only produce a shot list with a nice drawn story board, but also create complete sequences digitally rendered in 3D to get a previsualization – even before shooting the first piece of celluloid. This ranges from moving 2D cardboards through space to a madness of pretty-good-realism. The below video shows an example of a production with a “previs” before starting the actual production and the final result:
This way it is possible to have a more accurate planning of the later-on shot. During my work at Berlin based company rise|fx we had the same tasks: previsualize in a 3D real-time engine to check for dolly setups, green screen sizes, camera angles, right lenses, etc.
… but it was still only done before shooting. On set the cameraman, director and actor didn’t get another chance to get an advantage out of the technology. The basic setup could be done better and planning the shot before was easier. But now it’s again every man for himself/every woman for herself…
The expensive previs stops here. Everyone is stuck with his intuition, a portion of luck and hopefully a pile of money to have enough time of reshooting it over and over again until it fits later on with the not-yet-existing CG.
But here we are today. Big studios like WETA with Peter Jackson shoot augmented steady cams, ILM does on-set integration of Caribbean Pirates… With augmented reality on set we can overcome the limits of the classic post-production way! How this works in detail, how it helps the overall movie production and the final quality… and even more will be the second part next week! Stay tuned! :-)