Overview of the Visual Effects Process




Special thanks to everyone at Centropolis FX for all of the help and information they have provided in creating this article!
A typical movie might have 1,000 to 1,500 shots. A shot might be one or two seconds long, or 30 to 60 seconds long. In "The Patriot," the longest shot that the team handled was about 1,000 frames. A given scene in a movie might be filmed with a number of different cameras so that there are wide vistas, close-ups, changes of perspective and so on. In the final movie, these different viewpoints are mixed together to create the scene. Therefore a single scene might contain dozens of individual shots. By splicing all of the shots together in the correct order you create the complete movie.

Hollywood movies are typically shot on 35mm film at 24 frames per second. The first step of the visual effects process is deciding which of the shots need to have visual effects applied to them. For example, in "The Patriot," 150 of the shots in the film required visual effects. The rolls of film for these shots are sent to CFX to be scanned. CFX uses a Kodak Cineon scanner capable of up to 4,096 dots of resolution (4,096 X 3,112 dots per frame). "The Patriot" was scanned at 2K resolution (2,048 x 1,556 dots per frame), and the 150 shots and all of the intermediate files consumed about 1.6 terabytes of disk space.

Aspect Ratios
The 35mm film that is used to film movies is the same as the 35mm film you use in a 35mm camera. Instead of being packaged in small 24 or 36 frame rolls, movie film comes in big reels, but it is exactly the same stuff. A movie camera exposes the film at a rate of 24 frames per second.

35mm film is 35mm wide and includes the sprocket holes along both edges. It has 24mm of usable film width once the space for the sprocket holes is subtracted. Therefore a typical frame of a film might be 24mm by 18mm -- it has an aspect ratio of 4:3 when shot.

When projected on a screen in a movie theater, the aspect ratio is much wider -- 16:9 for example. This is much wider than the 4:3 aspect ratio of the film. To compensate, a movie camera uses a special anamorphic lens that compresses a 16:9 image onto the 4:3 film. If you look at the film, therefore, a frame of a movie looks odd -- everything is tall and thin. Once the film is projected through a similar (but reversed) lens at the movie theater, it stretches out to its full 16:9 width and looks normal on the screen.

Once it is digitized, a shot may go through a number of 3-D artists. A big part of realistic computer generated effects is the creation of 3-D models and characters, a process that includes a number of steps. These include:
  • 3-D Tracking - The tracking department uses markers added to the scene to create a 3-D model of the scene and then a 3-D camera. The goal is for the 3-D camera to exactly mimic the motion of the real camera so that the 3-D elements added to the scene look right and move correctly as the camera moves in the actual scene.
  • 3-D modeling - A 3-D model is a collection of shapes that form the outside of the object being modeled. For example, the soldiers in "The Patriot" are formed from a combination of spheres, cylinders and other shapes that are molded on the computer screen into exactly the right configuration to look like a soldier.
  • 3-D setup - Setup is the process of adding a "skeleton" of bones and joints to a 3-D model so that the different shapes in the model move correctly with respect to one another. In some cases the bones and joints are created by hand. In other cases they come from motion capture data (see below).
    Motion Capture
    To gather motion capture data, an actor is fitted with a suit that has reflective markers or lights at every joint. The actor moves on a special stage and 3-D cameras watch the actor from a number of different angles. Computer software is then able to track all of the markers and, with the help of a technician, bind them together into a stick figure that accurately duplicates the motion of the actor.

    The stick figure is the bones and joints that then control and animate a 3-D model. This video shows the transformation of motion capture data to a realistic soldier. A variety of different poses and activities are captured to provide variety, as shown in this video of four soldiers.

  • 3-D animation - In the animation process, an artist choreographs the movement of a 3-D character.
  • 2-D painting - Painting is used extensively by any visual effects team. With painting the team can:
    • Create matte paintings for backdrops
    • Paint out wires, harnesses, brackets and other safety equipment
    • Paint over the "holes" sometimes created by rotoscoping
    • Touch up things like grass that has lawn mower tracks in it (see Example 2 below)
  • 2-D compositing - Compositing is the process of adding all of the different layers of a shot together to create the final shot. In the example shown in the previous section, the composited layers included the rotoscoped piece of the original shot, the water, the painting of the town, the boats on the water, the cannon fire from the boats, the people in the town, the smoke over the town and a new explosion. The compositing artist incorporates all of the elements in the right order so they overlay each other properly to create the final shot.
A visual effects team is responsible for all of the effects shots in a single film. At CFX, a team consists of a producer, several supervisors (for example a 3-D supervisor, a 2-D supervisor, etc.) and a number of artists. One of the first tasks for the team is the research and development process. For example, in the production of "The Patriot," a fair amount of R&D time was spent on creating soldiers and then groups of soldiers that look realistic when added to a scene. The team then uses what it creates during the R&D process to manipulate the shots it is responsible for. This is not a quick process -- a film takes about a year to create.

With the help of the supervisors and artists on the team as well as the film's director, the producer's job is to look at all of the shots, understand what the director wants to change in each one and then estimate the amount of time that all of the tasks will take. From these estimates a huge production schedule is built in a tool like Microsoft Project and the team executes against this schedule.