CFX began life as a company in 1996 and today has over 100 employees dedicated to the creation of digital visual effects in a wide variety of Hollywood movies. The team consists of artists, technicians, producers and managers who work together to create scenes that are realistic, stunning and totally convincing to the audience.
The word digital in the phrase digital visual effects means that CFX primarily uses computer hardware and software to create its effects. The general term used in the industry for what CFX does is CG, short for computer-generated. For example, when talking to the artists you will hear them say things like, "That entire scene is CG," or "Those are all CG soldiers," or "The actors are real, but everything else is CG." Computer-generated effects make imaginary characters like Godzilla possible, and they also create almost every effect that used to be done using models. The advantages of CG effects are their realism, flexibility and relatively low cost (compared to the alternatives).
The team at CFX works with the movie's director both during filming (production) and then extensively after filming (post-production) to create the effects. Involvement during production helps the director make creative decisions so that the effects can be integrated into the film more easily, and also allows CFX to add different markers and other features to each scene to make post-production work easier. For example, in certain scenes the camera might be fitted with encoders that will allow for easier integration of the effects that CFX creates. During post-production, the director works extensively with CFX to make sure that the effects in each shot have exactly the right look for the film.
CFX uses a variety of tools and techniques to create visual effects. The different capabilities of the company include:
Combined together, these techniques allow a visual effects team to create nearly anything that the director can imagine. It is an amazing set of tools!
The easiest way to understand what a visual effects team can do is to look at an example. The first example that we will use involves a surprising number of techniques to completely change the landscape from a wooded field to a seaside town! This is a shot from the movie "The Patriot."
To get a feeling for the total transformation, take a look at these two short videos. The first shows the original shot as filmed. The second shows the scene as it appears in the movie:
The shot arrives as a piece of film, and the first step is to scan this film into digital form so it can be manipulated. Scanning is done frame by frame (movie film is shot at 24 frames per second), and then each frame is stored as an individual full-color image with a resolution of 2,048 by 1,556 pixels. The section of video shown in the two previous videos is 20 seconds long, or about 480 frames.
First, a variety of things might be done to the original shot to clean it up, correct the color and so on. Since the camera pans across the scene in this shot, the next step is to build a 3-D model of the camera so that all of the visual effects the team will create will mesh with the original scene exactly.
One of the first steps taken to add visual effects to the scene is the rotoscoping. An artist sits at a computer and, frame by frame, outlines the portion of the original shot that will be used in the final version. The breastworks, a portion of the field and several of the running soldiers will all be used, but the explosion seen on the left and everything else will be removed. The artist will outline these elements and essentially "lift" them out of the frame.
Another camera crew has created an ocean shot -- in this case the ocean is not far from the CFX studio in Hollywood! Once the ocean shot is cleaned up and color-corrected, it and the rotoscoped scene can be integrated:
Another artist in the paint department has been working on a matte painting of the town. This is a high-resolution digital image created using a painting/illustration package.
Once the town is added in, the scene looks like this:
In this shot there are a number of boats encircling the town and firing on it. The boats are all computer generated. Each one is modeled and then added to the shot:
The cannon fire for the boats is its own stand-alone effect that is created separately and then added to the shot.
A variety of other effects are added to the shot, including things like smoke over the town, people in the town, a large explosion on the right, etc.
The final step in the process is compositing all of the different components of the shot (the rotoscoped foreground, the water, the town, people in the town, the boats, the cannon fire, the smoke, the explosion, birds flying overhead and so on) layer by layer to create the final shot as it will appear in the movie. Once digitally composited, the shot is written back to film so that it can be spliced into the movie.
The small images used here don't really do justice to the shot -- when you see "The Patriot" on a huge screen in a theater, you can see the incredible detail that has been added to make this shot look totally realistic.
As you might imagine, creating a scene like this is time-consuming. Even though the shot only appears on screen for 20 seconds, hundreds of hours go into the visual effects to create a rich, convincing visual image. In this video, several of the artists who worked on the shot describe how long the shot took. It is an amazing process that involves an entire team of people, and this process may be repeated several hundred times in a single film! In "The Patriot," 150 shots in the film involve visual effects like these.
A typical movie might have 1,000 to 1,500 shots. A shot might be one or two seconds long, or 30 to 60 seconds long. In "The Patriot," the longest shot that the team handled was about 1,000 frames. A given scene in a movie might be filmed with a number of different cameras so that there are wide vistas, close-ups, changes of perspective and so on. In the final movie, these different viewpoints are mixed together to create the scene. Therefore a single scene might contain dozens of individual shots. By splicing all of the shots together in the correct order you create the complete movie.
Hollywood movies are typically shot on 35mm film at 24 frames per second. The first step of the visual effects process is deciding which of the shots need to have visual effects applied to them. For example, in "The Patriot," 150 of the shots in the film required visual effects. The rolls of film for these shots are sent to CFX to be scanned. CFX uses a Kodak Cineon scanner capable of up to 4,096 dots of resolution (4,096 X 3,112 dots per frame). "The Patriot" was scanned at 2K resolution (2,048 x 1,556 dots per frame), and the 150 shots and all of the intermediate files consumed about 1.6 terabytes of disk space.
Once it is digitized, a shot may go through a number of 3-D artists. A big part of realistic computer generated effects is the creation of 3-D models and characters, a process that includes a number of steps. These include:
A visual effects team is responsible for all of the effects shots in a single film. At CFX, a team consists of a producer, several supervisors (for example a 3-D supervisor, a 2-D supervisor, etc.) and a number of artists. One of the first tasks for the team is the research and development process. For example, in the production of "The Patriot," a fair amount of R&D time was spent on creating soldiers and then groups of soldiers that look realistic when added to a scene. The team then uses what it creates during the R&D process to manipulate the shots it is responsible for. This is not a quick process -- a film takes about a year to create.
With the help of the supervisors and artists on the team as well as the film's director, the producer's job is to look at all of the shots, understand what the director wants to change in each one and then estimate the amount of time that all of the tasks will take. From these estimates a huge production schedule is built in a tool like Microsoft Project and the team executes against this schedule.
Digital visual effects at the Hollywood level require an incredible technology infrastructure that includes both hardware and software. The machine room at CFX shows just how extensive the hardware investment is. The team needs massive amounts of hardware for four reasons:
All of this means that the machine room at CFX is an amazing place!
This quick video walk-through of the machine room shows you the scope of the equipment required for the team at CFX to do its work.
The workhorses at CFX are a collection of SGI Onyx machines. These machines contain multiple processors and specialized graphics pipelines that speed the rendering and compositing tasks. Typically, a compositing artist will use an Onyx as his or her "desktop" machine, and other artists will run background rendering tasks on them as well. There is also a Linux render farm formed out of dozens of PC desktop machines. These machines work together to subdivide and execute rendering tasks.
A big part of "The Patriot" is the battle scenes, and it turns out that most of the battle scenes contain computer generated soldiers. A good part of the R&D process for this film involved the creation of the individual soldiers and then groups of soldiers that would look realistic and convincing in the movie.
Creating CG soldiers that look realistic in a theater is not easy because the human eye is extremely sensitive to repetition and duplication. People are also very good at detecting artificial motion, even at a distance.
To create realistic groups of CG soldiers, the team at CFX started with real actors trained to perform in revolutionary war reenactments. These actors were outfitted and then filmed on a motion capture stage. The motion capture data allowed the team to create stick figures acting in a number of realistic ways -- walking, loading guns, falling and so on. For example, this video shows the motion capture data for four soldiers performing different activities.
The team fleshed out these stick figures with 3-D models of soldiers and then created four different variations (for both armies) with different heights and widths:
The different soldier models were then applied to the motion capture data to animate them, and then the soldiers were formed into groups.
These two videos show the process:
One big part of the project was "dressing" the soldiers -- a process done by texture-mapping different fabrics and equipment onto the 3-D models. In "The Patriot," CG soldiers are only viewed from a certain distance, so advanced elements like cloth dynamics and facial expressions did not have to be incorporated into the models. If they had, the computational load -- already severe because of the number of soldiers -- would have increased dramatically.
The following shot shows how groups of soldiers get added to a scene. Here is the scene as it was shot:
If you look carefully, you can see a number of bushes in the far field, as well as something as mundane as lawn mower tracks in the grass in the foreground! The visual effects for this scene involved everything from changing the sky to redoing the grass to removing the bushes and then adding in two armies. The following pictures show the process.
These small MPEG files have no way to show the richness and depth of the final scene as it would appear in a theater, nor do they demonstrate the realism of the two armies. When you see the scene in a theater, it is amazing!
This example shows how the film's director can look at a scene and decide to completely change it using computer generated effects. In this case, the film crew shot a scene on a street in Charleston, SC.
This video shows the shot as filmed. The actors were dressed in period clothing, and the street was covered with a layer of dirt to hide the asphalt and take it back in time. Compare it to this video of the final shot as it appears in the movie -- the transformation is amazing!
The director made a decision to transform the end of this shot into something completely different -- instead of buildings down the left side of the street, what the director wanted is a dock, like this:
To create this effect, CFX did the following. First, using rotoscoping, the actors and the wall were lifted out of the frame of the original shot:
A new sky was added in to form the backdrop for the new scene:
Models of the docked boats were filmed against a green screen, rotoscoped and then added into the shot, and several buildings (both real and CG models) were added on the right:
Various crates and barrels were added to the dock to break it up, and there's even a new cloud in the sky if you look above the buildings:
CG flags were created and added to the ships:
All of this creates the final image:
These videos are interesting:
A scene transformation like this shows just how far a director can go in getting the right look and mood for a shot. About 90% of the scene is different from what was originally filmed. The magic of CFX is that the team can work together with the director to create completely new realities that look totally believable and convincing on the screen. The amount of effort it takes to work at this level is enormous, but as you can see in a film like "The Patriot," the final results are spectacular!