Industrial Light & Magic (ILM) is a name that evokes an excited reaction from most movie fans. Ever since George Lucas created the company in 1975 to provide visual effects for the first Star Wars movie, ILM has been synonymous with movie magic. In fact, the visual effects created by ILM have appeared in eight of the 10 highest grossing movies ever. ILM has won an amazing 28 Academy Awards: 14 for Best Visual Effects and another 17 for Technical Achievement!
You have seen ILM's spectacular work in such movies as:
- "Star Wars"
- "The Empire Strikes Back"
- "Return of the Jedi"
- The "Indiana Jones" movies
- Star Trek: Generations
- Star Trek: First Contact
- "Jurassic Park"
- "The Lost World"
- "Men In Black"
- "The Mask"
- "Forrest Gump"
- "The Perfect Storm"
- "Star Wars: Episode I: The Phantom Menace"
- "Star Wars: Episode II: Attack of the Clones"
- "Star Wars: Episode III: Revenge of the Sith"
- Harry Potter and the Goblet of Fire
- Mission Impossible III
- Pirates of the Caribbean: Dead Man's Chest
ILM has pioneered many groundbreaking effects and processes in the last 30 years. It would be hard to cover everything that ILM has done in any one article (or even in a single book), so in this article we'll focus on the incredible and revolutionary water effects used in "The Perfect Storm" as an example of their work. "The Perfect Storm" provided ILM a unique opportunity -- integrating artificial, computer-generated water with real water. Given that foaming, blowing, churning ocean water is one of the most dynamic elements in nature, the challenge was immense!
In this edition of How Stuff Works, you'll learn how ILM works with the filmmakers, how they plan out the effects, how the effects are created and how they are incorporated into the movie. You will also learn about simulations and particle systems.
You might think that it is easy for a company like ILM to create computer-generated (CG) effects for any film. Actually, it is an intense process that requires tremendous amounts of research and lots of painstaking labor. ILM has a staff of more than 1,000 skilled employees, including:
- Visual effects supervisors
- Technical directors
- Software developers
- Art directors
- Camera operators
- Stage technicians
Each of these people has an impressive array of abilities. For example, Habib “Particle Man” Zargarpour, the associate visual effects supervisor for "The Perfect Storm," has a bachelor's degree in applied science and mechanical engineering and another bachelor's degree in industrial design (see sidebar for more information on Habib). The huge computer models that simulate the water in "The Perfect Storm" were all researched and written during the movie's production by a team that included Habib, staff scientist John Anderson and many other people. Talent like this is the key to ILM's continuing excellence in CG (computer-generated) effects.
The Film Process
The first thing that happens in the creation of any new film is the studio's decision and announcement that it is starting the film. The studio chooses a director for the film, and he or she begins to look for companies to handle the various components of the film.
In almost any film today, computer-generated (CG) effects play an important role, even though you may not even detect them when you watch the film. For example, in 90 shots in "The Perfect Storm," everything is computer generated -- the water, the boats and even the actors -- and everything looks totally real. ILM bids on work for a variety of films, and pitches the company's ability to provide the effects to the director.
The director, in this case "The Perfect Storm’s" Wolfgang Petersen, met with the team, and agreed to farm out the visual effects to ILM.
A film typically follows a standard pattern of steps as it is developed. The steps go something like this:
- The story and/or the script gets created.
- From the script, the team creates a set of storyboards. A storyboard is a collection of still drawings, words and technical instructions that describe each shot. A shot is a piece of the movie from a single camera angle. For example, if two people are talking to each other in a scene, and the camera switches 10 times back and forth to show the two actors' heads while they are talking, that single scene contains 10 shots. A typical film might have 2,000 shots in it, and they get spliced together one after the other to create the whole film.
- Then the film team splits into several groups: One group works on designing and constructing any sets needed in the film. Another group scouts out and prepares for on-location shots. One group starts the research and development process on the elements for all of the different CG shots. Another group works on preparing for all the shots involving stunts. Another group works on any physical special effects, such as animatronics -- for example, the robotic shark in "Jaws" was an animated model. Another group works on wardrobe. The filming group prepares the cameras, lights, sound equipment, etc. for the different shots. In "The Perfect Storm," ILM had three roles on the team. First, ILM developed the 3-D animated storyboard for the film. Second, ILM created all of the 90 pure CG shots. Third, ILM integrated water and other effects into 250 shots that contained live actors or ships shot on a blue screen (see How Blue Screen Effects Work for details).
- The process of filming all of the live-action shots is called production. Production involves actors and stunt people working on location, or on sets, to get all of the footage for the film.
- Once the production phase finishes, the film enters post-production. In post-production, many different things happen: The parts of the film in which digital effects need to be added are digitized. Shots are built in which CG elements and live action elements combine together in a single shot. Shots are built which contain miniature models that may also incorporate live action or CG elements. All the CG shots are rendered. Stunt shots get touched up to remove wires, safety harnesses, etc. The shots get cleaned and color-corrected.
- The entire film gets assembled from all of the shots.
- The musical score and any sound effects get added.
- The film gets mastered, duplicated and sent to theaters (see How Movie Distribution Works for details).
Planning the Film
Upon reaching an agreement with Petersen, ILM named a visual effects supervisor, Stefen Fangmeier, to oversee the project. Fangmeier assembled a crew of 120 ILM staffers to develop the 340 visual effects shots needed for "The Perfect Storm." Ninety of the shots in the movie are completely computer-generated (including virtual actors), and another 220 have CG water or other elements. In fact, only two of the stormy ocean shots in the entire film are completely real!
The film is carefully planned out, shot by shot, to match the overall vision that the director has for the film. ILM is involved even at this early stage, preparing a rough 3D animated storyboard known as an animatic. Animatics allow the director to ensure that the planned shot will work, both visually and practically.
Here is a good example of how the animatic can change the development of a shot. Many of the close-up shots of the boats occurred in a mammoth indoor wave pool, about a quarter of an acre (1,000 square meters) in size! Full-size boats would be controlled by giant gimbals in the pool during the shot. The camera needed to move quite a bit to simulate the rolling movement of another ship in order to match up with the virtual camera viewpoint used in the visual effects. The originally planned position of the camera did not work when simulated -- the camera's motion would have taken it through a wall! By tweaking items like this in the storyboarding phase, directors are able to avoid many potential pitfalls that could lengthen the filming process and exceed the projected budget.
How Water Works
Water, it turns out, is an extremely complicated substance. Go down to the beach one day and really look at the water, and you can quickly understand how complicated it is. Watch a wave curl up onto the beach. You will see, among other things:
- The swells, well away from shore, that will become waves
- The waves forming and curling forward
- The wave's color and translucence as the water thins toward the top
- The foam at the top of the wave
- The receding foam that follows the wave and dissipates
- The foam forming in front of the wave
- The bubbles in the water that form part of the foam, and the way that further changes the color of the wave
- The water churning as the wave falls
- The movement of the water in the wave as it hits obstacles
- The interactions between multiple waves
- The water that splashes away in the wave's crest and as the wave falls
- Changes to the wave -- different foams, and water splashing away -- caused by the wind
ILM's job was to simulate all of those effects so closely that ILM's artificial water could be integrated into real water from live shots and you would be completely convinced that it was all real!
A big element of "The Perfect Storm's" planning process involved researching the water and other objects that the team needed to emulate with CG effects. In "The Perfect Storm," the team was creating an entire ocean environment along with simulated boats, actors, buoys and so on. The sheer amount of knowledge required is amazing! Here are some of the things that Habib Zargarpour and fellow ILMer John Anderson researched:
- All of the water effects discussed above
- The various types of waves
- Hull shapes
- Buoyancy characteristics
- Water density and surface tension
A good understanding of how wind and water and solid objects interact with each other was essential to the creation of a realistic sea. In addition to the physics of the objects, the team carefully observed and noted how light affected the appearance of the water and mist. All of this knowledge was put to the test when they began to create their virtual ocean.
Simulated Oceans and Maya
John Anderson, in-house scientist for ILM, developed a sophisticated simulation of the ocean that could be manipulated to create the type of rough seas that "The Perfect Storm" needed. The simulation needed to handle everything from rippling, still water, to waves the size of a 10-story building.
The set of scientific rules that simulates the way water acts and reacts is called, generically, fluid dynamics. Fluid dynamics governs everything from airflow over a wing to any type of moving water, as well as the way honey flows out of a jar -- it is a very broad field! A well-done water simulation obeys the laws of physics for water very well, and is incredibly accurate in the ways that it follows the laws of fluid dynamics. However, it also has to take into account the "look" of the water in different lighting conditions, as well as integrating the water in live shots.
Basic fluid dynamic simulation provided the foundation for the body of the ocean, and became known as the bottom water in the film. The way the ocean interacts with the rigid body objects, such as the boats, allowed the team to understand exactly how each object would move in the real world under the same circumstances.
The model of the Andrea Gail (the main boat in the film) is itself a wonder, with objects on the boat, such as cables and buoys, reacting to the wind and movement of the ocean as well.
In addition, to help capture the rolling motion of a stormy ocean, ILM placed a virtual camera into the simulation on a second boat. The virtual camera pointed to an invisible target object located on the boat that was the focus of the shot. The virtual camera had a certain degree of freedom that emulated the difficulty a person would have holding a camera while trying to stay trained on an object that is moving. This method is used in several shots to enhance the realism.
Most of the actual 3D modeling work took place using a commercial software application package called Maya, made by Alias|Wavefront. The cool thing about Maya is that it contains a complete programming language, C++, that allows animators and designers to write their own custom plug-ins. "The Perfect Storm" team at ILM wrote more than 30 plug-ins for Maya for this movie. They also wrote several stand-alone applications for specific aspects, such as shaders and particle systems, of the ocean scenes.
To make the top water (crests on the waves, the mist and foam and all the millions of water droplets in each and every splash), ILM relied heavily on particle systems. In the world around you, everything is made up of particles in some way. Essentially, a particle is a component of a larger object. Your body, for example, is made of trillions of cells working together, and each cell is a particle. Each cell, in turn, is made up of trillions of atoms, and each of atom is a particle as well. In things like water, dust, snow and rain, the entire visual effect that you see comes from the interaction of trillions of particles. They all react with each other, other objects, gravity and the air and wind to create the patterns we see.
Realistic models must manipulate huge numbers of particles if they want to look authentic. Calculating and displaying the path that each particle should follow is the key to creating a convincing CG version of a particle system. The number of computer operations needed to track each particle is gigantic, and can lead to scenes that require days of computation time per shot, using rooms full of high-speed computers!
In computer-generated graphics, most 3D objects are only visually representational of a real object. In other words, they are not constructed down to the atomic level to exactly match the original object. To make such detailed objects is well beyond the processing capabilities and time constraints of current systems. Typically, the object is simply a shell or framework with nothing inside. Most of the time, it doesn’t matter, as long as the object looks real. But if the object has to change drastically during the shot -- for example, an ocean wave cresting and spraying over another object -- then there has to be a way to represent that change realistically.
In "The Perfect Storm," ILM accomplished this feat by using multiple particle systems. Built right into the ocean simulation is a particle system that creates millions of particles each time a wave collides with an object, whether it is a boat, a person or another wave. The particle system is assigned properties, or behaviors, that tell each particle how it should act after it is emitted from the system.
Another integral part of the simulation that affects particle behavior is the collision model. Simply put, the collision model tells each particle, as well as every other object in the scene, what to do when it comes into contact with the surface of another object, or with another particle.
Let's take a look at a wave breaking over the Andrea Gail and how the Maya models simulate it:
- The wave swells and forms. (part of the ocean simulation)
- It collides with the Andrea Gail. (a rigid body object placed in the ocean simulation)
- The impact causes the wave to break over the boat. (the particle system emits millions of particles)
- As the water from the wave spreads out, it continually separates into smaller and smaller droplets. (Each particle has been assigned the particular properties of water, which include the dynamics of surface tension versus outside forces.)
- Some of the droplets eventually become mist, and blow away on the wind. (once again, following the physical laws assigned to the particles and using aerodynamic simulation to calculate the wind forces)
- Most of the water falls onto the deck of the Andrea Gail and trickles back into the ocean, creating foam. (The particles follow the rules of the collision model upon reaching the surface of the Andrea Gail object.)
The computers are performing all of those calculations individually for each particle it is tracking. By precisely controlling the rules applied to each particle, the whole thing can look totally realistic when viewed as a whole.
However, the simulation doesn’t always "look right," or give the director exactly what he wants. For example, the amount of spray coming off the bow of the Andrea Gail may not be as heavy as Petersen wishes. In such cases, ILM would adjust the hundreds of parameters of the simulators to spray out additional particles in very precise sequences. Also special invisible objects, called emitters, are often used to create environmental effects, such as fog and mist.
Shaders are sophisticated tools that analyze the light, density and surface color of an object, and determine exactly how it will appear. Used in conjunction with the particle systems, the shaders created water droplets that were not simply white dots, but instead were convincing, translucent objects.
The level of complexity that the particle systems added to each shot is amazing. In some of the shots with large waves, the number of particles per frame can be measured in the billions! Calculating and rendering such detailed graphics takes an incredible amount of time, and the disk space required to store the images exceeded that used by "Star Wars: Episode I."
Putting it All Together
Once the basics have been blocked out using the simulation and particle systems, all of the details are planned out. This is what makes the difference! Without all these exact details in place, the shot would look realistic but would still feel wrong, causing you to realize intuitively that the scene was using CG effects. You have probably seen this in other movies -- maybe the shadows were not quite right, or the coloration of the CG portion didn’t exactly match the real part.
It is amazing to look at all the different layers that ILM created for each shot:
In the all-CG shots, nothing else needs to be done. But many of the shots combine CG and live action. Most of the live action was filmed in a water tank with a blue screen surrounding it, on Stage 16 at Warner Bros. Studios (the largest indoor tank in the world). The blue screen has a grid of red dots in white squares, called targets on it, which are used to align the virtual camera’s viewpoint with the live camera. Most of the targets had to be removed manually frame by frame, making for a long and arduous process.
The CG elements are color-corrected and blended with the live footage. Some scenes require a little CG sleight of hand -- adding CG waves or swells -- to mask unwanted live elements. Additional mist and foam patterns are composited onto each frame. Finally, all of the live and CG elements are composited together to form the complete shot.
The shots are edited together and the film is released. You go to the theater, or you rent the VHS or DVD release of "The Perfect Storm," and you are stunned by the most amazing and realistic weather effects ever created, unable to tell what is real and what is computer-generated. And that is the greatest compliment ILM could ask for.
For more information on ILM, "The Perfect Storm," visual effects and particle systems, check out these interesting links: