How Rotoscoping Works

Artist and tech enthusiast Max Fleischer was the art editor for Popular Science Monthly when he came up with the idea of rotoscoping, with the aim of creating more fluid and realistic motion in animated cartoons. He enlisted the help of his many talented brothers (Dave, Joe, Lou and Charlie) to develop and test what would become a rotoscope device. Max filed a patent for the process and the related mechanism in 1915, which was granted in 1917.

The rotoscoping process required starting with film footage. For the Fleischers' first try, they went to the roof of an apartment building, with a hand-crank projector they had converted into a film camera, and filmed over a minute of test footage of Dave in a clown costume (sewn by their mother). Once that footage was made and developed, the rotoscope mechanism they had pieced together was used to project the film one frame at a time through a glass panel on an art table. Max would place tracing paper over the other side of the glass panel and trace over the still image. When done, he would move the film to the next frame and start a new drawing over the next image. The patent mentioned a possible mechanism to allow the artist to move to the next frame by pulling a cord from his current position.

Once all the images were drawn, they needed to be photographed one at a time. For their clown footage test, Max used the projector as a camera once again, this time exposing each drawn image to one frame of film by manually removing and replacing a lens cap for just the right amount of time, then incrementing the film. They had the film developed, played it back using the projector and found that the process had worked. And the animated clown, who would later be dubbed Koko, was born.

Max went on to animate, and his brother Dave to direct, many successful cartoons, starting around 1919 with the "Out of the Inkwell" series featuring Koko the Clown. Their later works included iconic characters Betty Boop and Popeye in the 1930s and the famously realistic (and expensive to produce) "Superman" short features in the 1940s. Rotoscoping was used to varying degrees in all of them to produce life-like character motion while still allowing the creativity and exaggeration that animation makes possible.

Three Betty Boop cartoons ("Minnie the Moocher," "The Old Man of the Mountain" and "Snow-White") even incorporated rotoscoped footage of Cab Calloway as different characters. The first two also opened with live-action film of Cab Calloway and his orchestra, and "Minnie the Moocher" includes the earliest known footage of Cab Calloway performing [source:Fleischer Studios].

Fleischer also employed the technique in "Gulliver's Travels," which was released in 1939, making it the second full feature length animation produced in the U.S.

The first American full feature length animation was Disney's "Snow White and the Seven Dwarfs." Before it was animated, reference footage was taken of actors performing on rudimentary studio sets in costume, including young dancer Marjorie Celeste Belcher (later known as Marge Champion) as Snow White, dancer Louis Hightower as the Prince, comics Eddie Collins and Billy House as Dopey and Doc and actor Don Brodie as the disguised queen. They then rotoscoped key frames at the beginning and end of character actions and animators elaborated the characters and scenery and created all the in between shots on transparent animation cels.

The technique was also used in "Peter Pan" with footage of several actors, including Hans Conried as both Mr. Darling and Captain Hook, Buddy Ebsen as a pirate, dancer Roland Dupree as Peter Pan and Margaret Kerry as Tinker Bell. It was employed in some other Disney films, as well.

Although it was put to good use, rotoscoping wasn't always well received. As we mentioned earlier, some considered it a cheat or shortcut, and others complained that characters modeled on live actors differed in look and movement from other moving characters, such as traditionally animated animals. Disney even tried to hide the use of rotoscoping in "Snow White and the Seven Dwarfs" to an extent. At the premier, they seated Belcher in the balcony away from the crowd and told her not to discuss her role in the film. She apparently didn't know that the footage taken of her was rotoscoped, and learned about the process decades later when she visited a Disney exhibit about the movie. Whatever techniques were employed to make it, the film was undeniably a masterpiece of animation and has become a classic.

And it was far from the last cartoon to use the technique. Some have used much heavier rotoscoping, resulting in movies that look like film footage that has been painted. The "Lucy in the Sky with Diamonds" segment of the Beatles' "Yellow Submarine" (1968) included some obvious rotoscoping. And animator Ralph Bakshi employed the technique in several films including "Wizards" (1977), "The Lord of the Rings" (1978), "American Pop" (1981), "Fire and Ice" (1983) and "Cool World" (1992). Columbia Pictures reportedly downplayed the use of rotoscoping in "American Pop" upon its release.

A-Ha's video for their song "Take On Me" in 1985 also used rotoscoping to memorable effect, alternately showing people in live-action footage and sketchy comic book animation. That same year, it was used in INXS's "What You Need" video to add animation and color effects.

More recently, Richard Linklater's movies "Waking Life" (2001) and "A Scanner Darkly" (2006) were both filmed with live actors and turned into animations using a process called interpolated rotoscoping. Many animators worked on the films, but to speed the process and create a smooth result, software created by MIT grad student Bob Sabiston was used that interpolated the images for frames between rotoscoped key frames.

The process is also used by animators to learn how to animate motion from scratch. But it's not just an animation tool. Rotoscoping is also used in live-action films.

Rotoscoping isn't just for making animated films. It was used to add special effects to Alfred Hitchcock's 1963 film "The Birds," to place all of the glowing lightsaber blades into the original "Star Wars" trilogy films and to add effects to many other movies.

In the days of movies shot and edited on actual film, rotoscoping was sometimes used to paint special effects onto animation cells over live-action footage, but it was also used to create mattes (or masks) to allow filmmakers to combine elements from one filmed scene with elements of another completely different filmed scene. A filmmaker might want to superimpose footage of a person on a soundstage over footage of a location background, such as the ocean or outer space or in front of an explosion. This combining of images or effects that were not initially shot together into one film frame is called compositing.

This sort of rotoscoping was an incredibly time consuming task that, like its animation equivalent, involved projecting each frame of film onto a glass plate and manually tracing items one frame at a time. An animator or special effects artist would trace any elements that needed to be isolated onto a transparent cell. The traced elements would be filled in with paint to create mattes that could be placed on top of another frame (such as footage of the desired background) to effectively block out that area on the frame and leave a spot for the foreground effect.

The process would often involve creating multiple reels of film, for example the original foreground and background image shots, film of the blacked out mattes and film of a negative version of the mattes with a blacked out background. A new reel of film would be exposed multiple times to combine all of the elements together. The background image would be exposed to the film with the black matte in front of it, and the foreground element would be exposed to the same frame of film with the negative version of the matte in front of it to expose the foreground element to the film in the spot that was left for it during the first exposure. In this way, the rotoscoped mattes were used to combine the foreground element and the background onto each frame of film.

It took a lot of work to get it right. Typical 35 millimeter film is projected at a rate of 24 frames per second, which translates to 1,440 frames per minute, so mattes would have to be painted for dozens, hundreds or even thousands of frames. In a single action shot, the moving items change position a little in each subsequent frame (a matte that changes position and shape from frame to frame is called a traveling matte). It can get even more complicated when multiple things are interacting and overlapping on screen, requiring multiple mattes per frame.

Not only were a lot of frames involved, but the outlines and painted mattes for each one needed to be meticulous, the lighting had to match up (or be matched up later through color correction), and all the physical elements (the mattes, film and equipment) had to be lined up exactly or the results would move oddly or otherwise look out of place.

But when done well, rotoscoping could be used to show things on film that would have been difficult or impossible to shoot in real life. And it is also for more mundane but necessary purposes, like removing cables, microphones, staging markers or other items left in a shot, intentionally or not, which might throw the audience out of the moment while watching the film.

The technique is still used to this day to an extent, but thankfully, more modern methods for rotoscoping, creating mattes and compositing elements came into being in the computer age.

The existence of ever smaller, cheaper and more powerful computers has brought us into a great new era in filmmaking. CGI (computer-generated imagery) makes it possible to put nearly anything on screen and make it look at least somewhat believable. And despite the existence of other special effects techniques (some of which we'll talk about in the next section), new graphics software has also brought us new ways to rotoscope that are far less time consuming than they were with physical film.

Some of the concepts are the same, but the media and tools are mostly digital (including the film, the paint and the software). Rather than a physical animation cel overlaid onto a glass table displaying projected film, graphics software often allows you to work in virtual layers, where one layer is the digitized film image, and the others contain whatever animation or effects you want to place in each frame. Then rather than photographing the final product, you save or export it as a new digital file.

Graphics software applications let you do many of the same things you could do with physical film media and equipment. You can paint over each frame of a digitized video using a mouse, trackpad or graphics tablet to create a traditional rotoscoped animation, where you can keep the regular film and animation together or remove the film leaving only the animation. Or you can create complex mattes to composite filmed or computer generated objects into each frame.

Mattes can be created on individual frames using selection and paint tools on the software's tool bar. You can outline using splines (lines or curves that can be manipulated by dragging various points) to make it easier to slightly alter the matte on the next image rather than completely redrawing a new one. Some things can even be handled over multiple frames automatically by the software, which might be able to take the information in beginning and ending keyframes and use that to figure out what information needs to be in the frames in between and generate them. A lot of software can track the position of items from frame to frame to know where to place any composited items that have been added to prior frames. The newer software has become increasingly good at recognizing and properly outlining and masking shapes over multiple frames without animators having to trace exact lines around each and every one, which is a giant time saver.

Modern graphics software also allows you to easily change colors, to morph items from one thing into another, to blur and soften edges with a few key strokes, and to clone (or copy) parts of the screen image from one place to another. This latter tool is useful in things like removing cables and stray boom mics, since you can place other parts of the background (a wall or the sky) over them to make it look like they were never there.

Early software tools that could be used for rotoscoping and similar techniques began appearing in the 1990s, including Colorburst, Commotion and Matador. More modern software that can be used for various levels of rotoscoping includes Adobe Flash, Adobe Photoshop, Adobe After Effects, Imagineer Systems' Mocha, Silhouette, Autodesk's Flame and Smoke, Blackmagic's Fusion and Foundry's Nuke, Ocula and Mari, among others. Some of these are expensive and require very powerful and likewise expensive computers and other equipment. But some, like Adobe Flash and After Effects, aren't outside the realm of possibility for a home hobbyist or independent filmmaker.

There are some notable alternatives to rotoscoping for compositing special effects into films, and one of them is color keying, which is sometimes called chroma keying, and is often referred to by the color of the background on which the actors are filmed (such as blue screen or green screen). Blue screen color keying has been around since 1940 when it was used in the film "The Thief of Bagdad."

Another version of the same idea was used exclusively by Disney, where they filmed actors on a white background and lit them with sodium vapor lamps. This proprietary method was used to film and composite effects in "The Parent Trap," "The Absentminded Professor" and "Mary Poppins."

Green overtook blue as most commonly used when post production work went digital, since green is easier to light and digital cameras are better able to pick up detail in front of that color. Whatever the color involved, color keying is used to create traveling mattes more automatically by filming actors and other foreground items in front of a single colored backdrop and then using film or digital processing to remove that color (or everything that isn't that color) to produce mattes for background and foreground elements. It eliminated the need to manually outline and matte elements frame by frame and made the process much easier, although it comes with problems of its own. For instance, you have to make sure your actors aren't wearing anything that's the color of the backdrop. Plus most things are multicolored, so faint traces of those colors might be removed from your foreground subjects, requiring color correction.

And it isn't foolproof. Rotoscoping is sometimes used to fix mistakes on set, such as someone or something you are filming moving outside of the color screen area. If someone accidentally waves an arm out of the area, rotoscoping can be used to make a traveling matte of the part that isn't in front of the color screen to composite it into the film properly.

Another comparable and even newer technique is motion capture, sometimes referred to as mocap, and more recently dubbed performance capture. It's akin to both rotoscoping and color keying in that it's used to composite new moving elements (actors in particular) into scenes, and like the rotoscoping of old, it is often used to lend characters realistic motion and appearance. But mocap is a thing of the digital age that's bringing us much more realistic graphics and motion than anything that came before.

Originally developed to study motion, the process has been used in film and video games since the mid-1990s. It involves digitally capturing the motion of a live actor to create a 3-D computer model of the body and its motions. In its earlier incarnations, an actor would don a bodysuit covered with reflective markers or sensors and then perform alone on a sound stage surrounded by cameras. The data from the session would be fed into computer software that would generate a moving 3-D model of the performer and performance. Facial expressions, costumes and other details would be added by animators and other special effects professionals via graphics software in post-production. The technique was used to model some shots of Batman in "Batman Forever" (1995), crowd scenes in "Titanic" (1996), Jar Jar Binks in "Star Wars Episode I: The Phantom Menace" (1999) and Gollum (performed by motion capture virtuoso Andy Serkis) in "The Fellowship of the Rings" (2002).

But every mocap blockbuster seems to bring innovations that make the process even better. One major improvement was facial performance capture, which was used for the titular character in Peter Jackson's "King Kong" (2005), Davy Jones in "Pirates of the Caribbean: Dead Man's Chest" (2006) and all the Na'Vi in James Cameron's "Avatar" (2009). This was at first achieved with the use of sensors or reflective markers on the face, but for "Avatar," the actors wore form fitting helmets that had cameras in front of the actor's faces (attached via a thin arm) and had dots painted onto their faces. The camera captured every facial change, including lip and eye movements, and fed the data into software. The data was used to imbue each computer generated character with as much of the actor's real performance as possible, rather than relying entirely on post production animation using reference footage. The actors still had to perform on a set (wearing the helmets and bodysuits covered with infrared LED sensors and being filmed by Weta's "The Volume" motion capture camera system), but they were able to interact with each other.

Another big breakthrough came with "Pirates of the Caribbean: Dead Man's Chest" (2006) where Bill Nighy was able to do his entire performance on set with other actors (albeit in a marker-covered body suit and with facial markers). Effects house Industrial Light and Magic was able to add all of the CGI elements of Davy Jones later using the data that was captured on-set. In "Rise of the Planet of the Apes" (2011) and "Dawn of the Planet of the Apes" (2014) (also both starring Andy Serkis in mocap roles), performance capture was done outdoors on location using bodysuits with infrared LEDs and the helmet cameras.

And yet another incredible innovation was first used on "Avatar." A virtual camera with a monitor allowed James Cameron to view the actors' as their computer-generated characters along with the CG setting in real-time while the performances were happening. The technology was also used in Peter Jackson's "The Hobbit: An Unexpected Journey" (2012) and its sequels. The final, much more realistic results still come later after many, many man-hours of post-production, but it can be used to better visualize how a shot will really look and give direction accordingly. Plus the immediate on-set graphics are bound to get better with time.

Motion capture has also been used to create fully animated films, including "Final Fantasy: The Spirits Within" (2001), Robert Zemeckis's "The Polar Express" (2004) and "Beowulf" (2007). The former two suffered from complaints that the characters fell into "Uncanny Valley" territory, but those sorts of complaints seem to be diminishing with newer more realistic animations.

Both rotoscoping and motion capture can allow for an actor's performance to come through in action rather than just voice on a graphically rendered character. But performance capture has the potential to turn an actor into any sort of creature the story requires, without the need for hours in makeup and costuming, and with an entirely realistic appearance and movement.

Still, it's an expensive process, and isn't always necessary to tell a good story. Although just about every film has special effects we don't notice, like the stripped out boom mic or day for night shots or more snow on the ground that is generally present during a spring shoot, they don't all have or need entirely CG characters.

All of these techniques give animators and other filmmakers ways to produce creative and stunning visuals. In live-action film, they allow for scenes that would be expensive, difficult, dangerous or impossible to film in a real-world setting. And new and improved digital tools are making it possible to create more realistic effects than ever, faster and more cheaply than before.

Despite all the new shiny toys, rotoscoping still exists as a viable tool in the arsenal of the filmmaker or graphic artist. It can be adopted for stylistic purposes to deliver a certain look and feel, as with Richard Linklater's "Waking Life" and "A Scanner Darkly." But it's also always on standby, ready to be used for filming flubs and other post-production compositing needs.