In the Olympics, you'll see everything from tiny gymnasts all the way to brawny, barrel-chested weightlifters; these competitors run the gamut of body types. So, the first factor that can affect athletes' prowess is their basic anatomy. Anthropometrics (the study of the human body's measurements) can help us study how this plays out.
Anthropometric factors can include an athlete's height, weight, body mass index and muscle mass. As you've no doubt noticed, different body types may lend themselves to different sports. A large, tall man, for example, could be ideal for sumo. Solid, although shorter, may mean you've got a wrestler in the making. Tall, but slender, could suggest he's ideal for volleyball. Small and thin might make for a good marathoner.
As for female athletes, shorter and skinnier women often appear on the gymnast mats. Shorter yet stockier women can be great on the wrestling mats. Tall, sinuous females may make for good swimmers, while taller and stockier females may excel at athletics, such as the hammer throw.
Now, of course, it's not a one-size-fits-all equation -- plenty of Olympians break the mold, and morphology is certain not a cut-and-dry factor -- but choosing a sport that complements your basic anatomy never hurts.