is the peculiar “winner’s curse” phenomenon, in which top results in small initial
studies don’t always pan out in later studies. This is a close cousin of the “Sports
Illustrated curse,” in which star rookies
featured on the magazine’s cover end up
with a crash-and-burn second season.
There’s a simple statistical explanation, says epidemiologist Teri Manolio of
the National Human Genome Research
Institute of the National Institutes of
Health in Bethesda, Md. Researchers
will naturally try to replicate the most
extreme top-scoring results in an initial
study. But these huge effects probably
owe their super-high ranking in part to
a true effect and in part to sheer random
luck. Small follow-up studies — designed
to look for these big effects — will miss the
more subtle, true effects, Manolio says.
Thus initial studies may appear
flawed, even if they aren’t. The solutions — increasing sample sizes and recognizing that extreme initial results are
likely overinflated — are beginning to take
hold. “It’s happening,” Manolio says, “but
it’s happening slowly.”
The trouble with ancestry
Complications from race and ancestry
can also play a role in genome-wide association studies. That’s because people with
European, Asian and African ancestries
have different genetic patterns. These
patterns can be misleading. “There is a big
debate about this in the genetics community,” says geneticist Eric Jorgenson of the
University of California, San Francisco.
“Does race matter? Individual genotypes
are what matter, but at the same time, race
is correlated with genotype.”
Take a simplified example: Suppose
most people of European ancestry in a
sample had blue eyes and also happened
to have disease X, while most people of
Asian ancestry were brown-eyed and disease-free. A naïve analysis might conclude
that the blue-eyes SNP is responsible for
disease X, even if eye color and disease are
That is, the methods are likely to nab
the wrong SNP suspects, simply because
these innocent SNPs tend to show up in
the same situations as truly guilty SNPs.
This genetic-mixing issue shows up in
other kinds of studies, too. But it’s a particular problem for studies of the entire
genome because of the huge number of
ancestry-related SNPs being tested.
Traditionally, researchers have
addressed this genetic-mixing problem
largely by balancing the number of study
volunteers belonging to different racial
groups. But this strategy goes only so far,
Jorgenson says. Genetic heritage is more
complicated than skin color or grandparents’ birthplace, and the ancestral variation in the gene pool can’t be conveyed
with a simple check-off survey box.
Some nifty statistical tricks, however,
can help researchers spot and fix this
problem in their analyses, Jorgenson
says. For example, one method comes up
with a mathematical summary of every
volunteer’s personal genetic ancestry and
incorporates that into the analysis. This
effectively allows researchers to “strip”
each volunteer of his or her genetic ancestry and simply investigate the important
genetic patterns that are left over.
Ancestry can cause other problems.
Waves of migration out of Africa — starting about 60,000 years ago — were led by a
relatively small number of people, resulting in a narrower gene pool in the new
communities. Plus, as populations spread
throughout Europe, Asia and the Americas, settlers faced limited mating choices,
further reducing genetic variability.
These conditions — founder effects
and bottleneck populations — meant that
new emigrant groups had less genetic
diversity than the original African population. Over time, these effects became
more pronounced. People with recent
African ancestry now have more variability across their genome than do people
with European and Asian ancestry.
Problems arise when people with different genetic ancestries are included
in one study, Jorgenson says. Scanning
a group with greater genetic variability
requires more refined tools. “If you’re
applying genome-wide association studies
to a bottleneck population with less variability, you can use a wider-tooth comb,”
he says. “Populations with more variability need a finer-tooth comb.” Current
methods may miss disease-linked SNPs in
African-Americans, especially if the SNPs
are associated with rare gene variants.
The ideal solution would be to sequence
every letter of volunteers’ genomes — thus
providing the finest-toothed comb possible. Cost and logistics are still prohibitive for this approach, however. Still,
the more SNPs that manufacturers can
squeeze onto their SNP chips, the more
likely that important SNPs will be caught,
Jorgenson says. And some manufacturers
are already starting to design chips that
incorporate sets of SNPs suitable for different genetic ancestries.
Genome-wide association studies
might indeed prove to be a bonanza for
modern gene hunters. But in all the excitement, researchers shouldn’t forget the
value of good old-fashioned study design,
Khoury warns. “I think people are being
lulled into a zone of comfort,” he says, as
some researchers rely on million-SNP
chips, large sample sizes and multiple replication studies to cover up study flaws.
And there’s still a nagging question:
After you’ve bagged your gene, what do
you do? “To me, this is the biggest stumbling block,” Khoury says. “You still have
to work out the biology of that hit.... That’s
actually where the hard work begins.”
It’s clear that clinical applications
are still years away, Manolio says. Some
companies are starting to sell personalized genetic tests based on results from
genome-wide association studies. But
researchers hardly know what the study
results mean themselves; any immediate
translation into personalized medicine
will naturally be problematic.
“There is a lot of missing heritability
in our results right now,” Boehnke says.
“If your goal [with these studies] is personalized medicine and developing your
own personal genetic report card, we’re
definitely not there yet. I don’t know
whether we ever will be.” s
Regina Nuzzo is a freelance writer based
in Washington, D.C.
s Thomas a. pearson and Teri a. Manolio.
“How to interpret a genome-wide association study.” JAMA, 19 March 2008.