|
"Getting Risk Right": An Interview with Geoffrey Kabat
Seeking To Learn
From Epidemiology At Its Best And Worst
This month The Epidemiology Monitor re-interviews
Geoffrey Kabat, cancer epidemiologist at Albert Einstein College
of Medicine, following publication of his latest book—Getting Risk
Right, a thoughtful examination of the scientific process involved in
identifying and assessing health risks. The Monitor interviewed Kabat
earlier in 2009 when he published his first book entitled “Hyping
Health Risks”. (See https://tinyurl.com/huv6vpb
)
Kabat
has long been concerned with the challenges facing epidemiologists in
doing solid research and having these results represented accurately
in the public domain. Beginning with the basic question “Why do
things that are unlikely to harm us get the most attention?”, Kabat
makes use of detailed case studies to explore the factors that
contribute to epidemiology both at its best and its worst. The public,
the media, and the scientists all appear to contribute to the problem
Kabat highlights. His analysis should be of interest to those
epidemiologists in the public health community hoping to contribute to
meaningful scientific advances.
EM:
The over-hyping of health risks is something you have been studying
closely for some time and explored in your previous book Hyping
Health Risks. What specifically motivated you to write this new
book and how does it pick up where the last one left off?
GK: The first
book, Hyping Health Risks, took a critical look at 4 prominent
environmental exposures that received a great deal of attention in the
1980s and 90s
What I tried to show, and explain to myself, was how in
each case a scientific question in the area of public health could get
distorted and inflated as a result of limited or flawed epidemiologic
studies combined with what was made of published results by the media,
advocacy groups, regulators, and scientists themselves.
As I was finishing the first book, there were other questions, like
cell phones and particulate air pollution, that caught my interest.
And my editor suggested other topics, such as BPA. But my strongest
feeling, as I thought about another book, was that I didn’t want to
repeat myself. To spend years writing a new book, I had to find an
animating idea that felt new and exciting.
As I grappled with new topics, it came to me that I wanted to contrast
instances where risks were hyped with examples of what science at its
best can achieve in the area of public health. And the HPV story came
to mind because I knew people at Einstein who had worked on HPV since
the early 1980s and because I noticed that epidemiologists got a gleam
in their eye when we talked about this success story. What struck me,
above all, is that the general public hears little about the process
that led to the development of a vaccine – how long it took and how
many twists and turns there were in the path leading to the
identification of the specific types of HPV that cause cancer and then
to the development of a vaccine -- and that achievements like this
tend to be taken for granted. On the other hand, there is enormous
attention rooted in fear that is directed at potential risks that
often turn out to be of little or no consequence. So that idea – of
the contrast between instances where scientific research relating to
health risks gets enormous attention but fails to uncover important
new knowledge or make progress, and those where unimaginable progress
is made – provided the central tension of the book. These represent
two extremes. Most research lies somewhere in the middle. But I felt
that we could learn something by contrasting the two extremes.
EM:
You devote roughly the first third of the book to discussing issues of
bias and methodologic and disciplinary pitfalls. Can you summarize
the thrust of this introductory section?
GK: The first third of the book lays the groundwork for the remainder
of the book – the case studies that are the heart of the book. I
start with the question, “Why do things that are unlikely to harm us
get the most attention?” and refer to the large number of contested
issues involving things that might be adversely affecting our health,
including vaccines, GMOs, pesticides and other chemicals in the
environment, cell phones, salt, obesity, smokeless tobacco,
e-cigarettes, “fracking,” etc. But rarely does the quality of the
studies enter into the public discussion. You would never know from
the media that there is a lively discussion going on the quality of
scientific research in the biomedical field. Specifically, there is
increasing recognition within the scientific community of what has
been referred to as a crisis characterized by fierce competition for
funding and professional advancement, a lack of reproducibility of
published papers, and a lack of transparency.
EM:
What was your main goal for this introductory section of the book?
GK: My goal in the first 3 chapters was to provide a framework for
understanding the many factors that can influence published findings
and how they get reported to the public. I cover methodological
biases inherent in observational studies, as epitomized by the work of
John Ioannidis and colleagues, cognitive biases such as those
described by Daniel Kahneman, professional and political biases, and
“bandwagon effects.” These different biases can interact and reinforce
each other. My intention in this section was to provide a description
of the landscape in which research is conducted and the kinds of
pitfalls it is subject to.
EM:
Do you feel these pitfalls apply equally across all areas of public
health research?
GK: No, a crucial point, which rarely gets attention, is that all
areas are not equal when it comes to the reproducibility of findings.
This is shown nicely in a 2012 analysis by Tsilidis et al. that showed
that among studies using biomarkers to assess associations with
cancer, infectious agents (HPV, H pylori, HBV) had robust
findings. In contrast, studies examining IGF/insulin and markers of
inflammation had considerably lower reproducibility. Finally, studies
of diet and environmental exposures had very poor reproducibility.
EM:
Having set the stage, can you encapsulate the contrast between your
two sets of cases studies?
GK: In the first two case studies I retell the story of roughly twenty
years of research devoted to the issues of cell phones and brain
cancer, and the possible health effects of exposure to
“endocrine-disrupting chemicals” in the environment. In my view, both
questions have their origin in dramatic findings that galvanized the
attention of scientists but were misleading.
EM:
What specifically were the dramatic findings that led scientists down
the wrong path for so many years?
GK: The cell phone question arose due to a man in St. Petersburg,
Florida, who, after his wife died from brain cancer, brought a lawsuit
against a wireless company and went on Larry King Live. The
endocrine disruption hypothesis came about in large part in the early
1990s due to three observations: incidents in which exposure to
industrial runoff affected the sex of alligators and other wildlife;
the DES experience in the middle of the last century; and the alleged
decline in sperm counts. None of these findings turned out to be
relevant to the general population, as I explain in detail. But they
led to a line of hundreds of research studies, which sometimes
produced “suggestive” results and kept the bandwagon going.
EM:
What were the main factors that contributed to the propagation and
continued study of these misleading hypotheses?
GK: Importantly, the results of studies generated concern in the
public and led to regulatory attention. I think one must acknowledge
that the fact that these two issues were so much in the public eye and
caused so much concern helped to perpetuate a line of research that
has failed to produce solid evidence in favor of either hypothesis.
Being in the spotlight helped to keep what was a weak and
poorly-justified hypothesis alive and consume scarce funding.
For
me, when I had a back-and-forth with the eminent expert on male
reproductive function, Richard Sharpe, who is one of the originators
of the endocrine disruption hypothesis, he put the issue in a way that
sheds a glaring light on how a field can go wrong. “In retrospect, I
consider that circumstances helped me because I ended up disproving my
own hypothesis/ideas early on in the ED saga. Plus, I was lucky that
the question that drove me was ‘what causes these disorders,’ not ‘how
do EDCs cause these disorders?’ Such a simple difference, but it
takes your thought processes in a very different direction.”
EM:
Were there clues at the time that the original hypotheses were poorly
justified? Would you suggest that these situations could be avoided if
hypotheses are more solidly justified before being pursued?
GK: That’s an interesting question. I think, to some extent, the
degree of distortion could have been reduced. Scientists are supposed
to be skeptical and to be critical of the evidence, and, I think that
regarding cell phones, and the earlier question of EMF, there was a
tendency to have a narrow focus on weak epidemiologic studies and
difficult-to-interpret in vitro studies. What I think should have
gotten more attention was the nature of the type of energy involved
(i.e., microwaves and extremely low-frequency electromagnetic fields)
and the plausibility that these types of very weak energy could be
inducing biological effects. I’m not saying that this consideration
should have been determinative, but it should have been taken into
account. Certainly, as time passed and more robust studies were done
(particularly, involving whole animals exposed to radiofrequency
emissions), there was a tendency for this strong null evidence to not
receive the weight it deserved and to latch on to weak findings from a
minority of epidemiologic studies. So, to an extent, yes we could
have been more skeptical from the outset, but, of course, the picture
becomes clearer as more high-quality studies are done. Similar
considerations could have helped put the endocrine disruption
hypothesis in a critical perspective.
EM:
And what about the second set of case studies?
In contrast, the
second set of case studies tells of work that was carried out over
decades to understand 1) a mysterious disease
involving irreversible kidney
damage in the Balkans and 2) the etiology of cervical cancer. Both
questions were difficult and required considering multiple hypotheses,
excluding explanations that did not fit with the evidence, confirming
findings, and refining one’s hypothesis. Over time, there were false
leads, methodological and technical obstacles that had to be overcome,
and disputes between different disciplines. But over time, scientists
in different parts of the world collaborated and overcame obstacles
and confirmed the links in a chain of causation. In both cases the
work led to new knowledge but also to undreamed of consequences for
public health.
EM:
What were those consequences?
GK: In the
first case, we now know that an herb used in traditional medicine in
major cultures going back two thousand years (Aristolochia)
causes irreparable kidney damage and cancer of the upper urothelial
tract, and the mechanism of cancer induction is now known (i.e., it is
a highly specific signature mutation in TP53). In the second case,
work over more than 30 years has led to the development of vaccines
that protect against HPV infection and have the potential to virtually
eliminate cervical cancer, with over half a million new cases and over
a quarter of a million deaths each year, mostly in countries in south
Asia and Africa.
EM:
So what do you believe is the main contrast between to the two sets of
studies?
GK: The stories highlight how science that tackles an important
question typically only makes progress by dint of painstaking work by
different groups of scientists over time. As Harald zur Hausen has
said, there was “no eureka moment.” Furthermore, because forging the
links in the chain is painstaking and unglamorous, it is not
newsworthy. Rather than the reports of the latest threat or
breakthrough, we should give greater attention to the hard work of
science that, if it pursues an important problem, can make
life-changing advances.
EM:
It seems a key distinction between these sets of case studies is
whether or not the media got involved early on and played a role in
influencing the research before the scientific process had enough time
to play out and reach solid conclusions. Would you say the media or
the scientists shoulder more of the blame in these situations?
G.K. The media is the media. What sells is what is unexpected,
startling, and novel. That is not going to change. There is actually
a good deal of high-quality reporting, if you know where to look for
it, but that is not going to reach the vast majority of the
population. So, I don’t think one can have great expectations about
the mass media changing. As a scientist, I’m more concerned about the
quality of scientific studies and the seeking out of media coverage
for results that really have very little claim on the public’s
attention. There is a pretense that the public needs to know about
studies, but often the results really are so uncertain that one has to
question what use they are to anyone. So, I come down on the side of
feeling that we need much higher standards for what gets published and
to stop utilizing the public to boost the stature of our work.
EM:
Today more than ever, scientists are under tremendous pressure to
publish or perish. Do you think systemic reforms are needed within
academia in order to balance such a desire for higher standards in
publishing with the increasingly competitive modern academic
environment?
GK: I definitely think that systematic reforms are in order. These
issues have been widely discussed, for example, in an informal survey
of scientists conducted by Vox
https://tinyurl.com/joyw7xx , and, most recently in a paper
entitled “A manifesto for reproducible science.”
https://tinyurl.com/jcya7f7
(The Epidemiology Monitor covered the results of the Vox.com survey
mentioned above in detail in our September 2016 issue:
https://tinyurl.com/z6sfj5s)
EM:
What is the key lesson that can be taken from these case studies about
how to best investigate and report on scientific results?
GK: Having in mind models of what science can achieve at its best can
provide a standard by which to judge the extravagant claims based on
flimsy evidence, which get so much attention.
EM:
How does having a standard of what science can achieve actually help
us to determine that some claims are extravagant and others are
reasonable to pursue? After all, isn’t determining what is flimsy
evidence from what is promising evidence the real challenge here?
That is not always easy to determine in the early days of research
findings.
GK: You are right, in the sense that the case studies that I recount
only achieve maximum clarity in retrospect. In the 1970s virologists
were dismissive of Harald zur Hausen’s hypothesis that papilloma
viruses might be the cause of cervical cancer. And Richard Sharpe put
forward the idea in the early 1990s that “living in a sea of
estrogens” might explain abnormal reproductive development. All one
can do is to keep in mind alternative hypotheses and not develop
tunnel vision, blocking out explanations that don’t fit with one’s
hypothesis. As I emphasize in the book, quoting the biophysicist John
Platt, keeping in mind alternative hypotheses is the best way to
protect against selecting data that appear to support a favored
hypothesis.
EM:
You describe a litany of factors contributing to misrepresentation of
health risks. Is there any one factor you believe is most central to
the problem? If not one, which do you feel are the most detrimental to
promoting good science?
GK: Two things
appear to me to be of paramount importance. First, we have to avoid
becoming wedded to a particular hypothesis, even if it is in vogue and
provides a source of funding. Once we block out alternative and
possibly more promising hypotheses, we become prisoners of
confirmation bias. Second, the politicization of science is a serious
danger. By politicization, I mean allowing an ideological stance or
policy considerations to influence one’s interpretation of the
evidence on a particular question. We need to continuously strive to
distinguish good -- that is, reproducible -- science from politics and
from policy. Jeremy Berg, the new editor-in-chief of Science
magazine, has recently made this point.
https://tinyurl.com/hm8yrln
■
|