NIH Gordon Lecturer Highlights Enormous Waste In Epidemiologic and
Biomedical Research And Suggests 7 Preventive Measures
The topic is not a new
one and has been tracked for some time by Michael Bracken,
professor of epidemiology at the Yale School of Public Health.
Speaking as the Robert S Gordon Jr Lecturer on “Inefficiency
and Waste in Biomedical Research: How Prevalent Is It, What Are Its
Causes, and How Is It Prevented? Bracken described ways in which
research work is wasted. For example, 50% of research is never
published, another 50% is flawed, and a final 50% may be redundant or
unnecessary, according to one estimate. Thus, only 12.5% is not
wasted.
The
Gordon Lecture at NIH has been given for over 20 years as a tribute to
Robert S Gordon for his outstanding contributions to epidemiology and
his distinguished service to NIH. A list of previous lecturers is
provided in an accompanying article in this issue.
False Positives
One
important finding about research waste relates to false positives.
Early research in an area is often later found to be wrong or
exaggerated. “Repeat studies tend to no effect,” Bracken told the NIH
audience, “and the risk of being misled is very high.”
Bracken discussed the topic of false positive results in many
observational studies whose results were overturned in clinical
trials, and said that implementing larger studies did not
automatically remove bias. He reflected on the angst these false
positives have created in the epidemiology community and even offered
the opinion that the much discussed and criticized article several
years ago by Gary Taubes in Science entitled “Epidemiology
Faces Its Limits” actually rendered a service to the profession.
Wasteful Replication
An
important contributor to the waste in research is the work done to
replicate previous findings. According to Bracken, such replication
reaches a point at which the replication is no longer necessary
because the evidence is in. Yet investigators continue to repeat
studies unnecessarily. One potential solution to this problem is to
track research prospectively so that the accumulating body of evidence
in any area becomes more apparent.
SIDS Example
A
striking example of the potential public health impact of excessive
research and the delay it causes is the one related to sudden infant
death syndrome. By some estimates, many follow up studies about the
baby sleeping position were redundant and delayed public health
recommendations. Had the evidence been properly tallied, an estimated
10,000 babies in the United Kingdom, and another 50,000 in Europe,
Australia, and the US could have been saved.
Preventive Measures
Among the measures advocated by Bracken to help prevent wasted
research are the following:
1.
Reduce the play of chance by designing studies with more stringent
alpha and beta values in much larger studies. Bracken recommends using
alpha errors of 0.01 or 0.005 rather than 0.05 and a beta error of
0.10 not 0.20
2.
Pay more attention to threats from multiple comparisons. Bracken
reminded the audience that when Bradford Hill was describing
standards for relying on research findings, there was often only one
hypothesis per study being tested. Today, it is not uncommon for 200
associations to be examined. “A different world” said Bracken.
3.
Strive harder to avoid bias, which subverts all research.
4.
Comply with appropriate standards and guidelines.
5.
Systematically review extant evidence in a published protocol before
starting new research.
6.
Follow the protocol outcomes when reporting findings or declare the
differences.
7.
Systematically update new findings in the context of all available
research
To
listen to the videocast of the Gordon Lecture, visit:
https://tinyurl.com/gnfssar ■
|