|
Epi Wit & Wisdom Articles
Translating Epi Data Into Public
Policy is Subject of Hopkins Symposium
Focus is on Lessons Learned From
Experience
As the story goes, Senator
Muskie was getting so frustrated with hearing testimony from
scientists saying “...on the one hand...and on the other hand...” that
he quipped what he needed was a one-handed scientist! The challenges
and successes as well as the frustrations encountered by scientists
and policy makers in translating evidence into public policy were the
subjects of a one day workshop on July 29, 1998 in Washington, DC
sponsored by The Johns Hopkins School of Public Health and the
American College of Preventive Medicine and multiple other
co-sponsors.
The day unfurled with a series
of eight speakers commenting on the role and value of epidemiology in
decision-making, especially regulatory decision-making, and on the
clash of cultures between epidemiology, the law and politics. The day
ended with a series of six presentations on the lessons learned from
case studies involving dioxin exposure and cancer in workers,
electromagnetic fields and cancer, folate supplementation for the
prevention of birth defects, needle exchange programs for the
prevention of HIV infections, particulate air pollution standards, and
radiation exposure and cancer. In contrast to past discussions in
which epidemiologists have debated the pros and cons of getting
involved in policy matters, the speakers at this workshop seemed to
take as given that epidemiologists wanted to be involved in policy
matters and should be involved in policy making. Thus, the focus of
the day was spent on what should be done to improve the process of
translating evidence into policy, and this orientation made the
workshop unique. Here is a synopsis of the workshop through the words
of the speakers:
Alfred Sommer, Dean and
Professor, Johns Hopkins School of Public Health
Sommer was given the honor of
presenting the first talk and setting the stage for the workshop.
Sommer began by stating the basic tenet that data can drive public
policy, but noted the historical record is not encouraging about how
often this occurs. He reminded the audience, as if anyone needed it,
that other factors besides data play into decision-making. He cited
two recent examples in the US related to national tobacco legislation
and needle exchange programs to make his point. Sommer drew on his
personal experience with investigations of vitamin A to remind
epidemiologists that data do not speak for themselves, and that
scientists should follow up on their leads and build a web of
compelling evidence over time. He encouraged scientists to engage in
the messy political arena using solid evidence, without becoming
wild-eyed advocates.
Joseph Rodericks, Senior Vice
President ENVIRON International Corporation
Rodericks spoke on how decisions
are made about acceptable levels of risk. He traced the history of our
thinking about risk and noted the evolution in earlier days from
seeing risk as having a threshold with “safe limits” and when
regulations could be hammered out in three days to now when no
exposures are considered perfectly safe and the challenge is to
determine how much risk is acceptable. Decisions are no longer made in
three days and require 800 pages in the Federal Register, he said, and
explained that conflicts in the area of risk management often arise
because warring parties often have other agendas and the arguments are
about risk in form only. The real struggles are about power and
ideology. Other problems arise because the public lacks trust in
public institutions and it perceives risk differently than in the
quantitative terms used by experts.
Paul Portnoy, Economist,
President and Senior Fellow, Resources for the Future
Portnoy spoke on the use of
epidemiologic evidence in cost-benefit analyses. He reminded the
audience why this is important since it is estimated that the US
currently spends $150 - 160 billion a year to comply with federal
environmental regulations and that not all of these regulations are
beneficial. Agencies need to justify their regulations, hence the
growing interest and need for cost-benefit analyses of environmental
regulations. Portnoy noted that epidemiology has “an almost essential
role” to play in cost-benefit analyses for a health program because it
helps to determine if and how changes in exposure brought about by the
regulations cause changes in health effects. “Epidemiologic analysis
is not a panacea, and it is hard to control for everything when you
are ‘committing’ epidemiology,” (i.e., doing epidemiologic analysis)
according to Portnoy, “but from 20 years of experience, I will put my
money on epidemiologic analysis every time,” he said.
Moyses Szklo, Professor, Johns
Hopkins and Editor, American Journal of Epidemiology
Szklo spoke on the challenges
posed in interpreting weak associations or small effects in
observational studies when the weak risk may actually lead to a large
population attributable risk. Szklo included, among the “threats to
the assessment of small RR/large AR effects,” such issues as the
imprecision of the estimates, confounding effects, bias, heterogeneity
of effects and colinearity. He discussed consistency between studies
and the reasons why there might be a lack of consistency, including
changes in the circumstances of exposure, in the timing of the
studies, and in the susceptibility to the risk factors among the
populations studied. In his view, policy decisions often boil down to:
How much do we stand to gain if we are right? How much to lose if we
are wrong?
Lynn Goldman, Assistant
Administrator, Environmental Protection Agency
Goldman gave her perspectives on
the use of epidemiology in the regulatory arena. She believes there
has been a “tremendous change” in the role of epidemiology in the last
five years since the administrator of the Agency, Carol Browner, has
made public health protection the number one goal. Goldman believes
that any wall between decision makers and scientists, which exists
because the former fears complexity and the latter fears loss of
objectivity, is not intended to be there. She argued for more
interaction between decision makers and scientists in order to produce
better decisions. Goldman criticized persons who make a career out of
writing op-ed pieces that point out the now well-known weaknesses in
epidemiologic studies (ecologic fallacies, exposures that are
estimates only, confounding, lack of known biologic mechanism,
association vs. causality, and weak associations). In a theme that was
to be echoed by others throughout the day, Goldman called for applying
a standard to how persons criticize studies.
Arthur Bryant, Public interest
lawyer
Bryant highlighted how science
is very important for legal decisions and noted how it is being
misused by the law at present. Sometimes courts give greater weight to
scientific evidence than they should and sometimes less. For example,
some courts have stated that epidemiologic studies provide the only
valid proof of adverse health effects, and if there are no
epidemiologic studies, experts cannot testify that a particular
exposure caused an injury. Or sometimes inconclusive epidemiologic
studies are allowed to trump all other studies, even good in vitro
studies that may have different results. Alternatively, epidemiologic
evidence is given less weight than it deserves, according to Bryant,
when studies with less than twofold risk are excluded, or when only
studies which have been peer-reviewed are allowed to be considered.
According to Bryant, the overall result is that these actions prevent
people from testifying in court and this hurts plaintiffs more than
defendants because plaintiffs tend to have fewer resources to produce
studies. To counter the current misuse, Bryant argued for better
science education for judges, greater use of court appointed experts,
and feedback to judges when they get the science wrong. “Write to
judges, publish articles; call symposia; and file amicus briefs” when
you are convinced that the legal system has mistakenly interpreted the
best available scientific information, said Bryant. All of this so
that the legal system can be “based on the truth” to the maximum
extent possible.
Genevieve Matanoski, Professor,
Johns Hopkins University
Matanoski discussed the
conflicts that arise from the fact that epidemiologists and policy
makers come from different cultures. She noted that translating
scientific evidence into public policy should not be guided on the
scientist’s side by a “hand-over-the-baton type of mindset” when the
findings have been produced. Rather the paradigm should be one of
“complete integration.” If scientists take the position that
collaborating with policy makers will mean that everyone is biased,
then this closes the door to cooperation. “We need communication
first, and we can worry about bias later as the least of our
problems,” said Matanoski. “To stop clashes, to move forward,
epidemiologists need to take a different stand,” she added. The choice
is clear, she said—“you can work on something with public health
interest or you can stay isolated and you will not have this problem.”
Paul Gilman, Executive Director
of the Commission on Life Sciences, National Research Council
Gilman spoke on “Improving
Communications Between Researchers and Policy Makers.” He noted how
there is now some familiarity with epidemiology at the highest levels
of the Congress. As evidence for his conclusion, he searched bills and
reports in Congress for the word epidemiology between 1973 and 1998.
During the 1970’s and 1980’s, the term epidemiology appeared an
average of 25 times in each Congress compared to 53 times in the
1990s. According to Gilman, Congress is now familiar with epidemiology
and comfortable with it as a means of understanding problems and
identifying culprits, but it is less comfortable with it as the
support for policy initiatives and regulatory actions. He cited the
recent controversy surrounding regulations for airborne particulate
matter in which “everyone was acting like a bunch of kids in a food
fight.” Given the rudimentary understanding of epidemiology in the
Congress, Gilman offered rules of thumb for communicating credibly
with Congress which include being candid about everything and
disclosing all information. He warned that disclosure is the norm in
Washington, and holding data confidential or not making data available
would be viewed with suspicion. He also called for more norms in
epidemiology. He said, “while it might be fun pointing out flaws in
studies and it is okay for graduate seminars, it is not great for
briefings and hearings.”
Leeka Kheifets, Electric Power
Research Institute
The final session of the day was
devoted to presentations of lessons learned from the case studies.
Speaking about electromagnetic fields and cancer, Kheifets and her
group of reviewers recommended to epidemiologists seeking to translate
evidence into policy that they 1) stop promising quick answers; 2)
design, analyze and present results to enhance policy; 3) be critical
and keep an open mind, 4) allow for pre- and post-publication peer
review; 5) evaluate and communicate uncertainty, and 6)
participate in policy making and appreciate its complexities. Contrary
to other speakers who sought to temper the widespread criticism in the
field, she urged epidemiologists to be proud of the critical tradition
in epidemiology and to urge others to do the same in their
disciplines. For users of epidemiologic data, she recommended that
they 1) allow sufficient time for multidisciplinary and long range
examination of the issues; 2) accept that there are no yes/no answers
when science is uncertain; 3) mistrust the “study of the week;” 4)
value review panels as providing an ongoing monitoring process and not
the definitive last word on a topic; and 5) defer regulatory decisions
until the science is better understood.
Joseph Rodericks, Senior Vice
President, ENVIRON International Corporation
Speaking about dioxin exposure
and cancer, Rodericks and his group made recommendations about data
development. They called for clear statements about the policy
questions and recommended that policy makers themselves help to define
questions. They also called for study designs that are relevant for
policy and for appraisal of the limits of studies ahead of time. On
improving how we use the results of studies, Roderick’s group
recommended communication of the results in terms directly relevant to
the policy question, communication about the strengths and weaknesses
of the findings, and the inclusion of uncertainty analysis in the
results.
Manning Feinleib, Georgetown
University
Feinleib chaired the session on
folate supplementation and the prevention of birth defects. He called
this a “good news story” in which there was an orderly development of
epidemiologic evidence over a 40 year period. His group reiterated the
point made earlier by Sommer that the accumulation of epidemiologic
evidence can ultimately be convincing and that a consensus can be
reached about what actions to take even when uncertainty remains.
David Vlahov, Johns Hopkins
University
Vlahov presented on the
experience with needle exchange programs. On this topic, there has not
been any question about the effectiveness of needle exchange programs.
They are now widely seen as effective, yet recently President Clinton
and Secretary Shalala refused to translate these findings into policy
by requesting federal funds for needle exchange programs. Vlahov and
colleagues noted that this research was taking place simultaneously
with a war on drugs with zero tolerance, that the affected community
of drug users was stigmatized and did not really serve as a
constituency, and that there was vocal opposition to this particular
intervention among minority communities. For the future, Vlahov and
colleagues recommended that epidemiologists need to consider the
ramifications of their research and its context. Along with others,
they highlighted the need for epidemiologists to learn how to
communicate to the public and to policy makers.
Daniel Greenbaum, Health Effects
Institute
Greenbaum spoke on the
controversy surrounding the setting of particulate air pollution
standards. Among the lessons learned were 1) the value of emerging
tools of meta and pooled data analysis (this field has more than 800
studies in the last 30 years!); 2) the Hill criteria on causality are
a useful guide to the evaluation of evidence but cannot be an absolute
standard; and 3) that while an expert panel can serve as a scientific
referee to separate the wheat from the chaff, the political debate has
no scientific referee and will be shaped by many factors. Perhaps the
most contentious issue surrounding particulate air standards was the
issue of data access and re-analysis. He noted that while
investigators do harbor concerns about property rights, abuse of data
and confidentiality, withholding data breeds suspicion. While not
every study can be re-analysed he said, it may be good to do for some
key studies.
Genevieve Matanoski, Professor,
Johns Hopkins University
Matanoski presented the final
set of recommendations stemming from the experience with radiation
exposure and cancer. Her group along with many others recommended 1)
teaching epidemiologists to talk with the public and with policy
makers; 2) supporting interdisciplinary teams; 3) supporting forums
and journals which address the scientific and policy aspects of public
health problems; and 4) planning for future science and policy needs
at least 10 years in advance.
While it is difficult to
identify overall themes for the day, several of the points made
pertain either to the limitations of epidemiology or the shortcomings
of its practitioners. As for epidemiology, the speakers did not dwell
on the well-recognized problems in the discipline, choosing instead to
remind the audience how necessary and useful epidemiologic studies
are. This generally sympathetic attitude was epitomized by Portnoy who
talked about “committing epidemiology.” The most pointed remarks were
directed at epidemiologists themselves for failing to engage more
fully in the process of translating evidence into policy. Time and
again speakers seemed to be saying that if epidemiologists really want
to influence policy, they need to get serious about policy making and
change their behaviors in a number of important ways to make
translations of evidence into policy more frequent and more
successful.
Published August/September 1998
v
|
