Despite the much touted evidence-based policy and practice movement of recent years, a new National Research Council report reaches the striking conclusion that “studies of knowledge utilization have not advanced understanding of the use of evidence in the policy process much beyond the decades-old National Research Council (1978) report.” That report, entitled “Knowledge and Policy: The Uncertain Connection” failed to find systematic evidence that social science evidence was being used. The findings from both reports are bleak and apply to all the sciences says the NRC since knowledge from all sciences is potentially relevant to policy choices.

2012 Report

According to the latest report, scientists have attempted to improve or better understand the use of science in policy making in two ways—either by strengthening the scientific evidence itself or by studying the process in a scientific specialty called knowledge utilization. According to the committee, “…the inevitable indeterminancy and context specific nature of use prevents these two efforts from providing a fully satisfactory understanding of the use of science or a satisfactory guide on how to strengthen that use in policy making.”

Reasons for Failure

In the process of describing some of the reasons for failure, the report noted that one of the reasons has to do with the limited way in which scientists may think about the process of policy making. According to the NRC, “some mixture of politics, values, and science will be present in any but the most trivial of policy choices. It follows that use of science as evidence can never be a purely “scientific” matter; and it follows that investigating use cannot exclusively focus on the methods and organizational settings of knowledge production or on whether research findings are clearly communicated and how.” 

Relevance of Values

According to the NRC, the political and value considerations that enter into the policy making process have been seen as outside the scope of science.  It added, “understanding whether, why, and how…scientific knowledge is used…is uniquely suited to the methods and theories of the social sciences. Making ‘use’ of scientific knowledge is what people and organizations do. And what people and organizations do is the focus of social science.”

New Approach

To make better progress, the committee constructed a new framework for thinking about the policy process and for conducting further research which builds into the model recent developments in social science, the role of values, and political considerations.

New Insights

In the process of achieving its mandate the NRC Committee provides several important insights about science, the policy process, and the intersection of the two domains. Furthermore, the report takes a less pessimistic view of the endeavor to understand and improve use than do other observers who have concluded that “…externally valid evidence pertaining to the efficacy of specific knowledge exchange strategies is unlikely to be forthcoming.”  To the contrary, the NRC report describes an alternate way to frame the issue and perhaps make more headway than has been made in the past.

Challenges To Learning More

Among the challenges in addressing whether, why, and how science is used in policy are the different perspectives of the disciplines and investigators who study the interface of science and policy making. This variability leads to difficulties in defining the phenomena of policy making and use and to different framings of the issue, according to the Committee.

Poor Models

Also, there is no generally accepted explanatory model of policymaking, but instead several different kinds of models, including descriptive, rational, and political models. After considering all of these approaches, the NRC committee concluded “…it is clear that the various models and frameworks do not coalesce into anything remotely resembling a powerfully predictive, coherent theory of policy making...” And the committee adds, “This conclusion is consistent with the fact that policy choices are context dependent.”

Two Communities Metaphor

One popular concept for addressing the intersection of science and policy has been the “two communities” metaphor which posits that scientists and policy makers constitute two separate groups of actors with separate purposes, cultures and values. According to the report, “differences between the two communities are associated with a contrasting list of supply-side and demand-side problems.”
 

It notes that this framing of the use problem “offers little guidance as to which of the long list of factors, from either side, best explains variance in use, let alone how the factors interact and whether they apply only in specific settings or have general applicability.”

Interaction Model

Among the strategies used by investigators seeking to bridge the gap between the two communities are different communication strategies and different researcher-user collaborations. These strategies are known as translation, brokering, and interaction. In explaining each of these, the committee notes that the interaction model goes beyond transfer, diffusion, and dissemination and even beyond translation and brokering. The interaction label covers a family of ideas directed to systemic changes in the means and opportunities for relationships between researchers and policy makers.  Interaction models appear to have considerable promise and the committee quotes one observer who believes they are the “most likely” models to help us understand how research actually gets used.

Committee Recommendations

In addition, the NRC committee offered its own views about how progress could be made in this area. These include a reframing of the problem from how to increase the use of science to one of how to  help improve the process of policy making. The committee speculates  that perhaps an excessive focus on the first formulation has distracted scientists from focusing on the second. Anchoring its view of the problem from the perspective of the policy maker, the committee offers its own framework as follows:

New Framework

“Our proposed research framework is based on a view of policy makers engaged in an interactive, social process that assembles, interprets, and argues over science and whether it is relevant to the policy choice at hand, and if so, using that science as evidence supporting their policy arguments. Policy argument as a form of situated, practical reasoning directly leads to a concern with how evidence, in the specific way now defined, is used rather than how it is produced.”

The committee’s recommendations are all about paying more attention to what happens during actual policy arguments when science presumably has the opportunity to make a difference. The recommendations fall into three categories.

Three Pronged Approach

As a necessary first step, it calls for paying more attention to investigating what constitutes valid arguments from the policymakers’ perspective and from that of the persons they need to persuade. Second, the committee calls for better understanding of the decision process itself, particularly in light of what is being learned about the psychological processes in decision-making.

Systems Perspective

Perhaps the centerpiece of the NRC report’s contribution is the call for the use of a systems perspective to investigate the use of science in policymaking. It describes such an approach as one of “…an iterative learning process in which we replace a reductionist, narrow, short-run, static view of the world with a holistic, broad, long-term dynamic view, reinventing our policies and institutions accordingly.  Such an approach has already been showcased at an NIH symposium on childhood obesity.
http://tinyurl.com/c8h9zl8

New Metaphor

Because evidence does not reside only in the world where science is produced but rather emerges in the world of policy making where the committee says it is interpreted, made sense of, and used, then evidence-influenced politics is potentially a more informative metaphor than evidence-based policy.

Reader Comments:
Have a thought or comment on this story ?  Fill out the information below and we'll post it on this page once it's been reviewed by our editors.