ODI Logo ODI

Trending:

Trending

What we do

Search

Newsletter

Sign up to our newsletter.

Follow ODI

Does evidence matter?

Date
Time (GMT +01) 11:00 12:30

Speakers:

David Halpern – Strategy Unit, Cabinet Office

Erik Millstone – Science Policy Research Unit

Chair:

John Young – Programme Manager, RAPID, ODI

    1. This was the first meeting of the series, ‘Does Evidence Matter?’ John Young outlined that the aim of the session was to focus on a few key questions: Why is evidence important in policy making? What sort of evidence? How to get it? Is the current emphasis on evidence- based policy in government resulting in better policies?
    2. David Halpern’s presentation, Evidence Based Policy: “Build-on” or “Spray-on”? focused on evidence-based policies in the UK – examples of success and failure, the characteristics of what distinguished the different outcomes and how we can do it better. Is policy built on a base of evidence or is evidence sprayed onto what policymakers were going to do anyway?
    3. The UK has many specific cases where evidence has improved policy with positive outcomes –literacy, labour market participation and pre-schooling, for example. Several successful policy strategies have been based on evidence from other countries. He explained that, in addition to what the evidence was, how the evidence is marshaled matters for the outcome. However, even in cases where evidence did influence policy, it is often difficult to attribute changes in policy to a specific evidence-based strategy. The UK also has examples where there have been gaps between the policy and the strategies that would be suggested by existing evidence – the crime and justice sector and primary healthcare issue are two.
    4. What drives impact? David Halpern highlighted five issues:
  • the evidence must exist – and good evidence takes time to marshal;
  • someone must know the evidence exists; 
  • the evidence must have policy implications – “so what?” is a common response to many research papers;
  • the issue must be relevant to public interests;
  • it must be in the “zone of proximal development” – evidence must be within existing frameworks of understanding.
What would improve the situation?
  • Bridge the division between analysts and policymakers;
  • encourage experiments and variance – so we can see what does and doesn’t work; 
  • be realistic about how knowledge spreads;
  • do the groundwork for next time;
  • statistics are important – and so is the capacity to understand them; 
  • continue communicating;
  • reform the Research Assessment Exercise.
Other suggestions for researchers were:
  • talk to policymakers – and keep talking; 
  • look for policy windows;
  • take a long-term perspective – you are likely to have more impact in two years than two weeks;
  • use intermediaries;
  • work inside government.
Evidence is actually used much more than people think. But, evidence is only one of a number of factors that influences policymaking. Erik Millstone spoke on what evidence can and cannot do using the case of BSE as an example. He described it as one of the biggest evidence-policy failures in recent times. Policy decisions are usually a hybrid of political and technical considerations. The government claimed policy was based on the best scientific advice. But, the BSE case was one where the evidence was not clear – scientists were not sure if BSE could affect people. Bureaucrats were privately worried, but did not always let policymakers know the whole story. The science was misrepresented to the public – ministers argued beef was safe. For policymaking to be evidence-based requires both technical information and social information (e.g. whether a policy is actually being implemented). Understanding is never complete – there are always some gaps and there are always risks. In the BSE case, the problem was that the policymakers became “addicted to their own narrative: 'Our knowledge of BSE is sufficiently extensive, comprehensive and secure to guarantee that British beef is perfectly safe'. Eventually the credibility of the policymakers was destroyed by the evidence and the BSE case undermined public faith in science-based policymaking. What kind of policymaking model is best? Neither a decisionist nor a technocratic model was desirable. There was agreement that there needed to be interaction between policymakers, implementers and scientists. The “iterative” model was preferred. Comments focused on a range of issues:
  1. It is important to distinguish between public policy statements and practice on the ground when considering the use of evidence.
  2. Clearly the political context matters. Policymakers are under diverse pressures and evidence is one set of issues that influences them.
  3. Elites and vested interests affect whether evidence is used and, if so, what evidence is used.
  4. Personal dynamics matter – it is often difficult to speak truth to those in power.
  5. Policymakers are much more constrained in their actions than researchers – and therefore linking research and policy is not as straightforward as researchers might hope. This raises the importance of intermediaries who both know the evidence (new thinking as well as existing evidence) and are less constrained than policymakers.
  6. We should not focus solely on what we can measure or learn from stakeholders. Some important issues cannot be measured and some stakeholders are invisible and would be ignored if evidence was the sole guide to policymaking. This is especially important in the humanitarian policies, where communities needing support are less visible, vocal and powerful.
  7. Who is an expert? Using evidence is important, but it is dangerous to create a “cult of expertise”. It is important to draw on a range of knowledge.
  8. Public opinion increasingly matters in UK policymaking. Policy change often occurs when the public understand issues. It is therefore important to de-mystify scientific advice.

Description

This event in the 'Does Evidence Matter' series examined the following questions:

  • Why is evidence important in policy making?
  • What sort of evidence?
  • How to get it?
  • Is the current emphasis on evidence-based policy in government resulting in better policies?