Why data for policy?
Evidence has influenced policy shifts in global health, for example, introduction of: voluntary male medical circumcision programmes that prevent HIV transmission; laws enforcing wearing automobile seatbelts and motor bicycle helmets that have reduced deaths and brain injuries from traffic accidents; and legislation to control tobacco use that has dramatically reduced lung cancer rates.
In of the above examples, scientists communicated their findings sufficiently clearly to eventually convince advocates and policymakers to champion and implement these laws; and scientists have demonstrated reductions in morbidity and mortality after the laws came into force.
National governments recognize they need data to govern. They invest in statistical offices and information systems so they can target resources to meet documented priorities.
Health policymakers respond to evidence of growing challenges, such as HIV/AIDS, opioid use, and the Zika virus, while they maintain and monitor control of obesity and diabetes, cholera, malaria, and maternal and infant mortality.
To highlight areas for policy focus, governments need up-to-date, reliable and relevant information about morbidity and mortality trends, and differences in health outcomes between geographic regions, racial groups or gender. Comparative data from international databases, such as WHO maintains, anchor new policy directions.
Participants in policymaking
- Policymakers or lawmakers conceive and develop policy agenda and argue for policy adoption. They are elected or nominated to prepare laws that protect the health of the people they represent.
- Programme managers interpret policy directives, implement and evaluate policies and suggest refinements and expansion. They work for governmental, non-governmental or private agencies at the state, provincial or district level, or for international institutions.
- Policy watchdogs are individuals and institutions who look for gaps in policy and policy implementation and lobby for policy change. They include advocacy and community groups, non-governmental organizations, media, individuals devoted to specific causes, and whistle-blowers.
- Data generators are statisticians, information technology specialists, or data managers who run health information systems and prepare regular performance reports. They maintain routine health facility records for governmental, non-governmental and private institutions, or run censuses, civil registration systems, disease surveillance, or regularly undertake large and small-scale surveys. They work nationally or internationally.
- Data analysts include statisticians, epidemiologists, sociologists or health economists who design qualitative and quantitative studies and analyse, model and present data to provide evidence for policy. They work in academic, governmental and other research institutions anywhere in the world.
- Data brokers are policy analysts who serve as intermediaries between data generators and analysts and policymakers. They gather data and evidence to address policy issues, analyse secondary data and big data, conduct systematic reviews, and prepare policy briefs. Brokers work for academic and governmental institutions or for independent policy units to advise local, national or international lawmakers.
- Evaluators work with programme managers to evaluate policy implementation at any level national or internationally. They develop frameworks, select indicators, interpret data and provide quantitative and qualitative, contextualized data on why certain outcomes are achieved or not.
Stages in policymaking
We identify six stages through which policy develops. Although we present the stages in sequence, they are seldom linear and don’t necessarily result in policy preparation or adoption. Their order and timing depend: on political will to pursue specific policy solutions; availability of appropriate and well-presented evidence; and competing priorities for resource allocation. Some policies fail early but get re-introduced years later when public opinion changes, for example, legislature for gay rights in the US. Other policies may start with one set of expectations and then be co-opted to address a different issue.
Stage 1: policymaker becomes aware of issue
Policymaking begins when a lawmaker recognizes an issue and considers developing or amending policy to address it. Issues usually correspond to political agendas but may arise organically through advocacy by policy watchdogs or reports from data brokers.
Members of the public may observe unprecedented traffic accidents at a particular location and work with police to propose speed limit changes. Professional organizations or non-governmental organizations may identify inadequacies in human resources and campaign to train and employ suitable health workers. Academics may demonstrate inequalities in people’s access to health care and press policymakers to address gaps in delivery.
The media may spotlight health problems through investigative reporting and help mobilize communities to consider options to decrease adverse outcomes. International organizations, such as WHO, provide evidence from different countries to highlight global issues.
Stage 2: policymaker adopts issue
To understand why they should prioritize an issue, lawmakers need to know the size of the problem, where and when it occurs, the most vulnerable groups, and how people perceive the issue. Convinced of its importance, the champion lawmaker firmly adopts the issue, develops an agenda and engages other policymakers, stakeholders, and constituent groups.
To make a firm commitment, lawmakers ascertain what hard evidence exists that makes agenda setting a high priority. Increasingly, international data sharing contributes to raising awareness about policy imperatives, as well as potential policy solutions.
Questions relevant to stages 1 and 2
Types of data to answer the questions
Methods to gather the data
What is the scale of the issue?
Who does the issue affect and how?
When and how often does the issue occur?
Where does the issue occur?
How do people perceive the issue?
Why should the issue be prioritized?
Proponents offer evidence about the importance of the issue (collected by data generators and analysts)
Data brokers advise policymakers about the quality and validity of data
Routine data collected through health information systems
Epidemiological and social surveys
Focus group discussions
Stage 3: policymaker formulates a bill
Having agreed an agenda, lawmakers propose and formulate policy options. They expect data brokers to review successes and failures of interventions implemented elsewhere and consider how interventions might work or be adapted to context. This includes systematic reviews and grading of evidence from journal articles and grey literature and examination of experts’ perspectives about best practices.
Lawmakers may request that data analysts gather and analyse new data—either qualitative or quantitative—to test acceptability of policy options, for example, to undertake focus groups of likely programme recipients, or public opinion surveys to check the public’s and business’ perspectives on policy direction.
Data brokers will analyse routine data or mine data sets that have not been analysed for this purpose. They may consider social determinants that could underlie the problem, for example, unsafe communities that prevent families from playing outside, or lack of viable transportation that impacts access to grocery stores and physical activity.
Stage 4: policymaker advocates for the bill
The champion policymaker then articulates the policy proposal, or bill, and attempts to persuade other lawmakers to adopt it as law. The champion builds support for the bill using bargaining, persuasion, and compromise.
Other lawmakers raise questions that require data brokers to collect and provide additional information. For example, scientists have had to produce significant data about safety of routine immunization against communicable diseases to convince policymakers to continue enforcement.
Questions relevant to stages 3 and 4
Types of data to answer the questions
Methods to gather the data
What interventions have addressed the issue successfully elsewhere?
Who will the proposed policy target?
When will the policy be implemented?
Where will the policy focus?
How much and what type of resources will policy implementation require?
Why will the chosen policy combination succeed?
Data brokers synthesize and present evidence for potential strategies to address the issue and advise on their cost and benefits
Synthesis of clinical trials or intervention studies
National and international consultations on experience with potential strategies
Cost benefit analyses
Data brokers prepare policymakers to answer questions from other lawmakers to assist them in their decision-making
Policy briefs collating answers to potential questions from the data gathering described above
Assessment of resources required for implementation
Stage 5: managers plan implementation
Once a bill passes into law, bureaucracies translate the law into guidelines or rules and regulations. National, state or local governments implement new legislation, such as agency activities and public expenditures, through public programmes.
Stage 6: managers assess implementation
Evaluators support lawmakers and programme managers to conduct systematic evaluation of a policy—its actual impacts, costs and whether it achieved its intended results. They inform policymakers of future policy options and suggest refinements they might consider. When a policy does not achieve expected results, data generators and evaluators may provide nuanced data analyses to show what it has achieved, for example for population subgroups.
Questions relevant to stages 5 and 6
Types of data to answer the questions
Methods to gather the data
What monitoring and evaluation framework is most appropriate?
Who will benefit and how?
Where will the evaluation happen?
When will the evaluation be undertaken and how frequently?
How will the stakeholders know if the policy is successful?
Why is the evaluation necessary?
Programme managers and data generators agree on reliable and relevant indicators to monitor policy implementation
Specification of the logical framework and the indicators of input, output, outcomes, and impact.
Programme managers and programme evaluators measure the indicators and indicate changes in impact over time
Monitoring and evaluation of implementation using health information systems, targeted surveys and trials and focus groups
Responsibilities of data specialists
- Data generators, mainly during policy recognition and agenda setting: maintain the health information system; undertake surveys to address specific issues; ascertain the public’s opinions on government services; present and visualize data; clarify data limitations.
- Data analysts, translating data into evidence at all stages of policy development: Provide advice on design of qualitative and quantitative studies; analyse data paying attention to trends and inequalities; develop models to estimate and predict results of policy options; analyse big data available through social media; present and visualize data; provide and explain statistical inference; describe data limitations.
- Data brokers, mainly during agenda setting, and policy formulation and adoption: Assess whether stakeholders’ interpretations are valid; decide whether available data provide sufficient evidence; request additional data generation to justify policy options; conduct secondary data analysis, meta-analyses and systematic reviews; undertake interviews, focus group discussions and polls of public opinion; make inter-country comparisons; analyse big data available through social media; prepare policy briefs, press releases, and social media.
- Evaluators, during policy implementation and evaluation: Provide advice on the monitoring and evaluation framework; select and justify the indicators to be used; provide advice on data collection and analysis; prepare timely and comprehensive reports.
We advise data specialists and policymakers to:
- Communicate: Data specialists and policymakers can network to share and appreciate each other’s perspectives through discussion fora, and staff exchanges between research institutions and government departments.
- Invest in flexible open information systems: Much data that health information systems collect are not directly useful to decision makers. Data generators can build improvements into waves of data collection, while measuring indicators consistently over time.
- Formulate policy questions that clarify data needs: If data specialists and data users collaborate to formulate policy relevant questions. there will be better fit between questions and data sources and also temper policymakers’ expectations of the time it takes to collate and analyse data.
- Tailor data collection and analysis to the time available: When time is short, instead of requesting a full-scale dedicated survey, the data broker can undertake secondary analyses and actively mine existing, and sometimes under-used, data-sets. Focus groups and in-depth interviews provide snapshots of opinions and explain quantitative findings. Triangulating information from multiple sources can provide additional insights.
- Explore and discuss data limitations: Scientists must provide a margin of error for their primary conclusions and clarify the time period and populations to which their findings apply. They should control data quality and assess findings for consistency over time and between sources. They may need to collect more data to explain unexpected results. Data specialists can advise lawmakers about how to recognize reliable evidence. Researchers should declare personal bias and why they chose to study a topic and they should watch out for unconscious bias when they interpret their findings.
- Present and discuss data limitations: Unlike research papers, policy briefs are short and contain only information essential to make a clear argument. Infographics can summarize the same information on a single page or poster using a combination of text, diagrams, graphs and maps. Findings can be disseminated as infographics, posters, flyers, interactive internet features, videos, or power-point presentations.
The complete chapter on which we based this page:
Brindis C.D., Macfarlane S.B. (2019) Challenges in Shaping Policy with Data. In: Macfarlane S., AbouZahr C. (eds) The Palgrave Handbook of Global Health Data Methods for Policy and Practice. Palgrave Macmillan, London.
Lukaslehner maintains a collection of COVID-19 policy trackers and data. It covers cross-country research in the areas of non-pharmaceutical interventions, economic and social policy responses, public attitudes, politics and media coverage.
The Our World in Data Coronavirus Government Response Tracker collects publicly available information on 17 indicators of government responses, spanning containment and closure policies (see the project’s working paper)
Covidvis, a collaborative based at the University of California Berkeley, publishes a webpage Visualizing the Impact of SARS-CoV-2 Intervention Strategies
The International Monetary Fund publishes a policy tracker that summarizes the key economic responses governments are taking to limit the human and economic impact of the COVID-19 pandemic. The tracker includes 196 economies.
Verhulst and Zahuranec of GOVLab. Using data for COVID-19 requires new and innovative governance approaches.
Erondu and Hustedt. COVID-19 policies not backed by data do more harm than good.
The US National Academies Press’s Coronavirus Resources Collection includes a number of rapid expert consultation reports for decision makers
Advocacy and policy change evaluation theory and practice. Gardner and Brindis describe the concepts, designs, methods, and tools needed to conduct effective advocacy and policy change evaluations.
A systematic review of barriers to and facilitators of the use of evidence by policymakers. Oliver et al present the results of a systematic review of the barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area.
Evidence based policy and practice: cross sector lessons from the UK. Nutley et al describe some key lessons to have emerged from the experience of trying to ensure that public policy and professional practice are better informed by evidence.
Conceptualising the information needs of senior decision makers in health. Davies et al propose approaches to identify the needs of senior health decision makers.