Can we trust the COVID-19 data?

The deluge of COVID-19 data over the past few months has left many people confused. Never before have politicians, scientists and the general public shared such a real-time interest in data and evidence based on data.

It is incredible that we can access an online COVID-19 dashboard or tracker and see the daily and cumulative distribution of deaths worldwide. On the other hand, it is clear that counting deaths is not straightforward, that scientists differ in their predictions of numbers of infections, that researchers have to withdraw erroneous results of clinical trials. and that politicians and scientists argue about the interpretation of the same data.

Most pages of this website emphasize the importance of collecting good data and of checking its quality. This section provides signposts to other pages.

How were the data managed?

  • The data quality and information integrity page provides advice on preventing, monitoring and correcting errors when collecting and managing data and suggestions about how the data and resulting information should be documented and can be assessed.

Return to contents ⇒

How were the data reported?

Return to contents ⇒

How were the data modeled?

Return to contents ⇒

How was the evidence reported?

There has been a proliferation of academic papers describing the characteristics of the COVID-19 pandemic, assessing prevention strategies, and evaluating tests, treatments and vaccines. Some of these have been the result of world class research and a few have been fallacious. The golden rule is to ascertain whether or not the evidence has been reviewed by peer scientists but in a few cases even this process fails. For example, the Lancet had to withdraw a paper that associated hydroxychloroquine sulphate to higher risk of death from COVID-19, and the New England Journal of Medicine (NEJM) withdrew a paper suggesting that cardiovascular diseases increased the chances of in-hospital Covid-19 death.

Even when the quality of research is high, the results of similar evaluations of treatments, for example, can differ. This can result from differences in methodology or simply by chance. When sample sizes are small, the differences between similar studies will be greater than when sample sizes are large.

  • Systematic review. The data to implement programmes page describes how scientists review several published research studies addressing similar questions and synthesize the evidence they provide. They do this using a standardised process called systematic review. The Cochrane initiative has initiated a process of Cochrane Covid rapid reviews, the results of which are available on their website.
  • Expert review. Scientists comment on published research findings, for example by writing commentaries and letters in journals or online blogs. Science Media Centres in Australia, Japan, Germany, New Zealand, United States, United Kingdom deliberately invite experts to give their opinions on the latest findings to help decision-makers, press officers, journalists, scientists and the general public interpret the findings. The UK Science Media Centre publishes daily commentaries on its COVID-19 page.

Return to contents ⇒

Resources

O’Neil C. 10 reasons to doubt the Covid-19 data. The pandemic’s true toll might never be known.

A dozen rules of thumb prepared for journalists by the Royal Statistical Society.

The Humanitarian Data Exchange publishes a set of FAQs to support organizations and staff around the world working with data in the COVID-19 response. Data Responsibility in the COVID-19 Response  

Science Media Centre for Canada. Journalists’ resources for covering COVID-19

The UCSF IGHS COVID-19 Research Watch puts together a weekly summary of original COVID-19 research focused on public health, including epidemiology, non-pharmaceutical interventions, and presentation patterns.

Balsari S. Which Covid-19 data can you trust?

Royal Statistical Society: Significance. This online magazine has up to date articles on how to interpret COVID-19 data.

Richardson  S, Spiegelhalter D. Coronavirus statistics: what can we trust and what should we ignore? 

A statistician’s guide to coronavirus numbers. The Royal Statistical Society collated this essential guide for understanding statistics about COVID-19.