Measurement of indicators

Components of an indicator

A single item of data comprises an observation (or measurement) made of a characteristic on an observational unit.

For example: this birth was attended by a skilled birth attendant; this three-year-old child weighs 12.1 kilograms; this man shows signs of tuberculosis; three people live in this household. All data items have three attributes, that is: the unit of observation (the birth, child, man, household); the characteristic observed or measured (type of birth attendance, weight, diagnosis of tuberculosis, number of people in a household), and the value of the observation (presence of a skilled attendant, 12.1 kilograms, signs of tuberculosis, three people).

When observers make multiple observations of the same kind on different units, they use indicators to summarise the data.

For example: 65 per cent of births occurred with the support of a skilled birth attendant; the average weight of three-year-old children was 12.4 kilograms; 12 per cent of the male population showed signs of tuberculosis; 95 out of 161 households had three or more people living in them.

Indicators summarise and allow users to interpret raw data; that is they provide information that the raw data cannot. Indicators facilitate comparisons of data across population sub-groups, time and space.

Types of indicators

Indicators are calculated from raw data observed on a sample selected from a target population. Sometimes they are calculated from data for an entire population. Indicators based on samples are estimates of the true population value and are reported with a margin of error, usually expressed as a 95 per cent confidence interval.

Proportions

Epidemiologists classify people as having a condition, or not, during sample surveys. They count the proportion of people surveyed in a geographical area at a specific time who. For example the proportion who: test positive for a condition such as anaemia or malaria; or possess an attribute such as having access to safe drinking water. They report the proportion as an indicator of a condition’s prevalence in the population which they designed the survey to represent.

Researchers classify study participants by possible socioeconomic and demographic determinants of the condition. For example, they may use categorical data such as education, type of residence or gender, and report indicators as proportions of the sample falling into specific categories.

A proportion is the number of units that have an attribute at a specific point in time (or over a period of time) divided by the total number of units in the population at that time (expressed as percentage, per thousand or, for rare attributes, per ten thousand), for example:

To describe the prevalence of a condition

• Prevalence of stunting (height for age <-2 standard deviation from the median of the World Health Organization (WHO) Child Growth Standards) among children under 5 years of age (SDG 2.2.1)
• Prevalence of malnutrition (weight for height >+2 or <-2 standard deviation from the median of the WHO Child Growth Standards) among children under 5 years of age, by type (wasting and overweight) (SDG 2.2.2)

Rates

Epidemiologists also count numbers of new cases of a condition such as measles or breast cancer which occur in a population during a time period. They divide this number by the average number of people who were at risk for the condition during the time period and report an incidence rate which indicates the frequency with which the condition occurred de novo in the population in the time period. Similarly, demographers express the numbers of births and deaths occurring in a population during a time period as rates per population.

A rate is the number of occurrences of an event in a time period divided by the average number in the population in which the event could occur (expressed as per thousand or, for rare events per ten or hundred thousand), for example:

Counts

Health managers calculate the mean of counts of the numbers of people attending outpatient clinics or patients admitted to hospital, or express the number of health facilities per population, that is as a density.

A count is the number of cases, events, items in a time period and/or a geographic area, for example:

Ratios

A ratio is one count divided by another, for example:

Presence or absence

A record of presence/absence indicates existence, for example:

Average

Researchers report measurements that have a population distribution, such as blood pressure or age, as a mean or median of a sample of person-specific measurements – with an indication of the range of variation across individuals. They can also report measurements as the proportion of observations falling in a range of interest. For example, among 55,015 babies born alive in Scotland in 2012, 90.1 per cent were of normal birthweight (that is, between the 5th and 95th percentile of the birthweights of all babies of the same gestational age).(8)

An average is the sum of all the measurements in a sample divided by the number of values, for example:

MedianThe value below which half the observations lie and above which half the observations lie

Composite indicators

A composite indicator is formed from two or more indicators on the basis of an underlying model of the multi-dimensional concept that is being measured, for example:

• Coverage of essential health services (defined as the average coverage of essential services based on tracer interventions that include reproductive, maternal, newborn and child health, infectious diseases, non-communicable diseases and service capacity and access, among the general and the most disadvantaged population) (SDG 3.8.1)

Qualitative indicators

Facts, assertion, opinions expressed as quotations or narratives e.g. opinions about care provision; patients’ understanding of their health

Whereas the majority of indicators to track national and global health goals and targets are quantitative, understanding the explanatory factors behind the numbers often requires qualitative research and analysis. Qualitative researchers undertake in-depth interviews or host focus group discussions with selected participants. They summarise their findings as people’s judgements and perceptions about a topic, such as their confidence in the quality of their health care.

Qualitative data could summarise the range of opinions of participants in a focus group discussion about whether health workers respected their privacy, or their reasons for not attending antenatal care or not using condoms during unsafe sex.

These data could be expressed in words or as the number of persons reporting positive or negative feelings about a given issue. Such information provides insights for community organizers or health planners to determine how to deliver health education and care.

Qualitative information can be close to the lived reality of stakeholders while quantitative indicators are perceived to be objective but distanced.

Measuring and estimating indicators

Data for indicators derive from many sources, some reliable, others not. For health sector programming, the primary sources are censuses, civil registration systems, surveys, health management information systems and surveillance systems.

Calculating indicators can be straightforward but depends on the availability of suitable data. Prevalence rates for long-standing health conditions can be calculated directly from well-designed surveys and disaggregated by sub-population characteristics, such as age, sex and socioeconomic status, and by geographical region.

Disease incidence rates can be calculated from data generated by high-quality surveillance systems that accurately record the number of new cases over time.

Mortality rates can be calculated directly from civil registers so long as death registration is complete. But death registration is the exception in most low- and middle-income countries which rely on other methods of estimation. For example the Demographic and Health Surveys estimates mortality and morbidity notably for children under five years.

Coverage rates can be derived from facility records if the numerator (persons receiving a service) and denominator (local catchment population targeted for the service) are accurate. When facility records are unreliable, analysts derive coverage indicators from household surveys. They rely on participants’ recall of their receipt of a service or corroboration, for example from a hand-held immunization record.

Analysts control indicator estimates for factors that might distribute differently among comparison groups, for example, they may adjust rates and ratios for age – in effect by assuming that age is represented equally in all comparison groups.

Estimates of the same indicator for the same population can vary far beyond differences that can be explained by sampling error. Such differences usually result from inadequacies or fundamental differences in methods of data collection or from non-response and poor recall.

Modelling

Analysts draw on data from several sources to build models to estimate indicators. They study associations with factors that might influence the indicator. They build global models to estimate national-level indicators when data are missing or unreliable. For example, international modelers estimate maternal mortality which many countries report unevenly.

Researchers also create composite indicators such as disability-adjusted life-years (DALYs). DALYs estimate disease burden as the number of years lost by all persons in a specified population, over their lifetimes, due to ill-health, disability or early death.

The value of estimates based on modeling can also vary enormously depending on the underlying databases, sources and statistical methods used by the institutions publishing them.

Presenting and interpreting indicators

Although a single value of an indicator for a given population at a given time can impact the way people think, it needs to be set in context.  For example, indicator values can be: 1) compared with values for earlier time periods to show a trend; 2) distributed across population groups to show differences or disparities; or 3) displayed on a map to demonstrate geographical patterns.

It is advisable to treat all indicators with caution. We offer some advice about interpreting published indicators:

Quality

Is the indicator value feasible? What is the quality of the data on which it is based? Was the indicator calculated using standard methods (e.g. WHO metadata)? What is the range of uncertainty?

Interpretation

How does the indicator value differ from earlier measurements for the same population? Does the indicator show a positive or negative change? Does it show a trend in a particular direction? How does the value differ between areas and sub-groups? How does the value compare with values reported by other countries?

Never generalise the indicator value beyond the time period and population group to which it refers. Check that the definition of the indicator was the same as for any other values with which you want to compare. Treat values based on small samples with extreme caution.

Source chapter

Macfarlane S.B., AbouZahr C., Tangcharoensathien V. (2019) National Systems for Generating and Managing Data for Health. In: Macfarlane S., AbouZahr C. (eds) The Palgrave Handbook of Global Health Data Methods for Policy and Practice. Palgrave Macmillan, London.

Bonita e al. Basic epidemiology

Last J. A dictionary of epidemiology Edited by Miquel Porta

World Health Organization. 2018 Global reference list of 100 core health indicators (plus health-related SDGs) .

World Health Organization. Global Health Observatory (GHO) data

United Nations. Sustainable Development Goal indicators.