What are household surveys?
Surveys describe changing population health data which other sources cannot provide. Surveys can, for example, measure the extent of undiagnosed chronic diseases in a population, and their risk factors. This is especially important in situations where people do not access or cannot access health facilities in which health workers can diagnose chronic conditions such as diabetes, cancers and cardiovascular diseases.
From the mid-fifties, governments have initiated two types of general national health household survey.
National health interview surveys (NHIS)
National health interview surveys describe people’s behaviour and opinions, health status and health service coverage. Trained interviewers ask individuals in sampled households a range of socio-demographic and health questions including: perceived health, risk-taking behaviour and utilization of the health system.
National health examination surveys (NHES)
National health examination surveys provide additional information about undiagnosed chronic health conditions in a population. Interviewers also examine respondents, for example by taking measurements of blood pressure and anthropometry, or blood specimens to measure blood glucose.
Household surveys do not stand alone; they complement data from other components of national health information systems. Censuses count entire populations (and ask some questions) every ten years. Civil registration systems document vital events continuously as they occur. Health management information systems record people’s interactions with the health system. But interviewers conducting household surveys reach out and spend time with a sample of people.
Why household surveys?
Although it was not until the early-twentieth century that statisticians began to accept that sample surveys could describe populations, surveys now rank ‘among the most important innovations in social science research of the last century’.
Many high-income countries now run regular or continuous NHESs and well over 100 LMICs run a DHS, a MICS or both every four or five years. The European Union is establishing a common European NHES (EHES), with standardized protocols, and is building capacity for comparable NHESs across European countries. So why is the world running these costly and time-consuming surveys?
Household surveys provided data for 21 of the Millennium Development Goal indicators and will provide data for most of the health-related Sustainable Development Goal (SDG) indicators, including Universal Health Coverage (UHC). Health facility data in many countries are inadequate to measure service coverage for all 16 UHC tracer indicators grouped as: reproductive, maternal, neonatal and child health, infectious and non-communicable diseases. The Eastern Mediterranean Regional Office of WHO, for example, is supporting countries to undertake a new wave of NHESs to include measurement of UHC. Tunisia’s 2016 NHES included service coverage and household spending on health.
National sample surveys
Early national surveys focused on socio-demographic data, aiming to complement census information and provide estimates of changing population demography. The United States (US) 1935/36 National Health Survey was probably the world’s first government-funded national morbidity survey, focussing on self-reported chronic disease and disability.
The National Sample Survey in India included morbidity in its seventh round in 1953-54. but only undertook a full-scale morbidity survey in its 28th round in 1973-74. This survey subsequently included morbidity in its decennial surveys on social consumption, with a morbidity and health care survey in 2004. Modules cover morbidity, utilization of preventive and curative care, and household expenditures on health. India undertook its first dedicated National Family Health Survey from 1992-93, on maternal, child and reproductive health, which it repeats every six to seven years.
Development of surveys
The Japanese National Health and Nutrition Survey began in 1948 to monitor the impact of food shortages after the Second World War and is the oldest running annual NHES in the world. The UK ran a Survey of Sickness from 1943 to 1952 and included a health module in its General Household Survey from 1971. The US undertook an NHIS in 1957, which it has operated continuously since, and began an NHES in 1959, which became a National Health and Nutrition Examination Survey (NHANES) in 1971–1974. NHANES has run continuously since 1999 and has set a standard for other NHESs.
In Europe, West Germany introduced an NHIS in 1957, followed by France in 1960, Finland in 1964, the Netherlands in 1974, and Switzerland in 1980. In the rest of the world, Chile, Colombia, and Taiwan conducted national health surveys as health manpower surveys in the mid-1960s. Indonesia undertook an NHIS in 1972, Australia in 1977, Canada in 1978, Pakistan in 1982, China in 1989, Singapore, New Zealand, and Russia in 1992, Taiwan in 1993, and Brazil in 1998. Since the 1990s, most OECD and some other countries have organized NHISs.
Romania was the first European country to run an NHES in 1959 followed by Finland in 1965, Germany in 1984, the Netherlands and Slovakia in 1987, England and Latvia in 1991, Denmark and Norway in 1994, Scotland in 1995, and the Czech Republic and Ireland in 1998. Egypt ran an NHES in 1981, Iran and Thailand in 1991, and South Korea in 1998.
Since 2000, over 20 countries have set up NHESs or similar surveys in most regions of the world. From 2002, well over 100 countries have conducted surveys using the WHO STEPwise approach to non-communicable disease surveillance.
World fertility Surveys (WFS)
Although researchers have undertaken numerous small-scale health surveys from the mid-1950s onwards, many governments in LMICs did not have the resources or capacity to organize regular national health surveys. From 1972 for about a decade, the International Statistical Institute—with financial support from UNFPA, the US and UK—ran the World Fertility Survey (WFS). The WFS provided funding and a protocol for 42 developing countries and 20 developed countries to run comparable nationally representative, mostly household, surveys focussed on fertility, child mortality and family planning. The programme also built some national capacity to undertake surveys.
Demographic and Health Surveys (DHS)
The Demographic and Health Surveys (DHS) Program, supported by the US Agency for International Development (USAID), took over from the WFS in 1984. DHS still has a focus on demography, maternal, infant, child and reproductive health. But it has gradually added optional modules, such as: nutrition, HIV/AIDS prevalence, women’s empowerment, domestic violence and tobacco use. While it began as an interview survey, DHS has added anthropometric measurements and some biomarker data such as testing for anaemia, HIV infection, malaria, blood glucose, blood pressure, lead exposure, and immunity from vaccine-preventable diseases. DHS had by 2017 supported over 300 nationally representative and internationally comparable surveys in 44 countries in sub-Saharan Africa, 12 in North Africa/West Asia/Europe, 15 in South and Southeast Asia, two in Oceania and 15 in Latin America and the Caribbean.
Multiple Indicator Cluster Survey (MICS)
In 1995, the UN Children’s Fund (UNICEF) established the Multiple Indicator Cluster Survey (MICS), to measure indicators to monitor the 1990 World Summit for Children. These surveys collect information related to women and children covering, for example, health, education, child protection and water and sanitation, and measure some of the indicators that DHS measures. Standard MICSs record anthropometric data but not biomarkers. By 2015, MICS had supported about 300 household surveys in ten countries in East Asia and the Pacific, 41 in sub-Saharan Africa, 41 in Latin America, and 19 in Europe and Central Asia.
Living Standards Measurement Study (LSMS)
From 1980, the World Bank has supported the Living Standards Measurement Study (LSMS). LSMS is a multi-purpose household survey, not primarily a health survey, describing poverty and living standards with customized modules. Countries can choose to include a health module which covers health-related behaviour and utilization of health services, health expenditures, insurance expenditures and access to health services. By 2017, almost 40 countries had undertaken over 100 LSMSs.
STEPwise surveys (STEP)
The WHO established the STEPwise approach to surveillance (STEPS) in 2002 to support countries that do not already run NHESs to collect data about non-communicable diseases. The steps cumulatively increase the content of household surveys from: 1) asking people to self-report behavioural risk factors; 2) taking physical measurements and blood pressure; and 3) taking specimens to measure fasting blood glucose, total cholesterol levels, and urinary sodium. The programme emphasizes that these steps need not happen within the same survey but the protocols build capacity to include non-communicable diseases in household surveys, that is to become full-scale NHESs and publish data for cross-country comparisons.
Multiple other surveys
The above are national population health surveys. But governments conduct other national and local health surveys focused on specific conditions. Such surveys describe the prevalence, for example of HIV/AIDs, tuberculosis, mental health, oral health, and nutrition. WHO provides exhaustive guidance across all its offices and regions. It also coordinates multi-country household surveys such as the World Health Survey 2002-04, and conducts surveys of many specific conditions. WHO contributes to developing comparable indicators and survey instruments, and building capacity to organize national household surveys.
From sample to population
The specific objectives of a cross-sectional national household survey are usually simple. Primary objectives are to estimate multiple indicators either as proportions: for example, the proportion of the sample who were ex-smokers, or who attended a health facility in the last month; or as an average, for example, of body mass index, haemoglobin or blood sugar. A secondary objective may be to disaggregate these indicators by pre-selected population sub-groups, for example, province, sex, or age group. Investigators select the types of indicators to meet the overall aims of their surveys. They craft questions and measurements based on definitions and metadata used in previous surveys or from comparable external surveys, adapting for culture. EHES, for example, is attempting to standardise survey tools across European countries.
Investigators present results in multiple cross-tabulations providing disaggregated estimates of the indicators with measures of variation, or confidence intervals. Although researchers may subsequently seek to demonstrate causality, this is not the goal of a national household survey. Because the sample size is usually large and the survey carefully designed, researchers are free to make complex analyses of published survey data.
Principles of sampling
The basic principles of probability sampling are that: investigators select units randomly from a defined population; that the probability of a unit being selected is known (not necessarily equal); and that investigators can, therefore, estimate population characteristics with a confidence interval. The width of the confidence interval, or the precision of the estimate, depends on the sample size.
Organizers of household surveys choose sample size by balancing precision against the cost of alternative sampling methods.
- They can specify precision for any population characteristic and then choose the sample size using a standard sample size calculator (which requires some basic preliminary knowledge of the characteristic, such as its expected value and likely variation). But since NHIS/NHESs include many indicators, investigators use an overall sample size that provides adequate precision for all indicators.
- The more sub-groups by which investigators plan to disaggregate indicators, the larger the sample size they will require.
- A survey needs a larger sample size if it seeks to capture rare events such as maternal deaths. The sampling method affects both precision and cost. For example, for the same sample size: estimates based on random selection of clusters of households are less precise but cheaper than estimates based on random selection of individual households.
Stratified multi-stage cluster sampling
Most national household surveys use stratified multi-stage cluster sampling. Organizers select clusters of units at each of several stages. For example by 1) selecting administrative units from a list, or sampling frame, of all such units and 2) selecting households from a list, or sampling frame, of all households in each selected administrative unit.
Standard DHSs use two-stage cluster sampling with some stratification – adapted to country circumstances.
- Investigators start with a recent list of census enumeration areas (EAs) which they have grouped within strata such as urban/rural or province.
- During the first stage, they randomly select EAs within a stratum with their probability of being selected determined by their population size. Imagine each EA takes a lottery ticket for each household, so larger EAs have higher chance of holding a winning ticket.
- Next, enumerators visit the selected EAs to prepare lists of households and map their locations. They then select a fixed number of households (usually 20-30) from each selected EA; these are clusters of secondary sampling units. When enumerators visit a household, they include in their sample all members who meet their selection criteria, for example all women aged 15 to 49 years.
- Sometimes investigators deliberately over-represent specific population groups, for example minority ethnic groups. This is to ensure these groups are represented in sufficient numbers to make estimates about their characteristics.
Two preconditions for sampling are that the organizers can define what they mean by a household and that they have access to a comprehensive sampling frame from which to select them.
- For the first stage, the DHS recommends EAs but, if lists of EAs are not up-to-date or incomplete, DHS suggests using electoral zones, or lists of other administrative units such as villages and city zones.
- At the second stage, if there are no adequate lists, survey enumerators may demarcate and list households themselves but this increases the cost of the survey.
Another possibility, for each stage, that DHS suggests is to use a gridded high resolution satellite map with estimated numbers of structures for each grid.
Household surveys may unintentionally exclude migrant, nomadic and undocumented populations, and usually intentionally omit people living in residential homes or prisons. SANHANES defined a household as consisting of ‘a person, or a group of persons, who occupied a common dwelling (or part of it) for at least four days a week and who provided for themselves jointly with food and other essentials for living’. They classified a household member as a person who slept in the household for at least four nights a week.
Sample size
The sample size comprises the number of clusters (the number of EAs) and the number of households (and therefore individuals) in a cluster.
- Precision varies according to the degree of similarity of households within a cluster (their intra-cluster correlation). If there were no intra-cluster correlation, estimates would be as precise as if the households were drawn at random from the population in one stage, that is without clustering.
- The complexity of sampling logistics determines cost, for example the more EAs (clusters) selected, the more field work required.
According to the DHS program website, the number of households covered by a DHS ranges from 5,000 to 30,000 households. With an average cluster size of say 25, this means the number of clusters ranges from 200 to 1,200 clusters per country.
The sampling method affects population estimates in two ways.
- If the sampling scheme has over-represented segments of the population, then the estimated values for those segments must be weighted. For example, in some situations, the sampled proportions of women aged 15 to 49 years in each province may not reflect the actual proportion of women across provinces. To make estimates about the characteristics of the total population, investigators must weight the observed numbers of women in each province by the actual number of women in each province. DHS provides and explains these weights in their reports.
- The precision of the estimate is reflected by the design effect. The design effect is a ratio of the precision of the estimate for the design against the precision that an investigator would have obtained using simple random sampling. A design effect of 1 implies the sampling was as efficient as simple random sampling; the higher the design effect, the less efficient the design. Investigators guestimate a design effect to make the sample size calculation, that is they increase the sample size to reach the same precision as for simple random sampling. They can calculate the design effect for different population characteristics retrospectively from the data.
Total survey quality
The ability of a survey to represent its population depends on how investigators select the sample and on how they implement the design. The investigator needs to control for errors arising through the entire process from design to dissemination.
The concept of total survey error includes errors arising, for example, from choice of sampling frames, interviewees’ responses and interviewer measurements, and in data management. In principle, survey costs should be balanced not just against precision but also to minimize total survey error.
Beimer sets TSE in the broader context of a total survey quality framework within which survey organizations usually include:
- Accuracy (TSE).
- Credibility (as judged by the survey community).
- Comparability (demographic and across geographies and time).
- Usability/interpretability (well-documented with metadata).
- Relevance (data satisfy users’ needs).
- Accessibility (ease of users’ access to the data).
- Timeliness/punctuality (adheres to schedules).
- Completeness (data rich enough for analysis without too much burden on respondents).
- Coherence (possible to combine estimates with different sources).
The total survey quality framework provides dimensions for investigators to consider when designing and budgeting a study – that is to optimise survey quality.
Ethical considerations
All survey organizers must obtain approval from the national institutional review board (IRB). If a survey is designed and carried out with an international partner, it must comply with their IRB and governmental regulations. IRB approval requires informed consent and voluntary participation with assurance of total privacy and confidentiality.
It is good practice for investigators to publish their procedures for dealing with ethical issues, for example, the DHS describes on its webpage procedures for maintaining confidentiality and how to handle biomarker referral treatment and counselling.
Specific to taking biomarkers are: the safety of the health workers handling the samples; decisions about when, how and who to inform about specimen results; banking and using tissues; and providing information about follow-up and possible retesting. Pappas and Hyder explore ethical considerations for using biological and physiological markers in NHES in ‘less developed countries’ and conclude that ‘While ethical principles may be global, implementation of those principles must be carefully considered within local contexts in which the health examination survey takes place.’
Challenges and opportunities
Coordination
Ministries of health run many surveys related to specific health conditions. But governments usually integrate NHIS/NHESs within a national programme of socio-economic household surveys. These are usually coordinated by a national statistical office (NSO). The NSO agrees and supervises a long-term plan for household surveys across sectors. It attempts to harmonise the content, frequency and timing and therefore, the cost of the surveys. Centralising survey operations builds specialised capacity to design and analyse surveys. Coordination ensures: 1) use of the same national sampling framework; 2) shared technologies for data capture and management; and 3) skilled use of sophisticated statistical software.
Non-response
Governments that have operated national surveys for decades are experiencing a crisis in participant response. In the US, for example, there are striking declines in response rates for national household surveys of all kinds, including NHIS and NHANES. From 1999-2000 to 2012-14 rounds, the conditional response rate (the proportion of individuals who agree to participate from the participating households) to the NHANES interview fell from 81.9 to 71.0 per cent and for the examination from 76.3 to 68.5 per cent. Meyer et al. point out that there is a decline in the proportion of questions answered and in the accuracy of the answers.
Technology
Advances in technology continue to revolutionize every stage of a household survey. Investigators take as given the availability of comprehensive software and computers with sufficient processing power to manage large quantities of data.
- Census sampling frames are usually available with geo-positioning of households and, if not, investigators can use grid sampling.
- Computer-assisted personal interviewing with tablets allows real-time data capture and validation. Investigators can produce survey results quickly and visualize them using data dashboards.
- Once they have tidied and described the data with metadata, investigators can make the data publically available on the Internet for others to use. Assessment of data quality and choice of appropriate analyses, however, depend on technical human skills.
Data linkage
Sakshaug et al. point out that while declining response to household surveys, poor coverage and increasing costs threaten data quality, there is growing demand for data.(40) A solution to enhance the value of survey data, and to reduce survey content and cost, is to link an individual’s survey records with their administrative records. Linkage depends on: 1) the existence of consistent administrative records of good quality; 2) unique individual identifiers; 4) the technical capacity to establish the linkages; and 4) sufficient procedures to protect personal data.
The DAnish National COhort Study (DANCOS) combines individual data from the Danish NHIS with all Danish registers on health and welfare for the entire adult population. Linking data is possible because Danish law allows record linkage and provides data protection. In most other countries, as Sakshaug et al. point out, obtaining consent to link data is challenging and consent rates vary considerably in different contexts.(40) Danish law also allows linkage of records for non-responders making it possible to research reasons for non-response.
Disaggregation and sub-national estimation
The SDGs require countries to demonstrate that health services reach all segments of the population and improve their health status. Investigators must design surveys so that they can disaggregate indicators by population sub-groups and provide reports sub-nationally. This requires a larger sample size but not only that, two-stage cluster sampling does not permit estimation at the cluster level.
The cluster design is intended to produce estimates by strata (for example province) but intra-cluster correlation may bias estimates for individual EAs. Researchers can obtain small area estimates by modelling information from censuses with relationships determined from survey data. Researchers have used this approach for example to develop small area maps of nutrition in Tanzania. But a district cannot obtain reliable planning information from a national household survey. Langston et al. make the case for small population-based health surveys to inform district and sub-district management based on their experience in Rwanda.
Publishing and using data
It is becoming standard practice for governments to make anonymized data available for others to use. DANCOS, for example, makes NHIS data, along with registry data, available for researchers to explore. The US National Center for Health Statistics publishes data for all its major national health surveys. Survey organizers must analyse the data thoroughly and make indicators promptly available to decision makers. To maximize use of the data, they may synthesize indicators from several sources and develop estimates for time periods not covered by the survey.
Contents
Source chapter
The complete chapter on which we based this page:
Macfarlane S.B. (2019) National Household Surveys: Collecting Data Where People Live. In: Macfarlane S., AbouZahr C. (eds) The Palgrave Handbook of Global Health Data Methods for Policy and Practice. Palgrave Macmillan, London.
Additional resources
Survey methodology. This book by Robert Groves, Floyd Fowler, Mick Couper, James Lepkowski, Eleanor Singer and Roger Tourangeau provides “state-of-the-science presentation of essential survey methodology topics and techniques”.
Household sample surveys in developing and transition countries. Department of Economic and Social Affairs Statistics Division. New York: United Nations; 2005.
European Health Examination Survey. Guidelines for health examination surveys 2016
Human Science Research Council (HSRC). SANHANES: Health and Nutrition 2013
Bethlehem J. The rise of survey sampling. The Hague: Statistics Netherlands; 2009.
World Health Organization. New health examination survey to strengthen health information in the region.
ICF International. Demographic and Health Survey sampling and household listing manual. MEASURE DHS, Calverton, Maryland, U.S.A.: ICF International; 2012.
Biemer P. Public Opinion Quarterly. Total survey error. Design implementation and evaluation;74(5):817-48.
The DHS Program. Demographic and health survey interviewer’s manual 2006
The IHME Global Health Data Exchange inventory of surveys
Examples of sites that provide data from National Household Health Surveys:
- The Health Survey for England
- European Health Interview & Health Examination Surveys Database
- Canadian Community Health Survey
- The US National Health and Nutrition Examination Survey
- Demographic and Health Surveys (DHS) Program
- Multiple Indicator Cluster Survey