Mdg's

Policy and Practice Tracking progress towards the Millennium Development Goals: reaching consensus on child mortality levels and trends Child Mortality Coordination Group a Abstract The increased attention to tracking progress towards the Millennium Development Goals (MDG), including Goal 4 of reducing child mortality, has drawn attention to a number of interrelated technical, operational and political challenges and to the underlying weaknesses of country health information systems upon which reliable monitoring depends. Assessments of child mortality published in 2005, for almost all low-income countries, are based on an extrapolation of the trends observed during the 1990s, rather than on the empirical data for more recent years. The validity of the extrapolation depends on the quality and quantity of the data used, and many countries lack suitable data. In the long run, it is hoped that vital registration or sample registration systems will be established to monitor vital events in a sustainable way. However, in the short run, tracking child mortality in high-mortality countries will continue to rely on household surveys and extrapolations of historical trends. This will require more collaborative efforts both to collect data through initiatives to strengthen health information systems at the country level, and to harmonize the estimation process. The latter objective requires the continued activity of a coordinating group of international agencies and academics that aims to produce transparent estimates — through the consistent application of an agreed-upon methodology — for monitoring at the international level. Keywords Child mortality/trends; Data collection/methods; Households; Development; Goals; Infant mortality/trends; Developing countries (source: MeSH, NLM). Mots clés Mortalité de l’enfant/orientations; Collecte données/méthodes; Ménages; Développement; Objectif; Mortalité nourrisson/ orientations; Pays en développement (source: MeSH, INSERM). Palabras clave Mortalidad en la niñez/tendencias; Recolección de datos/métodos; Hogares; Desarrollo; Metas; Mortalidad infantil/ tendencias; Países en desarrollo (fuente: DeCS, BIREME). Bulletin of the World Health Organization 2006;84:225-232. Voir page 231 le résumé en français. En la página 231 figura un resumen en español. .232 Introduction The Millennium Development Goals (MDGs) aim to reduce the striking inequalities between the rich and poor countries, and between the rich and poor populations within countries.1 A set of indicators has been selected to monitor progress towards achieving these goals. One of the most prominent goals for 2015 (MDG-4) is the reduction of child mortality by two-thirds from the level in 1990. Child mortality rates can be estimated using data from a variety of a sources, including population censuses, vital statistics systems and household surveys. Since the early 1990s, household surveys have become an increasingly important source of data for assessing and monitoring progress in improving child survival in low- and middle-income countries.2 Estimates of child mortality trends based on various data sources and different estimation methods are regularly published.2,3 Reports on progress in child survival by country, based on such estimates, are published annually and trends are extrapolated to assess whether or not the MDG is likely to be achieved.4–7 An important strength of the MDGs is the attention now being given to measurable indicators of progress and an institutionalized system of reporting. Increased commitment to tracking progress in child mortality has drawn attention to a number of interrelated technical, operational and political challenges and to the underlying weaknesses of health information systems in many countries, upon which reliable monitoring depends. This paper discusses the Members of the Child Mortality Coordination Group: Kenneth Hill, Harvard University, USA; Trevor Croft, Blancroft Research International, USA; Gareth Jones, Edilberto Loaiza, Attila Hancioglu, Neff Walker, Endre Bakka, Tessa Wardlaw, United Nations Children’s Fund, USA; John Wilmoth, François Pelletier, Cheryl Sawyer, Thomas Buettner, United Nations Population Division, USA; Emi Suzuki, Eduard Bos, The World Bank, USA; Mie Inoue, Kenji Shibuya, Ties Boerma, World Health Organization, Switzerland. Correspondence to Dr Kenji Shibuya, World Health Organization, CH-1211, Geneva 27 , Switzerland (email: shibuyak@who.int). Ref. No. 06-029744 (Submitted: 5 January 2006 – Final revised version received: 30 January 2006 – Accepted: 30 January 2006 ) Bulletin of the World Health Organization | March 2006, 84 (3) 225 Special Theme – Estimating Mortality Tracking progress on child mortality Child Mortality Coordination Group challenges in monitoring child mortality, and makes proposals for further improvements in monitoring progress towards the MDG on child mortality during the next decade. wards reducing child mortality is monitored annually with great accuracy. Data availability Reliable annual reporting on mortality — number of deaths by age, sex and cause — is possible only where there is comprehensive and accurate recording of births and deaths through a civil registration system. Such a system exists in only 72 countries representing around one third of the world’s population and primarily the high-income countries.13,14 For the other two thirds of the world, figures for child mortality are usually derived from estimates, based on the extrapolation of past trends or on modelling. The world health report 2005 — Make every mother and child count summarizes data availability and shows the extent to which estimates of child mortality in developing countries, for recent years, rely on extrapolations of past trends rather than on empirical data.13 Table 1 summarizes the median estimates of the U5MR for 2003 and data availability for WHO’s 192 Member States by quintiles of the U5MR. There are four primary sources of empirical data for the U5MR: vital statistics systems based on civil registration, sample registration systems, household surveys and censuses. Vital registration or sample registration systems provide numbers of deaths by age and sex obtained by direct reporting of individual deaths shortly after they occur. These are usually reported on an annual or biennial basis for a single Monitoring progress: technical challenges From a technical perspective, the simplicity and focus of the MDG indicators masks some significant challenges. In high-mortality countries, health information systems, having suffered from a history of underinvestment, are too weak and fragmented to routinely generate estimates.8,9 Furthermore, many healthrelated MDG indicators are difficult to measure and the trends to which the MDGs on health allude are impossible to monitor because of the lack of a suitable 1990 baseline.10 The mortality rate in under-5-year-olds (U5MR), in terms of which the target for MDG 4 is defined, is however one of the better MDG targets because extensive data are available and it is relatively easy to measure. Estimates of child mortality, together with expected trends, are published regularly, usually annually, by various international organizations.5–7,11,12 However, readers are generally not informed about the sources of data and methods of estimation. The underlying methodology is generally explained only in footnotes that sometimes include information on the availability of the empirical data that underlie the estimates. This gives readers the false impression that the progress to- point estimate. In the case of a survey or a census, the empirical data are based on retrospective data. Interviews, usually with the mother, provide information on the survival history of children in the household. Mortality information may be gathered for a specific period prior to the census or survey interview (although this approach is not recommended), through a full birth history that records the date of birth and, if appropriate, the age at death of each child, or through questions on the aggregate numbers of children ever born to the respondent and children still alive.2 It should be noted that a single survey generally provides multiple estimates for different points in time prior to the survey. Table 1 shows that the higher the estimated mortality the fewer the recent data points. Whereas vital registration data are available on an annual basis in countries with lower mortality, almost all estimates of child mortality in highmortality countries rely on extrapolations of past trends rather than empirically observed data.2 The global progress on MDGs clearly depends on the trends in child mortality in such countries. This lack of empirical data is partly the result of data collection being infrequent in high-mortality low-income countries. Fig. 1 shows the data collection effort in countries by the U5MR quintile from 1950 to 2000. The number of countries in the Q1 group with data collected for a given year approached 100% in the early 1980s. The number Table 1. Availability of data by under-five mortality quintile (Q), 1980–2003 (as of April 2005) Quintile based on under-five mortality rate (both sexes) in 2003a Q1 Q2 Q3 Q4 Q5 a No. of countries Under-five mortality rate Median Range No. of countries which have at least one data point 1980–89 VSS b 1990–99 VSS 38 36 34 19 2 b 2000–03 VSSb Survey/ censusc 38 30 22 13 1 0 0 5 6 5 Survey/ censusc 2 13 24 15 27 Survey/ censusc 0 10 29 32 37 Latest No. of available countries year without (average) data points during 2000–03 2001 2000 1999 1998 1997 1 8 14 20 33 39 38 38 38 39 5.5 14.4 31.4 77.7 168.6 3.0–7.9 7.9–21.4 21.4–40.8 40.8–118.3 118.3–283.5 38 35 29 19 1 Under-five mortality rate is the probability (expressed per 1000 live births) of a child born in a specific year dying before reaching five years of age according to current age-specific mortality rates. b The columns “VSS” (vital statistics with civil registration/sample registration system) show the number of countries which have at least one data point from either system, available at WHO. c The columns “Survey/census” show the number of countries which have at least one data point from either surveys or censuses with child mortality data available at WHO; only the most recent estimates are taken into account from each. Source: (13) 226 Bulletin of the World Health Organization | March 2006, 84 (3) Special Theme – Estimating Mortality Child Mortality Coordination Group Tracking progress on child mortality Fig. 1. Number of countries with data sources available at WHO, by under-five mortality quintile (Q), 1950–2000 (at April 2005) 40 of countries with data in the Q2–Q4 groups has increased since 1980 and has stabilized to a range between 90% for the Q2 group and slightly over 50% for the Q4 group. The Q5 group has shown only a gradual increase of countries with data to about 25%. The United Nations Children’s Fund (UNICEF) Multiple Indicator Cluster Surveys (MICS) contributed to the large increase in the data collected in 2000. The limited availability of empirical data on recent U5MR levels in highmortality countries is partly because information on child mortality in surveys and censuses is collected retrospectively and refers to a specific period prior to the census or survey interview.13 In general, the time-lag between data collection and publication is approximately two years for vital statistics, but tends to average four years or longer for household surveys because of this additional reference period. As of April 2005, the median of the mid-point of the most recent empirical child mortality data lies six years prior to 2004 for household surveys or censuses in 123 countries, whereas the vital registration data are now available, on average, up to 2001.11,16 Therefore, at best, the data used for estimating child mortality in 2003 in virtually all low- and many middle-income countries tell us what was happening in the late 1990s, the remainder being based 30 No. of countries 20 10 0 1950 Q1 Q2 1955 Q3 1960 Q4 1965 Q5 1970 1975 Year 1980 1985 1990 1995 2000 Note: Data sources are not included before 1980 for former socialist countries such as USSR, Czechoslovakia or Yugoslavia. Source: (15). WHO 06.30 largely on extrapolations using even older data. For most countries, whether or not they are on track to reach the MDGs is determined by an extrapolation of the trends observed during the 1990s with a baseline estimate for 1990. New data collection initiatives in the new millennium are beginning to provide data to increase our confidence in both the recent and the 1990 baseline estimates. Validity of extrapolation models The question then is how accurate are these estimates' Since the compilation of the U5MR for 2003, new results from the Demographic and Health Surveys (DHS) have become available for a dozen developing countries. A simple validation technique for the extrapolation model is to withhold these new data from the model parameterization, and to compare them with extrapolated mortality for the period 2000–03. This is obviously not sufficient to validate the entire set of extrapolations, but it does illustrate the performance of current methods in the countries where the vital statistics system is not complete. Figs. 2–4 show comparisons of U5MRs based on the standard extrapolation method 2 with those observed from the latest surveys in Guinea,17 Nigeria 18 and the United Republic of Tanzania,19 for the period 1980–2003. As expected, the results are not generalizable. For the majority of countries where consistent and frequently collected recent data are available, the model predictions are quite consistent with the observed values (e.g. Fig. 2). However, there is a large discrepancy between the estimates obtained by extrapolations and survey estimates in countries such as Nigeria (Fig. 3), where extrapolated figures showed a steady decline in child mortality, whereas recent Fig. 2. Comparisons between extrapolated and recently observed under-five mortality rates: Guinea Probability of dying by age 5 per 1000 live births 350 300 250 200 150 100 50 0 1980 1985 Fitted line 1990 Year 1995 2000 2005 Input data Observed values (Demographic and Health Survey 2005-preliminary) WHO 06.31 Note: observed values from (17). Bulletin of the World Health Organization | March 2006, 84 (3) 227 Special Theme – Estimating Mortality Tracking progress on child mortality Child Mortality Coordination Group Fig. 3. Comparisons between extrapolated and recently observed under-five mortality rates: Nigeria Probability of dying by age 5 per 1000 live births 350 300 250 200 150 100 50 0 1980 empirical data suggest that mortality was in fact higher and showing no sign of decline across the period. Conversely, in the United Republic of Tanzania, the extrapolated model suggested unchanged child mortality levels during the 1990s, although recent data seem to support the hypothesis that child mortality has declined steadily as shown in Fig. 4. The validity of extrapolations clearly depends on the quality and quantity of input data, and there is a need to strengthen the empirical basis for estimation of mortality in countries with fewer and less consistent data. The quality of input data used to generate child mortality estimates and to populate and validate models is improving. Better empirical data support better modelling efforts and vice versa. We need the two to complement one another. 1985 Fitted line 1990 Year 1995 2000 2005 Improved data collection methods Point estimates of mortality in children under five years old will obviously be improved if data are collected more frequently. However, the quality of the data is equally important. So what are the options' Given the difficulty of implementing a fully functioning vital registration system — the gold standard of mortality data — sample vital registration has been proposed as an interim solution to strengthen both the quantity and quality of mortality information.20 However, aside from successful implementation in India (and perhaps China), setting up a representative sample registration system requires a pre-existing registration system and substantial resources to sustain the activities. Although such efforts should be made as a part of a long-term commitment to strengthening country health information systems, it would be difficult for high-mortality countries to implement them in a relatively short time. Population censuses provide a good opportunity to gather data nationwide, but the interval between censuses — generally only once every 10 years — is too long for monitoring the U5MR. Thus, in the absence of a system that routinely provides reliable enumeration of vital events at the national level, household surveys remain the major tool for assessing recent child mortality levels. Currently, the most common approach used in household surveys is to include a full birth history from which 228 Input data Observed values (Demographic and Health Survey 2003) WHO 06.32 Note: observed values from (18). direct estimates of child mortality can be obtained. However, household surveys using direct estimation require intensive training and supervision of interviewers and are thus expensive. The estimates are also known to be prone to some systematic errors that may result in an artificially low estimate of U5MR for the most recent period. Household surveys also require relatively large sample sizes to provide statistically reliable estimates. The problem of wide confidence intervals is not simply that such estimates are imprecise. They may also lead to inappropriate interpretation of the figures. For example, using point estimates for child mortality may give the impression that the U5MRs are substantially different in different settings or at different times whereas, in fact, such differences may not be statistically significant because the confidence intervals overlap. The uncertainty ranges for the point estimates directly derived from surveys or vital registration systems are often presented by taking into account errors due to a finite sample size. On the other hand, the trend line fitted by the standard method does not necessarily encompass all uncertainties associated with estimates, because many data sources are affected by systematic errors (bias) as well as random errors (sampling). Therefore, the extrapolation line should be interpreted with great caution. Age patterns of child mortality MDG-4 deals with mortality in children aged under five years, primarily because it reflects overall child mortality and can be measured more accurately and reliably than, for example, the infant mortality rate. On the other hand, there is a growing interest in determining age-specific mortality among neonates, infants and children aged 1–4 years. First, empirical data suggest that U5MR is generally a good predictor of infant mortality except in some countries in western Africa. Second, as U5MR declines, the proportion of deaths of infants under one year of age, and particularly the proportion of deaths among neonates, increases. In fact, nearly 4 million neonatal deaths occur annually worldwide, now accounting for an estimated 37% of all deaths in children aged under five years.13,21 To achieve the MDG-4, the reduction of deaths in the first year of life is crucial, in particular the reduction of deaths among neonates.22 Data sources for estimating infant mortality and neonatal mortality are largely the same as for U5MR; they come primarily from household surveys and in some cases from vital statistics. However, the empirical basis for agespecific mortality rates is more limited.11 Better insight into the age patterns of mortality may enable some cause-specific patterns to be identified and, hence, lead Bulletin of the World Health Organization | March 2006, 84 (3) Special Theme – Estimating Mortality Child Mortality Coordination Group Tracking progress on child mortality Fig. 4. Comparisons between extrapolated and recently observed under-five mortality rates: United Republic of Tanzania Probability of dying by age 5 per 1000 live births 350 300 250 200 150 100 50 0 1980 to a better understanding of the epidemiological transition i.e. the systematic shift in cause-of-death patterns, within childhood mortality. The standard approach to developing extrapolated estimates of mortality in the first year of life is to derive infant mortality rates from extrapolated U5MRs using Coale–Demeny model life tables.2 However, because many countries (particularly in sub-Saharan Africa) no longer fit the model life tables, the approach needs to be modified, including updating the model life tables to reflect patterns observed in recent data, re-assessing the general relationship between infant mortality and U5MRs, and establishing a rule for how best to smooth out the “heaping” (the tendency to report deaths occurring at around 12 months as occurring at exactly 12 months) at age one. Further work on age-specific mortality rates, possibly coupled with an analysis of cause-ofdeath structure, should be a priority for assessing child mortality. 1985 Fitted line 1990 Year 1995 2000 2005 Input data Observed values (Demographic and Health Survey 2004-preliminary) WHO 06.33 Note: observed values from (19). Political challenges: towards consistent estimates across international agencies In the past, readers of annual reports published by different UN agencies were often confused by the inconsistencies between the figures given for some countries in the various reports. This was because, prior to 2004, the U5MR was estimated with some degree of independence by UNICEF, WHO, the World Bank and the United Nations Population Division, leading to each producing different figures (Table 2).5,7,12,13 Such a discrepancy is not surprising because the estimates would be substantially different depending on the data used and the extrapolation methodology selected. For example, more recent data are not necessarily shared by all agencies. Until recently, WHO put more weight on data from vital registration whereas other agencies weighted survey data more heavily. However, it is imperative that the international agencies disseminate internally consistent estimates on child mortality to enhance appropriate use of such figures in MDG monitoring and evaluation, and policy formulation. Indeed, there is an urgent need to develop a system through which the international agencies will speak with a single voice and produce estimates that are consistent across agencies. It is also essential that these estimates be of the highest possible quality, use transparent methods and are developed and reviewed by an independent expert group. As a step towards this goal, four UN agencies responsible for monitoring child mortality trends have established the Child Mortality Coordination Group. This group aims to conduct a critical review of current procedures used in each institution for compiling data and arriving at oint estimates; to discuss mechanisms for data distribution that would ensure that each organization has all available data (e.g. from DHS, MICS, the World Bank’s Living Standards Measurement Study, vital statistics, censuses and other sources) as soon as they become publicly available; and to harmonize and coordinate the estimation and projection methodology and results. Since inception, the Coordination Group has been actively working to harmonize and carry out joint estimations. Starting from empirical data and standard regression output for extrapolations,2 each country-by-country estimate is critically reviewed. If a discrepancy remains, more detailed assessments of the quality of input data and the appropriateness of statistical methods are carried out (Table 2). Through this process, UNICEF, WHO, the World Bank and the United Nations Population Division have been working towards jointly producing a consistent set of U5MRs by country. To this end, several reports have recently been published based on such mortality rates for the period 1990–2003.13,23–25 The Group has initiated joint activities on a regular basis to improve estimation through reviews of currently-used methods, and as means of improvement: creation of a common database; and more focus on country capacity-building and training to improve data availability and quality, including workshops for the regions with weaker health information systems. One of the major efforts of this Coordination Group is to set up a common database, including input data from agencies and institutions specializing in different sources of mortality data (censuses, vital statistics and household surveys), metadata and estimation processes. By doing so, the estimation process will be reproducible and eventually become publicly available to ensure its transparency and accuracy. UNICEF, WHO and partners in the Coordination Group have also begun to develop a common metric of uncertainty that can be used in future mortality estimates. The process builds on previous work by various groups and organizations and will lead to production of a set of guidelines and standards for calculating uncertainty associated with an estimate. These approaches will provide not only comparable mortality estimates, but also comparable data on the uncer229 Bulletin of the World Health Organization | March 2006, 84 (3) Special Theme – Estimating Mortality Tracking progress on child mortality Child Mortality Coordination Group Table 2. Consistency in under-five mortality rate: percentage difference in estimates compared with United Nations Children’s Fund (UNICEF) estimates Agency Year of publication No. of countriesa Relative difference compared with UNICEF estimates