Can Community Health Workers Report Accurately on Births and Deaths? Results of Field Assessments in Ethiopia, Malawi and Mali

INTRODUCTION: Most low-income countries lack complete and accurate vital registration systems. As a result, measures of under-five mortality rates rely mostly on household surveys. In collaboration with partners in Ethiopia, Ghana, Malawi, and Mali, we assessed the completeness and accuracy of reporting of births and deaths by community-based health workers, and the accuracy of annualized under-five mortality rate estimates derived from these data. Here we report on results from Ethiopia, Malawi and Mali. METHOD: In all three countries, community health workers (CHWs) were trained, equipped and supported to report pregnancies, births and deaths within defined geographic areas over a period of at least fifteen months. In-country institutions collected these data every month. At each study site, we administered a full birth history (FBH) or full pregnancy history (FPH), to women of reproductive age via a census of households in Mali and via household surveys in Ethiopia and Malawi. Using these FBHs/FPHs as a validation data source, we assessed the completeness of the counts of births and deaths and the accuracy of under-five, infant, and neonatal mortality rates from the community-based method against the retrospective FBH/FPH for rolling twelve-month periods. For each method we calculated total cost, average annual cost per 1,000 population, and average cost per vital event reported. RESULTS: On average, CHWs submitted monthly vital event reports for over 95 percent of catchment areas in Ethiopia and Malawi, and for 100 percent of catchment areas in Mali. The completeness of vital events reporting by CHWs varied: we estimated that 30%-90% of annualized expected births (i.e. the number of births estimated using a FPH) were documented by CHWs and 22%-91% of annualized expected under-five deaths were documented by CHWs. Resulting annualized under-five mortality rates based on the CHW vital events reporting were, on average, under-estimated by 28% in Ethiopia, 32% in Malawi, and 9% in Mali relative to comparable FPHs. Costs per vital event reported ranged from $21 in Malawi to $149 in Mali. DISCUSSION: Our findings in Mali suggest that CHWs can collect complete and high-quality vital events data useful for monitoring annual changes in under-five mortality rates. Both the supervision of CHWs in Mali and the rigor of the associated field-based data quality checks were of a high standard, and the size of the pilot area in Mali was small (comprising of approximately 53,205 residents in 4,200 households). Hence, there are remaining questions about whether this level of vital events reporting completeness and data quality could be maintained if the approach was implemented at scale. Our experience in Malawi and Ethiopia suggests that, in some settings, establishing and maintaining the completeness and quality of vital events reporting by CHWs over time is challenging. In this sense, our evaluation in Mali falls closer to that of an efficacy study, whereas our evaluations in Ethiopia and Malawi are more akin to an effectiveness study. Our overall findings suggest that no one-size-fits-all approach will be successful in guaranteeing complete and accurate reporting of vital events by CHWs.