Countdown to 2015 country case studies: systematic tools to address the “black box” of health systems and policy assessment

BACKGROUND: Evaluating health systems and policy (HSP) change and implementation is critical in understanding reproductive, maternal, newborn and child health (RMNCH) progress within and across countries. Whilst data for health outcomes, coverage and equity have advanced in the last decade, comparable analyses of HSP changes are lacking. We present a set of novel tools developed by Countdown to 2015 (Countdown) to systematically analyse and describe HSP change for RMNCH indicators, enabling multi-country comparisons. METHODS: International experts worked with eight country teams to develop HSP tools via mixed methods. These tools assess RMNCH change over time (e.g. 1990–2015) and include: (i) Policy and Programme Timeline Tool (depicting change according to level of policy); (ii) Health Policy Tracer Indicators Dashboard (showing 11 selected RMNCH policies over time); (iii) Health Systems Tracer Indicators Dashboard (showing four selected systems indicators over time); and (iv) Programme implementation assessment. To illustrate these tools, we present results from Tanzania and Peru, two of eight Countdown case studies. RESULTS: The Policy and Programme Timeline tool shows that Tanzania’s RMNCH environment is complex, with increased funding and programmes for child survival, particularly primary-care implementation. Maternal health was prioritised since mid-1990s, yet with variable programme implementation, mainly targeting facilities. Newborn health only received attention since 2005, yet is rapidly scaling-up interventions at facility- and community-levels. Reproductive health lost momentum, with re-investment since 2010. Contrastingly, Peru moved from standalone to integrated RMNCH programme implementation, combined with multi-sectoral, anti-poverty strategies. The HSP Tracer Indicators Dashboards show that Peru has adopted nine of 11 policy tracer indicators and Tanzania has adopted seven. Peru costed national RMNCH plans pre-2000, whereas Tanzania developed a national RMNCH plan in 2006 but only costed the reproductive health component. Both countries included all lifesaving RMNCH commodities on their essential medicines lists. Peru has twice the health worker density of Tanzania (15.4 vs. 7.1/10,000 population, respectively), although both are below the 22.8 WHO minimum threshold. CONCLUSIONS: These are the first HSP tools using mixed methods to systematically analyse and describe RMNCH changes within and across countries, important in informing accelerated progress for ending preventable maternal, newborn and child mortality in the post-2015 era. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12889-016-3402-5) contains supplementary material, which is available to authorised users.