A mixed methods evaluation of a digital antenatal quality improvement intervention in Nepal

BACKGROUND: Quality antenatal care plays an important role in reducing maternal and perinatal mortality. Electronic decision support systems (EDSS) aim to promote adherence to evidence-based practices. The evaluation aimed to assess the effectiveness of the intervention on improving quality of antenatal care (ANC) and understanding of implementation process and intervention effects. METHODS: The study was conducted in primary health care settings in Nepal. Theory of change framework was used for mixed-method evaluation. Quantitative and qualitative data were collected at multiple time points. Data were integrated using concurrent triangulation to develop explanations about the implementation process and the observed effects. RESULTS: Guided by our theory of change, we identified eleven themes that help to understand how the EDSS was used in the real context, and why the EDSS was not much effective in bringing the desired changes that was required to improve quality of ANC. EDSS appear to slightly influence ANC staff behavior, however, training ANC staff in using the EDSS did not necessarily lead to confidence in using it and ANC staff used the EDSS as recordkeeping system. We found mixed evidence on ANC staff believe that the EDSS offers benefits to their work. Inflexibility of EDSS during ANC consultation caused difficulties adapting to women's need and challenging to integrate the EDSS in ANC staffs’ workflow and daily activities. Inadequate linkage of the EDSS with existing health systems impaired the EDSS guided clinical performance. CONCLUSIONS: There is some indication that the EDSS have the potential to increase the quality of care. However, the EDDS alone are not enough to bring the adequate change in ANC staff delivery of quality care. Active participation of government and grassroot level understanding of health system would be essential considering conducting further research taking in account the resources, and other factors that hinders the EDSS utilization. KEY MESSAGES: • Concurrent triangulation helps to understand that digital health intervention is not enough to bring intended changes in the quality-of-service delivery at health facility. • Understanding of context and ground reality is essential to improve the quality of care and enhance the performance of healthcare providers.