Utilizing a mixed-methods approach to assess implementation fidelity of a group antenatal care trial in Rwanda

BACKGROUND: The Preterm Birth Initiative (PTBi)–Rwanda conducted a cluster randomized controlled trial to assess the impact of group antenatal care (group ANC) on preterm birth, using a group ANC approach adapted for the Rwanda setting, and implemented in 18 health centers. Previous research showed high overall fidelity of implementation, but lacked correlation with provider self-assessment and left unanswered questions. This study utilizes a mixed-methods approach to study the fidelity with which the health centers’ implementation followed the model specified for group ANC. METHODS: Implementation fidelity was measured using two tools, repeated Model Fidelity Assessments (MFAs) and Activity Reports (ARs) completed by Master Trainers, who visited each health center between 7 and 13 times (9 on average) to provide monitoring and training over 18 months between 2017 and 2019. Each center’s MFA item and overall scores were regressed (linear regression) on the time elapsed since the center’s start of implementation. The Activity Report (AR) is an open-ended template to record comments on implementation. For the qualitative analysis, the ARs from the times of each center’s highest and lowest MFA score were analyzed using thematic analysis. Coding was conducted via Dedoose, with two coders independently reviewing and coding transcripts, followed by joint consensus coding. RESULTS: A total of 160 MFA reports were included in the analysis. There was a significant positive association between elapsed time since a health center started implementation and greater implementation fidelity (as measured by MFA scores). In the qualitative AR analysis, Master Trainers identified key areas to improve fidelity of implementation, including: group ANC scheduling, preparing the room for group ANC sessions, provider capacity to co-facilitate group ANC, and facilitator knowledge and skills regarding group ANC content and process. These results reveal that monitoring visits are an important part of acquisition and fidelity of the “soft skills” required to effectively implement group ANC and provide an understanding of the elements that may have impacted fidelity as described by Master Trainers. CONCLUSIONS: For interventions like Group ANC, where “soft-skills” like group facilitation are important, we recommend continuous monitoring and mentoring throughout program implementation to strengthen these new skills, provide corrective feedback and guard against skills decay. We suggest the use of quantitative tools to provide direct measures of implementation fidelity over time and qualitative tools to gain a more complete understanding of what factors influence implementation fidelity. Identifying areas of implementation requiring additional support and mentoring may ensure effective translation of evidence-based interventions into real-world settings.