Clinical evaluation of the use of an mhealth intervention on quality of care provided by Community Health Workers in southwest Niger

BACKGROUND: Under the World Health Organization’s (WHO) integrated community case management (iCCM) Rapid Access Expansion Program (RAcE), World Vision Niger and Canada supported the Niger Ministry of Public Health to implement iCCM in four health districts in Niger in 2013. Community health workers (CHWs), known as Relais Communautaire (RCom), were deployed in their communities to diagnose and treat children under five years of age presenting with diarrhea, malaria and pneumonia and refer children with severe illness to the higher-level facilities. Two of the districts in southwest Niger piloted RCom using smartphones equipped with an application to support quality case management and provide good timely clinical data. A two-arm cluster randomized trial assessed the impact of use of the mHealth application mainly on quality of care (QoC), but also on motivation, retention and supervision. METHODS: A two-arm cluster randomized trial was conducted from March to October 2016 in Dosso and Doutchi districts. The intervention arm comprised 66 RCom equipped with a smartphone and 64 in the paper-based control arm. Trained expert clinicians observed each RCom assessing sick children presenting to them (264 in intervention group; 256 in control group), re-assessed each child on the same set of parameters, and made further observations regarding perceptions of motivation, retention, supervision, drug management and caregiver satisfaction. The primary outcome was a QoC score composed of diagnostic and treatment variables. Other factors were assessed by questionnaires. RESULTS: On average, the mHealth equipped RCom showed a 3.4% higher QoC score (mean difference of 0.83 points). They were more likely to ask about the main danger signs: convulsions (69.7% vs 50.4%, P < 0.001); incapacity to drink or eat (79.2% vs 59.4%, P < 0.001); vomiting (81.4% vs 69.9%, P < 0.01); and lethargy or unconsciousness (92.4% vs 84.8%, P < 0.01). Specifically, they consistently asked one more screening question. They were also significantly better at examining for swelling feet (40.2% vs 13.3%, P < 0.01) and advising caretakers on diarrhea, drug dosage and administration, and performed (though non-significantly) better when examining cough and breathing rates, referring all conditions, getting children to take prescribed treatments immediately and having caregivers understand treatment continuation. The control group was significantly better at diagnosing fast breathing, bloody diarrhea and severe acute malnutrition; and was somewhat better (non-significant) at treating fever and malaria. With treatment in general of the three diseases, there was no significant difference between the groups. On further inspection, 83% of the intervention group had a QoC score greater than 80% (25 out of 31), whereas only 67% of the control group had comparable performance. With respect to referrals, the intervention group performed better, mostly based on their better assessment of danger signs, with more correct (85% vs 29%) and fewer missed, plus a lower proportion of incorrect referrals, with the reverse being true for the controls (P = 0.012). There were no statistically significant differences in motivation, retention and supervision between the two groups, yet intervention RCom reported double the rate of no supervision in the last three months (31.8% vs 15.6%). CONCLUSIONS: Results suggest that use of the mHealth application led to modestly improved QoC through better assessment of the sick children and better referral decisions by RCom, but not to improvement in the actual treatment of malaria, pneumonia and diarrhea. Considering mHealth’s additional costs and logistics, questions around its viability remain. Further implementation could be improved by investing in RCom capacity building, building organization culture and strengthened supervision, all essential areas for improving any CHW program. In this real-world setting, in poor and remote communities in rural Niger, this study did not support the overall value of the mHealth intervention. Much was learned for any future mHealth interventions and scale-up.