Development and validation of an online tool for assessment of health care providers’ management of suspected malaria in an area, where transmission has been interrupted

BACKGROUND: The alertness and practice of health care providers (HCPs) in the correct management of suspected malaria (CMSM) (vigilance) is a central component of malaria surveillance following elimination, and it must be established before malaria elimination certification can be granted. This study was designed to develop and validate a rapid tool, Simulated Malaria Online Tool (SMOT), to evaluate HCPs’ practice in relation to the CMSM. METHODS: The study was conducted in East Azerbaijan Province, Islamic Republic of Iran, where no malaria transmission has been reported since 2005. An online tool presenting a suspected malaria case for detection of HCPs’ failures in recognition, diagnosis, treatment and reporting was developed based on literature review and expert opinion. A total of 360 HCPs were allocated to two groups. In one group their performance was tested by simulated patient (SP) methodology as gold standard, and one month later by the online tool to allow assessment of its sensitivity. In the other group, they were tested only by the online tool to allow assessment of any possible bias incurred by the exposure to SPs before the tool. RESULTS: The sensitivity of the tool was (98.7%; CI 93.6–99.3). The overall agreement and kappa statistics were 96.6% and 85.6%, respectively. In the group tested by both methods, the failure proportion by SP was 86.1% (CI 80.1–90.8) and by tool 87.2% (CI 81.4–91.7). In the other group, the tool found 85.6% (CI 79.5–90.3) failures. There were no significant differences in detecting failures within or between the groups. CONCLUSION: The SMOT tool not only showed high validity for detecting HCPs’ failures in relation to CMSM, but it had high rates of agreement with the real-world situation, where malaria transmission has been interrupted. The tool can be used by program managers to evaluate HCPs’ performance and identify sub-groups, whose malaria vigilance should be strengthened. It could also contribute to the evidence base for certification of malaria elimination, and to strengthening prevention of re-establishment of malaria transmission.