Utility of System Generated Syndromic Surveillance Alerts to Detect Reportable Disease Outbreaks

OBJECTIVE: In light of recent outbreaks of pertussis, the ability of Florida Department of Health’s (FDOH) Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE-FL) to detect emergent disease outbreaks was examined. Through a partnership with the Johns Hopkins University Applied Physics Laboratory (JHU/APL), FDOH developed a syndromic surveillance system, ESSENCE-FL, with the capacity to monitor reportable disease case data from Merlin, the FDOH Bureau of Epidemiology’s secure webbased reporting and epidemiologic analysis system for reportable diseases. The purpose of this evaluation is to determine the utility and application of ESSENCE-FL system generated disease warnings and alerts originally designed for use with emergency department chief complaint data to reportable disease data to assist in timely detection of outbreaks in promotion of appropriate response and control measures. INTRODUCTION: Reportable disease case data are entered into Merlin by all 67 county health departments in Florida and assigned confirmed, probable, or suspect case status. De-identified reportable disease data from Merlin are sent to ESSENCE-FL once an hour for further analysis and visualization using tools in the surveillance system. These data are available for ad hoc queries, allowing users to monitor disease trends, observe unusual changes in disease activity, and to provide timely situational awareness of emerging events. Based on system algorithms, reportable disease case weekly tallies are assigned an awareness status of increasing intensity from normal to an alert category. These statuses are constantly scrutinized by county and state level epidemiologists to guide disease control efforts in a timely manner, but may not signify definitive actionable information. METHODS: Within the ESSENCE-FL query portal, the Merlin Reportable Diseases Data Source was selected with a weekly time resolution by event date. Case Classification included all confirmed, probable and suspect cases, reported and not yet reported, during the time period of week 35, 2011, to week 35, 2012. The ESSENCE Weighted Moving Average (EWMA 1.2) detector was used to classify weekly counts as either of normal, warning or alert status based on previous weeks’ counts, indicating the possibility of an emerging outbreak. These weekly statuses were then compared with outbreaks reported in Merlin’s fully integrated outbreak reporting system and with outbreak reports submitted to EpiCom, Florida’s EpiX or health alert network. An ESSENCE-FL generated warning or alert was considered valid if a corresponding outbreak of 2 or more epi-linked pertussis cases were reported in either Merlin’s outbreak module or in EpiCom. For the sake of brevity in this abstract, the analysis of pertussis is presented, while other reportable disease conditions of immediate interest will be presented at the conference. RESULTS: Examination of 494 pertussis cases reported from September 2011 to September 2012 showed that of 53 weeks, 38 weeks contained normal case counts, 11 weeks generated warnings, and 4 weeks produced alerts. The number of warnings that corresponded to actual outbreaks was 6 of 11, whereas 2 of the 4 alerts matched reported outbreaks. Of the remaining 38 weeks, 12 had outbreaks reported with no warning or alert generated by ESSENCE-FL. When comparing confirmed outbreak status with ESSENCE-FL weekly data count status, warning/alert versus normal, it was found that the sensitivity of ESSENCE-FL to detect a true outbreak was 40.0% while the specificity was 78.8%. This comparison generated a positive-predictive value of 53.3% and a negative predictive value of 68.4%. CONCLUSIONS: The ability of ESSENCE-FL to act as a first alert system for emerging disease events using Merlin reportable disease data should be considered with constraint. While warnings or alerts about potential pertussis outbreaks were generated correctly about half the time, the nearly one-third of reported outbreaks with no warning or alert makes the utility of the alerts questionable as far as initiating immediate action without prior verification of the alert. Florida does not currently have a requirement for centrally documenting all outbreaks, so it is likely that outbreaks occurred but were not recorded, precluding verification of all outbreaks.