System Assessment and Evaluation

Knowing how systems perform under various scenarios is important. We need to examine with which level of sensitivity and how quickly they can detect an outbreak or recognize a bioterrorism attack. Knowing the error rate of a system can help make decisions regarding how much effort is needed to investigate an alarm. The performance of the algorithms for outbreak characterization determines the amount of information they provide (e.g., sets of affected individuals, the outbreak size, and disease spreading rate), which provide important input for response planning. Substantial costs can be incurred when developing or managing syndromic surveillance systems and investigating possible outbreaks based on the outputs of these systems (Reingold, 2003). For example, as reported in (Doroshenko et al., 2005), the annual cost of the NHS Direct Syndromic Surveillance System is about $280,000 and the usefulness of surveillance systems for early detection and response is yet to be established. Assessing the performance of surveillance systems is of significant importance for improving the efficacy of the investment in system development and management (Buehler et al., 2004).