A framework to characterize the performance of early warning index alarm systems for patient monitoring

In [Scully, C.G., and Daluwatte, C., Evaluating performance of early warning indices to predict physiological instabilities. J Biomed Inform. 75 (2017) 14–21], a framework was presented to characterize the performance of warning indices to provide information on the 1) probability a critical health event will occur when a warning is given (analogous to positive predictive value) and 2) proportion of warned events to all events (analogous to sensitivity). This framework also provides information about the timeliness of the warnings with respect to event occurrence and the warning burden of the system. • In the current work, we provide information on how this framework can be used when cases without events are present in a dataset to examine the proportion of warned non-events to all non-events (analogous to false positive rate). • Information on steps to apply the method, software, data and results for the case study are also provided to enable implementation of the framework. • Application and extension of the framework is demonstrated and discussed by adding non-event records to our previous case study comparing two warning strategies to predict physiologic instabilities.