Statistical insights for crude-rate-based operational measures of misdiagnosis-related harms

In longitudinal event data, a crude rate is a simple quantification of the event rate, defined as the number of events during an evaluation window, divided by the at-risk population size at the beginning or mid-time point of that window. The crude rate recently received revitalizing interest from medical researchers who aimed to improve measurement of misdiagnosis-related harms using administrative or billing data by tracking unexpected adverse events following a “benign” diagnosis. The simplicity of these measures makes them attractive for implementation and routine operational monitoring at hospital or health system level. However, relevant statistical inference procedures have not been systematically summarized. Moreover, it is unclear to what extent the temporal changes of the at-risk population size would bias analyses and affect important conclusions concerning misdiagnosis-related harms. In this article, we present statistical inference tools for using crude-rate based harm measures, as well as formulas and simulation results that quantify the deviation of such measures from those based on the more sophisticated Nelson-Aalen estimator. Moreover, we present results for a generalized multibin version of the crude rate, for which the usual crude rate is a single-bin special case. The generalized multibin crude rate is more straightforward to compute than the Nelson-Aalen estimator and can reduce potential biases of the single-bin crude rate. For studies that seek to use multibin measures, we provide simulations to guide the choice regarding number of bins. We further bolster these results using a worked example of stroke after “benign” dizziness from a large data set.