2159. Methodological Threats to the Standardized Infection and Standardized Antimicrobial Administration Ratios

BACKGROUND: Standardized ratios, such as the CDC’s Standardized Infection Ratio (SIR) and Standardized Antimicrobial Administration Ratio (SAAR), are used to assess hospital infection control and stewardship programs. While there has been a focus to improve adjustment factors in the prediction models for these ratios, there remain limitations to these methods in assessments of program effects. METHODS: We use a previously published mathematical model of an intensive care unit to simulate transmission of Staphylococcus aureus and the administration of antimicrobials to treat it. This approach allows for the calculation of an MRSA LabID SIR and Anti-MRSA adult ICU SAAR score with perfect adjustment, where the only difference between the simulated ICUs is due to random chance. We then evaluated the interpretations and statistical significance of these ratio measures as gauges of hospital program performance. RESULTS: Over a single year of 200 simulations, the models produced SIR/SAAR scores ranging from 0.47 to 1.73, with a median of 0.99, representing a considerable spread of scores obtained due to chance. The P-values measuring if these measures were different from 1.0 were significant in 86% of those facilities. Extending the simulation past one year exacerbated this tendency to over-identify scores as significant, and also showed that 53.5% of hospitals had improving (26%) or worsening (27.5%) of scores due to regression to the mean. This scenario could be falsely interpreted as the result of interventions put in place in response to their first year scores. CONCLUSION: Standardized ratio methods did not provide clear and actionable information, even with perfect adjustment. Statistically significant fluctuations occurred due to chance which could be mistakenly been attributed to actions taken by the hospital. Several methods, such as the use of percentiles rather than p-values, or presenting simulation-based projections of facility data, may help alleviate these problems. DISCLOSURES: All authors: No reported disclosures.