Evaluating Patient-Specific Antibiograms

BACKGROUND: Generating antibiograms has been standard practice in many hospitals for decades, and many guidelines recommend updating the data on a yearly basis. While effective at summarizing a hospital’s susceptibility data across all patients, likelihoods of susceptibility are not the same for all patients. Traditional antibiograms do not account for the numerous patient-specific factors (age, length of stay, diagnoses, previous antibiotic exposure and susceptibility results, etc.) that are known to influence a patient’s risk of having a resistant organism. We have built models that use patient-specific information to generate patient-specific antibiograms. Three methods for evaluating a model’s performance are presented. METHODS: Calculating Brier scores is the commonly used method to evaluate the performance of predictions that give the percentage likelihood of a binary event. We used Brier scores and two new methods we created (dispersion scores and susceptibility histograms) to evaluate patient-specific antibiograms we built. As an example of the methods, we present data from Mar-Jul 2016 on 3012 E. coli isolates and their susceptibility to levofloxacin. RESULTS: In the standard institutional antibiogram for the time period 75% of E. coli isolates were susceptible to levofloxacin. Our patient-specific antibiogram had a dispersion score of 73 (100 representing perfect dispersion, the standard antibiogram has a dispersion score of 0). In the susceptibility histogram the patient-specific antibiogram showed a predicted susceptibility of 90% or greater for 1716 (57%) of the 3012 isolates, and the actual susceptibility for that group was 96%. It also showed a predicted susceptibility less than 10% for 438 (15%) of the 3012 isolates, and the actual susceptibility for that group was 1%. Brier scores were 61% better for the patient-specific antibiogram (Brier = 0.24) than for the standard antibiogram (Brier = 0.62). CONCLUSION: By these methods patient-specific antibiograms are better than standard antibiograms at providing numerical predictions of the likely susceptibility of E. coli to levofloxacin. The methods can be used to guide and evaluate improvement to the models that generate patient-specific antibiograms. DISCLOSURES: S. Overly, Teqqa, LLC: Employee, Salary. S. Hayes, Teqqa, LLC: Employee, Salary. J. Mehta, Teqqa, LLC: Employee, Salary. D. Peterson, Teqqa, LLC: Employee, Salary