2198. Weak Interobserver Reliability in the Clinical Diagnosis of Pneumonia Among Infectious Disease Trained Physicians

BACKGROUND: Pneumonia remains a common global cause of death. Varied definitions of pneumonia rely on clinical features, imaging, and microbiological data. The Center for Diseases Control/National Healthcare Surveillance Network (CDC/NHSN) definition is used to study population-level trends. Prior studies have revealed discordance in components of the definition, yet reliability of overall clinical diagnosis has not been evaluated, nor has it been compared with the surveillance definitions. This study was designed to determine the overall concordance in the diagnosis of pneumonia by Infectious Diseases (ID) clinicians and the agreement between this and the surveillance definition. We then set out to determine which clinical features were weighted most heavily in provider decision-making. METHODS: Using an iterative approach with input from ID and Pulmonary Medicine physicians, we designed and refined an adjudication tool for diagnosis of pneumonia that consolidates clinical features, laboratory data, and imaging. Cases were analyzed by strict CDC/NHSN surveillance criteria and adjudicated independently by ID-trained physicians based on overall clinical opinion. Kappa coefficient (κ) was used to determine diagnostic reliability, and a random forest model was used to identify clinical factors most heavily weighted by physicians. RESULTS: Twenty-eight cases were adjudicated by three ID-trained physicians. Overall, interrater agreement was low (κ = 0.438). In comparing providers’ clinical adjudication with CDC/NHSN criteria, agreement was even worse (κ range 0.125 to 0.378). Among specific clinical features, positive culture growth strongly informed clinician diagnosis of pneumonia, while chest imaging did not play a significant role. CONCLUSION: Overall agreement in the clinical diagnosis of pneumonia is poor, even among ID-trained physicians. Culture results more strongly inform clinician decision-making than does chest imaging. The surveillance definition used by the CDC/NHSN has only weak agreement with in-practice clinical assessment. These results underscore the need for more precise diagnostic tools in cases of suspected pneumonia. [Image: see text] [Image: see text] DISCLOSURES: All authors: No reported disclosures.