The first step is recognizing there is a problem: a methodology for adjusting for variability in disease severity when estimating clinician performance

BACKGROUND: Adoption of innovations in the field of medicine is frequently hindered by a failure to recognize the condition targeted by the innovation. This is particularly true in cases where recognition requires integration of patient information from different sources, or where disease presentation can be heterogeneous and the recognition step may be easier for some patients than for others. METHODS: We propose a general data-driven metric for clinician recognition that accounts for the variability in patient disease severity and for institutional standards. As a case study, we evaluate the ventilatory management of 362 patients with acute respiratory distress syndrome (ARDS) at a large academic hospital, because clinician recognition of ARDS has been identified as a major barrier to adoption to evidence-based ventilatory management. We calculate our metric for the 48 critical care physicians caring for these patients and examine the relationships between differences in ARDS recognition performance from overall institutional levels and provider characteristics such as demographics, social network position, and self-reported barriers and opinions. RESULTS: Our metric was found to be robust to patient characteristics previously demonstrated to affect ARDS recognition, such as disease severity and patient height. Training background was the only factor in this study that showed an association with physician recognition. Pulmonary and critical care medicine (PCCM) training was associated with higher recognition (β = 0.63, 95% confidence interval 0.46–0.80, p < 7 × 10(− 5)). Non-PCCM physicians recognized ARDS cases less frequently and expressed greater satisfaction with the ability to get the information needed for making an ARDS diagnosis (p < 5 × 10(− 4)), suggesting that lower performing clinicians may be less aware of institutional barriers. CONCLUSIONS: We present a data-driven metric of clinician disease recognition that accounts for variability in patient disease severity and for institutional standards. Using this metric, we identify two unique physician populations with different intervention needs. One population consistently recognizes ARDS and reports barriers vs one does not and reports fewer barriers. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01543-7.