Development and Assessment of an Information Technology Intervention to Improve the Clarity of Radiologist Follow-up Recommendations

IMPORTANCE: It is challenging to ensure timely performance of radiologist-recommended additional imaging when radiologist recommendation language is incomplete or ambiguous. OBJECTIVE: To evaluate whether voluntary use of an information technology tool with forced structured entry of recommendation attributes was associated with improved completeness of recommendations for additional imaging over time. DESIGN, SETTING, AND PARTICIPANTS: This cohort study of imaging report data was performed at an academic quaternary care center in Boston, Massachusetts, and included consecutive adults with radiology examinations performed from September 12 to 13, 2019 (taxonomy validation), October 14 to 17, 2019 (before intervention), April 5 to 7, 2021 (1 week after intervention), and April 4 to 7, 2022 (1 year after intervention), with reports containing recommendations for additional imaging. A radiologist scored the 3 groups (preintervention group, 1-week postintervention group, and 1-year postintervention group) of 336 consecutive radiology reports (n = 1008) with recommendations for additional imaging. INTERVENTION: Final implementation on March 27, 2021, of a voluntary closed-loop communication tool embedded in radiologist clinical workflow that required structured entry of recommendation attributes. MAIN OUTCOMES AND MEASURES: The a priori primary outcome was completeness of recommendations for additional imaging, defined in a taxonomy created by a multidisciplinary expert panel. To validate the taxonomy, 2 radiologists independently reviewed and scored language attributes as present or absent in 247 consecutive radiology reports containing recommendations for additional imaging. Agreement was assessed with Cohen κ. Recommendation completeness over time was compared with with 1-sided Fisher exact tests and significance set at P < .05. RESULTS: Radiology-related information for consecutive radiology reports from the 4 time periods was collected from the radiology department data warehouse, which does not include data on patient demographic characteristics or other nonimaging patient medical information. The panel defined 5 recommendation language attributes: complete (contains imaging modality, time frame, and rationale), ambiguous (equivocal, vague language), conditional (qualifying language), multiplicity (multiple options), and alternate (language favoring a different examination to that ordered). Two radiologists had more than 90% agreement (κ > 0.8) for these attributes. Completeness with use of the tool increased more than 3-fold, from 14% (46 of 336) before the intervention to 46% (153 of 336) (P < .001) 1 year after intervention; completeness in the corresponding free-text report language increased from 14% (46 of 336) before the intervention to 25% (85 of 336) (P < .001) 1 year after the intervention. CONCLUSIONS AND RELEVANCE: This study suggests that supplementing free-text dictation with voluntary use of a structured entry tool was associated with improved completeness of radiologist recommendations for additional imaging as assessed by an internally validated taxonomy. Future research is needed to assess the association with timely performance of clinically necessary recommendations and diagnostic errors. The taxonomy can be used to evaluate and build interventions to modify radiologist reporting behaviors.