Measuring learning gain during a one-day introductory bronchoscopy course

BACKGROUND: Rigorous assessment of medical knowledge and technical skill inspires learning, reinforces confidence, and reassures the public. Identifying curricular effectiveness using objective measures of learning is therefore crucial for competency-oriented program development in a learner-centric educational environment. The aim of this study was to determine whether various measures of learning, including class-average normalized gain, can be used to assess the effectiveness of a one-day introductory bronchoscopy course curriculum. METHODS: We conducted a quasi-experimental one-group pre-test/post-test study at the University of California, Irvine. The group comprised 24 first-year pulmonary and critical care trainees from eight training institutions in southern California. Class-average normalized gain, single-student normalized gain, absolute gain, and relative gain were used as objective measures of cognitive knowledge and bronchoscopy technical skill learning. A class-average normalized gain of 30% was used to determine curricular effectiveness. Perceived educational value using Likert-scale surveys and post-course questionnaires was determined during and 3 months after course participation. RESULTS: Mean test scores of cognitive knowledge improved significantly from 48 to 66% (p = 0.043). Absolute gain for the class was 18%, relative gain was 37%, class average normalized gain 〈g〉 was 34%, and the average of the single-student normalized gains g(ave) was 29%. Mean test scores of technical skill improved significantly from 43 to 77% (p = 0.017). Absolute gain was 34%, relative gain was 78%, class average normalized gain 〈g〉 was 60%, and the average of the single-student normalized gains g(ave) was 59%. Statistically significant improvements in absolute gain were noted in all five elements of technical skill (p < 0.05). Likert-scale surveys, questionnaires, and surveys demonstrated strong perceived educational value. CONCLUSION: The effectiveness of a one-day introductory bronchoscopy curriculum was demonstrated using a pre-test/post-test model with calculation of normalized gain and related metrics. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s00464-010-1161-4) contains supplementary material, which is available to authorized users.