Development and evaluation of a pediatric hospital medicine board review course

BACKGROUND: The American Board of Medical Specialties recognized Pediatric Hospital Medicine (PHM) for subspecialty certification in 2016, with the first certification exam in 2019. To address the need for exam preparatory materials, we designed and evaluated a novel PHM board review course that was offered both in-person and online. METHODS: Course content was based on the American Board of Pediatrics (ABP) PHM certifying exam outline. Course objectives were developed from published PHM core competencies and the 2012 ABP general pediatrics content objectives. National experts served as faculty, presenting didactic sessions, and contributing to a question bank for high-yield review. For program evaluation, we applied the Kirkpatrick Model, evaluating estimated exam pass rates (Level 4), participant learning (Level 2) via post-presentation practice questions, and participants’ ratings of presenters (via five-point Likert scale) and satisfaction (Level 1). RESULTS: There were 112 in-person and 144 online participants with estimated pass rates of 89 and 93%, respectively. The mean correct response for the post-presentation knowledge questions was 84%. Faculty effectiveness ratings ranged from 3.81 to 4.96 (median score 4.60). Strengths included the pace of the course, question bank, and printed syllabus. Suggestions for improvement included question bank expansion, focus on “testable” points rather than general information, and challenges with long days of didactic presentations. CONCLUSIONS: This novel PHM board review course demonstrated effectiveness. Hospitalists preferred focused “testable” information, an active learning environment, and a robust question bank. Future preparatory courses should consider including more opportunities for practice questions, focused content review, and learner engagement. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-022-03862-1.