Cargando…
Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency
Introduction Faculty are required to assess the development of residents using educational milestones. This descriptive study examined the end-of-rotation milestone-based evaluations of anesthesiology residents by rotation faculty directors. The goals were to measure: (1) how many of the 25 Accredit...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Cureus
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6207168/ https://www.ncbi.nlm.nih.gov/pubmed/30410826 http://dx.doi.org/10.7759/cureus.3200 |
_version_ | 1783366484196589568 |
---|---|
author | Chemtob, Chloe M Tanaka, Pedro Keil, Martin Macario, Alex |
author_facet | Chemtob, Chloe M Tanaka, Pedro Keil, Martin Macario, Alex |
author_sort | Chemtob, Chloe M |
collection | PubMed |
description | Introduction Faculty are required to assess the development of residents using educational milestones. This descriptive study examined the end-of-rotation milestone-based evaluations of anesthesiology residents by rotation faculty directors. The goals were to measure: (1) how many of the 25 Accreditation Council for Graduate Medical Education (ACGME) anesthesiology subcompetency milestones were included in each of the residency’s rotations evaluations, (2) the percentage of evaluations sent to the rotation director that were actually completed by the director, (3) the length of time between the end of the residents' rotations and completion of the evaluations, (4) the frequency of straight line scoring, defined as the resident receiving the same milestone level score for all subcompetencies on the evaluation, and (5) how often a resident received a score below a Level 4 in at least one subcompetency in the three months prior to graduating. Methods In 2013, the directors for each the 24 anesthesia rotations in the Stanford University School of Medicine Anesthesiology Residency Program created new milestone-based evaluations to be used at the end of rotations to evaluate residents. The directors selected the subcompetencies from the list released by the ACGME that were most appropriate for their rotation. End-of-rotation evaluations for the post-graduate year (PGY)-2 to PGY-4 from July 1, 2014 to June 30, 2017 were retrospectively analyzed for a sample of 10 residents randomly selected from 22 residents in the graduating class. Results The mean number of subcompetencies evaluated by each of the 24 rotations in the residency equaled 17.88 (standard deviation (SD): 3.39, range 10–24, median 18.5) from the available possible total of 25 subcompetencies. Three subcompetencies (medical knowledge, communication with patients and families, and coordination of patient care within the healthcare system) were included in the evaluation instruments of all 24 rotations. The three least frequently listed subcompetencies were: “acute, chronic, and cancer-related pain consultation/management” (25% of rotations had this on the end-of-rotation evaluation), “triage and management of critically ill patient in non-operative setting” (33%), and “education of patient, families, students, residents, and others” (38%). Overall, 418 end-of-rotation evaluations were issued and 341 (82%) completed, with 63% completed within one month, 22% between month one and two, and 15% after two months. The frequency of straight line scoring varied, from never occurring (0%) in three rotations to always occurring (100%) in two rotations, with an overall average of 51% (SD: 33%). Sixty-one percent of straight line scoring corresponded to the residents’ postgraduate year whereby, for example, a post-graduate year two resident received an ACGME Level 2 proficiency for all subcompetencies. Thirty-one percent of the straight line scoring was higher than the resident’s year of training (e.g., a PGY-2 received Level 3 or higher for all the subcompetencies). The remaining 7% of straight line scoring was below the expected level for the year of training. Three of seven residents had at least one subcompetency rated as below a Level 4 on one of the evaluations during the three months prior to finishing residency. Conclusion Formal analysis of a residency program’s end-of-rotation milestone evaluations may uncover opportunities to improve competency-based evaluations. |
format | Online Article Text |
id | pubmed-6207168 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | Cureus |
record_format | MEDLINE/PubMed |
spelling | pubmed-62071682018-11-08 Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency Chemtob, Chloe M Tanaka, Pedro Keil, Martin Macario, Alex Cureus Anesthesiology Introduction Faculty are required to assess the development of residents using educational milestones. This descriptive study examined the end-of-rotation milestone-based evaluations of anesthesiology residents by rotation faculty directors. The goals were to measure: (1) how many of the 25 Accreditation Council for Graduate Medical Education (ACGME) anesthesiology subcompetency milestones were included in each of the residency’s rotations evaluations, (2) the percentage of evaluations sent to the rotation director that were actually completed by the director, (3) the length of time between the end of the residents' rotations and completion of the evaluations, (4) the frequency of straight line scoring, defined as the resident receiving the same milestone level score for all subcompetencies on the evaluation, and (5) how often a resident received a score below a Level 4 in at least one subcompetency in the three months prior to graduating. Methods In 2013, the directors for each the 24 anesthesia rotations in the Stanford University School of Medicine Anesthesiology Residency Program created new milestone-based evaluations to be used at the end of rotations to evaluate residents. The directors selected the subcompetencies from the list released by the ACGME that were most appropriate for their rotation. End-of-rotation evaluations for the post-graduate year (PGY)-2 to PGY-4 from July 1, 2014 to June 30, 2017 were retrospectively analyzed for a sample of 10 residents randomly selected from 22 residents in the graduating class. Results The mean number of subcompetencies evaluated by each of the 24 rotations in the residency equaled 17.88 (standard deviation (SD): 3.39, range 10–24, median 18.5) from the available possible total of 25 subcompetencies. Three subcompetencies (medical knowledge, communication with patients and families, and coordination of patient care within the healthcare system) were included in the evaluation instruments of all 24 rotations. The three least frequently listed subcompetencies were: “acute, chronic, and cancer-related pain consultation/management” (25% of rotations had this on the end-of-rotation evaluation), “triage and management of critically ill patient in non-operative setting” (33%), and “education of patient, families, students, residents, and others” (38%). Overall, 418 end-of-rotation evaluations were issued and 341 (82%) completed, with 63% completed within one month, 22% between month one and two, and 15% after two months. The frequency of straight line scoring varied, from never occurring (0%) in three rotations to always occurring (100%) in two rotations, with an overall average of 51% (SD: 33%). Sixty-one percent of straight line scoring corresponded to the residents’ postgraduate year whereby, for example, a post-graduate year two resident received an ACGME Level 2 proficiency for all subcompetencies. Thirty-one percent of the straight line scoring was higher than the resident’s year of training (e.g., a PGY-2 received Level 3 or higher for all the subcompetencies). The remaining 7% of straight line scoring was below the expected level for the year of training. Three of seven residents had at least one subcompetency rated as below a Level 4 on one of the evaluations during the three months prior to finishing residency. Conclusion Formal analysis of a residency program’s end-of-rotation milestone evaluations may uncover opportunities to improve competency-based evaluations. Cureus 2018-08-24 /pmc/articles/PMC6207168/ /pubmed/30410826 http://dx.doi.org/10.7759/cureus.3200 Text en Copyright © 2018, Chemtob et al. http://creativecommons.org/licenses/by/3.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Anesthesiology Chemtob, Chloe M Tanaka, Pedro Keil, Martin Macario, Alex Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title | Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title_full | Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title_fullStr | Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title_full_unstemmed | Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title_short | Analysis of Milestone-based End-of-rotation Evaluations for Ten Residents Completing a Three-year Anesthesiology Residency |
title_sort | analysis of milestone-based end-of-rotation evaluations for ten residents completing a three-year anesthesiology residency |
topic | Anesthesiology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6207168/ https://www.ncbi.nlm.nih.gov/pubmed/30410826 http://dx.doi.org/10.7759/cureus.3200 |
work_keys_str_mv | AT chemtobchloem analysisofmilestonebasedendofrotationevaluationsfortenresidentscompletingathreeyearanesthesiologyresidency AT tanakapedro analysisofmilestonebasedendofrotationevaluationsfortenresidentscompletingathreeyearanesthesiologyresidency AT keilmartin analysisofmilestonebasedendofrotationevaluationsfortenresidentscompletingathreeyearanesthesiologyresidency AT macarioalex analysisofmilestonebasedendofrotationevaluationsfortenresidentscompletingathreeyearanesthesiologyresidency |