Cargando…
Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator
Introduction. Medical simulators are used for assessing clinical skills and increasingly for testing hypotheses. We developed and tested an approach for assessing performance in anesthesia residents using screen-based simulation that ensures expert raters remain blinded to subject identity and exper...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Hindawi Publishing Corporation
2016
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4879220/ https://www.ncbi.nlm.nih.gov/pubmed/27293430 http://dx.doi.org/10.1155/2016/9348478 |
_version_ | 1782433651918635008 |
---|---|
author | McNeer, Richard R. Dudaryk, Roman Nedeff, Nicholas B. Bennett, Christopher L. |
author_facet | McNeer, Richard R. Dudaryk, Roman Nedeff, Nicholas B. Bennett, Christopher L. |
author_sort | McNeer, Richard R. |
collection | PubMed |
description | Introduction. Medical simulators are used for assessing clinical skills and increasingly for testing hypotheses. We developed and tested an approach for assessing performance in anesthesia residents using screen-based simulation that ensures expert raters remain blinded to subject identity and experimental condition. Methods. Twenty anesthesia residents managed emergencies in an operating room simulator by logging actions through a custom graphical user interface. Two expert raters rated performance based on these entries using custom Global Rating Scale (GRS) and Crisis Management Checklist (CMC) instruments. Interrater reliability was measured by calculating intraclass correlation coefficients (ICC), and internal consistency of the instruments was assessed with Cronbach's alpha. Agreement between GRS and CMC was measured using Spearman rank correlation (SRC). Results. Interrater agreement (GRS: ICC = 0.825, CMC: ICC = 0.878) and internal consistency (GRS: alpha = 0.838, CMC: alpha = 0.886) were good for both instruments. Subscale analysis indicated that several instrument items can be discarded. GRS and CMC scores were highly correlated (SRC = 0.948). Conclusions. In this pilot study, we demonstrated that screen-based simulation can allow blinded assessment of performance. GRS and CMC instruments demonstrated good rater agreement and internal consistency. We plan to further test construct validity of our instruments by measuring performance in our simulator as a function of training level. |
format | Online Article Text |
id | pubmed-4879220 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2016 |
publisher | Hindawi Publishing Corporation |
record_format | MEDLINE/PubMed |
spelling | pubmed-48792202016-06-12 Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator McNeer, Richard R. Dudaryk, Roman Nedeff, Nicholas B. Bennett, Christopher L. Anesthesiol Res Pract Research Article Introduction. Medical simulators are used for assessing clinical skills and increasingly for testing hypotheses. We developed and tested an approach for assessing performance in anesthesia residents using screen-based simulation that ensures expert raters remain blinded to subject identity and experimental condition. Methods. Twenty anesthesia residents managed emergencies in an operating room simulator by logging actions through a custom graphical user interface. Two expert raters rated performance based on these entries using custom Global Rating Scale (GRS) and Crisis Management Checklist (CMC) instruments. Interrater reliability was measured by calculating intraclass correlation coefficients (ICC), and internal consistency of the instruments was assessed with Cronbach's alpha. Agreement between GRS and CMC was measured using Spearman rank correlation (SRC). Results. Interrater agreement (GRS: ICC = 0.825, CMC: ICC = 0.878) and internal consistency (GRS: alpha = 0.838, CMC: alpha = 0.886) were good for both instruments. Subscale analysis indicated that several instrument items can be discarded. GRS and CMC scores were highly correlated (SRC = 0.948). Conclusions. In this pilot study, we demonstrated that screen-based simulation can allow blinded assessment of performance. GRS and CMC instruments demonstrated good rater agreement and internal consistency. We plan to further test construct validity of our instruments by measuring performance in our simulator as a function of training level. Hindawi Publishing Corporation 2016 2016-05-11 /pmc/articles/PMC4879220/ /pubmed/27293430 http://dx.doi.org/10.1155/2016/9348478 Text en Copyright © 2016 Richard R. McNeer et al. https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Research Article McNeer, Richard R. Dudaryk, Roman Nedeff, Nicholas B. Bennett, Christopher L. Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title | Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title_full | Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title_fullStr | Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title_full_unstemmed | Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title_short | Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator |
title_sort | development and testing of screen-based and psychometric instruments for assessing resident performance in an operating room simulator |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4879220/ https://www.ncbi.nlm.nih.gov/pubmed/27293430 http://dx.doi.org/10.1155/2016/9348478 |
work_keys_str_mv | AT mcneerrichardr developmentandtestingofscreenbasedandpsychometricinstrumentsforassessingresidentperformanceinanoperatingroomsimulator AT dudarykroman developmentandtestingofscreenbasedandpsychometricinstrumentsforassessingresidentperformanceinanoperatingroomsimulator AT nedeffnicholasb developmentandtestingofscreenbasedandpsychometricinstrumentsforassessingresidentperformanceinanoperatingroomsimulator AT bennettchristopherl developmentandtestingofscreenbasedandpsychometricinstrumentsforassessingresidentperformanceinanoperatingroomsimulator |