Cargando…
Assessing communication skills during OSCE: need for integrated psychometric approaches
BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curricu...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7887794/ https://www.ncbi.nlm.nih.gov/pubmed/33593345 http://dx.doi.org/10.1186/s12909-021-02552-8 |
_version_ | 1783652048077586432 |
---|---|
author | Piumatti, Giovanni Cerutti, Bernard Perron, Noëlle Junod |
author_facet | Piumatti, Giovanni Cerutti, Bernard Perron, Noëlle Junod |
author_sort | Piumatti, Giovanni |
collection | PubMed |
description | BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills. METHODS: Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs. RESULTS: EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students’ communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum. CONCLUSIONS: Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students’ CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-021-02552-8. |
format | Online Article Text |
id | pubmed-7887794 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-78877942021-02-22 Assessing communication skills during OSCE: need for integrated psychometric approaches Piumatti, Giovanni Cerutti, Bernard Perron, Noëlle Junod BMC Med Educ Technical Advance BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills. METHODS: Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs. RESULTS: EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students’ communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum. CONCLUSIONS: Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students’ CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-021-02552-8. BioMed Central 2021-02-16 /pmc/articles/PMC7887794/ /pubmed/33593345 http://dx.doi.org/10.1186/s12909-021-02552-8 Text en © The Author(s) 2021 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. |
spellingShingle | Technical Advance Piumatti, Giovanni Cerutti, Bernard Perron, Noëlle Junod Assessing communication skills during OSCE: need for integrated psychometric approaches |
title | Assessing communication skills during OSCE: need for integrated psychometric approaches |
title_full | Assessing communication skills during OSCE: need for integrated psychometric approaches |
title_fullStr | Assessing communication skills during OSCE: need for integrated psychometric approaches |
title_full_unstemmed | Assessing communication skills during OSCE: need for integrated psychometric approaches |
title_short | Assessing communication skills during OSCE: need for integrated psychometric approaches |
title_sort | assessing communication skills during osce: need for integrated psychometric approaches |
topic | Technical Advance |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7887794/ https://www.ncbi.nlm.nih.gov/pubmed/33593345 http://dx.doi.org/10.1186/s12909-021-02552-8 |
work_keys_str_mv | AT piumattigiovanni assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches AT ceruttibernard assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches AT perronnoellejunod assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches |