Cargando…

Assessing communication skills during OSCE: need for integrated psychometric approaches

BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curricu...

Descripción completa

Detalles Bibliográficos
Autores principales: Piumatti, Giovanni, Cerutti, Bernard, Perron, Noëlle Junod
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7887794/
https://www.ncbi.nlm.nih.gov/pubmed/33593345
http://dx.doi.org/10.1186/s12909-021-02552-8
_version_ 1783652048077586432
author Piumatti, Giovanni
Cerutti, Bernard
Perron, Noëlle Junod
author_facet Piumatti, Giovanni
Cerutti, Bernard
Perron, Noëlle Junod
author_sort Piumatti, Giovanni
collection PubMed
description BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills. METHODS: Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs. RESULTS: EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students’ communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum. CONCLUSIONS: Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students’ CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-021-02552-8.
format Online
Article
Text
id pubmed-7887794
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-78877942021-02-22 Assessing communication skills during OSCE: need for integrated psychometric approaches Piumatti, Giovanni Cerutti, Bernard Perron, Noëlle Junod BMC Med Educ Technical Advance BACKGROUND: Physicians’ communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills. METHODS: Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs. RESULTS: EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students’ communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum. CONCLUSIONS: Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students’ CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12909-021-02552-8. BioMed Central 2021-02-16 /pmc/articles/PMC7887794/ /pubmed/33593345 http://dx.doi.org/10.1186/s12909-021-02552-8 Text en © The Author(s) 2021 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
spellingShingle Technical Advance
Piumatti, Giovanni
Cerutti, Bernard
Perron, Noëlle Junod
Assessing communication skills during OSCE: need for integrated psychometric approaches
title Assessing communication skills during OSCE: need for integrated psychometric approaches
title_full Assessing communication skills during OSCE: need for integrated psychometric approaches
title_fullStr Assessing communication skills during OSCE: need for integrated psychometric approaches
title_full_unstemmed Assessing communication skills during OSCE: need for integrated psychometric approaches
title_short Assessing communication skills during OSCE: need for integrated psychometric approaches
title_sort assessing communication skills during osce: need for integrated psychometric approaches
topic Technical Advance
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7887794/
https://www.ncbi.nlm.nih.gov/pubmed/33593345
http://dx.doi.org/10.1186/s12909-021-02552-8
work_keys_str_mv AT piumattigiovanni assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches
AT ceruttibernard assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches
AT perronnoellejunod assessingcommunicationskillsduringosceneedforintegratedpsychometricapproaches