Cargando…

Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study

OBJECTIVES: To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed. DESIGN: Qualitative text analysis supplemented by quan...

Descripción completa

Detalles Bibliográficos
Autores principales: Holm, Ellen Astrid, Al-Bayati, Shaymaa Jaafar Lafta, Barfod, Toke Seierøe, Lembeck, Maurice A, Pedersen, Hanne, Ramberg, Emilie, Klemmensen, Åse Kathrine, Sorensen, Jette Led
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BMJ Publishing Group 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8319975/
https://www.ncbi.nlm.nih.gov/pubmed/34321296
http://dx.doi.org/10.1136/bmjopen-2020-047019
_version_ 1783730558220632064
author Holm, Ellen Astrid
Al-Bayati, Shaymaa Jaafar Lafta
Barfod, Toke Seierøe
Lembeck, Maurice A
Pedersen, Hanne
Ramberg, Emilie
Klemmensen, Åse Kathrine
Sorensen, Jette Led
author_facet Holm, Ellen Astrid
Al-Bayati, Shaymaa Jaafar Lafta
Barfod, Toke Seierøe
Lembeck, Maurice A
Pedersen, Hanne
Ramberg, Emilie
Klemmensen, Åse Kathrine
Sorensen, Jette Led
author_sort Holm, Ellen Astrid
collection PubMed
description OBJECTIVES: To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed. DESIGN: Qualitative text analysis supplemented by quantitative descriptive analysis. SETTING: Internal Medicine Departments in Zealand, Denmark. PARTICIPANTS: 48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents). INTERVENTION: This study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either ‘no reasons for concern’ or ‘possibly some concern’, thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee. RESULTS: Out of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7–1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed. CONCLUSIONS: A purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees’ performances based on the MSF reports.
format Online
Article
Text
id pubmed-8319975
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher BMJ Publishing Group
record_format MEDLINE/PubMed
spelling pubmed-83199752021-08-02 Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study Holm, Ellen Astrid Al-Bayati, Shaymaa Jaafar Lafta Barfod, Toke Seierøe Lembeck, Maurice A Pedersen, Hanne Ramberg, Emilie Klemmensen, Åse Kathrine Sorensen, Jette Led BMJ Open Medical Education and Training OBJECTIVES: To examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed. DESIGN: Qualitative text analysis supplemented by quantitative descriptive analysis. SETTING: Internal Medicine Departments in Zealand, Denmark. PARTICIPANTS: 48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents). INTERVENTION: This study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either ‘no reasons for concern’ or ‘possibly some concern’, thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee. RESULTS: Out of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7–1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed. CONCLUSIONS: A purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees’ performances based on the MSF reports. BMJ Publishing Group 2021-07-28 /pmc/articles/PMC8319975/ /pubmed/34321296 http://dx.doi.org/10.1136/bmjopen-2020-047019 Text en © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. https://creativecommons.org/licenses/by-nc/4.0/This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ (https://creativecommons.org/licenses/by-nc/4.0/) .
spellingShingle Medical Education and Training
Holm, Ellen Astrid
Al-Bayati, Shaymaa Jaafar Lafta
Barfod, Toke Seierøe
Lembeck, Maurice A
Pedersen, Hanne
Ramberg, Emilie
Klemmensen, Åse Kathrine
Sorensen, Jette Led
Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title_full Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title_fullStr Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title_full_unstemmed Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title_short Feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
title_sort feasibility, quality and validity of narrative multisource feedback in postgraduate training: a mixed-method study
topic Medical Education and Training
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8319975/
https://www.ncbi.nlm.nih.gov/pubmed/34321296
http://dx.doi.org/10.1136/bmjopen-2020-047019
work_keys_str_mv AT holmellenastrid feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT albayatishaymaajaafarlafta feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT barfodtokeseierøe feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT lembeckmauricea feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT pedersenhanne feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT rambergemilie feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT klemmensenasekathrine feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy
AT sorensenjetteled feasibilityqualityandvalidityofnarrativemultisourcefeedbackinpostgraduatetrainingamixedmethodstudy