Cargando…

Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy

INTRODUCTION: Self-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we a...

Descripción completa

Detalles Bibliográficos
Autores principales: Balvardi, Saba, Semsar-Kazerooni, Koorosh, Kaneva, Pepa, Mueller, Carmen, Vassiliou, Melina, Al Mahroos, Mohammed, Fiore, Julio F., Schwartzman, Kevin, Feldman, Liane S.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer US 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9362520/
https://www.ncbi.nlm.nih.gov/pubmed/35922607
http://dx.doi.org/10.1007/s00464-022-09466-6
_version_ 1784764738812510208
author Balvardi, Saba
Semsar-Kazerooni, Koorosh
Kaneva, Pepa
Mueller, Carmen
Vassiliou, Melina
Al Mahroos, Mohammed
Fiore, Julio F.
Schwartzman, Kevin
Feldman, Liane S.
author_facet Balvardi, Saba
Semsar-Kazerooni, Koorosh
Kaneva, Pepa
Mueller, Carmen
Vassiliou, Melina
Al Mahroos, Mohammed
Fiore, Julio F.
Schwartzman, Kevin
Feldman, Liane S.
author_sort Balvardi, Saba
collection PubMed
description INTRODUCTION: Self-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we aimed to contribute evidence regarding the validity of intraoperative performance assessment tools for video-based self-assessment by general surgery trainees when performing laparoscopic cholecystectomies. METHODS AND PROCEDURES: Using a web-based platform, general surgery trainees in a university-based residency program submitted recorded laparoscopic cholecystectomy procedures where they acted as the supervised primary surgeon. Attending surgeons measured operative performance at the time of surgery using general and procedure-specific assessment tools [Global Operative Assessment of Laparoscopic Skills (GOALS) and Operative Performance Rating System (OPRS), respectively] and entrustability level (O-SCORE). Trainees self-evaluated their performance from video-review using the same instruments. The validity of GOALS and OPRS for trainee self-assessment was investigated by testing the hypotheses that self-assessment scores correlate with (H1) expert assessment scores, (H2) O-SCORE, and (H3) procedure time and that (H4) self-assessment based on these instruments differentiates junior [postgraduate year (PGY) 1–3] and senior trainees (PGY 4–5), as well as (H5)simple [Visual Analogue Scale (VAS) ≤ 4] versus complex cases (VAS > 4). All hypotheses were based on previous literature, defined a priori, and were tested according to the COSMIN consensus on measurement properties. RESULTS: A total of 35 videos were submitted (45% female and 45% senior trainees) and self-assessed. Our data supported 2 out of 5 hypotheses (H1 and H4) for GOALS and 3 out of 5 hypotheses (H1, H4 and H5) for OPRS, for trainee self-assessment. CONCLUSIONS: OPRS, a procedure-specific assessment tool, was better able to differentiate between groups expected to have different levels of intraoperative performance, compared to GOALS, a general assessment tool. Given the interest in video-based learning, there is a need to further develop valid procedure-specific tools to support video-based self-assessment by trainees in a range of procedures. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00464-022-09466-6.
format Online
Article
Text
id pubmed-9362520
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Springer US
record_format MEDLINE/PubMed
spelling pubmed-93625202022-08-10 Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy Balvardi, Saba Semsar-Kazerooni, Koorosh Kaneva, Pepa Mueller, Carmen Vassiliou, Melina Al Mahroos, Mohammed Fiore, Julio F. Schwartzman, Kevin Feldman, Liane S. Surg Endosc 2022 SAGES Oral INTRODUCTION: Self-review of recorded surgical procedures offers new opportunities for trainees to extend technical learning outside the operating-room. Valid tools for self-assessment are required prior to evaluating the effectiveness of video-review in enhancing technical learning. Therefore, we aimed to contribute evidence regarding the validity of intraoperative performance assessment tools for video-based self-assessment by general surgery trainees when performing laparoscopic cholecystectomies. METHODS AND PROCEDURES: Using a web-based platform, general surgery trainees in a university-based residency program submitted recorded laparoscopic cholecystectomy procedures where they acted as the supervised primary surgeon. Attending surgeons measured operative performance at the time of surgery using general and procedure-specific assessment tools [Global Operative Assessment of Laparoscopic Skills (GOALS) and Operative Performance Rating System (OPRS), respectively] and entrustability level (O-SCORE). Trainees self-evaluated their performance from video-review using the same instruments. The validity of GOALS and OPRS for trainee self-assessment was investigated by testing the hypotheses that self-assessment scores correlate with (H1) expert assessment scores, (H2) O-SCORE, and (H3) procedure time and that (H4) self-assessment based on these instruments differentiates junior [postgraduate year (PGY) 1–3] and senior trainees (PGY 4–5), as well as (H5)simple [Visual Analogue Scale (VAS) ≤ 4] versus complex cases (VAS > 4). All hypotheses were based on previous literature, defined a priori, and were tested according to the COSMIN consensus on measurement properties. RESULTS: A total of 35 videos were submitted (45% female and 45% senior trainees) and self-assessed. Our data supported 2 out of 5 hypotheses (H1 and H4) for GOALS and 3 out of 5 hypotheses (H1, H4 and H5) for OPRS, for trainee self-assessment. CONCLUSIONS: OPRS, a procedure-specific assessment tool, was better able to differentiate between groups expected to have different levels of intraoperative performance, compared to GOALS, a general assessment tool. Given the interest in video-based learning, there is a need to further develop valid procedure-specific tools to support video-based self-assessment by trainees in a range of procedures. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00464-022-09466-6. Springer US 2022-08-03 2023 /pmc/articles/PMC9362520/ /pubmed/35922607 http://dx.doi.org/10.1007/s00464-022-09466-6 Text en © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle 2022 SAGES Oral
Balvardi, Saba
Semsar-Kazerooni, Koorosh
Kaneva, Pepa
Mueller, Carmen
Vassiliou, Melina
Al Mahroos, Mohammed
Fiore, Julio F.
Schwartzman, Kevin
Feldman, Liane S.
Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title_full Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title_fullStr Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title_full_unstemmed Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title_short Validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
title_sort validity of video-based general and procedure-specific self-assessment tools for surgical trainees in laparoscopic cholecystectomy
topic 2022 SAGES Oral
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9362520/
https://www.ncbi.nlm.nih.gov/pubmed/35922607
http://dx.doi.org/10.1007/s00464-022-09466-6
work_keys_str_mv AT balvardisaba validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT semsarkazeroonikoorosh validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT kanevapepa validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT muellercarmen validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT vassilioumelina validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT almahroosmohammed validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT fiorejuliof validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT schwartzmankevin validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy
AT feldmanlianes validityofvideobasedgeneralandprocedurespecificselfassessmenttoolsforsurgicaltraineesinlaparoscopiccholecystectomy