Cargando…

Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial

IMPORTANCE: In cardiac device implant training, there is no common system to objectively assess trainees’ ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators’ early learning curve. OBJECTIVE:...

Descripción completa

Detalles Bibliográficos
Autores principales: Mascheroni, Jorio, Stockburger, Martin, Patwala, Ashish, Mont, Lluís, Rao, Archana, Retzlaff, Hartwig, Garweg, Christophe, Verbelen, Tom, Gallagher, Anthony G.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: American Medical Association 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10472192/
https://www.ncbi.nlm.nih.gov/pubmed/37651144
http://dx.doi.org/10.1001/jamanetworkopen.2023.22750
_version_ 1785100022932570112
author Mascheroni, Jorio
Stockburger, Martin
Patwala, Ashish
Mont, Lluís
Rao, Archana
Retzlaff, Hartwig
Garweg, Christophe
Verbelen, Tom
Gallagher, Anthony G.
author_facet Mascheroni, Jorio
Stockburger, Martin
Patwala, Ashish
Mont, Lluís
Rao, Archana
Retzlaff, Hartwig
Garweg, Christophe
Verbelen, Tom
Gallagher, Anthony G.
author_sort Mascheroni, Jorio
collection PubMed
description IMPORTANCE: In cardiac device implant training, there is no common system to objectively assess trainees’ ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators’ early learning curve. OBJECTIVE: To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT). DESIGN, SETTING, AND PARTICIPANTS: In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022. INTERVENTION: Training with PBP vs SBT. MAIN OUTCOME AND MEASURES: The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined. RESULTS: Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees. CONCLUSIONS AND RELEVANCE: In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT05952908
format Online
Article
Text
id pubmed-10472192
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher American Medical Association
record_format MEDLINE/PubMed
spelling pubmed-104721922023-09-02 Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial Mascheroni, Jorio Stockburger, Martin Patwala, Ashish Mont, Lluís Rao, Archana Retzlaff, Hartwig Garweg, Christophe Verbelen, Tom Gallagher, Anthony G. JAMA Netw Open Original Investigation IMPORTANCE: In cardiac device implant training, there is no common system to objectively assess trainees’ ability to perform tasks at predetermined performance levels before in vivo practice; therefore, patients are potentially exposed to risks related to operators’ early learning curve. OBJECTIVE: To assess the effect on implant performance quality of novel metrics-based simulation training to proficiency (proficiency-based progression [PBP]) vs traditional simulation-based training (SBT). DESIGN, SETTING, AND PARTICIPANTS: In this prospective randomized trial, conducted between March 8, 2022 and November 24, 2022, novice implanters were randomized (blinded) 1:1 to participate in an SBT curriculum (procedural knowledge e-learning and in-person simulation training) at an international skills training center, with proficiency demonstration requirements at each training stage for advancing (PBP approach) or without the requirements. Ultimately, trainees performed a cardiac resynchronization therapy (CRT) implant using virtual reality simulation. The procedure was video-recorded and subsequently scored using previously validated metrics by 2 independent assessors blinded to group. Physicians who had already implanted more than 20 pacemakers or defibrillators and fewer than 200 CRT systems as the first operator were eligible. Thirty-two implanters from 10 countries voluntarily enrolled in the training program and were randomized; 30 (15 per group) started and completed training. Data analysis was performed from November 27 to December 22, 2022. INTERVENTION: Training with PBP vs SBT. MAIN OUTCOME AND MEASURES: The primary outcome comprised 4 objectively assessed performance metrics derived from the video-recordings: number of procedural steps completed, errors, critical errors, and all errors combined. RESULTS: Baseline experience of the 30 participants (19 [63%] male; mean [SD] number of years in implant practice, 2.0 [1.8]; median [IQR] number of implanted pacemakers or defibrillators, 47.5 [30.0-115.0]; median [IQR] number of implanted CRT systems, 3.0 [1.25-10.0]) was similar between study groups. Compared with the SBT group, the PBP group completed 27% more procedural steps (median [IQR], 31 [30-32] vs 24 [22-27]; P < .001) and made 73% fewer errors (median [IQR], 2 [1-3] vs 7 [5-8]; P < .001), 84% fewer critical errors (median [IQR], 1 [0-1] vs 3 [3-5]; P < .001), and 77% fewer all errors combined (errors plus critical errors) (median [IQR], 3 [1-3] vs 11 [8-12]; P < .001); 14 of the 15 PBP trainees (93%) demonstrated the predefined target performance level vs 0 of the 15 SBT trainees. CONCLUSIONS AND RELEVANCE: In this randomized trial, the PBP approach to novice implanter training generated superior objectively assessed performance vs SBT. If implemented broadly and systematically, PBP training may ensure safe and effective performance standards before trainees proceed to (supervised) in vivo practice. Future studies are needed to verify implications on procedure-related patient complications. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT05952908 American Medical Association 2023-08-31 /pmc/articles/PMC10472192/ /pubmed/37651144 http://dx.doi.org/10.1001/jamanetworkopen.2023.22750 Text en Copyright 2023 Mascheroni J et al. JAMA Network Open. https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article distributed under the terms of the CC-BY-NC-ND License.
spellingShingle Original Investigation
Mascheroni, Jorio
Stockburger, Martin
Patwala, Ashish
Mont, Lluís
Rao, Archana
Retzlaff, Hartwig
Garweg, Christophe
Verbelen, Tom
Gallagher, Anthony G.
Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title_full Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title_fullStr Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title_full_unstemmed Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title_short Effect of Metrics-Based Simulation Training to Proficiency on Procedure Quality and Errors Among Novice Cardiac Device Implanters: The IMPROF Randomized Trial
title_sort effect of metrics-based simulation training to proficiency on procedure quality and errors among novice cardiac device implanters: the improf randomized trial
topic Original Investigation
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10472192/
https://www.ncbi.nlm.nih.gov/pubmed/37651144
http://dx.doi.org/10.1001/jamanetworkopen.2023.22750
work_keys_str_mv AT mascheronijorio effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT stockburgermartin effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT patwalaashish effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT montlluis effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT raoarchana effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT retzlaffhartwig effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT garwegchristophe effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT verbelentom effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial
AT gallagheranthonyg effectofmetricsbasedsimulationtrainingtoproficiencyonprocedurequalityanderrorsamongnovicecardiacdeviceimplanterstheimprofrandomizedtrial