Cargando…

Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study

BACKGROUND: The increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the requir...

Descripción completa

Detalles Bibliográficos
Autores principales: Pranaat, Robert, Mohan, Vishnu, O'Reilly, Megan, Hirsh, Maxwell, McGrath, Karess, Scholl, Gretchen, Woodcock, Deborah, Gold, Jeffrey A
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5628287/
https://www.ncbi.nlm.nih.gov/pubmed/28931497
http://dx.doi.org/10.2196/medinform.7883
_version_ 1783268851518013440
author Pranaat, Robert
Mohan, Vishnu
O'Reilly, Megan
Hirsh, Maxwell
McGrath, Karess
Scholl, Gretchen
Woodcock, Deborah
Gold, Jeffrey A
author_facet Pranaat, Robert
Mohan, Vishnu
O'Reilly, Megan
Hirsh, Maxwell
McGrath, Karess
Scholl, Gretchen
Woodcock, Deborah
Gold, Jeffrey A
author_sort Pranaat, Robert
collection PubMed
description BACKGROUND: The increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the required EHR functions. Despite this rapid growth, little has been published on the training or assessment tools to appraise the safety and efficacy of scribe-related EHR activities. Given the number of reports documenting that other professional groups suffer from a number of performance errors in EHR interface and data gathering, scribes likely face similar challenges. This highlights the need for new assessment tools for medical scribes. OBJECTIVE: The objective of this study was to develop a virtual video-based simulation to demonstrate and quantify the variability and accuracy of scribes’ transcribed notes in the EHR. METHODS: From a pool of 8 scribes in one department, a total of 5 female scribes, intent on pursuing careers in health care, with at least 6 months of experience were recruited for our simulation study. We created three simulated patient-provider scenarios. Each scenario contained a corresponding medical record in our simulation instance of our EHR. For each scenario, we video-recorded a standardized patient-provider encounter. Five scribes with at least 6 months of experience both with our EHR and in the specialty of the simulated cases were recruited. Each scribe watched the simulated encounter and transcribed notes into a simulated EHR environment. Transcribed notes were evaluated for interscribe variability and compared with a gold standard for accuracy. RESULTS: All scribes completed all simulated cases. There was significant interscribe variability in note structure and content. Overall, only 26% of all data elements were unique to the scribe writing them. The term data element was used to define the individual pieces of data that scribes perceived from the simulation. Note length was determined by counting the number of words varied by 31%, 37%, and 57% between longest and shortest note between the three cases, and word economy ranged between 23% and 71%. Overall, there was a wide inter- and intrascribe variation in accuracy for each section of the notes with ranges from 50% to 76%, resulting in an overall positive predictive value for each note between 38% and 81%. CONCLUSIONS: We created a high-fidelity, video-based EHR simulation, capable of assessing multiple performance indicators in medical scribes. In this cohort, we demonstrate significant variability both in terms of structure and accuracy in clinical documentation. This form of simulation can provide a valuable tool for future development of scribe curriculum and assessment of competency.
format Online
Article
Text
id pubmed-5628287
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-56282872017-10-20 Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study Pranaat, Robert Mohan, Vishnu O'Reilly, Megan Hirsh, Maxwell McGrath, Karess Scholl, Gretchen Woodcock, Deborah Gold, Jeffrey A JMIR Med Inform Original Paper BACKGROUND: The increasing adoption of electronic health records (EHRs) has been associated with a number of unintended negative consequences with provider efficiency and job satisfaction. To address this, there has been a dramatic increase in the use of medical scribes to perform many of the required EHR functions. Despite this rapid growth, little has been published on the training or assessment tools to appraise the safety and efficacy of scribe-related EHR activities. Given the number of reports documenting that other professional groups suffer from a number of performance errors in EHR interface and data gathering, scribes likely face similar challenges. This highlights the need for new assessment tools for medical scribes. OBJECTIVE: The objective of this study was to develop a virtual video-based simulation to demonstrate and quantify the variability and accuracy of scribes’ transcribed notes in the EHR. METHODS: From a pool of 8 scribes in one department, a total of 5 female scribes, intent on pursuing careers in health care, with at least 6 months of experience were recruited for our simulation study. We created three simulated patient-provider scenarios. Each scenario contained a corresponding medical record in our simulation instance of our EHR. For each scenario, we video-recorded a standardized patient-provider encounter. Five scribes with at least 6 months of experience both with our EHR and in the specialty of the simulated cases were recruited. Each scribe watched the simulated encounter and transcribed notes into a simulated EHR environment. Transcribed notes were evaluated for interscribe variability and compared with a gold standard for accuracy. RESULTS: All scribes completed all simulated cases. There was significant interscribe variability in note structure and content. Overall, only 26% of all data elements were unique to the scribe writing them. The term data element was used to define the individual pieces of data that scribes perceived from the simulation. Note length was determined by counting the number of words varied by 31%, 37%, and 57% between longest and shortest note between the three cases, and word economy ranged between 23% and 71%. Overall, there was a wide inter- and intrascribe variation in accuracy for each section of the notes with ranges from 50% to 76%, resulting in an overall positive predictive value for each note between 38% and 81%. CONCLUSIONS: We created a high-fidelity, video-based EHR simulation, capable of assessing multiple performance indicators in medical scribes. In this cohort, we demonstrate significant variability both in terms of structure and accuracy in clinical documentation. This form of simulation can provide a valuable tool for future development of scribe curriculum and assessment of competency. JMIR Publications 2017-09-20 /pmc/articles/PMC5628287/ /pubmed/28931497 http://dx.doi.org/10.2196/medinform.7883 Text en ©Robert Pranaat, Vishnu Mohan, Megan O'Reilly, Maxwell Hirsh, Karess McGrath, Gretchen Scholl, Deborah Woodcock, Jeffrey A Gold. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 20.09.2017. https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Pranaat, Robert
Mohan, Vishnu
O'Reilly, Megan
Hirsh, Maxwell
McGrath, Karess
Scholl, Gretchen
Woodcock, Deborah
Gold, Jeffrey A
Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title_full Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title_fullStr Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title_full_unstemmed Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title_short Use of Simulation Based on an Electronic Health Records Environment to Evaluate the Structure and Accuracy of Notes Generated by Medical Scribes: Proof-of-Concept Study
title_sort use of simulation based on an electronic health records environment to evaluate the structure and accuracy of notes generated by medical scribes: proof-of-concept study
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5628287/
https://www.ncbi.nlm.nih.gov/pubmed/28931497
http://dx.doi.org/10.2196/medinform.7883
work_keys_str_mv AT pranaatrobert useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT mohanvishnu useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT oreillymegan useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT hirshmaxwell useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT mcgrathkaress useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT schollgretchen useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT woodcockdeborah useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy
AT goldjeffreya useofsimulationbasedonanelectronichealthrecordsenvironmenttoevaluatethestructureandaccuracyofnotesgeneratedbymedicalscribesproofofconceptstudy