Cargando…

The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital

BACKGROUND: Speech recognition (SR) technology, the process whereby spoken words are converted to digital text, has been used in radiology reporting since 1981. It was initially anticipated that SR would dominate radiology reporting, with claims of up to 99% accuracy, reduced turnaround times and si...

Descripción completa

Detalles Bibliográficos
Autores principales: du Toit, Jacqueline, Hattingh, Retha, Pitcher, Richard
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2015
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4464850/
https://www.ncbi.nlm.nih.gov/pubmed/25879906
http://dx.doi.org/10.1186/s12880-015-0048-1
_version_ 1782376036238884864
author du Toit, Jacqueline
Hattingh, Retha
Pitcher, Richard
author_facet du Toit, Jacqueline
Hattingh, Retha
Pitcher, Richard
author_sort du Toit, Jacqueline
collection PubMed
description BACKGROUND: Speech recognition (SR) technology, the process whereby spoken words are converted to digital text, has been used in radiology reporting since 1981. It was initially anticipated that SR would dominate radiology reporting, with claims of up to 99% accuracy, reduced turnaround times and significant cost savings. However, expectations have not yet been realised. The limited data available suggest SR reports have significantly higher levels of inaccuracy than traditional dictation transcription (DT) reports, as well as incurring greater aggregate costs. There has been little work on the clinical significance of such errors, however, and little is known of the impact of reporter seniority on the generation of errors, or the influence of system familiarity on reducing error rates. Furthermore, there have been conflicting findings on the accuracy of SR amongst users with English as first- and second-language respectively. METHODS: The aim of the study was to compare the accuracy of SR and DT reports in a resource-limited setting. The first 300 SR and the first 300 DT reports generated during March 2010 were retrieved from the hospital’s PACS, and reviewed by a single observer. Text errors were identified, and then classified as either clinically significant or insignificant based on their potential impact on patient management. In addition, a follow-up analysis was conducted exactly 4 years later. RESULTS: Of the original 300 SR reports analysed, 25.6% contained errors, with 9.6% being clinically significant. Only 9.3% of the DT reports contained errors, 2.3% having potential clinical impact. Both the overall difference in SR and DT error rates, and the difference in ‘clinically significant’ error rates (9.6% vs. 2.3%) were statistically significant. In the follow-up study, the overall SR error rate was strikingly similar at 24.3%, 6% being clinically significant. Radiologists with second-language English were more likely to generate reports containing errors, but level of seniority had no bearing. CONCLUSION: SR technology consistently increased inaccuracies in Tygerberg Hospital (TBH) radiology reports, thereby potentially compromising patient care. Awareness of increased error rates in SR reports, particularly amongst those transcribing in a second-language, is important for effective implementation of SR in a multilingual healthcare environment.
format Online
Article
Text
id pubmed-4464850
institution National Center for Biotechnology Information
language English
publishDate 2015
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-44648502015-06-14 The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital du Toit, Jacqueline Hattingh, Retha Pitcher, Richard BMC Med Imaging Research Article BACKGROUND: Speech recognition (SR) technology, the process whereby spoken words are converted to digital text, has been used in radiology reporting since 1981. It was initially anticipated that SR would dominate radiology reporting, with claims of up to 99% accuracy, reduced turnaround times and significant cost savings. However, expectations have not yet been realised. The limited data available suggest SR reports have significantly higher levels of inaccuracy than traditional dictation transcription (DT) reports, as well as incurring greater aggregate costs. There has been little work on the clinical significance of such errors, however, and little is known of the impact of reporter seniority on the generation of errors, or the influence of system familiarity on reducing error rates. Furthermore, there have been conflicting findings on the accuracy of SR amongst users with English as first- and second-language respectively. METHODS: The aim of the study was to compare the accuracy of SR and DT reports in a resource-limited setting. The first 300 SR and the first 300 DT reports generated during March 2010 were retrieved from the hospital’s PACS, and reviewed by a single observer. Text errors were identified, and then classified as either clinically significant or insignificant based on their potential impact on patient management. In addition, a follow-up analysis was conducted exactly 4 years later. RESULTS: Of the original 300 SR reports analysed, 25.6% contained errors, with 9.6% being clinically significant. Only 9.3% of the DT reports contained errors, 2.3% having potential clinical impact. Both the overall difference in SR and DT error rates, and the difference in ‘clinically significant’ error rates (9.6% vs. 2.3%) were statistically significant. In the follow-up study, the overall SR error rate was strikingly similar at 24.3%, 6% being clinically significant. Radiologists with second-language English were more likely to generate reports containing errors, but level of seniority had no bearing. CONCLUSION: SR technology consistently increased inaccuracies in Tygerberg Hospital (TBH) radiology reports, thereby potentially compromising patient care. Awareness of increased error rates in SR reports, particularly amongst those transcribing in a second-language, is important for effective implementation of SR in a multilingual healthcare environment. BioMed Central 2015-03-04 /pmc/articles/PMC4464850/ /pubmed/25879906 http://dx.doi.org/10.1186/s12880-015-0048-1 Text en © du Toit et al.; licensee BioMed Central. 2015 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
du Toit, Jacqueline
Hattingh, Retha
Pitcher, Richard
The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title_full The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title_fullStr The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title_full_unstemmed The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title_short The accuracy of radiology speech recognition reports in a multilingual South African teaching hospital
title_sort accuracy of radiology speech recognition reports in a multilingual south african teaching hospital
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4464850/
https://www.ncbi.nlm.nih.gov/pubmed/25879906
http://dx.doi.org/10.1186/s12880-015-0048-1
work_keys_str_mv AT dutoitjacqueline theaccuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital
AT hattinghretha theaccuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital
AT pitcherrichard theaccuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital
AT dutoitjacqueline accuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital
AT hattinghretha accuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital
AT pitcherrichard accuracyofradiologyspeechrecognitionreportsinamultilingualsouthafricanteachinghospital