Cargando…

Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol

BACKGROUND: Adverse events (AEs) in acute care hospitals are frequent and associated with significant morbidity, mortality, and costs. Measuring AEs is necessary for quality improvement and benchmarking purposes, but current detection methods lack in accuracy, efficiency, and generalizability. The g...

Descripción completa

Detalles Bibliográficos
Autores principales: Rochefort, Christian M., Buckeridge, David L., Tanguay, Andréanne, Biron, Alain, D’Aragon, Frédérick, Wang, Shengrui, Gallix, Benoit, Valiquette, Louis, Audet, Li-Anne, Lee, Todd C., Jayaraman, Dev, Petrucci, Bruno, Lefebvre, Patricia
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5314632/
https://www.ncbi.nlm.nih.gov/pubmed/28209197
http://dx.doi.org/10.1186/s12913-017-2069-7
_version_ 1782508555579949056
author Rochefort, Christian M.
Buckeridge, David L.
Tanguay, Andréanne
Biron, Alain
D’Aragon, Frédérick
Wang, Shengrui
Gallix, Benoit
Valiquette, Louis
Audet, Li-Anne
Lee, Todd C.
Jayaraman, Dev
Petrucci, Bruno
Lefebvre, Patricia
author_facet Rochefort, Christian M.
Buckeridge, David L.
Tanguay, Andréanne
Biron, Alain
D’Aragon, Frédérick
Wang, Shengrui
Gallix, Benoit
Valiquette, Louis
Audet, Li-Anne
Lee, Todd C.
Jayaraman, Dev
Petrucci, Bruno
Lefebvre, Patricia
author_sort Rochefort, Christian M.
collection PubMed
description BACKGROUND: Adverse events (AEs) in acute care hospitals are frequent and associated with significant morbidity, mortality, and costs. Measuring AEs is necessary for quality improvement and benchmarking purposes, but current detection methods lack in accuracy, efficiency, and generalizability. The growing availability of electronic health records (EHR) and the development of natural language processing techniques for encoding narrative data offer an opportunity to develop potentially better methods. The purpose of this study is to determine the accuracy and generalizability of using automated methods for detecting three high-incidence and high-impact AEs from EHR data: a) hospital-acquired pneumonia, b) ventilator-associated event and, c) central line-associated bloodstream infection. METHODS: This validation study will be conducted among medical, surgical and ICU patients admitted between 2013 and 2016 to the Centre hospitalier universitaire de Sherbrooke (CHUS) and the McGill University Health Centre (MUHC), which has both French and English sites. A random 60% sample of CHUS patients will be used for model development purposes (cohort 1, development set). Using a random sample of these patients, a reference standard assessment of their medical chart will be performed. Multivariate logistic regression and the area under the curve (AUC) will be employed to iteratively develop and optimize three automated AE detection models (i.e., one per AE of interest) using EHR data from the CHUS. These models will then be validated on a random sample of the remaining 40% of CHUS patients (cohort 1, internal validation set) using chart review to assess accuracy. The most accurate models developed and validated at the CHUS will then be applied to EHR data from a random sample of patients admitted to the MUHC French site (cohort 2) and English site (cohort 3)—a critical requirement given the use of narrative data –, and accuracy will be assessed using chart review. Generalizability will be determined by comparing AUCs from cohorts 2 and 3 to those from cohort 1. DISCUSSION: This study will likely produce more accurate and efficient measures of AEs. These measures could be used to assess the incidence rates of AEs, evaluate the success of preventive interventions, or benchmark performance across hospitals.
format Online
Article
Text
id pubmed-5314632
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-53146322017-02-24 Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol Rochefort, Christian M. Buckeridge, David L. Tanguay, Andréanne Biron, Alain D’Aragon, Frédérick Wang, Shengrui Gallix, Benoit Valiquette, Louis Audet, Li-Anne Lee, Todd C. Jayaraman, Dev Petrucci, Bruno Lefebvre, Patricia BMC Health Serv Res Study Protocol BACKGROUND: Adverse events (AEs) in acute care hospitals are frequent and associated with significant morbidity, mortality, and costs. Measuring AEs is necessary for quality improvement and benchmarking purposes, but current detection methods lack in accuracy, efficiency, and generalizability. The growing availability of electronic health records (EHR) and the development of natural language processing techniques for encoding narrative data offer an opportunity to develop potentially better methods. The purpose of this study is to determine the accuracy and generalizability of using automated methods for detecting three high-incidence and high-impact AEs from EHR data: a) hospital-acquired pneumonia, b) ventilator-associated event and, c) central line-associated bloodstream infection. METHODS: This validation study will be conducted among medical, surgical and ICU patients admitted between 2013 and 2016 to the Centre hospitalier universitaire de Sherbrooke (CHUS) and the McGill University Health Centre (MUHC), which has both French and English sites. A random 60% sample of CHUS patients will be used for model development purposes (cohort 1, development set). Using a random sample of these patients, a reference standard assessment of their medical chart will be performed. Multivariate logistic regression and the area under the curve (AUC) will be employed to iteratively develop and optimize three automated AE detection models (i.e., one per AE of interest) using EHR data from the CHUS. These models will then be validated on a random sample of the remaining 40% of CHUS patients (cohort 1, internal validation set) using chart review to assess accuracy. The most accurate models developed and validated at the CHUS will then be applied to EHR data from a random sample of patients admitted to the MUHC French site (cohort 2) and English site (cohort 3)—a critical requirement given the use of narrative data –, and accuracy will be assessed using chart review. Generalizability will be determined by comparing AUCs from cohorts 2 and 3 to those from cohort 1. DISCUSSION: This study will likely produce more accurate and efficient measures of AEs. These measures could be used to assess the incidence rates of AEs, evaluate the success of preventive interventions, or benchmark performance across hospitals. BioMed Central 2017-02-16 /pmc/articles/PMC5314632/ /pubmed/28209197 http://dx.doi.org/10.1186/s12913-017-2069-7 Text en © The Author(s). 2017 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Study Protocol
Rochefort, Christian M.
Buckeridge, David L.
Tanguay, Andréanne
Biron, Alain
D’Aragon, Frédérick
Wang, Shengrui
Gallix, Benoit
Valiquette, Louis
Audet, Li-Anne
Lee, Todd C.
Jayaraman, Dev
Petrucci, Bruno
Lefebvre, Patricia
Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title_full Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title_fullStr Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title_full_unstemmed Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title_short Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
title_sort accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol
topic Study Protocol
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5314632/
https://www.ncbi.nlm.nih.gov/pubmed/28209197
http://dx.doi.org/10.1186/s12913-017-2069-7
work_keys_str_mv AT rochefortchristianm accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT buckeridgedavidl accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT tanguayandreanne accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT bironalain accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT daragonfrederick accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT wangshengrui accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT gallixbenoit accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT valiquettelouis accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT audetlianne accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT leetoddc accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT jayaramandev accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT petruccibruno accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol
AT lefebvrepatricia accuracyandgeneralizabilityofusingautomatedmethodsforidentifyingadverseeventsfromelectronichealthrecorddataavalidationstudyprotocol