Cargando…

Identifying and classifying opioid‐related overdoses: A validation study

PURPOSE: The study aims to develop and validate algorithms to identify and classify opioid overdoses using claims and other coded data, and clinical text extracted from electronic health records using natural language processing (NLP). METHODS: Primary data were derived from Kaiser Permanente Northw...

Descripción completa

Detalles Bibliográficos
Autores principales: Green, Carla A., Perrin, Nancy A., Hazlehurst, Brian, Janoff, Shannon L., DeVeaugh‐Geiss, Angela, Carrell, David S., Grijalva, Carlos G., Liang, Caihua, Enger, Cheryl L., Coplan, Paul M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: John Wiley and Sons Inc. 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6767606/
https://www.ncbi.nlm.nih.gov/pubmed/31020755
http://dx.doi.org/10.1002/pds.4772
_version_ 1783454956612747264
author Green, Carla A.
Perrin, Nancy A.
Hazlehurst, Brian
Janoff, Shannon L.
DeVeaugh‐Geiss, Angela
Carrell, David S.
Grijalva, Carlos G.
Liang, Caihua
Enger, Cheryl L.
Coplan, Paul M.
author_facet Green, Carla A.
Perrin, Nancy A.
Hazlehurst, Brian
Janoff, Shannon L.
DeVeaugh‐Geiss, Angela
Carrell, David S.
Grijalva, Carlos G.
Liang, Caihua
Enger, Cheryl L.
Coplan, Paul M.
author_sort Green, Carla A.
collection PubMed
description PURPOSE: The study aims to develop and validate algorithms to identify and classify opioid overdoses using claims and other coded data, and clinical text extracted from electronic health records using natural language processing (NLP). METHODS: Primary data were derived from Kaiser Permanente Northwest (2008–2014), an integrated health care system (~n > 475 000 unique individuals per year). Data included International Classification of Diseases, Ninth Revision (ICD‐9) codes for nonfatal diagnoses, International Classification of Diseases, Tenth Revision (ICD‐10) codes for fatal events, clinical notes, and prescription medication records. We assessed sensitivity, specificity, positive predictive value, and negative predictive value for algorithms relative to medical chart review and conducted assessments of algorithm portability in Kaiser Permanente Washington, Tennessee State Medicaid, and Optum. RESULTS: Code‐based algorithm performance was excellent for opioid‐related overdoses (sensitivity = 97.2%, specificity = 84.6%) and classification of heroin‐involved overdoses (sensitivity = 91.8%, specificity = 99.0%). Performance was acceptable for code‐based suicide/suicide attempt classifications (sensitivity = 70.7%, specificity = 90.5%); sensitivity improved with NLP (sensitivity = 78.7%, specificity = 91.0%). Performance was acceptable for the code‐based substance abuse‐involved classification (sensitivity = 75.3%, specificity = 79.5%); sensitivity improved with the NLP‐enhanced algorithm (sensitivity = 80.5%, specificity = 76.3%). The opioid‐related overdose algorithm performed well across portability assessment sites, with sensitivity greater than 96% and specificity greater than 84%. Cross‐site sensitivity for heroin‐involved overdose was greater than 87%, specificity greater than or equal to 99%. CONCLUSIONS: Code‐based algorithms developed to detect opioid‐related overdoses and classify them according to heroin involvement perform well. Algorithms for classifying suicides/attempts and abuse‐related opioid overdoses perform adequately for use for research, particularly given the complexity of classifying such overdoses. The NLP‐enhanced algorithms for suicides/suicide attempts and abuse‐related overdoses perform significantly better than code‐based algorithms and are appropriate for use in settings that have data and capacity to use NLP.
format Online
Article
Text
id pubmed-6767606
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher John Wiley and Sons Inc.
record_format MEDLINE/PubMed
spelling pubmed-67676062019-10-03 Identifying and classifying opioid‐related overdoses: A validation study Green, Carla A. Perrin, Nancy A. Hazlehurst, Brian Janoff, Shannon L. DeVeaugh‐Geiss, Angela Carrell, David S. Grijalva, Carlos G. Liang, Caihua Enger, Cheryl L. Coplan, Paul M. Pharmacoepidemiol Drug Saf Original Reports PURPOSE: The study aims to develop and validate algorithms to identify and classify opioid overdoses using claims and other coded data, and clinical text extracted from electronic health records using natural language processing (NLP). METHODS: Primary data were derived from Kaiser Permanente Northwest (2008–2014), an integrated health care system (~n > 475 000 unique individuals per year). Data included International Classification of Diseases, Ninth Revision (ICD‐9) codes for nonfatal diagnoses, International Classification of Diseases, Tenth Revision (ICD‐10) codes for fatal events, clinical notes, and prescription medication records. We assessed sensitivity, specificity, positive predictive value, and negative predictive value for algorithms relative to medical chart review and conducted assessments of algorithm portability in Kaiser Permanente Washington, Tennessee State Medicaid, and Optum. RESULTS: Code‐based algorithm performance was excellent for opioid‐related overdoses (sensitivity = 97.2%, specificity = 84.6%) and classification of heroin‐involved overdoses (sensitivity = 91.8%, specificity = 99.0%). Performance was acceptable for code‐based suicide/suicide attempt classifications (sensitivity = 70.7%, specificity = 90.5%); sensitivity improved with NLP (sensitivity = 78.7%, specificity = 91.0%). Performance was acceptable for the code‐based substance abuse‐involved classification (sensitivity = 75.3%, specificity = 79.5%); sensitivity improved with the NLP‐enhanced algorithm (sensitivity = 80.5%, specificity = 76.3%). The opioid‐related overdose algorithm performed well across portability assessment sites, with sensitivity greater than 96% and specificity greater than 84%. Cross‐site sensitivity for heroin‐involved overdose was greater than 87%, specificity greater than or equal to 99%. CONCLUSIONS: Code‐based algorithms developed to detect opioid‐related overdoses and classify them according to heroin involvement perform well. Algorithms for classifying suicides/attempts and abuse‐related opioid overdoses perform adequately for use for research, particularly given the complexity of classifying such overdoses. The NLP‐enhanced algorithms for suicides/suicide attempts and abuse‐related overdoses perform significantly better than code‐based algorithms and are appropriate for use in settings that have data and capacity to use NLP. John Wiley and Sons Inc. 2019-04-24 2019-08 /pmc/articles/PMC6767606/ /pubmed/31020755 http://dx.doi.org/10.1002/pds.4772 Text en © 2019 The Authors. Pharmacoepidemiology and Drug Safety Published by John Wiley & Sons Ltd. This is an open access article under the terms of the http://creativecommons.org/licenses/by-nc-nd/4.0/ License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made.
spellingShingle Original Reports
Green, Carla A.
Perrin, Nancy A.
Hazlehurst, Brian
Janoff, Shannon L.
DeVeaugh‐Geiss, Angela
Carrell, David S.
Grijalva, Carlos G.
Liang, Caihua
Enger, Cheryl L.
Coplan, Paul M.
Identifying and classifying opioid‐related overdoses: A validation study
title Identifying and classifying opioid‐related overdoses: A validation study
title_full Identifying and classifying opioid‐related overdoses: A validation study
title_fullStr Identifying and classifying opioid‐related overdoses: A validation study
title_full_unstemmed Identifying and classifying opioid‐related overdoses: A validation study
title_short Identifying and classifying opioid‐related overdoses: A validation study
title_sort identifying and classifying opioid‐related overdoses: a validation study
topic Original Reports
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6767606/
https://www.ncbi.nlm.nih.gov/pubmed/31020755
http://dx.doi.org/10.1002/pds.4772
work_keys_str_mv AT greencarlaa identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT perrinnancya identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT hazlehurstbrian identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT janoffshannonl identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT deveaughgeissangela identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT carrelldavids identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT grijalvacarlosg identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT liangcaihua identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT engercheryll identifyingandclassifyingopioidrelatedoverdosesavalidationstudy
AT coplanpaulm identifyingandclassifyingopioidrelatedoverdosesavalidationstudy