Cargando…

An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation

BACKGROUND: Medication errors account for a large proportion of all medical errors. In most homes, patients take a variety of medications for a long period. However, medication errors frequently occur because patients often throw away the containers of their medications. OBJECTIVE: We proposed a dee...

Descripción completa

Detalles Bibliográficos
Autores principales: Heo, Junyeong, Kang, Youjin, Lee, SangKeun, Jeong, Dong-Hwa, Kim, Kang-Min
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9883737/
https://www.ncbi.nlm.nih.gov/pubmed/36637893
http://dx.doi.org/10.2196/41043
_version_ 1784879568962715648
author Heo, Junyeong
Kang, Youjin
Lee, SangKeun
Jeong, Dong-Hwa
Kim, Kang-Min
author_facet Heo, Junyeong
Kang, Youjin
Lee, SangKeun
Jeong, Dong-Hwa
Kim, Kang-Min
author_sort Heo, Junyeong
collection PubMed
description BACKGROUND: Medication errors account for a large proportion of all medical errors. In most homes, patients take a variety of medications for a long period. However, medication errors frequently occur because patients often throw away the containers of their medications. OBJECTIVE: We proposed a deep learning–based system for reducing medication errors by accurately identifying prescription pills. Given the pill images, our system located the pills in the respective pill databases in South Korea and the United States. METHODS: We organized the system into a pill recognition step and pill retrieval step, and we applied deep learning models to train not only images of the pill but also imprinted characters. In the pill recognition step, there are 3 modules that recognize the 3 features of pills and their imprints separately and correct the recognized imprint to fit the actual data. We adopted image classification and text detection models for the feature and imprint recognition modules, respectively. In the imprint correction module, we introduced a language model for the first time in the pill identification system and proposed a novel coordinate encoding technique for effective correction in the language model. We identified pills using similarity scores of pill characteristics with those in the database. RESULTS: We collected the open pill database from South Korea and the United States in May 2022. We used a total of 24,404 pill images in our experiments. The experimental results show that the predicted top-1 candidates achieve accuracy levels of 85.6% (South Korea) and 74.5% (United States) for the types of pills not trained on 2 different databases (South Korea and the United States). Furthermore, the predicted top-1 candidate accuracy of our system was 78% with consumer-granted images, which was achieved by training only 1 image per pill. The results demonstrate that our system could identify and retrieve new pills without additional model updates. Finally, we confirmed through an ablation study that the language model that we emphasized significantly improves the pill identification ability of the system. CONCLUSIONS: Our study proposes the possibility of reducing medical errors by showing that the introduction of artificial intelligence can identify numerous pills with high precision in real time. Our study suggests that the proposed system can reduce patients’ misuse of medications and help medical staff focus on higher-level tasks by simplifying time-consuming lower-level tasks such as pill identification.
format Online
Article
Text
id pubmed-9883737
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-98837372023-01-29 An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation Heo, Junyeong Kang, Youjin Lee, SangKeun Jeong, Dong-Hwa Kim, Kang-Min J Med Internet Res Original Paper BACKGROUND: Medication errors account for a large proportion of all medical errors. In most homes, patients take a variety of medications for a long period. However, medication errors frequently occur because patients often throw away the containers of their medications. OBJECTIVE: We proposed a deep learning–based system for reducing medication errors by accurately identifying prescription pills. Given the pill images, our system located the pills in the respective pill databases in South Korea and the United States. METHODS: We organized the system into a pill recognition step and pill retrieval step, and we applied deep learning models to train not only images of the pill but also imprinted characters. In the pill recognition step, there are 3 modules that recognize the 3 features of pills and their imprints separately and correct the recognized imprint to fit the actual data. We adopted image classification and text detection models for the feature and imprint recognition modules, respectively. In the imprint correction module, we introduced a language model for the first time in the pill identification system and proposed a novel coordinate encoding technique for effective correction in the language model. We identified pills using similarity scores of pill characteristics with those in the database. RESULTS: We collected the open pill database from South Korea and the United States in May 2022. We used a total of 24,404 pill images in our experiments. The experimental results show that the predicted top-1 candidates achieve accuracy levels of 85.6% (South Korea) and 74.5% (United States) for the types of pills not trained on 2 different databases (South Korea and the United States). Furthermore, the predicted top-1 candidate accuracy of our system was 78% with consumer-granted images, which was achieved by training only 1 image per pill. The results demonstrate that our system could identify and retrieve new pills without additional model updates. Finally, we confirmed through an ablation study that the language model that we emphasized significantly improves the pill identification ability of the system. CONCLUSIONS: Our study proposes the possibility of reducing medical errors by showing that the introduction of artificial intelligence can identify numerous pills with high precision in real time. Our study suggests that the proposed system can reduce patients’ misuse of medications and help medical staff focus on higher-level tasks by simplifying time-consuming lower-level tasks such as pill identification. JMIR Publications 2023-01-13 /pmc/articles/PMC9883737/ /pubmed/36637893 http://dx.doi.org/10.2196/41043 Text en ©Junyeong Heo, Youjin Kang, SangKeun Lee, Dong-Hwa Jeong, Kang-Min Kim. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.01.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Heo, Junyeong
Kang, Youjin
Lee, SangKeun
Jeong, Dong-Hwa
Kim, Kang-Min
An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title_full An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title_fullStr An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title_full_unstemmed An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title_short An Accurate Deep Learning–Based System for Automatic Pill Identification: Model Development and Validation
title_sort accurate deep learning–based system for automatic pill identification: model development and validation
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9883737/
https://www.ncbi.nlm.nih.gov/pubmed/36637893
http://dx.doi.org/10.2196/41043
work_keys_str_mv AT heojunyeong anaccuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT kangyoujin anaccuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT leesangkeun anaccuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT jeongdonghwa anaccuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT kimkangmin anaccuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT heojunyeong accuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT kangyoujin accuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT leesangkeun accuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT jeongdonghwa accuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation
AT kimkangmin accuratedeeplearningbasedsystemforautomaticpillidentificationmodeldevelopmentandvalidation