Cargando…

The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset

INTRODUCTION: This article was undertaken to explore the potential of AI in enhancing the diagnostic accuracy and efficiency in identifying hip fractures using X-ray radiographs. In the study, we trained three distinct deep learning models, and we utilized majority voting to evaluate their outcomes,...

Descripción completa

Detalles Bibliográficos
Autores principales: Beyaz, Salih, Yayli, Sahika Betul, Kılıc, Ersin, Doktur, Ugur
Formato: Online Artículo Texto
Lenguaje:English
Publicado: SAGE Publications 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10685786/
https://www.ncbi.nlm.nih.gov/pubmed/38033522
http://dx.doi.org/10.1177/20552076231216549
_version_ 1785151704017141760
author Beyaz, Salih
Yayli, Sahika Betul
Kılıc, Ersin
Doktur, Ugur
author_facet Beyaz, Salih
Yayli, Sahika Betul
Kılıc, Ersin
Doktur, Ugur
author_sort Beyaz, Salih
collection PubMed
description INTRODUCTION: This article was undertaken to explore the potential of AI in enhancing the diagnostic accuracy and efficiency in identifying hip fractures using X-ray radiographs. In the study, we trained three distinct deep learning models, and we utilized majority voting to evaluate their outcomes, aiming to yield the most reliable and precise diagnoses of hip fractures from X-ray radiographs. METHODS: An initial study was conducted of 10,849 AP pelvis X-rays obtained from five hospitals affiliated with Başkent University. Two expert orthopedic surgeons initially labeled 2,291 radiographs as fractures and 8,558 as non-fractures. The algorithm was trained on 6,943 (64%) radiographs, validated on 1,736 (16%) radiographs, and tested on 2,170 (20%) radiographs, ensuring an even distribution of fracture presence, age, and gender. We employed three advanced deep learning architectures, Xception (Model A), EfficientNet (Model B), and NfNet (Model C), with a final decision aggregated through a majority voting technique (Model D). RESULTS: For each model, we achieved the following metrics: For Model A: F1 Score 0.895, Accuracy 0.956, Specificity 0.973, Sensitivity 0.893. For Model B: F1 Score 0.900, Accuracy 0.960, Specificity 0.991, Sensitivity 0.845. For Model C: F1 Score 0.919, Accuracy 0.966, Specificity 0.984, Sensitivity 0.899. For Model D: F1 Score 0.929, Accuracy 0.971, Specificity 0.991, Sensitivity 0.897. We concluded that Model D (majority voting) achieved the best results in terms of the F1 score, accuracy, and specificity values. CONCLUSIONS: Our study demonstrates that the results obtained by aggregating the decisions of multiple models through voting, rather than relying solely on the decision of a single algorithm, are more consistent. The practical application of these algorithms will be difficult due to ethical, legal, and confidentiality issues, despite the theoretical success achieved. Developing successful algorithms and methodologies should not be viewed as the ultimate goal; it is important to understand how these algorithms will be used in real-life situations. In order to achieve more consistent results, feedback from clinical practice will be helpful.
format Online
Article
Text
id pubmed-10685786
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher SAGE Publications
record_format MEDLINE/PubMed
spelling pubmed-106857862023-11-30 The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset Beyaz, Salih Yayli, Sahika Betul Kılıc, Ersin Doktur, Ugur Digit Health Original Research INTRODUCTION: This article was undertaken to explore the potential of AI in enhancing the diagnostic accuracy and efficiency in identifying hip fractures using X-ray radiographs. In the study, we trained three distinct deep learning models, and we utilized majority voting to evaluate their outcomes, aiming to yield the most reliable and precise diagnoses of hip fractures from X-ray radiographs. METHODS: An initial study was conducted of 10,849 AP pelvis X-rays obtained from five hospitals affiliated with Başkent University. Two expert orthopedic surgeons initially labeled 2,291 radiographs as fractures and 8,558 as non-fractures. The algorithm was trained on 6,943 (64%) radiographs, validated on 1,736 (16%) radiographs, and tested on 2,170 (20%) radiographs, ensuring an even distribution of fracture presence, age, and gender. We employed three advanced deep learning architectures, Xception (Model A), EfficientNet (Model B), and NfNet (Model C), with a final decision aggregated through a majority voting technique (Model D). RESULTS: For each model, we achieved the following metrics: For Model A: F1 Score 0.895, Accuracy 0.956, Specificity 0.973, Sensitivity 0.893. For Model B: F1 Score 0.900, Accuracy 0.960, Specificity 0.991, Sensitivity 0.845. For Model C: F1 Score 0.919, Accuracy 0.966, Specificity 0.984, Sensitivity 0.899. For Model D: F1 Score 0.929, Accuracy 0.971, Specificity 0.991, Sensitivity 0.897. We concluded that Model D (majority voting) achieved the best results in terms of the F1 score, accuracy, and specificity values. CONCLUSIONS: Our study demonstrates that the results obtained by aggregating the decisions of multiple models through voting, rather than relying solely on the decision of a single algorithm, are more consistent. The practical application of these algorithms will be difficult due to ethical, legal, and confidentiality issues, despite the theoretical success achieved. Developing successful algorithms and methodologies should not be viewed as the ultimate goal; it is important to understand how these algorithms will be used in real-life situations. In order to achieve more consistent results, feedback from clinical practice will be helpful. SAGE Publications 2023-11-28 /pmc/articles/PMC10685786/ /pubmed/38033522 http://dx.doi.org/10.1177/20552076231216549 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by-nc/4.0/This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
spellingShingle Original Research
Beyaz, Salih
Yayli, Sahika Betul
Kılıc, Ersin
Doktur, Ugur
The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title_full The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title_fullStr The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title_full_unstemmed The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title_short The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset
title_sort ensemble artificial intelligence (ai) method: detection of hip fractures in ap pelvis plain radiographs by majority voting using a multi-center dataset
topic Original Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10685786/
https://www.ncbi.nlm.nih.gov/pubmed/38033522
http://dx.doi.org/10.1177/20552076231216549
work_keys_str_mv AT beyazsalih theensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT yaylisahikabetul theensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT kılıcersin theensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT dokturugur theensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT beyazsalih ensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT yaylisahikabetul ensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT kılıcersin ensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset
AT dokturugur ensembleartificialintelligenceaimethoddetectionofhipfracturesinappelvisplainradiographsbymajorityvotingusingamulticenterdataset