Cargando…

Automating hybrid collective intelligence in open-ended medical diagnostics

Collective intelligence has emerged as a powerful mechanism to boost decision accuracy across many domains, such as geopolitical forecasting, investment, and medical diagnostics. However, collective intelligence has been mostly applied to relatively simple decision tasks (e.g., binary classification...

Descripción completa

Detalles Bibliográficos
Autores principales: Kurvers, Ralf H. J. M., Nuzzolese, Andrea Giovanni, Russo, Alessandro, Barabucci, Gioele, Herzog, Stefan M., Trianni, Vito
Formato: Online Artículo Texto
Lenguaje:English
Publicado: National Academy of Sciences 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10450668/
https://www.ncbi.nlm.nih.gov/pubmed/37579152
http://dx.doi.org/10.1073/pnas.2221473120
_version_ 1785095249238949888
author Kurvers, Ralf H. J. M.
Nuzzolese, Andrea Giovanni
Russo, Alessandro
Barabucci, Gioele
Herzog, Stefan M.
Trianni, Vito
author_facet Kurvers, Ralf H. J. M.
Nuzzolese, Andrea Giovanni
Russo, Alessandro
Barabucci, Gioele
Herzog, Stefan M.
Trianni, Vito
author_sort Kurvers, Ralf H. J. M.
collection PubMed
description Collective intelligence has emerged as a powerful mechanism to boost decision accuracy across many domains, such as geopolitical forecasting, investment, and medical diagnostics. However, collective intelligence has been mostly applied to relatively simple decision tasks (e.g., binary classifications). Applications in more open-ended tasks with a much larger problem space, such as emergency management or general medical diagnostics, are largely lacking, due to the challenge of integrating unstandardized inputs from different crowd members. Here, we present a fully automated approach for harnessing collective intelligence in the domain of general medical diagnostics. Our approach leverages semantic knowledge graphs, natural language processing, and the SNOMED CT medical ontology to overcome a major hurdle to collective intelligence in open-ended medical diagnostics, namely to identify the intended diagnosis from unstructured text. We tested our method on 1,333 medical cases diagnosed on a medical crowdsourcing platform: The Human Diagnosis Project. Each case was independently rated by ten diagnosticians. Comparing the diagnostic accuracy of single diagnosticians with the collective diagnosis of differently sized groups, we find that our method substantially increases diagnostic accuracy: While single diagnosticians achieved 46% accuracy, pooling the decisions of ten diagnosticians increased this to 76%. Improvements occurred across medical specialties, chief complaints, and diagnosticians’ tenure levels. Our results show the life-saving potential of tapping into the collective intelligence of the global medical community to reduce diagnostic errors and increase patient safety.
format Online
Article
Text
id pubmed-10450668
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher National Academy of Sciences
record_format MEDLINE/PubMed
spelling pubmed-104506682023-08-26 Automating hybrid collective intelligence in open-ended medical diagnostics Kurvers, Ralf H. J. M. Nuzzolese, Andrea Giovanni Russo, Alessandro Barabucci, Gioele Herzog, Stefan M. Trianni, Vito Proc Natl Acad Sci U S A Social Sciences Collective intelligence has emerged as a powerful mechanism to boost decision accuracy across many domains, such as geopolitical forecasting, investment, and medical diagnostics. However, collective intelligence has been mostly applied to relatively simple decision tasks (e.g., binary classifications). Applications in more open-ended tasks with a much larger problem space, such as emergency management or general medical diagnostics, are largely lacking, due to the challenge of integrating unstandardized inputs from different crowd members. Here, we present a fully automated approach for harnessing collective intelligence in the domain of general medical diagnostics. Our approach leverages semantic knowledge graphs, natural language processing, and the SNOMED CT medical ontology to overcome a major hurdle to collective intelligence in open-ended medical diagnostics, namely to identify the intended diagnosis from unstructured text. We tested our method on 1,333 medical cases diagnosed on a medical crowdsourcing platform: The Human Diagnosis Project. Each case was independently rated by ten diagnosticians. Comparing the diagnostic accuracy of single diagnosticians with the collective diagnosis of differently sized groups, we find that our method substantially increases diagnostic accuracy: While single diagnosticians achieved 46% accuracy, pooling the decisions of ten diagnosticians increased this to 76%. Improvements occurred across medical specialties, chief complaints, and diagnosticians’ tenure levels. Our results show the life-saving potential of tapping into the collective intelligence of the global medical community to reduce diagnostic errors and increase patient safety. National Academy of Sciences 2023-08-14 2023-08-22 /pmc/articles/PMC10450668/ /pubmed/37579152 http://dx.doi.org/10.1073/pnas.2221473120 Text en Copyright © 2023 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by/4.0/This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY) (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Social Sciences
Kurvers, Ralf H. J. M.
Nuzzolese, Andrea Giovanni
Russo, Alessandro
Barabucci, Gioele
Herzog, Stefan M.
Trianni, Vito
Automating hybrid collective intelligence in open-ended medical diagnostics
title Automating hybrid collective intelligence in open-ended medical diagnostics
title_full Automating hybrid collective intelligence in open-ended medical diagnostics
title_fullStr Automating hybrid collective intelligence in open-ended medical diagnostics
title_full_unstemmed Automating hybrid collective intelligence in open-ended medical diagnostics
title_short Automating hybrid collective intelligence in open-ended medical diagnostics
title_sort automating hybrid collective intelligence in open-ended medical diagnostics
topic Social Sciences
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10450668/
https://www.ncbi.nlm.nih.gov/pubmed/37579152
http://dx.doi.org/10.1073/pnas.2221473120
work_keys_str_mv AT kurversralfhjm automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics
AT nuzzoleseandreagiovanni automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics
AT russoalessandro automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics
AT barabuccigioele automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics
AT herzogstefanm automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics
AT triannivito automatinghybridcollectiveintelligenceinopenendedmedicaldiagnostics