Cargando…

A proposal for developing a platform that evaluates algorithmic equity and accuracy

We are at a pivotal moment in the development of healthcare artificial intelligence (AI), a point at which enthusiasm for machine learning has not caught up with the scientific evidence to support the equity and accuracy of diagnostic and therapeutic algorithms. This proposal examines algorithmic bi...

Descripción completa

Detalles Bibliográficos
Autores principales: Cerrato, Paul, Halamka, John, Pencina, Michael
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BMJ Publishing Group 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9003600/
https://www.ncbi.nlm.nih.gov/pubmed/35410952
http://dx.doi.org/10.1136/bmjhci-2021-100423
_version_ 1784686165685698560
author Cerrato, Paul
Halamka, John
Pencina, Michael
author_facet Cerrato, Paul
Halamka, John
Pencina, Michael
author_sort Cerrato, Paul
collection PubMed
description We are at a pivotal moment in the development of healthcare artificial intelligence (AI), a point at which enthusiasm for machine learning has not caught up with the scientific evidence to support the equity and accuracy of diagnostic and therapeutic algorithms. This proposal examines algorithmic biases, including those related to race, gender and socioeconomic status, and accuracy, including the paucity of prospective studies and lack of multisite validation. We then suggest solutions to these problems. We describe the Mayo Clinic, Duke University, Change Healthcare project that is evaluating 35.1 billion healthcare records for bias. And we propose ‘Ingredients’ style labels and an AI evaluation/testing system to help clinicians judge the merits of products and services that include algorithms. Said testing would include input data sources and types, dataset population composition, algorithm validation techniques, bias assessment evaluation and performance metrics.
format Online
Article
Text
id pubmed-9003600
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher BMJ Publishing Group
record_format MEDLINE/PubMed
spelling pubmed-90036002022-04-27 A proposal for developing a platform that evaluates algorithmic equity and accuracy Cerrato, Paul Halamka, John Pencina, Michael BMJ Health Care Inform Review We are at a pivotal moment in the development of healthcare artificial intelligence (AI), a point at which enthusiasm for machine learning has not caught up with the scientific evidence to support the equity and accuracy of diagnostic and therapeutic algorithms. This proposal examines algorithmic biases, including those related to race, gender and socioeconomic status, and accuracy, including the paucity of prospective studies and lack of multisite validation. We then suggest solutions to these problems. We describe the Mayo Clinic, Duke University, Change Healthcare project that is evaluating 35.1 billion healthcare records for bias. And we propose ‘Ingredients’ style labels and an AI evaluation/testing system to help clinicians judge the merits of products and services that include algorithms. Said testing would include input data sources and types, dataset population composition, algorithm validation techniques, bias assessment evaluation and performance metrics. BMJ Publishing Group 2022-04-10 /pmc/articles/PMC9003600/ /pubmed/35410952 http://dx.doi.org/10.1136/bmjhci-2021-100423 Text en © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ. https://creativecommons.org/licenses/by-nc/4.0/This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ (https://creativecommons.org/licenses/by-nc/4.0/) .
spellingShingle Review
Cerrato, Paul
Halamka, John
Pencina, Michael
A proposal for developing a platform that evaluates algorithmic equity and accuracy
title A proposal for developing a platform that evaluates algorithmic equity and accuracy
title_full A proposal for developing a platform that evaluates algorithmic equity and accuracy
title_fullStr A proposal for developing a platform that evaluates algorithmic equity and accuracy
title_full_unstemmed A proposal for developing a platform that evaluates algorithmic equity and accuracy
title_short A proposal for developing a platform that evaluates algorithmic equity and accuracy
title_sort proposal for developing a platform that evaluates algorithmic equity and accuracy
topic Review
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9003600/
https://www.ncbi.nlm.nih.gov/pubmed/35410952
http://dx.doi.org/10.1136/bmjhci-2021-100423
work_keys_str_mv AT cerratopaul aproposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy
AT halamkajohn aproposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy
AT pencinamichael aproposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy
AT cerratopaul proposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy
AT halamkajohn proposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy
AT pencinamichael proposalfordevelopingaplatformthatevaluatesalgorithmicequityandaccuracy