Cargando…

Detecting shortcut learning for fair medical AI using shortcut testing

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models—their tendency to perform differently across subgroups of the population—...

Descripción completa

Detalles Bibliográficos
Autores principales: Brown, Alexander, Tomasev, Nenad, Freyberg, Jan, Liu, Yuan, Karthikesalingam, Alan, Schrouff, Jessica
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10354021/
https://www.ncbi.nlm.nih.gov/pubmed/37463884
http://dx.doi.org/10.1038/s41467-023-39902-7
_version_ 1785074830820769792
author Brown, Alexander
Tomasev, Nenad
Freyberg, Jan
Liu, Yuan
Karthikesalingam, Alan
Schrouff, Jessica
author_facet Brown, Alexander
Tomasev, Nenad
Freyberg, Jan
Liu, Yuan
Karthikesalingam, Alan
Schrouff, Jessica
author_sort Brown, Alexander
collection PubMed
description Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models—their tendency to perform differently across subgroups of the population—and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Diagnosing this phenomenon is difficult as sensitive attributes may be causally linked with disease. Using multitask learning, we propose a method to directly test for the presence of shortcut learning in clinical ML systems and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI.
format Online
Article
Text
id pubmed-10354021
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-103540212023-07-20 Detecting shortcut learning for fair medical AI using shortcut testing Brown, Alexander Tomasev, Nenad Freyberg, Jan Liu, Yuan Karthikesalingam, Alan Schrouff, Jessica Nat Commun Article Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models—their tendency to perform differently across subgroups of the population—and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Diagnosing this phenomenon is difficult as sensitive attributes may be causally linked with disease. Using multitask learning, we propose a method to directly test for the presence of shortcut learning in clinical ML systems and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI. Nature Publishing Group UK 2023-07-18 /pmc/articles/PMC10354021/ /pubmed/37463884 http://dx.doi.org/10.1038/s41467-023-39902-7 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Brown, Alexander
Tomasev, Nenad
Freyberg, Jan
Liu, Yuan
Karthikesalingam, Alan
Schrouff, Jessica
Detecting shortcut learning for fair medical AI using shortcut testing
title Detecting shortcut learning for fair medical AI using shortcut testing
title_full Detecting shortcut learning for fair medical AI using shortcut testing
title_fullStr Detecting shortcut learning for fair medical AI using shortcut testing
title_full_unstemmed Detecting shortcut learning for fair medical AI using shortcut testing
title_short Detecting shortcut learning for fair medical AI using shortcut testing
title_sort detecting shortcut learning for fair medical ai using shortcut testing
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10354021/
https://www.ncbi.nlm.nih.gov/pubmed/37463884
http://dx.doi.org/10.1038/s41467-023-39902-7
work_keys_str_mv AT brownalexander detectingshortcutlearningforfairmedicalaiusingshortcuttesting
AT tomasevnenad detectingshortcutlearningforfairmedicalaiusingshortcuttesting
AT freybergjan detectingshortcutlearningforfairmedicalaiusingshortcuttesting
AT liuyuan detectingshortcutlearningforfairmedicalaiusingshortcuttesting
AT karthikesalingamalan detectingshortcutlearningforfairmedicalaiusingshortcuttesting
AT schrouffjessica detectingshortcutlearningforfairmedicalaiusingshortcuttesting