Cargando…
A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8671500/ https://www.ncbi.nlm.nih.gov/pubmed/34907229 http://dx.doi.org/10.1038/s41467-021-27577-x |
_version_ | 1784615150803746816 |
---|---|
author | Zhou, Qianwei Zuley, Margarita Guo, Yuan Yang, Lu Nair, Bronwyn Vargo, Adrienne Ghannam, Suzanne Arefan, Dooman Wu, Shandong |
author_facet | Zhou, Qianwei Zuley, Margarita Guo, Yuan Yang, Lu Nair, Bronwyn Vargo, Adrienne Ghannam, Suzanne Arefan, Dooman Wu, Shandong |
author_sort | Zhou, Qianwei |
collection | PubMed |
description | While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by Generative Adversarial Network (GAN) models and to evaluate the effects on human experts when visually identifying potential adversarial images. Our GAN model makes intentional modifications to the diagnosis-sensitive contents of mammogram images in deep learning-based computer-aided diagnosis (CAD) of breast cancer. In our experiments the adversarial samples fool the AI-CAD model to output a wrong diagnosis on 69.1% of the cases that are initially correctly classified by the AI-CAD model. Five breast imaging radiologists visually identify 29%-71% of the adversarial samples. Our study suggests an imperative need for continuing research on medical AI model’s safety issues and for developing potential defensive solutions against adversarial attacks. |
format | Online Article Text |
id | pubmed-8671500 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-86715002022-01-04 A machine and human reader study on AI diagnosis model safety under attacks of adversarial images Zhou, Qianwei Zuley, Margarita Guo, Yuan Yang, Lu Nair, Bronwyn Vargo, Adrienne Ghannam, Suzanne Arefan, Dooman Wu, Shandong Nat Commun Article While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by Generative Adversarial Network (GAN) models and to evaluate the effects on human experts when visually identifying potential adversarial images. Our GAN model makes intentional modifications to the diagnosis-sensitive contents of mammogram images in deep learning-based computer-aided diagnosis (CAD) of breast cancer. In our experiments the adversarial samples fool the AI-CAD model to output a wrong diagnosis on 69.1% of the cases that are initially correctly classified by the AI-CAD model. Five breast imaging radiologists visually identify 29%-71% of the adversarial samples. Our study suggests an imperative need for continuing research on medical AI model’s safety issues and for developing potential defensive solutions against adversarial attacks. Nature Publishing Group UK 2021-12-14 /pmc/articles/PMC8671500/ /pubmed/34907229 http://dx.doi.org/10.1038/s41467-021-27577-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Zhou, Qianwei Zuley, Margarita Guo, Yuan Yang, Lu Nair, Bronwyn Vargo, Adrienne Ghannam, Suzanne Arefan, Dooman Wu, Shandong A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title | A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title_full | A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title_fullStr | A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title_full_unstemmed | A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title_short | A machine and human reader study on AI diagnosis model safety under attacks of adversarial images |
title_sort | machine and human reader study on ai diagnosis model safety under attacks of adversarial images |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8671500/ https://www.ncbi.nlm.nih.gov/pubmed/34907229 http://dx.doi.org/10.1038/s41467-021-27577-x |
work_keys_str_mv | AT zhouqianwei amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT zuleymargarita amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT guoyuan amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT yanglu amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT nairbronwyn amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT vargoadrienne amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT ghannamsuzanne amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT arefandooman amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT wushandong amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT zhouqianwei machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT zuleymargarita machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT guoyuan machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT yanglu machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT nairbronwyn machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT vargoadrienne machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT ghannamsuzanne machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT arefandooman machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages AT wushandong machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages |