Cargando…

A machine and human reader study on AI diagnosis model safety under attacks of adversarial images

While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhou, Qianwei, Zuley, Margarita, Guo, Yuan, Yang, Lu, Nair, Bronwyn, Vargo, Adrienne, Ghannam, Suzanne, Arefan, Dooman, Wu, Shandong
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8671500/
https://www.ncbi.nlm.nih.gov/pubmed/34907229
http://dx.doi.org/10.1038/s41467-021-27577-x
_version_ 1784615150803746816
author Zhou, Qianwei
Zuley, Margarita
Guo, Yuan
Yang, Lu
Nair, Bronwyn
Vargo, Adrienne
Ghannam, Suzanne
Arefan, Dooman
Wu, Shandong
author_facet Zhou, Qianwei
Zuley, Margarita
Guo, Yuan
Yang, Lu
Nair, Bronwyn
Vargo, Adrienne
Ghannam, Suzanne
Arefan, Dooman
Wu, Shandong
author_sort Zhou, Qianwei
collection PubMed
description While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by Generative Adversarial Network (GAN) models and to evaluate the effects on human experts when visually identifying potential adversarial images. Our GAN model makes intentional modifications to the diagnosis-sensitive contents of mammogram images in deep learning-based computer-aided diagnosis (CAD) of breast cancer. In our experiments the adversarial samples fool the AI-CAD model to output a wrong diagnosis on 69.1% of the cases that are initially correctly classified by the AI-CAD model. Five breast imaging radiologists visually identify 29%-71% of the adversarial samples. Our study suggests an imperative need for continuing research on medical AI model’s safety issues and for developing potential defensive solutions against adversarial attacks.
format Online
Article
Text
id pubmed-8671500
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-86715002022-01-04 A machine and human reader study on AI diagnosis model safety under attacks of adversarial images Zhou, Qianwei Zuley, Margarita Guo, Yuan Yang, Lu Nair, Bronwyn Vargo, Adrienne Ghannam, Suzanne Arefan, Dooman Wu, Shandong Nat Commun Article While active efforts are advancing medical artificial intelligence (AI) model development and clinical translation, safety issues of the AI models emerge, but little research has been done. We perform a study to investigate the behaviors of an AI diagnosis model under adversarial images generated by Generative Adversarial Network (GAN) models and to evaluate the effects on human experts when visually identifying potential adversarial images. Our GAN model makes intentional modifications to the diagnosis-sensitive contents of mammogram images in deep learning-based computer-aided diagnosis (CAD) of breast cancer. In our experiments the adversarial samples fool the AI-CAD model to output a wrong diagnosis on 69.1% of the cases that are initially correctly classified by the AI-CAD model. Five breast imaging radiologists visually identify 29%-71% of the adversarial samples. Our study suggests an imperative need for continuing research on medical AI model’s safety issues and for developing potential defensive solutions against adversarial attacks. Nature Publishing Group UK 2021-12-14 /pmc/articles/PMC8671500/ /pubmed/34907229 http://dx.doi.org/10.1038/s41467-021-27577-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Article
Zhou, Qianwei
Zuley, Margarita
Guo, Yuan
Yang, Lu
Nair, Bronwyn
Vargo, Adrienne
Ghannam, Suzanne
Arefan, Dooman
Wu, Shandong
A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title_full A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title_fullStr A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title_full_unstemmed A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title_short A machine and human reader study on AI diagnosis model safety under attacks of adversarial images
title_sort machine and human reader study on ai diagnosis model safety under attacks of adversarial images
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8671500/
https://www.ncbi.nlm.nih.gov/pubmed/34907229
http://dx.doi.org/10.1038/s41467-021-27577-x
work_keys_str_mv AT zhouqianwei amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT zuleymargarita amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT guoyuan amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT yanglu amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT nairbronwyn amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT vargoadrienne amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT ghannamsuzanne amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT arefandooman amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT wushandong amachineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT zhouqianwei machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT zuleymargarita machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT guoyuan machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT yanglu machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT nairbronwyn machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT vargoadrienne machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT ghannamsuzanne machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT arefandooman machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages
AT wushandong machineandhumanreaderstudyonaidiagnosismodelsafetyunderattacksofadversarialimages