Cargando…

Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury

Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate...

Descripción completa

Detalles Bibliográficos
Autores principales: Connor, Skylar, Li, Ting, Roberts, Ruth, Thakkar, Shraddha, Liu, Zhichao, Tong, Weida
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9679417/
https://www.ncbi.nlm.nih.gov/pubmed/36425225
http://dx.doi.org/10.3389/frai.2022.1034631
_version_ 1784834185550102528
author Connor, Skylar
Li, Ting
Roberts, Ruth
Thakkar, Shraddha
Liu, Zhichao
Tong, Weida
author_facet Connor, Skylar
Li, Ting
Roberts, Ruth
Thakkar, Shraddha
Liu, Zhichao
Tong, Weida
author_sort Connor, Skylar
collection PubMed
description Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time.
format Online
Article
Text
id pubmed-9679417
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-96794172022-11-23 Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury Connor, Skylar Li, Ting Roberts, Ruth Thakkar, Shraddha Liu, Zhichao Tong, Weida Front Artif Intell Artificial Intelligence Artificial intelligence (AI) has played a crucial role in advancing biomedical sciences but has yet to have the impact it merits in regulatory science. As the field advances, in silico and in vitro approaches have been evaluated as alternatives to animal studies, in a drive to identify and mitigate safety concerns earlier in the drug development process. Although many AI tools are available, their acceptance in regulatory decision-making for drug efficacy and safety evaluation is still a challenge. It is a common perception that an AI model improves with more data, but does reality reflect this perception in drug safety assessments? Importantly, a model aiming at regulatory application needs to take a broad range of model characteristics into consideration. Among them is adaptability, defined as the adaptive behavior of a model as it is retrained on unseen data. This is an important model characteristic which should be considered in regulatory applications. In this study, we set up a comprehensive study to assess adaptability in AI by mimicking the real-world scenario of the annual addition of new drugs to the market, using a model we previously developed known as DeepDILI for predicting drug-induced liver injury (DILI) with a novel Deep Learning method. We found that the target test set plays a major role in assessing the adaptive behavior of our model. Our findings also indicated that adding more drugs to the training set does not significantly affect the predictive performance of our adaptive model. We concluded that the proposed adaptability assessment framework has utility in the evaluation of the performance of a model over time. Frontiers Media S.A. 2022-11-08 /pmc/articles/PMC9679417/ /pubmed/36425225 http://dx.doi.org/10.3389/frai.2022.1034631 Text en Copyright © 2022 Connor, Li, Roberts, Thakkar, Liu and Tong. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Artificial Intelligence
Connor, Skylar
Li, Ting
Roberts, Ruth
Thakkar, Shraddha
Liu, Zhichao
Tong, Weida
Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title_full Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title_fullStr Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title_full_unstemmed Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title_short Adaptability of AI for safety evaluation in regulatory science: A case study of drug-induced liver injury
title_sort adaptability of ai for safety evaluation in regulatory science: a case study of drug-induced liver injury
topic Artificial Intelligence
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9679417/
https://www.ncbi.nlm.nih.gov/pubmed/36425225
http://dx.doi.org/10.3389/frai.2022.1034631
work_keys_str_mv AT connorskylar adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury
AT liting adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury
AT robertsruth adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury
AT thakkarshraddha adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury
AT liuzhichao adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury
AT tongweida adaptabilityofaiforsafetyevaluationinregulatoryscienceacasestudyofdruginducedliverinjury