Cargando…

Medical prediction from missing data with max-minus negative regularized dropout

Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generaliz...

Descripción completa

Detalles Bibliográficos
Autores principales: Hu, Lvhui, Cheng, Xiaoen, Wen, Chuanbiao, Ren, Yulan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10373302/
https://www.ncbi.nlm.nih.gov/pubmed/37521692
http://dx.doi.org/10.3389/fnins.2023.1221970
_version_ 1785078539080433664
author Hu, Lvhui
Cheng, Xiaoen
Wen, Chuanbiao
Ren, Yulan
author_facet Hu, Lvhui
Cheng, Xiaoen
Wen, Chuanbiao
Ren, Yulan
author_sort Hu, Lvhui
collection PubMed
description Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method.
format Online
Article
Text
id pubmed-10373302
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-103733022023-07-28 Medical prediction from missing data with max-minus negative regularized dropout Hu, Lvhui Cheng, Xiaoen Wen, Chuanbiao Ren, Yulan Front Neurosci Neuroscience Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method. Frontiers Media S.A. 2023-07-13 /pmc/articles/PMC10373302/ /pubmed/37521692 http://dx.doi.org/10.3389/fnins.2023.1221970 Text en Copyright © 2023 Hu, Cheng, Wen and Ren. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Hu, Lvhui
Cheng, Xiaoen
Wen, Chuanbiao
Ren, Yulan
Medical prediction from missing data with max-minus negative regularized dropout
title Medical prediction from missing data with max-minus negative regularized dropout
title_full Medical prediction from missing data with max-minus negative regularized dropout
title_fullStr Medical prediction from missing data with max-minus negative regularized dropout
title_full_unstemmed Medical prediction from missing data with max-minus negative regularized dropout
title_short Medical prediction from missing data with max-minus negative regularized dropout
title_sort medical prediction from missing data with max-minus negative regularized dropout
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10373302/
https://www.ncbi.nlm.nih.gov/pubmed/37521692
http://dx.doi.org/10.3389/fnins.2023.1221970
work_keys_str_mv AT hulvhui medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout
AT chengxiaoen medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout
AT wenchuanbiao medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout
AT renyulan medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout