Cargando…
Medical prediction from missing data with max-minus negative regularized dropout
Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generaliz...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10373302/ https://www.ncbi.nlm.nih.gov/pubmed/37521692 http://dx.doi.org/10.3389/fnins.2023.1221970 |
_version_ | 1785078539080433664 |
---|---|
author | Hu, Lvhui Cheng, Xiaoen Wen, Chuanbiao Ren, Yulan |
author_facet | Hu, Lvhui Cheng, Xiaoen Wen, Chuanbiao Ren, Yulan |
author_sort | Hu, Lvhui |
collection | PubMed |
description | Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method. |
format | Online Article Text |
id | pubmed-10373302 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-103733022023-07-28 Medical prediction from missing data with max-minus negative regularized dropout Hu, Lvhui Cheng, Xiaoen Wen, Chuanbiao Ren, Yulan Front Neurosci Neuroscience Missing data is a naturally common problem faced in medical research. Imputation is a widely used technique to alleviate this problem. Unfortunately, the inherent uncertainty of imputation would make the model overfit the observed data distribution, which has a negative impact on the model generalization performance. R-Drop is a powerful technique to regularize the training of deep neural networks. However, it fails to differentiate the positive and negative samples, which prevents the model from learning robust representations. To handle this problem, we propose a novel negative regularization enhanced R-Drop scheme to boost performance and generalization ability, particularly in the context of missing data. The negative regularization enhanced R-Drop additionally forces the output distributions of positive and negative samples to be inconsistent with each other. Especially, we design a new max-minus negative sampling technique that uses the maximum in-batch values to minus the mini-batch to yield the negative samples to provide sufficient diversity for the model. We test the resulting max-minus negative regularized dropout method on three real-world medical prediction datasets, including both missing and complete cases, to show the effectiveness of the proposed method. Frontiers Media S.A. 2023-07-13 /pmc/articles/PMC10373302/ /pubmed/37521692 http://dx.doi.org/10.3389/fnins.2023.1221970 Text en Copyright © 2023 Hu, Cheng, Wen and Ren. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Hu, Lvhui Cheng, Xiaoen Wen, Chuanbiao Ren, Yulan Medical prediction from missing data with max-minus negative regularized dropout |
title | Medical prediction from missing data with max-minus negative regularized dropout |
title_full | Medical prediction from missing data with max-minus negative regularized dropout |
title_fullStr | Medical prediction from missing data with max-minus negative regularized dropout |
title_full_unstemmed | Medical prediction from missing data with max-minus negative regularized dropout |
title_short | Medical prediction from missing data with max-minus negative regularized dropout |
title_sort | medical prediction from missing data with max-minus negative regularized dropout |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10373302/ https://www.ncbi.nlm.nih.gov/pubmed/37521692 http://dx.doi.org/10.3389/fnins.2023.1221970 |
work_keys_str_mv | AT hulvhui medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT chengxiaoen medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT wenchuanbiao medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout AT renyulan medicalpredictionfrommissingdatawithmaxminusnegativeregularizeddropout |