Cargando…

Beta Distribution-Based Cross-Entropy for Feature Selection

Analysis of high-dimensional data is a challenge in machine learning and data mining. Feature selection plays an important role in dealing with high-dimensional data for improvement of predictive accuracy, as well as better interpretation of the data. Frequently used evaluation functions for feature...

Descripción completa

Detalles Bibliográficos
Autores principales: Dai, Weixing, Guo, Dianjing
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515297/
https://www.ncbi.nlm.nih.gov/pubmed/33267482
http://dx.doi.org/10.3390/e21080769
_version_ 1783586784752435200
author Dai, Weixing
Guo, Dianjing
author_facet Dai, Weixing
Guo, Dianjing
author_sort Dai, Weixing
collection PubMed
description Analysis of high-dimensional data is a challenge in machine learning and data mining. Feature selection plays an important role in dealing with high-dimensional data for improvement of predictive accuracy, as well as better interpretation of the data. Frequently used evaluation functions for feature selection include resampling methods such as cross-validation, which show an advantage in predictive accuracy. However, these conventional methods are not only computationally expensive, but also tend to be over-optimistic. We propose a novel cross-entropy which is based on beta distribution for feature selection. In beta distribution-based cross-entropy (BetaDCE) for feature selection, the probability density is estimated by beta distribution and the cross-entropy is computed by the expected value of beta distribution, so that the generalization ability can be estimated more precisely than conventional methods where the probability density is learnt from data. Analysis of the generalization ability of BetaDCE revealed that it was a trade-off between bias and variance. The robustness of BetaDCE was demonstrated by experiments on three types of data. In the exclusive or-like (XOR-like) dataset, the false discovery rate of BetaDCE was significantly smaller than that of other methods. For the leukemia dataset, the area under the curve (AUC) of BetaDCE on the test set was 0.93 with only four selected features, which indicated that BetaDCE not only detected the irrelevant and redundant features precisely, but also more accurately predicted the class labels with a smaller number of features than the original method, whose AUC was 0.83 with 50 features. In the metabonomic dataset, the overall AUC of prediction with features selected by BetaDCE was significantly larger than that by the original reported method. Therefore, BetaDCE can be used as a general and efficient framework for feature selection.
format Online
Article
Text
id pubmed-7515297
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75152972020-11-09 Beta Distribution-Based Cross-Entropy for Feature Selection Dai, Weixing Guo, Dianjing Entropy (Basel) Article Analysis of high-dimensional data is a challenge in machine learning and data mining. Feature selection plays an important role in dealing with high-dimensional data for improvement of predictive accuracy, as well as better interpretation of the data. Frequently used evaluation functions for feature selection include resampling methods such as cross-validation, which show an advantage in predictive accuracy. However, these conventional methods are not only computationally expensive, but also tend to be over-optimistic. We propose a novel cross-entropy which is based on beta distribution for feature selection. In beta distribution-based cross-entropy (BetaDCE) for feature selection, the probability density is estimated by beta distribution and the cross-entropy is computed by the expected value of beta distribution, so that the generalization ability can be estimated more precisely than conventional methods where the probability density is learnt from data. Analysis of the generalization ability of BetaDCE revealed that it was a trade-off between bias and variance. The robustness of BetaDCE was demonstrated by experiments on three types of data. In the exclusive or-like (XOR-like) dataset, the false discovery rate of BetaDCE was significantly smaller than that of other methods. For the leukemia dataset, the area under the curve (AUC) of BetaDCE on the test set was 0.93 with only four selected features, which indicated that BetaDCE not only detected the irrelevant and redundant features precisely, but also more accurately predicted the class labels with a smaller number of features than the original method, whose AUC was 0.83 with 50 features. In the metabonomic dataset, the overall AUC of prediction with features selected by BetaDCE was significantly larger than that by the original reported method. Therefore, BetaDCE can be used as a general and efficient framework for feature selection. MDPI 2019-08-07 /pmc/articles/PMC7515297/ /pubmed/33267482 http://dx.doi.org/10.3390/e21080769 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Dai, Weixing
Guo, Dianjing
Beta Distribution-Based Cross-Entropy for Feature Selection
title Beta Distribution-Based Cross-Entropy for Feature Selection
title_full Beta Distribution-Based Cross-Entropy for Feature Selection
title_fullStr Beta Distribution-Based Cross-Entropy for Feature Selection
title_full_unstemmed Beta Distribution-Based Cross-Entropy for Feature Selection
title_short Beta Distribution-Based Cross-Entropy for Feature Selection
title_sort beta distribution-based cross-entropy for feature selection
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515297/
https://www.ncbi.nlm.nih.gov/pubmed/33267482
http://dx.doi.org/10.3390/e21080769
work_keys_str_mv AT daiweixing betadistributionbasedcrossentropyforfeatureselection
AT guodianjing betadistributionbasedcrossentropyforfeatureselection