Cargando…
Analysis of Information-Based Nonparametric Variable Selection Criteria
We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597280/ https://www.ncbi.nlm.nih.gov/pubmed/33286743 http://dx.doi.org/10.3390/e22090974 |
_version_ | 1783602310574768128 |
---|---|
author | Łazęcka, Małgorzata Mielniczuk, Jan |
author_facet | Łazęcka, Małgorzata Mielniczuk, Jan |
author_sort | Łazęcka, Małgorzata |
collection | PubMed |
description | We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both derived as approximations of Conditional Mutual Information (CMI) criterion. We show that both criteria CIFE and JMI may exhibit different behavior from CMI, resulting in different orders in which predictors are chosen in variable selection process. Explicit formulae for CMI and its two approximations in the generative tree model are obtained. As a byproduct, we establish expressions for an entropy of a multivariate gaussian mixture and its mutual information with mixing distribution. |
format | Online Article Text |
id | pubmed-7597280 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75972802020-11-09 Analysis of Information-Based Nonparametric Variable Selection Criteria Łazęcka, Małgorzata Mielniczuk, Jan Entropy (Basel) Article We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both derived as approximations of Conditional Mutual Information (CMI) criterion. We show that both criteria CIFE and JMI may exhibit different behavior from CMI, resulting in different orders in which predictors are chosen in variable selection process. Explicit formulae for CMI and its two approximations in the generative tree model are obtained. As a byproduct, we establish expressions for an entropy of a multivariate gaussian mixture and its mutual information with mixing distribution. MDPI 2020-08-31 /pmc/articles/PMC7597280/ /pubmed/33286743 http://dx.doi.org/10.3390/e22090974 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Łazęcka, Małgorzata Mielniczuk, Jan Analysis of Information-Based Nonparametric Variable Selection Criteria |
title | Analysis of Information-Based Nonparametric Variable Selection Criteria |
title_full | Analysis of Information-Based Nonparametric Variable Selection Criteria |
title_fullStr | Analysis of Information-Based Nonparametric Variable Selection Criteria |
title_full_unstemmed | Analysis of Information-Based Nonparametric Variable Selection Criteria |
title_short | Analysis of Information-Based Nonparametric Variable Selection Criteria |
title_sort | analysis of information-based nonparametric variable selection criteria |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597280/ https://www.ncbi.nlm.nih.gov/pubmed/33286743 http://dx.doi.org/10.3390/e22090974 |
work_keys_str_mv | AT łazeckamałgorzata analysisofinformationbasednonparametricvariableselectioncriteria AT mielniczukjan analysisofinformationbasednonparametricvariableselectioncriteria |