Cargando…

Analysis of Information-Based Nonparametric Variable Selection Criteria

We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both...

Descripción completa

Detalles Bibliográficos
Autores principales: Łazęcka, Małgorzata, Mielniczuk, Jan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597280/
https://www.ncbi.nlm.nih.gov/pubmed/33286743
http://dx.doi.org/10.3390/e22090974
_version_ 1783602310574768128
author Łazęcka, Małgorzata
Mielniczuk, Jan
author_facet Łazęcka, Małgorzata
Mielniczuk, Jan
author_sort Łazęcka, Małgorzata
collection PubMed
description We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both derived as approximations of Conditional Mutual Information (CMI) criterion. We show that both criteria CIFE and JMI may exhibit different behavior from CMI, resulting in different orders in which predictors are chosen in variable selection process. Explicit formulae for CMI and its two approximations in the generative tree model are obtained. As a byproduct, we establish expressions for an entropy of a multivariate gaussian mixture and its mutual information with mixing distribution.
format Online
Article
Text
id pubmed-7597280
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75972802020-11-09 Analysis of Information-Based Nonparametric Variable Selection Criteria Łazęcka, Małgorzata Mielniczuk, Jan Entropy (Basel) Article We consider a nonparametric Generative Tree Model and discuss a problem of selecting active predictors for the response in such scenario. We investigated two popular information-based selection criteria: Conditional Infomax Feature Extraction (CIFE) and Joint Mutual information (JMI), which are both derived as approximations of Conditional Mutual Information (CMI) criterion. We show that both criteria CIFE and JMI may exhibit different behavior from CMI, resulting in different orders in which predictors are chosen in variable selection process. Explicit formulae for CMI and its two approximations in the generative tree model are obtained. As a byproduct, we establish expressions for an entropy of a multivariate gaussian mixture and its mutual information with mixing distribution. MDPI 2020-08-31 /pmc/articles/PMC7597280/ /pubmed/33286743 http://dx.doi.org/10.3390/e22090974 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Łazęcka, Małgorzata
Mielniczuk, Jan
Analysis of Information-Based Nonparametric Variable Selection Criteria
title Analysis of Information-Based Nonparametric Variable Selection Criteria
title_full Analysis of Information-Based Nonparametric Variable Selection Criteria
title_fullStr Analysis of Information-Based Nonparametric Variable Selection Criteria
title_full_unstemmed Analysis of Information-Based Nonparametric Variable Selection Criteria
title_short Analysis of Information-Based Nonparametric Variable Selection Criteria
title_sort analysis of information-based nonparametric variable selection criteria
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597280/
https://www.ncbi.nlm.nih.gov/pubmed/33286743
http://dx.doi.org/10.3390/e22090974
work_keys_str_mv AT łazeckamałgorzata analysisofinformationbasednonparametricvariableselectioncriteria
AT mielniczukjan analysisofinformationbasednonparametricvariableselectioncriteria