Cargando…

Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality

The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classificat...

Descripción completa

Detalles Bibliográficos
Autores principales: Mirkes, Evgeny M., Allohibi, Jeza, Gorban, Alexander
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597215/
https://www.ncbi.nlm.nih.gov/pubmed/33286874
http://dx.doi.org/10.3390/e22101105
_version_ 1783602292946108416
author Mirkes, Evgeny M.
Allohibi, Jeza
Gorban, Alexander
author_facet Mirkes, Evgeny M.
Allohibi, Jeza
Gorban, Alexander
author_sort Mirkes, Evgeny M.
collection PubMed
description The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm [Formula: see text] , but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for [Formula: see text] at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data.
format Online
Article
Text
id pubmed-7597215
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75972152020-11-09 Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality Mirkes, Evgeny M. Allohibi, Jeza Gorban, Alexander Entropy (Basel) Article The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm [Formula: see text] , but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for [Formula: see text] at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data. MDPI 2020-09-30 /pmc/articles/PMC7597215/ /pubmed/33286874 http://dx.doi.org/10.3390/e22101105 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Mirkes, Evgeny M.
Allohibi, Jeza
Gorban, Alexander
Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title_full Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title_fullStr Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title_full_unstemmed Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title_short Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
title_sort fractional norms and quasinorms do not help to overcome the curse of dimensionality
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597215/
https://www.ncbi.nlm.nih.gov/pubmed/33286874
http://dx.doi.org/10.3390/e22101105
work_keys_str_mv AT mirkesevgenym fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality
AT allohibijeza fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality
AT gorbanalexander fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality