Cargando…
Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality
The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classificat...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597215/ https://www.ncbi.nlm.nih.gov/pubmed/33286874 http://dx.doi.org/10.3390/e22101105 |
_version_ | 1783602292946108416 |
---|---|
author | Mirkes, Evgeny M. Allohibi, Jeza Gorban, Alexander |
author_facet | Mirkes, Evgeny M. Allohibi, Jeza Gorban, Alexander |
author_sort | Mirkes, Evgeny M. |
collection | PubMed |
description | The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm [Formula: see text] , but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for [Formula: see text] at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data. |
format | Online Article Text |
id | pubmed-7597215 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75972152020-11-09 Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality Mirkes, Evgeny M. Allohibi, Jeza Gorban, Alexander Entropy (Basel) Article The curse of dimensionality causes the well-known and widely discussed problems for machine learning methods. There is a hypothesis that using the Manhattan distance and even fractional [Formula: see text] quasinorms (for p less than 1) can help to overcome the curse of dimensionality in classification problems. In this study, we systematically test this hypothesis. It is illustrated that fractional quasinorms have a greater relative contrast and coefficient of variation than the Euclidean norm [Formula: see text] , but it is shown that this difference decays with increasing space dimension. It has been demonstrated that the concentration of distances shows qualitatively the same behaviour for all tested norms and quasinorms. It is shown that a greater relative contrast does not mean a better classification quality. It was revealed that for different databases the best (worst) performance was achieved under different norms (quasinorms). A systematic comparison shows that the difference in the performance of kNN classifiers for [Formula: see text] at p = 0.5, 1, and 2 is statistically insignificant. Analysis of curse and blessing of dimensionality requires careful definition of data dimensionality that rarely coincides with the number of attributes. We systematically examined several intrinsic dimensions of the data. MDPI 2020-09-30 /pmc/articles/PMC7597215/ /pubmed/33286874 http://dx.doi.org/10.3390/e22101105 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Mirkes, Evgeny M. Allohibi, Jeza Gorban, Alexander Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title | Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title_full | Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title_fullStr | Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title_full_unstemmed | Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title_short | Fractional Norms and Quasinorms Do Not Help to Overcome the Curse of Dimensionality |
title_sort | fractional norms and quasinorms do not help to overcome the curse of dimensionality |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597215/ https://www.ncbi.nlm.nih.gov/pubmed/33286874 http://dx.doi.org/10.3390/e22101105 |
work_keys_str_mv | AT mirkesevgenym fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality AT allohibijeza fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality AT gorbanalexander fractionalnormsandquasinormsdonothelptoovercomethecurseofdimensionality |