Cargando…

Conversion of adverse data corpus to shrewd output using sampling metrics

An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Trai...

Descripción completa

Detalles Bibliográficos
Autores principales: Ashraf, Shahzad, Saleem, Sehrish, Ahmed, Tauqeer, Aslam, Zeeshan, Muhammad, Durr
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer Singapore 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7417470/
https://www.ncbi.nlm.nih.gov/pubmed/32779031
http://dx.doi.org/10.1186/s42492-020-00055-9
_version_ 1783569499531771904
author Ashraf, Shahzad
Saleem, Sehrish
Ahmed, Tauqeer
Aslam, Zeeshan
Muhammad, Durr
author_facet Ashraf, Shahzad
Saleem, Sehrish
Ahmed, Tauqeer
Aslam, Zeeshan
Muhammad, Durr
author_sort Ashraf, Shahzad
collection PubMed
description An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Training with an imbalanced dataset poses challenges for classifiers; however, applying suitable techniques for reducing class imbalance issues can enhance classifiers’ performance. In this study, we consider an imbalanced dataset from an educational context. Initially, we examine all shortcomings regarding the classification of an imbalanced dataset. Then, we apply data-level algorithms for class balancing and compare the performance of classifiers. The performance of the classifiers is measured using the underlying information in their confusion matrices, such as accuracy, precision, recall, and F measure. The results show that classification with an imbalanced dataset may produce high accuracy but low precision and recall for the minority class. The analysis confirms that undersampling and oversampling are effective for balancing datasets, but the latter dominates.
format Online
Article
Text
id pubmed-7417470
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Springer Singapore
record_format MEDLINE/PubMed
spelling pubmed-74174702020-08-18 Conversion of adverse data corpus to shrewd output using sampling metrics Ashraf, Shahzad Saleem, Sehrish Ahmed, Tauqeer Aslam, Zeeshan Muhammad, Durr Vis Comput Ind Biomed Art Original Article An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Training with an imbalanced dataset poses challenges for classifiers; however, applying suitable techniques for reducing class imbalance issues can enhance classifiers’ performance. In this study, we consider an imbalanced dataset from an educational context. Initially, we examine all shortcomings regarding the classification of an imbalanced dataset. Then, we apply data-level algorithms for class balancing and compare the performance of classifiers. The performance of the classifiers is measured using the underlying information in their confusion matrices, such as accuracy, precision, recall, and F measure. The results show that classification with an imbalanced dataset may produce high accuracy but low precision and recall for the minority class. The analysis confirms that undersampling and oversampling are effective for balancing datasets, but the latter dominates. Springer Singapore 2020-08-11 /pmc/articles/PMC7417470/ /pubmed/32779031 http://dx.doi.org/10.1186/s42492-020-00055-9 Text en © The Author(s) 2020 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
spellingShingle Original Article
Ashraf, Shahzad
Saleem, Sehrish
Ahmed, Tauqeer
Aslam, Zeeshan
Muhammad, Durr
Conversion of adverse data corpus to shrewd output using sampling metrics
title Conversion of adverse data corpus to shrewd output using sampling metrics
title_full Conversion of adverse data corpus to shrewd output using sampling metrics
title_fullStr Conversion of adverse data corpus to shrewd output using sampling metrics
title_full_unstemmed Conversion of adverse data corpus to shrewd output using sampling metrics
title_short Conversion of adverse data corpus to shrewd output using sampling metrics
title_sort conversion of adverse data corpus to shrewd output using sampling metrics
topic Original Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7417470/
https://www.ncbi.nlm.nih.gov/pubmed/32779031
http://dx.doi.org/10.1186/s42492-020-00055-9
work_keys_str_mv AT ashrafshahzad conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics
AT saleemsehrish conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics
AT ahmedtauqeer conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics
AT aslamzeeshan conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics
AT muhammaddurr conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics