Cargando…
Conversion of adverse data corpus to shrewd output using sampling metrics
An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Trai...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer Singapore
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7417470/ https://www.ncbi.nlm.nih.gov/pubmed/32779031 http://dx.doi.org/10.1186/s42492-020-00055-9 |
_version_ | 1783569499531771904 |
---|---|
author | Ashraf, Shahzad Saleem, Sehrish Ahmed, Tauqeer Aslam, Zeeshan Muhammad, Durr |
author_facet | Ashraf, Shahzad Saleem, Sehrish Ahmed, Tauqeer Aslam, Zeeshan Muhammad, Durr |
author_sort | Ashraf, Shahzad |
collection | PubMed |
description | An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Training with an imbalanced dataset poses challenges for classifiers; however, applying suitable techniques for reducing class imbalance issues can enhance classifiers’ performance. In this study, we consider an imbalanced dataset from an educational context. Initially, we examine all shortcomings regarding the classification of an imbalanced dataset. Then, we apply data-level algorithms for class balancing and compare the performance of classifiers. The performance of the classifiers is measured using the underlying information in their confusion matrices, such as accuracy, precision, recall, and F measure. The results show that classification with an imbalanced dataset may produce high accuracy but low precision and recall for the minority class. The analysis confirms that undersampling and oversampling are effective for balancing datasets, but the latter dominates. |
format | Online Article Text |
id | pubmed-7417470 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | Springer Singapore |
record_format | MEDLINE/PubMed |
spelling | pubmed-74174702020-08-18 Conversion of adverse data corpus to shrewd output using sampling metrics Ashraf, Shahzad Saleem, Sehrish Ahmed, Tauqeer Aslam, Zeeshan Muhammad, Durr Vis Comput Ind Biomed Art Original Article An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Training with an imbalanced dataset poses challenges for classifiers; however, applying suitable techniques for reducing class imbalance issues can enhance classifiers’ performance. In this study, we consider an imbalanced dataset from an educational context. Initially, we examine all shortcomings regarding the classification of an imbalanced dataset. Then, we apply data-level algorithms for class balancing and compare the performance of classifiers. The performance of the classifiers is measured using the underlying information in their confusion matrices, such as accuracy, precision, recall, and F measure. The results show that classification with an imbalanced dataset may produce high accuracy but low precision and recall for the minority class. The analysis confirms that undersampling and oversampling are effective for balancing datasets, but the latter dominates. Springer Singapore 2020-08-11 /pmc/articles/PMC7417470/ /pubmed/32779031 http://dx.doi.org/10.1186/s42492-020-00055-9 Text en © The Author(s) 2020 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
spellingShingle | Original Article Ashraf, Shahzad Saleem, Sehrish Ahmed, Tauqeer Aslam, Zeeshan Muhammad, Durr Conversion of adverse data corpus to shrewd output using sampling metrics |
title | Conversion of adverse data corpus to shrewd output using sampling metrics |
title_full | Conversion of adverse data corpus to shrewd output using sampling metrics |
title_fullStr | Conversion of adverse data corpus to shrewd output using sampling metrics |
title_full_unstemmed | Conversion of adverse data corpus to shrewd output using sampling metrics |
title_short | Conversion of adverse data corpus to shrewd output using sampling metrics |
title_sort | conversion of adverse data corpus to shrewd output using sampling metrics |
topic | Original Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7417470/ https://www.ncbi.nlm.nih.gov/pubmed/32779031 http://dx.doi.org/10.1186/s42492-020-00055-9 |
work_keys_str_mv | AT ashrafshahzad conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics AT saleemsehrish conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics AT ahmedtauqeer conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics AT aslamzeeshan conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics AT muhammaddurr conversionofadversedatacorpustoshrewdoutputusingsamplingmetrics |