Cargando…

Formal definition of the MARS method for quantifying the unique target class discoveries of selected machine classifiers

Conventional binary classification performance metrics evaluate either general measures (accuracy, F score) or specific aspects (precision, recall) of a model’s classifying ability. As such, these metrics, derived from the model’s confusion matrix, provide crucial insight regarding classifier-data i...

Descripción completa

Detalles Bibliográficos
Autores principales: Restrepo, Felipe, Mali, Namrata, Abrahams, Alan, Ractham, Peter
Formato: Online Artículo Texto
Lenguaje:English
Publicado: F1000 Research Limited 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9350436/
https://www.ncbi.nlm.nih.gov/pubmed/35967970
http://dx.doi.org/10.12688/f1000research.110567.2
Descripción
Sumario:Conventional binary classification performance metrics evaluate either general measures (accuracy, F score) or specific aspects (precision, recall) of a model’s classifying ability. As such, these metrics, derived from the model’s confusion matrix, provide crucial insight regarding classifier-data interactions. However, modern- day computational capabilities have allowed for the creation of increasingly complex models that share nearly identical classification performance. While traditional performance metrics remain as essential indicators of a classifier’s individual capabilities, their ability to differentiate between models is limited. In this paper, we present the methodology for MARS (Method for Assessing Relative Sensitivity/ Specificity) ShineThrough and MARS Occlusion scores, two novel binary classification performance metrics, designed to quantify the distinctiveness of a classifier’s predictive successes and failures, relative to alternative classifiers. Being able to quantitatively express classifier uniqueness adds a novel classifier-classifier layer to the process of model evaluation and could improve ensemble model-selection decision making. By calculating both conventional performance measures, and proposed MARS metrics for a simple classifier prediction dataset, we demonstrate that the proposed metrics’ informational strengths synergize well with those of traditional metrics, delivering insight complementary to that of conventional metrics.