Cargando…
Interval Coded Scoring: a toolbox for interpretable scoring systems
Over the last decades, clinical decision support systems have been gaining importance. They help clinicians to make effective use of the overload of available information to obtain correct diagnoses and appropriate treatments. However, their power often comes at the cost of a black box model which c...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
PeerJ Inc.
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7924521/ https://www.ncbi.nlm.nih.gov/pubmed/33816804 http://dx.doi.org/10.7717/peerj-cs.150 |
_version_ | 1783659107347070976 |
---|---|
author | Billiet, Lieven Van Huffel, Sabine Van Belle, Vanya |
author_facet | Billiet, Lieven Van Huffel, Sabine Van Belle, Vanya |
author_sort | Billiet, Lieven |
collection | PubMed |
description | Over the last decades, clinical decision support systems have been gaining importance. They help clinicians to make effective use of the overload of available information to obtain correct diagnoses and appropriate treatments. However, their power often comes at the cost of a black box model which cannot be interpreted easily. This interpretability is of paramount importance in a medical setting with regard to trust and (legal) responsibility. In contrast, existing medical scoring systems are easy to understand and use, but they are often a simplified rule-of-thumb summary of previous medical experience rather than a well-founded system based on available data. Interval Coded Scoring (ICS) connects these two approaches, exploiting the power of sparse optimization to derive scoring systems from training data. The presented toolbox interface makes this theory easily applicable to both small and large datasets. It contains two possible problem formulations based on linear programming or elastic net. Both allow to construct a model for a binary classification problem and establish risk profiles that can be used for future diagnosis. All of this requires only a few lines of code. ICS differs from standard machine learning through its model consisting of interpretable main effects and interactions. Furthermore, insertion of expert knowledge is possible because the training can be semi-automatic. This allows end users to make a trade-off between complexity and performance based on cross-validation results and expert knowledge. Additionally, the toolbox offers an accessible way to assess classification performance via accuracy and the ROC curve, whereas the calibration of the risk profile can be evaluated via a calibration curve. Finally, the colour-coded model visualization has particular appeal if one wants to apply ICS manually on new observations, as well as for validation by experts in the specific application domains. The validity and applicability of the toolbox is demonstrated by comparing it to standard Machine Learning approaches such as Naive Bayes and Support Vector Machines for several real-life datasets. These case studies on medical problems show its applicability as a decision support system. ICS performs similarly in terms of classification and calibration. Its slightly lower performance is countered by its model simplicity which makes it the method of choice if interpretability is a key issue. |
format | Online Article Text |
id | pubmed-7924521 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | PeerJ Inc. |
record_format | MEDLINE/PubMed |
spelling | pubmed-79245212021-04-02 Interval Coded Scoring: a toolbox for interpretable scoring systems Billiet, Lieven Van Huffel, Sabine Van Belle, Vanya PeerJ Comput Sci Data Mining and Machine Learning Over the last decades, clinical decision support systems have been gaining importance. They help clinicians to make effective use of the overload of available information to obtain correct diagnoses and appropriate treatments. However, their power often comes at the cost of a black box model which cannot be interpreted easily. This interpretability is of paramount importance in a medical setting with regard to trust and (legal) responsibility. In contrast, existing medical scoring systems are easy to understand and use, but they are often a simplified rule-of-thumb summary of previous medical experience rather than a well-founded system based on available data. Interval Coded Scoring (ICS) connects these two approaches, exploiting the power of sparse optimization to derive scoring systems from training data. The presented toolbox interface makes this theory easily applicable to both small and large datasets. It contains two possible problem formulations based on linear programming or elastic net. Both allow to construct a model for a binary classification problem and establish risk profiles that can be used for future diagnosis. All of this requires only a few lines of code. ICS differs from standard machine learning through its model consisting of interpretable main effects and interactions. Furthermore, insertion of expert knowledge is possible because the training can be semi-automatic. This allows end users to make a trade-off between complexity and performance based on cross-validation results and expert knowledge. Additionally, the toolbox offers an accessible way to assess classification performance via accuracy and the ROC curve, whereas the calibration of the risk profile can be evaluated via a calibration curve. Finally, the colour-coded model visualization has particular appeal if one wants to apply ICS manually on new observations, as well as for validation by experts in the specific application domains. The validity and applicability of the toolbox is demonstrated by comparing it to standard Machine Learning approaches such as Naive Bayes and Support Vector Machines for several real-life datasets. These case studies on medical problems show its applicability as a decision support system. ICS performs similarly in terms of classification and calibration. Its slightly lower performance is countered by its model simplicity which makes it the method of choice if interpretability is a key issue. PeerJ Inc. 2018-04-02 /pmc/articles/PMC7924521/ /pubmed/33816804 http://dx.doi.org/10.7717/peerj-cs.150 Text en ©2018 Billiet et al. http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Computer Science) and either DOI or URL of the article must be cited. |
spellingShingle | Data Mining and Machine Learning Billiet, Lieven Van Huffel, Sabine Van Belle, Vanya Interval Coded Scoring: a toolbox for interpretable scoring systems |
title | Interval Coded Scoring: a toolbox for interpretable scoring systems |
title_full | Interval Coded Scoring: a toolbox for interpretable scoring systems |
title_fullStr | Interval Coded Scoring: a toolbox for interpretable scoring systems |
title_full_unstemmed | Interval Coded Scoring: a toolbox for interpretable scoring systems |
title_short | Interval Coded Scoring: a toolbox for interpretable scoring systems |
title_sort | interval coded scoring: a toolbox for interpretable scoring systems |
topic | Data Mining and Machine Learning |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7924521/ https://www.ncbi.nlm.nih.gov/pubmed/33816804 http://dx.doi.org/10.7717/peerj-cs.150 |
work_keys_str_mv | AT billietlieven intervalcodedscoringatoolboxforinterpretablescoringsystems AT vanhuffelsabine intervalcodedscoringatoolboxforinterpretablescoringsystems AT vanbellevanya intervalcodedscoringatoolboxforinterpretablescoringsystems |