Cargando…

Building more accurate decision trees with the additive tree

The expansion of machine learning to high-stakes application domains such as medicine, finance, and criminal justice, where making informed decisions requires clear understanding of the model, has increased the interest in interpretable machine learning. The widely used Classification and Regression...

Descripción completa

Detalles Bibliográficos
Autores principales: Luna, José Marcio, Gennatas, Efstathios D., Ungar, Lyle H., Eaton, Eric, Diffenderfer, Eric S., Jensen, Shane T., Simone, Charles B., Friedman, Jerome H., Solberg, Timothy D., Valdes, Gilmer
Formato: Online Artículo Texto
Lenguaje:English
Publicado: National Academy of Sciences 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6778203/
https://www.ncbi.nlm.nih.gov/pubmed/31527280
http://dx.doi.org/10.1073/pnas.1816748116
_version_ 1783456730399637504
author Luna, José Marcio
Gennatas, Efstathios D.
Ungar, Lyle H.
Eaton, Eric
Diffenderfer, Eric S.
Jensen, Shane T.
Simone, Charles B.
Friedman, Jerome H.
Solberg, Timothy D.
Valdes, Gilmer
author_facet Luna, José Marcio
Gennatas, Efstathios D.
Ungar, Lyle H.
Eaton, Eric
Diffenderfer, Eric S.
Jensen, Shane T.
Simone, Charles B.
Friedman, Jerome H.
Solberg, Timothy D.
Valdes, Gilmer
author_sort Luna, José Marcio
collection PubMed
description The expansion of machine learning to high-stakes application domains such as medicine, finance, and criminal justice, where making informed decisions requires clear understanding of the model, has increased the interest in interpretable machine learning. The widely used Classification and Regression Trees (CART) have played a major role in health sciences, due to their simple and intuitive explanation of predictions. Ensemble methods like gradient boosting can improve the accuracy of decision trees, but at the expense of the interpretability of the generated model. Additive models, such as those produced by gradient boosting, and full interaction models, such as CART, have been investigated largely in isolation. We show that these models exist along a spectrum, revealing previously unseen connections between these approaches. This paper introduces a rigorous formalization for the additive tree, an empirically validated learning technique for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although the additive tree is designed primarily to provide both the model interpretability and predictive performance needed for high-stakes applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches.
format Online
Article
Text
id pubmed-6778203
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher National Academy of Sciences
record_format MEDLINE/PubMed
spelling pubmed-67782032019-10-09 Building more accurate decision trees with the additive tree Luna, José Marcio Gennatas, Efstathios D. Ungar, Lyle H. Eaton, Eric Diffenderfer, Eric S. Jensen, Shane T. Simone, Charles B. Friedman, Jerome H. Solberg, Timothy D. Valdes, Gilmer Proc Natl Acad Sci U S A Physical Sciences The expansion of machine learning to high-stakes application domains such as medicine, finance, and criminal justice, where making informed decisions requires clear understanding of the model, has increased the interest in interpretable machine learning. The widely used Classification and Regression Trees (CART) have played a major role in health sciences, due to their simple and intuitive explanation of predictions. Ensemble methods like gradient boosting can improve the accuracy of decision trees, but at the expense of the interpretability of the generated model. Additive models, such as those produced by gradient boosting, and full interaction models, such as CART, have been investigated largely in isolation. We show that these models exist along a spectrum, revealing previously unseen connections between these approaches. This paper introduces a rigorous formalization for the additive tree, an empirically validated learning technique for creating a single decision tree, and shows that this method can produce models equivalent to CART or gradient boosted stumps at the extremes by varying a single parameter. Although the additive tree is designed primarily to provide both the model interpretability and predictive performance needed for high-stakes applications like medicine, it also can produce decision trees represented by hybrid models between CART and boosted stumps that can outperform either of these approaches. National Academy of Sciences 2019-10-01 2019-09-16 /pmc/articles/PMC6778203/ /pubmed/31527280 http://dx.doi.org/10.1073/pnas.1816748116 Text en Copyright © 2019 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by-nc-nd/4.0/ https://creativecommons.org/licenses/by-nc-nd/4.0/This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) .
spellingShingle Physical Sciences
Luna, José Marcio
Gennatas, Efstathios D.
Ungar, Lyle H.
Eaton, Eric
Diffenderfer, Eric S.
Jensen, Shane T.
Simone, Charles B.
Friedman, Jerome H.
Solberg, Timothy D.
Valdes, Gilmer
Building more accurate decision trees with the additive tree
title Building more accurate decision trees with the additive tree
title_full Building more accurate decision trees with the additive tree
title_fullStr Building more accurate decision trees with the additive tree
title_full_unstemmed Building more accurate decision trees with the additive tree
title_short Building more accurate decision trees with the additive tree
title_sort building more accurate decision trees with the additive tree
topic Physical Sciences
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6778203/
https://www.ncbi.nlm.nih.gov/pubmed/31527280
http://dx.doi.org/10.1073/pnas.1816748116
work_keys_str_mv AT lunajosemarcio buildingmoreaccuratedecisiontreeswiththeadditivetree
AT gennatasefstathiosd buildingmoreaccuratedecisiontreeswiththeadditivetree
AT ungarlyleh buildingmoreaccuratedecisiontreeswiththeadditivetree
AT eatoneric buildingmoreaccuratedecisiontreeswiththeadditivetree
AT diffenderfererics buildingmoreaccuratedecisiontreeswiththeadditivetree
AT jensenshanet buildingmoreaccuratedecisiontreeswiththeadditivetree
AT simonecharlesb buildingmoreaccuratedecisiontreeswiththeadditivetree
AT friedmanjeromeh buildingmoreaccuratedecisiontreeswiththeadditivetree
AT solbergtimothyd buildingmoreaccuratedecisiontreeswiththeadditivetree
AT valdesgilmer buildingmoreaccuratedecisiontreeswiththeadditivetree