Cargando…
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity o...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7917081/ https://www.ncbi.nlm.nih.gov/pubmed/33659219 http://dx.doi.org/10.3389/fcimb.2021.610348 |
_version_ | 1783657622381002752 |
---|---|
author | Lüftinger, Lukas Májek, Peter Beisken, Stephan Rattei, Thomas Posch, Andreas E. |
author_facet | Lüftinger, Lukas Májek, Peter Beisken, Stephan Rattei, Thomas Posch, Andreas E. |
author_sort | Lüftinger, Lukas |
collection | PubMed |
description | Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST. |
format | Online Article Text |
id | pubmed-7917081 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-79170812021-03-02 Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data Lüftinger, Lukas Májek, Peter Beisken, Stephan Rattei, Thomas Posch, Andreas E. Front Cell Infect Microbiol Cellular and Infection Microbiology Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST. Frontiers Media S.A. 2021-02-15 /pmc/articles/PMC7917081/ /pubmed/33659219 http://dx.doi.org/10.3389/fcimb.2021.610348 Text en Copyright © 2021 Lüftinger, Májek, Beisken, Rattei and Posch http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Cellular and Infection Microbiology Lüftinger, Lukas Májek, Peter Beisken, Stephan Rattei, Thomas Posch, Andreas E. Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title | Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_full | Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_fullStr | Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_full_unstemmed | Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_short | Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_sort | learning from limited data: towards best practice techniques for antimicrobial resistance prediction from whole genome sequencing data |
topic | Cellular and Infection Microbiology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7917081/ https://www.ncbi.nlm.nih.gov/pubmed/33659219 http://dx.doi.org/10.3389/fcimb.2021.610348 |
work_keys_str_mv | AT luftingerlukas learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT majekpeter learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT beiskenstephan learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT ratteithomas learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT poschandrease learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata |