Cargando…

Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data

Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity o...

Descripción completa

Detalles Bibliográficos
Autores principales: Lüftinger, Lukas, Májek, Peter, Beisken, Stephan, Rattei, Thomas, Posch, Andreas E.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7917081/
https://www.ncbi.nlm.nih.gov/pubmed/33659219
http://dx.doi.org/10.3389/fcimb.2021.610348
_version_ 1783657622381002752
author Lüftinger, Lukas
Májek, Peter
Beisken, Stephan
Rattei, Thomas
Posch, Andreas E.
author_facet Lüftinger, Lukas
Májek, Peter
Beisken, Stephan
Rattei, Thomas
Posch, Andreas E.
author_sort Lüftinger, Lukas
collection PubMed
description Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST.
format Online
Article
Text
id pubmed-7917081
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-79170812021-03-02 Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data Lüftinger, Lukas Májek, Peter Beisken, Stephan Rattei, Thomas Posch, Andreas E. Front Cell Infect Microbiol Cellular and Infection Microbiology Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST. Frontiers Media S.A. 2021-02-15 /pmc/articles/PMC7917081/ /pubmed/33659219 http://dx.doi.org/10.3389/fcimb.2021.610348 Text en Copyright © 2021 Lüftinger, Májek, Beisken, Rattei and Posch http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Cellular and Infection Microbiology
Lüftinger, Lukas
Májek, Peter
Beisken, Stephan
Rattei, Thomas
Posch, Andreas E.
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title_full Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title_fullStr Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title_full_unstemmed Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title_short Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
title_sort learning from limited data: towards best practice techniques for antimicrobial resistance prediction from whole genome sequencing data
topic Cellular and Infection Microbiology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7917081/
https://www.ncbi.nlm.nih.gov/pubmed/33659219
http://dx.doi.org/10.3389/fcimb.2021.610348
work_keys_str_mv AT luftingerlukas learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata
AT majekpeter learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata
AT beiskenstephan learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata
AT ratteithomas learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata
AT poschandrease learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata