Cargando…

Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards

BACKGROUND: Predictive models to identify unknown methicillin-resistant Staphylococcus aureus (MRSA) carriage on admission may optimise targeted MRSA screening and efficient use of resources. However, common approaches to model selection can result in overconfident estimates and poor predictive perf...

Descripción completa

Detalles Bibliográficos
Autores principales: Lee, Andie S, Pan, Angelo, Harbarth, Stephan, Patroni, Andrea, Chalfine, Annie, Daikos, George L, Garilli, Silvia, Martínez, José Antonio, Cooper, Ben S
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2015
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4347652/
https://www.ncbi.nlm.nih.gov/pubmed/25880328
http://dx.doi.org/10.1186/s12879-015-0834-y
_version_ 1782359855672066048
author Lee, Andie S
Pan, Angelo
Harbarth, Stephan
Patroni, Andrea
Chalfine, Annie
Daikos, George L
Garilli, Silvia
Martínez, José Antonio
Cooper, Ben S
author_facet Lee, Andie S
Pan, Angelo
Harbarth, Stephan
Patroni, Andrea
Chalfine, Annie
Daikos, George L
Garilli, Silvia
Martínez, José Antonio
Cooper, Ben S
author_sort Lee, Andie S
collection PubMed
description BACKGROUND: Predictive models to identify unknown methicillin-resistant Staphylococcus aureus (MRSA) carriage on admission may optimise targeted MRSA screening and efficient use of resources. However, common approaches to model selection can result in overconfident estimates and poor predictive performance. We aimed to compare the performance of various models to predict previously unknown MRSA carriage on admission to surgical wards. METHODS: The study analysed data collected during a prospective cohort study which enrolled consecutive adult patients admitted to 13 surgical wards in 4 European hospitals. The participating hospitals were located in Athens (Greece), Barcelona (Spain), Cremona (Italy) and Paris (France). Universal admission MRSA screening was performed in the surgical wards. Data regarding demographic characteristics and potential risk factors for MRSA carriage were prospectively collected during the study period. Four logistic regression models were used to predict probabilities of unknown MRSA carriage using risk factor data: “Stepwise” (variables selected by backward elimination); “Best BMA” (model with highest posterior probability using Bayesian model averaging which accounts for uncertainty in model choice); “BMA” (average of all models selected with BMA); and “Simple” (model including variables selected >50% of the time by both Stepwise and BMA approaches applied to repeated random sub-samples of 50% of the data). To assess model performance, cross-validation against data not used for model fitting was conducted and net reclassification improvement (NRI) was calculated. RESULTS: Of 2,901 patients enrolled, 111 (3.8%) were newly identified MRSA carriers. Recent hospitalisation and presence of a wound/ulcer were significantly associated with MRSA carriage in all models. While all models demonstrated limited predictive ability (mean c-statistics <0.7) the Simple model consistently detected more MRSA-positive individuals despite screening fewer patients than the Stepwise model. Moreover, the Simple model improved reclassification of patients into appropriate risk strata compared with the Stepwise model (NRI 6.6%, P = .07). CONCLUSIONS: Though commonly used, models developed using stepwise variable selection can have relatively poor predictive value. When developing MRSA risk indices, simpler models, which account for uncertainty in model selection, may better stratify patients’ risk of unknown MRSA carriage.
format Online
Article
Text
id pubmed-4347652
institution National Center for Biotechnology Information
language English
publishDate 2015
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-43476522015-03-04 Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards Lee, Andie S Pan, Angelo Harbarth, Stephan Patroni, Andrea Chalfine, Annie Daikos, George L Garilli, Silvia Martínez, José Antonio Cooper, Ben S BMC Infect Dis Research Article BACKGROUND: Predictive models to identify unknown methicillin-resistant Staphylococcus aureus (MRSA) carriage on admission may optimise targeted MRSA screening and efficient use of resources. However, common approaches to model selection can result in overconfident estimates and poor predictive performance. We aimed to compare the performance of various models to predict previously unknown MRSA carriage on admission to surgical wards. METHODS: The study analysed data collected during a prospective cohort study which enrolled consecutive adult patients admitted to 13 surgical wards in 4 European hospitals. The participating hospitals were located in Athens (Greece), Barcelona (Spain), Cremona (Italy) and Paris (France). Universal admission MRSA screening was performed in the surgical wards. Data regarding demographic characteristics and potential risk factors for MRSA carriage were prospectively collected during the study period. Four logistic regression models were used to predict probabilities of unknown MRSA carriage using risk factor data: “Stepwise” (variables selected by backward elimination); “Best BMA” (model with highest posterior probability using Bayesian model averaging which accounts for uncertainty in model choice); “BMA” (average of all models selected with BMA); and “Simple” (model including variables selected >50% of the time by both Stepwise and BMA approaches applied to repeated random sub-samples of 50% of the data). To assess model performance, cross-validation against data not used for model fitting was conducted and net reclassification improvement (NRI) was calculated. RESULTS: Of 2,901 patients enrolled, 111 (3.8%) were newly identified MRSA carriers. Recent hospitalisation and presence of a wound/ulcer were significantly associated with MRSA carriage in all models. While all models demonstrated limited predictive ability (mean c-statistics <0.7) the Simple model consistently detected more MRSA-positive individuals despite screening fewer patients than the Stepwise model. Moreover, the Simple model improved reclassification of patients into appropriate risk strata compared with the Stepwise model (NRI 6.6%, P = .07). CONCLUSIONS: Though commonly used, models developed using stepwise variable selection can have relatively poor predictive value. When developing MRSA risk indices, simpler models, which account for uncertainty in model selection, may better stratify patients’ risk of unknown MRSA carriage. BioMed Central 2015-02-27 /pmc/articles/PMC4347652/ /pubmed/25880328 http://dx.doi.org/10.1186/s12879-015-0834-y Text en © Lee et al.; licensee BioMed Central. 2015 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Lee, Andie S
Pan, Angelo
Harbarth, Stephan
Patroni, Andrea
Chalfine, Annie
Daikos, George L
Garilli, Silvia
Martínez, José Antonio
Cooper, Ben S
Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title_full Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title_fullStr Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title_full_unstemmed Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title_short Variable performance of models for predicting methicillin-resistant Staphylococcus aureus carriage in European surgical wards
title_sort variable performance of models for predicting methicillin-resistant staphylococcus aureus carriage in european surgical wards
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4347652/
https://www.ncbi.nlm.nih.gov/pubmed/25880328
http://dx.doi.org/10.1186/s12879-015-0834-y
work_keys_str_mv AT leeandies variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT panangelo variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT harbarthstephan variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT patroniandrea variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT chalfineannie variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT daikosgeorgel variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT garillisilvia variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT martinezjoseantonio variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT cooperbens variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards
AT variableperformanceofmodelsforpredictingmethicillinresistantstaphylococcusaureuscarriageineuropeansurgicalwards