Cargando…
An empirical study using permutation-based resampling in meta-regression
BACKGROUND: In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation met...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2012
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3351721/ https://www.ncbi.nlm.nih.gov/pubmed/22587815 http://dx.doi.org/10.1186/2046-4053-1-18 |
_version_ | 1782232794556006400 |
---|---|
author | Gagnier, Joel J Moher, David Boon, Heather Bombardier, Claire Beyene, Joseph |
author_facet | Gagnier, Joel J Moher, David Boon, Heather Bombardier, Claire Beyene, Joseph |
author_sort | Gagnier, Joel J |
collection | PubMed |
description | BACKGROUND: In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression. METHODS: We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final P values between methods. RESULTS: We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The P values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases. CONCLUSIONS: We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase P values in meta-regression of multiple covariates for relatively small amount of trials. |
format | Online Article Text |
id | pubmed-3351721 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2012 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-33517212012-05-16 An empirical study using permutation-based resampling in meta-regression Gagnier, Joel J Moher, David Boon, Heather Bombardier, Claire Beyene, Joseph Syst Rev Methodology BACKGROUND: In meta-regression, as the number of trials in the analyses decreases, the risk of false positives or false negatives increases. This is partly due to the assumption of normality that may not hold in small samples. Creation of a distribution from the observed trials using permutation methods to calculate P values may allow for less spurious findings. Permutation has not been empirically tested in meta-regression. The objective of this study was to perform an empirical investigation to explore the differences in results for meta-analyses on a small number of trials using standard large sample approaches verses permutation-based methods for meta-regression. METHODS: We isolated a sample of randomized controlled clinical trials (RCTs) for interventions that have a small number of trials (herbal medicine trials). Trials were then grouped by herbal species and condition and assessed for methodological quality using the Jadad scale, and data were extracted for each outcome. Finally, we performed meta-analyses on the primary outcome of each group of trials and meta-regression for methodological quality subgroups within each meta-analysis. We used large sample methods and permutation methods in our meta-regression modeling. We then compared final models and final P values between methods. RESULTS: We collected 110 trials across 5 intervention/outcome pairings and 5 to 10 trials per covariate. When applying large sample methods and permutation-based methods in our backwards stepwise regression the covariates in the final models were identical in all cases. The P values for the covariates in the final model were larger in 78% (7/9) of the cases for permutation and identical for 22% (2/9) of the cases. CONCLUSIONS: We present empirical evidence that permutation-based resampling may not change final models when using backwards stepwise regression, but may increase P values in meta-regression of multiple covariates for relatively small amount of trials. BioMed Central 2012-02-23 /pmc/articles/PMC3351721/ /pubmed/22587815 http://dx.doi.org/10.1186/2046-4053-1-18 Text en Copyright ©2012 Gagnier et al; licensee BioMed Central Ltd. http://creativecommons.org/licenses/by/2.0 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Methodology Gagnier, Joel J Moher, David Boon, Heather Bombardier, Claire Beyene, Joseph An empirical study using permutation-based resampling in meta-regression |
title | An empirical study using permutation-based resampling in meta-regression |
title_full | An empirical study using permutation-based resampling in meta-regression |
title_fullStr | An empirical study using permutation-based resampling in meta-regression |
title_full_unstemmed | An empirical study using permutation-based resampling in meta-regression |
title_short | An empirical study using permutation-based resampling in meta-regression |
title_sort | empirical study using permutation-based resampling in meta-regression |
topic | Methodology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3351721/ https://www.ncbi.nlm.nih.gov/pubmed/22587815 http://dx.doi.org/10.1186/2046-4053-1-18 |
work_keys_str_mv | AT gagnierjoelj anempiricalstudyusingpermutationbasedresamplinginmetaregression AT moherdavid anempiricalstudyusingpermutationbasedresamplinginmetaregression AT boonheather anempiricalstudyusingpermutationbasedresamplinginmetaregression AT bombardierclaire anempiricalstudyusingpermutationbasedresamplinginmetaregression AT beyenejoseph anempiricalstudyusingpermutationbasedresamplinginmetaregression AT gagnierjoelj empiricalstudyusingpermutationbasedresamplinginmetaregression AT moherdavid empiricalstudyusingpermutationbasedresamplinginmetaregression AT boonheather empiricalstudyusingpermutationbasedresamplinginmetaregression AT bombardierclaire empiricalstudyusingpermutationbasedresamplinginmetaregression AT beyenejoseph empiricalstudyusingpermutationbasedresamplinginmetaregression |