Cargando…
Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis
There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the r...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer US
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9283158/ https://www.ncbi.nlm.nih.gov/pubmed/34272641 http://dx.doi.org/10.1007/s11121-021-01280-1 |
_version_ | 1784747272964145152 |
---|---|
author | Rijnhart, Judith J. M. Twisk, Jos W. R. Deeg, Dorly J. H. Heymans, Martijn W. |
author_facet | Rijnhart, Judith J. M. Twisk, Jos W. R. Deeg, Dorly J. H. Heymans, Martijn W. |
author_sort | Rijnhart, Judith J. M. |
collection | PubMed |
description | There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the robustness of the effect estimates against data analytical decisions is assessed. However, the uptake of multiverse analysis in empirical studies remains low, which might be due to the scarcity of guidance available on performing multiverse analysis. Researchers might experience difficulties in identifying data analytical decisions and in summarizing the large number of effect estimates yielded by a multiverse analysis. These difficulties are amplified when applying multiverse analysis to assess the robustness of the effect estimates from a mediation analysis, as a mediation analysis involves more data analytical decisions than a bivariate analysis. The aim of this paper is to provide an overview and worked example of the use of multiverse analysis to assess the robustness of the effect estimates from a mediation analysis. We showed that the number of data analytical decisions in a mediation analysis is larger than in a bivariate analysis. By using a real-life data example from the Longitudinal Aging Study Amsterdam, we demonstrated the application of multiverse analysis to a mediation analysis. This included the use of specification curves to determine the impact of data analytical decisions on the magnitude and statistical significance of the direct, indirect, and total effect estimates. Although the multiverse analysis methodology is still relatively new and future research is needed to further advance this methodology, this paper shows that multiverse analysis is a useful method for the assessment of the robustness of the direct, indirect, and total effect estimates in a mediation analysis and thereby to inform replication studies. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11121-021-01280-1. |
format | Online Article Text |
id | pubmed-9283158 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Springer US |
record_format | MEDLINE/PubMed |
spelling | pubmed-92831582022-07-16 Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis Rijnhart, Judith J. M. Twisk, Jos W. R. Deeg, Dorly J. H. Heymans, Martijn W. Prev Sci Article There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the robustness of the effect estimates against data analytical decisions is assessed. However, the uptake of multiverse analysis in empirical studies remains low, which might be due to the scarcity of guidance available on performing multiverse analysis. Researchers might experience difficulties in identifying data analytical decisions and in summarizing the large number of effect estimates yielded by a multiverse analysis. These difficulties are amplified when applying multiverse analysis to assess the robustness of the effect estimates from a mediation analysis, as a mediation analysis involves more data analytical decisions than a bivariate analysis. The aim of this paper is to provide an overview and worked example of the use of multiverse analysis to assess the robustness of the effect estimates from a mediation analysis. We showed that the number of data analytical decisions in a mediation analysis is larger than in a bivariate analysis. By using a real-life data example from the Longitudinal Aging Study Amsterdam, we demonstrated the application of multiverse analysis to a mediation analysis. This included the use of specification curves to determine the impact of data analytical decisions on the magnitude and statistical significance of the direct, indirect, and total effect estimates. Although the multiverse analysis methodology is still relatively new and future research is needed to further advance this methodology, this paper shows that multiverse analysis is a useful method for the assessment of the robustness of the direct, indirect, and total effect estimates in a mediation analysis and thereby to inform replication studies. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11121-021-01280-1. Springer US 2021-07-16 2022 /pmc/articles/PMC9283158/ /pubmed/34272641 http://dx.doi.org/10.1007/s11121-021-01280-1 Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Rijnhart, Judith J. M. Twisk, Jos W. R. Deeg, Dorly J. H. Heymans, Martijn W. Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title | Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title_full | Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title_fullStr | Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title_full_unstemmed | Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title_short | Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis |
title_sort | assessing the robustness of mediation analysis results using multiverse analysis |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9283158/ https://www.ncbi.nlm.nih.gov/pubmed/34272641 http://dx.doi.org/10.1007/s11121-021-01280-1 |
work_keys_str_mv | AT rijnhartjudithjm assessingtherobustnessofmediationanalysisresultsusingmultiverseanalysis AT twiskjoswr assessingtherobustnessofmediationanalysisresultsusingmultiverseanalysis AT deegdorlyjh assessingtherobustnessofmediationanalysisresultsusingmultiverseanalysis AT heymansmartijnw assessingtherobustnessofmediationanalysisresultsusingmultiverseanalysis |