Cargando…
Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools
BACKGROUND: Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program’s Office of Health Assessment and Translation (OHAT), the U...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7596989/ https://www.ncbi.nlm.nih.gov/pubmed/33121530 http://dx.doi.org/10.1186/s13643-020-01490-8 |
_version_ | 1783602232746311680 |
---|---|
author | Eick, Stephanie M. Goin, Dana E. Chartres, Nicholas Lam, Juleen Woodruff, Tracey J. |
author_facet | Eick, Stephanie M. Goin, Dana E. Chartres, Nicholas Lam, Juleen Woodruff, Tracey J. |
author_sort | Eick, Stephanie M. |
collection | PubMed |
description | BACKGROUND: Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program’s Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency’s (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB. METHODS: We selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence. RESULTS: The time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall “low” or “uninformative” using IRIS, due to “deficient” or “critically deficient” ratings in one or two domains. Similarly, all studies were rated “unacceptable” using the TSCA tool because of one “unacceptable” rating in a metric related to statistical power. Approximately half of the studies had “low” or “probably low ROB” ratings across all domains with the OHAT and Navigation Guide tools. CONCLUSIONS: Tools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide. SYSTEMATIC REVIEW REGISTRATION: This review has not been registered as it is not a systematic review. |
format | Online Article Text |
id | pubmed-7596989 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-75969892020-11-02 Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools Eick, Stephanie M. Goin, Dana E. Chartres, Nicholas Lam, Juleen Woodruff, Tracey J. Syst Rev Research BACKGROUND: Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program’s Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency’s (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB. METHODS: We selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence. RESULTS: The time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall “low” or “uninformative” using IRIS, due to “deficient” or “critically deficient” ratings in one or two domains. Similarly, all studies were rated “unacceptable” using the TSCA tool because of one “unacceptable” rating in a metric related to statistical power. Approximately half of the studies had “low” or “probably low ROB” ratings across all domains with the OHAT and Navigation Guide tools. CONCLUSIONS: Tools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide. SYSTEMATIC REVIEW REGISTRATION: This review has not been registered as it is not a systematic review. BioMed Central 2020-10-29 /pmc/articles/PMC7596989/ /pubmed/33121530 http://dx.doi.org/10.1186/s13643-020-01490-8 Text en © The Author(s) 2020 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. |
spellingShingle | Research Eick, Stephanie M. Goin, Dana E. Chartres, Nicholas Lam, Juleen Woodruff, Tracey J. Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title | Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title_full | Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title_fullStr | Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title_full_unstemmed | Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title_short | Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
title_sort | assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7596989/ https://www.ncbi.nlm.nih.gov/pubmed/33121530 http://dx.doi.org/10.1186/s13643-020-01490-8 |
work_keys_str_mv | AT eickstephaniem assessingriskofbiasinhumanenvironmentalepidemiologystudiesusingthreetoolsdifferentconclusionsfromdifferenttools AT goindanae assessingriskofbiasinhumanenvironmentalepidemiologystudiesusingthreetoolsdifferentconclusionsfromdifferenttools AT chartresnicholas assessingriskofbiasinhumanenvironmentalepidemiologystudiesusingthreetoolsdifferentconclusionsfromdifferenttools AT lamjuleen assessingriskofbiasinhumanenvironmentalepidemiologystudiesusingthreetoolsdifferentconclusionsfromdifferenttools AT woodrufftraceyj assessingriskofbiasinhumanenvironmentalepidemiologystudiesusingthreetoolsdifferentconclusionsfromdifferenttools |