Cargando…
Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5326492/ https://www.ncbi.nlm.nih.gov/pubmed/28236806 http://dx.doi.org/10.1186/s12874-016-0287-z |
_version_ | 1782510567483768832 |
---|---|
author | Ferrante di Ruffano, Lavinia Dinnes, Jacqueline Sitch, Alice J. Hyde, Chris Deeks, Jonathan J. |
author_facet | Ferrante di Ruffano, Lavinia Dinnes, Jacqueline Sitch, Alice J. Hyde, Chris Deeks, Jonathan J. |
author_sort | Ferrante di Ruffano, Lavinia |
collection | PubMed |
description | BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. METHODS: We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. RESULTS: One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. CONCLUSION: Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0. |
format | Online Article Text |
id | pubmed-5326492 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-53264922017-03-01 Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests Ferrante di Ruffano, Lavinia Dinnes, Jacqueline Sitch, Alice J. Hyde, Chris Deeks, Jonathan J. BMC Med Res Methodol Research Article BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. METHODS: We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. RESULTS: One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. CONCLUSION: Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0. BioMed Central 2017-02-24 /pmc/articles/PMC5326492/ /pubmed/28236806 http://dx.doi.org/10.1186/s12874-016-0287-z Text en © The Author(s). 2017 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Article Ferrante di Ruffano, Lavinia Dinnes, Jacqueline Sitch, Alice J. Hyde, Chris Deeks, Jonathan J. Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title | Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title_full | Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title_fullStr | Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title_full_unstemmed | Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title_short | Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
title_sort | test-treatment rcts are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5326492/ https://www.ncbi.nlm.nih.gov/pubmed/28236806 http://dx.doi.org/10.1186/s12874-016-0287-z |
work_keys_str_mv | AT ferrantediruffanolavinia testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests AT dinnesjacqueline testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests AT sitchalicej testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests AT hydechris testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests AT deeksjonathanj testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests |