Cargando…

Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests

BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these...

Descripción completa

Detalles Bibliográficos
Autores principales: Ferrante di Ruffano, Lavinia, Dinnes, Jacqueline, Sitch, Alice J., Hyde, Chris, Deeks, Jonathan J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5326492/
https://www.ncbi.nlm.nih.gov/pubmed/28236806
http://dx.doi.org/10.1186/s12874-016-0287-z
_version_ 1782510567483768832
author Ferrante di Ruffano, Lavinia
Dinnes, Jacqueline
Sitch, Alice J.
Hyde, Chris
Deeks, Jonathan J.
author_facet Ferrante di Ruffano, Lavinia
Dinnes, Jacqueline
Sitch, Alice J.
Hyde, Chris
Deeks, Jonathan J.
author_sort Ferrante di Ruffano, Lavinia
collection PubMed
description BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. METHODS: We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. RESULTS: One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. CONCLUSION: Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0.
format Online
Article
Text
id pubmed-5326492
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-53264922017-03-01 Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests Ferrante di Ruffano, Lavinia Dinnes, Jacqueline Sitch, Alice J. Hyde, Chris Deeks, Jonathan J. BMC Med Res Methodol Research Article BACKGROUND: There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results. METHODS: We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome. RESULTS: One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria. CONCLUSION: Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials. Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0. BioMed Central 2017-02-24 /pmc/articles/PMC5326492/ /pubmed/28236806 http://dx.doi.org/10.1186/s12874-016-0287-z Text en © The Author(s). 2017 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Ferrante di Ruffano, Lavinia
Dinnes, Jacqueline
Sitch, Alice J.
Hyde, Chris
Deeks, Jonathan J.
Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title_full Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title_fullStr Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title_full_unstemmed Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title_short Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
title_sort test-treatment rcts are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5326492/
https://www.ncbi.nlm.nih.gov/pubmed/28236806
http://dx.doi.org/10.1186/s12874-016-0287-z
work_keys_str_mv AT ferrantediruffanolavinia testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests
AT dinnesjacqueline testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests
AT sitchalicej testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests
AT hydechris testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests
AT deeksjonathanj testtreatmentrctsaresusceptibletobiasareviewofthemethodologicalqualityofrandomizedtrialsthatevaluatediagnostictests