Indirect comparisons of therapeutic interventions

HEALTH POLITICAL BACKGROUND: The comparison of the effectiveness of health technologies is not only laid down in German law (Social Code Book V, § 139 and § 35b) but also constitutes a central element of clinical guidelines and decision making in health care. Tools supporting decision making (e. g. Health Technology Assessments (HTA)) are therefore in need of a valid methodological repertoire for these comparisons. SCIENTIFIC BACKGROUND: Randomised controlled head-to-head trials which directly compare the effects of different therapies are considered the gold standard methodological approach for the comparison of the efficacy of interventions. Because this type of trial is rarely found, comparisons of efficacy often need to rely on indirect comparisons whose validity is being controversially debated. RESEARCH QUESTIONS: Research questions for the current assessment are: Which (statistical) methods for indirect comparisons of therapeutic interventions do exist, how often are they applied and how valid are their results in comparison to the results of head-to-head trials? METHODS: In a systematic literature research all medical databases of the German Institute of Medical Documentation and Information (DIMDI) are searched for methodological papers as well as applications of indirect comparisons in systematic reviews. Results of the literature analysis are summarized qualitatively for the characterisation of methods and quantitatively for the frequency of their application. The validity of the results from indirect comparisons is checked by comparing them to the results from the gold standard – a direct comparison. Data sets from systematic reviews which use both direct and indirect comparisons are tested for consistency by of the z-statistic. RESULTS: 29 methodological papers and 106 applications of indirect methods in systematic reviews are being analysed. Four methods for indirect comparisons can be identified: 1. Unadjusted indirect comparisons include, independent of any comparator, all randomised controlled trials (RCT) that provide a study arm with the intervention of interest. . 2. Adjusted indirect comparisons and 3. metaregression analyses include only those studies that provide one study arm with the intervention of interest and another study arm with a common comparator. While the aforementioned methods use conventional metaanalytical techniques, . 4. Mixed treatment comparisons (MTC) use Bayesian statistics. They are able to analyse a complex network of RCT with multiple comparators simultaneously. . During the period from 1999 to 2008 adjusted indirect comparisons are the most commonly used method for indirect comparisons. Since 2006 an increase in the application of the more methodologically challenging MTC is being observed. For the validity check 248 data sets, which include results of a direct and an indirect comparison, are available. The share of statistically significant discrepant results is greatest in the unadjusted indirect comparisons (25,5% [95% CI: 13,1%; 38%]), followed by metaregression analyses (16,7% [95% CI: -13,2%; 46,5%]), adjusted indirect comparisons (12,1% [95% CI: 6,1%; 18%]) and MTC (1,8% [95% CI: -1,7%; 5,2%]). Discrepant results are mainly detected if the basic assumption for an indirect comparison – between-study homogeneity – does not hold. However a systematic over- or underestimation of the results of direct comparisons by any of the indirectly comparing methods was not observed in this sample. DISCUSSION: The selection of an appropriate method for an indirect comparison has to account for its validity, the number of interventions to be compared and the quality as well as the quantity of available studies. Unadjusted indirect comparisons provide, contrasted with the results of direct comparisons, a low validity. Adjusted indirect comparisons and MTC may, under certain circumstances, give results which are consistent with the results of direct comparisons. The limited number of available reviews utilizing metaregression analyses for indirect comparisons currently prohibits empirical evaluation of this methodology. CONCLUSIONS/RECOMMENDATIONS: Given the main prerequisite – a pool of homogenous and high-quality RCT – the results of head-to-head trials may be pre-estimated by an adjusted indirect comparison or a MTC. In the context of HTA and guideline development they are valuable tools if there is a lack of a direct comparison of the interventions of interest.