Assessing the order of magnitude of outcomes in single-arm cohorts through systematic comparison with corresponding cohorts: An example from the AMOS study

BACKGROUND: When a therapy has been evaluated in the first clinical study, the outcome is often compared descriptively to outcomes in corresponding cohorts receiving other treatments. Such comparisons are often limited to selected studies, and often mix different outcomes and follow-up periods. Here we give an example of a systematic comparison to all cohorts with identical outcomes and follow-up periods. METHODS: The therapy to be compared (anthroposophic medicine, a complementary therapy system) had been evaluated in one single-arm cohort study: the Anthroposophic Medicine Outcomes Study (AMOS). The five largest AMOS diagnosis groups (A-cohorts: asthma, depression, low back pain, migraine, neck pain) were compared to all retrievable corresponding cohorts (C-cohorts) receiving other therapies with identical outcomes (SF-36 scales or summary measures) and identical follow-up periods (3, 6 or 12 months). Between-group differences (pre-post difference in an A-cohort minus pre-post difference in the respective C-cohort) were divided with the standard deviation (SD) of the baseline score of the A-cohort. RESULTS: A-cohorts (5 cohorts with 392 patients) were similar to C-cohorts (84 cohorts with 16,167 patients) regarding age, disease duration, baseline affection and follow-up rates. A-cohorts had ≥ 0.50 SD larger improvements than C-cohorts in 13.5% (70/517) of comparisons; improvements of the same order of magnitude (small or minimal differences: -0.49 to 0.49 SD) were found in 80.1% of comparisons; and C-cohorts had ≥ 0.50 SD larger improvements than A-cohorts in 6.4% of comparisons. Analyses stratified by diagnosis had similar results. Sensitivity analyses, restricting the comparisons to C-cohorts with similar study design (observational studies), setting (primary care) or interventions (drugs, physical therapies, mixed), or restricting comparisons to SF-36 scales with small baseline differences between A- and C-cohorts (-0.49 to 0.49 SD) also had similar results. CONCLUSION: In this descriptive analysis, anthroposophic therapy was associated with SF-36 improvements largely of the same order of magnitude as improvements following other treatments. Although these non-concurrent comparisons cannot assess comparative effectiveness, they suggest that improvements in health status following anthroposophic therapy can be clinically meaningful. The analysis also demonstrates the value of a systematic approach when comparing a therapy cohort to corresponding therapy cohorts.