Evidencing the impact of cancer trials: insights from the 2014 UK Research Excellence Framework

INTRODUCTION: An impactful clinical trial will have real-life benefits for patients and society beyond the academic environment. This study analyses case studies of cancer trials to understand how impact is evidenced for cancer trials and how impact evaluation can be more routinely adopted and improved. METHODS: The United Kingdom (UK) Government allocates research funding to higher-education institutions based on an assessment of the institutions’ previous research efforts, in an exercise known as the Research Excellence Framework (REF). In addition to each institution’s journal publications and research environment, for the first time in 2014, allocation of funding was also dependent on an evaluation of the wider, societal impact of research conducted. In the REF2014, impact assessment was performed by evaluation of impact case studies. In this study, case studies (n = 6637) submitted by institutions for the REF2014 were accessed and those focussing on cancer trials were identified. Manual content analysis was then used to assess the characteristics of the cancer trials discussed in the case studies, the impact described and the methods used by institutions to demonstrate impact. RESULTS: Forty-six case studies describing 106 individual cancer trials were identified. The majority were phase III randomised controlled trials and those recruiting patients with breast cancer. A list of indicators of cancer trial impact was generated using the previous literature and developed inductively using these case studies. The most common impact from a cancer trial identified in the case studies was on policy, in particular citation of trial findings in clinical guidelines. Impact on health outcomes and the economy were less frequent and health outcomes were often predicted rather than evidenced. There were few descriptions identified of trialists making efforts to maximise trial impact. DISCUSSION: Cancer trial impact narratives for the next REF assessment exercise in 2021 can be improved by evidencing actual rather than predicted Impact, with a clearer identification of the beneficiaries of cancer trials and the processes through which trial results are used. Clarification of the individuals responsible for performing impact evaluations of cancer trials and the provision of resources to do so needs to be addressed if impact evaluation is to be sustainable.