Tracking the impact of research on policy and practice: investigating the feasibility of using citations in clinical guidelines for research evaluation

OBJECTIVES: To investigate the feasibility of using research papers cited in clinical guidelines as a way to track the impact of particular funding streams or sources. SETTING: In recent years, medical research funders have made efforts to enhance the understanding of the impact of their funded research and to provide evidence of the ‘value’ of investments in particular areas of research. One of the most challenging areas of research evaluation is around impact on policy and practice. In the UK, the National Institute of Health and Clinical Excellence (NICE) provide clinical guidelines, which bring together current high-quality evidence on the diagnosis and treatment of clinical problems. Research referenced in these guidelines is an indication of its potential to have real impact on health policy and practice. DESIGN: This study is based on analysis of the authorship and funding attribution of research cited in two NICE clinical guidelines: dementia and chronic obstructive pulmonary disease. RESULTS: Analysis identified that around a third of papers cited in the two NICE guidelines had at least one author based in the UK. In both cases, about half of these UK attributed papers contained acknowledgements which allowed the source of funding for the research to be identified. The research cited in these guidelines was found to have been supported by a diverse set of funders from different sectors. The study also investigated the contribution of research groups based in universities, industry and the public sector. CONCLUSIONS: The study found that there is great potential for guidelines to be used as sources of information on the quality of the research used in their development and that it is possible to track the source of the funding of the research. The challenge is in harnessing the relevant information to track this in an efficient way.