Identifying potential indicators to measure the outcome of translational cancer research: a mixed methods approach

BACKGROUND: In a context where there is an increasing demand to evaluate the outcome of bio-medical research, our work aims to develop a set of indicators to measure the impact of translational cancer research. The objective of our study was to explore the scope and issues of translational research relevant to evaluation, explore the views of researchers on the evaluation of oncological translational research, and select indicators measuring the outcomes and outputs of translational research in oncology by consensus. METHODS: Semi-structured interviews amongst 23 researchers involved in translational cancer research were conducted and analysed using thematic analysis. A two-round modified Delphi survey of 35 participants with similar characteristics was then performed followed by a physical meeting. Participants rated the feasibility and validity of 60 indicators. The physical meeting was held to discuss the methodology of the new indicators. RESULTS: The main themes emerging from the interviews included a common definition for translational research but disagreements about the exact scope and limits of this research, the importance of multidisciplinarity and collaboration for the success of translational research, the disadvantages that translational research faces in current evaluation systems, the relative lack of pertinence of existing indicators, and propositions to measure translational cancer research in terms of clinical applications and patient outcomes. A total of 35 participants took part in the first round survey and 12 in the second round. The two-round survey helped us select a set of 18 indicators, including four that seemed to be particularly adapted to measure translational cancer research impact on health service research (number of biomarkers identified, generation of clinical guidelines, citation of research in clinical guidelines, and citation of research in public health guidelines). The feedback from participants helped refine the methodology and definition of indicators not commonly used. CONCLUSION: Indicators need to be accepted by stakeholders under evaluation. This study helped the selection and refinement of indicators considered as the most relevant by researchers in translational cancer research. The feasibility and validity of those indicators will be tested in a scientometric study. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12961-015-0060-5) contains supplementary material, which is available to authorized users.