Development of a Group Judgment Process for Forecasts of Health Care Innovations

IMPORTANCE: Health care costs have increased substantially over the past few decades, in part owing to the development and diffusion of new medical treatments. Forecasting potential future technologic innovations can allow for more informed planning. OBJECTIVE: To assess the predictive validity of a structured formal method for forecasting future technologic innovations in health care. DESIGN, SETTING, AND PARTICIPANTS: This pilot study combined an untested, unvalidated combination of a consensus process and group judgment process to evaluate forecasts made in 2001 for technologic innovations by 2021 in Alzheimer disease (AD) and cardiovascular disease (CVD). Six experts in AD and 7 experts in CVD composed the judgment group. The study was conducted in 2017-2018. MAIN OUTCOMES AND MEASURES: Year 2001 forecasts for 2021 that were judged by experts as being close to correct, directionally correct, or not correct, as well as innovations that occurred since 2001 that were not predicted. RESULTS: Four forecasts of innovations in AD, each considered to be between 30% and 40% likely to be achieved by 2021, were judged to be close to correct. One forecast was considered to be directionally correct, with a likelihood of occurrence of 40%, in that it was overoptimistic. One innovation that occurred was missed: new imaging techniques (amyloid β plaque and tau tangle positron emission tomographic imaging). Five forecasts of CVD innovations were considered to be at least 50% likely to occur by 2021, and of these, 2 were judged to be close to correct, 1 was judged as being directionally correct, and 2 were judged as being not correct (although in one of these forecasts, the overarching innovation has been achieved but with a different noninvasive imaging modality). Of 7 additional forecasts considered to be less likely to be achieved by 2021, 4 were judged to be close to correct and 3 were judged as being directionally correct. Two innovations occurred but were missed: transcatheter aortic valve replacement and cardiac resynchronization therapy. Across both conditions, 15 of 17 innovations forecasted were judged to be close to correct or directionally correct, 2 were judged to be incorrect, and there were 3 missed innovations. CONCLUSIONS AND RELEVANCE: Expert elicitation provided a useful, but not fully accurate, lens into future innovation.