An international and interdisciplinary Delphi study to measure digital public health system maturity

BACKGROUND: The Covid-19 pandemic showed how crucial digital public health (DiPH) interventions are for continuous treatment and system sustainability. Several studies highlighted how the digital system transformation might improve access to and quality of healthcare services. However, there is no consensus on how to best assess the national readiness for the digitalization of such systems. Therefore, this Delphi study aimed to collect indicators to measure the maturity of DiPH systems nationally. METHODS: The three-stage international and interdisciplinary Delphi study included 82 experts. During the first round, participants proposed indicators to measure maturity among the technological and legal requirements of DiPH tools, the willingness of the general public to use them, and the degree of application of DiPH tools to the national healthcare system. The suggestions were qualitatively assessed and presented for ranking on a 4-point Likert scale during the second round. Experts were also able to rephrase or add indicators. In the third round, experts could only rank indicators. RESULTS: Finally, 96 indicators to measure the maturity of DiPH systems were finally. Of these, 31 fall under the legal and 24 under the technological requirements. 26 assess the social level and 15 the degree of application. All dimensions can be clustered into 3-4 sub-groups each. The qualitative analysis highlighted the importance of data protection regulations, affordable smartphone and internet access, interoperability between interventions, financial incentives, user motivation and literacy, and whether interventions were implemented as local pilots or national services. CONCLUSIONS: The study identified indicators for interdisciplinary assessing DiPH system maturity. While this Delphi can’t answer how best to integrate these indicators into national health system evaluation procedures, this approach is an essential first step towards an index to measure the maturity of DiPH systems. KEY MESSAGES: • When assessing the digital public health system's maturity, one needs to include legal, technological, and social influencing aspects and not only evaluate the health services implementation degree. • International and interdisciplinary Delphi study panels are vital for finding consensus on indicators to create tools of international importance and usability to measure health system maturity.