“We cobble together a storyline of system performance using a diversity of things”: a qualitative study of perspectives on public health performance measurement in Canada

BACKGROUND: There have been longstanding calls for public health systems transformations in many countries, including Canada. Core to these calls has been strengthening performance measurement. While advancements have been made in performance measurement for certain sectors of the health care system (primarily focused on acute and primary health care), effective use of indicators for measuring public health systems performance are lacking. This study describes the current state, anticipated challenges, and future directions in the development and implementation of a public health performance measurement system for Canada. METHODS: We conducted a qualitative study using semi-structured interviews with public health leaders (n = 9) between July and August 2021. Public health leaders included researchers, government staff, and former medical officers of health who were purposively selected due to their expertise and experience with performance measurement with relevance to public health systems in Canada. Thematic analysis included both a deductive approach for themes consistent with the conceptual framework and an inductive approach to allow new themes to emerge from the data. RESULTS: Conceptual, methodological, contextual, and infrastructure challenges were highlighted by participants in designing a performance measurement system for public health. Specifically, six major themes evolved that encompass 1) the mission and purpose of public health systems, including challenges inherent in measuring the functions and services of public health; 2) the macro context, including the impacts of chronic underinvestment and one-time funding injections on the ability to sustain a measurement system; 3) the organizational structure/governance of public health systems including multiple forms across Canada and underdevelopment of information technology systems; 4) accountability approaches to performance measurement and management; and 5) timing and unobservability in public health indicators. These challenges require dedicated investment, strong leadership, and political will from the federal and provincial/territorial governments. CONCLUSION: Unprecedented attention on public health due to the coronavirus disease 2019 pandemic has highlighted opportunities for system improvements, such as addressing the lack of a performance measurement system. This study provides actionable knowledge on conceptual, methodological, contextual, and infrastructure challenges needed to design and build a pan-Canadian performance measurement system for public health. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13690-022-00931-1.