Case-mix adjustments for patient reported experience and outcome measures in primary care: an empirical approach to identify patient characteristics as case-mix adjusters based on a secondary analysis of an international survey among patients and their general practitioners in 34 countries

BACKGROUND: Case-mix adjustment of patient reported experiences (PREMs) and outcomes (PROMs) of care are meant to enable fair comparison between units (e.g. care providers or countries) and to show where improvement is possible. It is important to distinguish between fair comparison and improvement potential, as case-mix adjustment may mask improvement potential. Case-mix adjustment takes into account the effect of patient characteristics that are related to the PREMs and PROMs studied, but are outside the sphere of influence of the units being compared. We developed an approach to assess which patient characteristics would qualify as case-mix adjusters, using data from an international primary care study. RESULTS: We used multilevel analysis, with patients nested in general practices nested in countries. Case-mix adjustment is indicated under the following conditions: there is a main effect of the potential case-mix adjuster on the PREM/PROM; this effect does not vary between units; and the distribution of the potential case-mix adjuster differs between units. Random slope models were used to assess whether the impact of a potential case-mix adjuster varied between units. To assess whether a slope variance is big enough to decide that case-mix adjustment is not indicated, we compared the variances in the categories of a potential case-mix adjuster. Significance of the slope variance is not enough, because small variances may be significantly different from zero when numbers are large. We therefore need an additional criterion to consider a slope variance as important. Borrowing from the idea of a minimum clinically important difference (MCID) we proposed a difference between the variances of 0.25*variance (equivalent to a medium effect size). We applied this approach to data from the QUALICOPC (Quality and costs of primary care in Europe) study. CONCLUSIONS: Our approach provides guidance to decide whether or not patient characteristics should be considered as case-mix adjusters. The criterion of a difference between variances of 0.25*variance works well for continuous PREMs and PROMs, but seems to be too strict for binary PREMs and PROMs. Without additional information, it is not possible to decide whether important slope variation is the result of either differences in performance between general practices or countries, or cultural differences. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s41687-023-00667-8.