Raking of data from a large Australian cohort study improves generalisability of estimates of prevalence of health and behaviour characteristics and cancer incidence

BACKGROUND: Health surveys are commonly somewhat non-representative of their target population, potentially limiting the generalisability of prevalence estimates for health/behaviour characteristics and disease to the population. To reduce bias, weighting methods have been developed, though few studies have validated weighted survey estimates against generally accepted high-quality independent population benchmark estimates. METHODS: We applied post-stratification and raking methods to the Australian 45 and Up Study using Census data and compared the resulting prevalence of characteristics to accepted population benchmark estimates and separately, the incidence rates of lung, colorectal, breast and prostate cancer to whole-of-population estimates using Standardised Incidence Ratios (SIRs). RESULTS: The differences between 45 and Up Study and population benchmark estimates narrowed following sufficiently-informed raking, e.g. 13.6% unweighted prevalence of self-reported fair/poor overall health, compared to 17.0% after raking and 17.9% from a population benchmark estimate. Raking also improved generalisability of cancer incidence estimates. For example, unweighted 45 and Up Study versus whole-of-population SIRs were 0.700 (95%CI:0.574–0.848) for male lung cancer and 1.098 (95%CI:1.002–1.204) for prostate cancer, while estimated SIRs after sufficiently-informed raking were 0.828 (95%CI:0.684–0.998) and 1.019 (95%CI:0.926–1.121), respectively. CONCLUSION: Raking may be a useful tool for improving the generalisability of exposure prevalence and disease incidence from surveys to the population. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01626-5.