Use of online and paper-and-pencil questionnaires to assess the distribution of orthorexia nervosa, muscle dysmorphia and eating disorders among university students: can different approaches lead to different results?

PURPOSE: Administration of questionnaires to assess the diffusion of disordered eating behaviours via the web is becoming common today. The aim of this study is to assess whether two different approaches of administering a test to assess traits of eating disorders (EDs), orthorexia nervosa (ON) and muscle dysmorphia (MD) by email recruitment and online completion (web-based survey—WBS) and by in person recruitment and paper-and-pencil completion (paper-based survey—PBS), gives different results. METHODS: During 2 consecutive academic years, a self-reported questionnaire consisting of questions about personal characteristics and three tests for the evaluation of ON (ORTO-15), MD (MDDI-ITA), and EDs (EAT-26) were administered to two groups of undergraduates, respectively, as a WBS and a PBS. RESULTS: The WBS response rate was 6.7% (N = 137), and the PBS response rate was 86.5% (N = 372). The WBS group showed a statistically significant higher prevalence of students with eating disordered behaviours (21.2% vs 5.4%) and registered a higher mean score on the EAT-26 test (13.5 ± 11.1 vs 6.0 ± 8.0); no differences between the two groups emerged for ON and MD prevalence and test scores. Moreover, in the WBS group, the number of students with one or more tests with test scores above the cut-off values was significantly higher (46.0% vs 32.3%). CONCLUSION: The choice of the approach to administer a questionnaire to assess the diffusion of EDs and related issues must take into account all the factors that can result in selection bias and that can affect the reliability of the results. LEVEL OF EVIDENCE: Level V, descriptive cross-sectional survey. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40519-021-01231-3.