Using risk of bias domains to identify opportunities for improvement in food- and nutrition-related research: An evaluation of research type and design, year of publication, and source of funding

PURPOSE: This retrospective cross-sectional study aimed to identify opportunities for improvement in food and nutrition research by examining risk of bias (ROB) domains. METHODS: Ratings were extracted from critical appraisal records for 5675 studies used in systematic reviews conducted by three organizations. Variables were as follows: ROB domains defined by the Cochrane Collaboration (Selection, Performance, Detection, Attrition, and Reporting), publication year, research type (intervention or observation) and specific design, funder, and overall quality rating (positive, neutral, or negative). Appraisal instrument questions were mapped to ROB domains. The kappa statistic was used to determine consistency when multiple ROB ratings were available. Binary logistic regression and multinomial logistic regression were used to predict overall quality and ROB domains. FINDINGS: Studies represented a wide variety of research topics (clinical nutrition, food safety, dietary patterns, and dietary supplements) among 15 different research designs with a balance of intervention (49%) and observation (51%) types, published between 1930 and 2015 (64% between 2000–2009). Duplicate ratings (10%) were consistent (κ = 0.86–0.94). Selection and Performance domain criteria were least likely to be met (57.9% to 60.1%). Selection, Detection, and Performance ROB ratings predicted neutral or negative quality compared to positive quality (p<0.001). Funder, year, and research design were significant predictors of ROB. Some sources of funding predicted increased ROB (p<0.001) for Selection (interventional: industry only and none/not reported; observational: other only and none/not reported) and Reporting (observational: university only and other only). Reduced ROB was predicted by combined and other-only funding for intervention research (p<0.005). Performance ROB domain ratings started significantly improving in 2000; others improved after 1990 (p<0.001). Research designs with higher ROB were nonrandomized intervention and time series designs compared to RCT and prospective cohort designs respectively (p<0.001). CONCLUSIONS: Opportunities for improvement in food and nutrition research are in the Selection, Performance, and Detection ROB domains.