Development and reliability assessment of a new quality appraisal tool for cross-sectional studies using biomarker data (BIOCROSS)

BACKGROUND: Biomarker-based analyses are commonly reported in observational epidemiological studies; however currently there are no specific study quality assessment tools to assist evaluation of conducted research. Accounting for study design and biomarker measurement would be important for deriving valid conclusions when conducting systematic data evaluation. METHODS: We developed a study quality assessment tool designed specifically to assess biomarker-based cross-sectional studies (BIOCROSS) and evaluated its inter-rater reliability. The tool includes 10-items covering 5 domains: ‘Study rational’, ‘Design/Methods’, ‘Data analysis’, ‘Data interpretation’ and ‘Biomarker measurement’, aiming to assess different quality features of biomarker cross-sectional studies. To evaluate the inter-rater reliability, 30 studies were distributed among 5 raters and intraclass correlation coefficients (ICC-s) were derived from respective ratings. RESULTS: The estimated overall ICC between the 5 raters was 0.57 (95% Confidence Interval (CI): 0.38–0.74) indicating a good inter-rater reliability. The ICC-s ranged from 0.11 (95% CI: 0.01–0.27) for the domain ‘Study rational’ to 0.56 (95% CI: 0.40–0.72) for the domain ‘Data interpretation’. CONCLUSION: BIOCROSS is a new study quality assessment tool suitable for evaluation of reporting quality from cross-sectional epidemiological studies employing biomarker data. The tool proved to be reliable for use by biomedical scientists with diverse backgrounds and could facilitate comprehensive review of biomarker studies in human research. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-018-0583-x) contains supplementary material, which is available to authorized users.