Involvement of methodological experts and the quality of clinical practice guidelines: a critical appraisal of clinical practice guidelines and a questionnaire survey of the development groups in Japan

OBJECTIVE: To evaluate whether the involvement of methodological experts improves the quality of clinical practice guidelines (CPGs) after adjusting for other factors. SETTING: The quality of Japanese CPGs published in 2011–2019 was assessed using the Appraisal of Guidelines, Research, and Evaluation (AGREE) II instrument. A questionnaire survey targeting CPG development groups was conducted through postal mail. PARTICIPANTS: 405 CPGs were retrieved from a Japanese CPG clearinghouse. Questionnaires were distributed to the 405 CPG development groups. Of the 178 respondents, 22 were excluded because of missing values. Finally, 156 participants representing their CPG development groups were included in the analysis. PRIMARY AND SECONDARY OUTCOME MEASURES: CPG quality was assessed using the AGREE II tool. The characteristics of CPGs, including publication year, development organisation, versions, number of members in the development group and involvement of methodological experts, were corrected from the description in the CPGs and the questionnaire survey. We performed multiple logistic regressions using the quality of CPGs as the dependent variable and the involvement of experts as the independent variable, adjusting for other possible factors. RESULTS: A total of 156 CPGs were included. Expert involvement was significantly associated with the AGREE II instrument scores in domains 1 (β=0.207), 2 (β=0.370), 3 (β=0.413), 4 (β=0.289), 5 (β=0.375), 6 (β=0.240) and overall (β=0.344). CONCLUSION: This study revealed that the involvement of methodological experts in the CPG development process improves the quality of CPGs. The results suggest the importance of establishing a training and certification programme for experts and constructing expert referral systems that meet CPG developers’ needs to improve the quality of CPGs.