Development and analysis of quality assessment tools for different types of patient information – websites, decision aids, question prompt lists, and videos

OBJECTIVE: Our working group has developed a set of quality assessment tools for different types of patient information material. In this paper we review and evaluate these tools and their development process over the past eight years. METHODS: We compared the content and structure of quality assessment tools for websites, patient decision aids (PDAs), question prompt lists (QPLs), and videos. Using data from their various applications, we calculated inter-rater concordance using Kendall’s W. RESULTS: The assessment tools differ in content, structure and length, but many core aspects remained throughout the development over time. We found a relatively large variance regarding the amount of quality aspects combined into one item, which may influence the weighting of those aspects in the final scores of evaluated material. Inter-rater concordance was good in almost all applications of the tool. Subgroups of similar expertise showed higher concordance rates than the overall agreement. CONCLUSION: All four assessment tools are ready to be used by people of different expertise. However, varying expertise may lead to some differences in the resulting assessments when using the tools. The lay and patient perspective needs to be further explored and taken into close consideration when refining the instruments. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-023-02204-5.