European Public Assessment Report (EPAR) summaries for the public: are they fit for purpose? A user-testing study

OBJECTIVES: Apply ‘user testing’ methodology to test the readability of a European Public Assessment Report (EPAR) summary—which describes how the decision was made by the European Medicines Agency to approve a medicine. DESIGN: User testing uses mixed methods (questionnaire and semistructured interview), applied iteratively, to assess document performance—can people find and understand key points of information. SETTING AND PARTICIPANTS: Testing was undertaken with 40 members of the public in four consecutive rounds of 10. Inclusion criteria, matched across rounds, included range of ages and educational attainment. TESTED DOCUMENTS: In round 1 we tested 19 key points of information in a printed version of the EPAR summary for Bondronat (a cancer medicine). This was then revised to address the findings, and tested in round 2. In round 3 we tested the summary on-screen, and in round 4, tested a revised on-screen version, after addressing findings from both rounds 1 and 3. PRIMARY OUTCOME MEASURE: The target followed European guidance for medicine leaflets: for each point of information 90% of participants should be able to find, and of those, 90% able to show understanding of the point. RESULTS: For the original EPAR summary, 6 of the 19 points of information reached the target (both paper-based and on-screen). After revisions to format and content, using good practice in information writing and design, 14 and 16 points, respectively, met the target. The problems related to both finding (dependent on layout, headings and design) and understanding (words and sentences used, as well as design). We devised a new heading structure, increased use of bullet points, replaced difficult and technical words and divided long sentences. CONCLUSIONS: People had difficulty finding and understanding key messages in the summary, but user testing identified the problems, and application of good practice resulted in a revised format which performed well.