Consumer Evaluation of the Quality of Online Health Information: Systematic Literature Review of Relevant Criteria and Indicators

BACKGROUND: As the quality of online health information remains questionable, there is a pressing need to understand how consumers evaluate this information. Past reviews identified content-, source-, and individual-related factors that influence consumer judgment in this area. However, systematic knowledge concerning the evaluation process, that is, why and how these factors influence the evaluation behavior, is lacking. OBJECTIVE: This review aims (1) to identify criteria (rules that reflect notions of value and worth) that consumers use to evaluate the quality of online health information and the indicators (properties of information objects to which criteria are applied to form judgments) they use to support the evaluation in order to achieve a better understanding of the process of information quality evaluation and (2) to explicate the relationship between indicators and criteria to provide clear guidelines for designers of consumer health information systems. METHODS: A systematic literature search was performed in seven digital reference databases including Medicine, Psychology, Communication, and Library and Information Science to identify empirical studies that report how consumers directly and explicitly describe their evaluation of online health information quality. Thirty-seven articles met the inclusion criteria. A qualitative content analysis was performed to identify quality evaluation criteria, indicators, and their relationships. RESULTS: We identified 25 criteria and 165 indicators. The most widely reported criteria used by consumers were trustworthiness, expertise, and objectivity. The indicators were related to source, content, and design. Among them, 114 were positive indicators (entailing positive quality judgments), 35 were negative indicators (entailing negative judgments), and 16 indicators had both positive and negative quality influence, depending on contextual factors (eg, source and individual differences) and criteria applied. The most widely reported indicators were site owners/sponsors; consensus among multiple sources; characteristics of writing and language; advertisements; content authorship; and interface design. CONCLUSIONS: Consumer evaluation of online health information is a complex cost-benefit analysis process that involves the use of a wide range of criteria and a much wider range of quality indicators. There are commonalities in the use of criteria across user groups and source types, but the differences are hard to ignore. Evidently, consumers’ health information evaluation can be characterized as highly subjective and contextualized, and sometimes, misinformed. These findings invite more research into how different user groups evaluate different types of online sources and a personalized approach to educate users about evaluating online health information quality.