Evaluation criteria for health apps supporting medication adherence in early-stage technology development – a scoping review

INTRODUCTION: Health apps offer an approach to improve the patients’ management of their medication. Although the Digital Healthcare Act (DVG) has created a claim in the statutory health insurance (SHI), the large number of health apps available and their varying quality make it difficult for service providers and especially for medical laypersons to select an adequate high-quality medication app. Manufacturers need guidance for the development of high-quality apps right from the start. Various general evaluation concepts for health apps have been available to date. However, the requirements that should be met by healthcare depend largely on the field of application and the type of apps. This article aims to provide an overview of the international evidence on specific criteria for the evaluation of medication apps. METHODS: Within the framework of a scoping review, a systematic search was conducted in PubMed and EMBASE on January 29, 2020. The search was limited to publications from 2007 onwards as well as to English and German articles. Additionally, a semi-systematic research of reference lists of the previously included articles as well as a structured search of websites of relevant stakeholders were conducted. Inclusion criteria were the following: the publication deals with health apps that can be used on smartphones and focus on supporting medication intake; the publication does not refer to evaluation criteria for a single app exclusively. The included publications were examined in a qualitative content analysis searching for evaluation criteria and categorizing them according to the framework criteria of the DVG and the Digital Health Applications Ordinance (DiGAV). RESULTS: 2,542 articles were identified in the systematic search (999 in PubMed, 1,543 in EMBASE, 560 duplicates). A total of 16 studies met the inclusion criteria. The semi-systematic research and the structured search identified one further study. A catalog of criteria was developed based on the included 17 studies. This catalog covers the general topics “patient orientation” (data protection and security, consumer protection, user friendliness) and “quality/core functions of medication apps” (reminder, self-monitoring, (drug) information, motivation to change behavior, drug/patient safety, robustness) as well as “interoperability/cooperation”. Due to its specific importance for medication apps, the subcategory “motivation for behavioral change” stands out beneath the general topic “quality/core functions of medication apps”. This category aims to evaluate the design of individual functions with regard to their potential to actually change the behavior of app users. DISCUSSION: The criteria for the evaluation of health apps mentioned in the DiGAV intersected with the criteria identified in the literature research. However, the area of positive health care effects was hardly covered by the included studies. In the development of the criteria catalog, it was not possible to weight the identified criteria. Therefore, the catalog should be understood as a supporting checklist for service providers, manufacturers, and/or users. CONCLUSIONS: A large variety of possible evaluation criteria for medication apps could be shown. Future research should focus on the possibilities of weighting these diverse evaluation criteria, using not only clinical studies but also methods to identify preferences.