Challenges With the Use of Digital Sham: Systematic Review and Recommendations

BACKGROUND: Digital therapeutics (DTx) are software-based products that prevent, manage, or treat a medical condition and are delivered through a smartphone app, web application, or wearable device. Clinical trials assessing DTx pose challenges, foremost among which is designing appropriate digital shams (or digital placebos), which should ideally mimic DTx (in terms of design, components, and duration of treatment) while omitting the active principle or component. OBJECTIVE: The objective of our review was to understand how digital shams are being used in clinical research on DTx in neuroscience, which is the most common therapy area for DTx. METHODS: We conducted a systematic literature review of DTx in neuroscience (including neurodevelopmental, neurodegenerative, and psychiatric disorders) with a focus on controlled clinical trials involving digital shams. Studies were identified from trial registries (ClinicalTrials.gov, the European Union Clinical Trials Register, and Trial Trove) and through structured searches in MEDLINE and Embase (both via the Embase website) and were limited to articles in English published from 2010 onward. These were supplemented by keyword-based searches in PubMed, Google, and Google Scholar and bibliographic searches. Studies assessing DTx in neuroscience (including neurodevelopmental, neurodegenerative, and psychiatric disorders) were included. Details related to the publication, DTx, comparator, patient population, and outcomes were extracted and analyzed. RESULTS: Our search criteria identified 461 neuroscience studies involving 213 unique DTx. Most DTx were extended reality based (86/213, 40.4%) or mobile device based (56/213, 26.3%); 313 were comparative, of which 68 (21.7%) used shams. The most common therapeutic areas assessed in these studies were stroke (42/213, 19.7%), depression (32/213, 15%), and anxiety (24/213, 11.3%). The most common treatments were cognitive behavioral therapy or behavioral therapy (67/213, 32.4%), physical rehabilitation (60/213, 28.2%), and cognitive training (41/213, 19.2%). We identified the following important issues related to the use of digital shams in neuroscience: shams were not validated before use in studies, they varied widely in design (from being nearly identical to the DTx to using different software programs altogether), and the level of patient engagement or satisfaction with the sham and the impact of the sham on study outcomes were infrequently reported. CONCLUSIONS: Digital shams are critical for the clinical development of DTx in neuroscience. Given the importance of sham controls in evaluating DTx efficacy, we provide recommendations on the key information that should be reported in a well-designed DTx trial and propose an algorithm to allow the correct interpretation of DTx study results. Sham-controlled studies should be routinely used in DTx trials—in early-phase studies—to help identify DTx active components and—in late-phase studies—to confirm the efficacy of DTx. The use of shams early in development will ensure that the appropriate sham control is used in later confirmatory trials.