Evaluating the effectiveness of behavior change techniques in health-related behavior: a scoping review of methods used

Behavior change interventions typically contain multiple potentially active components: behavior change techniques (BCTs). Identifying which specific BCTs or BCT combinations have the potential to be effective for a given behavior in a given context presents a major challenge. The aim of this study was to review the methods that have been used to identify effective BCTs for given behaviors in given contexts and evaluate their strengths and limitations. A scoping review was conducted of studies that had sought to identify effective BCTs. Articles referring to “behavio(u)r change technique(s)” in the abstract/text were located, and ones that involved identification of effective BCTs were selected. The methods reported were coded. The methods were analyzed in general terms using “PASS” criteria: Practicability (facility to apply the method appropriately), Applicability (facility to generalize from findings to contexts and populations of interest), Sensitivity (facility to identify effective BCTs), and Specificity (facility to rule out ineffective BCTs). A sample of 10% of the studies reviewed was then evaluated using these criteria to assess how far the strengths and limitations identified in principle were borne out in practice. One hundred and thirty-five studies were identified. The methods used in those studies were experimental manipulation of BCTs, observational studies comparing outcomes in the presence or absence of BCTs, meta-analyses of BCT comparisons, meta-regressions evaluating effect sizes with and without specific BCTs, reviews of BCTs found in effective interventions, and meta-classification and regression trees. The limitations of each method meant that only weak conclusions could be drawn regarding the effectiveness of specific BCTs or BCT combinations. Methods for identifying effective BCTs linked to target behavior and context all have important inherent limitations. A strategy needs to be developed that can systematically combine the strengths of the different methods and that can link these constructs in an ontology of behavior change interventions.