Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014–2018)

BACKGROUND: A Risk Evaluation and Mitigation Strategy (REMS) is a drug safety program for certain medications with serious safety concerns required by the U.S. Food and Drug Administration (FDA) of manufacturers to implement to help ensure the benefits of the medication outweigh its risks. FDA is encouraging “the research community to develop novel methods for assessing REMS,” conveying the unmet need for a standardized evaluation method of these regulatory-mandated healthcare programs. The objective of this research is to evaluate FDA REMS assessment plans using established implementation science frameworks and identify opportunities for strengthening REMS evaluation. METHODS: A content analysis was conducted of publicly available assessment plans for all REMS programs (N = 23) approved 1/1/2014–12/31/2018 for new drug applications (NDAs) and biologics license applications (BLAs) requiring FDA-mandated Elements to Assure Safe Use (ETASU). Blinded reviewers critically appraised REMS assessment measures (n = 674) using three established implementation science frameworks: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); PRECEDE-PROCEED (Predisposing, Reinforcing, and Enabling Constructs in Educational/Environmental Diagnosis and Evaluation – Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development); and CFIR (Consolidated Framework for Implementation Research). Framework constructs were mapped to REMS Assessment categories as defined by FDA Guidance for Industry to evaluate congruence. RESULTS: REMS assessment measures demonstrated strong congruence (> 90% mapping rate) with the evaluative constructs of RE-AIM, PRECEDE-PROCEED, and CFIR. Application of the frameworks revealed that REMS assessment measures heavily emphasize implementation and operations, focus less on health outcomes, and do not evaluate program context and design assumptions. CONCLUSIONS: Implementation science frameworks have utility for evaluating FDA-mandated drug safety programs including the selection of primary measures to determine whether REMS goals are being met and of secondary measures to evaluate contextual factors affecting REMS effectiveness in varying organizational settings. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12913-021-06808-3.