Evaluating the Impact of the U.S. National Toxicology Program: A Case Study on Hexavalent Chromium

BACKGROUND: Evaluating the impact of federally funded research with a broad, methodical, and objective approach is important to ensure that public funds advance the mission of federal agencies. OBJECTIVES: We aimed to develop a methodical approach that would yield a broad assessment of National Toxicology Program’s (NTP’s) effectiveness across multiple sectors and demonstrate the utility of the approach through a case study. METHODS: A conceptual model was developed with defined activities, outputs (products), and outcomes (proximal, intermediate, distal) and applied retrospectively to NTP’s research on hexavalent chromium (CrVI). Proximal outcomes were measured by counting views of and requests for NTP’s products by external stakeholders. Intermediate outcomes were measured by bibliometric analysis. Distal outcomes were assessed through Web and LexisNexis searches for documents related to legislation or regulation changes. RESULTS: The approach identified awareness of NTP’s work on CrVI by external stakeholders (proximal outcome) and citations of NTP’s research in scientific publications, reports, congressional testimonies, and legal and policy documents (intermediate outcome). NTP’s research was key to the nation’s first-ever drinking water standard for CrVI adopted by California in 2014 (distal outcome). By applying this approach to a case study, the utility and limitations of the approach were identified, including challenges to evaluating the outcomes of a research program. CONCLUSIONS: This study identified a broad and objective approach for assessing NTP’s effectiveness, including methodological needs for more thorough and efficient impact assessments in the future. CITATION: Xie Y, Holmgren S, Andrews DMK, Wolfe MS. 2017. Evaluating the impact of the U.S. National Toxicology Program: a case study on hexavalent chromium. Environ Health Perspect 125:181–188; http://dx.doi.org/10.1289/EHP21