The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme

BACKGROUND: Evaluation of health technology programmes should be theoretically informed, interdisciplinary, and generate in-depth explanations. The NASSS (non-adoption, abandonment, scale-up, spread, sustainability) framework was developed to study unfolding technology programmes in real time—and in particular to identify and manage their emergent uncertainties and interdependencies. In this paper, we offer a worked example of how NASSS can also inform ex post (i.e. retrospective) evaluation. METHODS: We studied the TORPEDO (Treatment of Cardiovascular Risk in Primary Care using Electronic Decision Support) research programme, a multi-faceted computerised quality improvement intervention for cardiovascular disease prevention in Australian general practice. The technology (HealthTracker) had shown promise in a cluster randomised controlled trial (RCT), but its uptake and sustainability in a real-world implementation phase was patchy. To explain this variation, we used NASSS to undertake secondary analysis of the multi-modal TORPEDO dataset (results and process evaluation of the RCT, survey responses, in-depth professional interviews, videotaped consultations) as well as a sample of new, in-depth narrative interviews with TORPEDO researchers. RESULTS: Ex post analysis revealed multiple areas of complexity whose influence and interdependencies helped explain the wide variation in uptake and sustained use of the HealthTracker technology: the nature of cardiovascular risk in different populations, the material properties and functionality of the technology, how value (financial and non-financial) was distributed across stakeholders in the system, clinicians’ experiences and concerns, organisational preconditions and challenges, extra-organisational influences (e.g. policy incentives), and how interactions between all these influences unfolded over time. CONCLUSION: The NASSS framework can be applied retrospectively to generate a rich, contextualised narrative of technology-supported change efforts and the numerous interacting influences that help explain its successes, failures, and unexpected events. A NASSS-informed ex post analysis can supplement earlier, contemporaneous evaluations to uncover factors that were not apparent or predictable at the time but dynamic and emergent.