Measuring the Impact of a Moving Target: Towards a Dynamic Framework for Evaluating Collaborative Adaptive Interactive Technologies

BACKGROUND: Website evaluation is a key issue for researchers, organizations, and others responsible for designing, maintaining, endorsing, approving, and/or assessing the use and impact of interventions designed to influence health and health services. Traditionally, these evaluations have included elements such as content credibility, interface usability, and overall design aesthetics. With the emergence of collaborative, adaptive, and interactive ("Web 2.0") technologies such as wikis and other forms of social networking applications, these metrics may no longer be sufficient to adequately assess the quality, use or impact of a health website. Collaborative, adaptive, interactive applications support different ways for people to interact with health information on the Web, including the potential for increased user participation in the design, creation, and maintenance of such sites. OBJECTIVE: We propose a framework that addresses how to evaluate collaborative, adaptive, and interactive applications. METHODS: In this paper, we conducted a comprehensive review of a variety of databases using terminology related to this area. RESULTS: We present a review of evaluation frameworks and also propose a framework that incorporates collaborative, adaptive, and interactive technologies, grounded in evaluation theory. CONCLUSION: This framework can be applied by researchers who wish to compare Web-based interventions, non-profit organizations, and clinical groups who aim to provide health information and support about a particular health concern via the Web, and decisions about funding grants by agencies interested in the role of social networks and collaborative, adaptive, and interactive technologies technologies to improve health and the health system.