Development and Testing of a Multimedia Internet-Based System for Fidelity and Monitoring of Multidimensional Treatment Foster Care

BACKGROUND: The fields of mental health, child welfare, and juvenile justice are jointly faced with the challenge of reducing the prevalence of antisocial behavior among adolescents. In the last 20 years, conduct disorders have moved from being considered intractable difficulties to having complex but available solutions. The treatments for even long-standing offending behavior among adolescents are now well documented and supported by a growing and compelling body of evidence. These empirically validated interventions are being widely disseminated, but the replication of the results from clinical trials in community settings has yet to be documented. The treatments, which produced impressive effects in a research context, are difficult to replicate without intensive monitoring of fidelity by the developers. Such monitoring is a barrier toward adoption; as the distance between the adopter and developer increases, so does cost. At the same time, states, communities, and agencies are under increasing pressure to implement those intervention services that have been shown to be most effective. The use of the Internet offers a potential solution in that existing reporting and data collection by clinicians can be subject to remote supervision. Such a system would have the potential to provide dissemination teams with more direct access to higher-quality data and would make adopters more likely to be able to implement services at the highest possible conformity to research protocols. OBJECTIVE: To create and test such an innovative system for use with the Multidimensional Treatment Foster Care (MTFC) program, which is an in-home treatment (alternative to a residential- or group-home setting) for antisocial youths. This research could advance the knowledge base about developing innovative infrastructures in community settings to disseminate empirically validated treatments. METHODS: The fidelity system was used and reviewed by parent and professional users: 20 foster parent users of the Parent Daily Report function, 9 professional MTFC program supervisors, and 4 MTFC consultants. All participants rated the system’s ease of use, quality of the website, and observational videos recorded at agency meetings. In addition, foster parents entered data on child behavior. RESULTS: All professionals and foster parents rated the system as very easy to use. We found particularly high levels of use by parents. Professionals rated the computer-collected videos of clinical meetings as being of high quality and easily codeable. CONCLUSIONS: The project developed a user-friendly and secure Web-based system using state-of-the-art computer-based protocols for recording questionnaire and observational data generated by community-based MTFC staff and foster parents, with positive satisfaction and utilization results.