The Active Recovery Triad monitor: evaluation of a model fidelity scale for recovery-oriented care in long-term mental health care settings

OBJECTIVE: The Active Recovery Triad (ART) model is a recently developed care model for people who are admitted to an institutional setting for several years and receive 24-h mental health care and support. This study focuses on the ART monitor, a model fidelity scale that measures the degree of compliance with the ART model. Our aim is to evaluate the psychometric properties of the ART monitor and to further improve the instrument. METHODS: Fifteen teams at the start (n = 7, group 1) or in the process (6 months to three years) of implementing care according to the ART model (n = 8, group 2) were audited using the ART monitor. Auditors were trained care workers, peer workers, and family peer workers. Auditors and team members provided feedback on the instrument. The content validity, construct validity and inter-rater reliability of the ART monitor were investigated. Based on the outcomes of these psychometric properties, the ART monitor was finalized. RESULTS: Regarding content validity, auditors and teams indicated that they perceived the ART monitor to be a useful instrument. In terms of construct validity, a significant difference (t(13) = 2.53, p < 0.05) was found between teams at the start of the implementation process (group 1, average score of 2.42 (SD = 0.44)) and teams with a longer duration of implementation (group 2, average score of 2.95 (SD = 0.37)). When allowing for a one-point difference in scores, 88% of the items had an inter-rater agreement over 65%. Items with a relatively low inter-rater reliability, in combination with feedback from auditors and teams regarding content validity, provided direction for further improvement and revision of the instrument. CONCLUSIONS: We concluded that the revised ART monitor is feasible and useful in mental health care practice. However, further evaluation of its psychometric properties will be needed. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12888-022-03949-5.