Assessing stable validity and reliability of the Japanese version of the individualized supported employment fidelity scale: A replication

BACKGROUND: The Japanese version of the individualized Supported Employment Fidelity scale (JiSEF) was developed by modifying the 25‐item Individual Placement and Support Fidelity Scale (IPS‐25). While a preliminary study partly confirmed the concurrent validity with vocational outcomes, this replication study aimed to examine the stability of the concurrent validity and the inter‐rater reliability of the JiSEF and to test its convergent validity with IPS‐25. METHODS: Fidelity assessments were conducted in 2016 (n = 17), 2017 (n = 13), and 2018 (n = 18) to examine the employment rate and the fidelity scores at the agency level. We also evaluated the fidelity scores for the IPS‐25 in 2018. We examined the associations between the fidelity scale scores and vocational outcomes for the concurrent validity and between the fidelity scales for convergent validity. The inter‐rater reliability was examined in the 2016 and 2017 assessments. RESULTS: High intraclass correlation coefficients (0.93 in 2016 and 0.92 in 2017) were obtained for the inter‐rater reliability. The JiSEF score in each year was associated with the agency employment rate (r = 0.710, P = 0.001 in 2016; r = 0.722, P = 0.005 in 2017; and r = 0.665, P = 0.003 in 2018). A supplementary longitudinal data analysis also confirmed the association between the JiSEF score and the employment outcomes. Additionally, the JiSEF was significantly correlated with the IPS‐25 (r = 0.760, P < 0.001). CONCLUSIONS: This study stably replicated good inter‐rater reliability and concurrent validity of the JiSEF. Additionally, the convergent validity was confirmed. Further studies with large samples are needed to confirm these findings.