Reliability and validity of a novel caregiver-assessed skills system based on the ALSO conception in children with autism spectrum disorders

BACKGROUND: In responding to the potential challenges in the transition from childhood to adulthood for individuals with autism spectrum disorders (ASD) in China, a novel conception of “ALSO” was proposed to bridge the transitional needs and early intervention. To facilitate the application of ALSO in early intervention, ALSOLIFE skills assessment system (ALSOLIFE Assessment) was developed to enable caregivers to evaluate their children’s skills guided by the ALSO conception. Given that the critical shortage of qualified professionals in China, many caregivers of children with ASD must function as home therapists. To address the practical needs of Chinese families of children with ASD, ALSOLIFE Assessment is designed as a free, online, technology-assisted, self-operated and behavioral intervention approaches supported system. The assessment report then further served as the basis for caregivers to deliver the tailored educational intervention to their children. Although ALSOLIFE Assessment provides caregivers a home-based intervention program, it is still unclear whether its evaluation is reliable and accurate. Therefore, we conduct this study to investigate the reliability and validity of the ALSOLIFE Assessment. METHODS: A total of 1,050 children with ASD (1 to 10 years old) were recruited from 31 provinces of mainland China. Their caregivers participated simultaneously as ALSOLIFE Assessment evaluators. The testing results of Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP) and Psychoeducational Profile-3 (PEP-3) were also collected for a portion of the child participants (VB-MAPP, N=34; PEP-3, N=31) to check criterion-related validity. RESULTS: The reliability and validity of the ALSOLIFE Assessment satisfied psychometric requirements after the reduction from 511 to 464 items. Exploratory factor analysis (EFA) of ALSOLIFE Assessment scoring data yielded six factors, and confirmatory factor analysis (CFA) verified the best fitting construct model is a bifactorial model with one general factor and six group factors. Compared to the VB-MAPP and PEP-3, the ALSOLIFE Assessment exhibited good criterion-related validity across three levels: total scores, 6 skill domains, and 22 skill subdomains. The reliability tests indicated its strong internal consistency, test-retest reliability, and inter-rater reliability. CONCLUSIONS: The reliability and validity of the revised ALSOLIFE Assessment (464-items) satisfied psychometric requirements. It is essential to develop validated and comprehensive evaluation tools, embedded on the needs of Chinese families of children with ASD.