Applying Item Response Theory (IRT) Modeling to an Observational Measure of Childhood Pragmatics: The Pragmatics Observational Measure-2

Assessment of pragmatic language abilities of children is important across a number of childhood developmental disorders including ADHD, language impairment and Autism Spectrum Disorder. The Pragmatics Observational Measure (POM) was developed to investigate children’s pragmatic skills during play in a peer–peer interaction. To date, classic test theory methodology has reported good psychometric properties for this measure, but the POM has yet to be evaluated using item response theory. The aim of this study was to evaluate the POM using Rasch analysis. Person and item fit statistics, response scale, dimensionality of the scale and differential item functioning were investigated. Participants included 342 children aged 5–11 years from New Zealand; 108 children with ADHD were playing with 108 typically developing peers and 126 typically developing age, sex and ethnic matched peers played in dyads in the control group. Video footage of this interaction was recorded and later analyzed by an independent rater unknown to the children using the POM. Rasch analysis revealed that the rating scale was ordered and used appropriately. The overall person (0.97) and item (0.99) reliability was excellent. Fit statistics for four individual items were outside acceptable parameters and were removed. The dimensionality of the measure showed two distinct elements (verbal and non-verbal pragmatic language) of a unidimensional construct. These findings have led to a revision of the first edition of POM, now called the POM-2. Further empirical work investigating the responsiveness of the POM-2 and its utility in identifying pragmatic language impairments in other childhood developmental disorders is recommended.