Homogeneity Test of the First-Order Agreement Coefficient in a Stratified Design

Gwet’s first-order agreement coefficient (AC [Formula: see text]) is widely used to assess the agreement between raters. This paper proposes several asymptotic statistics for a homogeneity test of stratified AC [Formula: see text] in large sample sizes. These statistics may have unsatisfactory performance, especially for small samples and a high value of AC [Formula: see text]. Furthermore, we propose three exact methods for small pieces. A likelihood ratio statistic is recommended in large sample sizes based on the numerical results. The exact E approaches under likelihood ratio and score statistics are more robust in the case of small sample scenarios. Moreover, the exact E method is effective to a high value of AC [Formula: see text]. We apply two real examples to illustrate the proposed methods.