Reliability of classification for post-traumatic ankle osteoarthritis

PURPOSE: The purpose of this study was to identify the most reliable classification system for clinical outcome studies to categorize post-traumatic—fracture—osteoarthritis. METHODS: A total of 118 orthopaedic surgeons and residents—gathered in the Ankle Platform Study Collaborative Science of Variation Group—evaluated 128 anteroposterior and lateral radiographs of patients after a bi- or trimalleolar ankle fracture on a Web-based platform in order to rate post-traumatic osteoarthritis according to the classification systems coined by (1) van Dijk, (2) Kellgren, and (3) Takakura. Reliability was evaluated with the use of the Siegel and Castellan’s multirater kappa measure. Differences between classification systems were compared using the two-sample Z-test. RESULTS: Interobserver agreement of surgeons who participated in the survey was fair for the van Dijk osteoarthritis scale (k = 0.24), and poor for the Takakura (k = 0.19) and the Kellgren systems (k = 0.18) according to the categorical rating of Landis and Koch. This difference in one categorical rating was found to be significant (p < 0.001, CI 0.046–0.053) with the high numbers of observers and cases available. CONCLUSIONS: This study documents fair interobserver agreement for the van Dijk osteoarthritis scale, and poor interobserver agreement for the Takakura and Kellgren osteoarthritis classification systems. Because of the low interobserver agreement for the van Dijk, Kellgren, and Takakura classification systems, those systems cannot be used for clinical decision-making. LEVEL OF EVIDENCE: Development of diagnostic criteria on basis of consecutive patients, Level II. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1007/s00167-015-3871-6) contains supplementary material, which is available to authorized users.