INTER-RATER RELIABILITY OF NOVICE LINKERS USING AN INNOVATIVE SEQUENTIAL ITERATIVE LINKING METHOD TO LINK PROSTHETIC OUTCOMES TO THE INTERNATIONAL CLASSIFICATION OF FUNCTIONING, DISABILITY AND HEALTH

OBJECTIVE: When linking outcomes to the International Classification of Functioning, Disability and Health (ICF), inter-rater reliability is typically assessed at the conclusion of the linking process. This method does not allow for iterative evaluation and adaptations that would improve inter-rater reliability as novices gain experience. This pilot study aims to quantify the inter-rater reliability of novice linkers when using an innovative, sequential, iterative linking method to link prosthetic outcomes to the ICF. METHODS: Across 5 sequential rounds, 2 novices independently linked outcomes to the ICF. A consensus discussion followed each round that informed refinement of the customized ICF linking rules. The inter-rater reliability was calculated for each round using Gwet’s agreement coefficient (AC1). RESULTS: A total of 1,297 outcomes were linked across 5 rounds. At the end of round 1 inter-rater reliability was high (AC1 = 0.74, 95% confidence interval (95% CI) 0.68–0.80). At the end of round 3, inter-rater reliability (AC1 = 0.84, 95% CI 0.80–0.88) was significantly improved and marked the point of consistency where further improvements in inter-rater reliability were not statistically significant. CONCLUSION: A sequential iterative linking method provides a learning curve that allows novices to achieve high-levels of agreement through consensus discussion and iterative refinement of the customized ICF linking rules.