Coding reliability and agreement of International Classification of Disease, 10(th) revision (ICD-10) codes in emergency department data

INTRODUCTION: Administrative health data from emergency departments play important roles in understanding health needs of the public and reasons for health care resource use. International Classification of Disease (ICD) diagnostic codes have been widely used to code reasons of clinical encounters for administrative purposes in emergency departments. OBJECTIVE: The purpose of the study is to examine the coding agreement and reliability of ICD diagnosis codes in emergency department records through auditing the routinely collected data. METHODS: We randomly sampled 1 percent of records (n=1636) between October and December 2013 from 11 emergency departments in Alberta, Canada. Auditors were employed to review the same chart and independently assign main diagnosis codes. We assessed coding agreement and reliability through comparison of codes assigned by auditors and hospital coders using proportion of agreement and Cohen’s kappa. Error analysis was conducted to review diagnosis codes with disagreement and categorized them into six groups. RESULTS: Overall, the agreement was 86.5% and 82.2% at 3 and 4 digits levels respectively, and reliability was 0.86 and 0.82 respectively. Variations of agreement and reliability were identified across different emergency departments. The major two categories of coding discrepancy were the use of different codes for same condition (23.6%) and the use of codes at different levels of specificity (20.9%). CONCLUSIONS: Diagnosis codes in emergency departments show high agreement and reliability, although there are variations of coding quality across different hospitals. Stricter coding guidelines regarding the use of unspecified codes are needed to enhance coding consistency.