Reliability of the National Institutes of Health (NIH) Stroke Scale Between Emergency Room and Neurology Physicians for Initial Stroke Severity Scoring

Introduction: In patients with acute ischemic stroke (AIS), the National Institutes of Health Stroke Scale (NIHSS) is essential to establishing a patient’s initial stroke severity. While previous research has validated NIHSS scoring reliability between neurologists and other clinicians, it has not specifically evaluated NIHSS scoring reliability between emergency room (ER) and neurology physicians within the same clinical scenario and timeframe in a large cohort of patients. This study specifically addresses the key question: does an ER physician’s NIHSS score agree with the neurologist’s NIHSS score in the same patient at the same time in a real-world context? Methods: Data was retrospectively collected from 1,946 patients being evaluated for AIS at Houston Methodist Hospital from 05/2016 - 04/2018. Triage NIHSS scores assessed by both the ER and neurology providers within one hour of each other under the same clinical context were evaluated for comparison. Ultimately, 129 patients were included in the analysis. All providers in this study were NIHSS rater-certified. Results: The distribution of the NIHSS score differences (ER score - neurology score) had a mean of -0.46 and a standard deviation of 2.11. The score difference between provider teams ranged ±5 points. The intraclass correlation coefficient (ICC) for the NIHSS scores between the ER and neurology teams was 0.95 (95% CI, 0.93 - 0.97) with an F-test of 42.41 and a p-value of 4.43E-69. Overall reliability was excellent between the ER and neurology teams. Conclusion: We evaluated triage NIHSS scores performed by ER and neurology providers under matching time and treatment conditions and found excellent interrater reliability. The excellent score agreement has important implications for treatment decision-making during patient handoff and further in stroke modeling, prediction, and clinical trial registries where missing NIHSS scores may be equivalently substituted from either provider team.