Inter-Rater Reliability of Historical Data Collected by Non-Medical Research Assistants and Physicians in Patients with Acute Abdominal Pain

OBJECTIVES: In many academic emergency departments (ED), physicians are asked to record clinical data for research that may be time consuming and distracting from patient care. We hypothesized that non-medical research assistants (RAs) could obtain historical information from patients with acute abdominal pain as accurately as physicians. METHODS: Prospective comparative study conducted in an academic ED of 29 RAs to 32 resident physicians (RPs) to assess inter-rater reliability in obtaining historical information in abdominal pain patients. Historical features were independently recorded on standardized data forms by a RA and RP blinded to each others’ answers. Discrepancies were resolved by a third person (RA) who asked the patient to state the correct answer on a third questionnaire, constituting the “criterion standard.” Inter-rater reliability was assessed using kappa statistics (κ) and percent crude agreement (CrA). RESULTS: Sixty-five patients were enrolled (mean age 43). Of 43 historical variables assessed, the median agreement was moderate (κ 0.59 [Interquartile range 0.37–0.69]; CrA 85.9%) and varied across data categories: initial pain location (κ 0.61 [0.59–0.73]; CrA 87.7%), current pain location (κ 0.60 [0.47–0.67]; CrA 82.8%), past medical history (κ 0.60 [0.48–0.74]; CrA 93.8%), associated symptoms (κ 0.38 [0.37–0.74]; CrA 87.7%), and aggravating/alleviating factors (κ 0.09 [−0.01–0.21]; CrA 61.5%). When there was disagreement between the RP and the RA, the RA more often agreed with the criterion standard (64% [55–71%]) than the RP (36% [29–45%]). CONCLUSION: Non-medical research assistants who focus on clinical research are often more accurate than physicians, who may be distracted by patient care responsibilities, at obtaining historical information from ED patients with abdominal pain.