Validation of chief complaints, medical history, medications, and physician diagnoses structured with an integrated emergency department information system in Japan: the Next Stage ER system

AIM: Emergency department information systems (EDIS) facilitate free‐text data use for clinical research; however, no study has validated whether the Next Stage ER system (NSER), an EDIS used in Japan, accurately translates electronic medical records (EMRs) into structured data. METHODS: This is a retrospective cohort study using data from the emergency department (ED) of a tertiary care hospital from 2018 to 2019. We used EMRs of 500 random samples from 27,000 ED visits during the study period. Through the NSER system, chief complaints were translated into 231 chief complaint categories based on the Japan Triage and Acuity Scale. Medical history and physician’s diagnoses were encoded using the International Classification of Diseases, 10th Revision; medications were encoded as Anatomical Therapeutic Chemical Classification System codes. Two reviewers independently reviewed 20 items (e.g., presence of fever) for each study component (e.g., chief complaints). We calculated association measures of the structured data by the NSER system, using the chart review results as the gold standard. RESULTS: Sensitivities were very high (>90%) in 17 chief complaints. Positive predictive values were high for 14 chief complaints (≥80%). Negative predictive values were ≥96% for all chief complaints. For medical history and medications, most of the association measures were very high (>90%). For physicians’ ED diagnoses, sensitivities were very high (>93%) in 16 diagnoses; specificities and negative predictive values were very high (>97%). CONCLUSIONS: Chief complaints, medical history, medications, and physician’s ED diagnoses in EMRs were well‐translated into existing categories or coding by the NSER system.