Verification of a clinical decision support system for the diagnosis of headache disorders based on patient–computer interactions: a multi-center study

BACKGROUND: Although headache disorders are common, the current diagnostic approach is unsatisfactory. Previously, we designed a guideline-based clinical decision support system (CDSS 1.0) for diagnosing headache disorders. However, the system requires doctors to enter electronic information, which may limit widespread use. METHODS: In this study, we developed the updated CDSS 2.0, which handles clinical information acquisition via human–computer conversations conducted on personal mobile devices in an outpatient setting. We tested CDSS 2.0 at headache clinics in 16 hospitals in 14 provinces of China. RESULTS: Of the 653 patients recruited, 18.68% (122/652) were suspected by specialists to have secondary headaches. According to “red-flag” responses, all these participants were warned of potential secondary risks by CDSS 2.0. For the remaining 531 patients, we compared the diagnostic accuracy of assessments made using only electronic data firstly. In Comparison A, the system correctly recognized 115/129 (89.15%) cases of migraine without aura (MO), 32/32 (100%) cases of migraine with aura (MA), 10/10 (100%) cases of chronic migraine (CM), 77/95 (81.05%) cases of probable migraine (PM), 11/11 (100%) cases of infrequent episodic tension-type headache (iETTH), 36/45 (80.00%) cases of frequent episodic tension-type headache (fETTH), 23/25 (92.00%) cases of chronic tension-type headache (CTTH), 53/60 (88.33%) cases of probable tension-type headache (PTTH), 8/9 (88.89%) cases of cluster headache (CH), 5/5 (100%) cases of new daily persistent headache (NDPH), and 28/29 (96.55%) cases of medication overuse headache (MOH). In Comparison B, after combining outpatient medical records, the correct recognition rates of MO (76.03%), MA (96.15%), CM (90%), PM (75.29%), iETTH (88.89%), fETTH (72.73%), CTTH (95.65%), PTTH (79.66%), CH (77.78%), NDPH (80%), and MOH (84.85%) were still satisfactory. A patient satisfaction survey indicated that the conversational questionnaire was very well accepted, with high levels of satisfaction reported by 852 patients. CONCLUSIONS: The CDSS 2.0 achieved high diagnostic accuracy for most primary and some secondary headaches. Human–computer conversation data were well integrated into the diagnostic process, and the system was well accepted by patients. The follow-up process and doctor–client interactions will be future areas of research for the development of CDSS for headaches. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s10194-023-01586-1.