A user evaluation of speech/phrase recognition software in critically ill patients: a DECIDE-AI feasibility study

OBJECTIVES: Evaluating effectiveness of speech/phrase recognition software in critically ill patients with speech impairments. DESIGN: Prospective study. SETTING: Tertiary hospital critical care unit in the northwest of England. PARTICIPANTS: 14 patients with tracheostomies, 3 female and 11 male. MAIN OUTCOME MEASURES: Evaluation of dynamic time warping (DTW) and deep neural networks (DNN) methods in a speech/phrase recognition application. Using speech/phrase recognition app for voice impaired (SRAVI), patients attempted mouthing various supported phrases with recordings evaluated by both DNN and DTW processing methods. Then, a trio of potential recognition phrases was displayed on the screen, ranked from first to third in order of likelihood. RESULTS: A total of 616 patient recordings were taken with 516 phrase identifiable recordings. The overall results revealed a total recognition accuracy across all three ranks of 86% using the DNN method. The rank 1 recognition accuracy of the DNN method was 75%. The DTW method had a total recognition accuracy of 74%, with a rank 1 accuracy of 48%. CONCLUSION: This feasibility evaluation of a novel speech/phrase recognition app using SRAVI demonstrated a good correlation between spoken phrases and app recognition. This suggests that speech/phrase recognition technology could be a therapeutic option to bridge the gap in communication in critically ill patients. WHAT IS ALREADY KNOWN ABOUT THIS TOPIC: Communication can be attempted using visual charts, eye gaze boards, alphabet boards, speech/phrase reading, gestures and speaking valves in critically ill patients with speech impairments. WHAT THIS STUDY ADDS: Deep neural networks and dynamic time warping methods can be used to analyse lip movements and identify intended phrases. HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE AND POLICY: Our study shows that speech/phrase recognition software has a role to play in bridging the communication gap in speech impairment. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13054-023-04420-x.