Characterizing eye-gaze positions of people with severe motor dysfunction: Novel scoring metrics using eye-tracking and video analysis

Nonverbal communication with people who have physical disabilities is difficult. Eye-tracking technologies have recently been developed and applied to help people with physical disabilities in their communication. However, the eye-gaze patterns of people with severe motor dysfunction (SMD) have not been analyzed in detail. To clarify characterization of people with SMD, we aimed to develop gaze position-based evaluation metrics and analyze detailed eye-gaze patterns of people with SMD. We developed two new scoring metrics: (1) saliency score based on three saliency maps—spectral residual (SR), fine grained (FG), and motion (Mo); and (2) the distance score, which represents to what extent people can chase an object in a video. The evaluation was performed on 102 participants, consisting of 35 subjects with profound intellectual and multiple disabilities (PIMD; SMD with IQ < 20), 19 with severe physical disabilities (SPD; SMD with IQ ≥ 20), and 48 healthy individuals. We observed that two saliency scores (SR and FG) and the distance score showed significant differences between the PIMD/SPD and healthy groups for the entire video, whereas Mo scores did not. Moreover, the distance score was analyzed separately for each scene, where scenes were categorized into three patterns—running, explanation, and hiding—according to the behavior of the moving objects. In the SPD and healthy groups, the explanation scenes accounted for the highest percentage of all scenes with the best distance score (63.6% and 61.9%, respectively), whereas in the PIMD group, the running scenes accounted for the highest percentage (54.5%). In conclusion, the new metrics were successful in quantitatively assessing the gaze responsiveness of people with SMD, which could not be assessed using a conventional metric, gaze-acquisition time. This study is expected to expand the possibilities of nonverbal communication using eye-tracking devices for people with SMD.