Multiple-Time-Scale Analysis of Attention as Revealed by EEG, NIRS, and Pupil Diameter Signals During a Free Recall Task: A Multimodal Measurement Approach

Attention plays a fundamental role in acquiring and understanding information. Therefore, it is useful to evaluate attention objectively in such fields as education and mental health. Aimed at extracting objective indicators of attention from physiological signals, this study examined the characteristics of electroencephalography (EEG), near-infrared spectroscopy (NIRS), and pupil diameter signals during a free recall task. The objective was to clarify the temporal characteristics of these signals in relation to attention. We used a free recall task as a cognitive task with an attentional load. The participants attempted to memorize and then recall 13 serially presented words. Our hypothesis was that the significant physiological responses should differ depending on the time scale of the attention evaluation. The physiological responses were compared on the basis of differences between success and failure to recall a word on a short time scale, in terms of the attentional state among five serial position groups on a middle time scale, and on the basis of differences between trials with many and few words recalled on a long time scale. We found that the response of each physiological signal depended on the attention in the different time-scale comparisons. (1) The P300 amplitudes of the EEG signals for the words that were recalled were significantly higher than those for the words that were not recalled. (2) Pupillary dilation differed significantly depending on the serial position group. (3) Functional connectivity in the right hemisphere revealed by NIRS was significantly stronger in trials with many words recalled than in those with few words recalled. Different temporal characteristics of physiological signals with respect to attention were suggested by multimodal measurement and multiple-time-scale analysis. Consideration of these characteristics should help in the development of applications requiring objective attention evaluation.