Bimodal Data Fusion of Simultaneous Measurements of EEG and fNIRS during Lower Limb Movements

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have temporal and spatial characteristics that may complement each other and, therefore, pose an intriguing approach for brain-computer interaction (BCI). In this work, the relationship between the hemodynamic response and brain oscillation activity was investigated using the concurrent recording of fNIRS and EEG during ankle joint movements. Twenty subjects participated in this experiment. The EEG was recorded using 20 electrodes and hemodynamic responses were recorded using 32 optodes positioned over the motor cortex areas. The event-related desynchronization (ERD) feature was extracted from the EEG signal in the alpha band (8–11) Hz, and the concentration change of the oxy-hemoglobin (oxyHb) was evaluated from the hemodynamics response. During the motor execution of the ankle joint movements, a decrease in the alpha (8–11) Hz amplitude (desynchronization) was found to be correlated with an increase of the oxyHb (r = −0.64061, p < 0.00001) observed on the Cz electrode and the average of the fNIRS channels (ch28, ch25, ch32, ch35) close to the foot area representation. Then, the correlated channels in both modalities were used for ankle joint movement classification. The result demonstrates that the integrated modality based on the correlated channels provides a substantial enhancement in ankle joint classification accuracy of 93.01 ± 5.60% (p < 0.01) compared with single modality. These results highlight the potential of the bimodal fNIR–EEG approach for the development of future BCI for lower limb rehabilitation.