Two-in-one system and behavior-specific brain synchrony during goal-free cooperative creation: an analytical approach combining automated behavioral classification and the event-related generalized linear model

SIGNIFICANCE: In hyperscanning studies of natural social interactions, behavioral coding is usually necessary to extract brain synchronizations specific to a particular behavior. The more natural the task is, the heavier the coding effort is. We propose an analytical approach to resolve this dilemma, providing insights and avenues for future work in interactive social neuroscience. AIM: The objective is to solve the laborious coding problem for naturalistic hyperscanning by proposing a convenient analytical approach and to uncover brain synchronization mechanisms related to human cooperative behavior when the ultimate goal is highly free and creative. APPROACH: This functional near-infrared spectroscopy hyperscanning study challenged a cooperative goal-free creative game in which dyads can communicate freely without time constraints and developed an analytical approach that combines automated behavior classification (computer vision) with a generalized linear model (GLM) in an event-related manner. Thirty-nine dyads participated in this study. RESULTS: Conventional wavelet-transformed coherence (WTC) analysis showed that joint play induced robust between-brain synchronization (BBS) among the hub-like superior and middle temporal regions and the frontopolar and dorsomedial/dorsolateral prefrontal cortex (PFC) in the right hemisphere, in contrast to sparse within-brain synchronization (WBS). Contrarily, similar regions within a single brain showed strong WBS with similar connection patterns during independent play. These findings indicate a two-in-one system for performing creative problem-solving tasks. Further, WTC-GLM analysis combined with computer vision successfully extracted BBS, which was specific to the events when one of the participants raised his/her face to the other. This brain-to-brain synchrony between the right dorsolateral PFC and the right temporo-parietal junction suggests joint functioning of these areas when mentalization is necessary under situations with restricted social signals. CONCLUSIONS: Our proposed analytical approach combining computer vision and WTC-GLM can be applied to extract inter-brain synchrony associated with social behaviors of interest.