Distinct mechanisms of temporal binding in generalized and cross-modal flash-lag effects

It remains unknown how the brain temporally binds sensory data across different modalities and attributes to create coherent perceptual experiences. To address this question, we measured what we see at the time we experience an event using a generalized version of the flash-lag effect (FLE) for combinations of visual attribute (bar orientation, face orientation, or face identity) and probe modality (visual or auditory). We asked participants to judge the content of rapidly and serially presented images seen at the same time as a briefly presented visual (flash) or auditory (click) probe and estimated the “time windows” contributing to decisions using reverse correlation analysis. We also used displays in which the visual attribute of a stimulus continuously changed and measured FLEs around abrupt flip in change direction and at the initiation and termination of a sequence. We consistently found clear latency-difference effects, which depended on visual attribute for the visual probe but did not for the auditory probe. The intra-modal FLE can be explained in terms of differential latency and temporal integration, but the cross-modal FLE is suggested to operate via a distinct mechanism; the content of a successive visual stream experienced after the awareness of a click is interpreted as simultaneous with the click.