Speech perception involves fusion of multiple sensory inputs, but fusion is not automatic, likely depending on several external and internal factors (e.g., attention, noise, age). In this study, we exploited a specific paradigm in which a short audiovisual context made of coherent or incoherent speech material is displayed before an incongruent…

Visual and auditory information jointly contribute to face categorization processes in humans, and gender is a socially relevant multisensory category specified by faces and voices that is detected early in infancy. We used an eye tracker to study how gender coherence in audio and visual modalities influence face scanning in 9- to 12-month-old…

Seeing articulatory gestures enhances speech perception. Perception of auditory speech can even be changed by incongruent visual gestures, which is known as the McGurk effect (e.g., dubbing a voice saying /mi/ onto a face articulating /ni/, observers often hear /ni/). In children, the McGurk effect is weaker than in adults, but no previous…

Multimodal approaches have been shown to be effective for many learning tasks. In this study, we compared the effectiveness of five multimodal methods for second language (L2) Mandarin tone perception training: three single-cue methods (number, pitch contour, color) and two dual-cue methods (color and number, color and pitch contour). A total of…