Multimodal Perception – The McGurk Effect

wnu

First play this video with your eyes open:

What do you hear?  Probably “da da” repeated a few times.

Now replay the video, but this time close your eyes and simply listen to the audio.  It’s likely that you now hear “ba ba” repeated.

For the coup de grâce, play the video a third time but with the audio muted.  You’ll see that the man is actually mouthing “ga ga”.

The video takes an image of a man uttering “ga ga” and overlays audio of him pronouncing “ba ba”, and somehow our brains mesh these two to come up with “da da”.  It’s a phenomenon known as multimodal integration, or the intermingling of senses by the brain in interpreting language.  Sight and sound do not typically occur in isolation from one another, nor did they evolve independently.  The brain makes use of both simultaneously to provide the “best” understanding possible.

Unconvinced?  Watch this video for a thorough demonstration.

June 2, 2009 1:01 am

Older Conceptions »
::the open end:: Copyright © 2016 All Rights Reserved.