Kubovy Perception Lab


What if the sound and video do not go together in time? How far could one "offset" the sound from the moment of impact, either before or after, until the video no longer had any influence on the perceived duration of the sound? We designed this study to answer these questions.

We hypothesized that making the sound come before the impact would destroy the effect, whereas making the sound come after the impact would matter less (at reasonably small offsets). Why? Light travels much faster than sound, by many orders of magnitude. Thus, sometimes we see a visual event happen before we hear the sound it makes. Think of a thunderstorm where you see a flash of lightning and then hear the thunder come later. You can actually estimate the distance to the strike by how many seconds later the sound arrives! Sound, however, never travels faster than light in our day to day experience. Thus, it would be physically possible to see a strike and hear the sound later (if it happened far away), but there is no way a strike could create a sound before the impact occurred!

Our results confirmed our hypothesis. As expected, the video has the largest effect on an associated note when the sound and the event go together in time. This makes sense-- in daily life, sounds and the events that cause them usually come very near one another in time. When the audio lagged we still see an effect, although it is diminished. When the audio leads, however, the effect is no longer statistically significant.


Back to Sensory Integraton