next up previous contents
Next: The specific bimodality Up: The bimodality of Previous: The bimodality of

The synergetic bimodality of speech

Setting communications parameters at threshold level, Risberg and Lubker observed that when a speaker appeared on a video display, but with the sound turned off, subjects relying on speech-reading correctly perceived 1% of test words [290]. When the subjects could not see the display, but were presented with a low-pass filtered version of the speech sound, they got 6% correct. Presented with the combined information channels, the performance jumped to 45% correctly perceived test words. This observation exemplifies the remarkable synergy of the two modes of speech perception. However, little is known about the process that integrates the cues across modalities, although a variety of approaches and of models to multimodal integration of speech perception have been proposed [208,332,40] and tested [211,213,128,210,291]. Our understanding of this process is still relatively crude, but its study is very active and controversial at present (see the 21 remarks to and in [209]!). For an extensive presentation of the various integration models proposed and tested in literature, see the ICP-MIAMI 94-2 Report by Robert-Ribes.

Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995