Next: Virtual Reality?
Up: Synchronization
Previous: Object synchronization
Apart of building the representation of the environment, senses play
important communication roles. For the communication purposes, raw
information signals are grouped into units which convey complex content.
These groupings can be created, retrieved and manipulated in an endless
way, reflecting unlimited complexity of information which can be created.
We can differentiate at least among several levels of information
complexity which the sensory processing system must be dealing with:
- Elementary signals
- These are signals which can not be broken into
simpler ones, as regards their spatial, temporal, or other specific
properties.
- Simple syntactical units
- This is a sequence of elementary signals
producing an elementary component which will be a base for complex
information representation.
- Semantic sequences
- A sequence of syntactical units enables
representation of complex information.
All these types of stimulations can appear as inputs to the visual and
acoustical systems. The question arises how integrated audio-visual
processing of them is organized. The information complexity can be paired
with the object synchronization properties to produce a picture of
audio-visual integration. The following rules can be formulated in this
respect:
- For synchronized audio-visual objects, the higher the complexity of
information conveyed, the more enhanced their integrated representation
becomes. This results in higher system performance in extracting the
information from bimodal perception than from single senses.
- For nonsynchronized objects, the processes involved are more
complicated. In the case of lack of temporal synchronization, the
building of an integrated representation is highly disturbed. However,
in the case of spatial desynchronization, the building of an integrated
representation is preferred and the conflicting data coming from single
senses are attenuated. Usually the visual system is dominant over the
acoustical system, that is its data has more weight for building of the
integrated representation.
Next: Virtual Reality?
Up: Synchronization
Previous: Object synchronization
Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995