A design space for multimodal systems

Following [243], concurrency of processing and the fusion of input/output data are the two main features of a multimodal system. Consequently, Nigay and Coutaz took these features as the basis for their design space and classification method.

The design space has been defined along three dimensions: Levels of abstraction, Use of modalities, and Fusion. In order to keep the model as simple as possible, the permitted values of each dimension have been limited. The result is shown in figure 4.1 . From a maximum of eight combinations (3 dimensions, 2 values each: ), four have been named and are further investigated in the article.

  
Figure 4.1 : The multi-feature system design space (Taken from [243])

The design space is then used for a classification scheme for multimodal systems. By introducing weighted features of a system, the location of it within the design space can easily be determined. This is useful in order to test the effect of different features and commands, thus ``measuring'' the usability of an interface.

The second part of the paper deals with data fusion and concurrent processing. Therefore, a multi agent architecture is introduced which will not be further reviewed here. For data fusion, three levels of fusion have been identified:

Lexical fusion:

Binding of hardware primitives to software events; only temporal issues (e.g., data synchronization) are involved

Syntactic fusion:

Sequencing of events; combination of data to obtain a complete command

Semantic fusion:

Functional combination of commands in order to generate new, more complex commands

• the introduction of four salient classes of systems which can be used as the extrema of a reference space, and
• the chance to characterize and reason about the I/O properties of interactive systems by using their reference space.

Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995