next up previous contents
Next: Handwriting-visual control Up: Bi- and Multimodal Previous: Bi- and Multimodal

Visual-gestural control

Gestural control is a relevant aspect in multimedia systems. With the current state-of-the-art human movement tracking technology, it is possible to represent most of degrees of freedom of a (part of the) human body in real-time. This allows for a growing number of applications, from advanced human-computer interfaces in multimedia systems to new kinds of interactive multimedia systems (e.g., [175,247]).

The classification of gestural information patterns in human-machine interaction is an important research topic. A short overview of existing literature is given in Appendix D, mainly referred to robotics and cognitive/psychological studies. The recent research on virtual environments has given a strong acceleration to the research in this field --- see for example [247]. From another point of view, it is also important to analyze human activities which involve significant gesture communication. A significant field is related to the study of the languages developed over centuries for dance and music. This field is important also from the point of view of the study of the involved emotional communication [252,11]. The main areas regard

dance gesture languages and classifications of movement patterns [55,297,51,336,340];
languages for music conducting [232];
the study of the movement involved in music instruments performance [9]; and
the recent research efforts on ``Hyper-Instruments'' at MIT and in several other research centers [234,196,360] to simulate existing musical instruments and to find out new routes of communicating movement and emotional contents in art performance.

A complete survey on all these four aspects is [234].

Different taxonomies of gesture can be defined with respect to different aspects of gesture communication. The following are some main examples, explored in WP 2.3:

  1. Types of movement detected:
  2. Types of sensoring devices:
  3. Semantics:

    The important aspect is here the meaning associated to different gestures. The use of metaphors and analogies with other modalities is a fundamental aspect to devise evoking associations to gestures.

next up previous contents
Next: Handwriting-visual control Up: Bi- and Multimodal Previous: Bi- and Multimodal

Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995