next up previous contents
Next: Event Handling Architectures Up: Computer Input Modalities Previous: Other input devices

Generalized input devices

In fact we see a translation of 'user input actions' to 'events'. Clicking a 'cancel'-button with the mouse has exactly the same result as pressing the ESC-key. This suggest that it is useful to design an extendable set of events, independent of the used input device. Such as:

Input from other input devices (pen, speech, external knobs, joysticks, etc.) can all be translated to such events, such that it makes no difference for the application where the input comes from (see also the Meta Device Driver, figure 3.3 , on page 3.3 gif ). This is the first step of implementing multimodality. Some examples are:

In order for this translation to be correct, there is a lot of interaction between input device and application. Pressing a mouse button has different results, depending on the coordinates. Translation into actions is only possible if the 'mouse driver' knows which screen area is covered by the appropriate button. Similarly, it is very useful for a speech recognition system to know what words are expected in some appliciation. The schematic becomes as follows:

Figure 3.1 : A schematic model for generalized input devices

The arrows in this picture represent different flows of information. The applications perform some initialization (e.g. choice of pen resolution), or supply information to devices when their input is requested. If for instance an application cannot accept sound input, it is not necessary for the audio input system to supply any data. A similar argument holds for XY tablets. In general, the on and off switching of a peripheral sampling process may save CPU time.

A basic function of drivers is to translate data into a hardware-independent form. As an example, mouse-button clicks may be translated into keyboard key presses. Also, spoken words may be processed by a speech recognition system to simulate mouse-button or keyboard-key presses too. As an additional functionality, drivers might translate hardware coordinates (like tablet XY coordinates) into application coordinates (in this example the screen bitmap).

In practice, it is always necessary that there is two-way communication between the applications and the input devices, usually with some drivers in between. These drivers in principle can take any form and degree of software complexity, from simple pass-through interfaces to complex handwriting or speech recognition systems.

Note that the same schematic can be applied to general output devices (COM).

Figure 3.2 : A schematic model for generalized output devices

next up previous contents
Next: Event Handling Architectures Up: Computer Input Modalities Previous: Other input devices

Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995