In fact we see a translation of 'user input actions' to 'events'. Clicking a 'cancel'-button with the mouse has exactly the same result as pressing the ESC-key. This suggest that it is useful to design an extendable set of events, independent of the used input device. Such as:
Input from other input devices (pen, speech, external knobs, joysticks,
etc.) can all be translated to such events, such that it makes no
difference for the application where the input comes from (see also the
Meta Device Driver, figure 3.3 
). This
is the first step of implementing multimodality. Some examples are:
In order for this translation to be correct, there is a lot of interaction between input device and application. Pressing a mouse button has different results, depending on the coordinates. Translation into actions is only possible if the 'mouse driver' knows which screen area is covered by the appropriate button. Similarly, it is very useful for a speech recognition system to know what words are expected in some appliciation. The schematic becomes as follows:
Figure 3.1
: A schematic model for generalized input devices
The arrows in this picture represent different flows of information. The applications perform some initialization (e.g. choice of pen resolution), or supply information to devices when their input is requested. If for instance an application cannot accept sound input, it is not necessary for the audio input system to supply any data. A similar argument holds for XY tablets. In general, the on and off switching of a peripheral sampling process may save CPU time.
A basic function of drivers is to translate data into a hardware-independent form. As an example, mouse-button clicks may be translated into keyboard key presses. Also, spoken words may be processed by a speech recognition system to simulate mouse-button or keyboard-key presses too. As an additional functionality, drivers might translate hardware coordinates (like tablet XY coordinates) into application coordinates (in this example the screen bitmap).
In practice, it is always necessary that there is two-way communication between the applications and the input devices, usually with some drivers in between. These drivers in principle can take any form and degree of software complexity, from simple pass-through interfaces to complex handwriting or speech recognition systems.
Note that the same schematic can be applied to general output devices (COM).
Figure 3.2
: A schematic model for generalized output devices
Esprit Project 8579/MIAMI (Schomaker et al., '95)