One of the most interesting research areas in robotics is the development of fully autonomous systems. In order to achieve autonomy, the mobile systems have to be equipped with sophisticated sensory systems, massive computing power, and what might be called an ``intelligent controller''. Because this is not possible with today's technology, many researchers try to incorporate the human operator in the sense and control loop, thus replacing parts of the system where appropriate. For instance, the recognition and decision capabilities of humans are much better than that of a mobile robot at the present time. Although a mobile system might be equipped with a vision system, the image processing and object recognition still takes to much time to drive autonomously with reasonable speed. On the other hand, the sensory systems of the robot which act locally can perceive and process data which is usually not detectable by humans, like the reflection of ultrasound or gas molecules which might be ``smelt'' by a special detector. Thus, a teleoperation system where parts of the controller will run autonomously whereas others will need the operator's interaction seems to be a very promising approach.
Figure 6.1
: A sketch and a real picture of the mobile robot system PRIAMOS
The control of a mobile vehicle (or robot) with advanced multimodal methods
and sophisticated I/O devices is both, very attractive and useful. Take,
e.g., the mobile platform PRIAMOS (shown in
figure 6.1
Figure 6.2
: The controller configuration of PRIAMOS (taken
from [188])
The control concept of PRIAMOS which is shown in
figure 6.2 
to the real
execution.
The software and hardware architecture of PRIAMOS is suitable to be extended with multimodal functionalities. One of several possible system configurations will be described in the following scenario (which is only fictitious at the moment):
``The mobile robot's platform operates in either one of two different modes. In Autonomous Mode, the robot uses an internal map and its sensory system to navigate without any help from the operator who only supervises the movements. In the Supervisory Mode, the operator remotely controls the robot. For this task he uses a joystick which not only controls all degrees of freedom (x- and y-translation, z-rotation) but also reflects several characteristics of the robot's environment.The scenario described above has not been realized yet, but it is based exclusively on techniques which are either already existing or under current development. The most difficult problem will be the complexity of this system, which also complicates the integration of all the separate system components. We are convinced that the techniques and methods which are going to be developed in will contribute significantly to this integration process.Therefore, the joystick has been equipped with force feedback capabilities. Whenever the robot approaches an obstacle which is sensed by the ultrasonic sensors, the operator is able to feel it when the distance falls short of a certain threshold. Thus, collisions can be avoided without charging the operator's ``vision system'' with additional information.
This is especially necessary because the operator is already burdened with the images of the active stereo vision system KASTOR. Both cameras send their images independently to one of the two displays in the Head-Mounted Display (HMD) that the operator is wearing. Thus, he gets a full 3D impression of the remote world. The camera head can also operate in autonomous or supervisory mode. In the latter, the operator uses his second hand to move the cameras in the desired direction. By fully controlling all 18 degrees of freedom with a kind of master-slave manipulator or a 6D mouse, the operator is able to focus any object in the view of the cameras.
When using the helmet, the operator is no longer able to use a keyboard for additional command input. This is no drawback, because all commands can simply be spoken as the control system is equipped with a natural language processing (NLP) unit. Although only keywords and values can be recognized by it, it is comfortable and easy to use because it is a speaker independent system which doesn't have to be trained. Therefore, simple commands like ``stop'' or ``faster'' can be executed in a very intuitive way.
Instead of the force feedback described above, another way to inform the operator about nearby obstacles is to convert the signals of the ultrasonic sensors into acoustical information. For instance, the reflection of a wall detected by the sensors can be artificially created by a sound generator, thus giving the operator the impression of ``walking'' through the remote environment. Of course, alarm signals are transmitted to him by acoustical stimulus, too.''
\
Esprit Project 8579/MIAMI (Schomaker et al., '95)