VRR: What do you think is the greatest obstacle facing the VR industry and why?In real life, humans use machines, e.g. a car or a machine tool, and interact with them in a multimodal way, i.e. by means of perception (vision/hearing/haptic perception), and action (knob/wheel/lever/button control). In the case of virtual reality, the purpose is simply to substitute the sensory-motor flow elicited by a physical ambient environment with a computer generated sensory-motor flow which imitates/emulates (a) physical reality to a sufficient degree. Flight simulators are a good example. The purpose is not to improve or optimize man-machine interaction but to reproduce the standard one, with the goal of training the user or simply entertain him. Imitation/emulation is a possible paradigm of man-machine interaction, but it is not the only one. Due to the decoupling of the sensory-motor flow from the physical world, new environments can be generated based on environments which are physical in itself, but of a completely different scale (molecular surface landscapes vs galactic environments based on astronomical data), and which would normally never elicit an ambient sensory-motor flow.
Latta: Lack of basic research. The issues of having the most intimate form of human computer interface in Virtual Reality necessitate a thorough understanding of human perceptual, muscle and psychological systems. Yet that research foundation does not exist.
Dr John Latta interviewed in Virtual Reality Report, 2 (7) p 4.
Taken from: [195, page 43,]
More generally, one can define paradigms in which multimodality is used for improving the interaction between the user and reality via a computer:
Figure 4.9
: The computer as a mediator (or agent,
see 5.2
In [123], the following applications of VR are described in detail:
Ellis (in [89]) and Null and Jenkins(in [247]) from NASA labs on Virtual Reality (VR) argue that Virtual Environments (they use this name instead of VR), presented via head-mounted computer-driven displays, provide a new medium for man-machine interaction. Like other media, they have both physical and abstract components. Paper, for example, as a medium for communication, is itself one possible physical embodiment of the abstraction of a two-dimensional surface onto which marks may be made. The corresponding abstraction for head-coupled, virtual image, stereoscopic displays that synthesize a coordinated sensory experience is an environment. It is important to collocate VR (or VEs, following NASA) with respect to man-machine interaction.
In summary, we can understand multimodal interaction as a process characterized as follows: (i) the computer is able to capture the largest possible part of the human motor outflow, (ii) human movements are as little constrained as possible, (iii) the human receives from the computer a perceptual inflow which maximally utilizes the different available channels, (iv) the inflow and the outflow are optimally tuned, in relation with the specific task.
Esprit Project 8579/MIAMI (Schomaker et al., '95)