Next: Speech Recognition/Handwriting Synthesis:
Up: Handwriting-speech control
Previous: Handwriting Recognition/Speech Recognition:
[COM]
- HOC
- (a) hand/arm musculature: position and compliance control
(b) speech musculature, vocal chords, respiratory system: vocal
sound production
- CIM
- (a) XY digitizer, handwriting recognition algorithm
(b) Microphone, speech recognition algorithm
- COM
- Voice-like sound is being produced by the computer, representing
recognized text or the state of the recognition agent
- HIC
- The user gets:
(a) Immediate feedback on speech and handwriting by the intrinsic
feedback loop (see Figure 1.1 )
(b) Auditory feedback
At NICI, a number of experiments have been performed with an
experimentation platform called PenBlock running under Unix. In the case of
isolated characters (handprint), a spoken spelled letter can be produced,
after each character that is entered. If the character is not recognized
with sufficient likelihood, an ``uh?'' sound is produced. The meaning of this
is picked up quickly by the subjects, and makes more explicit the fact that
an active deciding 'agent' is trying to assess the incoming handwriting
products, instead of an infallible machine-like tool. However, the
usability of feedback in the form of spoken recognized letters must be
studied in more detail. A more neutral ``click'' sound, after entering each
character was considered more acceptable by a small number of informal
subjects. Variations on this theme can be designed, giving only speech
feedback in case of doubt by the recognition agent.
Other experiments have been performed on a PC, in a small project dubbed
``PenTalk'' together with Tulip Computers. In this project, the user wrote
isolated cursive words (Dutch) which were pronounced after on-line
recognition by a speech synthesizer program. Preliminary experiments
indicated that the use of speech feedback is limited, especially if the
delay is more than about half a second after writing the word. In later
versions of the program, speech synthesis synthesised speech feedback was
deferred to a drag-and-drop type function. The philosophy was that if it
could not be immediate, speech feedback was better placed under active user
control. The user decides when and what must be spoken. This has the
advantage that the speech synthesis algorithm can pronounce sentences and
paragraphs, effectively coding the prosody of speech, which is not possible
in isolated words. Based on these preliminary findings it is concluded that
speech as an immediate feedback in handwriting recognition is probably most
useful as a way of multimodally informing the user on the state of the
recognition process or the quality of the handwriting input. Auditory
feedback in any case has the advantage that the user does not have to look
away from the pen-tip, i.e., where the action takes place. Other forms of
(non-vocal) auditory feedback may be considered in .
Next: Speech Recognition/Handwriting Synthesis:
Up: Handwriting-speech control
Previous: Handwriting Recognition/Speech Recognition:
Esprit Project 8579/MIAMI (Schomaker et al., '95)
Thu May 18 16:00:17 MET DST 1995