# L. Schomaker c.s., Abstracts

Schomaker, L.R.B., & Thomassen, A.J.W.M. (1986). On the use and limitations of averaging handwriting signals. In H.S.R. Kao, G.P. van Galen, & R. Hoosain (Eds.), Graphonomics: Contemporary research in handwriting (pp. 225-238). Amsterdam: North-Holland.

The averaging of handwriting signals is subject to a number of special restrictions that in general do not apply to other signals such as electrophysiological recordings. Specific problems are presented by the choice of the entities to be averaged, by the choice of time-reference points and by duration variability. Knowledge of the signal production mechanism and of temporal and spatial characteristics of the handwriting signal is needed to solve these questions. It is noted that the signal is deterministic at the local level, which justifies the use of averaging techniques. The problem of stroke duration variability is dealt with by applying time-axis normalization prior to averaging. Examples of averaging at the stroke, letter and word level are presented. Results indicate that at up to four letters can be averaged without noticeable distortion in the spatial domain. Eventually, however, variability in stroke duration will dictate the choice of time-reference points for the time-axis normalization in multi-letter handwriting segments. This will 'warp' the original time path. If care is taken in the selection of time references, time-axis normalization and averaging can be useful in movement analysis, pattern matching and simulation of handwriting.

Thomassen, A.J.W.M., & Schomaker, L.R.B. (1986). Between-letter context effects in handwriting trajectories. In H.S.R. Kao, G.P. van Galen, & R. Hoosain (Eds.), Graphonomics: Contemporary research in handwriting (pp. 253-272). Amsterdam: North-Holland.

In fluent, cursive handwriting, context effects may appear at various levels. First, at the highest, central level, specific allographs (defined as alternative shapes for the same character, or grapheme) may be selected from motor memory when they are to be produced in the neighbourhood of specific other allographs, or following a space. Second, at an intermediate level, there may be mutual influence amongst neighbouring character representations during their temporary storage immediately preceding their execution. Third, the dynamics prevailing during the actual performance of a writing task may further influence the execution of the selected allographs. The present study investigates context effects of the two latter kinds. Specific hypotheses regarding the modification of spatial features of strokes as a function of adjacent strokes are tested both within letters and between letters in cursively written words. Testing is done by using a newly developed averaging technique. The results are interpreted in terms of performance effects similar to those coarticulation in speech.

 1987

Teulings, H.-L., Schomaker, L.R.B., Morasso, P., & Thomassen, A.J.W.M. (1987). Handwriting-analysis system. In R. Plamondon, C.Y. Suen, J.-G. Desche^nes, & G. Poulin (Eds.), Proceedings of the Third International Symposium on Handwriting and Computer Applications (pp. 181-183). Montreal: Ecole Polytechnique. ISBN 2-553-00197-5.

Teulings, H.-L., Thomassen, A.J.W.M., Schomaker, L.R.B., & Morasso, P. (1987). Experimental protocol for cursive script acquisition: The use of motor information for the automatic recognition of cursive script. Report 3.1.2., ESPRIT Project 419.

 1988

Maarse, F.J., Schomaker, L.R.B., & Teulings, H.-L. (1988). Automatic identification of writers. In G.C. van der Veer & G. Mulder (Eds.), Human-Computer Interaction: Psychonomic Aspects (pp. 353-360). New York: Springer.

A major problem in the automatic recognition of handwriting, as in speech recognition, appear to be the inter-individual differences. Although the use of feature extraction and abstract coding can alleviate the problem to some extent, the range of idiosyncratic handwriting properties is so wide that an early-stage narrowing of the domain of possible recognition solutions is necessary. This can be done by extracting global features from the handwriting and automatic identification of the person or class of hand writers before the actual recognition stage starts. In this paper, a list of useful handwriting parameters for person identification is presented. It is shown in what way person identification can be added to a handwriting recognition system. As an illustration, a method is described in which persons where identified on the basis of global characteristics of a single line of handwriting.

Teulings, H.L., Schomaker, L.R.B., & Maarse, F.J. (1988). Automatic handwriting recognition and the keyboard-less personal computer. In F.J. Maarse, L.J.M. Mulder, W.P.B. Sjouw, & A.E. Akkerman (Eds.), Computers in psychology: Methods, instrumentation, and psychodiagnostics (pp. 62-66). Amsterdam: Swets & Zeitlinger.

Thomassen, A.J.W.M, Teulings, H.-L., Schomaker, L.B.R., Morasso, P., & Kennedy, J. (1988) Towards the implementation of cursive-script understanding in an on-line handwriting-recognition system. In D.G. XIII (Ed.), ESPRIT '88: Putting the technology to use (pp. 628-639). Amsterdam: North-Holland.

As a preliminary to the prototype of an on-line handwriting-recognition system, a flexible series of low-level analysis procedures has been developed for the processing of totally unconstrained cursive script. The present paper describes these procedures in a non-technical fashion. A distinction is made between (1) continuous procedures yielding estimates of the global properties of the handwriting signal and (2) discrete procedures which deal with appropriately parse segments of handwriting. The latter procedures are based in part on knowledge of the human motor system which is responsible for the generation of natural cursive script. Low-level analysis is achieved by a set of independent modules in a parallel control structure and acting in an iterative mode. Both bottom-up and top-down information streams are incorporated in this approach.

Thomassen, A.J.W.M., Teulings, H.-L., & Schomaker, L.R.B. (1988). Real-time processing of cursive writing and sketched graphics. In G.C. van der Veer & G. Mulder (Eds.), Human-Computer Interaction: Psychonomic Aspects (pp. 334-352). New York: Springer.

Thomassen, A.J.W.M., Teulings, H.L., Schomaker, L.R.B., Morasso, P., & Kennedy, J. (1988). Towards the implementation of cursive-script understanding in an on-line handwriting-recognition system. In Commission of the European Communities: D.G. XIII (Ed.), ESPRIT '88: Putting the technology to use. Part 1 (pp. 628-639). Amsterdam: North-Holland.

 1989

Schomaker, L.R.B., Thomassen, A.J.W.M., & Teulings, H.-L. (1989). A computational model of cursive handwriting. In R. Plamondon, C.Y. Suen, & M.L. Simner (Eds.), Computer Recognition and Human Production of Handwriting (pp. 153-177). Singapore: World Scientific.

This paper presents a computational model for the production of handwriting, starting with allograph codes as input and ending with a target pen-tip trajectory as output. In the model, a distinction is made between a symbolic level of processing and a quantitative level of processing. At the symbolic level, a grammar for the connection of cursive allographs determines abstract codes for connecting strokes. At the quantitative level, a translation of symbols into a sequence of parametrized strokes takes place. A parsimonious stroke parametrization in the velocity domain is used, that is based on planning in Cartesian space and allocation of time to movement components along the spatial axes. With the basic model settings used, simulation results already show a satisfactory correspondence with original handwriting samples.

 1990

Schomaker, L.R.B. & Plamondon, R. (1990). The Relation between Pen Force and Pen-Point Kinematics in Handwriting. Biological Cybernetics, 63, 277-289.

This study investigates the spectral coherence and time-domain correlation between pen pressure (axial pen force, APF) and several kinematic variables in drawing simple patterns and in writing cursive script. Two types of theories are prevalent: "biomechanical" and "central" explanations for the force variations during writing. Findings show that overall coherence is low $(<0.5)$ and decreases with pattern complexity, attaining its lowest value in cursive script. Looking at subjects separately, it is found that only in a small minority of writers "biomechanical coupling" between force and displacement takes place in cursive handwriting, as indicated by moderate to high negative overall correlations. The majority of subjects displays low coherence and correlation between kinematics and APF. However, APF patterns in cursive script reveal a moderate to high replicatability, giving support to the notion of a "centrally" controlled pen pressure. The sign of the weak residual average correlation between APF and finger displacement, and between APF and wrist displacement is negative. This indicates that small biomechanical effects may be present, a relatively higher APF corresponding to finger flexion and wrist radial abduction. On the whole, however, variance in APF cannot be explained by kinematic variables. A motor task demanding mechanical impedance control, such as handwriting, apparently introduces a complexity that is not easily explained in terms of a passive mass-spring model of skeleto-muscular movement.

The human reader of handwriting is unaware of the amount of back-ground knowledge that is constantly being used by a massive parallel computer, his brain, to decipher cursive script. Artificial cursive script recognizers do not have access to a comparable source of knowledge or of comparable computational power to perform top-down processing. Therefore, in an artificial script recognizer, there is a strong demand for reliable bottom-up processing. For the recognition of unrestricted script consisting of arbitrary character sequences, on-line recorded handwriting signals offer a more solid basis than the optically obtained grey-scale image of a written pen trace, because of the temporal information and the inherent vectorial description of shape. The enhanced bottom-up processing is based on implementing knowledge of the motor system in the handwriting recognition system. The bottom-up information will already be sufficient to recognize clearly written and unambiguous input. However, ambiguous shape sequences, such as $m$ vs $n..$ or $d$ vs $cl$, and sloppy stroke patterns still require top-down processing. The present paper discusses the handwriting recognition system as being developed at the NICI. The system contains six major modules: (1) On-line digitizing, pre-processing of the movements and segmentation into strokes. (2) Normalization of global handwriting parameters. (3) Extraction of motorically invariant, real-valued, feature values per stroke to form a multi-dimensional feature vector and subsequent feature vector quantization by a self-organizing two-dimensional Kohonen network. (4) Allograph construction, using a second network of transition probabilities between cell activation patterns of the Kohonen network. (5) Optional word hypothesization. (6) The system has to be trained by supervised learning, the user indicating prototypical stroke sequences and their symbolic interpretation (letter or N-gram naming).

In the automatic recognition of unrestricted handwriting the ambiguities can be solved by top-down processing. However, automatic systems never have access to the extended background knowledge available to human readers. In order to replace this higher-level information we need to improve the reliability of the bottom-up processing. A handwriting-recognition system can be split up into six discrete blocks: (1) digitizing, word segmentation, pre-processing, and segmentation into strokes, (2) normalization of global handwriting parameters, (3) extraction of features per stroke, (4) allograph recognition, (5) optional word hypothesization, and, in order to allow recognition (6) a learning phase. The present paper discusses the design of three of these processing blocks: normalization, allograph recognition, and learning and briefly specifies feature extraction. Normalization concerns orientation, size, and slant. However, various alternative algorithms can be chosen and some algorithms yield more reliable results than others. A mechanism is proposed that will, sooner or later, find the most appropriate normalization algorithms. Consequently, the features extracted from each stroke in the handwriting pattern will be more uniform within a writer and even between writers. In the recognition phase, handwriting patterns are segmented into allographs using an algorithm that can handle allographs with various numbers of strokes and with optional connection strokes between them. In order to teach the recognizer the allographs a method has been designed that builds non-interactively a lexicon of allographs by automatically discovering the allographs in a large corpus of cursive script.

 1991

Schomaker, L.R.B. (1991). Simulation and Recognition of Handwriting Movements, Doctoral Dissertation (NICI TR-91-03). Nijmegen University, The Netherlands.

This study concerns the processes that take place from the moment that a writer wants to write down a given word, until one can inspect the finished result. The approach followed is based on the assumption that new insights can be gained by trying to build a working generative computer model of handwriting. Chapter 1 deals with the theoretical aspects of modeling processes of motor control. Chapter 2 discusses pen-tip kinematics during cursive writing: How reproducible are replications of writing movements recorded on different occasions? Only if movements are actually reproducible, it makes sense to develop a handwriting production model. Chapter 3 presents a symbolic generative model of handwriting. The problem that has to be solved concerns the transformation of discrete entities, i.e., the symbolic representation of a planned letter shape (allograph), into a continuous multi-dimensional time function: Pen-tip movement. This problem is tackled with the assumption that velocity-based strokes are the basic segments in handwriting. In the model, a parsimonious parametrization of the strokes is used, which is based on transforming a shape factor into differential timing. Consequently, a grammar providing rules for generating connecting strokes between two planned letters is proposed. In Chapter 4, a kinetic aspect of writing is studied: What happens to axial pen force during the production of several types of movement patterns. It appears that pen-force fluctuations are not a passive biomechanical phenomenon. Pen-force control and compliance appear to be embedded in the "motor programs" for letter production, in an idiosyncratic, writer-dependent fashion. In Chapter 5, a change of perspective takes place. It is noted that there are some limitations inherent to a symbolical modeling approach, especially with respect to low-level processes in handwriting control. A review of basic artificial neural-network models is presented and their potential use both in modeling handwriting movement control and in handwriting recognition is assessed. In the chapters 6-8, three basic issues are raised with respect to motor modeling: The coding of quantity, the representation of time, and the representation of the effector system by neurally inspired models. A final interesting and relevant problem is computer recognition of handwriting movements which is the focus of Chapter 9. An algorithm is proposed which performs recognition by actively constructing letter (allograph) hypotheses on the basis of chains of individual strokes, instead of storing prototypical allographs and performing template matching.

Schomaker, L.R.B., (1991). Stroke- versus Character-based Recognition of On-line, Connected Cursive Script. In G.E. Stelmach (Ed.), Proceedings of the Fifth Handwriting Conference of the IGS: Motor Control of Handwriting (pp. 213-217). Tempe AZ: Arizona State University.

 1992

Teulings, H.-L., & Schomaker, L.R.B. (1992). Un-supervised learning of prototype allographs in cursive script recognition using invariant handwriting features. In J.-C. Simon & S. Impedovo (Ed.), From Pixels to Features III (pp. 61-73), Amsterdam: North-Holland.

An on-line cursive-script recognizer must first obtain knowledge about the letters to be distinguished. This knowledge may be based upon a common cursive-script style or it may be extracted from a large corpus of real handwriting. The latter approach seems more flexible when different styles of handwriting and large vocabularies are concerned. In order to train a recognizer, the (velocity-based) strokes belonging to separate letters in their context have to be identified. For that purpose, a corpus of someones normal handwriting is available together with the letters of the words. In this paper, various methods of segmenting on-line recorded cursive script are briefly reviewed. A promising method of unsupervised learning, inspired by simulated annealing or relaxation has been implemented and tested. Only those variations that tend to yield consistent patterns of stroke features for each letter, are accepted. The simulated-annealing procedure is valuable but not perfect. It is concluded that a combination of knowledge sources may be combined to obtain acceptable reliability.

Schomaker, L.R.B. (1992). A neural-oscillator model of temporal pattern generation. Human Movement Science, 11, 181-192.

Most contemporary neural network models deal with essentially static, perceptual problems of classification and transformation. Models such as multi-layer feed-forward perceptrons generally do not incorporate time as an essential dimension, whereas biological neural networks are inherently temporal systems. In modelling motor behaviour, however, it is essential to have models that are able to produce temporal patterns of varying duration and complexity. A model is proposed, based on a network of pulse oscillators consisting of neuron/interneuron (NiN) pairs. Due to the inherent temporal properties, a simple NiN net, taught by a pseudo-Hebbian learning scheme, could be used in simulating handwriting pen-tip displacement of individual letters.
Keywords: spike oscillator, spiking oscillator, neural pulse oscillator network

 1993

Abbink, G.H., H.-L. Teulings, & L.R.B. Schomaker (1993). Description of on-line script using Hollerbach's generation model. Proceedings of the Third International Workshop on Frontiers in Handwriting Recognition (IWFHR-3), Buffalo, USA: CEDAR, May 25-27, (pp. 217-224).

In most on-line cursive script recognizers, the pen trajectory is chopped into smaller basic elements. These basic elements are identified and combined in order to recognize the pen-tip trajectory as a whole. Much research seems to be devoted to the question of how these basic elements should be represented. In the present research, the basic elements are so-called 'dual strokes', two successive velocity-based strokes which form shapes that can be described well with Hollerbach's model for cursive-script generation. Once such a dual-stroke description is retrieved, various features can be estimated by analytical expressions. As an example, a derivation of an expression of the curvature at the middle segmentation point of a 'dual stroke' is given. A preliminary version of a dual-stroke cursive script recognizer showed good results for oscillating types of connected-cursive handwriting.

Helsper E & Schomaker LR (1993). Off-line and on-line handwriting recognition: the role of pen movement in machine reading. Optical Character Recognition in the Historical Discipline: Proceedings of an International Work group organized by NHDA and NICI. Goettingen: Max-Planck Institut fuer Geschichte, pp. 39-51.

In computer recognition of text, there are a number of levels of increasing difficulty: (1) printed text, (2) handwritten and fully separated letters and digits, and (3) connected-cursive handwriting. Software for machine reading of printed text is commercially available. Techniques for reading isolated handwritten characters and digits are already in use for the processing of bank checks and postal codes on envelopes. Most writings, however, including all historical material are produced with the purpose of being read by humans, not machines. The most challenging area in this respect is the automatic processing of scanned cursive script. This paper shows illustrations of stroke-order reconstruction from off-line script samples.

Helsper, E., Schomaker, L., & Teulings, H.L. (1993). Tools for the recognition of handwritten historical documents. History and Computing, 5(2), 88-93.

An optical cursive script recognition system for historical documents faces the problem of recognizing handwritten words on paper, including interfering lines and blots. Recognition would be greatly simplified if the original pen movements were available. Therefore, a technique is proposed to recover pen movements from a scanned image. The image of a writing pattern is thus translated into an ordered sequence of strokes, where each stroke can be classified by its shape. We use a simulated neural network to learn the associations between pen position and current direction on the one hand, and its desired direction and speed on the other hand. The network uses no knowledge about what letters are present. For the training phase, we use words of which both bit images and recorded pen trajectories are available. After training, the network can retrace the strokes, given some starting points. Having retraced all strokes,, one or more chains of strokes can be reconstructed. Within these chains, the order of the strokes is known, and chains are assumed to be ordered in time from left to right. Improving robustness to cope with different kinds of scripts will be the next step.

In this study, comparisons are made between a number of stroke-based and character-based recognizers of connected cursive script. In both approaches a Kohonen self-organizing neural network is used as a feature-vector quantizer. It is found that a "best match only" character-based recognizer performs better than a "best match only" stroke-based recognizer at the cost of a substantial increase in computation. However, allowing up to three multiple stroke interpretations yielded a much larger improvement on the performance of the stroke-based recognizer. Within the character-based architecture, a comparison is made between temporal and spatial resampling of characters. No significant differences between these resampling methods were found. Geometrical normalization (orientation, slant) did not significantly improve the recognition. Training sets of more than 500 cursive words appeared to be necessary to yield acceptable performance.

Schomaker, L.R.B., Teulings, H.-L., Helsper, E.H., & Abbink, G.H. (1993). Adaptive recognition of on-line, cursive handwriting. Proceedings of the Sixth International Conference on Handwriting and Drawing. Paris, July, 4-7: Telecom, (pp. 19-21).

A handwriting pattern is considered as a sequence of ballistic strokes. Replications of a pattern may be generated from a single, higher-level memory representation, acting as a motor program. Therefore, those stroke features which show the mot invariant pattern are probably related to the parameters of the higher-level representation, whereas the more noisy features are probably related to the parameters derived at the lower levels (top-down hierarchy). This hierarchy of invariances can be revealed by the signal-to-noise (SNR) ratio, the between-parameter correlations and the between-condition correlations. Similarly, at the higher level, a sequence of strokes may act as a unit from which individual strokes are derived (sequence hierarchy). This second type of hierarchy of invariances can be revealed by the between-stroke correlation patterns. The present research largely confirmed the top-down hierarchy, even for upward and downward strokes, separately. Downward strokes were more invariant than upward strokes in terms of vertical stroke size. However, contrary to vertical stroke size, the horizontal stroke size cannot be considered to be an invariant. Furthermore, both vertical and horizontal sizes shows substantial between-stroke correlation. In contrast, the stroke durations did not show any between-stroke correlation. This suggests that velocity-based stroke segmentation is reliable, in spite of its discrete partitioning of the handwriting movements.

 1994

Guyon, I., Schomaker, L., Plamondon, R., Liberman, M. and Janet, S. (1994). UNIPEN project of on-line data exchange and recognizer benchmarks, Proceedings of the 12th International Conference on Pattern Recognition, ICPR'94, pp. 29-33, Jerusalem, Israel, October 1994. IAPR-IEEE.

In this paper, we report on the status of the UNIPEN project of data exchange and recognizer benchmarks for on-line handwriting. This project started two years ago at the initiative of the International Association of Pattern Recognition (Technical Committee 11). The purpose of the project is to propose and implement solutions to the growing need of handwriting samples for on-line handwriting recognizers used by pen-based computers. Researchers from several companies and universities have agreed on a data format, a platform of data exchange and a protocol for recognizer benchmarks. The on-line handwriting data of concern may include hand print and cursive from various alphabets (including Latin and Chinese), signatures and pen gestures. These data will be compiled and distributed by the Linguistic Data Consortium. The benchmarks will be arbitrated by the US National Institute of Standards and Technologies. A brief introduction to the UNIPEN format is given. Furthermore, we explain the protocol of data exchange and benchmarks.

Schomaker, L.R.B. (1994). User-interface aspects in recognizing connected-cursive handwriting. Proceedings of the IEE Colloquium on Handwriting and Pen-based input, 11 March, London: The Institution of Electrical Engineers, Digest Number 1994/065, (ISSN 0963-3308).

There are at least two major stumbling blocks for user acceptance of pen-based computers: (1) the recognition performance is not good enough, especially on cursive handwriting, and (2) the user interface technology has not reached a mature stage. The initial reaction of product reviewers and potential user groups to pen-based computers varies. The realistic assessment is that this is a technology with a large potential, but still too brittle for serious real-world use. The application area of form filling, for instance, is characterized by acceptable digit recognition rates, but the essential punctuations such as decimal points and commas are still handled poorly. Similarly, there are problems in handling text input. Novice users seem to have a very high expectation of the pen technology. They start quickly writing a paragraph of text in their style (mostly a mix of hand print and cursive) and are amazed at the resulting strings of question marks and characters which they do not remember having entered. Also, whereas users seldomly blame the keyboard for their typing errors, the pen computer is blamed for not recognizing script which shows evident shape errors after close inspection. It seems to be clear that there are problems both at the system and at the user side.

Schomaker, L, Abbink, G. & Selen, S. (1994). Writer and Writing-Style Classification in the Recognition of On-line Handwriting. Proceedings of the European Workshop on Handwriting Analysis and Recognition: A European Perspective, 12-13 July, 1994, London: The Institution of Electrical Engineers, Digest Number 1994/123, (ISSN 0963-3308).

The Kohonen self-organized map of velocity-based strokes was found to be a useful processing stage in earlier work. However, a problem is posed by the huge variation in writing styles. In order to develop style-specific recognition of cursive handwriting, it is useful to apply a form of writer or writing-style classification in order to restrict the set of possible and relevant character shapes. In this paper, the histogram of stroke usage in the Kohonen self-organized map of velocity-based strokes was used to classify writers. Using average-linkage hierarchical clustering, the smallest clusters(n > 1) consisted of individual writers.

 1995

Schomaker, L.R.B, Muench, S. & Hartung, K. (Eds). (1995). A Taxonomy of Multimodal Interaction in the Human Information Processing System. Report of the Esprit Project 8579 MIAMI (187 p.), Nijmegen: NICI.

This document has been prepared in the ESPRIT/BRA project 8579, Multimodal Integration for Advanced Multimedia Interfaces (MIAMI), in order to serve as a basis for future work. The basic terms which will be used in MIAMI are defined and an overview on man-machine-interfaces is given. The document consists of six chapter and extended appendices. In the Introduction, a general model for human-computer interaction is given. Four components are identified: Human Output to Computer (HOC), Computer Input Modalities (CIM), Computer Output Media (COM), and Human Input from Computer (HIC). Furthermore, we identify the necessary cognitive modules of abstract knowledge representation in both man and machine. Chapter 2 deals with human Perception characteristics (HIC). Chapter 3 describes Control and Manipulation by the user (HOC). Chapter 4 introduces aspects of Interaction design and the effects of 'modality blending'. Chapter 5 addresses the central or or Cognitive component, which is needs to be described in human-computer interaction, and Chapter 6, finally describes some possible future scenarios in multimodal interaction. The Appendices give an introduction into the uni- and bimodal interaction expertise present in the MIAMI project: Binaural Technology, Audio-Visual Speech Synthesis, Audio-Visual Speech Recognition, Gesture Taxonomies, and Two-dimensional Movement in Time. The report is completed with an extensive bibliography in the relevant scientific and technological fields.

Schomaker , L.R.B, Nijtmans, J., Camurri, A., Lavagetto, F., Morasso, P., Benoît, Guiard-Marigny, T., Le Goff, B., Robert-Ribes, J., Adjoudani, A., Defée, I., Muench, S., Hartung, K., Blauert, J. (1995). DI2 - Progress Report of the Esprit Project 8579 MIAMI (122 p.), Nijmegen: NICI.

A number of experiments in multimodal control are described. While in the fields of perception research and experimental psychology, many findings on unimodel interaction are studied, our approach is oriented towards practical multimodal solutions. Three types of experiments were performed.
• Spatiotemporal integration of simple visuo-acoustical patterns: The aim is to derive a numerical description about to what extent the spatial and temporal attributes of visual and acoustic representations may deviate in order to form an integrated single-object perception.
• Visual and acoustical integration in speech perception and automatic recognition: A comparison is made between unimodal (acoustical) and bimodal (sound + lip movement) recognition of speech by humans and by existing recognition algorithms.
• Visual and haptic integration: In this set of experiments, haptic and visual integration is tested. A manipulator with tactile feedback is used to perform simple tele-operating manipulations with controlled visual participation via camera and monitor. The images were varied in detail, quality and amount of noise. Results are compared for a number of control-interface devices.

 1996

Schomaker, L.R.B., & Van Galen, G.P. (1996). Computer models of handwriting. In: Dijkstra & De Smedt (Eds.), Computational Psycholinguistics: AI and connectionist models of human language processing (pp. 386-420). London: Taylor & Francis.

In this chapter, a general introduction is given in the area of modeling handwriting. It is noted that most existing models of handwriting focus on the peripheral aspects of the handwriting process. Also, the majority of models is directed at a superficial regeneration of experimental handwriting data. In such an approach, the essence of handwriting as a process that evolves in real time is lost, and there is the risk that the model is more an exercise in curve fitting rather than being a tool for gaining insight in the psychomotor aspects of the handwriting process. An essential problem in handwriting production is the fact that there are symbolical levels of processing, in which discrete representations or system states are involved, as well as analog or continuous levels of processing, in which force, amplitude and time play a central role. A conceptual model of the handwriting process is presented, addressing aspects of seriality and parallelism, and taking into account the existing empirical psychomotor knowledge. Finally, two simulation models are presented, which address two different levels of chaining and shaping processes in handwriting production.

Vuurpijl, L. & Schomaker, L. (1996). Coarse writing-style clustering based on simple stroke-related features. Proceedings of the 5th International Workshop on Frontiers in Handwriting Recognition (pp. 29-34), Sept. 2-5, 1996, Colchester, England.

Two methods are presented for the automatic detection of generic writing styles like e.g. {\em mixed}, {\em cursive} and {\em hand print}. Such techniques can be used for e.g. a dispatch process which assigns specialized$recognition systems to a writer with unknown writing style. Based on a set of handwritten words, three features are determined: a {\em cursivity index}$c$, which indicates the tendency of a writer to write cursive, and two distance measures$d_c$and$d_h$. The distance measures represent the distance between the stroke feature vectors in the input handwriting$ and the strokes stored in two style-specific Kohonen Self-Organizing Maps (SOM)$SOM is tuned for the writing style {\em hand print}, and the other for {\em curs$ The first method uses a linear decision criterion to classify feature vector $\{c,d_c,d_h\}$ into one of the three writing styles. The second method uses non-linear decision boundaries found via agglomerative hierarchical clustering of the three-dimensional feature vectors. This method is able to make a more fine-grained distinction between writing sty$classifications, especially within the lumped category 'mixed'. Hartung, K., Muench, S., Schomaker L., Guiard-Marigny, T., Le Goff, B. MacLaverty, R., Nijtmans, J., Camurri, A., Defee, I., and Benoit, C. (1996). DI3 - Development of a System Architecture for the Acquisition, Integration, and Representation of Multimodal Information. Report of Esprit Project 8579/MIAMI, March 25, 1996. Nijmegen: NICI. (121 p.).  1997 Vuurpijl, L. & Schomaker, L. (1997). Coarse writing-style clustering based on simple stroke-related features. In: A.C. Downton & S. Impedovo, Progress in Handwriting Recognition (pp. 37-34). London: World Scientific. ISBN 981-02-3084-2 Mackowiak, J., Schomaker, L. & Vuurpijl, L. (1997). Semi-automatic determination of allograph duration and position in on-line handwriting words based on the expected number of strokes. In: A.C. Downton & S. Impedovo, Progress in Handwriting Recognition (pp. 69-74). London: World Scientific. ISBN 981-02-3084-2 In this paper, a semi-automatic character labeling ('truthing') procedure is presented which uses an initial unsupervised algorithm to estimate the starting stroke and number of strokes per allograph, followed by minimal user interaction to improve the estimates (in this study, a stroke is defined as the trajectory of the pen tip between two consecutive minima in the absolute pen-tip velocity). Such a procedure is very useful since the complete manual labeling of handwriting takes in the order of one hour per one-hundred written words. Even without making use of detailed shape information the proposed algorithm already yields a usable segmentation which facilitates the manual labeling process. Vuurpijl, L. & Schomaker, L. (1997). Finding structure in diversity: A hierarchical clustering method for the categorization of allographs in handwriting, Proceedings of the Fourth International Conference on Document Analysis and Recognition, Piscataway (NJ): IEEE Computer Society, p. 387-393. ISBN 981-02-3084-2 This paper introduces a variant of agglomerative hierarchical clustering techniques. The new technique is used for categorizing character shapes (allographs) in large data sets of handwriting into a hierarchical structure. Such a technique may be used as the basis for a systematic naming scheme of character shapes. Problems with existing methods are described and the proposed method is explained. After application of the method to a very large set of characters, separately for all the letters of the alphabet, relevant clusters are identified and given a unique name. Each cluster represents an allograph prototype.  1998 Schomaker, L.R.B. (1998). Entre écrire des formes sur une ardoise et le traitement d'information utilisable. Actes du 1er Colloque International Francophone sur l'Écrit et le document, p. 9-13. Montreal: Acfas. ISBN 2-89245-148-5. L'introduction commercial du concept de 'pen computing' ou les petits ordinateurs qui sont equippé d'un stylo pour l'entrée des données etait un processus assez pénible pour l'industrie impliqué. Dans ce tableau, les origines de cet échec sont analysés, partant de trois points de vue de recherche. Aprés un introduction concernant l'histoire recemment du ordinateur á stylo, le premier perspectif donné concerne le processus psychomotorique gouvernant l'ecrire avec le stylo. Le deuxième aspect concerne l'interface utilisateur á stylo (IUS). La dernière perspective est celle-ci du reconaissance des formes dans l'écriture registré en-ligne. Á la base de cet tableau sont deux observations: (1) notre connaissance des formes qui se trouvent dans l'écriture Occidentale et multi-culturelle est assez superficiel et doit etre élargi et approfondi; et (2) la développement des algorithmes pour la reconnaissance automatique de l'écriture dans le contexte des agendas électroniques ne peut pas réussir si l'interface utilisateur et l'application ne sont pas connus précisement dès le début de cette entreprise. Les solutions sont évident: (1) le construction d'une taxonomie explicite des formes (allographes) dans l'écriture Occidentale, et (2) l'integration des recherches sur le terrain de la reconnaissance automatique avec ceux dans le terrain de la ergonomie cognitive des interface utilisateurs sur les ordinateurs miniaturisés. Schomaker, L.R.B. (1998). From handwriting analysis to pen-computer applications. IEE Electronics Communication Engineering Journal, 10(3), pp. 93-102. In this paper, pen computing, i.e., the use of computers and applications in which the pen is the main input device, will be described from four different angles. In the first section, a brief overview will be given on the hardware developments in pen systems. After concluding that the technological developments in this area did not lead to the expected user acceptance of pen computing, the reasons underlying this market failure are explored: The second part deals with Pen-User Interfacing (PUI) aspects. Problems of pen-user interface design are described. Existing and new applications are summarized. The third part is concerned with the handwriting process and product. The last part deals with automatic recognition methodologies. Four basic factors determining handwriting variation and variability are identified. A handwriting recognition approach using segmentation into velocity-based strokes is handled in somewhat more detail. A large-scale project (UNIPEN) concerns the benchmarking of the performance of on-line handwriting recognition algorithms which is crucial for the advancement of the state of the art in this area. Schomaker, L. & Segers, E. (1998). A method for the determination of features used in human reading of cursive handwriting. Proceedings of IWFHR'98, 12-14 August, Taejon, Korea, pp. 157-168. Schomaker, L., Hoenkamp, E. & Mayberry, M. (1998). Towards collaborative agents for automatic on-line handwriting recognition. Proceedings of the Third European Workshop on Handwriting Analysis and Recognition, 14-15 July, 1998, London: The Institution of Electrical Engineers, Digest Number 1998/440, (ISSN 0963-3308), pp. 13/1-13/6. A important problem in many areas of complex information processing is to integrate heterogeneous information from different sources. For handwriting recognition we accomplished such an integration through an agent architecture of three co-operating agents. One agent represents the bottom-up knowledge as generated by a shape-based classifier of unistrokes. The second agent contains top-down (syntactic) knowledge about the language to be interpreted. The third agent is the user who produces written symbol shapes and who can be queried by the other agents. The chosen application paradigm is the real-time input and execution of Scheme code. Although this is not a practical domain, it contains all the essential elements of a full text-entry application. Notable improvements can be gained over a pure bottom-up approach to pen-based computing. Vuurpijl, L. & Schomaker, L. (1998). A framework for using multiple classifiers in a multiple-agent architecture. Proceedings of the Third European Workshop on Handwriting Analysis and Recognition, 14-15 July, 1998, London: The Institution of Electrical Engineers, Digest Number 1998/440, (ISSN 0963-3308), pp. 8/1-8/6 Novel pattern recognition techniques using multiple agents for the recognition of handwritten text are proposed in this paper. The architecture of a distributed system dispatching recognition tasks to a set of recognizers and combining their results is discussed. The concept of intelligent agents and innovative multi-agent architectures for pattern recognition tasks is introduced for combining and elaborating the classification hypotheses of several classifiers. This concept is being developed in the IART project, where intelligent agent architectures are built for pattern recognition tasks. Vuurpijl, L. & Schomaker, L. (1998). Multiple-agent architectures for the classification of handwritten text. Proceedings of IWFHR'98, 12-14 August, Taejon, Korea, pp. 335-346. (Loe?)  1999 Hoenkamp, E.C.M., Stegeman, O., and Schomaker, L.R.B. (1999). Supporting content retrieval from WWW via 'basic level categories'. In Proceedings of the 22rd International Conference on Research and Development in Information Retrieval (pp. 311-312). New York: ACM Press (SIGIR forum, 33 (3)). Plamondon, R., Lopresti, D.P., Schomaker, L.R.B. and Srihari, R. (1999). On-line handwriting recognition. In: J.G. Webster (Ed.). Wiley Encyclopedia of Electrical & Electronics Engineering, 123-146, New York: Wiley, ISBN 0-471-13946-7. This chapter gives an overview of the field of on-line handwriting recognition. The introduction explains some backgrounds of the field: on-line vs off-line recognition; recognition vs verification; occidental vs oriental languages; and pen-pad applications. A section on handwriting, i.e., process modeling and understanding deals with psychophysical aspects. A section on preprocessing elaborates on the following techniques: filtering, stroke detection, character and word segmentation, restoration and beautification. Furthermore data compression and ink preprocessing are elucidated. The next section of the chapter focuses on the handwriting recognition itself: Gesture, character and word classification; the differences between recognizing cursive vs hand-printed styles, and processing of sentences. Structural and statistical methods are explained, where a distinction is made between explicit statistical methods and implicitly statistical methods (such as neural-network classifiers). Finally, hidden-Markov models as applied to on-line handwriting recognition are explained. The final section is concerned with post processing of classification results: combination of methods, lexical, syntactical and semantic approaches. Schomaker, L., de Leau, E. & Vuurpijl, L. (1999). Using pen-based outlines for object-based annotation and image-based queries. In: D.P. Huijsmans and A.W.M. Smeulders (Eds.). Visual Information and Information Systems, New York: Springer, pp. 585-592. A method for image-based queries and search is proposed which is based on the generation of object outlines in images by using the pen, e.g., on color pen computers. The rationale of the approach is based on a survey on user needs, as well as on considerations from the point of view of pattern recognition and machine learning. By exploiting the actual presence of the human users with their perceptual-motor abilities and by storing textually annotated queries, an incrementally learning image retrieval system can be developed. As an initial test domain, sets of photographs of motor bicycles were used. Classification performances are given for outline and bitmap-derived feature sets, based on nearest-neighbour matching, with promising results. The benefit of the approach will be a user-based multimodal annotation of an image database$ yielding a gradual improvement in precision and recall over time.

Schomaker, L., Vuurpijl, L. & de Leau, E. (1999). New use for the pen: outline-based image queries. Proceedings of the 5th International Conference on Document Analysis and Recognition (ICDAR '99). Piscataway (NJ): IEEE. pp. 293-296.

Schomaker, L. & Segers, E. (1999). Finding features used in the human reading of cursive handwriting International Journal on Document Analysis and Recognition, 2, 13-18.

This paper first summarizes a number of findings in human reading of handwriting. A method is proposed to uncover more detailed information about geometrical features which human readers use in the reading of Western script. The results of an earlier experiment on the use of ascender/descender features were used for a second experiment aimed at more detailed features within words. A convenient experimental setup was developed, based on image enhancement by local mouse clicks under time pressure. The readers had to develop a cost-effective strategy to identify the letters in the word. Results revealed a left-to-right strategy in time, however, with extra attention to the initial, leftmost parts and the final, rightmost parts of words in a range of word lengths. The results confirm high hit rates on ascenders, descenders, crossings and points of high curvature in the handwriting pattern.

Christian Benoit, Catherine Pelachaud, Lambert Schomaker & Bernhard Suhm (1999). Audio-visual and Multimodal Speech Systems. Esprit project EAGLES. (under revision: will appear as a chapter in a book).

This chapter on multimodal interfaces and multimodal speech systems (as defined above) is structured as follows. After this introductory section, Section 1 presents results from a literature review and survey of multimodal systems that the authors of this chapter performed. Section 2 discusses evaluation of multimodal systems. Challenges in evaluating multimodal systems are identified, known evaluation methodologies are reviewed, and issues in evaluating certain kinds of multimodal systems are discussed, including talking heads and synthetic conversational agents. The next four sections discuss aspects of the various types of multimodal systems, according to what speech- and non-speech modalities speech input and output is associated with: Section 3 describes systems that combine speech input with information from the visual channel (face detection, face recognition, tracking of facial features, and lip-reading), Section 4 describes systems that combine speech with visual output (e.g., talking heads), Section 5 describes systems that combine speech input with other input modalities (defined as multimodal interface above), and Section 6 describes systems that combine speech output with other modalities (defined as multimedia systems above). These sections focus on concepts and issues; the details of the technology necessary to implement such systems is reviewed in Section 7. Finally, Section 8 presents established standards and common resources for multimodal systems.

 2000

M. van Erp and L. Schomaker (2000). Variants of the Borda count method for combining ranked classifier hypotheses. Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition (7th IWFHR), Nijmegen: iUF, p. 443-452.
ISBN 90-76942-01-3

The Borda count is a simple yet effective method of combining rankings. In pattern recognition, classifiers are often able to return a ranked set of results. Several experiments have been conducted to test the ability of the Borda count and two variant methods to combine these ranked classifier results. By using artificial data, domain-specific results were avoided. The results show the strength of the Borda count when many errors occur in the results, but also show its weakness in case of a limited number of large ranking errors.

L. Schomaker, D. Mangalagiu, L. Vuurpijl and M. Weinfeld (2000). Two tree-formation methods for fast pattern search using nearest-neighbour and nearest-centroid matching. Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition (7th IWFHR), Nijmegen: iUF, p. 261-270.
ISBN 90-76942-01-3

This paper describes tree-based classification of character images, comparing two methods of tree formation and two methods of matching: nearest neighbor and nearest centroid. The first method, Preprocess Using Relative Distances (PURD) is a tree-based reorganization of a flat list of patterns, designed to speed up nearest-neighbor matching. The second method is a variant of agglomerative hierarchical clustering (HCLUS) which aims at finding a hierarchical structure of centroids in the pattern space. Results indicate that the PURD method is a very fast, effective and convenient method for the speedup of 1NN search, from which it is, however, difficult to derive usable character prototypes. HCLUS can be used to obtain very fast search with acceptable classification rate while providing character prototypes, however, at the cost of significant training efforts.

Vuurpijl and L. Schomaker (2000). Two-stage character classification: A combined approach of clustering and Support Vector Classifiers Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition (7th IWFHR), Nijmegen: iUF, p. 423-432.
ISBN 90-76942-01-3

This paper describes a two-stage classification method for (1) classification of isolated characters and (2) verification of the classification result. Character prototypes are generated using hierarchical clustering. For those prototypes known to sometimes produce wrong classification results, a "support vector classifier" (svc) is trained. The svc can be used to increase the confidence that a classification is correct and furthermore decide on a classification if the confidence using the standard method is too low. Experiments with the iUF UNIPEN database yield 94% recognition rate. In cases where both classifiers agree, the error rate is zero.

F. Wang, L. Vuurpijl and L. Schomaker (2000). Support Vector Machines for the classification of Western Handwritten Capitals Proceedings of the Seventh International Workshop on Frontiers in Handwriting Recognition (7th IWFHR), Nijmegen: iUF, p. 167-176.
ISBN 90-76942-01-3

In this paper, new techniques are presented using Support Vector Machines (SVMs) for multi-class classification problems. The issue of decomposing a N-class classification problem into a set of 2-class classification questions is discussed. In particular, the technique for normalizing the outputs of several SVMs is presented. Based on these techniques, support vector classifiers for the recognition of Western handwritten capitals are realized. Comparisons to several other classification methods are also presented.

 2001

Schomaker, L.R.B. (2001) Image Search and Annotation: From Lab to Web. Document Electronique: Methodes, demarches et techniques cognitives, pp.373-375, ISBN 2-909285-17-0.

In earlier work, a method for image-based queries and search was proposed which is based on drawing of object outlines on photos, by the users of an image-search system. By exploiting the actual presence of the human users with their perceptual-motor abilities and by storing textually annotated queries, an incrementally learning image retrieval system can be constructed. As a proof of concept, a large collection of paintings of the Dutch Rijksmuseum was put on an Internet website, embedded in Java software which allowed for the drawing of object outlines and the typing of textual annotation [4]. The proposed outline-matching scheme can be used to search and organize the annotated objects in a growing and enriched database of objects which are present within the painting collection. Figures 1 display a query for 'dogs', based on the outline drawn in a painting by the Dutch painter Jan Steen. The object can be extracted from the image context. Different annotators produce multiple selections of objects on different paintings in the database. Both at the level of perception (geometry) and cognition (semantics) the human input is invaluable. It is believed that such an approach will ultimately lead to better 'machine intelligence': It has become clear in several domains (cf., the openMind initiative) that only large amounts of labeled data will provide the performance levels which are required if these techniques (speech recognition, handwriting recognition, object recognition) need to evolve from their current rudimentary stage.

Schomaker, L.R.B. (2002). [Book Review on] "Plausible Neural Networks for Biological Modelling (2001). H.A.K. Mastebroek & J.E. Vos (Eds.)", BCN Nieuwsbrief, 46. Groningen: BCN - School of Behavioral and Cognitive Neuroscience [in Dutch].

van Erp, M., Vuurpijl, L. & Schomaker, L. (2002). An overview and comparison of voting methods for pattern recognition, Proc. of the 8th IWFHR, Piscataway: IEEE, ISBN 0-7695-1692-0, pp. 195-200.

In pattern recognition, there is a growing use of multiple classifier combinations with the goal to increase recognition performance. In many cases, plurality voting is a part of the combination process. In this article, we discuss and test several well known voting methods from politics and economics on classifier combination in order to see if an alternative to the simple plurality vote exists. We found that, assuming a number of prerequisites, better methods are available, that are comparatively simple and fast.
In this inaugural address, the incommensurability of methods from logic and geometry are illustrated, using the example of automatic recognition of cursive script. The schism started with the separation of the AAAI and the IAPR research fields, currently leading to inelegant hybrid systems and models. It is noted that the current state of the art is far from being able to produce reliable and autonomous systems. Furthermore, it is noted that systems have to be completely and manually re-engineered for each application variant and context of usage. A second theme of this inaugural address concerns the distinction between builders (engineers), 'knowers' (scientists) and thinkers (philosophers). It is noted that there exists a pecking order in which the builders are in a most unfortunate position.

L. Schomaker, M. Bulacu & M. van Erp (2003). Sparse-parametric writer identification using heterogeneous feature groups. ICIP'2003: IEEE International Conference on Image Processing (Vol. I), pp. (I) 545-548.

This paper evaluates the performance of edge-based directional probability distributions as features in writer identification in comparison to a number of non-angular features. It is noted that angular features outperform all other features. However, the non-angular features provide additional valuable information. Rank-combination was used to realize a sparse-parametric combination scheme based on nearest-neighbor search. Limitations of the proposed methods pertain to the amount of handwritten material needed in order to obtain reliable distribution estimates. The global features treated in this study are sensitive to major style variation (upper- vs lower case), slant, and forged styles, which necessitates the use of other features in realistic forensic writer identification procedures.

M. Bulacu, L. Schomaker & Vuurpijl, L. (2003). Writer identification using edge-based directional features. ICDAR'2003: International Conference on Document Analysis and Recognition, pp. 937-941.

This paper evaluates the performance of edge-based directional probability distributions as features in writer identification in comparison to a number of non-angular features. It is noted that the joint probability distribution of the angle combination of two "hinged" edge fragments outperforms all other individual features. Combining features may improve the performance. Limitations of the method pertain to the amount of handwritten material needed in order to obtain reliable distribution estimates. The global features treated in this study are sensitive to major style variation (upper- vs lower case), slant, and forged styles, which necessitates the use of other features in realistic forensic writer identification procedures.

M. Bulacu & L. Schomaker (2003). Writer Style from Oriented Edge Fragments. Proc. of the 10th Int. Conference on Computer Analysis of Images and Patterns (CAIP'03), pp. 460-469.

In this paper we evaluate the performance of edge-based directional probability distributions extracted from handwriting images as features in forensic writer identification in comparison to a number of non-angular features. We compare the performances of the features on lowercase and uppercase handwriting. In an effort to gain location-specific information, new versions of the features are computed separately on the top and bottom halves of text lines and then fused. The new features deliver significant improvements in performance. We report also on the results obtained by combining features using a voting scheme.

L. Schomaker & M. Bulacu (2004). Automatic writer identification using connected-component contours and edge-based features of upper-case Western script. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp.xx-xxx.

In this paper, a new technique for off-line writer identification is presented, using connected-component contours (COCOCOs or CO3s) in upper-case handwritten samples. In our model, the writer is considered to be characterized by a stochastic pattern generator, producing a family of connected components for the upper-case character set. Using a codebook of CO3s from an independent training set of 100 writers, the probability-density function (PDF) of CO3s was computed for an independent test set containing 150 unseen writers. Results revealed a high-sensitivity of the CO3 PDF for identifying individual writers on the basis of a single sentence of upper-case characters. The proposed automatic approach bridges the gap between image-statistics approaches on one end and manually measured allograph features of individual characters on the other end. Combining the CO3 PDF with an independent edge-based orientation and curvature PDF yielded very high correct identification rates.

R. Ekker, E.C.D. van der Werf & L.R.B. Schomaker (2004). Dedicated TD-learning for stronger gameplay: applications to Go In: A. Nowe, T. Lennaerts, K. Steenhaut (Eds.), Proceedings of Benelearn 2004 Annual Machine Learning Conference of Belgium and The Netherlands (Brussels, BE, January 8-9), pp. 46-52.

This paper presents a study of several dedicated Temporal-Difference (TD) reinforcement learning algorithms for deterministic zero-sum games of perfect information such as the game of Go. The algorithms include TD(mu) by Beal (2002), which separates good play from bad play, TD-leaf(lambda) and TD-directed(lambda) by Baxter et al. (1998), which exploit game tree searching, as well as Baird's residual algorithm (1995) for preventing instability during training. We show that dedicated TD learning algorithms provide faster training and the acquisition of more 'genuine' knowledge of the game resulting in significantly higher playing strength than players trained by standard TD.

L. Schomaker (2004). Anticipation in cybernetic systems: A case against mindless anti-representationalism. Proc. of IEEE Systems, Man & Cybernetics (SMC'04), pp. xx-xxx.

The developments in behavior-based robotics and in ecological psychology have had a strong effect on theoretical development in some research communities. A new belief has emerged under the name of anti-representationalism, which is strongly opposed to the notion of representations in cognition. This notion is spurred by the inarguably fruitful insight that behavioral complexity can be brought about by simple mechanisms at a low systemic level. Although there are many problems with constructed formal representations in the toy models of traditional artificial intelligence, there is a fundamental problem with extreme anti-representationalism, as well. Representations actually do exist in the biological neural-information processing system. In this paper, a review of neural representation mechanisms will be given, looking at perception and motor control in biological systems. Subsequently, it will be illustrated that already in simple animal behaviors, simple 'representation-less' reactivity does not suffice. Anticipation exists even in jumping spiders, requiring the existence of a representation as the computational basis for the prediction of future system states.

Niels, R., Vuurpijl, L. & Schomaker, L.R.B. (2006, in press). Automatic allograph matching in forensic writer identification. International Journal of Pattern Recognition and Artificial Intelligence, x(x), p. xxx-xxx.

A well-established task in forensic writer identification focuses on the comparison of prototypical character shapes (allographs) present in the handwriting. In order for a computer to perform this task convincingly, it should yield results that are plausible and understandable to the human expert. Trajectory matching is a well-known method to compare two allographs. This paper assesses a promising technique for so-called human- congruous trajectory matching, called dynamic time warping. In the first part of this paper, experiments are shown that indicate that DTW yields results that correspond to the expectations of human users. Since DTW requires the dynamics of the handwritten trace, the on-line dynamic allograph trajectories need to be extracted from the \off-line" scanned documents. For the second part of this paper, handwritten images are employed generated from relatively large on-line data sets, which provide the true trajectories. This allows for a quantitative assessment of the trajectory extraction techniques rather than a qualitative discussion of a small number of examples. Our results show that DTW can significantly improve the results from trajectory extraction when compared to traditional techniques.

Schomaker, L.R.B., Franke, K. & Bulacu, M. (2006, in press). Using codebooks of fragmented connected-component contours in forensic and historic writer identification, Pattern Recognition Letters, x(x), p. xxx-xxx.

Recent advances in 'off-line' writer identification allow for new applications in handwritten text retrieval from archives of scanned historical documents. This paper describes new algorithms for forensic or historical writer identification, using the contours of fragmented connected-components in free-style handwriting. The writer is considered to be characterized by a stochastic pattern generator, producing a family of character fragments (fraglets). Using a codebook of such fraglets from an independent training set, the probability distribution of fraglet contours was computed for an independent test set. Results revealed a high sensitivity of the fraglet histogram in identifying individual writers on the basis of a paragraph of text. Large-scale experiments on the optimal size of Kohonen maps of fraglet contours were performed, showing usable classification rates within a non-critical range of Kohonen map dimensions. The proposed automatic approach bridges the gap between image-statistics approaches and purely knowledge-based manual character-based methods.

Bulacu, M. & Schomaker, L.R.B. (2007, in press). Text-independent Writer Identification and Verification Using Textural and Allographic Features, IEEE PAMI, July 2007, x(x), p. xxx-xxx.

The identification of a person on the basis of scanned images of handwriting is a useful biometric modality with application in forensic and historic document analysis and constitutes an exemplary study area within the research field of behavioral biometrics. We developed new and very effective techniques for automatic writer identification and verification that use probability distribution functions (PDFs) extracted from the handwriting images to characterize writer individuality. A defining property of our methods is that they are designed to be independent of the textual content of the handwritten samples. Our methods operate at two levels of analysis: the texture level and the character-shape (allograph) level. At the texture level, we use contour-based joint directional PDFs that encode orientation and curvature information to give an intimate characterization of individual handwriting style. In our analysis at the allograph level, the writer is considered to be characterized by a stochastic pattern generator of ink-trace fragments, or graphemes. The PDF of these simple shapes in a given handwriting sample is characteristic for the writer and is computed using a common shape codebook obtained by grapheme clustering. Combining multiple features (directional, grapheme and run-length PDFs) yields increased writer identification and verification performance. The proposed methods are applicable to free-style handwriting (both cursive and isolated) and have practical feasibility, under the assumption that a few text lines of handwritten material are available in order to obtain reliable probability estimates.