NWO/DiD project 'Global Currents' - ALICE - University of Groningen
Dutch | English
The 'Global Currents' project is a grant provided by the Netherlands Organization for Scientific Research (NWO),
the Natural Sciences and Engineering Research Council (Canada), the Social Sciences and Humanities Research Council,
the Foundation for Innovation (Canada) and the National Endowment for the Humanities (US).
Our task in this project is the design and application of algorithms for the induction of semantic concepts
from the visual surface of historical document images. Until today, there has been a predominant focus on the
textual transcription (transliteration) of historical documents. This focus has ignored the wealth of visual information
that is available on each of the scans of the original document pages. Word shape itself may be informative, such as,
e.g., a Greek word in an arabic text. However, there are many more special visual items: Calligraphy,
author markings, illuminated capitals, schematic drawings, special symbols and glyphs, even 'doodles', that can give an insight
into the underlying meaning and provenance of a text. Additionally, the spatial ordering of visual elements on a page, i.e., their layout,
will often be stochastically regular and may give important clues to the meaning. The recurrence of visual elements,
in large and heterogeneous document collections allows for a tracing of visual memes over networks
of authors by means of 'big-data mining'. In this project we will apply and extend the knowledge that has been developed in
our Monk system
for handwritten word retrieval from large historical manuscript collections.
- Project supervisor for The Netherlands: prof. dr. Lambert Schomaker
- Postdoctoral researcher (vacancy)
Institute for Artificial Intelligence & Cognitive Engineering (ALICE)
Faculty of Mathematics and Natural Sciences
University of Groningen, The Netherlands.
The other project partners in Canada and the United States are:
- prof. Mohamed Cheriet of Ecole de technologie supérieure(Montréal)
- prof. Andrew Piper (McGill University) (principal investigator)
- prof. Elaine Treharne (Stanford University)
An example of the relation between content and the spatial layout of
visual elements on a page in a historic document (an act).
If payment has occurred, the clerk places a unique sign, representing the Latin word
'solvit' in the left margin. Because this is an arbitrary glyph without individual
letters, it is better to handle this case as a generic image recognition problem
than as a case of optical character recognition ('OCR').
Also indicated are the expected positions of, respectively, the start of the
paragraph (Item), followed by a meaningful verb (semantic anchor),
with a name and a date at the bottom of the act.
This is but one of the many possible examples of visual information in
document images (from
Ritsema van Eck & Schomaker, 2012).
Drawings and diagrams in historical documents are usually related to the surrounding words.
This text is about bubbles, 'bullae'. There is a relation between that word and the round
shape of the diagram. There are other related words such as 'forma circulari' (see manually
annotated version on the right). Modern methods of pattern recognition and machine learning
allow to relate the visual and textual elements. (Example from letter by
Gisbert Cuper (1674), Royal Library (The Hague), ms. 72 C 18, f. 20 recto,
courtesy of dr. Jetze Touber).