Monk [Dutch]

The Monk system is developed at the University of Groningen by a research group at the Artificial Intelligence institute ALICE, under supervision by prof. dr. Lambert Schomaker. In cooperation with the Dutch National Archive we have created methods for accessing historical archive collections which are difficult to process by traditional OCR methods, for example due to their historical character types or due to the fact that the material is handwritten. The system consists of two major components: (1) a setup for the storage and web-based annotation of scanned page images and parts thereof; (2) a set of (handwriting and text) recognition algorithms as well as retrieval and search methods.

  • Try our [Monk search engine]

    An example hit list is here.

    Indexed books and overview

    • From the Archief van het Kabinet der Koningin, 1898-1945, Periode 1814-1988 (totale omvang 598 meter; 4660 inventarisnummers)
      we started with the book: Indices op het verbaal, boek 7823, 1903, bladz. 1-1040

    • From Alg. Rijksarch. 1e Afd. Admiraliteitscolleges
      Nr. 1177, Journaal, beginnend 7 Sept. 1779, Capiteyn J.J. van Hoeij, [... s'Lands Schip van Oorloge Rotterdam, 50 stukken canon en 300 manschappen ...] (272 pages)

    • To the internal project site with demos, annotation and search tools for handwritten or historical manuscripts. Please contact me for obtaining access to this site. Volunteers who can read Dutch or Latin are invited to participate in transcription and annotation.

    • There is also a static index of Monk's collections.

    • In the summer of 2009, the Monk system went into autonomous training mode, using a queuing system for addressing remote HPC resources and starting re-learning tasks on the basis of user activity, autonomously for the greater part, in '24/7' mode.

    • In the summer of 2010, we have ingested two different 15th century texts, on from Belgium (Schepenbank, Stadsarchief Leuven) and one from the archives of the province of Gelderland, with good success. This broadens the applicability of our algorithms, without any change to the basic methods. For each new collection, some customized layout analysis is needed, after which the scans enter into a generic pipeline and are exposed to the web.

    • As of April 2013, Monk contains 32 books/documents and 20,000 page scans of handwritten manuscripts. The machine-learning system has learned over 16,000 word-image classes for (lexical) words, terms, abbreviations and word contractions. The total number of harvested and human-confirmed word labels is 370,000. These numbers are continuously growing at a rate that depends on the available computing resources: Monk is a 24/7 machine-learning effort.

    • Currently, Monk runs on the test bed of the Target project at the High-Performance Computing center of the University of Groningen. The disk size, 'single mount point', is more than 2 petabyte, of which Monk is currently using less than 10%.

      In May 2014, Monk started processing Chinese characters, handwritten and wood-block printed, of the Harvard Yenchin collection for Grace Fong in the Digging into Data project 'Global Currents' with Andrew Piper et al. By the end of 2014, Monk was processing 300 Chinese manuscripts.

    • In December 2015, Arabic machine printed text was provided by Maxim Romanov. Fifty .pdf documents of typically 500-800 pages are now being ingested. The lexicon is +320 words after two weeks, with 15k instances and growing quickly.

    • April 2016, the system started to work on printed hieroglyphs, mixed with French machine-printed text.

    • In May 2016, we started to process war diaries for the NIOD, The Netherlands, in a pilot project. The text concerns connected-cursive Western, mostly in Dutch, 1940-ies style, for a single writer.

    • In August 2016, Monk starts to process Hieratic script on papyrus, for Leiden University

    Acknowledgements

    This web site is made possible thanks to grants by the Nederlandse Organisatie van Wetenschappelijk Onderzoek (NWO), project NWO/Catch "Scratch" and project NWO/EW "Learning to learn", project SNN/Target, Groningen, thanks to the Nationaal Archief, The Hague (dhr. H. van Schie) and a continuous support from the University of Groningen.

    People

    The following people have contributed to Monk since 2004: Lambert Schomaker, Henny van Schie, Marius Bulacu, Tijn van der Zant, Sveta Zinger, Fons Laan, Jean-Paul van Oosten, Michiel Holtkamp.

    The following annotators have produced substantial amounts of line transcriptions, word and character labels: J.A. Schomaker, H. van Schie, F. Laan, L. Schomaker, S. Zinger and several anonymous volunteers.


    schomaker with-affiliation ai.rug.nl