Learning to Communicate via Social and Linguistic Interactions


Project summary

Humans learn to communicate through social and linguistic interaction. When developing artificial agents whose purpose is to communicate with humans or with other agents, it is important to use social cues to enhance the natural quality of communication. To date, artificial intelligence models of communicative agents either have no social interaction component, or their behavior tends to follow fixed and pre-specified patterns that does not resemble human interaction behavior. In this project, we aim to develop an agent-based model that simulates naturalistic social and linguistic interactions in order to learn and use language for communication purposes.

To achieve this, we will first analyze a corpus containing naturalistic observations of social interactions among children and their social environment. This corpus was collected as part of the CASA MILA project, and annotated concerning the verbal and non-verbal interactions addressed to and produced by the children (Vogt & Mastin, 2013). The purpose of the analysis is to construct a statistical input generation engine for producing naturalistic linguistic and social input based on available verbal and non-verbal signals provided by the children’s communication partners. Second, we will design an interaction-based agent model, which can learn the meaning of words and learn how to use these words together with non-verbal signals in a naturalistic manner.  Third, a multi-agent framework will be developed in which agents equipped with the interaction-based model can interact with each other while learning how to interact socially and linguistically.


Vogt, P. and Mastin, J.D. (2013) Anchoring social symbol grounding in children’s interactions. Künstliche Intelligenz 27(2): 145-151. DOI: 10.1007/s13218-013-0243-6. Abstract Preprint

Haque, M. M., Vogt, P., Alishahi, A., and Krahmer, E. (2016). A connectionist model for automatic generation of child-adult interaction patterns. In Papafragou, A., Grodner, D., Mirman, D., and Trueswell, J., editors, Proceedings of the 38th Annual Conference of the Cognitive Science Society, Austin, TX. Cognitive Science Society. PDF

Research team

PI/supervisor: Dr. Paul Vogt

Co-supervisor: Dr. Afra Alishahi

PhD student: Moinuddin Haque

Promotor: Prof. dr. Emiel Krahmer (TiCC)


This project was funded by the Netherlands Organisation for Scientific Research (NWO) the Natural Artificial Intelligence programme.