The function of eye gaze in multimodal interaction management. An interdisciplinary study on speech, music and signed language

Research in a variety of disciplines (including linguistics, psychology, behavioral sciences and empirical musicology) has pointed at the important role of eye gaze as a strong coordinating device in interaction management. Gaining insights into the ways in which gaze may be used to claim speakership (turn-taking), to designate next speakership (turn-giving) or to give feedback to a current speaker (backchanneling), requires fine-grained data on the visual behavior of all participants involved in a communicative interaction. The proposed project implements a recently developed technique of multifocal eye-tracking (Brône & Oben 2015), which uses mobile eye-tracking systems to simultaneously capture two or more participants eye movements in (semi-)spontaneous settings, to gain access to this continuous stream of visual information. This technique will be used by an interdisciplinary consortium, consisting of linguists, musicologists and educators, to inquire into the regulatory function of eye gaze in combination with other semiotic means in three-party interactions in spoken language , sign language and musical performance.

Keywords:multimodal interaction management, eye-tracking

Disciplines:Developmental psychology and ageing, Linguistics, Theory and methodology of literary studies