Eye-Tracking in Interaction. Studies on the Role of Eye Gaze in Dialogue

Several subdisciplines and programs in linguistics have demonstrated a long-standing interest in the study of non-verbal communication in relation to speech. The most widely studied form of non-vocal communication is by far (co-speech) gesture, defined as the use of hands, arms and other body parts, typically in conjunction with speech. An ever-growing community of scholars studies gesture, both in terms of its inherent cognitive-functional characteristics (e.g. Cienki & Müller 2008, Kendon 2004) and its relation to the speech channel (McNeill 2005; Streeck 2009, Müller et al. 2013, Hadar 2013, Casasanto, in press). Examples of the institutionalization of this line of research are the Gesture journal and the Gesture Studies book series (both published by John Benjamins), as well as the founding of the International Society for Gesture Studies in 2002(ISGS: http://www.gesturestudies.com/). Within the broadly defined category of gesture, hand gestures have received most attention because of their broad semiotic potential, ranging from highly conventionalized emblems to deictics (e.g. pointing gestures) and non-representational, speech production related beats (cf. Kendon’s continuum, Kendon 1982; McNeill 2000, 2005). The proposed volume, however, zooms in on a different semiotic channel within the visual modality that has been shown to have multiple communicative functions as well, viz. eye gaze in face-to-face interaction. Ever since the publication of early pioneering work by Kendon (1967), Argyle & Cook (1976) and Goodwin (1980, 1981), different disciplines have shown a steady interest in the role of eye gaze in conversation, including conversation analysis, psycholinguistics and research on human-computer interaction. In conversation analysis, for instance, gaze behaviour in interaction is studied among others in relation to participant roles, display of participation, affiliation, activity types and turn allocation (see Rossano 2012 for an extensive overview of the field, with relevant references). Dialogue models in psycholinguistics and research on human-computer interaction have focused primarily on the role of (mutual) gaze in conversational attention and the turn-taking machinery (Novick et al. 1996, Cassell et al. 1999, Vertegaal et al. 2001, Thórisson 2002, Louwerse et al. 2009, Bailly et al. 2010, Jokinen 2010, Hjalmarsson & Oertel 2012). One key methodological issue in the study of eye gaze pertains to the reliability of gaze estimations. Although estimations based on video recordings may be useful for a basic segmentation of the distribution of visual attention (e.g. looking at an interlocutor vs. looking away), they are notoriously coarse-grained and unreliable for more detailed analysis (Kendon 2004, Streeck 2009). For instance, video-based analysis does not provide useful information on short fixations on a target (of 200ms or less), saccades (i.e. fast movements of the eye) and visual scan paths. In order to be able to include this level of detail in the analysis, a different methodological paradigm is needed, viz. eye-tracking. Eye-tracking or the measuring of gaze points and eye movements during online behaviour has established itself as an instructive paradigm for studying comprehension processes involved in a range of semantic and pragmatic phenomena (e.g. by studying an individual reading). The recently developed mobile eye-tracking systems (e.g. table-top eye-trackers or eye-tracking glasses) allow researchers to study eye gaze in naturally occurring interactive settings, in which eye gaze is not only indicative of comprehension processes, but can function as a strong communicative instrument as well (supra). Thus, by broadening the perspective from processing to production, the traditional scope of eye-tracking may also be extended to include face-to-face communication, or the measuring of visual attention distribution in conversation, both on the part of a speaker and a hearer. This opens up a vast area of research currently being explored in (cognitive) interaction studies, including the role of gaze as a directive instrument (‘gaze cueing’, Emery 2000, Frischen et al. 2007, Lachat et al. 2012), shared and mutual gaze in interaction (Richardson & Dale 2005, Brennan et al. 2008, Richardson et al. 2007, 2009, Neider et al. 2010, Broz et al. 2012), the correlation between gaze and gesture (Gullberg & Holmqvist 2006, Gullberg & Kita 2009), gaze as a disambiguation instrument (Hanna & Brennan 2007), interactive alignment in various semiotic channels (Oben & Brône, forthc.), and discourse management (Raidt et al. 2007, Bailly et al. 2010, Jokinen et al. 2009, Jokinen 2010, Hirvenkari et al. 2013, Brône et al. 2013) (see Kreysa & Pickering 2011 for an extensive review of the different strands in eye-tracking-based interaction studies).Despite the above-mentioned growing interest in applying eye-tracking technology to the study of face-to-face interaction, there is to date no publication that brings together the different strands of research in this strongly interdisciplinary program. With the proposed volume, we aim at bringing together high-quality papers from the leading scholars in the field (many of whom have been mentioned above). Envisioned as an ‘innovating reader’, the volume addresses key questions of interdisciplinary relevance (e.g. to what extent can the analysis of fine-grained eye gaze data, obtained with eye-tracking technology, inform conversation analysis, and vice versa?), positioning (e.g. what is semiotic status of eye gaze in relation to linguistic signaling?), and methodology (e.g. can we strike a balance between experimental control and authenticity in setting up dialogue settings with eye-tracking technology?). The exploration of these and other questions contributes to the demarcation of a burgeoning research program.

Jaar van publicatie:2018