Multimodality and Cognitive Linguistics

Intermedial cognitive semiotics: Some examples of multimodal cueing in virtual environments
Asunción López-Varela
In Multimodality and Cognitive Linguistics. Page 388-401.
Special issue of Review of Cognitive Linguistics 11:2 (2013)
Edited by Maria Jesus Pinar Sanz
University of Castilla-La Mancha
John Benjamins Publishing Company

In this chapter, intermediality is explored from an interdisciplinary perspective that uses neuroscience as well as cognitive-semiotic concerns and insights from online digital communication, presenting it as a process where biophysical, technological, and interpersonal factors interact. Shared attention as well as spatial and temporal cueing – eye contact and the sonic modality – are explored from a task-oriented and social interactive dimension. The spatiotemporal impact of the mediating context is highlighted by moving from the role of visual cueing, in a brief reference to Al Davison’s autobiographical graphic novel The spiral cage, to a more detailed analysis of Annie Abrahams’ (2010) online project A fragmented relation, where cueing is dependent not just on spatial frames but also on the temporal dynamics introduced by the aural dimension recorded in an online environment. The paper tangentially touches upon the role of affect in communication.

Multimodal Metaphor and Intersubjective Experiences: the importance of eye-contact in Davison’s graphic novel ‘The Spiral Cage’ and in Annie Abrahams Net-Project ‘On Collaboration‘
Asunción López-Varela
In Masucci, Lello e Di Rosario, Giovanna, Lavori del Convegno Palazzo degli Artista Italiani. Ofcina di Letterature Electrónica. Napoli, pp. 307-324

Multimodality has been studied within many different disciplines: cognitive linguistic, conceptual metaphor theory, sociosemiotics, interface design, and human-computer interaction. Up to date very few studies concern themselves with the way humans perceive the interplay between the modalities, that is, how do we acquire information and how do we integrate it, an ability termed ‘intermediality’.

Filed under: Articles / Texts, Asunción López-Varela, fragmented relation, multimodality