Multimodal discourse reception: ​Approaches and methods

Multimodal research has traditionally focussed on the analysis of texts and their production, resulting in a wealth of theoretical and analytical constructs as well as approaches. On the one hand, this research has pushed the field forward; on the other hand, however, it has resulted in different, and at times contradictory, accounts of how different modes work individually and together in making meaning. One of the reasons for the divergent accounts is a relative lack of empirical evidence coming from the other end of the multimodal communication process, i.e. reception.

Over the last two decades, however, scholars from a variety of fields, including Linguistics, Communication Science, Media Studies and Critical Discourse Studies, have started to take on this challenge by conducting research that can be roughly divided into four main categories (Castaldi, 2025a). Engagement studies aim to investigate how people naturally engage with multimodal texts by looking at data such as comments and reactions to multimodal texts and discourses (e.g. Angouri and Wodak 2014; Way 2015; He 2019; Castaldi and Foundouka, 2025). Recontextualisation studies involve showing a multimodal text to participants and exploring how they recount and evaluate (i.e. recontextualize) what they see (e.g. Forceville 1996, 1999; Infantidou and Tzanne 2006; Abdel-Raheem 2017). Audience research approaches aim to explore the effects of texts on audiences (e.g. Castaldi 2021, 2025b). Finally, experimental studies employ some manipulation of the multimodal input: the data can vary, from task completion (e.g. Cohn and Paczynski 2013; Hart 2018) to psychophysiological measures (e.g. Chua, Boland and Nisbett 2005; Bucher and Schumacher 2006; Holsanova, Rahm, and Holmqvist 2006; Holsanova, 2014; Bucher 2017; Drijvers et al. 2019; Tseng, Laubrock and Bateman 2021).

The aim of the workshop is to provide an overview of these four approaches and to consider the most appropriate research questions and methodological designs to address them.