What is Multimodality?

Multimodality

In this course, Dr Agnieszka Lyons (Queen Mary University of London) explores multimodality. In the first module, we introduce multimodality and how we can use a range of semiotic resources and multimodal modes to create meaning. In the second module, we apply multimodal analysis to the use of gestures is conversation. In the third module, we explore multimodal interaction analysis and visual narrative analysis. In the fourth module, we learn about the social semiotical approach, which stemmed from Michael Halliday’s work around the functionality of language. In modules five, six and seven, we analyse various multimodal texts from the social semiotical perspective, focusing on the ideational metafunction, interpersonal metafunction and textual metafunction, respectively. In the eighth module, we discuss how visuals are used in the news. In the ninth module, we apply multimodal analysis to comic books, which are considered a true multimodal genre.

What is Multimodality?

In this module, we introduce multimodality, focusing in particular on: (i) linguist Guy Cook’s perspective on the importance of language in human life; (ii) contexts where language is insufficient; (iii) the increase in visual communication aided by technology; (iv) the theory of multimodality, which considers communications between people not only in the context of language but also related to other ‘modes’, such as visuals and gestures; (v) the fluidity of modes, which are shaped culturally and socially overtime and are not universal, as seen in an image by E.M. Ellis, Jenny Green and Inge Kral., "Documentation of the verbal arts of the Western Desert people of Australia", Endangered Languages Archive (2010-13); (vi) the development of multimodality from the end of the 19th century to the present via the thinkings of Ferdinand de Saussure, Charles Goodwin, Gunther Kress, Theo van Leeuwen, Kay O’Halloran, Charles Sanders Pierce, and Michael Halliday.