Digital Humanities and Digital Communication: Multimodality and Transmediality Summer School
Modena, 7-11 June 2021
Ongoing developments in digital technologies have originated a wide array of new media forms often involving multiple semiotic modes (e.g., verbal, visual, aural, spatial, and gestural). Working with multimodal and multimedia technologies as objects of research, tools for research or tools for public communication poses interesting questions. How do multiple semiotic resources contribute to meaning-making? What are the challenges of compiling and analyzing multimodal/multimedia corpora? What are the challenges and opportunities of multimodal analysis and multimodal digital communication?
The issue involves the whole range of social practices observed and produced in the humanities: from content management systems to corporate and institutional web pages, from social media to digital performing arts, including the complex meaning-making processes of face-to-face interaction and the multimodal nature of written texts. The complexity of digital artefacts and of cultural objects in general also requires analytical tools where multimodality can play a role in data visualization or analysis, while ultimately supporting an understanding of complexity.
The current development in media production is also characterized by transmediality, i.e. the possibility of disseminating ‘content’ across as many platforms or media as possible; for instance launching a new (cultural) product over the Internet, live events, gadgets etc. The term “Transmediality” is used here to include different phenomena which are non-media specific, and can therefore be realized in a large number of different media. Originating in the world of literary discourse, the term has extended to a great variety of different discourses where constellations of texts are at play, such as for example cross-media campaigns in political discourse, branding or public science. In the study of transmediality, it is important to take into account not only the dynamics of the multiple media actors involved, but also their interaction with algorithmic personalization and with the affordances of different platforms. These three components together play a fundamental role in shaping public opinion by affecting collective debates.
Over the past few years this social dynamics has been investigated especially using big data and digital methods, highlighting phenomena such as the spreading of misinformation, patterns of opinion manipulation and polarization in public debate. On the other hand, little research has explored the linguistic and semiotic dimensions of these phenomena, determining the need to identify new theoretical and analytical perspectives, such as multimodal social semiotics or multimodal discourse/interaction analysis.