This book, edited by Anthony Baldry (Campobasso: Palladino, 2000), now a collector’s item, brings together twenty research articles presented at international congresses at the University of Pavia between 1994 and 1999. It is concerned with the nature and functions of discourses produced in multimedia environments where different technologies (such as personal computers, television, video and telephone) are integrated for use across a range of educational contexts including distance learning.
One common thread underlying the majority of the research papers in this collection is the systemic functional perspective whereby meaning across semiotic resources is seen to be metafunctionally organised (Halliday 1973; Halliday 1978). In other words, choices from each semiotic resource are seen to simultaneously fulfil the textual, interpersonal and ideational metafunctions. The complexity of multimodal analysis becomes apparent ─ how to theorise and analyse metafunctional choices across resources which integrate to make meaning. In dynamic moving texts such as video, the problem becomes a matter of tracking transitional phases where the salient choices from different semiotic resources shifts over time. However, as the
Introduction claims, the theoretical frameworks afforded in social semiotic theory (Halliday 1994; O’Toole 1994; Kress and van Leeuwen 1996) together with the use of IT now mean that the complexity of multimodal analysis is becoming manageable. This is one of the challenges this volume addresses: the use of computer technology as a tool for dealing with the complexity of multimodal analysis as exemplified by
HyperContext Web, an interactive web-based software program designed and described by Marco Piastra and Luca Lombardi (
The HyperContext Web Project: Dynamic Authoring for Distance Learning).
The articles specifically concerned with intersemiosis and the analysis of the interaction and interdependence across semiotic resources include Anthony Baldry’s discussion of dynamic multimodal analysis and the need for a historical and comparative approach to text analysis, and the use of software to establish specialised multimodal corpora in the
Introduction and
English in a Visual Society: Comparative and Historical Dimensions in Multimodality and Multimediality, Elena Nalon’s analysis of the use of visual images of women’s bodies in perfume advertisements in
Multimodal Meaning Making: Perfume Advertisements and the Human Body, Silvia Porcelli’s discussion of the interaction of language and visual images in cartoons in
Visual and Verbal Resources in Cartoons, Christopher Taylor’s exploration of the use of interactive software in the field of translation in
Text Analysis and Translation: An Interactive, Self Access Computer Application Incorporating a Functional Approach, and Paul Thibault’s comprehensive transcription and descriptive framework for the analysis of moving images in
The Multimodal Transcription of a Television Advertisement: Theory and Practice.
While primarily aimed for students who face new ways of learning English in the electronic medium, the volume is also concerned with exploring a range of issues arising from the multiple forms of discourse afforded by such new technologies. Thus the research into multimodality and multimediality are fused with issues surrounding possible advances in the practices of distance learning, continuing education, linguistics and software development. The essence of the contributions are therefore centred around three areas: first, the use of technology for classroom teaching and distance education across a range of disciplines for native and non-native speakers; second, the theorisation, analysis and interpretation of the range of multimodal discourses produced in such environments; and third, the use of new technologies and software applications to advance the theory and practice of multimodal analysis of both static and dynamic and printed and electronic texts in educational and research contexts. The papers are organised alphabetically in the book due to the diverse range of research interests.
The following is a brief list of the different genres which are investigated followed by specific papers which deal with the problem of interacting semiotic resources in multimodal texts. In the first case, the ranges of multimodal genres which are investigated include economics texts, dynamic and static scientific texts (Anthony Baldry), children’s story telling (Francesca Bianchi), educational databases on the internet (Elena Caldirola), computer mediated conversations (Katia Carraro), CNN news broadcasts (Mauro Marangon) advertisements (Elena Nalon), cartoons (Silvia Porcelli) and television advertisements (Paul Thibault). The use of technology is investigated in a range of educational contexts which include virtual lectures and interactions in the humanities (Elena Cotta Ramusino), technology for teacher training (Marina Dossena), self access modules for distance education (Deborah Lidgley), hypertext to teach poetry (Ivana Marenzi) and scientific texts (Maria Pavesi and Anthony Baldry), development of dynamic authoring system software for distance learning (Marco Piastra and Luca Lombardi), software for teaching English to non-native speakers (Carol Taylor Torsello) and software for text analysis and translation (Chris Taylor). Other uses of technology include specialised corpora (Laura Gavioli) and internet texts (Mersini Karagevrekis) for teaching English for Specific Purposes (ESP), on-line information retrieval for language teachers (Annalisa Golfredi) and the use of corpora for linguistic analysis and language teaching (Maria Antonietta Spanu).
