Image/Text Relations in Narrative and Information Texts for Children in Print and Electronic Media

Department of Linguistics

(Top L to R) Talia Gill, Clare Painter, Len Unsworth, Chris Cleirigh, Jim Martin
(Bottom L to R) Ariane Welch, Sumin Zhao

J R Martin, Linguistics, Sydney
J R Martin is Professor of Linguistics (Personal Chair) at the University of Sydney. His research interests include systemic theory, functional grammar, discourse semantics, register, genre, multimodality and critical discourse analysis, focussing on English and Tagalog - with special reference to the transdisciplinary fields of educational linguistics and social semiotics. Publications include Working with Discourse (with David Rose) Continuum, 2003; Re/Reading the Past (Edited with Ruth Wodak) Benjamins, 2003; Negotiating Heteroglossia (a special issue of Text Edited with Mary Macken-Horarik) Mouton de Gruyter, 2003; Language Typology: a functional perspective (Edited with A Caffarel & C Matthiessen) Benjamins 2004; and Interpreting Tragedy: the language of September 11th , 2001 (a special double issue of Discourse & Society Edited with John Edwards) Sage 2004. He has recently completed a book on evaluation (with Peter White, in press with Palgrave) and a book on genre (with David Rose, in press with Equinox). Professor Martin was elected a fellow the Australian Academy of the Humanities in 1998, and awarded a Centenary Medal for his services to Linguistics and Philology in 2003.

Len Unsworth, Education, UNE
Len Unsworth is Professor of English and Literacies Education at the University of New England. Len was a classroom teacher in Queensland before moving into teacher education at the University of Western Sydney, and then the University of Sydney prior to taking up his present position. He has a PhD in linguistics from the University of Sydney. As well as an extensive number of journal articles, Professor Unsworth has published several books including Literacy learning and teaching (Macmillan, 1993), Researching language in schools and communities (Continuum, 2000) and Teaching multiliteracies across the curriculum (Open University Press, 2001). Two further books are in press: e-literature for children and classroom literacy learning (Routledge) and [with Angela Thomas, Alyson Simpson and Jenny Asha] Teaching children’s literature with Information and Communication Technologies (McGraw-Hill/Open University Press). In 2004 Professor Unsworth was presented with a Citation of Merit Award for research by the Australian Literacy Educators’ Association.


Clare Painter, English, UNSW
Clare Painter is a Senior Lecturer in the School of English at the University of New South Wales, and has an MA (Hons) and PhD in linguistics from the University of Sydney. Her research and teaching interests are in systemic-functional theory, children's language and literacy development, educational linguistics, multimodality and children's literature, all of which bear upon the project. She has published numerous papers on child language and language education issues and is the author of Into the Mother Tongue (Pinter 1984), Learning the Mother Tongue (Oxford U.P., Deakin U.P., 1985/1991), Learning though Language in Early Childhood (Continuum 2000) and co-author (with J.R.Martin & C.M.I.M.Matthiessen) of Working with Functional Grammar (Arnold 1997).


Sumin Zhao, Linguistics, Sydney
Sumin Zhao completed an MLitt in Linguistics at Sydney University in 2003, focussing on image/text relations in contemporary Chinese women's magazines. She has joined the project as a PhD student focussing on history and social studies in print and electronic media.

Chris Cleirigh, Linguistics, Sydney
Chris Cleirigh completed a PhD in Linguistics at Sydney University, focussing on the phonology of Irish and is currently completing a monograph on the nature of physical, chemical, biological and semiotic systems. He has joined the project as a research assistant focussing on science in print and electronic media.

Ariane Welch, Semiotics, Sydney
Ariane Welch is currently completing her Honours program in Semiotics at Sydney University; as part of this work she is focussing on the expression of affect in images in narrative picture books.

This project focuses on image/text relations in print and electronic materials designed for children in primary school and early secondary school. Both narrative and information texts will be considered. The research is informed by systemic functional linguistic theory and its social semiotic applications to other modalities of communication. To date, the main focus of the project has been on relations between images in narrative picture books. Work on the affective dimension of these relations is currently underway. As this theme nears completion our research is shifting to consideration of the mutually informing relation between words and images in narrative and information texts, including CD-ROM and web based materials. Information texts recontextualising science, history and culture for children are being considered, concentrating on themes from the Australia, Asia and the Pacific.

Research Aims & Proposed Outcomes

To develop an account of the ways words and images interact to construct the meanings of narrative and information texts in print and electronic media. To formulate from this account an accessible metalanguage necessary for the development of multiliteracies in primary education. Specifically the project aims to:

• extend our current work on meaning links across images in picturebooks, to include inter-image relations in information books;
• draw on this work and on existing functional grammars of language and image to construct an account of the 'intermodal' grammar and cohesive devices interrelating meanings of images and verbal text;
• determine how this account of intermodal grammar and cohesion would need to be enhanced to account for the dynamic and/or hyperlinked images in multimedia texts;
• develop an educationally accessible metalanguage – a language to enable teachers and students to talk about language and image properties and their meaning-making interactions;
• indicate implications for ways these descriptions might be used in developing children’s critical multiliteracies in primary education.

The expected outcomes of the study will include:

• A model of image/text relationships in the intermodal construction of meaning in print and electronic text formats.
• An educationally accessible metalanguage for describing such semiotic inter-relationships
• Developmental guidelines based on text complexity for mediating the learning of metalanguage as a resource for students’ multiliteracies development.

The benefits of these outcomes include:

• A new framework through which teachers can relate their knowledge of language and image in traditional text forms to the increasing availability of multimodal and inter-media texts as resources for children’s learning and entertainment;
  More inter-state consistency in references to metalanguage in syllabi concerned with literacy development;
• More effective teaching of the new literacy practices required of students dealing with 21st century intermodal texts using an accessible intermodal metalanguage;
• An explicit functional semiotic basis for more effective design of multimodal texts.

Australian Research Council Discovery Project DP0558112

• Martin, J R & D Rose 2003 Working with Discourse: meaning beyond the clause. London: Continuum.
• Martin, J R & M Macken-Horarik [Eds.] Negotiating heteroglossia: social perspectives on evaluation. (Special Issue of Text 23.2, Ed). 2003.
• Martin, J R & R Wodak 2003 Re/reading the past: critical and functional perspectives on discourses of history Amsterdam: Benjamins (Discourse Approaches to Politics, Society and Culture).
• Martin, J R & P R R White in press The Language of Evaluation: appraisal in English. London: Palgrave. in press.
• Martin, J R & D Rose in press. Genre relations: mapping culture London: Equinox.
• Painter, Clare 1984 Into the Mother Tongue. London: Pinter.
• Painter, Clare 1991 Learning the Mother Tongue. Oxford: Oxford University Press.
• Painter, Clare 1999 Learning through language in early childhood London: Cassell.
• Unsworth, L. Ed. 2000 Researching language in schools and communities. London: Continuum/Cassell.
• Unsworth, L. 2001 Teaching multiliteracies across the curriculum: changing contexts of visual and verbal texts in classroom practice. Buckingham: Open University Press.
• Unsworth, L. 2004 e-literature for Children and Classroom Literacy Learning. London Routledge.

• Department of Linguistics & Semiotics Program at the University of Sydney
• Department of English at the University of New South Wales
• Education Faculty at the University of New England