Visual Communication 9783110255492, 9783110255485

Citation preview

David Machin (Ed.) Visual Communication

Handbooks of Communication Science

Edited by Peter J. Schulz and Paul Cobley

Volume 4

Visual Communication Edited by David Machin

DE GRUYTER MOUTON

The publication of this series has been partly funded by the Università della Svizzera italiana – University of Lugano.

ISBN 978-3-11-025548-5 e-ISBN 978-3-11-025549-2 Library of Congress Cataloging-in-Publication Data A CIP catalog record for this book has been applied for at the Library of Congress. Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.dnb.de. © 2014 Walter de Gruyter GmbH, Berlin/Boston Cover image: Oliver Rossi/Photographer’s Choice RF/Gettyimages Typesetting: Meta Systems Publishing & Printservices GmbH, Wustermark Printing and binding: Beltz Bad Langensalza GmbH, Bad Langensalza ♾ Printed on acid-free paper Printed in Germany www.degruyter.com

Preface to Handbooks of Communication Science series This volume is part of the series Handbooks of Communication Science, published from 2012 onwards by de Gruyter Mouton. When our generation of scholars was in their undergraduate years, and one happened to be studying communication, a series like this one was hard to imagine. There was, in fact, such a dearth of basic and reference literature that trying to make one’s way in communication studies as our generation did would be unimaginable to today’s undergraduates in the field. In truth, there was simply nothing much to turn to when you needed to cast a first glance at the key objects in the field of communication. The situation in the United States was slightly different; nevertheless, it is only within the last generation that the basic literature has really proliferated there. What one did when looking for an overview or just a quick reference was to turn to social science books in general, or to the handbooks or textbooks from the neighbouring disciplines such as psychology, sociology, political science, linguistics, and probably other fields. That situation has changed dramatically. There are more textbooks available on some subjects than even the most industrious undergraduate can read. The representative key multi-volume International Encyclopedia of Communication has now been available for some years. Overviews of subfields of communication exist in abundance. There is no longer a dearth for the curious undergraduate, who might nevertheless overlook the abundance of printed material and Google whatever he or she wants to know, to find a suitable Wikipedia entry within seconds. ‘Overview literature’ in an academic discipline serves to draw a balance. There has been a demand and a necessity to draw that balance in the field of communication and it is an indicator of the maturing of the discipline. Our project of a multi-volume series of Handbooks of Communication Science is a part of this coming-of-age movement of the field. It is certainly one of the largest endeavours of its kind within communication sciences, with almost two dozen volumes already planned. But it is also unique in its combination of several things. The series is a major publishing venture which aims to offer a portrait of the current state of the art in the study of communication. But it seeks to do more than just assemble our knowledge of communication structures and processes; it seeks to integrate this knowledge. It does so by offering comprehensive articles in all the volumes instead of small entries in the style of an encyclopedia. An extensive index in each Handbook in the series, serves the encyclopedic task of find relevant specific pieces of information. There are already several handbooks in sub-disciplines of communication sciences such as political communication, methodology, organisational communication – but none so far has tried to comprehensively cover the discipline as a whole.

vi

Preface to Handbooks of Communication Science series

For all that it is maturing, communication as a discipline is still young and one of its benefits is that it derives its theories and methods from a great variety of work in other, and often older, disciplines. One consequence of this is that there is a variety of approaches and traditions in the field. For the Handbooks in this series, this has created two necessities: commitment to a pluralism of approaches, and a commitment to honour the scholarly traditions of current work and its intellectual roots in the knowledge in earlier times. There is really no single object of communication sciences. However, if one were to posit one possible object it might be the human communicative act – often conceived as “someone communicates something to someone else.” This is the departure point for much study of communication and, in consonance with such study, it is also the departure point for this series of Handbooks. As such, the series does not attempt to adopt the untenable position of understanding communication sciences as the study of everything that can be conceived as communicating. Rather, while acknowledging that the study of communication must be multifaceted or fragmented, it also recognizes two very general approaches to communication which can be distinguished as: a) the semiotic or linguistic approach associated particularly with the humanities and developed especially where the Romance languages have been dominant and b) a quantitative approach associated with the hard and the social sciences and developed, especially, within an Anglo-German tradition. Although the relationship between these two approaches and between theory and research has not always been straightforward, the series does not privilege one above the other. In being committed to a plurality of approaches it assumes that different camps have something to tell each other. In this way, the Handbooks aspire to be relevant for all approaches to communication. The specific designation “communication science” for the Handbooks should be taken to indicate this commitment to plurality; like “the study of communication”, it merely designates the disciplined, methodologically informed, institutionalized study of (human) communication. On an operational level, the series aims at meeting the needs of undergraduates, postgraduates, academics and researchers across the area of communication studies. Integrating knowledge of communication structures and processes, it is dedicated to cultural and epistemological diversity, covering work originating from around the globe and applying very different scholarly approaches. To this end, the series is divided into 6 sections: “Theories and Models of Communication”, “Messages, Codes and Channels”, “Mode of Address, Communicative Situations and Contexts”, “Methodologies”, “Application areas” and “Futures”. As readers will see, the first four sections are fixed; yet it is in the nature of our field that the “Application areas” will expand. It is inevitable that the futures for the field promise to be intriguing with their proximity to the key concerns of human existence on this planet (and even beyond), with the continuing prospect in communication sciences that that future is increasingly susceptible of prediction.

Preface to Handbooks of Communication Science series

vii

Note: administration on this series has been funded by the Università della Svizzera italiana – University of Lugano. Thanks go to the president of the university, Professor Piero Martinoli, as well as to the administration director, Albino Zgraggen. Peter J. Schulz, Università della Svizzera italiana, Lugano Paul Cobley, London Metropolitan University

Contents Preface to Handbooks of Communication Science series

v

Part I David Machin Introduction

3

Part II

1

Göran Sonesson The cognitive semiotics of the picture sign

23

2

Charles Forceville Relevance Theory as model for analysing visual and multimodal communication 51

3

Ian Roderick Military hardware as affective objects: Towards a social semiotics of militainment television 71

4

Christina Konstantinidou and Martha Michailidou Foucauldian discourse analysis: Photography and the social construction of 91 immigration in the Greek national press

5

Helen Kelly-Holmes Linguistic fetish: The sociolinguistics of visual multilingualism

6

Paul Bowman Sex and Race go Pop

7

Inna Semetsky The visual semiotics of Tarot images: A sociocultural perspective

8

Dennis Puhalla Colour language hierarchy

9

135

153

195

Mary C. Dyson Applying psychological theory to typography: is how we perceive letterforms special? 215

173

x

Contents

Gilles Brougère 10 Toys or the rhetoric of children’s goods

243

Martin Conboy 11 Visual Aspects of British Tabloid Newspapers: ‘Image Crowding Out Rational 261 Analysis’? Gwen Bouvier 12 British press photographs and the misrepresentation of the 2011 ‘uprising’ in Libya: A Content Analysis 281 John A. Bateman 13 Looking for what counts in film analysis: A programme of empirical research 301 Jana Holsanova 14 In the eye of the beholder: Visual communication from a recipient 331 perspective H. Camilla Smith 15 Questioning bohemian myth in Weimar Berlin: Reinterpreting Jeanne Mammen and the artist function through her illustrations Der Maler und sein Modell ‘The Painter and his Model’ (1927) 357 Carey Jewitt 16 A multimodal lens on the school classroom

387

Ross P. Garner 17 Celebrating and critiquing “past” and “present”? The intersection between nostalgia and public service discourses in BBC1’s Ashes to Ashes 405

Part III Roger Sabin 18 Ally Sloper, Victorian comic book hero: interpreting a comedy type Marvin Carlson 19 Visual Communication in the Theatre

445

Vincent Campbell 20 Analysing impossible pictures: Computer generated imagery in science documentary and factual entertainment television 463

429

xi

Contents

Sarah Edge 21 Reading the Fenian photographs: A historically and culturally located 483 study Diane Carr 22 Interpretation, representation and methodology: Issues in computer game 501 analysis Susan Wright 23 The art of voice: The voice of art – understanding children’s graphicnarrative-enactive communication 517 Danielle Almeida 24 The political values embedded in a child’s toy: The case of “Girl Power” in the Brazilian doll Susi 539 Kay l. O’Halloran, Alvin Chua and Alexey Podlasov 25 The role of images in social media analytics: A multimodal digital humanities approach 565 Nathaniel Dafydd Beard 26 From static to dynamic: The changing experience of fashion imagery

589

Nurit Peled-Elhanan 27 The de-humanization of Palestinians in Israeli school books: a multimodal 603 analysis of layout, intertextuality and reading paths William Cannon Hunter 28 Visual communication in tourism research: Seoul destination image

625

Randall Teal 29 Thinking visuals: What the challenges of architectural representation can tell us about visual communication 645 Karel Kleisner and Timo Maran 30 Visual communication in animals: Applying a Portmannian and Uexküllian biosemiotic approach 659 Maximilian Rapp and Markus Rhomberg 31 The importance of Murals during the Troubles: Analyzing the republican use of wall paintings in Northern Ireland 677

xii

Contents

Rupert Cox 32 Transforming art and visual anthropology: Imitation, innovation and 697 inspiration in two Japanese art museums Maria Nikolajeva 33 Emotion ekphrasis: representation of emotions in children’s 711 picturebooks Paul Brighton 34 ‘ The Ocular Proof ?’: Television news and the pursuit of reality Biographical sketches Index

755

745

729

Part I

David Machin

Introduction 1 Introduction For several thousand years there has been scholarly debate about the nature of visual communication, about the nature of seeing and how this differs to language, through Plato, Kant, Freud, Wittgenstein, Peirce and Goodman. And over the past century at least, visual communication has been studied across many fields in different ways, asking quite varied kinds of questions and going about analysis in very different ways: archaeology has studied the meaning of cultural artefacts; anthropology the meaning of gesture and expression in humans and primates; psychology the way the brain and psyche process visual information; the philosophy of mind has analysed the nature of sight itself and of images in the mind; Classics has studied the meaning of classical art and iconography; art history the patterns, techniques and materials of painting; linguistics the study of sign language; semiotics the tools of visual communication; design studies the way to best make information accessible, appropriate and attractive; film and media studies have looked at the nature of the visual industries and the kinds of representations they produced, with a view to the kinds of ways that they construct the world and persuade people. The list could go on and on. Yet a number of commentators point to a more current growing popularity in the analysis of visual communication (Elkins 2003; Pauwels 2012; Rose 2012), across a wider range of disciplines, and increasingly in a way that shows awareness of the need to look outside of our own specialist fields for fresh ideas. It is noted that this is evident in the growing number of academic journals that address the visual which have emerged in the last decade: Visual Communication, the Journal of Visual Culture, Visual Communication Quarterly, and also in the form of journals that deal with the visual side of their own academic field such as Journal of Visual Communication in Medicine, Journal of Visual Literacy, Journal of Visual Communication and Image Representation, Journal of Visual Culture and Gender, Journal of Visual Cognition, Visual Culture in Britain, and Visual Art Practice. To some extent this increase in visual journals should be seen as part of a general trend in both the flourishing of academic journal titles which has, in itself, been part of a wider trend in the increase in pressure for academics to publish, to get promotions, attain tenure or to just hold on to their jobs. Even graduate students realise the need to publish to find a job. This in one part explains the huge profusion of academic journals. But on the other hand it is clear that there has been an actual surge in interest in visual aspects of communication and culture across academic fields, with an increased awareness in the value of interdisciplinarity, and such journals provide sites where scholars are able to publish their work

4

David Machin

and find that of others. And it does appear that each of these more specialist approaches is often able to increase our knowledge of the visual as they come at it with their own specific set of questions and interests. And, as Smith (2008: 7) points out in regard to the field of Visual Studies, such specialisms have grown out of a frustration with what can be the stifling processes of disciplinary policing. Although as a note of caution he does point to the way that such new fields can then themselves become subject to new sets of border controls. There has also been a wave of handbooks and introductory texts, many of which clearly reflect the sense that scholars need to look outwards from their own fields to be aware of the kinds of visual methods and approaches being pursued in other fields. One of the first of these was Van Leeuwen and Jewitt’s (2000) Handbook of Visual Communication, which brought together a number of scholars from different fields to show how they would deal with the visual. More recent have been Spencer’s (2010) Visual Research Methods in the Social Sciences, Rose’s (2012) highly successful Visual Methodologies – which offers a range of visual methods from the humanities, including semiotics, art history and discourse analysis – and Pauwels and Margolis’ (2012) Handbook of Visual Research Methods which brings together systematic approaches to analysis from the Social Sciences and Reavey’s (2011) Visual Methods in Psychology which presents the diverse ways of dealing with visual data in psychology. Each of these excellent volumes presents handbook-style contents from a particular scholarly perspective as do the journals above. For example, Visual Communication and Visual Studies carry different kinds of work, the former with a tendency towards design and semiotics and the latter with a tendency towards cultural theory. So, too, this present volume comes from my own scholarly experience that I will discuss shortly. But what this volume contributes specifically is a resource that gives access to work in visual communication in the form of a larger collection of original research papers written by leading scholars from a much wider range of scholarly fields. These chapters are part of authors’ ongoing innovative research in their own specialism. They are designed to get the authors to foreground their assumptions, theories and methods, making them accessible to non-specialists. Bringing these chapters together has also allowed me to make some of my own reflections on visual enquiry: what this is about, and whether, indeed, there is, or should be, a broader field of visual communication studies, or integrated set of visual methods. It may be one thing to share our approaches and avoid pointless repetition and reinventing of the wheel, but another to expect there to be anything meaningful in suggesting that the huge range of interests and specialist questions that are asked about visual communication could be brought together in one place. From my own perspective, as an academic writing mainly within linguistics, pragmatics and critical discourse networks, I have become aware of the growing interest amongst my own colleagues in the role of the visual. Formerly, those working in the study of grammar, sociolinguistics and stylistics saw their job as

Introduction

5

analysing language. But there was a realisation from the late 1990s that most of the texts they examined contained visual components and that words located in space, such as shop signs and advertisements, could communicate beyond their traditional linguistic sense as they brought with them meanings of global references and higher status languages. For the most part, however, such scholars have found it difficult to move out to the exploration of different kinds of visual methods, preferring those that emerge from within their own networks and which tend to be based on or around theories of language. As happens in other fields where a “new” realm of investigation is “discovered” it can then herald a new flurry of activity that can, to those outside looking in, appear rather arbitrary. New network leaders will emerge in this pioneering area of research. New terminologies will appear to account for the very same things already documented decades before in a different field. Ginette (1982) has written of certain tendencies in academia towards “the passion to name which is a mode of self expansion and self-justification: it operates by increasing the number of objects in one’s purview”, even if terms for these same objects have been established and challenged in other subject areas. In my own field of linguistics, the specialism of “multimodality” has over the last decade seen linguists draw models from their own field to attempt to identify the building blocks of the visual: a visual grammar. But it soon became clear that these scholars were largely ploughing the same furrow as over a century of semiotics, yet still not asking very basic and important questions about the nature of visual signs that had long been standard fare in this long standing tradition (Machin 2009). Pauwels (2012: 4) too has argued that effort is often put into trying to appropriate a field through a process of renaming and imposing field specific theories rather than integrating them into existing work. (p. 4). And it can even be the case that the leaders of the new field begin to neglect existing skills and expertise to take on a weakly informed sense of how others investigate the visual. Visual collections can therefore offer a resource for those who wish to know what has been done on the visual in different fields so that they can have methods to take away with them. If they lay claim to developing a new analytical approach then they can place it next to what already exists out there. Spencer (2008), using the example of the field that calls itself Visual Studies, also makes an excellent case for the way that different sections of emerging academic fields may then go on to produce genealogies, often arbitrarily identifying founding thinkers as a process of legitimising their own movement. Such a process can, of course, exclude as much as they include both important and influential work that suits the cause less well. Elkins (2003), writing about his own field of Visual Studies in the humanities, points to another limitation that can be found within disciplines. He was concerned by the way a limited number of models and topics of analysis seemed to have been settled upon. He suggests that in Visual Studies there has become a

6

David Machin

lazy overreliance on a few theorists such as Foucault, Lacan and Barthes and a very limited set of interests that focus on TV, photographs and the Internet. He also argued that there was too much interest in social meanings and theory rather than form. And indeed it can be quite bewildering, moving from one academic field to another, to see the reliance on one or two theorists that appears arbitrary and isolated. Such observations are important for this collection. We need to be open to different kinds of theories and also to different kinds of instances of visual communication. And from my own perspective I would agree with the view that there has been a tendency to theorise about visual phenomena rather than providing more concrete accounts of their composition, design and purpose. In fact, the huge impact of Kress and Van Leeuwen’s (1996) Reading Images may lie exactly in their attempt to provide tools that allow a much more precise way to describe and document the details of images, suggesting a wider feeling that there has been a lack of this kind of work. Bringing a collection of visual research together in this way, I am also mindful of the comments of Mitchell (1994) who asked the question as to the politics of why we should want to study the visual in a way that is set apart from other forms of communication. What really is the justification and use in analysing the visual apart from other modes of communication, from language, sound and materiality? Most of the communication we come across happens in different modes simultaneously. The chapters in this volume, while they offer an insight into the different kinds of research going on in the visual, may therefore help us to look outwards and share our models, theories and approaches in dealing with gesture studies, painting, architecture, fashion, psychoanalysis, toys, guns, archives, classroom design and memory. But, for me, they also raise the issue of whether it would be meaningful to ever think of these papers as part of a coherent field of “Visual Communication Studies”. Elkins (2003) himself argued that what comes to be dealt with under the rubric of Visual Studies in the humanities is not so much a defined discipline but a broader set of overlapping concerns. In the case of the chapters in this book the overlap may be little more than the fact that the topics of study involve visual perception which hardly seems the making of a concrete field of enquiry given that the visual embraces much of human culture.

2 What is Visual Communication? One way to think more about the meaningfulness of integrating Visual Communication Studies is to consider for a moment what we mean by visual communication. We often hear that our society is increasingly saturated by the visual. Presumably, therefore, it is a society that is saturated by acts of “visual communication” and dominated by the “image”. But what does such an assumption actually mean in itself? Thinking about what this statement glosses over brings us to what this

Introduction

7

volume is about and why it is useful to always be mindful of wider forms of research being done in an area, of the different kinds of questions that get asked, or could be asked, about the visual. In fact the matter of the “image” or “the visual” is itself not as clear as might first be assumed. Images can comprise photographs, abstract paintings, mirror images, the image we see through our eyes, dreams, memories, verbal descriptions. Even writing itself comprises an image, composed of kinds of typeface, alignments, font colour and paper quality. When we hear words, even if we cannot see the speaker, we will be in a particular setting which will lend meaning to those words. And work in Conversation Analysis has considered the way that such context works on understanding. But clearly not all of the phenomena we have listed are visual communication per se. What is important to note from this, however, is that “the visual”, and “the image”, may be less easy to bring together in one place as a field of study, to integrate these in ways that on first sight may seem possible. But this does not mean that we cannot learn from each other, expand our own horizons, by bringing out research questions and processes together in one place for such purposes. When we refer to visual communication we mean acts of representation where on the one hand we have a text maker who uses a set of visual resources to communicate a set of ideas, attitude and values to others. But on the other hand, as semiotics has taught us (Cobley 2010), things like natural phenomena can communicate to us by virtue of their form or cultural meanings such as the moon, or patterns on an animal to mean danger. Visual communication can occur through the clothes we wear, through gesture, a children’s board game, through the way we lay out our homes, through the way houses are built to suggest austerity and conformity or their opposites. Visual communication occurs through computer software, the way weapons are made to look, the design of cars and bicycles, shoes, the way we are taught to look at the sky at night. And it is not so much that humans experience these things through the visual apparatus alone but also through our “haptic sense”. In other words through our physical sense of being in spaces comprised of, and in relation to, these phenomena. All of this kind of communication involves the exchange and promotion of ideas, attitudes and values. For example, windows in schools were once positioned at a high point so that children could not look out, cutting them off from their everyday world. More recently there has been a trend towards more open classrooms with much larger windows, often spanning the whole wall. Each of these choices communicates ideas, attitudes and values as to what education is about and about the identities ascribed to the learners themselves, whether potentially unruly children are to be controlled with perceptions limited, framed and disciplined, or where innocence and imagination are to be given free-reign. Each of these choices has psychological and physiological effects on us as we understand their meanings or feel the experience of living with them.

8

David Machin

In these classrooms, other aspects of visual communication will take place: how the desks are arranged, the gestures used by the teacher, the postures struck by the children at their desks, the importance and prominence given to technology and the colours of objects, furniture, walls and toys. Children’s toys also communicate particular kinds of ideas and values. When I was a child I was given construction toys to play with. In the spirit of the Enlightenment I was tutored in the importance of the rational, to build massive, powerful structures that in some way changed or controlled nature. My own 18 month old daughter has a toy telephone that has a face with huge eyes and wheels encouraging her to make interpersonal connections with the values of both communication and transport. She also has a pair of jeans that carry a large stamp in the manner that a military munitions crate would be marked. We might view such as stamp as simply fashion. But why do we find a military stamp and not one from, say, a crate of fruit or medical equipment? In all of these cases there is communication through visual resources that tell us about how the world is, how we should act in it, and what kinds of identities we should foster. Such military references can point to the values of adventure, “rough and tumble” and durability, as opposed to those of death, maiming, destroyed families or occupation. All of these are instances of visual communication. So when we say society is increasingly visual, what is it that we mean? The claim that we live in a more visual society tends to mean, as Elkins (2003) points out, a rather limited view of the visual. We tend to be talking about photographs and film. And we tend to mean that more variety of visual communication is placed before our eyes. So we come across masses of advertisements each day, showing images of different locations on the planet. We can quickly scan through thousands of images on Social Networking sites, but also more words. News and lifestyle magazines can use more images, but also more words. It is a more communication saturated world in some senses. But is it also more about a changed relationship with the visual, given the possibilities offered by technology, and also as Kress and Van Leeuwen (2001) argue, a changed role of the visual and its relationship to words and language. Most of the texts we now read are made up of bullet points and visually designed – no longer meant to be read out loud but to communicate through a different kind of relationship between words and visual design. My old electricity bill was formerly filled with numbers and writing. My new one has very little text or numbers and includes lots of white space and a diluted blue square containing the words “save money” in a friendly, rounded, forward-leaning font. Here the ease of paying is communicated not through words but through the meaning of visual layout and colour. Diluted colours point to more muted moods as opposed to the emotional intensity of those that are highly saturated. Letter forms suggest something light, dynamic and soft as opposed to those that are more angular, heavy and fixed. Here, these visual components are harnessed instead of words and numbers to point to the need to pay, presenting the facts as something easier and less formal than in the case of the earlier bills.

Introduction

9

Such forms of visual representation are not simply new ways to reveal and show. Tufte (1997) has shown that all kinds of graphical representations used to represent information are not simply value neutral but shape, control and also misrepresent the world they claim to simply record and document. He shows that this can be with sometimes tragic consequences where charts have meant that the way that diseases spread, due to the need for simplification, conceal what actual processes are taking place. Tufte (2003) has made the same criticism of Powerpoint which reduces all issues to a simplified set of bullet points, glossing complexity. Again he points to the consequences of this kind of shaping of information. We also find changing uses of images and of genres of images. In my own work I have looked at the way that lifestyle magazines and news photographs have become more stylised. Formerly, photographs for features, advertising and fashion shoots would be different where the former would appear to bear witness to a specific place at a moment in time. Fashion shoots would be posed and in minimalist settings, advertising would present sensuous and idealised worlds. But now they have become much more alike. The symbolic work of the image in advertising has now moved across to replace the former documentary, or “bearing witness” role of the factual image (Hansen and Machin, 2008). This is not “more” visual communication, but “changing” visual communication, which, like the design of classrooms, speaks about broader changes in cultural values and corresponding transformations in social life. When we say that we live in a “visual society”, an “image-saturated society”, we really need to consider the massive array of communicative activities and artefacts that comprise visual communication. Much of the theory and research that deals with what visual communication is, why it is made, how it communicates and what it does to us, is about images in the form of the photograph and film. But, in fact, a wider view of visual communication is one which does not disconnect the visual from other modes of communication and is the very study of human action and culture. Smith (2008) points out that when we talk about “Visual Culture” and the range of topics, places and times that are included, along with the close interrelationship of the visual with other kinds of communication, this gives a sense that is both “everywhere thus nowhere, wholly over-determined and almost meaningless simultaneously” (Smith 2008: 3). In this collection I have, in the first place, chosen established scholars to represent different methodological approaches and also very different areas of visual analysis. I wanted these to offer a resource in the manner of the collections by Van Leeuwen and Jewitt and Margolis and Pauwels, but which was much more broadly interdisciplinary. I also looked for scholars who were producing innovative work searching out new directions for research from within their own fields. In the second place, I chose these papers to help demonstrate the breadth of work, the great range of questions that are asked of the visual and the difficulty in even pinning “the visual” down. In my own field of linguistics the visual turn to

10

David Machin

multimodality led theorists to claim to be working towards integrated or holistic models of visual communication. Of course this should in part be understood as being part of the tradition in linguistics to create all-encompassing models of language and grammar. But it should also be seen as part of a limited perspective on what comprises visual communication. In the chapters in this volume we look at design, perception, reception, physiological responses to the visual, the visual as something multisensory, the problem of visual truth and biology as visual communication. It is important to share these approaches and questions, but the question remains as to whether it is useful to view these as part of any kind of coherent Visual Communication Studies.

3 Contents of the volume The contents of this collection have been organised into two sections. The first section presents different methodological and theoretical approaches to visual communication and the second presents different topics of analysis. Below, summaries of the chapters are given in the order that they appear in the volume. In each case I have labelled them according to their main theoretical or methodological identity (although many combine several) or by the topic of analysis for the papers in the second part of the volume.

3.1 Chapters in the first section 1.

Semiotics – Göran Sonneson In this chapter the author distinguishes semiotics from other approaches to the visual, such as art history and psychology, since semiotics is concerned with generalities pertaining to all semiotic phenomena – in other words, rules that would apply to all pictures themselves. He explains carefully why these approaches are fundamentally different. He then proposes a model that brings together cognitive psychology and semiotics to explore some fundamental rules of pictures. In order to explore the way that such a model can work he applies the model to cases of abstract art, looking for the way that viewers understand phenomena such as shape, colour and form. This chapter provides a fascinating investigation into what kinds of rules may influence the perception and understanding of this kind of art.

2.

Images and Relevance Theory – Charles Forceville In this chapter the author shows how relevance theory (from linguistics) can be one fruitful way to look at the way that images communicate and how people view them. Relevance theory offers a number of maxims that govern communication to do with such features as being optimally informative, being

Introduction

11

relevant and avoiding ambiguity. This allows speakers and listeners to best grasp ideas, information and attitudes. The author shows how the theory would need to be adapted for the analysis of images, showing how this might then allow us to carry out a more accurate and valuable kind of visual analysis. Which elements of images are most important to analyse? The answer to this question will allow us to work out which parts of images are decodable and which need more background knowledge of cultural conventions. 3.

Multimodality – Ian Roderick This author draws on a developing multimodal approach to visual communication to analyse television programmes that showcase military hardware on the US Military Channel. Multimodality comprises a loose set of observations and rules developed within linguistics which nevertheless allow more precise levels of observation of elements and features of media. Here the author analyses sound, lighting, iconography and interpersonal alignment to show how these television programmes represent guns and aircraft through a kind of pleasurable consumption and reverence.

4.

Discourse analysis (Foucauldian) – Christina Konstantinidou and Martha Michailidou Here the authors use Foucauldian analysis to provide an analysis of photographic representations of immigrants in the Greek Press. This kind of analysis involves looking for the different themes that dominate reporting, revealing the frames of reference used. Here photographs are used to activate such frames in different ways at different times. The paper looks at the visual semantic association of immigration with negativity and asserts that news reporting plays a pivotal role in the production of immigration as a social and political problem and the construction of the immigrant as a multifaceted threat.

5.

Sociolingiustics – Helen Kelly Holmes Here the author is interested in the visual properties of language and its use in branding and advertising. Sociolinguistics has a tradition of looking at the status ascribed to different languages, how they can have different strengths and meanings. She looks at the operation of visual multilingualism in examples such as the use of French words in English market advertisements for Stella Cidre, the cider version of the Belgian lager, Stella Artois. The paper shows that such a theoretical approach is useful for allowing us to rethink the uses of language that are now common in branding, how these trigger and reified different kinds of associations and meanings. It also raises the question of the influence of branding on language status.

12

David Machin

6.

Cultural Studies/Critical Theory – Paul Bowman This author is interested in the way that different forms of representation are welded to different kinds of media technology. Drawing on critical theory, the author shows how a number of concepts address the way we deal with identity and difference. The chapter first demonstrates the ways in which cinema and film in general can be shown to be significant “cultural technologies” and how they have complex effects on culture and identity. It then uses this discussion to analyse two music videos in terms of what they could be said to reveal about gender, sexuality, race and ethnicity in the contemporary, media-saturated world.

7.

Psychoanalytical – Inna Semetsky This author uses Jungian theory and the idea of a kind of collective unconscious, combined with a semiotic analysis, to reveal much about the meaning of Tarot cards. She relates the designs to shared anxieties and shows how these crop up in other media sites, in images that are designated as iconic, such as the smouldering World Trade Centre on Sept 11, 2001.

8.

Cognitive – Dennis M. Puhalla This chapter addresses the way that we perceive colour and how understanding more about colour recognition and processing can help us to understand more about the way we view images and also allow us to improve visual designs. The aim of the paper is to move towards a more systematic understanding of colour relationships. Using statistical analysis, the chapter presents empirical evidence supporting the proposition that strategic colour organisation based on colour’s properties and conventional harmonies communicates a distinct visual message. The chapter presents guiding principles related to colour selection and organization which it argues are utilised in design and strengthen a person’s cognitive ability to extract information, increase retention and assist comprehension.

9.

Design and functionality – Mary Dyson Drawing on methods from experimental psychology where subjects have to assess different types of images, this author is interested in the way that people perceive and discriminate between typefaces in order to provide principles to aid graphic designers in improving and clarifying for the purposes of accessibility of information. These methods are not normally used by designers themselves, but the author shows that they can reveal fascinating insights into the patterns of type perception, including the facilitating of further font tuning.

10. Anthropology – Gilles Brougère This author carries out a rhetorical analysis of contemporary children’s toys as cultural artefacts. In the spirit of anthropology he is interested in the ways

Introduction

13

that such objects offer insight into deeper cultural values and assumptions about childhood. And in the tradition of rhetorical analysis it is a study of the means by which such artefacts communicate these values. He shows how we can connect everyday objects to wider cultural values and how toys construct the nature of play and fun in ways that reveal specific cultural assumptions about childhood and imagination. Amongst other things, he asks why toy animals and telephones may be given wheels. In such cases, what is the cultural value in pushing and pulling? And why are different kinds of animals represented as the same size and with the same textures? 11. Media studies – Martin Conboy This author provides an analysis of the visual style of tabloid newspapers. This is done by an historical analysis that looks at the emergence of the use of visual techniques in the press. He also shows that in order to understand the tabloid visual style we need to place tabloid newspapers within a broader contemporary mediasphere where the visual aspects of the tabloids play an important part in reinforcing broader trends towards an entertainment drivenmedia culture and a marginalisation of rational analysis. 12. Content analysis – Gwen Bouvier This author uses a quantitative method to ask a set of specific questions of a sample of newspapers in order to find out how the uprising in Libya in 2011 was represented in press photographs. The analysis is compared to the actual nature of the events that took place in Libya, showing that the identity of those who sought to overthrow Gaddafi was distorted and simplified and the acts of NATO backgrounded, therefore obscuring the central role of oil in the conflict. 13. Film narrative analysis – John Bateman This chapter draws on functional socio-semiotic theory to look at film narrative. Assessing a number of key theories in film studies, the paper shows that new technologies offer new possibilities to analyse film form by allowing researchers to systematically document instances of contents, styles and structures within film. But the author argues that there have been no theories able to address these kinds of concrete features alongside more abstract interpretations of film. The chapter proposes that suitable motivations can be provided for selecting and grouping quantitatively measurable filmic phenomena. This may then offer a theoretically more robust notion of just what measurable phenomena “count” within a film or films, which in turn may support more precise empirical hypotheses for increasingly fine-grained investigations of the workings of film. 14. Eye tracking – Jana Holsanova Using eye tracking technology, this chapter seeks to understand images from the point of view of how viewers interact with them. It shows that interaction

14

David Machin

with the visuals is dynamic. Not all viewers see the same image content. The author looks at the way that eye tracking can reveal the way that task, goal and purpose of the visual exploration of images play an important part in how we perceive them. And through this analysis it is possible to identify the role of certain design features and the role of expertise, emotion and mental imagery. 15. Art history paintings – Camilla Smith Bringing together an art historical approach to paintings, along with a biographical analysis, this chapter looks at the way that an analysis of a preserved artist’s studio can give insights into the meaning of their work, revealing details about their personal lives and the broader social context in which they lived and worked. The author shows how such details are able to challenge previous interpretations of art and the kinds of artistic community that was thought to create a form of alignment for the artist. The author analyses objects and features of the studio space along with some of the art work produced there. 16. Education studies – Carey Jewitt This chapter looks at how we can draw on the analysis of gesture, visual communication, classroom layout and the use of technology to understand the nature of the classroom and learning experience for school children. The author points out that the relevance of the visual has not traditionally been acknowledged in the teaching curriculum, nor in the way that teachers establish knowledge and learning. She draws on a multimodal approach to visual analysis, along with educational theory, to look at the way that these kinds of visual features can less obviously shape curriculum and knowledge, as well as the classroom practices that underpin learning and teaching in the secondary school. 17. Television and Cultural studies – Ross Garner This author gives a close reading of the visual cues found in a television programme that represents the past. Analysis shows that the past is represented through a limited number of icons and associations and that it is both romanticised and presented in a way that allows the viewer at the same time to alight with a socio-political criticism of a supposedly less enlightened time in British history.

3.2 Chapters in the second section 18. Comics – Roger Sabin This chapter analyses one particular character from Victorian comics in order to understand why he was so popular and considered funny at the time. The

Introduction

15

author describes the analysis as a kind of cultural archaeology. On the one hand this means considering the cartoon in the context of other entertainments forms of the time and in terms of how and where it was read. On the other hand this involves a reading of visual features both in terms of their meanings and in terms of the styles of the artist, as well as in terms of the subject matter of the cartoons. Analysis points to the way that the character, in terms of form and subject, relates to kinds of popular entertainment forms and also to a range of social and political issues of the time such as changing notions of citizenship, unemployment levels and elite attitudes to the poor in times of war. 19. Theatre – Marvin Carlson In this chapter the author notes that much of the way that theatre is debated in scholarly fields is in terms of the language used by noted play writers, or in terms of splendid performances by lead actors. But the true meaning of the word theatron in its Greek origins is to see. The author goes on to look at the importance of the visual in theatre, looking at its historical development. Giving examples, the chapter explores different kinds of theatre space that have been created and through what means this has been accomplished, including simply, but intriguingly, how “stage”, “backstage” and “audience space” are maintained. 20. Digital images in documentaries – Vincent Campbell This chapter looks at the use of computer-generated footage in documentary film and entertainment television in relation to scientific truth. Some science documentaries reconstruct long extinct species or reproduce scenes impossible for the human eye to apprehend, such as images of galaxies lying in deep space. But what does this do to the idea of the documentary as truth or as quality television? And given our cultural emphasis on “seeing as truth”, how do these simulations contribute to, enhance or distort public understandings? The author shows how these new developments in computer-generated footage require us to revisit existing theories of documentary film. 21. Historical photograph collections – Sarah Edge This chapter looks at a photographic archive of Irish people from the mid19th century. The author works from the assumption that photographs are not themselves evidence of history but are themselves historical. Therefore, it is the role of the photographic historian to rebuild the system of the signs which have been captured in the photograph in order to make the meanings they once held “intelligible” to the modern viewer. The chapter shows how a collection of photographs representing Irish people can been seen as reflecting a rather complex set of imperialist values about Irishness at the time.

16

David Machin

22. Video games – Diane Carr This chapter considers video games and what kinds of methods are required to analyse them. It reviews existing theories and considers elements of video games that need to be included in any such analysis. Using the example of a Resident Evil game, the author explains how we need to look at both the visual landscape and characters in the game and the games as playable texts – in other words how visual meaning emerges through play. Simply analysing how the games progress is not enough in itself, even though it may have been the case in older media forms. The author proposes a structural analysis of the game which involves looking at the organisation of its constituting units and the ways in which these units inter-relate in time and space, taking into account that the relationships between these units might not be constant, stable or consistent through play. 23. Children’s drawings – Susan Wright This chapter looks at the way that drawing and talking about their artworks liberates young children to communicate their thoughts and feelings. Whilst also allowing their imaginative, expressive and creative capabilities to surface. The author argues that drawing is the visual equivalent of dramatic play and of graphic speech and that the use of pictorial signs is a fundamental function of early cognitive, affective and social development. The chapter considers examples of children’s graphic-narrative-enactive communications, revealing them as complex thinkers, theory makers, as philosophers, dramatists, technicians, aesthetes and fantasy creators. 24. Advertising – Daniel Almeida This chapter looks at advertisements for a Brazilian girls’ doll. It shows that detailed analysis, based on social semiotics, allows us to draw out the kinds of ideas, values and identities communicated by the doll as it is placed in different kinds of settings. From these observations the author is able to make connections both to the ideology of consumerism and to the changing role of women in Brazilian society. In Brazilian society there has been a move towards possibilities for professional careers, but nevertheless these are signified more through lifestyles and consumer objects than actual concrete activities and through these messages Brazilian women are not encouraged to be anything other than passive. 25. Social media images – Kay O’Halloran, Alvin Chua and Alexey Podlasov This chapter uses socially-created content and social media networks (Twitter, Wikipedia, Foursquare, Instagram) for analysing and understanding different forms of visual communication and social life in urban spaces. Computational algorithms have been used to analyse and cluster the social media data,

Introduction

17

thereby taking a multimodal digital humanities approach. The study found that this approach, which integrated the semantic content across text-based and image-based social networks, allows us to understand more about the roles of semiotic resources in social media – in this case, for language and photos; and, in the future, potentially for video data. 26. Fashion – Nathaniel Dafydd Beard This chapter, written by a scholar working in the fashion industry, provides a sense of the changing visual display of fashion clothing through film, for most part online. Fashion is not so much about how we look and reference visual data, but is very much about escapism, dreams and fantasy. The role of the fashion promoter is to balance this with facilitating a product to appear potentially obtainable and purchasable. The author looks at the changing world of the visual representation of fashion in photography, in movies and more recently in behind-the-scenes programmes – often running alongside fashion shows – which are shown online. The author suggests that fashion communicated through the visual may now be more real than the actual experience of fashion obtainable through visiting stores or being present at live catwalk shows. He discusses the consequence of this. 27. School books in Israel – Nurit Peled This chapter carries out a close analysis of the visual in two text books used in Israeli schools. Carrying out a semiotic analysis of photographs and page composition, the author shows how these book on the one hand present Israel in terms of human rights and the values of dignity. On the other, Palestinian people are represented as sub-human, as dangerous, or often completely deleted from events, while at the same time suppressing any sense that Israel has had an aggressive policy towards them. What this chapter shows is that such an ideology may not necessarily be immediately obvious at first glance but that more careful and rigorous observations which are placed against historical facts are revealing. The use of the visual in such school books clearly plays an important role in legitimising Israeli actions against Palestinians and in tutoring school children to support them. 28. Tourism brochures – William Hunter In this chapter the author, writing in the field of tourism studies, combines theory on destination branding with media theory of pseudo-events, plus the idea of the “tourist gaze” in looking at the South Korean capital, Seoul. The chapter carries out a content analysis of images used in brochures and guidebooks for the city to look at which features and locations are foregrounded and which suppressed. It identifies the main themes that recur in the images – such as modernist city-scapes that serve more as an indication of urban clon-

18

David Machin

ing than anything locally specific – futuristic structures and landscape design, “technology”, “tradition”, and the grandeur of nature. 29. Architecture – Randall Teal In this chapter, the author, an architect, looks at the problem of pictorially representing buildings for planning – in artists’ impressions and in presentations for committees. The problem discussed is that it is impossible to give a sense of the feeling of scale and the kinds of viewing position that actually exist when the structure is completed and related to our “haptic sense” of position in the world. Put simply, we cannot judge a building solely by its visual character. The author looks at different kinds of representations of buildings in images, which attempt to represent them in different ways, considering the strategies by which these overcome the problem. He also questions the way we have become trained to look at such images in a manner that makes us less inclined to look beyond the immediacy that they claim to offer. 30. Animal communication – Karel Kleisner and Timo Maran This chapter reconsiders standard thinking in biology that the appearance of organisms is largely an adaptation serving a survival function. The authors put forward a biosemiotic perspective inspired by theories in philosophical zoology. In the study of animal display and visual communication, biosemiotics focuses on the qualitative perspective in both the logic behind the emergence of such signs as well as their perception and interpretation by other animals. The chapter discusses the development, perception, and evolution of semantic organs such as eyespots on butterfly wings, vertebrate eyes, and other bodily and facial displays. 31. Graffitti and murals in Northern Ireland – Maximilian Rapp and Markus Rhomberg This chapter provides a historiographical reading of mural painting in Northern Ireland. These murals have been one way for communities and political leaders to communicate their aims and their view on events during troubled times. They have been especially important for those groups otherwise prohibited from access to other mainstream media and, of course, before there were other media forms such as social media. They were also ways to mark out territory and declare no-go zones for security forces. The authors look at the murals from the 1980s, showing how they have been used to mobilise new recruits to fight for a united Ireland and what kinds of visual devices and representations were used to do this. 32. Museum layout and display – Rupert Cox In this chapter the author is interested in the role of different senses in the seeing of museum displays. The author investigates “haptic visuality” which

Introduction

19

is a way of seeing that is circumscribed by the sensory environment of the institution and also by the “tactility” of the artwork itself. He also considers this in the context of viewers engaging with reproductions of art in a way that gets them to consider the work not as an original but as a reproduction that borrows from the original not only by virtue of the fidelity of its likeness but through the manner of its reproduction and by its presentation. The chapter considers the way that a reproductioncan allow an art-work to be re-seen. This is one way that we can challenge the authority of either the anthropologist or an arrangement of images to represent a culture. 33. Literary analysis of picture books – Maria Nikolajeva In this chapter the author analyses children’s picture books. Her aim is to consider how various emotions can be conveyed through the interaction of word and image in picture books addressed to young readers. Drawing on ideas from cognitive literary theory, combining recent achievements in brain research with issues in literary criticism – including why reading offers aesthetic pleasure – she investigates how emotions are simulated yet can elicit responses from children as if they were real. 34. News footage – Paul Brighton Written by a journalist, this chapter represents a practitioner perspective on television news. It reflects on the visual representations found in news bulletins. It asks what, ideally, a viewer might wish to expect from news programmes in terms of being informed. It then looks at the kinds of images that are used. The author examines symbolic images and the use of archive footage. He explores how the kind of footage available can also have a huge influence over what is chosen as news. All of this, he suggests, raises issues about trust and authenticity, raising the question as to whether news is more akin to impressionist or modernist painting than it is to a more realist form of painting such as purveyed by Constable or Poussin.

References Cobley, Paul (ed.). 2009. The Routledge Companion to Semiotics. London: Routledge. Elkins, James. 2003. Visual Studies: A Skeptical Introduction. London: Taylor and Francis. Genette, G. 1982. Figures of Literary Discourse, New York, Columbia University Press. Hansen, Anders and David Machin. 2008. Visually branding the environment: climate change as a marketing opportunity. Discourse Studies 10(6): 777–794. Kress, Gunther and Theo van Leeuwen. 1996. Reading Images: The grammar of visual design. London: Routledge. Kress, Gunther and Theo van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

20

David Machin

Machin, David. 2009. Multimodality and Theories of the Visual. In: Carey Jewitt (eds.), The Routledge Handbook of Multimodal Analysis, 181–190. London: Routledge. Margolis, Eric and Luke Pauwels (eds.). 2012. Sage Handbook of Visual Research Methods. London: Sage. Mitchell, William J. Thomas. 1994. Picture Theory: Essays on Verbal and Visual Representation. Chicago: University of Chicago Press. Pauwels, Luke. 2012. An Integrated Conceptual Framework for Visual Social Research. In: Eric Margolis and Luke Pauwels (eds.), Sage Handbook of Visual Research Methods, 3–23. London: Sage. Reavey, Paula (ed.). 2011. Visual Methods in Psychology: Using and interpreting images in qualitative research. London: Routledge/Taylor & Francis. Rose, Gillian. 2012. Visual Methodologies: An Introduction to Researching with Visual Materials. 3rd edition. London: Sage. Smith, Marquand. 2008. Introduction, Visual Culture Studies: History, Theory Practice. In: Marquand Smith (ed.), Visual Culture Studies, 1–16. London: Sage. Spencer, Stephen. 2010. Visual Research Methods in the Social Sciences: Awakening Visions. London: Routledge. Tufte, Edward R. 1997. Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire: Graphics Press. Tufte, Edward R. 2003. The Cognitive Style of PowerPoint. Cheshire: Graphics Press. Van Leeuwen, Theo and Carey Jewitt (eds.). 2000. Handbook of Visual Analysis. London: Sage.

Part II

Göran Sonesson

1 The cognitive semiotics of the picture sign Abstract: The picture is as original a human creation as language, and yet there has not been any study of pictures as such until recently. In the sixties of the last century, the picture was discovered as a subject of study at the same time, but independently, by French structuralism and by perceptual psychology in the tradition of James Gibson. My own contribution has consisted, from the beginning, in bringing these two traditions to bear on each other, adding the inspiration from those few philosophers, notably Edmund Husserl, who have taken a serious interest in pictures. More recently, it has been possible to realise some experimental studies with children and apes from a semiotical point of view. The present chapter describes some of the main methods and traditions of pictorial semiotics and then goes on to illustrate a cognitive semiotic approach, consisting, in this case, of a detailed text analysis of a picture by Rothko, which is based on results from psychology.

1 Introduction Unlike an age-old approach to pictures such as art history, pictorial semiotics is concerned with picture signs as such, not with singular pictures. This means that it aims to determine what is true of every picture, of some more general categories of signs of which the picture signs are instances, such as iconic and visual signs, and of such special kinds of picture signs as paintings and photographs (construction types), posters and picture post cards (circulation types), art and advertisements (function types). This cannot be done by scrutinizing, however much in detail, a single picture. It is possible, of course, to examine closely a whole series of pictures, preferably such as are superficially very different, to draw conclusions about general properties, which can be further probed by adding ever more items to the study. Or, as member of the human species, we may turn our reflection back on our own capacity for understanding pictures. Finally, it is possible to set up artificial real-world situations involving picture understanding to find out what happens in clearly determined cases. Nevertheless, in this chapter, after some preliminary discussion of the relationship of pictorial semiotics to the psychology of pictures, and some further exploration of the methods and models of semiotics, we are going to have recourse to a case study in order to show what is peculiar about semiotics. And to bring home the essential points, we will pick as our subject an abstract work of art.

24

Göran Sonesson

2 Elements of visual semiotics 2.1 Semiotics as a science of generalities According to a by now well-established consensus, there are two kinds of sciences: those that are dedicated to the formulation of rules and laws, and those that study singular facts. The first kind, known as the natural sciences, is supposed to express its findings in the language of mathematics, while the second kind, the humanities, has recourse to more qualitative ways of conveying its discoveries. But something is clearly wrong with this easy identification of two divisions of the sciences: linguistics is certainly concerned with rules and regularities, although it does not generally express them quantitatively. English grammar is about the rules of the English language. General linguistics is about universals of all languages. This also applies to what Saussure called “the other semiotic sciences”. The semiotics of pictures has to tell us what is generally true of all pictures. General semiotics is concerned with generalities pertaining to all semiotic phenomena (cf. Sonesson 2008; 2012). As a contrast, art history is basically about a set of singular pictures, fixed by historical accident. The consensus does not hold any place even for disciplines coming to their own well before linguistics and semiotics: the social sciences. Psychology, notably, may make a more extensive use of measurement than the traditional humanities, but essentially, it is about qualitative generalities, just like semiotics, not simply about mathematical correlations, which is the subject matter of the natural sciences, even when given fancy names such as quarks, black holes, or whatever. When psychology got interested in pictures, just as when semiotics took it up, the picture emerged as a quite different kind of object compared to that of art history. Interestingly, this happened almost at the same time, at the middle of the last century, when James Gibson (1971) first introduced the picture as a specific object of perception to psychology, rejecting the simplistic use of the picture as a substitute for the world of perception, and when Roland Barthes (1964) first presented his pioneering, if largely misguided, approach to a semiotics of pictures. The similarity between these enterprises consists in both being geared to qualitative generalities, at least by implication. Still, they are different. Sonesson (1989) tried to amalgamate these two initiatives. Daddesio (1994) later suggested that semiotics and cognitive science (as a kind of amplified psychology) would need to be combined into cognitive semiotics, semiotics being too absorbed into the study of the objective structures of signs, and psychology being dedicated exclusively to the modes of access to these structures. Since then a lot has been written about cognitive semiotics as a way of bringing together the theories and findings of semiotics and cognitive science (cf. Sonesson 2009; Zlatev 2011). In this sense, what is proposed here is really a cognitive semiotics of the picture sign.

The cognitive semiotics of the picture sign

25

2.2 The methods of semiotics Semiotics is a discipline the aim of which is to lay bare the rules and regularities which govern the use of signs and other meanings in human beings as well as in other species; as well as to discover the emergence of these meanings in evolution, child development, and socio-cultural history. Pictorial semiotics, as a subdomain of general semiotics, has to account for the similarities and differences between pictures and other signs and meanings, structurally as well as diachronically, in the threefold sense indicated above. Semiotics, then, is neither a method nor a model, but it of course makes use of different models and methods. One wellknown model used by semiotics is the structuralist one, according to which everything is organized into binary oppositions, a particular variant of which is the Greimasean Square; and another familiar one is the hierarchy of trichotomies characterising the Peircean approach. The method most commonly used in traditional semiotics is what I have elsewhere called “text analysis” (Sonesson 2008; 2012). It consists is isolating a singular instance from the stream of perception and taking it to be one out of a whole range of (possible) instances obeying the same laws of construction and then to proceed to the exhaustive analysis of this instance by means of a set of general principles. Such a procedure can only be valid if it is applied repeatedly to a series of instances, if the general principles are modified according as new instances yield to the analysis, and if the principles are reapplied to the instances earlier considered, in the light of the new insights resulting for the later analyses. In his study of the Panzani publicity, Barthes (1964) certainly claimed to go through the first part of this procedure; but he never tried out the analysis on other instances to verify the principles. In fact, there is not any other such detailed analysis by Barthes himself using the particular notions defined by him in the Panzani analysis, and although Barthes had a abundant following in many countries, nobody seems to have asked themselves whether new instances of pictures require a modification of the model. Indeed, the vague definitions given by Barthes to such notions as connotation and denotation hardly make it possible to know whether the model fits the picture or not.1 More serious semioticians, such as Jean-Marie Floch (1984; 1986; 1990) and Felix Thürlemann (1982; 1990) may have come closer to applying the same model to several instances of the object studied, but in fact they seem to modify the model in relation to each instance, without taking into account the changes taking place. Thus, for instance, Floch postulates that all pictures must be divided into two main areas, and most of the time he makes the division opposing the right to

 There are clear definitions of connotation and denotation in the work of Louis Hjelmslev, to which Barthes refers, but in fact, he does not use the terms at all as defined by Hjelmslev (cf. Sonesson 1989: 119 ff., 179 ff.).

26

Göran Sonesson

the left part (though not necessarily in the middle). However, there is at least one instance, the analysis of the “Nude” by Eduard Bourbat, where Floch suddenly decides to make the binary division in terms of a piece in the middle and its background, without commenting on this very different perceptual configuration. Different kinds of pictures may indeed require different models, on some level, but these differences should then be systematically correlated with different kinds of picture signs. Perhaps the only extant example of trying out different models and accounting for them in terms of the different nature of the pictures is found in my own early work (Sonesson 1988, 1989a, b). The principles from which text analysis departs are never arbitrary, even though they may have to be modified by repeatedly being confronted to new instances.2 We start out from our intuitions as users of pictures and other signs. Another possibility, therefore, takes the point of departure in the intuitions themselves. In analogy with text analysis, I called this “system analysis”. This is the method often used by philosophers. The phenomenologist Edmund Husserl talked about “free variation in the imagination”. He was thinking of very general facts, like colour being incompatible with the absence of extension. Thus, for instance, you cannot think about a colour without also conceive it as occupying a particular spot in space. A colour cannot have just a hue, a chroma or a saturation, but must occupy a particular place on all three scales. On the other hand, there is no obligation for a cube to have white dots on all its sides (it does not have to be a die). But the same principle is used in linguistics to show that in Japanese /r/ and /l/ are the same sound, since the substitution of one for another does not change the meaning of a word, although it does so in English. “Leap” and “reap” are two different words with entirely different meaning in English. A Japanese would not hear the difference, or he would hear the two words as dialectical varieties of the same word. There are no Japanese words which carry different meanings, which are only different in having the sound /r/ instead of /l/. The kind of reasoning employed by Umberto Eco (1968, etc.) to argue that iconical signs are arbitrary is of this kind, as are, in part, the arguments presented by Sonesson (1989a) to refute this idea. So is also, in a stricter sense, the application of Peircean categories to pictures (Deledalle 1979; Jappy 2010), though imagination is here limited by the hierarchy of trichotomies. This is to say that there is a restriction on the freedom of the free variation in the imagination, for we must always come up with three terms, which are different in a way which is in some way analogous to the distinction between iconical, indexical and symbolic signs.3 As I have described it elsewhere, “text classification” is really a combination of text analysis and system analysis. It consists in deriving a set of concepts from

 In spite of what is said by Hjelmslev and repeated by the Greimaseans. That Hjelmslev’s own principles are not as arbitrary as he thinks is shown by Sonesson (1989a).  On the three types of signs, see the following sections 2 and 3.

The cognitive semiotics of the picture sign

27

the reflection on our knowledge as users of signs and other meanings, and to use these concepts to cross-classify a number of instances. Thus, signs having an absent element or a present element may be cross-classified with signs being combined by means of contiguity or a relation of part to whole, the result being illustrated by four concrete instances. Thus, in a drawing showing Captain Haddock with a bottle instead of a pupil is his eyes, Groupe µ (1992), which introduced this method, would say that there is a presence of a bottle in an unexpected place, and this bottle appears as an integral part of (“conjoined with”) a body where a different part would be expected. The race then is then open to find illustrations of the three other combinations of the dimensions presence/absence and conjoined/ disjoined. Of course, it may be argued, as I have done (Sonesson 2004), that most figures of pictorial rhetoric supposes, at the same time, the presence of one thing expected (the pupils), and the presence of something unexpected (the bottles), and that there are many intermediary cases between complete separation of things (disjointness) and total amalgamation (conjointness), going from mere contiguity to the integration into a whole. This is a different model, but it is still an instance of text classification. In both cases, unlike what happens of text analysis, none of the instances are exhaustively described, but only characterised with respect to the concepts picked out beforehand. Finally, the “experimental method” allows us to draw conclusions about the properties of pictures, by means or constructing an artificial text and studying the reaction to this text, or by having numerous individuals constructing artificial texts under given conditions. There were some examples of this kind of study in early pictorial semiotics, notably by Lindekens (1976) and by Krampen (1983) and collaborators, but on the whole, this method has been little used. A few psychologists, with little notions of semiotics, notably Judy Deloache (2004, etc.) and collaborators, have studied experimentally children’s’ notions of pictures. Recently, I have myself been involved, with several collaborators, in some experimental studies, all forthcoming (except for Zlatev et al. 2013), which study pictorial understanding in children and apes (e.g. Hribar et al. in press), as have one of my students (Lenninger 2012).

2.3 The models of pictorial semiotics The first model, if it could be so-called, of pictorial semiotics was of course that used by Barthes in analysing the Panzani publicity. Serious pictorial semiotics begins with a criticism of this model, which was accomplished more or less independently by the Greimas school (notably by Floch), by Fernande Saint-Martin, by Groupe µ, and by myself (Sonesson 1989a: 114 ff.). The resulting models, however, are fairly different. The Greimasean model preserves the structuralist heritage of binary oppositions. According to Floch, all pictures may be separated into two parts (the left and right part, but not necessarily divided at the middle, or else

28

Göran Sonesson

perhaps the central and the marginal part), and sometimes one or both of these parts may be separated into two subparts. The separation is supported by a series of binary oppositions. For instance, there may be vertical lines and dark colours in one of the parts, and the opposite in the other part. At a later stage, each of these oppositions may be put in opposition to others, forming the Greimasean square. We may end up with continuity opposed to discontinuity, both opposed to non-discontinuity and non-continuity. The model of Groupe µ (1992) is rhetorical: they start out from what is taken for granted and study the different figures resulting from the divergences to such expectations. Thus, for instance, if Captain Haddock has bottles instead of pupils in his eyes, there is a presence of bottles that is not expected in the context of eyes, and the bottle is somehow amalgamated with a human body. Or, contrary to what we would expect in the Lifeworld, one and the same figure contains parts which are normally parts of a cat, and parts which are normally parts of a coffee pot (a work by Julien Key). Saint-Martin’s (1987) model is based on the forms discerned by Gestalt psychology as well as the shapes defined by mathematical topology (neighbourhood, closure, seriation, etc.). She and her followers divide the canvas into many squares and count the figures and colour spots filling in each of the squares. She also takes note of the placement of figures in relation to the two diagonals, which can be construed on each picture. In fact, Saint-Martin’s model is a series of different tools that may be applied to pictures. It gives more the impression of being a varied tool box than a model with a specific research interest. However, all the parts of her model are clearly geared to the perceptual appearance of the pictures. It is no accident, I think, that the pictures analysed by her and by her followers are all (as far as I know) abstract paintings. My own model recognizes the limited, but real, value of structuralist models, but is based on perceptual and cognitive psychology more generally. The autonomy postulate characterised the Greimas school, as well as Groupe µ at the beginning. According to this postulate, semiotics, like Saussurean linguistics, should create its models without taking into account knowledge gathered within other sciences. The autonomy postulate is implicitly rejected by Saint-Martin, as well as in the later work of Groupe µ, and explicitly so, in my work (cf. Sonesson 1989a). This means that insights resulting from, for instance, the cognitive sciences can by used and further verified in the application of semiotic models. The systematic implementation of this idea is cognitive semiotics. An important result of semiotics proper, however, which seems to have been discovered independently by the Greimas school and by Groupe µ, is the division of the standard pictorial signs into two layers: the iconic function, which is thought to be the feature by means of which the picture depicts one thing or another, normally corresponding to a fragment of a scene as perceived in real experience; and the plastic function, which is the meaning conveyed by the picture as a surface on which colour spots are disposed in a particular way, i.e. the picture

The cognitive semiotics of the picture sign

29

considered as it if had been a piece of abstract art. The discovery consists in realising that this latter function is found in all pictures. Indeed, as Floch points out, studying the plastic function of any picture amounts to treating figurative pictures as abstract art. Something that is, on one level, a sun or a flower, and even a human body huddled together, may on another level appear as a circle or another roundish shape.4 It seems that for the Greimas school, as well as for Groupe μ, iconicity is that aspect of the picture that creates the illusion of seeing a scene from the real world, as given in direct perception, while the plastic layer, whose expression is constituted by the flat surface of the picture, generally is taken to embody more abstract meanings. However, this is then no more the iconicity in Peirce’s sense, which serves to define a relationship between an expression and a content, and that can be applied to both the pictorial function, which creates the illusion of seeing reality of perception, and to the plastic function, whose contents are generic concepts, having the actual properties, not the illusory properties, of the flat surface as expression. While what is here called the iconic function connects relatively concrete expressions and contents corresponding to familiar objects within the individually perceptible world of our experience, plastic language, such as we just characterized it uses shapes whose properties are abstract, to convey to us meanings often being at even higher levels of abstraction (“roundness” for “nature”, etc.). Then the latter may well be iconic, to the extent that they are synesthetic and/or physiognomic, which is, as will be shown in the next section, part of the secret of abstract art. To avoid confusion, however, it is important not to use the same term for two properties which do not have to coincide: we will call the rendering of perceptual appearances the “pictorial function”, and oppose it to the “plastic function” which has to do with the meanings of shapes and colours. The pictorial function is always iconic, but there are also other types of iconicity, as will be seen in the following.

3 A case study: “Without title” by Rothko For an art historian, the “Untitled” of 1951–55, by Mark Rothko (one of many “Untitled” of his production), is not only a single object, essentially incomparable to any other object in this world, possessing its own particular “aura”, but a unique event, located in space and time, and assigned to a specific individual. All these determinations may certainly be reassessed from the semiotic perspective (but then they acquire a new meaning), and the historian may, while continuing to pursue his particular purpose, borrow elements of method from the semiotician. However, for the semiotician (if he is not a disguised art historian), Rothko’s work cannot consti-

 For a full overview of the history of pictorial semiotics, cf. Sonesson 2010a.

30

Göran Sonesson

tute a purpose in itself: it is only a means, or better yet, a pretext of an analysis. In effect, it can only be used to illustrate concepts derived from other studies, or to suggest very provisionally their existence.5 More specifically, however, we will use this example to introduce many of the current issues of pictorial semiotics. This means that this case study will be more than ordinarily theory-laden.

3.1 Pictures and other iconic signs The picture is certainly a sign in the sense defined by Husserl and Piaget, as I have shown elsewhere (Sonesson 1989a; 2010b), that is, the expression of the picture, which is directly given but not thematic, is doubly differentiated from its content, which, while given in an indirect manner, is the thematic part. It is in effect as pictures that many works of art become objects of semiotic study, because the picture is a type of sign, and the sign remains a central part of semiotics, even if we admit that the latter has to deal with meanings also in a broader sense. Although the picture is also a carrier of another kind of meaning, plastic meaning (as we saw in 1.3 above), it becomes a picture sign by virtue of embodying the picture function, which is of course a kind of iconic sign function, but conveying iconicity in a special way. What is peculiar about this kind of iconicity is that it produces the illusion of perceiving the content of the sign transposed directly onto the level of expression, as a kind of quasi-perception. You look at the photograph of your loved one, and in a sense you see her before your very eyes, although you know that she is back in Mexico for the moment. But you don’t see her as if she was really here, nor as if she were a ghost or she had been tele-transported to this very place. She is, as Hussserl put it (quoted in Sonesson 1989a), “perceptually imagined”. This should be sufficient to distinguish pictoriality from the kind of iconicity in which Peirce was mostly interested, the one manifested in mathematical formulas and diagrams. For instance, the “x” which appears on both sides of an equals sign in an equation only fulfils it function because it stands for the same number in both cases, even though we do not know which number that is to begin with. To give a another example of this abstract kind of iconicity, think of the way in which the general ideas of “up” and “down” are designated in Blissymbolics (a visual sign system conceived as an international language of communication but in fact used with aphasics) which consists in putting a line over ‾ and under _ the line of writing on which the other signs are placed. No quasi-perception is needed, only the grasping the general idea of the line of writing standing for the whole of the world of our experience. Nevertheless, a corollary of this notion of pictoriality is that different parts of the content in question can be located in distinct parts of the expression, but only  The first version of this analysis was published in French in Sonesson 1994.

The cognitive semiotics of the picture sign

31

with reference to the whole of the expression corresponding to the whole of the content, so that a space is reserved in the expression where the perceptual properties of the content that do not appear in the corresponding expression can be perceived as missing at their place (cf. Sonesson 1989a: 301 ff.). Thus, in the cat/ coffee pot, there is a part that is seen as a spout when the whole is defined as a coffee pot but as a tail when the whole is seen as a cat. But not only that: we can see where the whiskers are lacking on the cat. Nothing of that is necessary or even possible in the case of equations or Blissymbolics. Clearly, “Untitled” is not a picture, since it does not perform a pictorial function in this sense. Its semiotical character therefore remains, for the moment, undetermined. It is certainly possible to attribute a pseudo-pictorial function to “Untitled”, for which the perceiver has to take most of the responsibility, as in the clouds or the stains on the wall that Leonard talked about. Since the title is “Untitled”, we are free to “anchor” the forms however we want. It has been suggested to me that one may recognize a portrait of a Russian soldier, whose fur cap is rendered by the black spot. Alternatively, one could see a landscape with clouds, the highest of which is a harbinger of storm. In a similar vein, Chave (1989) has tried to derive certain of Rothko’s works from a series of Renaissance paintings, and even from Rothko’s own previous figurative paintings. At a more theoretical level, Floch (1984) has applied the same reasoning, in the case of a work of Kandinsky, and would perhaps do the same, in the case of Rothko’s work. However, even if Chave’s argument would turn out to be historically correct, this does not tell us anything at all about the meaning of abstract works. The reasoning of Floch, although more sophisticated, must necessarily presuppose a redundancy between the pictorial representation, and the other non-pictorial layer, which only exists in abstract paintings, plastic language (cf. 1.3 and Sonesson 1989a: 152 ff.). In the end, it is to the plastic layer itself we need to turn, in order to investigate its semiotical nature. There is much more to iconicity than the pictorial function. According to Peirce’s definition, there is iconicity when the sign is based on the properties that two objects entertaining a relationship have as such, independently of one another, even if these properties are not visual, or even perceptual (cf. Sonesson 1995). Remember that to Peirce, the “x” of a mathematical equation is iconic, because the whole sense of an equation depends on “x” each time standing for the same number. This is a fact of mathematics, which might be conveyed by other means than visually available formulas. But Peirce even claims that Franklin and Rumford are icons of each other, not because of any accidental similarity of appearance, but because both are Americans. Iconicity thus to Peirce has a much wider scope than is nowadays appreciated – and I think there is every reason to follow him in this. Iconicity is simply the relation between two items whose list of properties are compared and found to coincide in part – and some of those coincidences are taken to be salient. In this perspective, there is still the possibility of

32

Göran Sonesson

discovering an iconicity other than pictoriality in a work such as “Untitled”. Indeed, we will see later that “Untitled” is an eminently iconic sign.

3.2 Surfaces with markings Although he was a psychologist, Gibson’s contribution to the study of picture was mainly of the kind called above system analytical. As he observes, a picture is first of all a way of disposing marks on a surface, in other words, a surface treated in a certain way. He proceeds to draw a distinction between two broad sets of pictures depending on how the marks have been transferred to the surface: they are “chirographic” if the marks were produced with the hands, possibly prolonged by a simple instrument, such as a pen, a pencil, or a chisel; and they are “technographic”, if the marks were created with the help of a complex tool or a machine, a camera, a video, a computer, etc. (Gibson 1978: 229 f.; 1980: xii ff; Gubern 1987: 46 ff.). In this sense, all pictures are indexical, signs based on the adjacency between the expression and the content, although this content is not always identical to the content of the iconic sign, as it is in the case of the photograph (cf. Sonesson 1989b). One could even argue that by becoming less iconic works of art become more and more indexical: the paintings of Jackson Pollack, for example, are both traces of “actions”, and “expressions” (of moods). Now these observations apply already less easily to Rothko, although it is possible to observe the directionality of brushstrokes in different portions of “Untitled”. In describing the picture as a surface treated in a particular way, Gibson contrasts it to two other types of surfaces that have undergone treatment, the ornament and the surface covered with mud and dirt. As I have noted elsewhere (Sonesson 1989a: 255 ff.), the ornament and the dirt spots are opposed as order to disorder, whereas the picture obeys a principle of organization derived from another space, that of the referent. When observing something ordered, we tend to ascribe the treatment undergone by the surface to an active and conscious agent, while the disorderly traces would rather be attributed to an instance of nonpremeditation. But it is certain that, for an archaeologist from another civilization, including from our own past, a work of Pollock would not be a “picture”, but a dirty surface, while a painting by Balla, Mondrian, Vasarely, or some representative of Op art, would be classified as an ornament. The case of Rothko remains somewhat intermediate: the archaeologist would perhaps take it to be a halfpainted wooden fence. The basic question is really about the difference between a board painted blue and a work of Yves Klein: the first is nothing, but the second, although it is not a picture, carries a meaning. The difference may be largely social: we know that, according to Mukařovský (1974), the artwork is an artefact, which acquires a real life only from the moment that it is seen by someone who realises a “concretisation”, filling in the empty indeterminate spaces of the work. Every act of produc-

The cognitive semiotics of the picture sign

33

tion and concretisation of an artefact is overdetermined by the standards, cannons, and repertories of exemplary works current in a given society. Following the Russian formalists, the Prague school also observes that the standards are perpetually renovated: a new standard is introduced whenever the previous standard is unable to produce works that break with established patterns of perception (cf. Sonesson 1999). In this sense, each artefact maintains an opposition to earlier works that are governed by other standards, as well as, more subtly, to works that, within the same normative system, are considered to be exemplary. Elsewhere, I have suggested three main ways of distinguishing types of picture signs: we may separate them on the basis of “the mode of construction, the channels of circulations, or their socially defined purposes” (Sonesson 1989b). Basically, construction has to do with technography as opposed to chirography, and all their subcategories. The channels of circulation separate a picture post card from a poster in the street: they reach us on different trajectories, more or less often and more or less actively. The socially recognized purposes (which may be different from the purposes of a particular user) serve to tell apart genres such as publicity pictures, pornography, and caricature. At least in certain historical periods, there is an expectancy in particular societies for specific construction types, circulation categories and purpose-defined genres to go together: thus, the prototype of art in the 19th century was the oil painting normally reaching us at the artistic exhibition and having the purpose to create an experience of beauty. Today, however, art is a very complex category, whose socially recognized aims are very unclear, and whose channels of circulation are relatively well defined, at least since the genesis of the notion of art in the modern sense. And yet the history of modernism has consisted in an incessant upheaval of the construction categories conceivable within the space defined by the socially recognized purpose of art as well as of its channels of circulation (cf. Sonesson 1999). When Duchamp drew a moustache on a reproduction of Mona Lisa, he did nothing that had not been done before in cartoons; the innovation consisted in circulating this work in the channels defining art, and ask for it to be recognized as an object realizing the socially recognized goals of art. Like most modernists, Rothko often spoke of what he wanted in terms that recall the formulas of the Russian formalists (1989: 123 ff. Even Chave refers to Sklovskij). The meaning of “Untitled” is probably also to be found in its relations with non-modernism, and in particular to modernist movements prior to its creation. “Untitled” participates in the great modernist denial of the pictorial function. In this sense, “Untitled” can be said to form part of a series of artefacts that also comprehends pictures. Given the expectancies at the time of its creation, its lack of pictorial function, and its particular way of lacking such as function, was conspicuous. Yet it may be more pertinent to ask how it is opposed to earlier and contemporary modernist works. Instead of representation, painters such as Mondrian and Malevich introduced a kind of personal ornament: the order and the repetition

34

Göran Sonesson

take the place of the referential illusion. In opposition to this standard, other artists, such as the Dada, rather quickly invented a kind of disorder. We recognise here the two kinds of surfaces that Gibson opposes to the picture. However, comparing them, we discover that even the Dadaist chaos has a finite character, which disappears in the works of American abstract expressionism. In the case of Rothko, however, it will seem that the dirty surface hides an ornament.

3.3 From prototypes to Gestalten There are four “bodies”, or basic forms, in Rothko’s work, all four belonging to the same type, the rectangle: namely, the shape created by the entire space inside the real or imagined frame (R1), the black shape on top (R2), the red form at the bottom (R3) and the white form between the black form and red form (R4). The latter form can possibly be analysed into two similar rectangles (R4a/R4b). There are also some elements that lack sufficient consistency to form “bodies”, the orange background and a few lines that do not create enclosures (see Figure 1). We will come back to these. But first we must justify the name “rectangle” applied to forms that seem quite irregular, with the exception of the totality and, in a way, the three lower limits of the red form. These forms are not properly speaking rectangles, but they refer to the rectangle as their limiting case or, as I will now say by adopting the terminology of cognitive psychology, their prototype. In a way, this is still the Gestalt, as defined by the Gestalt psychology, but we now know from Rosch’s (1973) studies that the configuration is a special case of the prototype, which is the form usually taken by all thought. Indeed, we do not normally think in terms of categories defined by necessary and sufficient criteria. Instead, a category is characterized by the best example that manifests it. Around the most characteristic case are

Fig. 1: Basic forms.

The cognitive semiotics of the picture sign

35

situated the other members of the class at different distances. Thus, the sparrow is a prototypical bird, because it can fly, is oviparous and has feathers, wings and a beak, while the chick, the ostrich, the penguin and the kiwi, that do not possess all of these properties, form part of the same category, while being relatively far from its central zone. The square, the triangle, the circle, and the other “good forms” are categories in the same way (cf. Rosch 1973, 1978). However, already at the time when the emphasis was still placed on the perceptual character of configurations, Sander and Volkelt (1962) discovered the principles of operation of the concrete instance compared to the ideal prototype: if the distance between the occurrence and the prototype is small, the difference is not observed; next, it will be seen as a not quite perfect example of the configuration in question; and even further from the ideal, it can be experienced as in an equilibrium between two different prototypes. The latter case, realised for example by the Mandala or the super-ellipse of Piet Hein, is found more often at the cognitive level than that of perception. Indeed, in the real world of our experience, at least as it is filtered by our culture, certain properties tend to appear together: they are correlated. It is therefore, as Rosch says, the most likely correlations of properties that define the prototypes. One might think that the least probable correlations of properties must have a role to play also in a theory of meaning. Rosch does not mention it, but it seems that these correlations exist in structuralist analyses, from Levi-Strauss to the school of Greimas (cf. Sonesson 1989a: 90 ff.). The complex, neutral and contradictory terms of these analyses are mostly not logical contradictions; nevertheless, they bring together unlikely combinations of properties. I will call such a least likely correlation of properties an “antitype”. In the present case, only the totality (R1) appears to be within the limits allowed for a perfect configuration. Because of its quasi-adhesion to the lower boundary of the rectangle of the totality, the red form (R3) also is relatively close to the prototype. However, a greater degree of deviation from the prototype can be observed in the black rectangle (R2) and in the two white rectangles (R4). At this level, however, one can hardly speak of anti-typical characters (which I have yet found in abundance in other analyses, cf. Sonesson 1988, 1989a, b), unless a tendency to form an ellipse cannot be recognize in the white rectangle on the right (R4b). According to Rosch’s research, the forms are not only more or less prototypical, but they also are oriented in directions more or less conforming to the prototype. For example, an animal or a car is more naturally conceived from the longitudinal side, while we tend to imagine a human being from the front. Similarly, a rectangle is first conceived as resting on its extended side. This means that the single rectangle in “Untitled” whose shape is prototypical, the total form (R1), has an orientation that is not, while the red rectangle (R3) comes closest to the ideal orientation. As for the other rectangles, they show different types of deviation, the white rectangles (R4) mainly because of their positions being slightly slanted, and

36

Göran Sonesson

the black rectangle (R3) because its baseline is out of sync, which means that they cannot serve as a base on which the shapes rest.

3.4 The prototypicality of the colours We have already noted, implicitly, that all the rectangles have different dominant colours, or, to be more exact, that different chromatic dominants predominate (none of the areas being completely homogeneous) in each rectangle.6 For now we will ignore the non-dominant colours, and other factors that determine the phenomenological colours (brightness and saturation, cf. Groupe μ 1992: 73 ff.), to concentrate on the prototypicality of the dominant colour. Indeed, Berlin and Kay, Rosch herself, and other researchers have been able to show that colour is prototypical in two ways: first, because each colour has a focal, or prototypical, area, and then because some colours are more prototypical than others (see Holenstein 1985). Each dominant colour, just like a prototype, is characterized by good examples around which are ordered more or less marginal cases. Thus, our red rectangle (R3) is really orange with a red tinge (opposed to the background with its orange having a yellow tinge), while our white rectangles could more accurately be described as cream-coloured. But this only means that the instances manifested are situated at a greater or lesser distance to the prototypes of red and white, respectively. More substance is given to this assertion, however, by the fact that the chromatic dominants themselves are more or less prototypical. We know that if a language distinguishes only two colours, these will always be “black” and “white”, and if a third colour is added, it is always “red”. However, black and white are not colours per se, but extreme levels of saturation, which means that they represent the non-colour, opposed to red, which is the colour to represent all colours, the macro-red, as it has been called (see Holenstein 1985: 36). In this perspective, “Untitled” shows relatively prototypical cases of macroblack and macro-white, representing the central instances of all “cold” and “warm” colours, and the macro-red, which is the perfect representative of chromatics. It is tempting to see, also, in the orange background, a somewhat deviant case of the second dominant colour introduced (or third, as the case may be), yellow. However, I think that this shade of orange is too far from the focal area to function as a yellow at present: on the contrary, in the context, it rather represents a developed stage, the introduction of this colour being among the latest. For now, however, we will merely summarize the characteristics of prototypicality of the four different “bodies” that we distinguished in “Untitled” (Table 1).  Speaking of colour, it must be remembered that we refer here as elsewhere, not to the original work, but to the reproduction from the Tel Aviv Museum, which is anyway the only means by which most of us can have access to the work in question.

The cognitive semiotics of the picture sign

R1 R2

Shape Prototypic Far from prototypic

R3 R4

Close to prototype Rather far from prototype

Orientation Non-prototypic Divergent in relation to the prototype Prototypic Divergent in relation to the prototype

37

Colour Prototypic scale Prototypic Close to prototype Close to prototype

Tab. 1: The prototypic relations of the rectangles in “Untitled”.

4 The system behind it all So far, I have made use of the notion of prototype, derived from Rosch, which has a direct ancestor in the notion of Gestalt. In the following, our analysis will be even more theory-laden, both because it will invoke other theoretical models, and because it will open up for further studies which can only be realised by means of experimental methods: those involving hierarchies of prototypicality, as they develop in the child.

4.1 The hierarchy of visual shapes So far, I tried to isolate the configurations, but we can not stop there. Cognitive psychology and studies of child development clearly show that perception does not start from simple elements, but from complex, holistic properties. However, the configuration is merely a particularly strong case of holistic properties, the kind having an internal organization and clear boundaries in space (cf. Sander and Volkelt 1962: 43 ff.; Sonesson 1989a, 81 ff.). We know, however, that in children’s first experience more diffuse properties that nevertheless refer to the totality, and not to “simple properties”, make their appearance. From this point of view, an old study by Lotte Hoffmann (1943) on the interpretation of drawings by children is highly relevant. The task assigned to participants was to reproduce a series of simple figures, with the help of some elementary objects that could be combined in different ways. The “texts” of these interesting experiments consisted of the reductions that children had the models undergo. In interpreting these studies, I discovered a fragment of the hierarchy of visual properties, comparable from this point of view to the hierarchy of phonological features proposed by Jakobson (1943, cf. Figure 2 and Sonesson 1989a: 173 ff.). However, although the prototype concept was unknown at the time of Hoffmann, we see that the results (and those of Jakobson) are amenable to interpretation by means of this concept. Thus, the most highly prototypical shape, in the same sense in which red is the colour prototype, is the circle, specifically the circle that is perfectly round and solid. This is the shape that was used by the children when there

38

Göran Sonesson

Fig. 2: Shapes and visual properties.

was no correlation between the model and the figure created, i.e. to represent “any object whatsoever”, but also later, when other shapes to which it is opposed had been isolated, to represent roundness, in opposition to what is straight, stiff, or angular. At this point, the circle, as an ideally round object, becomes the representative of any object containing an element of roundness. What we have just been doing, reducing the areas of “Untitled” to prototypes, is, in principle, comparable to what Hoffmann’s children were doing when they used combinations of elementary objects to represent simple figures. As adults, we have access to a much more elaborate system of distinctions than that employed even by the older children studied by Hoffmann. The system underlying the work of Rothko seems more “primitive” than the one we normally use, but it is not as elementary as our hierarchy. Thus, the “equivalents” created by Rothko – to use his own terminology – are not circles, or any round shapes, but straight shapes, the prototype of which, according to Hoffmann’s study, is not the rectangle or the square, but the straight line. On the other hand, Rothko’s forms are “solid”, i.e., they do not contain contours, but colour spots. We see at once that they are different also from our prototypical case of solid figures, because their boundaries are not clearly defined. For now, however, it is important to remember that Rothko’s shapes (here and in fact often elsewhere) are discontinuous, straight and solid, and, therefore, fairly elementary, while also representing something closer to a developed state, i.e., to “civilization”. Even when reducing a figure to its closest prototype, as I emphasized elsewhere (Sonesson 1988), we also need to account for the distance between the instance manifested and the prototype. Indeed, a configuration can be decomposed into a series of global features that, without exhaustively explaining the Gestalt effect, serve to support it redundantly, as well as other traits that are in opposition to the sense of the configuration. I have already indicated, above, that

The cognitive semiotics of the picture sign

39

the orientation and colour of certain rectangles may be perceived as traits that are in opposition to the sense of the global configurations formed by the rectangles. But we must go further by noting that the elementary constituents of the rectangles themselves are opposed to the overall meaning of the rectangles. With the sole exception of the whole rectangle (R1), whose boundaries are straight, all components of the rectangles of “Untitled”, i.e. the lines, manifest traits whose overall sense is opposed to that of the configurations: they are more or less undulating and diverge from the vertical and horizontal coordinates of the surface. In fact, they are not even lines, since there is no sudden change in colour, but actually fields of transition. The effect is also supported by the existence of irregular areas of a third colour that often accompany the limits. This is indeed also true of the red rectangle (R3), whose three “straight” lines disappear at a closer look (which does not invalidate the overall impression). Unlike the configurations that they help to form, these traits go in the direction of softness and blur whose prototypical form is the circle. Hoffmann’s experiments also suggest that the rectangle itself is more “round” than the straight line, since the closure that makes the difference between the line and the rectangle is usually represented by the circle. These observations are somewhat paradoxical. The total configurations (except the rectangle of the totality, R1) are all on the side of the straight line, but all the holistic non-configurational features (except those of R1) return to the circle. Between configurations and properties, there is thus an anti-typical relationship. I have already had occasion to observe similar examples (cf. Sonesson 1988), but in these cases, the divergence was supported by an explicit opposition between a circle and a square. Our hierarchy suggest, however, that the opposition may exist even if it is not manifested directly in the text, in the form of a contrast. But before returning to this fact, we must consider some details of Rothko’s work that we have until now put in brackets.

4.2 An excursus on signs and grounds According to Piaget, topological relationships characterize the first space perceived by the child (cf. Saint-Martin 1987). Elsewhere, I have proposed to reduce the basic concepts of semiotics to the simple relations of topology that are neighbourhood, separation, envelopment, nesting, order, and continuity (cf. Sonesson 1989a: 96 ff.). One could see in the neighbourhood a kind of magma of perception, the closest semiotic denomination of which is indexicality. This may be the moment to remind the reader of the classical division of signs made by Peirce (following upon many other thinkers who made similar distinctions in other terms): icons, indices, and symbols. In very general (non-Peircean) terms, “iconicity” is the relation between two items that is based on (an experience of) similarity between one or several properties of the items. “Indexicality” is a relation founded on contiguity (or neighbourhood) existing between two items or

40

Göran Sonesson

properties thereof. If such a neighbourhood exists between two things that are experienced as being parts of the same whole, we may prefer to call this relation factoriality (as between Captain Haddock’s eye and his pupil; see 1.3 and Sonesson 1989a, 49 ff.; 1998), but it will still pertain to indexicality.7 Finally, if there is neither similarity nor contiguity between the two items involved, but some kind of regularity, which may be a simple habit, an explicit convention, or something in between, we have to do with “symbolicity”. Peirce thinks of all these as signs, because his notion of sign is very broad (as he realised late in life), more or less equivalent to any kind of meaning, including perception. Expanding upon an earlier tradition (which goes at least as far back as Augustine), I have suggested (cf. 2.1) that a sign is a kind of meaning which is differentiated in the double sense of the expression not going over into the content in time or space (as it does in perception), and being experienced as of a different nature (not another banana, however far from the prototype, but only a sign for it); and where one item, which is directly given, is not in focus, which instead the other item is, which is only indirectly given. Iconicity, indexicality, and symbolicity as characterized above then clearly apply, not only to signs, but to all kinds of meaningful relationships, to begin with perception. Using a term employed early on by Peirce, I would say that iconicity, indexicality, and symbolicity are grounds, i.e. they are the point of view taken on a relation between two items, which may define signs, in case the two items, according to other criteria, form signs. The neighbourhood may be currently perceptible and actually perceived, as in the case of an arrow pointing the way, or it may have existed at a time anterior to the time of perception, as in the case of footprints, or photography. If these indexicalities become signs (according to the criteria defined above), one can speak of “performative” or “abductive” indices, according as the neighbourhood is created by the sign itself, or, conversely, is a condition for the use of signs (cf. Sonesson 1989b; 1998). Indeed, an arrow or a pointing finger works by creating a neighbourhood which did not exist before to the thing they point to, but the footprints can be interpreted because we know beforehand about the relationship between feet and the different kinds of surfaces on which they are susceptible to leave their marks. There is no independently existing category of pointed-out objects, but there certainly is a category of feet, which (if we take this only to mean human feet) can be differentiated into big feet and small feet, male feet and female feet, flat-feet and feet of traditional Chinese women, etc. The former are performative, because they create the relationship they are about (following Austin’s classic definition); they latter are abductive, because they rely on our ability to draw conclusions from one singular fact to another on the foundation of habitual relationships (following Peirce’s characterization of abduction).

 In topological terms, this is a difference between neighbourhoods and envelopments.

The cognitive semiotics of the picture sign

41

4.3 The indexicalities of the areas Indexicality, whether it becomes a sign or not, is often an important contributing factor in the sense of visual signs performing a pictorial function. While representing the appearances of the real world, the picture normally only shows a part, the rest being indicated by means of abductive indices. If we recognise the scene, or even the type of scene, that is shown in the picture, we can anticipate, by abduction, more or less how the scene continuous outside the margins of the picture. Even an abstract work may contain abductive indices permitting us to complete shapes, such as configurations, which appear only in part on the surface, e.g. a truncated circle. Even if the object is not recognized as a specific object of our world of experience, we still tend to think of a shape, for instance a circle, which reaches the margins of the canvas, as continuing, in some sense, outside it. In “Untitled” no such indices are found, however, since all shapes appear therein completely, the background being, by definition, the only exception, since the background can have no clear boundaries, continuing beyond the painted surface as well as behind the figures. However, the latter effect is amplified in “Untitled”, because of the lack of density of the layer showing the painted background “behind” the different coloured areas. Unlike the indices mentioned before, this is the result of a juxtaposition of signs within the painted space: it is therefore a performative index. Indeed, Rothko’s work includes several other performative indices, which are the result of other juxtapositions of colour areas, the black colour showing through behind the white, the cream colour appearing at the edge of the red rectangle, and even a suggestion of white being visible behind the black area. In the present context, these indices seem to indicate (more or less real) continuity to other colours behind those that are manifested, thus adding a sense of depth to the composition. It is for reasons of indexicality that the white area seems closer to us, and the red area farther away. The boundaries of the rectangles are blurred and permeable at depth, just as we have seen before that they are on the surface. There are “passages” between the areas in all dimensions, not, as prescribed by Cubism, in a few places, but at all borders. All other rectangles are included in the rectangle of the totality (R1), with which they, therefore, have a relation of factoriality, i.e. they entertain a relation of parts to the whole and/or vice-versa. However, as noted elsewhere (cf. Sonesson 1989a, 43 ff.), in the relation of part to whole, it is often not indifferent which part is concerned. Thus, two of the enclosed rectangles (R2 and R3) occupy the respective ends of the rectangle of the totality, while the third (R4) is situated slightly above the horizontal axis at the centre of the composition, or more accurately, in the middle of the third horizontal band resulting from an imaginary decomposition of the work into four bands (see Figure 1). The red and black rectangles, on other hand, go slightly beyond the limits of the bottom and top band, respectively, of such a decomposition.

42

Göran Sonesson

Leaving aside a number of other possible decompositions, we will now attend to some contiguity relations subsisting between the rectangles. The whole composition is characterized by a quasi-parallelism between the different figures. Thus, the three sides of the black rectangle (R2), and the red rectangle (R3), are in parallel with the sides of the totality (R1). In addition, the three enclosed rectangles form a series of figures of the same type and with the same orientation whose dominant sides, namely the horizontal sides, are parallel. On the other hand, the enclosed rectangles reinforce the dominant dimension of the totality, the verticality (R1), the continuity of which is taken up by their short sides, relayed by the red line along the orange background on the left, echoed by the remnant of a line almost parallel to the right. Between these last two lines, one could possibly perceive a ghost rectangle (R5), much like the red rectangle below, but the right boundary is in fact so vaguely indicated, that it is more reasonable to imagine the composition being unbounded to the right. However, it is the relative degree of contiguity of these different rectangles that will be important later: while the red rectangle (R3) is, apparently, securely anchored to the limits of the totality (R1), the black rectangle (R3) maintains a certain distance, more pronounced in the upper right, and less visible on the left (which reinforces the effect of opening to the right), and the white rectangle floating freely in the air. We will therefore tend to assign a more fixed position to the red rectangle, and a more mobile one to the white rectangle. We could have gone further into the study of the parts of a whole, by building a tree reflecting the relations of inclusion and dependence that unite the different parts together. Indeed, when studying the “Blue Nude IV” by Matisse (Sonesson 1989a, 310 ff.) and the “Nude” by Billy Brandt (unpublished workshop), I was able to discover very interesting facts concerning the differences that may exist between the relations of constitution and dependence of real bodies and those found in the bodies reproduced in the pictures.8 But what makes this kind of analysis less profitable in the present case is not merely the absence of a real world object with which we can compare the work, more fundamentally, it is the absence of internal organization of the “body” of the elemental composition, except for the white rectangle (R4), which actually consists of two rectangles, whose difference is too small to allow us to determine a dependency relationship. Leaving aside the relationship tree, we will therefore go in search of oppositions and identities, the organization of which we will try to lay bare.

4.4 Oppositions and identities The opposition, centrepiece of any structuralist semiotics, is a neighbourhood of a particular type, namely a coupling in which the two entities have properties  Comparable models were independently created in cognitive science by, for instance, Marr and Biedermann, but only with reference to real perception.

The cognitive semiotics of the picture sign

43

that, from one point of view or another, are seen as being opposed to each other. As has often been noticed about the phonological oppositions, it is only on the basis of a fundamental identity that differences can be seen. The opposition is a neighbourhood in which the identical features of integrating members are taken for granted, while their opposite traits are accentuated. By reversing the proposition, we rediscover the identity (the “mini-isotopy” of the Greimasians), in which traits that are not similar retreat to the background. Contrary to what one tends to think, all oppositions are not of the same type as those of phonology (cf. Sonesson 1989a, 76 ff.; 1998). Thus, it is necessary to distinguish between oppositions in praesentia, or contrasts, and oppositions in absentia, or oppositions in the narrow sense, according as both terms are present in the work perceived, and the proximity between them is real, or only one term appears in the work in question, the other existing only in the consciousness of a group of people who are able to understand the work. While being in absentia, oppositions may be systemic, if they existed in a system prior to the work, i.e. within a set of units and rules regulating the combination of units (such as phonological oppositions), or intertextual, if they existed prior to the work in question in a previous work (this is the case of an picture that refers to another picture while modifying it). One must also distinguish regulatory oppositions, that add meaning to already constituted units, from constitutive oppositions, creating the units that they define (which are nothing other than the terms of these oppositions). According to Jakobson, the oppositions between the phonemes are constitutive (“purely negative”), while those that can be found between words are regulatory. In the case of a picture, it is usually possible to identify the objects represented, and certain peculiarities of their mutual relations, regardless of oppositions, which, therefore, merely serve to regulate a set of meanings already constituted elsewhere. While analysing “Untitled” above, I have had no recourse to explicit oppositions of any sort, contrary to what I have considered necessary in the case of other analyses (cf. Sonesson 1988; 1989a, b). If my analysis is not entirely wrong, we must conclude that the contrasts contained in the work are not constitutive, but regulative, and that, moreover (as I have already suggested), the oppositions are either systemic or intertextual. Although “Untitled” is not properly speaking a picture, it could no doubt participate in an intertextual opposition, possibly of a specific nature, and certainly of a generic type. While the work in question could be related to a Renaissance painting, as Chave (1989) has attempted to do with other works by Rothko, the differences between the two works, which remain quite large, should be studied as carefully as the identities (cf. Sonesson 1989a, 76 ff.). On the other hand, it is certain that the work in question (and all the works of Rothko) maintain rather close relations with the works of modernist painting prior to its creation, and in particular, with Cubist works. We will return to this aspect later on.

44

Göran Sonesson

Before continuing, it should be noted that, more than the oppositions, it is the identities that are immediately apparent here: All areas directly perceived are rectangles of different extensions and colours, approaching more or less, as we have noted above, the ideal prototype. It is in fact on the assumption of these identities that we can find the fundamental differences in the approximations of these forms to the prototypical rectangle and its orientation (Figure 2), as well as the relative prototypicality of the relations of these forms to the background (Table 2). Let us, for now, attend to the systemic constitutive oppositions, and then move on to the regulatory contrasts. There can be no question here of oppositions specific to this painting, or to Rothko’s work: on the contrary, the relevant oppositions pertain to the system of the visible, i.e. to the pan-human organizations of colours and shapes. In this system, a first relationship opposes black and white, which together are opposed, as a non-colour, to macro-red, the colour par excellence. Finally, red as a prototypical, and thus elementary, colour is opposed to orange as a marginal or elaborate colour. These oppositions exist prior to the work of Rothko, and probably to all of human culture, but it is quite remarkable that all its terms are manifested in “Untitled” (the first two oppositions in full, the third being represented by its extreme cases). Furthermore, the way they are located in the picture, and the mass attributed to each term, prove to be highly relevant: black vs. white occupies the upper half of the surface, the large amount of black contrasting with the exiguity of the white colour; the lower half is occupied by the contrast between the prototypical colour and marginal colours, the respective masses being almost the same, although the orange phantasm-rectangle is overflowing on top. The opposition between colour and non-colour, on the other hand, seems to be represented more particularly by black vs. red, whose special relationship is induced by their almost equal masses and their symmetrical positions in relation to the rectangle of the totality. This shows how a regulatory contrast, specific to the work in question, may be constructed on the basis of a universal constituent opposition (see Table 2). Indeed, we will see later how a constitutive opposition, projected onto two differently localized units, creates an expectation of other correlations between these units, which can be realized in the form of contrasts and oppositions, according to the logic of the prototype and the antitype. We certainly know less about the constitutive system of the perception of shapes than about that of colours, but researches by Hoffmann, Sander and Volkelt, Köhler, Arnheim, Lindekens, etc., allow us to formulate some hypotheses. Let us start with an initial opposition, which we can call the macro-strait vs. the macro-round, associating with the first term not only that which is straight but also the angular, jagged, impermeable, and the digital and/or discrete (e.g. the contours), etc., and assigning to the second term not only round things but all that is undulating, with gradual and blurred limits, the blurring of colours, closure, etc.

The cognitive semiotics of the picture sign

45

Tab. 2: Oppositions and identities between the different rectangles in “Untitled”.

As we have already observed, “Untitled” only realizes one of the terms of this opposition at the level of configurations, the rectangle that represents the macrostraight, but there is no reason to doubt the availability, for any subject of perception, of the opposition in question. In terms of features, however, it is the other term of the opposition, the macro-round, which is plentifully, though not exclusively, realised, and as it manifests itself in the features contributing to forming

46

Göran Sonesson

the macro-straight configurations, the contradictions reveal themselves as being intrinsic to the forms as such, as we shall now see.

4.5 The contradictions and the meaning In the world of our experience, as we noted above, some properties tend to appear together. It is was precisely as being the most likely correlations of properties that were defined the prototypes. However, the researches on colour universals, as well as the studies on shapes conducted by Hoffmann, suggest that at an elementary level of development the properties of the world are perceived in bigger chunks, in conglomerations whose probability ties are thicker than in the adult world. Beyond the circle, there is the macro-round, in the same way that the colours dissolve into the macro-red. One might therefore speculate that in abstract art, and in the world of children, the expectation of a coincidence of related properties appearing in large blocks is greater than in representational art and in the adult world, perhaps simply because our familiarity with everyday things contribute, in the latter two cases, to moderate our expectations. Unlike the child, however, the artist has the opportunity to play on these expectations. By consulting the tables of opposition in “Untitled” (Table 2), we will indeed discover that all rectangles contain internal contradictions: while all configurations are on the side of the macro-strait, almost all their holistic nonconfigurational properties propound the macro-round redundantly. The only exception is the rectangle of the totality, R1, whose features realize, in almost all cases, the macro-strait, just like the configuration to which they belong. All other configurations maintain an adversarial relationship with their own features, not in a logical sense, but in the way of antitypes, i.e. they realise the least probable correlations of properties within a hierarchical system of proto-forms. So there is non-coincidence of the configurations and their features. Since the colours are very prominent in this composition, one wonders naturally whether there is also non-coincidence between the colours and shapes. However, we do have at this moment the means to answer this question: the lack of coincidence, if there is one, can neither take place at the level of shapes, nor at the level of colours, but must occur at a level common to both, i.e., the level of meaning, both of the opposition macro-strait vs. macro-round, and the oppositions of colours. Let’s start with the opposition between the macro-strait and macro-round, exemplified by the two figures, “takete” and “maluma”, one angular and one round, whose names are always correctly attributed, because the sounds “mean” the same things as the figures, which form a distinction proven experimentally by Köhler (1929) and confirmed more recently by Ramachandran (2011: 75–116; cf. Sonesson 1989a, 163 ff.). Most of the properties appearing in the plastic language of “Untitled” are iconic, to the extent that they are synesthetic and/or physiognomic. This is to say that they involve a kind of iconicity that spans the different

The cognitive semiotics of the picture sign

47

sense modalities.9 This is the sense in which “Untitled”, although it is not a picture, is still largely iconic. Moreover, many experiments show that of the two corresponding prototypes, the circle is seen as the Urform, as something elemental, natural, soft, warm, generous, quiet, happy, dynamic, female, something that protects and embraces, while the rectangle appears as something more elaborate and developed, hard, cold, serious and studious, static, adult and male (see Sander and Volkelt 1962, Arnheim 1966; Lindekens 1971; Sonesson 1989a: 163 ff.). This is obviously true of two “nebulae of sense,” which are hardly comparable to the linguistic content, or the fairly well-defined content of pictorial language. Some of these meanings, however, are far from arbitrary: round shapes are indeed more “basic” and therefore “natural” than straight forms, as they emerge before others in the hierarchy of child development. The function of the contrasts, present in the specific “texts”, is to bring out the potential iconicity, which is there along with many other features of another sort. Since the colour system appears to be more developed, we can not expect to find a complete homology between the latter and the shapes. Indeed, there is no basis for positing a stage “before” the opposition of black vs. white, the two instances being equally “elementary”, and although red is more “elaborate” than the two non-colours, it is more “elementary” than orange. So we will suppose, at least temporarily, a scale whose first member is the pair consisting of black and white, followed by red, and much further on by orange. We therefore find, in R1, a range of colours, with a predominance of “primitive” colours, which is in opposition to its rather “elaborate” shapes; in R2, there is coincidence between the “natural” colour and the shapes having the same predominance, and the same is true in R4, while the balance of R3, which contains extreme traits of macro-strait as well as of macro-round, lean a little towards the “elaborate” pole, because of its colour being relatively “elaborate”. Finally, R5, considered a phantom rectangle, is fully contradictory, its “highly elaborate” colour being opposed to the configuration with open contours on the right side. Armed with these correlations, we can try to find the “drama” to which this composition is, according to Rothko, an “equivalent”. It must be remembered that, among the three rectangles, R4 realises most fully the relationship between form/ background, and R3 does it quite well, while R2 performs this function with difficulty. This should perhaps be seen as a reading order, starting with R4, and ending with R2. If so, we will proceed from a kind of Urform disguised in an elaborate state, quite free in relation to the totality, to a more stable form, but having conflicting content, leading on to a form whose prevalence is again on the “natural” side but which is better integrated into the whole. But without doubt it is possible to “read” many other “dramas” in these same nebulae of sense.  As I was careful to note in 3.2., iconicity involves the experience of similarity (and no doubt we should add this qualification also to the other grounds): it only exists in the peculiar human niche of the world, and, in some cases, in some particular socio-cultural varieties thereof.

48

Göran Sonesson

5 Conclusion One task, in this study, was to relate a number of general concepts, or models, to Rothko’s work called “Untitled”, in order to explore the possibilities of these models, and to gather ideas for the construction of different models. We have seen that the prototypes, as had been suggested by previous analyses, play an important part in this kind of work, and that oppositions, identities and indexicalities also find their place. We also explored the function of anti-typical relations, which turned out to be quite different from what we have found elsewhere, and we ended with some suggestions that need to be developed in further analyses, the most important of which is the differentiation of the modes of signification of surfaces beyond the simple model of iconicity and plastic language. While subordinating Rothko to the goal of semiotics, I have not yet been able to avoid saying something, however fragmentary, which may throw some light of this particular work. From a semiotical point of view, however, the aim was double: using theoretical insights already gained, and discovering other issues that remain to be probed. At another level, this study has a much wider scope. It was intended to show, not only that singular cases, in semiotics, serve to offer intricate issues permitting the advancement of theory, rather than the particular case ever being subserved by theory, but also that semiotics has the means to explain, at least to some extent, the meaning of those dirty and/or ornamental surfaces which our culture call abstract art. Beyond that, the intention was also to demonstrate that semiotics does not have to dwell in its own autonomous heaven, but may practise free intercourse with all kinds of scholarly endeavours. This is what, with a rather awkward terms, we call cognitive semiotics.

References Arnheim, Rudolf. 1966. Towards a psychology of art. London: Faber and Faber. Barthes, Roland. 1964. Rhétorique de l’image. Communications 4: 40–51. Also in Roland Barthes, 1982. L’obvie et l’obtus, 25–42. Paris: Seuil. Chave, Anna. 1989. Mark Rothko. New Haven/London: Yale University Press. Daddesio, Thomas C. 1994. On minds and symbols: the relevance of cognitive science for semiotics. Berlin: Mouton de Gruyter. Deledalle, Gérard. 1979. Théorie et pratique du signe: introduction à la sémiotique de Charles S. Peirce. Paris: Payot. Deloache, Judy. 2004. Becoming symbol-minded. Trends in cognitive sciences (8)2: 66–70. Eco, Umberto. 1968. La struttura assente. Milan: Bompiani. Floch, Jean-Marie. 1984. Petites mythologies de l’œil et l’esprit. Paris: Hadès. Floch, Jean-Marie. 1986. Les formes de l’empreinte. Périgueux: Pierre Fanlac. Floch, Jean-Marie. 1990. Sémiotique, marketing et communication. Sous les signes, les stratégies. Paris: PUF. Gibson, James. 1971. The information available in pictures. Leonardo 4(1): 27–35.

The cognitive semiotics of the picture sign

49

Gibson, James. 1978. The ecological approach to visual perception of pictures, Leonardo 4(2): 227–235. Gibson, James. 1980. A prefatory essay on the perception of surfaces versus the perception of markings on a surface. In: Margaret Hagen (ed.), The perception of pictures, volume I: Alberti’s window, xi–xvii. New York: Academic Press. Gibson, James. 1982. Reasons for realism. Selected essays of James J. Gibson. Edited by Edward Reed and Rebecca Jones. Hillsdale/London: Lawrence Erlbaum Associates. Groupe µ. 1992. Traité du signe visuel, Pour une rhétorique de l’image. Paris: Seuil. Gubern, Roman. 1987. La mirada opulenta. Barcelona: Gustavo Gili. Hoffman, Lotte. 1943. Vom schöpferischen Primitivganzen zur Gestalt. München: Beck’sche Verlagsbuchhandlung. Holenstein, Elmar. 1985. Sprachliche Universalien. Eine Untersuchung zur Natur des menschlichen Geistes. Bochum: Brockmeyer. Hribar, Alenka, Göran Sonesson, and Josep Call. in press. From sign to action: Studies in chimpanzee pictorial competence. Semiotica. Husserl, Edmund. 1939. Erfahrung und Urteil. Prag: Academia Verlagsbuchhandlung. Jakobson, Roman. 1942. Kindersprache, Aphasie und allgemeine Lautgesetze. Uppsala: Uppsala University. Jappy, Tony. 2010. Regards sur le poème muet. Perpignan: Presses universitaires de Perpignan. Krampen, Martin (ed.). 1983. Visuelle Kommunikation und/oder verbale Kommunikation. Hildesheim: Olms. Köhler, Wolfgang. 1929. Gestalt psychology. New York: Liveright. Lindekens, René. 1971. Eléments pour une sémiotique de la photographie, Paris/Bruxelles, Didier/Aimav. Lindekens, René. 1976. Eléments de sémiotique visuelle. Paris: Klincksieck. Lenninger, Sara. 2012. When similarity qualifies as a sign. A study in picture understanding and semiotic development in young children. Lund: Lund University [Dissertation]. Mukařovský, Jan. 1974. Studien zur strukturalistischen Ästhetik und Poetik. München: Hanser Verlag. Ramachandran, V. S. 2011. The tell-tale brain: a neuroscientist’s quest for what makes us human. 1st ed. New York: W.W. Norton. Rosch, Eleanor. 1973. On the internal structure of perceptual and semantic categories. In Timothy Moore (ed.), Cognitive development and the acquisition of language, 111–144. New York/ London: Academic Press. Rosch, Eleanor. 1978. Principles of categorization. In Eleanor Rosch and Barbara Lloyd (eds.), Cognition and categorization, 27–48. Hillsdale: Lawrence Erlbaum Associates. Saint-Martin, Fernande. 1987. Sémiologie du langage visuel. Sillery, Québec: Presses de l’Université du Québec. Sander, Friedrich and Hans Volkelt. 1962. Ganzheitspsychologie. München: Verlag C.H. Beck. Sonesson, Göran. 1988. Methods and models in pictorial semiotics. Lund: The semiotics project. Sonesson, Göran. 1989a. Pictorial concepts. Lund: LUP. Sonesson, Göran. 1989b. Semiotics of photography. On tracing the index. Lund: The semiotics project. Sonesson, Göran. 1994. Les rondeurs secrètes de la ligne droite. A propos de “Sans titre” de Rothko in Les Nouveaux Actes Sémiotiques 34–36, 41–76. Sonesson, Göran. 1995. On pictorality. The impact of the perceptual model in the development of visual semiotics. In Thomas Sebeok and Donna Jean Umiker-Sebeok (eds.), The semiotic web 1992/93: Advances in visual semiotics, 67–108. Mouton de Gruyter, Berlin/New York. Sonesson, Göran. 1998. Index – Indexicality – Opposition. Entries in: Paul Bouissac, in collaboration with Göran Sonesson, Paul Thibault and Terry Threadgold (eds.), Encyclopedia of Semiotics. New York: Oxford University Press.

50

Göran Sonesson

Sonesson, Göran. 1999. The Culture of Modernism. In Marie Carani and Göran Sonesson (eds.), VISIO 3, 3: Modernism: 9–26. Sonesson, Göran. 2004. Rhétorique du monde de la vie. In: Anne Henault and Anne Beyaert (eds.), Ateliers de sémiotique visuelle, 83–100. Paris: PUF. Sonesson, Göran. 2008. Beyond methods and models. Semiotics as a distinctive discipline and an intellectual tradition. Signs – International Journal of Semiotics: 277–319. Available at http://vip.iva.dk/sis/index.php?journal=signsandpage=articleandop=viewandpath%5B%5D= 17andpath%5B%5D=51. Sonesson, Göran. 2009. Here comes the semiotic species. Reflections on the semiotic turn in the cognitive sciences. In Brady Wagoner (ed.), Symbolic transformations. The mind in movement through culture and society, 38–58. London: Routledge. Sonesson, Göran. 2010a. Pictorial semiotics. In: Thomas Sebeok and Marcel Danesi (eds.), Encyclopedic dictionary of semiotics, 737–763. Berlin/New York: De Gruyter Mouton. Available at: http://www.degruyter.com/view/EDS/pictorial_semiotics?rskey= UXxz91andresult=1andq=anddbq_0=Pictorial%20semioticsanddbf_0=eds-fulltextanddbt_0= fulltextanddbop=and. Sonesson, Göran. 2010b. Semiosis and the Elusive Final Interpretant of Understanding, Semiotica 179(1/4): 145–258. Sonesson, Göran. 2012. Semiotics inside-out and/or outside-in: How to understand everything and (with luck) influence people. Signata 2: 315–348. Thürlemann, Félix. 1982. Paul Klee. Analyse sémiotique de trois peintures. Lausanne: L’Age d’Hommes. Thürlemann, Félix. 1990. Vom Bild zum Raum. Beiträge zu einer semiotischen Kunstwissenschaft. Köln: DuMont. Zlatev, Jordan. 2011. What is cognitive semiotics? Semiotix XN-6. Available at: http://www.semioticon.com/semiotix/2011 Third edition. /10/what-is-cognitive-semiotics/. Zlatev, Jordan, Alenkær Madsen, Elainie Lenninger, Sara Persson, Tomas Sayehli, Susa, Sonesson, Göran van de Weijer, Joost. 2013. Understanding communicative intentions and semiotic vehicles by children and chimpanzees. In Cognitive Development 28 (2013) 312– 329.

Charles Forceville

2 Relevance Theory as model for analysing visual and multimodal communication Abstract: Visual and multimodal discourse becomes an ever-more popular object of research. This is a welcome development, since the proliferation of communication that is not (exclusively) verbal makes the need for an academic discipline focusing on images and multimodal texts highly pertinent. However, while many studies in this fledgling field are concerned with exposing supposedly hidden ideologies, considerably less work attempts to systematically chart how it is that we understand the non-verbal and multimodal in the first place. In this chapter I argue that the new discipline will benefit from adopting Sperber and Wilson’s Relevance Theory as the basic model for analysis. This will require adaptation of the model – which has hitherto been mainly used to analyse verbal, face-to-face communication – to multimodal mass-communication. The chapter mounts tentative proposals how this can be done. Adopting the model will moreover help provide insight into different modes’ affordances and limitations to communicate and persuade.

1 Introduction In this chapter I will promote the usefulness of Relevance Theory as a model for the analysis of visuals and multimodal discourse. The discussion will not be restricted to visuals alone, since most of the time visuals are accompanied by information in one or more other modalities. This means I will begin by saying something about multimodality. If the study of multimodal discourse is to develop into a respectable scholarly humanities discipline, each of the modes/modalities partaking in multimodal discourse must also be theorized seperately. The question is thus first of all what, and how, a mode can communicate on its own. Now before anything else this means we are faced with the formidable question of how “mode” is to be defined. Various attempts have been made (e.g. Jewitt 2009, Kress 2010, Elleström 2010), but the definitions diverge, and it is too early to accord any of them authoritative status. However, if there is no agreement on what constitutes a mode, any dimension of discursive meaning could qualify for modal status, and that would make the concept useless. I will not venture my own definition here but be practical and consider as modes: (1) written language; (2) spoken language; (3) visuals; (4) music; (5) sound; (6) gestures; (7) olfaction; (8) touch. To this list all the boundary problems typical of any categorization adhere: why not split up the visual mode into color, size, orientation, and a host of others? where does sound end and music begin? shouldn’t gestures be subdivided into facial expres-

52

Charles Forceville

sions, arm/hand movements, and bodily postures? However, these problems are not crucial, as long as it is acknowledged, first, that many categories allow for subdivisions and, second, that they tend to have fuzzy borders. Still, most categories have prototypical members, and it is via these that we can distinguish categories from one another (Lakoff 1987). Elaborating on my earlier work (Forceville 1996: chapter 5, 2005, 2009; see also Yus 2008), I will here sketch how discussions of visual and multimodal discourse can be embedded in a more general theory of communication and cognition: Sperber and Wilson’s Relevance Theory/RT (Sperber and Wilson 1986, 1995; Wilson and Sperber 2004, 2012). The focus of attention will be the visual mode, sometimes accompanied by the written verbal mode, but the idea is that the reasoning developed here is generalizable to other (combinations of) modes. Such a project should benefit both multimodality theory, which urgently needs more rigorous analytic models than have hitherto been proposed for it, and RT, which while claiming to hold for all forms of communication has been mainly applied to its spoken verbal varieties. Let me acknowledge straightaway that while I intend to do justice to RT, its pioneers and advocates may not agree with parts of this project, if only because my use of the theory must be selective. Extending concepts that were developed primarily with the verbal modality in mind to cover a very different modality such as the visual means that some of these concepts need to be relaxed, or may even be found to be inapplicable. So be it. Scholars’ models are there to account for reality, and if there is a mismatch between them it is the model which will have to be adapted. That being said, I also do not want to alienate nonlinguists by using more of the sometimes daunting RT terminology than is necessary. So while I am happy to stand corrected if I should make demonstrable mistakes and hope that RT scholars will help further improve my proposals, I take full responsibility for what some might consider a bastard version of RT.

2 A crash course in Relevance Theory Relevance Theory is much indebted to Grice’s (e.g. Grice 1975) Cooperative Principles. Grice distinguished four “maxims” that govern communication: the maxim of quantity (make your contribution optimally informative); the maxim of quality (don’t lie, don’t say things for which you have insufficient evidence); the maxim of relation (be relevant); and the maxim of manner (be perspicuous and thus avoid obscurity, ambiguity, prolixity, and be orderly) – see Wilson and Sperber (2012: 3). Sperber and Wilson (henceforth S&W) contend that actually the maxim of relation, or relevance, is the only necessary one, the other Gricean maxims being subservient to it. Moreover, they see the Relevance Principle not as prescriptive, a rule to be obeyed, but as hardwired in humans’ brains, something we simply can’t help relying on. In this section I will briefly, and with no claim to exhaustiveness,

Relevance Theory as model for analysing visual and multimodal communication

53

discuss those concepts of RT that I think are essential for theorizing visual and multimodal discourse.

2.1 Communicative versus informative intention Typical communication is ostensive-inferential (S&W 1995: 50–54): it is clear to both sender and addressee that the sender wants the addressee to be aware that she directs a message to him and that he is to infer relevant information from this message. (To avoid ambiguity in the choice of pronoun, I adopt S&W’s convention to make the communicator consistently female and the addressee consistently male.) Within ostensive-inferential communication, S&W distinguish between two types of intentions, defined as follows: Informative Intention: to make manifest or more manifest to the audience a set of assumptions I (S&W 1995: 58, emphasis in original). Communicative Intention: to make it mutually manifest to audience and communicator that the communicator has this informative intention (S&W 1995: 61).

Specifically, “the true communicative intention is […] the intention to have one’s informative intention recognised” (1995: 29, emphasis in original). The communicative intention, then, is the sender’s intention to get one or more addressees to accept certain assumptions I as (probably) correct, while the informative intention pertains to the contents of these assumptions. If I tell you that I won the lottery, and you understand my utterance, I have succeeded in fulfilling my informative intention; but only if you believe me, I have also succeeded in fulfilling my more encompassing communicative intention. Put differently: if you understand what I say to you, I have informed you; but only if you accept what I say, I have communicated with you. It is to be noted that the informative intention is also fulfilled while the communicative intention is not when, for instance, the addressee eavesdrops on the sender or when a sender quasi-accidentally gives an addressee access to the contents of a message while seemingly addressing somebody else (e.g. via a “mistakenly” sent reply-to-all e-mail). Even though the receiver may obtain valuable, indeed highly relevant information, in neither of these cases does the sender address the listener or reader. This means that the assumptions that are to be communicated are not made “mutually manifest” to the sender and the addressee, and so these examples do not exemplify ostensive-inferential communication. For my purposes it is useful to emphasize one specific dimension of ostensiveinferential communication mentioned in passing by S&W, namely the signal(s) that one person wants to communicate something to another in the first place. In Forceville (1996) I considered this an aspect of the communicative intention. However, since this is incommensurate with S&W’s claim that fulfillment of the communicative intention presupposes fulfillment of the informative intention, it seems wiser to refrain from burdening the communicative intention with the signaling of

54

Charles Forceville

a desire to communicate as such. Let me briefly expand on this matter (see also S&W 1995: 51). It may be the case that Mary makes mutually manifest her desire to communicate, but not (yet) her informative intention: she may clear her throat, whistle at Peter to draw his attention, run toward him frantically waving her arms, or emphatically try to catch his eye. In ordinary circumstances this will be a preamble to fulfilling her informative, and hopefully her communicative intention, but this need not be so. Peter may not hear her in a storm, she may stand behind sound-proof glass, or – having suffered a stroke – she may only produce incomprehensible babble. In these cases Mary’s intention to engage in ostensive communication, but not her informative, let alone her communicative intention, may be recognized by Peter.

2.2 Effect and effort RT presupposes that human beings are a goal-directed species par excellence. We have micro- and macro-goals and are acutely aware that so do our fellow human beings. Inasmuch as we stand to gain from cooperation, at least from the withingroup variety (a tenet held by socio-biologists – see Tomasello 2008, De Waal 2009 – and other scholars taking Darwin’s evolution theory as the best explanation for human behavior; see Boyd et al. 2010), it is not surprising that we are in principle prepared to help each other achieve our goals. Effective communication is crucial in this respect. The central idea of RT is that a communicator cannot help but presume to be optimally relevant to her addressee. For a message to be relevant to a given addressee, that message must have an “effect” on the sum total of knowledge, beliefs, and emotions (called the “cognitive environment”, S&W 1995: 38) of that addressee. The time at which the message is communicated and the place where the addressee processes it is also part and parcel of the addressee’s cognitive environment. Typical examples of relevant messages are telling an addressee something he did not know (“today the shops close at 4 o’clock”), strengthening an assumption he already entertained (“remember you promised to repair my bike?”), or weakening an assumption he held (“I will probably not have time after all to go out for a drink with you tonight”). Importantly, sharing an emotion or evaluation with an addressee also counts as an effect (“isn’t this Gorgonzola exquisite?” “this is the saddest story I ever heard”). The greater the effect a message has on the cognitive environment of the addressee, the greater its relevance. Put differently, the greater the real or potential difference the uptake of the message will have on the addressee’s future decisions, actions, and attitudes, the greater its relevance. This means that a message’s relevance can vary from being minute and extremely short-lived to being vast and life-changing. But the benefits of relevance are offset by its costs, called “effort” in RT. The more mental effort an addressee must invest to recover a message’s effect, the less relevant it becomes. Relevance thus arises from a balance between effect and effort.

Relevance Theory as model for analysing visual and multimodal communication

55

One of the most interesting and far-reaching claims of RT is that since he trusts his interlocutor to be optimally relevant, an addressee will typically stop interpreting a message as soon as it achieves relevance. The addressee assumes that the sender has selected the best possible or available stimulus, given the circumstances, to get across the message. It is thus neither too short or simple, nor too long or complex. So as soon as the addressee deems he has found an interpretation of the message that strikes him as relevant, he will stop searching for further relevance. This means that, typically, in face-to-face conversation the first interpretation the addressee hits on is the one the sender intended him to derive. Thanks to this mechanism, communication can be as quick and, usually, efficient as it is (just imagine that an addressee would have to work out the possible meanings of each individual sentence, or indeed each individual word, in a message …). In poetry, a text-genre known to require rereading, pondering, and reassessing, a reader may be aware that he needs to invest more effort to achieve optimal relevance than stopping at the very first interpretation. Consequently, he may settle for an aesthetically pleasing ambiguity that he would find intolerable in most other forms of communication (see Pilkington 2000).

2.3 (En/de)coding versus inferencing Each verbal message contains “objective” information: knowledge of the denotation of the words in it and of the grammatical rules that determine the relation in which these words stand. Both the sender and the addressee of the message need to be in possession of the linguistic code (knowledge of vocabulary and grammar) to encode, respectively decode, this information. But this information, which depends on the “logical form” of a sentence (“it is in virtue of its logical form that a conceptual representation is involved in logical processes and enters into relations such as contradiction or implication with other conceptual representations”, S&W 1995: 72), needs to be further developed before it achieves the status of “explicit information” or, in S&W’s terminology, “explicatures”. Explicatures are propositions that are capable of being evaluated as true or false. Inferring explicatures comprises reference-attribution, disambiguation, and enrichment. If Mary tells Peter, “John will come soon”, Peter must instantly decide which “John” Mary means (answer: the first one he thinks of, John Baker, since otherwise, he trusts, Mary would have said “John Carpenter” to keep him from thinking she means John Baker [= reference attribution]). He also needs to decide whether, in the given situation (which is part of both Mary’s and Peter’s mutually manifest cognitive environment) “soon” means “within minutes”, “within hours” or “within days” (= enrichment). And in this same situation, say, it is also clear that Mary refers to John arriving, not coming in a sexual sense (= disambiguation). But Peter knows that Mary has uttered “John will come soon” with a reason: she expects him to derive certain relevant effects from this information. That is,

56

Charles Forceville

after having derived the explicatures of the message, he now needs to infer the relevant information it contains. He will do this by combining her message with information in his cognitive environment. Let us imagine that John will come to dinner. This is mutually manifest to Mary and Peter. In that light, it may be that Mary intends Peter to derive the implicated message: “I need to stop reading and start laying the table”, or “I am to go and run to get a bottle of wine from the liquor store”, or “Mary suggests we now stop our marital row in order be in a decent mood when our guest arrives”, or “Mary wants to reassure me that though John is late, he will turn up”, or all of them, or any of a range of others. These inferences, which are based on combining explicatures and assumptions that are mutually manifest to the interlocutors are not of the explicit but of the implicit kind, and are are called “implicatures”. S&W distinguish between strong implicatures, which must be derived for the message to make sense, and weak implicatures, which the addressee derives at his own discretion. Mary’s utterance presupposes a number of things – actually very many things. For one thing, Mary presupposes that Peter understands English. If Mary knew that this is not the case, she would not have chosen the phrase “John will come soon” in the first place (she might have tried another language, or attempted to mime the information). More generally, she presupposes that Peter in one way or another has an interest in John’s coming or not coming, and in the implications of this event. Even more generally, Mary takes numerous facts and evaluations pertaining to Peter, herself, and John, as well as their relationships to be mutually manifest to Peter and herself. The more the uptake of a message depends on the addressee’s derivation of explicatures and self-evident, hard-to-miss-or-disagree-about implicatures, the more the message counts as “strong” communication. By contrast, the more its uptake relies on implicatures whose recruiting is not very self-evident, the more the message verges toward “weak” communication. One of the most insightful aspects of the RT model is in my view the following: the closer the message is to the “strong” end of the continuum, the more the responsibility for the derivation of implicatures resides with the sender of the message, while conversely the closer it is to the “weak” end of the continuum, the more the responsibility for deriving it rests with the addressee. If I shout “Fire!” at you, I take strong responsibility for conveying the message that there is a fire, which will probably lead you to get out of the building as soon as possible. If there then should turn out to be no fire at all, you would be fully justified to be indignant with me, saying I lied. If, by contrast, I say, “I seem to perceive a somewhat smoky smell” you take far more responsibility yourself for deriving the message “there is a fire”, and you shouldn’t complain if this was not the case after all.

2.4 Relevance is always relevance to an individual Most of the examples S&W (1995) use pertain to verbal exchanges between two individuals, Mary and Peter, who know each other very well (they may be married).

Relevance Theory as model for analysing visual and multimodal communication

57

This is essential, since it means that Mary and Peter have a lot of mutually manifest information and history in their cognitive environments due to their shared past experiences. They thus can often be very concise in their exchanges, since many things need not be spelled out between them. In the scenario of John about to arrive, if instead of her husband Peter Mary had addressed a neighbor who happened to have dropped by, she might have said, “John Baker, a friend of ours, will arrive in the next fifteen minutes or so for dinner”, hoping that the neighbor would derive the implicature he should go home. If Mary uttered this sentence to both Peter and the neighbor, she would expect each of them to derive different implicatures. Face-to-face verbal communication is the prototype form of communication in S&W’s model. In this form of communication, implicatures are not just derived by the addressee from combining explicatures, shared experiences, and awareness of the sender’s goals, but can also be triggered by the sender’s intonation, facial expression (see de Brabanter 2010), and body posture, as well as the spatio-temporal circumstances in which the exchange takes place. When sender and addressee are no longer in the same space (as in telephone conversations) and/or time-frame (as in voice-mail messages), this inevitably affects what is, and is not, possible in the communication between them. In the latter cases, facial expressions and body postures cannot play a role, while in the last example there is not (as in a telephone conversation) an opportunity for immediate feedback or the correction of errors. Matters become considerably more complicated in mass-communication. Let us consider mass-communication to start, for theoretical purposes, when the audience consists of more than one person simultaneously. This makes the example of Mary addressing both Peter and the neighbor a specimen of mass-communication, albeit an untypical one. I proposed that her utterance presumably achieved a different effect in both of them – but Mary probably had a fairly good idea of how it did. By contrast, when the US president delivers her inaugural speech, televised live around the world, her audience consists of many millions of addressees who will, in terms of command of English, gender, class, material welfare, geographical location and a host of other circumstances each have their own, very different cognitive environments. The presidential message (delivered multimodally, drawing minimally on spoken language, visuals and gestures) will thus be processed marginally or vastly differently by each of the millions of addressees, because each of these latter has his own unique cognitive environment and interests. A consequence of this is that even in mass-communication, “relevance is always relevance to an individual”. This crucial point was made by S&W (S&W 1995: 142), but in my view they insufficiently emphasized it, possibly because their focus on simultaneous, primarily verbal communication between two individuals did not foreground the necessity to investigate the implications of their insights when considering masscommunication (see also Forceville 1996: chapter 5).

58

Charles Forceville

2.5 Some points for consideration Although the basic tenets of RT are fairly simple, its ramifications are by no means so, and there has been fierce debate about its merits and problems (see the discussions in Behavioral and Brain Sciences 10(4), 1987, and Language and Literature 6(2), 1997). In this section I will restrict myself to a few brief comments on issues that directly bear on the use I make of RT in the analysis of visual and multimodal discourse. Babbling and lying. RT describes communication; it does not prescribe rules for its correct use. In this sense RT differs from Grice’s cooperative principles, which are formulated in a “thou shalt (not) …” manner. Critics might object that a madwoman’s incomprehensible babble cannot be accounted for by RT. A strategy to counter this criticism could be to argue that the madwoman’s babble still comes with the presumption of relevance (she believes herself to be relevant); or alternatively, we could maintain that the babble is self-directed and thus does not constitute ostensive-inferential communication. “Lying”, though, is unproblematically accommodated in RT: liars hope that their addressees are unaware of the misleading nature of the explicatures and implicatures of their messages, and that these addressees therefore presume the senders to be optimally relevant. RT thus “has no independent maxim or convention of truthfulness” (Wilson and Matsui 2012: 209). The best possible stimulus. According to RT the sender chooses the best possible stimulus that is commensurate with her preferences and/or her abilities. There can be numerous reasons why a sender’s stimulus is not ideal. A speaker may for instance choose a stimulus requiring more effort on the part of the addressee than a more straightforward formulation would have taken for reasons of politeness. Or perhaps she has a far from perfect command of the language she uses, and therefore may have to select her words from the limited stock of words she masters. Or she may be gagged and hand-cuffed, and only hope to warn her prospective savior that the bad guy is standing behind the door with a hatchet, ready to kill him, by uttering muffled sounds at the top of her voice. Or she may subtly want to inform one person in a group at a party of something without alerting the others, making her favor “weak” communication which she hopes will be picked up by her intended addressee and ignored by the others. But given the circumstances it is nonetheless the best stimulus she can, or is willing to, provide. Emotional effects. Within cognition studies, there is now increasing consensus that emotions are part and parcel of cognition, and RT is no exception. It may well be that the main, or even only, effect of a stimulus is the communication of an emotional response (“This is beautiful/terrifying/disgusting …”). Even if the intended effect is primarily the activation of certain assumptions in the addressee, these often have affective dimensions as well; arguably, the reverse is no less true. I note in passing that Michael Tomasello emphasizes that successful communication does not only presuppose the transmission of information, but also the mutu-

Relevance Theory as model for analysing visual and multimodal communication

59

ally manifest awareness of the communicator’s attitude to this transmitted information (Tomasello 2008: chapter 3). “Symptomatic” communication? What could be an RT approach to relevant information communicated without its sender being consciously aware of conveying it? Undoubtedly, communicators may unwittingly transmit crucially important information. Bordwell and Thompson (2008) coined the label “symptomatic meaning” for this: a communicator may not be aware of saying something important that happens to be highly relevant to the addressee. But this is not ostensive-inferential communication. The communicator did not intend to convey this information – just as certain clouds do not intend to tell those knowledgeable about clouds that it is going to rain, even though this conclusion may well be both significant and correct. For this reason I propose to exclude “symptoms” from the realm of communication: they are by-effects of communication, not parts of it. Tanaka (1994) discusses another borderline case, namely that of “covert” communication. Consider the following situation: we are both at a cocktail party and stand talking in different groups. I know that you are within hearing distance, and I say loudly about you, but not to you, “s/he is such an intelligent person!” This, again, does not count as ostensive-inferential information. After all, even though I may deliberately speak loudly in the hope that you will hear me, I do not make mutually manifest that I want to communicate my praise to you (i.e. the communicative intention is not fulfilled). That being said, it is sometimes difficult or even impossible to distinguish between weak communication and symptomatic meaning.

3 RT and visual and multimodal communication In this paragraph, I will revisit the RT concepts discussed in the previous section with an eye to their applicability to visual and multimodal discourse. My central point is that RT can well model visual communication – whether or not accompanied by language or other modes – but that this requires taking into account the affordances and constraints of the visual mode as well as the fact that most visual and multimodal communication is of the mass-medial kind.

3.1 The presumption of relevance The maker of a picture tries to be optimally relevant to her envisaged audience. Whether the picture is an illustration in a book, a political cartoon, an advertisement, or a hand-drawn map, its maker wants to attract the audience’s attention (ostensive communication), convey information and/or attitudes (informative intention) and thus to have an effect on this audience at no unnecessary effort (communicative intention). Usually, this effect is a quite precise one. The political cartoon criticizes a specific person or state of affairs at a certain time and place in

60

Charles Forceville

history; the illustrations in the manual of your new printer indicate which cord goes where; the advertisement promotes a specific brand or service by focusing on some of its supposed qualities. These are mass-communicative pictures, whose genre their envisaged addressees usually recognize unproblematically. The pictorial message comes with the presumption of relevance. All signal: “hey, look at me, I’m worth your attention”.

3.2 Explicatures and implicatures In order to make RT work for pictures, we need to adopt the distinction between the processes of en/decoding, pragmatically inferring explicatures, and pragmatically inferring implicatures. I think this is feasible. We first need to recognize the elements in the picture that have been depicted. In the Tintin panel reproduced in Figure 1, we must recognize the patches of light green as grass, the darker green as trees in a wood or jungle, the grey-white entity in the middle as a simple hut with a thatched roof and a brown door. Furthermore, we presumably recognize the two creatures minimally as a young man and a dog, respectively. We can equate this with the decoding stage. Everybody in possession of the required visual literacy “code” is able to do this. Viewers whose cognitive environment comprises knowledge of Tintin derive more information: even when they see the panel in decontextualized form, as here, such persons will recognize the redhead as Tintin and the dog as his loyal companion Snowy. We further assume, even though we only see part of Tintin’s left arm, that the hero actually has a complete arm, and that Tintin is walking (not running or flying). That is, we routinely perform reference assignment, disambiguation, and enrichment. Does this amount to claiming that we derive explicatures from this picture? In S&W’s theory explicatures are propositions, and thus can be evaluated as true or false. Critics might insist that only propositional sentences are capable of being true or false, which would then lead to the conclusion that pictures never have explicatures, because they do not have a logical form from which propositions can be generated. But I will suggest that we can say that the panel invites explicatures such as “these are a young man and a dog” or “these are Tintin and Snowy”, or even “Tintin and Snowy walk toward a hut in a forest”. So we would have to say either that this panel conveys multiple propositions, or that the only intended explicature is “Tintin and Snowy walk towards a hut in a forest”, the other propositions then being implicated premises (see Forceville 2005 for more examples of implicated premises). The next step is inferring implicatures. Strong implicatures (implicatures for whose derivation the sender takes responsibility) include typically that the young man/Tintin and the dog/Snowy will enter the hut (rather than walk past the hut, or transform into dinosaurs in front of it). Further pragmatic inferences of the implicature variety will be fed by hypotheses about Tintin’s goals in this particular situation, which build on information from the previous panels in the album, on

Relevance Theory as model for analysing visual and multimodal communication

Fig. 1: Panel from Hergé, Tintin et les Picaros 1976), original in color. © Hergé/Moulinsart 2013.

61

Fig. 2: Cartoon of Barack Obama, original in color. Accessed 24/12/12 http://noisyroom. net/blog/2012/06/30/obama-lies-taxes-rise/.

our knowledge of the protagonist’s character, and our understanding of typical story-plots. In Figure 2, the decoding of the picture involves the assessment that a man with a shirt, tie, and jacket with an exceptionally long nose is depicted. Reference attribution, enrichment, and disambiguation lead to the conclusion that this is Barack Obama, at the moment of writing president of the United States. This assessment hinges on resemblance between the depicted person and Obama. Moreover, the red-and-blue color scheme evokes the original poster in his 2008 election campaign and thus also cues “Obama”. The long nose intertextually refers to Pinocchio. Pinocchio is well-known for the fact that he pathologically lies and that by way of punishment his nose grows a little longer with every lie. I propose that all this amounts to the explicature “Obama is a liar”. The textual “lies” anchors (in the sense of Barthes 1986; see also Unsworth and Cléirigh 2009) this interpretation, but for many viewers this text is redundant. The next stage is the inference of implicatures, such as that the communicator of this picture counsels viewers to mistrust Obama, or not to vote for him again. Again, the envisaged addressee needs to be able to recruit a lot of knowledge about the world from his cognitive environment. Somebody who does not recognize Obama, and has no idea that he is/was the president of the United States, and/or is not familiar with the Pinocchio story, will not derive the intended inferences. But it may also be the case that the viewer full well knows about Obama, but nonetheless fails to recognize him in this picture, and is not familiar with the color scheme and the style of the campaign poster. This would perhaps be similar to

62

Charles Forceville

the failure of communication due to mis-hearing, or not knowing, a crucial word in an interlocutor’s utterance. It is also important that the viewer recruits information that pertains to the moment and place of the discourse under consideration (see Forceville 2005). The meaning of political cartoons, specifically, is very much tied to the situation of the moment of publication, drawing on knowledge that is briefly very salient and thus easily recruited, but quickly forgotten as time passes. This, incidentally, is why cartoons later re-published in book collections or on websites invariably require verbal elucidation to remind the audience of the precise socio-historical context in which they originally appeared. Finally, it might be objected that one would have to get used to a certain style of drawing a house, a man, a beautiful girl, Barack Obama, before being able to derive pertinent explicatures from pictures. To the extent that this is true, one could counter that, in language, a listener unfamiliar with regional language variations has a less-than-perfect command of the code, and therefore is unable to derive certain explicatures in such dialectical utterances. Once that person has mastered certain words or expressions, he has solved this problem. In my view, this is not really different from having, in some cases, to learn the code of a cartoon style (say, Japanese Manga). Here is another example: an architect’s blueprint is a type of drawing that is supposed to be very precise indeed – unlike, for instance, the child’s stylized house with a few token windows and a door. But one has to learn the code to derive the explicatures, and it is explicatures, explicit content, that count here.

3.3 Relevance to the individual in the mass-audience Most communication involving pictures is mass-communication. As we saw, masscommunication is more complex than face-to-face communication in the sense that there is no opportunity for adapting the message in the case of misunderstanding or incomprehension. Experienced mass-communicators of course anticipate this, so they carefully consider the form and contents of their message before releasing it, and may already know fairly precisely who their target audience is. In this way they minimize the risk of miscommunication. But mass-communication remains hazardous, simply because mass-communicators usually have to go for a “one size fits all” message, which “lands” differently in the numerous different cognitive environments of its numerous different addressees (exceptions include customized ads aimed at individual internet users, based on their Google search history). Specifically, mass-communicators have little control over the weak implicatures their mass-audience of individuals will derive. As all communicators, a picture maker (whether an advertiser, a newspaper cartoonist, a graphic designer, or the passer-by who draws a little map upon your asking for directions) aims for a more or less specific effect in her audience. The friendly passer-by has a relatively easy job: she wants to help an audience of one

Relevance Theory as model for analysing visual and multimodal communication

63

(you) to reach your destination with the aid of her drawn-on-the-spot map. Since this is Mary-and-Peter communication, she can fine-tune her visual or multimodal (visuals + phrases such as street names) message in interaction with you. But in multimodal communication involving pictures this is an exceptional situation, since usually the audience is a mass-audience. When the communicator is a lecturer standing in front of a class of two hundred students, her verbal communication, gestures, and the diagrams and pictures on her PowerPoint constitute masscommunication in which there is at least a shared time-and-space frame. In most mass-communication, by contrast, the audience does not share spatio-temporality with the maker. Nonetheless the picture maker, just as the USA president delivering her televised speech, still has a fairly specific idea about her audience. The advertiser aims at selling goods or services to a target audience; the cartoonist wants the reader/viewer of the paper/magazine/site where it appears to critically ponder, and momentarily see the humor of, a certain political state of affairs; the graphic designer of a book desires to attain clarity and perhaps aesthetic appreciation of its layout in her (prospective) reader; the architect hopes to achieve both admiration for the beauty of her building and awareness of its clever, functional design. As always, each of these messages is highly contextualized. The picture appears at a certain moment in time and place, for a more or less specific audience: the target audience of a specific product flipping through a specific magazine; the reader of a left-wing journal in a certain country after a certain newsworthy event has happened; the buyer of a Russian classic novel in hardback or of an American management book in paperback; you reading this very chapter sitting in your armchair or behind your laptop having downloaded an illegal copy. The context within which a mass-audience accesses a message, that is, is to a considerable extent anticipated by its maker. First of all, the message belongs usually to a recognizable genre. Genre is for discourse what activity-type, as formulated by Goffman (1974), is for action: as soon as we recognize that a given discourse belongs to a genre with which we are familiar, we know what conventions to recruit in interpreting it, and what responses are appropriate to it (Fokkema and Ibsch 2000). When we decide that something is an ad, we know the sender is trying to sell us something (see Pateman 1983, Forceville 1996); if we see somebody fall in a melodrama we feel sorry, if we see somebody fall in a comedy we laugh. Genre is an element of context whose importance cannot be overestimated. Genreattribution moreover occurs mostly subconsciously and in milliseconds, and is in my view the single most important element in the addressee’s cognitive environment steering his strategy of interpretation of any pictorial or multimodal message (Forceville 2006). Whereas context is endless, and ever-changing, genre-attribution is quite stable and reliable. Indeed, I submit that “genre” more than any other contextual factor helps constrain what the relevance theorist Robyn Carston calls “free pragmatic processes: […] pragmatic processes that contribute to what a

64

Charles Forceville

speaker [or picture-maker, ChF] is taken to have explicitly communicated but which are not triggered or required by any linguistic [visual, ChF] property or feature of the utterance [picture, ChF]” (Carston 2010: 265). This genre-attribution is first of all cued by text-internal semantic and “syntactic” information (I use quotation marks because the notion of grammatical rules applies at best metaphorically to non-verbal modalities: visuals do not have a grammar in the sense that language has, pace Kress and Van Leeuwen 1996, 2006; see Forceville 1999). But in addition, the spatio-temporal circumstances in which we encounter a discourse enormously facilitate correct genre-attribution. We usually know what genre to expect before we read/see/hear/feel the discourse itself, due to where and when we encounter it. The multiplex screening of the blockbuster Hollywood film, the JC Decaux billboard showcase on the street or the pop-up internet-banner, the bottom-right corner on p. 3 of your newspaper featuring a political cartoon, the manual accompanying your newly-bought machine, the menu at the entrance of a restaurant presenting information about its dishes, prices, and atmosphere … we already know a lot before actually accessing this information – although of course those out to trick us make deceptive use of these generic cues. In turn, where and when we are likely to encounter a discourse is largely determined by the agency or institution controlling such discourses (see Altman 1999 for a lucid account of how this works in film). The attribution of meaning to a message thus is guided by the chain “discourse-genre-medium-institution”, of course always embedded in the presumption of relevance for the addressee. Consider Figure 3. Decoding this picture involves assessing that there is a single human being standing on a big flight of steps, hands behind the back. Most viewers, moreover, would realize that the human being is a royal person (thanks to the crown, which is yellow in the original, and the ermine cloak, which is orange). For viewers knowledgeable about Dutch culture, more specific information can be decoded: This is the Dutch (now: ex-) queen, Beatrix. These enrichments enable certain explicatures, such as: “Beatrix stands on a flight of steps”, “Beatrix is alone”, “Beatrix looks toward the left”. Such viewers also have the following knowledge in their cognitive environment: as soon as, after elections, a new coalition cabinet has been formed, the monarch, surrounded by the ministers, poses on the steps of one of the royal palaces, “Paleis Huis ten Bosch”, for the official press photograph. But yet other assumptions, retrieved from his cognitive environment, are recruited by the envisaged viewer: this is a cartoon on the “political cartoon” page of the left-wing weekly magazine Vrij Nederland. It is made by Peter van Straaten, which the cognoscenti know because of one or more of the following pieces of information: there is both the Barthesian anchoring “Peter” above the picture and his signature in the bottom right corner; Van Straaten “always” draws the cartoon in Vrij Nederland; and his cartoons have a recognizable style. The cartoon would

Relevance Theory as model for analysing visual and multimodal communication

65

Fig. 3: Political cartoon by Peter van Straaten, Vrij Nederland 27/1/’07, p. 80, original in color.

moreover be typically accessed in the week of 27 January 2007, in Vrij Nederland. All these fragments of knowledge thus belong to the cognitive environment of the envisaged viewer: the reader of this Vrij Nederland issue, who is fully aware, at this moment in time, that it is taking ages for the new coalition government to be formed. This information leads viewers to infer the following explicature, the one intended by Van Straaten: the Dutch Queen Beatrix impatiently waits for her new government to finally turn up for the official press photograph on the stairs of Paleis Huis ten Bosch, the photograph symbolizing that a new government has been formed.

3.4 Communicative versus informative intention in masscommunication Communication without temporal synchronicity and spatial co-presence may be aimed specifically at an individual (think of a letter, e-mail, SMS, voice-mail), but often it is not. The presidential speech, the Hollywood film, the advertising billboard, and the tweet are aimed at whoever perceives them as potentially relevant. Clearly each mass-communicator wants to reach as many members of her target audience as possible. Thus we can understand any attention-grabbing device of such a message as signaling its intention to engage in ostensive-inferential com-

66

Charles Forceville

munication. Radio jingles, loud noises, scantily-clad ladies or ditto muscle men, big sizes, primary colors, unusual formats …; the list of devices used to lure people into paying attention is long. Each of them can be considered mass-communicative equivalents of clearing one’s throat, gesturing, or starting to speak in face-to-face communication. In contrast to face-to-face communication, however, it is often quite easy to decline the invitation to heed the message; if so, the informative intention is not fulfilled. If it is, whether the communicative intention is subsequently fulfilled depends, as always, on whether the assumptions conveyed in the message are fruitfully processed in the cognitive environment of the addressee. Misinterpretation may be due to a faulty derivation of explicatures (due, for instance, to inadequate knowledge of the code, or incorrect procedures of disambiguation), or to the derivation of implicatures not intended by the sender (e.g. due to associations evoked by the stimulus in the addressee’s cognitive environment that were not foreseen by the sender).

3.5 Effect and effort in visual communication The mechanisms of effect and effort apply in much the same way in visual and multimodal mass-communication as they do in face-to-face verbal communication. Just as a narcissist at a party may bore us to death with her presumption of the relevance of her self-aggrandizing chatter, so we may find a TV programme completely irrelevant. In the latter case, however, we have the opportunity to zap away after a few seconds, while we may not so conveniently be able to flee the party bore. (If the bore is a lecturer lecturing, we may, due to considerations of politeness and institutional rules, not so easily run off, even though this is a form of masscommunication.) But equally, just as we are likely to pay close attention to the story of our beloved ones about something important that has befallen them because they are superbly relevant to us, so we may be prepared to invest a lot of effort grasping the text and diagrams of a text book whose contents we will be examined about, or the text and pictures in the manual of the DVD recorder we hope to get working. That is, there may be socio-cultural or practical reasons that encourage us to invest an inordinate amount of effort in interpreting a mass-media message.

4 Whys and wherefores In this chapter I have fleshed out Relevance Theory’s claim that it offers a model for all communication, including visual and multimodal communication. But since the medium is the message, its central tenets must be tested, and where necessary expanded or adapted, for discourses in genres and media that deviate from the prototypical form of oral verbal communication in Sperber and Wilson’s work.

Relevance Theory as model for analysing visual and multimodal communication

67

Why is it useful to adopt RT as the overall model for communication? RT combines the attractiveness of high plausibility and precision in explaining how its central concepts apply to real-life communication. This precision allows for the systematic analysis of all forms of communication in all (combinations of) modes in all media. More specifically, it enables attesting which elements in a message (fail to) contribute what information towards achieving relevance for one or more addressees. Since communication is geared towards the optimization of relevance, all factors in the discourse that relate to this must be addressed. An asset of the model is that it permits comparisons between modes and media. I have argued that, like language, pictures can convey explicit content – and propose to theorize this explicit content in terms of explicatures. This proposal deserves extended discussion: if it is found to be untenable, we would have to conclude that after addressees have decoded visuals (the first stage of interpretation), they skip the derivation of explicatures and immediately move on to the derivation of implicatures. This is a crucial issue, for it would mean that pictures unaccompanied by information in other modalities, specifically the verbal one, could never serve as independent proof for anything, since we can construe no propositions from them that are evaluable as true or false. The pertinence of this with regard to visual evidence in documentary films as well as in court cases should be clear. I have only begun the work to be done. Let me sketch some contours of the broader research program by asking further pertinent questions: Communicators and addressees. Who are they? Is the communicator a single agent? If not, who/what is formally the sender of the message? Often this is impossible to determine. Someone is formally the sender (the journalist of her newspaper article, the author of her novel, the director of her film), but in mass-communication the communicator is usually part of an institution, in which sometimes many individuals have co-shaped a message. And who is/are the communicator(s) in forwarded mail? A re-tweet? A re-released film? Who are the interlocutors in a TV show or radio show (see Scannell 1996)? Are they the TV show’s host and his/her guests? The studio audience? The TV audience? Discussing such questions with reference to the RT model can alert us to essential dimensions of mass-communication, but also suggest refinements of that model. Institutions and media. Mass-communication takes place in an institutional context. Who have access to the media to send their messages? In what way is the best possible stimulus a communicator can or is willing to provide to achieve relevance guided or constrained by the technical dimensions of the medium, by budgets, formats, scheduling slots, censorship – often in ways, audiences have no way of knowing about? How is the situation for websites? Blogs? Social media? Modalities. What are the modalities a given discourse can draw on? Is a discourse monomodal (such as wordless music and music-less pantomime) or multimodal (drawing on two or more modalities, as in film, see Forceville 2006)? What does each modality contribute to the optimization of relevance? To what extent

68

Charles Forceville

does a modality allow for a distinction between the levels of en/decoding, and the derivation of explicatures and (strong/weak) implicatures? While arguably it makes sense to talk about en/decoding non-verbal sound (in a film, or a radio play, for example), could we say the same about music? Do ostensively used textures and smells require decoding, and do they have explicatures and implicatures? What modalities can be combined, and what does combining modalities yield in terms of meaning potential? Genre and intertextuality. I have proposed that genre is the single most important element in the pragmatics of context-activation for discourse interpretation. Indeed, the very first thing a would-be addressee of a mass-communicative message does is assess the genre to which the message belongs. He then activates, from his cognitive environment, knowledge of the genre conventions that allow him to derive pertinent explicatures and implicatures. Something similar holds for having (or not having) knowledge of a pertinent “intertext” in one’s cognitive environment. Given that individuals possess varying degrees of knowledge about genres and intertexts, they are bound to come up with slightly or vastly differing interpretations of a mass-communicative message. Decoding visual signals and deriving their explicatures and implicatures. It could be argued that the only explicatures in Figures 1–3 are “Tintin and Snowy walk toward a hut in a forest”, “Obama is a liar”, and “Queen Beatrix is impatiently waiting for the new Dutch government to be formed”, respectively, and that all other information is part of the implicated premises. Alternatively, we could say that these three pictures are all “multipropositional”. Clearly the distinctions between decoding and inferring require further thought. All of these questions can be addressed within RT, although (1) the model needs to be expanded; and (2) expert knowledge from many other disciplines needs to be invoked. Systematic research within the RT model will also help define categories and clarify the always interesting fuzzy border areas between them. When does communication stop being communication? Where does music bleed into sound? Which elements in the visual mode are potentially meaningful – and under what conditions? Which elements in a picture are universally decodable (at least by sane adults) and which require more or less knowledge of (sub)cultural conventions? Acknowledgments: I thank Belén Soria (University of Granada) for generously commenting on earlier drafts of this paper. Moreover I have benefited from discussing specific sections with Billy Clark and Francisco Yus. But it remains the privilege of the author to be solely responsible for all mistakes and controversial opinions.

Relevance Theory as model for analysing visual and multimodal communication

69

References Altman, Rick. 1999. Film/Genre. London: British Film Institute. Barthes, Roland. 1986 [1964]. Rhetoric of the image. Translation by Richard Howard. In: Roland Barthes, The Responsibility of Forms, 21–40. Oxford: Blackwell. Bordwell, David, and Kirstin Thompson. 2008. Film Art. 8th edition. Boston: McGraw-Hill. Boyd, Brian, Joseph Carroll, and Jonathan Gottschall (eds.). 2010. Evolution, Literature & Film: A Reader. New York: Columbia University Press. Brabanter, Philippe de. 2010. Uttering sentences made up of words and gestures. In: Belén Soria and Esther Romero (eds.), Explicit Communication: Robyn Carston’s Pragmatics, 199–216. Basingstoke: Palgrave MacMillan. Carston, Robyn. 2010. Explicit communication and ‘free’ pragmatic enrichment. In: Belén Soria and Ester Romero (eds.), Explicit Communication: Robyn Carston’s Pragmatics, 217–285. Basingstoke: Palgrave MacMillan. Elleström, Lars (ed.). 2010. Media Borders, Multimodality and Intermediality. Basingstoke: Palgrave MacMillan. Fokkema, Douwe, and Elrud Ibsch. 2000. Knowledge and Commitment: A Problem-Oriented Approach to Literary Studies. Amsterdam: Benjamins. Forceville, Charles. 1996. Pictorial Metaphor in Advertising. London: Routledge. Forceville, Charles. 1999. Educating the eye? Kress and Van Leeuwen's Reading Images: The Grammar of Visual Design (1996). Language & Literature 8(2): 163–178. Forceville, Charles. 2005. Addressing an audience: Time, place, and genre in Peter Van Straaten’s calendar cartoons. Humor 18(3): 247–278. Forceville, Charles. 2006. Non-verbal and multimodal metaphor in a cognitivist framework: Agendas for research. In Gitte Kristiansen, Michel Achard, René Dirven, and Francisco Ruiz de Mendoza Ibáñez (eds.), Cognitive Linguistics: Current Applications and Future Perspectives, 379–402. Berlin: Mouton de Gruyter. Forceville, Charles. 2009. Relevanz und Prägnanz: Kunst als Kommunikation. Translation by Martina Plümacher. Zeitschrift für Semiotik 31(1–2): 47–63. Goffman, Erving. 1974. Frame Analysis: An Essay on the Organization of Experience. New York: Harper & Row. Grice, H. Paul. 1975. Logic and conversation. In: Peter Cole and Jerry L. Morgan (eds.), Syntax and Semantics 3: Speech Acts, 41–58. New York: Academic Press. Jewitt, Carey (ed.). 2009. The Routledge Handbook of Multimodal Analysis. London: Routledge. Kress, Gunther. 2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge. Kress, Gunther, and Theo Van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London: Routledge. Kress, Gunther, and Theo Van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. 2nd, revised edition. London: Routledge. Lakoff, George. 1987. Women, Fire and Dangerous Things: What Categories Reveal About the Mind. Chicago: University of Chicago Press. Pateman, Trevor. 1983. How is understanding an advertisement possible? In Howard Davis and Paul Walton (eds.), Language, Image, Media, 187–204. Oxford: Blackwell. Pilkington, Adrian. 2000. Poetic Effects: A Relevance Theory Perspective. Amsterdam: Benjamins. Scannell, Paddy. 1996. Radio, Television & Modern Life: A Phenomenological Approach. Oxford: Blackwell. Sperber, Dan, and Deirdre Wilson. 1986. Relevance Theory: Communication and Cognition. Oxford: Blackwell.

70

Charles Forceville

Sperber, Dan, and Deirdre Wilson. 1995. Relevance Theory: Communication and Cognition. 2nd edition. Oxford: Blackwell. Tanaka, Keiko. 1994. Advertising Language: A Pragmatic Approach to Advertisements in Britain and Japan. London: Routledge. Tomasello, Michael. 2008. Origins of Human Communication. Cambridge: MIT Press. Unsworth, Len, and Chris Cléirigh. 2009. Multimodality and reading: The construction of meaning through image-text interaction. In Carey Jewitt (ed.), The Routledge Handbook of Multimodal Analysis, 151–163. London: Routledge. Waal, Frans de. 2009. The Age of Empathy: Nature’s Lessons for a Kinder Society. New York: Harmony. Wilson, Deirdre, and Dan Sperber. 2004. Relevance Theory. In Laurence R. Horn and Gregory Ward (eds.), The Handbook of Pragmatics, 607–632. Malden: Blackwell. Wilson, Deirdre, and Tomoko Matsui. 2012. Recent approaches to bridging: Truth, coherence, relevance. In Deirdre Wilson and Dan Sperber, Meaning and Relevance, 187–209. Cambridge: Cambridge University Press. Yus, Francisco. 2008. Inferring from comics: A multi-stage account. In Pelegrí Sancho Cremades, Carmen Gregori Signes, and Santiago Renard (eds.), El Discurs del Comic, 223–249. Valencia: University of Valencia.

Ian Roderick

3 Military hardware as affective objects: Towards a social semiotics of militainment television Abstract: This paper seeks to critically examine the visual presentation of military artefacts in militainment versions of what Fürsich (2003) has called nonfiction entertainment. While not equating technology to discourse, it is argued that militainment programming such as that which can be found on The Military Channel reliably draw upon regularized sets of semiotic resources so as to visually present military hardware in a manner that at once constitutes both a relationship between viewers and artefact based upon reverence and, more broadly, a shared orientation towards state violence as an object for pleasurable consumption. Following Simondon and actor-network theory, to adequately account for the viewer-weapon relation, the social semiotic conception of interaction is extended to include nonhuman actors as well as human ones. In short, the paper asks how militarism is realized in visual discourse through the constitution of affective relations with and through objects as effective devices of killing.

1 Introduction In this chapter I should like to begin to mark out how social semiotics can be applied to analyze how television entertainment programming promotes support for military procurement programmes and, ultimately, state violence. In particular, I am interested in the ways in which militarism is disseminated through the use of military hardware as mediating objects between public and military. As a practice of discourse analysis, I will argue that social semiotics is particularly well suited to this because fundamentally semiosis is understood as an interaction as well as a representation. In short, the ideological success of militainment programming is its ability to foster affective relations between viewers and objects and this is accomplished through the articulation of semiotic resources that not only represent weapons as actors but also position the weapons as interactants.

2 Encountering military artefacts To those who have seen it, what is striking about Fiona Banner’s Harrier and Jaguar installation at Tate Britain is the manner in which it effectively breaks with the conventions of how we have come to visually encounter military hardware. In the 2010 exhibit, Banner suspended a Sea Harrier that had once been in service in Bosnia nose

72

Ian Roderick

down from the ceiling and a Jaguar that had been in service in Desert Storm was placed “belly-up” on the exhibition floor. Compared to a trussed bird and a submissive animal respectively, the exhibit counters how military hardware conventionally comes to appear and what that appearance is intended to accomplish. Conversely, this chapter, in focussing upon what has come to be referred to by journalists and academics as militainment 1 television, examines how military hardware does conventionally come to appear and how that appearance is tied to a pleasurable form of consumption of state violence (Stahl 2010: 6). While militainment has been used to describe any form of media that spectacularizes warfare, my focus is upon how military technologies and techniques have been adapted to the genres of reality television. Of particular interest are the ways in which the visual presentation of military hardware extends beyond treating them as effective technologies for killing to being objects of affect. In such programming, each weapon is never simply discharged but is enthusiastically discussed, appreciated, compared, and admired thus taking on the status of aesthetic object. In short, weapons are not simply presented as utilitarian tools but instead militainment endows weapons with emotive as well as technical-rational value. It will be argued then that militainment television programming reliably draws upon regularized sets of available semiotic resources so as to visually present military hardware in a manner that facilitates an approving if not reverent orientation. A social semiotic approach is therefore proposed to examine the ways in which the weapon as technical object comes to be constituted as something with which the militainment viewer is able to enter into a “positive” interactive relationship. In one way, the manner in which the weapons are presented abstracts them from their practical use and the gruesome consequences of their use but in another, the weapons are closely tied to their capabilities by clinically examining just what they can do and speculating what would happen to those who might have been the targets. However, the weapons being presented are not only understood as objects of knowledge but also have an ontological status and are therefore implicated in the political subjectification of the viewers themselves. The act of representing the weapon to the viewer makes that weapon come into existence for the viewer, what Simondon (1980) terms the genesis of the technical object, as a political-aesthetic object but also as an actor. As such, it is necessary to radically expand the way in which interaction is understood so as to include nonhuman as well as human participants as interactants. It is proposed therefore that what militainment television does is a two-fold process of semiotizing the process of constituting the weapon system as a distinct technical object and, by extension, resemiotizing (Iedema 2001; 2003) the relationship between militainment viewer and military artefact as an operation of recruiting and enrolling the viewers as popular militarists.

 See Stahl (2010: 6–7) for an etymology of the neologism.

Military hardware as affective objects

73

3 A social semiotic approach Social semiotics is a critical approach to communication that has developed from a wide range of interdisciplinary scholarship. Those engaged in social semiotics tend to draw to varying degrees from both the Parisian school of semiotics and the systemic functional school of linguistics associated most closely with Michael Halliday. In adopting the Hallidayan view of language as one of a number of possible semiotic modes, social semiotics has distinguished itself from earlier Barthesean conceptions of semiotics by moving beyond a focus on codes and signs to stress instead the way people draw from available semiotic resources to both produce and interpret communicative artefacts and events. So while signs are typically construed as meaningful because they have an already attributed meaning semiotic resources have instead “meaning potential”, which is derived from both previous conventionalized use and those possible uses afforded to users within the specific social milieu in which the communicative act takes place (see Van Leeuwen 2005: 3–6). While the legacy of Parisian semiotics has been to treat the various semiotic modes – image, sound, verbal text, etc. as distinct forms that each “bear” meaning, the recent emphasis upon multimodality in linguistics and CDA has foregrounded the ways in which meaning emerges through the interaction of multiple semiotic modes. In this way, social semiotic analysis tends to compare and contrast the semiotic modes and explore the forms of cohesion that are available to both produce and interpret texts. In privileging the concept of resource over intrinsic systematicity or code, social semiotics attends to the ways in which social actors are able improvise with as well as regulate the use of semiotic resources. Social semiotics also takes from Halliday the notion that any communicative act can, at some level, be understood as functional. Social semiotics contends that semiotic resources are socially organized to accomplish specific communication tasks. These tasks are realized through what Halliday has termed metafunctions: the ideational, the interpersonal, and the textual. Accordingly each communicative act draws upon semiotic resources to do three things simultaneously: 1) produce representations, 2) create interactions between social actors, and 3) synthesise those representations and interactions into specific kinds of texts. Every act of semiosis, therefore, performs each of these functions which are in turn realized by three corresponding types of meaning: representational, interactional and compositional, respectively. In this way, because meaning is as much interactional as it is representational, all communicative acts constitute the reality of relations (Mackenzie 2005: 384), what Simondon terms relationality, as much as they do the reality of the representation itself. For the most part, social semiotics has tended to treat objects as carriers of connotative meaning and equate human actors with social actors. In part, this reflects a bias towards thinking of agency as an intrinsically human property. As

74

Ian Roderick

a result, the interactional has tended to be limited to three kinds of interactions: between human communicative interactants, between represented human social actors, and between human communicative interactants and represented human interactants. I would propose a much more radicalized understanding of interaction, following from Simondon’s mechanology and from actor-network theory, which extends the social semiotic conception of interaction to non-human actors as well as human ones. This does not mean that grammatically there cannot be subject-object structures but simply that we need to broaden our thinking as to what constitutes a social relation and that social relations can be formed with as well as through objects.

4 The individuation of the object and the production of relationality For Simondon, technology is profoundly relational. A technical object cannot have any sort of autonomous existence independent of the associated milieu in which it is embedded. Both the object of knowledge itself and the knowing subject are mutually implicated in one another such that they are always already co-constituted in relation to one another. The being of one is always defined in relation to the being of the other. Therefore, what Simondon proposes is not simply a social constructionist argument about technology in which the social writes itself onto or even into the object but rather a theory of socio-technical relations in which the one is always implicated in the other. Rather than understanding the technical object as having arrived before us in an already finalized form, Simondon asks that we consider it as some“thing” inprocess: “[e]very technical object undergoes a genesis” such that the “unity, individuality, and specificity of a technical object are those characteristics which are consistent and convergent with its genesis” (Simondon [1958] 1980: 18). What gives the object its specificity, its “being”, is already implicated in its “becoming”. The “thingness” of the technical object is therefore not a priori to its social existence but, at the same time, the social is always already constituted in and through the object. As Del Lucchese (2009: 181) succinctly puts it, “[b]eing is not what ‘is’ […] Being is what becomes in and through relationality”. Technological objects, in this way, are not defined in terms of intrinsic properties but rather are constituted as discrete entities only insofar as they can be individuated and this is always accomplished relationally. Simondon terms this process of individuation, transduction. As Mackenzie (2005: 393) summarizes, transduction “leads to individuated beings, such as things, gadgets, organisms, machines, self and society, which could be the object of knowledge”. By proposing that technological objects undergo a process of genesis, Simondon is attempting to alert us to the conditions under which artefacts

Military hardware as affective objects

75

come into existence and how those conditions are both a part of the object’s being and are themselves transformed by the emergence of the object. The technical object comes into existence or becomes “as a specific type that is arrived at at the end of a convergent series” (Simondon 1980: 18). Such a series is understood by Simondon as a movement from an abstract to a more concrete mode of existence whereby “the technical being becomes a system that is entirely coherent with itself and entirely unified”. Simondon thus conceives the genesis of the object to be the product of twinned processes of concretization and differentiation. Through individuation, therefore, technical objects are at once concretized as individuated entities and thus both constituted as part of but also differentiated from a lineage of antecedent technical objects. At the same time, these lineages are themselves the product of concretizations such that for example, the knight on horseback can be discursively constituted as the antecedent to the armoured tank. Mackenzie notes that within the SCOT literature, with its rejection of deterministic, humanist accounts of technology as a kind of negating force, there has been a tendency to equally by-pass considerations of the role that more enthusiastic promotional discourses or quite simply “hype” might play in constituting the technical object: “the cost has been been an immunity to the idea that the hype might be constitutively part of technology as we know it, that hype can get under the skin, so to speak, of technologies, whatever their materiality or sociality” (Mackenzie 2005: 383). For Mackenize, attending to any hype that might surround the technical object affords consideration of affect, which he finds lacking in SCOT accounts of technology. As he continues, hype “does connote an excess of feeling or affect, an over-attachment to an image or figure, usually circulating through mass media or publicity mechanisms” (Mackenzie 2005: 383). Mackenzie (2005: 393) proposes that alternatively Simondon’s conception of transduction foregrounds the equally constitutive role that affective relations play in the genesis of the technical object since it “extends to perception, feeling, acting, knowing and thinking as ontogenetic processes”. If we forge relations with and through technology then this is accomplished affectively as much as it is socially. In this way, “collectives can come into being around technical activity” (Mackenzie 2005: 394) and it is in this way that technology, as a materialization of sociality, is inherently political. What I am proposing to do for the remainder of this chapter is identify some of the primary ways semiotic resources are drawn upon to produce the technofetishist hype essential to militainment and popular militarism. In adopting a Simondonian approach to the (re)presentation of technical objects, my intent is to argue that the meaningfulness of objects is not given or self-evident but neither are they tabula rasa, that meaning does not just “stick” to objects, and that the techno-fetish relation therefore constitutes more than just a relationship of displaced desire between object and subject but rather is implicated in a broader set of relations that define a particular orientation to the political space of communal

76

Ian Roderick

life. Socio-technical relations are always expressly political since “[r]elations to others, to machines or technical ensembles, and relations to self are knotted together in Simondon’s account through processes of individuation […]” (Mackenzie 2005: 395). If we want to understand militainment as more than simply taking pleasurable interest in the military and that there are real consequences in terms of both political-subject formation and the kinds of technoculture that we can foster, then greater attention needs to be given to the production of individuation and relationality between viewer and military artefact.

5 Militainment Militainment characterizes a wide range of different mass media forms from films that collaborate with and venerate the military to news coverage that depends upon largely decontextualized combat news footage for its news value. Militainment, at some level, depends upon the blending or hybridizing of fact and fiction. Fictional representations of combat such as Saving Private Ryan (Spielberg 1998) or more recently The Hurt Locker (Bigelow 2008) and even the Transformers (Bay 2007) film franchise must be deemed realistic in terms of the weapons, uniforms, language, and so forth that appear on the screen. Factual representations such as the reporting of the repatriation of Private Jessica Lynch must be spectacular enough to hold the attention of the audience. As Stahl (2010: 6–7) notes, the neologistic quality of the word points to the ability to join together two supposedly incommensurate things – militarism and entertainment – in the same vein as “infotainment” and “advertorial”. Perhaps initially inflected with some implied critique, any ironic tone in these latter two terms seems to eventually disappear. Accordingly, that militainment is produced in such a wide variety of forms seems to suggest that in actuality these two things are not so incommensurable as we would like to imagine and, perhaps more than anything else, speaks to the degree to which PR and publicity has come to permeate contemporary culture. The variety of militainment that will be addressed here falls on the factual end of the cline. It can be characterized, following from Fürsich (2003), as nonfiction entertainment. Fürsich (2003: 132) coins the term to describe a genre of media content that is “situated between traditional journalistic or public service values and commercial interests” and tends to readily incorporate freely-provided content that is characteristically promotional in intent. The tendency to seamlessly integrate publicity material into nonfiction entertainment means that it is characteristically “non-critical, non-political, combined with a focus upon celebratory worldwide understanding” (Fürsich 2003: 132). In the case of militainment, it often means that the only individuals who will appear discussing the weapon in question are the employees and paid consultants representing the defence contractor along with military personnel assigned to that particular procurement program.

Military hardware as affective objects

77

The result is a form of “documentary” that depoliticizes and decontextualizes military conflict and uncritically supports the defence contractor, in particular, and the military-industrial-entertainment complex,2 in general. Nonfiction entertainment is largely a televisual genre and effectively describes much of the programming content produced for specialty cable channels such as those owned by Discovery Communications Inc. and A & E Networks. The militainment programming to be discussed below all appear on The Military Channel, which is owned by Discovery. First coming to air in 1999 as Discovery Wings, it was rebranded as The Military Channel in 2005.3 As the name would suggest, the channel specializes in the delivery of content produced by Discovery Inc. itself and smaller independent production companies that can easily be characterized as militainment. The specific form of militainment programming that this chapter focuses upon showcases and profiles weapons and weapon systems for a lay audience. This sort of programming tends to be either historical, detailing a particular weapon from a particular conflict or offering a countdown of the “best” of a particular weapon over a period of time, or contemporary, simply showcasing the new and “innovative” or providing a breakdown of the best in a particular category. The two programmes that are discussed here, FutureWeapons and Ultimate Weapons, both focus upon current and new armaments, discussing in detail what makes them effective in combat. The immediate origins of this type of content can be traced to the reporting of the first Gulf War from 1990–1991. Immediately after the war, A&E Networks aired Brute Force: Weapons at War (with hosts deliberately chosen based upon their perceived positive connection to the military: George C. Scott in the first season and then Gerald McRaney in the second). Stahl (2010: 44–45) argues that the televisualization of Operation Iraqi Freedom in 2003 “progressively favoured first-person relationships with weapons, fanning the gaze out through an array of machines”. The same argument has also been made of the first Gulf War in 1991 as viewing audiences were given facts and figures for the Patriot Missile System, the M1 Abrams Tank, the JDAM and so forth much in the same manner as star athletes. Watching news reports on CNN and the major networks, the television audience quickly became accustomed to broadcasts of Pentagon released grainy optical sighting system videos of ordnance striking wherever the cross-hairs happened to be. Indeed, in arguing that state-sponsored warfare has transformed from spectacular to what he characterizes as interactive, Stahl (2010: 44) acknowledges that “[i]ntegrating the citizen subject into the machinery of sighting has been a  See Der Derian, James (2001) and also Machin, David and Van Leeuwen, Theo (2009).  Though it was trademarked in 2002 after the name became available with the failure of another independent specialty cable channel of that name failed. The relatively easy shift from Wings to Military Channel is not surprising, given the close integration between the aerospace industry and the military-industrial complex – much of the programming of Wings was already militainment.

78

Ian Roderick

motif since Desert Storm in 1991”. It is this notion of militainment as a means of integrating the viewer with the object of the gaze and through it, integrating the viewer into a position of supporting militarist state policy that is of crucial concern here. Militainment therefore works semiotically to create a militaristic relationality through a processs Iedema terms resemiotization whereby the representation of the weapon as object is in turn implicated in a set of broader relationships in support of state violence.

6 Resemiotization: semiosis as relationality Drawing in part from Jackobson’s conception of intersemioticity and Actor-Network Theory4 concepts of delegation and translation, Rick Iedema has introduced the term of resemiotization to describe the process by which meanings are translated from one domain to another into increasingly more durable and therefore presumably less negotiable materialities. Resemiotization highlights “the role of material reality in communication” (Iedema 2001: 24) and the ways in which, through delegation, the material is expressive and the expressive is in turn materialized. Looking at communication through the lens of resemiotization necessitates explicitly attending to “how meaning making shifts from context to context, from practice to practice, or from one stage of a practice to the next” (Iedema 2003: 41) as a series of delegations or displacements and that practices of meaning making are therefore intrinsically intersemiotic or what is more frequently termed multimodal. But these delegations are not limited solely to what are typically thought of as semiotic modes (talk, writing, images, music, etc.) and instead extend to the material itself. As an example, Iedema cites Latour’s (1992: 235) example of the door closer and the delegation of the program of keeping the door closed from verbal request, to written notice, to the installation of a hydraulic door closer. Thus, in drawing from ANT, Iedema (2001: 36; 2005) foregrounds the intrinsic heterogeneity and materiality of semiosis by characterizing it as being generative of “meaning-materiality complexes”. Therefore, when we attend to the process of resemiotization, we are, in effect, conceptualizing semiosis as an emergent, heterogeneous, relational phenomenon. We are, in fact, not so far off from what Akrich and Latour (1992: 259) characterize as semiotics in that we are not treating meaning as reducible to signs. According to Iedema (2003: 31), the multimodal turn in social semiotics has been facilitated in part because, in keeping with Halliday’s systemic-functional linguistics, “semiosis [is] not analysed in terms of discrete building blocks or structures, but in terms of socially meaningful tensions and oppositions which could be instantiated in

 Actor-Network Theory or ANT is often situated under the umbrella of social construction of technology (SCOT).

Military hardware as affective objects

79

one or more (structural) ways”. If semiosis is already understood as the product of relations and multimodality has increasingly stressed the heterogeneity of those relations at the level of the text, then it should not be such a great leap to begin to think of meaning-making transductively. Consequently, since the “hype” is a constitutive element of the artefact itself (not just what can be “known” about it but instead how the object comes into “being”) then a social semiotic approach is ideally suited to critically analysing the individuation of the technical object and, ultimately, the kinds of relationality in which it is implicated. In the case of militainment, military artefacts can be understood as more than carriers of techno-fetishist hype since the very processes of hype contribute to the artefacts “being”. Furthermore, in keeping with the concept of relationality, that hype not only constitutes the object itself but a broader set of transindividual relations brought about through the resemiotization of state violence for pleasurable consumption. In other words, what is being resemiotized in the militainment programming is not only the representation of a technological object as an instrument of state violence but the representation and ultimately generation of a political constituency premised upon a kind of camaraderie with and through the military hardware. Accordingly, I will be arguing, militainment does more than entertain; it works to enrol the audience as popular militarists.

7 The social semiotic resources of individuation In the production of the militainment television programming content being discussed here, semiotic resources are drawn upon and organized so as to present the weapons in question as both objects of knowledge and interaction. The technofetishist hype of militainment is realized through the regularized selection of sets of semiotic resources that tend to delegate agency and affect to the weapon itself. This hype is, in turn, understood to resemiotize the individuation of the weapon as technical object into a broader relationality of popular militarism in which object and viewer are co-constituted. In keeping with the functional approach of social semiotics, the political project of popular militarism can be explored in such programming through an analysis of ways in which resources are used to accomplish representational and interactional meanings conducive to the generation of affective relations with and through objects presented as effective devices of killing. Programmes such as FutureWeapons and Ultimate Weapons literally represent military hardware to their audiences, providing factual information about each weapon in terms of manufacturer, origins, capabilities, and usage. Of particular interest is the reliably consistent manner in which the weapon being profiled is initially presented to the viewing audience. Kress and Van Leeuwen (2006: 79) have argued that most, if not all, visual representations can be construed as adopt-

80

Ian Roderick

ing either a narrative or a conceptual structure. In the case of the two programmes, a conceptual representational structure is consistently utilized to present each weapon to the audience. So while narrative representational structures present social actors as participants engaged in some sort of activity, conceptual processes represent social actors in terms of their attributes. And while the viewers are ultimately shown the weapons in action, what is being emphasized and presented for the audience are their attributes. This is most obvious in the case of Ultimate Weapons, which, in keeping with the weapon-as-athlete motif, actually presents at the start and end of each profile what looks like a trading card, with the name and image of the weapon on top and ratings for the weapon across five key attributes on the bottom of the card. So for example, for the Dillon Aero Gatling Gun, it gets 3 bullet holes out of 5 for Accuracy, 2 for Range, 4 for Cost, 3 for Destructive Power, and 2 for Portability (30 June 2009). Less obvious but consistently used in both programmes is the use of extreme closeups of each weapon so that the viewer is initially only shown partial shots before seeing the weapon as a whole. This form of visual presentation is consistently done with everything from small arms to larger weapons systems such as submarines and aircraft. In effect, the weapons are being presented through what Kress and Van Leeuwen (2006: 87–92) term analytical representation structures. Analytical processes are a sub-type of conceptual structure whereby the participants are represented as a part-whole relation. The parts function as Possessive Attributes that are in turn carried by the whole. Thus when being presented with an automatic shotgun in FutureWeapons (17 March 2007), the shotgun is initially seen as a series of partial shots as it is loaded and fired by the host (drum magazine, trigger, butt, rear sight, rear and front sight, rotating magazine) before being presented in a full sideways shot. Following from Kress and Van Leeuwen (2006), it can be argued that the viewers are being afforded the opportunity to analyse the shotgun (as Carrier) based upon its Possessive Attributes of weapon-ness. This part-whole analytical structure very much redounds with the Simondonian conception of the concretization of the weapon as technical object and is the most frequently used form of visually introducing the weapon in both programmes. At the same time, while the weapons tend to be represented visually using analytical structures and therefore as objects that can “offer” information to the viewing subject, linguistically there is a tendency to present the objects as having a stronger agency of their own. Van Leeuwen (2008: 28–29) maintains that in the representation of any given social action, some actors may be excluded as participants in the text either through suppression or backgrounding. In the case of suppression, a social actor has been excluded entirely and there is no reference to the actor in the text. A backgrounded actor, on the other hand, is represented but their role in the represented action is de-emphasized in the text. Correspondingly, social action can therefore be represented as being either caused by (agentalised) or independent of (de-agentalised) human agency (Van Leeuwen, 2008: 66–68).

Military hardware as affective objects

81

Accordingly, as each weapon is introduced, the weapons themselves are frequently agentalised by backgrounding or even suppressing the human actors within the voice-over narration. Elsewhere, I (Roderick 2010) have discussed how military robots tend to be visually represented as autonomous agents but in the case of weapons such small arms, the operator will obviously be visually present. Even so, there still seems to be a predilection for imagining the weapon as being able to act in some way autonomously. In keeping with the techno-fetishist quality of militainment, the enthusiastic talk about these weapons often gives them a “life” of their own. So for example, the host of FutureWeapons (17 March 2007) introduces the TDI Kriss submachine gun in this way: Narrator: I am about to try out a radical new submachine gun design. There is a new weapon coming to the world of close quarter battle. It’s named after the Indonesian sword that cuts down its enemies. In the past, some submachine guns were simply known as “spray and pray”. Shooting them on full auto was not so much a tactical manoeuvre. More like an act of faith. Even though this prototype fires the massive .45 cal. pistol round, this is one gun that doesn’t need divine intervention.

After the first line, the host effectively backgrounds himself and suppresses all other human actors. The weapon comes into the world without human agency, it is named without a “namer” and its namesake “cuts down” its enemies independently of human hand and even past submachine guns have been known without a “knower”, shot without a “shooter”, and believed in without a “believer”. Ultimately, the Kriss submachine gun can, as it is verbally represented, seemingly operate independently of both divine and human intervention. The same kind of agentalisation can be found in Ultimate Weapons (2 June 2009). Narrator: At Number 5, it’s the airborne version of the heavy artillery arsenal. It’s the AC-130U gunship also known as the Spooky. Having a heavily armed, fully loaded gunship above a conflict zone provides a strong psychological as well as military advantage. It can offer support for ground troops below or launch full-scale artillery strikes of its own. And when the gunship in question is the AC-130 Spooky, attack is name of the game. Air Force Tech Sgt Gregg Jordan: It’s an attack aircraft and the weapons put the attack into it. If it wasn’t for the weapons being on there, it would have no bite.

Again, there is no identified “knower”, no “possessor”, no “armer”, no “supporter”, and no “player”. The only human actors identified are the ground troops who are beneficialised (see Van Leeuwen 2008: 32–35) by the support of the AC130U. Aside from stating the rather obvious, the closing comments to the segment

82

Ian Roderick

offered by Technical Sergeant Jordan reiterate the notion that the weapons do it all independently of human intervention. Representationally then, the programming discussed works to introduce the weapons being showcased to the viewing audience in terms of attributes and capabilities. The viewer is shown quite literally what the weapon can do though always to an inanimate object rather than a flesh and blood being.5 Visually, this is most often accomplished through the use of tight close ups at multiple angles leading to a full shot of the weapon, thus realizing a conceptual-analytic structure. Verbally, the weapons are typically presented not only as extremely effective in what they do but also in a techno-deterministic manner such that the humans play a diminished role in their actual operation. What the two semiotic modes do together is realize a representation of military hardware that lends itself to enthusiastic discussion that attributes not only capabilities but agency too. Militainment does more than promote military hardware as effective. It also, by its very interpellative function, promotes them as affective objects as well. This depends upon a kind of subjective investment in and delegation to the weapons being promoted and is realized semiotically by the formation of interpersonal relationships with the individual weapons as they are profiled. In other words, transduction, as a process of individuation “leads to individuated beings such as things, gadgets, organisms, machines, self and society, which could be the object of knowledge” (Mackenzie 2005: 393) but can also be the objects of affect. By calling these weapons affective objects, I therefore mean that they become objects of attachment and that the viewers therefore come to feel interconnected to and through these objects. In considering the representation of social actors in visual communication, Machin (2007: 109) proposes that we need to be concerned with the ways in which semiotic resources are used to accomplish three kinds of viewer-participant orientation: how selected resources position “the viewer as regards the participants in the image, how we are encouraged to relate to them, and how we are encouraged to assess them”. The first orientation is largely realized through gaze, angle of interaction, and distance. The second entails how depictions determine the kinds of participants that are represented in terms of individuals vs. groups, categorization, and non-representation/exclusion. The third, then, considers the kinds of action and roles that the participants perform, following from Halliday, as processtypes. Together, these three forms of viewer-participant orientation define the attitudinal, axiological and, by extension I argue, affective orientation the viewer is expected to adopt towards the represented participants.

 By and large, the programs are quite clinical in how they demonstrate the effectiveness of the weapons. Vehicles, buckets filled with water, mannequins, blocks of ballistic gelatin come under fire but with the exception of a watermelon being exploded by sniper fire, there is a complete absence allusion of the viscera and, by extension, suffering.

Military hardware as affective objects

83

The viewer-weapon orientation is the most obvious facet of the way in which the techno-fetishism of militainment is realized. What is evident in FutureWeapons and Ultimate Weapons is that the viewer is consistently invited to adopt a venerating orientation to the weapon as it is introduced. This is accomplished through a frequent meeting of “gaze”, angles of interaction that often suggest reverence, and a distance between viewers and object that connotes intimacy. Kress and Van Leeuwen (2006) have argued that the Hallidayan schema of offer/demand interaction in speech acts is also realized in visual forms of communication through the direction of the gaze of represented participants. Essentially, if the represented participant is gazing towards the viewer, some sort of social relationship or interaction is being implied between the two. If on the other hand there is no “mutual gaze” between viewer and the represented participant, then there is no relationship being implied and the participant takes on the status, in essence, of object. The degree to which the gaze is direct or oblique gives an indication of how direct or “demanding” the represented participant’s relationship with the viewer actually is. Conversely, the degree to which the viewer’s gaze is not met by the represented participant conveys a relationship of offering – not meeting the viewer’s gaze, the participant is literally in no position to make demands of the viewer. In both programmes, even though it would be a gross violation of range practice, the viewer will frequently end up looking down the barrel of the weapon being showcased either before or after it is trialled. Though this is frequently done in FutureWeapons, it is also a familiar motif in Ultimate Weapons. For example, when the Stryker MGS is profiled, it includes a series of shots of the Stryker moving rapidly across the screen with the main gun pointing toward the viewer (30 June 2009). Likewise, when the Cheytek sniper rifle is demonstrated, the viewer actually first encounters the rifle by looking directly at the muzzle of the weapon (26 May 2009). In FutureWeapons, when it is a small or light arms weapon, this point-ofview shot is even more frequently used with the host, having handled the weapon in the lead-up to discharging it, then points it directly at the camera before the trial actually begins. Alternatively, when the weapon being profiled is much larger like the B2 Bomber, the shots both programmes use are likely to be front-on from the outset. Because looking tends to be equated with power, having the weapon “look back” at the viewer gives the weapon not only a more menacing quality but also gives the object “life” which is essential for it to have fetish status: “It really can “do things” and you would not want to be at the receiving end of this”. This is enhanced by the fact that the windscreen glass is often front lit so that the glare causes the glass to be opaque and thus prevents the viewer from seeing into the vehicle much in the same way that sunglasses can be used for intimidation by preventing seeing into the wearer’s eyes. In addition to the degree of returned gaze, the vertical angle of the interaction also provides information about the implied relationship between viewer and

84

Ian Roderick

weapon. Upward-looking low angle shots tend to make things look bigger than they are and therefore more impressive in relation to the viewer. Moreover, the meaning potential of low angles tends to realize asymmetric relations of power and privilege in which the viewer is subordinated to the elevated represented participant. While obviously not every shot of the weapon is going to be a low angle shot, nonetheless, many are. In episodes focusing upon heavy weapons and aircraft it is both usual and easy to employ low vertical angles of interaction. Looking up at the PZH 2000 or the MLOS self-propelled guns readily puts the viewer in a position of feeling diminutive in comparison (Ultimate Weapons 30 June 2009). However, this angle of interaction is also frequently used to present smaller weapons as well. For example, when the FutureWeapons host fires the TDI Kriss (March 17, 2007) and the H&K MP-7 (Jan 29, 2007), in each case, close ups of the firearms are taken from a low vertical angle. This is obviously not done to make them look bigger since their compact size is one of their special features. Instead, the low angle works to present these small arms as nonetheless powerful, intimidating, and impressive. The need to use low angles of interaction even results in it being used to introduce the Dragon Fire II mortar though it is only waist high to the host. While horizontal and vertical angles tend to realize engagement and status, distance in visual communication tends to realize degrees of intimacy. Close social relations tend to be represented through close physical distance while more impersonal social relations are represented through greater physical distance. A closeup suggests that the viewer can get physically close and therefore has a social relationship with the represented participant that permits such closeness. A longshot, on the other hand, suggests detachment from the represented participant. What we find in both programmes, as already discussed above in reference to the use of part-whole analytical structures is that when the weapon is initially introduced, there are a series of extreme close-ups as the viewer examines parts of the weapon before seeing it in its entirety. The viewer is therefore initially brought up close and given an intimate look at the weapon. Indeed, on at least two occasions the host of FutureWeapons makes reference to “getting up close and personal” with the weapon being considered. The second kind of viewer-participant orientation requires that we consider how the weapons are represented in the programmes as kinds of participant. For Machin (2007: 118), in visual communication, this sort of orientation is realized in three key ways: individuals and groups, categorization, and non-representation. What concerns us most here is individualization-collectivization whereby participants can be represented as either individuals or in groups but to varying degrees. The specificity of the individual can be modulated by changes in distance for example. Likewise, groups or crowds can be represented as undifferentiated and homogeneous to varying degrees. Furthermore, as Machin (2007: 119) points out,

Military hardware as affective objects

85

it is also possible for represented participants to be in some aspects individualized and in others collectivized. Given the express purpose of the programmes is to showcase unique and innovative weapons as they are developed by key players in the arms industry, it should come as no surprise that the weapons are almost exclusively individualized. Viewers are almost invariably presented one model of the weapon and see it trialled alone, even in the case of weapons that have long since left the prototype stage. At the same time, in keeping with my argument, this individualization must be understood as a resemiotization of the individuation of the technical object. It is always “this” weapon that the viewer is being introduced to rather than a new class, category or line. The one weapon does not stand for all of that new type; it is the new type. The third form of viewer-participant orientation addresses agency and types of action. I have already discussed above how the agency of the weapons is determined verbally by the suppression of actors. Visually it is not so easy to suppress or background the role of human actors when the weapon is hand operated but in the case of larger weapons this is more readily accomplished. When FutureWeapons (17 March 2007) showcases the Grizzly Armoured Personnel Carrier, it is recorded only from the exterior, moving at an angle towards the camera but backlit. The next shot is of the vehicle stopped under the same unusual lighting conditions but with more detail and recorded in one continuous pan from rear to front and then up to the driver’s windscreen. The third shot is a tight close up of the front nose of the vehicle in full daylight and zooms in to an extreme close-up as the vehicle moves forward. The fourth shot starts at the front wheel and then pans up and across the passenger area of the vehicle (the back of an armed passenger can be seen aiming a rifle in the other direction) and then down to the rear wheels. The final introductory shot shows the front half of the vehicle moving away from the camera and then zooms out to show the entire vehicle from the side. This is the first full view in proper lighting. The only human actor visible in this sequence is the passenger. Likewise, when the B2 stealth bomber is profiled in Ultimate Weapons (2 June 2009), the bomber is revealed front-on with the opening of the hanger doors. The next shot is of the back of the bomber backlit so that it can only be seen in its distinctive profile. The third shot is a fast pan front-on from wing tip to wing tip. The fourth is then a front-facing fast zoom from medium close up to close up of the nose and cockpit area of the aircraft. The fifth is a pan shot that only reveals the undercarriage and landing gear. The sixth and final shot (most likely lent footage) is a full panning shot of the aircraft taking off past and flying away from the camera. It would not be difficult to argue that in these cases, the human operators of the vehicles have been suppressed and that this is more a matter of non-representation that contributes to the first kind of viewer-participant orientation. However,

86

Ian Roderick

that interpretation does not fully consider the agency being accorded to the vehicles in the profile segments and how it semiotizes the individuation of the weapon as technical object for the viewers. Clearly, in segments such as these, it is the weapon that is performing the action. Furthermore, the types of actions that the weapons typically perform can be said to shift from existential whereby actors are represented as being or existing (the views of the weapon using analytical structures) to material in which actors do something that has a material consequence (when the weapon does what it was designed to do such as discharge rounds into or drop bombs on a target). Militainment, as I have already suggested, depends a great deal upon claims to truth and realism. Realism, of course, shapes and gives verisimilitude to the “interaction” between the viewer and the profiled weapon but it is also implicated in positioning the viewer in relationship to the program and the producers. Documentary in form, the programmes draw upon what Kress and Van Leeuwen have called “modality cues” which function interpersonally to produce “shared truths aligning readers or listeners with some statements and distancing them from others. It serves to create an imaginary ‘we’” (Kress and Van Leeuwen 2006: 155). Modality, therefore, as Machin (2007: 48) notes, is not about the truth or falsity of a given representation but rather the shared orientation to that representation based upon its claims to truthfulness. Visual modality is constituted through the modulation of detail, background, depth, colour, and tone. Since the programming puts the weapons in the spotlight, in the initial presentation they are almost invariably presented with a maximum articulation of detail with a high degree of illumination. At the same time, since it is the weapon that is to be foregrounded, the articulation of the background and depth is often reduced. This is most apparent when the muzzle of the firearm is brought into sharp focus with a very short focal distance such as the in the case of the Cheytek profile on Ultimate Weapons. Since, overall, a naturalistic visual coding orientation (see Machin 2007: 58–59) is employed, tonal range and colour tend to be “true-to-life” in appearance. Sound modality, like visual modality, can function as naturalistic, abstract, or sensory according to Machin (2010: 175–180). Naturalistic modality entails sound being “true” to the “real” world. As Machin (2010: 180) notes, this depends very much on culturally determined conceptions of “naturalism”. Abstract modality uses sound to represent or symbolize the essence of something. Machin (2010: 180) gives the example of a musical instrument being used to represent a particular form of movement. Finally, sensory modality entails the affective qualities of sound such as the creation of mood. Interestingly, despite the claims of realism that these programmes depend upon, they tend not to employ highly naturalistic sound modality. The soundscape that is created tends to be “hifi” such that the sounds that can be heard can be heard distinctly but there is no background noise (see Machin 2010: 217). Sounds

Military hardware as affective objects

87

do not compete and the diegetic sounds are essentially limited to those of the weapon itself. While the viewer can invariably hear the sound of the mechanisms within the weapon operate and then the sound of it discharging, other sounds that one might hear outdoors on a range or even the sound of the operator walking across the ground are not recorded. The effect is that nothing is audibly competing with the weapon for the viewer’s attention. But it also creates a sound environment more typical of a soundstage than the locations in which the programmes are actually recorded. Music also features in the programming and it is primarily electronic. The choice of electronic music gives the programmes a rather technological and inorganic abstract sound modality. Finally, the music also clearly contributes to sensory modality sounds. The musical often conveys confidence and aggression with the use of power chords, quick tempo, increasing intensity and volume as the segment “climaxes” with the weapon being discharged. Indeed, it might also be argued that the sounds of “lock and load” as the mechanical systems of the weapons are articulated also function as sensory modality sounds. The militainment I have been detailing here therefore creates opportunities for the viewer to “interact” with weapons. As hype, the programmes can be said to be introducing viewers to the various weapons in a way that semiotically realizes the individuation of the weapon as technical object. This is accomplished representationally and interactionally. At the level of representation, the semiotic resources employed in the programmes concretize the weapons by visually making the attributes come together as whole weapon systems. At the same time, the programming also functions interactionally by orienting the viewer in relation to the weapons as represented participants thus building an affective relationship with the weapons as technical objects. Furthermore, the use of modality resources equally orients the viewer in relation to the programming itself and in turn extends these affective relations with military-technical objects into the genesis of a “we” founded upon a collective admiration of the military and its gadgetry.

8 Conclusion If we think of militainment television as more than simply taking an interest in the military and that there are real consequences in terms of both political-subject formation and the kinds of technoculture that we can foster, then greater attention needs to be given to the production of relationality between viewer and military technical object. What a social semiotic analysis should reveal is precisely the production of relationality across both concepts and materials. In the case of the militainment programming that I have addressed here, there are three overlapping and interconnected processes of translation or resemiotization being produced: firstly, there is the resemiotization of state violence as an object for popular con-

88

Ian Roderick

sumption, secondly, there is resemiotization of the genesis of the object as an object of popular militarism, and thirdly, there is the resemiotization of the audience as a collective in relation to the pleasurable consumption of state violence and the adoration of objects in support of militarism. At the same time, also in keeping with a social semiotic approach, it would be too reductive to treat this relationality as static or finalized. Meaning is always produced of potentials so semiosis is always an open rather than closed system. Likewise, Simondon’s approach to the individuation of the technical object also means that we need to rethink durability of object and meaning. Since transduction and therefore the technical object are always in a process of becoming there is therefore potential to “rewrite” the relationality in which the object is embedded so that it might become something else. Technology may be society made durable, but if the technology itself is not intrinsically durable, then there is no need to treat that claim as somehow deterministic. This is not to be so naïve as to claim that technical objects are value neutral and as such weapons become either good or bad only in use but rather to recognize that what might currently be transduced into relations of popular militarism could affectively be transduced from fetish object to abject object.

References Akrich, Madeline and Latour, Bruno. 1992. A Summary of a Convenient Vocabulary for the Semiotics of Human and Nonhuman Assemblies. In Weibe Bijker and John Law (eds.), Shaping Technology/Building Society, 259–264. Cambridge: MIT Press. Der Derian, James. 2001. Virtuous War: Mapping the Military-Industrial-Media-Entertainment Network. Boulder: Westview. Fürsich, Elfiede. 2003. Between Credibility and Commodification: Nonfiction Entertainment as a Global Media Genre. International Journal of Cultural Studies 6(2): 131–52. FutureWeapons. 2007. No Escape. Season 2, Episode 3. 29 January. FutureWeapons. 2007. Mission Invisible. Season 2, Episode 4. 5 February. FutureWeapons. 2007. Close Quarter Combat. Season 2, Episode 10. 17 March. FutureWeapons. 2007. Guns. Season 3, Episode 5. 20 December. Iedema, Rick. 2003. Multimodality, Resemiotization: Extending the Analysis of Discourse as Multi-Semiotic Practice. Visual Communication 2(1): 29–57. Iedema, Rick. 2001. Resemiotization, Semiotica, 137(1/4): 23–39. Kress, Gunther and Van Leeuwen, Theo. 2006. Reading Images: The Grammar of Visual Design. Second edition. New York: Routledge. Latour, Bruno. 1992. Where Are the Missing Masses? The Sociology of a Few Mundane Artifacts. In: Weibe Bijker and John Law (eds.), Shaping Technology/Building Society, 225–258. Cambridge: MIT Press. Lucchese, Fillipo Del. 2009. Monstrous Individuations: Deleuze, Simondon, and Relational Ontology. Differences: A Feminist Journal of Cultural Studies 20(2/3): 179–193. Machin, David. 2010. Analyising Music Music: Image, Sound, Text. Thousand Oaks: Sage. Machin, David. 2007. Introduction to Multimodal Analysis. New York: Bloomsbury Academic.

Military hardware as affective objects

Machin, David and Theo Van Leeuwen. 2009. Toys as Discourse: Children’s War Toys and the War on Terror. Critical Discourse Studies 6(1): 51–63. Mackenzie, Adrian. 2005. Problematising the Technological: The Object as Event? Social Epistemology 19(4): 381–399. Roderick, Ian. 2010. Mil-Bot Fetishism: The Pataphysics of Military Robots. Topia: Canadian Journal of Cultural Studies 23/24: 286–303. Simondon, Gilbert. 1980. On the Mode of Existence of Technical Objects. Trans. by Ninian Mellamphy. [Paris: Aubier, Editions Montaigne, 1958]. London: Univeristy of Western Ontario. Stahl, Roger. 2010. Militainment, Inc.: War, Media, and Popular Culture. New York: Routledge. Ultimate Weapons. 2009. Sniper Rifles. Season 1, Episode 1. 26 May. Ultimate Weapons. 2009. Shock and Awe. Season 1, Episode 2. 2 June. Ultimate Weapons. 2009. Heavy Metal. Season 1, Episode 6. 30 June. Van Leeuwen, Theo. 2008. Discourse and Practice: New Tools for Critical Discourse Analysis. New York: Oxford University Press. Van Leeuwen, Theo. 2005. Introducing Social Semiotics. New York: Routledge.

89

Christina Konstantinidou and Martha Michailidou

4 Foucauldian discourse analysis: Photography and the social construction of immigration in the Greek national press Abstract: This chapter outlines the theoretical premises and analytic strategies of Foucauldian discourse analysis and its usefulness in the study of photography. It then turns to the case of the visual representation of immigration in the Greek press in order to provide an example of the application of Foucauldian methodology. More specifically, the aim of the chapter is to test the usefulness of Foucauldian discourse analysis for the study of photography in the context of the concrete problematisations of immigration which appeared in the Greek press from the official outburst of the economic crisis in late 2009 to the 2012 general elections. The chapter argues that Foucault’s notion of discourse can be productive for the analysis of systems of visual imagery but the Foucauldian approach needs to be supplemented by social semiotics in order to analyse important aspects of photography otherwise neglected and throw light more fully on the relationship between photography and the social construction of reality. To this end, the chapter utilises analytical tools from Foucauldian discourse analysis, photography theory and media and migration studies in an effort to produce a multi-methodological reading of the construction of the discursive object lathrometanastis (a blanket term recently established to denote illegal immigrants) in the contemporary Greek press.

1 Introduction One of the most important aspects of visual communication is photography. Over the past 30 years the debate about the nature of photography has become central to the study of visual culture (Elkins 2007). Of central importance has been a theoretical debate as to whether photography should be understood, and analysed, primarily as a system of representation, as a form of discourse, or as a cultural and social phenomenon with specific use. While in early twentieth century debates it was assumed that photography represents the means by which one might grasp “objective reality” and the photograph was equated with a supreme form of authentication or evidence of the “truth” of events,1 the late 1970s and early 1980s saw the development of what might be called a “critical photographic discourse”,

 This is what Wright calls a “forensic” approach to the photograph, “in which we collate “factual” fragments and place them in temporal succession, in order to reconstitute the process of events” (Wright 1989: 66).

92

Christina Konstantinidou and Martha Michailidou

part of the rising postmodern critique of cultural and social systems (Batchen 1997: 4). This trend has brought with it a diversification of approaches to the study of the photographic image; among these, semiotics and Foucault’s work have been central for the production of a critical deconstruction of the truth status and representational strategies of photography. Photography and art theorists such as John Tagg, Alan Sekula, Rosalind Krauss, Griselda Pollock, Victor Burgin, and Suren Lalvani among others, drawing from Foucault’s conceptualisation of discourse, offered valuable insights on the social function of photography, analysing the material, institutional and ideological influences of photographic practices and thus contributing to the foundation of a coherent theoretical framework for the analysis of the photographic image. This chapter is an exploration of the usefulness of foucauldian discourse analysis as applied to the visual representation of immigration in the Greek press, focusing specifically on the period of the Greek crisis from 2009 onwards. The entry and permanent or temporary settlement of migrants and ethnic Greeks has been the object of heated debate between authorities, politicians, activists, policy makers, academics and the media in Greece since the early 1990s. A considerable amount of research has been conducted on the representation of immigrants in the Greek media, particularly in the printed media. Most of the findings point out the linguistic and semantic association of immigration with negativity (crime, violence, low social status, urban decay, deviance, poverty, country invasion, security threat etc.). Indeed, all relevant research asserts that news reporting plays a pivotal role in the production of immigration as a social and political problem and the construction of the immigrant as a multifaceted threat. Furthermore, a number of recent studies argue that negative stereotyping also takes place in the “progressive” press, as, in its pages, immigration is consistently associated with victimisation. This association is instrumental for the development of an antiracist humanitarian discourse and the denunciation of the resurgence of nationalism, racism, and xenophobic practices (Konstantinidou 2001). Greek media discourse on immigration can be divided into three broad chronological periods, which correspond to different phases of construction and conceptualisation of the relationship between the Greek self and the immigrant Other. Dominant discursive conceptualisations of immigrants as ‘ethnic’ or ‘cultural’ Balkan Others from the early 1990s to the early 2000s give way, from the mid 2000s onwards, to more “phenotypic” racialised conceptualisations of immigrants as Black or Asian (Muslim) Others. Since the outburst of the economic crisis in December 2009, however, we have been witnessing a further qualitative shift in media discourse on immigration. This may be understood, following Stanley Cohen’s ([1972] 2011) pioneer definition, as a “moral panic”, whereby immigration is produced, but also contested, through an amplification spiral, as an urgent social problem or even a threat, marking a shift from a society defined by consensus to one structured by deep political, economic, social and cultural crisis.

Foucauldian discourse analysis

93

In our view, this amplification and intensification of public discourse on immigration can be productively analysed as a “dispositif” or “apparatus”, “a thoroughly heterogeneous ensemble consisting of discourses, institutions, architectural forms, regulatory decisions, laws, administrative measures, scientific statements, philosophical, moral and philanthropic propositions – in short, the said as much as the unsaid […] which has as its major [strategic] function at a given historical moment that of responding to an urgent need” (Foucault 1980: 194). Hence, our aim is to examine the way the Greek press contributes to this dispositif, problematising immigration in a way that, while producing an apparently truthful account of it, it simultaneously reaffirms the production of the immigrant as threat or victim and ultimately as Other, a tendency embodied by the paradigmatic discursive object lathrometanastis. Lathrometanastis (pl. lathrometanastes) is a neologism with deeply discriminatory overtones, associated with a cluster of negative images that connotes irregularity and illegality and constructs a root metaphor of immigrants as a mass of devious and eventually dehumanised subjects.2 The chapter is organised in three parts. In the first part, we broadly set the theoretical basis of our research. We begin by setting out some key concepts of Foucault’s notion of discursive formation and offering a brief introduction to Foucault’s analytical strategies. We then focus on strands of photography theory which have been influenced by the work of Foucault, briefly defining and explaining a number of crucial interrelated concepts. Our aim is to demonstrate that the function of photographs is intricately tied to the institutions producing and using them and, through this, to the establishment and support of knowledge/power relationships, not by reflecting or distorting the “real” world, identities and social relations but, rather, by their active participation in creating and changing them. The second part concentrates on the case study of the representation of immigration in Greek newspapers, considering the process through which textual and visual discursive elements constitute the immigrant as “one of [Greece’s] deepest and most recurring images of the Other”, to paraphrase Said’s famous quote (2003: 1). The third section contains our concluding remarks.

 “Metanastis” means “immigrant”. The prefix lathro, deriving from the adjective lathraios meaning “furtive, stealthy, clandestine”, objectifies the displaced people by adding to the concept of “irregular immigrants” the derogatory connotation of lathremporion (lathraion-emporio) which means “illegal cargo, contraband, smuggling”. Moreover, the concept of “stealthy” connotes the state of being furtive in order to escape observation or the act or action of proceeding furtively, secretly or imperceptibly and, most interestingly, “the sneaky movement of an animal”: “the stealthy movement of a crocodile” – “cats are among the stealthiest of stalkers”. See MerriamWebster online dictionary, http://www.merriam-webster.com/dictionary/stealthy.

94

Christina Konstantinidou and Martha Michailidou

2 Theoretical and analytical framework 2.1 Foucauldian discourse analysis This section seeks first to provide a brief outline of Foucault’s notion of discourse and concise definitions of the key terms involved in his analytical strategies, before turning to the way these terms have been incorporated into the study of photography as a technology of truth. Putting forward a series of new conceptualisations about the character of power, subjectivity and knowledge in modern societies, Foucault elaborated a fundamentally novel idea of discourse, moving away from linguistic, grammatical or micro-sociological conceptions. Put briefly, according to Stuart Hall (1997: 44), “what interested him were the rules and practices that produced meaningful statements and regulated discourse in different historical periods”. Foucault’s notion of discourse evolved significantly from the earlier to the later phase of his own research (Howarth (2000: 48–49), McHoul and Grace ([1993] 2002: 26–31), Kendall and Wickham (1999)). In the narrower, quasi-structuralist, archaeology, performed in his earlier works The Order of Things (1970) and especially The Archaeology of Knowledge (1972), Foucault analyses discourse as “a group of statements in so far as they belong to the same discursive formation; it does not form a rhetorical or formal unity, endlessly repeatable, whose appearance or use in history might be indicated (and, if necessary, explained); it is made up of a limited number of statements for which a group of conditions of existence can be defined” (Foucault 2002 [1972]: 131). In The Archaeology of Knowledge Michel Foucault posits the “statement” as the elementary unit of archaeological analysis. According to Foucault (2002 [1972]: 89–132), statements are differentiated from logical propositions, grammatical sentences and speech acts even though they may contain one or more of these. A statement cannot be solely defined on the basis of its logical properties, nor by the speaking subject pronouncing it, and does not necessarily suffice for the accomplishment of a speech act. Instead, a statement consists in its enunciative function, or rather “is” an enunciative function which groups together a series of signs in a way that brings about specific transformations of the spatiotemporal field within which it is located and operates. Indeed, in The Order of Discourse Foucault gives an exposition of discourses as consisting of series of events which are systematically linked through the regularity of their enunciation, their positioning within specific epistemological and institutional limits and so on. A discursive event is an historically specific enunciation which takes place at the level of the material, as it: is always at the level of materiality that it takes effect, that it is effect; it has its locus and it consists in the relation, the co-existence, the dispersion, the overlapping, the accumulation,

Foucauldian discourse analysis

95

and the selection of material elements. It is not an act or the property of a body; it is produced as an effect and within the dispersion of matter (Foucault 1981: 69).

A statement then does not merely represent a person’s attribution of ephemeral meaning to a series of signs, and through this to a specific situation. Rather, according to Foucault, it is: an enunciative function that involved various units […] and instead of giving a ‘meaning’ to these units, this function relates them to a field of objects; instead of providing them with a subject, it opens up for them a number of possible subjective positions; instead of fixing their limits, it places them in a domain of coordination and coexistence; instead of determining their identity, it places them in a space in which they are used and repeated (Foucault [1972] 2002: 119).

So, to put it crudely, a statement is an enunciative function the occurrence of which does not just contingently order pre-existing (social and/or linguistic) elements (or floating signifiers), but the occurrence of which is the “production” of specific states of affairs in the manner just mentioned. The ontological status of statements as events with their own irreducible effectivity and materiality is due to their organisation into discursive formations of enunciation with their specific “positivity” (Foucault [1972] 2002: 141–145). This positivity organises the limits of the possibilities of “meaningful enunciations” of the speaking subject, the limits of the possibilities of “interpretations” of actually occurring utterances, and the assignation of “truth status” or value to these utterances. A statement therefore can only be a statement because it is produced by and located within a particular assemblage of enunciation, the essence of which consists in the organisation of a historically particular “system of enunciability” and corresponding “system of functioning” (Foucault [1972] 2002: 146). To put it differently, the meaning of statements is to be found in their historical organisation, not in the analysis of their constituent elements (words or signs); statements are speaking positions which are both actually and analytically “prior” to the words and signs composing them. Archaeological research then tries to identify the historically specific correlation of parameters which have allowed the appearance of some statements (as opposed to others, or silencing others, designating them to the realm of the unsayable) and the way these statements have operated as social relationships to position the subjects of enunciation in ways which produce, sustain, and modify recognisable configurations of knowledge and power. The function of statements then is to be found in the way that they draw from, correlate with, and shape wider “strategies” (Foucault [1972] 2002: 71–78 and 196– 219, 1979). These strategies do not simply constitute the social and political context of enunciation of particular statements; rather, strategies can only exist insofar as particular statements articulate them and thus bring them into operation at any one time. In his, later, genealogical approach, Foucault makes a shift from “language” to “practices” and from “epistemes” to “dispositifs” or “apparatuses”, expanding

96

Christina Konstantinidou and Martha Michailidou

the scope of his analysis beyond the realm of the so-called sciences of man, into the domain of the institutions and practices which were founded on particular claims of truth. Discussing his concept of the dispositif, Foucault explains: What I’m trying to pick out with this term is, firstly, a thoroughly heterogeneous ensemble consisting of discourses, institutions, architectural forms, regulatory decisions, laws, administrative measures, scientific statements, philosophical, moral and philanthropic propositions – in short, the said as much as the unsaid. Such are the elements of the apparatus. The apparatus itself is the system of relations that can be established between these elements. Secondly, what I am trying to identify in this apparatus is precisely the nature of the connection that can exist between these heterogeneous elements. Thus, a particular discourse can figure at one time as the programme of an institution, and at another it can function as a means of justifying or masking a practice which itself remains silent, or as a secondary re-interpretation of this practice, opening out for it a new field of rationality. In short, between these elements, whether discursive or non-discursive, there is a sort of interplay of shifts of position and modifications of function which can also vary very widely. Thirdly, I understand by the term “apparatus” a sort of-shall we say formation which has as its major function at a given historical moment that of responding to an ‘urgent need’. The apparatus thus has a dominant strategic function (Foucault 1980: 194).

The function of these “heterogeneous elements”, of these dispositifs, consists in the production and possibility of enunciation of specific “problematisations”. The concept of problematisation is proposed by Foucault (1994)3 in order to capture the processes, intellectual and institutional, through which particular issues come to be thought of and publicly articulated as problems which may then be solved through a series of specific solutions. As Foucault puts it, “This development of a given into a question, this transformation of a group of obstacles and difficulties into problems to which the diverse solutions will attempt to produce a response, this is what constitutes the point of problematisation and the specific work of thought” (1997: 118).

2.2 Photography as a technology of truth John Tagg, drawing from Foucault, argues that what Barthes calls the “evidential force” of photography is a complex historical outcome that can only be exercised by photographs within certain institutional practices and particular historical relations (Tagg 1988: 4). As he argues, the very idea of what constitutes evidence has a history, which implies definite techniques and procedures, concrete institutions and specific social relations; that is, relations of power. Therefore, the fact that a photograph can come to stand as evidence rests on a series of historical, social, and semiotic processes, or, in other words, on particular discursive strategies.

 See also Rabinow and Rose (2003) for a discussion of the centrality of the concept in the development of Foucault’s work on ethics.

Foucauldian discourse analysis

97

Hence, photography and especially its habitual use in documenting reality that “offers itself as neutral with regard to social difference […] is in fact profoundly […] shaped by it” (Dyer 1997: xix). In the same vein, Sekula has argued that: […] the photograph is an ‘incomplete’ utterance, a message that depends on some external matrix of conditions and presuppositions for its readability. That is, the meaning of any photographic message is necessarily context-determined (Sekula 1982: 85).

Therefore, in the very moment of being perceived the meaning of photography is established through discursive and interpretive conventions which exist outside the image – conventions which are socially and institutionally constructed and serve an ideological function (Burgin 1982: 45–46). Above all, many photography theorists argue that objectivity, as a defining characteristic of photography, is a myth. In Tagg’s own words: Like the state, the camera is never neutral. The representations it produces are highly coded, and the power it wields is never its own. As a means of record, it arrives on the scene vested with a particular authority to arrest, picture and transform daily life … This is not the power of the camera but the power of the apparatuses of the local state which deploy it and guarantee the authority of the images it constructs to stand as evidence or register a truth (Tagg 1988: 64).

Despite the fact that the natural resemblance of an image to the reality it represents is being steadily revised and the conceptualisation of the photograph as an “analogue” of reality has been abandoned in photography theory (Eco 1982: 33), journalism – among the first institutions to identify itself with photography – relies on this historical legacy in order to mobilise the evidential force and corresponding truth claim of photography. Thus, common sense belief in photography’s ability to give unbiased and credible accounts of events and in photojournalism as a form of eye witnessing remains deeply embedded in journalistic practices, playing a pivotal role in the social construction of reality. Photojournalism, then, remains one of the few institutions privileging the “forensic approach” to photography. Considering discourses as broad patterns of institutional thought or “regimes of truth” which define and limit what can and can should read and cannot be said about a specific topic, our aim will be to find which regimes of truth around immigration do the Greek newspapers uphold, what is the function, role and place occupied by photojournalism in these problematisations, and how do these contribute to the construction of the object lathrometanastis in contemporary Greece.

3 The analysis: Visual construction of immigration in Greek newspapers from 2009 to 2012 This part of the chapter will be devoted to the way we have tried to carry out these theoretical and methodological commitments in the process of our research. It

98

Christina Konstantinidou and Martha Michailidou

will account for our selection of primary material out of the archive of visual representations of immigrants in the digital and print versions of major Greek newspapers and their weekly supplement magazines and the way this material was analysed. For the purposes of this chapter, we will geographically and chronologically restrict the object of inquiry (Greek discourse on immigration from 2009 to 2012), confine our analysis to the discourse produced by one institution, the media, and, more specifically, the discourse of the political press, and focus our research on visual representations. Following Howarth (2000: 128), the primary aim of our analysis is “to delineate the historically specific rules and conventions that structure the production of meanings” of immigration in the contemporary Greek press. Based on the theoretical and analytic premises outlined in the previous section, we conceptualise discourse on immigration as “made up of a limited number of statements for which a group of conditions of existence can be defined” (Foucault [1972] 2002: 131), constituting the product of an “archive” or system of “discursive practices”, i.e. “a body of anonymous, historical rules, always determined in the time and space that have defined a given period, and for a given social, economic, geographical, or linguistic area, the conditions of operation of the enunciative function” (Foucault [1972] 2002: 131). Press discourse, linguistic and visual, as part of the general discourse on immigration belongs to the “apparatus” or “dispositif”, in the Foucauldian sense of term, that literally constructs the object lathrometanastis in a manner similar to the one in which European culture, according to Said’s frequently cited quotation, “manage[d] – and […] produce[d] – the Orient politically, sociologically, militarily, ideologically, scientifically, and imaginatively during the post-Enlightenment period” (Said 2003: 3). News photographs are component elements of this system and can be conceived as discursive statements (Rose ([2001] 2007: 136–186)) within this historically specific discourse on immigration. This perspective requires “to work hard not to see pictures as rigid and fixed things – beginning and ending at their frames” (Lister and Wells 2001: 64), but to see them as parts of a system of production of meaning, of a discursive formation with its own positivity. Rose ([2001] 2007) proposes two methodological approaches of foucauldian discourse analysis with regard to photography. As she notes ([2001] 2007: 139–140), the first approach “tends to pay rather more attention to the notion of discourse as articulated through various kinds of visual images and verbal texts than it does to the practices entailed by specific discourses”. The second form of discourse analysis “tends to pay more attention to the practices of institutions than it does to the visual images and verbal texts. Its methodology is usually left implicit. It tends to be more explicitly concerned with issues of power, regimes of truth, institutions and technologies”. Even though the distinction is not clear-cut, since as she notes, it is not difficult to find work that examines visual images, verbal texts, institutions and social practices together, she believes that it is still methodologically helpful.

Foucauldian discourse analysis

99

This chapter follows the first approach proposed by Rose, concentrating primarily on the press photographs and text,4 and especially on the pivotal role photography, as a special kind of discursive technology, plays in news reporting in order to construct the discursive object lathrometanastis through an apparently objective account of immigration. This perspective, fundamentally shaped by the work of Foucault, will also make selective use of Barthian visual semiotics, especially regarding the double meaning layering of images (Barthes 1961, 2003).5 In the same vein, conceptualising the photographs under consideration as “puzzles that call for analysis” (Rajchman 1988: 90), we will make use of Kress and Van Leeuwen’s visual grammar (Kress and Van Leeuwen 1996; Van Leeuwen 2001), which offers valuable insights with regard to the significance of historically produced patterns of photographic compositionality.

3.1 Greek newspapers: corpus of analysis and research context Greek newspapers and their Sunday supplements are a privileged space of public discourse on immigration and are crucial for the visual construction of the relationship between Self and Other. Greek newspapers can be generally classified along a range of common distinctions such as frequency of publication (dailies vs. Sunday editions), coverage (national vs. regional), style (broadsheet vs. tabloid), position in the political spectrum (conservative vs. progressive, left vs. right), and assumed readership (popular vs. quality). Despite the rapid growth of tabloids since the 1980s, the increase of “soft news” content (entertainment, sensational human stories, celebrities, sports etc), and the adoption of popular forms of presentation (such as “screaming” headlines and extensive use of photographs and graphics), all major Greek national papers – tabloid and broadsheet – are nevertheless deeply involved in politics, often as actors in political clientelism and power brokering.6 The majority of the photographs were chosen out of the daily and Sunday print and online editions of the national daily evening papers with the highest circulation (Ta Nea, Eleftherotypia, Ethnos, Eleytheros Typos) and of two morning

 An in depth analysis of the institutional workings responsible for the production of these texts would be extremely interesting but would necessitate the use of other, primarily ethnographic, methods of inquiry, which fall outside the remit of this chapter. We will however return to this issue in our discussion of the need of multi-methodological perspective in the concluding section of the chapter.  We are here following Dreyfus and Rabinow’s (1982: 17) and Hall’s (1997: 43) argument that Foucault’s work remains indebted to some degree to Saussure and Barthes while in other ways departing radically from them, since it was much more historically grounded than the semiotic approach.  For an analysis of the political clientelism of the Greek media system, see Hallin and Papathanassopoulos (2002).

100

Christina Konstantinidou and Martha Michailidou

elite newspapers Kathimerini7 and To Vima.8 The corpus included all the editions published by these titles from January 2009 to June 2012, except for Eleftherotypia, which ceased publication in December 2011. The photographs were chosen by virtue of appearing in a textual context (news story, feature, reporting, editorial, photo-essay etc.) dealing in some way with the theme of immigration. This decision posed considerable difficulties, as themes can relate to each other in complex and unpredictable ways, which can change over time and between newspapers.9 Thus, content and photographs that appeared in other journalistic themes, even if they were discursively and visually linked to the imagery of immigration were also excluded. During the period of our analysis, immigration is represented by the papers as either a local or national issue, in which case items are classified under domestic news (especially during the preelection campaigns of 2009, 2010 and 2012), or as part of a wider European or international theme, in which case news items are classified under the foreign news category (as was done with items concerning the so called “Lampedusa crisis” during the Arab spring period and the refugee crisis caused by Somalia’s civil war). Since our primary focus was on discourse concerning the Greek crisis, we concentrated on photographs appearing in domestic news items. Yet, we realised that on many occasions the same photographs also appear in Greek or foreign media through the web, in print or online editions, different news items, at different time periods and in news about different countries. Therefore, the photographs

 Kathimerini is practically the only morning broadsheet still financially viable, with a daily circulation of approximately 46,500 copies, representing 81 % of the total morning newspapers’ circulation in 2011.  The newspapers were chosen on the basis of the size of their circulation and their political identity. We included major titles from the entire dominant, mainstream, political spectrum, and excluded newspapers owned by political parties and organisations (Rizospastis, owned by the Greek Communist Party, and Avgi, the morning newspaper of the left coalition SYRIZA whose combined circulation in 2011 stood at approximately 10,400 daily). Tabloid titles, such as Espresso, To Proto Thema, Realnews etc., characterised by a simpler writing style, eye-catching headlines and dominated by pictures were excluded, although sometimes their circulation is higher than that of major political newspapers. Yellow journalism tends to emphasize topics such as crime and sex, gossip about politicians, media celebrities and sport stars and its techniques may include exaggerations of news events, scandal mongering and sensationalism. Although their aggressive involvement in politics may be decisive in the formation of the public debate, we decided not to include these newspapers, primarily because we believe that due to their special characteristics, they merit an analysis specifically directed to their specific language and its textual and visual characteristics.  For instance, during the period of analysis the journalistic theme “illegal trade” was frequently articulated in relation to the theme lathrometanastis because of immigrant involvement in street selling. Similarly, Mavrommatis et al. (2010) found that in even though, in 2008, the issue of human trafficking appeared as part of the theme “illegal immigration”, later on in 2009, “human trafficking” became a separate journalistic theme, a change probably due to increasing legal and political recognition of the importance of human trafficking.

Foucauldian discourse analysis

101

from the foreign news section were not excluded unless they were totally irrelevant to the issue of immigration. During the period of analysis, several events were considered newsworthy and were discursively articulated with the more general ongoing debate over immigration. Because of the domination of news values that promote conflict, rarity, spectacularity, emotionality, dramatisation and personalisation, most of the events reported are constructed 10 as violations of social rules on a judicial or a moralpolitical level. This daily news construction, as has been well documented, on the one hand mobilises a sense of what constitutes normality (Hall et al. 1978: 83), and on the other hand contributes to the construction of periods of media/crime waves11 eventually producing or fuelling a moral panic (Cohen [1972] 2011).12 Our research shows that, between 2009 and 2012, two main categories of news on immigration prevail in the narratives of the Greek press: firstly, news items constructing immigrants as “perpetrators” of illegal or criminal acts, such as illegal border crossing of undocumented migrants, squatting of public places or aban According to the social production of news approach (Hall et al. 1978: 53–59), news values are not given characteristics of events, actors, or messages that exist outside of, and prior to the discursive realm. “News” is the end-product of a complex process which begins with a systematic sorting and selecting of events and topics according to a socially constructed set of categories. In this process, the bureaucratic organisation of the media, the professional ideology of what constitutes good news – the news practitioners’ sense of news values, the classification of an event to specific social context (framing) and the presentation of the item to its assumed audience – give to the event its proper meaning and newsworthiness. According to Hall et al. (1978: 57) this process is crucial in two ways, since “the media define for the majority of the population what significant events are taking place, but, also, they offer powerful interpretations of how to understand these events”.  According to Fishman (1981), media reporting on crime, through its thematic classification, constructs new types of crime (immigrants’ crime, crimes against the elderly, drug-induced crime, inner-city crime, racist crimes etc) with specific offenders, victims and circumstances. If a topic has enough potential to retain media attention for a sufficient number of days, then, in all probability, it will constitute a crime wave. This means that, for a crime wave to arise, it must somehow be founded on real incidents, but in addition, the participation of all media in “making incidents meaningful only as ‘instances of themes’” (Fishman 1981: 105) is necessary, as well as that of law enforcement agencies (mainly the police) and various specialists (criminologists, psychologists, psychiatrists etc.) who take part in the public debate. While the subject remains on the daily agenda, events within this thematic category will continue to be reported, confirming the “reality” of the crime wave. It is in this sense that, according to Fishman’s definition (Fishman 1981: 100), crime waves constitute “media waves”.  Simplifying generalisations of crime waves carry a particular ideological weight. By means of exaggeration and stereotype construction, symbolisation and dramatisation, groups such as immigrants, soccer fans, anarchists, drug addicts etc. are mystified and demonised. It is indeed quite often through the cumulative effect of the intermediation of “primary definers” in the public debate on crime prevention and control, the penal system and generalised social lawlessness, that a moral panic, according to Cohen’s (1980) definition, is generated, a climate, that is, of social insecurity which eventually favours moves towards more autocratic forms of crime controlling, generalised social repression and violations of fundamental human rights.

102

Christina Konstantinidou and Martha Michailidou

doned buildings, working without a permit, disseminating diseases, participation in petty crime or serious criminal acts (robberies and homicides, trafficking, drug related etc.). Secondly, the Greek newspapers also abound with news on immigrants constructed as “victims” of macro-sociological and economic conditions, such as globalisation, forced displacement, poverty, international conflicts, neoliberal economic policies, state institutions’ practices, police sweeping, public pillorying, smugglers’ exploitation, prejudices, racist attacks and hate crimes of extremeright vigilant groupings or fascist thugs, and so on. The first category dominates the conservative press while the second category prevails in centre and centre-left newspapers, although the distinction is far from clear-cut and depends on political conjuncture.13 News on immigrants as social “actors”, family members, citizens, employed or unemployed and so on only occur in media content as long as they can be framed into a conflictual frame, articulated through reference to a bipolar schema of villain/victim or as part of a meticulous strategy to nullify the negative image of immigrants by reversing the stereotype. As is well documented, however, simply reversing existing stereotypes, journalistic discourse, even in its most progressive version, fails to transgress and challenge, let alone deconstruct, the register of values which constitutes the dominant national (Greek) identity, and to therefore challenge social processes through which the notion of deviance is attached to certain behaviours or people. Moreover, by introducing the distinction between good (integrated, settled, legal) and bad immigrant, progressive discursive practices reproduce symbolically the stigma, though in inverted form (Konstantinidou 2001), even when they hit out with particular vehemence against the “exploiters” of foreign workers or “Nazi thugs”. In summary, the immigrants’ categorisation as such “reduces every probable complexity to a group generalisation and encloses every probable social and temporary fluidity in an absolute definition, within the limits of a character which is per se cause, effect and process” (Guillaumin 1972: 183–184, 1995). Such news construction practices usually end up by representing immigration as an “objective” security or humanitarian “problem” and immigrants as a (photographic) “object” of contemplation or an imaginary or cultural “abstraction”14 and not as political “subjects” for the reader to enter into  Batziou (2011: 85), analysing the Greek press of 2005, argues that the dominant thematic category framing immigrants in Greek newspapers was crime (30.5 %), a category in which she classified, rather curiously, representations of immigrants as both perpetrators and victims of criminal activities (the latter accounting for 63.7 % of the category).  Auclair (1982: 57–58) claims that the news story’s “main character”, either as offender or victim, is perceived in relation to a very specific act and through the unique role the collective imaginary attributes to him. S/he is reduced, that is, to the “spectrum of functions” (faisceau de fonctions) which the dominant culture ascribes to him/her, in other words to an abstraction. According to his approach this is one of the operational conditions of simplistic thought or common sense. Hence, the immigrant who makes the news is always represented through his signification, first in direct, if not exclusive, relation to his status as foreigner (legal or illegal) and his nationality and second, something usually forgotten, by the individual being spread all over with the sole act which made him known, thus exclusively consolidating his identity and personality with the stereotype of the

Foucauldian discourse analysis

103

any kind of imaginary relationship with or as “active agents who enact their own politics of integration” (Lentin 2012: 235).

3.2 The problematisation of immigration The analysis of this heterogeneous and contradictory press archive provides a perspective on the ways in which immigration is discursively problematised in Greek media narratives on a par with the primary definers’ views and practices but also according to the constraints imposed by each paper’s professional practices and technologies. The findings of previous empirical research led us to periodise Greek media discursive construction of immigration into three consecutive phases. In the first phase, the stereotypes of the “Albanian criminal”15 or the masses of the Balkan poor16 are the dominant conceptualisations of immigration. In the second phase, the stereotype of the Balkan other loses its dominance17 and is gradually replaced by the image of African or Asian Muslim18 lathrometanastes. In the third phase, which we will explore in some detail in the following section, a moral panic about illegal immigration is constructed in the discourse of the Greek press. Broadly, from 2009 until 2012, in all news items we can distinguish two main clusters of problematisation, very similar to the twofold discourse Mavrommatis et al. (2010) found in Greek newspaper narratives in 2008 and 2009. These are the conservative/anti-immigrant thesis based on the unambiguous articulation of immigration as a threat and the humanitarian or human and civil rights crisis thesis. These are not necessarily mutually exclusive; “humanitarian” problematisations of immigration can be either “anti” or “pro-immigrant” and they may meet the “threat” problematisations, their internal contradictions built into the core of their problematisations. Hence, for analytic purposes, the newspapers can be classified all along a loose continuum, which covers the distance between the conservative anti immigrant problematisation and the progressive/pro-immigrant thesis founded

immigrant. The foundation of a cause and effect relationship between status or nationality and behaviour follows from the simple discursive correlation of the two phenomena. This stereotyping and identity construction goes far beyond news reporting. During the period of our analysis, a feature length movie (Constantine Giannaris’ Man at sea, 2011) and a theatre play (Reppas and Papathanassiou’s Homeland in the Greek National Theatre, 2012) were promoted on the basis that “immigrants play themselves”.  Askouni 1997; Vamvakas 1997; Voulgaris et al. 1997; Dimitras 1995; Greek Human Rights Watch 1995–1996; Karydis 1996; Mikrakis 1998, 1994; Païdoussi 1998; Petraki 1998; Triandafyllidou et al. 1997; Triandafyllidou and Mikrakis 1995; Triandafyllidou 1996; Fragoudaki and Dragona 1997; Moschopoulou 2005; Kiliari 1997; Konstantinidou 2001; Pavlou 2001. For a detailed review see Stratoudaki (2007).  Kapllani and Mai (2005).  Maroukis (2008), Tsaliki (2008).  Skoulariki (2009).

104

Christina Konstantinidou and Martha Michailidou

on humanitarian discourse. On the conservative end we can place Eleftheros Typos, a right-wing tabloid, which consistently develops a nationalist populist problematisation on immigration. Eleftheros Typos is a paradigmatic case of what Kundnani (2001: 48) terms “an interface between state and popular racism, providing a forum where the bureaucratic language of the first is translated into the populist language of the second”. The newspaper reports widely on so-called generalised lawlessness and lathrometanastes’ participation in crime, on city centre decay and ghettoised urban areas, on border invasion and “countless” jobless lathrometanastes, while calling the government to tighten its “inadequate” repressive policies and measures, to deport all lathrometanastes, to close the borders etc. Eleftheros Typos is followed by two elite morning newspapers, the centre-right and mildly nationalist Kathimerini and the centrist and mildly nationalist To Vima, and two evening newspapers, Ta Nea and Ethnos, oscillating between a more or less centre-right and centre-left nationalist identity.19 These moderate newspapers, which more or less, operate within the same framework, articulate the threat and humanitarian crisis problematisations in very complicated ways depending on the specific event and the primary definers’ involvement.20 At the other end of the continuum we can place the centre-left Eleftherotypia, which provided the most consistent progressive anti-racist retort until December 2011 when it ceased publication.21 Eleftherotypia was committed to a position against racist ideology and xenophobic perceptions and practices, and shifted the weight of news reporting in the most singular manner by replacing the threat stereotype of the anti-immigrant discourse with the public image of the innocent victim of humanitarian crisis. Indeed, during the study period, Eleftherotypia is the only newspaper that adopts a clear “pro-immigrant” stance, reporting daily on the rise of far right parties, extreme right Golden Dawns’ practices and racist attacks against immigrants. Simultaneously, the newspaper contributes to public debate on immigration through its editorial and features and by reporting widely on leftist parties, refugee organisations and anti-racist NGOs’ discourses and activities. Additionally, Eleftherotypia constantly criticises leading parties and successive governments for not observing European treaties and international laws con See also, Rougheri et al. (1999).  For example, according to Mavrommatis et al. (2010), the dual narrative approach, can mobilise the threat and the humanitarian framing at the same news story by differentiating between trafficker/perpetrator and migrant/victim. On the one hand, the traffickers are presented as criminals, as members of organised crime who make a fortune at the expense of illegal immigrants. On the other hand, newspapers’ references to deaths of migrants who tried to cross the border and coastguard interrogation abuses portray undocumented migrants as victims of illegal networks and state repression and as sufferers who were forced to escape war, poverty and despair.  Since the closure of Eleftherotypia in 2011 no major newspaper represents this political or ideological stance. Given the electoral rise of the left and especially of the Coalition of the Radical Left (Syriza) whose official newspaper Avgi has a negligible circulation, this vacuum in the political spectre of Greek press stands out as a true paradox.

Foucauldian discourse analysis

105

cerning refugees and asylum seekers or minor migrants, for the implementation of restrictive immigration policies and for abusing securitising practices (detention centres, border patrols, police sweeping and maltreatment, racist behaviour of police etc). As far as the general context of immigration problematisation is concerned, the whole period from 2009 to 2012 can be seen as a cyclic process where the debate on immigration, more or less regularly, appears, disappears and reappears in the media, depending, apart from the wider international or domestic conjuncture, firstly on specific newsworthy human interest stories, secondly on governmental activities and policies, and thirdly on the political agenda and practices of various parties or social actors in close relation to election campaigns. Additionally, this periodisation also reveals that the theme of immigration is also articulated in relation with the theme of the “Greek debt crisis” from 2011 onwards. More specifically, during 2011–2012, as public debate around the deepening of the Greek economic crisis gets more and more polarised between pro-memorandum and antimemorandum positions,22 a polarisation in which the Greek press plays an active role (Leandros et al. 2011), the issue of immigration, which had up to then enjoyed relative discursive autonomy from the frame of Greek debt crisis news, slipped gradually towards the centre of the public and media discourse on Greek crisis. Indeed, while at the beginning of the period analysed (2009–2010), the journalistic theme of the Greek debt crisis had developed separately from the problematisation of immigration, during 2011 an important transformation occurs. This displacement and redefinition can be clearly traced to a sequence of prime news stories that occurred during 2011 and 2012. One of the most prominent of this type of news stories, where all the right elements were there for mixing “crisis” and “immigration”, was the case of the 45 day hunger strike and occupation of Athens Law School and Hypatia Mansion by more than two hundred undocumented immigrants in January 2011, requesting the legalisation of all migrants in Greece. This event gained enormous newsworthiness not only because of its duration but also by the highly controversial occupation of the Law School and the involvement of actors from various ministries (of Education, Employment, Citizen Protection and Health) and the whole political spectrum, fuelling the public and media discourse around immigration policy, human rights, state authority, humanitarian ethics, Greek crisis, academic freedom and law and order for more than 40 days.

 Even as this chapter is being composed (August 2012), Greece, among other European Union Member states – the so-called, by the offensive acronym, “PIIGS” (Portugal, Ireland, Italy, Greece and Spain), experiences the most severe economic downturn since the Great Depression of the 1930s, the Second World War and the civil war period that followed. The fiscal crisis and deterioration of the international economic situation in 2008 drove Greece in a vicious cycle of implementation of successive packages of austerity measures, European bailouts loans that require more austerity measures and structural reforms (the so-called memorandum policy), pushing Greece in deep recession.

106

Christina Konstantinidou and Martha Michailidou

Moreover, the redefinition of the immigration problematisation within the context of the economic crisis can be identified in the statements of the Ministers of Health and Citizen Protection (former Public Order) in March 2011 and again in April 2012, asserting the existence of a “hygienic bomb”23 threatening Greek society because of the deplorable living conditions in “immigrant ghettos” in Athens inner city, and declaring the implementation of a “zero tolerance” policy, declarations that were widely reported and debated. The discursive articulation of the themes of economic crisis, criminality, immigration, urban decay and ghettoisation can also be traced in diverse news items about crime of allegedly foreign perpetrators against Greek victims. The debate over immigration continued throughout 2012 and crystallised during the long pre-electoral period (April–June). Irregular migration was constructed as an “exceptional crisis within the Crisis” by all Greek media, orchestrating public discourse and fuelling the law and order and anti-immigration rhetoric. As Nick Malkoutzis notes in the online English edition of Kathimerini, “Judging by the content of the debate in Greece over the past few days, one might think that the most pressing issue facing the country ahead of the upcoming general elections is illegal immigration rather than the economy” (30/3/2012).24

3.3 Classification of the photographs according to genre In the analysis of the visual material, we are not concerned with specific events or with establishing whether and to what extent the media representations correspond to objective reality. Our aim is to investigate the way press photographs contribute to the construction of otherness within the aforementioned twofold ambivalent immigration problematisation. Therefore, the various newsworthy events along with other routine news features and editorials (information on daily petty criminality, timeline of racist attacks, debate on detention centres and border fences, legalisation process, statistics on immigration etc) will be considered as a multilayered canvas constructed through well established journalistic and editorial practices (illustration, pagination etc.) upon which the photographs are “weaved” into the media news items in order to offer the media narrative on immigration an indisputable truthfulness either as evidential or as symbolic statements or nodal points. Within the photographic archive explored, we located three major genres of photographic imagery, each of which follows different rules and logic: – The “generic photojournalist images”, mostly as representations of “types” of people, places, actions or events, found in the weekday and online editions most commonly as illustrations of routine news reporting, features and editorials  The “hygienic bomb” metaphor was first used by New Democracy’s Minister of Public Order back in 2009, soon after the defeat of the party in the European elections.  http://www.ekathimerini.com/4dcgi/_w_articles_wsite3_1_30/03/2012_435450.

Foucauldian discourse analysis

–

–

107

The “photo essay” genre, a set or series of higher quality news-reporting photographs, usually found in a full page or a two-page spread in the Sunday supplements of Eleftherotypia, Kathimerini and To Vima The “fine art-documentary photography” genre, in the form of tributes or special articles dedicated to well known photographers like Magnum reporter Jim Goldberg (Eleftherotypia, 21/2/2010), found in the Sunday supplements of Eleftherotypia, Kathimerini and To Vima.

The distinction between the three genres of visual imagery can be less clear than the classification here might imply, since these different modes of representation can overlap at times. Nevertheless, it is useful because it allows us to clarify the newspapers’ differing signifying strategies, narrative intentions and discursive practices. In particular, this distinction highlights, what Ludnstrom calls “knowledge from within, knowledge from without. The engaged humanist and the objective reporter”: Documentary or journalistic photography rest on two related – and as it seems just as necessary as contradictory – distinctions or approaches. One, where the presence of the photographer is the guarantee for the quality and believability and substance of the documentary, the other where the photographer’s absence is the guarantee for precision and quality. The perspective of the insider, the perspective of the outsider. Participate or observe (Lundstrom 1999: 62).

It is obviously beyond the scope of this analysis to examine in detail all kinds of visual imagery of Greek newspapers pertaining to the immigration issue. Therefore, in what follows, we will consider in more detail the generic photojournalist images, as these appear regularly and in different occasions in all newspapers as constitutive elements of daily news.

3.4 Generic photojournalist images Machin (2004) and Machin and Jaworski (2006: 351) drawing on Halliday (1978), describe the ITN and Getty Images archives as containing footage and pictures that have broad “meaning potential” which can be actualised in news items. As they suggest (Machin and Jaworski 2006: 363), the archival visual material provides typical versions of the phenomenal world which we take to be “real” in the specific frame of news reporting. Additionally, these generic, decontextualised, and anonymised images not only construct particular visions of the world but, in so doing, they also iconize and legitimize them. Following Jameson’s ([1984] 1991) discussion of the way collective memory of the US in the 1950s had been iconicised by a handful of selective images, they suggest that our knowledge of our own societies, is equally essentialised by the repeated use of generic, stylised news video footage sequences or photographs (Machin and Jaworski 2006: 363). This visual material, they argue (Machin and Jaworski 2006: 347), not only has a power-

108

Christina Konstantinidou and Martha Michailidou

ful ability to index the real world “out there”, but, also, to restrict or even override the accompanying linguistic message, therefore to limit its meaning potential crucially. The generic photojournalist and archival images we found in the Greek press function in a very similar way to the Getty Images commercial collections analysed by Machin. They are produced by editorial photojournalistic practices to serve as undeniable records of reality, as documented evidence of the people, places, things, actions and events they depict. These photos, claiming a truth directly linked to their evidential/realistic attributes, constitute the most frequent kind found in weekday and online editions of the papers, most commonly conferring verisimilitude to routine news reports, features and editorials. Usually, these photos are taken from the newspapers’ own archives or bought from a press or photo agency network. This genre can be divided in two main categories. In the first, most of the photographs appear as being “out of context”, “out of space” and “out of time” rather than as part of a specific story. As Kambouri (2009: 98) and Batziou (2011) have also noted, in the case of these numerous photographs there is no obvious connection between the image (for example the person, place or action depicted) and the news item. Moreover, the origin of many of these photos is hard to trace, as Greek newspapers rarely mention the copyright holder of the photograph, and they are often published without identifiers as to the event, date, location, or the photographer. Nevertheless, photographs like this are easily recognised as photographs of “immigrants” or “would-be immigrants”; their meaning is so established first because of the news item as part of which they appear, which operates as an immediate textual framework limiting the array of possible interpretations, and secondly, because of their previous use in the press or in other media, especially in the web, because, that is, they are informed by the positivity of an already established media discourse on immigration. In addition, very often the same photo appears in a newspaper and in its online edition (for example the boat people icons), as part of different news items or at different periods of time and in news about different countries. Hence, many generic photographs instead of being verbally or contextually linked to specific events, places, people and situations, by the effect of this very archivisation, acquire the status of conventional visual icons of the journalistic theme “immigration” and related sub-themes such as “asylum seekers”, “boat people”, “immigration detention centres”. The second category of generic images seem more in context, in the sense that the photographs’ apparent indexicality relates to specific events, places, people and situations in news reporting as indicated by the title, the caption or the news item content. Nevertheless, many of these photographs, during the period of our analysis, became in turn iconic “out of context” images, appearing in later editions

Foucauldian discourse analysis

109

Fig. 1: Children in detention centre, Kyriakatiki Eleftherotypia, 22/5/2011, p. 22. © Elefterotypia & Milos Bicanski/Getty Images News/Ideal Image.

in a vignette-like way either in order to make shorthand allusions to the specific event first represented by them or to represent a similar event visually, discursively or ideologically associated with the journalistic theme “immigration”.

110

Christina Konstantinidou and Martha Michailidou

Fig. 2: Patrol in Aegean Sea, Ethnos website, 2012. © Ethnos.

Fig. 3: Boat people, Ethnos, 29/7/2011. © Ethnos.

Foucauldian discourse analysis

111

Fig. 4: Hunger strike in Athens, Ta Nea, 4/3/2011. © Ta Nea & Orestis Panagiotou/AMNA.

This process of decontextualisation and recontextualisation is not limited only within an array of journalistic functions, as the following case illustrates. An iconic photo of a 2011 immigrant hunger strike that appeared many times in Greek newspapers illustrating what became known as the “Hypatia case”, after the house occupied by the immigrants who then went on a hunger strike (Figure 4), soon

112

Christina Konstantinidou and Martha Michailidou

Fig. 5: “Even one victim of racist violence is too much”, 2011 UNHCR campaign against racism. © UNHCR & Orestis Panagiotou/AMNA.

afterwards became part of a UNCHR photographic campaign against racist violence (see Figure 5). The photographs’ tendency towards decontextualisation and archivisation leads to the standardisation and iconisation of the representation of immigration in a twofold manner. First, through the constant use of visual stereotypes, which may be spatial, cultural or physiognomic, the photographed person’s individuality is concealed and the people represented are reduced to social “types” (immigrants) (Van Leeuwen 2001: 95) (see Figure 6) or dehistoricised “symbols” like the children behind bars in detention centres (Figure 1). Furthermore, this archivisation results in what Malkki terms “dehistoricising universalism”, “a context in which it is difficult for people in the refugee category to be approached as historical actors rather than simply as mute victims” (1996: 378). This is further compounded by the fact that, while in the foreign news section the category of refugee is usually well distinguished through text (captions, title etc.) (Figure 7), in domestic news, the Greek press seems to “confuse” the legal status of people coming into Greece, using the terms refugees, asylum seekers or (economic) migrants indiscriminately. Most importantly, these categories, regularly, seem to disappear and collapse in the same vague grouping: lathrometanastes. Even more significantly, since the term latrometanastis is commonly used to

Foucauldian discourse analysis

113

Fig. 6: “Immigrants residence in Greece and right to vote”, Ta Nea, 5/2/2010. © Ta Nea.

designate anyone who enters the Greek borders without legal documents, either seeking asylum or not, the once “morally untouchable category of the political refugee” (Cohen, 1994, cited in Cohen [1972] 2011: xxii) becomes not only objectified but also invisible, and the solidity of the legal category of refugee and asylum seeker destabilized (see Voutira 2010). Hence, during the period of our analysis, refugee and migration status are discursively blurred and it is almost impossible

114

Christina Konstantinidou and Martha Michailidou

Fig. 7: “Asylum seekers”, Elefterotypia, 29/1/2009. © Elefterotypia.

to visually identify immigrants, asylum seekers and refugees within the national territory. As a whole these fundamental archival practices create, in a very subtle way, a specific vocabulary of visual statements, a “veritable vocabulary of visual topoi” (Van Leeuwen and Jaworski 2002: 256) that organise the conceptualisation of

Foucauldian discourse analysis

115

immigrants along specific, and specifically localised, gender and racial identities. In what follows we will outline the ways in which the (migrant) subjects’ identities, in all their constitutive complexities, are simplified and essentialised into specific types.

3.5 Visual essentialisations of identity The first, and most frequently recorded photographic practice of articulation of identity, consists in the over-representation of men and, relatedly, the striking absence of women and children, verifying partly the findings of Lalioti (2005). In this kind of photographs, as Lalioti notes, it seems as if the “typical immigrant” in the visual imagery of the Greek press is male in his early 20s and 30s,25 is in some public space, he appears to be part of a group of other immigrants, usually lying or sitting on the ground or floor and looking tired, wretched or desperate. Unsurprisingly, the subject is rarely depicted smiling (Lalioti 2005: 442–443), and is very often “looked down upon”. In this photograph appearing in Eleftherotypia, for instance, a human “tsunami of desperates is knocking on the doors of the EU” (Figure 8). Kress and Van Leeuwen’s (1996) visual social semiotics on the photograph’s dual capacity to operate as a sign and as a site of performance are very helpful here for they permit us to explore the role of the most common journalistic visual resources26 in newspapers’ discourse. According to their approach, the distinction between making eye (or gesture) contact with the viewer or not, the distance (size of frame), and the point of view constitute important resources for representing and maintaining an interaction between the producer and the viewer of the images. Thus, in most of these long distance shots of groups of young men, the depicted people seem to address the viewer indirectly: the newspaper “offers” the depicted people to the viewer as items of information or objects of contemplation (Kress and Van Leeuwen 1996: 124). Similarly, producing an image also involves the selection of a particular angle, a “point of view”, and this implies the possibility of expressing distinct subjective attitudes towards represented participants (Kress and Van Leeuwen 1996: 135). The horizontal angle, depending on whether the producer situates her/himself in front of the represented participants (people, spaces, objects) or by the sidelines, encodes whether the image-producer (and hence, the viewer) is involved with the represented participants (frontal angle) or

 According to national statistics regarding the registered immigration, the male to female ratio of incoming migrants is balanced overall, though with considerable variations depending on the nationality and the legal status of the migrant (Mediterranean Migration Observatory, Hellenic Migration Policy Institute and Baldwin-Edwards 2004: 5).  For the useful distinction between codes and resources in social semiotics, see Jewitt and Oyama (2001: 134–135).

116

Christina Konstantinidou and Martha Michailidou

Fig. 8: “Tsounami of desperates hits the EU”, Elefterotypia, 15/2/11. © Elefterotypia.

not (oblique angle), in which case, the producer/viewer is “detached” from the represented participants (Kress and Van Leeuwen 1996: 141–145). Similarly, if a portrayed person is seen from a high angle, then the relation between the producer of the image, and therefore the viewer, and the represented person is depicted as one in which the producer/viewer has power over him: the represented person is seen from the point of view of power. This power over the represented is compounded by the fact that most of the photographs depict masses of newcomers, facilitating the visualisation and interpretation of immigrants in terms of numbers, rather than individuals, typical of the media representation of migrants “as a miserable sea of humanity”. The second discursive strategy, one which functions towards particular versions of humanisation of lathrometanastes, is one that’s features heavily women and infants and consists of photographs representing victimisation and photographs representing the celebration of cultural diversity or integration. The “victimisation” type uses elements of the most sentimentalist visual vocabulary, pertaining to the media narrative of human suffering and the humanitarian visual imagery, commonly used in campaigns by humanitarian organisations. According to Imogen Tyler (2006: 194) many humanitarian organisations, in their effort to generate forms of recognition that work against identification of the asylum-seeker as a hate figure and a dehumanised, undifferentiated foreign mass

Foucauldian discourse analysis

117

or flood, attempt to turn media hypervisibility in favour of asylum-seekers. One of the central strategies employed by them is the use of photographic close-ups of asylum-seekers’ faces and first-person accounts of their lives. These visual technologies of familiarisation aim to help the public recognise “the human face” of specific asylum-seekers and thus enable the reader to identify with “the victims” of repressive asylum laws. In other words, humanitarian discourse strategies seek to force recognition of asylum-seekers as individuals, assuring the public that asylum-seekers are humans “just like us”. In this strategy, the visual technologies of photographic close up and face-to-face portraits are used to counter the dehumanising figurations or identifications of governmental and media rhetoric produced by racist or xenophobic discourses.27 As Konstantinidou argued (Konstantinidou 2007, 2008), drawing on Taylor (1998: 92) and Wright (2002), the insistence on spectacular images of suffering and the combination of a humanitarian discourse of compassion to “innocent distant victims” with populist and Christian Orthodox conceptualisations of the self, is one of the most important constituents of the Greek newspapers’ signifying practices. Thus, the images of children or a Pietalike mother carrying a young child in her arms, very often behind fences in deportation centres (Figure 1), captured in smugglers boats etc. stand as a metonymic representation of the plight of the forcibly displaced people in the modern world, constituting a fundamental element of the visual vocabulary of the generic archival photographs. Placing women, children and especially little girls at the centre of the media narrative, these images attempt to humanise the figures of asylum seekers or immigrants that are negatively represented in the conservative media and far right posters, blogs etc. This kind of “humanistic” representational strategy very often depicts people looking directly at the viewer’s eyes, and the image creates a visual form of direct address demanding that the viewer enter into some kind of imaginary relation or a pseudo-social bond with the subjects represented, constituting an “image act” (Kress and Van Leeuwen 1996: 122–123) of humanisation. Furthermore, most of the photographs of this genre use visual codes which signify “involvement” between the people depicted and the photographer/viewer (Kress and Van Leeuwen 1996: 130–146), as they are close-ups or medium shots, using frontal angles, addressing their subjects at eye level. Children play a pivotal role in this imagery of humanisation, being put in the centre of the photograph, often as a component of a mother-child visual composition (Figure 9). Crucially, the children are often depicted to be cradling Greek national symbols, especially the flag. Here, humanisation is conditional on the  Malkki (1995: 10) notes that a similar array of pictorial themes appears in a 1985 wall calendar entitled “Refugee Women” put out by the United Nations High Commissioner for Refugees (UNHCR) where “all of the “beautiful, glossy color photographs” depict one of three things: close-up portraits of individual women; women cradling babies or children; and women sewing, weaving, cooking, etc.”

118

Christina Konstantinidou and Martha Michailidou

Fig. 9: “Twice without a country”, Elefterotypia, 7/11/2010. © Elefterotypia.

adoption of key markers of national identity, the complete incorporation into the imaginary community of a homogeneous consensual Greekness (Figure 10). Through the corpus it is obvious that most of the photographs depicting people construct patterns of identity using visual markers of gender, age, race, culture, and religion. As will by now have become evident, non-white people of allegedly “African or Asian features” dominate in press representations of immigration, despite the fact that, according to existing statistical data, the majority of immigrants in Greece come from Balkan countries and most specifically from Albania, as Kambouri (2009: 98) and Lalioti (2005) have also noted. The issue is not simply one of over-representation of blackness or under-representation of whiteness, however. Rather, the persistent visualisation of immigrants as non-white and the consistent use of visual tropes through which the immigrant subjects of the photographs are constructed as “irrevocably” different, result in the racialisation of immigration tout court, standardising and typifying immigrants as a single homogeneous and irrevocably different category of racial subjects (Figure 11). Race, then, transverses all the images of the archive, imposing a conceptual coherence on even apparently differentiated representations. In the imagery of the corpus, race is articulated in both its “biological” and its culturalist version. In the first articulation, race is constructed as a given, as what Hall calls “the very

Foucauldian discourse analysis

119

Fig. 10: “The children of Greece”, Kathimerini, 25/3/2009, p.1.© Kathimerini & Icon Press/ Antonis Pasvantis.

obviousness of the visibility of race”,28 visually signified by skin colour. This constancy of the visible signifiers of race in the Greek press imagery of immigration produces an effect of epidermalisation (Fanon [1986] 2008), a term which as Gilroy has commented “refers to a historically specific system for making bodies meaningful by endowing them with qualities of colour: It suggests a perceptual regime in which the racialised body is bounded and protected by its enclosing skin. The observer’s gaze does not penetrate that membrane but rests upon it and, in doing so, receives the truths of racial difference from the other body” (Gilroy 2004: 46). In the second, culturalist articulation, race is visually signified by stereotypical markers of “ethnicity” and “religion” (Asian, Muslim, Islamic), occasionally producing visual statements of what Rattansi (1992: 2) calls “the multicultural celebration of diversity [reproducing] the “saris, samosas and steel-bands syndrome” (Figure 12). Here, the complexities of nationality, ethnicity, ethnic identification and religious faith are eschewed in the development of culturally essentialist representations of exotic or threatening varieties of ethnic otherness. At a historical conjuncture when biological racism is no longer part of the vocabulary of “mainstream” politics and biological constructions of race as pheno See https://www.msu.edu/course/ams/280/hall3.html.

120

Christina Konstantinidou and Martha Michailidou

Fig. 11: “Indigenous and voters”, Elefterotypia, 10/1/2010. © Elefterotypia.

Fig. 12: “If there was a mosque in Athens we could pray indoors”, Ta Nea, 17/11/2011. © Ta Nea.

Foucauldian discourse analysis

121

typic difference are delegitimised in mainstream political discourse,29 this “racial schematisation of the visible field” (Butler 1993: 16) silently lends weight to a latent racist and sexist discourse, which, depending on the newspaper’s politics, constructs immigration either as an out of control security threat against the nation and its racial and territorial integrity or as a humanitarian crisis with immeasurable consequences on social cohesion and national well-being. The third discursive strategy is one of spatialisation of immigration. The typical photographs of this genre revolve around three spatial signifiers which are fundamental to the localisation of the problem of immigration. Thus the photographs can be classified, depending on their use of spatial signifier, in three categories. These are the “border”, a category which consists of photographs involving “boat people”, the “Greek – Turkish Border Fence” issue, police border patrols (including Frontex), people dead in border crossing, truck climbing stowaways, etc.; the “inner city/ghetto”, which consists of photographs involving migrants in the city within the ghettoisation narrative, police “sweeping” and arresting operations, street selling, “ghetto” images etc.; and the “camp”, consisting of photographs symbolising the boundary between Greek territory and what Agamben’s famous expression has described as the “space of exception” (1998: 106–107): the detention centres in desert territory, overcrowded yards and rooms, people and especially children behind fences, etc. Metaphors of exclusion and enclosure such as gates, fences and cages, hands grabbing barbed wire, are also very common, in this category (see Figures 13, 14, 15). Interestingly, the majority of photographs concern people and actions in public places (indoor or outdoor), connoting what might be conceptualised as the “inevitability of our meeting with the Other”. Indeed, in close relation with the rise of the anti-immigrant rhetoric and the problematisation of immigration as an urgent social and political issue in the electoral campaigns leading to the 2010 local and 2012 national elections, emerges what Mavrommatis et al. (2010) term a “geographical narrative of migration”. According to their analysis, the discourse on immigration slips, in 2009, towards a new narrative and starts to be territorialised and localised in two discreet zones: first, in specific areas in inner city Athens, and secondly, in areas near the national borders and the ports. As they notice, the emergence of a journalistic geographical discourse of immigration differs from previous “non-spatial” narratives, as it tends to conceptualise migration as a “problem in space” or a “spatial problem”. The emergence of the geographical narrative of immigration puts an end to non-spatial news stories as the territorial dimension starts to play an important role in public discourse and news storytelling and all actions are territorialized. Over the years, and as the meaning of specific areas becomes associated to the “tsunami” of

 The populist discourse of the rising far right, however, and especially and the neo-Nazi nationalist party Chrysi Avgi (Golden Dawn) which entered parliament with 6.92 % of the votes in June 2012 may signal the return of such discredited phenotypic notions of race.

122

Christina Konstantinidou and Martha Michailidou

Fig. 13: Photograph of rescued migrants in the Cretan coast in 2007. Ethnos, 19/5/2009. © Ethnos.

Fig. 14: “Historical centre: People and mice in Athens”, Ta Nea, 23/5/2011. © Ta Nea & Enri CanaJ.

Foucauldian discourse analysis

Fig. 15: “Illegal immigrants … guests in prisons”, Eleftheros Typos, 19/4/2012. © Eleftheros Typos & Phasma.

123

124

Christina Konstantinidou and Martha Michailidou

migration30 spatiality becomes a key component of the problematisation and policing of immigration. This strategic territorialisation of immigration ensures, as Hall et al. (1978: 118) put it, that “since the problem had been localised, it was amenable to local solutions”.

3.6 Framing devices: quantification and mapping The positivity produced through the visual markers of identity is enhanced through powerful and exemplary visual narratives and infographics (photos with maps, diagrams, statistical tables). The latter lend an aura of “scientific evidence” to the otherwise generic photographs, producing specific scientific “reality effects” and a special weight in contemporary Greek “politics of truth” (Tagg 1988: 189; Chouliaraki 2005: 275). Indeed, the infographics we collected during the study

Fig. 16: Infographic, Ta Nea, 21/7/2009 © Ta Nea.

 Pagani detention centre in Lesvos, Agios Panteleimon inner city neighbourhood in Athens, the historical centre of Athens, the port and railway areas of Patras, Chania town mosque, Agathonisi, Farmakonisi, New Manolada strawberry fields are only some of the places which become emblematic in the geographical meta-narrative of news reporting of immigration throughout the study period.

Foucauldian discourse analysis

125

Fig. 17: Infographic, Ta Nea, 31/3/2010 © Ta Nea.

period, are very interesting not only with regard to the way they consistently select and present photos of (racially) stereotyped people but because of the combination of the evidential force of photography with the rhetoric power of graphics, either in the form of population statistics (Figure 16) or in the form of visual maps of migration (Figure 17).

126

Christina Konstantinidou and Martha Michailidou

Press narratives of immigration as either a threat or a humanitarian crisis abound with accounts of quantitative data in order to provide news reporting with the truthfulness of scientific knowledge. In doing so, they testify to media reliance on primary definer’s conceptualisations of events, but also they draw from an established discursive regime of truth of western modernity that equates knowledge, government administration, and symbolic representation with the rise of specific techniques of measurement, like censuses, polls, and other official routines of surveying, counting, and classifying population (Urla 1993), operating as an “engine for making up people” (Hacking 2006) in the government of populations. The Greek press’s transcriptions of these “technologies of knowledge and power production” into public idiom provide an original visualisation of immigration as a “problem”. For example, let us briefly consider an image that appears in 21/7/2009 in the newspaper Ta Nea (Figure 16). The image, under the heading “explosion of lathrometanastes’ entries”, is a combination of two distinct statistical graphs: a series of bars illustrating lathrometanastes monthly apprehensions during 2008 (lower graph) and an orange line vector illustrating the total number of annual apprehensions from 2004 to 2008 (upper graph). What really gives a high truth modality to the image, however, is the background colour photograph of an old vessel apparently overflowing with young men in an unidentified port. Here, the photograph of the overflowed vessel embodies and vivifies empirically the published numbers of people apprehended in Greek territory. Moreover, semiotically speaking, the meaning of the image is anchored by the surrounding text reporting on the building of two detention centres in Greece and on the “explosive” situation of the “sinking” Aegean island of Lesvos because of the continual arrivals of immigrants from Turkey. This in turn reinforces the inevitability of the detention centres as solution to immigration as a problem. The evidential force of the infographic, as a whole, despite the chronologically arbitrary combination of its elements,31 is produced by the interweaving of statistical discourse within the discursive field of “news reporting of reality”, by the high visual recognisability of the specific or similar events in Greek media and by its symbolisation through repetitive use into a journalistic and illustrative theme of “boat people”, which once a theme becomes incorporated into news people’s professional routines and repertoires. The combination of photography with maps is the second kind of infographic frequently used in Greek press narratives. The map, as Neocleous (2003: 119) suggests, “is an intensely political object and the practice of cartography has been crucial to the state and its territorial imagination […] within the political discourse concerned with the acquisition and maintenance of state power”. William Walters

 The photograph depicts a ship captured in 2001 near the Ionian island of Zakynthos transporting approximately 1,000 Kurds in their attempt to go from Turkey to Italy.

Foucauldian discourse analysis

127

in his analysis of what he terms “the cartography of ‘illegal immigration’” (2009: 115), claims that these maps, used by media all over the western world, “contribute […] to the wider symbolic and semantic field within which political questions of human mobility are debated today” (2009: 114) and are closely linked to the political and social construction of immigration as a security problem. As he argues: cartographic practices, and the various ways in which they shape the meaning of key identities like ‘immigration’, ‘state’, ‘economy’ and ‘territory’, illustrate processes by which something called ‘illegal immigration’ is, on the one hand kept at a certain distance (if not completely detached) from that which our societies recognise as ‘the economic’, while on the other, maintained in an intimate relationship with themes of territory and security. (Neocleous 2009: 116)

In the example we chose from the Greek press (Figure 17), the infographic appeared in Ta Nea in 31/3/2010 (p.16) under the heading “The routes of illegal immigration”, while the banner of the page reads “Terror in Kato Patisia”32 and the title of the news item reads: “Ticket to the dream for 15,000 A. Thousands of desperate aliens risk in the routes of illegal immigration”. We chose this infographic as an example of powerful visualisation of the dominant problematisation on immigration, constructed as an out of control invasion of Europe. It is composed by a coloured close-up of a hooded dark skinned young man with his hands crossed before his face fading behind a conventional graphic map of the European Union member states belonging to the Schengen area, surrounded by sea and part of North and West Asia, North Africa and the Middle East. The first interesting point is that the map depicts the Schengen Area and the European Union33 as a united territory of national states, clearly separated from the rest of the world by grouping Africa, part of East Europe, Asia, Middle East, and the European SouthEastern countries that do not belong to EU, together in the same colour. The visual construction of the boundaries between “Europe and the Rest”, to paraphrase Hall’s famous expression, is highlighted by the bold colour arrows which represent the alleged different routes of illegal immigration to Europe (i.e. Central Mediterranean, West African, Centre-Orient European, South Eastern European), as well as the routes within the Schengen countries area (by dotted arrows). As Walters (2009: 122–123) observes, this mapping of illegal immigration, reminiscent as it is of classical maps of warfare, transposes the image of Europe as a “‘theatre’ of military operations” onto the transnational flows of people and goods,

 It must be noted that the upper banner which refers to “terror” in the centre of Athens, though usually linked to the journalistic theme of “ghettoisation” of the inner city by immigrants, in this particularly news item is linked to the news item of the previous page, concerning a bomb blast that killed a 15 year Afghan boy and gravely injured his sister and their mother. The bomb exploded when the boy opened a bag he found near a garbage bin in front of an institute for training public officials in the west of Athens.  In fact the map is full of errors, confusing European Union territory with that of the Schengen Agreement area.

128

Christina Konstantinidou and Martha Michailidou

allowing their conceptualisation as “metaphorical wars”. This mundane visual and conceptual framing, can be instrumental in the formation of the discursive object “immigration”; as Walters argues “It is quite likely that the “security continuum” and the “transfer of illegitimacy” – processes which Bifo identifies as profoundly structuring the political and governmental field of migration – are effected not just through the institutional and expert framing of things, but at the level of these little image-fragments”. As for the very evocative icon of the hooded dark skinned young man, this kind of image resonates in contemporary Greek visual vocabulary of immigration because it is already established in the collective imaginary, through the imaging of the “ghetto or suburb riots” and the iconography of “hood movies” and hip-hop music and culture, signifiers that abound in popular culture and epitomise a new, violent, male racial order. Thus, the infographic discursive construction constitutes the intersection of at least six interrelated discourses: immigration, European identity and cohesion, securitarisation, cultural diversity, racial violence and gender relations concisely and suggestively conferring a higher truth value to the essentialisation of immigrant identities discussed in the previous section.

4 Concluding remarks This chapter attempted to analyse the visual construction of immigration in the Greek press during the economic crisis from 2009 to 2012 utilising key tools of foucauldian discourse analysis. Our examination of the corpus of the Greek press during this period focused on the routine, generic photojournalist representations of immigration, which it then attempted to read in their textual context. This close reading required, first, the tracing and analysis of the distinctive visual codes and overall systems of composition of the photographs, a task which drew upon visual semiotics as a body of work able to do justice to the specificity of visual language. This reading revealed significant repetition and systematicity in the visual and textual representations of immigration, systematicity which contributes to the visual establishment of the neologism lathrometanastes for the blanket description of the varied, heterogeneous groups of migrants located in contemporary Greece. We conceptualised these repetitive, routine, generic representations as discursive statements which produce and reproduce the object lathrometanastes as a homogenised racialised and gendered grouping, as a grouping defined by its “race truth” (Nayak 2006: 423) and irrevocable difference from the Greek self. We defined these devices as a set of interlocking structures – as parts of a dispositif – which work through race and space, conferring the evidentiality of the photograph onto the texts which surround them, and argued that the increasing frequency and intensity of these statements should be analysed as a moral panic which problematises immigration as an urgent problem requiring an urgent solution. This problematisa-

Foucauldian discourse analysis

129

tion assigns lathrometanastes with specific gender and racial characteristics, envisages them as a human mass, and emblematically locates them, through a precise imaginary itinerary from borders to specific positions trapped within a ghettoised territory, either in the “historical” centre of the capital of Greece or at the liminal points of the leaky borders. We are aware that we have only been able to give a static picture of the discursive construction of immigration by the contemporary Greek press through its representation of the paradigmatic object lathrometanastis. This static picture, the result of the chapter’s methodological focus on the ways in which textual and visual elements operate as statements designating the object lathrometanastis with specific attributes and in so doing problematising it, is only part of an analysis of the discursive construction of immigration in contemporary Greece. This latter would require a much more extended and in depth examination of both the statics and the dynamics of the discursive production of immigration as an urgent problem in contemporary Greece, of its conditions of possibility and its relation to other discursive and non discursive terrains, of its points of connection with party political discourses and state regulations, with objectifying practices and knowledges. We have alluded to some conditions of possibility of the discursive object lathrometanastis, especially the ones rooted in professional media routines and practices, and their reliance on primary definers as sources, but have only made passing references to the conditions of possibility which arise outside media discourses and routines but may crucially interlock with them in the formation of the dispositif of immigration. The former would certainly complete the picture with regards to the conditions of production, the professional logics and routine practices of selection of the textual and visual elements which make up media up media representations. Such an account, only possible through the application of methods of field research, ethnographic work and in depth interviewing, would both shed more light on the specific empirical question which has been the point of departure of this chapter and would be additionally useful as a methodological corrective to a determinist understanding of discourse as a master narrative that somehow imposes itself in and of itself. The latter would necessitate the tracing of the initial emergence of the term lathrometanastis in the rhetoric of far right political parties, and an in depth analysis of the texts and imperatives problematising immigration outside the realm of the media, in party political rhetoric and state institutions, their articulation with other imperatives of government and their solidification into a term which acquired the status of common currency.

130

Christina Konstantinidou and Martha Michailidou

References Agamben, Giorgio. 1998. Homo Sacer. Sovereign Power and Bare Life. Stanford: Stanford University Press. Askouni, Nelli. 1997. Facing two contradictory figures of the racist other: elements from an analysis in teachers. In Anna Fragoudaki and Dragona Thaleia (eds.), «Our Country». Ethnocentrism in Education, 283–322. Athens: Alexandria [in Greek]. Auclair, George. 1982. Le Mana Quotidien. Structures et Fonctions de la Chronique de Faits Divers. Paris: Anthropos. Barthes, Roland. 1961. Le message photographique. Communications 1: 127–138. Barthes, Roland. 2003. Rhetoric of the image. In Liz Wells (ed.), The Photography Reader, 114–125. London/New York: Routledge. Batchen, Geoffrey. 1997. Burning with Desire: The Conception of Photography. Cambridge: MIT Press. Batziou, Athanasia. 2011. Picturing Immigration. Photojournalistic Representation of Immigrants in Greek and Spanish Press. Bristol: Intellect. Burgin, Victor (ed.). 1982. Thinking Photography. New York: Palgrave Macmillan. Butler, Judith. 1993. Endangered/endangering: Schematic racism and white paranoia. In Robert Cooding-Williams (ed.), Reading Rodney King. Reading Urban Uprising, 15–22. New York/ London: Routledge. Chouliaraki, Lilie. 2005. Media Discourse and the Public Sphere. In David R. Howarth and Jacob Torfing (eds.), Discourse Theory in European Politics, 275–298. New York: Palgrave Macmillan. Cohen, Robin. 1994. Frontiers of Identity: The British and the Others. London: Longman. Cohen, Stanley. [1972] 2011. Folk Devils and Moral Panics. Abingdon: Routledge classics. Dimitras, Panagiotis. 1995. Greek Albanophobia. Politis 122: 38–42. [in Greek]. Dreyfus, Herbert and Paul Rabinow (eds.). 1982. Michel Foucault: Beyond Structuralism and Hermeneutics. Brighton: Harverster. Dyer, Richard. 1997. White. London/New York: Routledge. Eco, Umberto. 1982. Critique of the Image. In Victor Burgin (ed.), Thinking Photography, 39–83. New York: Palgrave Macmillan. Elkins, James. 2007. Photography Theory (Art Seminar). London: Routledge. Fanon, Frantz. [1986] 2008. Black Skin, White Masks. London: Pluto Press. Fishman, Mark. 1981. Crime Waves as Ideology. In Stanley Cohen and Jock Young (eds.), The Manufacture of News. Deviance, Social Problems and the Mass Media, 98–117. London: Constable/Sage. Foucault, Michel. 1979. On Governmentality. Ideology and Conciousness 6: 5–21. Foucault, Michel. 1980. The Confession of the Flesh. In Colin Gordon (ed.), Power/Knowledge: Selected Interviews and Other Writings, 1972–1977, 194–228. Brighton: Harvester Press. Foucault, Michel. 1981. The Order of Discourse. In Robert Young (ed.), Untying the Text: a Poststructuralist Reader, 51–78. London: Routledge. Foucault, Michel. 1994. Dits et Ecrits, Vol. IV, 1954–1988. Paris: Editions Gallimard. Foucault, Michel. [1972] 2002. Archaeology of Knowledge. London: Routledge. Fragoudaki, Anna and Thaleia Dragona (eds.). 1997. «Our Country». Ethnocentrism in Education. Athens: Alexandria. [in Greek]. Gilroy, Paul. 2004. Between Camps: Nations, Culture and the Allure of Race. London: Routledge. Greek Human Rights Watch. 1995–1996. Hate Speech in the Greek Media. Athens: Greek Human Rights Watch. Guillaumin, Colette. 1972. L’Ideologie Raciste. Genese et Langage Actuel. Paris: La Haye/Mouton. Guillaumin, Colette. 1995. Racism, Sexism, Power and Ideology. (Critical Studies in Racism and Migration). London/New York: Routledge.

Foucauldian discourse analysis

131

Hacking, Ian. 2006. Making Up People. London Review of Books 28: 23–26. Hall, Stuart (ed.). 1997. Representation: Cultural Representations and Signifying Practices. London: Sage. Hall, Stuart, Chas Critcher, Tony Jefferson, John Clarke and Brian Roberts. 1978. Policing the Crisis: Mugging the State and Law and Order. London: Macmillan. Halliday, Michael A. 1978. Language as Social Semiotic. London: Edward Arnold. Hallin, Daniel C. and Stylianos Papathanassopoulos. 2002. Political clientelism and the media: Southern Europe and Latin America in comparative perspective. Media, Culture & Society 24: 175–195. Howarth, David. 2000. Discourse. Buckingham: Open University Press. Jameson, Fredric. [1984] 1991. Postmodernism, or, the Cultural Logic of Late Capitalism. London/ New York: Verso. Jewitt, Carey and Rumiko Oyama. 2001. Visual Meaning: A social semiotic approach. In Theo Van Leeuven and Carey Jewitt (eds.), Handbook of Visual Analysis, 134–156. London/Thousand Oaks/New Delhi: Sage. Kambouri, Helen. 2009. Images of Pakistani Migrants in the Greek and the Pakistani Migrant Press. In Christos Dermentzopoulos (scientific coordination) (ed.), Culture, Identity and movement: A Study in the social anthropology of the everyday life and popular representations of migrants from Pakistan in Nea Ionia, 87–134. Athens: Latsis Public Benefit Foundation. Kapllani, Gazmend and Nicola Mai. 2005. Greece belongs to Greeks!: The case of the Greek flag in the hands of an Albanian student. In Russell King, Nicola Mai and Stephanie SchwandersSievers (eds.), The New Albanian Migration, 153–172. Brighton: Sussex Academic Press. Karydis, Vasilis. 1996. Immigrant Criminality in Greece. Questions on Theory and Anticrime Policy. Athens: Papazisi (in Greek). Kendall, Gavin and Gary Wickham. 1999. Using Foucault’s Methods. London: Sage. Konstantinidou, Christina. 2001. Societal representations of crime. The criminality of albanian immigrants in Athens press. In Hans-Jörg Alberecht, Afroditi Koukoutsaki and Telemach Serassis (eds.), Images of Crime, Representations of Crime and the Criminal in Science, the Arts and the Media, 93–130. Freiburg: Max-Planck Institut. Konstantinidou, Christina. 2007. Death, Lamentation and the Photographic Representation of the Other During the Second Iraq War in Greek Newspapers, International Journal of Cultural Studies 10: 147–166. Konstantinidou, Christina. 2008. The spectacle of suffering and death: The photographic representation of war in Greek newspapers. Visual Communication 7: 143–169. Kress, Gunther and Theo Van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London/NewYork: Routledge. Kundnani, A. 2001. In a Foreign Land: The new popular racism, Race and Class 43: 41–60. Lalioti, Vassiliki. 2005. Photographing the “Other”: Immigration and new images of the Greek ethnic “self”. Visual Anthropology 18: 439–456. Leandros, Nicos, Dimitra Papadopoulou and Marianna Psylla. 2011. The ‘crisis’ in the press. A thematic and linguistic analysis. The Greek Review of Social Research 134–135: 237–255. [in Greek]. Lentin, Ronit. 2012. Turbans, Hijabs and Other Differences: ‘Integration from below’ and Irish Interculturalism. European Journal of Cultural Studies 15: 226–242. Lister, Martin and Liz Wells. 2001. Seeing Beyond Belief: Cultural studies as an approach to analysing the visual. In Theo Van Leeuven and Carey Jewitt (eds.), Handbook of Visual Analysis, 61–92. London/Thousand Oaks/New Delhi: Sage. Lundstrom, Jan-Erik. 1999. Realism, Photography and Visual Culture. In: Lars Kiel Bertelsen, Rune Gade and Mette Sandbye (eds.), Symbolic Imprints: Essays on Photography and Visual Culture, 50–64. Aarhus University Press.

132

Christina Konstantinidou and Martha Michailidou

Machin, David. 2004. Building the world’s visual language: The increasing global importance of image banks in corporate media. Visual Communication 3: 316–336. Machin, David and Adam Jaworski. 2006. Archive video footage in news: creating a likeness and index of the phenomenal world. Visual Communication 5: 345–366. Malkki, Liisa H. 1995. Purity and Exile: Violence, Memory and National Cosmology Among Hutu Refugees in Tanzania. London: University of Chicago Press. Malkki, Liisa H. 1996. Speechless emissaries: Refugees, humanitarianism, and dehistoricization, Cultural Anthropology 11: 377–404. Maroukis, Thanos. 2008. Clandestino. Undocumented Migration. Counting the Uncountable. Brussels: Research DG, European Commission, The Hellenic Foundation for European and Foreign Policy. Mavrommatis, George, Triantafyllia Abramidi, Georgina Alexopoulou, Niki Stoupa and Zoe Balomenou. 2010. The Emergence of Geographic Narrative on Migration: Immigration and journalistic discourses in daily and weekly press. Athens: Imepo. Mchoul, Alec and Wendy Grace. [1993] 2002. A Foucault Primer. Discourse, Power and the Subject. London/New York: Routledge. Mediterranean Migration Observatory, Hellenic Migration Policy Institute and Martin BaldwinEdwards. 2004. Statistical data on immigrants in Greece: An analytic Study of available data and recommendations for conformity with European union standards. Athens: Imepo. Mikrakis, Antonis. 1994. Greece: The others within, Social Science Information 33: 787–805. Mikrakis, Antonis. 1998. We and the ‘Others’, Different, Foreigners: The conflict on a symbolic level. In Sakis Karagiorgas Foundation (ed.), Social Inequalities and Social Exclusion, 728–737. Athens: Sakis Karagiorgas Foundation. [in Greek]. Moschopoulou, Anna. 2005. Immigrants Criminality. Representations of the Issue in Evening Press, 1990–1999. Athens – Komotini: Ekd. Sakkoulas. [in Greek]. Nayak, Anoop. 2006. After race: Ethnography, race and post-race theory, Ethnic and Racial Studies 29: 411–430. Neocleous, Mark. 2003. Imagining the State. Maidenhead: Open University Press. Païdoussi, Chryssa. 1998. The ‘Voice’ of the Socially Excluded in the Mass Media. The case of television news. In: Sakis Karagiorgas Foundation (ed.) Social Inequalities and Social Exclusion, 738–748. Athens: Sakis Karagiorgas Foundation. [in Greek]. Pavlou, Miltos. 2001. The Smugglers of Fear: Racist discourse and immigrants in the press of a potential metropolis. In: Athanasios Marvakis, Dimitris Parsanoglou and Μiltos Pavlou (eds.), Immigrants in Greece Athens: Ellhnika Grammata. [in Greek]. Petraki, Georgia. 1998. Social Exclusion: Old and new ‘readings’ of the social problem. In: Please add eds (eds.), Social Inequalities and Social Exclusion, 12–24. Athens: Sakis Karagiorgas Foundation. [in Greek]. Rabinow, Paul and Rose, Nikolas. 2003. Foucault Today. In Paul Rabinow and Nikolas Rose (ed.), The Essential Foucault: Selections from Essential Works of Foucault, 1954–1984, vii–xxxv. New York: New Press. Rajchman, John. 1988. Foucault’s Art of Seeing, October 44: 89–117. Rattansi, Ali. 1992. Introduction. In James Donald and Ali Rattansi (eds.), ‘Race’, Culture and Difference, 1–10. London: Sage/The Open University. Rose, Gillian. [2001] 2007. Visual Methodologies. An Introduction to the Interpretation of Visual Materials, Second Edition. London: Sage. Rougheri, Christina, Dimitris Angelidis, Panayote Dimitras, Maria Fola, Lola Kalandrakis, Mariana Lenkova, Louisa O’ Brien, Nafsika Papanikolatos, Hebe Roccou, Vassilis Sakellariou and Persa Zeri. 1999. Greek Media Monitoring. Positive and Negative Stereotypes in the Greek Press on Internal Minorities and Neighboring Peoples. Greek Helsinki Monitor. Available at: http://www.greekhelsinki.gr/bhr/english/index.html.

Foucauldian discourse analysis

133

Said, Edward W. 2003. Orientalism. London: Penguin. Sekula, Allan. 1982. On the invention of photographic meaning. In Victor Burgin (ed.), Thinking Photography, 85–109. Houndmills, Basingstoke/New York: Palgrave Macmillan. Skoulariki, Athina. 2009. Muslims, state and the never build mosque. Avgi daily newspaper. 7/6/2009 ed. Spinelli, Kalliopi. 1992. Allien immigrants in Greece. The Rights of Man, and the loss of freedom. Athens: Report to the Council of Europe. [in Greek]. Stratoudaki, Chara. 2007. Research on Migrants in Greece. Research Obsessions and Gaps. Athens: National Centre for Social Research. Tagg, John. 1988. The Burden of Representation. Essays on Photographies and Histories. Minneapolis: University of Minnesota Press. Taylor, John. 1998. Body Horror. photojournalism, catastrophe and war. Manchester: Manchester University Press. Triandafyllidou, Anna. 1996. Greek Migration Policy: A critical note, Review of Modern Greek Studies 1(1): 15–22. Triandafyllidou, Anna and Antonis Mikrakis. 1995. Greece: A ghost wanders through the capital. In Bemd Baumgartl and Adrian Favell (eds.), The New Xenophobia in Europe. London: Kluwer International Law. Triandafyllidou, Anna, Antonis Mikrakis and Marina Calloni. 1997. New Greek Nationalism. Sociological Research Online 2/1. Available at: http://www.socresonline.org.uk/2/1/7.html. Tsaliki, Liza. 2008. How does the word Albanian make you feel?’ A case study in the representation of the Albanian ethnic minority within the Greek media. Media@lse Fifth Anniversary Conference LSE, London. Tyler, Imogen. 2006. ‘Welcome to Britain’. Τhe cultural politics of asylum. European Journal of Cultural Studies 9: 185–202. Urla, Jacqueline. 1993. Cultural politics in an age of statistics: numbers, nations, and the making of Basque identity. American Ethnologist 20: 818–843. Vamvakas, Vasilis. 1997. Albanian immigrants in the daily press. Representations of a clandestine existense. Dokimes 5: 7–40. [in Greek]. Van Leeuwen, Theo. 2001. Semiotics and iconography. In Theo Van Leeuwen and Carey Jewitt (eds.), Handbook of Visual Analysis, 92–118. London/Thousand Oaks/New Delhi: Sage. Van Leeuwen, Theo and Adam Jaworski. 2002. The discourses of war photography: Photojournalistic representations of the Palestinian-Israeli war. Journal of Language and Politics 1: 255–275. Voulgaris, Yannis, Dimosthenis Dodos, Panagiotis Kafetzis, Christos Lyrintzis, Ilias Nikolakopoulos, Kaiti Michalopoulou, Michalis Spourdalakis and Konstantinos Tsoukalas. 1997. Perceiving and dealing with the «Other» in contemporary Greece. Findings of an empirical research. Helleniki Epitheorisi Politikis Epistimis 5: add pp. [in Greek]. Voutira, Eutihia. 2010. Refugees: Species under threat of extinction. Syghrona Themata 109: 32– 37. [in Greek]. Walters, William. 2009. Anti-Political Economy: Cartographies of “Illegal Immigration” and the Displacement of the Economy. In Jacqueline Best and Matthew Paterson (eds.), Cultural Political Economy, 113–138. London: Routledge. Wright, Terence. 1989. Lines of Descent: Photography for evidence or interpretation? Visual Sociology 4: 63–70. Wright, Terence. 2002. Moving Images: The media representation of refugees. Visual Studies 17: 53–66.

Helen Kelly-Holmes

5 Linguistic fetish: The sociolinguistics of visual multilingualism Abstract: Linguistic fetish refers to the phenomenon of using languages for symbolic (fetishised) rather than utility (instrumental-communicative) purposes in commercial texts. In such a context, form takes precedence over content, which may or may not be relevant to, or understood by, the target audience. In this chapter, building on and extending my previous work in this area (Kelly-Holmes 2010, 2005, 2000), I explore linguistic fetishization as a sociolinguistic practice, using a range of examples from a variety of media and contexts. The chapter starts by examining current thinking on visual multilingualism in sociolinguistics, before moving on to examine the commodification of such visual language in contemporary consumer culture using the notion of linguistic fetish. I then go on to examine three cases of linguistic fetish in visual multilingualism – the foreign language visual; the minority language visual; and visual English – and attempt an assessment of their sociolinguistic implications.

1 Introduction While visuality is becoming increasingly central to contemporary consumer culture and “marketing as a practice reflects this intense engagement with visuality” (de Burgh-Woodman and Brace-Govan 2010: 188), considerations and analyses of visual consumption or visual marketing do not generally consider the role of language in this process (cf. for example Schroeder 2006). This is despite the increasing use of language for its visual qualities and or symbolic purposes rather than for communicative or instrumental functions in contemporary consumer discourses such as advertising. Likewise, in the study of language, a preoccupation with its visual materiality is a relatively recent phenomenon. In this chapter, I look at what I term visual multilingualism, namely the use of languages primarily as images, with an analysis that is informed by sociolinguistics, the study of language use in its social context. I start by examining current thinking on visual multilingualism in sociolinguistics before moving on to examine the commodification of such visual language in contemporary consumer culture using the notion of linguistic fetish. I then go on to examine three cases of linguistic fetish in visual multilingualism – the foreign language visual; the minority language visual; and visual English – and assess their sociolinguistic implications.

136

Helen Kelly-Holmes

2 Visual multilingualism While the system of language has long been an object of study, with lexicographers, grammarians and syntacticians attempting to codify and record the relations between words, spellings, structures and meanings, attention to language as a social phenomenon is a relatively new phenomenon. It was only in the latter half of the 20th century that there was a decisive move away from studying language primarily as an objective, inanimate phenomenon to studying it in relation to its speakers, and more precisely to different types of speakers (differentiated by, for example, location, social class, educational background, gender, age) (cf. Spolsky 2009 for an overview here). From this “social turn” in the study of language grew the contemporary discipline of sociolinguistics, which itself has also moved away from its original focus on modernist constructs such as class to consider all dimensions of language in use, including the issue of concern in this chapter, namely visual uses of language. Probably the most well-known and widespread approach to analysing visual multilingualism is the “linguistic landscapes” approach, developed by Landry and Bourhis (1997) and advanced by many scholars since (e.g. Shohamy and Gorter 2009). The concept of linguistic landscape represents an attempt to account for the visual presence of particular languages in the public space as a reflection of and contribution to ethnolinguistic vitality, i.e. the relative strength of these languages in terms of their status and functions as “living languages” within their immediate speech communities and beyond, particularly in minority language spaces and sites of complex multilingualism. The technique involves documenting visual multilingualism (and also monolingualism) by systematically counting the presence and frequency of languages on public signs in various domains. As such, linguistic landscape analysis represented a departure from the focus on text semantics in the study of language use, since content and/or meaning were not initially important, a focus was on the visual presence or absence of particular languages in particular spaces. The idea behind the linguistic landscape approach was that the visibility of particular languages could reflect their relative position in the sociolinguistic hierarchy, and that a greater visibility of one particular language could send the message that this was the dominant language. Thus, the visibility and visual positioning of particular languages could be used as a way of revealing common-sense ideologies about language(s) that are prevalent in a particular society. As Heller (2008: 518) tells us, “our ideas about language are not neutral; we believe what we believe for reasons which have to do with the many other ways in which we make sense of our world and make our way in it”, and the languages we see around us everyday are considered to be part of this, whether or not we speak them or understand them or even recognise them, beyond an acknowledgement that they are not “our” language. The investigation of visual multilingualism, in

Linguistic fetish: The sociolinguistics of visual multilingualism

137

the form of linguistic landscape analysis, can thus be seen to be informed by and contributing to understanding “language ideologies”, which are considered to be “the discourses that attribute value to linguistic forms and practices, along with the processes of constructing social difference and social inequality with which they are associated” (Heller 2008: 518) in contemporary and historical perspective. A further indication of the power of visual multilingualism is the fact that status planning initiatives in minority language spaces have both responded to and initiated linguistic landscape studies by reversing what is seen to be an imbalance in visual multilingualism and/or monolingualism by introducing legislation which requires that the minority language is printed on public signs in addition to the relevant dominant language (e.g. Wales, Ireland) or that it is to appear on its own (e.g. Quebec). Linguistic landscape studies have not just concerned themselves with the presence and frequency of languages on public signs, but have also focussed on the visual positioning of different languages on the signs, and, again, this notion that the positioning of languages is important has also been taken up by language planners, who now often require that the existing visual hierarchy is reversed by placing the minority language on the top of the sign above the dominant language (e.g. in Ireland, where Irish is placed above English) (cf. Shohamy and Gorter 2009 and Gorter, Marten and van Mensel 2011, for examples of studies). Clearly, then, such approaches represent a concern with the material properties of the printed language, rather than with what is being communicated by the printed language, since, in many cases, the content is the same (e.g. on road signs, where the same place name is given in parallel versions). Linguistic landscape analyses have developed considerably since the pioneering study of Landry and Bourhis (1997) (for an overview of the development of linguistic landscape research, cf. Jaworski and Thurlow 2010). In a recent volume, Jaworski and Thurlow (2010) advocate a move from focusing on the linguistic landscape to considering the semiotic landscape, in order “to emphasize the way written discourse interacts with other discursive modalities: visual images, nonverbal communication, architecture and the built environment” (Jaworski and Thurlow 2010: 4). They are interested in “defining landscape as a way of seeing” (Jaworski and Thurlow 2010: 4), which is “subject to a number of competing ‘scopic regimes’ or ‘visual subcultures’” (Jay 1998: 4 in Jaworski and Thurlow 2010: 4). This implies that how we see is mediated by social and cultural practices and structures. This also applies to language: in the same way that how we hear language – in terms of hearing and differentiating accents, dialects and other languages – is mediated by cultural practices and social structures, how we “see” another language or variety depends on the particular “scopic regime” to which we are subject; our gaze is dependent on our particular habitus in Bourdieu’s terms (1991). Thus, how we see language is pre-formed and then either reinforced or challenged by the visual multilingualism or monolingualism we encounter on a daily basis – hence the desire by language planners to reverse existing visual hierarchies in order to change language ideologies within the particular habitus.

138

Helen Kelly-Holmes

Jaworski and Thurlow’s (2010) approach reflects a new focus in sociolinguistics on multi-modality, in recognition of “the essentially multimodal nature of all human meaning making. Multimodality, here, is about recognizing that language is not at all at the centre of all communication” (Iedema 2003: 39). Thus, it is not just changes in our environment, economy and technology, which are leading to greater multimodality, but a recognition of meaning (and of language) as multimodal. This is quite a radical mainstreaming, since it appears to be taking the focus off language as the immediate object of study. This means on the one hand that our sociolinguistic environment is experienced as increasingly multi-modal while, at the same time, there is also a recognition among sociolinguists that far more linguistic encounters are multi-modal than was previously considered the case in analyses which looked at speech and writing without considering in full the existence, impact and inter-connectedness of other modes, what Iedema (2003) terms “the material and historicized dimensions of representation” (Iedema 2003: 50). Thus, the way a piece of language looks is also a mode of meaning, not just the semantic meaning of what it is referring to (cf. Kress and Van Leeuwen 2001, 2006). In addition, the sociolinguistic techniques for studying linguistic phenomena primarily experienced and documented as verbal practices have increasingly been applied to written language also. For example, Eastman and Stein’s (1993) concept of “language display”, which looks at the conscious use of a language not linked to the ethnic identity of the speaker. Eastman and Stein (1993) argue that language display is in fact most successful where there is limited or even no knowledge of the language being displayed on the part of the speaker. Language display “represents symbolic rather than structural or semantic expression” (Eastman and Stein 1993: 200), with the focus on form rather than content; what is being said, is not important, instead it is the language in which it is being said – this is the effect which the speaker is trying to achieve. Language display, while originally conceived of in relation to oral data, has increasingly been combined with written data in the study of visual multilingualism (cf. Kelly-Holmes (2005), and Coupland (2012) for a recent study of the display of Welsh in the linguistic landscape of Wales). Language display is just one example of the growth of interest in “‘spectacular’ uses of language, in which a variety is begged, borrowed, or stolen by speakers who don’t normally claim it” (Sweetland 2002: 515). Where previously, with the focus in sociolinguistics firmly on stable and long-standing oral speech communities, they would have been dismissed as idiosyncratic and insignificant, such practices have, Sweetland tells us, recently become “[…] prized [in sociolinguistics literature] for their value in understanding the social meanings that adhere to language varieties and the many ways in which speakers can put such ideologies to work” (2002: 516). “Inauthentic language” (Sweetland 2002) in the form of, for example, “styling the other” (Bell 1999), “mock” language (cf. Hill 1999), “crossing” (Rampton 1995), and language display (Eastman and Stein 1993) etc. are increasingly being studied as written phenomena and for visual effects rather than just verbal effects.

Linguistic fetish: The sociolinguistics of visual multilingualism

139

3 Linguistic fetish The concept of linguistic fetish has been developed to explain multilingualism in economically driven displays, such as marketing and advertising texts. It is important from the outset to acknowledge that all language relations, as highlighted above in relation to linguistic landscapes, to a greater or lesser extent reflect some type of power relations. The local (and global) political economy of language (Gal 1989), the linguistic market (Bourdieu 1991) and other terms have been coined to explain the interrelationship between language choice, language ideologies, and political, social and economic power. Such relations permeate all domains of language use – the school, the government, the media, the university, and even the family – not just explicitly economically driven displays and choices. Thus, while the focus in linguistic fetish is on recognisably market-driven texts, it is important to see linguistic fetish within a wider field of power relations. Fetishisation involves “the capacity of creating [symbolic] value – a value greater than it contains” (Marx and Engels, 1959 (1894): 392). This leads to a situation where there is “form without content”, (Marx, 1959 (1894): 393), or where form or symbolic meaning take precedence over content or utility. Utility in terms of language can (crudely) be seen in terms of its communicative function, or use as a means of communication between speakers. Where the utility value of the language is not fetishised, then the content, the (semantic) meanings themselves are the essence – rather than the form. In linguistic fetish, the symbolic or visual value of a language takes precedence over its communicative value, and this symbolic value is the product of existing linguistic hierarchies and regimes, ways of seeing. As Heller points out in relation to the commodification of languages in the current era: “[…] the new economy’s valuing of increasingly commodified cultural artefacts and symbolic resources blurs the relationship between political symbols and exchange goods” (Heller 2008: 512) Linguistic fetish, then, results in a highly refined version of multilingualism, informed by a culturally determined gaze. It is a type of decoration or linguistic colour by numbers, that has everything to do with the producer’s perception of the consumers’ own linguistic culture or habitus, as “visuality is inducted into the service of creating a homogenous cultural perspective” (de Burgh-Woodman and Brace-Govan 2010: 177). Linguistic colour by numbers describes the very contained, prescribed parameters by which words from other languages are used in advertisements. The choice is driven primarily not by what the word means, but how it looks, and what associations the language may have among target addressees, and how these can be triggered by the appearance of the foreign word. Reflecting this, Harald Haarman (1989) coined the term “impersonal bilingualism” to explain the occurrence of foreign words in Japanese advertising. He concluded that their usage had nothing to do with “everyday” or “real” bi- and or multilingualism in Japan and instead were to do with exploiting symbolic associations of

140

Helen Kelly-Holmes

these languages, which were not widely understood or taught in Japan at the time of his study, would have among the Japanese public. The commodification of visual multilingualism can be seen as part of a wider process of commodification of the visual: “visuality is crucial to the current economic structures”, and “visual consumption constitutes a key attribute of an experience economy organized around attention” (Schroeder 2006: 6). Jaworski et al. (2003) have analysed the importance of “linguascaping”, which involves designing “the sounds and visual representations of special or unique language codes and varieties (accents, dialects) often used for symbolic purposes” (Jaworski et al. 2003: 3), of tourist destinations to enhance the tourist experience and ultimately add value to it. Thus, language too can form part of the tourist gaze (Urry 2002). This realisation also changes our understanding of language. Jaworski and Thurlow (2011) show how language has become “extra-visual”, while Machin and Van Leeuwen (2003) tell us that language has become a “surface value” as a result of global media genres, which spread homogenous mediascapes with local languages used simply for instrumental functions. While linguistic fetish emphasizes the distinction between the communicative and the symbolic or between the textual and visual in communication, it is important to remember that this differentiation is somewhat articifical, since most communication consists of both symbolic and communicative aspects. However, as Juliane House (2003) has argued, it can be useful to try to distinguish the use of “language for communication” from the use of “language for identification”, as she puts it. In addition, Wee (2003: 212) also tells us that while “it is obviously possible for one and the same language to be seen as having both instrumentalist and non-instrumentalist functions, […] it is also important to keep the functions distinct”. Thus, while accepting the interconnection between use value and symbolic value, between the textual and the visual, linguistic fetish proposes and recognises the primacy of the symbolic in the commodification of visual multilingualism in contemporary advertising. We will now look at three examples of linguistic fetish. The distinctions between them are slightly artificial, as shall become clear, and they all share commonalities in approach. However, each one highlights a particular dimension to linguistic fetish which is interesting and useful to explore. We will start with the first and most obvious case of linguistic fetish, linked to the discussion above, namely the use of “foreign” languages as visuals; then we will move onto two other cases, the first involving the use of “visual English”. Although English is of course a foreign or second language in most parts of the world, its fetishisation is, it has been argued, a special case that is not linked to a particular ethnosymbolic identity, unlike the use of other languages as visuals. The final case involves the use of minority languages as visuals.

Linguistic fetish: The sociolinguistics of visual multilingualism

141

4 Foreign languages as visual The basis for the use of the foreign language fetish, as described above, is the understanding that “an image of language may be structured only from the point of view of another language, which is taken as the norm” (Bakhtin 1981: 360). Thus, the habitus, in Bourdieu’s (1991) terms, of the advertisee is the norm against which foreignness or otherness is measured and against which the “foreign” or other language is understood. It is the blank paper that gets coloured in. Lawrence Venuti, in his study of translation, argues that the objective of translation is to erase as much as possible of the original context and to domesticate words and concepts for the target audience, particularly in the Anglophone world. The text should appear “seemingly untranslated [by] suppressing the linguistic and cultural difference of the foreign text, assimilating it to dominant values in the target language culture” (1994: 218). In contrast, the use of foreign languages in advertising is not about erasure (Irvine and Gal 2000) of difference, but in fact about the display (Eastman and Stein 2003) and highlighting of difference. As Kress and Van Leeuwen (2006: 12) tell us, “we have available the culturally produced semiotic resources of our societies, and are aware of the conventions and constraints which are socially imposed on our making of signs”, and foreign language visuals and their histories are part of those semiotic resources. Figure 1 shows us a good example of the foreign language fetish in the screen shot of the Stella Artois homepage, which is advertising the launch of Stella Cidre, the Stella Artois brand being better known for its beer. The message of the adver-

Fig. 1: Homepage of www.stellaartois.com/cidre/.

142

Helen Kelly-Holmes

tisement is that “Cidre” (the French word for cider) is completely different to Cider (the English word), not just linguistically, but also materially – it is a completely different drink. The main vehicle for highlighting this difference is the rendering of the word in French for an Anglophone audience. The slogan of the campaign is “C’est cidre. Not –> cider”. It assumes a fairly widespread knowledge of the basics of the French language in the Anglophone world not just as a result of education, but also as a result of a type of “mock” French (like mock Spanish – cf. Hill 1999) spread by the portrayal of the French speaker in mainstream media products in the Anglophone world. The ad also offers a phonetic guide for Anglophones (cidre => see-dra), creating a manifest intertextual (Fairclough 2001) link with travel and tourist guides, where this kind of ethnocentric phoneticisation is a common practice. The ad also assumes the need for this pedantic explanation and domestication, and derives its humour from hyperbolising the ethnocentrism and monolingualism of the Anglophone on the one hand, and the chauvinistic exclusionary tendency of the Francophone, on the other, obsessed equally and pedantically with the particularities of their language and culinary habits. Finally, the link between the visual French and the product is spelled out explicitly: “Stella Artois Cidre is a premium crafted Belgian cider made with hand-picked apples”. So, the instrumental or authoritative message is delivered in English, which further highlights the visual functions of the French language in the ad. This example shows us how linguistic fetish is all about understanding foreignness from the point of view of one’s own habitus, as part of the culturally available repertoire of signs, to paraphrase Kress and Van Leeuwen (2006). The foreign words on display in the text are, in Bakhtin’s words “structured only from the point of view of another language, which is taken as the norm” (Bakhtin 1981: 360), in this case English. English is the blank canvas on which a little French is painted. The repeated use of “Cidre” serves no instrumental function, since its literal meaning is the same as that of the English word cider; instead its use shows us how, according to Adorno: “foreign words become the bearers of subjective contents: of the nuances. The meanings in one’s own language may well correspond to the meanings of the foreign words in every case; but they cannot be arbitrarily replaced by them because the expression of subjectivity cannot simply be dissolved in meaning” (1991: 287). Stella Artois is utilising a common practice in foreign language fetish, whereby the name of the product in the language of the country of origin “becomes the bearer of subjective contents”, in Adorno’s words. Familiar examples are “Bier” (for German beer); “Auto” (for German cars); “Technik” (for German technology); “Kaas” (for Dutch cheese); as well as all of the French vocabulary that permeates the language of cosmetics (“parfum” etc.) (cf. Kelly-Holmes 2005 for an in-depth discussion of these). The commodification of languages in the new economy (and in the “new communication landscape” in Kress and Van Leeuwen’s 2001 terms) involves “the

Linguistic fetish: The sociolinguistics of visual multilingualism

143

naturalisation, uniformisation, and objectification of social groups (and hence their cultures, their boundaries and their ‘languages’” (Heller 2008: 505). Linguistic fetish thus allows the “foreign” language to become domesticated in Venuti’s (1994) terms as one of the available resources for sign-making in the habitus.

5 Visual English The contemporary functioning of “visual” English illustrates well the dissolving of cultural, political and semiotic boundaries, as highlighted by Kress and Van Leeuwan (2006, 2001), which both result from and contribute to the global flows of capital, media, ideas, people and technology and their accompanying instruments of homogenisation (Appadurai 1996), foremost among which are advertising and the English language. English, Kachru (1996) tells us, travels via “channels which bypass the strategies devised by language planners” and turns up in the most unexpected places: in countries which are considered “outer circle” (Kachru 1996), i.e. locations where English usage is the legacy of colonisation and where the language enjoys special status in education, business and many other domains; and, in countries which are considered “expanding circle” (Kachru 1996), where English is learned as a foreign language; in situations where English is well-known and understood; and, in situations where understanding is limited to cultural and educational elites, or younger generations, or is confined to specific domains. Unlike “foreign” languages, English has come to be ideologised as separate to multilingualism and not indexical of any particular country or “otherness”. Instead, its fetish is that of neutrality, globality etc. Wee (2010: 424) highlights that “a language may … be considered neutral because it represents the language of an Other”, i.e. it is neutral because it belongs to someone else outside of the habitus in question; however, with English, its supposed neutrality is based on the ideology that it represents the language of no-one and simultaneously the language of everyone: “In the USA and in many other parts of the world, English has become a neutral language, a code that can be used to avoid whatever implications may result from choosing a more identity-laden language” (Callahan 2005: 284). In complex multilingual situations, English has acquired neutral status, even though of course its usage, spread and heritage are far from neutral. So, instead of attempting to fulfil the impossible task of addressing everyone in their own language in for example a country like India, marketers opt for English, thus enhancing its “neutral” status and fitness for this purpose (cf. Kelly-Holmes 2005; Louw 2004; Cheshire and Moser 1994). Louw (2004) describes in detail the move to “neutral” English in commercial discourse in South Africa – a country with an officially multilingual policy – together with the side-lining of the negatively connoted, though widely understood, language of Afrikaans. Elizabeth Martin (2011) sums up the “display” functions of English as follows: “… superior technology, chic and modern lifestyles, adventure, international and …

144

Helen Kelly-Holmes

Fig. 2: Toyota’s .co.jp site.

the sense of belonging to a ‘global village’” (Martin 2011: 267). The example reproduced in Figure 2 illustrates many of these features. It shows the home page of Toyota’s .jp site, i.e. its dedicated site for Japanese customers. Although considered one of the largest global brands, Toyota is Japanese owned and is widely recognised as a Japanese origin brand. In common with many brands that construct themselves as global, Toyota operates a global gateway site (www.toyotaglobal.com), (reproduced in Figure 3) which is available in English, and which is intended to guide consumers into their regional, national or linguistically relevant site. The use of English on global gateway sites has become a common-sense assumption, even for brands which do not originate in English-speaking countries, for many of the reasons highlighted above: English is the neutral choice, the global choice, the language of technology etc. To not have a global gateway site in English, but to have it instead in the corporation’s home language (in this case Japanese) or even in another international lingua franca or supercentral language (de Swaan 2001) such as French or Spanish, without an English option would in the contemporary era be remarkable. The use of English in the gateway site reflects a combination of symbolic and instrumental factors: on the one hand the desire to exploit the “global” fetish of visual English, on the other, the desire – and necessity – to exploit the distribution and functioning of English in the world today. Japanese does appear on the site, indicating a link, not to a translated version of the global gateway site, but to the local Japanese site. The use of English on the global gateway site shows the interrelationship between communcative and symbolic functioning, between textual and visual concerns in language choice: English

Linguistic fetish: The sociolinguistics of visual multilingualism

145

Fig. 3: Homepage of Toyota-global.com.

is present in order to communicate with as many customers as possible; English is also displayed to index the brand’s global presence and credibility. While symbolic and instrumental concerns are reflected in the choice of English for the gateway site, the use of visual English in the .co.jp site, the site for Japanese consumers of the Japanese brand, would appear to be completely driven by symbolic concerns, as illustrated in the slogan accompanying the Auris (“Not authority but Auris”). English appears in a number of places on the site, indexing – in instrumental and symbolic ways – Toyota’s global presence. For example, terms such as “Hybrid”, “Pick Up”, and the homogenising slogan “Find us on Facebook”, create the “global village” feel, alluded to by Martin (2011). The use of English in advertising by a Japanese company for a Japanese market is a good example of visual English and its associated fetishes of globality. Toyota is not just a Japanese brand for Japanese consumers, it is also a global brand for Japanese consumers, and this global credibility is indexed by the visual English, even where the words are not understood. Doulton (2004: 285) cites mass media as the main source of contact with English words in Japan: “nearly all gairaigo (Western loanwords) are first introduced to the Japanese public by a very small number of individuals with most Japanese people having never heard or read the word before, and having no role to play in its borrowing. Because of this presumptuous use of foreign words by the media, the meanings of many words used are poorly understood.” As in the case of other expanding circle countries, in Khachru’s terms, in which English is spoken and learned as a foreign language, English comes via “fashion magazines and commercial advertisements of modern technology” (Doul-

146

Helen Kelly-Holmes

ton 2004: 286) and “for the most part, new English expressions are introduced in the media without the audience having the necessary knowledge to understand them.” (Doulton 2004: 287). The presence of English on Toyota’s Japan website is not a reflection of an advertiser trying to service a bilingual community; instead, it is a reflection of an advertiser displaying its globality through the display of English, in Haarmann’s words, it is “impersonal bilingualism”.

6 Minority languages as visual The use of minority languages for their visuality is driven primarily by the quest for authenticity in marketing. Jones et al remind us that “two distinct strategies may be used for claiming authenticity. One is to subject one’s creative voice to the perpetuation of tradition […] A second route to authenticity […] is to be original and offer a distinctive approach” (Jones et al 2005: 893). The minority language fetish is clearly an example of the former. We are, Coupland (2003) tells us, in an era that values authenticity, and language – including visual language – is “a means of achieving authenticity” (Coupland 2003: 417). The minority language visual is an example of “language indexing authentic cultural membership” (Coupland 2003: 421). This type of essentialist authenticity is, according to Coupland (2003: 424), a “divisive and socio-politically dangerous notion”, but it is one that is frequently utilised in marketing discourses, particularly in the context of tourism. Language indexing cultural membership is of course crucial in all of the examples cited so far, particularly in terms of the foreign language fetish; however, it is particularly pronounced in relation to the display of minority languages in tourism discourses. The cultural membership here could be in terms of the product or brand itself – the language is indexing the brand’s credibility in terms of a culture that is different to that of the notional addressee of the advertising or marketing message. The language use could also be about positioning the brand and the consumer as having the same authentic cultural membership. Here the issue of the relationship between the symbolic and instrumental functions of the minority language becomes once more blurred, since there may well be comprehension of the content. However, in situations where there are no monolingual speakers of a minority language, which is the case for many of the autochthonous European minority languages (e.g. Irish, Welsh, Breton, Sámi languages), then the choice to use the minority language rather than the majority language is necessarily a marked choice, driven by symbolic rather than communicative imperatives. As Wee points out, distinguishing between instrumental and non-instrumental views of language: “a language is viewed non-instrumentally to the extent that it is seen as forming an integral part of one’s ethnic or cultural identity, and if its existence in a community is justified in terms of its symbolic value in allowing the community members to maintain a sense of identity” (Wee 2003: 211–212).

Linguistic fetish: The sociolinguistics of visual multilingualism

147

Fig. 4: Homepage of Louismulcahy.com.

In a study of the labelling of souvenirs in the multilingual minority language space of Finnish Sámiland (Pietikäinen and Kelly-Holmes 2011), the minority Sámi language was found to work symbolically and visually towards “localizing and authenticating the product”, while the instrumental functions were carried out generally by English or other languages of wider communication (e.g. German or French). Visual Sámi thus becomes part of the multiple modes of supporting and authenticating (predominantly) English language labels. In this case, Sámi operates non-instrumentally and English operates instrumentally, using Wee’s dichotomy. The visual language, here Sámi, needs to be narrated in order to become meaningful, not in terms of its semantic content (i.e. what is actually being said), but in terms of pointing out what language it actually is, like the heading or title for an illustration (cf. Pietikäinen and Kelly-Holmes 2011). In Figure 4, we can see another example of what we are calling the minority language visual. It is the homepage of the Louis Mulcahy pottery workshop, located in the designated Irish-speaking or Gaeltacht region of Ireland. As can be seen from the website, the Irish language is used in the slogan, located on the right hand side of the page below the Facebook link:

The slogan translates as the “sparrow pottery”. The decision to use an Irish slogan, without an English translation, in this global, high status domain could be seen as a normalizing move which presents Irish as being as fit for this purpose

148

Helen Kelly-Holmes

as any other language, particularly its rival English (see discussion above in relation to linguistic landscapes). However, it can be argued that the position of the slogan and the use of old Gaelic script mark the usage as graphic rather than textual (for a wider discussion of this, cf. Kelly-Holmes 2005). It is irrelevant whether the Irish version is understood. It still enhances the product because it indexes authentic cultural membership for both Irish customers (who can be part of this) and overseas customers (who wish to buy an authentic Irish product) As well as the slogan for the pottery workshop itself, Irish is used in a number of product names and the Irish words are not marked by italics, thus in paralinguistic terms, they are normalized. However, the usage is generally fetishized (Kelly-Holmes 2005), or functions as a type of linguascaping (Jaworski et al. 2003), since the use of Irish on the front page is limited to a number of tokens, with the main or serious business taking place in English, thus reinforcing the linguistic status quo: the use of Irish here makes sense in relation to the norm, the blank page, which is English, and from which it departs (cf. Bourdieu 1991, Bakhtin 1981). Driving this type of linguistic fetish is the current context of a language ideological shift from “bilingualism as deficit” to “bilingualism as added value” (Jaffe 2006: 51), and even more than this, a “valorization of multilingualism” at the global level, and the development of global markets for “the expression of ‘authentic’ minority language identities” (p. 51). In the current era, where brands need to be distinctive, visual multilingualism can be part of that distinction. In such a context, a smattering of the language, what Jacquemet (2005) terms “transidiomatic practices” can be enough to add difference and authenticity, no matter how fleeting.

7 Conclusion Visual English, foreign and minority language fetish all confirm the increasing exploitation of the visuality of language in commercial domains, underpinned by “the new communicational landscape” (Kress and Van Leeuwen 2001: 46), in particular, the World Wide Web, which “has put a premium on understanding visual consumption” (Schroeder 2006: 5). In this increasingly multi-modal context, it becomes, as seen in the examples above, harder to distinguish visual from textual functions of text. Consequently, it is not surprising that the visual is becoming a mainstream concern, and something few linguists can afford to ignore, particularly since the Web is part of people’s everyday sociolinguistic practices and also a site of growing sociolinguistic inquiry (cf., for example, Androutsopoulos 2006 and Thurlow and Mroczek 2011). Adorno (1991), writing about language fetish in translated texts, i.e. the deliberate insertion or non-replacement of a foreign word, argued that such practices

Linguistic fetish: The sociolinguistics of visual multilingualism

149

add a splash of colour to the otherwise grey, monolingual text and offer the reader temporary release from the monolingual prison. In similar fashion, we could argue that visual multilingualism is a playful practice that enriches and potentially challenges the monolingual’s world. However, we are still left with a need to question which languages are displayed, and which ones are not, and what these reveal about deep-seated and common-sense ideologies about languages and speakers. Finally, it is interesting to note the growth of visual multilingualism, which heightens and exploits linguistic difference in order to differentiate a product or brand, in parallel with a minimising of exposure to “real” multilingualism in the “new communication landscape” of the Web. For instance, as a result of IP address tracking, the archiving of previous language choices and other factors, consumers are increasingly guided through the web in a linguistic “filter bubble” (Pariser 2011), which protects them from being exposed to the “functional” or utilitarian multilingualism of the Web. Consequently, it can be argued, the effects of visual multilingualism are heightened, and the view of the other through an ethnocentric lens is reinforced. Note: This chapter is produced as part of the “Peripheral Multilingualism” Project, funded by the Academy of Finland.

References Adorno, Theodor. 1991. Notes to Literature. Volume 2. New York: Columbia University Press. Androutsopoulos, Jannis. 2006. Introduction: Sociolinguistics and computer-mediated communication. Journal of Sociolinguistics 10: 419–438. Appadurai, Arjun. 1996. Modernity at large: Cultural dimensions of globalization. Minneapolis: University of Minnesota Press. Bakhtin, Mikhail. 1981. The dialogic imagination: Four essays by M. M. Bakhtin. Trans. by Caryl Emerson and Michael Holquist. Austin: University of Texas Press. Bell, Allan. 1999. Styling the other to define the self: A study in New Zealand identity making. Journal of Sociolinguistics 3: 523–541. Bourdieu, Pierre. 1991. Language and symbolic power. Oxford: Polity Press. de Burgh-Woodman, Helene and Brace-Govan, Janice. 2010. Vista, vision and visual consumption from the Age of Enlightenment. Marketing Theory 10(2): 173–191. Callahan, Laura. 2005. Talking Both Languages: 20 Perspectives on the Use of Spanish and English Inside and Outside the Workplace. Journal of Multilingual and Multicultural Development 26(4): 275–295. Cheshire, Jenny and Moser, Lisa-Marie. 1994. English as a cultural symbol: the case of advertisements in French-speaking Switzerland. Journal of Multilingual and Multicultural Development 15(6): 451–469. Coupland, Nikolas. 2012. Bilingualism on display: The framing of Welsh and English in Welsh public spaces. Language in Society 41: 1–27. Coupland, Nikolas. 2003. Sociolinguistic authenticities. Journal of Sociolinguistics 7(3): 417–431.

150

Helen Kelly-Holmes

De Swaan, Abram. 2001. Words of the world: The global language system. Cambridge: Polity Press. Daulton, Frank E. 2004. The creation and comprehension of English loanwords in the Japanese media. Journal of Multilingual and Multicultural Development 25(4): 285–296. Eastman, Carol M. and Roberta F. Stein. 1993. Language display: Authenticating claims to social identity. Journal of Multilingual and Multicultural Development 14(3): 187–202. Fairclough, Norman. 2001. Language and Power. Second edition. Harlow: Longman. Gal, Susan. 1989. Language and political economy. Annual Review of Anthropology 18: 345–367. Gorter, Durk, Heiko F. Marten and Luk van Mensel (eds.). 2011. Minority languages in the linguistic landscape. Basingstoke/New York: Palgrave-MacMillan. Shohamy, Elana and Durk Gorter (eds.). 2009. Linguistic landscape: Expanding the scenery New York: Routledge. Haarmann, Harald. 1989. Symbolic Values of Foreign Language Use, From the Japanese Case to a General Sociolinguistic Perspective. Berlin/New York: Mouton de Gruyter. Heller, Monica. 2008. Language and the nation-state: Challenges to sociolinguistic theory and practice. Journal of Sociolinguistics 12: 504–524. Hill, Jane H. 1999. Styling locally, styling globally: What does it mean? Journal of Sociolinguistics 3: 542–556. House, Juliane. 2003. English as a lingua franca: A threat to multilingualism? Journal of Sociolinguistics 7: 556–578. Iedema, Rick. 2003. Multimodality, resemiotization: extending the analysis of discourse as multisemiotic practice. Visual Communication 2: 29–57. Irvine, Judith T. and Susan Gal. 2000. Language ideology and linguistic differentiation. In Paul V. Kroskrity (ed.), Regimes of language: Ideologies, polities, and identities, 35–83. Santa Fe: School of American Research Press. Jacquemet, Marco. 2005. Transidiomatic practices: Language and power in the age of Globalisation. Language and Communication 25: 257–277. Jaffe, Alexandra. 2006. Minority language movements. In: Monica Heller (ed.), Bilingualism: A social approach, 50–70. Basingstoke/New York: Palgrave Macmillan. Jaworksi, Adam and Crispin Thurlow. 2010. Introducing semiotic landscapes. In Adam Jaworksi and Crispin Thurlow (eds.), Semiotic Landscapes: Language, image, space, 1–40. London and New York: Continuum. Jaworski, Adam, Crispin Thurlow, Virpi Ylänne-McEwen, and Sarah Lawson. 2003. The uses and representations of local languages in tourist destinations: A view from British television holiday programmes. Language Awareness 12: 5–29. Jones, Candace, N. and Josè Luis Alvarez. 2005. Manufactured Authenticity and Creative Voice in Cultural Industries. Journal of Management Studies 42(5): 895–899. Kachru, Braj. 1996. World Englishes: Agony and ecstasy. Journal of Aesthetic Education 30(2): 135–155. Kelly-Holmes, Helen. 2005. Advertising as multilingual communication. Basingstoke/New York: Palgrave-MacMillan. Kress, Gunther and Theo Van Leeuwen. 2006. Reading images: The grammar of visual design. London/New York: Routledge. Kress, Gunther and Theo Van Leeuwen. 2001. Multimodal Discourse: Modes and meaning of contemporary communication. London: Edward Arnold. Landry, Rodrigue and Richard Bourhis. 1997. Linguistic landscape and ethnolinguistic vitality. Journal of Language and Social Psychology 16(1): 23–49. Louw, P. Eric. 2004. Anglicising Postapartheid South Africa. Journal of Multilingual and Multicultural Development 25(4): 318–332. Machin, David and Theo Van Leeuwen. 2003. Global schemas and local discourses in Cosmopolitan. Journal of Sociolinguistics 7: 493–512.

Linguistic fetish: The sociolinguistics of visual multilingualism

151

Martin, Elizabeth. 2011. Multilingualism and Web advertising: Addressing French-speaking consumers. Journal of Multilingual and Multicultural Development 32(3): 265–284. Marx, Karl. 1954 [1867]. Capital, Volume 1. London: Lawrence and Wishart. Pariser, Eli. 2011. The Filter Bubble. New York: Penguin Press. Pietikäinen, Sari, and Helen Kelly-Holmes. 2011. The local political economy of languages in a Sámi tourism destination: Authenticity and mobility in the labelling of souvenirs. Journal of Sociolinguistics 15(3): 323–346. Rampton, Ben. 1995. Crossing: Language and ethnicity among adolescents. (Real language series). London/New York: Longman. Schroeder, Jonathan E. 2006. Editorial: Introduction to the special issue on aesthetics, images and vision. Marketing Theory 6: 5–11. Spolsky, Bernard. 2009. Ferguson and Fishman: Sociolinguistics and the Sociology of Language. In Ruth Wodak, Barbara Johnstone and Paul Kerswill (eds.), Handbook of Sociolinguistics, 3–15. Thousand Oaks: Sage. Sweetland, Julie. 2002. Unexpected but authentic use of an ethnically-marked dialect. Journal of Sociolinguistics 6: 514–538. Thurlow, Crispin and Kristin Mroczek. 2011. Fresh perspectives on new media sociolinguistics. In Crispin Thurlow and Kristin Mroczek (eds.), Digital Discourse: Language in the New Media, xix–xliv. New York: Oxford University Press. Urry, John. 2002. The tourist gaze. 2nd edition. London: Sage Publications. Venuti, Lawrence. 1994. Translation and the Formation of Cultural Identities & Preliminary Remarks to the Debate. Current Issues in Language and Society 1/3: 201–224. Wee, Lionel. 2010. Neutrality in language policy. Journal of Multilingual and Multicultural Development 31(4): 421–434. Wee, Lionel. 2003. Linguistic Instrumentalism in Singapore. Journal of Multilingual and Multicultural Development 24(3): 211–224.

Paul Bowman

6 Sex and Race go Pop Abstract: This chapter examines the question of the relations between media representations, their interpretations, and popular cultural practices and attitudes in relation to them. It begins by stating the centrality of filmic representations to popular culture today, and moves on to analyse, first, the gendered and sexualised images as seen in a large proportion of music videos by way of a reading of Ayo Technology by 50 Cent and Justin Timberlake, a reading that uses Laura Mulvey's hugely influential feminist account of visual pleasure. Secondly, the chapter opens its frames of reference to include a consideration of the place of masculinity and ethnicity in popular cultural discourses, drawing on the work of Rey Chow and her concept of “coercive mimeticism”. Mulvey’s approach was constructed in film theory and Chow’s notion of coercive mimeticism was constructed in postcolonial studies, but this chapter shows their usefulness and applicability in any analysis of the visual aspects of popular culture.

1 Introduction: cinema as a cultural technology The connections between mass-mediated forms of popular culture, such as film or pop music, and matters that could be called political are often difficult to discern. This is so even though they can be said to be overwhelmingly “visual” matters. More precisely, they are matters of “representation”, and more specifically still, matters of representation, which are ineradicably wedded to particular types of media technology. To elucidate these propositions, this chapter first shows some of the ways in which cinema and film in general can be shown to be significant “cultural technologies”, and some of the ways in which they have complex effects on culture and identity. It then goes on to analyse two music videos in terms of what they could be said to show us about gender, sexuality, race and ethnicity in the contemporary media-saturated world. More and more of the world can be said to be media-saturated. And this is not a particularly new thing. It is rather that the types of media that are dominant in different times and places change. The 19th Century was arguably the century of the dominance of literature and of the British Empire. Literature was the dominant cultural form; Britain the dominant national force. To the extent that this is the case, it can also be said that the 20th Century can be regarded as the century of the United States of America and cinema. The USA emerged as the dominant (or hegemonic) cultural and economic force and presence, with the cinema as the dominant cultural form or technology. The complex processes involved are often distilled into the word “Hollywood”, a term which evokes the global ideological

154

Paul Bowman

hold of the USA and the channelling of that ideology through the film form and the cinematic apparatus. We might ask, what then is the 21st Century? Historians of all areas of life – from culture to economics, from military to market, from language to technology – have proposed that the 21st Century seems likely to be the century of China and the internet. We will have to defer a sustained consideration of the question of “China”, at this point – although the effects of the changing status of China is a fascinating question (Chow 1993; Park 2010). Instead, let’s begin our visual cultural analysis from the question: what kind of cultural form or technology is the internet? As I write these words in 2012 – over 20 years after the birth of the World Wide Web – the jury is, in fact, still out. Rather than trying to predict the future by trying to anticipate the cultural significance and development of the Internet, let us first consider the fate of the older 20th Century cultural technology of film in the context of the emergence of the Internet. Some cultural theorists have recently started talking of the transition from a “cinematic” age to a “post-cinematic” age. Such an idea will doubtless undergo further revision and elaboration and find a much more precise formulation as time goes on. But for now, Steven Shaviro’s Post-Cinematic Affect (Shaviro 2010) leads the way in this regard, by engaging with the effects of post-cinematic technologies on our experiences, orientations, emotions, feelings and lives. “Post-cinematic” technologies include all that is associated with the rise of interactivity, gaming, multimedia, and the proliferation of different internet platforms, as well as various new types of text, such as the music video, the new ways, modes and contexts of experiencing and consuming them and the effects they have on consciousness and perception. Shaviro considers the rise to dominance of these “post-cinematic” technologies in terms of a transformation of “affects”: mutations of experiential landscapes, emotional geographies, and perceptual and sensorial ecosystems. Using a famous term developed by the pioneer of cultural studies, Raymond Williams, (yet developing this term in ways informed more by the French philosopher Gilles Deleuze), Shaviro characterises this as an epochal transformation in dominant “structures of feeling”. In other words, the rise of the post-cinematic context has transformed our lives in ways related to our day-to-day and moment-to-moment experience. If such post-cinematic technologies have transformed structures of feeling, this is not the first time this has happened. For instance, we might consider the emergence of cinema itself. Rey Chow opens her 1995 book Primitive Passions (Chow 1995) with a reconsideration of the famous story of the turn towards a writing career of the monumental figure of Chinese literature, Lu Xun. Whilst he was a medical student at the very beginning of the 20th Century, Lu Xun watched with horror the cinema newsreels depicting atrocities committed in the Russo-Japanese War in Manchuria, including the executions of Chinese people. Lu Xun’s account of his response to these sights is complex and provocative. Indeed, his response

Sex and Race go Pop

155

to these first cinematic newsreels actually prefigures many of the dominant questions that have arisen in the face of cinema and other forms of viewing or “passive consumption” of mass media messages. For instance, Lu Xun asked, how could the witnesses to executions be so “passive”; how could audiences, of any kind, “do nothing”; and, more to the point, what could he himself do here and now to address such wrongs and escape the incapacity and passivity of being nothing but a viewer? These first problems, arising very early on in response to the first cinematic experiences of news reports, arguably set out many of the entrenched problems associated with the cinema, especially the problem of a sense of incapacity, castration, helplessness and passivity. Rey Chow’s analysis of Lu Xun’s emotional and intellectual response is far reaching and immensely important (Chow 1995). But the point I want to single out and draw attention to here is one that Chow emphasises about the significance of the fact that this new technology (the “cinematic” apparatus) precipitated a peculiar response from Lu Xun: in response to the media images, he turned away from his chosen career path of medicine and towards “literature”, believing that he could do more to improve the health of China by cultural or ideological intervention than by medical intervention. Central to Chow’s reading of this famous narrative is the following: Xun’s response to the new cultural technology (cinema) sends him into a relationship with an “older” technology (literature). From this, Chow proposes that it is possible to perceive the effects of “cinema” in (and on) Xun’s “literature”. From this point, one may broaden the perspective and begin to explore the significance of the emergence of “cinema” within subsequent developments in “literature”. Indeed, we might even be tempted to regard the majority of 20th Century literature as “postcinematic”, in that it is literature produced in a cultural world that the cinematic apparatus has intervened into – and has in fact dominated and transformed. In other words, this is the same as to say that after the birth of cinema, literature could never be the same again. In this sense, Lu Xun’s story is exemplary of the epochal mutation entailed in the shocks and jolts that are such a central part of life in modernity. The shock of the new attendant to the emergence of cinema had effects in untold ways in untold numbers and kinds of context – so much so that literature itself, in modernity, since the birth of cinema, might best be regarded as post-cinematic. Of course, this reverses the chronological periodization and emphasis that organizes Shaviro’s book. For, the idea of the “post-cinematic” that Shaviro uses in his book is one which points to all that new stuff that comes “after” cinema: computers, the Internet, the dynamism and interactivity of gaming or web 2.0; before that, cable and satellite TV, multiple (indeed myriad) television and radio channels, video, DVD, and all the rest. Nevertheless, as was implicit even in the very first theorisations of the word “postmodern” by such philosophers as Jean-François Lyotard, one of the key points about the postmodern is that everything you can say about the features of the “post” are actually already there, at

156

Paul Bowman

the outset, before the emergence of the period of “the post” as such (Lyotard 1984): so you can see elements of “postmodernity” at the origins (and throughout) the historical period called “modernity”. Postmodern thinkers such as Lyotard have long pointed out that the postmodern is implied in and active in the very emergence of the modern, right from the start. Rey Chow’s reading of Lu Xun’s affective response to these early experiences of (or encounters with) cinema demonstrates this explicitly. The new technology intervenes into, informs and thereby transforms the cultural landscape in ways which have knock on (albeit unpredictable) effects on other forms of cultural production and reception. To see this at a basic level, one need merely consider the extent to which so many literary best-sellers today have clearly been written with the production requirements of the standard Hollywood film form firmly in mind. This is but one register of the hegemony or dominance of the cinematic form and its “hegemonization” even of other cultural realms, such as literature. In any case, Shaviro argues that contemporary cultural conditions are such that the cinematic epoch is coming to a close. We are now at the end(s) of the cinematic. This is being registered “within” cinema, even though cinema remains strongly influential across all of its inheritors – all of the new technologies that are taking cinematic technologies forward in new directions. This is why the times are to be regarded as “post-cinematic” and not “anti” or “non-cinematic”. Cinema is on the wane while other technological forms are on the rise, just as the USA is on the wane in terms of its global hegemony, while China is on the rise in terms of economic and military strength. Thus, gaming, all things interactive, the music video, and other new arrivals on the audio-visual technological scene, all remain hugely informed by cinematography, but they move away from its technological limitations. Meanwhile, cinema attempts to incorporate the new technological advancements within itself: from DVD menus, extras, commentaries, outtakes, integrated marketing strategies with other realms (gaming, animation, toys and merchandise, spin off series) and other supplements, all the way to the inclusion of forms of interactivity that ultimately signal the demise of the older form. According to this perspective, films like Blade Runner (1982) or S1m0ne (2002) are not post-cinematic, whilst The Matrix (1999) or even the Korean film Old Boy (2003) are. The former are films “about” future technologies, whilst the latter “incorporate” future technologies into themselves, insofar as both films famously affect the styles of computer simulated choreographies in their most famous fight scenes, albeit in different ways: The Matrix employs the sharpness and precision of arcade game fights, whilst Old Boy incorporates the two-dimensional plane of older forms of computer game, but it counterbalances this with the inclusion of all of the scrappiness, imprecision, stumbling, gasping, moaning and, indeed, “messy brawling” that almost all action films exclude or repress (as exemplified by the ultra-precise choreography of The Matrix or The Bourne Identity trilogy (2002, 2004, 2007)).

Sex and Race go Pop

157

2 What’s the big deal about cinema? Or: what does cinema “do”? 2.1 Case study 1: activity, passivity, gender and sexuality Now, whether post-cinematic or classically cinematic, one important question is that of what the cinema “does”, or what the cinematic apparatus “does”: what effects this type of media has on people, what difference it makes to culture and society. As already indicated, one abiding argument made especially by Marxist thinkers is that the cinema makes us “passive” (Adorno and Horkheimer 1986). Some thinkers have been concerned that societies dominated primarily by the imposition of viewing relations, in which we’re all spectators, not only make us passive but actually make us acquiescent to or even enthusiastic for the worst kinds of political power. In the worst cases we can become enthusiastic for populist or fascist dictators who exploit the cinematic apparatus to make us think that they are charismatic, wise, authoritative, avuncular or loveable father figures or suchlike. (The film theorist Bazin argued that this manipulation of cinematic effects was the root of Stalin’s success in the Soviet Union.) At the very least, it is clear that we can have our heart-strings plucked by formulaic and clichéd devices of emotional or affective manipulation. The cover of Guy Debord’s classic, The Society of the Spectacle – one of the most influentially pessimistic Marxist texts about the effects of a media saturated society – has an image of rows of transfixed viewers sitting in a cinema, all facing towards the screen, all equally and identically enthralled. This has become one of the defining images of the positions which see a media saturated society as one which produces passivity, not only in audiences but effectively in everyone (Debord 1994). But this argument about passivity, docility or plasticity is not the end of the story about cinematic effects. One of the most influential analyses of the ideological effects of the cinematic apparatus was set out by Laura Mulvey in 1975. In an essay entitled “Visual Pleasure and Narrative Cinema” (Mulvey 1975), Mulvey uses psychoanalytic (not Marxist) theory to argue that “the unconscious of patriarchal society has structured film form”. Specifically, she argues, the regular repeated “image of the castrated woman [is used by Hollywood film] to give order and meaning to its world”. The men fight for the woman. They fight over the woman. The drama circles around the woman. So, “woman” is “tied to her place as bearer of meaning”, but she is not the “maker of meaning”. The woman is objectified. She is the motive force of the action, but she is essentially excluded from it. Specifically, argues Mulvey, in classic Hollywood, woman is the object of the gaze. She is there “to be looked at”. And this has significant implications, she argues. For it clarifies the extent to which filmic and other media images work to reinforce patriarchal, sexist or misogynistic ideologies.

158

Paul Bowman

Fig. 1: Woman “to-be-looked-at”, in the music video for “Ayo Technology” by Justin Timberlake and Fifty Cent”.1

A clear example of what Mulvey calls the classic effect of such a style can be seen in many pop music videos. Perhaps the best, to my knowledge, is that of the video for the song “Ayo Technology”, performed by Justin Timberlake and Fifty Cent. In this video, set in London, we see the male performers play ersatz James Bond characters. However, although they are slick-suited and sporting various forms of weaponry and technological gadgets, like Bond, they are not spying on villains, criminals, terrorists, or other stock kinds of antihero; rather they are spying on various female characters. Early shots in the video see them looking through night vision goggles and peering through the sights of large guns at women who are scantily clad and, in unusual locations, performing strangely incongruous erotic dances. An early moment sees Fifty Cent spying through a gunsight from a rooftop, looking down at a girl dressed only in her underwear and high heels as she gets into a sports car. Other scenes see Timberlake spying through binoculars from a chauffeurdriven car at a woman who is writhing in underwear, back-lit, in silhouette, in the window of a city flat or apartment. Later on in the video, Timberlake and Fifty Cent also appear to command futuristic sci-fi-like technologies, which can evidently act on women at a distance: Fifty Cent controls a virtual computer akin to the device

 http://music.yahoo.com/blogs/framed/archive/22.html.

Sex and Race go Pop

159

that Tom Cruise uses to see events with omniscience in Minority Report (2002). For Tom Cruise’s police officer character in the movie, the computer functions to help him master imponderable amounts of data and to perform calculations that attempt to predict the future. Fifty Cent, however, uses a similar-looking device in such a way as to make women become sexually aroused. Eventually, the video devolves down to the protagonists entering a private or luxury and exclusive lap-dancing club. This has all the hallmarks of a traditional upper-class London “gentleman’s club”, plus strippers. At this point, we see the male protagonists enjoy lap-dances whilst they themselves are blindfolded, as if inverting the original form of pleasure: at the start of the video, the men enjoy the scopophilia of looking and desiring. By the end, in contrast to the common understanding of the etiquette of lap-dancing establishments, in which customers can typically look but not touch, the males, in being blindfolded, are evidently now allowed to touch. Apparently, being blindfold will enhance this experience for them. (This scene is intercut with other scenes in which Timberlake also appears elsewhere: erotic scenes on the stairways, in the doorways and on the landings of a residential apartment building.) There is much that could be said about such a video, and many others like it. From the perspective offered by Laura Mulvey, the video first illustrates the desiring, objectifying, controlling aspirations of “the male gaze”. The male gaze is a voyeuristic, “scopophilic”, controlling gaze, she argues. It is a sexualised and sexualising gaze. It literally “targets” the female form and objectifies it. In this case, the gun-sight through which Fifty Cent spies is clearly a phallic image. The gun targets the woman, and the act of sighting the women is in itself an enactment of power. He could easily “shoot”. He could easily “take” her. He feels in control. The decision is in his hands. In a sense, therefore, he is already in control, by virtue of his viewing position. Similarly, the night-vision binoculars used by Timberlake confer upon the screen the green tint that has been associated (in film and television, in news, fact and fiction) not only with security cameras and military weapons, but also with the visual look of various much-publicised celebrities’ private/personal pornographic sex tapes. Many of these came to light at around the same time as this music video. The Paris Hilton sex tapes are perhaps the most famous (and perhaps most cruel) example of a rash of “private” sex videos to emerge around that time. Moreover, at these points, the camera angle changes to that of a hand-held private or amateur video, which emphasises the pornographic allusions. (This song, Ayo Technology, was reputedly first titled “Ayo Pornography”, but the decision was taken to rename it and change the lyrics accordingly because the word “pornography” would damage its ability to receive prime time TV and radio airplay, and hence maximise sales and revenues.)

 http://www.wired.com/underwire/2007/08/50-cent-and-jus/.

160

Paul Bowman

Fig. 2: Fifty Cent strangely blindfolded in a lap-dance club – as if the women who are employed to be looked at are not desirable to him, or need not be looked at; implying that “the most” desirable women are those who have to be spied on.2

So, the video performs a certain male fantasy of desire, control and sexualising objectification. At the same time as this, and in a way that is entirely consistent with Mulvey’s argument, the video depicts women as reciprocally (or perhaps even primarily) performing for a male gaze – and, crucially, even when a literal male gaze is not normally assumed to be present. The case of the female figure writhing at the window is the clearest example. For, via this scenario, the video seems to suggest, this is precisely the way that beautiful women will behave when they undress; that even when they believe themselves to be home alone and even when they are simply changing their clothes, they are still basically “asking for it”. The idea of identity as “performance” – that is, as something that is not simply natural or inevitable, but is rather a culturally obligatory performance – was perhaps most widely popularized in the arguments of the early work of Judith Butler. Butler’s arguments have since become widely accepted – to a greater or lesser extent – in cultural and media studies (as well as the humanities more widely). It is a perspective that has significant implications for what we might call the politics of media and culture – for part of the argument is that we learn how to “perform”, how to “be”, from what we “see”.

Sex and Race go Pop

161

Fig. 3: The male gaze is controlling, patriarchal, objectifying and “targeting”.3

There are many implications and ramifications here. The video reiterates a version of masculinity as gaze and femininity as “to-be-looked-at-ness” – or male as controlling and female as controlled. With the lyrical repetition of “oh, she wants it, oh, I’m gonna give it to her”, and similar sentiments, it repeats the misogynistic perspective that women are “asking for it”. But, the question then becomes one of “our agency” in front of the text. Or in other words: once we see that the text is patriarchal, misogynistic and sexist, the question is, does that mean that the viewer will be or become patriarchal, misogynistic and sexist? Is the viewer “passive”? Does the filmic text manipulate us the way that the virtual technology used by Fifty Cent in the video manipulates the woman? I am reluctant to propose that the watching of videos, films, TV or other media, is going to generate a case of “monkey see, monkey do”. In other words, I am not proposing that such videos in and of themselves cause or deepen misogyny or patriarchy. However, this is certainly a possibility that needs to be entertained. For, according to the implications of Mulvey’s approach, what such texts do is that they “normalise” these patriarchal viewing relations: we – male and female – become “used to” (habituated, acclimatised) viewing the world this way, to apprehending the female as body, as object, as sex, and to regarding the male as power-

 http://music.yahoo.com/blogs/framed/archive/22.html.

162

Paul Bowman

ful, controlling, gazing; we get used to regarding the woman as “wanting” to masquerade and “perform” her femininity as her sexuality for the male gaze, etc. Another clear example of this can be seen in many Beyoncé songs and videos, in which apparently “feminist” sentiments are uttered – declarations about “independence” and “strength”, for instance, which might at first glance seem to be feminist. But the problem is that these sentiments emerge within songs which are otherwise entirely organised by the performance of desire for a man. One song, which claims to celebrate strong independent femininity, nevertheless repeats the phrase “if you like[d] it then you should have put a ring on it”. In other words, the song is organised by a kind of bitterness – a bitterness about and a desire for male commitment. Such resentment hardly seems to be a feminist sentiment, or even the sentiment of a truly “free” and “independent” person. Rather, in this performance, we see a combination of contradictory sentiments which show that the celebration of independence is in fact a grudging resentful response to the disappointment elicited when the (absent) male refused to “put a ring on it” – i.e., to commit, to get engaged or married. To this extent, all the words about “feminism” or “strength” in the song are a mere replaying of the most patriarchal of assumptions or stereotypes about males and females: that men will not commit, and that all women really want is a man. Many other of Beyoncé’s “feminist” or “post-feminist” songs replay this logic: the character who frantically performs her desire through myriad costume changes and insanely energetic erotic performances in front of an entirely stationary and unresponsive “cool” male character; or the woman who ditches her partner because he is not up to scratch, singing about how she will be “over you in a minute” and that she will have “another [one of] you in a minute” – all of which confirm that ultimately what is desired is standard patriarchal heterosexual domesticity. But just because these popular cultural media texts are evidently patriarchal, sexist and “heteronormative” (in that they reiterate the message that heterosexual norms are both the standard and the objective), does this mean “either” that we are passive before such texts “or” that we ourselves will “become” patriarchal, sexist and heteronormative ourselves by virtue of our exposure to them? There are many approaches to culture and identity which propose that we may “become” what we are exposed to. However, many of these are simplistic (or indeed what is termed “essentialist”), in that they propose a kind of “monkey see, monkey do” relationship between what (certain types of) people are exposed to and what they will do – especially if they “enjoy” what they see. And, it is important to note, we may well enjoy Beyoncé’s or Timberlake’s videos, and for any number of reasons: we may find the beat irresistible; we may be enthralled by the faces and bodies in the videos; or the sheer complexity and rapidity of the flashing and changing scenes in a music video may be compelling. But does our enjoyment make us a prisoner or a puppet? The situation is surely more complex.

Sex and Race go Pop

163

2.2 Segue: the commodification of sex and ethnicity bell hooks once provided an interesting account of the problems of rap, hip hop music and black youth identity in the 1980s and early 90s. Hip hop was arguably one of the main contexts in which black youth culture gained anything like cultural visibility and prominence. During the 1990s and up to the early 2000s, hip hop and rap could actually be said to have utterly transformed mainstream popular music and global popular culture in myriad ways. But there was a problem at the heart of it. According to hooks, the black rapper “character” and the hip hop music video “style” became entangled with a damaging image of blackness. The logic, according to hooks, is that of a vicious circle. The circle is this: first, hip hop gains prominence as a nominally black musical genre. It is associated with poor black youth and also with anger and protest. It is organised by a connection with “the street”. This all gives hip hop and rap a strong identity. But it also becomes a cliché, a stereotype, a cheap commodity. It becomes another way of defining the black: angry, dangerous, poor, politicized, apparently aggressive, often violent. Over time, sexuality comes to the fore too. Black rap and hip hop videos (along with white spin offs and related enterprises) increasingly involve the tried and tested marketing device of always including sexy female “eye candy”. So what becomes produced is a genre which can all too easily act as a stand in for black culture per se. This genre – and this interpretation of what black culture “is” – becomes reduced to violence, guns, money and girls. Now, what hooks proposes is not that “people” (viewers, listeners) simply “change” and “become” more violent or more sexist. Rather, it is that certain generic and formulaic rules, certain sorts of lyrics, types of imagery and styles of video (involving guns, girls and money) gain a dominance and act as a kind of stranglehold. To be a success, artists perceive that the easy – or the only – route is to produce texts and performances that now conform to this new norm. In other words, what Adorno and Horkheimer called “the culture industry” produces cultural and media effects, effects which play themselves out in people’s daily lives, fantasies and desires. Nevertheless, despite the importance of bell hooks’ arguments and insights, the media are not simply “external” to us. The audiovisual media texts that permeate everyday popular culture are not simply fictional or fake, with no relation to our hearts and minds, our actions and inactions. As all of the foregoing discussions have implied, audiovisual media texts are potentially hugely important visà-vis individual and collective identity-formation. To see this, let us turn to another concise media text – a text which, so to speak, dramatizes in a hyperbolic and comic way the effects on identity of attractive media images. What the text lacks in seriousness, it more than makes up for in providing a thought-provoking scenario about identity.

164

Paul Bowman

2.3 Case study 2: ethnicity, sexuality, identity and coercive mimeticism The text I want to focus on here is the music video for the song Pretty Fly for a White Guy (Offspring 1998). It is a comic and frivolous text – both in the lyrical content and in the video. But what it explores and dramatizes is both recognizable and serious, and raises some fundamental questions about identity, desire and phantasy. The song is structured by the refrain “all the girlies say I’m pretty fly for a white guy”. In the video, these words are uttered by a white “wannabe”. The lyrics narrate the tale – or rather, the situation – the plight – of an apparently affluent, suburban white American teenager, a teenager who nevertheless fantasises about and evidently fetishises edgy nonwhite ethnicity. In the video, we see several of the scenarios which define his phantasy. Whether black African-American or Latino, our eponymous “white guy” wannabe wants-to-be “that”: he identifies “with”, he fantasizes “as” that. He wants to be one of “them”. Unfortunately, what is absolutely clear here is that the one thing he is not is “pretty fly”. Rather, he is presented as ridiculous, a fool, utterly lacking in self-awareness or self-knowledge – living, as the lyrics put it, “in denial”. So, the song is all about getting it wrong, wanting the impossible, and denying that impossibility. The reason for wanting the impossible boils down to a phantasy.5 This is dramatized in the call-and-response (and commentary) that opens and permeates the song. The song opens with it: a female chorus chant “Give it to me baby”. In the video, our hapless hero responds in the affirmative. This call and response is repeated. It is a chant of female call and male response that dramatizes what is evidently a male sexual phantasy about specifically ethnic female desire. It is followed by the gravelly-voiced claim: “And all the girlies say I’m pretty fly for a white guy”, whereupon the song “proper” begins. This, it soon becomes clear, is the structuring fantasy (or “phantasy”) of our misrecognizing, fantasising white guy. This is what he wants. This is what he thinks it would be like if only he were the ethnic he wants to be. This is what he wants to see and hear. He imagines the call. He “performs” a response. So, in the video representation, the song runs: repeated female chant (“Give it to me baby”); he answers (“uh huh, uh huh”). This is followed by the voice of his phantasy, which asserts his conviction that “all the girlies say I’m pretty fly for a white guy”.

 http://uk.viva.tv/musik/artists/the-offspring-209023/videos.  I use the “ph” spelling here to highlight the psychoanalytic/cultural theoretical specificity of this usage. The “ph” spelling is more likely to be used in British-English rather than AmericanEnglish academic contexts in any case. However, elsewhere I use the “f” spelling, usually where the “ph” spelling would appear awkward, but also where psychoanalytic specificity is not necessary.

Sex and Race go Pop

165

Fig. 4: Opening moments of the video for “Pretty Fly for a White Guy”.4

After this intro, we are “counted-in” in incorrect Spanish (“Uno, dos, tres, cuatro, cinco, cinco, seis”). If we had been in any doubt up until now, this “miscount” – this moment of getting it just a bit but fundamentally wrong – not quite getting the Spanish right – clarifies things for us. This is a joke. This is about misrecognition, getting it wrong. Moreover, the girls in the video are clearly nonexistent fantasy constructions: there never were girls thronging around him on the way to his car, by the side of the road, or covered in glittering paint by the pool. They are entirely his phantasy. An initial assessment of the song, taking into account any mirth it might produce – and the extent to which we might share, understand, or “get” the joke – suggests that this popular cultural text is saying something quite precise about identity, about “cultural” identity, “identity performativity” and ethnicity. And this appears to be something quite different from what is widely supposed to be held by many thinkers, from Judith Butler to Homi Bhabha and beyond. For, the text is saying at least one, or perhaps all, of the following: 1.) that a white ethnic cannot – or should not – try to “perform” another ethnic identity; 2.) that trying to be other than white for the white is ridiculous; 3.) that trying to do or to be so is premised on “not getting it”, on “denial”; 4.) that white ethnicity is not like other ethnicities – not porous, not dilutable, not “hybridisable” or “fragile;” and 5.) that the only compensation for the sadness and disappointment that this might cause for our wannabe is the contemporary Confessional: “At least you know you

166

Paul Bowman

can always go on Ricki Lake”, say the lyrics. Indeed, don’t worry, be happy, add The Offspring: “the world needs wannabes”. So, “hey, hey, do the brand new thing”. The song is very clear on this. After staging the fantasy scenario, after being miscounted-in, the narrative voice begins to tell us all about it. The lyrics begin by addressing us in terms of a shared lot, a common problem that we all recognise: “You know it’s kinda hard just to get along today”. “We all know this”, right? Furthermore: “Our subject isn’t cool, but he thinks it anyway”. Isn’t this a familiar story? How many of us are guilty of it ourselves? We may recall Lacan’s contention that, in love, “You never look at me from the place from which I see you”. Conversely, “what I look at is never what I wish to see” (Lacan quoted in Chow 1998: 81). Moreover, as Rey Chow points out, this “dialectic of eye and gaze” need not be “literally” intersubjective; a man may fall “in love, not with a woman or even with another man, not with a human being at all but with a thing, a reified form of his own fantasy” (1998: 78). As The Offspring put it: “He may not have a clue, and he may not have style / But everything he lacks, well he makes up in denial”. Is this his problem: “denial”? “Denial” is surely the most abused, misused, bandied-about psychobabblistic term ever. Everyone, it seems, risks living in denial. Overcoming denial is indeed an abiding concern of an enormous range of popular cultural texts and discourses. But, if denial is deemed to be the problem, what is deemed to be the solution? The popular answer is: come to terms, recognize, accept. But how? By talking about yourself; by “confessing”. Go on Ricki Lake. Even if you are “fake”, you “can” have a moment of real-world, recognized, “authentic” success (“fame”), by coming clean, by confessing, publicly: the only authentic redemption in a world which thrives on the production of fakes and wannabes, say The Offspring. If we can laugh at all of this it is also because we can recognize all of this. According to the implications of the argument of Michel Foucault in The History of Sexuality, Volume 1 (Foucault 1978), this familiarity and recognisability comes from the fact that The Offspring song plays with the material thrown up by and circulating in and as a discursive constellation – a very old discursive constellation, says Foucault, which came together in the 18th Century. In this discursive formation, the terms ethnicity, identity, authenticity and autobiography – or confession – encounter each other in an overdetermined chiasmus. In it, whenever issues of identity and ethnicity arise as a (self-reflexive, “personal”) problem, this discursive constellation proposes that the route out is via the self-reflexive sidedoor of autobiographical (self) confession. There is more to this than observing that engaging with ethnicity requires an engagement with one’s own identity, one that ought to lead into a searching selfinterrogation and ideally a deconstruction of questions of authenticity and autobiography – although this is certainly a part of it. For the Foucauldian point is that precisely such discourses of the self, especially in terms of the brands of self-

Sex and Race go Pop

167

referentiality that nowadays feed chat shows like Ricki Lake, can be seen to have emerged decisively in modernity. And they emerged with an attending argument about self-referentiality’s subversive relation to power and its emancipatory relation to truth. That is, it refers us to the implications of Foucault’s argument about what he called “the repressive hypothesis” – namely, that almost irresistible belief that power tries to silence us and demands our silence (Foucault 1978: 18; Chow 2002: 114). As Foucault argued, however, almost the exact opposite is the case. Or rather, even if there are places where power demands silence or discipline, these are more than matched by an exponential explosion and proliferation of discourses – in this case, discourses about the self. These discourses include arguments about self-referentiality’s subversive relation to power and its emancipatory relation to truth, which relates to the Enlightenment idea that an introspective turn to the self is emancipatory: the ingrained idea (whose prehistory is the Catholic Confessional, and whose contemporary ministers Foucault finds in the psychiatrist and psychoanalyst) that seeking to speak the truth of oneself is the best method of getting at our essential truth “and” the best way to resist power. Similarly, modern literary self-referentiality emerged with an attending discourse of resistance – a discourse which regarded literature “as such” as resistance to the instrumentalization of technical and bureaucratic language, first and foremost. And, by the same token, self-referentiality emerged as an apparently ideal solution to the knotty problem of representing others. For, how do you represent others truthfully, adequately, ethically? The answer given here is: “you” don’t. “They” should represent themselves. Here, the self-reflexivity of self-referentiality is regarded not as apartheid but as the way to bypass the problems of representing others – by throwing the option open for everyone to speak the truth of themselves. However, in Foucault’s phrase: “the “Enlightenment”, which discovered the liberties, also invented the disciplines” (Foucault 1977: 222; see also Chow 2002: 113). In other words, the desire to refer to the self, to discuss the self, to produce the self discursively, the impulse to indulge in autobiography and confession, can be regarded as a consequence of disciplinarity. Psychiatry demands that we reveal our selves. As does psychoanalysis, as do ethnographic focus groups, as do corporate marketing focus groups, not to mention the Confessional, the criminologist and Ricki Lake. And so on. Autobiography and confession are only resistance if power truly tries to repress the production of discourse. Which it doesn’t – at least not everywhere. The point is, autobiography and confession are genealogically wedded – if not welded – to recognisable disciplinary protocols and – perhaps most significantly – proceed according to the terms of recognisable metanarratives. Thus, says Chow: When minority individuals think that, by referring to themselves, they are liberating themselves from the powers that subordinate them, they may actually be allowing such powers to work in the most intimate fashion – from within their hearts and souls, in a kind of voluntary

168

Paul Bowman

surrender that is, in the end, fully complicit with the guilty verdict that has been declared on them socially long before they speak. (Chow 2002: 115)

Of course, in thinking about postcoloniality, ethnicity, social semiotics and cultural politics, it is very difficult “not” to think about oneself. Indeed, even in full knowledge of Foucault, there remains something of a complex “imperative” to do so, even (perhaps especially) if, like me, one does not have a blatantly postcolonial ethnicity in the classic sense – even if, that is, like me, one has an entirely hegemonic socio-cultural identity: an ethnicity without ethnicity, as it were; the “hegemon” of a hegemony; that is, the “norm”. For, surely one must factor oneself into whatever picture one is painting, in terms of the “institutional investments that shape [our own] enunciation” (Chow 1993: 2). Indeed, suggests Chow: the most difficult questions surrounding the demarcation of boundaries implied by ‘seeing’ have to do not with positivistic taxonomic juxtapositions of self-contained identities and traditions in the manner of ‘this is you’ and ‘that is us’, but rather, who is ‘seeing’ whom, and how? What are the power relationships between the ‘subject’ and ‘object’ of the culturally overdetermined ‘eye’? (Chow 1991: 3)

Might acknowledging as much make “me” pretty fly for a white guy? As thinkers like Robyn Wiegman and Rey Chow have pointed out: the white subject who nowadays endeavors to compensate for the historical ‘wrong’ of being white by taking on politically correct agendas (such as desegregation) and thus distancing himself from his own ethnic history, is seldom if ever accused of being disloyal to his culture; more often than not, he tends to be applauded for being politically progressive and morally superior. (Chow 2002: 116–117)

Chow proposes that we compare and contrast this with nonwhite ethnic subjects – or rather, in her discussion, with nonwhite ethnic critics, scholars and academics. These subjects, she argues, are pressured directly and indirectly to behave “properly” – to act and think and “be” the way “they” are supposed to act and think and be, “as” nonwhite ethnic academic subjects. If they forget their ethnicity, or their nationalistically or geographically – and hence essentialistically and positivistically – defined “cultures” and “heritages”, such subjects are deemed to be sellouts, traitors – “inauthentic”. But, says Chow, if such an ethnic scholar “should […] choose, instead, to mimic and perform her own ethnicity” – that is, to respond or perform in terms of the implicit and explicit hailing or interpellation of her as an ethnic subject as such, by playing along with the “mimetic enactment of the automatized stereotypes that are dangled out there in public, hailing the ethnic” (2002: 110) – “she would still be considered a turncoat, this time because she is too eagerly pandering to the orientalist tastes of Westerners” (2002: 117), and this time most likely by other nonwhite ethnic subjects. Thus, the ethnic subject seems damned if she does and damned if she doesn’t “be” an ethnic subject. Of course, this damnation comes from different parties,

Sex and Race go Pop

169

and with different implications. But, in any eventuality, Chow’s point is that, in sharp contradistinction, “however far he chooses to go, a white person sympathetic to or identifying with a nonwhite culture does not in any way become less white” (2002: 117). Indeed, she claims: When it comes to nonwhite peoples doing exactly the same thing […] – that is, becoming sympathetic to or identified with cultures other than their own – we get a drastically different kind of evaluation. If an ethnic critic should simply ignore her own ethnic history and become immersed in white culture, she would, needless to say, be deemed a turncoat (one that forgets her origins). (Chow 2002: 117)

It is important to be aware that it is not just whites who pressure the nonwhite ethnic to conform. Chow gives many examples of the ways that scholars of Chinese culture and literature, for instance, relentlessly produce an essentialist notion of China, which is used to berate modern diasporic Chinese (and their cultural productions). This essentialism takes the form of evoking an essence “that none can live up to”, precisely because “they are alive” and as such contaminated, diluted, tainted or corrupted by non-Chinese influences. At least one side of this key difference between the white and the nonwhite is dramatized in the Offspring song. Whilst postcolonial critics often recount cases in which nonwhite ethnic subjects are pressured directly and indirectly to start to behave “properly” – to act and think and be the way “they” are supposed to act and think and be as nonwhite ethnic subjects – in other words, to be both “interpellated”, in Althusser’s sense, and “disciplined“, in Foucault’s sense – I think that the very intelligibility of the Offspring song and its fairly unequivocal condemnation of the white-wannabe-nonwhite suggests that the white guy who shows too much interest in nonwhite culture, rather than being “applauded for being politically progressive and morally superior”, can quite easily and will quite frequently be deemed not only “disloyal to his culture” but ridiculous. “Yet, he remains no less white”. In fact, it seems, “he can become no less white”. But he is still a traitor. Thus, corroborating Chow’s thesis, white ethnicity is here presented as absolutely immovable and essentially (or wholly/holy) incorruptible. All of this, Rey Chow calls “coercive mimeticism” (2002: 107). Coercive mimeticism designates the way in which the interpellating, disciplining forces of all different kinds of discourses and institutions “call” us into place, “tell” us our place, and work to “keep” us in our place. As Chow writes of the ethnic academic subject: Her only viable option seems to be that of reproducing a specific version of herself – and her ethnicity – that has, somehow, already been endorsed and approved by the specialists of her culture (Chow 2002: 117)

Accordingly, coercive mimeticism ultimately works as “an institutionalized mechanism of knowledge production and dissemination, the point of which is to manage a non-Western ethnicity through the disciplinary promulgation of the supposed difference”. (2002: 117)

170

Paul Bowman

As we see through the Offspring song, this disciplinary mechanism extends far beyond the disciplines proper, far beyond the university. In Chow’s words: unlike the white man, who does not have to worry about impairing his identity even when he is touched by a foreign culture, the ethnic must work hard to keep hers; yet the harder she works at being bona fide, the more of an inferior representation she will appear to be. (Chow 2002: 124)

Reciprocally, we might add, the harder the white guy tries to be nonwhite, the “more” white he will appear. In trying to be other – so say the interpellating voices, tropes, discourses and institutions – he is of course, just being “silly”. Whether this means that the white attempt to be like the other is silly, or that the other is silly – or both – is debatable. What is not debatable is that in all cases “authenticity” ultimately translates as a hypothetical state of non-self-conscious and non-constructed essential “being”. The fact that this is an essentialism that is essentially impossible does not mean that it does not “happen;” rather it means that “ethnicity” becomes an infinitely supple rhetorical tool. It is available (to anyone and everyone) as a way to disparage both anyone who is not being the way they are supposed to be and anyone who “is” being the way they “are” supposed to be. As Chow explains, “ethnicity can be used as a means of attacking others, of shaming, belittling, and reducing them to the condition of inauthenticity, disloyalty, and deceit” (2002: 124). Ironically, such attacks are “frequently issued by ethnics themselves against fellow ethnics, that is, the people who are closest to, who are most like them ethnically in this fraught trajectory of coercive mimeticism” (2002: 124). What this means is that the most contempt, from all quarters, will always be reserved for he or she who does not stay in their place, play their proper ethnicity. All too often, criticism is levelled “individually”, as if it is a “personal” issue, “despite the fact that this historically charged, alienating situation is a collectively experienced one” (2002: 124). Such is the disciplining, streaming, classifying force of coercive mimeticism. Such are the “uses of ethnicity”. In the words of Etienne Balibar: “the problem is to keep ‘in their place’, from generation to generation, those who have no fixed place; and for this, it is necessary that they have a genealogy” (Balibar quoted in Chow 2002: 95). As such, even the work of sensitive, caring, deeply invested specialists, and expert ethnic scholars – even ethnic experts in ethnicity – themselves can function to reinforce ethnicized hierarchies, structured in dominance, simply by insisting on producing their field or object in its difference. What is at stake here is the surely significant fact that even the honest and principled or declared aim of studying others otherwise can actually amount to a positive working for the very forces one avowedly opposes or seeks to resist.

Sex and Race go Pop

171

3 Conclusion: Visible space and/as power The two case studies discussed above may seem very different. One focused on gender performativity and the force of “the male gaze”. The other focused on issues around the performance of ethnicity. But both are unified in what they reveal about the visual field’s relation to power. If these case studies indicate anything, it is the extent to which the space of visual media is steeped in power relations. There are codes of propriety in gender and ethnic performance – and even where one might least expect it, such as in the supposedly irreverent (con)texts of popular music videos. In other words, although our case studies took us far and wide – from Hollywood to the Internet, from 19th Century British colonialism and imperialism to 20th Century Chinese literature, from the semi-pornographic codes of a contemporary pop video to the irony of some late 20th Century rock/pop, via complex poststructuralist theory – each reading pointed to the conclusion that the visual and performative space of popular culture is saturated with power, power that cajoles and coerces us to identify with this and to disidentify with that, and to “perform” ourselves according to the dictates of dominant cultural discourses about gender and ethnicity.

References Adorno, Theodor W. and Max Horkheimer. 1986. Dialectic of Enlightenment. London: Verso. Chow, Rey. 1991. Woman and Chinese Modernity: The politics of reading between West and East. Minneapolis:University of Minnesota Press. Chow, Rey. 1993. Writing diaspora: Tactics of intervention in contemporary cultural studies. Bloomington: Indiana University Press. Chow, Rey. 1995. Primitive Passions: Visuality, Sexuality, Ethnography, and Contemporary Chinese Cinema. New York: Columbia University Press. Chow, Rey. 2002. The Protestant Ethnic and the Spirit of Capitalism. New York: Columbia University Press. Debord, Guy. 1994. The Society of the Spectacle. New York: Zone Books. Foucault, Michel. 1977. Discipline and Punish: The Birth of the Prison. New York: Pantheon Books. Foucault, Michel. 1978. The History of Sexuality, Volume 1. London: Penguin. Lyotard, Jean-François. 1984. The Postmodern Condition: A Report on Knowledge. Minneapolis: University of Minnesota Press. Mulvey, Laura. 1975. Visual Pleasure and Narrative Cinema. Screen 16(3): 6–18. Park, Jane Chi Hyun. 2010. Yellow Future: Oriental Style in Hollywood Cinema. Minneapolis: University of Minnesota Press. Shaviro, Steven. 2010. Post-Cinematic Affect. Washington: Zero Books.

Inna Semetsky

7 The visual semiotics of Tarot images: A sociocultural perspective Abstract: This chapter addresses the visual semiotics of a specific medium represented by Tarot images and investigates their communicative and signifying potential. The chapter argues that Tarot is not just a cultural game but a system of signs grounded in a complementary relation between word and image. Using such sources as Carl Jung’s archetypal psychology, Charles Sanders Peirce’s triadic semiotics and Yuri Lotman’s approach to cultural memory, this chapter demonstrates that the meanings implicit in the Tarot pictures accord with the unorthodox grammar of the symbolic language of the collective unconscious. Importantly, the chapter considers the implications of Tarot pictorial semiotics at the level of present-day social reality and specifically focuses on three images of the Major Arcana, namely: the Devil, the Tower, and the Star. The chapter concludes by presenting an impending Fifth Age of Understanding wherein the integration of visual communication takes place leading to humankind's intelligent and ethical actions.

1 Introduction This chapter addresses the visual semiotics of a specific medium represented by Tarot images and investigates their communicative and signifying potential. Tarot pictures have been historically excluded from “high” visual art forms such as paintings and relegated to the very “low” end of popular culture (Auger 2004) despite the fact that the Bibliotheque Nationale in Paris holds a collection of exquisitely painted Tarot cards documented in the French Court ledger as dating back to 1392. The collection located at the Pierpont Morgan Library in New York contains thirty-five picturesque cards from a full deck of seventy-eight, whose origin goes back to the middle of 15th century. Semiotics – a study of signs, both verbal and non-verbal, and their signification or meanings – considers images and pictures as belonging to the category of signs (Posner 1989; Sonesson 1989): “pictures have a continuous structure [that] induces the reader to […] read the picture as if it were a written text” (Posner 1989: 276). Tarot cards contain meaningful attributes in the format of pictorial or “graphic symbols [that remain] a semiotically still largely unexplored field of research” (Nöth 1995: 219). Yet pictures are polysemous, that is, they are open to a variety of meanings depending on broad contexts that may adopt a specific cultural code. According to Roland Barthes’ now-classic example of the photo of the bald eagle, a physical image serves as a signifier, while the concept per se of

174

Inna Semetsky

the bald eagle is the signified. The photographic image of an eagle as such, that is, representing the level of denotation, is a sign or a signifier. But importantly it is also a signified at the higher-order level of possible connotations; as such, the signified is polysemic and may connote a plurality of meanings. Functioning as a potential signified, the sign is characterized by a surplus of signification: it may mean either patriotism, or be a symbol of the American flag, or represent endangered species, or whatever else might be associated with it in a given cultural code, thereby producing a sign called by Barthes an associative total. Despite the form remaining the same, the conceptual content – or a sign’s meaning – is polyvalent. An analogous associative process is a backbone behind reading and interpreting the Tarot cards. Famous philosopher of language Sir Michael Dummett, a pioneer of the so called linguistic turn in philosophy, devoted several monumental volumes to the history and mystery of Tarot; however he addressed it from a rather limited perspective of a merely cultural game (e.g. Dummett 1980). Conversely, French philosopher Antoine Faivre presented “Tarot [as] a specific art [… and] a subject of extensive literature, both scholarly and popular [that] increasingly suffuses our culture [and] through a hermeneutic of situations and characters […] opens out upon a gnosis” (1994: 96) as deep inner knowledge, which is however expressed not in words but in images. This chapter adopts the latter perspective, which has been extensively researched and summarized in my 2011 book titled Re-Symbolization of the Self: Human Development and Tarot Hermeneutic. The relationship between word and image remains historically, philosophically, and ideologically troubled. Shlain (1989) contrasts the “linear, sequential, reductionist, and abstract thinking” (1998: 1; italics in original) pertaining to the verbal mode with “holistic, simultaneous, synthetic, and concrete” (Shlain 1998: 4; italics in original) medium of images. Importantly, images “approximate reality […] The brain simultaneously perceives all parts of the whole integrating the parts synthetically into a gestalt. The majority of images are perceived in an all-at-once manner” (Shlain 1998: 4; italics in original). The complementarity between word and image as a feature of semiotics is important in the hermeneutic process of Tarot readings (Semetsky 2011). During their interpretations, the meanings of pictures are narrated and articulated; however they are not exhausted by verbal expressions. The words are abstract – but the feelings and emotions that they elicit are concrete and particular and determine our actual actions at the level of cultural practices. These feelings may betray words. While human consciousness expresses itself in propositional thought and verbal language as a prerogative of Cartesian Cogito, feelings and emotions may be buried deep in the unconscious, whose mode of expression is, according to psychologist Carl Gustav Jung, universal or archetypal; thus exceeding personal unconscious posited by Freud as simply repressed and reducible to Oedipal drama. Jungian archetypes “populate”, metaphorically speaking, the field of “collective

The visual semiotics of Tarot images

175

unconscious” and may appear in our dreams in the form of multiple archetypal images. For Jung, the effects of the archetypes “can only be explained by assuming them to be deposits of the constantly repeated experiences of humanity” (Jung CW 7, 109). While Jung did not specifically focus on Tarot, he noticed that despite their “obscure origin […] the set of pictures on Tarot cards were distantly descended from the archetypes of transformation” (Jung in Gad 1994: 179). Sally Nichols, who was Jung’s student in Zurich, has explicitly connected archetypes with the Tarot images in her book Jung and Tarot: an Archetypal Journey (Nichols 1980); and Jung’s biographer Laurens van der Post, in his introduction to Nichols’ book, pointed out her “profound investigation of Tarot, and her illuminated exegesis of its pattern as an authentic attempt at enlargement of possibilities of human perceptions” (in Nichols 1980: xv). Jungian psychologist Irene Gad (1994) connected Tarot cards with the stages of human development along the Jungian process of individuation towards becoming authentic selves and considered their archetypal images “to be […] trigger symbols, appearing and disappearing throughout history in times of transition and need” (1994: xxxiv).

2 The archetypal images of Tarot A Tarot deck comprises 78 images, 22 Major and 56 Minor, called Arcana. The meaning of the word Arcana derives from Latin arca as a chest; arcere as a verb means to shut or to close; symbolically, Arcanum (singular) is a tightly shut treasure chest holding a secret – its implicit meaning. According to Jung, the universal meanings of common human experiences are hidden in the collective unconscious or “objective psyche” shared at a deeper level by all members of humankind and manifest through archetypal, symbolic and latent, images (Jung 1959). The collective unconscious acts as a symbolic “home” for archetypes that transcend cultural or temporal barriers. The archetypes are structural elements of the collective psyche and “possess a certain autonomy and specific energy which enables them to attract, out of the conscious mind, those contents which are better suited to themselves” (Jung CW 5, 232). Jung saw archetypes as skeletal patterns, filled in with imagery and motifs that are “mediated to us by the unconscious” (CW 8, 417), the variable contents of which form different archetypal images. He was adamant that the “symbolic process is an experience in images and of images” (Jung CW 8i, 82). He insisted that “there are things in the psyche which I do not produce, but which produce themselves and have their own life … there is something in me that can say things that I do not know and do not intend” (Jung 1963: 183), that is, which act at the unconscious level beyond one’s voluntary control or conscious will. Contemporary post-Jungians consider the archetypes to be both the structuring patterns of the psyche and the dynamical units of information (cf. Semetsky 2008a, 2008b) implicit in the contents of the collective unconscious.

176

Inna Semetsky

The archetypal images reflected in the Tarot Arcana are the vehicles for/of information embedded in the collective unconscious, and the unconscious is capable of spontaneously producing images “irrespective of wishes and fears of the conscious mind” (Jung CW 11, 745). Thus Tarot pictures lay down a “route to the unconscious. This alternative route uses some of the materials, shapes, signs, and symbols used by artists and our dreams […] The major arcana […] are visual aids to the unconscious. They are vivid shorthand portraits” (Hederman 2003: 27). The pictorial images comprising the Major and Minor Arcana in a Tarot deck drawn by Pamela Colman Smith are shown below in Figure 1. In contrast to a number of popular publications that usually reduce the wealth of potential meanings embodied in Tarot images to their description by a set of keywords, my research (Semetsky 2011) demonstrates that there cannot be a forever-fixed meaning attached to an image; rather meanings are contextualized depending on a particular situation, as well as inferred from specific positions occupied by this or that picture in a typical Tarot layout. While these positions traditionally describe common semantic categories, their “contents” embodied in the imagery do vary. Hence, while the meanings of the Tarot images are codified, their codification is never a fixed given but represents a dynamic evolving “process” that includes a pragmatic dimension. It is our practical experiential “usage [that] renders [a potential meaning] more precise and extends its convention” (Guiraud 1975: 25). Importantly, there is: a deep affinity between communication […] and perception. […] Psychoanalysis – particularly Lacan’s school – considers the manifestations of the unconscious as a mode of communication and a language. Parapsychology, too, postulates the notion of subliminal messages which are not conscious. These notions have been taken up by literary criticism, the study of myth, the psychosociology of behavior […] under the heading of ‘depth psychology’, and semiology must take this into account. […] Codification […] is a process [and] depending on each particular case, signs are more or less motivated, and sign systems more or less structured. (Guiraud 1975: 22–25)

Jacques Lacan was correct when he said that the unconscious is structured as a language. Even if this language is subtle and non-verbal, it has a communicative potential. Exceeding a solely linguistic representation, it expresses itself in the form of images as iconic information. The reading and interpretation of images makes the implicit meanings explicit, as if creating them anew. Importantly, the collective unconscious encompasses future possibilities, and “[a] purposively interpreted [image], seems like a symbol, seeking to characterize a definite goal with the help of the material at hand, or trace out a line of future psychological development” (Jung CW 6, 720), that is to perform a prospective, “prognostic” function in addition to the one symptomatic, or “diagnostic” as related to the experience of the present. The meanings implicit in the Tarot pictures are not arbitrary but accord with the unorthodox grammar of the symbolic language of the unconscious (Semetsky

The visual semiotics of Tarot images

177

1

Fig. 1: Images of Major and Minor Arcana.

 All illustrations, unless otherwise specified, are from the Rider-Waite Tarot Deck, also known as the Rider Tarot and the Waite Tarot. Reproduced by permission of US Games Systems Inc., Stamford, CT 06902, USA. Copyright 1971 by US Games Systems, Inc. Further reproduction prohibited.

178

Inna Semetsky

2010a; 2013) above and beyond verbal expressions of the conscious mind. As Jung made clear, “it is not the personal human being who is making the statement, but the archetype speaking through him” (Jung 1963: 352). In the Four Archetypes Jung says: “You need not be insane to hear his [archetype’s] voice. On the contrary, it is the simplest and most natural thing imaginable. […] You can describe it as mere ‘associating’ […] or as a ‘meditation’ [and] a real colloquy becomes possible when the ego acknowledges the existence of a partner to the discussion” (CW 9, 236– 237). Such “discussion” as a symbolic dialogue between consciousness and the unconscious takes place during Tarot readings when the subtle language of images is being interpreted or translated into verbal expressions thereby communicating meanings implicit in the images. Tarot pictures form a pattern in this or that specific layout which thus functions as a material embodiment of the archetypes by means of which such silent discourse (Semetsky 2010a) can communicate itself. We can see (Figure 1) that nearly every one of Tarot pictures has an image of a living being, a human figure situated in different contexts. The images symbolically encode information that affects human behaviour when this or that of Jungian archetypes becomes active either at the individual or collective levels. Among Major Arcana the Fool embodies the archetype of the Eternal Child; the Empress is the Great Mother; Hierophant represents the archetype of Persona; the Devil embodies a powerful Shadow (see further below), etc. The human figure travels through the cards, and Tarot Arcana comprise a pictorial text that represents universal human experiences accompanied by what Jung called a feeling-tone. This visual text has its own syntax, semantics, and pragmatics even if not in written linear alphabetic form. Yet, “[b]efore there was writing there were pictures” (Shlain 1998: 45). Each Tarot picture embodies “a meaning that is […] still beyond our grasp, and cannot be adequately expressed in the familiar words of our language” (Jung in Nöth 1995: 119) but needs a symbolic medium for its very expression, and a Tarot reader functions as a “bi-lingual” interpreter translating the semiotic, nonverbal but visual, messages implicated in pictures into their verbal counterparts.

3 Visual communication From the perspective of semiotics as the study of signs, communication exceeds a solely verbal mode of expression as in linguistics but covers extra-linguistic modes, including pictures and images. Earlier empirical studies by Russian semioticians equated the phenomenon of Tarot readings squarely with fortune-telling; the cards were used for a specific pragmatic purpose by a fortune-teller whose task was considered to simply “exert a strong influence on the person whose fortune is being told” (Lekomceva and Uspensky 1977: 70). Playing cards were regarded as a simple semiotic system with a limited vocabulary, in which “divination of past and present is a game” (1977: 71) and the future is foretold. In comparison with natural

The visual semiotics of Tarot images

179

languages, the formal structure of “the system of cartomancy [as] a language with finite number of states” was considered analogous to “certain artificial languages with a limited semantics” (1977: 73). In the comprehensive study by Heeren and Mason (1984), the authors, adopting a sociolinguistic method, presented both the ethnography of communication used by contemporary spiritual readers, as well as a therapeutic discourse as guiding their analysis, which focused on the precise form of readers’ statements. They distinguished between the style of everyday life, interview style and visionary style. The latter was characterized as “the most unusual and distinctive” (1984: 197) even as the authors acknowledged the ethnomethodological “et cetera principle” (1984: 200) employed by many readers. They described this principle as analogous to one that functions in “everyday conversation when participants are expected to ‘fill in’ meanings when others make ambiguous statements. [… It] means that one supplies contextual information to make sense of the indexical expressions of others. So it is in […] readings. Readers are not expected to spell out precise details of the persons and situations in their clients’ life. Instead clients must play an active part by trying to access the applicability of visionary statements to their life” (1984: 200). The original study by Aphek and Tobin (1989) has advanced cards reading to the level of complex, dynamic, meta-semiotic systems (cf. Semetsky 2001, 2008a), in which the multiple meanings of the cards were seen as representing “the possible semantic, cultural and social attributes of an umbrella term or theme attributed to that particular card” (Aphek and Tobin 1989: 13). They addressed the dynamic character of Tarot readings within which communication and the perception perform the function of “an individualized autopoetic process” (Aphek and Tobin 1989: 3). Still, as just one of many branches of fortune-telling in general, a case of Tarot readings was considered to be simply “a specific instance of persuasive dyadic human communication” (1989: 175; emphasis added), in which both participants were considered as being assigned “well-defined roles” (1989: 33). The Encyclopedic Dictionary of Semiotics (Sebeok 1994, 1: 99–100) describes Tarot readings “as a branch of divination based upon the symbolic meaning attached to individual Tarot cards or modern decks, interpreted according to the subject or purpose of a reading and modified by their position and relation to each other from their specific location in a formal ‘layout’ or ‘spread’ (1994: 99). What, however, has been overlooked in the aforementioned studies is an almost complete absence of any informational content in their examples of readings. Informational content is a necessary component for a sign to function as a genuine sign, that is, to stand for or to mean something other than itself. My research reconceptualises the phenomenon of Tarot readings and the interpretation of images from the viewpoint of Charles Sanders Peirce’s logic as semiotics and the importance allotted by him to the dynamic process of signs’ evolution and growth called “semiosis”. For Peirce, the whole world is composed of signs that

180

Inna Semetsky

Fig. 2: A genuine sign.

permeate nature, culture and human mind (Semetsky 2013). Due to the evolution of meanings, signs grow and become other as more fully developed. As embedded in a stream of semiosis, we humans too are signs and we can acquire information and come to mean more than before. While analytic philosophy of language grounded in the dyadic logic of the excluded middle presented verbal signs as the sole means of communication and representing reality directly, the semiotic, or pragmatic, turn (cf. Bernstein 2010) is not illogical. The process of reasoning however is indirect or mediated. Peirce’s logic is triadic or analogical, and involves interpretation versus direct representation common to the modernity for which, historically, “there could be no tertium quid” (Merrell 2002: 204) as an indirect, mediated, relation between the two, usually perceived as opposites, things. Rather than being a simple dyad, a genuine (contra degenerate) sign has, according to Peirce, a triadic structure (Fig. 2), in which it is an interpretant that mediates between a sign and its object and functions as the included middle. It is an interpretant that creates a meaning for the sign by means of “filling” it with a specific informational content the meaning of which, as a result of interpretation, can be communicated. If there is a lack of in-formation, then no trans-formation is possible, so the reading remains just a reading without any implication of habit change by virtue of creating a new meaning for – or making sense of – often disturbing and traumatic experiences taking place in human lives, in culture. In my recent volume Re-Symbolization of the Self: Human Development and Tarot Hermeneutic (Semetsky 2011) I have presented real-life “case studies” with the interpretations of cards’ meanings in 15 reading sessions. Importantly, one and the same image, however situated in different contexts, may acquire different connotations, that serve to illustrate the concept of polysemy peculiar to Tarot and address the “polyvocality” of visual semiotics with its wealth of potential meanings. For Peirce, a sign can be represented by image, index or symbol, and a perfect sign combines all three semiotic elements, so that image (or icon) is always already

The visual semiotics of Tarot images

181

present in the products of the intellect such as concepts, meanings and conscious judgments. Tarot images are signs that establish a syntactic structure of a layout in the form of a pictorial, extra-linguistic, text. They are purposeful and meaningful semantically and are capable of changing their meanings dynamically depending on the context they are situated in. Functioning as an unorthodox symbolic text communicating messages in a pictorial mode, the Tarot spread can be read and interpreted: importantly, it is “the constitution of messages [that] forms the subject matter of semiotics” (Sebeok 1991: 22). From the Peircean perspective, the function of the semiotic “interpretant” is performed by the language of Tarot images and symbols, which – like any language in general – is represented by a structured semiotic system whose role is to pair expressions (functioning as public aspects, e.g. physical marks as actual pictures) and messages (private aspects as thoughts and concepts), that is, to have a means for mapping between them (cf. Jackendoff 2001) when narrating the meanings of the images. For Peirce, it is his third category of interpretation that, by means of performing a mediative or symbolic function, creates or “brings information […] [it] determines the idea and gives it body” (Peirce CP 1. 537) at the level of practical action. It is the Tarot symbolism – the universal language of signs (Semetsky 2006a) – that establishes an unorthodox communicative link functioning as a semiotic interpretant and performing a function of the “included third” of Peirce’s genuine triadic sign capable of creating a semiotic bridge to the Jungian collective unconscious. A Tarot reader converts the pictorial language of the unconscious into verbal expressions, thus facilitating the trans-formation of in-formation into consciousness. The signs may vary, however the means of symbolic communication remains the same: what takes place is an indirect, mediated, connection akin to the acting principle of synchronicity posited by Jung in collaboration with the famous physicist and Nobel laureate Wolfgang Pauli. Synchronicity addresses the problematic of meaningful patterns generated in nature and culture alike along the multiple paths of human experiences. The concept of the unconscious extends its boundary tending to the notion of “‘field’ in physics [as] a more general form of ‘connections’ in nature” (Pauli 1994: 164). In his 1952 letter to Jung, Pauli expressed his belief in the gradual discovery of a new, what he called “neutral”, language that functions symbolically to describe the psychic reality of the archetypes and would be capable of crossing over the psycho-physical dualism2 that continues to haunt human culture since Descartes. Indeed, mind and matter do not oppose each other but can be reconciled within the process of Peircean semiosis: ‘‘Mind is First, Matter is Second, Evolution is Third’’ (Peirce CP 6. 7).  See Atom and Archetype: The Pauli/Jung Letters 1932–1958. Edited by C. A. Meier, with a preface by Beverley Zabriskie (2001, Princeton University Press). This particular letter is designated in the book as 56P, pp. 81–83. See also my article “The language of signs: Semiosis and the memories of the future” in SOPHIA: International Journal for philosophy of religion, metaphysical theology and ethics, 2006, 45/1: 95–116.

182

Inna Semetsky

The art of Tarot readings belongs to the right hemisphere as a sympathetic system capable of what Peirce called the synthetic consciousness in contrast to the left, cerebral, hemisphere, oriented to solely analytic reason. Still, it is when the integration of both sides takes place that the mind is capable of fulfilling its full creative potential. Shlain remarked that the right side of the brain is “wise […] more familiar with needs and drives stemming from the earlier stages of evolution” (1998: 18). The right hemisphere is permeated by “feeling-states [that] are authentic […] An internal voice verifies the experience […] The right hemisphere is […] the portal leading to the world of the invisible” (Shlain 1998: 19; italics in original) comprising a “signosphere” (cf. Deely 2001) that exceeds, yet encompasses, our empirical observable world. The expanded world of semiotic reality becomes available to observation that includes now the deep level of collective psyche full of potential significance at the level of “practical bearings” (Peirce CP 5. 402) in agreement with Peirce’s pragmatic maxim as his theory of meaning. When symbolically represented in Tarot images, the transpersonal and transcendental realm of the collective unconscious is being brought, so to speak, down to earth by virtue of its embodiment in the material reality, confirming Jung’s insight that “psyche and matter are two different aspects of one and the same thing” (Jung CW 8, 418). They are united during readings, thus defying the ghost of the dualistic schism both in theory and, significantly, in practice. The levels of sociocultural praxis as encompassing human behaviours, decision making or choosing a particular course of action is of utmost significance! Jung was adamant that the general rules of human conduct are “at most provisional solutions, but never lead to those critical decisions which are the turning-points in a man’s life. […] ‘The diversity and complexity of the situation makes it impossible for us to lay down any theoretical rule for ethical behaviour’” (Jung in Neumann 1969: 13).3 It is not only in theory but, significantly, in our practical experience that each and every Tarot reading (Semetsky 2011) becomes a step toward the realisation of the deepest meaning of a particular situation at the level of social reality.

4 Social reality The field of communication phenomena as part of the typology of cultures calls for the identification of specific semiotic systems representing their “languages”. Accordingly, human experiences cross over the boundaries of individual minds and expand to the level of the whole of culture. In this respect culture partakes of a text containing collective memory (cf. Lotman 1990) that unfolds in the process of the evolutionary process of communication during which the nature of objects is

 From Foreword by C. G. Jung to E. Neumann’s book Depth Psychology and a New Ethic. Jung’s Foreword copyright © 1968 by the Bollingen Foundation, New York.

The visual semiotics of Tarot images

183

Fig. 3: Major Arcana images.

gradually revealed to human subjects-in-kinship. In semiotic terms, memory is considered a specific capacity to preserve and reproduce information. In this respect a Tarot deck represents a semiotic system in the format of a nonverbal “lexicon”, and each Tarot layout comprising the constellations of images is a symbolic text full of information and implicit “memories” of which we becomes aware due to an unorthodox communicative link between consciousness and the unconscious. The journey through the cards’ imagery not only contributes to individual selfknowledge but also expands to the relationships with significant others in diverse social contexts and amidst events taking place in human culture. Cultural events become meaningful signs “filled” with a semiotic or communicative potential; they carry social and political significance and represent symbolic texts to be read and interpreted. As pictorial artefacts, Tarot images embody the patterns of thoughts, affects, emotions, feelings and behaviors implicit in collective experiences and social events that transcend times, places, language barriers, and disparate beliefs and values. As Marshall McLuhan famously made clear, the medium is the message. The medium of Tarot pictures becomes, rather than being “merely a passive conduit for the transmission of information, an active force in creating new social patterns and new perceptual realities” (Logan 1986: 24). Often, the action of Jungian archetypes is such that they can possess the psyche in the guise of the unconscious Shadow. Jung saw how powerfully this archetype worked behind the scenes, implicitly affecting the psyche and explicitly influencing people to behave in a neurotic or compulsive manner. Among Major Arcana, the Shadow archetype corresponds to the Arcanum XV called the Devil, the fallen angel, the dark precursor in the evolution toward two subsequent cards called the Tower and the Star (Figure 3). In the guise of the Shadow, the Devil can easily possess one’s psyche and, importantly, the Shadow can often become projected onto others (Semetsky 2005b),

184

Inna Semetsky

and one may very well attribute to significant others those qualities that one is tempted to deny in oneself. The concept of Shadow describes a cluster of impulses, complexes, shameful and unacknowledged desires, self-indulgences and being a slave to one’s own primitive instincts. Sexual compulsion, poor impulse control, or plain old greed are some behavioural patterns that may manifest in real life. It may be a fear or a superficial complex of superiority when in fact deep inside one feels rather inferior. On the picture, the two naked figures chained to the Devil’s throne in the underworld appear to have lost the ability of clear judgment and seem helpless. The Devil’s heavy chains represent our self-destructive tendencies and weaknesses; bondage and fear. In interpersonal relationships, the Devil can reflect upon co-dependency issues. It may be a deeply ingrained fear of breaking free, similar to battered women unable to leave and continuing to stay in abusive relationships, overwhelmed by submissiveness or sexual/economic dependency. At the collective level, the Shadow encompasses those outside “the norm” of the established order and social system, such as “criminals, psychotics, misfits, scapegoats” (Samuels 1985: 66). It is not only that these shadowy figures appear to stand outside culture, but importantly culture itself fails to assimilate its own Shadow. The Devil Arcanum is a symbol of the ultimate slave morality, in the Nietzschean sense, in the relationship between the oppressor and those oppressed. It represents a moment of psychological denial and the implementation of scapegoat policy by the dominant culture or nation, while in the meantime projecting onto some generic Other one’s own inferior and shadowy qualities. The scapegoat psychology is associated with what Erich Neumann called old ethics, and it is an ethical attitude indeed that is central with regard to the Shadow archetype. While the ego-consciousness focuses on indubitable and unequivocal moral principles, these very principles crumble under the “compensatory significance of the shadow in the light of ethical responsibility” (Jung in Neumann 1969: 12; italics in original). The neglect of this responsibility tends to precipitate multiple evil consequences in the world. While old ethics is the ethics of illusionary perfection and absolute Good that necessarily leads to the appearance of its polar opposite, absolute Evil, the new ethics consists in recognising our own dark side, that is, making the darkness conscious. The old ethics is “partial” (Neumann 1969: 74) as belonging solely to the isolated and self-centred conscious Ego; the new ethics devoted to the integration of the unconscious Shadow is, however, holistic and ideally represented by the Jungian archetype of the Self. The Self – as a whole – emerges only when the opposites exist as a harmonious whole and neither side is suppressed or eliminated. In his book Tarot: Talisman or Taboo? Reading the World as Symbol, Irish philosopher and monk Mark Patrick Hederman (2003) points out that the Devil’s (or Satan’s) other name is Lucifer, he who brings the Light in order to illuminate the darkness. In this allegory, “the evil that is the shadow side of everything that is bright and good remains hidden” (Hederman 2003: 176) or invisible. The perpetual

The visual semiotics of Tarot images

185

presence of the shadow must be recognised, made visible, and integrated into consciousness; otherwise, it will fall into the depth of the unconscious where it will continue to crystallise. The absence of freedom, lack of hope, and total powerlessness will reach their critical mass and will start acting from within the psyche, spreading spontaneously until reaching a destructive climax. Non-incidentally, the picture that immediately follows the Devil in the deck, called the Tower (Figure 3), represents this upcoming climax. In the Tarot feminist interpretation (Gearhart and Rennie 1981), the imagery of the Tower signifies radical intervention, revolution and the overthrowing of false consciousness, violent social conflict and change, destruction of the old order on a grand scale, and release from imprisonment in the patriarchal structure during the very process of its demolition. Jung spoke about the archetypal temenos in one’s psychic structure that in Greek means a sacred precinct like a temple; a synonym for it is a hermetically sealed vessel or, for that matter, the Tower. Temenos, as employed in Jungian analysis, acquired psychological connotations as the psychically charged area surrounding a complex, and may be experienced sometimes through the symbolism of any closed container such as a womb or a prison. Because the vessel is sealed hermetically, the force looking for its way out will be ultimately felt as acting from within in an erratic, horrifying and unpredictable manner (Semetsky 2000) reflected in the imagery of this card. Regarding real events in human culture at the social level, the imagery of the Tower – which in some decks is called The House of Destruction – has an uncanny resemblance with the image of the destroyed towers on September 11 (Figure 4; Photo courtesy FEMA.):

Fig. 4: WTC towers on 11 September 2001.

Jean Baudrillard (2002), in his analysis of the spirit of terrorism, talks about the shift of the struggle into the symbolic sphere where an initial event – “as quite a good illustration of chaos theory” (2002: 23) – becomes subjected to unforesee-

186

Inna Semetsky

able consequences. Such a singular event – like the destruction of the Twin Towers on 9/11 – propagates unpredictably, causing the chain of effects “not just in the direct economic, political, financial slump in the whole of the system – and the resulting moral and psychological downturn – but the slump in the value-system” (2002: 31–32) per se. The collapse of the towers represents the fact that “the whole system has reached a critical mass which makes it vulnerable to any aggression” (2002: 33). No longer projected inward, the released darkness becomes directed into the outer space. Baudrillard points out that not only terrorism itself is blind but so were the actual towers – “no longer opening to the outside world, but subject to artificial conditioning” (2002: 43): air conditioning, or mental conditioning alike – similar to the Tower in the Tarot picture that was sealed at the top when suddenly struck by lightning. The Tower image is an embodiment of contradiction and the conflict of opposites; Jung used the notion of contradiction with regard to the meaning of the tower which he, at a symbolic level, identified with the tower of Babel as a symbol of false omnipotence and mistaken certainty, a priori condemned to destruction during the most powerful and confusing instance of contradiction and mutual misunderstanding: the confusion of tongues, indeed. Jung pointed out that it is the excess of pride and passion that “raises a man not only above himself, but also above the bounds of his mortality and earthliness, and by the very act of raising him, it destroys him. This ‘rising above himself’ is expressed mythologically in the building of the heaven-high tower of Babel that brought confusion to mankind” (Jung, CW 5, 171). The picture in Figure 5 below, from the deck called The Lovers’ Tarot, incorporates elements of the famous masterpiece by Pieter Brueghel the Elder:

Fig. 5: The Tower.4

 This image is from the deck called The Lovers’ Tarot, by Jane Lyle; Illustration Copyright © Oliver Burston 1982. Published by Connections (January 2000) in the UK and St. Martin’s Press in the US.

The visual semiotics of Tarot images

187

Fig. 6: WTC following the 11September 2011 attack.

During Tarot readings the appearance of the Tower card may indicate a catharsis, that is, a dramatic and forceful replay of the unconscious material that exceeds the boundaries of the current “circumference” of the mind and forces the darkness at a very deep level to break through into the surface of consciousness. Significantly, the enforced evacuation, breaking all defences, frees one from being incarcerated in the symbolic tower of one’s own making, whether it is psychological, ideological, cultural, or of any other belief system. Thunder and lightning, as per the image of the Tower, are universal signs of the wrath of gods; the symbolism of which also indicates a swift – and painful – alteration at the level of collective consciousness when it observes the aftermath of the destruction of the self-erected unstable structure. The ultimate destruction – a body turned into a lifeless skeleton – is seen in this other poignant and maximally real image of 9/11 also published on the Internet (Figure 6; U.S. Navy photo by Journalist 1st Class Preston Keres). Any unforeseen cataclysmic event that suddenly brings people down to earth by disturbing the existing norm and order of things through the abruptly terminated current psychological state or a break-up in a set of values privileged by a given culture, necessarily raises the level of consciousness. The breakdown in existing order simultaneously creates conditions for the potential production of a new order. Thus the image of The Tower card is a sign not only of a breakdown but a breakthrough when the darkness embodied in the preceding image of the Shadow-Devil is illuminated and made conscious. I agree with Mark Patrick Hederman (2003) who warns of a danger to ourselves and others if and when we choose to remain unconscious of the Shadow. If history and culture have taught us anything, it is that in the 20th century The Devil manifested as:

188

Inna Semetsky

a hell on earth and that this hell was a human creation. It was a hell of cruelty and mayhem resulting from the incapacity of the powerful people to decipher their unconscious motivation [… E]ach of us has to discover and explore the labyrinth of the dark, the unconscious […] Its language is incomprehensible, even inaudible to most. But, no matter how difficult it is to decipher, such work must be undertaken. We must recognize that most of our past, whether personal or historical, took place underground, in silent rivers, ancient springs, blind pools, dark sewers. While the task of making them accessible to our consciousness is difficult, it is nonetheless imperative. Even more so at the beginning of a new century when we hope to outline some plausible tracks into a better future. We have to read the signs of the times […] (Hederman 2003: 21).

The signs of the times may come from the earth, such as volcanos or earthquakes; or from water such as tsunamis; or from the air such as the attack on September 11; or from the fire when drought causes famine; in all cases the results are disastrous. Still, human lives can be saved because it is all four elements of nature – air, water, earth and fire corresponding to the four suits in a deck (swords, cups, pentacles, and wands) that are trying to communicate with us in the form of images representing real significant events that encode symbolic messages about the behavioural and cultural patterns which have caused (or will have caused) them. To decode these messages through the vibrant language of the unconscious embodied in the symbolic system of Tarot is not a utopian dream for the future but the reality of the present because this code is already available and can be used to communicate the meanings encoded in the Tarot imagery (Semetsky 2008b, 2011; 2013). Sure enough, the future can still be skewed because prevailing ideologies or grand meta-narratives are still here and remain the means “of imposing our own myopic architecture, of obliterating the splendour of what might have been: the future perfect” (Hederman 2003: 22). The least we can do is to have hope for a better future. But, in accord with Jung’s reference to the foreknowledge by virtue of symbols as purposive, healing, and numinous, the better future may already “will have been”. Tarot readings perform an amplifying function in agreement with the Jungian synthetic method (contra reductive Freudian psychoanalysis, as we said earlier) that implies the emergence of new meanings as carrying the utmost significance. The synthetic method reflects the future-oriented path to knowledge, and the archetypes do determine “the nature of the configurational process and the course it will follow, with seeming foreknowledge, or as if it were already in a possession of the goal” (Jung CW 8, 411). Ditto for Peirce: semiosis as the action of signs across nature, culture and human mind is evolutionary and includes all three time-dimensions: “A man denotes whatever is the object of his attention at the moment; he connotes whatever he knows or feels of this object, and is the incarnation of this form […]; his interpretant is the future memory of this cognition, his future self” (Peirce CP 7. 591). Peirce’s semiotics thus makes commonsensical the future dimension of experience peculiar to Tarot readings; yet, without the vulgarity associated with fortune-telling as a habitual perception of this phenomenon prevalent in contemporary culture.

The visual semiotics of Tarot images

189

Significantly, the polyvalence of the image that follows the Tower in a deck, called the Star, connotes the field of meanings which include hope, healing, inspiration, creativity, and the realisation of our spiritual dreams. Hence, we do understand the message that the Tower of destruction, which preceded the Star temporarily, was only a stage in the directed-forward evolution of consciousness and the development of humankind. We have learned our moral lesson embedded in the Tower. The presence of the Star in a deck, as a natural progression from the Tower, is a symbolic message that the Tower image, with its abysmal real-life effects, is a precursor to the renewal and the creation of new psychic space in human culture as aligned with nature. The image of the Star convenes our oneness with nature symbolised by the naked woman pouring waters from the two vessels. The vessels are red, this colour representing full flesh-and-blood humanity in unity with spiritual essence represented by water, colour blue. As the first figure in the sequence of the Major Arcana – importantly, feminine – without any clothes on, the Star is a symbol of being finally stripped of the outlived habits and values by bringing them to the level of conscious awareness – illuminating them, indeed. Reading and interpreting the meanings of Tarot images leads to transcending the current level of consciousness. Jung posited the transcendent function that arises from the tension between consciousness and the unconscious and effectuates their union. He called this function transcendent because of its ability to make the transition from the unconscious into consciousness, “from one attitude to another organically possible” (Jung, CW 8, 145). This new attitude embodied in the Star is represented by a sense of hope, healing, inspiration and the forthcoming new Aquarian age; in fact, this Arcanum is sometimes called The Star of Hope: the hope for new understanding and the revaluation of values! In the current global climate permeated by diverse beliefs, disparate values and cultural conflicts when different ideologies compete with each other, leading to destruction on the scale of the Tower on September 11, the universal value of hope is paramount. A revolution in the societal value system can in fact be brought about not by a physical upheaval but by our participation in the process of evolution. As signs among sings we are an organic part of semiosis as the growth and development of signs expressed symbolically in our journey through the Tarot images. Thus we can become able to transform the potentiality into our very reality by virtue of the lived-through meanings contained in the Tarot symbolism. The Star is a symbol of being finally stripped of the one-sided “left-brain” (cf. Shlain 1998) outlived values and typically masculine ideologies permeating social world. As Shlain indicates, “writing of any kind will realign the gender politics of any culture” (1998: 184); and it is the Tarot legible images that will realign our Western culture with its long-lost feminine dimension, which is symbolically represented by the female figure in The Star picture.

190

Inna Semetsky

5 The Fifth Age of Understanding Philo, a philosopher in ancient Alexandria, asserted that the real wisdom functions through Logos or the Word used later in the Gospel of John: In the beginning was the Word. This creative symbol, the Word, should not be understood literally as verbal or linguistic! Shlain argues, sigificantly, that “[I]n the beginning was the image” (1998: 432). A return to iconic, visual, information represents a shift toward an alternative – feminine, as expressed in The Star – mode of communication and action that “will prepare us for the next great communication revolution, in which we are already deeply engaged” (Shlain 1998: 429; italics in original). Learning the Tarot symbolic language by means of training the right-hemispheric mode of perception and expression will recover the lost respect for iconic information – and not only through the medium of personal computers. The interpretation of Tarot images employs the same “pattern recognition inherent in ideographic language [that] enmeshes users in a web of interpersonal relationships” (Shlain 1998: 185) and which – as embedded in the shared field of meanings – can therefore be improved. Reading and understanding the language of images “can bring our two hemispheres into greater equilibrium and allow both individuals and cultures to become more balanced” (Shlain 1998: 429). A return to iconic information represented through the ancient medium of Tarot images should now become the core of semiotic research in order to establish what contemporary semiotician Winfried Nöth designated as an “intercultural competence” (2010: 9). This is an urgent matter in the present geopolitical context that displays diverse and polyvalent signs of the times (cf. Semetsky 2010b) amidst cultural conflicts and clashes of values. The language of signs that can communicate in characters reflecting meanings and values as shared by humankind needs to be understood. Ultimately, Tarot functions in accordance with the universal, speculative, grammar; “speculum” expressing the correspondence between the two kinds of signs: words and things. For Peirce, everything in the world is a sign: yet, paradoxically, “nothing is a sign unless it is interpreted as a sign” (Peirce CP 2. 308). The presence of a Peircean interpretant, as the included middle, or the third in its relation to words and things, blends all three in a single event, a sign always having a triadic structure. Signs are relational, and they form a process-structure as a network of relations between events. The structure is dynamical because an interpretant as embedded in a triadic process necessarily becomes an object of the next sign (Figure 2). John Deely, in his survey of philosophy from ancient times to the turn of the 21st century (Deely 2001) speaks of the four Ages of Understanding, tracing the development of philosophical thought from antiquity to postmodern philosophers, the first of which, for Deely, was Charles S. Peirce. These four Ages, connected intellectually and historically by the theory of signs as their dominant theme, cover Ancient Greek philosophy, the Latin age of European philosophy from the 4th to

The visual semiotics of Tarot images

191

the 17th centuries, then the Modern period beginning with Descartes and Locke, followed by the Post-modern period that began with Peirce and continues today. It is only logical that in order to participate fully in the process of signs’ evolution and growth we have to learn how to read and understand the language of signs. Reading Tarot signs and learning from significant events – especially from the catastrophic ones as displayed in the Tower – will contribute to the integration of the iconic information and will bring a holistic dimension to moral life at personal, social and ecological levels. This holistic intelligence is necessarily equipped with deep Gnostic knowledge that encompasses meanings and values. It is this knowledge symbolically represented in Tarot images that should enable our “reading the emotional state, needs, and intentions of others” (Noddings 2010: 170). Thus it should move us closer into what I believe (and paraphrasing Deely) can become the New, fifth, Age of Understanding. This New Age shall provide us with unprecedented semiotic freedom to act intelligently and wisely in the world perfused with signs. This New Age of Understanding can become: a new Golden Age […] in which the right-hemispheric values of tolerance, caring, and respect for nature will begin to ameliorate the conditions that have prevailed for the too-long period during which left-hemispheric values were dominant. Images […] are the balm bringing about this worldwide healing (Shlain 1998: 432).

Tarot empowers us with the ability to make sense out of the chaotic flux of experiences when it is being unfolded in front of our very eyes. The images displayed in Tarot pictures “were conceived deep in the guts of human experience, at the most profound level of the human psyche. It is to this level in ourselves that they will speak” (Nichols 1980: 5) whenever we participate in the process of semiosis and, along such a lifelong path, continue to learn from our experiences (Semetsky 2011) so that becoming able to make decisions and choose ethical actions in unity and harmony with the Jungian unus mundus as one undivided world. The present 21st century is itself symbolic of this forthcoming transformation towards new understanding: is it a coincidence that the last Major Arcanum in a deck called the World is numbered XXI? Its symbolic meaning partakes of humanity comprising integrated Selves capable of living in harmony with Others. Such inter-relational human beings can and should take ethical responsibility for the social and natural world – the World – that they inhabit.

References Aphek, Edna and Yishai Tobin. 1989. The Semiotics of Fortune-Telling. (= Foundations of Semiotics 22). Amsterdam/Philadelphia: John Benjamin Publishing Company. Auger, Emily. 2004. Tarot and Other Meditation Decks: History, Theory, Aesthetics, Typology. Jefferson/London: McFarland & Company. Baudrillard, Jean. 2002. The Spirit of Terrorism/Requiem for Twin Towers. Trans. by Chris Turner. London: Verso.

192

Inna Semetsky

Bernstein, Richard. 2010. The Pragmatic Turn. Cambridge: Polity Press. Deely, John. 2001. Four Ages of Understanding: the First Postmodern Survey of Philosophy from Ancient Times to the Turn of the Twenty-First Century. Toronto: University of Toronto Press. Dummett, Michael. 1980. The Game of Tarot: from Ferrara to Salt Lake City. London: Gerald Duckworth & Co. Faivre, Antoine. 1994. Access to Western Esotericism. Albany: State University of New York Press. Gad, Irene. 1994. Tarot and Individuation: Correspondences with Cabala and Alchemy. York Beach: Nicholas-Hays. Gearhart, Sally and Susan Rennie. 1981. A Feminist Tarot. Watertown: Persephone Press. Guiraud, Pierre. 1975/1971. Semiology. Trans. by George Gross. London: Routledge & Kegan Paul. Hederman, M. Patrick. 2003. Tarot: Talisman or Taboo? Reading the World as Symbol. Dublin: Currach Press. Heeren, John W. and Marylee Mason. 1984. Seeing and Believing: A study of contemporary spiritual readers. Semiotica 50(3/4): 191–211. Jung, Carl G. 1953–1979. Collected Works, Vols. I-XX. Ed. by Herbert Read, trans. by R. F. C. Hull, Michael Fordham, Gerhard Adler, and William McGuire. (Bollingen Series.) Princeton: Princeton University Press. Jung, Carl G. 1959. The Archetypes of the Collective Unconscious. London: Routledge.. Jung, Carl G. 1963. Memories, Dreams, Reflections. Ed. by Richard Winston, Clara Winston and Aniela Jaffe. New York: Pantheon Books. Lekomceva, Margarita Ivanova and Boris Andreyvevich Uspensky. 1977. Describing a Semiotic System with a Simple Syntax. In: Daniel P. Lucid (ed.), Soviet Semiotics: An Anthology, 65–76. Baltimore: John Hopkins University Press. Lotman, Yuri M. 1990. Universe of the Mind, a Semiotic Theory of Culture. Trans. by Ann Shukman. Bloomington: Indiana University Press. Merrell, Floyd. 2002. Learning Living, Living Learning: Signs, Between East and West. New York/ Toronto: Legas. Neumann, Erich. 1969. Depth Psychology and A New Ethic. London: Hodder and Stoughton. Nichols, Salle. 1980. Jung and Tarot: An Archetypal Journey. York Beach: Samuel Weiser. Noddings, Nel. 2010. The Maternal Factor: Two Paths to Morality. Berkeley: The University of California Press. Nöth, Winfried. 1995. Handbook of Semiotics. (= Advances in Semiotics). Bloomington: Indiana University Press. Nöth, Winfried. 2010. The Semiotics of Teaching and the Teaching of Semiotics. In Inna Semetsky (ed.), Semiotics Education Experience, 1–20. Rotterdam: Sense Publishers. Pauli, Wolfgang. 1994. Writings on Physics and Philosophy. Ed. by Charles P. Enz and von Karl Meyenn, trans. by Robert Schlapp. Berlin: Springer-Verlag. Peirce, Charles Sanders. 1860–1911 [1931–1935]. Collected Papers by Charles Sanders Peirce. Ed. by Charles Hartshorne and Paul Weiss. Cambridge: Harvard University Press. Posner, Roland. 1989. What is Culture? Toward a semiotic explication of anthropological concepts. In Walter Koch (ed.), The Nature of Culture, 240–295. Proceedings of the international and interdisciplinary symposium, October 7–11, 1986 in Bochum. Bochum: Brockmeyer. Samuels. Andrews. 1985. Jung and the Post-Jungians. London/New York: Routledge. Sebeok, Thomas. 1991. Communication. A Sign is Just a Sign, 23–35. (= Advances in Semiotics.) Bloomington: Indiana University Press. Sebeok, Thomas (ed.). 1994. Encyclopedic Dictionary of Semiotics, Volume I. (= Approaches to Semiotics 73.) Berlin/New York: Mouton de Gruyter. Semetsky, Inna. 2000. Symbolism of the Tower as Abjection. Parallax 15 6(2): 110–122. Semetsky, Inna. 2001. Signs in Action: Tarot as a Self-organized system. Cybernetics & Human Knowing 8(1–2): 111–132.

The visual semiotics of Tarot images

193

Semetsky, Inna. 2005. Integrating Tarot Readings into Counselling and Psychotherapy. Spirituality and Health International 6(2): 81–94. Semetsky, Inna. 2006a. The Language of Signs: Semiosis and the memories of the future. SOPHIA: International Journal for philosophy of religion, metaphysical theology and ethics 45(1): 95–116. Semetsky, Inna. 2006b. Tarot as a projective technique, Spirituality and Health International 7(4): 187–197. Semetsky, Inna. 2008a. Simplifying Complexity: Know Thyself … and others, COMPLICITY: An International Journal of Complexity and Education 5(1). Available at: http://www.complexityandeducation.ualberta.ca/COMPLICITY5/documents/ Complicity_5_1_07_Semetsky.pdf. Semetsky, Inna. 2008b. The Transversal Communication, or: “Reconciling science and magic”, Cybernetics & Human Knowing: A Journal of Second Order Cybernetics, Autopoiesis & CyberSemiotics 15(2): 33–48. Semetsky, Inna. 2009. Whence Wisdom? Human development as a mythic search for meanings. In Marian De Souza, Leslie Francis, James O’Higgins-Norman, and Scott Daniel (eds.), International Handbook of Education for Spirituality, Care and Wellbeing, 631–652. (= International Handbooks of Religion and Education 3.) Dordrecht/Heidelberg/London/ New York: Springer. Semetsky, Inna. 2010a. Silent Discourse: The language of signs and “becoming-woman”. SubStance #121 [Special issue Spiritual Politics After Deleuze] 39(1): 87–102. Semetsky, Inna. 2010b. Interpreting the Signs of the Times: Beyond Jung. Social Semiotics 20(2): 103–120. Semetsky, Inna. 2011. Re-Symbolization of the Self: Human Development and Tarot Hermeneutic. Rotterdam: Sense Publishers. Semetsky, Inna. 2013. The Edusemiotics of Images: Essays on the Art-Science of Tarot. Rotterdam: Sense Publishers. Shlain, Leonard. 1998. The Alphabet versus the Goddess: The Conflict Between Word and Image. New York: Viking. Sonesson, Goran. 1989. Pictorial Concepts. Inquiries into the semiotic heritage and its relevance to the analysis of the visual world. Malabar: Krieger Publishing Company.

Dennis Puhalla

8 Colour language hierarchy Abstract: Tying colour to language structure is a proposition that establishes a clear rationale for strategically organizing colour. When colour is organized strategically, visual communication is clear. To support this proposition, statistical data was generated from the Intrinsic Colour Research Study focusing on colour’s naturally inherent properties and visual language communication. If colour is to function as language and carry a message, it must contain a colour syntax and semantic organisation. The three properties of colour govern the rules of order (syntax) in which colours are combined to form a visually fitting expression of thought. Semantics addresses meaning or the interpretation of a colour grouping. Colour is the element that ‘gives meaning’ to the syntax. Varying types of colour harmonies form the semantics of a colour organisation. The context of colour’s composition delivers informational content. When the principles of colour organisation functions as a unified whole, a colour language is formed.

1 Colour tactics Derivative and objective colour selection methodology is not a strategy found in the literature related to colour theory and application. Philosophy, social science, art and design have generated a considerable amount of published material on the subject of colour. Berlin, Kay, Hardin, Rosch, Munsell, Favre, Albers and Tufte are a few among the notable authors and researchers addressing the topic of colour. Colour is a variable that can increase clarity and significance. “Colours trigger very specific responses in the central nervous system and the cerebral cortex. Once they affect the cerebral cortex, colours can activate thoughts, memories, and particular modes of perception” (Gobé 2001: 77). As Hardin explains “Colour comes into its own in recognition of objects and in conveying specific biological information” (Hardin 1992: 376). Hardin also points out that among the operative principles in human factors research, colour contrast shortens recognition tasks and improves task accuracy. “Colour is a powerful aid to object recognition, and categorical colour difference seems particularly helpful” (Hardin 1992: 376). A research study on colour’s inherent properties and influence on perceived order provides definitive recognition of how colour establishes its own language. Statistical results from the research study, Intrinsic Colour, present empirical evidence supporting the proposition that strategic colour organisation based on colour’s properties and conventional harmonies communicate a distinct visual message. The outcome of this study conveys objective guiding principles related to colour selection and organisation. Therefore, the application of these principles

196

Dennis Puhalla

can be utilized and strengthen a person’s cognitive ability to extract information, increase retention and assist comprehension. However, minimal documentation exists identifying objective guiding principles related to the organisation of a colour language. Credible empirical findings are detailed in the Intrinsic Colour Research Study. Specifically, this study applied the principles of colour’s inherent properties and harmonious natural relationships. Accordingly, this research proposes a strategy for objective colour selection and ordering which is crucial to making information understandable. This study also initiates a complementary method of colour structuring methodology based on its innate language, additionally evoking an emotional, psychological response. Understanding colour recognition, related to perceived order and importance, could have useful applications to a naturalistic, real world display of visual information systems based upon objective colour selection strategy. Real world colour application also includes visual recognition tasks. These are tasks in which an individual looks at the context of colour organisation as a means of ordering information to determine importance. Colour adds significance to visual communication as it accentuates and makes perceptible the identification of that which is perceived first, as the main idea, before decoding additional elements in the message. Perceived hierarchy represents a sequence of visual commands in the language of colour. Therefore, colour adds functionality and definitive relationships to many informational entities – signalling systems, way-finding systems, maps, instrumentation, educational/instructional material, etc. Effective colour communication permits users to view, retrieve, access, decipher, interpret, understand, and experience a variety of information systems in a meaningful, valid and authentic manner. As stated earlier, colour communication implies meaningful usability. Generally, visual designers intuitively agree on the most pleasing colour combinations, though established theories about the specific nature of colour composition have no basis in empirical research. In short, there are no rules. The importance of this study was to find guiding principles of fact. It was, therefore, necessary to statistically determine the predominant tendency in viewers’ perception of colour organisation. The documented findings of the Intrinsic Colour Structures Study presented explicit evidence that addresses specific mechanisms for objective colour ordering phenomena within a dynamic digital environment. Accordingly, it may be assumed that the same logic would apply to print media. Colour should translate as a code for transmitting a message, making things easy to understand and to assimilate. Effective colour assembly strategies produce strong visual impact, improve clarity and define artifact identity. Meaningful colour communication should improve the efficiency and efficacy of the message perceived by the end-user. Simultaneously, colour must function successfully on multiple levels. Colour serves as a primary structural element in the organisation of print, digital, product and information design. In this capacity, colour creates appropriate spatial and navigational effects that are specific to the operational

Colour language hierarchy

197

tasks. The colour designer’s task is to identify ideal and normal sequences: what the user should see first; where the eye should move next; and how much time the viewer's attention should be held within each area. The findings of the Intrinsic Colour Structure research provide evidence that colour organisation options can be based on objective criteria rather than by opinion, trend, bias or inclination alone.

2 Colour language structure Colour is one of several attributes of form that can establish a consistent visual language that functions as a unified whole. However, it must be noted that colour interacts in the context of two and three-dimensional visual attributes of form – “line, shape, size, volume, texture/surface, and proportion”. The congruent relationship of the attributes of form needs to be legible and capable of signifying resolved intent. Each of these attributes has its own individual qualities but all may be characterized by their colour. Three distinctive variable elements of colour function as a language and are defined by colour’s intrinsic properties. The three intrinsic properties are hue, value and chroma. The properties of colour are the basis of forming a visual language colour system. Undeniably, colour is an attribute of form that significantly impacts perception. Consequently, colour evokes an emotional reaction. The reaction is a product of our environmental and cultural experiences. However, each of the three colour elements – hue, value, chroma – embraces specific visual characteristics and can be identified by objective criteria independent of subjective feelings. Establishing an objective rationale for colour organisation and structure is central to meaningful information processing. Uncertain or random colour ordering communicates ambiguous messages. Persons with a normally functioning visual system obtain a rather large amount of information about their surroundings from the visual sense with colour playing an important part in this sensory interpretation. Colour serves as an activating stimulus that intensifies visual consciousness and responsiveness. “Since perception of colour is the single most strongly emotional part of the visual process, it has great force and can be utilized to express and reinforce visual information to a great advantage” (Dondis 1973: 55). To some degree, the process of cognitive understanding and meaning is shaped by a hierarchical conceptual structure found in language and visual representation. Language transcends the boundaries of the verbal and written word. Language is any system of representation including representations found in written words, signs, symbols, electronically produced images and musical notes. All languages embody some method of structure and hierarchy. Hierarchical structuring is a graded or ranked factor, which determines importance. Since hierarchy is a stratified organisation of parts, it is fundamental to the principles of ordering and to logical comprehensibility.

198

Dennis Puhalla

Fundamental to cognitive processing and meaning are syntax and semantics. Early Twentieth Century work of Swiss linguist Ferdinand de Saussure saw language as a system of words acting as arbitrary signs. Language systems employ a syntax, which governs the rules of order in which words or other elements of structure are combined to form grammatical sentences and phrases. This could also include the study of relationships between signs and symbols and their intent. Semantics addresses meaning or the interpretation of a word or sentence. Thus, interpretation is dependent upon a systematic, orderly construction and arrangement of parts, as well as the relationships between signs/symbols and their intention. According to the American cognitive linguist and Professor of linguistics at the University of California, George Lakoff (1990: 222): “Semantics is a technical way to ‘give meaning’ to the uninterrupted symbols of the syntax”. Therefore, language construction is dependent upon hierarchical structuring. Implicit hierarchical structuring increases the human ability to perceive similarities and differences among things. Hierarchical structuring preserves meaning in writing, speaking and visual representation. A rather unique view on colour structure and organisation offers the possibility that the intrinsic properties of colour form a language. Consider the prospect that colour’s language provides meaning based upon the similarities and differences of its intrinsic properties. When colour is applied to two or three-dimensional form, the intrinsic properties are well known and utilized. There are three properties of colour – “hue, value and chroma”. Hue refers to the generic or family name of colour. For example, think of all of the different colours associated with a blue hue. Value refers to perceived lightness or darkness of a colour. Light or dark colours are derivatives of a hue that appear to be tints with white added or shades with black added. Chroma is comprised by two characteristics – brightness and saturation. Brightness is often confused with lightness since they are inextricably linked. Suffice to say that brightness is a perceived response to the intensity of a colour. A colour’s intensity is a factual measurement of light’s wavelengths. Fully saturated colours are ones that cannot become any more colourful. For example, there is a point when a blue colour cannot become any bluer. Fully saturated colours are also associated with the visible spectrum of light as well as the red, yellow and blue (RYB) colour pigment wheel (see Figure 2 RYB Colour Wheel). Combining specific colours to form a unified whole is a visual composition problem. Understanding the similarities and differences of a colour grouping is an important first step in establishing unity. A unified grouping of colour establishes a context, which in turn enables a clear link to content – the message. Without a unified whole, visual chaos and message ambiguity is certain. Imagine small bits of paper differing in colour only scattered about. Perhaps that message is festive, but it is purely an emotional response to the chaotic colour arrangement. Tying colour to language structure is a proposition that establishes a clear rationale for strategically organizing colour. When colour is organized strategi-

Colour language hierarchy

199

cally, visual communication is clear. To support this proposition, statistical data was generated from the Intrinsic Colour Research Study focusing on colour’s naturally inherent properties and visual language communication. Accordingly, if colour is to function as language and carry a message, it must contain a colour syntax and semantic organisation. Colour syntax may be derived from the strategic organisation of its three properties – hue, value, and chroma. The three properties of colour govern the rules of order (syntax) in which colours are combined to form a visually fitting expression of thought. Semantics addresses meaning or the interpretation of a colour grouping. Colour is the element that ‘gives meaning’ to the syntax. Varying types of colour harmonies form the semantics of a colour organisation. Vibrant and highly contrasting colours send a message different from pale and weak contrasting colours. Colour harmony is a consistent, orderly arrangement of parts designed to show mutual relations and differences of the three properties – hue, value and chroma. Colour harmonies are derived from the relative positioning of colours found within the visible spectrum. While these harmonies offer a structured rational approach to organisation, they also provide a wide range of emotional characteristics as well. Visualize a colour harmony comprised of the hues red, yellow and blue – a primary colour harmony. If these colours are made to appear vibrant with high contrast, the emotional response elicited is not the same as when these hues are made to be muted with minimal contrast and significantly light in appearance. A colour organisation strategy can be useful if it describes the rationale on which colour relationships are based and suggests ways this rationale can help us to think structurally about colour composition. For example, a regional map may show city areas and boundaries with colour. If no one city is more important than another, the colour composition should be represented in a manner in which no one colour dominates the other. In order to create this composition of colour, differing colours must reside in the same visual space. This is accomplished by carefully manipulating the colours to be similar in value and saturation. In this instance, colour would not employ a hierarchical structure. Therefore, the colour language supports the message. Strategic colour organisation can be defined and articulated objectively, leading to visual order and importance establishing a visual hierarchy. Controlling the visual relationships of hue, value and chroma contrast can significantly assist the perception of importance and dominance within a particular colour composition. The basis of orderly colour representation and hierarchical structuring refers to the observed degree of colour contrast found within its three variables. Each of the three properties of colour intrinsically “governs the rules of order” in which the elements of those structures formulate a perceived visual hierarchy. Managing the structure of colour’s three properties determines a logical order in which things are seen and interpreted. This theory is founded upon long standing principles of colour’s visual appearance and representation. The visible spectrum, usually referred to as a red, yellow,

200

Dennis Puhalla

blue colour wheel, is the root of contrasting hues. By altering the value and chroma/saturation, the intrinsic properties of hues, a multitude of derivative colours can be realized. In other words, the intrinsic properties of colour constitute a systematic method of structuring the derivative colours.

3 Intrinsic colour structure Generally, colours are commonly defined in the context of their visual appearance. Based on visual appearance, an ordinary visible colour spectrum is derived from red, yellow and blue. The visible spectrum is that portion of the electromagnetic spectrum between 380 nm and 700 nm (nm – a metric unit of length equal to one billionth of a meter) that can be seen by the human eye. The colours referred to here are derived from the Red, Yellow, Blue (RYB) colour system, Figure 1, RYB colour system. The regularly referred to colours in the RYB spectrum are red, orange, yellow, green, blue and violet, Figure 2, RYB colour wheel.

Fig. 1: RYB colour system.

Fig. 2: RYB colour wheel.

“Hue” is the family or generic name of colour. It is a term used for colours contained within the visible spectrum. For example, a blue hue is the root of all blues that appear light, dark, bright, or dull. As a result, there are a limitless number of blue colours found within the hue that vary in their value and chroma. The vast numbers of blues are therefore derivatives of the parent colour, blue. “Value” of a colour is characterized by its relative lightness or darkness. A colour appears light if it is placed in the context of a colour that is darker. Generally, light colours that appear to have white in them are called tints, while colours that appear to have black in them are called shades. These lighter and darker colours are derivatives of hues within the visible spectrum. “Chroma”, or purity, is the third property of colour. Derivative colours of chroma have two distinguishing characteristics – saturation and brightness. Colours at full saturation cannot be made more colourful. Desaturated colours appear to have a gray tone. A tone is a derivative colour of a fully saturated hue. It is a

Colour language hierarchy

201

colour that has been altered with gray. As colours appear grayer they become dull or less bright. The terms brightness and intensity are often used interchangeably. However, consider the possibility that brightness is dependent upon our perception of a colour, while intensity is dependent upon the wavelength of a colour. One deals with cognitive processing and the other deals with scientifically measured calculations. The perception of colour unites three distinct qualities, defined as hue, value and chroma. The unification of the three properties is clearly articulated in the Munsell Colour Notation System. Albert Munsell established a colour notation system based upon numerical nomenclature scales with visually uniform steps for each of the three colour properties. Munsell’s method was not based on instrument measuring; rather, it was formulated by visual comparisons. The attributes constitute specific colour relationships and colour order. In its entirety, the Munsell Notation System configuration is a three-dimensional model represented as a non-symmetrical sphere. The three-dimensional model includes vertical and horizontal bands of colours. A neutral gray scale from white to black makes the centre vertical axis of the sphere. The full chroma hue extends horizontally from the neutral gray scale in the required number of gradients that constitute equal degrees of saturation contrast until it reaches full chroma/saturation. An example of this system follows in Figures 3, Munsell ordering system, along with a detail in Figure 4, Munsell value and chroma. Two hues, “yellow” and “blue”, are depicted in Figure 3. The value of the fully saturated yellow is positioned equal to a neutral gray at the very top left on the scale. The full chroma blue is positioned equal to a neutral gray near the middle of the scale at the far right. Within each of the two hue families, colours are organized vertically according to value – light to dark. Chroma/saturation is organized horizontally from less colour to more colour illustrated in Figure 4.

Fig. 3: Munsell ordering system.

Fig. 4: Munsell value and chroma.

202

Dennis Puhalla

Fig. 5: Typical colour selection tool.

In the Munsell System, colours can contrast in saturation without altering the value. In other words, the value represented along each of the horizontal rows is the same while the saturation changes along the row from less colourful to more colourful. It should be noted that value and chroma/saturation are inextricably linked in the value scale. When the value of the colour is altered, so is the saturation. Visually, there is a detectable difference between the derivative colours that are desaturated with grays and those that change specifically in values of white or black. Although the origin of the Munsell Notation System dates back to 1905, it is still an internationally accepted, leading colour system. This system is widely employed today. Anthropologists use it as a colour reference for coding samples and artifacts. Many imaging software applications use it as a method for colour selection. For example, most colour selection interface systems employed in digital imaging software programs are organized according to a hue and its derivatives in value and saturation. A variety of software programs use a terminology and a colour management system based upon the properties defined by Munsell as demonstrated in the Figure 5, Typical colour selection tool.

4 Experimental research study Employing the structural characteristics of the Munsell System, an experimental research study was conducted to determine if colour combinations could be identified and objectively qualified and quantified. The research was conducted in 2005 under the operational title, Intrinsic Colour Structures (Puhalla 2005). It was hypothesized that the intrinsic properties of colour form specific categories generating visual hierarchical structures. These hierarchical colour structures within a category are comprised of inherent contrasting relationships verifying that intrin-

Colour language hierarchy

203

sic colour organisation can establish a visual hierarchy affecting perceived order and importance. The primary focus of this research was to find empirical evidence that colour combinations can be categorized by its three properties – hue, value and chroma. The study sought to establish whether the three properties could be organized to produce a visual hierarchy – order of visual importance. If so, the cognitive ability to comprehend specific systems of colour contrast would be intrinsic and independent of emotion and colour preference. Determining the role colour plays in a visual communication system, is an essential organisational factor. Colour’s language must be sensible and well-defined. Within the Intrinsic Colour Structures experimental research framework, the three independent variables (hue, value and chroma) were tested. The dependency of these attributes was compared visually in terms of contrast and similarity through a grouping of colours in derivative structured patterns. Derivative colour combinations were formulated by each hue, value and chroma category. Extraneous variables of size, shape, space, position, signs, symbols and motion were constrained. As discussed earlier, the Munsell Notation System is one of a few organisational structures that provides a categorical arrangement specifically defined by the three visual attributes of colour.

5 Research stimuli and colour selection logic The stimuli for Intrinsic Colour Research Study were based on colour harmonies (colour structures). The colour harmonies consisted of three categories – hue, value and chroma. Colours selected from each of the three categories were derived from their similarities and contrasting relationships. The constructed relationships were derived from principles of the Munsell Notation System. This colour system employs a clear organisation that unites colour’s three properties. It forms the bases for objective colour organisation. Therefore, the colour selection method for this research study was based on objective logic rather than subjective or personal appeal. Assembling a colour system may be managed by simply observing the relationship of contrast among the colours’ intrinsic properties. The first category was derived from a hue with contrasting value. The second category was derived from one hue with contrasting saturation. The third category was derived from contrasting hues. Generally, the degree of contrast needed to be decisively noticeable within each property. As a result, colour categories were formulated with objective criteria based upon the three variable properties of colour. The value contrast category comprised colour variables that were constrained in chroma and hue, no contrast. The chroma category comprised colour variables that were constrained in value and

204

Dennis Puhalla

hue, no contrast. The hue category consisted of four contrasting relationships. Therefore, the colour stimuli in the experiment were controlled. Other variables needed to be constrained. The additional constrained variables included participants selected, sample size, setting, equipment, and typeface. The goal of the study was to produce quantitative results for task effectiveness and task efficiency. Task effectiveness was the participants’ response to the colour stimuli itself. Task efficiency measured the participants’ lapsed time or the time needed to recognize a colour hierarchy. Participants in the experiment were asked to view and respond to colour samples (stimuli) within each of the three categories – hue, value and chroma. Each of the three categories was comprised of six colour structures. Participants were presented three colours at a time. Figure 6, Colour categories and organisation, illustrates the three colour organisation strategy and sequence.

Fig. 6: Colour categories and organisation.

The colour combinations were digitally constructed and viewed on a monitor in a controlled setting. The “value” structures consisted of six colours that were derived from the primary (red, yellow, blue) and secondary (orange, green, violet) colours. Similarly, the “chroma” colour structures consisted of six colours that were derived from the primary and secondary colours. The rationale for selecting the primary and secondary hues was based on their light to dark value dimension within the visible spectrum. The “hue” structure was formulated on variable value and chroma configurations of contrast.

6 Intrinsic colour structures research findings The data was compiled by applying specific statistical formulas to analyze effectiveness and efficiency. Compelling quantitative data analysis proved the hypothesis to be true. When colour is organized according to its intrinsic properties, it establishes a sensible visual language that communicates distinctly, independent of emotion and personal colour preference. The data summaries revealed a strong preference for chroma/saturation contrast. Participants selected the most saturated colour within each of the six struc-

Colour language hierarchy

205

tures as shown in Figure 6. However, the time variance analysis within the chroma category verified significant differences in response time. It is not certain what contributed to the time variance within the saturation category. One might speculate the time variances within the saturation category are related to particular hues, since they were comprised of six different hue families. This appears to suggest that some hue families are not as convincing and decisive as others. The value structures detailed a fluctuating response rate from the participants. Among the value structures, participants selected the darkest contrasting colour. Within the value category, there was a relatively similar response time to determine the dominant colour stimulus. The hue category consisted of five contrasting variable structures. Given the composition of the five contrasting categories, participant response to stimuli demonstrated a pattern that recognized contrast in all three variables (hue, value and chroma). However, one of the six hue structures followed a different composition structure. Purposefully, this structure was composed of contrast in hue with no contrast in value and saturation. It was hypothesized that the response to this structure would be erratic and inconsistent. This proved to be true. Forty percent of the participants recognized no one colour to be dominant. This finding is significant. Contrast of hue alone was not enough to discern a clearly dominant colour. Demonstrating indecisiveness among the participants, the timed reaction to this particular colour structure was significantly greater than the other seventeen combinations. It is clear that this colour combination did not demonstrate a hierarchical order. Apparently, the participants’ selection of a colour in this combination was based on personal preference not hierarchical structure. The remaining five hue categories were consistently similar in colour selection and response time. Subsequently, when two colours were similar in value and saturation contrast and the third darker and more saturated, a significant percentage of the participants were in agreement. Namely, the dominant colour was the darker more saturated one.

7 Research inferences The purpose of this study was to investigate which derivative colour combinations in hue, value or chroma communicate visual order for the purpose of meaningful visual colour language. The principles of colour selection and organisation are founded upon objective criteria originating from derivative colour combinations. Derivatives were obtained from the context within each of the three attributes of colour. The derivative colour combinations can establish visual order of importance and contribute to the human ability to understand a message. Colour can be assembled according to certain rules or “codes.” Colour and communication represent the intentions and purposes of the initiator. If users do not have a clear

206

Dennis Puhalla

understanding of colour codes, translation is more than likely invalid. Without intelligible visual recognition of colour language in application, interpretation and purpose most certainly will be lost. This Intrinsic Colour Research Study provided a rationale for the basis of understanding a process of colour structure and order. Generally, the literature in design and art accepts the proposition that there are conventional colour harmonies. However, within these harmonies, hierarchical structuring is not readily clear or obvious. The harmonies alone do not produce an intrinsic visual order. For example, a primary colour harmony composed of unaltered red, yellow and blue does not in itself establish a visual hierarchy. Many individuals most likely would respond to the colour red as the most dominant. This reaction is probably based on the individual’s cultural background, symbolic interpretation and/or personal preferences. Some might respond to the blue colour for the same reasons. However, if these same colours were altered by value and saturation levels, one colour could be made decisively dominant, as found in the Intrinsic Colour Research experiment. By analysing the hue, value and chroma scheme of a primary harmony or any other fundamental harmony, colour-coding and categorization are possible. It is the derivative colours within a harmony that establish order and hierarchy. This is accomplished by controlling the contrasting relationships of colour’s properties. Therefore, hierarchical structuring within these schemes must be constructed. Managing the ordinary relationships found within these colour harmonies in relationship to hue, value and chroma is the key to establishing a visual hierarchical order.

8 Colour harmonies Models of harmonious and inventive colour combinations are commonly known. It is not unusual to find some of these harmonies integrated into some imaging software programs. From these harmonious models, a multitude of colour combinations are possible. Managing the properties of colour within these harmonious systems, enables a consistent visual colour language that is capable of producing whichever type of decisive or expressive response required. Therefore, the development of a colour language strategy uses a method that begins with the conventional colour harmonies. Colour harmonies are traditionally defined as having an internal orderly structure comprised of a dynamic equilibrium between homogeneous and heterogeneous parts. Conventional colour harmonies are commonly known to be primary, secondary, tertiary, split complementary triad schemes; as well as complementary, analogous, monochromatic and divergent schemes. Monochromatic harmonies are derivative colours from one hue family. For example, a monochromatic harmony consists of the many colours associated with

Colour language hierarchy

207

Fig. 7: Monochromatic colour derivatives from white and black.

Fig. 8: Monochromatic derivatives from neutral grays.

Fig. 9: Primary, secondary and tertiary harmonies.

a blue hue – light, dark; bright, dull. These colours can be formed in several ways as long as the colours remain in the same hue family. One hue modified by white or black constitutes a value structure creating light dark contrast as shown in Figure 7, Monochromatic derivatives from white and black. Altering a hue with a neutral gray also forms monochromatic harmonies. Many desaturated colours can be derived from differing grays – varying degrees of white and black mixtures. Once again, the value and chroma structures consistently reside within the same hue family. Therefore, the various colours may be classified as a monochromatic harmony Figure 8, Monochromatic derivatives from neutral grays. In the value structure, the homogeneous part is the hue family and the heterogeneous parts are the contrasting value and/or saturation.

208

Dennis Puhalla

Fig. 10: Analogous harmonies.

Fig. 11: Complementary harmony.

Fig. 12: Split complementary harmony.

Primary, secondary and tertiary triad harmonies are illustrated in Figure 9. Each of these triads is equally spaced on the RYB colour wheel, thereby forming a colour triad. As demonstrated in Figures 7 and 8, each of the hues within these triads can generate an extensive number of monochromatic harmonies by altering the degrees of contrast in value and saturation. Analogous harmonies are adjacent colours on the colour wheel. They may encompass neighboring colours of two hues, Figure 10, Analogous harmonies. In this example, the analogous colours orange and orange-yellow create additional colours derived from their mixture. Simultaneously, the two colours are mixed with a neutral gray as shown in the bottom row. Complementary colours are opposite on the colour wheel, Figure 11, Complementary harmonies. The colour derivatives in the complementary scheme produce gray. The gray colour will be different for each complementary pair because the gray is derived from two hues rather than a mixture of white and black. The example also illustrates a range of light and dark colours derived from the mixture of red and green complements. Split complements are formed by three hues. In this case, the complementary pair as shown in the previous example is altered. Figure 12, Split complementary harmony, exemplifies a yellow hue opposite two hues rather than its complementary colour purple. Derivative colours generated by the split pairs will generate brown colours with low saturation levels. A divergent colour harmony is based upon one of the conventional harmonies. It is distinguished by one hue differing in its customary or expected location. In

Colour language hierarchy

Fig. 13: Divergent colour harmony.

Fig. 14: Derivatives of a tertiary harmony.

209

210

Dennis Puhalla

Figure 13, Divergent colour harmony, one colour in each of the conventional harmonies is skewed. For example, as discussed previously, red, yellow, blue constitute a primary harmony. As illustrated in Figure 13, the colour red is skewed toward red-orange in the primary harmony. From these conventional harmonies, countless derivative colours can be structured to convey a consistent colour language with or without a hierarchy by manipulating the value and saturation. Managing the contrasting relationships within the colour harmony can evoke a wide range of expressive and poignant visual effects. The inherent structure of the colour wheel and perceivable attributes of colour provide communicative colour combinations. These colour formulations are at the heart of the chromatic displays found in nature and serve as the root of effective colour structures. Innumerable colour configurations can be formulated by the conventional colour harmonies. Figure 14, Derivatives of a tertiary harmony, explains how hue, value and saturation derivatives are formed from a tertiary colour harmony. Tints, shades and tones can be organized within the tertiary harmony in a number of ways that offer varying degrees of contrast. The example presents a limited number of thousands of possibilities.

9 Colour language hierarchy Using the derivatives from the tertiary harmony shown in Figure 14, a system of order can be visually attained. A stratified organisation of colour parts is accomplished by altering the value and saturation levels within the tertiary harmony. The integrity of the colour harmony is not compromised and the communicative potential is maximized. Figure 15, Hierarchy structures formed by tertiary harmony, is an example of three different hierarchical structures. The first grouping of three colours represents one dominant colour while the other two colours reside in equilibrium. Both of these two colours are subordinate to the dominant colour. This visual phenomenon is caused by the value and saturation similarity of the two subordinate colours and strong value and saturation with the one principal colour. The second grouping of colour reveals a three-tier structure. Each of the three colours is contrasting in hue, value, and saturation at three differing levels. Therefore, the blue

Fig. 15: Hierarchy structures formed by tertiary harmony.

Colour language hierarchy

211

is first in visual importance; the orange second in visual importance; the green last in importance. The third grouping of colour represents a state of evenness caused by the similarity of value and saturation within each of the three contrasting hues. In this example, no one colour is more visually important than another.

10 Colour: context and content Colour does not function in a vacuum. The matter of visual communication is made even more complex as other attributes of form are added. The other attributes of form include size, shape, texture, proportion, orientation and position. Additionally, biological, physiological, psychological, social, culture and colour symbolism play a significant role. However, when individuals respond to visual stimuli coded by the categorical representation of hue, value and chroma, meaning is constructed objectively. These categorical representations are means in which objective visual communication forms a language and delivers a message. The derivative colours contained within a harmony form the syntax and semantics of colour’s visual language. Hue, value and chroma are the essential parts that bring visual meaning to an object. The organisation of a colour composition elicits a perceptual image of an object. Cognitive processing assigns meaning in response to the manner in which colour is organized and presented. Understanding the role of colour as a means to communicate perceived order and importance has useful applications to a naturalistic, real world display of visual information systems. One example of a real world colour application includes visual recognition tasks. These are the tasks in which an individual looks at the context of colour organisation as a means of ordering information to determine relevance. Colour adds significance to visual communication as it accentuates and makes perceptible the identification of that which is perceived first, as the main idea, before decoding additional elements in the message. Perceived hierarchy represents a sequence of visual guides in the language of colour. For example, most weather maps display a poorly transmitted colour language. They are often confusing and difficult read without referring to a colour legend. Another example is found in some visual communication software applications. Templates are provided for the user. Often the templates contain colour pallets that appear to have no specific logic other than personal preference for certain colour combinations. On the other hand, the London underground transportation map is an excellent example of a clean graphical system that incorporates colour effectively, as displayed in Figure 16. In this case, colour adds functionality. It provides welldefined information about a complex operation. By focusing attention on colour’s intrinsic properties, criteria for colour selection become objective, not subjective. In other words, selection methods are based

212

Dennis Puhalla

Fig. 16: London underground transportation map.

upon the contrasting relationships of colours derived from hue, value and chroma that are found within a specific colour harmony. This method of colour selection provides the framework of a clear, empirically defined colour combination system. Effective colour communication permits users to view, retrieve, access, decipher, interpret, understand, and experience a variety of information systems in a meaningful, valid and reliable manner. Understandably, it is acknowledged that colour choices are often based upon trends, styles, emotion and tradition. However, the colour selection strategies founded upon objective criteria provide opportunities for clear, intelligible information processing. As illustrated by the findings of the Intrinsic Colour Structures Study, the natural/intrinsic ordering of colour within the categories of hue, value and chroma provides a guiding principle of fact for the purpose of useful colour selection methodology. The context of colour’s composition delivers informational content. When the principles of colour organisation functions as a unified whole, a colour language is formed. These principles of fact are rooted in simple, common and ordinary relationships of colour’s properties and harmonies. Usually, observing and understanding the simple/ordinary state of things is not a simple/ordinary task. If colour is to function as a coherent language, purposeful visual observation and analytical thinking are absolutely indispensable.

Colour language hierarchy

213

References Albers, Josef. 1975. Interaction of Colour. New Haven/London: Yale University Press. Berlin, Brent and Paul Kay. 1991. Basic Colour Terms: Their Universality and Evolution. Berkeley: University of California Press. Dondis, Donis A. 1973. A Primer of Visual Literacy. Cambridge: MIT Press. Favre, Jean-Paul and Andre November. 1979. Colour and und et Communication. Zurich: ABC Zurich. Gobé, Marc. 2001. Emotional Branding; The New Paradigm for Connecting Brands to People. New York: Allworth Press. Hardin, Clyde. 1988. Colour for Philosophers: Unweaving the Rainbow. Indianapolis: Hackett Pub. Co. Hardin, Clyde. 1992. The Virtues of Illusion. Philosophical Studies 68: 371–382. Kay, Paul and Maffi Luisa. 1999. Colour Appearance and the Emergence and Evolution of Basic Colour Lexicons. American Anthropologist 101: 743–760. Lakoff, George. 1990. Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. Chicago/London: The University of Chicago Press. Munsell, Albert and Alex Munsell. 1979. A Colour Notation. Baltimore: Munsell Colour. Puhalla, Dennis. 2008. Perceiving hierarchy through intrinsic colour structure. Visual Communication Journal 7(2): 199–228. Puhalla, Dennis. 2005. Colour as cognitive artifact: A means of communication, Language and message. Dissertation, North Carolina State University. Rosch, Eleanor and Barbara Lloyd. 1978. Cognition And Categorization. Hillsdale: Lawrence Erlbaum Associates. Tufte, Edward R. 2001. The Visual Display of Quantitative Information. Cheshire: Graphics Press. Tufte, Edward R. 1990. Envisioning Information. Cheshire: Graphics Press.

Mary C. Dyson

9 Applying psychological theory to typography: is how we perceive letterforms special? Abstract: One debate among psychologists is whether we have a few general mechanisms that we apply to our various activities, or more specialized mechanisms that have adapted to our particular needs. My research looks for parallels between our perception of letters and typefaces (by readers and designers) and other perceptual activities. To date the work has drawn on research into the perception of speech, faces and music. We are all experts in speech and face perception; similarly the fact that we can all recognize letters regardless of the typeface makes us experts in letter perception. But musical training changes perceptual abilities. Based on the assumption that typographic design training has a similar outcome, my research is trying to characterize the nature of designers’ perceptual skills. In order to explore the parallels, the work adapts the experimental methods applied to the original psychological studies.

1 Summary My research looks for parallels between our perception of letters and typefaces (by readers and designers) and other perceptual activities. Because I am concerned with perception, I have drawn upon examples of research into other areas of perception, both visual and auditory, i.e. the perception of faces, music, and speech. These research studies suggest to me avenues to explore in relation to how we perceive visual forms, and also stimulate ideas concerning particular methods of investigation. This approach leads to novel experiments within the field of typography. As a teacher, I am interested in how we train students in the visual discriminations that are required of typographers and what characterizes typographic expertise. But I am also concerned with the more general question of how we, as readers, recognize letters regardless of the typeface which changes their visual form. Although readers are not typographic experts, they are experts in letter perception, just as listeners are experts in speech perception. How do we process letters and how might typographers’ perceptual skills differ from those of readers? The outcomes of this research are intended to have theoretical implications, contributing to models of reading, but also to have practical applications: a better understanding of reading could inform the teaching of reading and reading materials; gaining insight into how we process typefaces could feed into their design; identifying the nature of typographers’ perceptual skills could facilitate training. Overall, my aim is to understand how we perceive letterforms.

216

Mary C. Dyson

2 Introduction 2.1 Why this question? One debate among psychologists is whether we have a few general mechanisms that we apply to our various activities or more specialised mechanisms that have adapted to our particular needs. General mechanisms provide a simpler, more parsimonious explanation of how we perceive the world and can be intuitively appealing. For example, my PhD thesis looked for an explanation as to why we notice the peaks and troughs in a melody (the high and low notes) through a visual analogy. Isn’t it likely that the extremes of a visual display also stand out and we might therefore apply a similar process when listening and looking?1 It would seem to be more efficient for us to re-use perceptual mechanisms in different circumstances, rather than switch depending on the particular activity. But the alternative argument is that there may be activities that require more specialization and particular skills may be acquired, such as learning to recognize different faces or understanding speech. Within these areas of research typical questions are “Is face processing special?” or “Is speech special?” So have we developed perceptual mechanisms that are different from those we use for processing objects in general or listening to other sounds? In my work, I am considering these two explanations of how we perceive variation in typography. Is the way in which readers process typefaces special, or are there parallels in other forms of perception? In addition, I am exploring how typographic expertise may be distinct from letter expertise, i.e. how to characterise perceptual abilities resulting from typographic design training. Quite a few of my studies therefore recruit students from the Department in which I teach2 in order to explore the effects of training in typographic design. For example, do typographers perceive typefaces in a different manner to non-typographers? If they do, this may have implications for how we may more effectively and efficiently teach typography and encourage the development of particular ways of analyzing and discriminating visual characteristics of text. Typographers will also need to be aware that their perceptions may not match those of the users of their designs and they must therefore take account of this, e.g. through user testing. My research was initially motivated by encountering models of reading and experiments conducted to test such models that appeared to avoid explanations of how variations in fonts3 that are typical of our normal reading material are  Although this hypothesis was not supported by experimental evidence, the general idea of drawing on research from both visual and auditory perception has remained with me and pervades my more recent research.  Department of Typography & Graphic Communication, The University of Reading, UK.  The term “font” is used to refer to variations in the visual form of letters. Font traditionally refers to a particular size and style or variant (e.g. italic) of a typeface family (e.g. Times), but has taken on a more general meaning with word processing software etc. The psychological literature cited

Applying psychological theory to typography

217

handled. For example, McClelland and Rumelhart's Interactive Activation Model (1981: 383) describes detectors for visual features of letters, and assumes that “the basic results do not depend on the particular font used” but also acknowledges that this “skirts several fundamental issues about the lower levels of processing”. These observations on the lack of attention to fonts may be regarded as the “natural inclination of a person to think the details of his craft matter” (Oden 1989: 145). This may be true, but a more compelling argument for pursuing my research has been to further understand one component of the complex skill of reading: the early stages of reading, where the lower level of processing occurs. To recognise a word, a reader must have processed the letters to create abstract letter identities (Besner et al. 1984; Grainger et al. 2008). A skilled reader recognises most words within a fraction of a second despite the letters being in different fonts (Rayner and Pollatsek 1989: 3). With relative ease, we translate variant visual forms (such as different fonts and cases) into invariant representations (Figure 1).

Fig. 1: Letters in different fonts are converted into a single letter identity.

2.2 What informs my approach? The means by which we recognise letters in different fonts has been researched and is described as “font tuning”. This process whereby the particular characteristics of a font can be used to facilitate letter recognition was proposed and extensively explored by Sanocki (1987, 1988a, 1988b, 1991a, 1991b, 1992). Fonts generally have a consistency in their design and Sanocki suggests that the perceptual system can become tuned to a particular font over time and develop a set of font parameters. These parameters are used to transform font-specific input into font-invariant form. Sanocki conducted a series of experiments to address how variations in visual forms are translated into font invariant forms (Figure 1) and produced the reliable finding that mixing fonts leads to less efficient letter identification than

typically uses the word font and I therefore adopt the terminology of the studies when referring to them. I use typeface when introducing more generic issues within typography as this also reflects normal usage.

218

Mary C. Dyson

using the same fonts. These results have been variously described by Sanocki as a “font-regularity effect” (1987) and “font-consistency effect” (1991b). A similar skill to font tuning is perceiving and understanding speech from many different talkers with considerable variation in the acoustic properties of speech (Martin et al. 1989: 676). The acoustic pattern of a vowel from one talker overlaps with the acoustic pattern of a different vowel spoken by someone else. Yet we are able to identify vowels correctly across different talkers. Although commonly described in terms of the opposite effect (i.e. variability), a similar regularity is found in vowel identification when mixed talkers are compared with a single talker (e.g. Assmann et al. 1982; Mullennix et al. 1989). If the talker is fixed, vowel identification is improved. In looking at how we process letters, I am asking whether there are specialised skills or the application of more general visual and auditory perceptual mechanisms. My research objectives are therefore theoretically oriented, but could have a range of practical applications in for example, informing the design of material for beginner readers, more generally improving ease of reading, or training typographers (as described above).

2.3 How I research the question 2.3.1 Rationale The methods that I have applied to looking at how we process letters and typefaces draw upon psychological techniques that are designed to generate reliable and objective measures of the processes, and that remain broadly within the paradigms used for related research (e.g. face perception). This approach to revealing our perceptions is necessary because we are not able to accurately report what may take place in less than a second, or we may not be consciously aware of the strategies we use in judging visual material. Adopting a scientific approach to the study of typography complements the craft knowledge of designers and more applied research. Over time, I have developed a desire to bridge the gap between disciplines (e.g. Dyson 1999a, 1999b; Dyson et al. 2000) aided by a better understanding of both “sides”. I have done this from within typography inspired by the work of typographically aware psychologists (Prof Pat Wright 4 and Prof James Hartley5).

 Professor Patricia Wright – BSc PhD Lond MA Cantab FBPsS CPsychol exFISTC Hon FSTC http:// psych.cf.ac.uk/contactsandpeople/honorary_staff/wright.html [Accessed 29 July 2012].  Prof James Hartley http://www.keele.ac.uk/psychology/people/hartleyjames/ [Accessed 29 July 2012].

Applying psychological theory to typography

219

2.3.2 Materials I conduct experiments that are computer-based so that I can control and record all aspects of the procedure. All material is displayed on screen; the content does not form meaningful text as I use either single paragraphs of dummy text material, or single strings of letters. To create paragraphs of dummy text I use a freely available dynamic text tool which generates random sequences of words.6 Some of the experiments require judgements of fonts; others require letters to be recognised. Where fonts are being judged, words may be recognisable (see Figure 9a), although some may appear unusual as low frequency words and proper nouns are included. When I am looking at letter processing, this needs to be separated from other aspects of the reading process (e.g. word recognition or sentence comprehension). Using random letter strings removes the semantic (top-down) influences.

2.3.2 Tasks Of the four studies described in this chapter, two use an identification task which consists of saying whether a paragraph of dummy text or a string of letters (Hamdurefonsiv)7 is most like one font or another font, i.e. a form of categorisation. The participant is provided with examples of the two fonts (to show what they look like) and must then repeatedly select which of these two is closest to each of the examples they are shown. I use this task for two purposes: to identify the boundary of the font categories, i.e. where there is a switch from one font to another; to explore whether this boundary changes with prior exposure to one or other of the fonts. These results provide some insight into how typeface characteristics are perceived and the effects of context on this perception. Three of the four studies include a discrimination task where two paragraphs or two letter strings are judged to be either the same or different. The studies involve two types of judgements: whether the two sets of material are presented in the same or a different font; whether two letter strings contain the same letters or different letters (where one letter has been changed). Figure 3a illustrates an example of different letter strings and Figure 3b, same letter strings. I use this task to further explore how typeface characteristics are perceived, and whether typographers’ judgements differ in a qualitative way from those of non-typographers. This task is also designed to reveal whether we make use of typeface characteristics when we identify letters.  Adhesiontext™ is available at http://www.adhesiontext.com/ [accessed 3 January 2012].  This is a variation on Hamburgerfontstiv, hamburgevons, etc, which are keywords used when creating a font as they contain the basic strokes used in the Latin alphabet, including stems, diagonals and rounded strokes.

220

Mary C. Dyson

In some of my studies described below, the length of time that the material is presented on screen (the viewing duration) is critical. This is because the task must not be too easy or too difficult to be able to demonstrate the effects I am interested in. The viewing duration is therefore set in two ways: (i) through pilot studies or (ii) an adjustment stage prior to the main experiment. In the latter case, I am aiming to tailor the experiment to the particular participant and set a level that is reasonably comfortable for them.8 In addition to varying the viewing duration, I also can vary the degree of difference between the two sets of material; a larger difference should make the task easier. Participants receive feedback on the accuracy of their responses after each judgement in the form of a tick or cross which hopefully helps to familiarise them with the nature of the task.

2.3.3 Measures of performance With the identification task, responses are summarised by calculating the percentage identified as one particular font.9 The discrimination task uses a rating scale from “sure same”, “same” to “different”, and “sure different”. I encourage participants to use all four responses and to distinguish between responses where they are more confident and those where they are less certain. The analysis of these responses is based on Signal Detection Theory (Green and Swets 1966) which when applied to psychology concerns making perceptual decisions based on equivocal evidence (McNicol 1972: 10). When we are asked to make a decision as to whether two strings of letters, presented in quick succession, are the same or different letters, we are unlikely to be certain of our answer. Signal Detection Theory can model this uncertainty by looking at a range of responses, providing us with a more valid measure of our perceptual processes. The measure I use is p(A)10 which is an index of discrimination ranging from 0.5 (guessing) to 1 (perfect discrimination); this is a gauge of sensitivity. This measure is separate from another aspect of decision-making which is bias, where one response may be favoured over another. For example, if a participant finds it easier to say that two sets of material are different than to say they are the same11 they

 There is quite a large variation across participants in the viewing durations that generate performance between chance (guessing) and perfect performance. These differences are likely to be influenced by a range of factors such as familiarity with the task, confidence, etc. rather than perceptual ability per se.  The percentage identification of the other font is the mirror image of this data.  This stands for the proportion of area under an ROC (Receiver-Operating Characteristic) curve. See McNicol (1972) for further explanation.  This is a plausible scenario as spotting anything that differs can prompt a different response, whereas determining that everything is the same may be more difficult.

Applying psychological theory to typography

221

may be biased towards different responses and give more “sure different” responses than “sure same” responses. Their criterion for deciding to respond same may be stricter than for deciding on different. Consequently, they may give the response “different” to some cases where the two sets are the same. If we were to calculate the percent correct responses (which is typically done in psychology experiments), we would naturally count the “different” response to same material as incorrect. But if this participant responds “different” to same material and “sure different” to different material, they are discriminating between same and different but their bias (strict criterion) is interfering with their judgments. Calculating p(A) avoids this interference and therefore provides a better measure of discrimination because it removes any differences in criterion among participants and within a single participant over time. Reaction time is commonly measured by psychologists as an indication of whether one condition is easier than another. In my font tuning experiment described below, I have included the time taken to respond same or different because the majority of evidence for the effect I am looking for is from reaction time data. However, this measure can be affected by bias and this cannot be removed. Using the same example as above, a participant may respond more quickly when making a different response because they need accumulate less evidence. Similarly, a participant may decide to take more care when needing to compare two strings of letters in different fonts than two strings in the same font so their responses are slower. For this reason, I use reaction time data only as a secondary measure.

2.3.4 Participants When recruiting typographers for my studies, I ask for volunteers from final year undergraduates on our Graphic Communication programme, Masters students in Typeface Design, Book Design and Information Design, and PhD students and reimburse them for their time. The studies are submitted for ethical review according to the procedures specified by the University of Reading Research Ethics Committee and are given permission to proceed. In line with the ethical demands of the studies, it is made clear to participants that the study, their participation and the results are not part of any assessment on their programmes. Non-typographers are recruited from within the University of Reading and are also typically students. Whilst students may not be representative of the general population in many respects, I believe that the processes they employ in perceiving letters are likely to reflect those of most adult readers (of the Latin alphabet). In one study described below, I compare typographers and non-typographers with the aim of distinguishing between processes used in reading and those used to analyse typefaces.

222

Mary C. Dyson

2.3.5 Critique of my methods My approach, which uses artificial test conditions, is not always well received by typographers. Although not necessarily made explicit, criteria that are commonly used by typographers to evaluate empirical research within the field are: (i) the extent to which the experimental materials and methods approximate to normal reading practices; (ii) how closely the results fit with what designers know about visual forms. The former view is likely to be shared by applied psychologists but my work has a more theoretical orientation. This requires a means of isolating the components of the reading process, as described above, to ensure the research has internal validity, even though it may not have ecological validity. Questioning the reliability of the results if they appear surprising is a positive reaction that I encourage. A good scientist questions findings that do not fit with previous knowledge, and investigates further, trying to replicate what they have found. If my results run counter to designers’ tacit knowledge, I believe further checking is required.

3 My studies 3.1 Font tuning This work tries to directly address how typefaces fit into models of the early stages of reading, where the lower level of processing occurs (as described above).

3.1.1 Updating font tuning studies The original research into font tuning was started about twenty five years ago, but there has been a minor revival of work (Gauthier et al. 2006; Wong et al. 2005; Wong and Gauthier 2007; Walker 2008). This later work has been able to use more sophisticated fonts than the basic shapes12 used by Sanocki, due to the proliferation of digital fonts now available to all PC users. A recent review by Sanocki and Dyson (2012) also explores the evidence for font tuning and how this meshes with type design principles. My research into font tuning takes a similar line to the above authors in looking at whether we tune to current digital fonts, which are obviously of a higher visual quality than the fonts created by Sanocki (Figure 2a), but are also more similar to each other in basic shapes (Figure 2b). I use my discrimination task, which differs from the methods used by all the previous researchers.  These were constrained by the computer technology of the time to a height of 20 pixels and width of 8 pixels.

Applying psychological theory to typography

Fig. 2a: Fonts used by Sanocki.

223

Fig. 2b: Fonts used by Dyson.

The participant is asked to say whether a string of letters contains the same letters as the previous string or different letters. To ensure that my participants attempt to identify all the letters, and not just recall the word or letter shapes, the order of the letters are jumbled from one string to the next. If the string contains different letters, just one letter has been changed. A visual mask follows each letter string. This is used to halt the processing of the test material (by having to process the mask) and therefore provides an accurate measure of processing time. Two typical sequences of screens are shown in Figure 3a and 3b. The comparison of interest is whether participants find it easier to identify letters (and thereby discriminate between same and different strings) if the two strings are in the same font as opposed to different fonts. As this effect seemed to be quite elusive, I tried many iterations of the above task using different pairs of fonts, different ways of introducing variation from trial to trial, and different numbers of letters in the string.

Fig. 3a: Sequence of screens in font tuning experiment showing 5-letter strings where the second string is different from the first and displayed in a different font.

224

Mary C. Dyson

Fig. 3b: Sequence of screens in font tuning experiment showing 5-letter strings where the second string is the same as the first and again in a different font.

3.1.2 Bodoni and Verdana The results shown in Figure 4 (accuracy) and Figure 5 (reaction time) come from an experiment with non-typographers which uses Bodoni and Verdana, the two fonts shown in Figure 2b, and varies the number of letters in the strings between 4, 5 and 6. The task is understandably easier, i.e. there is greater accuracy, when there are only 4 letters in the string, but these results also indicate that the advantage for same fonts (the font tuning effect) is only found when there are 4 letters. There would therefore seem to be some limit on the number of letters that are identified

Fig. 4: Accuracy of discriminating same and different letter strings when presented in the same or different fonts and varying in length from 4 to 6 letters. The discrimination index p(A) has been transformed for statistical analysis and a perfect score is π (≈ 3.14159) and chance is π/2 (≈ 1.571). The vertical bars show the standard errors of the means which provide an indication of the amount of variation in the scores across participants.

Applying psychological theory to typography

225

Fig. 5: Speed of responding to the letter strings.

more easily by being in the same font (or are more difficult to identify through being in a different font). If we include the time that participants take to respond (Figure 5), the results confirm a font tuning effect, as responses are faster when in the same font. Although the difference seems to be largest when there are only 4 letters, this is not supported statistically. Processing two strings of letters in the same font therefore facilitates a quicker response, but the consistency in visual form only helps in identifying 4 letters. As introduced in the method section above, faster reaction time might reflect a change in the criterion for responding, e.g. when fonts are the same, there is greater confidence in initiating a response. The accuracy results are not affected by this bias and therefore should provide a more reliable measure of how letters are identified. However, the additional processing required for 5 or 6 letters significantly reduces accuracy. Although performance is still better than guessing, the difficulty of the task may have prevented the consistency in fonts from helping letter identification.13

3.1.3 Evidence for font tuning From these results, I can infer that font tuning occurs with two text fonts in current use, but the effect is particularly sensitive to aspects of the task such as the level of difficulty. The method of adjustment I used attempted to control this by varying the duration of the letter strings, but a compromise was necessary in finding one duration for 4, 5 and 6 letters. Given the greater difficulty with more letters, this compromise may not have been suitable.  This is referred to as a floor effect, which can result from performance near to the lowest score (in this case chance performance at 1.571).

226

Mary C. Dyson

I have subsequently demonstrated font tuning in typographers using Garamond and Bodoni in a more complicated experiment with 4-letter strings which involved judgements of letters (as in this study) and of fonts. Sanocki’s theory may therefore still be relevant to current fonts, and our perceptual system can become tuned to a particular font, but there may be particular circumstances that elicit this response. Walker (2008) proposes that readers may be able to exercise strategic control over the use of font parameters based on the likelihood of the font changing.

3.2 Categorical perception Font tuning aims to explain how readers process typefaces. The next study explores typographic expertise.

3.2.1 Cross-modality and within-modality comparisons A special status for speech perception was originally attributed to the occurrence of categorical perception with speech sounds, but not non-speech (Rosen and Howell 1987: 118). Categorical perception is a psychophysical phenomenon whereby we perceive categories where none physically exist. Although the initial work identified phoneme boundaries from an acoustic continuum (Liberman et al. 1957), categorical perception has now been demonstrated in many visual domains. In pursuit of some general principles of perception, Bornstein (1987) lists evidence for parallels between visual and auditory categorical perception. A classic example of categorical perception in the visual domain is colour perception where we perceive discrete hues rather than gradually changing ones (Bornstein 1987). Figure 6 illustrates the colour spectrum which is a continuum of light frequencies, which we categorise into blue, green, red, etc. Colour perception corresponds to changes in a single dimension (the wavelength of light) but categorical perception also occurs when there are multidimen-

Fig. 6: Colour spectrum representing a continuum of light frequencies, categorised as discrete hues.

Applying psychological theory to typography

227

Fig. 7: Some of the many ways in which a sans-serif typeface can differ from a serif typeface.

sional changes. An example of this is categorical perception of faces (Beale and Keil 1995; Campanella et al. 2003). Faces vary in the size and shape of features (eyes, nose, mouth) and also the relative distance between them. Typefaces also vary along multiple dimensions:14 weight, contrast, proportions, basic shapes, terminals and serifs (Baines and Haslam 2005). Figure 7 illustrates these differences. The ability to distinguish among typefaces and recognise categories of typefaces (e.g. Humanist, Garalde, Transitional, Didone)15 or identify individual typefaces is part of typographic training. The influence of training on categorical perception has been demonstrated in the field of music. Trained musicians show categorical perception of musical intervals and those without musical training do not (Burns and Ward 1978). I therefore chose to investigate whether students with some education in typographic or graphic design might perceive typefaces categorically (Dyson 2011).

3.2.2 Creating test material Because faces and typefaces do not have a natural continuum, i.e. they do not exist as continuously varying shapes in the real world, creating appropriate material for testing is challenging. The interpolation procedure needs to produce a linear continuum with equal steps between points on the continuum, which is not straightforward with multidimensional stimuli (Newell and Bülthoff 2002). A 12 This variation is a classic problem when investigating the relative legibility of different typefaces. If the aim is to evaluate which typefaces are better for reading (i.e. the practical implications of the research findings outweigh the theoretical understanding) then using existing typefaces, which differ in various ways from each other, is a valid approach. However, to determine which features of typefaces influence legibility requires manipulation of individual typeface features. This is now relatively easy to do, but the validity of this approach is questionable from a design perspective. There is an interrelationship of elements in a well-designed typeface, within and among letters. For example, if the serifs are removed, the modulation between thick and thin elements may need adjusting. Legge (2007: 113) refers to this as the “underlying design rationale”. This means that standard psychophysical methods employed by psychologists and vision scientists, whereby only one aspect of a stimulus is changed, are problematic.  British Standard 2961: 1967, Typeface classification and nomenclature.

228

Mary C. Dyson

Fig. 8a: Continuum from Times (1) to Helvetica (12).

Fig. 8b: Continuum from Garamond (1) to Bodoni (12).

step continuum from Times New Roman to Helvetica (Timevetica) is illustrated in Figure 8a and from Garamond to Bodoni (Garadoni) in Figure 8b. Timevetica was chosen as the two ends are common examples of a serif and sans-serif typeface;

Applying psychological theory to typography

229

Garadoni uses two serif typefaces which avoids some of the difficulties in the transition from serif to no serif. These continua were developed and modified by trained type designers as the results of the interpolation must be a set of plausible fonts. Equal steps are necessary as a hallmark of categorical perception is that physically equal steps are not perceived as equal. Equal physical differences are perceived as larger or smaller depending on whether or not they are put into the same category.

3.2.3 Testing for categorical perception Both identification and discrimination tasks are used to test for categorical perception. Participants in my experiments are asked to identify examples from along the continuum, saying whether a paragraph is most like Times or Helvetica (Figure 9a) or most like Garamond or Bodoni (Figure 9b). A category boundary is determined where half the responses are from one end of the continuum and half from the other (Figure 10). For Timevetica, the boundary is around step 7 on the continuum (when averaged across eight participants). Discrimination is measured by comparing two examples which may come from the same point on the continuum or from different points (Figure 11) and participants say whether the paragraphs are in the same font or a different font. If participants find it harder to tell the difference between two fonts from one side of the category boundary, as compared to two fonts that straddle the boundary, there is

Fig. 9a: Paragraph near mid-point of Timevetica continuum (step 6).

Fig. 9b: Paragraph near mid-point of Garadoni continuum (step 6).

230

Mary C. Dyson

Fig. 10: Percentage of responses as Times and Helvetica (mirror image) for each step on the Timevetica continuum.

Fig. 11: Sequence of screens in discrimination task (from step 4 and 6 on Timevetica continuum).

Fig. 12: Accuracy of discriminating pairs of fonts separated by 2 steps.

evidence of categorical perception. The results for Timevetica (Figure 12) show that discrimination is better when the pairs are 5&7, 6&8, 7&9. These pairs are around

Applying psychological theory to typography

231

the category boundary (7) and contrast with the poorer discrimination of pairs within a category (e.g. 1&3, 2&4 at the Times end and 9&11, 10&12 at the Helvetica end).

3.2.4 Categorical perception of typefaces through training So there is some evidence that fonts are perceived categorically by people who have been trained to attend to differences among typefaces. The training presumably increases perceptual sensitivity to those differences that are relevant to the categorisation (see Figure 7) leading to more efficient discrimination. As I did not carry out the same experiment with non-typographers, the question remains as to whether readers without typographic training develop similar categories through exposure to common typefaces. Readers do not normally have experience in attending to font information (Gauthier et al. 2006).

3.3 Adaptation to specific fonts Because normal reading does not require categorisation of fonts, I return to how fonts may be processed when reading taking a different approach to that of font tuning.

3.3.1 Use of font information Because we process letters by forming abstract letter identities (Figure 1), we should decrease our sensitivity to differences that are not relevant to categorising the letter’s identity. But in some contexts preserving information about the visual form of the letters, after they have been identified, may be helpful. Examples that have been proposed include recognizing an individual’s handwriting (Bruce, Green, and Georgeson 2003: 272) or spotting brand names or corporate identities (Walker and Hinkley 2003). A counter argument is that fonts have little communicative value compared with voices which convey the talker's gender, age, etc. (Goldinger et al. 2003).16

 A body of research does exist on the meaning conveyed by visual form, variously described as the atmosphere value (Ovink 1938), congeniality (Zachrisson 1970), semantic qualities (Bartram 1982), and personality (Shaikh 2007) of typefaces.

232

Mary C. Dyson

3.3.2 Effects of context in vision and audition One way of exploring whether visual form has any effect following letter identification is to look at the after effects of exposure to particular forms. Low-level adaptation occurs when there is repeated exposure to, for example, a colour, darkness, or a specific spatial frequency, and we then perceive things in a different way. If we adapt to yellow (by staring at a yellow area for some time), and then look at a neutral white area, we will see the complementary colour (blue). Adaptation can also occur at a higher level with multidimensional stimuli. Exposure to a face can produce a bias in subsequent face identity that is in the “opposite” direction17 (Webster and MacLin 1999; Leopold et al. 2001). In speech research, the identification of a test word immediately following an introductory sentence is influenced by the properties of the sentence (Ladefoged and Broadbent 1957). Using a similar idea, I explore whether exposure to a particular font might affect subsequent perceptions (in this case, identifying the font).

3.3.3 Testing for effects of context with fonts Using the continuum between Garamond and Bodoni (Figure 8b), participants with some typographic training are asked to identify examples of the letter string “Hamdurefonsiv” as most like Garamond or most like Bodoni. This stage finds the category boundary for each individual and is necessary in order to establish a baseline with which to compare subsequent perceptions. The next stage introduces a statement before each test word (in either Garamond or Bodoni) that requires a true or false response (Figure 13). This task is included to encourage reading of the statements so that the font is processed.18 Following the true/false response, participants are asked to judge whether Hamdurefonsiv is in a font most like Garamond or most like Bodoni. I found that the statement’s font has an effect on subsequent categorisation of fonts on the continuum. Having read a statement in Garamond, participants are less likely to judge a font towards the middle of the continuum (steps 5–9) as Garamond, than if they had not read the statement. Figure 14 illustrates this result by plotting the percentage of Garamond responses (across all 8 participants) for each font on the continuum (shown in Figure 8b). The line corresponding to Garamond responses, following a statement in Garamond, lies further to the left than the baseline indicating a reduction in categorising the more ambiguous fonts as  As faces do not have opposites, these experiments construct an artificial continuum of faces using interpolation (morphing).  Unlike speech, which is more difficult to ignore in an experiment, participants may not attend to what is displayed on screen. In the hope of further motivating participants to attend to the statements, I include the number of correct answers after every 36 trials.

Applying psychological theory to typography

233

Fig. 13: Sequence of screens to look at effect of reading either Garamond or Bodoni on subsequent font categorisation.

Fig. 14: Percentage of responses as Garamond for each step on the continuum before reading a statement (baseline) and after statements in Garamond and Bodoni.

Garamond. There is a much smaller (non-significant) effect when reading a statement in Bodoni, which is illustrated by the closer correspondence between the baseline and the responses following a Bodoni statement.

234

Mary C. Dyson

3.3.4 Adaptation to fonts: at what level? Having processed letters with the stylistic characteristics of Garamond, these characteristics appear to be less salient in the subsequent test word. The change in the perception or categorisation of fonts may be evidence of adaptation to a font. This may be low-level adaptation to specific spatial frequencies which reduces our sensitivity to these frequencies (Sekuler and Blake 1994: 167). Letters have a broad spatial frequency spectrum and the spatial frequency components used in identifying them vary according to the size of letters and also the font (Majaj et al. 2002). More intricate designs include higher spatial frequency components. All spatial frequencies, including relatively high ones, are necessary to resolve the edges of small features, such as the intersections of lines or terminals (Fiset et al. 2008: 1166). Alternatively, the results may be more comparable to the higher-level configural 19 after effects found with faces (Leopold et al. 2001). Face perception has been considered special because faces are recognised holistically, rather than through the individual parts (Farah et al. 1998). Furthermore, evidence from neuropsychological studies has been used to infer that a different brain area is used to process faces, compared to other objects (Farah, Levinson and Klein 1995). Face adaptation demonstrates an asymmetry between distorted and undistorted adapting images: adapting to an undistorted face has little effect on distorted faces, but adapting to a distorted face affects perception of an undistorted face (Webster and MacLin 1999). This is explained in terms of adaptation having a normalizing role; if the adapting face is fairly normal then little normalization is required and there is therefore less of an after effect.

Fig. 15: Re-creation of the more extreme distortions used by Webster and MacLin (1999).

 The nature of configural processing is discussed below in relation to inverted faces and typefaces.

Applying psychological theory to typography

235

I would not go as far as suggesting that an area of the brain has evolved to process typefaces, but an explanation of how we process different fonts to identify letters has been couched in terms of normalization of font information (Gauthier et al. 2006: 555). As Bodoni has less of an adapting influence than Garamond, this implies that less normalisation may be required for Bodoni. Whilst we probably all have a view as to what is a normal face, if compared with the types of distortions used in experiments (see Figure 15), readers and typographers would likely consider most text typefaces as normal.20 I have therefore yet to develop a convincing argument as to why there is this asymmetry in the results.

3.4 Inverting fonts Finally I look at whether typographers have developed a perceptual expertise that is not present in readers.

3.4.1 Characterising expertise One of the main reasons to attribute a special status to face recognition is the face inversion effect. Whilst most objects or scenes are more difficult to recognise when they are inverted, faces are considerably more difficult. Thompson (1980) provided a compelling demonstration of the difficulty of perceiving facial expressions in inverted faces21 (Figure 16). But the evidence that this is unique to face processing has been questioned (e.g. Diamond and Carey 1986; Valentine 1988).

Fig. 16: The illusion of normal facial expressions when viewed upside down. The effect is evident when viewing the images in the upright orientation.

 Most designers select text typefaces intended for continuous reading with regard to their legibility so unusual letterforms would generally be avoided.  Thompson increased the impact of his illusion by using an image of Margaret Thatcher.

236

Mary C. Dyson

The study by Diamond and Carey is of particular interest to me because it explores the effect of expertise on perception, though using a different task. They compared dog experts and novices on their recognition memory for photographs of faces and dogs in upright and inverted orientations. Their task requires participants to try to remember the photographs and then to pick out which ones they have seen before. Experts who specialised in the particular breeds in the photographs were impaired by inverted photographs compared to upright, whereas novices were not. As experts are better overall at recognizing dogs, the task was made easier for novices so that the overall level of performance would be similar.

3.4.2 Configural processing Although the exact nature of the processing that is disrupted when objects are inverted is not resolved, the effect is “traditionally attributed to disruption of configural/holistic processing” (Kimchi and Amishav 2010: 1035). Configural refers to the spatial relations among components and can be contrasted with analytic or the processing of parts. Holistic processing, in its extreme form, does not process parts, but the unified whole or gestalt (Amishav and Kimchi 2010: 743). However configural processing can include processing of parts, with configural information dominating.

3.4.3 Testing inverted fonts Persisting with my parallel between face perception and typeface perception, I looked at whether a font inversion effect is affected by expertise with type, acquired through typographic design training (Dyson 2012). The results from dog breeders and novices lead me to predict that typographers will be more impaired in their judgements of fonts when text is inverted than non-typographers. The reasoning is that typographers are trained to attend to stylistic characteristics of typefaces, which include the spatial relations, e.g. positioning of the thick and thin strokes. This appears to fit the description of configural processing. The task used was again discrimination of different fonts using the Garamond continuum in an upright and inverted form. The inversion flips the text along the horizontal axis, which is an inversion and mirror-reversal (see Figure 17). If reading the text, this would maintain the normal reading direction for the Latin alphabet of left to right. This type of inversion has been used in an experiment which looked at reading inverted text (Kolers 1975, 1976). As I assume that those without typographic design training will be likely to find the task more difficult in the upright orientation, the extent of the difference between fonts is varied. This ranges from 3- to 6-steps on the continuum shown

Applying psychological theory to typography

237

Fig. 17: Examples of two paragraphs in inverted form which differ by 4 steps on the continuum.

Fig. 18: Accuracy of discriminating pairs of fonts when upright and inverted for typographers and non-typographers.

in Figure 8b. So, for example, font 4 can be paired with font 7, font 8, font 9, or font 10. For each participant, an adjustment procedure determines which step size will result in performance of around 75 % (which is half way between guessing and perfect performance). This step size is then used for their experiment. The typographers are more impaired in their judgements of fonts when the letters are inverted relative to upright than non-typographers (see Figure 18). So this does support the idea that typographic training has resulted in a difference in perceptual abilities that is something more than being better at discriminating between fonts. The way in which typographers make their judgements about fonts is disrupted by inversion, but this is not the case for non-typographers. What might the inversion have done to prevent typographers from attending to the characteristics that they normally use?

238

Mary C. Dyson

Fig. 19: The types of distortions in spatial relations within letters that occur when text is inverted.

3.4.4 Use of spatial relations within typefaces In line with the argument put forward above regarding the nature of the disruption caused by inversion, typographers may process the fonts in a configural manner and use spatial relationships to make judgements, rather than just recognising components, such as serifs or terminals. The inversion reverses the pattern of contrast between thick and thin strokes and this is particularly evident in fonts from the Bodoni end of the continuum. The uneven size of counters is more apparent when inverted; the angle of stress may be changed (see Figure 19). Non-typographers usually look at letters to determine their identity and recognise words, which does require some processing of spatial relations, e.g. an L and T share components (vertical and horizontal lines) but are distinguished by the relationship between the components. But the more subtle variation in the width of strokes, or size of counter, is unlikely to be attended to by non-typographers. In contrast, by the age of 10, we all have some expertise in discriminating faces based not only on the size and shape of eyes, nose and mouth, but also the distances among them (Diamond and Carey 1986: 117).

4 Concluding remarks These studies have aimed to characterise readers’ and typographers’ perceptual abilities with a view to developing a description of how we perceive variation in

Applying psychological theory to typography

239

typographic forms. But a distinction remains between typographers’ attention to visual form and readers need to ignore visual form. Because we do not attend to fonts when reading, exploring font processing in an objective way can be a challenge. My approach has been to draw on research in other fields of perception and explore various features of perceptual experience. The reader of this chapter may be left wondering what practical use the results of these studies might have. Does it help to know that we may be able to make use of the consistency in the stylistic features of a typeface when we read? A paragraph is normally set in a single typeface so we do not need to switch between different visual forms. When there are changes, such as a word in italic or bold for emphasis, or a change in font to distinguish a heading, we may find it slightly more difficult to process the letters, but this disruption might be useful in signalling change. But a lot of on-screen reading now involves skimming texts or switching from one source or device to another. We may be expending quite a lot of effort in adjusting to the different visual forms, even if we apparently have no trouble reading. I am not proposing that everything should use the same font, but it would be helpful to know what types of variation in fonts require a re-tuning of our perceptual system and which do not. Would people who have difficulty reading or are learning to read, find it easier if they did not have to adjust to perceptually distinct fonts? My interest in typographic expertise is probably a natural response to observing and commenting on the irreversible changes that occur as students start attending to previously unnoticed visual attributes. I am intrigued to pursue the idea that there may be qualitative changes in how typographers perceive certain things, rather than just being slightly more observant. But these differences are understandable if they are the result of particular strategies for analysing visual forms that have been encouraged as part of their training. Having insight into the nature of the changes might inform techniques for teaching design. Acknowledgments: This research has received funding from the University of Reading Research Endowment Trust Fund and the Undergraduate Research Opportunities Programme which supported Sarah Nadin, Caroline Stott and Clare Leake as Research Assistants. My thanks to Clare for preparing many of the illustrations in this chapter. I have also been assisted in carrying out the studies by Joseph Thompson and José Marconi Bezerra de Souza. The typeface continua were devised and developed by graduates of the MA Typeface Design programme at the University of Reading: Malou Verlomme, Daniel Rhatigan and Jo De Baerdemaeker. Much of the work has been inspired by discussions with Anthony J Watkins.

240

Mary C. Dyson

References Amishav, Rama and Ruth Kimchi. 2010. Perceptual integrality of componential and configural information in faces. Psychonomic Bulletin & Review 17: 743–748. Assmann, Peter F., Terrance M. Nearey and John T. Hogan. 1982. Vowel identification – orthographic, perceptual, and acoustic aspects. Journal of the Acoustical Society of America 71: 975–989. Baines, Phil and Andrew Haslam. 2005. Type & Typography. (2nd edition.) London: Laurence King. Bartram, David. 1982. The perception of semantic quality in type: differences between designers and non-designers. Information Design Journal 3: 38–50. Beale, James M. and Frank C. Keil. 1995. Categorical effects in the perception of faces. Cognition 57: 217–239. Besner, Derek, Max Coltheart and Eileen Davelaar. 1984. Basic processes in reading: computation of abstract letter identities. Canadian Journal of Psychology 38: 126–134. Bornstein, Marc H. 1987. Perceptual categories in vision and audition. In: Stevan Harnard (ed.), Categorical Perception: The Groundwork of Cognition, 287–331. Cambridge: Cambridge University Press. Bruce, Vicki, Patrick R. Green and Mark A. Georgeson. 2003. Visual Perception: Physiology, Psychology and Ecology. (4th edition.) Hove: Psychology Press. Burns, E. M. and W. D. Ward. 1978. Categorical perception – phenomenon or epiphenomen: evidence from experiments in the perception of melodic intervals. Journal of the Acoustical Society of America 63: 456–468. Campanella, S., C. Hanoteau, X. Seron, F. Joassin and R. Bruyer. 2003. Categorical perception of unfamiliar facial identities, the face-space metaphor, and the morphing technique. Visual Cognition 10: 129–156. Diamond, Rhea and Susan Carey. 1986. Why faces are and are not special – an effect of expertise. Journal of Experimental Psychology: General 115: 107–117. Dyson, Mary C. 1999a. Typography through the eyes of a psychologist. Hyphen 2(1): 5–13. Dyson, Mary C. 1999b. Typography's contribution to the design of human-computer interfaces. Point 8: 22–25. Dyson, Mary C. 2011. Do designers show categorical perception of typefaces? Visible Language 45: 193–220. Dyson, Mary C. and Caroline A. Stott. 2012. Characterizing typographic expertise: Do we process typefaces like faces? Visual Cognition 20: 1082–1094. Dyson, Mary C., Roy Brigden, Jonathan P. Bowen, Katie Jenkins, M. Palmer and Will Phillips. 2000. Interfacing disciplines in the design of web interfaces. In Stephen A. R. Scrivener, Linden J. Ball and Andrée Woodcock (eds.), CoDesigning 2000: Adjunct Proceedings, 35–40. Coventry: Coventry University. Farah, Martha J., Karen L. Levinson and Karen L. Klein. 1995. Face perception and within-category discrimination in prosopagnosia. Neuropsychologia 33: 661–674. Farah, Martha J., Kevin D. Wilson, Maxwell Drain and James N. Tanaka. 1998. What is “special” about face perception? Psychological Review 105: 482–498. Fiset, Daniel, Caroline Blais, Catherine Éthier-Majcher, Martin Arguin, Daniel Bub and Frédéric Gosselin. 2008. Features for identification of uppercase and lowercase letters. Psychological Science 19: 1161–1168. Gauthier, Isabel, Alan C.-N. Wong, William G. Hayward and Olivia S. Cheung. 2006. Font tuning associated with expertise in letter perception. Perception 35: 541–559. Goldinger, Stephen D., Tamiko Azuma, Heather M. Kleider and Virginia M. Holmes. 2003. Fontspecific memory: more than meets the eye? In: Jeffrey S. Bowers and Chad J. Marsolek (eds.), Rethinking Implicit Memory, 157–196. Oxford: Oxford University Press.

Applying psychological theory to typography

241

Grainger, Jonathan, Arnaud Rey and Stéphane Dufau. 2008. Letter perception: from pixels to pandemonium. Trends in Cognitive Sciences 12: 381–387. Green, D. M. and J. A. Swets. 1966. Signal Detection Theory and Psychophysics. New York: Wiley. Kimchi, Ruth and Rama Amishav. 2010. Faces as perceptual wholes: The interplay between component and configural properties in face processing. Visual Cognition 18: 1034–1062. Kolers, Paul A. 1975. Memorial consequences of automatized encoding. Journal of Experimental Psychology: Human Learning and Memory 1: 689–701. Kolers, Paul A. 1976. Reading a year later. Journal of Experimental Psychology: Human Learning and Memory 2: 554–565. Ladefoged, Peter and Donald E. Broadbent. 1957. Information conveyed by vowels. Journal of the Acoustical Society of America 29: 98–104. Legge, Gordon E. 2007. Psychophysics of Reading in Normal and Low Vision. Mahwah: Lawrence Erlbaum Associates. Leopold, David A., Alice J. O’Toole, Thomas Vetter and Volker Blanz. 2001. Prototype-referenced shape encoding revealed by high-level after effects. Nature Neuroscience 4: 89–94. Liberman, Alvin M., Katherine Safford Harris, Howard S. Hoffman and Belver C. Griffith. 1957. The discrimination of speech sounds within and across phoneme boundaries. Journal of Experimental Psychology 54: 358–368. Majaj, Najib J., Denis G. Pelli, Peri Kurshan and Melanie Palomares. 2002. The role of spatial frequency channels in letter identification. Vision Research 42: 1165–1184. Martin, C. S., J. W. Mullennix, D. B. Pisoni and W. V. Summers. 1989. Effects of talker variability on recall of spoken word lists. Journal of Experimental Psychology: Learning Memory and Cognition 15: 676–684. McClelland, James L. and David E. Rumelhart. 1981. An interactive activation model of context effects in letter perception; Part 1. An account of basic findings. Psychological Review 88: 375–407. McNicol, D. 1972. A Primer of Signal Detection Theory. London: George Allen & Unwin Ltd. Mullennix, John W., David B. Pisoni and Christopher S. Martin. 1989. Some effects of talker variability on spoken word recognition. Journal of the Acoustical Society of America 85: 365–378. Newell, Fiona N. and Heinrich H. Bülthoff. 2002. Categorical perception of familiar objects. Cognition 85: 113–143. Oden, Gregg C. 1989. Letters of Credit – Tracy, W. American Journal of Psychology 102: 145–149. Ovink, Gerrit Willem. 1938. Legibility, Atmosphere-Value and Forms of Printing Type. Leiden: Sijthoff. Rayner, Keith and Alexander Pollatsek. 1989. The Psychology of Reading. Hillsdale: Lawrence Erlbaum. Rosen, Stuart and Peter Howell. 1987. Auditory, articulatory, and learning explanations of categorical perception in speech. In Stevan Harnard (ed.), Categorical Perception: The Groundwork of Cognition, 113–160. Cambridge: Cambridge University Press. Sanocki, Thomas. 1987. Visual knowledge underlying letter perception: font-specific schematic tuning. Journal of Experimental Psychology: Human Perception and Performance 13: 267–278. Sanocki, Thomas. 1988a. Font regularity constraints on the process of letter recognition. Journal of Experimental Psychology: Human Perception and Performance 14: 472–480. Sanocki, Thomas. 1988b. Looking for a perceptual schema – effects of changing letters and fonts on recognition. Bulletin of the Psychonomic Society 26: 517–517. Sanocki, Thomas. 1991a. Intrapattern and interpattern relations in letter recognition. Journal of Experimental Psychology: Human Perception and Performance 17: 924–941. Sanocki, Thomas. 1991b. Looking for a structural network: effects of changing size and style on letter recognition. Perception, 20: 529–541.

242

Mary C. Dyson

Sanocki, Thomas. 1992. Effects of font- and letter-specific experience on the perceptual processing of letters. American Journal of Psychology 105: 435–458. Sanocki, Thomas and Mary C. Dyson. 2012. Letter processing and font information during reading: Beyond distinctiveness, where vision meets design. Attention Perception & Psychophysics 74: 132–145. Sekuler, Robert and Randolph Blake. 1994. Perception. (3rd edition.) New York: McGraw Hill. Shaikh, Audrey Dawn. 2007. Psychology of onscreen type: investigations regarding typeface personality, appropriateness and impact on document perception. PhD thesis, Department of Psychology, Wichita State University. Thompson, Peter. 1980. Margaret Thatcher: a new illusion. Perception 9: 483–484. Valentine, Tim. 1988. Upside-down faces – a review of the effect of inversion upon face recognition. British Journal of Psychology 79: 471–491. Walker, Peter. 2008. Font tuning: A review and new experimental evidence. Visual Cognition 16: 1022–1058. Walker, Peter and Lisa Hinkley. 2003. Visual memory for shape-colour conjunctions utilizes structural descriptions of letter shape. Visual Cognition 10: 987–1000. Webster, Michael A. and Otto H. MacLin. 1999. Figural aftereffects in the perception of faces. Psychonomic Bulletin & Review 6: 647–653. Wong, Alan C.-N. and Isabel Gauthier. 2007. An analysis of letter expertise in a levels-ofcategorization framework. Visual Cognition 15: 854–879. Wong, Alan C.-N., Isabel Gauthier, Brion Woroch, Casey DeBuse and Tim Curran. 2005. An early electrophysiological response associated with expertise in letter perception. Cognitive Affective & Behavioral Neuroscience 5: 306–318. Zachrisson, Bror. 1970. The problem of congeniality in typography. In: Typographic Opportunities in the Computer Age: Papers of the 11th Congress of the Association Typographique Internationale Prague, June 1969, 47–51. Prague: Typografia Prague.

Gilles Brougère

10 Toys or the rhetoric of children’s goods Abstract: Toy is mainly an object addressed to children. It must persuade children or parents who is its addressee. To mean this, to analyse this kind of communication, which specify the toy, I use the notion of rhetoric. Rhetoric is the way the toy means who is the recipient, how the object seduce, catch and capture the consumer and/or the user, a child or an adult but for the child. To analyse this rhetoric we need to read the object to find the different ways the recipient is present in the toy itself. We analyse 1) the image, the representation and the transformation of the referent using to make a toy, 2) the affordances for play. Nowadays the toy is not an isolated object, but inserted in the circle of the childhood mass culture. Toys have to be interpreted also as a nexus of this popular culture.

1 Introduction The text that follows is based on many years of research in the analysis of toys as a support for communicating meaning, among other things. For over thirty years I have analysed toys in general or specific toys in particular in order to understand the place children occupy in our society and the role of material culture in the construction of contemporary childhood. My socio-anthropological approach (Brougère 2006) does not isolate the meanings embodied in the toy and produced by its use from the social context that makes this object possible. I willingly subscribe to a sociology of childhood or a social strudy of childhood (Qvortrup et al. 2009) that preferentially takes into account the questions of material culture and consumption. The objects reflect practices which they make possible and shape new practices through their use (Dasio 2009). Being a child is to use the objects of childhood, but it is also to produce them through practices that lead designers to propose new objects (Brougère 2003). I consider the child as both product and producer of his or her objects, rejecting both the idea of a child as the product of contemporary mass culture, and that of the child as its all-powerful creator. By following the work of David Buckingham (2011) and distancing ourselves from images of the child as a victim or the child as almighty we take into account the relational dimension of consumption. The child takes part in a network of social interactions, interdependencies (Elias 1978) that allow him or her to both act on and be acted upon by others. The toy takes on meaning in this network which involves adults responsible for the child (parents, teachers), the producers of the object in the broad sense (designers, manufacturers, distributors, communications agencies, the media, etc.) and children themselves as users (Brougère 2003, 2006), and sometimes as decision makers but also, as we shall ultimately see, co-producers.

244

Gilles Brougère

But to achieve this objective it was necessary to pass through the analysis of the object “toy” in order to understand its dynamic, the elements of the analysis that will be reviewed in this text and that are illustrated in numerous texts published in French and more rarely in English (Brougère 2004, 2006). Indeed, the idea that the meanings only make sense linked to specific practices (of production, consumption, transformation) should not prevent us from taking the time to highlight them, to analyse them as such. My method of conducting research on toys has always led me to (temporarily) remove the object, the toy, from its context to look at it from every angle, make it talk, explore the different levels of meaning. This is in order to avoid overly hasty approaches (which are numerous when studying toys) that consist of stressing, to name just one example, the gendered dimension of Barbie or some other toy, then to move on to another subject. What is of interest is to see how Barbie specifically produces (and so differently from other toys) this gendered dimension or this way of addressing itself to little girls. It is a matter of careful reading that implies taking into account the different dimensions that make up the toy, to focus on the smallest details and in so doing follow children who are able to accentuate a specific and inconspicuous aspect of a toy which to them seems essential. Thus, an older study on the Care Bears (Brougère 2003) showed the importance children place on their lock of hair, an aspect that children feel renders them particularly cute. The analysis of the toy is necessary for understanding how the child fits into society by means of a material culture. And in this the toy seems to have an important role in the extent that, more than other objects and due to its relative futility, it is the child’s affair. Paradoxically, it is because it is of little importance, completely frivolous, that the toy matters to us. More easily left to the expectations, choices and demands of the child than other objects, it shows particularly well how an object addresses itself to the child, it features the child. But few of us pursue this approach. Indeed, the research is scant. A few old lines of inquiry given in the context of the emerging multimodal research field (Van Leeuwen 2009), the analyses of specific objects (Barbie, Japanese toys), the analyses by Kline (1993) on the transformation of toys starting in the 1980s can help us on this course. We note the lack of work in this direction as attested to for 20 years by the conference of the International Toy Research Association. This prominent organisation demonstrates the domination of research oriented towards the use of the objects, play, its effects, particularly educational, and sometimes the designer's approach more than the analysis of its products. Though indeed different works can be of use here and there, those who wish to take on the study of toys will not find any document that allows them to prepare for this endeavour. This text will attempt to fill this void, but from our point of view, a perspective that I invite the reader to keep in mind, that which interests itself in toys in order to better understand the child within a consumer society, with the importance that the material culture of mass consumption takes on. As

Toys or the rhetoric of children’s goods

245

such, our analysis focuses on the toys that are the most representative of this mass culture even though it can undoubtedly be applied to other objects that are, at least in appearance, on the margins of mass culture. It is a matter of offering a set of tools and an approach to analysing the toy of contemporary mass culture and placing it in the construction of childhood today.

2 What is meant by “toy” Before starting the analysis of the toy we need to know what is meant by the term, especially as it will guide us in this analysis. We limit ourselves to the more usual sense of the term toy, according to the Webster online dictionary, “something for a child to play with,” ignoring older meanings or more or less metaphorical derived meanings. In this sense the term corresponds to the word jouet used in my works in French. Starting from this first (and perhaps too broad) definition we will specify its characteristics. The first and fundamental element is that the toy addresses itself to the child, which distinguishes it from other objects (playthings, games) that may also address themselves to adults. The object we take into account is an object that is explicitly intended for the child and which is socially perceived as such, and clearly designed for this. This element is essential since we base our analysis in a socio-anthropology of childhood and it will lead us to extract lessons to apply them to other objects intended for children. In one sense our analysis aims to answer the question: what is an object intended for a child? Or how can we address ourselves to the child within the framework of material culture? The second element, so essential that it appears in the definition, is that the object is also defined by its function, to play. The problem is that play is an ambiguous notion with unclear boundaries (Brougère 2005). What is play? Where does play stop, where does it start? Play appears as a paradoxical function and it is perhaps this that allows the wide variety of toy objects, linked to the diversity of play activities and more generally pastimes of children. One may wonder if beyond play, the toy should be thought of as an object of entertainment for the child. None the less, play is at the centre and it is in the margins that we will find a variety of activities between play and amusement. The question of the “function” of play will therefore be essential to analysing an object (that which renders it unique) and is the first question we take into account. However, we will quickly recognize that the different dimensions of the toy are deeply intertwined and that we cannot isolate its function. Though these two dimensions feature the definition of toy, they miss its unique qualities compared to another plaything or a game that may likewise allow the child to play and be intended for him or her. There is a cultural dimension essential to the toy that does not seem to appear in the definitions but that is at the

246

Gilles Brougère

centre of our representations of a toy: the toy is not a purely functional object allowing one to play (the term “game” is usually used for this). It is an image (usually tangible) of the real or imaginary world that a child will manipulate through an activity associated with play in the broadest sense of the word. This has it not be generally based on a narrow functionality like that of a game of checkers or a video game, but open to uses that are based on what it represents (such as a baby doll prompting play of parenthood but not exclusively so). We can discuss – but it is an approximation – play without rules, open play. In any case the toy represents, puts forth an image and this is an essential dimension which undoubtedly gives it its unique qualities and, more so, its richness. This dimension (connected to the two mentioned above) will be central to our analysis. We can thus say that the toy features, gives dimension to, gives an image of the world so that the child can play with it. It is in this way that it can be considered a visual communication, the toy communicating an image to the child, with the unique quality that the expected response is its manipulation, an action, play. This leads us to limit our analysis to objects produced (by adults) with this objective and to not take into account the toys produced directly by children. What interests us is how a society produces, through objects, the meanings that we have discussed (representation of a world manipulated by the child). Thus we believe that Barbie should be analysed as the image of a woman (or girl) intended for a child such that he (or rather she) plays with it. The image dimension cannot be isolated from the rest and so we cannot avoid noticing that the long legs attract play and are not limited to the expression of an aesthetic norme. To these dimensions linked to the definition of a toy as an object produced by adults for children (under the possible influence of the latter – we will get back to this), is added another dimension which reflects our choice to understand contemporary children's culture, namely the highlighting of the toys most linked to contemporary mass culture. We are interested in the toy found at the heart of child consumption and in the circulation of content through a variety of objects and media.

3 An object to play with As indicated by the definition, being intended for the child and having a play objective are interlinked in the toy. And one of the ways to address the object to the child is to focus on the play function that renders the child the user. The analysis of the toy object, in that it is not a simple image, therefore consists of taking into consideration this functional dimension. But it is a matter of doing so based on the object (such as seen by parent and children buyers) and not from observation of its use. We should read in the object that which we can do with it, assuming that this can be seen. Of course this can be seen in an advertisement or the packaging, but our analysis here limits itself to the object.

Toys or the rhetoric of children’s goods

247

Play appears primarily as enrolment of the child’s actions in the toy. It is not a matter of what the child will actually do with the object which he or she can redirect as much as he or she does not need a toy to play, any object being able to potentially become an object of play. We do not aim at the richness, the diversity of child's play, but only its enrolment in the object, that which the object affords the child as its play, at least as it is conceived by the designer of the object. This is what can be called, following a tradition coming from the psychologist James Gibson (1979), “affordances”. The toy is primarily intended for the child because it provides affordances, structures for the play that is proposed. The toy communicates possibilities for action by its form or different devices visible and the first level of analysis is to identify them. Such is the case of adding wheels to an animal or a telephone. Here an affordance is visible referring to the action to pull, push, or make it roll. Doing so is a very old and traditional dimension of play that is embodied in the object even though this modifies the image profoundly. Who has seen a horse or a telephone with wheels if not in the world of children? In fact this functionality at the same time that it affords the possibility of a play action, designates the child as the recipient of the object labelled by its dimension of false resemblance (it’s not a real horse, you cannot make a call). Thus the discussion of the toy is a discussion of play, an embodiment of play functionalities that very often reflect ancient traditions of play, restructured in new ways, whose difficult to find origin is in part linked to children's culture, that produced by the children themselves, generation after generation. So it is with warlike opposition, a classic structure of play, which can be seen in figurines representing two opposing camps. Another form of play, featured on the side of little girls, is parenting through the doll and its accessories and, to not take only one of its functions, the styling of hair requiring an ample and manageable head of hair as well as the appropriate accessories. This affordance can be found on a pony that will, through its mane, create this same play relation to hair styling. My Little Pony by Hasbro therefore creates in the object itself a significant affordance for play, hair styling, by giving volume to the mane. This means that the analysis of the object must highlight all the elements that render possible, encourage the manipulations involved in the play repertoire of childhood, and sometimes, but only rarely come to enrich and renew it. What characterizes this dimension is its historical stability. As revolutionary as Barbie was in the history of the toy (Brougère 2003), she remains conservative at the level of play affordances: thus we find styling hair, dressing, as well as the bath (in a pool as opposed to the same function attached to a baby doll). To analyze this doll is to show how she (not generally but through references, specific products) offers play manipulations. To analyze only the image is to pass over the object; the importance of the play dimension appears clearly in the accounts we possess regarding her creation (Lord 1984). Objects are transformed, adapted to make play activity possible but also to make it visible, to feature it. Thus the use of soft materials used for stuffed animals

248

Gilles Brougère

since the advent of the Teddy Bear in the early 20th century affords the possibility of a gentle touch, the expression of a potential emotional relationship. Though this meets certain realism for long-haired animals such as bears, today it is used for a wide variety of animals allowing one to find on the market plush pigs, a necessary transformation to enable the emotional affordance belonging to stuffed animals. Here we mean the play function in a broad sense. This is the case with collecting as an affordance embodied in objects associated together in a series, such as Littlest Pet Shop, a success in recent years, differences and similarities, visible signs of the dynamic of collecting that brings together objects that have a point in common (same size, same type of representation, all are pets) and differences linked to the diversity of animals represented. All the signs are that much easier to read since most toys demonstrate little innovation at the level of play dynamics. Finding new play principles is rare, difficult and does not guarantee recognition. A new doll, as original as she may be, does what a doll does, it highlights cuteness, childhood and calls upon “parental” care. The traditions are there, the toys reproduce known forms of play, a play culture that varies with age and gender. Through play, children are targeted in their diversity, girl or boy, younger or older. The affordance to manipulate the object demonstrates the ability of the child, and a toy for an infant should take this into consideration. Early childhood development toys depict (for the adults more so than for the children) an active baby, manipulating, sorting, reacting (the sounds allow one to hear the effectiveness of the action). We could consider, following Robin Bernstein (2011: 11–12), that the toy offers in its own characteristics a script: “The term script denotes not a rigid dictation of performed action but rather a set of invitations that necessarily remain open to resistance, interpretation and improvisation.” The affordances that we have mentioned invite possible actions that presume interpretation and improvisation in the child's play (that which is characteristic of play with a toy unlike games that imply following a set of sometimes very specific rules). And this is even truer when this script is undoubtedly influenced by play invented by children. It is based on this play, the play activities of children (or of themselves when they were children) that designers offer objects that make such activities possible (at least in their eyes). Thus at the origin of Barbie there was play with cardboard dolls for which one cuts out paper clothes. Based on this pre-existing play script Mattel offers an object that makes this possible in three dimensions (Lord 1984). The threedimensional representation taken from a German doll (Hanquez-Maincent 1999) enriches a pre-existing script around fashion, but will allow other play actions that the object will incorporate in a back-and-forth between adult constructions and child improvisations. If we have focused on specific affordances (such as hairdressing or the wheels of a phone), the toy is characterized by the fact that its affordances are not only

Toys or the rhetoric of children’s goods

249

functional in a narrow sense. What is it that makes us parent it, that makes us pretend to call, if not the appearance itself of the toy, the image that it offers, that of a baby or of a telephone? And it is here that we find the unique qualities and the point of the toy, the importance of the image that prompts play, the play script embodied not only in the directly functional elements, but in the image. That which the toy represents is all the more important since it is that which allows one to play. The two dimensions mentioned, function and image, intertwine: function (like the wheels of the phone) transform the image, the image is itself a function, at the source of the play activity. To analyze the “functional” elements is therefore to highlight that which contributes to the play script, both the directly functional elements and the images that give meaning to the play activity.

4 An object that represents the world These scripts, these possibilities for action, these affordances or transformations of the action as devices visible on the object are only part of what the toy has to say. They are inserted into the images, representations: houses, vehicles, human beings and their environment, animals (and even more so animals resembling humans). Toys offer an image of the world for the child. There is a selection of realities considered suitable for the child (such as animals), and a transformation of images that are only partially realistic. To analyse the toy is to highlight the processes that transform the world in order to put it within the grasp of the child, to make it playable. The processes are numerous. The first is undoubtedly the miniaturization that evokes a very ancient way to place the adult world within the reach of children. A Lilliputian reproduction of the world refers to childhood, and is intended for the child, even if this is not always sufficient since adults also have their miniatures, models or trinkets. Not all objects are miniatures; we can on the contrary, in the representations of insects for example, find the reverse process which is to oversize the objects to make them easy to handle by the child. Again we see that image and affordance go hand in hand and are distinguished only for the purposes of analysis. As such, miniaturization that produces a “childlike” image of the world is also an affordance, a means to the manipulation of objects. Among the various ways of transforming an object to address the world to the child, we can call upon animism which consists of animating inanimate objects (such as the telephone already mentioned that not only has wheels, but also eyes, a nose, a mouth), anthropomorphism, animals – starting with the teddy bear – being humanised or rather made to look like children in a process of “pedomorphism”, putting into a child-like form. We can call up all the work on colours (today bright and primary), and shapes (often rounded for younger children). The result is that the toy transforms the world to offer it to the child or to give the

250

Gilles Brougère

child (or his or her parents for the youngest children) the desire to possess it. A world that is often caricatural and simplified, imaginary, fanciful is thus produced. But the toy must be situated in the context of a broader children’s culture insofar as this work of transformation of reality, of the imaginary for the child is not unique to the toy. Children’s literature, comic books, cartoons and films, in their own way, produce worlds designed for children and offer to toys images already intended for the child. We will return to this point. We can take the example of Mr Potato Head by Hasbro that originally refers to a children’s activity (or one shared between children and adults) consisting of turning a potato into a character. In its first version Hasbro offers only the accessories that allow one to humanize the potato (nose, mouth, eyes, arms, feet). For safety reasons, it had to offer these accessories with a plastic potato. The image is that of a potato that one can humanize following the scripts issuing here from traditional play activities. Mr Potato Head then became an imaginary character of such success as to be found in different opuses of the film Toys Story by Pixar, which could only increase its notoriety. One also finds that he diversifies and Hasbro offers various characters in the form of Mr Potato Head. Analysing this toy therefore comes back to connecting several dimensions, those of functions (putting eyes, nose, mouth, etc. to transform the potato into a character), but also the complex dimension of representation, which leads us from the image of a potato to the creation of a character in the shape of a potato which can in turn evoke other characters (e.g. from Star Wars). Analysis of the toy cannot be general and though each analysis encounters the same dimensions, each toy involves a specific approach in the way in which function and image connect. It may be noted that unlike the horse, the car does not need a derealising functional addition to function as a toy. It can be realistic and afford an essential play function; this is undoubtedly one of the reasons for its success, which connects the “natural” functionality of the object and promotion of the representation. But the toy is immersed in a history of representations and their reception. We find more and more imaginary cars (starting with those from Disney’s Cars), showing that promotion of the real automobile object is no longer enough, and that the promotion by its embodiment in an imaginary world has become more important today than it was yesterday.

5 Featuring of the recipient The two dimensions converge, come together to place the object in relation to childhood and children’s culture, as much in its play dimension as in that of elaborate representations by other media starting with children’s literature that has developed an often explicit dynamic of addressing itself to the child (NièresChevrel 2009). This can lead to an object that moves away from the faithful repro-

Toys or the rhetoric of children’s goods

251

duction of reality, to be easily manipulated like the horse on wheels, or place it in a children’s imagery such as My Little Pony which transforms it into a virtual doll thus associating to it the play dynamic linked to styling hair, as we have seen above. Elsewhere, realism encounters this dynamic as in the case of the automobile. But it is a reduced model, a false resemblance, two modes of addressing the child, a little human that pretends to mistake it for real. The dynamics of this address, combining image and visible function (or affordance), vary according to age. For younger children it is the exaltation of childhood aimed at adults that will be highlighted. For older children it will feature a seductive (and largely imaginary) adult life as a model like Barbie or a warrior. Between these two dynamics, worlds combine one or the other in varying ways, for example when human life is translated into animal form. To describe this evolution and sometimes this tension between the dynamics that surround the toy depending on the age of the child (and the role played by the adult in making the choice) we can look to the work of Gary Cross (2004), which shows the transition from cute (older historically but also chronologically in the life of the child) to cool in the values underlying childhood and its objects. We can therefore in the analysis trace the tracks of cute (round shapes, representation of the baby, etc.) and cool (rupture with adult good taste, an excess of characteristics belonging to the adult world whether violence or sexuality). Hello Kitty, the symbol of cuteness (Allison 2006) marks the objects of the former dimension. On the opposite end we can consider that Barbie’s chest and long legs are cool. These are two ways of addressing oneself to the child (a girl in both cases), which express themselves through symbols to be decoded as much as supports for play, transformation of the image, and featuring of the recipient. Thus Barbie is formatted for play as we have seen (easy manipulation due to her legs, hair to be styled, accessories for dressing or bathing); she provides a representation of the world marked by excess, exaggerating female traits, thus approaching her to the Drag Queen (Rogers 1999), and finally addresses herself to children who refuse the cute of childhood in favour of the cool that creates a rupture with it. However, this Barbie effect has been lessened, since mothers have known it in their youth, so it shocks less, since it addresses itself to younger children (and transforms itself for this at the risk of becoming cute) from where comes the competition of Bratz, dolls which exaggerate the very exaggeration of Barbie referring to the codes of rap and television musicals. This features cool and thus renews the address to the child that projects into adult life (idealised, unrealistic, of course) and not into childhood as other (cuter) toys offer. Thus is the child as recipient embodied in the object itself and this appears as the result of the two dimensions analysed. It can be broken down in the elements of the object that mark the recipient in a more specific manner (such as colour, pastel or bright, blue or pink), but this will find itself reinforced at the level of packaging and advertising. Thus the doll, intended for girls, is more often the representation of a female, relinquishing Ken and the male doll as the exception

252

Gilles Brougère

or more likely the accessory. The doll really does give a representation of the child as a girl, toys intended for boys favour male representations, and younger children can cultivate the ambiguity of asexual animals. The recipient is quite present, in a very explicit manner, in an object at the interface of a representation of the world intended for a child and the featuring of play functions referring to play according to gender and age. It is therefore a matter of deciphering in the toy as much function and image as recipient, but in that this is embodied in the object itself, it is present through the tangible elements.

6 The rhetoric of the toy The different elements that we have identified constitute a set of meanings oriented towards an address to the child in the sense that its goal is to convince the child (and/or the adult buyers) that the toy is made for, is intended for him or her. The goal is to persuade the recipient directly and indirectly, to seduce, to capture (Cochoy 2004). It is indeed rhetoric in the sense defined by Patrick Chareaudau: Any targeted captation is directed toward a partner of the exchange, the representative or the recipient as one would want to call him or her, in short, towards this other of the act of communication, without whom the subject communicating him or herself would not exist. It assumes that this other is not a natural partner (one must set him or her as the recipient of a message) and is not assured in advance to the values of intentionality that the message covers (one must persuade, seduce him or her) (Chareaudau 1997: 78).

The toy must construct the child as the recipient of the message in a dynamic of persuasion, seduction and captation through the means available to it as an object, those we have discussed above, coming from the connection between representation or image and featuring of the function or affordance. What we are trying to highlight is the rhetoric of the toy in the way of addressing itself to the child (and/ or his or her parents or other toy providers). We could certainly evoke objects devoid of rhetoric, with no featuring (in action or image) of the child. The object would be purely functional, in the sense that it would content itself to implement the constraints of play as would the game of checkers. It is a dynamic that can be found precisely in some objects called games (including games of skill) more than in the toys that I gladly define now as an object endowed with rhetoric understood as discourse addressing the child. As such the spinning top, defined by its function, can be considered devoid of rhetoric at first glance (it is therefore a game and not a toy). But concrete objects offered to children are very often marked, reworked, adding images. Though the absence of rhetoric is possible, it seems rare and difficult since the toy so much is deemed to give rise to a relation with childhood, that which makes this object unique. By refining the analysis we can distinguish two main rhetorics to target the child.

Toys or the rhetoric of children’s goods

253

6.1 The rhetoric of education The first rhetoric, more present in toys of the past (Manson 2001), has perhaps become marginal today, it is the rhetoric of education in the broadest sense. The child is targeted as a being under development, a future adult, the subject of an education that defines its uniqueness. Objects capture him or her through this dimension: they are the early childhood development toys, educational toys (or games) and more generally toys that claim (through their realism, for example) to help the child discover the world. We can add to the list creative games, scientific games and some board games (we will find more games here than toys). We can thus define in this frame the importance of parenting as enrolment of the little girl in her destiny as a mother, highlighted by the desire to transmit parenthood in its most realistic dimension compatible with the toy. The concern for realism in the representation of the baby not only sexed but close to that of a newborn infant reveals how toys can be designed in order to show the world as it is and not as desired by the child. We can place in this same dynamic the initiation to fashion linked to older dolls. A doctor’s briefcase can likewise be viewed as a way to place the child in the real world to enable him or her to understand and master it. The representation of school may be taken literally this relation between child and education, and can even be made into a learning support. Image and function converge to anchor themselves in this rhetoric that often seems to target more the parents or adult educators than the child itself. Children’s literature, comic books, films experience (and especially have experienced in the past) the same rhetoric. Today, it appears to be lagging behind, disappearing largely when the products are chosen by children and not adults. But some toys are still chosen by adults and this is especially true when the children are young. Children themselves can enjoy learning and seek products that go in this direction.

6.2 The rhetoric of fun The second rhetoric which seems to us as largely dominant in the contemporary toy without, however, being new, we call the rhetoric of fun, marked by the dominance of entertainment and amusement. A toy should be fun, should generate more pleasure than learning. It addresses the child as the subject for entertainment and not learning and this is reflected in the same way in a part of children's literature, the comic books, and more so in television series. It is the notion of “fun food”, used to designate food that not only addresses the child but highlights to this end values of entertainment, even play, well away from any serious or healthy vision of nutrition, which prompted me to use the concept of the rhetoric of fun (Brougère 2010, 2012; Cook 2005; de la Ville and ali. 2010). It consists of using the shape, colour, the insertion of a play principle or a few other techniques for food while still remaining such, to become something

254

Gilles Brougère

else, an object for pleasure and entertainment, which can serve as a marker for membership in a peer group, possibly breaking with an intergenerational dynamic of consumption within the family. Fun translates a new rapport to the world that makes entertainment the norm. Television is undoubtedly that which has given fun a considerable place in our society, the educational dimension, without being absent, being marginalized here. And the relation between the toy and television has become very important, not only through advertising but also through shows that feature toys. Fun is expressed through and largely based on the culture of peers who will collectively construct what is fun and what is not. It relies on different processes that we have already highlighted: a break with realism, play dynamic, shapes and colours that refer more to childhood than to reality, use of the characters of children’s culture, which uses the work of rendering something entertaining already carried out by children’s literature, comic books, films. Finding in the form of toys the characters of one of the productions by Pixar, through Buzz Lightyear is to prolong this production, give it a continuation in the life of the child, his or her play. Though the process has today increased in magnitude, it is not new, the American toys studied by Gary Cross (1997) show how the phenomenon started to be present between World War I and II, the first significant and still fragile and limited divergence from the dominance of the rhetoric of education. As with any categorisation, this one has its limitations. It is a tool for thinking of a reality that does not obey our categories and we find ourselves instead facing two poles in a continuum of more and less (fun, educational). Numerous products combine fun and educational dimensions, this is true of early childhood toys, certain worlds such as Babar, for example, representations put forth by Playmobil. As such, Fisher Price products surrounding the name “Fun to Learn” attempting to associate closely education and fun. It is the fun side that the toy pushes insofar that play remains central and allows the free action of the child. We can thus continue the analysis and track the rhetoric of fun belonging to the toy, a way to address itself to the child by highlighting the fun it generates both through the functions (the play script offered) and through enticing images, whether they be cute or cool. But the analysis remains insufficient if it isolates the toy from other objects of childhood. Indeed, what characterizes this mass culture is a circulation between media and products, which makes it that the toy uses images, worlds, characters produced by other supports and likewise characters coming from toys find themselves (such as Barbie or Mr Potato Head) on other supports.

7 The transmedia dimension This use by the toy of images produced by other supports must be taken into account in the analysis. The presence of Harry Potter, Hello Kitty or Spiderman as

Toys or the rhetoric of children’s goods

255

toys implies that we should take an interest in what I have called the circulation of images between the different supports of children’s mass culture (Brougère 2008). In other words the toy can build its own images (as in the cases discussed above) or use images produced by other supports (books, films, television shows, video games, etc.). In this case, the rhetoric is based on pre-existing rhetoric produced in another context. The work of addressing itself to the child, of enticing, most often through fun, the transformation of the world to adapt it for the child, is already done. The only thing left to do is to turn this into a toy, to make manipulation and play possible by giving it form. The analysis must thus focus both on that which circulates and adaptations specific to the toy. It is difficult to define what circulates as this can be embodied in different supports, and in a transmedia dynamic as analysed by Jenkins (2006) the story does not circulate since it is the product of the group of supports. As such Bernstein (2011) shows that we cannot reduce Uncle Tom’s Cabin to the novel by Harriet Beecher Stowe, one must take into account the plays, illustrations, and countless objects (handkerchiefs, trinkets) derivatives that the author refers to as “Tomitudes”. Jenkins limits himself to media with relatively noble examples (The Matrix), showing how the story is constructed from one medium to another. If we look at the toy, we must go beyond the story in the strictly narrative sense to take into account the construction of a world of values, images, characters or stories that produce this ensemble to which refer as much Harry Potter (endowed with a structured narrative) as Hello Kitty, a character literally without background. Bernstein, in analysing the Raggedy Ann doll which involves books, dolls and various objects, describes the phenomenon as follows: Gruelle succeeded because what he sold, and what millions of Americans bought, was ultimately neither Raggedy Ann books nor dolls, but “Raggedyness”. Gruelle contained Raggedyness in books and dolls; he also occasionally packaged Raggedyness in a musical score or theatrical performance. […] After Gruelle’s death in 1938, the Gruelle estate expanded licensing of Raggedy characters. Raggedyness then entered containers as diverse as stationery, piggy banks, pencil sharpeners, sleepwear, toothbrushes, dishes, lamps, figurines, and many other commodities. […] Raggedyness became diffuse and almost infinitely flexible1 (Bernstein 2011: 151).

We can therefore consider that that which circulates could be called as the case may be “Harrypotterness”, “Pokémonness” or “Hellokittyness”, and is not limited to characters or stories, but to characteristics, values that are spread and constructed through different objects. What characterizes this children’s mass culture is the taking into account of the recipient (or multiple recipients) in the very structure of the world, which allows us to speak of rhetoric in this way of targeting an audience explicitly.  There is now a website dedicated to the sale of “Raggedyness” under the name of Raggedy Land: http://www.raggedy-ann.com/.

256

Gilles Brougère

For many toys, the rhetoric therefore consists of taking a world that is already constructed in this dynamic and adding a play dimension of greater or lesser extent according to the product. The worlds undoubtedly lend themselves more or less to a play transformation. However, what characterises the circulation between media and supports is a complex game of duplication of the same and modification, of repetition and transformation. Usually, each medium and support faithfully reproduces content from other media – especially content from the original platform – for economic reasons and also due to the stakes surrounding brand and illustration ownership. The proprietors – when they have the means – watch over and check the faithfulness of the reproduction to the original. The public equally expects the new medium to provide what it appreciated in others. The Harry Potter reader doesn’t expect the film to be an original interpretation or a specific reading of the novel, but a highly faithful representation in images of what appears in the book. This general principle comes up against the obvious impossibility of guaranteeing a loyal transposition from one medium to another. Each medium, bound by specific means and constraints, generates an in-depth transformation of the content from one platform to another, adding its personal touch. This is one of the reasons why what is in circulation does not exist outside the circulation itself. These two principles need to be taken into consideration together. The transposition from one platform to another that considers itself faithful to the original is also an interpretation, a process of interpretative reproduction, to coin a phrase defined by Corsaro (1997) when analyzing children’s practices, especially surrounding play. It seems to cover a greater cultural movement, also at work in this case. We are confronted with a circular merchandising strategy that manufactures the elements of this culture – a group of characters, stories and worlds – which are transposed onto a multitude of platforms. Pokémon is an example of the advent of a mass world culture in which toys and videogames play a leading role. What is behind Pokémonness is a cultural world that cannot be transcribed in a single product. It takes on its true meaning if we connect the original product (a gameboy video game) with the television series that develops the story, card games that represent well the acts of collecting and challenging an opponent, figurines bringing to light the dynamic of collecting, stuffed animals that develop the cute dimension of the universe (Brougère 2006). Pokémonness is the fusion of collection, combat and cute through numerous but specific characters illustrating the concepts of transformation, training and affection. This complex and improbable connection can only be expressed in all its richness through a multitude of supports and defines Pokémonness. The toys incorporate these values but can develop them in a dimension linked to the unique qualities of the object. As such, the stuffed toys will explore and develop the emotional and cute dimension of the pocket monsters.

Toys or the rhetoric of children’s goods

257

To analyse the toy is therefore to understand the connection between the characteristics of the universe and those of the toy, but also to try to grasp what it can bring to the world and how it enriches it. Behind this circulation lies also the question of script already mentioned following Bernstein’s (2011) example. The toy appears to be linked to a script, the story of Pokémon, Harry Potter or Spiderman. However, we should qualify this by coming back to the idea that affordances, the dynamics of actions embodied in the toy and even more so the social and cultural representations that accompany the images (such as the relation between the image of a baby and play of parenting) are already scripts. The toy is defined by the integration, of greater or lesser strength and visibility, of a play script which will allow the child to follow it, to interpret freely or invent something completely different. These play scripts are largely readaptations in the toy of the play repertoire produced generation after generation by children, undoubtedly in a complex relation between child production and adult influence. The worlds that we have mentioned seem to offer other scripts (coming from culture produced by adults) to the play scripts. But this is only in appearance since if we analyse the stories offered by these worlds, they are sometimes explicitly play scripts (as in the case of Beyblade, the game of spinning tops or Yu-Gi-Oh, the card game) or more often a more or less elaborate transposition of a play script. In this way Pokémon transposes traditional play, according to the creators (catching insects, shutting them up in matchboxes, collect them, possibly making them engage in combat), Disney’s Cars seems to feature playing with toy cars, etc. We find another circulation between play scripts and stories intended for children, each influencing the other and vice versa. Therefore we can find in the toy, but also in the worlds that it reuses, play scripts coming from children’s play culture.

8 The unique aspects of the analysis of the toy To summarize, we can consider that analysing the toy is first to identify in a tangible image, a three-dimensional object, the opportunities for manipulation, the play it affords but also the ways it addresses itself to the child, that which we have named rhetoric. The specific representation, usually removed from all realism, and which results from a transformation of something real, is to be understood in the connection between the dynamics of action (to do, to play) and rhetoric (persuade, seduce, even capture). To do this many objects are part of a complex circulation of contents (set of values, characters, stories) that though it allows to immediately place them in the rhetoric of fun, implies a process of transformation so that a play action is possible. In this the toy contributes to the circulation by enriching the universe by making it playable. This leads to introducing it more

258

Gilles Brougère

strongly in the world of the child through its transformation into a script adopted by the child. The principles of the analysis of the toy should be thought of as specific, connecting the dimensions specific to toys, but they also allow us to offer tools to analyze other objects, especially all those that address themselves to the child by using the notion of rhetoric. Numerous objects of all kinds develop the rhetoric of fun to address themselves to the child and the toy is no longer alone in providing entertainment. This is the case with fun food, but also decorations, stationery and even clothing. However, the rhetoric of fun is rarely alone and as in the opposition to the rhetoric of education for the toy, it meets other dynamics of address such as the rhetoric of health for foods, that of functionalism for the world of decoration. Finally, it is important to remember that these rhetorics are based on the play culture of the child itself and are not to be understood as purely adult productions. They testify to a strong interdependence between that produced by the child (his or her play and diverse activities) and that produced by adults for children. The script put forth by the adult is even more likely to prevail when it meets the very scripts that children produce in their play. This is what the analysis of the toy, among other things, must track.

References Allison, Anne. 2006. Millennial Monsters. Japanese toys and the global imagination. Los Angeles: University of California Press. Bernstein, Robin. 2011. Racial Innocence. Performing American Childhood from Slavery to Civil Rights. New York: New York University Press. Brougère, Gilles. 2003. Jouets et compagnie. Paris: Stock. Brougère, Gilles. 2004. How much is a Pokémon worth? Pokémon in France. In Joseph Tobin (ed.), Pikachu’s Global Adventure. The Rise and Fall of Pokémon, 187–208. Durham: Dude University Press. Brougère, Gilles. 2005. Jouer/Apprendre. Paris: Economica. Brougère, Gilles. 2006. Toy houses: a socio-anthropological approach to analysing objects. Visual Communication 5(1): 5–24. Brougère, Gilles. 2008. La ronde de la culture enfantine de masse. In Gilles Brougère (ed.), La Ronde des Jeux et des Jouets. Harry, Pikachu, Superman et les Autres, 5–21. Paris: Autrement. Brougère, Gilles. 2010. Fun food et culture matérielle enfantine, paper presented to the international conference “Alimentation, Cultures Enfantines et Education”, Angoulême, 1–2 april. Available at: http://www.univ-paris13.fr/experice/images/stories/MEMBRES/ PUBLICATIONS/funfood_colloque_04_2010.pdf. Brougère, Gilles. 2012. Ludo-aliments: des produits immergés dans la culture enfantine. In Louise Hamelin-Brabant and André Turmel (eds.), Les Figures de l’Enfance: un Regard Sociologique, 294–314. Québec: Presses Inter Universitaires. Buckingham, David. 2011. The Material Child-Growing up in Consumer Culture. Cambridge: Polity Press.

Toys or the rhetoric of children’s goods

259

Charaudeau, Patrice. 1997. Le Discours d’Information Médiatique. La Construction du Miroir Social. Paris: Nathan. Cochoy, Frank. 2004. La captation des publics, entre dispositifs et dispositions, ou le petit chaperon rouge revisité. In Frank Cochoy, La Captation des Publics. C’est pour mieux te Séduire mon Client …, 8–19. Toulouse: Presses Universitaires du Mirail. Cook, Daniel T. 2005. How food consumes the child in the corporate landscape of fun. Agency and culture. Paper presented at the annual meeting of the American Sociological Association, Philadelphia. Available at: http://cas.uchicago.edu/workshops_old/money/PDF/ Cook2005_funfood.pdf. Corsaro, William. 1997. Sociology of Childhood. Thousands Oaks: Pine Forge Press. Cross, Gary. 1997. Kids’ Stuff. Toys and the changing world of American childhood. Cambridge: Havard University Press. Cross, Gary. 2004. The Cute and the Cool. Wondrous Innocence and Modern American Children Culture. Oxford: Oxford University Press. Dasio, Nicoletta. 2009. La liaison tumultueuse des choses et des corps: un positionnement théorique. In Marie-Pierre Julien and Christine Rosselin, Le Sujet contre les Objets … tout contre. Ethnographie des Cultures Matérielles, 21–83. Paris: CTHS. de la Ville, Valérie-Inès, Brougère, Gilles and Boireau, Nathalie. 2010. How can food become fun? Exploring and testing possibilities … Young Consumers 11(2): 117–130. Elias, Nobert. 1978. What is Sociology? New York: Columbia University Press. Gibson, James J. 1979. The Ecological Approach to Visual perception. Boston: Houghton Miflin. Hanquez-Maincent, Marie-Françoise. 1998. Barbie, poupée totem. Paris: Autrement. Jenkins, Henry. 2006. Convergence Culture. Where Old and New Media Collide. New York: New York University Press. Kline, Stephen. 1993. Out of the Garden. Toys and Children’s in the Age of TV Marketing. Toronto: Garamond Press. Lord, M. G. 1984. Forever Barbie. The Unauthorized Biography of a Real Doll. New York: Avon Book. Manson, Michel. 2001. Jouets de Toujours. Paris: Fayard. Nières-Chevrel, Isabelle. 2009. Introduction à la Littérature de Jeunesse. Paris: Didier. Qvortrup, Jens, Corsaro, William, Honig, Michael-Sebastian (eds.). 2009. Handbook of Childhood Studies. Basingstoke: Palgrave Macmillan. Rogers, Mary F. 1999. Barbie Culture. London: Sage. Van Leeuwen, Theo. 2009. The world according to Playmobil. Semiotica 173(1–4): 299–315.

Martin Conboy

11 Visual Aspects of British Tabloid Newspapers: ‘Image Crowding Out Rational Analysis’? Abstract: The tabloid newspaper has been analysed most often in terms of its language and its general contribution to vernacular culture. To a large extent this is a legitimate means of focusing attention on the sophistication of the language of the tabloid and its relationship to longer patterns of engagement between the popular and print culture. However, this type of analysis has often neglected the visual appeal of the genre which is perhaps its primary attraction. Visual elements within popular culture have had as long a gestation as the linguistic and it is true to say that the tabloid has emerged from popular traditions very much in terms of a visual/linguistic partnership, one reinforcing the claims of the other. This chapter will examine the visual component of the precursors of tabloid newspapers in Britain and look at the impact of the shifts which they introduced.

1 Introduction The tabloid newspaper has been analysed most often in terms of its language and its general contribution to vernacular culture. To a large extent this is a legitimate means of focusing attention on the sophistication of the language of the tabloid and its relationship to longer patterns of engagement between the popular and print culture. However, this type of analysis has often neglected the visual appeal of the genre which is perhaps its primary attraction. Visual elements within popular culture have had as long a gestation as the linguistic and it is true to say that the tabloid has emerged from popular traditions very much in terms of a visual/linguistic partnership, one reinforcing the claims of the other. Much of the power of the tabloids has accrued through the evolution of a particular style of language and its relationship to the broader patterns of popular print culture which have developed over centuries. But to this we can add the important complementary aspect of the visual; from image to layout, this visualization of the popular has played an essential role. This chapter will examine the visual component of the precursors of tabloid newspapers in Britain and look at the impact of the shifts which they introduced. It will also consider the place of tabloid newspapers in Britain as part of a broader contemporary mediasphere within which the visual aspects of the tabloids plays an important part in reinforcing broader trends towards an entertainment driven

262

Martin Conboy

media culture. In doing so, this piece will consider the assertion that in the tabloid format, “image is crowding out rational analysis” (Bird 2000).

2 Historical antecedents Despite their self-evident location in the contemporary world, tabloid newspapers are merely the latest and most marketable permutation of mass mediated popular taste in periodical form. As print culture evolved in the early modern period, the best way to appeal to a broader and more popular audience was to build upon accepted patterns of folk culture (Burke 1978) and to frame printed material as much as possible in a language with which the audience would be familiar. Much of this early popular print culture in the sixteenth and seventeenth centuries deliberately followed the rhythms and patterns of the spoken language of the ordinary people (Ong 1982). Popular print culture had always incorporated aspects of the visual in its display and this allowed it to reach a wider audience than the literate minority. Images and visual layout helped ease the passage from folk to commercial print culture. Early graphic devices mapped onto established traditions of vulgar populism; attracting people through the display of the visual. Even while literacy rates were low, there were still markets for printed matter which could be read, often out loud, or sung. This appealed to a community of readers and listeners not only by means of the language which they employed but also in the illustrations they could combine with the text. Woodcut illustrations were used to draw the power of the image into popularizing the process of reading religious material (Watt 1991) indicating a full spectrum of image deployment from the secular to the sacred. Early printed ballads, broadsheets and ephemeral productions such as the almanacs which sold straight to the streets and markets of the Early Modern period were the typographic and stylistic precursors of popular and later tabloid journalism (Shepard 1978: 24) Such illustration also assisted the commercial effectiveness of popular print as street hawkers needed to attract the attention of a passing public.

3 The evolution of the popular newspaper in Britain Despite the creative energy invested in making printed material more attractive to wider audiences, for the best part of two hundred years, regular periodical print publication remained restricted to the wealthier classes and the poor had to make do with the irregularity of ephemeral literature. The lifting of taxes and duties on the press from the 1830s onwards made newspaper and magazine production a

Visual Aspects of British Tabloid Newspapers

263

much more profitable activity and one which was, over time, increasingly targeted at lower socio-economic groups. Previously, formal newspapers and magazines had assumed that their readership was a relatively homogeneous upper middle class with interests and opinions which, although varied, were nevertheless characteristic of the range of bourgeois opinion at that time (Black 1991). This had been signaled in their language as well as in the absence of what their readership may well have considered a distracting illustration directed towards the semi-literate. The excess of words is best expressed by the Times’ famous insistence on the verbatim reporting of Parliament during most of the nineteenth century as an indicator of its elite discursive position. What was to emerge as popular print in the nineteenth century had its foundations firmly shaped by the two-way flow between the popular press and the chap book production of lively, illustrated stories throughout the eighteenth and nineteenth centuries. By the eighteenth century, for example, the Newgate Calendar was providing regular accounts of criminal trials and a key attraction was their illustrated content. It was very influential in developing the shape and style of sensational crime stories which were to drive the content of much of the popular weekly press in the nineteenth century such as the police gazettes. In Britain, the Sporting Courier and Cleave’s Weekly Police Gazette experimented with typography to draw attention to their more sensational stories; sex, sport and sensation became the three staples of this emerging popular form. Weekly illustration in the Sunday press was of particular importance in establishing the reach and the tenor of popular commercial periodicals. The first regular Sunday newspapers were the Observer (1791) and the Weekly Dispatch. By the 1840s with the appearance of Lloyd’s Illustrated London Newspaper (1842) and the News of the World (1843) Sunday papers had come to stand out on account of their lack of respectability, continuing a tradition of the coupling of image and low moral standing at the same time as they broke established traditions of Sabbath observation. The disreputable nature of the weekly Sunday newspapers was further amplified by their early acceptance of small, personal advertising imported from the American press in the 1860s. They first blended on a regular basis the woodcut and bold headings which would develop into the modern tabloid headline and they drew upon a popular cocktail of radical rhetoric and sensationalism from the varied traditions of illegal unstamped publications, the last dying speeches of convicted felons before execution and the contemporary printed ballad sheets. Information had begun its journey towards entertainment. Another area where Britain led the way in this two-way negotiation within the development of popular journalism was in the Illustrated London News (1842) and the Pictorial Times (1843) which from their launch were the first to develop a quick transfer of wood engravings from sketches – a technique which had been revived in the 1820s (Wiener 2011: 47). From the 1850s these provided illustrated news on

264

Martin Conboy

a weekly basis drawn from photographs and by this point they were beginning to rival the Sunday newspapers in their popularity.

4 American influences It is often claimed that American influences led the push towards popular mass newspapers in Britain (Wiener 1990). In a more recent assessment Wiener (2011) takes his Anglo-American comparison from 1829 when newspapers on both sides of the Atlantic were devoid of illustrations, headlines or crossheadings to break up the constant diet of undifferentiated print – not even to accompany the plentiful advertisements which were located on the front page. Horace Day’s New York Sun from 1833 was the first successful penny paper and the first mass newspaper to address its urban readership in a tone, “based on the language of the common man” (Whitby 1982: 25). It was tabloid-sized and comprised three columns rather than six as was the practice common for all newspapers. Michael Schudson argues that two models of journalism could be identified from this point onwards; a journalism of analysis and a journalism of entertainment (1978: 89). This latter variety quickly began to establish formal and stylistic characteristics. Editorials in some of the cheaper papers concentrated on short sentences for easy reading. Interviews and stories which exposed an interest of prurient readers in the affairs of their social superiors were first explored by Gordon Bennett in his reports in the Herald, illustrated by some of the first newspaper interviews in 1836, on the background to the trial of the socialite, Robinson, accused of murdering a prostitute named Jewett. There was also a good deal of early sensational reporting of gunfights, spiritualism and boxing. In this period, from a stylistic perspective, there was a good deal of what has been called “editorial billingsgate” (Mott 1961: 310) designed to echo the rough and tumble of popular name-calling. Accompanying these innovations was the editorial desire expressed by Bennett to provide “a correct picture of the world” (Crouthamel 1989: 47). In the case of American and later British popular newspapers, it was, as we will see, a pictorial metaphor that was to become increasingly literalized. Bennett’s use of woodcuts and wood engravings in his New York Herald led the way in illustration in daily newspapers although these had been developed from the Penny Magazine of London from 1832 which was exported to the USA. Its stories did not require contemporary illustration of the type which Bennett would develop. They still relied on more generic and slower paced reproduction. Things took a bold step forward from these traditions with the front page woodcut illustration of President Jackson’s funeral in 1845. Bennett soon began to produce sketches of living protagonists to enliven his pages from the 1850s. It was during the Mexican war of 1846 that headlines in multiple deck headlines were introduced and the gold rush in 1848–9 made them a common feature

Visual Aspects of British Tabloid Newspapers

265

in both the Sun and the Herald. These headlines grew in size and quantity depending on the scope of the story (Mott 1961: 292). Towards the end of the century, under the influence of Pulitzer’s New York World from 1883, there was an increase in the prominence of crusades which were designed to promote the newspaper at the same time as chiming with the concerns of readers, together with human interest stories with a great emphasis on sensationalist gossip and scandal, crime and disaster reporting and a great deal of illustration. A liberal use of adjectives in all stories demonstrated a preference for hyperbole in these papers. Once Pulitzer discovered that in Hearst’s New York Journal, he had a rival who was willing to attempt to outperform him in all aspects of his populist ambition, “yellow journalism” was born. It included a further intensification of the language of popular journalism. It turned, in an interesting formulation, “the high drama of life into a cheap melodrama” (Emery and Emery 1992: 191) although as Gripsrud has demonstrated with regard to tabloids in general, melodrama is not an insignificant social phenomenon especially when included in the fabric of popular newspapers (1992). The yellow journalism of the period of competition between Pulitzer and Hearst brought about a significant set of shifts in typographical layout which enabled the papers to be more easily and more rapidly digested. Scare-heads in either red or black made even the most trivial story scream with attention-grabbing force. It displayed a relish for the deployment of pictures and even faked pictures when the story required them! The typographical features also had an impact on the style of language used, compressing space and thereby making it more simple, snappy, condensed, lively and more consistently populist in tone. In this they attempted to recreate the urban vernacular of mass-readership America. New technologies and layout techniques could by-pass the norms of reasoned debate and go straight for the visceral appeal of the image with a short exclamatory headline to drive home a ready-made conclusion. Representing the world as spectacle and morality play at one and the same time drew upon the visualization of these processes in the new layouts of the popular press.

5 The early twentieth century consolidation As American popular journalism entered the twentieth century, it began to consolidate the language and the layout of that language in ways which would come to inform the first truly tabloid newspapers. The first American tabloid in spirit as well as format encapsulated the extreme of the process of visualization within popular journalism. Proprietor Patterson urged on by British media baron, Northcliffe launched the Illustrated Daily News from 1919. At first it was composed almost completely of pictures apart from a large banner headline and a small number of captions accompanying the pictures.

266

Martin Conboy

It established the content and style of the early tabloid and Bessie (1938) stresses the way in which populist editorial ambitions could be enhanced by an astute use of language: Each day play up one story, preferably related to crime, sex or heroic achievement […] Stand vigorously upon an editorial platform which is broad enough to permit policy to be liberal, conservative, patriotic or moral as the occasion seems to demand but never forget that the tabloid’s causes must be popular causes, at least in name. Practice a limited, colorful vocabulary. In sum, produce a picture newspaper which is a compendium of extraordinary happenings and simple comment, exciting, entertaining, reassuring and couched in the living language of the day (Bessie 1938: 131).

In 1924 Hearst launched a rival, the Daily Mirror and this was swiftly followed by the Evening Graphic from Macfadden. These new tabloids were described with perceptive honesty as a “journalistic mirror” of 1920s America (Bessie 1938: 119). Philip Payne who edited both the Daily News and the Daily Mirror is quoted by Stevens as saying that his goal was to illustrate every story with a photograph – “the very essence of tabloidism”. Stevens (1991) argues that this created an instant product-consumer bond since photography, “conveyed a story in a flash and made the reader feel he was part of the event” (Stevens 1991: 119–120). These newspapers, in a frenzy of competition extended the borders of acceptable visual practice to new dimensions. They were not above creating photographs of fictitious or rumoured events to support a story if there were no photographs in existence. These images became known as composmographs and were described in euphemistic fashion by their exponents as “interpretative reporting” (Emery and Emery 1992). Perhaps the most notorious visual episode of the period was the picture in the Daily News of Ruth Snyder in 1928 headlined “DEAD”. The photograph of a convicted murderer had been snapped by a journalist, witness to the execution, who had lifted his trouser leg to enable a hitherto concealed camera to take a photograph at the moment of her death. The tabloids’ pioneering of the techniques of photojournalism led to skepticism that it could be used to enhance serious news: “[…] photographs of events and personalities reproduced which are trite, trivial, superficial, tawdry, salacious, morbid or silly” (Taft 1938: 448).

6 New Journalism When Engel comments that what became known from the 1880s as the “New Journalism”, “had always been there; it was now merely breaking out of its bounds”, (1996: 46) it is to the increased regularity of this journalism on a daily basis that he is referring. It developed a style of journalism which was stridently self-promotional in its crusades, entertainment values and sensationalism – “the

Visual Aspects of British Tabloid Newspapers

267

phenomenon existed by calling attention to itself” (Koss 1981: 343). One of the ways in which this became visually apparent was in the typographical assimilation of American influences in the use of cross-heads and multiple deck stories for the first time consistently in the British press. T. P. O’Connor began his Star in January 1888 on his return from America where he had observed at first hand the rush to popular forms of journalism. From his radical perspective, he understood that it was in the layout as well as in the language where the interests of the people could be articulated in a way which could distinguish his paper in the market and it is hard to overlook his emphasis on the very visualized appeal of the project: We live in an age of hurry and of multitudinous newspapers […] To get your ideas across through the hurried eyes into the whirling brains that are employed in the reading of a newspaper there must be no mistake about your meaning: to use a somewhat familiar phrase, you must strike your reader right between the eyes (O’Connor 1889: 434).

By the end of the century the first changes were in the layout and structure of the language. The development of large type headlines necessitated a particular sort of abbreviated language to attract attention – “straight between the eyes” – as a marked contrast to the staid and monotonous presentation of the early century. Wiener (2011) sees the New Journalism as the culmination of a half-century of developments in the visual aspects of newspapers and stresses that these encapsulate not only a debate about the newspaper’s aesthetics but a much broader dialogue about the democratizing of knowledge, particularly political and cultural knowledge. The tight frame of traditional newspapers had been an exclusive device to restrict the circle of those drawn into debate on contemporary affairs. The tabloid therefore has always been a potentially political project not just an aesthetic construct although how people were drawn into the circle of debate and what happened to the diet of discussion are different and complex stories.

7 Popular and elite attitudes to image in the press Tabloid journalism communicates in a semiotic combination of typeface, visual impact, brand and linguistic style. By themselves none of these are determinant of tabloid style as the Daily Mirror demonstrated in 1903 when it was launched as a tabloid picture paper but took another 30 years before it was reformulated into something which we would recognize as a tabloid newspaper today. In fact, before the twentieth century photography and graphic illustration could be signifiers of very up-market or even politically influential publications such as early scientific journals in the nineteenth century and the highly respectable and didactic Penny Magazine which propagandized on behalf of the Society for the Diffusion of Useful Knowledge in the 1830s. The Picture Post and other photo-magazines from the

268

Martin Conboy

1930s provided photo-essays and pictorially driven documentary with a social edge which continued the seriousness of this particular visual tradition. Yet away from scientific and educational publications, there was a general journalistic avoidance of image in all but magazines, weekly popular papers, publications aimed at a family and/or a female readership. The failure of the Daily Mirror as an illustrated paper for women within twelve months of its launch seemed to confirm the gap between general journalism and the optimum deployment of the image. The late-coming of the Times to the inclusion of photographs from 1914 confirmed the impression that the elite press especially was keen to retain its status as a high brow purveyor of the written word through an unwillingness to engage with photographs or other illustrations until well after their popular cousins (Wright 2003: 65). Despite this limited resistance, the whole movement of newspapers has been towards a greater “visualization” of content which involves image as well as layout. This becomes noticeable in British popular newspapers from the 1930s before the tabloid takes it up as a speciality. Becker argues that the preponderance of photography in the modern tabloid press is one of the main reasons why the “popular” is denigrated as it has often been seen as “a direct appeal is seen as a threat to reason and to the journalistic institution’s Enlightenment heritage” (1992: 82). This means it is worth revisiting the ways in which photography was introduced into the daily popular press. Halftone printing process allowed print and photograph to be reproduced on one page. It was available from the 1880s but only the American “yellow press” made daily use of the technique; photographs were central to the sensationalizing agenda of these papers. Linotype printing was in common use by 1890s. The tabloids of the 1920s continued the trends developed by the yellow press with larger, brasher displays of photos in the 1920s. This meant that photography became associated with the inferior, popular end of the journalism market “photographs […] defeated their own object, disseminating news” (Taft 1938: 448).

8 The tabloid proper Like many of the developments of twentieth century journalism, the tabloid had its roots in the nineteenth century. The pharmaceutical company Burroughs Wellcome and Co. copyrighted the term “tabloid” in 1884 to advertise their marketing of compressed medicines in the form of small tablets. The term itself is a contraction of “tablet” and “alkaloid”. It was on New Year’s Day in 1901 that the first copy of a tabloid newspaper appeared. Harmsworth had been invited by Pulitzer to New York to oversee the production of an experimental, one-off, tabloid-sized version of his New York World. He showed his appreciation of the commercial appeal of the instant news digest when his slogan boasted: “All the news in sixty seconds”. Presciently, he announced that it would be the newspaper of the century.

Visual Aspects of British Tabloid Newspapers

269

In retrospect, we can see that this was not hyperbole but an accurate assessment of the future of not only the newspaper but for journalism in all its formats. The Daily Mirror from 1903 was the first attempt to produce a regular popular newspaper in this format targeting a female readership with a heavy emphasis on illustration while remaining every column inch a respectable paper for the middle class female reader. In broader terms of connecting in its language to a popular reader and aiming at the lower end of the social spectrum, this was not however a precursor of the tabloid in anything other than size; neither was it a commercial success. Although the original Daily Mirror (1903) and later the Daily Sketch (1908) preceded it, the first British newspaper which we would recognise as tabloid in format as well as content was the relaunched Daily Mirror from the mid-1930s. In the 1930s, under pressure from increasing competition for the popular reader, it was the incorporation of bigger and more frequent pictures into popular journalism which constituted the most significant change of the period. On August 7 1933 Christiansen on the Daily Express produced a cleaner, better spaced newspaper with more and bigger headlines and more cross-heads to break up the page. It was followed by all its rivals including the re-launched Daily Mirror which from 1934, under the editorial direction of Bartholomew, triggered the so-called tabloid revolution with its signature heavy black bold type for its headlines, pin-ups, young style, simplified language and a prominent use of pictures to reach a new readership. The success of the re-launched Daily Mirror was built on a formula based on two American tabloids, the New York Daily Mirror and Daily News, skillfully adapted to a British cultural context and combined with the advice of an American advertising agency, J. Walter Thompson deploying the old Northcliffe formula of the telegraphic sentence within a modern layout (Edelman, 1966: 40). The first stage of the British version of the “Tabloid Revolution” took place between 1934–37. Curran and Seaton have argued that this constituted a “key moment in the incorporation of the popular press by the entertainment industry” (1993: 53) with its slashing of political, social and economic news to formulate a newspaper of popular appeal to the working classes. Yet there are others who would claim that making newspapers more readable by improving their typeface is simply a courtesy to their readers and done to a large extent to maintain pace with changes in presentational style and layout elsewhere in the media environment. In terms of popular tabloid newspapers proper, the visual aspect was the first and most obvious signifier of change in modern Britain. The prominent use of large type, bold headlines and photographs to lead stories sought to capture a particular audience; one more in tune with the increased pictoriality of popular culture in the first half of the twentieth century. However it has had a different and more lasting impact which is still being felt today. It shifted public engagement with politics while allowing newspapers to link themselves increasingly with the dynamics of a more broadly diffused popular media culture.

270

Martin Conboy

The Daily Mirror began to articulate a more stridently proletarian voice than any other commercial newspaper. In addition, it was able to do this in a form which advertisers found of appeal to the new sense of solidarity among the working classes as the Second World War drew to an end. It was a format which meshed with the language and the aspirations of its idealized readers, as Smith has commented: “a paper that will conveniently stuff into the pocket of overalls, and that can be read in brief intervals between manual work” (Smith 1975: 233). The most significant, recent development in the history of British tabloid journalism was the relaunch of the Sun in 1969. Thomas has summarized the epochdefining pitch for a new, downmarket popular newspaper in Murdoch’s conviction that that the Mirror had become too highbrow for its readers by the 1960s. By this point, Cudlipp had begun to reposition the Mirror, if not upmarket, then certainly towards a more serious market with features such as, “World Spotlight” and “Mirrorscope” with serious engagement with photo-essays and illustration of news across the globe. With former Daily Mirror journalist Larry Lamb, Murdoch set out to produce an alternative that was explicitly based on an updated version of their rival’s irreverent approach of previous decades (Thomas 2005: 72). Lamb scrapped all specialist news reporters except political journalists and specialized in human interest, sensation and sex significantly narrowing its news agenda (1989: 22–23) and adding a more garish populism and more consistent celebrity coverage as the norm. The Sun targeted younger readers, dropped the increasingly serious ambitions of the Mirror, embraced the permissiveness of the age and provided a disrespectful, anti-establishment, entertainment-driven agenda. The popular cultural credentials of the Sun were reinforced from the start by its astute exploitation of television advertising to reinforce itself as a brand in the public mind and by its increasing synchronicity with the agenda of popular television programmes and domestic television stars in its own content. Its front pages became renowned for their use of large, bold and irreverent headlines sometimes comprising a single word. The Sun from 16 November 1969 became a belated vehicle of the Zeitgeist of the 1960s as it began to change both the face of journalism and the face of Britain. Emblematic of this shift on 17 November 1970 was its first Page 3 Girl. Murdoch claimed it was “a statement of youthfulness” (Taylor 50: 1992) and it provided a further development of the tabloid as something quite distinctive within British culture. From the Daily Graphic in the 1920s onwards there had been an incorporation of nudity into the mainstream of the press via men’s magazines (Benwell 2003). Tabloids developed the spectacle of the female body as part of what Holland (1998) has called the “sexualization of popular culture”. This visualization of the female has spread through the other pages of more contemporary mainstream newspapers into issues of celebrity, diet, fashion, agony aunts, photoplays but it seems to remain a dividing line which demarcates the world of the popular tabloids from the elite press. Engel (1996: 17) comments that each generation of successful, popular newspapers has been undercut by a new wave which has been able to capture the reader-

Visual Aspects of British Tabloid Newspapers

271

ship of a lower social grouping and integrate that readership within the widest range of appropriate other media such as advertising, cinema and television. The Daily Star certainly attempted to outperform the Sun in this respect in the basement of tabloid taste. It was aimed at the lowest end of the market in terms of sensation and dubious journalism ethics, sexualization of popular culture with appeal to an even younger market than the Sun’s. From 1978 the Daily Star was described as having, “a circus layout that fairly burst from the pages […] the paper used more taping, more italics, more reverses, and more graphics in conjunction with sensational heads and stories to give a sense of excitement and power” (Taylor 1992: 45).

9 Tabloidization and the visual The term “tabloidization” first emerges in Webster’s dictionary in 1991. It is a critical attempt to come to terms with the fruits of a longer process which has to an extent impacted upon all forms of journalism both high and low brow. It is generally considered to be driven by changes in the style as well as the approach to journalism in an era dominated by hyper competition and the permeation of popular culture into all areas of public life (Conboy 2007). It is sometimes used synonymously with the expression “dumbing down” and is equally contentious in its implication that journalism has somehow become debased as it enters the early years of the twenty first century. In definitional terms, tabloidization may refer to an increase in news about celebrities, entertainment, lifestyle features, personal issues, an increase in sensationalism, in the use of pictures and sloganized headlines, vulgar language and a decrease in international news, public affairs news including politics, the reduction in the length of words in a story and the reduction of the complexity of language and also a convergence with agendas of popular and in particular television culture. Tabloids are characterized as being primarily to do with a combination of format and language: “editorial matter is presented in emotive language in easy-to-consume formats” (Rooney 2000: 91). The first key developments in the tabloiding of British newspapers are defined initially and literally by the reformatting into tabloids of the Sun in 1969 and the Daily Mail in 1971. The Daily Express completed the middle market shift to the format in 1977, the News of the World from 1984 and the Sunday Express went tabloid from 1992. The next stage was the tabloidization of most of the elite press, with the Independent leading the way in 2004 followed by the Times and the Guardian’s shift to the slighter larger “Berliner” format in 2005 leaving now only the Daily Telegraph holding out as a genuine broadsheet in the general national daily market. The second key development within tabloidization is the spread of the characteristic tabloid style and news values to the elite press. McLachlan and Golding

272

Martin Conboy

(2000) chart the growth in visuals in relation to text as one indicator of tabloidization, squeezing text out of the frame. Other strategies which conform to the tabloidization model according to their study have included the inclusion of more lifestyle and consumer coverage and an increasing number of their journalists being the media star treatment with a photograph accompanying their by-line and freer rein to write from a more personalized perspective. Tunstall argues that objectivity even on elite newspapers has shifted considerably towards more of a house-style orientation than anything absolute and interprets this as part of their need to differentiate themselves within the media market: “Newspapers, in any case, need to ‘take it further’ than T.V. and radio. They need to go beyond the neutral and objective rituals of balance” (Tunstall 1996: 197–198). Here he is talking about the increasing drive to editorialize, provide stronger opinions and more commentary as the focus for speed of coverage and proximity reside ever more clearly with television. However, a large component of that shift involved an incremental transition to a more visual style, more illustration and the eventual reformulation of the broadsheets into compacts between 2004–2005. There is a popularizing arc at work within certain aspects of news media which could be described as a sort of progression if it were not crisscrossed by other more complicating aspects which pull the arc in certain other directions like a series of competing magnetic fields. This arc is determined by the presence of news media within economic markets and to an extent this means that what maximizes profit tends to be adopted. So, including images has in general terms been something which has made media products more “popular” with all the complexity which that term carries. This tendency has flowed from magazines into popular newspapers and then gradually into other media forms including broadsheet newspapers which have even tried to obscure the dreaded “tabloidization” thesis by the use of the euphemism “compact”.

10 Celebrity and tabloid culture Celebrity news began sedately with illustrations of royal holidays in the Illustrated London News in the 1840s and the gossip columns beginning with the work of Yates in various publications in the 1850s. These provide a precursor to the later informal visualization of celebrity. They develop into a new lease of life in the 1880s and 1890s as part of the surge driving the New Journalism with its growing interest in personality and the personal touch. It reaches a first crescendo in the newspaper wars of the popular press in the 1920s and 1930s. Celebrity news is a perfect conduit for the tabloids as it allows the fluid interchange between fact and fiction as well as the easy generic transfer from information to entertainment (Hill 2005: 15) upon which their values are centred. Tabloid tendencies to cross-reference celebrity and entertainment issues can be witnessed

Visual Aspects of British Tabloid Newspapers

273

increasingly as part of the repertoire of the elite press and broadcast journalism in all its forms. They constitute a strategy to reach new audiences in a crowded market and a changing cultural environment. Celebrities are increasingly providing an alternative public sphere. The form of intimacy thus generated between reader and tabloid journalism is an important element in establishing and developing a resonant sense of mediated community in Britain. The popular representation of and intrusion into the lives of celebrities forms part of the developing tabloid culture from stylized Hollywood portraits approved by the studios of the 1920s and 1930s, to the paparazzi and the contemporary combination of gossip and photography. British tabloids have brought us “Princess Diana in the gym” and the “toe-sucking Princess”; off-screen antics of reality television shows and off-pitch peccadilloes of footballers and, sometimes their wives. Royal stories may have peaked in the 1990s in the British tabloids with the colourful and controversial adventures of Diana, Princess of Wales but there were still notable stories beyond the 1990s as tabloids became more desperate than ever to milk Royal scandal or even fabricate it to boost sales such as in the “Spy in the Palace” coup by the Daily Mirror and the Burrell affair (Coward 2007). Technology brings not just speed but more importantly for the tabloids and their substitute for democratization, proximity. Bringing the famous closer to us through the telephoto-lens and the smart phone for our perusal and ultimately judgment is a gift to tabloid media. Other features of the tabloid such as the use of celebrity to act as a hook in news stories, the relating of news to features of more general popular culture, the preference for the nationally specific against the international and the downplaying of “political” coverage have all migrated to television. Bird, from a more integrated, almost anthropological perspective sees the contemporary tabloid as unimaginable outside other historical and cultural phenomena: […] They complement the star system, the other popular media, the class system, and the gender system. They exist because of television, newspapers, movies, and a vast range of folk narratives and values (Bird 1992: 1–2).

11 Televising the tabloid The popularizing arc has extended to television, perhaps inevitably as it has generated its own aesthetic to distinguish itself from the traditions of radio and newspapers to style itself the most entertainment oriented of news media. This has been identified in the polemic of Neil Postman (1986) which explored the relationship between the visualization of communication as a mode of entertainment and his view of its detrimental effect on informed democratic debate. The first experiment in adopting television news to changing patterns of consumption and lifestyle was the regular breakfast television news in 1983 providing

274

Martin Conboy

a domestication of television news with a highly practised visual aspect to the presentation. This saw a shift in the hitherto rather austere image of the newscaster. These new-style journalists, easy on the eye and with tones suited to new style of “sofa journalism” were very different creatures to their counterparts in other broadcast news programmes. Breakfast television news introduced a softer range of human-interest angles on the stories they covered and because of its manwoman double-act it was dubbed “Ken and Barbie” journalism (van Zoonen 1998: 40). After breakfast television news the presentational styles and formats of mainstream television news, started a process of rapid transformation. Regional journalism began to become more oriented to a lighter style with more human interest and good news stories. On prime time mainstream news there was an increased use of text on the screen, a sound-bite syntax which reduced the length of individual news items, the use of graphics to illustrate news, newscasters and reporters on first name terms with each other if not quite yet with the politicians they interview, are all indications of the popularization of the news format. The perceived “tabloidization” of television news and current affairs generates negative critiques as well as more optimistic assessments of these changes. Television journalism’s contemporary developments fit easily into a catalogue of criticism attached to the perceived trend towards tabloid format, content and philosophy: “image crowding out rational analysis” (Bird 2000: 221) while from a more positive perspective perhaps, MacDonald (2000) emphasizes its emotive elements, while Langer (1998) explores the broader remit and relevance of the “other” news and Harrington (2008) argues for a reassessment of the complexity of the format against the severity of critical accounts. The whole tabloidization debate includes commentary and judgment on the visualization of the newspaper and the spread of cognate styles of presentation to other media. With tabloid newspapers as with television news, it’s not about the increase in reliance on the image, after all photojournalism had a long and illustrious tradition as social and cultural commentary before the rise of television journalism, and more the purposes to which the visual has been put. The BBC has not been immune from moves to introduce more visually appealing television newscasters and reporters. This has embroiled the Corporation, in line with commercial rivals, in arguments over the suitability of older women as newsreaders, in particular when Moira Stuart resigned in 2007 as it was alleged she was fearful of being moved out for a more photogenic and younger replacement. The logic of more aesthetically-pleasing TV presenters seems to fit within a general movement of broadcast news towards entertainment values and may even represent a part of a bid to retain some of a younger audience which is turning off from mainstream news and politics. The most pervasive influence on the form and scope of journalism of the twentieth century was undoubtedly the tabloid, especially in its elaboration of first, an appeal to a broadly working class readership and subsequently its incorporation

Visual Aspects of British Tabloid Newspapers

275

of a more general popular culture. In fact, when any form of popular media makes the transition to tabloid, its style becomes shaped into a format which has a protean potential to migrate and influence other media genres. Biressi and Nunn (2007) assert that beyond newspapers and indeed beyond news media, we are now enveloped within a broadly tabloid culture. The language and visual style of both the popular newspaper in Britain and then the tabloid have had an incremental impact on newspapers generally over the last hundred years. Not only have broadsheet newspapers been driven for commercial reasons to adopt a “compact” format but the emphasis and style of the language of these newspapers have been orientated more towards the news values of the tabloids as these newspapers all try to emphasize their congruence with popular culture in an era of unprecedented competition in the media. It may be, as Bromley and Tumber (1997) have speculated, that after the gradual convergence of tabloid and broadsheet styles, a process of re-specialisation may see different newspapers (particularly in their on-line manifestations) starting to diverge considerably in their tone, style and coverage once again.

12 Tabloid trends and democracy As a consequence of the above trends, debate about the broader significance of the politics of the tabloid is a contested terrain. Sparks (2000) emphasizes the political aspect of this process as he sees it, with the tabloid news devoting, “relatively little attention to politics, economics, and society and relatively much to the personal and private lives of people, both celebrities and ordinary people, and relatively little to political processes, economic developments, and social changes” (Sparks 2000: 10). In addition we might add that it stresses the pictorialization of such stories and their headlining. On the other hand, McNair (2003) sees changes in the content and style of the elite press as a positive move towards a more inclusive even democratic journalism culture: “Less pompous, less pedagogic, less male, more human, more vivacious, more demotic” (McNair 2003: 50). As has been argued elsewhere (Conboy 2006) there is nothing necessarily reactionary or ignorant in the techniques used by the tabloids in themselves. A recent article in the British Journalism Review, addressed to a readership located predominantly in journalism, emphasizes the positive attributes of this kind of journalism while expressing caution over their irresponsible handling of immigration , asylum and Europe within that idiom: […] thanks to the exploits of Parry and co – and, more recently, Mazher Mahmood’s encounter with Sven-Göran Eriksson as described in the News of the World – I gained a new appreciation (and occasionally, a grudging admiration) for what the tabs do. There is a certain gaudy brilliance to stunts like these. A weird hybrid of investigative journalism and the print equiva-

276

Martin Conboy

lent of reality TV […] whiff of old-fashioned muckraking […] the tabs appoint themselves the official voice of the people […] Reality tabloidism is bold, enterprising, lively and fun. But is it good journalism? There’s no reason why it can’t be (Geary 2008: 43).

13 Conceptualizations of image in the tabloid world Although targeted at a mass audience, the tabloids in their idiom and visualization have a consistent appeal couched within a textual display of intimacy with idealized individual readers. The visual aspect helps to generate that “up close” feel of these papers and of the tabloid genre more broadly. The tabloid trend is encouraged by the technologies of digital photography and the general visualization of culture which can be seen in the increased prominence of images in newspapers, in both print and online versions. Becker (1992: 90) has explored the ways in which photographic aesthetics function within a tabloid frame. She highlights the irony that most often it is in celebrity photographs where the subjects are enabled to enhance the performativity of celebrity whereas the traditional “look” of tabloid intrusive photography is best used to enhance their news coverage, bringing this within the tabloid idiom: The higher truth revealed in the candid moment is a notion that is repeated and expanded in the tabloids’ photographic coverage of news events (Becker 1992: 90).

In terms of broader news coverage, technical proximity comes to equate to an aesthetic lack: Technical ‘flaws’ like extreme graininess and underexposure have actually become conventions of the tabloids’ style, visually stating the compromises the newspaper will accept in its commitment to presenting the ‘real’ story. The techniques work to enhance the appearance of candour, lending additional support to the construction of these photographs as authentic (Becker 1992: 91).

Today’s front pages continue to be a rich cross-referencing of celebrity culture, the rhetoric of the tabloids and their dependence on visualizations of popular culture and humour and continue within Becker’s aesthetic frame. The visual in newspaper terms is a combination of image, layout and format (page size). But the tradition of visualization emerging from the tabloids into other media formats has its own range of specific attributes. The popular moves into the tabloid shape and what emerges culturally from within that shape has already been affected by the dynamics of that import. We could take a lead from Halliday’s metafunctional semiotics – ideational, interpersonal and textual (1978) with a view to constructing a theory of how the visuality of the tabloid has come to combine into a specific communication genre. Tabloids combine, a representation of social relations, interaction with a media institution and invitation to participate in an ideological continuum.

Visual Aspects of British Tabloid Newspapers

277

Bird (2009) sees tabloidization as less of a problem than the technological innovations which threaten journalism as a whole and one might argue that the highly visualized aesthetic of the tabloid has long been part of journalism’s survival strategy. Furthermore, heavy visualization assists the branding of tabloids and creates a further blurring with another related media field, that of marketing. Tabloid front pages act as a marketing device, a poster for the wares on sale because of the traditional street appeal to casual buyers; the louder, the brasher, the more contemporary, the better.

14 Conclusion The emergence of a distinctive tabloid idiom indicated a shift in the social parameters of journalism. This idiom blends sensation and a calculated disrespect and suspicion of authority, particularly political authority, within an overall concentration on the broadest appeal of contemporary popular culture. Focusing on the visual aspect of tabloid media can reveal the distinctive nature of this sub-genre of journalism and reflect upon the social implications of its attempts to articulate a populist, media-centric version of contemporary society for a mass audience. Glynn argues that tabloids produce “popular counterknowledge” which engenders skepticism of official viewpoints (Glynn 2000: 177) and in our contemporary tabloid culture, the visual is enlisted in this process to deconstruct our heroes and expose our establishment figures of respectability from politicians to football managers. Debrix (2008) defines the tabloid era as a time where “image outweighs substance” in a similar fashion to Bird but he is also, as is this author, keen to stress the analytical over the judgemental. Along with Langer (1998) he sees the tabloid genre now most completely represented in the televisual form: The plurality of messages, the hybridity of forms, and the eclecticism of styles that seem to characterize tabloid TV extend well beyond the television medium and affect many other popular cultural genres that, in the 1990s, increasingly defined themselves against the background of television’s upbeat real-time mode of information and entertainment (Debrix 2008: 6).

Tabloid culture literally visualizes the world with its powerfully persuasive and entertaining combination of methods, techniques, textual or visual tricks perfecting in negotiation with markets and audiences down the best part of two centuries. It has adapted to the pressures of the postmodern age so successfully that Glynn can claim that we are living in the golden age of tabloid culture. In combination with their language, the tabloids provide a powerfully iconic representation of Britain in this densely visualized media era.

278

Martin Conboy

References Baynes, Ken (ed.). 1971. Scoop, Scandal and Strife: A study of photography in newspapers. London: Lund Humphries. Becker, Karin. 1992. Photojournalism and the tabloid press. In: Peter Dahlgren and Colin Sparks (eds.), Journalism and Popular Culture, 131–153. London: Sage. Benwell, Bethan. 2003. Masculinity and Men’s Lifestyle Magazines. Oxford: Blackwell. Bessie, Simon. 1938. Jazz Journalism. New York: Dutton. Bird, Elizabeth S. 1992. For Enquiring Minds: A cultural study of supermarket tabloids. Tennessee: The University of Tennessee Press. Bird, Elizabeth S. 2000. Audience demand in a murderous market: tabloidization in US television news. In Colin Sparks and John Tulloch (eds.), Tabloid Tales: Global debates over media standards, 213–228. Oxford: Rowman and Littlefield. Black, Jeremy. 1991. The English Press in the Eighteenth Century. Aldershot: Gregg Revivals. Bromley, Michael and Howard Tumber. 1997. From Fleet Street to cyberspace: the British “popular” press in the late twentieth century. European Journal of Communication Studies 22(3): 365–378. Burke, Peter. 1978. Popular Culture in Early Modern Europe. London: Temple Smith. Conboy, Martin. 2006. Tabloid Britain: Constructing a community through language. Abingdon: Routledge. Conboy, Martin. 2007. Permeation and Profusion: Popular journalism in the new millennium. Journalism Studies 8(1): 1–12. Coward, Ros. 2007. What the butler started: Relations between British tabloids and monarchy in the fall-out from the Paul Burrell trial. Journalism Practice 1(2): 245–260. Crouthamel, James L. 1989. Bennett’s New York Herald and the Rise of the Popular Press. New York: Syracuse University Press. Curran, James and Jean Seaton. 2003. Power Without Responsibility: The press, broadcasting and new media in Britain. London: Routledge. Debrix, François. 2008. Tabloid Terror: War, culture and geopolitics. Abingdon: Routledge. Edelman, Maurice. 1966. The Mirror: A political history. London: Hamish Hamilton Ltd. Engel, Matthew. 1996. Tickle the Public: One hundred years of the popular press. London: Gollancz and Prentice Hall. Emery, Michael and Edwin Emery. 1992. The Press and America: An introspective history of the mass media. Englewood Cliffs: Prentice Hall. Geary, James. 2006. In praise of the tabs [sort of]. British Journalism Review 17(1): 41–44. Glynn, Kevin. 2000. Tabloid Culture: Trash taste, popular power, and the transformation of American television. Durham: Duke University Press. Gripsrud, Jostein. 1992. The aesthetics and politics of melodrama. In: Peter Dahlgren and Colin Sparks (eds.), The Press and Popular Culture, 84–95. London: Sage. Halliday, Michael. 1978. Language as Social Semiotic: The social interpretation of language and meaning. London: Arnold. Harrington, Stephen. 2008. Popular news in the 21st century: Time for a new critical approach. Journalism: Theory, Practice and Criticism 9(3): 266–284. Holland, Patricia. 1998. The politics of the smile: “soft news” and the sexualization of the popular press. In Cynthia Carter, Gill Branston and Stuart Allan (eds.), News, Gender and Power, 17–32. London: Routledge. Holland, Patricia. 1998b. The direct appeal to the eye? Photography and the twentieth-century press. In Asa Briggs and Paul Cobley (eds.), The Media: An introduction, 412–425. New York: Addison Wesley. Hill, Annette. 2005. Reality Television. Abingdon: Routledge.

Visual Aspects of British Tabloid Newspapers

279

Koss, Stephen. 1981 [1984]. The Rise and Fall of the Political Press in Britain. 2 Volumes. Chapel Hill: University of North Carolina Press. Lamb, Larry. 1989. Sunrise: The remarkable rise of the best-selling, soaraway Sun. London: Papermac. Langer, John. 1998. Tabloid Television: Popular journalism and the ‘other’ news. London: Routledge. MacDonald, Myra. 2000. Rethinking Personalization in Current Affairs Journalism. In Colin Sparks and John Tulloch (eds.), Tabloid Tales: Global debates over media standards, 251–266. Oxford: Rowman and Littlefield. McLachlan, Shelley and Peter Golding. 2000. Tabloidization in the British press: a quantitative investigation into changes in British newspapers. In Colin Sparks and John Tulloch (eds.), Tabloid Tales: Global debates over media standards, 75–90. Oxford: Rowman and Littlefield. McNair, Brian. 2003. An Introduction to Political Communication. London: Routledge. Mott, Frank L. 1961. American Journalism: A history of newspapers in the US through 200 years 1690–1960. Basingstoke: Macmillan. O’Connor, Thomas P. 1889. The new journalism, New Review, 1 October: 423–434. Postman, Neil. 1986. Amusing Ourselves to Death. Public discourse in the age of showbusiness. London: Methuen. Rooney, Dick. 2000. Thirty years of competition in the British tabloid press: the Mirror and the Sun. In Colin Sparks and John Tulloch (eds.), Tabloid Tales: Global debates over media standards, 91–110. Oxford: Rowman and Littlefield. Schudson, Michael. 1978. Discovering the News: A social history of American newspapers. New York: Harper. Sekula, Allan. 1984. Photography against the grain: Essays and photo works, 1973–1983. Nova Scotia: Press of the Nova Scotia College of Art and Design. Shepard, Leslie. 1973. The History of Street Literature. London: David and Charles. Smith, Anthony. 1975. Paper Voices: The popular press and social change 1935–1965. London: Chatto and Windus. Sparks, Colin. 2000. Introduction: the panic over tabloid news. In Colin Sparks and John Tulloch (eds.), Tabloid Tales: Global debates over media standards, 1–40. Oxford: Rowman and Littlefield. Stevens, John. 1991. Sensationalism and the New York Press. New York: Columbia University Press. Taft, Robert. 1938. Photography and the American Scene: A social history, 1839–1889. New York: Dover. Taylor, Sally J. 1992. Shock! Horror! The tabloids in action. London: Black Swan. Thomas, James. 2005. Popular Newspapers, The Labour Party and British Politics. Abingdon: Routledge. Tunstall, Jeremy. 1996. Newspaper Power: The new national press in Britain. Oxford: Clarendon. Watt, Tessa. 1991. Cheap Print and Popular Piety, 1550–1640. Cambridge: Cambridge University Press. Whitby, Gary. 1982. The penny press and the origins of American journalistic style. Studies in Journalism and Mass Communication: A Review from the Texas Journalism Education Conference: 23–34. Wiener, Joel H. (ed.). 1988. Papers for the Millions: The New Journalism in Britain, 1850–1914. New York: Greenwood Press. Wiener, Joel H. 2011. The Americanization of the British Press, 1830–1914: Speed in the age of transatlantic journalism. Basingstoke: Palgrave Macmillan. Wright, Jeff. 2003. The Myth in the Mirror, British Journalism Review 14(3): 59–66. Zoonen van, Liesbet. 1998. “One of the girls”: The changing gender of journalism. In Cynthia Carter, Gill Branston and Stuart Allan (eds.), News, Gender and Power, 33–46. London: Routledge.

Gwen Bouvier

12 British press photographs and the misrepresentation of the 2011 ‘uprising’ in Libya: A Content Analysis Abstract: This chapter carries out a content analysis of newspaper photographs in the British Press that represented the uprising and NATO strikes in Libya in 2012. It begins by placing the events in Libya in context: by showing first the complex history of Libya, revealing that the “peoples’ uprising” was rather comprised of one particular regional political interest and was lead by elites concerned to hold onto the oil money and second at the NATO power’s negotiations to gain new access to oil agreements. The chapter then analyses the photographs from four newspapers over a one week period. Photographs foreground the role of an ordinary ragtag people's army, although without showing actual battle and with a focus on small disorganised groups. The democratising NATO aircraft are shown in hangars or in flight but are not seen in bombing raids. Other images show Western politicians and women and children. The chapter shows how these photographs were one important tool in the political misrepresentation of Libya in the British press, justifying intervention and silencing the motivation of oil.

1 Introduction In 2011 the British news media reported on an “uprising” in Libya. Reports revealed that the Libyan people were now challenging the rule of the dictator Gaddafi, long an unpopular figure with Western democratic nations. He was formerly aligned with the Soviet Union and against the West, known for the brutal way he treated his population, and symbolic of all that was “unconventional, belligerent and aggressive in terms of foreign relations”. (Metz 2004: 184) This event was presented in the context of other “uprisings” in Egypt and Syria that took place around the same time. There was a narrative presented that this was a popular and united, rather than factional, people’s push with a common goal of attaining Western democratic type social structures. Yet exactly who was driving these uprisings, how they were different and the same across the three countries, and why they were happening now was not being clearly explained. Nor, importantly, were the events in Libya, and the decision to support the rebels, placed within the context of power relations in the Middle East nor connected to the nature of the oil interests of Britain, France, Italy and the US who financed the airstrikes and the provision of arms to help defeat Gaddafi’s forces. All that was clear, as events developed, was that these ordinary people needed the help of the Western powers as the human toll increased and Gaddafi struggled to keep control over the coun-

282

Gwen Bouvier

try. The narrative suggested that the conflict only persisted through the brutality of a desperate ruthless dictator as he struggled to hang on as the popular uprising gradually and happily swallowed up his reign. In this chapter I look at the role of press photography in the representation of this uprising and the NATO involvement that obliterated Gaddafi’s attempt to retain control. The scholarly literature on the role of photography in news in general and specifically in the representation of war has suggested that far from documenting conflicts, photographs rather tend towards clichéd moments from wellestablished news frames and memorable moments, and are for the most part the result of photo opportunities that are well managed by governments. Using content analysis and some closer readings of individual images, this chapter asks what kinds of images frame the conflict in Libya for newspaper readers. It asks who was shown to participate, in what settings, what they were shown to be doing and how they were evaluated. These representations are placed first against the actual nature of the events – i.e. who actually comprised the “uprising”, why it took place at this particular time, and what the Western powers had to gain – and second in the context of the scholarly literature on the role of press photography in times of war. The chapter points to the way that photojournalism does indeed tend towards clichés, simplifications and ethnocentric views. But specifically in this case these processes completely misrepresent and gloss over the nature of this conflict that was clearly shaped in the interests of Western powers and their oil companies.

2 Who participated in the uprising? As regards the identity of the rebels and who took part in the “uprising”, we need to place this in the longer history of Libya. Libya was formerly formed by the colonial power Italy, bringing together the three provinces Cyrenaica, Tripolitania and Fezzan. (Malcolm and Losleben 2004: 8) The factional division between these provinces, themselves long since dissolved before Gaddafi, is one part of understanding the origins of the “rebels”. (Dziadosz 2011) The King, supported by the British and overthrown by Gaddafi in 1969 in the military coup, was from the province of Cyrenaica, which is the province that contains 80 % of Libya’s oil. It is from Cyrenaica that the core of the opposition movement arose that were described as “rebels”. (Mittelstaedt and Windfuhr 2011) These also included and were supported by government elites who were controlling oil. (Hamilton 2011; Vira et al. 2011: 10) There had always been tensions between Gaddafi and Cyrenaica. Under Gaddafi, who came from the poorer province of Tripolitania, wealth had tended to flow out of Cyrenaica and into Tripolitania and specifically into the hands of those close to Gaddafi, or in positions of power in his government. (CFR 2011) However, in the case of Libya it would be wrong to create a picture of these

British press photographs

283

provinces as being comprised of a single coherent ethnic or cultural voice given the tribal and fragmented nature of Libya’s population. (Obeidi 2001: 43, 115; Blanchard and Zanotti 2011: 4) The fight back by Gaddafi’s forces in the uprising endured simply as he did still have support of local tribal-based groups, much of the army, and of powerful groups without connections to Cyrenaica along with mercenaries from Sudan, Niger and Mali. (Vira et al. 2011: 22, 26) This fight back provided a need for a change in narrative by the British press. The events had been presented first as an uprising of a “people” but this then turned into a need to protect them from destruction, leading to the need for the intervention of the NATO forces. This also created complications for the NATO leaders as they needed to be seen to be supplying the means, rather than carrying out the actual battle. To some extent, it could be argued, what happened was in the way of old style imperialism.

3 Why the uprising happened at this particular time Why the “uprising” happened at this particular time is a complex story, but a number of issues are salient. After coming to power, Gaddafi nationalised oil in 1972 which was of course not popular with the Western oil companies operating there. Colonel Qaddafi proved to be a problematic partner for international oil companies, frequently raising fees and taxes and making other demands. (Krauss 2011) A new government with close ties to NATO could have provided an easier partner for Western nations to deal with. When Gaddafi came to power the wealth from the oil was diverted away from Cyrenaica and largely placed in the hands of an elite which ruled Libya (Vira et al. 2011: 67). One of the observed features of Libya has been the massive inequalities in wealth distribution given its oil supplies and small population. However, by 2007 Gaddafi had made the decision to make sweeping reforms into how this oil money was distributed, talking critically about how much of the profits were channelled into a small number of hands of members of the country’s elite. (Pfeiffer 2008) This wealth was to be rechanneled into public projects such as schools and health services, although this was something he had on previous occasions stated that he would do and had not come through on. (Haggar 2009: 127) But there was evidence that in the 21st century, Gaddafi had changed his outlook. Libya had shown a trend towards improved relations with the Western powers and a reengagement with the global economy. This included the abandonment of arms production in 2003, and the developments of international investments and more open relations with the US (BBC May 15th 2006). There had even been plans to sell a large segment of Libyan oil to a US equity company in 2007.

284

Gwen Bouvier

But according to The Economist (Apr 1st 2011) Gaddafi’s decision to redistribute oil money was not being met with enthusiasm by the elite that had so far been its beneficiaries, and therefore they speculated, could well mark the end of his regime. During the uprising the elite who controlled his government, says The Economist, were defecting to the other side. These people were the very owners of the Libyan oil company Tamoil and risked having their entire international-oriented lifestyles, with condos in Monaco, international bank accounts and children at Oxford, stopped (The Economist Apr 1st 2011; Hamilton 2011). Al Jazeera (Mar 3rd 2009) reported that the majority members of the Libyan cabinet had voted against Gaddafi’s plans to redistribute the oil money. Since it was leaders from Cyrenaica who were central in Gaddafi’s downfall, one problem to be faced by the new regime would be how to accommodate their demands for power and autonomy. Immediately after the defeat of Gaddafi, it became apparent that more than one person and more than one faction came to claim leadership for the eastern province, but all had the common claim to its wealth and oil.

4 Why the Western powers helped the rebels Finally we come to why the Western powers wanted to help the rebels. Moore (2011), writing in The Telegraph at the time when it was clear that the Western powers would intervene, cynically explains that “a defensible position about intervention, especially intervention in the Muslim world, is being built up”. International relations expert Joshi (2010: 18) pointed out that Turkey had immediately criticised the intervention as “an oil grab” and that a French minister had already publicly announced that France would benefit from the commercial and energy deals that would come after Gaddafi’s defeat (Joshi 2011: 19). Krauss (2011) points out that Italian Foreign Minister Frattini had also made it clear that as regards the distribution of Libyan oil, once Gaddafi was removed, the Italian oil company Eni “will have a No. 1 role in the future”. Kovalyova (2011) for Reuters reported a spokesman for BP stating that they “fully intend to return to Libya to fulfil our contract when conditions allow”. The New York Times (Aug 22nd 2011) in an article headed The Scramble for Access to Libya’s Oil Wealth Begins, explains how the very countries that had provided the biggest military investment to support the “uprising”: BP of Britain, Total of France, Eni of Italy, Repsol YPF of Spain and OMV of Austria, “now stand to gain the most once the conflict ends”. In contrast, the countries that had questioned the support of the uprising and proposed a negotiated end to the unrest were not welcome. The New York Times reports that a spokesman for the Libyan oil company Agoco had announced that “we may have some political issues with Russia, China and Brazil” (Krauss 2011). According to the Guardian, countries like Britain had invested around £ 1.25 billion in the war

British press photographs

285

effort and expected a return on their money, although the newspaper points out “the costs of the whole war are swathed in mystery” (Hopkins 2011). At the end of the conflict, British PM David Cameron announced that “the Libyan people have taken their country back” and that “the future of Libya belongs to its people” (HM Government 2012). Of interest is that one of these people to whom this future seems to belong more than others is discussed by the Financial Times (Kerr 2011). Rafik Al-Nayed is the head of the Libyan Investment Authority (LIA), the holding company that now manages the country’s oil revenues and fund in the international finance market. Al-Nayed was formerly a senior executive of Oilinvest (based in The Netherlands and the holding company for Libyan Tamoil) and one of those who had much to lose under Gaddafi’s plans to change the way oil money was distributed. The newspaper reported on the way that this may be beneficial for continuity so that the new regime does not alienate itself from the country. But it speaks mountains on the links between the intervention of the NATO forces and the kinds of people who were making promises.

5 The news media, war and photography Media researchers have been highly critical of news representations of war and conflict. While there may be public expectation that news media should provide transparent coverage of wars, their causes, the events of which they are comprised, their progress and broader social and political contexts, a number of barriers have been identified to this taking place. Taylor (1992), MacArthur (2004) and Carruthers (2000) point to the way that journalists have minimal access to battlegrounds and are therefore reliant on official military and political sources for information as to what is actually taking place and for definitions of the nature of the war. Studies of war coverage show high degrees of dependency on official briefings with few other sources of information (Herman and Chomsky 1988; McChesney 2004). Realtime war reporting is not much better as very little is conveyed about the real war (Ignatieff 1998). Embedded journalists who operate alongside a military unit will have no actual access to wider events nor even to the meaning of what they see in front of them. Such journalists will also tend to align with the viewpoint of “our boys” with whom they share food, conversation and sleeping space (Tumber and Palmer 2004). In fact, sometime after the Libyan uprising was over, the head of BBC news announced that its own coverage of the conflict had been less than satisfactory due to the way reporters had been embedded with rebel groups and had tended to take their view of events, as reported in the Daily Mail (Revoir, 2012). The problem of access to actual visual footage and images also featured where there was an acknowledged reliance on user generated content for visuals which was presented with no caveats. Additionally, the reporting of contemporary wars is carefully managed by governments. The more open access of the news media to Vietnam, as journalists

286

Gwen Bouvier

played part in exposing brutality and cruelty of war, mobilising the public against it, was a lesson learned (Kimball 1988). Governments subsequently carefully select journalists and manage their access to locations and sources (Carruthers 2000: 122). Even 24 hour news coverage about conflicts will reveal little other than the views of officials, longer sequences of embedded footage where reporters shown kitted out in military gear bravely reveal the view from an armoured vehicle, or generic footage of explosions. There will be little sense of the broader death toll, level of suffering, casualties and experience of the locals, but an emphasis is likely to be placed on the views given by retired generals, former soldiers and politicians (Bouvier 2007). The conflict will therefore be unconnected to the broader historical socio-political context and emphasis will be given to military strategy and manoeuvres. Media scholars have long shown that news reporting often involves placing events into well-worn and easily understood news frames for audiences, what Galtung and Ruge (1965) called “unambiguous” and “meaningful” news. Labels created by politicians may therefore be reproduced uncritically by reporters as they provide simple frames such as good versus evil, or civilised versus barbaric, into which they can sell the story (Parenti 2002). Kellner (2004) looked at reporting of 9/11 and identified a news frame the media used was “the clash of civilisations”. This focuses on the threat of terrorism to social order (2004: 44). Such coverage contained little analysis of what happened and why events became subordinated to the news frame. Machin and Niblock (2006), considering the news reporting of Iraq, show how the entire history of the involvement of the Western powers in the region with the interests of access and control to oil is completely glossed over to present a view of a civilising and fundamentally humanitarian effort by the allies to overthrow an evil regime. Definitions of official sources are taken throughout. Finally, other authors (McChesney 2004) have pointed out that news organisations such as CBN and Fox news, or news agencies such as Reuters and Associated Press, are billion-dollar a year media corporations generally integrated with other forms of corporate interests in stocks and shares, petroleum, pharmaceuticals etc. In such cases it is unlikely that news organisations are going to be critical of actions by the military which supports the interests of corporate activities. In sum, media scholars make a strong case that news coverage of wars take the point of view that serves the interest of their own government. This may not be due to the bias of reporters, but due to the nature of their access to material, to the definitions of official sources, and through a need to provide material that is understandable to news audiences. Media scholars have also turned their attention specifically to the role of photojournalism in the representation of war. These studies are equally critical of the representation of warfare, seeing Vietnam as one exception to what is otherwise a highly controlled form of reporting. These scholars generally place their work in a broader discussion within media and cultural studies regarding the nature of the

British press photographs

287

photograph as evidence. Photojournalism is often associated with key moments where important events are documented for the public good, revealing instances of injustice or horror calling attention for some kind of intervention or action by politicians, or which record key moments in history. An example would be Nik Ut’s famous image of the napalmed naked child during the US war in Vietnam. But theorists have challenged the extent to which photographs can ever be simply recordings of reality or “evidence”. Rather, photographs are always selected moments. They can lie by omission (Knightley 2003). Photographs are also always the result of a range of representational choices by the photographer in terms of framing, angle, focus, composition, etc. (Burgin 1982). Yet as viewers we tend not to see these kinds of choices as we look at an image. Such images simply appear as a window on the world. Writers like Sontag (2004) have famously pointed to the way that such arbitrarily shaped representations can come to stand for complex and ambiguous events. Parry (2010) notes that this means that when politicians make reference to the way that images “say it all”, this is simply not the case. For Burgin (1982) the point of view of the photographer, or at least the photograph, becomes “our” point of view. The decisions, processes and conventions that form the image we see remain invisible. And the photograph remains largely thought of as a document. Media researchers, in the context of the wider work on news representations of war, have shown that Vietnam was indeed an exception where images revealed the effects of war on society, the effects of weapons on bodies, and the toll of suffering and horror on the minds of the soldiers themselves (Machin 2007). Controversial images which escaped military censorship included napalmed victims and dishevelled looking soldiers (Griffin 2010). Such public exposure of the cost of war was made possible by the loss of ability of government departments to effectively manage news portrayals (Hallin 1986). This was to change as governments were to make sure, as did those of the wars earlier in the 20th century, that the public would not be shown images that would create a negative impression. Scholars investigating the press photographs of different conflicts have made a number of observations. Kozol (2004: 2) notes a different focus was placed in representing the 1991 Gulf War, as news organisations were orchestrated into accepting military censorship. There was an absence of dead bodies and an overrepresentation of sophisticated military technology and its efficiency (Taylor 1998). This meant that the effects of this weaponry on actual human bodies were left unexplored, as was the purpose behind the bombing (Griffin and Lee 1995). The impression was that this was a clean, clinical war. Following 9/11, the War on Terror focused visually on drumming up military support, rather than wanting to present images of casualties or hardship in Afghanistan (Zelizer 2005). Kozol (2004: 32) shows that news media tended to use images of Afghani women in burkas as grounds for the liberating military mission, and as indicators of a lack of modernity and the barbarism of Islam. Roth et al.

288

Gwen Bouvier

(2010) looked at the portrayal of “the innocent victims of war” in Afghanistan, through depiction of women and children, arguably focusing attention away from the actual identities of the enemy and overall objectives. Using photographic coverage to establish narrative themes that sit comfortably with official discourse was also seen in the portrayal of the Iraq war (Schwalbe 2006). Once again, the “mustering and deployment of the American military arsenal overshadow[ed] any fuller or more complex range of depiction” (Griffin 2004: 381). Rather than inflaming audiences by showing them the true nature and cost of war, e.g. the display of military coffins draped with the US flag, or suffering and maimed civilians, the visual documentation of Iraq steered away from graphic representations of war. Machin (2007) showed that photographs of the subsequent occupation of Iraq by the Allied forces emphasised the humanitarian role of the troops and inability of the Iraqis to help themselves. Militia were shown as unruly and disorganised, suggesting the need for the civilising role of the calm and organised US soldiers. Such images of the “militia” operating within the people as rogue, disorganised, and dangerous appeared to distract from who exactly these people might be, what their aims and objectives could be, and why exactly they must be removed. This appears to become typical of the way in which conflicts are now represented visually for the British public where the actual military objective is absent, and in its place lies vaguer activities such as humanitarianism. Parry (2012) too has shown that where we do see images of suffering and more graphic images in the Iraq conflict, these do really contribute towards the documentation of events but rather serve the purpose of portraying the human costs of wars. In other words, emphasis is on personal suffering and tragedy rather than causes and events in the conflict itself. Here we may see images of a child’s toy in rubble, for example. A number of studies have looked specifically at the news photographs that represent moments in the Middle East conflict between Palestine and Israel (Jaworski and Van Leeuwen 2002; Parry 2010). In these cases there is a clear sense that the violent nature of Israel may be represented in a way that backgrounds their agency, and that images that do show death and suffering are more as evidence of the “human cost” and tragedy, as opposed to a rigorous call to responsibility. In summary, what the literature shows us is that war reporting in general and war photojournalism in particular will do little more, post Vietnam, than present the public with clichés about generic militia, sanitised warfare, and embedded journalists unable to provide a broader meaning, and contribute to an overall lack of socio-historical context. Where visually they do represent horror and suffering, this will certainly not be as regards the actions of their “own boys” but as evidence of the human toll of other conflicts. It is in this context that the analysis in this chapter is placed.

British press photographs

289

6 Content Analysis Content Analysis is a way of asking a set of systematic questions of a larger set of data. For example if we want to know how often something is reported on one news channel as compared to another over a defined period of time. We could ask, for example, how often we see images of suffering children. Content Analysis consists of counting instances that are found within the research data which can be of any kind: textual, verbal, visual or sound and from a variety of sources. Analysis allows us to establish different kinds of patterns exhibited in the data. In simple terms, in the case of newspaper texts, for example, we can use content analysis to establish the numbers of kinds of persons represented, if they were not represented, what they are depicted as doing, etc. We can also identify less concrete issues such as kinds of evaluations, whether positive or negative. In this case the investigator will define what kinds of factors in the text would indicate such evaluation. One instance where this might be useful would be to investigate the quantity and quality of representation of political parties leading up to an election. Content Analysis could reveal how much a party was represented in different newspapers, if its policies were covered, and if there was a sense of positive or negative evaluation. In this last sense, this might involve documenting the kinds of adjectives used to describe politicians. Content Analysis could also allow a comparison of representations in different modes of communication. So for example, we could compare how politicians are evaluated in texts and in photographs. We may find that while politicians are represented and evaluated equally in texts, we see them very differently visually – perhaps one always appearing in larger groups whereas another always appearing thoughtful in close-up. Such representations could arguably impact upon the public sense of politicians being personable as opposed to remote and official. Content Analysis begins with a clearly defined data source, e.g. a series of newspaper photographs published over a pre-determined period. A number of variables are identified as being suitable to provide the kinds of insights the researcher seeks from the study. The researcher groups these variables in a set of coding categories called a “coding scheme”. These may take a form of a set of questions such as “how many people are in the image?”. The images in the whole sample are all then carefully examined and are all attributed the relevant codes attached to it (based on the coding scheme) (Rose 2007: 61–2). Categories may be pre-set (to reflect the aims and theoretical framing of research) by the researcher or based on an initial reading of texts. In many cases categories emerge from a combination of these two processes (Tonkiss 2004: 369). In other words, as we start to use our coding scheme we discover that new questions need adding and others need to be expanded. A pilot can be conducted on a randomly selected subsample of the total population before the study begins (Neuendorf 2002: 146). The latter can prove useful to change the coding scheme for the final data analysis if problems are revealed.

290

Gwen Bouvier

In this chapter, Content Analysis is used to analyse photographs from four British national newspapers in the period from 19/03/2011 to (and including) 25/03/2011. The four British newspapers are: The Times, Guardian, The Daily Express, The Daily Mirror. This time was chosen since this was a point in the conflict where NATO military operations were starting. Doing a Visual Content Analysis allows us to make generalised statements about the kind of representations we find in a defined population, in the sense that we can ask whose realities are being shown, and how “truth” in the images is the result of particular systems of representation (Berelson 1971: 115). In the case of our photographs of Libya, there are a number of overarching questions. In the first section of this chapter the way in which news coverage has misrepresented the nature of the conflict was considered, laying out some of the actual nature and origins of the conflict. Content Analysis of the photographs will show the role played by these in the construction of the uprising in Libya. It will allow us to think about the way that the photos fit with and contribute to the broader moral and political evaluations of the conflict that has been considered above. In the analysis that follows, photographs have been analysed in terms of a range of coding criteria that allows observations to be made in terms of participants, actions, settings, foregrounding, and evaluation. These will be indicated as the data is analysed.

7 Analysis of the press photographs of the ‘uprising’ in Libya Over the week, 120 photographs were found in the four newspapers. Interestingly, it was the tabloid Mirror that carried the least images – only 14 as compared to 36 in the Guardian and 48 in The Times with 22 in the Express. But this reflects a proportional lower coverage in the tabloids as a whole, as opposed to simply less emphasis on the visual. The analysis of these images that follows considers first what and whom are seen in the images, second what settings are represented, and finally what the participants are depicted as doing. These kinds of questions can allow us to think about what is foregrounded and what is backgrounded. In press images of Vietnam, images of soldiers and airplanes shooting and bombing, villages in flame, civilians and soldiers maimed and dying can be seen. Fairclough (2000) suggests that we can always be certain that ideological work is being done when the microprocess of social practices are missing or abstracted. In other words, it is important to be aware when the specific details – of who is involved in sets of events, what they do, and where – are not represented in full, or are missing or transformed.

British press photographs

291

7.1 Who and what can be seen in the images? To begin with, most of the images found over the week (80 %) depicted people. The other 20 % show items such as military materials, either intact (8 %) or destroyed and possibly on fire (12 %) – although the tabloid newspapers carried proportionally more images that contained destroyed hardware. Of the intact equipment, NATO aircraft (19 images) from archive images in flight or outside of hangars can be seen. None of these are engaged in combat. In the Vietnam war some of the more famous images depict actual shots of US aircraft engaged in bombing raids, taken from the ground and from within other aircrafts. They also showed the results of such raids on civilians. The destroyed equipment show burned out or burning tanks and other vehicles from the NATO bombings. In fact, these images of the destroyed tanks appear as the only real documented images of the effects of the bombing. Captions inform that these are the burned out wrecks of Gaddafi’s forces. Of the participants there were relatively fewer women in the images, as 80 % depicted men. Of the images of people, in 20 % of cases it is hard to discern their age, due to clothing and distance or angle. Images were coded for whether participants across the four newspapers represented different age groups, such as children, youth, or young adults. Across the newspapers, mainly early adult and middle aged looking men were found as opposed to youth or elderly. It was also asked what kind of role the participants play in the images. Most of the participants across the newspapers were politicians or rebels. Their presence occupied about 25 % each of all images. In these images we see 14 that show NATO politicians: 1 of Obama, 2 of Sarkozy, 3 of Cameron, 1 of Clegg, 1 of Fox, and 4 of Foreign Secretary Hague. These are all shown in headshot apart from 3 body shots of Hague leaving Downing Street (and all these images are for the most part in the Guardian and in The Times). There are 17 images of Gaddafi, although 8 of these are in the Express. The Guardian showed one image of rebel leader Abdul Fattah Younis surrounded by rebel soldiers, 1 showed Gheriani, spokesman for the revolutionary council, and 1 image showed a large crowd of officials including UN Secretary General Ban Ki-Moon. Overall there is a great emphasis visually in terms of politicians on Gaddafi himself. The Guardian shows a wider variety of persons, and the Times has a much more British focus. The Express and the Mirror are almost exclusively showing Gaddafi. Other participants are NATO forces at 10 %. There were images of military personnel in standby, watching television in a canteen, or pilots posed next to aircrafts or posing in cockpits. There were also smaller numbers of images of protesters and onlookers. Rebels, protesters and onlookers mostly appear dishevelled and wear tatty clothes and improvised assemblies of military and civilian clothing, such as jeans and sports clothing. In only one image did a rebel soldier wear what appeared to be more formal military clothing and equipment.

292

Gwen Bouvier

Of the participants, there were no images of middle class persons, smarter professional types, or anyone appearing to represent an official role found (with the exception of the rebel leader mentioned above). This is mindful of Machin’s (2007) observations on press photographs of the Iraq occupation where civilians and settings suggested lack of order, lack of organised society or those associated with official community or society. His conclusion was that this was part of the evidence suggesting the need for the Allied powers to step in and provide order. Across the sample, ten images of children were found. In her own research Parry (2010: 79) pointed to the use of images of women and children in press photographs to signify the human toll of war, even if these had little value in actually documenting the concrete nature of what had taken place in terms of actual combat and overall political meaning. In the current sample, children were generally represented in hospital beds as peering, frightened between intimidating armed men as part of the human shield. These children certainly fit with Parry’s observations and their presence is clearly one of resonance with Western values of childhood innocence and the need for protection. As regards women too, we find them praying and fearful and certainly representing the threat to innocence and family. Participants are represented for the most part as individuals (40 %) or in a small group of three or four (30 %) of all images. So a typical image here would show a single rebel standing next to a burned tank or two rebels holding their weapons aloft. Larger numbers of people are found much less frequently in the image, at 14 %. These images tended to show groups of rebels of five or more, or protesters. Again this follows the pattern of representation in Iraq as regards giving a sense of only smaller scale and fragmented incidents. It can be contrasted to photographs from WW2, which often showed large armies massing and manoeuvring. Such images were to convey a sense of power and national might against an equally powerful enemy. These images of smaller groups in Libya suggest something less organised and fragmented, and certainly more personalised. Further evidence for this personalisation can be found when looking at the positioning of participants in the image. 62 % of the participants are positioned in the foreground of the image; 30 % are in the middle; and only 6 % appear in the further distance. So the pattern in photography here is an emphasis on the individual and small groups, photographed in closer-up. Theorists on the semiotics of proximity in photography point to the way that closeness has the effect of individualising the participants, whereas distance can anonymise them (Kress & Van Leeuwen, 2006). But other authors (Van Leeuwen, 2009) would also point to the otherwise generic nature of the representation of the participants who are for the most part scruffy adult men with improvised military uniforms mixed with civilian clothing. Such people are not individualised as specific people with specific lives, such as teachers, dentists, writers, but as generic rebels. One notable difference between the newspaper titles was that the Guardian and Express had proportionally the highest amount of foregrounded images at

British press photographs

293

70 % each, although all are high in this regard. But this supports interviews recently carried out with photographers supplying the Guardian with photographs of conflicts who suggested that there is a preference for images that are able to personalise complex issues (Machin & Polsa, 2014).

What settings appear in the photographs? Arguably, this was difficult to code. This is particularly so since most of the images are closer shots of individuals or small groups of people, in which there tends to be very little background to be seen. Closer shots may contain no background at all. A close shot of 6 men waving weapons looking slightly up at them may show only sky as a context. Other images are cropped and only show rubble and burned out tanks. Broadly, nearly 60 % were coded as being in Libya although this was often done simply as captions said this to be so. Of note is that images that represented unofficial looking groups such as rebels wearing ragged clothing and protesters and onlookers were all in Libya, whereas over half of the official type images which included official looking soldiers or politicians were in the UK. Images located outside of Libya might depict an aircraft in a hangar or in flight. Figure 1 shows a generic shot of an RAF Tornado used in the conflict, which shows little context. Captions might point broadly to NATO aircraft. Such images are mindful of Griffin’s (2004) account of images of the first war in Iraq, where

Fig. 1: Generic image of RAF Tornado like those used in newspapers Photo © DM Parody (www.dotcom.gi/photos).

294

Gwen Bouvier

there was a great presence of archive images of weapons and training used in news. Such images often signalled up high technological capabilities of weapons. In Libya too, images of aircraft come from archives and point to clean technology and measure in contrast to the chaos seen on the ground. And what is telling is that all of the 19 images of aircraft in the sample show only one airplane. One could imagine the different effect if they were to show a scene of hundreds waiting to go out to bomb or even a sky full of planes, as is found in WW2 photography – for its own propaganda purposes. Such singular images of a plane in a hangar with one pilot remove this sense of force. In addition, such archive images gloss over who was actually doing the bombing. In trying to code for whether images were taken in cities or in the countryside, it was found that 13 % was in urban settings and 15 % was rural. The rest was unclear, which are mainly the close ups. Where urban settings can be seen, these do not tend to tally with photographs the author had seen of larger urban areas in Libya with higher buildings and complex road systems but less distinct and smaller scenes. Overall, there is a sense of lack of context.

7.2 What are people depicted as doing in the images? Again this was not so easy to code. In images of other wars such as WW2 and Vietnam it is clear that soldiers are seen marching, building bridges, as wounded and waiting evacuation, as giving sweets to children, as bombing a village. In the sample here, 40 % of the participants could be said to be carrying out a military related activity. But most of these are clearly posing, raising weapons and looking at the camera as in Figure 2 below. But in others, participants simply stand around, aimlessly but nevertheless all still holding weapons. In the case of NATO military this may be comprised of sitting in, or standing against a plane. In only one case do we see someone pointing a gun at another person, which is depicted in the Guardian and described as a rebel protecting a suspected Gaddafi supporter against another armed rebel. Another way to think about the representation of actions is in terms of Halliday’s (1978) linguistic classification of verb processes or transitivity. This provides a way for drawing out a little more what this all means. Some verbs such as “build” suggest a clear material outcome and agency. Other verbs suggest behaviours without material outcomes such as “walking” or “waiting”. Others represent verbal processes, such as “talking”, “replying”. Others yet represent mental processes such as “worrying”. The latter of these have been described as important in linguistics as they are ‘focalisers’ which give access to the mental worlds of participants. Mental processes are often used to create a sense of empathy or association with persons. So in an account of war, one may be given access to the internal states of “our boys” and their families at home, but not to those of the enemy. Many of the participants in the photos are not carrying out material processes,

British press photographs

295

Fig. 2: Image from The Times 21/03/2011 showing two rebels with guns posing Jack Hill / The Times / N.I. Syndication.

with the exception of the image mentioned above in which a rebel points a gun and someone shields someone else. In all the other images, behavioural verbs such as waiting and walking and raising weapons can be seen, and additionally some verbal processes as some men appear to shout, expressing a kind of joyful aggression and excitement – although as stated for the most part these appear highly posed. An attempt was made to code for any obvious types of emotion being shown by the participants. But in 70 % of cases participants are best described as ‘blank’ and emotionless. Even those raising weapons do not appear either particularly joyful or aggressive. Although 14 % of participants were coded as appearing aggressive, who would be those men raising weapons. We do see the grief of a women and the fear of children in the typical ‘human toll of war’ images. In this sense, no particular access is given to the internal worlds of these participants. It is the behavioural verbs that dominate. As mentioned above, Machin (2007) showed that the participants in the Iraq occupation are represented as aimless and passive, justifying the need for the Allies to bring order and shape. In Libya, expressions of joy and mild aggression on the faces of some of the rebels can be seen, although many seem expressionless. And they represent broadly a ragtag force, chaotically wandering the landscape, rowdily raising their weapons. Women and children show fear. This too suggests a need for the civilising NATO involvement.

296

Gwen Bouvier

7.3 Summary of image contents In summary, the photographs from the sample for the most part show images of rebels toting weapons, either slightly aggressive or aimlessly. No actual combat is shown. No one is shooting, or getting shot. No one is directly under threat. We see NATO aircraft and personnel, although not in action and always in standby roles. Many images of burning Gaddafi weapons can also be seen. Leaders are also represented, although in the tabloids we see only Gaddafi. In The Times it is for the most part British leaders, and only the Guardian shows other Libyan political figures. Earlier in the chapter, Fairclough’s (2000) point about the need to be aware of the deletion and abstraction of the micro details of social practice were considered. In this press photography here, there is a sense of simplification. Rebels are represented visually as a generic group, even if they are shown in smaller fragmented groups. In fact, during the conflict these people played a very small role in actual battle terms, as military groups would tend to surrender and defect as the situation developed. The opposition to these rebels and the “uprising” is represented in the form of Gaddafi himself. What is also absent from the images are the actual bombing raids. Only burned out tanks are shown. But we are given no sense of the death and injury toll. Missing is also any sense of the more institutional sections of the Libyan population. This is a conflict photographed for the most part by journalists attached to groups of rebels – which the BBC later admitted lead to rather poor levels of actual reporting and understanding of what was going on (Revoir 2012). While all coverage is particularly simplified, tabloid coverage is particularly so, being comprised of Libya with its burning tanks, rebels and Gaddafi.

8 Conclusion The scholarly literature on war reporting shows that what we find as news is often highly controlled by governments, with an emphasis on official definitions, accounts of the details of tactics and manoeuvres by embedded journalists, and a complete lack of actual context. In this case, press photography of the Libya uprising and NATO intervention is clearly no exception. Different from other cases such as Iraq (Parry 2010 and Machin 2007) is the lack of call for humanitarianism and sense of the human toll of war. Rather, intervention is required due to the evilness of Gaddafi and the fragmented and disorganised nature of the rebel forces. The results of NATO intervention and bombing are represented only as successfully destroyed Gaddafi equipment, and not in terms of deaths and injury. As such, the actions of the wandering rebels with their hand weapons are foregrounded over the fury of the bombing raids of squadrons of NATO planes. In Fairclough’s (2000) sense, how could images have documented this conflict more appropriately and

British press photographs

297

in line with the micro processes? Of course, visually it is difficult to capture complex issues that span over many decades. But one start could have been to show the identities and roles of those manoeuvring not to lose power over the oil money. What could have been shown are images of Western politicians making deals with oil companies over the years, connecting the actions of NATO in Libya to those in other countries like Iraq and Afghanistan. We could have then seen, as we did in Vietnam, the view from the NATO planes as the bombing took place and its effects on the people below. WW2 marked a change in the nature of war, where more casualties became civilians as opposed to soldiers themselves. It is a terrifying prospect, however inhumane Gaddafi may himself have been, that a British public is denied access to the realities of a conflict carried out in its name, both in terms of historical context and in terms of the actual goings on in that conflict. But as Sontag (2004) has argued, one problem with the image is that it can powerfully communicate that something has indeed been shown. As Parry (2010: 70) points out, such images give the sense that they “say it all”.

References Al Jazeera. 2009. Libya delays Gaddafi oil plan. News Africa 03/03/09. Available at: http://www.aljazeera.com/news/africa/2009/03/200933183343913989.html. BBC. 2006. US to renew full ties with Libya. BBC News World 15/05/2006. Available at: http://news.bbc.co.uk/1/hi/world/africa/4773617.stm. Berelson, Bernard. 1971. Content Analysis in Communication Research. New York: Free Press. Blanchard, Christopher and Jim Zanotti. 2011. Libya: Background and U.S. Relations. US Congressional Research Service. Available at: http://fpc.state.gov/documents/organization/157348.pdf. Bouvier, Gwen. 2007. Breaking News: The First Hours of the BBC Coverage of 9/11 as a Media Event’. In Thomas Pludowski (ed.), How the World’s News Media Reacted to 9/11, 51–83. Spokane: Marquette Books. Burgin, Victor. 1982. Thinking Photography. London: Macmillan. Carruthers, Susan. 2000. The Media at War: Communication and Conflict in the Twentieth Century. Basingstoke: Macmillan/Palgrave. CFR. 2011. The New Arab Revolt. Manhattan, NY: Council on Foreign Relations. Dzdiadosz, Alexander. 2011. Neglect, tribalism, history fuel east Libya revolt. Reuters Feb 23. Available at: http://www.reuters.com/article/2011/02/23/us-libya-east-rebellionidUSTRE71M6RM20110223. Fairclough, Norman. 2000. Discourse, Social Theory, and Social Research: The Discourse of Welfare Reform. Journal of Sociolinguistics 4(2): 163–195. Galtung, Mari and Johan Ruge. 1965. The Structure of Foreign News. Journal of Peace Research 2: 64–91. Griffin, Michael. 2004. Picturing America’s “War on Terrorism” in Afghanistan and Iraq. Journalism 5(4): 381–402. Griffin, Michael and Jongsoo Lee. 1995. Picturing the Gulf War: Constructing an Image of War in Time, Newsweek, and U.S. News & World Report. Journalism & Mass Communication Quarterly 72(4): 813–825.

298

Gwen Bouvier

Haggar, Nicholas. 2009. The Libyan Revolution: Its Origins And Legacy. Ropley: O Books. Halliday, Michael. 1978. Language as Social Semiotic: The Social Interpretation of Language and Meaning. Maryland: University Park Press. Hallin, Daniel. 1986. The ‘Uncensored War’: The Media and Vietnam. Oxford: Oxford University Press. Hamilton, John. 2011. ‘Libya Protests: The Tangled Web Keeping Gaddafi in Power’. The Telegraph 23/02/11. Available at: http://www.telegraph.co.uk/news/worldnews/africaandindianocean/ libya/8343859/Libya-protests-The-tangled-web-keeping-Gaddafi-in-power.html#. Herman, Edward and Noam Chomsky. 1988. Manufacturing Consent: the Political Economy of the Mass Media. New York: Pantheon Books. HM Government. 2012. Statement on Libya 5 Sep 2011. Number 10. Available at: http://www.number10.gov.uk/news/statement-on-libya/ Accessed on 29/11/2012. Hopkins, Nick. 2011. UK operations in Libya: the full costs broken down. Guardian 26/09/11. Available at: http://www.guardian.co.uk/news/datablog/2011/sep/26/ uk-operations-libya-costs. Ignatieff, Michael. 2000. Virtual War: Kosovo and Beyond. London: Chatto and Windus. Jaworski, Adam & Van Leeuwen, Theo. 2002. The Discourses of War Photography. Journal of Language and Politics 1(2): 255–275. Joshi, Shashank. 2011. Six lessons from Libya. The Arab Spring Implications for British Policy. The Middle East Council. Available at: http://cmec.org.uk/wp-content/uploads/CMEC-ArabSpring.pdf. Kellner, Douglas. 2004. 9/11, Spectacles of Terror, and Media Manipulation. Critical Discourse Studies 1(1): 41–64. Kerr, Simeon. 2011. Libya fund seeks to free assets for loans. Financial Times 31/08/11. Available at: http://www.ft.com/cms/s/0/055dec10-d3f4–11e0-b7eb-00144feab49a.l. Kimball, Jeffrey. 1988. The Stab-in-the-Back Legend and the Vietnam War. Armed Forces and Society 14(3): 433–458. Knightley, Phillip. 2004. The First Casualty: The War Correspondent as Hero and Mythmaker from Crimea to Iraq. London: The John Hopkins University Press. Kovalyova, Svetlana. 2011. UPDATE 2-ENI leads Libya oil race as Gaddafi nears end. Reuters 08/08/11. Available at: http://www.reuters.com/article/2011/08/22/libya-oilidUSL5E7JM10T20110822. Kozol, Wendy. 2004. Domesticating NATO’s war in Kosovo/a: (in)visible bodies and the dilemma of photojournalism. Meridians: Feminism, Race, Transnationalism 4(2): 1–38. Krauss, Clifford. 2011. The Scramble for Access to Libya’s Oil Wealth Begins. NY Times 22/08/11. Available at: http://www.nytimes.com/2011/08/23/business/global/the-scramble-for-accessto-libyas-oil-wealth-begins.html?pagewanted=all&_r=0. Kress, Gunther and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. London: Routledge. MacArthur, John. 2004. Second Front: Censorship and Propaganda in the 1991 Gulf War. London: University of California Press. Machin, David. 2007. Visual Discourses of the Iraq War. In Adam Hodges and Chad Nilep (eds.), Discourse, War and Terrorism, 123–142. Amsterdam: Benjamins. Machin, David and Sarah Niblock. 2006. News Production. London: Routledge. Machin, David and Lydia Polsa. 2014. Visual Journalism. London: Palgrave. McChesney, Robert. 2004. The Problem of the Media: U.S. Communication Politics in the 21st Century. New York: Monthly Review Press. Malcolm, Peter and Elie Losleben. 2004. Libya. Tarrytown: Marshall Cavendish. Metz, Helen Chapin. 2004. Libya. Whitefish: Kessinger Publishing. Moore, Charles. 2011. Britain Shouldn’t Feel Guilty about the Part We Played in Ousting Colonel Muammar Gaddafi. The Telegraph 26/08/11. Available at: http://www.telegraph.co.uk/news/

British press photographs

299

worldnews/africaandindianocean/libya/8724942/Britain-shouldnt-feel-guilty-about-the-partwe-played-in-ousting-Colonel-Muammar-Gaddafi.html. Neuendorf, Kimberley. 2002. The Content Analysis Guidebook. London: Sage. Obeidi, Amal. 2001. Political Culture in Libya. Richmond: Curzon. Parenti, Michael. 2002. Monopoly Media Manipulation. Mediterranean Quarterly 13(2): 55–66. Parry, Katy. 2010. A Visual Framing Analysis of British Press Photography during the 2006 IsraelLebanon Conflict. Media, War and Conflict 3(1): 67–85. Parry, Katy. 2012. The First ‘Clean’ War? Visually Framing Civilian Casualties in the British Press during the 2003 Iraq Invasion. Journal of War & Culture Studies 5(2): 173–187. Pfeiffer, Tom. 2008. Libya daunting for U.S. firms despite better ties. Reuters 03/09/08. Available at: http://uk.reuters.com/article/2008/09/03/us-usa-libya-investmentidUKLT13607620080903. Revoir, Paul. 2012. Our coverage of the Arab Spring was over-excited, admits BBC. Daily Mail 25/06/12. Available at: http://www.dailymail.co.uk/news/article-2164536/BBCs-coverageArab-Spring-sporadic-ignoring-uprisings-failed-favour-big-stories-Libya-Egypt.html. Rose, Gillian. 2007. Visual Methodologies: An Introduction to the Interpretation Of Visual Materials. London: Sage. Roth, Andrew, Zoe Huffman, Jeffrey Huling, Kevin Stolle and Jocelyn Thomas. 2010. Covering War’s Victims: A Content Analysis of Iraq & Afghanistan War Photographs. Project Censored: Media Democracy in Action. Available at: http://www.projectcensored.org/top-stories/ articles/covering-wars-victims-a-content-analysis/. Schwalbe, Carol. 2006. Remembering Our Shared Past: Visually Framing the Iraq War on US News Websites. Journal of Computer-Mediated Communication 12(1). Available at: http://jcmc.indiana.edu/vol12/issue1/schwalbe.html. Sontag, Susan. 2004. Regarding the Pain of Others. London: Penguin. Taylor, Philip. 1992. War and the Media: Propaganda and Persuasion in the Gulf War. Manchester: Manchester University Press. Taylor, John. 1998. Body Horror: Photojournalism, Catastrophe and War. Manchester: Manchester University Press. The Economist. 2011. One reason Qaddafi might fold. Democracy in America 01/04/11. Available at: http://www.economist.com/blogs/democracyinamerica/2011/04/libya. Tonkiss, Fran. 2004. Analysing Text and Speech: Content and Discourse Analysis. In Clive Seale (ed.), Researching Society and Culture, 367–382. London: Sage. Tumber, Howard and Jerry Palmer. 2004. Media at War: The Iraq Crisis. London: Sage. Van Leeuwen, Theo. 2009. Discourse and Practice. Oxford: Oxford University Press. Vira, Varun, Anthony Cordesman and Arleigh Burke. 2011. The Libyan Uprising: An uncertain trajectory. Centre for Strategic and International Studies. Available at: http://csis.org/files/ publication/110620_libya.pdf. Von Mittelstaedt, Juliane and Volkhard Windfuhr. 2011. Chaos and Uncertainty in Libya’s Revolutionary Leadership. Der Spiegel 30/3. Available at: http://www.spiegel.de/ international/world/the-rebels-from-benghazi-chaos-and-uncertainty-in-libya-s-revolutionaryleadership-a-754035.html. Zelizer, Barbie. 2005. Journalism through the Camera’s Eye. In Stuart Allan (ed.), Journalism: Critical Issues, 167–176. Maidenhead: McGraw-Hill.

John A. Bateman

13 Looking for what counts in film analysis: A programme of empirical research Abstract: The moving audio-visual image as found in narrative films, documentaries, video and TV defines a class of artefacts including some of the most complex designed semiotic entities yet devised. A growing body of work is investigating how the organisation and workings of such artefacts, and particularly of narrative film, can be explored empirically. Although it is possible to characterise audiovisual artefacts in terms of their technical, measurable physical qualities, the bare availability of such data does not of itself resolve the fundamental issue of the relationship between, on the one hand, the analysis of measurables and, on the other, the aesthetic and cultural import of that which is measured. After overviewing the range of approaches currently being taken to the empirical analysis of film, this contribution argues that drawing on more refined notions of discourse as the level of analytic description most closely related to textuality offers a useful way forward.

1 Introduction The moving audio-visual image as found in narrative films, documentaries, video and TV defines a class of artefacts including some of the most complex designed semiotic entities yet devised. Such artefacts set enormous challenges for systematic investigation. A growing body of work is consequently investigating how film, particularly narrative film, can be explored empirically by employing automatic and semi-automatic analysis techniques. One principal motivation for this is that it offers a complementary approach to the hermeneutic and aesthetic research typical of work in film studies more broadly, thus relating to a long tradition of attempts to give aesthetics a sounder empirical foundation (cf. Flückiger 2011). In recent years this empirical orientation to film and audiovisual analysis has received a tremendous boost from the development of readily accessible technological tools for exploring audiovisual data. It is possible now as never before to characterise audiovisual artefacts in terms of their technical, measurable physical qualities. But the bare availability of such techniques does not of itself change the fundamental ontological issue of the relationship between, on the one hand, the analysis of measurables and, on the other, the aesthetic and cultural import of that which is measured. The measurable physical qualities and technical features exhibited by a film as that film unfolds do not allow direct derivation of those aesthetic, sociocultural, dramaturgical and philosophical interpretations that are generally of interest for film studies and other visual research. We generally want

302

John A. Bateman

to know what a film “means”, to us and for us, rather than what precise technical features it exhibits. At the same time, however, accounts pitched at more abstract levels of description have shown themselves to be extremely difficult to validate. Problematic for such approaches is precisely their distance from the empirically investigable technical details of the films they seek to characterise. Finding mechanisms that can bridge this distance constitutes the central challenge of employing empirical methodologies for film analysis. This chapter addresses this issue from the perspective of functional sociosemiotic theory, a general approach to human communication growing out of earlier linguistic work by Michael Halliday, Ruqaiya Hasan, Jim Martin, Christian Matthiessen and colleagues (cf. Halliday 1978; Halliday and Matthiessen 1999). A growing body of evidence suggests that many tenets of this approach are applicable to research on a broad range of semiotic artefacts stretching well beyond those more narrowly concerned with “verbal” language (cf. Kress and Van Leeuwen 1996, 2001). In particular, I will draw on that theory’s account of discourse organisation (Martin 1992; Van Leeuwen 2005), arguing both that film requires analysis in these terms by virtue of the kind of semiotic artefact that it is and that this offers precisely the kind of bridge between measurable quantities and abstract semiotic interpretation that is useful for driving empirical research. By setting out “systems of functional discourse discriminations” at work in film, I will propose that suitable motivations can be provided for selecting and grouping quantitatively measurable filmic phenomena. This may then offer a theoretically more robust notion of just what measurable phenomena “count” within a film or films, which in turn may support more precise empirical hypotheses for increasingly finegrained investigations of the workings of film. The chapter is organised as follows. First, I very briefly address some of the critiques that have been brought against empirical approaches to film from the perspective of film aesthetics; the charge of inappropriate reductionism for objects of cultural interpretation always needs to be considered before proceeding in this area as it often marks a fundamental theoretical, and often ideological, divide. Second, I offer a brief overview of some of the most prominent approaches to film that are currently adopting empirical, quantitative methods. Third, I draw on this overview to characterise a reoccurring problem that is beginning to be recognised within these empirically-based approaches. And fourth, I suggest that drawing on more refined notions of discourse as the level of analytic description most closely related to textuality may offer us a useful way forward. The chapter concludes with a brief summary and some suggestions for a research programme building on the discussion.

2 Why count what you cannot quantify? There is considerable divergence in opinions concerning both the possibility and utility of examining films from the perspective of their quantitatively measurable

Looking for what counts in film analysis

303

properties and physical features. A common position in the humanities is that the technical features sensible for an analysis only become visible with respect to an understanding of a film “as a whole” – including its themes, historical background and so on – and thus there is little chance of working “bottom-up” from the data to interpretation (cf. Wilson 1995 and our critique of his position in Bateman and Schmidt 2012: 58–72). Such arguments generally draw on relatively simple models of how “levels” in a description are related; there is indeed little chance of deriving abstract interpretations of data by simple compositions of measurable qualities, but this far from exhausts the range of mechanisms available for characterising complex semiotic artefacts. Since it is equally clear that the detailed organisation of semiotic artefacts has considerable bearing on those artefacts’ interpretation – particularly so, as we shall see below, in the case of film – more sophisticated notions of precisely how such interpretation proceeds on the basis of audiovisual evidence are required: interpretation cannot be considered as a purely “top-down” process either. Audiovisual evidence is susceptible to description at various levels of abstraction. Within film studies, schemes for the description of the technical and formal “devices” (cf. e.g. Bordwell 1989) deployed in film are well established (cf. standard reference works such as Reisz and Millar 1968; Bordwell and Thompson 2010). But these schemes are themselves still often a considerable distance from measurable features of the audiovisual artefact; this is revealed with particular force whenever attempts are made to objectively characterise categories for automatic processing (cf. Fuxjäger 2009). Even apparently straightforward notions such as “shot” face tensions concerning whether a “perceptual” unit or a “production” unit is intended: production-based definitions such as the segment of film between starting and stopping the camera will often not match one-to-one with shots as perceived. Similarly, it is often noted in introductory texts on film how camera distance might need to be considered with respect to the norms established within specific films: for example, if an entire film were to consist of shots within which figures are shown at a distance, then a shot that in another film might only count as a medium shot might to some extent work as a close-up. This becomes increasingly complex when human audiovisual perceptual systems are factored in: for example, the phenomenon of colour constancy shows “top-down” influences on colour perception that make straightforward measurement problematic (e.g. regions in the visual field may be perceived to be of a uniform colour because we “know” that a single object is involved). There are some good reasons for working with categories that are not straightforwardly related to physical measurement. For example, their flexibility is already supportive of more abstract interpretative analyses because they allow the analyst to bring to bear more knowledge that they might have concerning the film as a whole. But this flexibility also has a downside in that descriptions at this level become less well suited for performing broader, “empirically well-founded” inves-

304

John A. Bateman

tigations. By “empirical” here I mean constructing hypotheses concerning how films make their meanings which can be checked against larger samples of data – in this case, films – in order to see if those hypotheses are supported or refuted. Analyses of this latter kind demand a tighter relationship between the filmic material under consideration and the analytic categories among which patterns are being sought. Issues of standardisation, reliability, intersubjectively viable classification and other aspects well known from corpus-based approaches to language and quantitative studies become central. Such quantitatively motivated, corpus-based approaches to language have transformed linguistic research over the past two decades and we now know considerably more about the form and functioning of language in context. It is then likely that analyses of film that appeal to broader, reliably applicable analytic categories could similarly contribute to our understanding of the mechanisms of film and audiovisual media in general. But the complexity of audiovisual media presents extreme challenges for developing the correspondingly reliable schemes of technical description necessary for this methodology to be applied. Finding and relating appropriate levels of abstraction for description is therefore a core problem for systematic and empirical film analysis and interpretation. At present, a continuing lack of accounts in this area leaves film theory with a seriously weakened basis for theory building.

3 Empirical approaches to the analysis of film The use of objective measures for the purpose of empirical film analysis was most prominently pioneered in the work of Barry Salt, who over many years has provided detailed analyses of films in terms of objectively ascertainable measures such as average shot length (ASL), camera angles, camera distance (e.g. close-ups, long-shots, etc.) and camera movements (Salt 1983, 1974, 2007). Salt’s original premise was that such measures would correlate with particular directorial styles, historical epochs and distinct types of narrative events – examining films in this way was therefore expected to clarify a variety of properties of the medium and regular features of style that would not be accessible by simple observation. In many respects subsequent analyses have borne this out – for example, Salt’s detailed descriptions suggested that systematic changes in shot lengths can indeed be determined over the 100 years or so of film’s history: average shot length measures appear to have gone through several “epochs” each marking a significant change in cutting rates compared to that preceding. In other areas, however, more fine-grained characterisations appear necessary. Bordwell et al. (1985: 60–63) argued, for example, that it is the “range” of likely shot lengths that would show systematicity rather than the bare averages. Bordwell et al. also show how variations in shot lengths can be beneficially combined with other aspects of developing

Looking for what counts in film analysis

305

film technology, such as the introduction of sound, which for a considerable time may have led to longer average shots due to more cumbersome technical apparatus. The purpose of such analyses in general is similar to early quantitative approaches in stylistics which tried to explore properties of texts on the basis of the distributions of various countable properties of those texts. The classic case of such “statistical stylometrics” is authorship attribution, where texts of disputed authorship are examined to see if the usage of a broad range of linguistic features in the texts makes them more or less like the known texts of other authors. Although this is rarely the case for film, the essential idea that a work will exhibit distributions of measurable features that are distinctive for its “author” remains. Thus Salt and others have explored, for example, whether distinctive distributions of measurable features can be correlated with particular directors, such as Max Ophuls and Alfred Hitchcock. The same method is naturally applicable to other sources of regularity: studies by Salt and others have also considered whether films from different national cinemas exhibit differences. And, as mentioned above, all of these questions can be raised with a historical dimension: asking whether particular periods of film-making in particular national contexts differ in their statistical distributions to those of other periods. All such investigations are therefore intended as contributions to the poetics or aesthetics of film as a medium, attempting to find reoccurring properties by which distinctive empirically-grounded categories can be isolated and to offer ways of tracking possible changes in those categories over time – for example as film has become more of an “international” medium. A recent overview of the basic premises of this “statistical” approach to film aesthetics can be found in Baxter (2012). Quantitative questions can also be raised concerning film particularly as a narrative medium; as, for example, the film semiotician Christian Metz argued: […] it was precisely to the extent that the cinema confronted the problems of narration that, in the course of successive groupings, it came to produce a body of specific signifying procedures (Metz 1974: 95).

Thus, certain filmic structures that have developed can be seen as answering the challenge of narrating ever more complex stories. The development of this for film has been explored by Kristin Thompson (Bordwell et al. 1985: 174–193), who considers how the audiovisual moving image developed from its early beginnings of short films lasting only a few minutes, perhaps centred around a visual joke, to the extensive artefacts of today. Each extension required corresponding techniques so that the interest of viewers would be maintained. Such techniques came to include increasing reliance on cause-result chains, so that the events portrayed would “make sense” and be “motivated” within the story, as well as parallel alternating structures, either for drawing comparisons and contrasts or, in the hands of early film-makers such as D. W. Griffith, adding dramatic tension in the service

306

John A. Bateman

of chase or last-minute rescue scenes. With this increase in dramatic complexity, film could be sensibly related to other traditions, such as classical drama and its schemes of dramatic development in terms of conventionalised sequences of “acts”. Each act takes on a particular cluster of narrative functions in the unfolding of the drama. A well-known description of this kind influential for screenplay writers has been Vogler (1998). Vogler proposes a three-act organisation building on earlier work on the narrative structure of myths and fables mapped out across the classical beginning-middle-end scheme promulgated since the time of Aristotle. Other kinds of act structures have been suggested in literary theory and, specifically for Hollywood film, Thompson (1999) argues for a “four”-act structure made up of “setup”, “complicating action”, “development” and “climax and epilogue”; all four acts are suggested to be of roughly equal length. As with literary analyses of act structures, such forms can be seen as ways of exploring and comparing how particular works and genres of works achieve their narrative goals. This also relates closely to other work on narrative and, in particular, linguistic studies of narrative form such as that of Labov and Waletzky (1978), where very similar structures and their functional interpretation in terms of establishing crisis points and waves of evaluation and empathy are proposed on the basis of fine-grained linguistic analyses. For film, therefore, the question again raises itself of just “what evidence” might be drawn on for uncovering such patterns. Thompson proposes her four-act structure on the basis of detailed examination of a broad corpus of films but this nevertheless rests on broadly qualitative interpretation rather than an empirical foundation. Finding quantitative correlates of act structure would therefore help either decide between competing hypotheses or provide more discriminating descriptions, for example between genres. Salt’s selection of features was originally motivated by a combination of, on the one hand, standard practice in film studies and the established descriptive categories employed there and, on the other, practical considerations. Since manual analysis formed the basis of Salt’s earlier results, the features considered needed to be measurable “by hand”. However, even for the more straightforwardly measurable quantities such as shot length, performing such analyses manually is less than ideal. In order to cover a reasonable range of data, Salt employed various “sampling” strategies that reduced the quantity of data to be coded, proposing for example that counts for the first 40 minutes of a film rather than the entire film would be adequately representative. Bordwell and Thompson (1985) argue that this can lead to considerable inaccuracies due to film-internal dramaturgically motivated changes in shot lengths occurring later in a film. Nowadays it is possible to improve on this situation in several ways. First, it is possible to obtain shot lengths from film-data automatically with considerable (but not complete) accuracy; we will return to this growing application of automatic methods in a moment. Second, even without fully automatic methods, sub-

Looking for what counts in film analysis

307

stantial technical support now exists for hand-coding so that the basic data, at least for shot lengths, can be gathered in little more time than it takes to watch a film. This forms the basis for the “cinemetrics” approach to film analysis originally developed by Yuri Tsivian (www.cinemetrics.ly and Tsivian 2009). This takes Salt’s basic premises considerably further by moving towards genuinely large collections of film data. It does this by employing what is nowadays termed in many fields “crowdsourcing” techniques: that is, coding film data for shot lengths is seen as a community effort – the software and website provided by cinemetrics enables any user to “mark up” an entire film for shot lengths and upload the results to a growing database of results openly accessible to interested researchers. This transition from “private” data and proprietary formats to open data and tools is very much in accordance with current work on multimodal corpora and data analysis in other fields (cf. Bateman 2013). The current cinemetrics database includes data for over 10,000 films, making it possible for the first time to employ stronger statistical techniques in the search for patterns rather than relying on simple averages for a restricted range of films. In addition, a variety of “visualisations” are also provided for forming overviews of selections of this data and for the individual films covered. For example, one citation style for films promoted within cinemetrics includes as a matter of course a graphical representation of the “trendline” for average shot lengths across the duration of a film; entries for Michael Curtiz’s Casablanca (1942) or Stuart Baird’s (ASL 7.5)” and “Star Star Trek: Nemesis then appear as follows: “Casablanca Trek: Nemesis (ASL 3)” respectively. The trendlines are intended to give an indication of how the duration of shots varies over the course of a film: thus it appears that Casablanca begins with longer shots, has some variation as the film develops and then ends with longer shots, while Star Trek: Nemesis begins with short shots and maintains gradually increasing longer shots before ending again with very short shots. These might then be indicative of broader differences in genre or other plot points within the narrative structure – for example, action films may be hypothesised to end with shorter shots, constituting the “climax and epilogue” act, whereas a romantic film may call for longer shots. For more detail, the shot lengths for each film are maintained in full within the cinemetrics database; Figure 1 accordingly shows the cinemetrics visualisation of the full shotlength data for M. Night Shyamalan’s The Sixth Sense (1999). The availability of data sets on this scale provides much of what is necessary for addressing Bordwell et al.’s (1985: 60–63) claim that shot ranges and variation in shot lengths need to be considered rather than simpler metrics. Attention to variation, ranges of variation and the use of more sophisticated statistical analysis are all now well established in quantitative analyses of filmic technical features. Salt (2007), for example, gives several examples of how more careful statistical analysis can reveal reoccurring patterns in the data that are not evident in simple presentations of averages and means and warns against too simple interpretations

308

John A. Bateman

Fig. 1: Cinemetrics ASL visualisation (black-white reversed) for the film The Sixth Sense (1999, dir. M. Night Shyamalan); coded by Barry Salt and displayed with a (12th degree) trendline (ASL 8.6).

of the raw data. The correlation of trendlines of averages with plot development is one such case where extreme caution is required. As a consequence, Salt suggests instead focusing on overall properties of the data, such as the statistical distribution of shots of specific lengths in a film regardless of those shots’ sequential positioning. This also reveals interesting patterns of change over time – and also interesting commonalities. Salt suggests that the overwhelming majority of films fit within a standard statistical model found to hold when some variable, in the present case shot lengths, “is determined as a result of the probabilities associated with a large number of independent causative factors being multiplied together” (Salt 2007: 391). Removing information concerning the sequential unfolding of a film is, however, clearly disruptive for exploring narrative issues quantitatively and so even more sophisticated statistical measures may be necessary. Exploration of data sets of this kind has recently been taken considerably further by the research team of James Cutting, where several methods are being applied to uncover regularities including, but not restricted to, average shot lengths (Cutting et al. 2010, 2011). Cutting et al. present results based on 143 films released between 1935 and 2005 that relate variations and changes in shot lengths both to narrative structure and to patterns of attention allocation. More specifically, Cutting et al. proposed on the basis of their analysis that there was strong statistical evidence supporting Thompson’s proposal of a four-act structure described above. As noted before, this would be of considerable interest because it would establish a link between measurable properties of a film and its more abstract dramaturgical organisation. In order to compare films of very different lengths, Cutting et al. “normalised” films to a standard length of 1000 arbitrary units and then explored shot lengths within portions of the film created by dividing the film into four equal segments

Looking for what counts in film analysis

309

(i.e. forming boundaries at the 250, 500, and 750 marks). Clear genre differences appeared to result, with adventure and comedy showing a similar gentle decrease in shot length over the four segments, accentuated slightly in segments 1 and 4, while action films exhibit a sharp decrease across the entire film with a precipitous drop in segment 4 (Cutting et al. 2011: 5). There are then good grounds for believing there to be interesting correlations between, on the one hand, broader aesthetic categories and recipient-related properties (such as attention) and, on the other, fine-grained measurable physical features of film. Nevertheless, as the discussion in Cutting et al. (2012) makes very clear, the question of precisely which statistical methods are going to be most appropriate for discovering significant patterns is still open. Cutting et al. originally went further and proposed that there was also evidence for a clear, i.e. statistically significant reoccurrence, of a pattern of shot lengths in which shots were longer at the beginning and ending of acts. This seemed reasonable from both a production and a narrative perspective: at the beginning of an new act in the story, the viewer might need to be given more time to orient themselves, while at the end, there might need to be some explicit indication that a narrative pause was about to take place; this would also offer further support for Thompson’s four-act model. However, following a critique from Salt, the “U”-shaped pattern of shot lengths with borders at presumed act boundaries was revealed to be an artefact of their statistical method. Simply by dividing the film into four segments had created a statistical bias for obtaining longer values around the 250, 500 and 750 marks; when they repeated the statistics with films divided into three segments, they again found “U”-shaped patterns, but this time grouped around the 333 and 667 marks and not at the marks that had previously been used. Further studies are therefore evidently necessary because the distributions of other phenomena, such as the genre-specific differences mentioned above and the distribution of other technical film features, such as fades, dissolves and wipes, did appear to show sensitivity to an actlike structure, although not to all parts of that structure equally (Cutting et al. 2012: 144). Research of this kind is already benefitting from results drawn from another area, that of automatic audiovisual processing; Cutting et al. for example, employed automatic shot boundary recognition techniques for an initial segmentation of their data. A considerable research effort is currently being expended on automatic processing techniques of many kinds for the purposes of indexing, retrieval and knowledge extraction from audiovisual materials. There has been substantial development in recent years and the range of properties that can be isolated automatically has grown considerably both in number and variety. This then also makes data available that will be of use for quantitatively-supported film analysis. Approaches in this area typically work by analysing audiovisual sequences for large numbers (usually several hundred) of very “low-level” features additionally characterised, in the case of visual information, by their spatial position

310

John A. Bateman

and size (termed a Scale Invariant Feature Transform, or SIFT: Lowe 1999). These “measurements” then serve as the basis for training focused “event detectors” that fire when relatively more abstract features are judged to be present. Such features now include face detection, shot boundaries, scene boundaries, shot similarities, various classes of audio events, spoken language, text inserted in images, motion detection and many more. Several of these are precisely the kinds of technical details that are pursued in quantitative stylistic approaches such as Salt’s above or cinemetrics and so may be directly employed for exploring correlations between dramaturgical patterns and technical features for different genres, historical periods and directors as described above. Others, as we shall see, are rather more lowlevel and require further intermediate processing. Figure 2, for example, shows a screenshot of a tool developed in our own research for working with such automatic analysis results. The film being investigated plays in the video panel visible in the tool and, to the left of this, raw analysis results are shown for any detection track selected for closer examination. Below the video panel several of these results are displayed graphically: these can be selected and mixed freely as decided by the analyst. For ease of reference, the figure labels the particular tracks selected in this case. As such tools develop further, a broader range of measurable features will become available for research in precisely the same manner as previous work, such as cinemetrics, is now making shot lengths accessible. The provision of several parallel tracks of analysis is useful for several reasons. First, whereas the accuracy of full automatic recognition for individual tracks often still leaves much to be desired – a particular actor’s face may well fail to be recog-

Fig. 2: Screen shot of our augmented video player showing the results of several automatic analysis tracks for the film The Sixth Sense (1999, dir. M. Night Shyamalan); the image with Bruce Willis shown occurs at 00:09:10 into the film.

Looking for what counts in film analysis

311

nised or be recognised when it is not there – “combining” information from different tracks offers a significant method for improving accuracy: for example, if there is a hypothesis that a particular face has been recognised and there is a successful recognition of lip movement and in the audiotrack there is a reasonable probability recognition of the voice that has previously been associated with that character, then the original hypothesis that a particular face has been recognised can be raised to almost a certainty. Such a method may also be psychologically plausible for real viewers, as they will also combine possibly inaccurate partial information during perception in order to achieve more robust processing. Second, and as will be suggested in more detail below, combining information begins to offer ways of raising the level of abstraction of the information that can be recognised automatically. A dialogic conversation between two figures in a film, for example, is often a narratively significant episode but is far beyond what a low-level automatic analysis would deliver. But, when we again combine information – two successful face recognitions, two distinct voice recognitions, alternative shot similarities in terms of a visual pattern ABABA … (visible as repeated crossing arcs in the top coding line of Figure 2) and so on – then it becomes very likely that we have discovered a “narratively” significant film segment, thereby achieving the desired stepwise increase in levels of descriptive abstraction from quantitative to qualitative. Another style of empirically-based film research rather different to those we have seen so far focuses more directly on the film recipient, the spectator, than on objective details of the film as artefact. Most of this research is anchored in psychology, where films offer usefully controlled data for the investigation of a variety of issues involving human perception; an overview of important aspects of this emerging field is given in Smith et al. (2012). One of the primary methods employed in this area is eye-tracking, which offers very fine-grained real-time measurement of viewer responses to film. Here the research work of Tim Smith has been particularly influential. Smith’s experiments show in detail how features (but not exclusively) of the visual stimulus contribute to viewers’ allocation of “attention” while viewing. The idea that technical devices are deployed within films as strategies on the part of the film-maker for guiding the attention of the viewer to precisely those pieces of information which are important for interpretation is relatively common (cf. Bordwell 1989, 2007). Smith has now shown experimentally just how important this is and presents a detailed theory of the role of attention allocation during film-viewing (Smith and Henderson 2008; Smith 2012). The resulting “attentional model” of film perception offers convincing explanations of observed eye movement behaviour in a broad variety of films and circumstances. There is also work that examines viewers’ responses to films using physiological measurements, such as skin conductivity and pulse rate. Monika Suckfüll, for example, has conducted extensive data collection of viewers watching particularly

312

John A. Bateman

selected short films and film extracts in order to explore the emotional response of those viewers as a function both of types of individual viewers (Suckfüll and Scharkow 2009) and the organisation of the film extracts themselves. Results from data gathered with respect to the prize winning short animation film Father and Daughter (2000, dir. Michael Dudok de Wit) are presented in Suckfüll (2010); I return to these results below. Reception studies of this kind share the benefits of eye-tracking in that the data obtained is of a relatively high temporal resolution compared with methods such as post-viewing questionnaires. It is not possible for viewers to report on their fine-grained responses to films and those films’ internal organisation and so more direct measurements of affect are invaluable. Similarly, the fine-grained interpretative processes are often not accessible to viewers/hearers and so also cannot be probed with traditional post-viewing reception-study methods. Finally, a growing body of research is applying results and methods obtained within the brain sciences, particularly approaches that map spectators’ brain activity while they are viewing films. The currently most sensitive method applied here is functional magnetic resonance imaging (fMRI), which gives real-time views of the spatial distribution of brain activity. Results in this area more broadly, i.e. not focusing on film, appear to reveal considerable detail concerning the functions fulfilled by particular brain regions. The approach has now also been shown to be relevant for film in work by Uri Hasson and colleagues, where it has been demonstrated that highly similar courses of brain activity occur across viewers while watching particular film sequences (Hasson et al. 2004, 2008). Thus, while it has long been assumed that a significant part of filmic meaning is “inter-subjectively viable” – i.e. viewers will, by and large, respond to many film segments in similar ways – the brain imaging results strongly corroborate this belief and reveal a direct neurological correlate. The prime methodological component of this work relies upon the measurement of “inter-subjective correlations” across subjects during film-viewing. Hasson et al. report that “about 45 percent of the neocortex” (Hasson et al. 2008: 3) showed statistically significant intersubjective correlations. The brain areas involved range across language, emotion, visual processing, auditory areas, multisensory areas and more. Recent research by Talma Hendler and colleagues suggests further that emotional responses can be correlated more finely not with the activity of particular brain regions but with systematic “ synchronisation” of activity across clusters of regions (Hendler 2011; Raz et al. 2012); this has also now been observed during film viewing. Whereas some of these results could perhaps have been expected – for example, since the audiovisual stimulus is the same across subjects, it might be predictable that audiovisual processing would have something in common – for other areas, such as emotion, the results are more challenging. The neurological evidence strongly supports positions that have claimed film to guide and direct emotional responses in viewers in a highly effective fashion – a phenomenon aptly

Looking for what counts in film analysis

313

termed the “aesthetic orchestration of arousal” by Torban Grodal (Grodal 1999: 130). There is now a growing awareness of the need to achieve a theoretical and practical grasp of this “orchestration” (cf. Smith 1995, 1999) and a natural question to be addressed concerns just how the technical features of films contribute to the process.

4 So what counts? The design of descriptive categories that are both objectively measurable and useful for further analysis of a qualitative kind still presents substantial challenges. These challenges are not due, by and large, to problems of physical measurement – it is, for example, possible to measure camera angles, to recognize characters, to extract gaze vectors, to measure brightness, colour and sound frequency distributions, to state how characters are positioned and move with respect to objects and other characters, and so on. The problem is rather one of knowing which of these many variables in which combinations are going to be significant for an analysis – that is, which of these combinations are going to correlate with differences in genre, with differences over time as film style develops, and with differences in narrative form, both across entire films and within films as the story unfolds, possibly in terms of distinct acts or other dramaturgically relevant units, such as scenes and thematic episodes (making up acts) or events (making up scenes). This difficulty of relating particular technical film features to qualitative interpretations of these kinds is already raised in the following early critique from Bordwell and Thompson concerning Salt’s original statement of his quantitative approach: [Salt] ignores the possibility that a film’s style is not simply an aggregation of discretely measurable parameters but rather a system in which some parameters are interdependent and some are promoted to a level of salience that others never achieve (Bordwell and Thompson 1985: 227).

This criticism holds for all approaches where too direct an interpretation of lowlevel features is made and has important consequences for just how quantitativelyoriented film research can be pursued. For example, Bordwell and Thompson stress further that quantitative approaches must also take into consideration notions such as narrative function and the role of the spectator in order to begin grouping combinations of parameters appropriately (Bordwell and Thompson 1985: 227–228). They find it unlikely that the large quantities of data made available at the lowest levels of description will of themselves reveal interesting distinctions. While it is possible with sufficiently sophisticated statistical methods that valuable patterns will emerge, it is also the case that purely “bottom-up” approaches of this

314

John A. Bateman

kind may overlook functionally-motivated clusters in the data that could improve our ability to construct more discriminating hypotheses for investigation. Moreover, as the difficulties described above concerning Cutting et al.’s attempts to correlate variations in shot lengths with narratively motivated categories such as “acts” indicate, it is crucial when analysing quantitative data to have good motivations both for what is counted and how it is counted. The decision to explore acts following Thompson’s model was guided partly by the straightforward metric provided by Thompson herself, i.e. that the acts should be of approximately equal length; however, the subsequent analysis showed this to be problematic. There is suggestive finer-grained variation in the data but relating this to narrative interpretation will itself require narrative models that are more fine-grained than acts. Cutting et al. suggest that scenes, sequences of scenes, and events may provide more accurate guidance for searching for patterns of physical instantiation but, at the same time, these are far more difficult to determine (Cutting et al. 2011: 2). Questions of when one scene ends and another begins would require considerable elaboration before the empirically robust segmentations necessary for empirical work would be possible. Interestingly, very similar methodological stages can be observed in “data-driven” approaches in linguistics; earlier pure learning techniques are now often complemented with knowledge gained independently about the linguistic system and its organisation, while corpus work nowadays rarely approaches corpora without concrete hypotheses to be investigated with respect to the data. The open question for audiovisual material is then in what terms might corresponding audiovisual hypotheses be expressed. To make progress on this issue, the notion of “system” used by Bordwell and Thompson in the quotation above is crucial. By this, Bordwell and Thompson refer to the fact that choices of cues and other filmic devices in film are rarely, if ever, singular – entire sets of cues will be selected to pattern together in order to signal meanings that a viewer can follow. Moreover, the mobilization of aspects of material filmic form to carry patterns that lead viewers to particular interpretations rather than others is itself extremely flexible (for discussion and examples, see, e.g.: Metz 1974b: 133–134, Wulff 1999 and Bordwell 2004). As Edward Branigan phrases it, again employing a notion of “system”: Every process of signification is a formal play of differences. […] An important consequence of [this belief] is that there are no inherent meanings. For example, a dissolve in film does not inherently signify a short lapse of narrative time; in a particular system, however, a dissolve may mean just that. […] In addition, the meaning is not unique – other physical properties may be replaced by another device […] precisely because it is system, not material form, which determines meaning (Branigan 1984: 29).

There are many examples of this well known in film analysis and strong indications that these systems change over time. For example, whereas in early films introduction of dream sequences might necessarily involve particularly marked shot transitions, such as a close-up on a sleeping person following by the image

Looking for what counts in film analysis

315

going out of focus, nowadays there might be no formal marking beyond a simple cut (and sometimes not even that). Systems made up of contrasting alternatives may even be developed within individual films. Whereas, for example, a fade-toblack between two shots may conventionally indicate a scene boundary and/or a lapse of time, in Krzysztof Kieślowski Three colours: Blue (1993) a fade-to-black accompanied by loud orchestral music is regularly employed “within” scenes as a pronounced marker of an emotional response on the part of the main character (Bateman and Schmidt 2012: 80). The “relative” effect mentioned above of a medium shot acting like a close-up in the context of a film consisting otherwise entirely of long-shots is a further example of this flexibility of attributions of function. Finding these “systems” that contribute to filmic interpretation and which provide sense to the stream of technical features deployed then remains the major stumbling block for film analysis. More detailed characterisations of the “formal play of differences” underlying every “process of signification” have shown themselves to be particularly resistant to isolation: it is simply unclear which groupings of filmic technical features will do the job. Considerable work is needed to discover which combinations of which technical resources are carrying interpretable patterns, when, and under what conditions. Without this level of detail, hypotheses as necessary for psychological and related forms of investigation remain weak. Moreover, this is equally important for interpretative, hermeneutic analysis. There also the basis for discussion needs to be made as secure as possible; even when hypotheses are probed by example, the treatment of those examples needs to be as rich and empirically well founded as the material warrants, and for film this has not been developed nearly enough. Each of the empirical quantitative approaches introduced above considered relationships to more abstract systems of organisation in order to help characterise the “significance” of what they were counting or measuring. As we saw, Cutting et al. (2011) explicitly related both their results and the method they employed to form statistically-testable hypotheses to assumptions concerning the “act structure” of Hollywood films. Similarly, Smith’s (2012) attentional model requires an account of what film spectators will consider to be worth their attention if observed eye-tracking behaviour is to be explained – not all of the variability observed in data can be accounted for by purely “bottom-up” triggers such as visual properties or motion. Recent accounts of audiovisual processing such as the “information gain” model of Kerstin Schill and colleagues make a strong case that attention is strategically deployed in order to gain information relevant for the particular tasks and goals of the subject (Schill et al. 2001) – this builds a direct bridge between behavioural data and more abstract organisations, which may well include narrative concerns. Suckfüll’s (2010) account is particularly clear with respect to its reliance on more abstract, narrative underpinnings. The film she analyses, Father and Daugh-

316

John A. Bateman

ter, is commonly found to exert a strong emotional effect on viewers and Suckfüll consequently investigated whether her measurements of physiological responses could be correlated with more abstract, and particularly narratival, properties of the film. Points of narrative significance were used to make experimental hypotheses concerning where significant changes in physiological responses could be expected in viewers while watching the film. For this to be possible, it was necessary to have a more or less refined externally-defined model of the film’s aesthetic and dramaturgical form so that hypotheses could be generated independently of the measured physiological responses. Suckfüll drew here on the broadly psychologically-motivated framework for filmic narrative developed by Peter Wuss (1993, 2009). Wuss’ model proposes several kinds of narratively significant organisations; the two adopted as relevant in Suckfüll’s study are “plot points”, involving moments of conflict, dramaturgical turning points, areas of risk, etc. and “topic lines”, involving “intra-textual repetition”. For Father and Daughter Suckfüll identifies three such reoccurring motifs: “futility”, “encounter” and “effort”. At particular places within the film these motifs converge and Suckfüll hypothesised that these would then correspond to moments of strong narrative impact. These were then adopted as hypotheses concerning where in the film physiologically measurable responses would most likely occur (Suckfüll 2010: 45). These hypotheses were supported to a certain extent but there was also considerable further variation that was not yet accounted for by the narrative model. This raises a number of concerns. First, the narrative model adopted is limited with respect to its “granularity”: that is, the narrative organisations proposed are still some distance from particular combinations of technical features. This is also the case in Cutting et al.’s consideration of act structures: in the best case, these remain large organisational units that are a considerable distance from the blowby-blow dramaturgical and aesthetic development playing out within those acts. The resolution of the hypotheses resulting cannot, as a consequence, be very high. Second, the accuracy and reliability of the narrative model may itself be questioned. Suckfüll asks to what extent other analysts would have analysed the narrative organisation of her data in the same way to produce similar predictions concerning likely segments of significant effect. As a consequence, although useful, the precise guidance offered by Wuss’ model appears limited – Suckfüll cites Wuss as follows: [The topic lines’] existence can often only be proven through exact observation and careful structural comparisons between certain passages of film. … Thus there are no identifiable patterns from which one could orient oneself, but rather a broad field of different forms of narration […] (Wuss 2009: 68–69 cited in Suckfüll 2010: 46–47).

This relates back to Cutting et al.’s observations mentioned above concerning the difficulty of reliably segmenting films according to narratively motivated units smaller than acts, such as scenes. As is generally the case for quantitative studies,

Looking for what counts in film analysis

317

when the segmentation or coding of segments is not reliable, that is, when different analysts may produce different decisions on segment boundaries, then this is a poor basis for further empirical study. Since different analysts may have different ideas about just which events are significant, when those events first start, where they come to an end, etc. any segmentation that draws on such narrative interpretation may be suspect – not because that interpretation is wrong, but because it cannot be guaranteed that differing analysts are suggesting segmentations on the basis of the same criteria. Can the analyst be certain, for example, that different “coders” will segment Father and Daughter for aspects of “futility” reliably? A lack of reliable hypotheses is an increasingly serious problem when the data set grows; unreliable hypotheses will have unpredictable effects. It would clearly be beneficial for forming hypotheses for investigating a body of data empirically if more reliable foundations could be found for generating predictions.

5 Using discourse semantics as a focusing and selection mechanism To support more systematic interpretative analysis, it would be beneficial if we can find systems of contrasts that organise and pre-structure the filmic devices employed in a manner which is less open to subjective variation. The proposal that I will make here is that such contrasts can be drawn directly from an appropriate conception of “textuality” – that is, it is “texture” that is taken to define the context within which “discretely measurable parameters” function. In other words, the approach that will now be developed will define a set of analytic contrasts that are supportive of more abstract interpretation while, at the same time, being anchorable in phenomena observable in the film material; examples will follow below. To build this level of description, I draw on the highly differentiated semiotic account of “discourse” developed in the context of Hallidayan socio-functional linguistics. Discourse under this view is the semiotic resource that provides the mechanisms for relating particular, recognisable configurations of technical features – be they in language or some other semiotic mode – to their potential interpretations in context. Whereas possible interpretations may at first appear boundless, in fact semiotic artefacts are often designed to guide their receivers in specific, intended directions. Precisely this aspect of “guiding interpretation” is what is distinctive about the level of discourse. I will suggest that this linguistically-inspired view of discourse provides much of what we need for constructing the kinds of detailed descriptions of filmic artefacts required for hypothesis formation during empirical analysis. There has long been a connection with “discourse” drawn in film theory and there have been many calls to treat film as a kind of discourse, analysing film using discourse analytic techniques. The main challenge is to find appropriate

318

John A. Bateman

characterisations of filmic discourse organisation that can, on the one hand, serve to guide empirical analyses and, on the other, be sufficiently responsive to empirical results that improvements in the discourse account can follow. Until relatively recently, however, there were few discourse techniques sufficiently powerful to be applied to audiovisual media. This situation has now changed (cf. Bateman 2007; Tseng 2013; Wildfeuer 2012); there are several well developed frameworks that have considerable relevance for the analysis of film, one of which I will apply here. Within the socio-functional semiotic view of language, discourse is a semiotic resource just like all other levels in the language system. Resources of this kind are generally decomposable into more or less independent, but interacting, “functional regions”. A functional region within a resource is responsible for some particular area of meaning within the responsibilities of the resource as a whole: typically examples from grammar would include the region responsible for time and tense, the region responsible for making sentences fit into their textual context (with pronominalisation, word-orders of particular kinds, etc.), the region responsible for expressing activities of various types, and so on. Martin (1992) has set out a similarly articulated view of discourse for language, including various functional regions that are necessary for guiding discourse interpretation. This notion has now been applied to “filmic discourse” with several corresponding functional regions within this resource receiving detailed attention. This is suggested graphically in Figure 3. Each “ring” in the figure denotes an identified functional region; the first from the left is responsible for indications of time, duration and rhythm

Fig. 3: Graphical rendition of audiovisual discourse semantics and some of its functional regions. Any discourse is taken to be analyseable simultaneously with respect to all of the indicated functional regions.

Looking for what counts in film analysis

319

in film, the second for the basic narratological distinction between events in the story and the order and style of presentation of those events within filmic discourse structures, and so on. These regions taken together can be seen as complementary facets or “dimensions” of textuality: all apply to all instances of film simultaneously, viewing the filmic artefact’s guidance of interpretation from different, but interwoven, perspectives. In the remainder of this section, I will focus on just three of these facets – filmic discourse relations, filmic syntagmatic organisation or discourse structure, and filmic cohesion – in order to suggest how they offer a more reliable foundation for pursuing empirical analysis. All three will first be illustrated with respect to a single example taken from Father and Daughter, which I will also use further below. The plot line of the film can be summarised as follows. The narrative starts with a father and small daughter riding on bikes. They come to the sea and the father leaves in a rowing boat. The girl keeps coming back and looking out over the sea, repeating this through several cycles until she is herself an old woman. She then goes into what was the sea but which now appears to have become dry land, finds an old wrecked rowing boat, and falls asleep there. Then she wakes up, sees her father and runs to him, becoming a young girl again as she runs. As remarked above, the film commonly exerts a strong emotional effect on viewers, which cannot be attributed solely to its content. Filmic discourse relations: This functional region establishes systems of contrast between discourse semantic relationships such as temporal succession, similarity, contrast, spatial relationships and logical dependence; extensive discussion is given in Bateman and Schmidt (2012). Using this scheme it is possible to examine successive shots in films in a precisely analogous way to the examination of successive clauses in a text and to classify the filmic discourse relations that hold. The discourse relations provide those relationships available to viewers for “attaching” successive film segments into their growing understanding of the film while watching and listening. Viewers need to find particular relations in order to have “made sense” of what is occurring on screen. Since the number of possible relations is quite small, the range of possible interpretations is then already constrained considerably. We can show this functional region in operation with respect to the example extract from Father and Daughter set out in Figure 4. The figure shows keyframes from seven successive shots of the film. The extract starts with the daughter arriving with her family on bikes above the sea; her partner and children then run down the bank to the sea; there is a shot of a seagull taking wing; the man and the children sit at the foot of the bank by some mooring posts; the daughter is shown at the top of the bank looking towards the sea; there is a tilted angle shot of the tops of trees blowing in the wind; and then there is a shot of a snow covered field. Appropriate discourse relations are then sought for each pair of shots. First,

320

John A. Bateman

Fig. 4: Shots 63–69 (4:21–4:29) from Father and Daughter (2000, dir. Michael Dudok de Wit).

i.e. between shots 63 and 64, there is a probable “temporal continuity” with a spatial shift to a “neighbouring connected spatial region”. The continuous successive temporal relation then colours the interpretation of all the shots following until shot 68; this is supported by continuous music that only pauses and changes motif as shot 67 comes to an end. Shot 65 is temporally bound but only loosely spatially bound as somewhere in the vicinity: it is not constructed as a point-ofview shot and so may simply be a further action, caused by the fast approach of the children: it functions structurally as an insert since shot 66 then returns to the same location as shot 64. Shot 67 then similarly brackets shots 64–66 since it returns to the top of the bank, although only showing the daughter. The relation between shot 67 and 68 is then more complex: it is not a point-of-view shot, since the daughter is looking out to sea, but its tilted angle and the replacement of the music by the mournful sounds of the trees blowing in the wind nevertheless indicates a “projection”, or subjective relationship connected with the daughter. The relation between shots 68 and 69 is then one of marked “temporal discontinuity” since now there is snow on the ground; the music also begins again with a quite different phrasing; no identifiable spatial connection is established and so the relationship is “distant”. This web of relations across shots identifies how shots most likely “hang together” (since this is a discourse interpretation, it is always necessarily “defeasible”; cf. Bateman 2011) and as a consequence begins to create fine-grained segmentations that are narratively relevant for further interpretation. Filmic syntagmatic organisation: This component of the discourse semantics then provides a structure across which the patterns of discourse relations can be built; Bateman and Schmidt (2012) again offer a detailed introduction. The essential idea is that films present audiovisually identifiable chunks of “spatiotemporal” worlds which serve as a filmic backbone for driving interpretation. For example, when any identifiable location reoccurs, the viewer is faced with the task of relating this occurrence to previous occurrences. With respect to the film segment in Figure 4, almost all of the shots are spatially anchored in this way. Shot 63 is a place that

Looking for what counts in film analysis

321

is shown several times throughout the film: it is where the father and daughter first leave their bikes when the father leaves. Shots 64 and 66 are the mooring place, where the father’s rowing boat was originally tied up. Shot 65 is the “sea” shown at several points in the film. Shot 67 is the top of the hill, where the father last hugs the daughter. Only shot 69 represents an “unknown place”: i.e. a place that has not been introduced before; this of course strengthens its role as a scene or episode boundary. Recognition of places relies crucially on the visual perception system and is both robust and very fast. Much of the local coherence of film then emerges from relations that viewers establish over and across these identified locations: locations may form stations on an itinerary when linked by spatiotemporal continuity, for example, or may build alternations, as in chase sequences or telephone calls, and so on. Filmic cohesion: Finally, filmic cohesion describes the texture that is constructed by the “reappearance” of filmic elements as a film unfolds, identifying, tracking and classifying such reappearances as they occur. For cohesion, filmic elements include figures in a film, settings, particular technical features, audio components and so on. These elements are then placed in chains of reoccurrence establishing connections and linkages of various kinds across a film. Several of these linkages serve as support for other functional discourse regions, such as, for example, syntagmatic structure by virtue of the identification of locations. The potential relevance of cohesion for film analysis has long been noted (cf. Palmer 1989: 316), but little research building on these proposals followed until the development of a fully audiovisual system of cohesion by Chiaoi Tseng (Tseng 2008, 2013). Thus, although reoccurring elements of this kind are often referred to in analyses of film, Tseng’s filmic cohesion takes this further by combining it with Ruqaiya Hasan’s notion of cohesive harmony (Tseng 2008). It is not then individual patterns of reoccurrence, or “cohesive chains”, that are considered significant for filmic texture but rather “interconnections” across chains. Such interconnections have been found to support a variety of more abstract descriptions suited to narrative interpretation. Returning again to our film segment in Figure 4, cohesive chains are established for each of the four characters involved and for the locations as mentioned above. In addition, the tree tops blowing in the wind in shot 68 are also a recurring motif, although not specifically spatially anchored. It can be hypothesised that some of these places become specifically “charged” with associations by means of the chains in which they participate – for example, the top of the hill (shot 67) is the place where the father last hugs the daughter, and the trees at the top of the hill (shot 63) is where the daughter waited and waited when she was small. Empirically-relevant discourse interpretations: All of the functional regions described offer analyses that are substantially more abstract than straightforwardly measurable technical features but which are nevertheless reliably applicable by

322

John A. Bateman

suitably trained analysts. Work on their automatic recognition using combinations of lower-level tracks of the kind shown in Figure 2 – for example, by picking up reoccurring motifs for inclusion in cohesive chains or locating reoccurring places to build syntagmatic structures – is in progress. Of primary relevance here, however, is the potential use of these analyses in order to make stronger predictions concerning the patterns that might be found in quantitative data. To recap, the goal is to provide a more fine-grained narratively-relevant analysis that, on the one hand, is both reliable and robust when applied by different analysts and, on the other, may be related either to recognisable physical instantiations or to reliably measurable responses of viewers to those instantiations. As discussed above, Suckfüll generated predictions concerning where effects might occur in the film Father and Daughter on the basis of a rather more abstract analysis of the narrative structure of the film. I will now contrast some of those predictions with those that arise from applying the functional discourse account of textuality. Functional discourse analysis of any film proceeds along all discourse dimensions in parallel, employing descriptions from each of the facets of filmic discourse introduced above. The interaction of these descriptions usually allows a detailed model of a film’s texture to be constructed. For example, for Father and Daughter the analyses combine to yield 10 locally connected “thematic episodes”: within each episode locations are grouped together in an “itinerary”, i.e. a sequence of places visited in a fixed order, the same cohesive chains are revisited, and the discourse relations indicate temporal continuity. Only at the boundaries to the episodes are these connections or patterns broken, showing the overall texture particularly clearly. The use of the cohesive chains of recurring locations to build syntagmatic structure is suggested graphically in Figure 5. In this representation, the cohesive chains for each location are shown similarly to staves in musical notation: each horizontal line identifies a location where individual shots may be

Fig. 5: Fragment of film syntagmatic structure for three thematic episodes from Father and Daughter (2000, dir. Michael Dudok de Wit), separated by vertical dashed grey lines and with illustrative screenshots for the first episode.

Looking for what counts in film analysis

323

anchored, depicted by hanging the relevant shots below the lines like musical notes. Thus we can see that shots 8, 9 and 10 – which show the event of the father taking his leave of the daughter – mark out locations on the itinerary moving from the “waiting place”, to the “parting place” to the “mooring place”. This basic cycle then recurs throughout the film. The three groups of shots shown in the figure separated by vertical grey dashed lines correspond to three such cycles. One of these was also to be seen in Figure 4 above; again, within each episode, we see the same repeated sequence of locations: the tree where the daughter repeatedly waits, the place at the top of the hill where the father hugs the daughter for the last time, and the mooring place at the bottom of the hill where the rowing boat is first seen. Within and across these locations, the cohesion analysis then makes explicit identifiable elements, tracking them as they reoccur. This yields a very similar organisation for all of the episodes in the film: riding out to the parting place, waiting, and riding back with reoccurring and/or varying elements along the way – including those we have seen as well as other motifs such as encountering other characters riding bikes in the other direction and climbing a hill where considerable effort must be exerted. In terms of filmic discourse relations, the film consists of 10 locally connected units whose internal parallelism meets the conditions required to posit an overarching text-internal “comparison” or “similarity” discourse relationship (cf. Bateman 2007; Bateman and Schmidt 2012). Viewers are then guided to an interpretation that includes making comparisons as the same events unfold again and again, with the daughter going through various life-stages on the way. Despite the strong connection evident to the narrative development of the film, it is important to emphasise at this point that the analytic structures produced have been derived with respect to the distinctions defined within each functional discourse region – i.e. the filmic discourse relations, discourse syntagmatic structure and cohesion – and not by following the story or intuitions concerning what the film might be intending. This therefore runs counter to Wuss’ comment cited above that there are “no identifiable patterns from which one could orient oneself”. The patterns derived from the functional discourse analysis are precisely such patterns. Moreover, the framework that produced them can be applied in the same way to any film. As a consequence, a far higher inter-coder reliability may be expected than with the standard, more interpretative analytic categories employed in film analysis – in fact, the framework of analysis provided here does not allow, at this level of abstraction, substantial variation at all. This then meets the goal set out above of achieving a higher-level of descriptive abstraction that is both reliably applicable and sensitive to fine-grained narrative concerns. For the particular purposes of Suckfüll’s analysis, therefore, the structures derived may support more formally and more reliably the identification of the desired points of potential narrative impact with measurable physiological consequences. Entirely analogously to Suckfüll’s (2010: 45) use of narrative impact for

324

John A. Bateman

Tab. : Predictions expressed in terms of functional discourse discriminations with examples from Father and Daughter.

A B C

D E

F

G

the audiovisual material portrays something that is of itself “affecting” in some way – such as when the father hugs the daughter for the last time; focus on a cohesively established participant for whom identification or sympathy has been created – such as when the daughter fights against a strong headwind; an indicated or assumed cohesive identity is revealed to be wrong or identification is delayed – such as when a group of several girls is shown and it may not be immediately clear to the viewer which one, if any, is the daughter; a pattern of syntagmatic closure is established – such as when the waiting episode ends for the second time; a new narrative episode begins with chances for new action, new developments, new patterns – such as when each episode begins, typically indicated by a change of musical phrasing and several descriptive shots of landscape; a pattern of syntagmatic repetition is broken – such as when the daughter first goes beyond the place where her father left her or when she goes beyond the mooring place into what was the sea; an identity is found that places the identified entity in an unexpected or different cohesive connection with unexpected other entities: G established entity is cohesively linked with a previously unlinked place – such as when the daughter goes beyond the mooring place into the long grass; G established entity is cohesively linked or re-linked with entities previously in distinct cohesive chains – such as when the daughter finds the rowing boat or is reunited with her father.

deriving predictions following Wuss’ narrative account, the functional discourse analysis similarly proposes segments where “discourse functional” import (of diverse kinds) may be expected; some of these prediction types are listed in Table 1 with illustrative examples suggested for Father and Daughter. The combined structure of syntagmatic organisation, discourse relations and cohesive chains picks out very clearly the main pivot locations of the film, where points of change and stasis are set up: for example, over 16 % of the film by time is spent at the locations making up the area where the daughter parts from and then repeatedly waits for her father (the “parting place”: cf. shots 8, 63 and 67 above). This opens up the possibility of fine-tuning the account by exploring whether the predictions it gives rise to are supported or not by the experimental data. Interestingly, the discourse analytic structures do not pull out quite the same places of predicted effects as Suckfüll’s account and, when they do coincide, sometimes this is for other reasons than those suggested by Suckfüll. As an example, let us consider the “encounter” motif. Suckfüll identifies two such encountering events in the film, suggesting that both will be places where significant physiological responses might be expected. In the first, shown in the left-hand screenshot of Figure 6, the daughter passes an older woman on a bicycle on her way back from

Looking for what counts in film analysis

325

Fig. 6: Two encounters from Father and Daughter (2000, dir. Michael Dudok de Wit; shots 31 and 86 respectively) annotated for characters and motion; length of motion arrows indicates speed.

the parting place; in the second, shown on the right in Figure 6, the daughter, now herself an old woman, is passed at some speed by a young girl again riding in the opposite direction. Now whereas Suckfüll is able to suggest on the basis of the combination of narrative elements that both shots would give rise to a significant physiological response, in fact only the first elicited a response; the second did not. In contrast, the cohesive analysis suggests that neither is significant because of an “encounter”: in both cases, the person the daughter “encounters” is not picked up by a cohesive chain anywhere else in the film and so does not contribute to an interaction between chains. There are several other properties of the first shot, however, which make it a likely candidate for physiological responses. First, it occurs at the end of an episode and so marks a suitable moment for reorientation; second, it is itself a highly dynamic shot showing the daughter moving at speed on her bike; and third, that movement is in the opposite direction to the previous shots where the daughter was shown to be battling against a strong headwind. These taken together are very likely sufficient to bring about physiological reactions and so a clear prediction here would be that a very similar pattern of response would be measured even if there were no encounter at that point. The narrative positioning of the second encounter lacks these indicators of salience and so is predicted not to give rise to reactions. As suggested by the examples given in Table 1, there are several other points of prediction that can be derived from the discourse analysis. However, whether or not these are borne out in relation to the data Suckfüll has collected is not yet known. This is an interesting question for future research. These and other formally derived points of potential narrative impact need to be explored to see whether they give rise to any measurable effects and whether different kinds of effects can be noted to correlate with different kinds of discourse configurations. Such predictions are moreover not only relevant for the kinds of physiological measures reported in Suckfüll (2010), any kind of response data might be used, such as facial expressions, which are also being analysed by Suckfüll, and fMRI. It is precisely this generation of a rich array of hypotheses for investigation that may open the door to more extensive empirical analyses of just what features of films “orchestrate” our responses.

326

John A. Bateman

6 Conclusions and outlook One of the considerable challenges in the analysis of the audio-visual moving image is to relate abstract social interpretations and audience effects with the realtime succession of technical features exhibited by the artefact itself. Qualitative high-level analyses, recipient effects and the measurable details of form all have very different natures and there is no simple connection between them. Nevertheless, an increasing body of work is now exploring empirically the audio-visual moving image, and particularly narrative film, by employing automatic and semiautomatic analysis techniques. This provides a wealth of information concerning the artefacts under investigation but it is unclear precisely how such information can best be related to established hermeneutic or aesthetic studies of the medium. This chapter has addressed this long-standing source of debate from the perspective of functional socio-semiotic theory and, in particular, of that theory’s account of discourse. In order to drive more focused empirical analysis, particularly of physiological and other kinds of behavioural data, more fine-grained descriptions of the structure of these artefacts are required. I have suggested that descriptions of discourse-level patterns might serve as a source of testable predictions for film segments that can be expected to have an effect on their recipients during film reception. By setting out systems of functional discourse discriminations at work in film in this way, I have argued that more motivation can be provided for selecting and grouping quantitatively measurable filmic phenomena. This offers a theoretically more robust notion of just what measurable phenomena “count” within a film or films, which in turn may support more precise empirical hypotheses for an increasingly fine-grained probing of the workings of film. In short, I have set out what might contribute to a kind of “functional cinemetrics”, i.e. finding “countable” properties of film which are nevertheless strongly indicative of functional interpretations appropriate for film. Acknowledgments: Our research on automatic feature recognition reported here was funded as part of the project Bild-Film-Diskurs funded by the German Ministry for Education and Research (BMBF, contract number 01UA0813A). The augmented video tool and related components were designed and implemented by Arne Jacobs and Andree Lüdtke during the project while at the TZI, Bremen. The research on predicting points of emotional impact is supported by an internal grant from the University of Bremen.

Looking for what counts in film analysis

327

References Bateman, John A. 2007. Towards a grande paradigmatique of Film: Christian Metz reloaded. Semiotica 167(1/4): 13–64. Bateman, John A. 2011. The Decomposability of Semiotic Modes. In Kay L. O’Halloran and Bradley A. Smith (eds.), Multimodal studies: Multiple approaches and domains Routledge Studies in Multimodality, 17–38. London: Routledge. Bateman, John A. 2013. Using Multimodal Corpora for Empirical Research. In Carey Jewitt (ed.), The Routledge Handbook of multimodal analysis, 2nd edition. London: Routledge. Bateman, John A. and Karl-Heinrich Schmidt. 2012. Multimodal Film Analysis: How films mean. (Routledge Studies in Multimodality.) London: Routledge. Baxter, Mike. 2012. Picturing the Pictures: Hitchcock, statistics and film. Significance 9(5): 5–9. Bordwell, David. 1989. Historical Poetics of Cinema. In R. Barton Palmer (ed.), The Cinematic Text: Methods and Approaches, 369–398. New York: AMS Press. Bordwell, David. 2004. Neo-structuralist Narratology and the Functions of Filmic Storytelling. In Marie-Laure Ryan (ed.), Narrative across media. The languages of storytelling, 203–219. Lincoln/London: University of Nebraska Press. Bordwell, David. 2007. Poetics of Cinema. London/New York: Routledge. Bordwell, David and Kristin Thompson. 1985. Toward a Scientific Film History? Quarterly review of film studies 10(3): 224–237. Bordwell, David and Kristin Thompson. 2010. Film Art: An Introduction. (Ninth edition.) New York: The McGraw-Hill Inc. Bordwell, David, Kristin Thompson and Janet Staiger. 1985. The Classical Hollywood Cinema. Film Style and Mode of Production to 1960. New York: Columbia University Press. Branigan, Edward. 1984. Point of View in the Cinema: A theory of narration and subjectivity in classical film. (Approaches to semiotics 66.) Berlin: Mouton. Cutting, James E., Kaitlin L. Brunick and Jordan E. DeLong. 2011. How Act Structure Sculpts Shot Lengths and Shot Transitions in Hollywood Film. Projections: The journal for movies and mind 5(1): 1–16. Cutting, James E., Kaitlin L. Brunick and Jordan E. DeLong. 2012. On Shot Lengths and Film Acts: A revised view. Projections: The journal for movies and mind 6(1): 142–145. Cutting, James E., Jordan E. DeLong and Christine E. Nothelfer. 2010. Attention and the Evolution of Hollywood Film. Psychological Science 21(3): 432–439. Flückiger, Barbara. 2011. Die Vermessung ästhetischer Erscheinungen. Zeitschrift für Medienwissenschaft 5: 44–60. Fuxjäger, Anton. 2009. Wenn Filmwissenschaftler versuchen, sich Maschinen verständlich zu machen. Zur (mangelnden) Operationalisierbarkeit des Begriffs Einstellung für die Filmanalyse. Maske und Kothurn: Internationale Beiträge zur Theater-, Film- und Medienwissenschaft 55(3): 115–127. [Special issue: “Digital Formalism. Die kalkulierten Bilder des Dziga Vertov”. Edited by Klemens Gruber and Barbara Wurm with Vera Kropf.]. Grodal, Torben. 1999. Emotions, Cognitions, and Narrative Patterns in Film. In Carl Plantinga and Greg M. Smith (eds.), Passionate Views: Film, cognition, and emotion, chap. 6, 127–145. Baltimore/London: The John Hopkins University Press. Halliday, Michael A. K. 1978. Language as Social Semiotic. London: Edward Arnold. Halliday, Michael A. K. and Christian M. I. M. Matthiessen. 1999. Construing Experience through Meaning: A language-based approach to cognition. London: Cassell. Hasson, Uri, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin and David J. Heeger. 2008. Neurocinematics: The neuroscience of film. Projections: The Journal for Movies and Mind 2(1): 1–26.

328

John A. Bateman

Hasson, Uri, Yuval Nir, Ifat Levy, Galit Fuhrmann and Rafael Malach. 2004. Intersubject Synchronization of Cortical Activity during Natural Vision. Science 303(5664): 1634–1640. Hendler, Talma. 2011. Afilliate Lab Interview. Quarterly newsletter for the Center for Cognitive and Social Neuroscience 1–2. University of Chicago. Available at: http://ccsn.uchicago.edu/ static/ccsn/newsletter/9/Spring_2011.pdf. Kress, Gunther and Theo van Leeuwen. 1996. Reading Images: The grammar of visual design. London/New York: Routledge. Kress, Gunther and Theo van Leeuwen. 2001. Multimodal Discourse: The modes and media of contemporary communication. London: Arnold. Labov, William. and Joshua Waletzky. 1978. Narrative Analysis. In June Helm (ed.), Essays on the verbal and visual arts, 12–44. Seattle: University of Washington Press. [Proceedings of the 1966 Spring Meeting of the American Ethnological Society]. Lowe, David G. 1999. Object Recognition from Local Scale-invariant Features. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV), 1150–1157. Martin, James R. 1992. English Text: Systems and structure. Amsterdam: Benjamins. Metz, Christian. 1974a. Film Language: A semiotics of the cinema. Trans. by Michael Taylor. Oxford/Chicago: Oxford University Press and Chicago University Press. Metz, Christian. 1974b. Language and Cinema. Trans. by Donna Umiker-Sebeok. (Approaches to Semiotics 26.) The Hague: Mouton. Palmer, R. Barton. 1989. Bakhtinian Translinguistics and Film Criticism: The dialogical image? In R. Barton Palmer (ed.), The cinematic text: methods and approaches, 303–341. New York: AMS Press. Raz, Gal, Yonatan Winetraub, Yael Jacob, Sivan Kinreich, Adi Maron-Katz, Galit Shaham, Ilana Podlipsky, Gadi Gilam, Eyal Soreq and Talma Hendler. 2012. Portraying Emotions at their Unfolding: A multilayered approach for probing dynamics of neural networks. NeuroImage 60(2): 448–1461. Available at: http://dx.doi.org/10.1016/j.neuroimage.2011.12.084. Reisz, Karl and Gavin Millar. 1968. The Technique of Film Editing. London: Focal Press. Salt, Barry. 1974. Statistical Style Analysis of Motion Pictures. Film Quarterly 28(1): 13–22. Salt, Barry. 1983. Film Style and Technology: History and analysis. London: Starwood. Salt, Barry. 2007. Moving into Pictures: More on film history, style, and analysis. London: Starword. Schill, Kerstin, Elisabeth Umkehrer, Stephan Beinich, Gerhard Krieger and Christoph Zetzsche. 2001. Scene Analysis with Saccadic Eye Movements: Top-down and bottom-up modelling. Journal of Electronic Imaging 10(1): 152–160. Smith, Greg M. 1999. Local Emotions, Global Moods, and Film Structure. In Carl Plantinga and Greg M. Smith (eds.), Passionate Views: Film, cognition, and emotion, 103–126. Baltimore/ London: The John Hopkins University Press. Smith, Murray. 1995. Engaging Characters: Fiction, Emotion, and the Cinema. Oxford: Oxford University Press. Smith, Tim J. 2012. The Attentional Theory of Cinematic Continuity. Projections: The Journal for Movies and the Mind 6(1): 1–27. Smith, Tim J. and John M. Henderson. 2008. Edit Blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research 2(6): 1– 17. Smith, Tim J., Dan T. Levin and James E. Cutting. 2012. A Window on Reality: Perceiving edited moving images. Current Directions in Psychological Science 21(2): 107–113. Suckfüll, Monika. 2010. Films That Move Us: Moments of narrative impact in an animated short film. Projections 4(2): 41–63. Suckfüll, Monika and Michael Scharkow. 2009. Modes of Reception for Fictional Films. Communications 34(4): 361–384.

Looking for what counts in film analysis

329

Thompson, Kristin. 1999. Storytelling in the New Hollywood: Understanding Classical Narrative Technique. Cambridge: Harvard University Press. Tseng, Chiaoi. 2008. Cohesive Harmony in Filmic Text. In: Len Unsworth (ed.), Multimodal Semiotics: Functional Analysis in Contexts of Education, 87–104. London: Continuum. Tseng, Chiaoi. 2013. Cohesion in Film: Tracking film elements. Basingstoke: Palgrave Macmillan. Tsivian, Yuri. 2009. Cinemetrics, Part of the Humanities’ Cyberinfrastructure. In Michael Ross, Manfred Grauer and Bernd Freisleben (eds.), Digital Tools in Media Studies 9, 93–100. [Bielefeld: transcript]. Van Leeuwen, Theo. 2005. Introducing Social Semiotics. London: Routledge. Vogler, Christopher. 1998. The Writer’s Journey: Mythic structure for writers. Studio City: Michael Wiese Productions. Wildfeuer, Janina. 2012. Intersemiosis in Film: Towards a new organisation of semiotic resources in multimodal filmic text. Multimodal Communication 1(3): 233–304. Wilson, George M. 1995. Morals for Method. In Cynthia A. Freeland and Thomas E. Wartenberg (eds.), Philosophy and film, 49–67. New York/London: Routlege. Wulff, Hans J. 1999. Darstellen und Mitteilen: Elemente der Pragmasemiotik des Films. Tübingen: Gunter Narr Verlag. Wuss, Peter. 1993. Filmanalyse und Psychologie. Strukturen des Films im Wahrnehmungsprozeß. Berlin: Edition Sigma. Wuss, Peter. 2009. Cinematic Narration and its Psychological Impact: Functions of cognition, emotion and play. Newcastle: Cambridge Scholars.

Jana Holsanova

14 In the eye of the beholder: Visual communication from a recipient perspective Abstract: Why do not all viewers see the same image content? Why do viewers perceive and interpret visuals differently despite common perceptual and cognitive mechanisms? The article addresses a number of challenges in visual communication and underlies the need for empirical analyses of visuality and multimodality from a recipient perspective. Perception and interpretation of visuals is seen as an interactive meeting between the recipient, the multimodal message and the situational context. All of these three aspects modulate the process of meaning– making. First, the form and contents of the visual message serves as a starting for the process of meaning–making, Second, personal characteristics of the viewers modulate perception and interpretation of visuals. Viewers play an active role in the interaction with visuals, their interaction with visual messages is dynamic and they co-create its meaning. Inter-individual differences in perception of visuals arise thanks to variety of goals of the visual examination, the viewers’ background, interests, previous knowledge, expectations, domain knowledge or expertise, emotions and attitudes. In other words, both the bottom-up and top-down factors influence visual perception. Third, the context in which images are displayed, perceived and interpreted plays an important role for perception and interpretation of visuals. All these factors can be studied by using a multidisciplinary framework and integrated theories and methods.

1 Introduction Visuality and multimodality play an important role in modern society.1 In our everyday life, we encounter still and moving images, texts and graphical elements that are orchestrated in specific ways. These complex visual messages are designed to serve a certain purpose: to inform, explain, instruct, persuade, educate, sell and entertain. However, the knowledge about how we actually interact with these messages is rare. What catches our attention and what does not? How do we extract

 It has to be pointed out that there is a wide spectrum of images (Machin 2009). Mitchell (1986: 10) speaks in his book on Iconology about graphic images (pictures, statues, designs), optical images (mirror projections), perceptual images (sense data), mental images (dreams, memories, appearances, fantasmata) and verbal images (metaphors, descriptions, ideas). We will use the terms visuals and multimodal messages for both single images and for complex messages combining image, text, graphics, audio, video and animations.

332

Jana Holsanova

and integrate information from various sources during meaning–making? How do we navigate in order to find relevant information? The actual reception of the messages in a particular context of use has not been studied empirically in detail. There is still little knowledge about how users actually are affected by visual messages or whether the intended effects of a design have been achieved. In other words, there is still lack of empirical studies on recipients’ interaction with visual and multimodal messages. Despite the fact that designers have experience of what works for the readers or users (Machin 2009), designers’ predictions can built on rules of thumb, assumptions and theories that have not been tested empirically (Holsanova 2010) – take for example the placement of the advertisements on a newspaper spread. Wartenberg and Holmqvist (2005) report results from a large eye tracking study conducted for Society of Newspaper Design Scandinavia (SNDS) where they compared designers’ predictions with authentic reader behavior in 17 newspapers. The authors found that overall, designers were reasonably good at predicting actual reading order and reading time. However, designers were overconfident that color attracts attention, they underestimated the role of position for reading order, they overestimated the role of information graphics for reading order, underestimated the role of information graphics for deep reading and overestimated the time spent on photos.

2 Challenges of visual communication After the so-called “visual turn” coined by W. J. T. Mitchell, visual aspects of communication have been acknowledged as being important and worthy of an intense scrutiny. Researchers began to devote more scholarly attention to various kinds of “visuals” (Barnhurst 1994; Griffin 1991), including multimodal messages combining various modes of representation (Griffin 1992; Kress and Van Leeuwen 1996). Attention has also been drawn to the role of visual design and text design (Bucher 2007). But given the dominant role that visuality and multimodality plays in modern society, there is still lack of empirical studies on how recipients interact with visuals and media messages. There are a number of issues and challenges that must be considered in visual communication (Holsanova 2012). These issues and challenges will be addressed in detail below.

2.1 Visual communication can be studied from both production and reception perspectives Visual communication can be studied from both production and reception perspectives (Holsanova 1999; Bucher 2012; Holsanova 2013). “Production aspect” concerns the interplay between various resources (images, gestures, body language,

In the eye of the beholder

333

mimics, language, sounds, music), their contribution to the content of the message and their orchestration in order to achieve a certain effect, often referred to as “intersemiosis” (O’Halloran et al. 2012). Visuals and multimodal messages are created according to how an imaginary or hypothetical recipient would perceive the material. Through their composition, they offer a potential for users’ meaning– making (Kress and Van Leeuwen, 1996; cf. also Holsanova et al. 2006, Bateman 2008). “The reception aspect” is closely connected to recipients’ ability to select, attend to and process information, as well as the ability to integrate information from various resources and to fill in the “gaps”. It is the recipients who ultimately choose among the available information: what they want to explore, mentally process and interpret, in what order and how deeply (Holsanova 2013). Reception studies are therefore very important. Preferably, they should focus on the dynamic process of users’ actual interaction with the visuals and investigate general and individual patterns in the unfolding perception of visuals.

2.2 Recipients play an active role in the interaction with visuals and multimodal messages Perception and interpretation of visuals can be seen as an interactive meeting between the recipient, the multimodal message and the situation context (Holsanova 2010). That means that different people can perceive the same visual differently, and attribute very different meanings to it. It is therefore crucial to study how individual recipients interact with multimodal documents and create meaning. In order to illustrate this interaction, we will have a closer look at reader’s behavior on a newspaper spread. Figure 1 shows a typical example of how readers interact with a newspaper spread. Due to the page turning, the reader starts with an early brief glance at the right hand side of the spread (blue line), continues with a longer dwell on the image (the head of the bird, the bird’s eye and the chest), and a fixation on the top right object. After this short inspection (under one second), the reader continues with some reading in selected articles on the left hand side of the spread. The headlines serve as entry points. Note that few articles are read in full. The reading depth varies: a couple of articles are totally ignored, while others (for instance newspaper notices) are read in depth. Sometimes, only the headline is attended to and read by the reader. Some parts of the articles, for example the briefs, are read quite carefully. Usually, very few readers attend to the advertisements – unless they have a special interest or reason to inspect them. In this case, the brief of bottom left ad has been read – probably because the timing of the ad is crucial: it is time to submit the tax declaration. The example in Figure 1 also illustrates that readers do not process and interpret everything on the spread holistically, at one glance, as many believe. Reading and scanning are sequential processes where the focus of attention moves from

334

Jana Holsanova

Fig. 1: Reader’s visual interaction with a newspaper spread. The circles in Figure 1 indicate the position and duration of “fixations”, the lines connecting fixations represent “saccades”. The diameter of each fixation is proportional to its duration (the black dot in the bottom right corner represents one second). Fixations are stops or periods of time when the point of regard is relatively still. It is during fixations that we acquire useful information. Fixation duration is often used as a measure of how much cognitive effort is required for information processing. During saccades, the eyes move at a relatively rapid rate to reorient the point of regard from one spatial position to another. Saccades are very short, usually lasting from 20–40 ms. Our vision is suppressed and we are essentially blind during saccades. On the potentials and limitations of eye tracking methodology see Holsanova (2013). From Metro.

one place to another, and where much of the spread is not seen. Human attention has limited capacity and does not allow us to actively read and process information in multiple places simultaneously. To perceive, read and cognitively process information in detail, it is necessary to direct the gaze to relevant pieces of information – letters, words, parts of the image – so that they fall onto a high-resolution area of the eye, the fovea (Holsanova 2013).

2.3 Recipients’ interaction with visuals is dynamic Studies on image perception and scene perception show that they are viewed and attended to stepwise (Henderson 2007; Holsanova 2001, 2008). However, the dynamics of the perception and interpretation of visuals has not yet been widely

In the eye of the beholder

335

investigated. In audience research, the focus usually lies more on media effects than on the process of reception. By using traditional methods such as recall tests, questionnaires or interviews, researchers try to reconstruct reception indirectly, based on quantitative data (Bucher 2007). Direct methods and dynamic approaches are still very rare. What is the time-course in scene perception? In eye tracking research, there is agreement on viewers’ ability to get the initial holistic impression or the “gist” of the scene very early in the viewing process (Henderson, 2007; Locher et al. 2007). Several researchers have also found regularities characterized by specific phases of visual exploration. The visual exploration starts with an initial “global” phase consisting of a general survey where the eyes move with a series of relatively short pauses over the main portions of the image. This phase is then followed by a subsequent “local” phase of image exploration consisting of a series of long fixations, concentrated over small areas of the image, evidencing detailed examination of those sections (Buswell 1935: 142). We can also study the temporal and semantic build-up of the visual examination in more detail. What units do the viewers identify and at what level of abstrac-

Fig. 2: Image viewing. The visual pattern illustrates the path of image discovery. It shows which objects of the visual scene have been fixated by the viewer, in what order and for how long. The circles indicate the position and duration of the fixations, the diameter of each fixation being proportional to its duration. The lines connecting fixations represent saccades. This scanpath is a result of one minute of image viewing, one minute after start (Holsanova 2001). The motif comes from a children’s book by Sven Nordqvist (1990).

336

Jana Holsanova

Fig. 3: Multimodal score sheet (Holsanova, 2008: 111).

tion? The image content can be perceived and described on different levels and the fixation itself does not indicate what properties of an object in a scene have been acquired. Thus, in order to study the process of meaning-making process, we still need some kind of referential framework, to infer the ideas and thoughts to which these fixations and scanpaths correspond. Holsanova (2001) uses a dynamic, sequential method, and combines “eye movement data and verbal description data” as two windows to the mind. The eye movement data allow us to follow what image elements have been attended to, when and for how long (Figure 2). The verbal description data allow us to come closer to how the image has actually been perceived and experienced by the viewers. If we want to investigate the process of image viewing and image description in detail, we need to synchronize visual and verbal behavior. For this purpose, Holsanova (2001) developed a multimodal score sheet (see Figure 3). This timecoded score enables us to follow and compare the content of the attentional spotlight and extract clusters in the visual and verbal flow. With its help, we can examine the relationship between what is looked at and what is described verbally at a particular point in time. In result, we can witness a dynamic process of stepwise specification, evaluation, interpretation and reconceptualization of picture elements and of the picture as a whole. Informants started by looking at scene-inherent objects, units and gestalts. As their image viewing progressed, they tended to create mental units independently of the concrete image elements. They made large saccades, picking up information from different locations to support concepts that were distributed across the image. With the increasing cognitive involvement, observers and describers tended to return to certain areas, changed their perspective and reformulated or re-categorized the scene. Their perception of the image changed over time. Active mental groupings were created on the basis of similar traits, of symmetry and common activity. The process of mental zooming in and out could be documented, whereby concrete objects were re-fixated and viewed on another level of specificity or with another concept in mind (Holsanova 2001, 2008, 2011).

In the eye of the beholder

337

3 Why does our perception of visuals differ? There are a number of perceptual and cognitive mechanisms that all viewers have in common. At the same time, not all viewers see the same image content. This is due to the fact that we are not passive recipients. Visual perception is active, exploratory, creative and highly selective. Viewers filter what they perceive, select certain aspects of it and fill in other aspects on the basis of their expectations, goals, knowledge and expertise. The question of inter-individual differences has, however, so far not been addressed by researchers in visual communication. In the following, we will first briefly mention some of the common underlying perceptual and cognitive mechanisms (3.1) and then focus on a number of aspects that contribute to the individual variation in perception and interpretation of visuals (3.2).

3.1 Perceptual and cognitive mechanisms that we have in common First, let us mention “object and spatial vision”. There are two important cortical areas in the visual system with complementary functions for processing of visual information. These areas are called “what” pathway (responsible for object vision) and “where” pathway (responsible for spatial vision). The “what” pathway – in the ventral stream – is primarily concerned with information about the identity of objects in the scene, their form, color and shape, while the “where” pathway – in the dorsal stream– is concerned with information about spatial location, orientation and movement (Ungerleider and Mishkin 1982). Another important mechanism concerns “visual attention”. According to our present needs, interests and goals, we concentrate our attention on essential aspects of the visual scene, in a piecemeal manner. We focus at small parts, one at a time. The human ability to focus attention on a smaller part of the visual field has been discussed in the literature and likened to a spotlight (Posner 1980; Theeuwes, 1993) or to a zoom-lens (Findlay and Walker 1998). As Theeuwes (1993: 95) states: “[…] the locus of directed attention in visual space is thought of as having greater illumination than areas to which attention is not directed or areas from which attention has been removed”. According to Mozer and Sitton (1998: 369) “the current consensus is that the spotlight of attention turns off at one location and then on at another”. The explanations for such a spotlight differ, though. Traditionally, attention is viewed as a limited mental resource that constrains cognitive processing of (visual) information. Due to the limited capacity of visual attention, we can only focus on one or a few elements at a time (Treisman, Wolfe). In an alternative view, the concept is viewed in terms of the functional requirements of the task at hand (Allport 1989). In this latter view, attention is understood

338

Jana Holsanova

as preparation for action. Due to this explanation (based on perception–action integration), we select those parts that are relevant for our task at hand. Furthermore, according to current theories, perception of visuals is determined by two important mechanisms (or two different aspects of how our minds attend to visuals): “bottom-up” (image properties such as form, color, luminance, contrast, edge density and motion) and “top-down mechanism” (viewers’ personal characteristics such as interests, attitudes, goals and motives, prior knowledge, experiences, individual preferences and emotions) (Boeriis and Holsanova 2012). This issue is problematic, however, since fixated regions (for example, faces) often contain both low-level factors of visual saliency and high-level factors of potential semantic importance. The influence and interaction between bottom-up and topdown factors is the subject of a current debate in the field (Nyström and Holmqvist 2008; Henderson and Ferreira, 2004; Henderson et al. 2006; Foulsham and Underwood 2008; Harding and Bloj 2010). Also, as we will see later, this common mechanism contributes to inter-individual variation. Even principles of “constancy” and “perceptual organization” are important for perception of visuals. The human visual system extracts and groups similar features and separates dissimilar ones, segments the scene into coherent patterns, identifies objects, distinguishes between Figure and ground, perceives object properties and parts and recognizes perceptual organization (Palmer 1999). The principle of constancy says that the perception of an object or quality as constant under changing conditions (Wertheimer). Our ability to perceive Figures and meaningful wholes instead of collections of lines have been described in terms of “gestalt principles” of (a) proximity, (b) similarity, (c) closure, (d) continuation, and (e) symmetry (Köhler 1947). For instance, the proximity principle implies that features and objects placed close to each other appear as groups rather than a random cluster. The similarity principle means that there is a tendency for elements of the same shape, brightness or color to be seen as belonging together. The principle of closure implies that elements tend to be grouped together if they are parts of a closed Figure. The continuation principle means that units that are aligned appear as integrated perceptual wholes. Finally, the symmetry principle implies that regions bounded by symmetrical borders tend to be perceived as coherent Figures (cf. examples in Figure 4).

(a)

(b)

(c)

Fig. 4: The images illustrate the gestalt principles of continuity, proximity and similarity.

In the eye of the beholder

339

Fig. 5: Encoding and recall of a complex visual scene. Left: Inspecting a complex image. Right: Orally describing the image from memory while looking at a blank screen. The effect was strong showing significance both for local correspondence coding (74.8 percent correct eye movements, p = 0,002) and for global correspondence coding (54.9 percent correct eye movements, p = 0,005). From Johansson et al. 2006.

Fig. 6: Spoken scene description. Left: Listening to a scene description while looking at a blank screen. Right: Orally retelling the scene while looking at a blank screen. The subjects’ spatial pattern of eye movements was highly consistent with the original spatial arrangement. In the retelling phase, more than half of all mentioned referents to objects had correct eye movements, according to the conservative global correspondence criteria (55.2 percent correct eye movements, p = 0.004). According to the local correspondence criteria, we found 74.8 percent correct eye movements (p = 0.001). From Johansson et al. 2006.

Last but not least, let us mention “mental imagery”. Mental imagery refers to the human ability to see things with our mind’s eye and is apparent in a wide range of everyday situations. We “see” images in our minds when we recall episodes from our past, when we solve problems or when read poetry. This phenomenon has historically been very hard to study but with present day eye tracking technology a new window to understanding our mind has opened. Current research has demonstrated that mental imagery is accompanied by spontaneous eye movements and that these eye movements closely reflect the content and spatial layout of the imagined scene (Johansson et al. 2006). Johansson et al. (2006: 2012) report results from a number of eye tracking studies, viewers either inspected a complex picture (Figure 5) or looked at a blank screen while listening to a verbal scene description (Figure 6). Afterwards, while looking at a blank screen, they recalled the picture by orally describing it and

340

Jana Holsanova

recalled the scene description by orally retelling it. Results from this study revealed that participants to a very high degree executed eye movements to appropriate spatial locations while describing the picture from memory, while listening to the spoken scene description (that was never seen in the first place) and while retelling it from memory. The eye movement pattern reflected the viewers’ mental images. In their mind’s eye, the viewers ‘saw’ the scene as if it was in front of them and ‘painted’ it with their eyes on the blank screen. The results also indicated that the effect was equally strong during recall, irrespective of whether the original elicitation was spoken or visual. Furthermore, the effect was also equally strong during encoding and recall of the verbal scene description. The explanations to why these eye movements occur vary and have currently been the target of a hot topic debate. Is seeing something with the “mind’s eye” similar to retrieving a static photograph with a given structure? Do eye movements always become recruited in a specific and predictable manner? Or are they instead driven by different tasks, goals and individual preferences as they would be during active visual perception? It is still a puzzle which underlying cognitive mechanisms mediate eye movements during mental imagery. Nevertheless, mental imagery is an important part of our thinking and empirical studies of mental imagery are interesting since for applications in the areas of design, problem– solving and expert training.

3.2 What makes a difference in visual perception Despite the above-mentioned common perceptual and cognitive mechanisms, empirical studies show that we can perceive and interpret images differently. This is because of the viewer characteristics and the individual and cultural variation. Following aspects can play a role: the task or goal of the visual examination, previous knowledge and expertise, expectations, emotions and attitudes. Apart from viewer characteristics, even the context in which images are displayed, perceived and interpreted plays a role.

3.2.1 Task, purpose or goal influence visual perception One reason why viewers do not perceive visuals in the same way despite the fact that they are looking at the same image and possess the same perceptual and cognitive mechanisms underlying visual inspection is that their exploration is guided by a certain task or goal. Seminal studies of Yarbus (1967) demonstrate the role of the task and its influence on visual exploration behavior. Viewers who looked at Repin’s painting The unexpected visitor showing a family gathered in a living room and a person entering the room were given various instructions: “estimate the material circumstances

In the eye of the beholder

341

Fig. 7: Free viewing by four persons. The motive comes from http://upload.wikimedia.org/ wikipedia/commons/3/3c/Botticelli-primavera.jpg.

of the family shown in the picture”, “give the ages of the people shown in the picture”, “surmise what the family was doing before the arrival of the unexpected visitor”, “remember clothes worn by the people in the picture”, “remember position of the people and objects in the room”, “estimate how long the unexpected visitor had been away from the family”. In result, these instructions lead to different eye movement patterns. For instance, in response to the instruction “estimate the material circumstances of the family shown in the picture”, the observer paid particular attention to the women’s clothing and the furniture, whereas in response to the instruction “give the ages of the people shown in the picture”, all attention was concentrated on their faces (Yarbus 1967: 192). The task was thus guiding the viewer’s visual exploration of the image and the eye movement patterns showed which pieces of information had been considered relevant for the specific task and were therefore extracted by the viewers. A replication of a task effect can be seen in Figure 7–8. One and the same image is inspected during free viewing and with the instruction “What season of the year?” In both cases, the data from the inspection of the image are presented as a heat map. A heat map is a graphical representation of data where the individual values are represented as colors displaying areas of the image that have most frequently been fixated by viewers (in case of the actual viewers) or have been expected to attract the gaze (in the case of saliency algorithms). The warmer the color (yellow, orange, red), the more visual attention the particular area attracts.

342

Jana Holsanova

Fig. 8: Viewing by four persons with the instruction: What season of the year? The motive comes from http://upload.wikimedia.org/wikipedia/commons/3/3c/Botticelli-primavera.jpg.

Whereas most visual attention is devoted to the faces of the figures and to the gaze direction between the figures during free viewing, the task changes viewing behavior. Observers look at the clothing, at the trees full of fruits, at the soil and the leaves in order to find relevant information to answer the question. Castelhano et al. 2009 investigated how task instruction influences eye movement control. Twenty participants viewed color photographs of natural scenes under two instruction sets: visual search and memorization. The study demonstrates that viewing task biases the selection of scene regions and influences the viewing patterns. Recent research on eye movements in natural behavior has confirmed the importance of the task for the deployment of gaze (Hayhoe 2004; Hayhoe and Ballard 2005; Rothkopf et al. 2007). Human gaze is directed toward regions of the visual scene that are determined primarily by the task requirements. Kaakinen et al. (2011) investigate the role of psychological perspective taking and discuss the role of bottom-up versus top-down factors in image viewing. Thirty-three participants viewed photographs from either a potential homebuyer’s or a burglar’s perspective, or in preparation for a memory test. The results show that perspective had large effects on image viewing: perspective-relevant areas of the image were inspected very early during visual examination, for longer times and more frequently than perspective-irrelevant parts. Visually salient areas attracted the first fixation and more and longer visits than non-salient areas. How-

In the eye of the beholder

343

ever, when the viewers were asked later on to recall the image and recognize various elements of the photograph, their performance was better for perspectiverelevant than for perspective-irrelevant scene objects. The authors conclude that the role of bottom-up versus top-down factors varies as a function of viewing task and the time-course of scene processing. Simola et al. (2011) studied the role of saliency and higher cognitive processes in an applied study on the impact of salient advertisements on reading and attention on web pages. Since human vision is sensitive to motion, it was interesting to find out whether animation and onset of advertisements on websites attracts visual attention and disrupts reading. The authors found that the degree of animation affects attention toward the ads during reading. Dwell times on ads were the longest when the ad above the text was static and ad to the right was animated. They also showed that ads that appear abruptly after a random time interval attract attention of the viewers. Also, ads appearing close to the text region captured overt attention. Salient ads attracted visual attention and disrupted reading. The fact that during free browsing, ads were viewed more frequently and for longer time than during reading, can be interpreted in perceptual and cognitive terms as a competition between saliency and relevance. When the users are involved in reading, the low-level saliency is overridden by higher cognitive processes and does not attract so much attention. To conclude, the way viewers perceive, understand and remember the content of visuals is guided partly by visually salient (bottom-up) aspects and partly by semantically relevant (top-down) aspects (Holsanova 2013).

3.2.2 Gaze allocation models differ from the way human viewers perceive images and visual scenes If it were so that our gaze behavior is entirely guided by bottom–up processes then the result of a picture viewing by a human viewer would be the same as those generated by a gaze allocation model or algorithm. By using data from our eye tracking study (Johansson et al. 2012), we collected sessions from 21 viewers and compared their inspection of the image with the widely recognized gaze allocation saliency model proposed by Itti and Koch (2000). The bottom-up saliency assumption is based on the hypothesis that certain features of the visual scene inherently attract gaze. The saliency model extracts these low-level features (differences in intensity, edges, contrasts, colors and orientation), identifies the most important information in the visual input and predicts which areas of the image will get most attention.

344

Jana Holsanova

Fig. 9: Heatmap over image viewing (21 informants). The viewers have looked at the picture for 5 seconds.

Figures 9–11 below present authentic results from an image viewing session where 21 participants inspected the image for 5, 10 and 20 seconds. Figure 12 presents data generated by a saliency algorithm (Itti and Koch 2000) based on orientation, intensity and color information. In both cases, the data from the inspection of the image are presented as a heat map. Let us look at the heat maps in more detail in order to compare gaze distributions of actual viewers to distributions generated by a specific saliency model,

In the eye of the beholder

Fig. 11: Heatmap over image viewing (21 informants). The same viewers have looked at the picture for 20 seconds.

Fig. 10: Heatmap over image viewing (21 informants). The same viewers have looked at the picture for 10 seconds.

345

346

Jana Holsanova

predicting where in that scene viewers will look. During the first five seconds, viewers inspected the faces of the main figure (Petsson) on the left and on the right, his cat in the foreground and one of the birds in the tree. After 10 to 20 seconds, viewers’ attention was drawn to all three birds in the tree, the other Petsson figures, all cats, flowers on the left and cows in the background. This is quite intuitive because viewers are interested in living beings (people and animals) and look for these meaningful categories in the images. We can also note that gaze distributions show high similarity across viewers. In contrast, the result from a saliency map algorithm is more diffuse. Some of the objects attended to by human viewers receive some attention even here but since saliency algorithms are based on formal features (such as contrast, color and edges), some of the most attended areas ‘give no sense’ to a human viewer (for example the red area on the left hand side or in the top of the tree). The human viewers and the algorithm behave differently during inspection of the image. It seems thus that bottom-up saliency is not a good predictor for the direction of human gaze in image viewing (cf. also Rotkopf et al. 2007).

3.2.3 Domain knowledge and expertise influence image perception There is increasing evidence that expert knowledge influences perception and guides the interpretation of images. Consider the image in Figure 13. It can be admired for its aesthetic qualities by a biologically not informed viewer, if it is

Fig. 12: Heatmap generated by using Itti and Koch (2000) algorithm.

In the eye of the beholder

347

framed and hangs on the wall in a gallery. But for the biologically knowledgeable viewer, it is primarily a picture of diatoms found in lakes and oceans.

Fig. 13: Figure 13: Art or diatoms?

Previous research has shown that visual attention allocation is often influenced by pre-existing knowledge and expertise (van Gog et al. 2005). Experts use different visual exploration strategies than novices. This has consequences for meaning–making and can be applied for the design of multimodal messages (Chandler and Sweller 1991; Holsanova and Nord 2010; Mayer 2005). Studies have been conducted with artistically trained versus untrained participants looking at pictures and art (Antes and Kristjanson 1991; Vogt and Magnussen 2007), experts and novices playing chess (Charness et al. 2001; Reingold et al. 2001), domain specialists looking at photographs (Humphrey and Underwood 2009), diagrams (Lowe 1996), as well as experts and laymen studying mammograms (Nodine et al. 1996). In educational context, researchers often speak of image legibility and visual literacy (Pettersson 2008). For certain types of images (for example tables, graphs, charts and maps), it is necessary to learn the rules of how to extract information in order to understand the content. For an effective and correct interpretation of these logical images, the viewer must be able to connect what she sees with what she knows, ie. perceptual and conceptual knowledge. It has been shown that image understanding and learning from images is positively affected if the reader has prior conceptual knowledge in the specific field of expertise. For example, students with deeper expertise can more easily direct attention to the relevant information in an image. They are also not as much affected by high visual complexity compared to students with superficial expertise. Both conceptual knowledge within a domain and perceptual skills are needed to become an expert (Jarodzka and Dewhurst 2012).

348

Jana Holsanova

3.2.4 Cultural background may influence perception of visuals Recent studies suggest that eye movement patterns during scene viewing differ for people from different cultural backgrounds. These differences in scene viewing are mainly associated with differences in the prioritization of information (Chua et al. 2005). Masuda and Nisbett (2001) found that Japanese made more statements about contextual features compared to Americans, and that the recall was more influenced by the background context when viewing complex scenes. It has been claimed that Westerners attend more to focal objects, whereas East Asians attend more to contextual information (Chua et al. 2005). In a current eye tracking study, Rayner et al. (2009) examined whether there are cultural differences in how quickly eye movements are drawn to highly unusual aspects of a scene. American and Chinese viewers examined photographic scenes while performing a preference-rating task. For each scene, participants were presented with either a normal or an unusual/weird version. Even though there were differences between the normal and weird versions of the scenes, there was no evidence of any cultural differences while viewing either scene type. The present study, along with other recent reports, raises doubts about the notion that cultural differences can influence oculomotor control in scene perception. To sum up, cross-cultural differences in visual perception and cognition are still understudied and the results of the current studies are not univocal. Also, there is no clear-cut between individual and cultural differences. Thus, more empirical studies are needed to investigate the impact of cultural background for the perception of visuals.

3.2.5 Emotional aspects influence perception of visuals It has been shown that viewers have an ability to rapidly detect emotive stimuli (Humphrey et al. 2012). However, there is still a need for study of affective (emotional) responses to visuals. It is often stated that emotive images are powerful but these statements have not been proved empirically. The problem is that many emotional objects in a scene are also highly visually salient. Humphrey et al. raise in their article “Salience of the lambs” the issue of how dependent the effects of emotionality are on visual saliency and whether the combination of an emotional and visually salient object attracts more attention. It is a methodological challenge how to code and measure emotional aspects of visuals. In order to investigate this issue of visual communication, it is necessary to apply novel methods and combine expertise from several disciplines. Müller et al. (2012) use an interdisciplinary combination of three methods – iconology, eye tracking and psychophysiological reaction measurements – to analyze examples from press photography. Emotional reactions to visuals are included in their Visual

In the eye of the beholder

349

Communication Process Model. The proposed model integrates three processes: visual perception and attention, meaning attribution and emotional reactions to mass-mediated visuals.

3.2.6 Context influences perception and interpretation of visuals If we discuss the impact of contextual factors on perception of visuals we should mention pictorial, linguistic and physical/situational context (Holsanova 1996). Even the above-mentioned cultural context should be included. First, it makes a difference for the interpretation of visuals whether an image appears alone or as a part of a series of images. It’s meaning changes. Also, the composition of a visual plays a role for how it is perceived (Machin 2009). The concept of “visual crowding” is important in his context (Whitney and Levi 2011). If a complex scene contains visual clutter, viewers are unable to recognize objects. Visual clutter sets a fundamental limit on conscious visual perception and object recognition. Second, whenever an image is embedded in a linguistic context or combined with a label, headline, caption or annotation, the perception of the image is guided by this linguistic context. It serves partly as a kind of instruction for the reader how to proceed in scanning the image in order to find relevant information. It also suggests how the visual and verbal parts should be integrated in order to create meaning. It happens that one and the same image is used to illustrate different texts, arguing in an opposite direction. Third, physical context contributes to the process of meaning–making. An image can be interpreted in a different way when placed in a department store, in an art gallery, in Yellow pages, etc. In a classical categorization experiment by Labov (1973), participants were shown pictures of various containers and were asked to categorize the objects. In some instances, either flowers or potatoes were placed in the containers. When potatoes were present, the object was more likely to be categorised as a bowl, whereas it was more likely to be categorized as a vase when flowers were present.

3.2.7 Design influences how we perceive visuals Yet another important factor that influences perception, meaning–making and understanding of visuals is how the information is presented. In particular for explanatory, instructional and educational materials it is crucial that the users interact with the materials in an intended way and extract relevant information that is important for understanding. In the area of multimedia learning, researchers have formulated principles reducing cognitive load and promoting multimodal integration (Chandler and

350

Jana Holsanova

Sweller 1991; Mayer 2005). In the rhetorical tradition, procedures for multimodal document design have been suggested in connection to document genre (Bateman 2008). So far, only few studies have been conducted to test these principles in visual communication. Holsanova et al. (2009) present a naturalistic study on newspaper reading and study the effects of design principles on perception of complex visuals. A pair of information graphics (radial vs. serial format) have been designed and embedded into a newspaper prototype. They contained the same text, the same illustrations, but had a different overall layout. The radial format allowed for a variety of different reading orders whereas the serial layout was spatially structured and conceptually pre-processed in order to guide the readers. Two authentic-looking newspapers were used consisting of 15 spreads with built-in information graphics that were subject to manipulation. Informants read one of the newspapers at their own pace. Holsanova et al. (2009) proposed the “dual scripting principle”: “people will read a complex message more deeply when attentional guidance is provided both through the spatial layout (supporting optimal navigation) and through a conceptual organization of the contents (supporting optimal semantic integration)” (Holsanova et al. 2009: 1217). This principle involves not only providing layout cues but also restructuring the message itself in order to assist the reader and to optimize the proposed meaning-making. On the level of message surface, attentional guidance is provided by a suggested reading path. On the level of the message content, the material is rhetorically organized and conceptually preprocessed (from known information, over expert information to practical information). A coherent, conceptually pre-processed spatial format can guide the readers and contribute to easier processing of the materials. The data clearly show that different spatial layouts have a significant effect on readers’ behavior. As we can see in Figure 14, the radial layout, which allows for a variety of different reading orders, leads to initial reader interest followed by short reading times indicating a later disinterest in the content. Since the readers have several decisions to make – to choose the entry point, to decide about the reading path, to find relevant pieces of information, to create a connection between them and to integrated them mentally – the material might be judged as difficult to grasp and not worth deeper processing. This further suggests that the freedom of choosing entry points and reading paths is not an optimal strategy for attracting readers to stay with a complex material and get a deeper understanding of its contents. The serial layout that is spatially structured and conceptually pre-processed enhances reading and text–picture integration. Due to dual attentional guidance, the format supports navigation and semantic processing of the contents. This not only leads to catching the readers’ attention but also to sustaining their interest in the material. The readers followed the suggested reading path and the serial graphic was read more than twice as much as the radial graphic – in spite of the fact that the content and the position in the newspaper were exactly the same.

In the eye of the beholder

351

(a)

(b) Fig. 14: Typical example of reading of radial information graphics (a) and serial information graphics (b). From Holsanova et al. 2009.

Moreover, integrative saccades were almost twice as common in the serial format compared to the radial format. This is as a strong indication that readers prefer to be guided through complex information graphics. To conclude, different designs of multimodal news graphics lead to significant differences in user behavior, semantic integration of various modes, and comprehension of the complex material.

352

Jana Holsanova

4 Summary and conclusion Recipients play an active role in the interaction with visuals and complex multimodal messages. However, their dynamic interaction with the visuals has not been studied in detail. There is a need for empirical analyses of visuality from a recipient perspective (Holsanova 2013). The fact that not all viewers perceive the same image content – despite common perceptual and cognitive mechanisms such as object and spatial vision, constancy and perceptual organization, gestalt principles, limited capacity of visual attention – is due to the interplay between bottom-up and top-down processes. Apart from bottom-up processes where viewers’ attention is drawn to contrasts, colors, edges and movement, there are also top-down factors such as viewers’ expectations, goals, background knowledge and associations, contributing to inter-individual differences in visual perception. Furthermore, context in which visuals are embedded or diplayed plays an important role in perception and interpretation of visuals. When we use a an interdisciplinary framework, integrate a wide range of theoretical and methodological approaches to visual communication and multimodality and use a triangulation of methods we can investigate the following issues: dynamic aspects of image perception, attentional and cognitive processing underlying interaction with visuals, the role of domain knowledge and expertise, the role of the goal or task, the emotional impact of visuals and the role of context for perception and interpretation of visuals. In future research, more scholarly attention should therefore be devoted to individual perception and interpretation of visuals by using a multimodal, interdisciplinary framework and integrated methods.

References Allport, Alan. 1989. Visual Attention. In Michael Posner (ed.), Foundations of Cognitive Science, 631–682. Cambridge: MIT Press. Antes, James and Arlinda F. Kristjanson. 1991. Discriminating Artists from Nonartists by their Eyefixation Patterns. Perceptual and Motor Skills 73: 893–894. Barnhurst, Kevin. 1994. Seeing the Newspaper. New York: St. Martin’s Press. Bateman, John A. 2008. Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. London: Palgrave Macmillan. Boeriis, Morten and Jana Holsanova. 2012. Tracking Visual Segmentation: Connecting Semiotic and Cognitive Perspectives. Visual communication 11(3): 259–281. Boland, Julie, Hannah Chua, and Richard Nisbett. 2008. How We See It: Culturally different eye movement patterns over visual scenes. In: Keith Rayner, Deli Shen, Xuejun Bai, Guoli Yan (eds.), Cognitive and cultural influences on eye movements, 363–378. Tianjin: Tianjin People’s Press/Psychology Press. Bucher, Hans-Juergen. 2007. Textdesign und Multimodalität: Zur Semantik und Pragmatik medialer Gestaltungsformen. In Kersten Sven Roth and Jürgen Spitzmüller (eds.), Textdesign und Textwirkung in der massenmedialen Kommunikation, 35–64. Köln: UVK.

In the eye of the beholder

353

Buswell, Guy thomas. 1935. How People Look at Pictures: A study of the psychology of perception in art. Chicago: University of Chicago Press. Castelhano, Monica, Michael Mack and John Henderson. 2009. Viewing Task Influences Eye Movement Control During Active Scene Perception. Journal of Vision 9(3): 1–15. Chandler, Paul and John Sweller. 1991. Cognitive Load Theory and the Format of Instruction. Cognition and Instruction 8(4): 293–332. Charness, Neil, Eyal M. Reingold, Marc Pomplun and Dave Stampe. 2001. The Perceptual Aspect of Skilled Performance in Chess: Evidence from eye movements. Memory and Cognition 29(8): 1146–1152. Chua, Hannah, Julie Boland and Richard Nisbett. 2005. Cultural Variation in Eye Movements During Scene Perception. Proceedings of the National Academy of Sciences 102: 12629– 12633. Findlay, John and Robin Walker. 1999. A Model of Saccadic Eye Movement Generation Based on Parallel Processing and Competitive Inhibition. Behavioral and Brain Sciences 22: 661–674. Foulsham, Tom and Geoffrey Underwood. 2007. How does the Purpose of Inspection Influence the Potency of Visual Salience in Scene Perception? Perception 36(8): 1123–38. Foulsham, Tom and Geoffrey Underwood. 2009. Does Conspicuity Enhance Distraction? Saliency and eye landing position when searching for objects. The Quarterly Journal of Experimental Psychology 62(6): 1088–1098. Griffin, Michael. 1991. Defining Visual Communication for a Multi-media World. Journalism Educator 46(1): 9–15. Griffin, Michael. 1992. Looking at TV News: Strategies for research. Communication 13(2): 121– 141. Harding, Glen and Marina Bloj. 2010. Real and Predicted Influence of Image Manipulations on Eye Movements during Scene Recognition. Journal of Vision 10(2): 1–17. Hayhoe, Mary and Dana Ballard. 2005. Eye Movements in Natural Behavior. TRENDS in Cognitive Sciences 9(4): 188–193. Hayhoe, Mary. 2004. Advances in Relating Eye Movements and Cognition. Infancy 6(2): 267–274. Henderson, John M. 2007. Regarding Scenes. Current Directions in Psychological Science 16(4): 219–222. Henderson, John M. and Fernanda Ferreira (eds.). 2004. The Integration of Language, Vision, and Action: Eye movements and the visual world. New York: Psychology Press. Holmqvist, Kenneth, Jana Holsanova, Maria Barthelson and Daniel Lundqvist. 2003. Reading or Scanning? A study of newspaper and net paper reading. In Jukka Hyönä, Ralph Radach, and Heiner Deubel (eds.), The Mind's Eye: Cognitive and applied aspects of eye movement research, 657–670. Amsterdam: Elsevier Science. Holsanova, Jana. 1996. När säger en bild mer än tusen ord? Om samspelet mellan text och bild i gränssnitt. [When does a picture say more that a thousand words?] In Jonas Löwgren (ed.), Teman i människa-datorinteraktion, 65–76. Lund: Institution för Informatik, Lunds universitet. Holsanova, Jana. 1999. Olika perspektiv pa språk, bild och deras samspel. Metodologiska reflexioner. [Different perspectives on multimodality. Methodological considerations] In Inger Haskå and Carin Sandqvist (eds.), Alla tiders språk. Lundastudier i nordisk språkvetenskap A 55, 117–126. Lund: Lund University Press. Holsanova, Jana. 2001. Picture Viewing and Picture Description: Two Windows on the Mind. Doctoral dissertation. Lund University Cognitive Studies 83: Lund. Holsanova, Jana. 2008. Discourse, Vision, and Cognition. Amsterdam/Philadelphia: Benjamins. Holsanova, Jana. 2010. Myter och sanningar om läsning. Om samspelet mellan språk och bild i olika medier. [Myths and facts about reading. On the interplay between language and pictures in various media] Stockholm: Norstedts.

354

Jana Holsanova

Holsanova, Jana. 2011. How we Focus Attention in Picture Viewing, Picture Description, and During Mental Imagery. In: Klaus Sachs-Hombach and Rainer Totzke (eds.), Bilder, Sehen, Denken, 291–313. Köln: Herbert von Halem Verlag. Holsanova, Jana (ed.). 2012. Methodologies for Multimodal Research. Visual Communication 11(3). Holsanova, Jana. 2013. Reception of Multimodality: Applying eye-tracking methodology in multimodal research. In Carey Jewitt (ed.), Routledge Handbook of Multimodal Analysis. Second edition. Chapter 20, 285–296. London: Routledge. Holsanova, Jana and Andreas Nord. 2010. Multimodal Design: Media structures media principles and users meaning-making in printed and digital media. In Hans-Jürgen Bucher, Thomas Gloning, Katrin Lehnen (eds.), Neue Medie – neue Formate: Ausdifferenzierung und Konvergenz in der Medienkommunikation, 81–103. Frankfurt/New York: Campus. Holsanova, Jana, Henrik Rahm and Kenneth Holmqvist. 2006. Entry Points and Reading Paths on Newspaper Spreads: Comparing a semiotic analysis with eye-tracking measurements. Visual Communication 5(1): 65–93. Holsanova, Jana, Nils Holmberg and Kenneth Holmqvist. 2009. Reading Information Graphics: The role of spatial contiguity and dual attentional guidance. Applied Cognitive Psychology 23: 1215–1226. Humphrey, Katherine, Geoffrey Underwood and Tony Lambert. 2012. Salience of the Lambs: A test of the saliency map hypothesis with pictures of emotive objects. Journal of Vision 12(1): 1–15. Humphrey, Katherine and Geoffrey Underwood. 2009. Domain Knowledge Moderates the Influence of Visual Saliency in Scene Recognition. Br J Psychol. 100(2): 377–98. Itti, Laurent and Christof Koch. 2000. A Saliency-Based Search Mechanism for Overt and Covert Shifts of Visual Attention. Vision Research 40: 1489–1506. Johansson, Roger, Jana Holsanova, Richard Dewhurst and Kenneth Holmqvist. 2012. Eye Movements During Scene Recollection have a Functional Role, but they are not Reinstatements of those Produced During Encoding. Journal of Experimental Psychology: Human Perception and Performance 38(5): 1289–1314. Johansson, Roger, Jana Holsanova and Kenneth Holmqvist. 2006. Pictures and Spoken Descriptions Elicit Similar Eye Movements During Mental Imagery, both in Light and in Complete Darkness. Cognitive Science 30(6): 1053–1079. Kaakinen, Johanna, Jukka Hyönä and Minna Viljanen. 2011. Influence of a Psychological Perspective on Scene Viewing and Memory for Scenes. Quarterly Journal of Experimental Psychology 64(7): 1372–87. Kress, Gunther and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. London: Routledge. Köhler, Wolfgang. 1947. Gestalt Psychology: An Introduction to New Concepts in Modern Psychology. New York: Liveright Publishing Corporation. Labov, William. 1973. The Boundaries of Words and their Meanings. In Charles-James Bailey and Roger Shuy (eds.), New Ways of Analyzing Variation in English. Washington: Georgetown University Press. Locher, Paul, Elizabeth A. Krupinski, Claudia Mello-Thoms and Calvin F. Nodine. 2007. Visual Interest in Pictorial Art during an Aesthetic Experience. Spatial Vision 21(1–2): 55–77. Machin, David. 2009. Multimodality and Theories of the Visual. In: Carey Jewitt, (ed.), Routledge Handbook of Multimodal Analysis, 181–190. London: Routledge. Mayer, Richard E. 2005. Principles for Managing Essential Processing in Multimedia Learning: Coherence, signaling, redundancy, spatial contiguity and temporal contiguity principles. In Richard E. Mayer, (ed.), Cambridge handbook of multimedia learning, 183–200. New York: Cambridge University Press.

In the eye of the beholder

355

Mitchell, W. T. J. 1986. Iconology: Image, Text, Ideology. Chicago/London: The University of Chicago Press. Mozer, Michael and Mark Sitton. 1998. Computational Modelling of Spatial Attention. In Harold Pashler (ed.), Attention, 341–393. London: UCL Press. Müller, Marion G., Arvid Kappas and Bettina Olk. 2012. Perceiving Press Photography: A New Integrative Model, Combining Iconology with Psychophysiological and Eye tracking Methods. In Jana Holsanova (ed.), Visual communication [Special issue: Methodologies for multimodal research] 11(3): 307–328. Nodine, Calvin F., Harold Kundel, Sherri C. Lauver and Lawrence C.Toto. 1996. Nature of Expertise in Searching Mammograms for Breast Masses. Academic Radiology 3(12): 1000–1006. Nordqvist, Sven. 1990. Kackel i trädgårdslandet. Opal. Nyström, Marcus and Kenneth Holmqvist. 2008. Semantic Override of Low-level Features in Image Viewing – Both Initially and Overall, Journal of Eye Movement Research 2(2): 1–11. O’Halloran, Kate, Chua and Marissa Podlasov. 2012. Interactive Software for Multimodal Analysis. In: Jana Holsanova (ed.), Visual communication [Special issue: Methodologies for multimodal research] 11(3): 363–381. Palmer, Stephen E. 1999. Vision Science: Photons to Phenomenology. Cambridge: MIT Press. Posner, Michael. 1980. Orienting of Attention. Quarterly Journal of Experimental Psychology 32(1): 3–25. Rayner, Keith, Monica Castelhano, Jinmian Yang. 2009. Eye movements when Looking at Unusual/Weird Scenes: Are there cultural differences? Journal of Experimental Psychology: Learning, Memory and Cognition 35(1): 254–259. Rothkopf, Constantin A., Dana H. Ballard and Mary M. Hayhoe. 2007. Task and Context Determine Where you Look. Journal of Vision 7(14): 1–20. Simola, Jaana, Jarmo Kuisma, Anssi Öörni, Liisa Uusitalo and Jukka Hyönä. 2011. The Impact of Salient Advertisements on Reading and Attention on Web Pages. Journal of Experimental Psychology: Applied 17(2): 174–190. Theeuwes, Jan. 1993. Visual Selective Attention: A theoretical analysis. Acta psychologica 83(2): 93–154. Theeuwes, Jan, Arthur F., Kramer, Sowon Hahn, and David E. Irwin. 1998. Our Eyes Do Not Always Go Where We Want Them To Go: Capture of the eyes by new objects. Psychological Science 9(5): 379–385. Ungerleider, Leslie and Mortimer Mishkin. 1982. Two Cortical Visual Systems. In David J. Ingle, Melvyn Goodale and Richard Mansfield (eds.), Analysis of Visual Behavior, 549–586. Cambridge: MIT Press. Van Gog, Tamara, Fred Paas, and Jeroen J. G. van Merriënboer. 2005. Uncovering ExpertiseRelated Differences in Troubleshooting Performance. Combining eye movement and concurrent verbal protocol data. Applied Cognitive Psychology 19: 205–221. Vogt, Stine and Magnussen, Svein. 2007. Expertise in Pictorial Perception: Eye-movement patterns and visual memory in artists and laymen. Perception 36(1): 91–100. Wartenberg, Constance and Holmqvist, Kenneth. 2005. Daily Newspaper Layout-Designers’ Predictions of Reader Behaviour – A case study. (Lund University Cognitive Studies 126.) Available at: http://www.lucs.lu.se/LUCS/126/LUCS.126.pdf. Whitney, David and Dennis M. Levi. 2011. Visual Crowding: A fundamental limit on conscious perception and object recognition. Trends in Cognitive Sciences 15(4): 160–168. Yarbus, Alfred L. 1967. Eye Movements and Vision [First Russian Edition 1965]. New York: Plenum Press.

H. Camilla Smith

15 Questioning bohemian myth in Weimar Berlin: Reinterpreting Jeanne Mammen and the artist function through her illustrations Der Maler und sein Modell ‘The Painter and his Model’ (1927) Abstract: This paper focuses on nine drawings entitled Der Maler und sein Modell [The Painter and his Model] (1927) created by the artist Jeanne Mammen (1890– 1976). Mammen’s images appeared in the popular magazine Die Woche alongside a narrative by Hermann Krehan (1890–1972). It argues that these illustrations mockingly aligned male sexuality with artistic creativity during a time in which Mammen was actively engaging with, and contributing to women’s artistic practices. Moreover, by exploring the corresponding sympathies of Krehan’s narrative, it interprets the artist function as social mediator, emphasizing that Mammen’s art should be reconsidered to be both social and Socialist. Consequently, it moves away from previous emphasis on Mammen as a social outsider, in line with her admiration of male nineteenth-century role models, in particular, Gustave Flaubert. Mammen’s collecting habits have also been read as indexical signs of her hermetic existence. The studio apartment, in which she lived for over fifty years, still houses her library along with wax votives, masks, paintings and clay figures. Considerations of Die Woche as part of this collection further helps reevaluate this studio apartment as a lived, porous space in the heart of west Berlin between which objects, friends and Mammen herself constantly moved.

Berlin, by comparison – how splendid! A city like Berlin is an ill-mannered, impertinent, intelligent scoundrel, […]. An artist here has no choice but to pay attention. Elsewhere he is permitted to stop up his ears and sink into wilful ignorance. Here this is not allowed.1 Robert Walser, ‘Berlin and the Artist’ (1910)

1 Introduction Since the 1980s, the contested notions of authorship have played an important role in informing what is now recognised as the “new art history”, which has I am grateful to the DAAD for their financial support. Special thanks to the Förderverein der JeanneMammen-Stiftung e. V., in particular Cornelia Pastelak-Price who enabled extensive studies at Mammen’s studio apartment and many valuable discussions.  All translations are the author’s own unless otherwise stated. Walser is translated here by Susan Bernofsky.

358

H. Camilla Smith

Fig. 1: Page one of Jeanne Mammen’s illustrations for Hermann Krehan’s Der Maler und sein Modell ‘The Painter and his Model’ (circa 1927), watercolour and pencil, original size unknown. Die Woche, Heft 31, 29 Jg., Juli 1927. (c) VG Bild-Kunst, Bonn 2013.

Questioning bohemian myth in Weimar Berlin

359

Fig. 2: Page two of Jeanne Mammen’s illustrations for Hermann Krehan’s Der Maler und sein Modell [‘The Painter and his Model’] (circa 1927), watercolour and pencil, original size unknown. Die Woche, Heft 31, 29 Jg., Juli 1927. (c) VG Bild-Kunst, Bonn 2013.

360

H. Camilla Smith

encouraged uses of sociology, politics and literary theory to aid interpretation of art in relation to both its social production and the viewing subject. Whilst such interpretations are important, this paper seeks to readdress the balance between artist biography and the social conditions of art by focusing on the German artist Johanna Gertrude Luise Mammen (1890–1976), (known as Jeanne), and a series of nine watercolour and pencil drawings entitled Der Maler und sein Modell ‘The Painter and his Model’ (1927), which Mammen produced for a narrative written by the artist and set designer Hermann Krehan (1890–1972) (Figures 1–4). Mammen is still best known in Anglophone scholarship for her images documenting social sections of the Weimar Republic (Noun 1994; Lütgens 1997a) and her work continues to form part of the subject of international exhibitions.2 However, despite having been produced during this period, Der Maler und sein Modell has never been examined by scholars. Although Mammen’s drawings are “merely” illustrative, they appeared in a magazine entitled Die Woche ‘The Weekly’, which placed great emphasis on visual representation, dedicating whole pages to sculptures and reproductions of work by contemporaries Max Liebermann, Fritz Klimsch and Arthur Kampf in order to compete with its main rival the Berliner Illustrirte Zeitung ‘Berlin’s Illustrated Newspaper’. The magazine’s circulation was substantial and the potential importance of Mammen’s series of drawings should therefore not be underestimated. The article appeared during 1927 (Krehan and Mammen 1927, 31: 31–34) less than a decade after Mammen and her sister, Marie Louise (known as Mimi), had moved into their own two-room studio apartment at 29 Kurfürstendamm in Berlin, where they both lived and worked. Through closer examination of Der Maler und sein Modell this paper argues that Mammen was mockingly aligning male sexuality with artistic creativity during a time in which she was actively engaging with, and contributing to women’s artistic practices. Moreover, by exploring the corresponding sympathies of Hermann Krehan’s narrative, it interprets the ‘artist function’3 (Foucault 1979: 141–160) as social mediator; thereby emphasizing that Mammen’s art can be interpreted as social and to a degree, Socialist. The significance of Mammen’s studio apartment, where she lived and worked for over fifty years (1921–1976), further establishes an understanding of the artist function. Many scholars still assume Mammen first moved here in 1919 or 1920. However, her name alongside her profession as “Malerin” ‘painter’ (singular) as

 Recent exhibitions such as ‘Straßen und Gesichter 1918 bis 1933’, Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, 2012 and ‘Gefühl ist Privatsache: Verismus und Neue Sachlichkeit’, Kupferstichkabinett, Berlin 2011.  By “artist function” I mean the perceived role of the artist as constructed by social and cultural discourses in line with what Michel Foucault has termed the “author-function”. In this paper the artist is understood as contributing and responding to such constructions, which are also shown as being shaped by both historical contingency and enduring myths.

Questioning bohemian myth in Weimar Berlin

361

Fig. 3: Page three of Jeanne Mammen’s illustrations for Hermann Krehan’s Der Maler und sein Modell [‘The Painter and his Model’] (circa 1927), watercolour and pencil, original size unknown. Die Woche, Heft 31, 29 Jg., Juli 1927. (c) VG Bild-Kunst, Bonn 2013.

362

H. Camilla Smith

Fig. 4: Page four of Jeanne Mammen’s illustrations for Hermann Krehan’s Der Maler und sein Modell [‘The Painter and his Model’] (circa 1927), watercolour and pencil, original size unknown. Die Woche, Heft 31, 29 Jg., Juli 1927. (c) VG Bild-Kunst, Bonn 2013.

Questioning bohemian myth in Weimar Berlin

363

living at no. 29 only first appears in Berlin public address books in 1921.4 The centrality of this space for Mammen and its important preservation today, supports art historical investigations into the biography of the artist by lending further potential meaning to her works, as well as allowing for wider understandings of the social conditions in which the artist lived and worked. Similar investigations have formed the basis of recent art historical writing by Michael Cole and Mary Pardo (2005) and Mary Jane Jacob (2010) and successful exhibitions curated by Giles Waterfield (2009). By examining Der Maler und sein Modell alongside Mammen’s letters (1946–1975) and collected objects found in her studio apartment, this paper challenges previous scholars’ interpretations of Mammen’s life-long identification with outsider-figures and argues for the social reconfiguration of her studio as part of this. This approach questions readings of the artist’s studio as a mythical space5 and demonstrates how Mammen’s collecting practices connect her and her studio space to the world around her. Consequently, where relevant, this paper suggests how Der Maler und sein Modell helps us understand the artist’s attitudes towards art and the artist function until her death in 1976. Although moving between the historic specificity of text/image produced in 1927 and contexts thereafter is potentially problematic, I do not deny that this timeframe produced unique sets of social conditions, which complicate interpretations of Mammen’s protracted isolation: the National Socialist dictatorship, WWII and the establishment of the Federal Republic of Germany. However, unlike previous interpretations of Mammen’s oeuvre as a series of corresponding stylistic and political “breaks”; moving from the social realism of the 1920s, through cubo-expressionism during the 1930s and towards lyrical-abstraction in later life, I stress the “continued” social engagement of her work through this exploration of the role of art and artist function.

2 Jeanne Mammen or the art of ‘disappearing’ (Lütgens 1997b: 10) Scholarly interpretations of Mammen’s life and the artist’s own comments have led scholars Annelie Lütgens (1991), Hildegard Reinhardt (2002) and more recently, Carolin Leistenschneider (2010), to conclude that Mammen lived a socially withdrawn life which was guided by her enduring interest in late nineteenth-century French cultivations of the creator as ascetic outsider. Born in Berlin, Mammen

 Moreover, the previous occupant of Mammen’s studio apartment, renowned photographer Karl Schenker, is still listed as living at number 29 in 1920. For entries Mammen and Schenker see the Berliner Adressbuch der Jahre 1799 bis 1943, Teil I, p. 1877, through http://adressbuch.zlb.de/ accessed July 2012.  I am thinking in particular here of interpretations forwarded by scholars such as Caroline Jones (1996) in her discussion of the significance of studios for post-war American artists.

364

H. Camilla Smith

moved with her family to Paris in 1900, where she and Mimi later began their training as artists (Reinhardt 1991: 82–83). The family were forced to flee to Germany in 1914 and arrived in Berlin in 1916. Mammen’s enduring love of French literature is evidenced through her extensive collection of books in her studio apartment.6 These scholars concur that Mammen’s particular admiration of Gustave Flaubert’s novel La tentation de Saint Antoine, ‘The Temptation of St. Anthony’ (1849, 1856 and 1872) directly influenced her early symbolist imagery and, more crucially, her self-critical position as an artist throughout her life.7 Having trained as an artist at the Parisian Académie Julian in 1906 and at the Académie Royale des Beaux-Arts de Bruxelles in 1908, Hildegard Reinhardt convincingly argues that Mammen knew the work of Félicien Rops and Jean Delville, whose imagery influenced her own series of drawings in 1910 depicting Flaubert’s tortured St. Anthony (Reinhardt 2002: 10–11). Leistenschneider further emphasizes Mammen’s knowledge and ownership of the philosophical writing of Arthur Schopenhauer and Friedrich Nietzsche, which underpin symbolist pictorial tropes that allied artistic creation with asceticism and corporeal suffering (2010: 87). Mammen’s apparent uncomfortable relationship with other people, as well as her enduring knowledge of Flaubert’s text throughout her life (Kinkel et al. 1978: 93–102), are understood as compelling evidence that the artist cultivated a type of self-imposed isolation, legitimised through such artistic and literary Außenseiterrollenmuster ‘outsider role models’ (Leistenschneider 2010: 93). Mammen’s preserved studio apartment (Jochens 2011: 36–38) is used further as a way of legitimising this self-imposed isolation (Figure 5). Mammen is described as withdrawing into this space, where she lived alone after Mimi moved out around 1936. The studio apartment, a small space of 53.16 square metres is reached after crossing the courtyard and up four flights of stairs. There is no kitchen, only two hotplates, a sink with cold water, a sofa, chairs, a wardrobe, bookshelves, chest of drawers and a small bed. The toilet is in the upstairs corridor. Mammen’s frugal living conditions are perceived as a critical response to the development of Berlin around her, for she only reluctantly acquired a telephone near the end of her life “just in case” (Kuby et al. 1978: 116). She is interpreted as finding little affinity with the plethora of cinemas, cafés and amusement halls on Kurfürstendamm: a boulevard, integral to the western part of Berlin (Roters et al. 1978: 9). Moreover, this same studio apartment became a survival space of “inner immigration” during the National Socialist dictatorship (Lütgens 1991: 98–99; Roters et al. 1978: 53) and remarkably withstood the allied bombing of Berlin during WWII, thereby increasing perceptions of Mammen as introverted. Mammen’s collecting habits are also understood as indexical signs of a hermit existence (Leistenschneider 2010: 2 and 81–93). This studio space has been billed  According to the Förderverein archive’s unpublished list of Mammen’s library, the artist has over two hundred volumes by French authors.  Mammen’s library contains two editions of Flaubert’s text from 1849 and 1856.

Questioning bohemian myth in Weimar Berlin

365

Fig. 5: Jeanne Mammen in her studio-apartment on Kurfürstendamm (circa 1946). On the wall her own painting African Mask (circa 1939–42) (centre) and sculptures Small Head (circa 1945) and Musical Clown (circa 1942) (far left). Photographer Elsa Thiemann née Franke. Image courtesy of the Förderverein der Jeanne-Mammen-Stiftung e. V.

a “nest” and “second skin” (Roters et al. 1978: 9) housing collected artefacts for over fifty years: wax votives, glass, wood and clay figures arranged alongside masks, magazines, sculptures, paintings, seashells and driftwood (Figure 5). The bookshelves reveal further passions for literature, philosophy, folklore and art. Mammen had an intense and intimate relationship with these objects, some of which clearly played an important role in her artistry. Collected artefacts are represented in her works, or are transformed into art objects themselves through cutting, doodling and painting.8 Such subject-object relations share affinities with Walter Benjamin’s understanding of collecting, (in his case he collected primarily books), as anarchic and destructive. According to Benjamin, “[the collector] dreams his way not only into a bygone world, but at the same time into a better one […] in which things are liberated from the drudgery of usefulness”, or typical

 Such transformations include: Der Querschnitt with painted pages, writing in Jeanne’s hand and bits cut out of photographs; Mammen’s copy of Jean Giraudoux’s Belle from 1947 with ink doodles and the self-painted wardrobe in her bedroom.

366

H. Camilla Smith

monetary value (Benjamin [1931] 1999: 62). For Benjamin, as for Mammen perhaps, collecting served as a renewal of an “old world”, which acted to subvert the animosity she felt for the consumerism of the Kurfürstendamm. Indeed, Mammen’s own friends point towards her dislike of materialism through her own comments, “I hate having to go into shops and having to choose”, further affirming the artist’s ascetic tendencies (Kinkel and Kuby et al. 1978: 98 and 117). Such interpretations, however, lend themselves to what Brian O’Doherty has deemed a problematic fascination with artists’ studios. These studios essentially become “para-creations, footnotes to the departed painting”, which in some cases come to “stand for the art”, be it through examination of the physical spaces themselves, or gallerymuseum simulacrum9 (O’Doherty 2009: 6). Mammen’s studio apartment has been interpreted as an isolated “womb” therefore, in which her art was mystically conceived. In light of these previous approaches, Mammen’s illustrations for Krehan’s narrative in Die Woche appear misplaced. As Krehan’s title suggests, Der Maler und sein Modell ‘The Painter and his Model’ focuses on the relationship between the model and artist in the studio, emphasized by Mammen’s first illustration depicting a male artist copying a female model (Figure 1). The closeness of text and image suggests Mammen knew Krehan’s narrative, for her illustrations consolidate his textual exploration of the different types of model an artist encounters. Mammen therefore portrays the same artist with both male and female portrait sitters, be they a scrawny-suited male, modish flapper who is unsatisfied with her portrait, fearsome lion-tamer, or fat, unattractive capitalist portrayed flatteringly (Figures 2–4). Conversely, Mammen did not, as far as we know, accept commissions for portraits, nor invite models into her studio.10 However, her portrayal of the tensions between the artist and his sitters signifies the act of commissioning as artistic compromise and begins to point towards the optimum creative experience of social isolation, (as unadulterated inner contemplation), defined by Schopenhauer and Nietzsche (Leistenschneider 2010: 34–35). Moreover, as an artist himself, Krehan’s text, written in the first person, defends the artist ascetic through his zealous description of the “professional” relationship between the artist and model (Kosch 1960: 1096; Krehan and Mammen 1927, 31: 32–33). Consequently, Krehan dismisses the “bourgeois perception” of the studio as a site of unfettered sexual pleasure and instead emphasizes the model’s “protective male chaperone” and the artist’s overall “lack of money” (Krehan and Mammen 1927, 31: 34). Perhaps deliberately echoing an earlier Director of the Weimar Art Academy, Paul Schultze-Naumburg (Berger 1982: 112–114), where Krehan tells the reader he  For simulacrum I mean here in terms of intentionally distorted studio spaces such as Paul MacCarthy’s installation, The Box (1999): his studio preserved in a box in a gallery space, but turned on its side.  There is one possible early watercolour and pencil drawing suggesting a female model (Mimi?) posing in Mammen’s studio apartment (Merkert 1997: 240).

Questioning bohemian myth in Weimar Berlin

367

trained, the article argues that many of the models the artist is forced to paint are in fact “old” and “ugly” and therefore possess no seductive “threat” whatsoever. Mammen’s illustrations reinforce Krehan’s narrative. Consequently, she depicts an old hag trying to restrain a man who clumsily knocks over plant pots in a futile attempt to glimpse the artist-model encounter in a nearby studio. The accompanying caption emphasizes the man’s pointless curiosity further proclaiming, “Vorsicht! Lebensgefahr! (Warum in die Ferne schweifen –)” ‘Careful! Danger! Why gaze into the distance’. Mammen’s portrayal of the artist’s ill-fated sufferings is further signified by the assurance of academic acclaim conflicting with his abstract “avant-gardism”11 (Krausse 2006: 44) (Figure 2). The caption “Himmlischer Lohn – die Bilder werden gehängt!” ‘Heavenly reward-the pictures are to be hung!’ suggests the eventual “skying” of his work in a position in the academy, (if his work is accepted at all), where no one can see it. Paint-spattered (Figure 2), tearing out his hair (Figure 3) and crouched, rain-drenched under an umbrella (Figure 4), the painter’s tortured body, although ridiculous, can be considered similar to the visual manifestations of the outsider role models with which scholars Leistenschneider, Lütgens and Reinhardt associate Mammen’s early symbolist imagery. However, interpreting Mammen’s work in these terms undermines the significance of the artist’s playful and self-critical understanding of herself as a “female” practitioner.

3 Mammen’s mocking of phallic authority The magazine in which Der Maler und sein Modell appeared had a set format that was well established by 1927. The magazine had first appeared in 1899 and was published by the August Scherl publishing house in Berlin. It contained local and international news in pictures, reviews of theatre, fictional stories, puzzles and numerous advertisements. Articles such as Die Großstadt der Zukunft ‘The Metropolis of the Future’ with Strozoda’s fantastical illustrations (Dominik 1924, 15: 386– 388), or ‘Können wir uns mit Mars verständigen?’ ‘Can we come to an understanding with Mars?’ (Anon 1924, 15: 369) also indicate the importance of humour. The fact that Mammen’s work originally appeared in high quality colour further suggests the significant position the story occupied in a magazine, which in 1927 only boasted a few coloured pages in each edition.12 The article’s central role in the magazine is also signalled by Mammen’s mock-up drawing of an original front cover entitled College X [sic] ‘Colleague X’, portraying the depleted figure of a

 Simply put, overemphasis on colour and expression/distortion were considered ‘anti-establishment’ and greeted with hostility amongst conservative (academic) camps.  I am indebted to Adelheid Rasche for discussions on magazines during this period.

368

H. Camilla Smith

Fig. 6: Jeanne Mammen, College X [sic] [‘Colleague X’], undated (circa 1927), watercolour and pencil, 33.5 x 26 cm. Unsigned. Title page intended for Die Woche, Heft 31, 29 Jg., Juli 1927. (c) VG Bild-Kunst, Bonn 2013.

Questioning bohemian myth in Weimar Berlin

369

scrawny artist, bent double, painting at his easel (Figure 6).13 Moreover, articles by writers such as Clara Viebig and advertisements for face cream, female hygiene and slimming techniques in the magazine indicate that female readers were also important (Viebig 1924, 14: 348). Considering Krehan’s and Mammen’s article in the context of Die Woche suggests it was mocking two discursively constructed positions of the male artist: the artist as impoverished bohemian and the artist as potent creator. Following Irit Rogoff’s illuminating discussion on representation of the male artist in Germany, it can be argued that these characteristics function autonomously as distinguished sets of ideas but here, are interconnected around notions of creative “vision”, “authority” and “masculinity” (1991: 116–147) that would have been obvious to the magazine’s readers based on previous stories relaying (male) artistic agony (Münchhausen 1924, 18: 465). In emphasizing the artist’s “professionalism”, ensuing economic problems and complex negotiations between artist and patron, Krehan refutes myths of artistic bohemianism. He describes the hypocrisy of those who visit the artist’s studio in the hope of nocturnal adventure, yet barely acknowledge this figure during the daytime. Bohemianism is not a self-conscious position by the artist therefore, rather discursively constructed by the patrons who commission his work. As O’Doherty succinctly points out, ‘”the bourgeoisie […] consigns its alienated imagination not only to the artist, but to the magical space where art is pondered and brought into being” (2009: 6). Mammen’s illustrations signify this discursive recognition by parodying the long-established tradition of the male artist in his garret (Figure 1 and Figure 6). Such garret studios had again become popular in France during the nineteenth century through the likes of Paul Cézanne’s The Stove in the Studio (circa 1865) and Octave Tassaert, Interior of a Studio (1845). Although not dissimilar to symbolist emphasis on artist marginality and suffering, these works do not depict battles with metaphysical worlds or femme fatale figures, rather romanticise the social conditions within which the artist is compelled to create. College X (Figure 6 and Figure 1) can be considered as subtly referencing such works. For example, the artist’s canvases turned to the wall and stacked near to his stove are not unlike Cézanne’s, whereas his depleted figure and his black cat, share affinities with Tassaert’s slumped artist warming himself with his white cat by the fire. However, the dangling pink socks, the cheeky cat perching precariously on the artist’s shoulder, the playful spider silhouette and the obvious disparity between imagined putto and screaming child-reality, caught in the caption, Großmutters Goldkind hat noch nicht die richtige Einstellung zur Malkunst ‘Grandmother’s golden-child does not yet have the right attitude towards art’, all point towards Mammen’s pictorial ridicule of such enduring myths.

 This is the only preserved original drawing of the series. The whereabouts of the other nine originals are unknown.

370

H. Camilla Smith

In Berlin’s rapidly growing market economy, bohemian contrivance was outmoded. Artists could not afford to shut themselves away and sample the “exquisite dish” of loneliness, as author Robert Walser’s pre-war commentary already recognised ([1910] 2012: 62). Having lived with his brother, Karl, an artist, in the same apartment building as Mammen on Kurfürstendamm in 1910 (Ortmanns 2010: 41), Walser’s comment at the beginning of this paper suggests that he understood the importance of engaging with the metropolis if an artist were to make his mark. Upon arriving back in Berlin in 1916, Mammen too recalls in a letter to her friend, the painter Hans Thiemann how she realised that this was not a city for bohemians, unlike Paris, noting, ‘even the bohemians in Café des Westens were smartly turned out and ordered their drinks with a friendly, but distanced tone from Herrn Oba [sic] ‘waiter’. “They [so-called bohemians] were money grabbing and those who didn’t have any were regarded simply as wasters” (Delbrück et al. 1978: 144). Mammen’s comment significantly connects dwindling bohemian culture with increasing commercialism, sentiments of which are also found in the writing of her contemporaries Joseph Roth and Eugene Szatmari. Both writers report disparagingly regarding the mainstream changes in Berlin’s café culture, which now only boasted a boring cross section of Berlin’s metropolitan population seeking entertainment (Roth [1923] 2003: 135–139; Szatmari 1927: 114–124). Moreover, the fact that College X was in reality not reproduced on the cover of Die Woche suggests that using the motif of the artist’s garret to encourage magazine sales in 1927 was perhaps too risky. Instead, the magazine used the glamorous image of a woman. Whilst Mammen’s early works suggest an iconographic identification with outsider role models, by 1927 she and her sister were professional artists producing magazine illustrations, fashion plates and film posters and negotiating contracts with the Ullstein and Scherl publishing houses. The notion that she self-critically positioned herself as a solitary ascetic during this period, be this in line with artistic or literary role models, is therefore not convincing. Mammen’s illustrations lend further weight to renewed consideration of the artist alongside her female colleagues in Weimar Berlin. Having trained in Paris and Brussels, both her own education and collection of books suggest she was well aware of established discursive constructions of artist canonicity. This is evident in her collection of artist monographs, with books on Bosch (1922), Brueghel (1921), Rembrandt (1900), Picasso (1927) and Cézanne (1922), some of whose publishing dates suggest that they were perhaps acquired during the decade in which Die Woche illustrations appeared. Mammen’s correspondence with friends also reveals these interests. In a letter dated 1947 to émigré scientist Max Delbrück, Mammen asks if he has seen Alexander Korda’s 1936 film, Rembrandt. Described by Mammen as “ghastly”, she decides “One should never ever film very creative people; it is a big sin […]” (Delbrück et al. 1978: 126). In this film Rembrandt adheres to all artistic clichés, seducing his wife and smuggling criminals into his studio. Nine years later in 1956, Mammen’s friend, the artist and art historian Friedrich Ahlers-

Questioning bohemian myth in Weimar Berlin

371

Hestermann (1883–1973), published a small book on Pablo Picasso entitled Maler und Modell ‘Painter and Model’, which Mammen kept in her library. The book contains his commentary on Picasso’s drawings depicting all types of artist-model relationships converging around and mocking (potent) creativity (Ahlers-Hestermann 1956: 6). Ahlers-Hestermann’s critical engagement with such clichés in the book’s introduction, as well as his attack on Berlin’s stringent artistic conservatism, might be explained through his own advanced teaching methods (1956: 7– 8). He was a life drawing master between 1928 and 1933 in Cologne, teaching both men and women and became the newly appointed director of the Department of Fine Arts in Berlin’s Akademie der Künste the same year his Picasso book appeared (Manigold 1986: 28–29). Moreover, Ahlers-Hestermann’s monographs on artists such as Käthe Kollwitz (1952 and 1960)14 and his German-Russian wife, the artist Alexandra Povórina with whom Mammen was friends,15 indicate his potentially progressive, if not liberal attitudes towards women professionals. Moreover, it is clear he not only supported Povórina as an artist in her own right, but also her commitments as a lecturer at the Hochschule für Angewandte Kunst in Berlin Weißensee (Manigold 1986: 41–44). Consequently, it is perhaps unsurprising to find early examples of Mammen’s work similarly questioning the presumed phallic (physical and cultural) authority inherent in the act of creativity. Unlike Schopenhauer’s and Nietzsche’s writing, Krehan presents the necessary sexual abstinence of the artist ambivalently and in fact admits that “private” relationships between artist and model do happen (Leistenschneider 2010: 29–36). When artist and model come together it is because she is a “lover of his art” [my italics]: darning his socks and cooking his dinners (Krehan and Mammen 1927, 31: 33). Consequently, Krehan’s typological descriptions of “ugly”, “old” and “beautiful” models uphold artist-virility as the considered norm. When Krehan recounts how he is forced by Professor Schnell to approach the muscle-bound boxer, Paul Vierkant, to be his model, Krehan’s distress is signified by descriptions of Vierkant as a man “of muscular magazine-perfection […] who filled the whole doorframe” (1927, 31: 32). Krehan’s narrative therefore reinforces contemporaneous portrayals of the male artist as openly masculine (and heterosexual), which have since been identified by scholars in the self-portraits of painters such as Lovis Corinth, Otto Dix and Ernst Ludwig Kirchner posing with their female models (Duncan 1993: 81– 108 and Meskimmon 1999: 237–240). Moreover, Dix was said to have become “the most talked about portraitist in Berlin” precisely because of the exhibition of a self-portrait in which he clearly sought to emphasize his status as a professional

 Noteworthy in the Kollwitz publications is Ahlers-Hestermann’s (emotive) emphasis on her successful roles as mother and artist (Kollwitz 1952: 9–11).  Mammen owns an exhibition programme of Povórina’s retrospective at the Haus am Lützowplatz in her library with a preface written by Ahlers-Hestermann. There is a dedication in the front from Ahlers-Hestermann and their daughter Tatiana.

372

H. Camilla Smith

artist a year before Der Maler und sein Modell appeared.16 Interestingly, this 1926 portrait exhibited in Galerie Nierendorf has been interpreted as having a “defensive” quality about it, although ultimately Dix depicts himself as an archetypal dandy (O’Brien-Twohig and Keith Hartley 1992: 157 and 160). Mammen’s illustrations parody such portraits, for instead of mastery in her images; the artist is shown to be both helpless and clumsy, stepping onto paint tubes in his ill-fitting slippers in an attempt to calm a screaming child in a scene of uncomfortable domesticity (Figure 1). Opposite the strength and beauty of the female lion tamer brandishing her whip, the artist shrinks behind his canvas and when confronted with the criticisms of a female patron, he completely loses his nerves (Figure 3). Parodic images of role reversal in the artist’s studio were not particularly unusual. However, the fact that a female artist produced these drawings is significant (Bing 1910, 46: 117 and Weisgerber 1909, 17: 387). Indeed, two years earlier in 1925, Mammen completed an illustration for the story Das unschuldige Mädchen ‘The Innocent Girl’ for the magazine Der Junggeselle ‘The Bachelor’, depicting a depleted male artist opposite a confident, beautiful female model (Barr 1925, 37: 14). The caption between the model asking Ausgerechnet auf so’n Pinsel muß ich reinfallen! and the painter Aber liebes Kind, bei uns Malern hast Du doch die Lippenfarbe umsonst bekommen! ‘Trust me to fall for such a type!’ ‘Why my dear child, you did get your lip colour for free from us painters!’ reinforces the discordance between the painter’s unwavering optimism and his dissatisfied model. Mammen’s drawing shares extraordinary similarities with Joseph Hémard’s satirical illustrations for Henry Murger’s famous stories Scènes de la Vie de Bohème ‘Scenes of Bohemian Life’ (1847–9), which appeared in 1921. Crucially, Hémard’s images maintain the traditional artist-model status quo, whereas Mammen’s “Bohème” does not. Considered as such, I argue that Mammen’s images for Die Woche begin to allow a closer positioning of the artist in relation to her female contemporaries.

4 Reconsidering and retaining the female artist In emphasizing Mammen’s identification with nineteenth-century outsider role models, scholars Leistenschneider, Reinhardt and Lütgens emphasize that it is the notion of the eventual transcendence of the flesh that Mammen sought in literary figures like Flaubert’s St. Anthony. Consequently, Mammen’s uncomfortable relationship with her own body demonstrated by comments “my body is a burden” and “I only want to be a pair of eyes” in order to create a sense of her own invisibility, find parallels in Flaubert’s wrestling with his own body as a writer (Leistenschneider 2010: 88–89; Lütgens 1991: 206–208). Further compelling simi-

 For further examples of Dix’s potent creativity see Self-Portrait with Nude Model (1923) and SelfPortrait with Muse (1924).

Questioning bohemian myth in Weimar Berlin

373

larities are revealed through Mammen’s wish to, and Flaubert’s actual wearing of a monk’s cowl, signifying respective longing for a type of disembodied creativity (Leistenschneider 2010: 88). However, these parallels continue to define Mammen through her relationship to the male subject. As Leistenschneider readily suggests, there were no female outsider role models with whom Mammen could identify (2010: 93). Moreover, although Lütgens points out that Flaubert should not be considered simply as articulating similar forms of outsider ideal as other male symbolists, she still infers that ultimately Mammen was unable to sympathize with “revolutionary” female artists like Hannah Höch and found little affinity with her work (1991: 208). Conversely, I argue that because women artists were working from the real position of marginality within established cultural frameworks, not one cultivated from a position of luxury like men, this situation needed challenging. Evidence suggests that Mammen recognised the importance of being a woman artist and did not necessarily distance herself from the traditional “feminine” ideal. Although there are few self-portraits, photographs exist of Mammen in her studio apartment throughout her life (Figure 5). These range from casual snap-shots with friends, to formal photographic portraits for exhibition prospects taken by the Bauhaus photographer Elsa Thiemann.17 The function of many of these photographs is unclear, but none appear to have been used in popular magazines such as Der Querschnitt featuring photographs of artists such as Mira Sohn alongside images of growing woman professionals (1924, 6: between pages 392–393). Despite these ambiguities, however, we might consider such photographs as consolidating artistic authority through the adoption of performative strategies not unlike those revealed in the self-portraits of Mammen’s female contemporaries such as painter Lotte Laserstein (Meskimmon 1999: 238–240; Krausse 2006: 124–125; Rowe 2006: 68–88). Taken a few years after Mammen’s illustrations for Die Woche and when both sisters were established artists, the photograph from circa 1930 shows the sisters Mimi and Jeanne taking tea in their studio apartment (Figure 7). Sitting elegantly with their legs crossed, both women wear mid-length skirts, stockings with dainty shoes and sport fashionable bobbed hairstyles. Their body language and dress, coupled with the act of taking tea, point towards their middle class background (Reinhardt 1991: 81–82). However, the notable number of paintings and the room partition behind which is a small sink, reveal that this is a temporary “domestic” scene erected within their studio workspace. Both sisters also wear painters’ smocks over their fashionable dress, a similar smock in which Mammen is photographed over fifteen years later. Together, these potentially conflicting

 Another of Elsa Thiemann’s photographs of Mammen appears in the Almanach der Galerie Gerd Rosen in 1947, the same year in which Else, (maiden name Franke), married Mammen’s close friend and later correspondent Hans Thiemann. Else had met Hans Thiemann at the Bauhaus school, where he was a student in Wassily Kandinsky’s painting classes.

374

H. Camilla Smith

Fig. 7: Jeanne Mammen (right) with her sister Marie Louise Mammen (Mimi) in their studio apartment (circa 1930). Photographer unknown. Image courtesy of the Förderverein der Jeanne-Mammen-Stiftung e. V.

signifiers point towards a resolute understanding that historical perceptions of “femininity” still felt during the early thirties, do not undermine the cultural authority of women’s artistic practice. Mammen’s apparent lack of interest in clothes and her cross-dressing in photographs align her further with the androgynous body of the ascetic or homosexual (Leistenschneider 2010: 81–88 and 93; Lütgens 1991: 23 and 205). However, there is no conclusive evidence to suggest Mammen was a lesbian as scholars continue to assume. Leistenschneider further argues that Mammen also used artistic strategies of “uglification”18 that she associates with the “hard”, “jagged” and “spiteful” lines of Mammen’s contemporaneous self-portrait of circa 1933 (2010: 85; Merkert 1997: 187). Such interpretations underplay the popularity of cross-dressing amongst women at that time (Meskimmon 1999: 199–229), as well as Mammen’s notable sense of humour.19 In a second portrait of the sisters from the same set of

 Mammen’s friends are ambivalent about her appearance. Comments by Hans Thiemann to Mammen in letters from the 1970s suggest that she resembled the cabaret star Margo Lion (1899– 1989) (Mammen and Thiemann 1979: 29–31).  A number of unpublished photographs of Mammen in the Förderverein archive suggest meticulous attention to fashionable detail through jewellery, scarf rings, belts and hats. Moreover, her maintained interest in her appearance can be found in unpublished letters from the artist to Hans Gaffron dated 24. 09. 46 asking for stockings in her CARE-packages. Here Mammen is very specific

Questioning bohemian myth in Weimar Berlin

375

photographs, both women have removed their smocks and pose in the fashionable clothes they wear beneath.20 There are no obvious painterly attributes around them. However, it is Mammen who now assertively meets the viewer’s gaze and Mimi who stands in profile. Pictured full length, both sisters show a clear interest in contemporaneous fashions some of which could have been found immediately at 28 Kurfürstendamm, where Marie Latz had a renowned women’s fashion salon from 1919 until 1927 (F. C. Gundlach, Uli Richter and Katja Aschke 1993: 123 and 248). In challenging the viewer’s gaze, Mammen might be understood as establishing her position as active viewing subject, who is both viewed and confidently viewing, thereby undermining previous scholarly focus on Mammen’s own proclaimed invisibility (Lütgens 1991: 205). Crucially, the artist’s femininity is part of this subjectivity. Together, these two photographs highlight the playful changing appearance of the sisters, which is captured further through Mammen’s attitude towards her biography throughout her life. Noted for her own unwillingness to give biographic details, Mammen would wittily remark to Max Delbrück in 1970 that her biography was a “bioschraffie”, (schraffieren means to hatch or crosshatch in German), signalling the underlying humour implicit in these so-called “hard” and “jagged” lines that Leistenschneider identifies with Mammen’s self-representation (Delbrück et al. 1978: 142). Whilst there is no evidence to suggest that these photographs of the Mammen sisters were produced for wider viewing, they can nonetheless be understood within the growing field of women image-making and makers of the Weimar Republic and thereby move away from the artist’s identification with male outsider role models and instead towards inclusionary frameworks. If we allow Mammen’s work to be positioned alongside that of her female contemporaries, her mocking of phallic authority also points towards her recognition of the ongoing restrictions that women artists in Berlin still faced. Despite Krehan and Mammen’s ridicule of enduring myths, Anna Havemann points out that unlike other academies in Germany, (such as Weimar and Stuttgart), through the directorship of Anton von Werner (1845–1915) Berlin’s Akademie der Künste did in fact lag far behind its European counterparts. It still forbade women to participate in life drawing classes, even after their admittance to study in 1919 (Havemann 2011: 15; Krenzlin 1992, 1: 73–87). Indeed, the author and women’s rights campaigner Lenore Kühn (1878–1955) complained that such restrictions and conflicts were often the root cause of a lack of self-confidence for many woman artists, which ultimately helped sustain their continued cultural exclusion (Jankuhn 2004: 49–50). Mammen’s training at the Parisian Académie Julian meant that she had in fact been able to attend such classes and her comments suggest how informative this period of “constant sketching” was to her artistry (Kinkel et

that they should be a particular denier and ‘not too thick, not too thin’ and in ‘brown tones and dark, rather than light’.  Photograph in the Förderverein archive.

376

H. Camilla Smith

al. 1978: 93). Indeed, the role life drawing played in Mammen’s creative development is indicated by the one thousand, two hundred and seventy-four drawings she produced throughout the thirties, when she regularly attended a studio in the Hardenbergstraße. Part of the reason Mammen enjoyed these classes was that they appeared to have had few of the formal constraints associated with academic study, for you could simply “come and go as you pleased” (Kinkel et al. 1978: 95). In an on-going discussion in her letters to Hans Thiemann about her art education, Mammen recalls that despite pedagogic equalities, her colleagues in Brussels were in fact still somewhat bemused at the prospect of an eighteen year-old-Demoiselle like Mammen winning the class medal for composition (Mammen and Thiemann 1979: 32). It is perhaps significant therefore, that Mammen’s drawing of the male artist sketching in a hallway full of women for Die Woche bears a resemblance to the corridors of the Académie Royale des Beaux-Arts in Brussels (Lütgens 1991: plate V), where she received this prize (Figure 3). Seen alongside the conflicting interests of academic acclaim and “avant-gardist” abstraction, conflicts that raged throughout the first decades of the twentieth century (Figure 2), Mammen’s illustrations for Die Woche suggest that she was connecting institutional conservatism with enduring artist-model myths. What could be considered historical debates in progressive France by 1927, were still topical for Berliners. Moreover, Mammen’s mocking illustrations perhaps found their resonance with the female readers of a magazine in which Clara Viebig had so optimistically proclaimed that women “were now able to take up any profession [… they] wanted” (Viebig 1924, 14: 348). In reality however, there were clearly caveats. In the second half of the twenties, although training was still far from equal, women artists were in fact gaining notable public presence through the formation of cultural organisations. In the same year Der Maler und sein Modell appeared, two exhibitions dedicated to working women and female artists were held in Berlin (Havemann 2011: 21). Six years later, Mammen herself was to exhibit with one of Berlin’s most notable all-female organisations, the Verein der Berliner Künstlerinnen und Kunstfreundinnen ‘Organisation for Woman Artists and Female Art Friends’ that, since its establishment in 1867, provided women with formal training and growing exhibiting possibilities (Fuhrmann and Jestädt 1992, 2: 353–366). Although not a formal member, Mammen’s work in the Verein’s 1933 exhibition Frauen im Beruf ‘Women Professionals’, alongside twenty-nine other women artists, indicates that she was part of such exhibiting cultures (Züchner 1992, 2: 268; Cierpialkowski and Keil 1992, 2: 383) and crucially, points towards her fostering of friendships with other female artists.21 The notably provocative theme of this  Scholars overemphasize Mammen’s lack of contact with women as part of this hermetic framework. Unpublished letters in the Förderverein archive reveal Mammen was very close to Stefanie Nathan (the second wife of well-known satirical artist Albert Schäfer-Ast) whom she continued to write once ‘Steffie’ had immigrated to England (nine letters exist from between 1939 and 1970). Steffie was also a professional artist who illustrated magazines such as Das junge Deutschland, Nr. 4/5, vol. 2., 1919, the cover of Die Dame, Nr. 1, Januar, 1923 and Nr. 17, June 1923.

Questioning bohemian myth in Weimar Berlin

377

exhibition also suggests perhaps why Mammen deliberately contributed and carefully kept the exhibition reviews.22 Whilst no evidence exists to suggest close contact with the Berlin studios of Lotte Laserstein or Hannah Höch, the artist’s preservation of articles such as Frauen haben mehr Stolz: Wie Berliner Künstlerinnen leben ‘Women are more proud: How Berlin’s female artists live’ (1951), which compares Mammen’s (lack of) public presence to Höch and sculptor Renée Sintenis and upon which Mammen has scribbled a large exclamation mark, indicates that she maintained an interest in her discursive positioning amongst female contemporaries.23 Indeed, Mammen’s commissioning of two newspaper clipping services, Dr. Max Goldtschmidt’s Büro für Zeitungsausschnitte and Adolf Schustermann’s Zeitungsnachrichten-Büro to send her reviews of her work and her own exhibitions during the 1920s and early 1930s, demonstrates that it was during this time that she began to carefully consider the public reception of her work. As a result of her interest in fellow artists, Mammen would have been aware of the essentialist approval of Sintenis’ animal sculptures as “appropriate” (Scheffler 1920, 18: 184; Biermann 1933, 22: 15),24 as well as enduring continual comparisons of her own work to that of the “männliche” (male) style of counterparts Otto Dix and George Grosz (Osborne 1930; Lütgens 1991: 77–79). Certainly, contemporaneous reviews by male critics suggest Mammen’s work was perceived as being particularly vicious in its portrayal of men. In reviewing her first solo exhibition at Galerie Gurlitt, three years after Der Maler und sein Modell, Max Deri asks whether the artist’s works do not signify a “[…] woman’s revenge? Revenge for centuries of intellectual quashing?” (Deri 1930, 288). Consideration of Mammen’s pan-European education and early professionalism in Berlin therefore, points towards a closer alignment of the artist with her female contemporaries. Conversely, it might be considered odd that the artist was never actually an official member of such all-female organisations, nor did she own literature such as Hans Hildebrandt’s Die Frau als Künstlerin [Woman as Artist] (1928), nor indeed other books relating to women artists in her library. Sexual difference appears not to be the prime, or certainly not the only motivating factor behind Mammen’s 1927 illustrations.

 The Förderverein archive contains a large folder of press articles Mammen kept from 1930 onwards.  Albert Büsche’s article appeared on 24. 06. 51 in Sonntag Der Tagesspiegel, Förderverein archive.  Sintenis worked with Mammen’s close artist friends Hans Uhlmann and Hans Thiemann in the Berliner Neuen Gruppe (founded 1949) and also exhibited with Mammen (Reuter 1992, 1: 170–175).

378

H. Camilla Smith

5 The social context of Die Woche: recognising the importance of the artist as mediator Mammen’s concentration on subjects of social marginality such as gangsters and homosexuals during the 1920s in particular, has also been interpreted in relation to her identification with outsider role models25 (Leistenschneider 2010: 83). Mammen’s later abstract images are posited as ascetic signs and ciphers signifying her further withdrawal post-World War II (Leistenschneider 2010: 86). Such readings risk conflating gradual abstraction with perceived disengagement and infer that Mammen’s studio apartment space became a place of intensified social retreat (Leistenschneider 2010: 84–86; Roters et al. 1978: 9–13; Lütgens 1989: 76). However, Mammen’s illustrations for Die Woche point towards art and the artist function as rooted in the real world of work and achievement. Art should be understood as a socially mediatory practice and it is through this lens we should also interpret Mammen’s studio apartment. Mammen’s interest in portraying social outsiders gives an identity to sectors of society not traditionally deemed “worthy” artistic subject matter. Whilst not unusual amongst her Neue Sachlichkeit contemporaries for doing so, Mammen’s function as artist might be understood in relation to what Irit Rogoff has termed a “social mediator”. Rogoff convincingly argues that the artist Max Liebermann’s paintings reveal both his political preoccupations and social responsibility from the privileged position of an academician and the upper middle classes. Rogoff’s interpretative framework is helpful in pointing towards Mammen’s own mediatory position as an artist also from a middle class family but who, unlike Liebermann, turned to Gebrauchsgraphik ‘mass illustration’ as a way of addressing social issues. Like Liebermann therefore, Mammen portrays subject matter, which negated the “inherited values of culture as the forum in which only one dominant class’ values could be reflected” (Rogoff 1991: 131). Despite the difficult financial position Mammen and Mimi had to endure during this period, rather than identifying with, and thereby consolidating her own isolation, we should consider Mammen’s works as advocating forms of social inclusion. As Rogoff posits, “[…] any articulation of marginality is by definition part of a critique since it is inscribed with the issue of exclusion” (1991: 131). This critical inclusion is revealed through both the subject matter of Der Maler und sein Modell which, as I have argued, parodies social exclusion from the position of a female artist, and further through consideration of the wider mediatory position Die Woche occupies in the field of mass, visual reproductions.

 Rita Täuber, Ute Scheub and Gertrude Cepl-Kaufmann’s explorations of the femme flâneur also discuss Mammen and other female artists in the role of the flâneur figure who is able achieve a paradoxical ‘closeness’ to their subjects through forms of distanced vision.

Questioning bohemian myth in Weimar Berlin

379

Although it was ardently patriotic, Die Woche’s mediatory position can be gauged through its significant choice of authors and article subject matter, which reported stories of crime and poverty (Ewers 1924, 15: 364; Schwormstädt 1924, 15: 365–368). Notably Mammen’s only other known illustrations for the magazine were for the well-known author Kurt Münzer and his narrative, Licht und Schatten ‘Light and Shadows’, which appeared two months earlier in May 1927 (Münzer 1927, 22: 629–632; Merkert 1997: 252–253). Here, Mammen portrays the starving children, single mothers and disabled beggars of the city, which Münzer unambiguously blames on American capitalism (1927, 22: 629). From a good middle class Jewish family, parallels can be drawn between Münzer as author and Mammen as artist, for Münzer’s work also concentrated on social outsiders.26 The mediatory potential of this collaboration is signalled through the readers’ confrontation with Mammen’s images and Münzer asking middle class readers to question their privileged positions.27 Krehan’s and Mammen’s collaboration should also be considered mediatory. In reality, Krehan’s training as a fine artist made him a set designer for Berlin’s Kabarett der Komiker (KadeKo) in 1926 (Völker 2010). By its nature, the cabaret is noted for both its closeness to its audiences, as well as by the then contemporaneous critic Max Herrmann-Neisse for its “fighting, rebellious, satirical, attacking, […]” function (Völker 2010: 28). Working concurrently in KadeKo when he wrote Der Maler und sein Modell, it is easy to see why therefore, Krehan wrote a narrative whose focus was on parodying the outmoded cultural clichés of the socially disengaged artist. Moreover, the regular reviews of smaller theatre groups and cabarets in Die Woche suggest why Krehan might well have been known to the magazine’s readership and his views on the subject of art respected.28 Mammen’s interest in portraying social outsiders served as thematic markers of the constant absences and exclusions in monolithic cultural and social practices, to which, as a woman artist, she could perhaps relate. However, this critical position, as I have argued was not an isolated one, rather one which did show degrees of affiliation with female organisations. Crucially, Krehan’s and Mammen’s respective mediation is allied with both their own left-wing sympathies.29 Despite no conclusive evidence to suggest that Mammen was ever a member of the German Communist Party (KPD), the artist visited Russia in 1932 with sculptor and KPD member Hans Uhl-

 For example, Der weisse Knabe (1921), the story of theatrical child exploitation and also Mich hungert (1929) and Menschen am Schlesischen Bahnhof (1930).  Advertisements for ‘Benz Automobile’, Gold-fountain pens, gramophones and the ‘ICA Camera’, suggest a well-off readership.  Whether Mammen went to any KadeKo performances is unknown. However, given the cabaret’s close proximity on Kurfürstendamm from 1925, it is not improbable.  KadeKo held strong left-wing sympathies for which many of its members went into exile during 1933. In one of the last productions Krehan and his stage sets of Hitler caricatures were attacked by a group of SS men brandishing bier glasses (Völker 2010: 101).

380

H. Camilla Smith

mann, distributed left-wing material on Kurfürstendamm with Uhlmann in 1933 (Kuby et al. 1978: 115) and nurtured long friendships with like-minded writers such as Erich Kuby and scientists Kurt and Grete Wohl and Hans and Clara Gaffron who all shared similar left-wing democratic sympathies.30 Moreover, her left-wing sympathies also remain evident in later works, particularly in relation to her 1932 contributions to the Deutsche Zentral-Zeitung and in her 1946 and 1948 contributions to Ulenspiegel (Lütgens 1991: 139–141). Mammen’s underlying political sympathies might also be intimated further through her interpretation of “avant-gardism”. In Die Woche she notably depicts canvases portraying forms that are not dissimilar to those used by Russian Constructivists (Figure 2). In rejecting the notion of autonomous art, the Constructivists no longer considered themselves artists, rather engineers and exhibited works in the ‘Russian Art Exhibition’ in Galerie van Diemen in 1922 and at the ‘Great Berlin Art Exhibition’ in 1923. In her illustrations for Die Woche four years later, Mammen’s use of geometric abstraction could be understood as a further way in which she points towards social forms of art in line with her questioning of the artist’s status.31 Indeed, this early engagement with abstraction significantly becomes a politicised tour de force in the artist’s cubo-expressionist works during the 1930s and during the later decades of her life in relation to debates on artistic style between the eastern and western sectors of a post-World War II divided Germany. During these subsequent decades, Mammen’s abstraction is interpreted as politicised forms of Utopia (Lütgens 1989: 76; Lütgens 1991: 134–135, ff. 45, and 234). Despite never returning to her social realism of the 1920s therefore, her enduring friendships and interest in political discussion signal a maintained understanding of art and the artist function as politically and socially engaged. Final comments should be made in relation to Die Woche’s function as object within Mammen’s studio apartment. Editions of Die Woche form part of the artist’s collection of magazines, books, exhibition and theatre programmes which, not unlike Benjamin’s book collecting (Benjamin [1931] 1999: 69) have been interpreted as acting like “building blocks, into which [… he] disappeared, away from the world” (Roters et al. 1978: 9–13). However, parts of Mammen’s collection also signify the movement of objects back and forth from her studio, a space that was not hermetically sealed therefore, rather where objects were in fact “ingested and digested” (O’Doherty 2009: 26). A key example of this is Mammen’s Bücherkarren ‘book cart’, which she operated with Hans Uhlmann on a Kurfürstendamm side street, between 1933 and 1934 (Eschmann 2012: 59–64). Surviving photographs show Mammen standing by her cart and handwritten pages document the sale of

 Gaffron is said to have voted for the KPD in 1932 (Rürup 2008: 200).  Not all of Mammen’s work was considered ‘mediatory’. Die Rote Fahne decided in 1930 that the subject matter of her works was ultimately too ‘bourgeois’. 2. Beilage, Nr. 276, 26. 11. 30, Förderverein archive.

Questioning bohemian myth in Weimar Berlin

381

prints in commission from Galerie Gurlitt, as well as books and magazines and some of her own art work.32 In his research on book carts, Michael Eschmann emphasizes the social aspect of such book carts as Mammen’s, where items were often exchanged and people stood talking together (Eschmann 2012: 59–64). Unlike Benjamin’s books, which were marked by their quiet isolation, packed away in the privacy of boxes, objects from Mammen’s collection were inherently linked to the economic and social conditions in which she lived. The book cart suggests that some of these objects circulated as part of social fabric of Kurfürstendamm. Consequently, other objects in Mammen’s collection should be considered in these terms, such as the wax votive baby and child’s head both brought back from a church in the Canary Islands and the confectionary papers, (of which she had boxes full), collected by friends (Kinkel et al. 1978: 100). These objects reinforce social exchanges, be it directly with friends, through travelling, or indirectly through their original connection to daily practices such as church worship.33 Annelie Lütgens also highlights the way in which Mammen used the objects in her studio-space as a way of ordering her works into their own cultural context, the extent of which is evident in the backdrop of Elsa Thiemann’s photographs of the artist (Figure 5).34 Here, in placing her own sculptural forms and paintings alongside North America ethnographic objects, stencils, images of Asian dancers and European architecture, Mammen’s collecting deems as both anarchic and socialising. Social realignment of Mammen’s collecting practices points towards her studio apartment not as a space of mystical creation in which she shut herself away but rather as a lived, porous space in the heart of the hubbub of west Berlin between which objects, friends and Mammen herself constantly moved.

6 Conclusion: “One should remain unreservedly modern”35 Scholars and Mammen’s friends dispute the artist’s own regard for her Weimar production during her life. Whilst Mammen herself was surprised at the public’s enraptured “rediscovery” of her magazine illustrations during the 1970s,36 com The receipt book documenting what was sold is in the Förderverein archive and dates from 24. 8. 1933–20. 1. 1934. There is only one entry for 1934.  Jeanne travelled extensively throughout her life until well into her eighties, for example, Russia (1932), Italy (1954), Spain (1968), Morocco (1969), Canary Islands (1972) Avignon (1973) and Hamburg (1975).  I am indebted to Annelie Lütgens for discussions on Mammen’s studio.  A quotation from Arthur Rimbaud who Mammen greatly admired.  Mammen’s ambivalence is demonstrated by her incredulity in a letter to Thiemann that someone, “an idiot”, would pay 1,800 German marks for an original watercolour she produced for Simplicissimus. Mammen to Thiemann 24.04.70. (Delbrück et al. 1978: 141).

382

H. Camilla Smith

ments by her in fact point towards their enduring social function (Delbrück et al. 1978: 141). Upon seeing the prostitutes in Amsterdam during her many travels, Mammen recalls in a letter in 1967 to Thiemann how, “had I still worked for Simpl [the satirical magazine Simplicissimus], I would have drawn a whole series of them row upon row” (Delbrück et al. 1978: 137). Using Der Maler und sein Modell to come to wider conclusions about Mammen’s overall understanding of the role of art and the artist function is therefore helpful. In its satirical exploration of the bohemian male artist, Krehan’s narrative and Mammen’s illustrations bring to the foreground wider practices of social and cultural exclusion, particularly for women artists, in Weimar Berlin. Due to the longevity of painter and model debates in Berlin as well as the changing culture of the bohemian café life, the article’s mocking of cultural clichés would have been topical for Die Woche readers. Der Maler und sein Modell also points further towards the underlying complexities of Mammen’s perception of art and the artist function. Whilst her collecting practices reveal the tensions between the poles of introversion and social configuration, so does her art. Her interest in playing with ascribed gender roles in early photos, as well as her association with all-female artist organisations suggest her illustrations for Krehan’s article functioned as a parody of phallic authority. Whereas Annelie Lütgens posits Mammen’s art as her main way of “playing out the [feminine] decorative”, I have argued that by discursively positioning Die Woche and by examining photographic examples, Mammen actively contributed to women’s artistic practices (1991: 207). Yet the artist’s unwillingness to talk about herself or to help organise exhibitions (Mammen and Thiemann 1979: 29–30; Delbrück et al. 1978: 137), suggest that viewing her art as self-referential is not ultimately how we should interpret her work. Whereas previous interpretations of Mammen have forwarded one-dimensional understandings of the artist through her identification with male outsider role models throughout her life, I have argued for the artist function as social mediator, thereby emphasizing Mammen’s art as both social and to a degree, Socialist. Through this approach, interpretations of the artist ascetic and the mythical studio apartment have been challenged. Mammen’s own comments, “I really couldn’t give a hoot”, when asked about her legacy, demonstrate her utter refusal to be compared to “great” male artists and placed within “the” specialised field of art history (Kinkel et al. 1978: 100). Being an artist was natural: she had painted everything she could get her hands on since she was a child (Kinkel et al. 1978: 93). Moreover, Mammen’s reluctance to belong to, or be affiliated with any particular organisation or political party signifies her unwillingness to ascribe to institutionalised socio-cultural and political practices in both her life and works. Art and the artist should remain of the present, “modern”, as the comment by Arthur Rimbaud, whose work Mammen greatly admired, advocates (Hübner et al. 1978: 149–151). This does not mean fashionable or avantgarde, nor does it necessarily mean formalism as primary concern, rather socially engaged and engaging, as Krehan’s and Mammen’s Der Maler und sein Modell so

Questioning bohemian myth in Weimar Berlin

383

skilfully and convincingly suggests. And it is through this social engagement that I argue we should begin to look at Mammen’s work and her studio space anew. Acknowledgments: I am grateful to the DAAD for their financial support. Special thanks to the Förderverein der Jeanne-Mammen-Stiftung e.V., in particular Cornelia Pastelak-Price who enabled extensive studies at Mammen’s studio apartment and many valuable discussions.

References Ahlers-Hestermann, Friedrich. 1956. Pablo Picasso: Maler und Modell. Stuttgart: Reclam. Anon. 1924. Können wir uns mit Mars verständigen? Die Woche 26(15): 369. Barr, Fritz. 1925. Das unschuldige Mädchen. Der Junggeselle 37: 14. Bellony-Rewald, Alice and Michael Peppiatt. 1982. Imagination's Chamber: Artists and their studios. Boston: New York Graphic Society. Benjamin, Walter. 1931 [1999]. Unpacking my Library. A Talk about Book Collecting. In Hannah Arendt (ed.), Illuminations, 61–69. London: Pimlico. Berger, Renate. 1982. Malerinnen auf dem Weg ins 20. Jahrhundert: Kunstgeschichte als Sozialgeschichte. Place: DuMont Taschenbücher. Biermann, Georg. 1930. Renée Sintenis. Der Cicerone Halbmonatl. Zeitschrift für Künstler, Kunstfreude und Sammler 22: 15. Bing, Henry. 1910. Das Modell. Jugend 46: 117. Cierpialkowski, Darius and Carina Keil. 1992. Der Verein Berliner Künstlerinnen in der Zeit zwischen 1933 und 1945. In: Dietmar Fuhrmann (ed.), Profession ohne Tradition: 125 Jahre Verein der Berliner Künstlerinnen. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur. Volume 2, 383–395. Berlin: Kupfergraben [exhibition catalogue]. Cole, Michael and Mary Pardo (eds.). 2005. Inventions of the Studio, Renaissance to Romanticism. Chapel Hill: University of North Carolina Press. Delbrück, Max. 1978. Jeanne Mammen-Briefe. In: Eberhard Roters, Lothar Klünner, Max Delbrück, Johannes Hübner and Stanislaw Kubicki (eds.), Jeanne Mammen 1890–1976, JeanneMammen-Gesellschaft. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin, 121–149. Stuttgart-Bad Cannstatt: Cantz. Deri, Max. 1930. Zwei neue Künstler. Berliner Zeitung am Mittag 288, 21 Oktober: unpaginated. Berlin: Ullstein Verlag. Dominik, Hans. 1924. Die Großstadt der Zukunft. Die Woche 26(15): 386–388. Duncan, Carol. 1993. Virility and Domination in Early Twentieth-Century Vanguard Painting. In Carol Duncan (ed.), The Aesthetics of Power: Essays in critical art history, 81–108. Cambridge/New York: Cambridge University Press. Eschmann, Michael. 2012. Antiquare auf Rädern. Eine kleine Reminiszenz an den Berliner Bücherkarren. Marginalien Zeitschrift für Buchkunst und Bibliophilie 1: 59–64. Ewers, Hanns. 1924. Edle Steine und Menschen Schicksale. Die Woche 26(15): 364. Flaubert, Gustave. 1874. La tentation de saint Antoine. Paris: Charpentier et Cie. Foucault, Michel. 1979. What Is an Author? In: Josué V. Harari (ed.), Textual Strategies: Perspectives in Post-Structuralist Criticism, 141–160. Ithaca: Cornell University Press. Freiherr von Münchhausen, Börries. 1924. Im Künstlerzimmer. Die Woche 26(18): 465. Fuhrmann, Dietmar and Klaus Jestädt. 1992. Die Zeichen- und Malschule des Vereins der Berliner Künstlerinnen. In Dietmar Fuhrmann (ed.), Profession ohne Tradition: 125 Jahre Verein der

384

H. Camilla Smith

Berliner Künstlerinnen. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Volume 2, 353–366. Berlin: Kupfergraben [exhibition catalogue]. Giraudoux, Jean. 1947. Bella – Histoire des Fontranges. [Illustré de 22 bois originaux de Renefer]. Paris: Librairie Arthème Fayard. Gundlach F. C., Uli Richter and Katja Aschke. 1993. Berlin en vogue: Berliner Mode in der Photographie. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur. Tübingen: Wasmuth [exhibition catalogue]. Havemann, Anna. 2011. Gertrude Sandmann. Künstlerin und Frauenrechtlerin. Berlin: Hentrich & Hentrich, Centrum Judaicum. Hübner, Johannes. 1978. Nachruf. In Eberhard Roters, Lothar Klünner, Max Delbrück, Johannes Hübner and Stanislaw Kubicki (eds.), Jeanne Mammen 1890–1976, Jeanne-MammenGesellschaft. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin, 149–153. Stuttgart-Bad Cannstatt: Cantz. Jankuhn, Rudolf. 2004. Ursula Vehrigs (1893–1972). Ein Malerinnenleben. Berlin: Edition Kunsthog. Jacob, Mary Jane and Michelle Grabner (eds.). 2010. The Studio Reader: On the space of artists. Chicago/London: University of Chicago Press. Jochens, Birgit, Sonja Miltenberger, and Heike Stange. 2011. Von Haus zu Haus am Kurfürstendamm: Geschichte und Geschichten über Berlins ersten Boulevard. Berlin: Text. Verl. Ed. Jones, Caroline. 1996. Machine in the Studio. Constructing the postwar American artist. Chicago: University of Chicago Press. Kinkel, Hans. 1978. Begegnung mit Jeanne Mammen. In Eberhard Roters, Lothar Klünner, Max Delbrück, Johannes Hübner and Stanislaw Kubicki (eds.), Jeanne Mammen 1890–1976, Jeanne-Mammen-Gesellschaft. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin, 93–105. Stuttgart-Bad Cannstatt: Cantz. Kollwitz, Käthe. 1952. “Ich will wirken in dieser Zeit”. Auswahl aus den Tagebüchern und Briefen, aus Graphik, Zeichnungen und Plastik. With an introduction by Friedrich Ahlers-Hestermann. Berlin: Gebr. Mann. Kosch, Wilhelm [entry ‘Hermann Krehan’]. 1960. Deutsches Theater-Lexikon: biographisches und bibliographisches Handbuch. Volume 2, 1096. Klangenfurt: F. Kleinmayr. Krausse, Anna-Carola. 2006. Lotte Laserstein 1898–1993. Leben und Werk. Berlin: Dietrich Reimer Verlag. Krehan, Hermann and Jeanne Mammen. 1927. Der Maler und sein Modell. Die Woche 29(31): 31– 34. Krenzlin, Ulrike. 1992. “Auf dem ernsten Gebiet der Kunst ernst arbeiten” zur Frauenausbildung im künsterlischen Beruf. In Dietmar Fuhrmann (ed.), Profession ohne Tradition: 125 Jahre Verein der Berliner Künstlerinnen. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin. Volume 1, 73–87. Berlin: Kupfergraben [exhibition catalogue]. Kuby, Erich. 1978. Jeanne, die Freundin. In Eberhard Roters, Lothar Klünner, Max Delbrück, Johannes Hübner, and Stanislaw Kubicki (eds.), Jeanne Mammen 1890–1976, JeanneMammen-Gesellschaft. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin, 115–121. Stuttgart-Bad Cannstatt: Edition Cantz. Leistenschneider, Carolin. 2010. Ich möchte eine Mönchskutte haben: Askese im symbolistischen Frühwerk von Jeanne Mammen. Stuttgart: Ibidem-Verlag. Lütgens, Annelie. 1991. Nur ein Paar Augen sein: Jeanne Mammen, eine Künstlerin in ihrer Zeit. Berlin: Dietrich Reimer Verlag. Lütgens, Annelie. 1997a. The Conspiracy of Women: Images of city life in the work of Jeanne Mammen. In Katharina von Ankum (ed.), Women in the Metropolis: Gender and Modernity in the Weimar Culture, 89–105. Berkeley/Los Angeles/London: University of California Press.

Questioning bohemian myth in Weimar Berlin

385

Lütgens, Annelie. 1997b. Jeanne Mammen or the Art of Disappearing. In Jörn Merkert, Marga Döpping, Lothar Klünner, Klara Drenker-Nagels (eds.), Jeanne Mammen 1890–1976. Monographie und Werkverzeichnis, 9–26. Berlin/Köln: Berlinische Galerie/Wienand Verlag. Lütgens, Annelie. 2010. Vorwort. In Carolin Leistenschneider, Ich möchte eine Mönchskutte haben: Askese im symbolistischen Frühwerk von Jeanne Mammen. Stuttgart: Ibidem-Verlag. Lütgens, Annelie, Eckhart Gillen and Diether Schmidt. 1989. Zone 5: Kunst in der Viersektorenstadt 1945–1951. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur. Berlin: D. Nishen [exhibition catalogue]. Mammen, Jeanne and Hans Thiemann. 1979. Jeanne Mammen, Hans Thiemann. Jeanne-MammenGesellschaft, Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin. Staatliche Kunsthalle Berlin. Berlin: Die Kunsthalle [exhibition catalogue]. Manigold, Anke. 1986. Der Hamburger Maler Friedrich Ahlers-Hestermann, 1883–1973: Leben und Werk. Hamburg: Verlag Verein für Hamburgische Geschichte. Merkert, Jörn. 1997. Vorwort. In Jörn Merkert, (ed.), Jeanne Mammen 1890–1976: Gemälde, Aquarelle, Zeichnungen. Berlin/Köln: Berlinische Galerie/Wienand Verlag. Merkert, Jörn, Marga Döpping, Lothar Klünner, Klara Drenker-Nagels. 1997. Jeanne Mammen 1890–1976. Monographie und Werkverzeichnis. Köln: Wienand. Meskimmon, Marsha. 1999. We Weren’t Modern Enough. Women Artists and the Limits of German Modernism. London/New York: I. B. Tauris. Miltenberger, Sonja and Birgit Jochens. 2011. Von Haus zu Haus am Kurfürstendamm. Berlin: Textpunkt Verlag. Münchhausen, Borries Freiherr von. 1924. Im Künstlerzimmer. Die Woche 18(26): 465. Münzer, Kurt and Jeanne Mammen. 1927. Licht und Schatten. Die Woche 29(22): 629–632. Noun, R. Louise. 1994. Three Berlin Artists of the Weimar Era: Hannah Höch, Käthe Kollwitz, Jeanne Mammen. Iowa: Des Moines Art Center [exhibition catalogue]. O’Brien-Twohig, Sarah and Keith Hartley. 1992. Otto Dix 1891–1969. London: Tate Gallery [exhibition catalogue]. O’Doherty, Brian. 2007. Studio and Cube: On the relationship where art is made and where art is displayed. New York: Buell Center/FORuM Project. Osborne, Max. 1930. Jeanne Mammen Austellung bei Gurlitt. Vossische Zeitung, 04/11/30 unpaginated. Berlin. Ortmanns, Gudrun. 2010. Das Berlin des Robert Walser. Berlin: Edition A. B. Fischer. Reinhardt, Georg. 1991. Jeanne Mammen 1890–1976. Ein Lebensbericht. In Georg Reinhardt, Jeanne Mammen. Köpfe und Szenen 1920–1933, 81–112. Kunsthalle Emden, Stiftung Henri Nannen, 28/4–23/6 [exhibition catalogue]. Reinhardt, Hildegard. 2002. Jeanne Mammen: Das symbolische Frühwerk 1908–1914 – les tribulations de l’artiste. Ephraim Palais Berlin. Berlin: Jeanne Mammen Gesellschaft [exhibition catalogue]. Reuter, Ursula. 1992. Künstlerinnen der ersten Stunde. In Dietmar Fuhrmann (ed.), Profession ohne Tradition: 125 Jahre Verein der Berliner Künstlerinnen. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin, Volume 1, 170–175. Berlin: Kupfergraben [exhibition catalogue]. Rogoff, Irit. 1991. The Anxious Artist. Ideological mobilisation of the self in German Modernism. In: Irit Rogoff (ed.), The Divided Heritage. Themes and Problems in German Modernism, 116– 147. Cambridge/New York: Cambridge University Press. Roters, Eberhard. 1978. Jeanne Mammen – Leben und Werk. In Eberhard Roters, Lothar Klünner, Max Delbrück, Johannes Hübner and Stanislaw Kubicki (eds.), Jeanne Mammen 1890–1976. Herausgegeben von der Jeanne-Mammen-Gesellschaft in Verbindung mit der Berlinischen Galerie Berlin, 9–93. Stuttgart-Bad Cannstatt: Cantz.

386

H. Camilla Smith

Roth, Joseph. 2003. Café des Westens. In: Michael Hofmann (trans.), Neue Berlin Zeitung reproduced in What I Saw: Reports from Berlin 1920–1933. London: Granta Books. [Cited extract first published as an individual report in 1923]. Rowe, Dorothy. 2006. Representing herself. Lotte Laserstein between subject and object. In Christina Schönfeld (ed.), Practicing Modernity: female creativity in the Weimar Republic, 68–88. Würzburg: Königshausen & Neumann. Rürup, Reinhard. 2008. Schicksale und Karrieren. Gedenkbuch für die von den Nationalsozialisten aus der Kaiser-Wilhelm-Gesellschaft vertriebenen Forscherinnen und Forscher. Göttingen: Wallenstein. Scheffler, Karl. 1920. Rezension der Sintenis-Ausstellung in der Galerie Gurlitt. Kunst und Künstler. Illustrierte Monatsschrift für Kunst und Kunstgewerbe 18: 184. Berlin: Bruno Cassirer. Schwormstädt, Felix. 1924. Berlin um Mitternacht. Die Woche 26(15): 365–368. Szatmari, Eugene. 1927. Das Buch von Berlin. Was nicht im Baedeker steht. Bd. 1. München: R. Piper & Co. Täuber, Rita, Ute Scheub and Gertrude Cepl-Kaufmann. 2004. Femme Flâneur. Erkundungen zwischen Boulevard und Sperrbezirk. Exhibition Catalogue Verein August Macke Haus Bonn. Bonn: Verein August Macke Haus. Viebig, Clara. 1924. Die Woche und die Frauen. Die Woche 26(14): 348. Völker, Klaus. 2010. Kabarett der Komiker: Berlin 1924–1950. München: Edition Text + Kritik. Walser, Robert. 2012 [1910]. Berlin and the Artist. Trans. by Susan Bernofsky In Jochen Greven (ed.), Berlin Stories. New York: New York Review Books. Waterfield, Giles (ed.). 2009. The Artist’s Studio. London: Hogarth Arts, Compton Verney. Weisgerber, Albert. 1909. Der Maler. Jugend 17: 387. Züchner, Eva. 1992. Frauen Freunde und Kunstfreundinnen. Zwei Portrait-Ausstellungen am Ende der zwanziger Jahre. In Dietmar Fuhrmann (ed.), Profession ohne Tradition: 125 Jahre Verein der Berliner Künstlerinnen. Exhibition Catalogue. Berlinische Galerie, Museum für Moderne Kunst, Fotografie und Architektur, Berlin. Volume 1, 259–269. Berlin: Kupfergraben.

Carey Jewitt

16 A multimodal lens on the school classroom Abstract: This chapter shows how attention to multimodal communication, the use of modes, gesture, images and technologies allows research to denaturalise classroom interaction to examine the discourses of the classroom, teaching and curriculum knowledge. It points to the range and reach of multimodal texts and how these unfold across time, to illustrate how the practices, interaction and space of the classroom are constituted multimodally in ways that are significant for teaching and learning. Following a brief introduction to multimodality, the chapter provides three illustrative case studies from the school Science classroom, the English classroom, and the ‘digital’ classroom, to explore how teachers and students mobilize a range of multimodal resources. The chapter points towards how knowledge and pedagogy are ideologically shaped and materially realized through multimodal semiotic work. It concludes that education needs to attend to the specificity of how the visual is configured and elaborated and the pedagogical and ideological work it is used to achieve.

1 Introduction The chapter provides an illustrative case study from the school Science classroom to explore how teachers and students use a variety of multimodal resources to construct a narrative. The second case study turns a multimodal lens on the design of the English classroom, to demonstrate that educational spaces – a school classroom, museum or gallery exhibition – can also be understood as a set of modal affordances and constraints. The third case study stays with the English classroom to highlight the effects of digital technologies on the remediation of classroom time, space, and practices. Each of the short case studies presented in this chapter demonstrates the need to attend to teaching as a highly visual/multimodal practice. Throughout, the chapter points towards the ways that knowledge and pedagogy are ideologically shaped multimodally with respect to allowing space for students versus domination of the official curriculum criteria, for creativity or not, which contributes to the processes of teaching and assessment as either hidden and naturalised, or more open, as well as issues of a ‘modernisation agenda’.

2 The visual and multimodal classroom Until relatively recently, where image is acknowledged in educational settings (and research) it was often celebrated for its potential to interest and motivate learners

388

Carey Jewitt

rather than its contribution to establishing knowledge and supporting learning. This marginalisation of the visual is a paradox, as educational spaces are highly visual and multimodal environments and learning and teaching draw heavily on a range of visual and other non-linguistic texts, artefacts and resources. This chapter argues that attention to how visual and multimodal forms feature within education is essential to understanding how curriculum knowledge is shaped, and the classroom practices that underpin learning and teaching in the secondary school. Multimodality attends to the visual in configurations across gesture, gaze, body posture, sound, writing, music, speech and so on (Jewitt 2009). The assumption that meanings are made – distributed, interpreted and remade – through many representational and communicational resources (modes) is key to multimodality. From this perspective all modes are understood as partial and each as contributing to the production of knowledge in distinct ways. No single mode stands alone in the process of making meaning. Rather, each plays a discrete role in the whole: hence the need to attend to visual forms within the broader multimodal ensemble in which they are situated. These multimodal ensembles structure our encounters and the circulation of multimodal artefacts in educational spaces actively work to define and fix their associated social and cultural norms. Multimodality draws attention not only to artefacts and materials but also to practices of looking and other non-verbal interaction and their part in the orchestration of the social relations in learning environments as well as the constitution of the student or learner.

3 Multimodal texts, and textual cycles in the classroom Most classrooms include one form or another of visual display. Classroom displays are often discussed in terms of the ways in which they might create an attractive environment for learning. From a more critical perspective, display can be charged with naturalizing the classroom environment and with making opaque the exercising of power (Foucault 1991). These displays, brought into the lesson by the teacher, can relay the curriculum and disciplinary rules of the school. They provide a framework of what is to be learnt and what is valued both of which shape how learners are expected to act and behave. Visual displays and the arrangement of the classroom can therefore be understood as a teaching tool, a medium to communicate desired qualities and expectations, in a manner that is to be lived by the students. A multimodal perspective can be used to understand the classroom as a complex sign in which the resources of space, furniture, visual display, and so on are discursively organized. A wide range of multimodal materials (e.g. textbooks, work sheets, objects and models, drawings, websites) are produced, mobilised, circulated and consumed

A multimodal lens on the school classroom

389

within classrooms. Children and young people regularly make multimodal artefacts including drawings, three-dimensional models, PowerPoint presentations, photographs, digital videos, websites and so on. Up until the last decade or so, these have received little attention – with images relegated to mere illustration. However, there is increased interest in the role of the visual and multimodal in children’s learning with its potential for engaging with the visual aspects of writing and reading, as well as students’ production of multimodal models and digital multimedia materials (Jewitt and Kress 2003). Images and visual objects, visual experiences and practices are common in the classroom and have a central role in how knowledge is presented, supports dialogue and interaction between learners. In school science, for example, teachers use image and visual technologies to represent scientific ideas in powerful ways that make objects, phenomena and relationships visible (Kress et al. 2001). Observation and drawing are, and have been for many years, central to the teaching and learning of science. The Geography classroom draws on maps and a broad range of visual and spatial representations, including those made by students – most recently using digital representations with GoogleEarth and other Geographical Information Systems (GIS) as well as mobile technologies. Even in the English classroom, where common sense would have it that language is what really matters, teaching and learning are realised visually (Kress et al. 2005). Having sketched the multimodal character of teaching and learning the chapter presents a series of three case studies drawn from different research projects to show how multimodal texts, artefacts and technologies are embedded in classroom practices and interaction and how detailed attention to the use of modes, gesture, image etc. can enable researchers to denaturalise and reveal the hidden ideologies and discourses of the classroom and curriculum. In this first section I draw on an example from the school Science classroom to explore how teachers and students mobilize a range of multimodal resources to construct a naturalized narrative of blood circulation that speaks to the history of the subject and demonstrate the analytical potential of attending to teaching as a highly visual/multimodal practice.

4 The multimodal textuality of the Science classroom: Blood circulation The orchestration of modes in the classroom is linked to curriculum subject matter and historical pedagogic practices. That is the kinds of images used, how they are used, how image is combined with other modes and so on is different in the school Science classroom than it is in the school English classroom, for example. In other words texts in context connect with histories, assumptions, and ideological underpinnings of a curriculum subject beyond the boundary of the classroom – and in

390

Carey Jewitt

Fig. 1: The teacher’s manipulation of the model.

this way what is counted as knowledge becomes naturalised through mundane texts and artefacts. This illustrative example focuses on a short (5 minute) excerpt of the textual cycle of a year nine school science lesson on blood circulation in an Inner London school. Over the course of the lesson the scientific entity blood circulation was constructed from the interweaving of meanings created by the teacher’s shifting between modes and texts. The teacher starts with a spoken account of the circulation of blood, which he transposes through gesture onto an image on the whiteboard behind him. He then gives a more complex spoken description of the cycle of blood as a ‘double loop’, blood going to the lungs, and to the rest of the body. He added this second loop to the drawing, transforming it. Having described the route of the blood, the teacher placed a model of the upper part of the human body on the bench in front of him (Figure 1). The teacher handled the model heart while talking about the blood’s movement through the heart, and the contracting of the heart’s muscles. Cyclicity was represented through the teacher’s ‘animation’ of the model, and by his use of the verbs “goes”, “comes”. The teacher then directed the students to look at an image in their textbook. He used his finger to indicate the circulation of the blood on the image, providing a multimodal (visual, verbal, and gestural) summary of the lesson. The teacher then instructed the students to complete a series of activities from their textbook, focusing on the functions of the different organs, in particular the heart, in the process of blood-circulation.

A multimodal lens on the school classroom

391

Standing in front of the image on the white board he introduced the model of the insides of the human body, using his own body as a model for the outside: it mediated the transition between the image on the board and the model on the bench, in a layering effect: his body overlaid the image; the model overlaid his body. Speech, gesture and model were fully integrated. Speech provided an explanation, gesture indicated the players, it acted out and dynamized the verbal account, the model provided an analytical representation of the body as the physical location, and the relationship between the parts and the whole. The textbook image offered a detailed visual summary of all that had happened. During the lesson the teacher’s use of different modes repositioned the students in relation to the human body. The teacher’s talk and his use of the image he had drawn required the students to imagine themselves inside the body. His gesture and use of his body as a “model” asked them to imagine themselves as observers outside of the body. Through his gesture of the pumping action of the heart, he “took” the students a layer deeper – they were “in” the heart “seeing the movement of the valves”. With the model, the teacher pointed out and handled the organs, a dissection reminiscent of an autopsy. This required the students to imagine the model as once moving and placed them in the role of medical students looking in at the dead body – observers of the transformation from the process of experiencing the body as an external entity to experiencing it as an “internal entity”. The textbook image required the students to imagine the context of the rest of the body. Finally, the homework which the teacher set, to write “as a blood cell” of their journey around the body, required them to “move into” the cell itself – to “become” a blood cell. In this way all of the modes worked to achieve the rhetorical repositioning of students, and the construction of the entity the human body. The visual and gestural resources mobilised by the teacher required the students to undertake specific kinds of imaginative work in order to make sense of different aspects of the circulatory system. The image required them to imagine the movement of blood around a circuit; to envisage the size and position of organs, and the relationships between them, in a highly abstract fashion; to think of the organs in a circuit, and of the cells as collected up into one place as an organ; to transform the spatial relationship between the heart and lungs from a three-dimensional relationship into a two-dimensional relationship: front became up, behind became down. The teacher’s use of gesture and his body required the students to imagine what goes on “beneath the skin”. The model required the students to think about elements of the cycle (for instance, the size and position of the organs) differently to that suggested by speech and image. The teacher’s actions with the model realized movement and direction of actions, which were named in his speech, and required envisaging the agent of this action – the blood. Each mode both repositioned the students in relation to the human body, and demanded different cognitive work of them. The complex meaning of this part of the science curriculum was realized through this interweaving of modes.

392

Carey Jewitt

Throughout the lesson the students were required to shift their view of the body between that of an internal to an external entity. The internal view of the body as process realised through the teacher talk and image. Followed by the external view of the body as location realised through the teacher’s use of gesture. Followed by the internal view of the body as movement, heart as pump realised through gesture. Switching again to an external (distant) view of the body as parts realised through the manipulation of the model. Returning finally to an internal view (deeper still) or the blood as an entity – a cell, as was the focus of the students written homework. Each “shift” in view (from internal to external and back again) involved the student metaphorically re-entering the body at a “deeper” level. The direction of these moves can be, I want to suggest, interpreted as a pedagogic stance to the process of science as immersion and observation in science. While the visual elements of this multimodal lesson articulate a specific scientific stance, they also point to the visuality of the science classroom: something that the technology appears to being used to heighten across the curriculum and which is discussed later in this chapter. Here we can see the links and connections across modes and texts in the classroom, the intertextuality of texts that characterise the textual cycle in the school classroom. I am arguing that the canonical text, whether it is the textbook image of the circulation of blood or Macbeth, is produced through the actions of teacher and students – that is the multimodal ensemble that it is embedded within. I am also drawing attention to how talk and writing are not always the dominant forms of communication in the classroom and even when they are they do not “stand alone”. Taking a multimodal perspective to the textuality of learning contexts shows that a linguistic focus has the potential to produce a distorted view of the role of language in learning. In the next section I turn a multimodal lens on the design of learning spaces, classrooms and visual displays in the English classroom to demonstrate the potential of this approach to reveal how the ideologies of a subject or school become sedimented in the material affordances and constraints of the classroom space.

5 The multimodal space and visual displays of the classroom Just as the practices of the teacher discussed in the previous section can be seen as an orchestration of multimodal resources infused with historical practices and articulated through artefacts with particular constraints and affordances, so the design of an educational space – a school classroom, museum or gallery exhibition – can be understood as a set of modal affordances and constraints. The classroom, like any other physical, constructed site, is designed with built-in values and purposes that contribute (not determine) to shaping the work and behaviour

A multimodal lens on the school classroom

393

of the teachers and students who occupy it: values of student centred learning, the authority of the teacher or the canon, or the promotion of collaboration or individual personalisation. In other words “the pedagogical order of the classroom is mediated in its spaces” (Grosvenor 1999) at the same time as being constantly transformed by those who act in and on it. These multimodal configurations provide a material trace of social practices shaped through use over time: in part built into the material structure of a building and in part made through the interactions that occur within it (Grosvenor et al. 1999). These spaces operate “as a set of pathways and constraints, facilitating and frustrating parts of the educational mission” (Prosser 2007: 15). Many different, often conflicting, missions infuse education, including the development of independent learners, collaborative learners, digital competence, examination success, flexible learners and so on, and these translate into particular configurations of space and potentials (or constraints) on students’ movement in that space. Pathways are indicated through the arrangement of the space, the furniture used, the objects that are made available, as well as socially through norms and rules concerning who is permitted to enter and exit or move, where they are allowed to speak and so on. Focusing on the visual construction of learning environments raises three questions for education. First, what do the texts, objects and furniture in a learning environment represent as a part of a curriculum subject? What is included? What resources are students offered for their learning, and how are they positioned, “physically” and conceptually, in relation to this knowledge? Secondly, how are visual and other non-symbolic forms used to represent the subject? What is the visual used to explore? Which modes are foregrounded and by what means? Thirdly, what is given importance or made central through the resources of visual display and spatial-temporal arrangements of a learning environment?

6 The English classroom as a multimodal sign The illustrative example below demonstrates how teachers’ situated design of learning contexts can be examined from a multimodal perspective and how it enables central questions to be addressed on the relationship between discourses of subject knowledge and learning contexts. These two examples are drawn from the larger study Production of school English (Kress et al. 2005).

6.1 English as competence in language communication My first example is a classroom in an English department (of which the classroom focused on here is a typical teaching room) that is preoccupied – and not without reason – with getting students, mainly from ethnic minorities and working-class families, to perform well in GCSE exams at age 16. It assumes that it has to privilege

394

Carey Jewitt

a particular version of “official knowledge” that is curriculum based knowledge, with an emphasis on written language, formal grammar, and Shakespeare. This is materialized in the many official texts on the wall, which are mounted, framed and placed in prominent positions. The teacher selects and adapts the official policy (e.g. criteria and grades, key words in examination papers) and mediates these into sets of instructions for students that focus on examination. The ideological view of English presented is one of tradition and language and school-based knowledge that stands apart from English as it features in the diverse lived language experiences and concerns of the students. This creates a strong ideological barrier between English as a subject and the classroom, and the students’ use of language in and outside of the classroom. The layout of the classroom in neatly arranged group tables is another echo of departmental (and national) policies on group work and differentiation. In the process it conveys to students that they are the kind of people who need a basic level of assistance to cope with assessment demands. In this sense it provides an example of a strategy for distributing what is seen as “cultural capital” in what are seen in both economic and cultural terms as impoverished places. Distribution, in this context, involves rationing. The teacher’s use of visual resources and her arrangement of these in the classroom displays combine to produce English as a coherent, “bounded” and univocal space filled with language. The posters and displays are primarily written and word-processed and only a few include images. The content of the displays concerns writing genres for writing, instructions on how to start a sentence, and so on. There is only one poster with several images on it: these are of Leonardo DiCaprio in the Baz Luhrmann film production of Romeo & Juliet. The images are derived from popular culture but are in a very literal sense cut down and tightly framed, so that the very thing that appears to have engaged many young people in Lurhmann’s filmic reshaping of Shakespeare’s play – the display of masculinity (through the use of rap, dress and imagery of gang fighting) – is the subject of the teacher’s critique, and occupies a minor position in a classroom which is a strongly “boundaried” place clear, sparse, and uncluttered, like the version of English that is produced there. In this classroom “What English is” is constituted multimodally as a completely explicit entity. The curriculum is displayed, the language of examination is literally written on the walls, the deadlines are clear. The politics of the English curriculum as articulated in this classroom is “access to language”: language is, literally, made available as a resource. An instance of this is the posters on the cupboard doors that contain dictionaries and thesauruses that list ways of starting a sentence. The visual display of the classroom fits into an instructional genre. Policy in the form of the National Curriculum is selected, condensed and strongly framed. Everything else has vanished. The curriculum is so pervasive that it is difficult to overlay it with anything. This genre of instruction positions the teacher and student in a particular relation to the production of English, beyond the obvi-

A multimodal lens on the school classroom

395

ous power relation that they are in. The teacher represents English as a series of competencies that the students are to learn. The classroom display reminds them why they are there, why they must work. They are to aim for the teacher’s idealized student text.

6.2 English and the life-worlds of students The furniture and spatial arrangement of my second illustrative example classroom can be seen as an example of the organisation of the “progressive primary school” finding its way into the secondary school English and redefining the place to learn and the future of the school (Stuebing 1995). The teacher has brought soft furnishings into her classroom: a carpet, several soft swivel chairs, a soft armchair, and a sofa creating a home-like space in her classroom. She regularly re-organised the furniture and desks in the room, and the arrangements were always non-symmetrical. The teacher’s design of her classroom can also be seen as a rejection of formal secondary school arrangements and an attempt to create a learning environment in which the curriculum is, as she commented, “only a small part of what is actually going on within the whole learning experience”. The home furnishings of the classroom all serve to (attempt to) distance the classroom from the school as an institution to produce the classroom (and, by implication, “school English”) as a “pseudo-liminal” space that is neither school nor “out of school”, bordering on the style of a “teenager’s room”. This is echoed in the posters stuck quite literally all over the walls, overlapping, angled, struggling for space in a jumble of film and music icons of youthful despair and early death (Kurt Cobain, James Dean, Tupac Shakur and more). Alongside this cultural mix the teacher has displayed students’ texts, which are highly visual collages drawing on pop and film icons to represent the characters and relationships from, for example, Macbeth. These elements of the display re-construct the relationship between writing and text within the domain of school English. In contrast to the use of popular culture as a visual annotation of aspects of the English curriculum in the first classroom I discussed, the teacher has here re-framed English as popular culture (and in doing so she has re-framed popular culture as English). In short, her visual display serves to position English and popular culture as interconnected forms of “knowledge”. The meanings of popular culture are foregrounded and the position of English in this is backgrounded (changed). The display reflects the teacher’s comments on school English as being about “different ways of reading the world and reading the text”. The complex of signs realised via the visual display of the classroom is one of English as a space which collects the life-worlds of students. Here the world of popular culture is pervasive, and the curriculum is laid over it gently, offering the students a filter of “school English” for viewing this everyday world in another

396

Carey Jewitt

way. The question “what is English?” is raised and the role suggested to students is one of how to explore such a question in relation to the artefacts of their worlds. These two contrastive examples show how learning contexts are built up over time and nonetheless remain fluid, dynamic spaces created through the agentive work of the teacher as she or he is situated in the web of social, cultural and historical forces that they operate within (Jewitt and Jones 2005). This opens up the idea, the potential for the re-design of context, at the same time perhaps as it demonstrates the complex difficulty of change and re-design. The examples discussed in this chapter so far have hopefully given a sense of how attention to the role of images and multimodal forms within the classroom can contribute to understanding curriculum knowledge, classroom practices, and the significance of the classroom as a learning environment. In the following section the effects of digital technologies on the visual features in the school classroom with attention to the remediation of classroom time, space, and practices are briefly discussed.

7 Digital technologies in the school classroom The attention to the visual, spatial, and gestural that a multimodal perspective can provide is a useful lens through which to investigate the effects of digital technologies in the school classroom. In this example the focus is on the changes in the interaction of teachers and students in the English classroom, its forms and functions, and the resources used to mediate its constitution. These areas of concern are examined using video data from the Production of school English study collected in 2000 (Kress et al. 2005) and contrasted with video data from a study of the use of Interactive Whiteboards (IWBs) (Moss et al. 2007). Both videos show a secondary-school English lesson on poetry taught by the same teacher in the same classroom. In the earlier lesson the teacher is using an Overhead Projector and in the more recent lesson she is using an Interactive Whiteboard. Poetry and persuasive language remain the curriculum focal points in both lessons; however there are significant changes in the landscape of school English with respect to the pedagogic organisation of the classroom and roles, the display of texts, and the process of textual analysis, each of which is described and discussed below. (The findings described in this illustrative example resonate across the larger data sets of the projects and knowledge of the contemporary English classroom.)

7.1 Pedagogic organisation of the classroom and roles The spatial-arrangement of the classroom in both lessons (and historical moments) is similar in many respects. The desks are arranged to seat students in small groups of 3–5 in a cafeteria style. Students sit around these so that they can also

A multimodal lens on the school classroom

397

face the board at the front middle of the classroom. Overall, the balance of wholeclass work to small-group work remains broadly similar. Historical comparison of the environment of the classroom, however, indicates some significant changes in the work of teacher and students in the classroom that I suggest may mark a pedagogic rhetoric of democratisation. It is important to note that I am not suggesting that the digital, visual and multimodal “are” more democratic, rather that they are strongly associated with a discourse of democratisation and as is the spatial configuration that students are placed here. That is, small groups are used to signal the provision of the opportunity for peer learning and discussions which in turn suggest an empowerment of the students and a de-centralised role for the teacher who moves to a facilitative role. Specifically changes are realized in the teacher’s use of the classroom space, the re-organisation of the collective gaze of the class, and student movement in the classroom. In both lessons the teacher’s position and movement is strongly shaped by the technology of the IWB or the overhead projector (OHP), but differently so. In the classroom with the OHP, the teacher stood by the OHP some distance from the front of the class to annotate the poem for the majority of the introduction of the poem. In the IWB classroom, the teacher stood at the centre-front of the class at the IWB to introduce the poem; she manipulated and annotated the prepared PowerPoint directly on the board. These differences effect the organisation and orientation of teacher gaze and student gaze in the classroom as well as the authority of the poem on display. In the OHP classroom, students had to choose from multiple focal points: the learning objectives written on one side the board, the poem displayed on the other side of the board, the teacher at the OHP annotating the poem, the poem – as duplicated part-text – from their students’ anthologies. The teacher was annotating in real time and thus was engaged directly with the poem for the majority of her time, looking up to call on the students for comments. That is, the gaze of teacher and student was polycentric and often cut across one another. Further, the technology of display created a separation between the practice of annotation and the annotated text – what the teacher was doing could not be viewed by the students; what was displayed was a disembodied outcome on the board. To establish a collective gaze on the board, the teacher had to move to sit alongside the students. In the IWB classroom, teacher and student shared a centralised gaze (except during the short period when the teacher interacted remotely with the IWB). This shift from individual to collective gaze orientates the teacher and students to the work of analysis in different ways: with the OHP, the teacher demonstrated the process of annotation and the divide between teacher and student was marked in gaze orientations and the distinct spaces of interaction these created; in the IWB classroom, gaze suggested a shared teacher-and-student position to the poem. This rhetorical reshaping is aligned with the modernisation agenda, notions of explicitness, equity, participation and ownership, all of which serve to emphasise the role of the learner.

398

Carey Jewitt

In the lesson using the OHP, the students remained in their seats throughout; in contrast, students in the lesson using the IWB came up to the front to annotate texts displayed on the IWB. Student-made texts were scanned and displayed on the IWB. The pedagogic space of the front of the class (the IWB) was thus positioned as a space of action and display for students and teacher. In both lessons, students responded to the teacher’s questions and prompts, but the number and extent of student responses was higher in the lesson using the IWB. Of course, it would have been possible for the teacher to use the OHP to support student participation, but not within the same timeframe and with the ease afforded by her use of an IWB. Furthermore, the social discourses that technologies are embedded in can make the options for interaction more obvious in a new social environment: the OHP is embedded in rhetorical discourses of authoritative styles of presentation, while the IWB is embedded in rhetorical governmental and commercial discourses of interaction and participation. The lesson with the OHP involved the teacher in more real-time “board work” than the later lesson. Much of the text displayed on the IWB was teacher-made and produced in advance. Text-making and preparatory work have been a key element of teaching and created layers of activity across times and contexts regardless of the technology used. However, digital technologies have changed some of these practices and with it many of the texts used in the contemporary UK English classroom. The contemporary conditions described in this paper foreground the work of text design, and place previously “in-class work” outside of the classroom – into personal preparation periods or beyond the school into the home. Thus, some of the work of teaching that in the past would have been visible in the classroom becomes “hidden, behind-the-scenes work”. This paper suggests that this has the potential to strengthen and naturalise the authority of teachermade texts, to quicken the pace of lessons to the rhythm and flow of teacher and curriculum time and to further restrict space for student interjection. These different temporal and spatial forms have consequences for the work of the teacher and the students in the classroom. For example, with no extended time where the teacher writes on the board or the need to erase the contents of the board, moments where a teacher has their back to the class are eradicated. One consequence of this is the removal of spaces for students to behave badly, but also the reduction of regular “informal-open” classroom spaces for students to think, reflect, and chat that can now be filled with curriculum. The change thus results in both gains and losses for teaching and learning. In UK English classroom teachers are now expected to actively “design” all the spaces in the classroom.

7.2 The display of texts The rhetoric of “democratisation” is suggested by the contrast between the display and function of student texts in the classrooms. In the OHP lesson, canonical

A multimodal lens on the school classroom

399

English texts and teacher-made texts were displayed on the front and sidewalls of the classroom, with some student texts displayed on the back wall. Student-made texts were incorporated into the active pedagogic space of the classroom using the IWB. The teacher scanned student responses to the poem, including student poems, and displayed these immediately on the IWB. The student texts displayed on the IWB became an object of discussion that both the teacher and fellow students manipulated and annotated. A shared malleable text was created that opens up new pedagogic possibilities that can effect the configuration of authorship and authority in the classroom. The teacher’s annotation and marking of the student texts on the IWB transforms what is usually a semi-private activity into a public one. This makes both the marking criteria and process explicit. This can be understood as a multimodal version of the verbal IRF (Initiation, Response, and Feedback) process. It is a process that foregrounds the importance of assessment and examination in the everyday practice of the classroom. The sense of what can and needs to be displayed has changed in the time between the two lessons, as have the technologies of display. Using the OHP, writing and speech were in the foreground; with the use of the IWB, however, image, colour and layout have, alongside writing, become central to the pedagogic resources of the classroom. The changing semiotic landscape of the classroom has an effect on the curriculum and the pedagogic function of texts – what texts are presented, how texts are presented, and what can be done with them. The use of the OHP supported the display of the written poem with line numbers, in a photocopy from a book. Using the IWB the poem was integrated with images downloaded from the Internet in a teacher-made PowerPoint across several slides. More generally, comparison of the two data-sets, and observations of school English in 2010, suggest that changes in the relationship between image, speech and writing have extended and embedded in the English classroom. It is now common for English teachers (although there may be generational differences in this) to show a clip of digital video (often via YouTube) or to display an image – often downloaded from the Internet – to offer a route into a concept. Teachers frequently use PowerPoint presentations to present their argument, they annotate texts visually or they connect to a webpage. The use of image is also prevalent in students’ work in English, with the use of clipart, digital photographs – taken by students or downloaded from the Internet – designed as PowerPoint presentations and project work, both in class and out of school for homework. This reshapes the work of the teacher and the student. The contemporary teacher is involved in the pedagogic design of digital multimodal texts that were rarely seen in 2000. The student analysis of written “imagery” in poems is now often re-mediated by actual images. What is to be learned and how it is to be learned is being reshaped by teacher and student uses of the multimodal potentials of digital technologies. This prompts the question: what are the social and educational implications – the gains and losses?

400

Carey Jewitt

7.3 Textual analysis A significant difference for textual analysis is that the starting point for the introduction and the analysis of the poems is different. This difference appears to be underpinned by changes in the use and function of writing, speech and image in the classroom. The starting point for textual analysis in the OHP lesson was provided by a whole-class discussion of the poem title and students’ use of the dictionary to look up title words. The starting point for textual analysis in the IWB lesson was provided by a whole-class discussion of the image accompanying the poem displayed on the IWB and a whole-class brainstorming activity. The role of the dictionary in textual analysis has changed from its central position. In the era of the IWB, the meaning of words are anchored and defined through images downloaded from the Internet that the students are asked to match to words in the poem such as “Congregation”. Comparison suggests a general move towards capturing and displaying the work and opinions of students: from ephemeral talk to the concretised display of talk. In 2000 there is a firmer boundary between the work of reading the poem and analysing the poem than appears to be the case now (Jewitt, Bezemer and Kress 2011). The poem is read aloud twice before analysis for meaning begins in 2000, whereas the IWB provides a new visual starting point for analysis – the illustration is introduced before the poem. The potentials for meaning made possible by changes in the socio-technological environment of the classroom raise new decisions for teachers and students with implications for curriculum and pedagogy that the proposed project will investigate. These differences mark a significant trend for English and literacy as much as for thinking about both. There is the use of image rather than of writing as a starting point for discussion of the poem; the Oxford English Dictionary (OED) has disappeared from the students’ tables; with the IWB, images rather than the OED are used to define words considered difficult for the students. The poem is now displayed in “chunks” spread across the IWB screens – as words, lines, or title. In 2000 the poem was displayed on the OHP as a whole; that was then slowly “carved up” in a process of interrogation. With the IWB, the teacher works with the whole class and students interact with the meaning of the poem right from the start of the lesson – in matching image and word, for example – and in answering questions. In 2000, the teacher drew a strong boundary between “reading” the poem and “analysing” it. In the contemporary classroom, the 2000 boundary between speech, writing and image has been reconfigured, as has the boundary between reading and analysis. Boundaries of many kinds in the English classroom can be argued have either disappeared, reconfigured or relocated.

A multimodal lens on the school classroom

401

7.4 Summary: the remediation of the interactions of teachers and students Over the past decade, the interactions of teachers and students have, as this comparative analysis from a (micro) multimodal perspective demonstrates, changed in some significant ways (and remained the same in equally significant ways). The re-mediation of the teacher and student interaction via technological, social and policy change is, this chapter argues, central to how school English is constituted. Table 1 summarises the comparative elements to provide a snapshot comparison across complex changes, and the stabilities that persist in the interaction of teachers and students, its forms and functions, and the resources used to mediate its constitution.

Tab. : Comparative summary of elements of teacher and student interaction with OHP and IWB. OHP episode

IWB episode

Student position and movement

Seated in small groups No movement from seats

Seated in small groups Walking to front of class to interact with IWB

Teacher position and movement

Front of classroom, near OHP Or walking around the class to visit small groups

Front of classroom, near IWB Or side of classroom near computer

Teacher and student gaze

Poly-centric

Mono-centric

Teacher board work

Real-time

Prepared before lesson Real time annotation

Pedagogic organisation

Lesson pace

Increased

The display of texts Texts displayed on board

Canonical poem

Canonical poem Illustration accompanying poem Student poem Marking criteria and process explicitly demonstrated

Modes in use

Writing and speech

Image, colour and layout, and writing

Student texts

Displayed on back wall Private individual space of exercise book

Displayed in the active pedagogic space on IWB and annotated and manipulated Private individual space of exercise book

Rhetoric of display

Rhetoric of authority

Rhetoric of democratisation

402

Carey Jewitt

OHP episode

IWB episode

Textual analysis Starting points for the introduc- Whole class discussion of the tion and analysis of the poem poem title

Whole class discussion of the image accompanying the poem displayed on the IWB Whole class brainstorming activity

Meaning of words

Central role of dictionary: students’ use of the dictionary to look up title words

Anchored and defined through images displayed on the IWB

Relationship/boundary between the reading and analysis of the poem

Poem is read aloud twice before analysis for meaning begins Strong boundary between reading and analysis

Analysis for meaning begins with the illustration “before” the written poem is introduced or read Weak boundary between reading and analysis

The poem as a text

Poem displayed on the OHP as a whole Poem slowly “carved up” in the process of interrogation Distributed into  parts across the groups

Poem cut up into “chunks” spread across the IWB screens – as words, lines, or title.

Modes analysed

Writing

Image and writing

These findings resonate across further case studies of the IWB in use in the classroom. This technology can be broadly described as offering three distinct modal starting points and semiotic spaces for contemporary English: a written starting point, a visual starting point or a multimodal starting point (see Moss et al. 2007). Each of these modal starting points engages with distinct semiotic resources, modal affordances and involves the “interest” of the teacher in the multimodal orchestration of school English. The representational decisions that teachers make influence the kinds of demands made on the learner as well as the practices that they engage with which might lead to learning. Visual starting points offer relatively open ways into the production of school English that connect with the students’ experiences of imagery. Teachers’ discussions of images displayed on the IWB are common starters used to explore poetry and plays that were once introduced as written texts. While new routes and entry points are provided, the resources and experiences that the teacher draws on are different than if she were to start this discussion with a written or oral text – a shift in authority, a connection with a variety of experiences and knowledge outside of the cannon – new ways into the play. The ways in which imagery and writing appear in the English classroom has changed considerably over the past decade (Jewitt, Bezemer and Kress 2011).

A multimodal lens on the school classroom

403

Through investigation of the relationship between image, writing, action and layout they show that image and layout are increasingly meshed in the construction of content. This is exemplified by the exponential increase (3000 %) in the average number of images per page in English textbooks over the past couple of decades (Bezemer and Kress 2008). Their analysis also shows that the quality and function of image or animation in English textbooks is in a state of change. Image and word frequently attend to discrete aspects of meaning with images or as object of reflection. Increasingly concepts are introduced, established and analysed visually and writing is brought into new relationships with, or even exchanged for, visual and multimodal forms of representation (Jewitt 2008).

8 Concluding comments This chapter has argued for the importance of the visual and other non-linguistic or multimodal forms for learning and teaching. It has shown that the textual cycle of the classroom is highly visual and multimodal and that attention to all modes of representation and communication is beneficial to understanding the complexity of teaching and learning. It has pointed to some of the roles of the visual in the classroom –as aid-memoire, narrative backdrop, canonical representation, imaginative starting point, definition or anchor, and so on. The ways in which speech and gesture are firmly woven into the use of visual resources for meaning making in the classroom has been shown. The ways in which teachers design and activate visual displays and the spatial organisation of classrooms as meaningful subject discourses to frame learning has been illustrated. Finally how the use of digital technologies has impacted on the visual forms appearing in the contemporary classroom in new ways has been touched on. The visual is no longer – if indeed it ever was – an illustrative adjunct to word; images are used fully in representation; they are integrated in multimodal ensembles. This chapter suggests that the relationship between the visual and learning in the classroom is changing in ways significant for learning and teaching. Digital technologies (the IWB and technologies including mobile and tangible technologies) and access to the Internet shape how information and knowledge are created, recreated, mobilised and shared in the classroom. Increasingly image provides the first step in accessing topics and issues including the effects and uses of language. What the effect of this is on learning have barely begun to be recognised, as Kress argues – the world “shown” is not the same as the world “told” (Kress 2003). Education needs to attend to the specificity of how the visual is configured and elaborated and the pedagogical and ideological work it is used to achieve.

404

Carey Jewitt

References Bezemer, Jeff and Gunther Kress. 2008. Writing in Multimodal Texts: A social semiotic account of designs for learning. Written Communication 25(2): 166–195. [Special Issue on Writing and New Media]. Foucault, Michel. 1991. The Foucault Reader. London: Penguin. Grosvenor, Ian. 1999. On Visualising Past Classrooms. In Ian Grosvenor, Martin Lawn and Kate Rousmaniere (eds.), Silences and Images: The social history of the classroom, 83–104. New York: Peter Lang. Jewitt, Carey and K. Jones. 2005. Managing Time and Space in the New English Classroom. In Martin Lawn and Ian Grosvenor (eds.), Material Cultures of Schooling, 201–214. Oxford: Symposium Books. Jewitt, Carey. 2008. Technology, Literacy, and Learning: A multimodal approach. London: Routledge. Jewitt, Carey, Jeff Bezemer and Gunther Kress. 2011. Annotation in School English: A Social Semiotic Historical Account. National Society for the Study of Education Yearbook 110(1): 129–152. Jewitt, Carey and Gunther Kress. 2003. Multimodal Literacy. New York: Peter Lang. Jewitt, Carey (ed.). 2009. Routledge Handbook of Multimodal Analysis. London: Routledge. Kress, Gunther, Carey Jewitt, Jill Bourne, Anton Franks, John Hardcastle, Ken Jones, and Euan Reid. 2005. English in Urban Classrooms: A multimodal perspective on teaching and learning. London: Routledge Farmer. Kress, Gunther, Carey Jewitt, Jon Ogborn and Charalampos Tsatsarelis. 2001. Multimodal teaching and learning. London: Continuum Press. Moss, Gemma, Carey Jewitt, Ros Levačić, Vicky Armstrong, Alejandro Cardini and Frances Castle. 2007. The Interactive Whiteboards, Pedagogy and Pupil Performance Evaluation: An evaluation of the Schools Whiteboard Expansion (SWE) Project: London Challenge. London: DfES. Stuebing, Susan. 1995. Redefining the Place to Learn. Paris: OCED.

Ross P. Garner

17 Celebrating and critiquing “past” and “present”? The intersection between nostalgia and public service discourses in BBC1’s Ashes to Ashes Abstract: This chapter employs a social constructionist approach to theorise nostalgia as a discursive form and analyses how versions of this discourse become constructed within fictional television programmes. Rather than subscribing to zeitgeist-based understandings, which link nostalgia to identifiable underlying social anxieties, the discussion expands upon Paul Grainge’s (2000) work in this area and approaches mediated forms of nostalgia as textual constructions which are impacted upon by institutional structures including the individual remit and imagined audience of a television channel. To explore these issues, the chapter provides a case study of the BBC1 series Ashes to Ashes (2008–10). Here it is argued that the two forms of nostalgia articulated through the programme’s signifying elements (named here as ‘affective’ and ‘present-orientated’ nostalgia) help to fulfil the channel’s public service responsibilities by simultaneously celebrating and critiquing its construction of ‘the 1980s’ from the perspective of the present.

1 Introduction This chapter examines how forms of nostalgia are constructed through the representation and comparison of differing historical periods in fictional television, using the BBC programme Ashes to Ashes as a case study. Ashes to Ashes aired for three series on BBC1 – the BBC’s flagship television channel in the UK – between 2008 and 2010 and centred around DI Alex Drake (Keeley Hawes), a police psychologist from 2008 who, after being shot during a confrontation between herself and mentally-unstable criminal Arthur Leyton (Sean Harris), finds herself transported back in time to 1981 (the programme’s second and third series moved to 1982 and 1983 respectively). Across the three series, Drake’s narrative goal is to escape her entrapment in the 1980s and return to her daughter, Molly (Grace Vance), in the present. However, whilst stranded in “the past”, the character gradually warms to the initially alien environment she encounters despite its many negative connotations (see below). Each episode of Ashes to Ashes combines elements of the police drama series, such as the episodic investigation and resolution of crimes (Chapman 2009: 11– 13), with conventions from what William Burling (2006: 8) names the “temporal contrast” time travel plot. Temporal contrast narratives involve characters journey-

406

Ross P. Garner

ing from an originating spatiotemporal location to an-Other past (or future) time period(s) and foregrounds the differences between the worlds represented. The temporal contrast plot structure therefore overlaps with an understanding of nostalgia as “a periodising emotion” (Tannock 1995: 456) which opposes “[t]he Beautiful Past and the Unattractive Present” (Davis 1979: 18) as, through juxtaposing the narrative worlds constructed, forms of nostalgia can be identified. However, just as in the programme’s parent series Life on Mars (BBC/Kudos 2006–7), within Ashes to Ashes “[t]he idea of home is a shifting site” (Holdsworth 2011: 109) as neither “the 1980s” or “the present” are singularly romanticised. Expanding from this point, I here discuss how two contrasting forms of nostalgia, which I have named “presentorientated” and “affective” nostalgia, are constructed. “Present-orientated nostalgia” crystallises around Drake’s desire to return to the “present” whereas “affective nostalgia” counteracts this by valuing the “past” for the solidarity and camaraderie Drake shares with her colleagues (see also Castiglia 2000). Affective nostalgia centres around the character of DCI Gene Hunt (Philip Glenister), who quickly became established as a British TV icon and represents an old-fashioned form of (British) masculinity where women are “birds” and female police officers are “plonks”. Hunt’s character also incorporates discourses of working class taste (he has a love of steak and chips and a pint of bitter) and the then-industrialised North of England.1 His decidedly un-PC nature draws attributes from the “new lad” type – “a central figure in […] popular television in the UK in the mid-1990s” (Hollows 2003: 231) that is characterised by “a refusal of the “political correctness” associated not only with feminism but also the “nurturing” new man” (Hollows 2003: 232) and so appealed to this audience profile (see also Tincknell 2010: 164) as well as others (see below). This chapter argues that the tension generated between contrasting Hunt/“past” and Drake/“present”, and the associated constructions of nostalgia, results in both represented time periods being simultaneously celebrated and critiqued. Exploring Ashes to Ashes in terms of nostalgia raises a number of questions; issues concerning how representations of historical periods impact upon contemporary audiences’ understanding of the “past” (and the “present”) are raised, for example. Additionally, from a semiotic perspective, television has been characterised as “a messy thing” (Seiter 1992: 45) as the medium constructs meaning through a complex mixture of audio and visual signifiers alongside narrative and/ or generic codes (see also Creeber 2006). As Ellen Seiter (1992: 43–44) recognises, “[u]nlike language, television does not conveniently break down into discrete elements or building blocks of meaning; it has no equivalent of an alphabet”. Addressing this point thus requires considering how television constructs nostalgia through combining different signifiers. To manage these concerns, this chapter employs social constructionist theories (Gergen 1999; Edley 2001) to approach nos-

 See also Curzon (2012).

Celebrating and critiquing “past” and “present”?

407

talgia as a discursive form that is constructed according to the social, historical, and especially in the case of this chapter, institutional contexts where it is articulated. This approach is useful because television has also been theorised as a product of “complex … different histories – disciplinary, national, economic, technological, legislative” (Brunsdon 1998: 95), alluding to how different factors arising from its production context(s) impact upon its strategies for creating meaning. Adopting an institutional perspective towards how television encodes nostalgia therefore requires this chapter to move beyond established theoretical perspectives concerning nostalgia and visual media that are indebted to Frederic Jameson’s (1991) arguments concerning pastiche and la mode retro. Instead, I build upon Paul Grainge’s (2000) discussion of how production discourses impact upon nostalgic modes on television in two ways: firstly, whereas Grainge analyses an example from the commercially-funded US television industry, I examine how Ashes to Ashes’ status as a BBC product, produced under a public service remit, impacts upon constructions of nostalgia. Secondly, whereas Grainge addresses how “past television is re-contextualised, through textual, generic, personal and institutional practices” (Holdsworth 2011: 5), I instead discuss the “textual strategies” (Johnson 2005: 7) that a contemporary television series employs for representing different time periods. The chapter therefore argues that Ashes to Ashes’ constructions of nostalgia intersect with institutional discourses requiring primetime BBC1 programmes to attract “coalition” and “quality” audiences.

2 Nostalgia: Styles, modes and contexts Nostalgia can be a difficult concept to comprehend because of the variety of academic disciplines where it has been discussed (see Boym 2001: 3–18; Sprengler 2011: 11–37). As Michael Pickering and Emily Keightley (2006: 922) argue, “[d]eployment of the category [of nostalgia – RPG] in such different academic domains as psychology, history and cultural studies creates difficulties of application and reference within a coherent explanatory framework”. Relating specifically to television, existing research discussing nostalgia has split into two areas. One body of work has approached television and nostalgia from an audience perspective and demonstrated how nostalgic memories of old television shows provide a cornerstone for contemporary constructions of generational, gendered and/or fan identities (see Veith 2004; Hallam 2005; Keightley 2011). Elsewhere, Frederic Jameson’s (1991) complaints against nostalgia as a visual style arising from the historical conditions of postmodern societies have provided the dominant theoretical framework for text-based discussions of televisual nostalgia. Robin Nelson’s (1997: 74– 88) analysis of British TV drama series Heartbeat (ITV/Yorkshire Television 1992– 2009) demonstrates this point. By noting the programme’s inclusion of 1960s

408

Ross P. Garner

music and fashions alongside its intertextual allusions to British TV police dramas from the period, Heartbeat emblematises arguments regarding how historical understanding in postmodern societies has become replaced by the rise of “‘historicism’, namely, the random cannibalisation of all the styles of the past” (Jameson 1991: 18). From this perspective, social perceptions of “past” historical periods are drawn from a pastiche of “the glossy qualities of the image, […] the attributes of fashion” (Jameson 1991: 19) and deemed nostalgic due to their selectivity and rooting in mediated images of the “past” rather than historical accuracy. John Caughie (2000: 211) also employs Jameson’s arguments when critiquing television costume dramas such as Pride and Prejudice (BBC 1995). He argues that in these series “History becomes the present in costume, showing us only human continuities and lingering generalities of tone and style – the seduction of the image – without the formal distance and the historical particularity which might enable us to experience difference and change”. This line of analysis is identifiable in other work on period TV drama (see Barker 1994; Cooke 2003: 166–169) and mobilises Jameson’s arguments to posit that, in these genres, television viewers experience “History” as historicism in that differences between “then” and “now” become erased and representations of the past instead reflect contemporary attitudes and values. Uniting both of these positions is the assumption that nostalgia’s “visual modality […] reduc[es] […]articulation of details” (Abousnnouga and Machin 2011: 183) alluding to a “crisis” in historical representations in TV drama (see also Anderson 2001: 19) as heavily intertextual, stylised and/or presentinformed representations of “past” time periods take precedent.2 Not all period-set fictional television has been dismissed through Jameson’s arguments. Scholarly analyses of the American television series Mad Men (AMC/ Lionsgate Media 2007–), which is set in early 1960s New York and reflects upon issues such as gender roles and the emerging consumer society from a contemporary perspective, demonstrate this point. Gary R. Edgerton (2011a: xxvii) has posited that Mad Men “never wallows in the feel-good revisionism of works such as American Graffiti [… or] Happy Days” – the latter two examples exemplifying postmodernist arguments concerning how TV and film provide ahistiorical and stylised representations of “past” eras which primarily reference mediated images of the “past” (see also de Groot 2009: 9). Instead, Mad Men is seen to offer an “incarnation of [the] 1960s [… which] enacts a shrewd critique of the period’s values, one that has little patience with nostalgia” (Butler 2011: 55). However, despite these attempts to position Mad Men away from nostalgia’s tainted associations of providing simplified and romanticised representations of historical peri-

 Overlapping with this point, Jerome de Groot (2009: 3) has discussed “ a crisis of historical legitimacy” arising from “a shift in access … that allowed the individual to … conceptually and materially … engage with the “past” in a more direct fashion” due to technologies including the Internet and television.

Celebrating and critiquing “past” and “present”?

409

ods, overlaps between Mad Men and the aforementioned postmodernist critiques of mediated “pasts” are identifiable. Despite Jeremy G. Butler’s (2011: 64) disassociation between the series and “nostalgia”, his assertion that Mad Men “refers to the style of theatrical films” from the 1960s echoes Jameson’s argument that contemporary media representations of past historical eras are indebted to popular images of that period. Thus, although some scholars have attempted to position Mad Men away from nostalgia’s negative connotations, overlaps with Jameson’s arguments remain identifiable in this instance (see also Tudor 2012) and further demonstrate the ubiquity of his ideas when discussing nostalgia through specific TV case studies. What is disappointing about these discussions of Mad Men is that scholars haven’t linked the series’ ambivalence towards nostalgia to its production context. Mad Men is commissioned by AMC – a niche-orientated American cable channel which “produce[s] programming that [i]s edgy, sophisticated, and […] innovative” (Edgerton 2011b: 7) with a view to attracting “quality” audiences that appreciate complex representations (see Nelson 2007: 28). Locating Mad Men’s negotiated attitude towards nostalgia within its institutional setting suggests that the series’ “textual strategies” (Johnson 2005: 7) can be read as decisions “put in place at the point of production” (Johnson 2005: 7) by the programme’s producers to satisfy the imagined tastes of “quality” audiences. Discussing textual constructions of nostalgia in TV series as “production strategies” (Johnson 2005: 7) overlaps with Paul Grainge’s (2000) analysis of televisual nostalgia modes. Grainge adopts a different perspective towards TV nostalgia in that, rather than analysing the socio-cultural consequences of how historical periods become represented, he focuses upon how television institutions reappropriate “past” TV shows to activate nostalgia in contemporary audiences. Grainge therefore distinguishes between nostalgic “moods” and “modes”, arguing that the former account for mediated forms of nostalgia via using a “Zeitgeist model” (Collins 1995: 7) of interpretation as nostalgia becomes accountable “to a governing narrative or cultural temper” (Grainge 2000: 27). Helen Piper (2011: 413) demonstrates this line of argument by linking the increasing use of archived material and “past” TV shows on British television to “the more general surge of public interest in memory and memory practices which some have seen as characteristic of late modernity”. Similarly, Amy Holdsworth (2011: 2) has offered a variation upon this position since, despite arguing for “medium-specific interrogations of memory”, a framework indebted to a perceived industrial zeitgeist is identifiable: “[f]ear that television […] will inevitably itself disappear might be seen to lead towards an increased obsession with television memory and the nostalgia for television past. Anxieties about television […] might be seen to run parallel to present anxieties regarding history and memory in general”. (Holdsworth 2011: 4) Approaching televisual forms of nostalgia as “modes” instead recognises that “[w]hile the production of nostalgia may have grown in tandem with a sense of

410

Ross P. Garner

cultural crisis” (Grainge 2000: 27) – whether this be industrial or, following Jameson (1991), linked to social shifts concerning the ubiquity of consumer capitalism – cases of mediated nostalgia “cannot be reduced to this explanatory model” (Grainge 2000: 27). Nostalgic modes are constructed in specific media contexts and so need to be analysed using: an approach that resists a critical reduction where nostalgia modes become the reflex result of anxieties and dissatisfactions with the present. As a cultural style, nostalgia has developed in accordance with a series of cultural, demographic, technological, and commercial factors that have made “pastness” an expedient and marketable mode (Grainge 2000: 33).

Grainge (2000: 30) therefore demonstrates that US cable station “The Nostalgia Network” directly targeted “post-49-year-olds” through scheduling “lifestyle programs with acquired shows like The Love Boat, The Rockford Files, and The Streets of San Francisco” (Grainge 2000: 30). The channel’s nostalgic mode is accountable “first and foremost, not in relation to a political project or pervasive cultural temper, but to the massive expansion in cable television during the 1980s” (Grainge 2000: 30). Addressing the specificities of how production contexts construct nostalgic modes in response to “market segmentation and media syndication” (Grainge 2000: 31) implies that strategies can be employed to target constructions of televisual nostalgia towards specific audience profiles. Grainge’s approach needs expanding for a variety of reasons: not only does it solely focus on one example within the context of the commercially-funded American television industry, it also focuses upon how “old” television series become repositioned through constructions of nostalgia. This leaves room for further development: how applicable are Grainge’s arguments to TV series representing “past” historical periods? Moreover, given this article’s use of Ashes to Ashes as its case study, an important influence upon the series’ articulation(s) of nostalgia would be BBC1’s interpretation of its public service values and the impact these have on shaping content. The next section therefore outlines some of BBC1’s public service responsibilities.

3 BBC1’s public service responsibilities Purist notions of “public service” are difficult to sustain within a contemporary context as the BBC “has to operate and compete within a highly commercialised industry characterised by large media conglomerates” (Johnson 2012: 107). At a national level, the Corporation adapts to this environment by utilising “commercial” practices to accentuate and extend areas of its public service charter such as communicating its core values to audiences through such practices as developing individuated brands for each of its television channels (Johnson 2012: 96–101). The BBC’s TV services thus all presently respond to public service values concern-

Celebrating and critiquing “past” and “present”?

411

ing “universal service; […] the servicing of the interests of minorities including children, in addition to mainstream audiences; the reflection of national culture and identity; and the provision of quality programming which encompasses a preparedness to innovate and to not be driven by audience size” (Debrett 2009: 809). These values are then inflected differently according to the remit of each commissioning channel (see Smith and Steemers 2007: 48, on the intended audiences for BBC3 and BBC4 programming). BBC1’s service licence, which is issued by the independently-elected BBC Trust, defines “public service” through discourses including universality, “quality” and addressing a diverse range of audiences: BBC1’s remit is to be the BBC’s most popular mixed-genre television service across the UK, offering a wide range of high quality programmes. It should be the BBC’s primary outlet for major UK and international events and it should reflect the whole of the UK in its output (BBC Trust 2008: 1).

BBC1 is institutionally-required to produce programmes offering “broad demographic appeal” (Johnson 2005: 133) which engage and reflect the diversity of the UK population by uniting these divergent audience profiles into a “coalition” audience (Johnson 2005: 141). BBC1’s understanding of coalition appeal is subject to further differentiations, however, in terms of scheduling. Whereas early “Saturday-evening schedules are […] traditionally constructed for family viewing, consisting of game shows, variety, soap operas and mainstream drama series” (Johnson 2005: 133), Ashes to Ashes’ scheduling on weekday evenings (Thursdays, Tuesdays and Fridays for each series) at 9pm classifies the programme as post-watershed drama. This enables the series to include “material considered unsuitable for children” (Johnson 2005: 133) as well as complex approaches to narrative, representation or genre (Bignell 2004: 270) that subsequently overlap with “quality” imagined audience tastes.3 Arising from BBC1’s requirement to attract a coalition audience comprised of differing generations, genders, classes and races, Ashes to Ashes’ constructions of nostalgia must intersect with these demands. Linking Ashes to Ashes’ encoding(s) of nostalgia to the series’ production context suggests the suitability of employing a social semiotic methodology due to the theory’s stress upon how “[s]ign-makers and interpreters select from [semiotic] affordances according to their communicative needs and interests in a given context” (Abousnnouga and Machin 2011: 178). The next section develops these points by summarising this article’s methodology.

 Within scholarly analyses of television, the term “quality” is much debated and can be explored from a variety of perspectives as Nelson’s (2007) study demonstrates. This article solely explores the term “quality” as it is defined within the institutional context of BBC1 (see BBC Trust 2008).

412

Ross P. Garner

4 Methodology This chapter employs social constructionist theories to consider forms of nostalgia as discourses. Social constructionism has been employed across a variety of academic contexts (see Ray 2000: 19) and is premised upon rejecting “the positivist position [… where] language is unproblematically representational, a mirror of ontological reality” (Ray 2000: 26). Social constructionist theories critique positivism’s central assumption “that words can correspond to the world as it is” (Gergen 1999: 20) by instead questioning “how the mind could know its own contents (self-knowledge)” (Gergen 1999: 14) and positing that all concepts are “specific to particular traditions – lodged in culture and history” (Gergen 1999: 14). Social constructionism thus highlights how any term’s meanings – such as “nostalgia” – are produced through language’s social, cultural and historical operations (Gergen 1999: 45). Gestures towards a social constructionist understanding of nostalgia are identifiable in preceding work discussing the concept: Pickering and Keightley (2006: 934) recognise that “the meanings of nostalgia are multiple because culturally and historically specific” whilst different national and generational inflections of nostalgia have been demonstrated elsewhere (Schwarz 2009; Leow 2010). A social constructionist approach to nostalgia therefore requires recognising that the concept can be defined and analysed as a discursive form or an “interpretive repertoire”: “[a] relatively coherent way […] of talking about objects and events in the world. In discourse analytical terms, they are the “building blocks of conversation”, a range of linguistic resources that can be drawn upon and utilised in the course of everyday social interaction” (Edley 2001: 198). Whilst nostalgia resists concrete definitions due to its diffusion across a variety of historical, national (and media) contexts, recurrent linguistic resources which subjects and/or cultures employ are identifiable that enable articulations of nostalgic discourse. This discursive approach has been gestured towards elsewhere in discussions of televisual nostalgia (Kompare 2005: 101–125) and overlaps with Grainge’s (2000) approach by recognising that discourses of nostalgia constructed through the signifying elements of television programmes are impacted upon by institutional factors. To analyse nostalgic discourses in Ashes to Ashes, a methodology combining textual analysis with social semiotic assumptions concerning meaning production will be employed. To start, it is necessary to recognise that within semiotics “[t]he simplest way of marking difference is, of course, by means of a binary opposition” (Hall 1997: 31). Nostalgic discourses employ binary oppositions to construct meaning through opposing discursively-distinct constructions of “past” and “present”. Pam Cook (2005: 4) recognises this point by stating that nostalgia “is predicated on the acknowledgement that the past is gone forever. Nostalgia plays on the gap between representations of the past and actual past events, and the desire to overcome that gap and recover what has been lost”. Pickering and Keightley (2006: 936) challenge the suitability of this approach to studying mediated nostalgia,

Celebrating and critiquing “past” and “present”?

413

arguing that “[n]ostalgia in popular culture doesn’t necessarily operate with the dichotomous before/after scenario […] associated with classical sociology and critical theory”. Yet Pickering and Keightley overlook how genre categories, such as Ashes to Ashes’ employment of “temporal contrast” (Burling 2006: 8) conventions, may construct meaning through directly opposing representations of “past” and “present” and so enable constructions of nostalgia. Thus, by examining how Ashes to Ashes contrasts representations of “past” and “present”, constructions of nostalgia can be explored. Holdsworth’s (2011: 109) aforementioned observation concerning “shifting sites of home” provides an entry point for analysing Ashes to Ashes’ two divergent discourses: “present-orientated” and “affective” nostalgia. These discourses become identifiable through examining the intersection of a range of semiotic resources including audio-visual techniques such as camera movement/positioning and sound, construction of the mise-en-scene via choice of colour(s) and costuming, and the programme’s multiple narrative levels. However, recognising the contextual nature of how meanings are produced (Hodge and Kress 1988: 5; Chandler 2007: 217–218), once Ashes to Ashes’ constructions of nostalgia have been identified the discussion then outlines the production context to analyse how these discourses intersect with BBC1’s requirement to attract coalition and “quality” audiences.

5 Analysis 5.1 Present-orientated nostalgia: othering “the 1980s” 5.1.1 Establishing the temporal shift Immediately following Drake’s backwards temporal movement to 1981, Ashes to Ashes utilises a variety of semiotic devices to indicate the narrative shift between time periods and begins inscribing negative connotations to its reconstruction of the 1980s. This is partly achieved by mobilising the device of “allotting [a character] more […] screen time” (Nelson 1997: 41) than any other to align the audience with that character’s point-of-view. Episode 1.1 of Ashes to Ashes demonstrates this as audiences are first introduced to Drake in the context of the “present” and then follow her from being shot through to realising that she has apparently travelled back in time to 1981. From the outset, it is Drake’s perspective that audiences are aligned with. Following Drake’s shooting, episode 1.1 combines performance and audio-visual signifiers to foreground the location’s dislikeable nature. Firstly, Hawes/Drake’s costume changes from a stylish and understated dark blue suit, connoting the professional values associated with the character’s present-day career as a police psychologist, to a glaring (and revealing) bright red top and white fur coat which suggest displacement to a less desirable spatiotemporal loca-

414

Ross P. Garner

Fig. 1: Hawes’ confused facial expressions.

tion. The temporal shift also involves a change in colour palette from muted blues and greys (see also Holdsworth 2011: 108) to one incorporating popular memories of the early 1980s as a period “of lost bearings [… and] the thrill of decadence” (Sewall 1997: xix) as signified by the tasteless juxtaposing of bold and pastel shades.4 When read together, these changes to the mise-en-scene imply the undesirability of the world that Drake has arrived at. Hawes’ facial expressions further support this reading as these express disbelief and displeasure with her immediate surroundings (see Figure 1). The objectionable nature of Drake’s location is then further reinforced by the roaming, point-of-view shot that follows: Drake stumbles disorientated through a crowd of 80s yuppies as they sing along to Ultravox’s diegetically-audible hit “Vienna” and make sexually-suggestive comments to her (Figure 2). This sequence is also continually intercut with shots of Hawes visibly denoting Drake’s disbelief and disgust at the alien location that the character has awoken in.5 Ashes to Ashes also uses dialogue to communicate differences between “past” and “present”. Later in episode 1.1, when Drake arrives of the 1981 CID office for the first time, DS Ray Carling (Dean Andrews) directs a range of sexist comments towards Drake’s appearance that indicate the “entrenched misogyny” (Nelson 2010: 144) that Ashes to Ashes associates with this historical period. Unaware that

 Ashes to Ashes used these associations ironically by promoting its second series under the tagline of “The Eighties are back. And it’s criminal”.  See Tincknell (2010: 167) and Curzon (2012: 74) on how Life on Mars also establishes 1973 as an alien location immediately following the character’s temporal movement.

Celebrating and critiquing “past” and “present”?

415

Fig. 2: Drake’s disorientated point-of-view.

Fig. 3: DS Carling’s objectification of Drake.

Drake is in fact his new superior, Carling mistakes her for a prostitute and comments firstly on her “crackin’ pair of puppies” and then further degrades Drake by making similar objectifying comments as she inspects (or, as shot from his perspective – see Figure 3, seductively caresses) Hunt’s red Audi Quattro. Across both of these sequences, myriad semiotic resources establish Ashes to Ashes’ representation of 2008 as the location “to which [the protagonist] is desperate to return”

416

Ross P. Garner

(Holdsworth 2011: 107–108) and thus “it is [the character]’s central ambition to “get home” that makes the series, first and foremost, a nostalgic narrative” (Holdsworth 2011: 107). The combined effect of these elements is the construction of a discourse of “present-orientated nostalgia”. This discourse is “overtly rooted in the diegesis” (Cardwell 2002: 123) as Drake yearns to return to the narrative world that has been previously glimpsed earlier in the episode. Present-orientated nostalgia therefore foregroundes the differences between the programme’s spatiotemporal worlds through elements including visual design and the socio-cultural attitudes that are articulated through a range of upper- and working-class characters. The combined effect of these strategies therefore encourages audiences to identify with Drake’s desire to return to the “present”.

5.1.2 The present as “a less discriminatory Britain” Ashes to Ashes’ strategies for representing 1980s Britain do not construct “an idealised past” (de Groot 2009: 205) for audiences in the manner associated with adaptations of “classic” literature or period-set programmes such as Heartbeat (Nelson 2012: 28). Instead, Ashes to Ashes constructs present-orientated nostalgia by “visualiz[ing] a bleak environment of urban decline” (Nelson 2012: 20–21) on an episodic basis whilst also articulating this discourse through each of the programme’s selfcontained and continuous narrative levels (Chapman 2009: 8–9 outlines these in greater detail). For example, the series extends Robin Nelson’s (2007: 177) observation that, within Life on Mars’ episodic investigations, “interest is not focused upon the resolution but on the means to the end by a comparison of twenty-firstcentury police methods with those of 1973” into its representation of the 1980s. Ashes to Ashes’ first episode begins by establishing a “narrative context of contemporary policing that is structured through the “politically correct” discourses of […] gender parity” (Tincknell 2010: 164), and where psychological profiling and a caring attitude towards criminals achieves results (Tincknell 2010: 164). From here, each episode narrates clashes between Drake’s “modern” approach to both policing and wider socio-cultural issues (Episode 1.2 makes explicit that Drake is a feminist and has had lunch with Germaine Greer) with those located in the 1980s. Drake’s values and approach to police work directly contrast with those of “the decidedly non-politically correct Gene Hunt” (Chapman 2009: 12) since numerous minority groups including rape victims (episode 1.3), the mentally ill (episode 1.7) and undercover female officers (episode 3.4) are subjected to mistreatment and intimidation at the hands of Hunt et al. 1980s policing in Ashes to Ashes is frequently “horribly contrasted with the present day” (de Groot 2009: 205) as Drake is left “reeling with shock and an overwhelming sense of disbelief at what [s]he is witnessing” (Cook and Irwin 2012: 81) as Hunt and his team beat up witnesses and display insensitivity towards socially-marginalised factions. As the monologue that

Celebrating and critiquing “past” and “present”?

417

accompanies each episode’s opening credits imparts, Drake continually has to “fight” against the 1980s, its dominant approaches to policing and the wider “casual racism …and macho aggression” (Nelson 2010: 144) encountered on a day-today basis. Alongside othering “the 1980s” through its representational strategies, Ashes to Ashes also constructs present-orientated nostalgia by providing audiences with a “dramatic message […] that a huge amount of progress has been made […] in creating a fairer, less discriminatory Britain” (Cook and Irwin 2012: 82) via its narrative oppositions. However, rather than singularly validating Drake’s contemporary values, Ashes to Ashes instead counteracts present-orientated nostalgia through providing a “dialectical interrogation of […] norms of police activity” (Hills 2012: 105), as well as wider socio-cultural values, from both spatiotemporal locations. In other words, through juxtaposing both “past” and “present” attitudes towards issues including gender and community, the programme highlights the positive and negative aspects of each time period. This leads to the programme constructing a contrasting discourse of what I name “affective nostalgia” that celebrates the 1980s from the perspective of the present in terms of its levels of friendship and loyalty.

5.2 Affective nostalgia: evaluating the “present” 5.2.1 “Sticking with The Guv” Despite employing a range of narrative and representational strategies for othering its reconstruction of “the 1980s”, Ashes to Ashes complicates this perspective by also inviting its audiences to reflect upon what values have been lost from our society. However, although the programme uses popular memories of the decade such as its music and fashion styles, the series, as should be evident, constructs “nostalgia for the 80s” in a more complex manner than purely recalling popular culture artefacts. Instead, through articulating a discourse of affective nostalgia around Hunt’s character, Ashes to Ashes “invit[es] us […] to view a little more critically the fragmentation and privatisation of experience in twenty-first-century culture” (Nelson 2012: 29). Affective nostalgia has been conceptually alluded to, yet not defined through this nomenclature, in previous cultural studies work on memory. Christopher Castiglia (2000) alludes to the concept in his study of how gay men reconstruct homosexual experiences during the 1970s. He recognises that, despite the problems associated with recalling this period (such as how promiscuity led to consequences including the AIDS epidemic), respondents still construct fond memories of the era for its associated levels of camaraderie, intimacy and friendship. Affective nostalgia is therefore a nostalgic discourse that recognises the limitations of the period being remembered but still employs nostalgia’s “interpretive repertoire” to fondly reconstruct the levels of warmth and community experienced between subjects. Rather than articulating affective nostalgia for “the

418

Ross P. Garner

1980s” in general, though, Ashes to Ashes displays sensitivity towards socio-cultural shifts associated with 80s Britain by acknowledging this as the period when post-Second World War “consensus values” became deconstructed by identity politics including racial and gendered discourses (Thompson 2007: 33). The series instead regularly constructs affective nostalgia around Hunt. Various social groups, including rude boys (Episode 1.7), radical feminists (Episode 1.4) and students (Episode 2.3) as well as sections of the police force (see DI Jim Keats’ (Daniel Mays) role in series three), are represented to explicate “that Gene has become a “dinosaur”” (Chapman 2009: 17) and that his ideals are those of white, working class males rather than society in general. Despite (in some cases brutally) highlighting the shortcomings of his methods, Ashes to Ashes’ three series represent Hunt’s integrity and unity with his colleagues as something to be protected. Whether in the face of wider police corruption in series two, or juxtaposed with DCI Keats’ modernising attitudes throughout series three, Hunt is continually romanticised as the last of a dying breed. Ashes to Ashes therefore recognises the character’s negative attributes but celebrates other parts of his character such as the respect he shows towards crime victims (see episode 2.5) or those he perceives as similar to himself – especially fellow Mancunians (see episode 2.1). This discourse of affective nostalgia constructed around Hunt is especially foregrounded in series finales: episode 3.8 literally casts him as the “good” guy by revealing him as a self-appointed saviour who guides deceased police officers away from “Hell” (Lacey and McElroy 2012: 13). Similarly, episode 1.8 partly culminates in Hunt standing up to Lord Scarman (Geoffrey Palmer) and rejecting the character’s negative assessments of the Fenchurch East station. Chapman (2009: 17) discusses this sequence, arguing that Hunt’s “impassioned speech prompts loud and spontaneous applause from Gene’s colleagues, including Alex. Ideologically, therefore, Ashes to Ashes has come full circle since Life on Mars: it now defends Gene and his approach to policing in the face of liberal reformers” (Chapman 2009: 17). Alex’s clear diegetic admiration of Hunt’s tirade in this sequence (Figure 4) ideologically suggests support for “a more timeless version of policing that that centres on […] “doing the right thing”, one that supersedes the liberal perspective of the present” (Willis 2012: 60) and further examples of merging “past” and “present” are identifiable across the series. For example, Ashes to Ashes retains Life on Mars’ method for resolving its episodic procedural narratives by having the “crime … solved through a combination of … modern psychologically oriented, scientific thinking and Gene’s more instinctive and intuitive methods” (Chapman 2009: 8); the breakthrough in the investigation of Colin Mitchell’s (Jason Haigh) murder in episode 2.6 comes from combining Drake’s psychological profiling approach with DS Carling’s knowledge of expected criminal behaviour to identify that Mitchell’s killer must have been someone close to him. “Past” and “present” approaches to policing join together and complement each other here to achieve narrative resolution. However, examples such as that

Celebrating and critiquing “past” and “present”?

419

Fig. 4: Drake’s admiration for Hunt.

seen in episode 1.8 might suggest a conservative bias operating across the series by valuing Hunt’s attitudes over those diegetically-coded as “modern” (e.g. Scarman and Keates). Reading Ashes to Ashes as ideologically conservative is too simplistic, though, as this position overlooks how discourses of present-orientated nostalgia continually critique Hunt’s methods and attitudes. Yet, where constructions of affective nostalgia become accentuated, such as in series finales, Ashes to Ashes questions contemporary liberal attitudes by making viewers consider whether, despite Hunt’s racist, sexist and homophobic attitudes, his methods are justified. In other words, the series asks audiences to consider their own values by positioning “sticking with the Guv” as the only certainty in an increasingly fragmented world.

5.2.2 Critiquing contemporary individualism Ashes to Ashes’ interrogation of “the present” extends beyond demonstrating the blindspots in contemporary police work via romanticising Hunt, though, as the series also frequently critically reflects upon Drake’s individualistic attitude (especially during the first series). This commentary can, from one perspective, be accounted for via recourse to Ashes to Ashes’ industrial status as a “spin-off” as the series “is posited on the notion that Alex knows from the outset that she is inhabiting someone else’s imaginary world. She greets her new colleagues with a breezy “Good morning, imaginary constructs” and puts finger-quotation marks around “Gene” when she speaks to him” (Chapman 2009: 16). To offer audiences

420

Ross P. Garner

variation upon Life on Mars’ established format, episode 1.1 begins by having Molly read Alex’s psychological report on Sam Tyler aloud and so establishes Drake’s familiarity with events from its parent series (reading the series from this industrial perspective also provides an explanation for why in Ashes to Ashes “Glenister's Gene […] is promoted to the star role” (Chapman 2009: 17) alongside Hawes/Drake as spin-offs frequently trade on the established popularity of key characters – see also Lacey and McElroy 2012: 12).However, Drake’s “knowingness” towards her temporal shift positions the character as a less-than-sympathetic identification point for viewers (McFarlane, 2008). This character reading is identifiable throughout Ashes to Ashes’ three series as Alex continuously adopts a laissez-faire attitude towards the consequences of her actions: she randomly sleeps with people under police protection (episode 1.2), attempts to save her parents (played by Amelia Bullmore and Andrew Clover) from dying in a bomb explosion she witnessed as a child by arresting them for drug abuse on planted evidence (episode 1.8), or conducts investigations behind Gene’s back that cast suspicion on his intentions (see series three). These actions attach connotations of selfishness to Drake’s character since, especially in the context of episode 1.8, these actions are directly contrasted with the parallel narrative strand concerning CID’s inspection by Lord Scarman that threatens Gene and her colleagues’ careers; Drake’s inducing of a fit-up emblematises the irresponsible and corrupt behaviour Scarman is trying to stamp out. Thus, despite Hunt’s negative attributes, affective nostalgia again becomes constructed around his character as audiences are invited to reflect upon the individualism motivating Drake’s actions.

5.3 Nostalgia in context: Ashes to Ashes as BBC1 product Discourses of affective nostalgia in Ashes to Ashes may rightly be read as ideologically problematic as they suggest a validation of Hunt’s questionable values. However, their constant undercutting via the present-orientated nostalgia constructed through Drake’s point-of-view means that neither discourse is ever completely vindicated. These competing discourses can be accounted for via locating them within Ashes to Ashes’ production context as addressing BBC1’s public service requirements allows for sensitivity to be displayed towards how institutional issues impact upon constructions of nostalgia in television. John R. Cook and Mary Irwin (2012) suggest the insights that industrially-focused readings of nostalgia can provide by arguing that Life on Mars: works through a […] set of internal tensions, namely a constant oscillation between the seventies as a “golden” time of freedom (compared to now) and the seventies as a time of restrictive unsophistication (also compared to now). In a sense, the different attitudes this popular drama must attract and reach on the mainstream BBC 1 channel help provide a measure of explanation for this constant oscillation. The series has clear appeal to thirty- and early-forty-something adults who […] remember growing up in the seventies: as a popular drama for the mainstream

Celebrating and critiquing “past” and “present”?

421

BBC 1 channel, however, Life on Mars also has to embrace (and not alienate) other demographics, including, for example, the youth audience, ethnic minorities, older audiences who may remember the decade with less pleasure, and so on (Cook and Irwin 2012: 85).

Ashes to Ashes’ juxtaposition of present-orientated and affective nostalgia similarly provide multiple points of identification to different audience profiles. Whilst affective nostalgia towards the 1980s and Hunt provides points of identification for male “new lads” (Chapman 2009: 15) as well as politically right-wing viewers (Lacey and McElroy 2012: 5) and some female viewers (see McElroy 2012: 121–122) admiring his unreconstructed masculinity, present-orientated nostalgia counters this discourse by constructing alternative reading positions. Negotiation between both discourses allows for imagined audiences occupying differing political dispositions, as well as those positioned through other (e.g. racialised, gendered and/ or generational) identity discourses, to be united into a coalition audience through the series’ constructions of nostalgia. As a result of BBC1’s requirement to provide “appeals to a broad audience” (BBC Trust 2008: 2), Ashes to Ashes displays ambivalence towards both of its narrative worlds by equally celebrating and critiquing its discourses of nostalgia. The series’ ongoing arbitration between nostalgic discourses is also readable as a strategy designed to target “quality” audience niches. As previously suggested, internal documentation for BBC1 states the importance of “audience approval of BBC1 and perceptions of it as high quality and innovative” (BBC Trust 2008: 9; original emphasis) and Ashes to Ashes’ continual negotiation of nostalgic discourses assists in fulfilling this criteria. Firstly, positioning Drake between narrative oppositions of “past” and “present” adds internal conflict to her characterisation which, as the above discussion of Mad Men suggests, opens the character to “quality” readings (see also Newman 2006: 23). What’s more, the negotiation between discourses of nostalgia enacted in Ashes to Ashes suggests that this is a series that has been designed to be read by audiences as “thought-provoking and […] culturally challenging” (Nelson 2012: 20). Whereas period-set “nostalgic” TV dramas do not invite audience evaluations as “quality”, partly due to their “sentimentalised treatment […] of a mythologised past” (Nelson 2012: 20), the same cannot be said of Ashes to Ashes.6 The series’ critical attitude towards both “past” and “present” instead constructs a complex reading position that intersects with “quality” audience profiles (Nelson 2007: 178–179). Rather than recalling the 1980s in a idyllic manner (Nelson 2012: 28), Ashes to Ashes displays ambivalence towards both of its spatiotemporal locations and so provides appeals to “quality” audiences by asking viewers to consider what has been “lost” from both narrative worlds. By simultaneously celebrating and critiquing “past” and “present”, Ashes to Ashes  See also Holdsworth (2011: 97) who argues in relation to “clip shows” that “nostalgia television [i]s cheap and populist programming and corresponds with the commercial safety of reproducing past successes and familiar forms”.

422

Ross P. Garner

encodes appeals to a variety of audience profiles and attempts to unite these into a coalition audience for a post-watershed BBC1 series.

6 Conclusion This chapter has developed existing discussions of nostalgia and television. Moving beyond Frederic Jameson’s theory of nostalgia, which critiques how media texts reconstruct “past” time periods from the perspective of the present for providing pastiched and/or stylised representations of “history” that are indicative of a prevailing postmodern “mood” or zeitgeist, I have instead analysed constructions of nostalgia as discourses that are articulated through, and impacted upon by, television’s production contexts. Using the BBC1 series Ashes to Ashes as a case study, this approach has identified two divergent discourses of nostalgia that simultaneously celebrate and critique the programme’s representation of both “the 1980s” and “the present”. The series’ stance towards both periods needs to be read alongside BBC1’s public service discourses so that its equal romanticisation and critique of “past” and “present” becomes accountable to constraints placed upon the programme such as its requirement to appeal to a coalition audience. As a result of Ashes to Ashes’ requirement to not alienate and offend any potential identity groups (generational, racial, gendered etc.) watching, the series constructs a negotiated stance towards its discourses of nostalgia which results in singular readings of the programme as regressively celebrating “1980s” attitudes or reinforcing contemporary social values being unsustainable. Moreover, when further explored against the programme’s institutional backdrop, this position towards nostalgia also assists Ashes to Ashes in fulfilling other public service responsibilities for BBC1 by provoking critical reflection within audiences and so attempting to secure viewer evaluations as a “quality” drama. Of course, the arguments that this chapter has provided still invite wider critical reflection concerning the role and responsibilities of a popular medium such as television in providing audience(s) with representations of “past” time periods. What I have argued here, though, is that the institutional practices structuring these representations should not be overlooked.

References Abousnnouga, Gill and David Machin. 2011. The Changing Spaces of War Commemoration: A multimodal analysis of the discourses of British monuments. Social Semiotics 21(2): 175–196. Anderson, Steve. 2001. History TV and Popular Memory. In Gary R. Edgerton and Peter C. Rollins (eds.), Television Histories: Shaping Collective Memory in the Media Age, 19–36. Lexington: University of Kentucky Press. BBC Trust. 2008. BBC One service licence. BBC Online 7/04/08. Available at: http:// www.bbc.co.uk/bbctrust/our_work/services/television/service_licences/bbc_one.html.

Celebrating and critiquing “past” and “present”?

423

Barker, Simon. 1994. “Period” detective drama and the limits of contemporary nostalgia: Inspector Morse and the strange case of a lost England. Critical Survey 6(2): 234–243. Bignell, Jonathan. 2004. An Introduction to Television Studies. London: Routledge. Boym, Svetlana. 2001. The Future of Nostalgia. New York: Basic Books. Brunsdon, Charlotte. 1998. What is the “television” of Television Studies? In Christine Geraghty and David Lusted (eds.), The Television Studies Book, 95–113. London: Hodder Arnold. Burling, William. 2006. Reading time: The ideology of time travel in science fiction. KronoScope 6(1): 5–30. Butler, Jeremy. 2011. “Smoke gets in your eyes”: Historicising visual style in Mad Men. In Gary Edgerton (ed.), Mad Men: Dream Come True TV, 55–71. London: I.B. Tauris. Cardwell, Sarah. 2002. Adaptation Revisited: Television and the Classic Novel. Manchester: Manchester University Press. Caughie, John. 2000. Television Drama: Realism, Modernism, and British Culture. Oxford: Oxford University Press. Castiglia, Christopher. 2000. The Way We Were: Remembering the Gay Seventies. In Shelton Waldrep (ed.), The Seventies: The Age of Glitter in Popular Culture, 205–223. London: Routledge. Chandler, Daniel. 2007. Semiotics: The Basics. 2nd edition. London: Routledge. Chapman, James. 2009. Not “another bloody cop show”: Life on Mars and British television drama. Film International 7(2): 6–19. Collins, Jim. 1995. Architectures of Excess: Cultural Life in the Information Age. London: Routledge. Cook, Pam. 2005. Screening the Past: Memory and Nostalgia in Cinema. London: Routledge. Cook, John R and Mary Irwin. 2012. “Moonage Daydreams”: nostalgia and cultural memory contexts of Life on Mars and Ashes to Ashes. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 79–88. Cardiff: University of Wales Press. Cooke, Lez. 2003. British Television Drama: A History. London: BFI. Creeber, Glen. 2006. Analysing Television: Issues and methods in textual analysis. In Glen Creeber (ed.), Tele-Visions: An Introduction to Studying Television, 26–38. London: BFI. Curzon, John. 2012. Sam Tyler and the “New North”. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 69–77. Cardiff: University of Wales Press. Davis, Fred. 1979. Yearning for Yesterday: A Sociology of Nostalgia. London: Collier MacMillan. de Groot, Jerome. 2009. Consuming History: Historians and Heritage in Contemporary Popular Culture. London: Routledge. Debrett, Mary. 2009. Riding the Wave: Public service television in the multi-platform era. Media, Culture and Society 31(5): 807–827. Edgerton, Gary R. 2011a. Introduction: When our parents became us. In Gary Edgerton (ed.), Mad Men: Dream Come True TV, xxi–xxxvi. London: I.B. Tauris. Edgerton, Gary R. 2011b. The Selling of Mad Men: A production history. In Gary Edgerton (ed.), Mad Men: Dream Come True TV, 3–24. London: I.B. Tauris. Edley, Neil. 2001. Analysing Masculinity: Interpretive repertoires, ideological dilemmas and subject positions. In Margaret Wetherell, Stephanie Taylor and Simeon Yates (eds.), Discourse as Data: A Guide for Analysis, 189–228. London: Sage. Gergen, Kenneth J. 1999. An Invitation to Social Construction. London: Sage. Grainge, Paul. 2000. Nostalgia and Style in Retro America: Moods, modes and media recycling. Journal of American & Comparative Cultures 23(1): 27–34. Hall, Stuart. 1997. The Work of Representation. In Stuart Hall (ed.), Representation: Cultural Representations and Signifying Practices, 13–74. London: Sage.

424

Ross P. Garner

Hallam, Julia. 2005. Remembering Butterflies: The comic art of housework. In Jonathan Bignell and Stephen Lacey (eds.), Popular Television Drama: Critical Perspectives, 34–50. Manchester: Manchester University Press. Hills, Matt. 2012. “The medium is the monster … or the world?” Discourses of uncanny “old media” and immersive “new media” in Life on Mars. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 105– 115. Cardiff: University of Wales Press. Hodge, Robert and Gunther Kress. 1988. Social Semiotics. Cambridge: Polity Press. Holdsworth, Amy. 2011. Television, Memory and Nostalgia. Basingstoke: Palgrave MacMillan. Hollows, Joanne. 2003. Oliver’s twist: Leisure, labour and domestic masculinity in The Naked Chef. International Journal of Cultural Studies 6(2): 229–248. Jameson, Frederic. 1991. Postmodernism, or, the Cultural Logic of Late Capitalism. London: Verso. Johnson, Catherine. 2005. Telefantasy. London: BFI. Johnson, Catherine. 2012. Branding Television. London: Routledge. Keightley, Emily. 2011. From Dynasty to Songs of Praise: Television as cultural resource for gendered remembering. European Journal of Cultural Studies 14(4): 395–410. Kompare, Derek. 2005. Rerun Nation: How Repeats Invented American Television. London: Routledge. Lacey, Stephen and Ruth McElroy. 2012. Introduction. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 1–15. Cardiff: University of Wales Press. Leow, Joanne. 2010. The Future of Nostalgia: Reclaiming memory in Tan Pin Pin’s Invisible City and Alfian Sa’at’s A History of Amnesia. The Journal of Commonwealth Literature 45(1): 115– 130. McElroy, Ruth. 2012. Consuming Retrosexualities: The past live on screen, online now. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 117–129. Cardiff: University of Wales Press. McFarlane, Mhairi. 2008. Ashes to Ashes is dispiriting. The Guardian 15/02/08, TV and radio blog. Available at: http://www.guardian.co.uk/culture/tvandradioblog/2008/feb/15/ ashestoashes. Nelson, Robin. 1997. TV Drama in Transition: Forms, Values and Cultural Change. Manchester: Manchester University Press. Nelson, Robin. 2007. State of Play: Contemporary “High-End” TV Drama. Manchester: Manchester University Press. Nelson, Robin. 2010. Life on Mars. In David Lavery (ed.), The Essential Cult TV Reader, 142–148. Lexington: The University of Kentucky Press. Nelson, Robin. 2012. Life on Mars: Hybridity and innovation in a British television context. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 19–30. Cardiff: University of Wales Press. Newman, Michael Z. 2006. From Beats to Arcs: Toward a poetics of television narrative. The Velvet Light Trap 58, Fall: 16–28. Pickering, Michael and Emily Keightley. 2006. The modalities of nostalgia. Current Sociology 54(6): 919–941. Piper, Helen. 2011. Vintage Entertainment: Nostalgia, the archive and the disappearing pleasures of collective television viewing. Journal of British Cinema and Television 8(3): 411–429. Ray, Ruth. 2000. Beyond Nostalgia: Aging and Life-Story Writing. Charlottesville/London: University Press of Virginia. Schwarz, Ori. 2009. Good Young Nostalgia: Camera phones and technologies of self among Israeli youths. Journal of Consumer Culture 9(3): 348–376.

Celebrating and critiquing “past” and “present”?

425

Seiter, Ellen. 1992. Semiotics, Structuralism, and Television. In: Robert C. Allen (ed.), Channels of Discourse, Reassembled: Television and Contemporary Criticism. 2nd edition, 31–66. London: Routledge. Sewall, Gilbert. 1997. Revisiting the Eighties. In: Gilbert Sewall (ed.), The Eighties: A Reader, xi–xxiv. Reading, Massachusetts: Perseus Books. Smith, Paul and Jeanette Steemers. 2007. BBC to the Rescue! Digital switchover and the reinvention of public service broadcasting in Britain. Javnost: The Public 14(1): 39–56. Sprengler, Christine. 2011. Screening Nostalgia: Populuxe Props and Technicolor Aesthetics in Contemporary American Film. New York: Berghahn Books. Tannock, Stuart. 1995. Nostalgia critique. Cultural Studies 9(3): 453–464. Tincknell, Estella. 2010. A Sunken Dream: Music and the gendering of nostalgia in Life on Mars. In Ian Inglis (ed.), Popular Music and Television in Britain, 161–175. Farnham: Ashgate. Thompson, Graham. 2007. American Culture in the 1980s. Edinburgh: Edinburgh University Press. Tudor, Deborah. 2012. Selling nostalgia: Mad Men, postmodernism and neoliberalism. Society 49(4): 333–338. Veith, Eva. 2004. A Kind of German Star Trek: Raumpatrouille Orion and the life of a cult TV series. In Sara Gwenllian-Jones and Roberta Pearson (eds.), Cult Television, 223–240. Minneapolis: University of Minnesota Press. Willis, Andy. 2012. Memory Banks Failing! Life on Mars and the politics of re-imagining the Police and the Seventies. In Stephen Lacey and Ruth McElroy (eds.), Contemporary Landmark Television: Life on Mars: From Manchester to New York, 57–67. Cardiff: University of Wales Press.

Part III

Roger Sabin

18 Ally Sloper, Victorian comic book hero: interpreting a comedy type Abstract: Texts mean different things to different audiences at different times. This chapter takes as its subject a British 19th century cartoon character, Ally Sloper, in order to explore why and how he might have been thought to be amusing. This involves an approach rooted in comics studies, which itself draws upon methodologies and ideas associated with art history, textual analysis, social history and historiography. On one level, Sloper’s appeal will always be mysterious, because the past is gone and we can never fully reconstruct it. But on another we can piece together the fragments that are left in order to offer a picture of a pop culture icon who was enjoyed in a semi-ironic fashion by his audience, and who rose to become arguably the most famous entertainment figure of his day.

1 Introduction This essay takes as its subject the cartoon star of a number of Victorian comic books, namely Ally Sloper – a ne’er-do-well, con-man, and drunkard, whose name came to be understood as “one who slopes off down the alleyway to avoid the rent-collector”.1 He was “the most popular fictitious character of the day – possibly the most popular ever invented” (Junior 1896), according to one contemporary commentator, and was a comedy superstar and folk legend (ending up as an inspiration for Charlie Chaplin). The essay will approach the character from one isolated example: a cartoon drawn in 1885, and attempt to dissect the meaning of his appearance – his age, his physique, and his clothes. In so doing, it is hoped to restore some of the character’s “energy” and thereby suggest ways of understanding why he was thought to be so funny.2 Thus, the approach belongs to the growing field of “comics studies” (sometimes referred to as a discipline with a capitalised Comics Studies) and involves a kind of cultural archaeology. “Interpretation”, for the purposes of this essay, constitutes a way of understanding the text (in this case, the image) by considering the various things it might have meant to different people. The image, and the comic to which it belongs, becomes a lens through which to view a political

 This meaning originates in 1867: although Sloper had a prototype in a penny dreadful named “Arry Sloper”, it would have been obvious (following the Thackerayan and Dickensian tradition of “naming”), and subsequent cartoons and stories in the Half-Holiday made it explicit.  Sections of this essay draw from Sabin, Roger (2007) Ally Sloper: ‘Our Bibulous Hero’, SIGNS (Studies in Graphic Narratives) 1(1).

430

Roger Sabin

Fig. 1: Our main case study: Ally Sloper, detail from cover illustration to Ally Sloper’s Half-Holiday, May 9, 1885, drawn by William G. Baxter.

moment, in this case the late 19th century, and to explore the various constellations of power within that moment. There are many reasons why this is worth doing, but perhaps the main one is that it allows the reader to reflect on the contemporary context they find themselves in. If the philosopher Stuart Hall is right that our “identity” is in large part about where we choose to place ourselves within the stories that are told about history, then such an approach might help us thinkthrough “who we are” (Hall 1996: 4).

2 Context and research questions Comics studies is a relatively new field, and the kind of critical analysis described above has long been a staple of literary and art critical theory. But we can point to certain historians who have become prominent in the analysis of 19th century comics, for example, David Kunzle (1992), whose History of the Comic Strip: the Nineteenth Century was a foundational work; Pascal Levèvre and Charles Dierick (1998), whose edited volume Forging a New Medium: the Comic Strip in the Nineteenth Century is an essential textbook; and Thierry Smolderen (2009), whose Nais-

Ally Sloper, Victorian comic book hero

431

sances de la Bande Dessinée (“Births of the Comic Strip”) traces several possible origins for the form. One of the points of disagreement between historians is when, exactly, comics began. The standard gambit for most historical surveys is that America’s “Yellow Kid” (which originated in 1895) by Richard F. Outcault is the starting point. But others alight upon European creators such as Rodolphe Topffer (1799–1846) in Switzerland, Wilhelm Busch (1832–1908) in Germany, Apel-les Mestres (1854–1936) in Spain, and Charles Ross (1835–1897) – the creator of Ally Sloper – in Britain. But it doesn’t end there. Some go further back in time, to the age of prints (Gillray, Cruikshank, Daumier, etc.) and to Hogarth (1697–1764) – who, it is argued, was first to bridge the gap between image and novel. Smolderen’s (2009) book, mentioned above, finesses this history, and argues that any search for a “first” is pointless because comics should be seen as part of a continuum of cultural outputs that might include magic lantern shows, Punch and Judy performances, early photography, and so on. Our Ally Sloper image needs to be considered within the context of these debates. Further, we should note that the historical record is not complete, and so therefore we can only “restore energy” to a point. The past is gone – over-with – and we can only do our best to construct a narrative based on the scraps that are left. (Which is partly why we’ll never find Sloper as amusing as did the Victorians.) Similarly, meaning is never immanent – this essay is not about trying to reveal the “true intentions” of the image’s creators. Ideally, we have to look “outside the text” as well as “inside”, which means that we have to take the audience into account – rather difficult in the case of a 19th century text. For example, readers would have been comparing the Sloper cartoon to all sorts of exterior factors – what they saw of other comics, what they saw of other artforms (for example, the routines of music hall performers), what they saw of “real life” (did the humour “ring true”?) and what they saw of “ancillary material” (commentary in the press, advance previews, posters, etc.), to name a few. Finally, our analysis has to be partial in other ways – for example, it doesn’t look too closely at the nature of Sloper’s adventures, which would be an altogether bigger project. Nevertheless, a careful process of looking, backed up with historical research, can help establish some basic lines of interpretation. Our image was drawn by the most famous of the Sloper artists, William Baxter (1856–1888), though the Sloper “concept” has to be understood as having many creators. It is a detail from the cover cartoon to Sloper’s most successful comic, Ally Sloper’s Half-Holiday (1884– 1923). This comic was important in itself as a progression from the Punch formula – a weekly, selling mainly to working class rather than middle class readers, and with an emphasis on slapstick and “low humour” rather than satire. The HalfHoliday was the first such publication to star a continuing character – Sloper – and therefore benefitted from the kind of creative possibilities that seriality could bring. In other words, Sloper could be put in increasingly complex situations week

432

Roger Sabin

after week because the joke was already “pre-loaded” with information. (It is also pre-loaded in other ways: Sloper had other incarnations, some in cartoon form, but also on the stage, where performers imitated him in various guises.) So, looking at this image, we can begin by asking: What are the character’s most noticeable traits, and how can we read these off against the anxieties of the age? More to the point, how might Sloper be mediating those anxieties, and even subverting them? The first thing you might notice about the image is that it is very well drawn. The style is illustrational, in the Punch tradition, and the art is “inspectable” and very detailed. Baxter was a master at this kind of virtuoso penmanship, and the fact that this image was used on the cover of the Half-Holiday is an indication that the publisher, one Gilbert Dalziel, came from a family of famous engravers and therefore required a certain standard of art on his products. (Dalziel owned the copyright to the Sloper character at this time.) This kind of detail in the drawing meant that the body of the character could be delineated with attention to its grotesque nature – for example, the head is drawn out of proportion, which references both the “big-head” style popularised by Punch, plus the oversize pantomime masks that performers would wear on stage. This meant that facial expressions could be emphasised, and physical flaws shown up. In our image, Baxter is making sure that readers can get a good look at Sloper, with the intended repulsion and fascination being all part of the fun.

3 A Victorian “body” So, for example, readers could see immediately that Sloper was old. He is clearly somewhere in his late 50s or 60s – much older than the comedy stereotypes we are familiar with today, and old enough to be fast approaching death at a time when life expectancy was a lot less than the Biblical “three score years and ten”. To be more specific: it has been estimated that the average life span in 1840, in the Whitechapel district of London, was 45 years for the upper class and 27 years for tradesmen. Labourers and servants lived only 22 years on average (see Mitchell 1988). By the time of our image, life expectancy had improved slightly, but accurate statistics are hard to find, and are anyway complicated by factors such as the extremely high infant mortality rate and the persistence of sporadic lethal epidemics e.g. cholera. Why might Sloper’s advanced years have been amusing? Possibly because by that age, a person was supposed to be settled in society: to have built their personal “empire” and be contented in their lot. Not so for Sloper, who from his demeanour and shabby clothing was immediately recognisable as belonging not just to the working class, but also possibly to what Charles Booth’s famous London Poverty Map (1889) referred to as “The lowest class – vicious, semi-criminal”.3  See: http://booth.lse.ac.uk.

Ally Sloper, Victorian comic book hero

433

Here was somebody who was not settled at all, and evidently always “up to something” – his carnivalesque adventures taking in scams at the seaside, or moneymaking schemes involving new technology. (His other manifestations, including trying to pass himself off as a member of the gentry, are discussed below.) The image of Sloper the old man, in other words, went against the Victorian “improving ideal”, touted from pulpits across the country. If Samuel Smiles’ book Self-Help (1859), about how the working classes could improve themselves through education and hard work, was a kind of Victorian blueprint for this way of thinking – a quarter of a million copies sold by 1904 – then Sloper and his comics represented a comprehensive subversion of it. Sloper was “helping himself” alright – just not in the way that Smiles had hoped. Here was an individual who manifestly hadn’t improved himself, and who furthermore wasn’t interested in doing so by legitimate means. Such an attitude also went against the Liberal ideal that the working class should play more of a part in society as “citizens”, with all the responsibility that word implied, by dint of better education and rising literacy rates: the ideal, in other words, that an educated populace would be better able to undertake the obligations of a broadened franchise. (1884, the year of Half-Holiday’s founding, also happened to be the year of the Franchise Bill, which extended the vote to male householders in county constituencies.) But Sloper, whether educated or not, is the opposite of “a responsible citizen”. It’s clear that he will go to his grave scamming, and this is what gives his adventures added gusto and pathos: he has no choice but to carry on doing what he does. From his advanced years and his place in society, readers could deduce that nothing was going to change for him unless he made it change. True, he was outside of the grind and monotony of factory or office work. But he was caught in an equally pernicious cycle that meant he had to come back to some kind of new scheme week after week: a classic tragi-comic narrative. Was this scenario necessarily a conservative one? As ever, there are many interpretations. For example, it is true that Sloper gets stymied at the end of his weekly adventures, but he continues to come back for more – in preference to finding “proper work”. (He gets knocked down, but he gets up again, as post-punk outfit Chumbawamba would have it.) He’d prefer to stay outside the system, forging an identity that is not tied to any kind of job – and that is a challenging position, politically speaking. Moreover, occasionally, Sloper would be depicted enjoying his outsider status, smiling in the face of adversity or even enjoying the rewards of his dodgy dealings. (In our image, he looks very pleased with himself.) It’s possible to imagine, in these circumstances, that readers might have been entertained by the idea that he was flaunting his lack of conventional ambition – an “underachiever and proud of it”, in the words of Bart Simpson a century later. And like the fuss that accompanied Bart’s words (e.g. President HW Bush stating that he wished more Americans were like the Waltons and fewer like the Simp-

434

Roger Sabin

sons), we can speculate that the idea of a layabout “getting away with it” would probably have given the more conservative readers of the Half-Holiday pause for thought. This kind of interpretation has been finessed by historian Scott Banville in his essay “Ally Sloper’s Half-Holiday: The Geography of Class in Late Victorian Britain” (2008) in which he argues that Sloper can be seen as a resolutely subversive figure, attacking both the morality and privileges of the middle class and demonstrating that the working class deserved more of “what they had”. In other words, he is about “the refusal to get on and conform to middle class prescriptions as to behaviour” (2008: 167). Thus, in terms of Mikhail Bakhtin’s concept of carnival, the chaotic “world-turned-upside-down” aspects of Sloper’s adventures are always short-lived, and the status quo prevails: but the example of his rebellion lives on – as a symbolic spur to others.4 But there were other associations that readers could have made with Sloper’s advanced age. Britain’s Prime Minister in the years 1868–74, 1880–85, 1886, and 1892–94 was one William Gladstone, a man born in 1809. Gladstone is today revered by some historians as one of the greatest politicians the country has ever known, and has been cited as an inspiration by everybody from Churchill to Blair and Cameron. As a Liberal, he was associated with reforms aimed at bettering the lot of the common man. One nick-name for him was “the People’s William”; another was “The Grand Old Man” (GOM). Baxter was quick to reference Gladstone’s impact on the nation, and paid homage by referring to Sloper as the “Friend of Man” (FOM).5 So what did this relationship with Gladstone represent? It was certainly complex. At the start of Sloper’s existence, in Judy magazine at the end of the 1860s, he was not so identifiably old and Gladstonian. In this incarnation, typically written by his inventor, the aforementioned Charles Ross, and often drawn by Marie Duval, Sloper was not intended to be a symbol of any kind of politics (see Kunzle 1986). Furthermore, Judy was a Conservative publication, and commonly declared itself for Gladstone’s arch-rival, Disraeli. After 1884, when the Half-Holiday was founded, a new team of Sloper creators took over (although the Ross/Duval material continued to be reprinted in its pages), and developed the character in different comedic directions – foremost among them being Dalziel and Baxter (we can identify Baxter specifically as using “FOM” in his first Sloper cover cartoon). So, did the overt link with Gladstone mean that Sloper was now more “political”? It’s very questionable that he ever took on the mantle of Liberalism. Indeed, through the 1880s and 90s it is more true to say that Sloper was (ostensibly, at  See in particular Bakhtin’s (1984) Rabelais and his World. New York: John Wiley and Sons.  The letters “FOM” would appear after Sloper’s name, thus performing the double task of taking the mickey out of people who identified themselves via their awards, memberships of associations, etc. (“RA, VC, etc.”). It also associated Sloper with this class – or, rather, as a fake among them (on his mock gentrification, see below).

Ally Sloper, Victorian comic book hero

435

least) a conservative with a little “c” – a “little man” virulently opposed to extremism of any kind. Cartoons depicted him dressed as a policeman and fighting off the unemployed, oozing up to the royal family, and hugging “the dear old flag” in times of war. None of this was necessarily out of kilter with Gladstone’s Liberal politics, or out of kilter with Banville’s analysis of Sloper as a subverter of middle class morality, but it did place Sloper on what we would now recognize as the right wing of the political spectrum. However, whether audiences were expected to laugh at his more reactionary stances, à la Alf Garnett, is, again, debateable. Thus the association with Gladstone had limits, and it is interesting to note that several times in the pages of the Half-Holiday it is stressed that Sloper has no party affiliation.6 Thus, Sloper could both be seen as being “in tune” with Gladstone’s professed advocacy of the working man, or a complete subversion of it – a satire on the Liberal project, and a repudiation of Gladstone’s dour Christian moralism. Gladstone himself was occasionally caricatured in Half-Holiday, including one famous Baxter cover in which Sloper herds him into a school (a comment on the GOM’s education plans). It was even true that Gladstone was awarded the Sloper Award of Merit, a certificate given to eminent personalities of whom Sloper approved, which he was proud to display in his Scottish home (where it can be seen by tourists today). But was this simply a marriage of convenience? Clearly, for Gladstone it meant demonstrating he had a sense of humour (always crucial for a politician) and that he was “down with the common people”; for the creators of Sloper it was an easy comedic device that gave the character a patina of topicality. If age was the first thing that the reader would notice from Baxter’s beautifully detailed drawing, then the state of Sloper’s body might be the second. Here is a man with stick-thin legs and a pot belly, always depicted with an oversize head – the focus of which is a bulbous red nose. He is a grotesque, and as such perfectly in tune with Bakhtin’s notion of carnival. Also, he is a comedy type – familiar from the caricatural tradition of Hogarth, Rowlandson and Cruikshank (the latter being an acknowledged influence upon Charles Ross). Equally clearly, Sloper has the physique of a malnourished drinker, emphasised by the bottle often pictured sticking out of his back pocket: again, somebody who in all probability belongs to that class of people living in the wrong districts of Booth’s Poverty Map. (Sloper, in fact, was meant to be a resident of Battersea, which on the map is indicated to be an area including the very poorest neighbourhoods to those of mixed income.7)

 For example, Ally Sloper’s Half-Holiday January 4, 1896: “… Sloper is neither Whig nor Tory …”; and June 20, 1903: “[The Half-Holiday] has always steered clear of party politics”.  On the map, Battersea included the following shades: Black to represent ‘The Lowest Class – vicious, semi-criminal’; dark blue for ‘Very poor, casual. Chronic want.’; light blue for ‘Poor – 18s to 21s a week for a moderate family’; and purple for ‘Mixed. Some comfortable, others poor’. For the richer neighbourhoods elsewhere, the map used pinks and reds. Sloper was said to live “opposite the Dogs’ Home” in Mildew Court – not the most salubrious of addresses.

436

Roger Sabin

Such Londoners knew very well about the links between diet and health, and readers from all backgrounds would have empathized quite readily with Sloper’s vulnerability to something like rickets, to which his ambling gait may well allude – evident from our 1885 image – as well as to the more serious diseases that were endemic (typhoid, cholera, etc.).8 Sloper’s was hardly the physique of the “classical ideal” beloved of Victorian artists – Apollo he was not. Nor was he the image of the “fit workman” beloved of a particular (sentimental) strand of Victorian fine art that eulogised the industrial revolution.9 Above all, he was not a “muscular Christian”, as would be portrayed in the pages of the Boys Own Paper and others. Such well-fed, middle class fellows were meant to be role models, “leaders of Empire”, and would eventually form the basis for the officer class in the First World War. The “ruined” Sloper (and, indeed, “the ruin” was another nickname for him) didn’t fit the template, and the effect of picturing him in the company of such prime physical specimens would no doubt have been doubly amusing. (It is interesting to note that the spindly limbs, pot belly and big nose recur through the later history of comics: think of examples such as Reg Smythe’s “Andy Capp”, the Viz team’s “Sid the Sexist”, and Peter Bagge’s “Buddy Bradley”). Not only this, but we can extend the idea of physical degeneracy as a metaphor. For example, “muscular Christianity” was about “order” – in other words, it embodied (literally) the idea that a decent person keeps order, both over themselves and over the world. Sloper was, of course, anti- the very idea of order, and therefore his bodily subversion, if it can be seen that way, had further associations. So, when Sloper was depicted as – say – a sportsman, it was both an amusing satire on the cult of fitness, but also on the idea of “discipline on the sportsfield”. Similarly, if he was a soldier, it was about physical suitability and discipline. In fact, it was inconceivable that anybody in Sloper’s state would have been enlisted in real life, and this became something of a political crisis a few years later. Historians Simon Gunn and Rachel Bell point out that: “The difficulty of recruiting soldiers for the Boer War in 1899–1902, because of the high rejection rates of young working class men due to physical defects like flat feet and inadequate physique, caused public alarm” (Gunn and Bell 2002: 47–48). As for the drinking, Sloper’s nose isn’t just oversized, it’s swollen, gnarly and red (a condition known as rhinophyma) in the manner of a man with “a problem”. Such a nose was not a novelty in the history of cartooning, and had a long association with clowning and the notion of “the fool” as well as with excessive drinking  In fact, Sloper rarely gets ill in the comics, chiefly no doubt because this wouldn’t have much comedy value. However, on one occasion (October 9, 1886) he comes down with the flu, only to be tended by the Royal Physician, Dr William Gull. (The same William Gull identified as Jack the Ripper in the popular 1999 graphic novel From Hell.).  See Martin A. Danahay (2005) Gender at Work in Victorian Culture: Literature, Art and Masculinity. Aldershot: Ashgate.

Ally Sloper, Victorian comic book hero

437

(Sloper’s nose is reminiscent of the noses made famous in drawings by Cruikshank, himself an anti-booze fanatic in later life). Thus, Sloper – and particularly the dilapidated state of his body – could be read as a warning against the perils of alcohol. Indeed, drink could be the very cause of his undoing – a message very much in tune with the revival of the temperance movement in the 1880s. (As an aside, Baxter himself is believed to have died of an alcohol-related illness, aged only 31.) But is drink such an evil in the comic? Often we find that the portrayal of Sloper’s habits is done in a sympathetic, even winked-at, way. Boozing could be seen as a necessary method of self-medication against the horrors of poverty, or as a way to offset the drudgery of the working week. Similarly, the temperance movement’s crusades in the poorer districts of London were generally not welcomed, and Sloper would have stood as symbol of rebellion against them. Some of the funniest cartoons in the comic are of him subverting the movement in some way.10 Moreover, Sloper’s rebellion might not just have been about seeing him as the defender of the right to get drunk. It might also have involved the right to have fun. For example, Victorian pubs in the period were not simply drinking dens, and included all manner of entertainments – always a piano in the corner, but also, often, marionette stalls, stages for small-scale theatrical performances, and even pitches for flower-sellers.11 The campaigns by temperance organisations against the pubs, therefore, would have been opposed for much wider reasons: in short, they were killjoys, and Sloper was a saviour.12 Similarly, the music halls were making their money from the sale of drink during this period, and especially bearing in mind Sloper’s presence on the stage, this would have been another factor to take into account. Yet it is still true that, to make the “drinking comedy” more palatable in the Half-Holiday, Sloper is rarely shown in an incapacitated state. We know he’s a heavy indulger (from his physique and the bottle), but in the Baxter and Thomas cartoons he is typically shown with clear eyes, and to be fully able to carry out his scams. Put simply, if he was blitzed out of his mind, he couldn’t rip anybody off. True, it’s more common in the early strips by Ross and Duval that we see scenes of him grabbing onto lamp-posts for stability, collapsing having passed out,

 The fact that the temperance crusaders were known for making alliances with Gladstone’s Liberals would have added another, political, dimension.  Perhaps the academic who is most associated with research into this area is Ann Featherstone, whose paper ‘Performing the Pub’ at the 2001 British Association for Victorian Studies Conference at Lancaster University pointed to exciting new perspectives on leisure studies.  This explains why the Salvation Army, always linked with the temperance movement, used certain tactics to draw men away from the pubs, such as adapting working class popular music, and theatrical forms of presentation. See Walker, Pamela (2005) Pulling the Devil’s Kingdom Down: The Salvation Army in Victorian Britain. Berkeley: University of California Press.

438

Roger Sabin

Fig. 2: Cover to Ally Sloper’s Half-Holiday, October 3, 1885, showing Sloper in “mock gentrified” mode; here shooting pheasants in the Highlands with a “dook”. Main image drawn by W. G. Baxter.

Ally Sloper, Victorian comic book hero

439

and the like. But these early stories were reproduced less and less frequently in the Half-Holiday as time went on. Sloper, if he’s an alcoholic, is a high-functioning one. Possibly, we can extend this line of thinking to the depiction of the “half holiday” itself – the newly-established practice of allowing workers to go home at lunchtime on a Saturday. In the Half-Holiday, this is portrayed as a time for workers to relax by going to the seaside, the races, the football or whatever, and have a bit of harmless fun. In other words, we can say quite confidently that the publication was a quintessential expression of the new leisure age taking shape in Britain at the end of the 19th century. Yet, what we don’t see in its pages is much mention of the binge drinking in the pubs that was just as much of a feature of these holidays as anything else. Here is a quote from a newspaper article from slightly later (1904) that gives a very different perspective: “In fining a number of working men for being drunk on Saturday afternoon and evening, Alderman Dr Crosby, at the Guildhall yesterday, remarked that the Saturday half-holiday was a curse to working people instead of a blessing … The men, directly they drew their earnings, went to the public-house with them instead of taking them home to their wives …” (Anon 1904).13 Perhaps by turning a blind eye to the commonly “pissed-up” nature of the Saturday break, the producers of Half-Holiday were themselves making some kind of stand against what they saw as the “extremists” in the temperance movement. The comic’s consumers would have read between the lines and got the message – and this message fitted perfectly with the vision of Sloper as the Friend of Man who took on other kinds of extremists (the aforementioned unemployed, along with socialists, anarchists, and, occasionally, “new women”). It was a kind of drinking comedy much different to the depiction of booze in the popular prints of half a century before (commonly focused on dissipation, bodily effluvia and violence); and different again from the “Eight Ace” humour of Viz a century later (always focused on dissipation, bodily effluvia and violence). As such, the HalfHoliday was a promoter of what its creators no doubt saw as a moderate world (rather in the same way as The Sun today sees itself as speaking for the middle ground): a comic where extreme politics were not wanted and where advertisers could feel safe in investing their money (see Sabin 2003).

4 Clothes maketh the (con) man So that’s Sloper the old man, the malnourished man, and the heavy drinker. But there’s one other aspect to him that’s evident from the image, and that is his  The same conclusions about the boozy nature of the half-holidays can be drawn from the descriptions in Tressell, Robert’s (1914) classic novel The Ragged Trousered Philanthropists. London: Grant Richards Ltd (especially p. 375 of the Penguin 1999 edition).

440

Roger Sabin

clothes. These are clearly not the clothes of a labourer. So, why is he dressed this way, and what might they be telling us? In his “everyday attire”, as pictured in Baxter’s image, Sloper is shabby – typically dressed in an old swallow-tail suit, worn-out shoes, and an old stovepipe hat, crumpled with ill-use. He also carries an umbrella, pocked with holes and with broken spokes, with which he sometimes defends himself or uses exaggeratedly to make a point. Although these vestments are “neutral” to some degree, Sloper is immediately positioned as a man of leisure. Somebody from the working class who doesn’t work – rather like the idle rich at the other end of the social spectrum. It could even be argued that the suit and umbrella make Sloper more akin to a member of the lower middle class – especially the sub-group of “clerks” that had recently come into being thanks to the country’s changing economic circumstances. This was the demographic that Charles Ross had good reason to observe around the time when he was inventing Sloper, when he was working at the Admiralty. Yet everything else about Sloper speaks of a working class context, and it might be more sensible to think about the suit in terms of the huge market in second-hand clothes that existed in London – as attested to by Henry Mayhew and others. The true horror of Sloper’s poverty is, understandably, elided in the HalfHoliday, not least because it needs to appeal to the widest constituency, but this does not mean that he is anything other than proletarian. More than this, Sloper was a conman, and, as such, could be expected to “dress up” in order the better to fleece people. This would have been understood by streetwise readers, who were aware of the long history of such petty criminality. Historian Donald Thomas, drawing on Mayhew et al.’s London Labour and the London Poor (1861–62), paints this evocative picture of a London encounter: “As the visitor progressed on his journey through the West End, two “tall, gentlemanly men” crossed the Strand together and went into a restaurant. These were pickpockets, also well known to the police. “They were attired in a suit of superfine black cloth , cut in fashionable style. They entered an elegant dining-room, and probably sat down to costly viands and wines.” If such men were not quite at the top of their profession, they were certainly a cut above Bill Sykes and the Artful Dodger and did not appear in the least like the criminal of the Victorian imagination” (Thomas 2000: 43). So, in other words, for Sloper to dress in aspirational clothes would suit his purpose. It was this aspirational aspect to the character that was built upon in different ways by his creative custodians as time progressed. For, in the Half-Holiday, Sloper is perhaps only half of the time seen in his everyday attire, and then rarely on the covers. Rather, there is a move away from the old Ross/Duval model and towards a version of the character who is clothed in different ways to suit different comedic situations. Thus, we have Sloper the footballer, cricketer and rugby player; Sloper the soldier, sailor and policeman; Sloper the dancer, musician and music hall performer; and so on. We also see Sloper becoming “mock gentrified” in order to make comment on the manners of

Ally Sloper, Victorian comic book hero

441

the middle and upper classes. Thus, he wears an Eton schoolboy’s uniform on a trip to the school; a top hat, tails and monocle on a visit to a society event; ermine on becoming elevated to the House of Lords, et cetera. This mock-gentrification, at one level, was a way to laugh at Sloper for his gaucheness and the way he manages to muck things up when in the company of his “superiors”. But, possibly, too, it could be interpreted as a fantasy of the experiences that Sloper could have had if circumstances had been different – if he’d been “higher born”. Thus, the stories might be “counter-factuals”, representing the different lives – better lives – he could have had. Similarly, such gentrification could also be seen as a rags-to-riches story beloved of Victorian popular literature – but without the necessary graft in-between. Thus, it could be a satire of the kind of yarn tracing a passage “from pit to parliament”, or from the slums to the country house, and therefore a satire, once again, on the Samuel Smiles ideal. Historian Peter Bailey, arguably the leading scholar on the Sloper of this period, has written perceptively that the tactic of mock-gentrification could be interpreted as a method to reach middle class readers who might pick up on more subtle cues, such as the particular way in which Sloper wears his clothes, or his choices in matters of taste.14 They might be viewing his gaucheness from a different perspective, and catching him out for his lack of cultural capital. Laughing “at” Sloper as well as with him. Bailey points out that is where Baxter’s art came into its own – its “inspectability” being perfect for drawing out such nuances. Such a theory makes sense, especially bearing in mind that Half-Holiday’s cartoonists and gagsmiths themselves were largely drawn from the middle class (Baxter himself was relatively highly paid). But it’s also possible to argue that the working class could often be very clued-in to the manners and fashions of the classes above them. Historian Jonathan Rose, for example, has shown that large sections were avid consumers of (some kinds of) high culture, especially when it came to reading.15 For example, the “classics” were demonstrably not the preserve of the upper and middle classes when mass production meant that there was a thriving trade in tuppenny second hand editions – and thus the cultural capital afforded by knowing what such classics were about could be easily attained. Similarly, information on fashion was available, via the secondhand trade in newspapers, journals and magazines. Just because the working class couldn’t afford good clothes did not necessarily mean they didn’t know what they were. Another historian, Christopher Breward, has even suggested that Sloper was a fashion icon for this class: a dandy to be admired for his sartorial flair.16  Bailey, Peter (1983) Ally Sloper’s Half Holiday: Comic Art in the 1880s. In History Workshop 16. especially pp. 18–20.  Rose, Jonathan (2003) The Intellectual Life of the British Working Classes. New Haven: Yale University Press.  Breward, Christopher (1999) The Hidden Consumer: Masculinities, Fashion and City Life, 1860– 1940, Manchester: Manchester University Press.

442

Roger Sabin

Finally, Sloper’s fancy threads might also be interpreted on a much more basic level, as associating him with a long tradition of making comedic figures seem more absurd. In the words of one commentator from 1886: “Mr Punch sometimes wears a silk hat and frock coat like any other gentleman; Stenterello, so long as he keeps his queue, and the lines on his face, and has one front tooth missing, can figure as a modern Florentine. The more strongly caricatured were the features of Ally Sloper, the less dependent was he on his clothes for identification, while it was only more ridiculous to make him appear at appropriate seasons in Highland dress, or Eton collar and jacket, in boating flannels or racing jerseys” (Pennell, 1886: 518).17

5 Concluding thoughts To bring this discussion to an ending, we can say that, for every creative choice made about the construction of Sloper the comedy character, there was a huge variety of possible interpretations. Whether it was his age, his connection with Gladstone, the state of his body, his drinking, or his clothes, the humour could be “got” on different levels. This multivalency was changing all the time, as it transformed in the light of the relationship with previous versions of Sloper (last week’s, last month’s, last year’s), Sloper’s onstage iterations (comedians that imitated him, ventriloquists who used Sloper dolls, etc.), Sloper merchandising, plus the broader relationship with other comic characters, with other forms of popular culture, and, ultimately, with “real life”. But we need a post-script on the readership for the Sloper comics. It was argued at the start of this essay that the audience would ideally have to be taken into account as well as the text itself. But, unsurprisingly, 19th century audiences are often hard to define. We know that the Half-Holiday attracted a readership that was mainly working class, and we know that there was a sizeable middle class readership – including such luminaries as the aforementioned William Gladstone, plus people like William Morris and Frederic Leighton. But can we necessarily make any assumptions about the political leanings of this readership? For sections of the working class, Sloper could be seen as a hero, for the reasons we have outlined – a subverter of middle class privilege, and so on. But on the other hand, he could be seen as a kind of enemy – the lout on the block who drinks too much, makes too much noise at night, and who brings the working class into disrepute. For the middle class, certainly there might have been those who snobbishly laughed at Sloper – for his crude manners, his social faux pas, etc. – but equally there is evidence that some middle class readers regarded him as a symbol of a

 Pennell’s remarkable article is recommended to anybody with an interest in Sloper, and must be counted as one of the first instances of what has become known as ‘comics scholarship’.

Ally Sloper, Victorian comic book hero

443

resurgent working class and of a new “folk culture”. William Morris, it should be noted, founded The Socialist League. Similarly, if the Sloper comics did take a particular political line on a topical event, there’s absolutely no reason to think that the readership necessarily agreed with it. Today’s Media Studies discipline has offered numerous case studies of newspapers which compare political stances with surveys of readers: for example, in the 1992 general election, although The Sun told readers to vote Conservative, most did not.18 In other words, audiences “negotiate” the text, and always have done. Such speculations about the complexity of the readership are relevant because in 1885, when our image appeared, the debate about how much political power should be ceded to the working class was very much alive. The franchise was being progressively widened, as we have noted, and it was clear that this social group was flexing its muscles in ways that had not been witnessed before. The dawning of the age of modernity was ushering in new configurations of power, and pop culture was at the forefront of both transmitting and reflecting hopes and fears. Humour could be a safety valve, a comfort, and a weapon – and the Sloper comics played on this ambiguity with an extraordinary sense of zeitgeist-surfing dexterity. In conclusion, the achievement of the creators of Ally Sloper was to bring an ages-old archetype up to date. They manipulated tropes that would have been well-known, and made them relevant to a 19th century audience – an audience that we can assume was very “clued-up” and would have developed a semi-ironic relationship with their “hero”. Appearances mattered a great deal in this process, with Sloper’s visual degeneracy mirroring his moral degeneracy. As such, our chosen image suggests a rich hinterland of connections and associations, especially having to do with social class. Ally Sloper “worked” because he tapped into some of the key obsessions of his age, and because, thanks to the skill of his various writers and artists, he developed layers of meaning over time. This is why he must be counted as one of the most important creations both in the history of comics, and of popular culture.

References Anon. 1904. newspaper article, no further details. Reprinted in: Paul Sieveking, Man Bites Man: The Scrapbook of an Edwardian Eccentric – George Ives, 1980, London: Jay Landesman Limited. Banville, Scott. 2008. Ally Sloper’s Half-Holiday: The Geography of Class in Late-Victorian Britain. Victorian Periodicals Review 41(2): 150–173. Bailey, Peter. 1983. Ally Sloper’s Half Holiday: Comic Art in the 1880s. In: History Workshop 16. Oxford: Oxford University Press.

 See also McNair, Brian (2000) Journalism and Democracy. London: Routledge, pp. 144–145.

444

Roger Sabin

Breward, Christopher. 1999. The Hidden Consumer: Masculinities, Fashion and City Life, 1860– 1940. Manchester: Manchester University Press. Gunn, Simon and Rachel Bell. 2002. Middle Classes: Their Rise and Sprawl: London: Cassell. Hall, Stuart. 1996. Introduction: Who Needs ‘Identity’? In: Stuart Hall and Paul du Gay (eds.), Questions of Cultural Identity, 4, London: Sage. Junior, Judex. 1896. Letters to Local Somebodies. Brighton Society, Nov 21. Brighton: Brighton Society Publishing. Kunzle, David. 1986. Marie Duval: A Caricaturist Rediscovered. Woman’s Art Journal 7(1): 26–31. Mitchell, Sally (ed.). 1988. Victorian Britain: An Encyclopedia. New York: Garland. Pennell, Elizabeth. 1886. The Modern Comic Newspaper, The Contemporary Review 50: 509–523. Rose, Jonathan. 2003. The Intellectual Life of the British Working Classes. New Haven: Yale University Press. Sabin, Roger. 2003. Ally Sloper: The First Comics Superstar? Image and Narrative. Available at: http://www.imageandnarrative.be/graphicnovel/rogersabin.htm. Thomas, Donald. 2000. The Victorian Underworld. London: John Murray Publishers. Quotes from Henry Mayhew, Bracebridge Hemyng, John Binny and Andrew Halliday, 1861–62, London Labour and the London Poor. London: Charles Griffin.

Marvin Carlson

19 Visual Communication in the Theatre Abstract: This essay begins with a brief historical survey of visuality in the Western theatre from the Greeks to the present, then goes on to discuss hoe theatrical visuality has been theorized, particularly from the late nineteenth century onward. Especially important to the development of a critical vocabulary for such theorization was the rise of modern semiotics. During the twentieth century the dynamics of visual communication in the theatre became much more complex, primarily for two reasons. One was the widespread movement out of traditional theatre spaces, creating very different visual opportunities for such productions. Next, and even more important, came theatre’s ever-increasing use of other visual media, first film, then video, then live video on stage, and most recently, digital imagery. All of these and various combinations of them are now widely used in both regular and experimental theatre, and placed the visual at the very center of the contemporary theatre experience.

1 Visuality in the Greek theatre The Greek word theatron, from which the English “theatre” along with parallel terms in many other modern languages is descended, literally means “a place for seeing”. Indeed seeing was much more central to the experience of Greek drama than it has been for many subsequent Western manifestations of this art. The Greek theatres were enormous outdoor civic monuments, the centers of national festivals and pageantry and most of them, built into the slopes of hills near their sponsoring cities, provided to their audiences sweeping views out across the countryside beyond city and theatre, vistas often including such striking natural features as distant mountains or coastlines. Visual spectacle was an important part of the performance itself, with a chorus moving in rhythmic patterns within a circle in the midst of the audience and the major actors appearing larger than life, on elevated shoes, with elaborate costumes, masks, and wigs. Even so, it would be an exaggeration to say, as many theatre historians have done, that this etymology provides significant evidence that visual communication was central to the Greek theatre experience. Even the etymological evidence is ambiguous. True, the word theatron means a place to see, a place with a view (theoria), but the English word descended from theoria is of course theory, which leads us to realize that the “view” and the “seeing” involved may imply either a visual seeing or an intellectual one, both of which are clearly relevant to the operations of the drama (another Greek derived word, but this one focusing on the performer, from dran, “to do” or “to act”.

446

Marvin Carlson

Despite the visual impressiveness of the Greek theatre, there is not a great deal of evidence that the visual aspect of this art was of major interest to its public. Indeed, Aristotle, whose opinions on theatre and the drama have been the most important single influence upon Western thought on this subject, clearly suggests that the visual side of theatre is its least important elements. Spectacle (opsis) and song, he lists last among the elements of tragedy, both considered only “embellishments”. Among such “embellishments” song is given the “chief place”. As for spectacle, “of all the parts, it is the least artistic”. For communication, Aristotle gives it little credit, observing that “the power of Tragedy, we may be sure, is felt even apart from representation and actors. Besides, the production of spectacular effects depends more on the art of the stage machinist than on that of the poet” (Aristotle 1895: Section IV). Doubtless this lack of interest in spectacle results from the strong literary bias in Aristotle. His key statement on theatre, the Poetics, looks primarily at tragedy, and to a much lesser extent to comedy and the epic as different literary forms imitating human action. Since this work serves as the bedrock for subsequent Western dramatic speculations upon and theory about the theatre, this textual orientation and concomitant indifference to non-textual elements is a characteristic of the Western tradition. The Sanskrit Natyasastra, in contrast, which is perhaps the closest non-Western parallel theoretical work to the Poetics, provides far more detailed analyses of the non-literary aspects of the theatre – makeup and costuming, gestures, the theatre and its design.

2 Visuality in the medieval theatre The emphasis upon the literary side of theatre, especially in respect to its operations as a communicative system, has dominated Western theatrical theory and criticism ever since the Greeks. There is not an historical period in which visual display was not a significant part of the theatre experiment, but it has often been regarded, as Aristotle regarded it, as an “embellishment”, not really essential to the “power” of the play or to the communication of its message or effect. Theatre scholars have paid far more attention to the literary dramatic texts from the Italian, Spanish, or English renaissance or from the French neoclassic and baroque periods than to the flourishing of the visual aspects of theatre during these same periods. And yet this flourishing was undeniable. The ceremonies of the Catholic church were by tradition highly visible, and when in many churches in the late middle ages these took on distinct theatrical elements, that only increased this element. Elaborate flying machines were added to the already utilized pavilions, processions and costumes as early as the fourteenth century. Filippo Brunelleschi (1337–1446) was particularly ingenious at devising elaborate mechanical effects for both indoor and outdoor productions, most notably huge flying machines covered in lights and clouds and bearing groups of angelic figures (Ogden 2002: 106).

Visual Communication in the Theatre

447

A report from the town chronicler on the visual effects in one of the best documents of the late medieval passion plays, that at Valenciennes in 1547, gives some idea of the importance of this aspect of the production: We saw strange and wonderful things. The machines of Paradise and of Hell were absolutely prodigious and could be taken by the populace for magic […] For we saw Truth, the angels and other characters descend from very high, sometimes visibly, sometimes invisibly, appearing suddenly. Lucifer was raised from Hell on a dragon without our being able to see how. The rod of Moses, dry and sterile, suddenly put forth flowers and fruits. Devils carried the souls of Herod and Judas through the air. Here Jesus Christ was carried up by the Devil who scaled a wall forty feet high: there He became invisible. Finally He was transfigured on Mount Tabor […] The eclipse, the earthquake, the splitting of the rocks and the other miracles at the death of Our Lord were shown with new marvels (Nagler 1959: 47–49).

3 Visuality in the renaissance theatre As the princely courts of the renaissance increased in power and wealth during the 15th century, one favored way of displaying their culture and their financial status was by the staging of elaborate theatrical spectacles. The machinery of these built upon the religious spectacles of Brunelleschi and others and many of the leading artists of the period, already in most cases supported by these princes, turned their visual imaginations to elaborate court entertainments, the Italian intermezzi and later the English masques. Leonardo da Vinci, at the Medici court in Florence was, like Bunelleschi, both a painter and an inventive machinist, skills that he combined in the creation of spectacular visual effects. Da Vinci was, like many of the leading Italian artists of his time, fascinated by the development of visual perspective, as can be clearly seen in his famous Notebooks. A favorite subject for Italian artists developing the new art of perspective in the early 16th century was the “ideal city”, and the elaborate architectural perspectives and nodal points of Europe’s new baroque cities in the following century clearly drew upon these visual patterns. So too did the stage designs of the period, often created by architectural draftsmen and in many cases serving almost as visual models for the new city perspectives.1 As the century passed, visual spectacle became an ever greater part of court theatrical entertainments, the intermezzi, and in addition to mere spectacular display, these productions took on an increasingly communicative function. The burden of this communication was not carried, as in the revivals and imitations of classic dramas, by a literary text, but primarily by mimes or dancers with elaborate costumes and properties. The subject matter was almost invariably  A particularly striking example of this is a scenic design by Baldassare Lanci for Vedova, a performance at the Medici court in 1569, depicting an urban perspective in Florence which did not yet exist but which was being constructed at that same time by Giorgio Vasari.

448

Marvin Carlson

pastoral or mythological, but since these were court presentations, often connected with princely alliances through marriage or other major state occasions, equally important to the visual messages of the intermezzi were political messages, often celebrations of the sponsoring house or of its specific achievements. Such celebrations at the major courts were well documented, and a rich visual record of them remains. As the intermezzi became more elaborate, and more cohesive as narrative forms, many of their features, including the tradition of visual spectacle, were incorporated into the new genre of opera, which appeared in Italy at the close of the 16th century. The close ties between these two entertainments may be clearly seen in work generally considered the first opera, Jacopo Peri’s Dafne, first performed in Florence in 1597. Peri had also served as one of the composers as well as a performer in one of the most famous of the intermezzi, for a Medici wedding in 1589. The designs for this celebration, according to the British art historian Roy Strong, “are crucial, for they are the earliest mass-disseminated illustrations of what became a norm throughout Europe for theatrical visual experience for the next three hundred years, the proscenium arts behind which receded ranks of side wings, the vista closed by a back shutter” (Strong 1984: 140). The first permanent theatres in England, like Shakespeare´s Globe, placed their emphasis upon acting and text, with visual elements playing a distinctly secondary role. Even scenery and lighting was depicted not visually but orally. As elsewhere in renaissance Europe, the real development of scenic spectacle in England can from Italy, brought back of traveling designers like Inigo Jones. As in Italy, the most visually-oriented of the English renaissance theatre productions were at court – where the masques served much the same function as the Italian intermezzi – communicating impressions of royal glory and munificence, and drawing upon a generalized body of classical and mythological reference. The tension between text-based and spectacle-based performance resulted in a famous conflict in England between Jones and the major dramatist Ben Jonson, who felt, rightly, that his dramatic text was being continually subordinated to visual effect. After Jonson departed, the court masques under Jones and his successor John Webb, became even more essentially visual entertainments. The Jonson/Jones controversy over the superiority in theatre of the spoken text or visual display has re-emerged in various forms in almost every generation since then. So, for example, John Dryden in a prologue of 1674 lauded the Spartan virtues of his own Drury Lane theatre, a “Plain Built House”, which focused on literary values, and condemned the seductions of the rival Dorset Garden, where “Scenes, Machines, and empty Operas reign” (Avery 1968: 173). Similarly, the Prologue to Richard Steele’s 1701 drama The Funeral complained that: Nature’s Deserted and Dramatick Art, To Dazle now the Eye, has left the Heart; Gay Lights, and Dresses, long extended Scenes,

Visual Communication in the Theatre

449

Demons and Angels moving in Machines, All that can now or please or fright the Fair May be perform’d without a Writer’s Care (Avery: cix)

On the Continent, with the development of opera as a genre, the interest in visual display did not diminish, and indeed remains an important element of opera to this day, but it was soon replaced at the center of the theatrical experience by music, which had been a far less important part of the intermezzi than such visual elements as dance, costume, scenery and machines. No longer were the predominant figures in this genre visual artists like Da Vinci or Buontalenti but composers, like Monteverdi and Cavelli. Although relegated to a distinctly secondary position, visual spectacle remained an important part of the tradition of displaying power and wealth which was a major element of Renaissance Italian court life and which remained so in the many and often even more wealthy and ceremony-oriented courts that developed following the Italian and subsequently the French model, across Europe during the 17th and 18th centuries. Every such court had one or more court painters, and usually some organizer of elaborate visual spectacles for the court, not only operas, but ballets and a wide variety of festivals for royal weddings, funerals, baptisms and other state occasions. The most famous visual contributors to this tradition was the much admired and influential Galli-Bibiena family, masters of the ornate style of baroque painting and especially architecture. Only a few of their actual sets have survived, but hundreds of surviving sketches and engravings provide an excellent record of the monumental and intricate spaces created in the stage designs of this prolific family. Their work extended out from Italy to Austria, France, Germany, Czechoslovakia, the Netherlands, Sweden, England, and Russia. From the family founder, Giovanni, to his great-grandson Carlo, the work of this family extended from about 1690 until Carlo’s death in 1787, on the eve of the French Revolution, which signaled the coming to an end of the culture that had supported this artistic tradition.

4 Visuality in the baroque theatre During the baroque period, France mounted a challenge to Italy in a wide variety of cultural fields, including the theatre. In the theory and practice of dramatic literature, French neoclassic writers clearly had taken the lead by the beginning of the seventeenth century, and thanks to Lully, the French were able to mount a serious challenge to Italian opera by the end of that century. In the visual side of theatrical production, however, Italy clearly remained the European master throughout this period. In large part this was due to the ubiquitous Bibienas, but also to the innovations of Giacomo Torelli, the most influential designer of stage machinery and effects since the Renaissance. Visually, Torelli settings were not

450

Marvin Carlson

greatly different from the wing and drop arrangement developed by the intermezzi designers, but Torelli created an elaborate system of ropes, pulleys and wagons on tracks that allowed scenery to change in full view of the audience without a single visible human being involved. This magical effect became standard in theatres throughout Europe and may still be seen in a few surviving eighteenth century theatres, most famously the royal Swedish theatre at Drottningholm. In 1645 Torelli was invited to Paris by the Cardinal Richelieu, to introduce his new machinery to the operatic stage there. Although the developing aesthetic of neoclassic dramaturgy resisted such visual ornamentation, even leading dramatists like Corneille and Molière created what were called “machine plays”, where the emphasis was on spectacular visual effects. On the whole, however, French neoclassic theory encouraged in both its serious and comic drama an emphasis upon the language and the emotion. The major visual ornamentation was in the costumes, especially of the heroes and heroines of tragedy, whose elaborate skirts (for both sexes, but larger for women), ribbons, sashes, puffed sleeves, feathers and wigs were based upon the most fashionable and excessive court fashions. Thus, by the end of the 18th century, when the French Revolution served as a signal for the end of the age of monarchy, most European countries (even the independent English, to a certain extent) had become accustomed to looking to French as a model for theatrical organization and aesthetics, especially in the spoken theatre, since Italy still remained a strong competitor in the opera. Young writers in serious drama across the Continent looked to Racine as a model (in plot, in structure, even in verse patterns), while young comic writers modeled themselves on Molière. The French stage also provided the model for the visual surroundings of these dramatic texts, which tended to be simple and minimal. The side wings and back drop were almost universal, and most theatres had only a limited number of such settings – often a salon, a palace room, a harbor, an urban street and a prison – and so small and unparticularized was the repertoire that the same settings could be used over and over, providing only the most basic visual orientation for the acting (the great majority of neoclassic comedies for example needed only the basic salon set, the tragedies the neutral room in a palace). The relative simplicity of the visual side of spoken theatre, compared to the expected visual display of opera and ballet should not cause us to be unaware of a major source of visual communication in the spoken as well as the operatic theatre quite aside from whatever was being offered to the gaze of the spectator upon the stage. From the renaissance onward, an important part of the communication of the power and the wealth of the sponsoring prince or monarch was not just the offerings on stage, but the entire physical surroundings of the theatre itself. The attending audience member was encompassed by a dream world of paintings, statuary, curtains, columns, crystal chandeliers and elaborate gilded ornamentation. Nor was the visual feast limited to physical décor. Until the rise of the illusionistic realist theatre of the later nineteenth century, the development of

Visual Communication in the Theatre

451

modern stage lighting, and the concomitant darkening of the auditorium, the audience itself often provided as important a part of the visual experience of theatre as did anything on stage. Wealthier patrons communicated their social superiority to each other and to their inferiors not only by the elaborate dress they displayed at the theatre, but equally importantly by where they sat, the entire auditorium being socially coded in great detail. In the royal theatres in France, the King and Queen displayed themselves in boxes on either side of the stage, where they shared visual attention with the performers, and the boxes nearest to them, quite inferior for actually seeing the stage, but excellent for being seen, were the next most prestigious after those of the monarchs themselves.

5 Visuality in the romantic theatre The coming of romanticism revolutionized many aspects of the theatre, not least its visual aspects. The interest in “atmosphere,” common to various pre-romantic movements such as the Sturm und Drang in Germany, the Gothic movement in England, and the early melodramas and “dialogue pantomimes” in France encouraged a greater interest in visual effects in the theatre. This was certainly reinforced in France by the disappearance of the old aristocratic order, which has supported the increasingly sterile and abstract neoclassic tradition by a new audience that sought a more striking and immediate sensual experience in the theatre. The Comédie Française languished, while the new melodrama theatres, specializing in visual spectacle and impassioned acting, drew large and enthusiastic publics. The public interest in the visual is suggested by the great vogue in the early years of the nineteenth century for panorama and diorama theatres. Equally significant is the change in the training and orientation of scenic designers in the theatre. From the renaissance onward, the usual training for scenic designers (who did not study with other scenic designers) was architecture, producing countless monumental if rather sterile palaces, prisons, and bridges for the opera, and more modest settings for the spoken drama equally devoid of atmosphere or specificity (both key concepts for the romantics) and indeed of nature itself. In this context, David Garrick’s hiring of a landscape painter, Philip de Loutherbourg, as the chief designer of Drury Lane theatre in 1771 marked a major change in the function and role of the visual on the British stage. The sort of images beloved by the romantics, sunrises or sunsets, storms in the mountains or at sea, now became more and more common in theatres, and rivaled or often surpassed the plays or actors in attracting audiences. De Loutherbourg even created a mechanical theatre, the Eidophusikon, which proved more popular than regular theatres in \London, and which presented no plays or actors, but only “visions of nature” created by a combination of paintings, mirrors, bits of scenery moved by pulleys, and projections from Argand lamps.

452

Marvin Carlson

The success of the Eidophusikon inspired other experiments with purely visual theatrical entertainments, first in London and then in Paris and elsewhere. The most popular were the panoramas, huge paintings, primarily of cityscapes or military encounters which completely surrounded the spectators and sought to increase the illusion by employing natural lighting and adding three-dimensional elements in the foreground to blend into the two-dimensional representation. The panorama was then brought back to a more distinctly theatrical form with the creation of the diorama in 1822 by Louis Daguerre, one of the inventors of the photograph. Audiences of the diorama, about 300 in number, would enter a small theatre-like space facing a stage where 10–15 minute shows were presented, without actors or text, but with elaborate changing scenic effects, a landscape changing from winter to summer, a moonlit forest changing to day, a violent rainstorm followed by a sunlight sky and a rainbow. Panoramas and dioramas may still be seen today, but their great international vogue was in the early nineteenth century, when their popularity anticipated that of the cinema that in the next century began with the utilization of many of their techniques. The contemporary theatre also of course had close ties to them, as the example of de Loutherbourg demonstrates. This was especially the case in the theatres devoted to melodrama, the major new form that emerged in the opening years of the century, exactly at the same time as these visual spectacle houses. As a dramatic form melodrama, which dominated the nineteenth century theatre, is perhaps best remembered as a popular form, with clearly defined good and evil characters and strongly emotional plots, emphasized by music. Equally central to the form, however, was its very heavy reliance on elaborate scenery and visual effects. In part this grew from the sheer attraction of spectacular visual effects, tying this tradition into the long-standing theatre interest in such display that goes back to the medieval hell scenes and the court festivals of the renaissance. Guilbert de Pixérécourt, one of the founders of the melodrama, not only provided elaborate descriptions of such special effects – fires, explosions, floods, earthquakes – but even invented special machinery to achieve some of the more difficult of these. This interest in dazzling visual effects remained an important part of the melodrama throughout the century, and indeed became even more important as improvements in lighting and stage technology allowed authors and designers to achieve ever more elaborate displays. The leading melodrama author of the mid–late 19th century, Dion Boucicault, was particularly associated with such visual spectacle, and beginning in the 1860s many of his most successful dramas were known as “sensation-plays” for their reliance upon such spectacular effects as fires and floods. As Garrick’s appointment of De Loutherbourg to Drury Lane suggests, the preromantic experiments of the late eighteenth century and the full tide of romanticism which followed had an enormous effect upon the visual elements of theatrical productions. One of the fundamental differences between classicism and romanticism was the tendency of the former to deal with universals and general truths

Visual Communication in the Theatre

453

and that of the later to stress the individual, the specific. An obvious effect of this general difference upon theatrical production was the fading away of the old generic sets – the salon suitable for every upper class comedy, or the neutral palace room suitable for every Racinian tragedy. Instead came a desire for settings specific to each play, followed soon by a passion for historical accuracy. Victor Hugo, leader of the French romantics, arguing for historically precise and accurate physical settings in historical drama, calling the setting a “silent character”, without whom the scene was incomplete. “Would the poet dare to assassinate Rizzo elsewhere than in Mary Stuart’s chamber?” he asked, or “burn Joan of Arc elsewhere than in the Old Marketplace” (Hugo 1967: 3:63).

6 Visuality in the realist theatre The more middle-class audiences that dominated the European theatres of the nineteenth century generally felt that the best art should educate as well as entertain, and many leading theatre directors took advantage of this interest by pursuing Hugo’s vision of historically accurate designs. One of the most influential of the pioneers of this new “pictorial drama” was Charles Kean in London in the 1850s. Kean was a Fellow of the Society of Antiquaries and his Shakespearian productions were exhaustively researched, offering scrupulously detailed and exact reconstructions of historical costumes, settings and properties. A new kind of visual communication in the theatre was the result, with audiences looking to the theatre, as they might to an illustrated encyclopedia, to expand their visual knowledge of previous eras and foreign climes. This interest in historical and foreign visualizations become ever more elaborate and dominant as the century progressed, and its impact and impressiveness was steadily increased not only by a growing body of visual knowledge about other places and other times, but also by an ever increasing development of lighting and stage technology, allowing designers not only a greater range of visual material but a greater technical ability to realize their visions. Spectacle dramas more elaborate than anything the renaissance or baroque designs could have imagined became common, and once again in many theatres the scenic display moved to the center of the theatre experience. Despite the presence of Sarah Bernhardt in the title role of Sardou’s Theodora, written especially for her in 1884, the production’s monumental visual recreation of the Byzantine court was by far the most attractive and talked-about part of the productions. The audience expectation of a museumlike experience was indicated by the fact that for months after the opening, Sardou carried on a lively debate in the newspapers with historians over such questions as whether forks were known in Byzantium or whether the Empress’s cabinet might contain blue glass. Such was the depth of the production’s research, however, that the dramatist was rarely, if ever, caught in an error (Carlson 1972: 1930). At the very end of the

454

Marvin Carlson

century, in 1899, came one of the greatest of such spectacles, William Young’s Ben Hur, featuring real horses and chariots in a recreation of the Antioch Circus, galloping pell-mell toward the audience on a rotating treadmill with a vast unrolling cyclorama behind them. Such spectacle was, however, to be swallowed up before long by the cinema, which could achieve such effects even more stunningly. The silent film produced a rather primitive Ben Hur as early as 1907 and an epic rivaling or surpassing the stage version by 1925. By this time the visual spectacles of late romanticism and the melodrama were disappearing from the stage, unable to compete in this respect with the film. The last important refuge of detailed visual realism on the stage was the realistic social drama in the tradition of Ibsen, which is still the dominant form of the serious drama in the United States today. The various experimental forms that emerged in the early years of the twentieth century, futurism, surrealism, dada, and expressionism offered a very different visual experience for their audiences, since all were closely connected with movements in the visual arts and carried over stylistic features of those movements into theatrical production. This interpenetration of the arts also brought some of the greatest names in the art world, like Picasso and Chagall to contribute to stage design of the period.

7 The theorizing of theatrical visuality Although in certain periods the stage painter or even the machinist dominated the theatrical experience, it was not until the very beginning of the twentieth century that a few theorists began to consider seriously the contributions of the visual to a theatre discourse that had been almost exclusively concerned with the literary text ever since Aristotle. Wagner helped to prepare the way for this development by his championing of the “total work of art” (Gesamtkunstwerk) which would draw equally upon text, music, the visual arts, and dance for its unified effect. Although in practice Wagner had much less interest in the visual arts, his theories and his music had an enormous influence in directing the attention of theatre theorists and practitioners to these arts in the years that followed. In Russia, the symbolists and the artists of the Ballets Russes brought the visual arts to the center of theatrical expression. Fyodor Komissarzhevsky’s “synthetic” theatre closely echoed Wagner’s total work of art, seeking a unity of music and words, movement, the colors and lines of costume and décor, the effects of lighting (Komissarzhevsky 1990: 148). In Germany, Walter Gropius described the theatre work of the Bauhaus, which had an enormous influences on that art, as uniting movement, organic and mechanical bodies, form, light, color, and verbal and musical sound (Gropius 1922: 25); In France the leading director and designer Gaston Baty argued that if the theatre wished to present “an integral vision of the world” it must rely primarily upon plastic expression – color, light, gesture and so on – in order “to evoke the

Visual Communication in the Theatre

455

world beyond the text”, a comment much in line with the symbolists of the previous generation (Baty 1949: 219). The two major figures who introduced both a concern with a total work of art and a shift in emphasis from the dramatic text to the visual spectacle were Adolphe Appia in Switzerland and Edward Gordon Craig in England. Through their own innovative stage designs and use of light and also through their widely read theoretical publications on theatre art, these two figures exerted a profound international influence on the theatre of the early twentieth century and changed the widely-held view of scenery as mere decoration or illustration to a significant art form in its own right.

8 The contribution of semiotics An even more detailed and substantial contribution to a modern theory of design and indeed of theatre in general was provided in the 1930s and 1940s by members of the Prague Linguistic Circle. Drawing upon the structural linguistics developed by Ferdinand de Saussure as well as on the writings of the Russian formalists, these theorists developed a methodology called semiotics, concerned with human communication in all its aspects, not only linguistic, but cultural, sociological, and artistic. Although their subjects of study and their strategies of analysis varied considerably, all of those who considered the theatre regarded all of its elements – physical, visual and verbal – of equal importance in understanding how this art form communicates to its audience, thus providing both a justification and a systematic practical grounding for the theatrical analysis of this communication which allowed for the development of serious theatre analysis and theory in the later twentieth century The second World War, with its devastation of Czech cultural and intellectual life, brought the work of the Prague Circle to an end, but their concerns and modes of approach were taken up again by scholars in other countries after the war. This post-war semiotic theory was at first applied to literary analysis, then to the arts of painting, music, and the cinema before finally, in the 1960s, being applied by French and German theorists, to the theatre. From the beginning such analysis placed much emphasis on the physical and visual elements of the theatre. Thus when Roland Barthes, whose explorations of semiotic analysis date back to the 1950s, was asked in 1963 how such analysis might relate to theatre, he suggested that the theatre offered one of the greatest challenges to semiotic analysis, due to the density and variety of its signs, including setting, costume, lighting as well as the words, movements and gestures of the actors (Barthes 1972: 262). Theatre scholars in France and Italy soon took up this challenge. Particularly influential was a 1968 essay by Tadeusz Kowsan, “The Sign in the Theatre”, which proposed thirteen specific sign systems for analysis in the study of theatre as what Barthes called a “cybernetic

456

Marvin Carlson

machine”, a system of communication. Significantly, these systems were predominantly visual – nine out of the thirteen – mime, gesture, movement, makeup, hairstyle, costume, accessories, décor and lighting. The remaining three were auditory – word, tone, music, and sound effects (Kowsan 1968: 73). The visual took on a new centrality in semiotic analysis, as may be seen in Jim Carmody’s “Semiotic Reading” of Molière’s The Misanthrope, which was presented as a representative example of this type of analysis in Roach and Reinelt’s 1992 anthology of contemporary critical approachs. Visual analysis forms the heart of this essay, which is summed up in a sentence near the end: “[T]he scenography borrows visual ideas from a variety of periods and cultures, using the visual language of postmodernism to provide a hospitable setting for the intertextualities of Barrlett’s and Fall’s recreation of Molière’s classic text” (Carmody 1992: 126). Central to the analysis is a detailed description and symbolic analysis of the production’s major visual element, “an imposing piece of exercise equipment, a postmodern blending of Universal weight machine and high-tech guillotine”, which “stands close to center stage”. Carmody uses “this scenic element, with its wealth of b-cultural signifieds” (1992: 118) to build up a discussion of the overall semiotic and symbolic structure of the production, blending classic French references with others from contemporary Hollywood. During the next decade or two semiotic theory allowed for a more articulate exploration of the visual and other operations of the theatre experience, but also turned away from simple physical descriptions and analyses of productions to the dynamics of reception and of the cultural operations of the semiotic process. During these same years, theatrical performance was developing in two almost opposite directions, both challenging the entire Western tradition of theatrical performance but both greatly increasing the importance and complexity of the visual element in this performance. On the one hand, the emancipation of theatrical performance from the theatre, in the form of such manifestations as site-specific and more recently immersive theatre, increased the experience of the live in theatre and also increased the importance not only of the visual, but indeed of all the human senses, including touch and smell, rarely much utilized in the theatrical work of earlier times. On the other, the increasing availability and utilization of visual technology, especially the live video and digital images, increased the importance of the visual but challenged the traditional centrality of the living body of the actor.

9 Changing spatial configurations and the visual From the Renaissance to the late twentieth century the standard Western concept of a theatre was an enclosed space subdivided into a space for a seated public to watch and listen to the fictional events taking place in an opposing space, nor-

Visual Communication in the Theatre

457

mally framed by a proscenium arch, in some cases literally in the form of a huge picture frame. Throughout much of this history, visual focus was divided between these two spaces, the auditorium and the stage, but toward the end of the nineteenth century, reinforced by the rise of realism, the stage became the privileged focus of the audience’s vision. The surrounding auditorium space was plunged into darkness to discourage the visual field from expanding beyond the stage picture itself. A variety of experiments with stage space and with actor/audience relationships in the later twentieth century consciously set out to disrupt and challenge this traditional mode of experiencing the theatre. In productions generally grouped under the title of “site-specific”, performances and audiences were moved outside of theatres into an almost endless variety of other locations, the majority of them out-of-doors. The focus of the audience generally remained upon the space utilized by the performers, but here instead of the limited and controlled visual field offered by the traditional theatre, the field expanded to include the total surrounding environment. Although this encompassing world was also accessible to audiences in the classic Greek theatre, it was not the specific bearer of visual information that is implied in the very name “site-specific”. At least in theory, a sitespecific performance is designed for a unique site, indeed may be inspired by that site and its history, and thus could not be as meaningfully produced anywhere else. One of the first major modern examples of such work illustrates these features clearly. This is the famous Russian mass spectacle, The Storming of the Winter Palace, staged by Nikolai Evreinov in 1920 in the giant square before that palace and re-creating one of the great events of the recent Russian Revolution. The actual buildings involved in that event, along with such accoutrements as tanks and armored cars, not to mention a case of more than 2000, filled the visual field with images communicating as much as possible a sense of witnessing the event itself. In such an event one could see the literal fulfillment of Victor Hugo’s observation, that the visual surroundings of an historical event served to reinforce the historical re-enactment in a manner impossible to realize within the more limited means of a traditional stage. Over the next century, the concept of site-specific theatre developed in many different ways, but all involved a range of visual communication much richer and more varied than the old-style proscenium theatre could offer. Early site-specific work generally kept the audience in a traditional stable position, facing the performance as they had done in conventional theatre. Later, in such variations as promenade performances, they were required to move about to different locations, thus vastly increasing the perceptive field in which the performance was located. In the opening years of the twenty-first century the audience was given further freedom to explore the performative world in so-called immersive theatre, in which spectators were free to wander about an environment at their own whim, choosing

458

Marvin Carlson

or not to direct their gaze at a performer, or in fact moving about a performer to view her from another angle. The construction of visual meaning or at least the weaving together of visual material into a meaningful pattern is here far more under the control of the individual spectator than in conventional theatre and much closer to the way the world around us is perceived and interpreted in everyday life.

10 Mediatization and the visual in contemporary theatre If site-specific theatre and its variations has moved the viewing of theatre closer to the experience of real life, another major direction in contemporary experimental theatre has gone in the opposite direction but with equal or greater implications for the expansion of visual communication in the theatre, moving away from real life into the realm of simulation and the mediatized image. No matter how elaborate the scenic design of traditional theatre, an important part of the dramatic world, that located in the past or the off-stage present, by the conventions of the genre was almost never presented visually, but only verbally. The invention of photography, and subsequently, of the film provided a possibility for the theatre to present much more easily visual images of presumably offstage scenes, although until fairly recent times this possibility was not extensively developed. Doubtless technical difficulties somewhat slowed the development of successful mixtures of live action and film, although probably a greater obstacle was an aesthetic one. Ironically, the very period when the new technology first of photography and then of film was developing, the theatre was dominated by the aesthetic of realism, to which film and theatre bore quite different relationships. Film, like theatre, was able to present a temporal narrative, and was able almost at once to surpass theatre in certain aspects of realism by its spatial mobility and flexibility. Theatre, however, sought realism in realms inaccessible to the absent world of the film: in a strong attention to the physical and emotional reality of the objects and human beings actually present for the observation of spectators. The mixing of filmic sequences and living actors thus inevitably called attention to the constructed and conventionalized nature of the “reality” of each, and so long as realism remained the dominant approach in the imagined worlds of theatre and film, this mitigated strongly against their mixing. Occasionally photographs or films were used to add to the visual possibilities of the stage picture in the twentieth century theatre, but their technical means of production were so apparent to the audience that they were essentially incompatible with fourth wall realism. Not surprisingly, they were much more often utilized in productions that either made no attempt to work within the illusionistic model (such as the documentary theatre of Piscator) or else directly challenged illusion-

Visual Communication in the Theatre

459

ism by exposing its conventions, as in Thornton Wilder’s Skin of Our Teeth. The latter, one of the best known such anti-illusionistic plays of the century, in fact begins with a series of visual images which considerably increase the visual range of the audience, offering views both of the real “offstage” world of that audience (the façade of the theatre where the play is being presented) and the imaginary “offstage” world of the characters (the exterior of the Antrobus’s suburban home (Wilder 1972: 109–10). In the later twentieth century, the dominance of illusionistic staging lessened and, even more importantly, the totalizing effect of the illusionistic stage was increasingly challenged by theatrical performances that no longer sought to hide the methodologies of production, even in stagings that still drew upon certain of the conventions of realism. Still, technical limitations generally restricted the use of filmic images to what Thomas Irmir has called a “second level” of images, not yet fully integrated into the scene, but rather like visual “footnotes to the stage picture” (2003: 22). Their primary use, Patrice Pavis has suggested, was “to provide background or ironic comment on the stage action (1992: 130).

11 The impact of live video in the theatre A different role, both more central and more substantial, for such visual images began to appear toward the end of the century in the work of a number of European directors, perhaps most notably that of Frank Castorf in Berlin. Castorf utilized a live camera in a production for the first time in 1987, in a production of Ibsen’s Enemy of the People. In one memorable sequence, a single video monitor on an otherwise empty stage showed the figure of the actor playing Stockmann confined in a tiny basement room in the theatre, a striking image of isolation and despair. Following filmic rather than theatre visual practice, at times the entire room was shown, and at others only the face of the actor, or even a fragment of the face, more intimate even than a filmic close-up. Even in this first experiment, Castorf had clearly departed from the use of video as “footnote” or “background” and moved the image to a central position in the theatrical experience. By the early years of the new century, Castorf had begun to mount very large video monitors above his stage, on which were projected live images not only from backstage, but from locations outside the theatre, or indeed on the stage itself, so that an audience could simultaneously see an actor in the conventional way as well as a simultaneous film-style close up on the screen above the stage. Moreover one video cameraman filming that image was also visible on stage, so that the visual filed included not only the actor, and the image, but the means of production of the image. Thus the visual communication of the performance expanded also to provide visual information on the means of that communication. The visual complexity of this approach is suggested in a review of a Castorf staging of a Dostoevsky adaptation in 2003:

460

Marvin Carlson

Cameras, it turns out are mounted all over this labyrinthine container-set, parts of which we never see directly except as glimpses through open doors or peeks behind patched-up windows […] On occasion, cameras are picked up by actors who follow other actors into this Everyman-warren that is also a forlorn No Man’s Land (Remshardt 2002: 27).

One of the most elaborate and extensive examples I have witnessed of the use of live video to multiply the visual space of performance as well as to blend onstage and offstage space is the Marat/Sade created in 2000 by the Triumviratus Art Group of Varna, Bulgaria. Here, as in most of Castorf’s work of this period, a very large video monitor/screen was mounted directly above the stage, so that the audience was always offered a double focus of live and mediatized action. De Sade wore a kind of harness with a small video camera mounted on it about two feet out in front of his face. This was often used to provide close-ups of his own facial reactions, but also, when turned outward, to provide a “Sade’s eye view” of people and events on the stage to supplement the more normal total stage picture available to the audience. Small video cameras were also carried by the two nurses who served as Sade’s assistants, and these provided other potential perspectives and, more importantly, visual access to offstage space, since the nurses from time to time, unlike Sade, left the stage with various “patients”. They followed the actors/characters/inmates (often the distinction was unclear) into various offstage spaces, but these were never the “localized off” of contiguous spaces in the imaginary world of Charenton, but rather the real backstage of the theatre, the dressing and green rooms, and then the corridors and public spaces shared by the audience, and finally even outside, where at one point we saw on the video the actress who was playing Corday outside in a parking lot finishing a cigarette and warming up for her encounter with Marat (this image overlapping the onstage shouts of “Corday! Corday!” by the chorus, urging her to join them). This visual blurring between onstage and offstage, public and private space, encouraged by the use of live video, also involved a blurring between actor and character. A certain aura of madness hung over the dressing rooms, and in one particularly striking sequence the chorus began their challenge to Marat (“We want a Revolution, NOW!) with mutterings in the Green Room. The video then follows their growing anger and passion as they ran out of the backstage areas, but not onto the stage, rather through the theatre lobbies, until they burst into the auditorium itself, on the balcony near the stage, from which they raged at Marat among the startled spectators. The portable cameras that had been following them from the backstage space of the actors through the lobby space of the public continued to follow them into the auditorium, still running. Thus the theatre auditorium, largely banished from the performance’s visual field since the rise of realism, was reintroduced as a part of that field, and a remarkable collage of spaces was achieved, live and mediatized space, actor and audience space, onstage and offstage space, real and mimetic space, all simultaneously present. Another group, Big Art, based in the United States, has through technology made visuality so central to their productions that these cannot really be discussed

Visual Communication in the Theatre

461

in non-visual terms. Basically, Big Art combines live action and live video while mixing the live video images to create totally new visual combinations, so that the visual field is operating simultaneously on at least three levels. Camden Mason, the leader of this group, calls this technique “Real-Time Film”. In a major theoretical study of Big Art, Jason Farman calls their productions “surveillance spectacles”. Farman stresses that both in the experience and the analysis of such work, one must begin with the scoptic process, and he draws equally upon phenomenological and semiotic theory to discuss how Big Art requires that the spectator become a visual receptive and analytic body (2009: 185). Certainly such productions acknowledge an audience familiar with and comfortable with the mediatization of images, but the continual interplay of mediatization and of directly viewed bodies and indeed of the working video apparatus itself (and of the living bodies that operate it) provides a visual effect, and a view of space, quite different from the simple absorption of reality by image, or presence by absence, or of life by “liveness”. Rather these presumed alternatives are placed in constant tension, the eruptions of the bodies on video into the living space of the stage or even the auditorium emphasizing the porosity of the boundaries between the new visual spaces made accessible through digital technology and the spaces on or around the physical stage itself. The continual negotiations with and exposure of the mechanisms of visual communication in the theatre in placed by such experimentation at the very center of the theatrical experience, and the visual side of that experience, so often subordinated to the linguistic in the history of Western theatre, is moved to the position of prominence it more often holds in any of the world’s other theatre and performance traditions.

References Aristotle. 1895. The Poetics. Trans. by S. H. Butcher, section VI. London: Macmillan. Avery, Emmett L. 1968. The London Stage 1700–1729. Carbondale: Southern Illinois University Press. Barthes, Roland. 1972. Litérature et signification. In: Critical Essays. Trans. by Richard Howard. Evanston: University of Chicago. Baty, Gaston. 1949. Rideau baissé. Paris: Bordas. Carlson, Marvin. 1972. The French Stage in the Nineteenth Century. Metuchen: Scarecrow Press. Carmody, Jim. 1992. Alceste in Hollywood: A Semiotic Reading of The Misanthrope. In Janelle Reinelt and Joseph Roach (eds.), Critical Theory and Performance, 37–48. Ann Arbor: University of Michigan Press. Farman, Jason. 2009. Surveillance Spectacles: The Big Art Group’s Flicker and the Screened Body in Performance. Contemporary Theatre Review 19(2): 181–194. Gropius, Walter. 1922. Der Arbeit der Bauhausbühne. Die Bauhausbühne: Erste Mittelung. Weimar. Hugo, Victor. 1967. Preface à Cromwell, Oeuvres complètes, 18 vols. Paris: Le Club Francais du livre.

462

Marvin Carlson

Irmir, Thomas. 2003. Das Theater der neuen Sehens: Zum Einsatz von Video und Film bei Frank Castorf, Rene Pollesch und Olaf Nicolai. Quadratur: Kulturbuch 5. Kowsan, Tadeusz. 1968. The Sign in the Theatre. Diogenes 61: 52–80. Komissarzhevsky, Fyodor. 1990. Myself and the Theatre. New York: Dutton. Nagler, Alois Maria. 1959. A Source Book of Theatre History. New York: Dover. Ogden, Dunbar. 2002. The Staging of Drama in the Medieval Church. London: Associated University Presses. Pavis, Patrice. 1992. Theatre at the Crossroads of Culture. Trans. by Loren Kruger. London: Routledge. Remshardt, Ralf. 2002. Touring The Insulted and the Injured: Theater der Welt in Köln, Bonn, Düsseldorf and Duisburg. Western European Stages 14(3). Roach, Joseph and Janelle Reinelt (eds.). 1992. Critical Theory and Performance. Ann Arbor: University of Michigan. Strong, Roy. 1984. Art and Power: Renaissance Festivals 1450–1650. London: Boydell Press. Wilder, Thornton. 1972. The Skin of Our Teeth: Three Plays. New York: Harper and Row.

Vincent Campbell

20 Analysing impossible pictures: Computer generated imagery in science documentary and factual entertainment television Abstract: This chapter is concerned with the trend for an increasing use of Computer Generated Imagery (CGI) in science documentary and factual entertainment, and what analytical approaches are needed to understand and interpret the variety of uses of CGI in such programmes. A focus on science programmes foregrounds the intrinsic questions surrounding the documentary project, and its claims to the real, which have often been equated with scientific goals. The chapter explores models for attempting to understand the use of computer animation in documentary (e.g. Hight 2008, Hoeness Roe 2011). Through examining some examples of CGI science programmes, the chapter explores whether there are other representational traditions that might be having an identifiable influence on their use of CGI, such as traditions within particular scientific disciplines, sub-categories of documentary, and treatments of science in wider popular culture.

1 Introduction The analysis of documentary film and television has been problematized in a number of ways in recent decades. The production environment has changed dramatically, for instance, with on the one hand an explosion in the spaces for documentaries to be screened and viewed around the world on dedicated channels, but on the other an increased exposure of documentary to the pressures of commercial screen production, competition for audiences, and concentration of production into the hands of a small number of companies with concomitant impacts on what kinds of documentaries can be made. Related to this, new genres have challenged the status and position of documentary as a primary site of factual screen content and increasingly hybridised formats have appeared blending elements of documentary with genres such as reality television, game shows, and even soap operas, into new types of factual entertainment (see Kilborn 2003 for a discussion). Criticisms of these developments have come from all quarters. Documentary makers complain about commercial imperatives and the need for spectacle to attract audiences undermining quality and standards in factual screen content production (Palfreman 2002; Ellis 2005; Byrne 2007). Paralleling these problems, and perhaps a reflection on them, scholarship has developed in ways which have fundamentally challenged documentary’s “claims to the real” (Winston 1995), with particular

464

Vincent Campbell

strands of documentary research rejecting outright even the possibility of the documentary project, as in Trinh T. Minh-Ha’s oft-cited claim “there is no such thing as documentary” (Minh-Ha 1993: 90). One distinctive contemporary development that further complicates the discussion and analysis of factual screen content has been the trend for an increasing use of Computer Generated Imagery (CGI) in documentary and factual entertainment. Whilst CGI had been utilised before in documentaries, alongside conventional drawn imagery and animation, the production of Walking With Dinosaurs (UK, 1999) by the BBC represented a seminal moment in the potential for the use of CGI in factual entertainment. By placing CGI as the central visual component of its representation of dinosaurs the series took a significant risk with the technology but its reception by audiences, breaking records for factual programme viewing in Britain, showed CGI was not only not a barrier to audiences, but a potential means of attracting them. The proliferation of programmes utilising CGI since can be directly linked to the new commercial environment, in its capacity “to expand the scope of documentary representation and to make it more visually seductive, raising the bar in terms of primetime documentary spectacle” (Hight 2008: 22). The use of CGI is also of analytical interest, as it is possible to consider the extent to which CGI “has the capacity to enhance and extend […] the epistemological potentiality of the documentary form” (Honess Roe 2011: 225). This chapter is concerned with the analytical challenges posed by documentary and factual entertainment programmes that utilise CGI extensively. Specifically it is concerned with how such texts relate (if at all) to traditional analytical approaches to understanding documentary and factual entertainment, and what analytical approaches are needed to understand and interpret the variety of uses of CGI in such programmes. In order to focus the discussion the chapter concentrates on programmes concerning scientific subjects. A focus on scientific topics foregrounds the intrinsic questions surrounding the documentary project, and its claims to the real, which have often been equated with scientific goals. The tensions between scientific documentaries, often seen as the most conservative of documentary forms with their emphasis on authoritative exposition of scientific knowledge, and the use of techniques considered by many to be antithetical to documentary’s potential claims to the real, serve as a clear point of analytical interest. For instance, natural history programmes are widely heralded as exemplars in public discourses around television quality1 whilst at the same being critiqued by scholars as not documentaries (Bousé, 2003) and as not proper science programmes (Jeffries 2003), issues which will be returned to later in the chapter. Science documentary and factual entertainment programmes have also been the main type of factual programmes that have begun to utilise CGI extensively

 The face of BBC flagship natural history programmes, David Attenborough, was voted most trusted celebrity in Britain in 2006 by Reader’s Digest, for instance.

Analysing impossible pictures

465

over the last decade or so. Just as Walking With Dinosaurs made evident, the potential for CGI to be used to visualise subjects in ways impossible for conventional photography to capture is of particular potential benefit to scientific topics where the core subject matter, or elements of the subject matter (such as dinosaurs in “life”), may be of this kind, and thus it is not surprising to have seen a whole plethora of scientific programmes appearing that use CGI. As well as paleontological topics, scientific disciplines such as archaeology, medical and forensic sciences, geological and meteorological sciences, and astronomy have all seen CGI increasingly utilised in documentaries and factual entertainment programmes. Examples of some of these will be discussed in this chapter. In routinely depicting impossible pictures – whether it be extinct animals in motion, galaxies in deep space, or biological processes in human cells – scientific programmes offer a particularly significant challenge to making sense of the images that are produced. Whilst models for attempting to understand the use of animation in general and computer animation in particular in documentary have been developed, these have tended to focus on trying to relate CGI and animation to established notions of how conventional documentaries attempt to construct their claims to the real. In other words the focus has been on how to relate the CGI to representational modes either exclusively within documentary, or on occasion in relation to other modes of visual information design (Hight 2008). What these approaches don’t do however, is to consider whether there are other representational traditions that might be having an identifiable influence on the construction, positioning and framing of CGI within these programmes. These might include the representational traditions within particular sciences themselves, and how they have traditionally converted their subject-specialist knowledge for wider public consumption. Additionally, these might also include the representational traditions within the particular subcategory of documentary/factual programmes that the programme occurs within. Finally, these might include the representational traditions of other genres in which the subject matter regularly appears and/or which are explicitly being drawn on by the programmes. In order to make sense of CGI and the factual programmes that contain it, all of these elements need to be included in the analysis as this chapter will show through a consideration of a number of distinctive scientific topics and their treatment in a variety of documentaries and factual entertainment programmes that utilise CGI. Before discussing the use of CGI in specific documentaries and factual entertainment programmes, the chapter will begin by exploring the relationship between documentary, animation and CGI.

2 Documentary, animation and CGI The tensions over the boundaries of the category of documentary, and more fundamental claims to the real of “factual” film and television programmes, are particu-

466

Vincent Campbell

larly well illustrated by attempts to make sense of the use of animation in documentary. Despite animation featuring in documentary throughout film and television history, scholarly attention on its use is relatively recent (e.g. DelGaudio 1997; Wells 1997; Strøm 2003; Hight 2008; Bordwell 2009; Honess Roe 2011). One immediate issue for scholars addressing the use of animation in documentary, and entirely (or predominantly) animated documentaries, has been a perceived need to validate the labelling of such material as documentary. For instance, Ward says that the problem with doing this is “that there is a tendency to view documentary as a mode of discourse that will not allow such subjective, expressive aspects as we associate with […] animation” (Ward 2005: 82). More fundamentally, DelGaudio notes that since “an animated film “exists” only when it is projected – there is no pre-existing reality, no pro-filmic event captured in its occurrence – its classification as documentary can be problematic” (DelGaudio 1997: 190). Honess Roe states: The authenticity of a documentary and the power of its claim to be such a type of film are deeply linked to notions of realism and the idea that documentary images bear evidence of events that actually happened, by virtue of the indexical relationship between image and reality (Honess Roe 2011: 216).

This notion of “indexicality” is a core requirement of documentary, and whilst traditional animation might be problematic in terms of its fit with this definition (Honess Roe 2011: 216), the use of CGI more intrinsically challenges the centrality of indexicality. Indexicality: […] relies on the capacity of the photographic image, and of sound recording, to replicate what we take to be distinctive qualities of what they have recorded. That this is an assumption […] becomes increasingly clear as we bear in mind the ability of digitally produced sounds and images to achieve a similar effect: the sound we hear and the image we behold seem to bear the trace of what produced them. (Nichols 2001: 35, original emphasis)

Bordwell’s discussion of the problem of animation in documentary links indexicality to a second possible assumption of the documentary viewer that events depicted are “unstaged by the filmmaker” and that given “these two intuitions, unstaged events and direct recording of them, it would seem impossible to consider an animated film as a documentary” (Bordwell 2009). However, he argues that neither of these are necessary conditions for the classification of a text as a documentary, given that the use of scripted scenes and graphics are routinely used in texts “presented and labelled as documentaries” (Bordwell 2009). He argues that if “we see documentary films as tacitly asserting a state of affairs to be factual, we can see that no particular sort of images guarantees a film to be a doc” (Bordwell 2009, emphasis mine). Strøm goes further arguing for a shift from simplistic notions of photographic/ documentary “truth” to notions of “trust”, in that whilst it may no longer be “possible to believe in photography as a guarantee for truth” given the inherent manip-

Analysing impossible pictures

467

ulation of all images in documentary texts, “that does not mean we cannot trust the photography” rather it is “the context it is presented in and the credibility of the medium that presents it” which is more important (Strøm 2003: 54). Similarly Nichols asserts that it is the expectations raised through the documentary that what will be depicted is a “discourse of sobriety” (another being science) where viewers “expect to learn or be moved, to discover or be persuaded of possibilities that pertain to the historical world” (Nichols 2001: 39). The very label of “documentary” arguably carries a level of cultural and intellectual credibility that other genres of media content do not to the point where it is not only producers who clearly want to associate their texts with documentary (to elicit audience trust) but also scholars in trying to incorporate animated works into documentary analysis approaches, for instance attempting to position them within or in relation to Nichols’ model of documentary “modes”.2 Nichols’ modes have been reformulated over time but remain a paradigmatic focus of much documentary scholarship due to their versatility in encompassing the variety of techniques that have emerged in documentary (see Nichols 2001). An evident problem here is that “there seems to be a tendency to “squeeze” [a text] into a frame of reference in order to validate it, that is to say that if we manage to fit an animated film into one of Nichols’ categories we have, so to speak, proven that it is a documentary” (Rozenkrantz 2011). This problem is evident in the lack of consensus over which modes documentary animation belongs in, with claims that animation sits in the “reflexive” mode (DelGaudio 1997: 193), the “performative” mode (Strøm 2003: 52), and the “interactive” (or participatory) mode (Ward 2005: 95). Interestingly none of these attempts to fit animation into Nichols’ modes centre on its usage in the “expository” mode, even when they acknowledge the potential for its usage as such (e.g. Strøm 2003: 53; Ward 2005: 86). Yet, as the discussion below will show, the use of CGI for exposition alongside and in combination with other modes is a prevalent feature of contemporary science documentaries and factual entertainment programmes. Another approach has been to try to develop a set of distinctive modes within animated documentary (Wells 1997). Wells’ model reflects a scholarly emphasis on the subjective and expressive uses of animation at the expense of nuanced uses of animation for exposition, with three of his four modes, the “subjective”, the “fantastic” and the “postmodern” being closely overlapping modes focused on aesthetic experimentation, expression of inner states, surrealism and other exotic uses of animation (Wells 1997: 43–5). Only the first mode, the “imitative” is concerned with use of animation in ways that “conform to “naturalist” representation and use the generic conventions of some documentary forms” (Wells 1997: 41).

 As Rozenkrantz (2011) has suggested this may be in as much an effort to validate the credibility of the analyses themselves, as much as it is anything to do with validating the notion of animation in/as documentary.

468

Vincent Campbell

Honess Roe suggests, particularly with the inclusion of a postmodern mode, that Wells’ modes reflect a “trend in scepticism regarding the documentary project” contemporary to Wells’ development of his modes (Honess Roe 2011: 225). Here the lack of interest in or attention to animation for exposition can be related to some scholars’ rejection of the capability of even “conventional documentary representation (as in, live action) to access or show reality” (Honess Roe 2011: 225) let alone animation. Honess Roe rejects this position, proposing that animation in documentary can be more usefully analysed in terms of its functions within particular documentaries, and suggests these manifest themselves “in three key ways: mimetic substitution, non-mimetic substitution, and evocation” (Honess Roe 2011: 225). Mimetic substitution is where “animation illustrates something that would be very hard, or impossible, to show with the conventional live-action alternative and often it is directly standing in for live-action footage” (Honess Roe 2011: 226). In effect this is a use of animation akin to the documentary technique of using dramatic reconstructions where footage is not, or could not be, available, and where the animation is “made to closely resemble reality, or rather, the look of a live action recording of reality” (Honess Roe 2011: 226). A seminal example of this would be the series Walking With Dinosaurs which included scenes such as having the cameraman “hide” behind trees to avoid the gaze of the computer-generated dinosaurs and having a T. Rex “spit” on the camera “lens” (Honess Roe 2011: 226). Honess Roe’s other functions relate to the use of animation in ways which explicitly do not attempt verisimilitude but instead offer visual interpretations of other elements in the content, with “non-mimetic substitution” relating to visuals tied to specific documentary elements such as using images of animals linked to interviewees’ voices, and “evocation” referring more to visualisation of individuals’ subjective experiences (Honess Roe 2011: 226– 7). Although developing the focus to acknowledge that “animation is not used in the same way in all animated documentaries” (Honess Roe 2011: 225), Honess Roe’s functions betray a predisposition against thorough scrutiny of mimetic modes of representation, and a far greater interest in aesthetic experimentation and expression (i.e. the non-mimetic and evocative). Honess Roe does not elaborate on issues relating to which specific representational tropes are being reproduced in any given mimetic substitution, for instance, and as the discussion in this chapter will indicate this is a crucial additional factor to incorporate into analytical frameworks for this type of content. What counts as a mimetic image depends on the specific representational tradition images are being presented within, and is not necessarily simply equitable with documentary live action footage, particularly given that some things that are being visualised in science programmes are beyond the capacity of humans to visually witness “live” in an unmediated fashion, as is evident from the use of time-lapse, highspeed, infra-red and night vision cameras for instance. These visual techniques

Analysing impossible pictures

469

are sometimes reproduced in CGI in some of these programmes as well, blurring the potential for identifying clear boundaries between mimetic, non-mimetic and evocative functions of CGI further still. The final approach considered here attempting to codify animation in documentary focuses specifically on digital animation (Hight 2008). Hight’s model whilst in some ways usefully over-lapping with models previously discussed, adds additional elements taking the development of analytical frameworks for such programmes somewhat further. Hight makes an important distinction between computer “mediated” images, and computer generated images as illustrative of a continuum along which computer animation techniques may be utilised (Hight 2008: 13). At one end of the continuum, computer mediated images are “elements of the indexical and photographic [placed] within animation and morphing sequences during post-production” (Hight 2008: 13). This might consist of, for instance, compositing of text captions over live action footage or manipulation of live action footage to highlight particular components of the image or sound (e.g. the speed, colour/sound balance or focus might be altered to direct the viewers’ attention to particular elements within the image). Many of the traditional techniques of documentary montage are now done using computers, and have been augmented by sophisticated (and largely invisible to the viewer) computer mediation techniques, such as compositing. At the other end of the continuum are entirely computer generated images where key components in an image have been created within the computer (Hight 2008: 13) such as the dinosaurs in Walking With Dinosaurs (and which may be much more obvious to the viewer). Crucially, Hight recognises that the use of computer imagery from mediation to generation intersects with a “discursive continuum within animation more generally, that between photorealism and the exploration of purely symbolic or abstract forms” (Hight 2008: 13). This second continuum extends from “symbolism” at one end, focusing on “iconic and metaphoric forms of representation” (Hight 2008: 13) to “photorealism” at the other end with a goal of making the images otherwise indistinguishable from conventional documentary live action footage. Hight notes how this goal focuses on achieving photorealism, that is “replicating cinematography rather than human perception and experience of reality itself” (Hight 2008: 13). In the discussion of the programmes analysed in this chapter the “inherent tension” (Hight 2008: 13) between documentary claims to the real and representational conventions of how claims to the real are made in documentary are shown to be particularly foregrounded by science programmes where imagery is a product of negotiation between the competing narratives and representational traditions of documentary (and factual entertainment) as well as those of the sciences covered. Hight utilises this schematic approach to identify three key modes evident in the use of digital animation. The first is called the “symbolic expositional” mode and refers to the use of informational graphics and animations […] and “simplistic

470

Vincent Campbell

three-dimensional modular reconstructions of events” (Hight 2008: 14). The value of this analytical category is that such imagery does not involve the imitation of or mimetic substitution for live action footage but is fundamentally expository in terms of intent and design, and reflects long-used conventions in certain types of documentary. As Hight notes, in historical documentaries for instance, the “animated map” has become a convention used to the point of cliché” (Hight 2008: 14), and the use of diagrams and graphs is widespread in science documentaries (such as the contentious “hockey stick” graph in programmes about global warming). Being widely recognised tools for the simplified conveying of sometimes “complex natural and social phenomena” (Hight 2008: 14) beyond documentary, their use has always been part of the evidential claims to the real made by documentary. Digital animation techniques provide far greater scope, however, for both variation in the visual design and display of such information, and its incorporation into other elements of the documentary image, such as superimposing a computer generated map onto live action footage of a location (a common technique in archaeological documentaries, for instance). The second mode Hight identifies is called the “graphic verité” mode (Hight, 2008: 17) whereby digital animation is utilised in photorealistic dramatic reconstructions. Here, just as with conventional dramatic reconstructions in documentary, events are created in the absence of, or impossibility of acquiring, live action footage but with the aim of invoking the indexicality of live action footage. Hight notes a paradox here in such programmes appearing to both want to invoke the claims to the real of the representational strategies being so painstakingly reproduced through using this mode, whilst at the same time often being promoted and marketed in a way that foregrounds their artifice and constructed nature (Hight 2008: 18–19; see also Scott and White 2003). The third mode Hight identifies is called the “invasive surveillance” mode, and refers to the addition of digital animation techniques to existing technologies that “extend the range and penetration of the documentary lens”’ (Hight 2008: 19). Existing techniques that might be augmented (or reconstructed) by computer imagery include “special effects such as time-lapse photography, time-slice photography and motion-control photography within primetime nature documentary, the use of surveillance tools within investigative reporting” and also the variety of techniques that extend surveillance “into the interior of the human body” (Hight 2008: 19) such as x-rays, endoscopes, MRI scanners etc. Hight considers this as a kind of “penetrative voyeurism” which utilises “an intimacy mediated through the comforting sterility of digital technologies, perfectly matching the dispassionate penetration of the gaze of medical science itself” (Hight 2008: 21).3  This contrasts with the superficially similar concept of “penetrative animation” (Wells in Ward 2005: 93) referring to the capacity of animation to visualise other types of “internal” states, such as states of mind.

Analysing impossible pictures

471

These three modes may be used discretely or in combination, and the interplay of these approaches provides important opportunities for critical analysis in terms of how and where these techniques are utilised, whether they are used in different ways across different types of programmes, and in this case across different sciences. As indicated earlier, however, once we begin to look at specific examples, it is possible to see how these approaches need to incorporate the additional representational traditions that contribute to the way CGI is used and incorporated into documentary and factual entertainment, in order to identify and make sense of differences between sub-categories of programmes, and individual programmes. To illustrate this, the chapter now discusses three examples in three distinctive areas of science, placing a consideration of each example’s use of CGI in the context of documentary conventions, external generic influences, and representational traditions of the sciences on display.

3 Planet Dinosaur Planet Dinosaur (UK, 2011), a six part primetime series, is a recent and typical example of the “extinct animal show” (Campbell 2009) sub-category of programmes that have proliferated in the decade or so since the success of Walking With Dinosaurs. A defining characteristic of extinct animal shows is the use of extended sequences of photorealistic CGI depicting extinct animals as if captured by live location filming. Like its precursor Walking With Dinosaurs, Planet Dinosaur is predominantly presented in its overall style like a “blue chip” natural history documentary, “programmes devoted to observing “spectacular” animal behaviour displayed within “timeless” natural habitats and all relatively “untainted” by human intervention, whether presenters in front of the camera, producers or animal trainers behind them, or humans interacting with, or on, the “pristine” animal habitats depicted” (Cottle 2004: 83). Typical examples of these in conventional natural history programmes are the BBC programmes presented by David Attenborough, such as the contemporaneous Frozen Planet (UK, 2011), and these types of programmes carry with them a high level of authority with audiences (Jeffries 2003: 527). As mentioned earlier, blue chip natural history programmes are not an unproblematic category though, having been widely criticised in scholarship of natural history on screen for presenting a largely “romantic image of the natural world” (Jeffries 2003: 543), and offering “period piece” fantasies that leave out the people and politics that contribute to how the natural world is, in favour of a concentration on the spectacle of the animals and landscapes on display (Bousé 2000: 14–5). Planet Dinosaur thus attempts to frame its CGI depictions of extinct animals within a particular representational style of natural history films that have high audience appeal and trust, even though that category is highly contested in terms of its claims to the real.

472

Vincent Campbell

Fig. 1: Blood on the Lens in Planet Dinosaur.

Planet Dinosaur also uses the trope from Walking With Dinosaurs of breath and fluids from the dinosaurs appearing to hit the lens of the camera (for a discussion of this in Walking With Dinosaurs see Scott and White 2003), and in this case specks of blood from a carnivore’s kill not only hit the lens but are morphed into the programme’s logo on the title screen of each episode, making it a signature trope of this series (Figure 1). Other programmes have taken this conceit of presence further, for instance by placing an on-screen “time-travelling” presenter shown supposedly interacting with extinct animals, and in the series Prehistoric Park (UK, 2006) even having people bring extinct animals back to the present day. Such conceits themselves relate to a distinct category of natural history programmes with on-screen explorers like Jacques Cousteau and Steve Irwin, but both these and the use of blood on the lens type shots are invoking a particular trope of natural history films. The trope asserts a notion of authenticity or authority through the proximity of the filmmaker to the subject exemplified by the close-up, though this has been criticised as a kind of ‘false intimacy’ in natural history films (Bousé 2003) eliding the degree of construction and artifice evident in natural history film narratives that would be seen as unethical or problematic were the subjects human rather than animals (for a detailed critique see Bousé 2000: 20–28). In Frozen Planet, for instance, the revelation that a sequence of a polar bear mother giving birth to cubs in a den were shot in a zoo, rather than in the wild as the original sequence implied, drew some significant public criticism (Singh 2011). So, even though the use of blood on the lens shots foregrounds the artifice of CGI extinct animal shows (the animal, the blood and the lens all being computer generated) what is being

Analysing impossible pictures

473

reconstructed is this illusion of intimacy, the conceit of allowing viewers to see animals “from stroking distance” (Bousé 2003: 124) widespread in natural history films. An area of criticism and controversy in extinct animal shows in paleontological terms has been the veracity of the CGI reconstructions on display. This raises underlying questions about the relationship between these programmes and the representational tradition within palaeontology, paleoimagery, and in the specific case of Planet Dinosaur some interesting issues relating to the other aspect of its use of CGI. The paleontological questions such programmes raise relate to two levels of representational choices – choices of how periods of prehistory and their animals have been selected, and on a more immediate level choices relating to the specific ways in which extinct animals have been represented. In terms of the more immediate level, critics have questioned the representation of extinct animals filmed in life involving the incorporation of speculation, such as the sounds, colours, and behaviours of animals. Walking With Dinosaurs was particularly criticised in this sense with concerns that there were no attempts to explain or validate the decisions made behind the representations made within the programmes (see Campbell 2009 for a discussion). A separate “making of” programme was produced, and there was a similar supporting programme made for Planet Dinosaur (called How to Build a Dinosaur) but where this more recent series differed from its predecessor is that it includes clear sequences designed to address the veracity of the photorealistic CGI sequences presented. Unlike some other extinct animal shows that broke up the CGI sequences with presenters or experts explaining the science behind the depictions, Planet Dinosaur uses a different strategy. Retaining the voice-of-god narration (from actor John Hurt) photorealistic CGI sequences are followed by explanatory sequences using computer generated and mediated imagery clearly in a symbolic expositional mode (Figure 2). Photographic images of fossilised bones are depicted as if on a computer-screen’s window display, augmented with text, graphs, and other imagery as the narration explains the evidence behind the depictions just witnessed. The change in the type of imagery here is interesting, in showing a clear shift from asserting the validity of the depictions of dinosaur behaviour through reproducing natural history film styles, to using scientific display techniques when the goal is validating the scientific basis of those depictions. A final point is worth making about the representational choices made in a programme like Planet Dinosaur. Analysis across extinct animal shows has found that they exhibit clear patterns in their representation of periods of prehistory that reflect the representational traditions of paleoimagery, the specific practice of reconstruction of extinct animals based on the available scientific knowledge of the day (Campbell 2009). Planet Dinosaur conforms to several of these identifiable paleoimagery frames (see Campbell 2009 for a detailed discussion). For instance, these programmes (much like blue chip natural history in general) have a tendency

474

Vincent Campbell

Fig. 2: Symbolic Exposition in Planet Dinosaur.

to focus on “megafauna” – the largest and most exotic animals – but the predominance of megafauna is largely unquestioningly accepted by audiences and critics. Even when parallels between extinct animal shows and fiction films like Jurassic Park (USA, 1993) have been made, it has been the shared use of CGI rather than the shared focus on megafauna that has drawn commentary and critique. Planet Dinosaur begins with a programme about two dinosaur predators bigger than T. Rex and references to this and other – presumed to be already familiar – dinosaurs, like Diplodocus and Allosaurus, are made throughout the series. These assumptions of knowledge on the part of the audience are overt examples of ingrained paleoimagery frames in producers, content, and audiences. Other paleoimagery frames are also present in Planet Dinosaur, such as a particular focus on species illustrating evolutionary change, with a whole episode on feathered dinosaurs, and throughout the series there’s an evident focus on extinction, with animals reconstructed often only to show their demise at the end of episodes as their environment changes pushing them to extinction. In terms of the use of CGI, the point here is that the choices that have been made in terms of what and how to reconstruct extinct animals through CGI have also been shaped by representational traditions within paleoimagery that stretch over time, across different types of media (art, sculpture, film) and different types of content (scientific illustration, fiction film, factual entertainment television).

Analysing impossible pictures

475

4 Perfect Disasters Another prominent sub-category of documentary and factual entertainment programmes to have begun to use CGI extensively involves the depiction of extreme weather and disasters relating to natural hazards. Unlike natural history programmes, however, trends in factual programmes on weather and natural hazards (earthquakes, volcanoes etc.) have generally met with some significant public criticism, even as audiences respond favourably to them. Television critics have derogatively dubbed many of these programmes “weather porn” (Boddy 2000). Series such as Stormchasers (USA, 2009–2012) which follows tornado hunters in the USA, or the Witness series of programmes from National Geographic which uses minimal captioning/narration over visual imagery entirely captured by amateurs caught up in the midst of disasters, are criticised for placing the emphasis increasingly on the voyeuristic spectacle of extreme weather and natural disaster footage, rather than attempting to contextualise and explain either the sciences behind such events, or their socio-economic and cultural consequences. CGI has provided film-makers with the opportunity to extend the scope of depicting extreme weather events and natural disasters beyond the timeframe of the history of filmic and photographic capture. Alongside extinct animal shows regularly depicting the asteroid strike that wiped out the dinosaurs, a series of programmes have gone back in time to recreate environmental disasters from prehistory all the way back to events such as the creation of the Earth and the Moon (via a massive collision in the early solar system) in series such as Prehistoric Megastorms (USA, 2008) and Catastrophe (UK, 2008). Perfect Disasters (UK/USA, 2006) is an example of a subset of programmes that go in the other direction using CGI to depict hypothetical disasters occurring in the future, and had as one of its promotional taglines “this is not science fiction, this is science prediction”. Produced by Impossible Pictures, the company that made Walking With Dinosaurs for the BBC, the series’ format consisted of a number of hypothetical near-future scenarios mixing talking head sequences with experts explaining the science behind these scenarios intercut with dramatizations of groups of characters experiencing the events. The six episodes respectively hypothesise a “Super Tornado” hitting Dallas, Texas, a “Solar Storm” affecting New York City, a “Mega Flood” engulfing London, Sydney experiencing a “Firestorm”, Montreal facing an “Ice Storm”, and Hong Kong being buffeted by a “Super Typhoon”. CGI used in the series relates to all three of Hight’s modes (2008). First, photorealistic CGI is used to depict the natural hazards and their material consequences within the dramatized sequences of the hypothetical events. Second, the expository sequences, via narration and expert talking heads, are accompanied by symbolic expositional schematic CG visualisations, for instance using CG maps to illustrate how a super typhoon might begin, and why Hong Kong might be a likely location to be hit by one. Third, embedded within some of the dramatic sequences

476

Vincent Campbell

Fig. 3: Disaster as it happens in Perfect Disasters.

are examples of CGI as invasive surveillance. On occasion the dramatic sequences are effectively placed “on pause”, shifting from the narrative exposition of the disaster events to highlight the science of a particular experience, such as sequences where images of actors depicting particular traumas, such as drowning or hypothermia, are replaced with x-ray like schematics of their bodies whilst the narration explains the effects of those difficulties on the human body. As with Planet Dinosaur there appears to be a clear differentiation between the use of CGI for different kinds of claims to the real, with photorealism being used in dramatized sequences where verisimilitude is paramount (Figure 3), and more symbolic and invasive CGI being used for scientific exposition (Figure 4). A fuller explication of the uses of CGI in Perfect Disasters requires addressing other contributing factors however. As mentioned above, extreme weather and disaster programmes have been noted, and criticised, for particular aesthetic trends, and the use of speculative dramatizations in Perfect Disasters, in part a choice arguably shaped by those trends, additionally brings in the potential influence of dramatic representational traditions, specifically those of the disaster movie. Considering first the representational tropes of weather and disaster programmes one can contrast their development with the natural history film context of Planet Dinosaur. In natural history film-making the difficult, effortful and often risky process of capturing imagery is part of such programmes’ specific appeal and the expertise of wildlife camera-people is often explicitly foregrounded in such programmes. “Spit on the lens” in conventional natural history films is a marker of the skill of the cameraperson to have got so close to the animals being filmed. In weather programmes, the notion of validity from a degree of intimacy through presence –

Analysing impossible pictures

477

Fig. 4: Symbolic Exposition in Perfect Disasters.

imagery captured during the extreme weather event or disaster from “within” the event – is also evident but here the validity is not related to the perceived professionalism or expertise of the camera-person. In fact, it is arguably quite the reverse, that it is amateur footage captured spontaneously, even accidentally, that entreats a sense of authenticity in weather and disaster programmes. Linked to the relative unpredictability of disaster events, and thus the difficulty in getting professional footage of events as they happen (save news footage, though here it’s more often of the immediate aftermath of events), amateur footage often stands as the only available footage of disasters as they occur (as in the multiple instances of home video footage of the Boxing Day Tsunami). At one level then, the use of dramatized sequences enables Perfect Disasters to offer imagery which can reproduce this increasingly dominant visual trope within weather and disaster programmes of disasters “as-they-happen”, even deliberately constructing elements in imitation of the amateur recording of events such as clunky camera movements and focusing problems, to try and associate the contents with the perceived authenticity such imagery has with audiences. Another element here is the notion of authority of the non-expert witness to events, and whilst this too can be traced to trends in weather documentaries favouring not only amateur footage but also survivor stories as central content, it is arguably also a reflection of wider generic influences on the representation of weather and disaster in documentary and factual entertainment. Like Walking With Dinosaurs following in the wake of Jurassic Park, CGI disaster documentaries have followed a resurgence of the disaster movie, also linked to the use of CGI in films such as Twister (USA, 1996).

478

Vincent Campbell

Disaster movies have a long pedigree and have specific generic conventions (see Keane 2006) that are reproduced quite closely in Perfect Disasters’ dramatized sequences. For example, disaster movies are typically focused around a group of protagonists clearly identifiable as heroes (often scientists or specialists like firefighters) and villains (often politicians recklessly ignoring the disaster risk), and Perfect Disasters does this as well. Heroes are depicted rescuing their families, or escaping the perils of the disasters, for instance, in the episode “Ice Storm” this includes a power engineer driving through a cascading collapse of electricity pylons. Spectacular scenes of mass destruction occur, however, these are reserved for the built environment, with hardly any scenes depicting human deaths (even of the villains), and perhaps the shifts in life-threating scenes from dramatization to invasive CGI and sober expository discourse are as much a response to the potential problem of not delivering the disaster movie trope of protagonists’ deaths alongside the other tropes of disaster movies, as they are about shifting the focus from spectacular dramatizations to scientific exposition.4

5 Crime 360 The final example to be considered here is the true crime series Crime 360 (USA, 2008- ) which depicts police investigations into criminal (usually murder) cases from initial report to arrest of the prime suspect with the distinctive selling point of using CGI to depict the contribution of crime scene forensics to the investigations. The series draws on two distinctive strands within true crime documentary and factual entertainment in its overall composition. First, crime shows have been amongst the most prevalent to combine techniques of documentary with reality television formats often taking advantage of the increasing use of routine recording of police work, such as through cameras in police helicopters, cars, interview rooms and booking rooms, allowing the production of series such as World’s Wildest Police Videos (USA, 1998–2002). Shows like the long running COPS (USA, 1989–) take a different approach through using embedded camera-people producing verité-style live action footage of on-duty officers, though again the appeal of such shows is their presentation of real cops in real police situations. Crime 360, like similar series The First 48 (USA, 2004–), focuses on murder detectives working specific cases, and uses a predominantly verité style, augmented by scenes with the investigating police officers offering the core exposition of the procedures of the case through unscripted but at least partially staged sequences, such as discussions between the investigat-

 Whether the producers were concerned about scenes of death taking the voyeurism of future disasters too far, or whether this is simply to ensure the widest possible sales through conforming to variations in international broadcasting taste and decency guidelines, is unclear.

Analysing impossible pictures

479

ing officers about their next line of enquiry, or discussions between the detectives and forensic scientists that offer explanations of procedures like obtaining latent fingerprints through using superglue. It is notable that such programmes invest in a kind of authority, and accompanying visual style, quite distinct from that of either extinct animal shows like Planet Dinosaur, or disaster shows like Perfect Disasters. Here, claims to authority and authenticity relate to the degree to which the imagery and the narrative that accompanies it are seen to be focalised around or coming from literally the “Authorities” – the exposition coming from the police themselves at the crime scene, in the police station, and on the streets. Unlike the other two programmes discussed in this chapter, photorealistic CGI is not extensively used with live location footage dominating the content. However, the series’ title relates to a very specific application of photorealistic digital imaging technology used by the crime scene investigative teams that have been appropriated by the programme-makers and included as part of the distinctive appeal of the series. The series specifically follows cases where the crime-scenes are captured using either 3-D laser scanners and/or 360 degree digital cameras. These produce photorealistic images of crime scenes that can be navigated around and are used as part of the criminal investigation, for instance by comparing a suspect’s account of events with the crime scene imagery, or to consider the possibility of single or multiple suspects being sought. Interestingly there’s no overt reflexivity in this series about tensions over images as evidence, these new tools are treated as merely helpful extensions of existing crime scene imagery (like traditional crime scene photos, suspect mug-shots etc.). The main use of CGI in the series occurs in relation to the criminal forensics components of the cases. A typical example would be the use of CGI to depict a victim’s body in outline, perhaps with the skeleton on view, showing the trajectory and damage of a bullet or weapon as it entered the body. Like Perfect Disasters then, there’s clear evidence of invasive surveillance of the body in this series, although here photographic and film images of dead bodies are not shied away from, and the CGI clearly does function as a form of penetrative voyeurism, rather than as a side-stepping of depictions of the dead as it arguably is in Perfect Disasters. These sequences often start with an image from the 360 degree camera or 3-D scanner, or a more conventional crime scene or autopsy photograph, before replacing that image with a wire-frame representation, picking out particular aspects, such as items over a body dumped in a car trunk, in symbolic expositional mode, then moving to further invasive CGI of victims’ bodies. The one other type of CGI evident in the series occurs in episode openings where the original audio of the phone call to police that initiates the investigation is used, and is accompanied by a rotating image of a bright green audio waveform as a background image, a rare instance of a more overtly expressionistic use of CGI amidst otherwise highly expository use of CGI. The second main strand of true crime reflected in the series relates to its particular focus on criminal forensics. Moreover, the particular use of CGI in this

480

Vincent Campbell

series reflects the complex interplay between factual and fictional screen representations of forensic science. Crime 360 follows a distinct tradition of documentaries concerned with criminal forensics series such as The New Detectives (USA, 1996– 2005) and Forensic Files (USA, 1996–). The success of these documentaries in the late 1990s directly influenced the development of the fictional television series CSI: Crime Scene Investigation (USA, 2000–) in which forensic science became the focus of a police procedural drama series, rather than an incidental background component as it had been in earlier crime dramas. Where CSI, and a subsequent plethora of rival forensics-oriented police dramas, differ from the factual programmes lays in the presentation of the forensics. Forensic Files, The New Detectives and the like, tended to depict the forensics in relatively straightforward documentary fashion, using dramatic reconstructions for scenes from/relating to the crime, crime scene photography and film where available, and talking head interviews with those involved with the case, including the forensic experts explaining their techniques and the outcomes of their investigations in the respective cases. The processes of forensic investigation could be edited into the time frame of a television programme through the focus on completed cases even though it was often acknowledged that investigations took place over many years in some cases, and that some particular types of forensic investigation routinely take weeks and months to complete. CSI and its ilk, however, need greater immediacy and a sense of urgency typical of the “race against time” type scenarios of police procedurals, and take significant dramatic licence with the time it takes to conduct forensic investigations, speeding them up using special effects and CGI to make the forensics themselves part of the dramatic and visual spectacle, a typical example being tracing the flight of a bullet into a body and visualising the internal damage done to organs. The impact of the “race against time” investigative format of police dramas has impacted on the style and format of factual series, such as The First 48, and one could argue that Crime 360’s significant use of CGI to augment the sequences concerning forensic components of investigations is a direct recognition of audiences’ familiarity and enjoyment of CSI-type programmes and their depiction of criminal forensics. Here then, symbolic exposition might be the surface goal of much of the CGI here, but it also explicitly makes references to uses of this kind of CGI in popular fiction where the usage is primarily for spectacle and dramatic effect not scientific exposition. Crime 360 then, gives the audience a visual spectacle of genuine criminal forensics but through what has become a highly recognisable representational frame of crime drama, not of conventional science documentary. Like CSI, Crime 360 elides the procedures and processes of criminal forensics for narrative impetus and visual drama, even whilst at the same time asserting an expository position through its use of CGI contextualised by its other documentary and factual entertainment representational aspects.

Analysing impossible pictures

481

6 Conclusion Impossible pictures are becoming a routine feature of documentary and factual entertainment programmes through the widespread and multi-faceted usage of CGI. Attempts to make sense of such imagery have concentrated on intrinsic features of the CGI itself and relating these to models of representational modes in documentary. The examples discussed here demonstrate how the use of CGI is in fact influenced by a range of factors beyond conventions within CGI and documentary. First, the subject matter itself brings with it representational conventions shaping the relationship between the use of CGI and claims to the real, investing different levels of credibility and authority in different types of representation, such as reconstructions of dinosaurs in palaeontology or of crime scenes in forensic science, impacting on how CGI is used across different subjects. Second, these inter-relate with the conventions and specific tropes which have developed as markers of authenticity and authority within factual programmes in subject areas, such as the authenticity invested in amateur footage in weather and disaster programmes, and need to be incorporated into analysis. Third, and perhaps with the greatest implications given the trends in factual film and television production, whilst CGI can be used to extend the scope of visual representation in factual programmes, from flying through galaxies to flying through the human body, and reconstructing lost civilisations and extinct animals, the examples discussed here show that programme-makers draw at least as much on the tropes and conventions of fictional genres as they do those of documentary and the subject matter when designing their CGI. Impossible pictures appear, then, not from an unproblematic application of computer technologies to discrete documentary representational modes, but from a complex interplay of representational traditions, with the dramatic and narrative conventions of fiction and factual entertainment having a significant impact, and needing to be incorporated into attempts to analyse and understand them.

References Boddy, William. 2000. Weather porn and the battle for eyeballs: Promoting digital television in the USA and UK. In: John Fullerton and Astrid Söderbergh Widding (eds.), Moving Images: From Edison to the Webcam, 133–147. London: John Libbey. Bordwell, David. 2009. Showing what can’t be filmed. Observations on Film Art, 4 March. Available at: http://www.davidbordwell.net/blog/2009/03/04/showing-what-cant-be-filmed/. Bousé, Derek. 2000. Wildlife Films. Philadelphia: University of Pennsylvania Press. Bousé, Derek. 2003. False intimacy: Close-ups and viewer involvement in wildlife films. Visual Studies 18(2): 123–132. Byrne, Ciar. 2007. Top film-makers rally to save BBC documentaries. The Independent, 8th September, 11.

482

Vincent Campbell

Campbell, Vincent. 2008. Environmental catastrophe risk as factual entertainment television in Perfect Disaster. Paper presented at the ISSEI Language and the Scientific Imagination conference, University of Helsinki, August 2008. Available at: http://www.valt.helsinki.fi/ optek/issei/section4/Session_71_Bakir/71_Campbell.pdf. Campbell, Vincent. 2009. The extinct animal show: The paleoimagery tradition and computer generated imagery in factual television programs. Public Understanding of Science 18(2): 199–213. Cottle, Simon. 2004. Producing nature(s): On the changing production ecology of natural history TV. Media, Culture and Society 26(1): 81–101. DelGaudio, Sybil. 1997. If truth be told, can ‘toons tell it? Documentary and animation. Film History 9(2): 189–199. Ellis, John. 2005. Documentary and truth on television: The crisis of 1999. In Alan Rosenthal and John Corner (eds.), New Challenges for Documentary. 2nd edition, 342–360. Manchester: Manchester University Press. Hight, Craig. 2008. Primetime digital documentary animation: The photographic and graphic within play. Studies in Documentary Film 2(1): 9–31. Honess Roe, Annabelle. 2011. Absence, excess and epistemological expansion: Towards a framework for the study of animated documentary. Animation 6(3): 215–230. Jeffries, Michael. 2003. BBC natural history versus science paradigms. Science as Culture 12(4): 527–545. Keane, Stephen. 2006. Disaster Movies: The Cinema of Catastrophe, London: Wallflower Press. Kilborn, Richard. 2003. Staging the Real: Factual TV Programming in the Age of Big Brother. Manchester: Manchester University Press. Minh-Ha, Trinh T. 1993. The totalizing quest of meaning. In: Michael Renov (ed.), Theorizing Documentary, 90–107. London: Routledge. Nichols, Bill. 2001. Introduction to Documentary. Bloomington, IN: Indiana University Press. Palfreman, Jon. 2002. Bringing science to a television audience. Nieman Reports 56(3): 32–4. Rozenkrantz, Jonathan. 2011. Colourful claims: towards a theory of animated documentary. Film International, 6 May. Available at: http://filmint.nu/?p=1809. Scott, Karen D. and Anne M. White. 2003. Unnatural history? Deconstructing the Walking With Dinosaurs Phenomenon. Media, Culture and Society 25(3): 315–32. Singh, Anita. 2011. Frozen Planet: BBC “faked” polar bear birth. Daily Telegraph, 12 December, accessed 22 August 2012. Available at: http://www.telegraph.co.uk/culture/tvandradio/bbc/ 8950070/Frozen-Planet-BBC-faked-polar-bear-birth.html. Strøm, Gunnar. 2003. The animated documentary. Animation Journal 11: 46–63. Ward, Paul. 2005. Documentary: The Margins of Reality, London: Wallflower Pres. Winston, Brian. 1995. Claiming the Real: The Documentary Film Revisited. London: British Film Institute.

Sarah Edge

21 Reading the Fenian photographs: A historically and culturally located study Abstract: This chapter examines how critical visual theory can be applied to readings of historical photographs. It opens by outlining key poststructuralist theories, related to visual communication studies and maps how these have informed a cultural studies perspective on interpreting historical photographs. An approach which, as the case study in this chapter reveals, requires the critical historian to apply a cross disciplinary approach. The main part of the chapter is a case study on one of the earliest known uses of photography in prisons. It examines an archive, housed in Dublin in Ireland, of hundreds of photographs of Irish men. Arrested without trial during the Mid Victorian period as Fenians they have been photographed in their own clothes, pose and style. Drawing upon histories of fashion, nationalism, colonialism, among others, the case study rebuilds the complex meanings held in the photographs and examines their role in the creation of discourses on the Irish.

1 Introduction The meanings held in photographs are slippery, shifting and change in different historical locations, contexts and cultural environments. This chapter will open by introducing the reader to a number of different theoretical debates on the processes involved in unpacking the meanings held in historical photographs. It will then move on to demonstrate how these approaches can help us understand the meanings held in a series of photographs of Irish Fenians taken in Ireland in the mid-Victorian period. Throughout his life Roland Barthes considered how the photograph functioned as a specific form of communication. While drawing from the discipline of semiotics, his work identified the peculiarity of the photographic message which, he suggested, functioned as a “message without a code” (Barthes 1977: 15–31). Under this hypothesis he likens the ability to read a photograph to a language; thus the reading of a photograph, as he notes, will always be “historical” because “it depends on the reader’s “knowledge” just as though it were a matter of real language (langue), intelligible only if one has learnt the signs” (Barthes 1977: 28). Such an approach has been further theorised by Burgin who explains how The intelligibility of the photograph is no simple thing; photographs are texts inscribed in terms of what we may call ‘photographic discourse’, but this discourse, like any other, engages discourses beyond itself, the photographic text like any other is the site of a complex ‘intertex-

484

Sarah Edge

tuality’, an overlapping series of previous texts ‘taken for granted’ at a particular cultural and historical conjecture (Burgin 1982: 144).

Such a conception of photography significantly alters the work of the photographic historian. John Tagg has written extensively on historical photographs and confirms that photographs “are never “evidence” of history; they are themselves the historical” (Tagg 1988: 65). Under this principle it is the role of the photographic historian to rebuild the language of the signs which have been captured in the photograph in order to make the meanings they once held “intelligible” to the modern viewer. Photographic historians who have examined photographic archives from the mid-Victorian period have tended to draw upon a Foucauldian conception of discursive practices. This is primarily because the first widespread uses of photography were connected to those new “institutions of power” in which visual messages work repressively upon those depicted. Studies have been undertaken on medical photography, prison photography, anthropological and ethnographic photography and charity photography. However this is also the moment when the new genre of domestic photography, more related to spectacle, appears, and I am going to return to these distinctions in more detail below. Photographic historians have identified that it took photography, as a new medium, some time to establish itself in what Mitchell calls its “pre industrial phase” (2004: 57). Thus in this intermediate phase it is suggested that the distinctive characteristics of different genres, such as domestic or prison, have yet to be established.1 This appreciation of different genres is important to my study; however, it is also vital to be mindful of other approaches to photography from other historical periods. Stuart Hall’s research on journalistic post-war photographs of black immigrants arriving in Britain via boat trains in the 1950s aptly demonstrates the complexities of rebuilding historical meanings from photographic archives. Hall confirms that “the past cannot speak”, except through its “archive”. Such circumstances necessitate an “exercise in interpretation […] considerable tact, historical judgement – in essences, a politics of reading” (Hall 1992: 106). This is because historical photographs are often tainted by our contemporary perception of historical events. Hall’s study reveals how the internal codes of dress, expression, pose and location that have been caught in these images challenge our contemporary perceptions. Hall asks: “What do we make of these images now? For one thing they contradict our expectations. Why are the men and women so formally dressed? Why does everyone wear a hat? […] why do they look so respectable?  For instance John Tagg’s (1984) work on early English prison photography produced by the West Midlands Police Force in 1860–2 notes that because the conventions of convict photography were not yet established, to the modern eye the photographs look more like domestic than criminal photography.

Reading the Fenian photographs

485

Where are the street fighters, the rude boys, the Rasta, the reggae? (Hall 1992: 107). Historical photographs need their meanings rebuilt through, what Holland (2000: 121) has aptly described as “an act of decoding or historical detective work”.

2 Photography, imperialism and the Irish The Fenian Crisis took place in Ireland during the mid-1860s and instigated the assembly of the photographic archive to be considered here. This crisis was a pivotal moment in colonial relations between Britain and Ireland, representing the first widespread cultural threat to British imperial rule. McClintock’s impressive study on the rise of imperialist ideas in popular cultural discourses (such as cartoons and advertising) confirms how, in “the last decades of the nineteenth century the term “race” was used in shifting and unstable ways, sometimes as synonymous with “species” sometimes with “culture”, sometimes with “nation”, sometimes to denote biological ethnicity or sub-groups within national groupings: the “English” race compared, say, with the “Irish” race” (McClintock 1995: 52). In Ireland these discourses attempted to mark the Irish as a race of “‘Celtic Calbians’ quite distinct from Anglo Saxons” (McClintock 1995: 52) by drawing upon differences in terms of language or culture.2 McClintock’s study identifies how the whiteness of the Irish “presented a dilemma for pseudo-Darwinian imperial discourses” and cites the work of Claire Wills (1991) who explains how such discourses were “compounded by the absences of the visual marker of skin colour differences which was used to legitimate domination in other colonized societies” (Wills 1991: 21). Luke Gibbons’s work on neo-colonialism in Ireland confirms this, arguing that “here was a colony whose subject population was “native” and “white” at the same time” (Gibbons 1991: 95). Post imperialist studies of photography confirmed how skin colour was a key signifier of “racial” differences which could be scientifically recorded by the camera and then given cultural meanings.3 However in the context of Ireland, as noted above, Irish whiteness was clearly a visual problematic for photography. Nonetheless, because popular culture required the creation of a set of visual signs that could mark the Irish as different, other visual media were employed, namely drawings, painting, political cartoons and literary illustrations. The most extensive study of such visual discourses has been undertaken by L.P. Curtis, who suggests that the “process of simianizing Paddy’s features” begins “between 1840 and 1890 with the 1860s [when the photographs reviewed here were taken] serving as a

 Wills (1991) argues that the Irish accent was used as a marker. McClintock (1995: 53) has added to this by suggesting that ‘the domestic barbarism of the Irish’ was employed ‘as a marker of racial differences’.  Edwards (1992); Ryan (1997).

486

Sarah Edge

pivotal point in this alteration of the stereotype”. This visual discourse worked to suggest “that the Irish were inherently inferior and quite unfit to manage their own affairs” (Curtis 1997: 29). This visual history is verified by other studies on paintings and cartoons in which the process of creating a set of visual signs, such as low forehead, red hair and large hands, were employed to signify the difference of the Irish. Cowling’s (1989) impressive review of Victorian paintings explains how a shared conceptual understanding was forged between artists and viewer so that certain signs came to signify distinct groups of people from which the veracity of the painting was judged. Curtis’s (1997) study does consider a few photographs and he reviews photographs from the Fenian Files, approaching them as a source of evidence to determine if the Irish face had simian features. Under the subtitle “Irish Realties”, he concludes that “only about six of the three hundred or so photographs contained anything resembling acute midfacial prognathism”, confirming that “the ape-men and the monsters of Punch and Judy are not to be found in this collection”(Curtis 1997: 90). Curtis’s reading of these photographs reveals a common error which has been refuted by those academics cited in the opening of this chapter. He approaches them as a source of evidence of the past rather than as historically discursive productions in their own right, an approach which will now be explored in this chapter. In some related research I have examined emergent uses of photography in Ireland (Edge 2004). Ireland had a rather unique position in relation to imperialist uses of photography. Dublin was a modern commercial location with a rising middle class and photographic portraiture thrived; moreover, because Beard’s patent,4 which restricted the commercial development of photography within England, did not apply to Ireland, innovative commercial practices developed slightly earlier. One of these was the commercial production and sale of photographs of emerging Irish nationalist leaders. The first was a lithograph print of a daguerreotype of Daniel O’Connell, available for purchase in 1847. This was followed in 1848 by lithographs from daguerreotype of the leaders of the Young Irelanders. Thus supporters of Irish independence could purchase the photographs from different retail outlets in Dublin or through newspaper adverts. These images held specific meanings for the purchaser and viewer in relation to fashioning an internal conception of Irish nationalism. These expensive commercially produced images promoted a message of middle-class Irish masculinity, respectability, passivity, control and individualism. At the same time Ireland was also pioneering the employment of

 In 1841 an entrepreneuring industrialist, Richard Beard, purchased a patent that was to restrict the setting up of daguerreotype studios in England; Wales and ‘Her Majesty’s Colonies and Plantations Abroad’.

Reading the Fenian photographs

487

photography5 in its prison system,6 and introduced the photographing of convicts in 1857. This instigated a state interest in the new process of photography and by 1863 a Select Committee on Prison Discipline examined photography as a means of surveillance and control, noting how convicts were forcibly photographed after conviction; if a person was convicted a second time, a copy of the photograph was sent to the head of the police (Slattery 1992: 224–231) Historically, then, when the Fenian Crisis took hold in 1867, photography was already being used by the state to record, examine and control criminals and by middle-class Irish nationalists to promote their “cause”. The Fenian movement was officially founded in 1865 as the IRB (Irish Republican Brotherhood) but it was more commonly known as the Fenians, a name taken from a mythical legendary warrior band, Na Fianna. Histories confirm how an interest and commitment to Irish nationalism become widespread in the post-famine period, with Kee’s history confirming that between 1863 and 1864 membership increased within “the urban and rural poor” (Kee 1979: 23). The 1860s also represented a new moment in the visibility of Irish opposition to British imperialism in England with increased activity in London, including the failed bombing of Clerkenwell Prison. Sensational stories appeared in the popular press, instigated by an increased need to “make sense” of the Irish and their opposition to British rule. Breandan Mac Suibhne and Amy Martin’s (2005) study identifies how the British press began to explain such violence through concepts of “terror”. Noting how “the London Times and many other mainstream newspapers repeatedly described the bombing as “terror”, “terrorism” and “terrorist” (2005: 115). And as previously noted, Curtis confirms how this “behaviour” was now “explained” in cartoons which visually imagined the Irish as an inferior race the “missing link”, less civilised and evolved.7

3 Photographing the Fenians The Fenian photographs were taken in this historical and cultural climate. In late 1865 the British Government closed the offices of the movement’s newspaper, arresting the staff. In February 1866 it suspended habeas corpus to allow the arrest and detention of “suspected” Fenians without a trial and between 1866 and 1867

 Prison photography was not introduced into England until 1871 Prevention of Crimes Act (Bates 2009; 72).  There has been little interests in the specific history of photography in Ireland in this earliest period; however, I am indebted to a currently unpublished PhD by Peadar Slattery 1992 for much of my background information.  It is worth noting that such explanation of Irish nationalist violence have proved to have significant longevity.

488

Sarah Edge

hundreds of men were imprisoned in various Irish jails, the majority in Mountjoy and Kilmainham prisons in Dublin.8 The unconvicted “Fenians” were photographed and these photographs are the subject of this study. The photographs are housed in the “Irish Crime Records” and “Fenian Photographs” at the National Archives of Ireland in Dublin – an overall archive that includes miscellaneous information related to the political and cultural opposition to British rule waged by the Fenians: letters, official documents, unclassified photographs and significantly hundreds of personal files on Fenian suspects, each of which significantly include a photograph. Under such conditions of production the Fenian photographs seem to fit well within approaches that study Victorian photographs from a Foucauldian perspective. They were forcibly taken and owned by a state institution. They were part of an official filling system that recorded “facts” about the individual and were distributed across powerful institutions in order to track and identify the “Fenian criminal”. Under this hypothesis the gaze of the camera is likened to the gaze of the panopticon.9 However, mindful of Hall’s research, noted at the beginning of this chapter, it is also important to retain an open mind and critically consider what might have been “caught” in these photographs. Significantly other academic disciplines have revealed that a number of different gazes structured looking in this period (see Debord, 1994). Davidson argues that while the gaze of the panopticon was “designed so that everyone could be seen”, the gaze of spectacle was designed “so that everyone could see” (Davidson cited in Bennett, 1999: 337). In this early period photography can best be appreciated by recognising a distinction between the photographic gaze as surveillance – “to be seen” (institutional) and the photographic gaze as spectacle – ”to see” (domestic or celebrity). This chapter will move on to examine how the unusual conditions of production of the Fenian photographs may have resulted in them containing both types of gaze. The majority of photographs in the Fenian Files are cheaply produced and have been attached to a standard printed prison sheet (Form K) on which the “Particulars relating to the Convict” – size, age, etc – have been filled in by hand. The majority have been taken in the prison, initially by an outside commercial photographer, but at the height of the crisis by the prison warden himself. They are head-and-shoulder shots, usually seated and contain all the signs of cheap productions: a transportable cloth as a backdrop, no props and poor to adequate lighting.  In his history of Mountjoy, Tim Carey (2000) pays no attention to photography. However he does note the impact of the Fenian Crisis, confirming that it instigated that status of Mountjoy as ‘a political prison’. He also notes how, ‘for security reasons Fenians sentenced to penal servitude’ were forced to serve their time in England, with Mountjoy acting as a clearing-house after sentencing’ (163).  This is undertaken by relating photography to Jeremy Bentham’s design of the panopticon in 1768, a prison that allowed one guard to survey many prisoners: ‘seeing without being seen’.

Reading the Fenian photographs

489

The Form K “particulars sheets” indicate that although arrested they could not be classed as convicts because they had not been convicted of a crime. This is confirmed by the standardised text “Particulars relative to Convict”, Convict has been crossed out by hand. Around a quarter of the files do not have a photograph attached and some reveal a battle over ownership of the self-image, containing handwritten notes such as “refused to be photographed before it was made compulsory” or “refused to be photographed until his discharge”.10 Others confirm how delicate the process of photography was: “negative defaced in being burnished” (NIA, F.P. 284) or “negative was broken while printing” (NIA, F.P. 472). Mac Suibhne and Martin’s (2005) review of the contents confirms how: men in their teens and twenties predominate […] but a surprising number of older men saw the inside of jails; the average age of internees was 27 and 28 per cent of internees were over thirty. […] Periods of internment ranged from a few days to several months, the average interment lasting 157 days. Over half of the internees (607) were released on giving bail; 14 on condition that they “leave Ireland” and a further 390 on agreeing to go to a named country, generally America (258), England (124) and Scotland (4) (Mac Suibhne and Martin 2005: 103).

In 1867 the Irish Government introduced the compulsory photographing of those interned. It sent instructions to the Prison Office to supply photographs of all untried prisoners “who are now or at any time have been in custody under [IG’s] [Excellency] warrant”. Over the next few weeks letters chart this process. On 9 January 1868 a total of 374 photographs and descriptive particulars of untried prisoners were sent to Dublin Castle. On 15 January the prison office forwarded to the Under Secretary to the Lord Lieutenant of Ireland further descriptive particulars and photographs of 35 additional untried prisoners,11 and on 20 January an additional 115, with the comment “the whole is now complete” (NIA, F.F. 550R). It is precisely because pre-convicted Fenians, as the above indicates, had an unclear status in relation to criminality (an anomaly that existed as consequences of the colonial situation) that this unique archive exists. Because these men had not been convicted of a crime, they could not be photographed within the established codes and conventions of convict photography already well established in Ireland. Instead they had to be photographed in a manner that visually confirmed their non-convicted status. This was signified through the photographs’ difference from convict shots: the men were photographed exactly as they were when they had been arrested, in their own clothes with their own forms of expression and pose. It was this unique set of historical circumstances that has allowed this archive to trace for the photographic historian a record of the mid-Victorian fashion, style, pose and appearance, of ordinary Irish working men; to my knowledge no similar  John Phillip’s file, for example, records how he ‘refused while here as an untried prisoner but was afterwards photographed as a convict’ (NIA, F.P. 427).  NIA, F.P. 407R.

490

Sarah Edge

archive of such “ordinariness” exists precisely because there was no reason to record such commonplace details.12 The significance of these photographs is that they record messages that are closer to the gaze of spectacle than surveillance. In order to fully appreciate this difference it is useful to consider a further set of photographs related to this archive. If an internee was convicted he was photographed a second time. This latter photograph followed the established conventions of Irish convict photography. The Fenian Files in Dublin do not hold many copies of these photographs; however, a large archive of these pre- and postconviction photographs are housed in the New York Public Library.13 This collection consists of two volumes of photographs of prisoners held in Mountjoy taken in August 1857 and November 1866. The collection originally belonged to Thomas Aiskew Larcom (1801–1879) and it would appear that these were put together by or for Larcom while he was under secretary in Ireland (1853–1868). One is of convict photographs from 1857 with a title page of “Photographs of Some of the more serious prisoner”. The second, produced in 1866, is of Fenians. A letter from the photographer (probably the warden) to Larcom reads: “You asked me some months ago to get you the photographs of the convicted and untried political prisoners who have been confined in Mountjoy. I now send you a most unique ‘Book of Beauty’. […] The camera is bad, but I am about to get a better, a really good one.”14 Many of the pre-conviction photographs are duplicates of the photographs held in the Fenian Files in Dublin,15 verifying their source. Thus Larcom’s collection offers the critical photographic historian access to a type of “before and after” photograph of the Irish nationalist Fenian. Jeremiah O’Donovan Rossa, a well-known Fenian, recorded the following account of being “prepared” for his “post-conviction” photograph in Mountjoy Prison: “After being shaven I was led to have my picture taken. The photographer had a large black-painted pasteboard prepared, with my name across it in white, and, pinning it across my breast, he sat me in position. I remained sitting and looking according to instructions until he had done, and he never had the manners to tell-what artists never fail to tell me – that I made an exceedingly good picture” (O’Donovan Rossa 1882: 73). As someone more use to the taking of celebratory

 Tagg’s limited examples from the Birmingham police files also show the criminals in their own clothes, while Jennifer Green Lewis’s study on criminal photography reproduces two illustrations of criminal photography taking place, one from Harpers Weekly from 1873 (Green-Lewis 1996: 202) and a staged photograph taken by Jacob Riis in 1886 (Green-Lewis 1996: 209). Both representations depict the criminal being photographed in his own clothes. What I am suggesting is that it is the number of such photographs in the Fenian files that make it unusual as well as the relationship they have to the post- conviction photographs of the same individual in an Irish context).  These are accessible via the internet through the New York Public Library (http://nypl.org./ digital). For more information on this archive and digitisation see Baylis, Gail 2009.  Image ID: 1111276 NYPL digital library.  Duplicates of these post conviction photographs do not seem to be housed within Ireland.

Reading the Fenian photographs

491

promotion photographs, O’Donovan’s final remark offers a rather “knowing” ironic twist on the process. The conviction photographs carry quite different cultural codes and meanings in which the Irish Fenian has been “prepared” to ensure that the photograph will carried the established signs of criminality: a formulaic pose, a number, the removal of all signs of individuality such as fashion and style, and finally the removal of a sign of “vanity” and self grooming – the beard (and I am going to return to this). There is one example of a pre- and post conviction shot is held in the Fenian Archives in the file of John Devoy, also a well-known nationalist. In the first he is in his own clothes, hair styled and groomed and he is sporting a short beard. His pose seems to signify a smart but relaxed posture and this photograph carries most of the codes of a domestic photograph. In the other he is close-shaven, hair cut and posed in a convict uniform; his name and number has been placed across his body – see Figure 1 and 2 (NIA, F.P. 116).16 In some previous research I related these criminal-style photographs to Alan Sekula’s work on early convict photography in the American penal system, which confirms how that the “truth” value of the photographic process complemented theories of physiognomy and phrenology in which “prison photography was used

Fig. 1: NIA, F.P. 116 – John Devoy.

Fig. 2: NIA, F.P. 116 – John Devoy.

 Post-conviction shots are rare in this archive. One other exists: James O’Connor: NIA, F.P. 388 – James O’Connor.

492

Sarah Edge

to both establish and delimit the terrain of the other, to define both the generalized look – the typology – and the contingent instance of deviance and social pathology” (Sekula 1992: 345). Sekula and Tagg (1988) have both noted how the construction of a working-class criminal “type” through photography functioned to displace problems of crime from poverty onto “biological” class deviancy. The postconviction Fenian photographs had a similar rhetorical function. However, their location in Ireland and their naming as Fenians shifts this deviancy from class onto a national or racial deviancy (Edge 2004: 17–39). I now wish to return to this reading to interrogate it further. Hall’s work cited at the beginning of this chapter pays particular attention to styles of fashion caught by the camera in order to challenge assumptions about those depicted and it is useful to apply this hypothesis to these photographs. I have already indicated how the conditions of production make this archive of pre-conviction Fenians quite unique. In them the historian has access to a distinctive record of the fashion sense and style of young Irish working-class urban men. Fashion was a crucial visual signifier for signposting difference – class, gender, age, race, national, regional – in the mid-Victorian period and Breward’s study confirms how “the fashionable ideals promoted by the clothing trade shared a common language with other more imaginative texts, in which identification and assessment of urban “types” through visual coding was of central importance” (Breward 1999: 55). I would suggest that placed in this visual cultural climate where “style” could be read as an indicated of “natural” difference, the pre-conviction photographs allow the photographic scholar a unique chance to examine the complex relationship between signs of (Irish) differences in the material world and the organization of colonial constructions of Irish difference in an institutional site. Fashion signifies in complex ways in this archive. The archive includes photographs of Fenians from different sources which carry different meanings. Alongside these photographs taken by the prison services, the archive also holds quite a large number of photographs of prominent nationalists. These photographs are professional commercial photographs and a W. Lawrence of Sackville Street Dublin has taken many of them. These were produced under different conditions: created in commercial studios, these include the conventional props associated with respectable middle-class Victorian portraiture of the period.17 This can be illustrated by the photograph of Pat Egan, who is posed smartly dressed and reading a book, signifying his intellectual educated status: see Figure 3 (NIA, F.P. 146 – Pat Egan), M. Healy, a Member of Parliament, is photographed in expensive smart clothes in which his eyeglass signifies his intellectual status (NIA, F.P. 209 – M.

 Some are domestic photographs such as Hamilton (NIA, F.P. 197), and it is difficult to know how this was acquired, while others such as J. A. McCaulay (NIA, F.P. 336) seem to be personal posed ‘action’ tableaux.

Reading the Fenian photographs

493

Fig. 3: NIA, F.P. 146 – Pat Egan

Healy). There are numerous other examples in the archive. It is difficult to ascertain precisely how these photographs found their way into the archive, but I would propose that they were secretly purchased by the prison service or police force from the photographer’s stock or when on public display. This claim is supported by the archive, as in many cases the attached “particulars form” is blank, indicating that this individual had not been arrested. In this respect we can see how the photographs in this archive were also being used as a means of identification. Originally produced as self-promotion and spectacle but later employed as a form of surveillance, they confirm the polysemic nature of all photographs. These men were not common working-class Fenians but members of the newly-formed educated middle-classes and the internal signs of dress, fashion, pose, props and location caught in the photographs verify this class-based distinctions that existed in this early Irish nationalist movement. Significantly, the archive also reveals a certain unease as to the veracity of photography in relation to these class-based distinctions. A key example of this is recorded on the form of a William Hogan: a cheap prison photograph signifies that he was a common rather than middle/ upper-class Fenian. A hand written note alerts the viewer to not be deceived by what they see – “not so gentlemanly looking as in photo” (NIA, F.P. 219).

494

Sarah Edge

While the pre-conviction photographs hold internal codes related to how the viewer had learnt to interpret signs of class differences in an Irish context, the post-conviction removes all these signs and all that is left is class “equality” as common criminals, thus removing the problematic of the educated middle-class Fenian for the British state. In his history of Mountjoy Prison, Carey includes an important description, from the period, which comments on the power of this transformation and criminalisation of “political” prisoners; one group “presented a wretched appearance. Clothed in the convict uniform, their beards and moustaches shaven off, and their hair closely cropped, it was impossible to distinguish them from felons of the lowest class (Carey 2000: 163).

4 A Discursive Twist: Surveillance or Spectacle. The majority of photographs in the Fenian Files have been taken by the prison authorities and, as previously noted, offer the photographic historian a unique record of Irish male “ordinariness”. To return to Hall (1992) and Curtis (1997), I would suggest that these photographs do hold clues on how Irishness was being discursively constructed but in a slightly different way than first imagined. Because these photographs were taken under conditions of production more related to domestic photography, these men have been recorded wearing jackets, waistcoats and patterned scarves and sometimes carrying hats. They often have a self-controlled relaxed pose and look expressively into the camera. Significantly, they also often have beards which have been shaved and groomed into different styles.18 In this respect, similar to Hall’s study, these photographs retain messages on the “modern” independence and individuality of Irish men. Moreover, as Curtis and others note, there are no defining features that mark them as different from their white imperial neighbours and unlike other oversee photographs of this period they had no widespread commercial potential in terms of “factually recording” the differences of the Irish,19 because ironically they confirmed their sameness. Furthermore these photographs seem to record an urban style of dress (particularly through facial grooming and its varying styles), which also contradicts imperialist constructions of the Irish as underdeveloped, rurally based,20 undomesticated and uncivilized.  Of the just under 300 photographs of ordinary Fenians around fifty percent have beards these are clearly groomed into different styles and fashions.  Oversee institutional photography often found its way into commercial sales, where they were sold as curiosities.  Cullen’s examination of paintings pre the Fenian crisis notes how in ‘the first half of the nineteenth century in both literature and the visual arts, Ireland was frequently represented through its peasant class. National character, it was thought, exhibited itself most strongly and visibly in the rural poor’ (Cullen 1997: 116).

Reading the Fenian photographs

495

Such a visual reading of dress codes is supported by the particulars noted on the form where the occupations recorded are new and urban-based: draper’s assistant, labourer, tailor, carpenter, cotton spinner, brick layer, plasterer, furnaceman, railway labourer, porter, shopkeeper and shoemaker. Finally, such a textual reading, I would suggest, allows me to speculate on what maybe an ironic twist in relation to imperialists’ constructions of Irish difference. As noted above, these unusual photographs have recorded “evidence” of fashion as a sign of difference. Such personal markers operated as visual devices by which the individual could signpost his identity. Adams’s study notes how fashion was an important expression “of this anxious conjunction of discipline and performance in middle-class Victorian constructions of masculinity” with an “intractable element of theatricality in all male self fashioning which inevitably makes appeal to an audience, real or imagined (Adams 1995: 10–11) Hair and the control of hair held significant meaning in Victorian culture. Research has identified how in terms of gender middle-class women were required to “control” their hair.21 However, there seems to have been no consideration in terms of how hair might have signified in relation to masculinity. I would speculate that the control of facial hair through shaving operated in a similar manner, in which shaving and styling was an outward sign of control over nature. Moreover, while racial and cultural superiority might have been signified by shaving, certain styles of shaving might also have signified different class or regional masculinities.22 With this in mind a close review of the Fenian photographs indicates that these urban-based Irish men favoured stylized facial grooming, in which one particular style seems to have a contemporary resonance (Figures 4–8). In these photographs the facial hair has been trimmed and shaved into a distinctive style cut away from the mouth and nose, left framing the chin.23 It is significant that Curtis’s (1997) study on cartoons cited at the beginning of this chapter indicates how a similar distinctive style of beard became used a key marker of the Fenian threat in the 1860s.24 That facial hair had been employed as an early signifier of Irish difference is also suggested in Cowling’s study (1989).25  For example see Roberts’s reading of Charles Baxter’s painting Sisters (Roberts 1972: 45).  While shaving does have a long history with evidence from the Bronze Age, the actual styling of beards seems to have taken off with urbanisation and the commercialisation of the barber’s shop when in the 1890s special schools for barbers were established, shops opened and styles emerged (Tonkin).  Of the 144 photographs in which beards are recorded just under half resemble this particular fashion of styling.  This specific style of beard is employed as a ‘facial’ characteristic even in earlier cartoons see for instance ‘The Irish Frankenstein’ a Punch cartoon from 1843, Curtis reproduces a very interesting cartoon from Punch 1846 which, as he notes, contains two different physiognomies, one of which includes the whiskers as a sign of the well-armed nationalist (Curtis 1997: 32 and 127).  She refers to the comparative physiognomy work of James Redfield from 1852 whose work was ‘highly regarded, perhaps’ as she suggests ‘because it confirmed so many prejudices’ (Cowling

496

Sarah Edge

Fig. 4: NIA, F.P. 727.

Fig. 5: NIA, F.P. 565 – Thomas Lynch.

Fig. 6: NIA, F.P. 320 – Timothy Murphy.

Fig. 7: NIA, F.P. 261 – Peter Saunders.

Fig. 8: NIA, F.P. 308 – Michael Morrissey.

Fig. 9: NIA, F.P. 238 – James Kearney.

To develop this proposition it is worth returning to painting which, as I have suggested, was also being employed to define Irish difference. Ford Madox Brown’s painting Work (painted over ten years: 1852–63) is an important example here. The painting is located in the new modern industrial metropolis of London and is acknowledged as consciously engaging with emerging class and “racial” differences within the modern urban environment.26 John Walker’s (2006) detailed 1989: 35). In one illustration the profile of an Irish man seems to contain these chin-based whiskers that are then used to ‘scientifically’ compare him to the profile of a hairy terrier dog.  Treuherz (1987: 31–32) notes how the painting was ‘inspired by seeing a band of navvies digging up the pavement in Heath Street, Hampstead’ in which ‘Brown created an assemblage of types representing not just workers, but Victorian society as a whole […] Brown depicted rich and poor, idle and industrious, working and unemployed’.

Reading the Fenian photographs

497

research into the painting draws from Brown’s original recollections of constructing the painting which verify the importance of the new practice of “drawing from life” in this period, a practice devised to accentuate the realism of the image. In March 1857 Brown notes going to “Gray’s Inn Road in search of Irish people to serve as models” and he “spends two days painting the head of a navvy mixing mortar using the young Irishman as a model” (Walker 2006: 33). In the 1865 exhibition catalogue Brown details the iconography for his audiences and again the navvy is connected to the Irish worker. “Paddy [a common nick-name for an Irishman] with his Larry [a kind of hoe with holes in the blade used for mixing mortar] and his pipe in his mouth” (Walker 2006: 52). Work includes representations of four Irish navvies.27 Following the accepted visual language of the period, their Irishness has been signified by their red hair. However, and significant to my study here, this has been accompanied by a further sign; the three men (one navvy is a boy) are also depicted with the short trimmed red beards.28 Thus while the Fenian photographs could not record the redness of the hair it appears they have recorded a rather distinctive style of shaving favoured by the Irish. A style which was visible in the real world, akin to skin colour, and mobilized in popular cultural representations to mark the Irish as different. The uniqueness of the conditions of production surrounding the makings of the Fenian Archive means that the critical photographic historian has been left a trace of how urban Irish men chose to represent themselves through codes of fashion in the 1860s. This trace has allowed me to pose some questions related to the role of photography in the construction of imperialist discourses related to the Irish. On the surface, in contrast to other imperialized nations, it would appear that photography did not lend itself to the development of an imperialist rhetoric, precisely because the key marker of skin colour was absent. Nonetheless, as I have demonstrated, once the internal codes of the photographs in this archive are made intelligible they do appear to contain markers of Irishness which hold varied historical meanings. They reveal internal battles around class meanings; they reveal the beginnings of a long process of criminalizing the Irish political prisoner; and they contradict perceptions of the Irish as backward and rural. Finally they reveal the complex relationship that exists between external imperialist constructions of the Irish and internal construction of an Irish identity. This is most clearly demonstrated in a somewhat ironic twist of visual history in which the sign of the red

 There are two other navvies who have been identified as English.  Cullen (1997) notes how the painting also contains representations of the Irish as passive rural victims of the famine resting on the bank There are very few painted representations of the famine and Cullen (1997) identifies just two: George Fredrick Watts’s The Irish Famine was produced in 1849–50 and Walter Howell Deverell’s The Irish Vagrants produced in 1853–4. Both these painting, in common with Brown’s Work, employ the image of the Irish mother and her hungry child to engender sympathy and pity in relation to the condition of the Irish, these representations encouraging a benevolent rather than political understanding of the famine from the English viewer.

498

Sarah Edge

stylized beard has become the key symbol of Irish difference constantly re-circulated in popular cultural representations of the Irish leprechaun29. Finally, to return to Curtis (1997), this chapter has revealed that a relationship does appear to exist between these pre-conviction Fenian photographs and the “ape-man of the Punch cartoons” – though not in the way he imagined, because ethnic or racial difference is never based on biological fact that can be proved or disproved by a photograph but on how recordable signs of difference are given cultural meaning.

References Adams, James Eli. 1995. Dandies and Desert Saints Styles of Victorian Masculinity. Ithaca/ London: Cornell University Press. Barthes, Roland. 1977. Image Music Text. Trans. by Stephen Heath. London: Fontana. Baylis, Gail. 2009. A Few too Many? Some considerations on the digitisation of historical photographic archives. Available at: http://web.mit.edu/comm-forum/mit6/papers/ Baylis.pdf. Bennett, Tony. 1999. The Exhibitionary Complex. In David Boswell and Jessica Evans (eds.), Representing the Nation: Histories Heritage and Museum, 332–362. London: Sage. Breward, Christopher. 1999. The Hidden Consumer Masculinities, Fashion and City Life 1860– 1914. Manchester: Manchester University Press. Burgin, Victor (ed.). 1982. Thinking Photography. London: Macmillan. Carey, Tim. 2000. Mountjoy: The Story of a Prison. Cork: The Collins Press. Cowling, Mary. 1989. The Artist as Anthropologist. Cambridge: Cambridge University Press. Cullen, Fintan. 1997. Visual Politics: The Representations of Ireland 1750–1930. Cork: Cork University Press. Curtis, Lewis Perry, Jr. 1997. Apes and Angels: the Irishman in Victorian Caricatures. Washington D.C./London: Smithsonian Institute Press. Debord, Guy. 1994. The Society of Spectacle. New York: Zone. Edge, Sarah. 2004. Photographic History and the Visual Appearance of an Irish Nationalist Discourse 1840–1870. Victorian Literature and Culture 23(1): 17–39. Edwards, Elizabeth (ed.). 1992. Anthology and Photography 1860–1920. New Haven/London: Yale University Press. Gibbons, Luke. 1991. Race Against Time: Racial Discourse and Irish History, The Oxford Literary Review 13(1–2): 95–117. Green-Lewis, Jennifer. 1996. Framing The Victorians: Photography and the Culture of Realism. Ithaca/London: Cornell University Press. Hall, Stuart. 1992. Reconstruction work. Ten.8 2(3): 106–114. Holland, Patricia. 2000. Sweet it is to Scan …: Personal Photography and Popular Photography. In Liz Wells (ed.), Photography: A Critical Introduction, 118–164. London: Routledge. Kee, Robert. 1979. The Bold Fenian Men. London: Quartet.

 It is difficult to pinpoint exactly when this representation emerged; however, representations prior to the 1860s do seem to commonly depict the leprechaun with a long beard rather than the shaved framed beard noted here.

Reading the Fenian photographs

499

Mac Suibhne, Breandan and Amy Martin. Fenians in the Frame Photographing Irish Political Prisoners, 1865–68. Field Day Review 1 20045, 101–119. McClintock, Anne. 1995. Imperial Leather: Race Gender and Sexuality in the Colonial Contest. London/New York: Routledge. Mitchell, William J. Thomas. 2004. Benjamin and the Political Economy of the Photograph. In Liz Wells (ed.), The Photography Reader, 53–59. London/New York: Routledge. O’Donovan Rossa, Jeremiah. 1882. Irish Rebels in English Prisons: A record of Prison Life. New York: P J Kennedy. Roberts, Helen. 1972. Marriage Redundancy and Sin. In Martha Vicinus (ed.), Suffer and Be Still: Women in the Victorian Age, 45–77. London: Methuen. Ryan, James R. 1997. Picturing Empire: Photography and the Visualisation of the British Empire. London: Reaktion Books. Slattery, Peadar. 1992. Uses of Photography in Ireland 1839–1900. Unpublished PhD thesis, Trinity College, Dublin. Sekula, Allan. 1992. The Archive and The Body. In Richard Bolton (ed.), The Contest of Meaning, 343–389. Cambridge: MIT Press. Tagg, John. 1984. The Burden of Representation. Ten.8 14: 10–13. Tagg, John. 1988. The Burden of Representation: Essays on Photographies and Histories. London: Macmillan. Tonkin, Ellen. History of Shaving. (pamphlet) Nottingham: Castle Museum. Walker, John. 2006. Work: Ford Madox Brown’s Painting and Victorian Life. London: Francis Boutle Publisher. Wills, Clair. 1991. Language Politics, Narrative, Political Violence, The Oxford Literary Review 13: 20–61. Files and photographs held in the ‘Irish Crime Records’ and ‘Fenian Photographs’: National Archives of Ireland. The Fenian Photographs contain around 500 files. Photographs reproduced with kind permission from National Archives of Ireland and the Director of the National Archives of Ireland.

Diane Carr

22 Interpretation, representation and methodology: Issues in computer game analysis Abstract: In this chapter representations in the digital game Resident Evil 4 are explored using a combination of textual, structural and inter-textual analysis. The limitations and offers of these approaches are considered, and the theories that underpin each of these approaches are specified. Drawing particularly on Barthes’ work in S/Z, a fragment of the game is chosen for close analysis. Focusing on the seme or thematic thread of ‘work’, it becomes possible to discuss the game’s protagonist (Leon) and his zombified assailants in terms of their augmentations and abilities. The discussion raises questions about fantasy, affect, and discourses of disability within horror games.

1 Introduction Computer games are played, and play is a culturally and socially situated activity. Undertaking the textual analysis of a game in order to explore issues of representation does not necessarily involve ignoring these points. Textual analysis, like any other methodology, has its limitations. The specifics of these limitations, however, will depend on the particular model of textuality employed. In this chapter these issues are explored through an analysis of the survival horror game, Resident Evil 4. Resident Evil 4 is a zombie infested survival horror game that is played on a console. It features a narrative and it incorporates cinematic and generic conventions. During play it unravels as a series of spatial puzzles. Players encounter particular resources and obstacles, and must manage these in order to attain necessary goals. Progression through the game-as-space is conditional on a player’s performance and the attainment of these goals. These points about play have methodological implications for those interested in representation in games. Within film and media studies the analysis of representations of, for instance, gender, sexuality or ethnicity has generally involved varieties of textual analysis – see, for example, Stuart Hall’s Representation (1997) or Richard Dyer’s The Matter of Images (1993). How appropriate are these forms of analysis to computer games? These questions of methodology and representation will be explored in this chapter, which is based on a paper presented at the Digital Games Research Association’s 2009 conference. There will be an attempt to contextualize the discussion by referencing debates within the field of game studies (for an insight into the field, visit www.digra.org). There has been some consensus in the field regarding the limitations of textual analysis, yet papers proposing to define textual analysis,

502

Diane Carr

Fig. 1: Resident Evil 4 screenshot.

specify how it is done, or describe what it offers, remain rare (Consalvo and Dutton 2006). When debating the value, offers and limitations of textual analysis, it is helpful to engage with specific theories of textuality. In the analysis presented in this chapter, games are approached as playable texts, and particular theories of structural analysis (Barthes 1997: 79–124), textual analysis (Barthes 1974; Barthes 1977: 22–32, Barthes 1977: 1–12) and inter-textual analysis (Bennett and Woollacott 1987) are referenced. It should be noted that this is not an attempt to drag games into an older disciplinary domain. Rather, it is a raid from game studies into older fields in an effort to locate useful tools. It might be argued that new forms of media demand new methods of analysis, but revisiting existing theories of meaning and examining how they might fit, and where they might fail to apply, would be an important aspect of any such development. Thinking about the textual analysis of games involves considering the ways in which meaning emerges during play, and the idea of interpretation itself. The described procedures will be explored through the analysis of a particular section of Resident Evil 4. Produced in Japan, the Resident Evil series references horror films (such as Suito Homu, 1989) and the enormously influential American zombie movies of George Romero. The franchise spans media (novels, comics, films, games). Resi-

Interpretation, representation and methodology

503

Fig. 2: Third person shooter perspective in Resident Evil 4.

dent Evil 4 is a survival horror game featuring action adventure styled spatial puzzles, and a third person shooter perspective.1 The avatar/protagonist Leon battles his way through a vaguely described European location2 while tracking down the kidnapped daughter of a US president. Players (in the guise of Leon) must solve puzzles and locate keys in order to progress from one location to the next, while managing resources (ammunition, medical supplies) and avoiding or defeating the zombified residents. The undead have thrived in popular media in the decades since elements of Haitian culture were selected for misrepresentation in Hollywood’s White Zombie (1932).3 Zombies frequently feature in horror games, perhaps because (as Krzywinksa has argued, see Krzywinska 2002: 211) such games play with the dynamic between gaining and losing control, and the “evocation of hopelessness in the

 Official website www.res-evil.com/re-4/.  The location of the village is not stated, although it is ‘sort of’ Spanish – Maybe it’s Andorra? The game is set in ‘Europe, 2004’ but the currency is pesetas (not Euros). See also http://www. wiiliving.com/resident-evil-4-racist/ and ‘Resident Evil 4: Zombie Cult or a Tragic Result of the Language Barrier?’ (by C. De Las Casas, 2007) at http://www.associatedcontent.com/article/314563/ resident_evil_4_zombie_cult_or_a_tragic_pg2.html?cat=19).  Visit zombierama.com for a list of zombie movies from around the world.

504

Diane Carr

face of an inexorable predetermined force is crucial in maintaining horror-based suspense”. Suspense is particularly acute in zombie games thanks to the monsters’ shambolic and faltering yet relentless progress. A problem with the zombies in Resident Evil 4, however, is that they are not technically undead. The villagers are live people infected by a mind controlling parasite, rather than animated corpses.4 The rage-infected monsters of the film 28 Days Later (2002) suffer from similar problems of classification. In both cases, however, the infected “perform the role of the zombie in recognizable patterns” (Rogers 2008: 120). The villagers that Leon encounters are “too slow to be serious, but too numerous to stop” (Leverette 2008: 203) and they inhabit a franchise rife with zombie associations (McIntosh 2008; Krzywinska 2008; Paffenroth 2006).

2 Textual analysis, digital games and digital game studies In the call for papers for the 2007 DiGRA conference it was stated that “Games are situated in culture and society [and to] truly understand the phenomenon of digital games, it is not enough to merely study the games themselves”. As this implies, textual analysis is not particularly popular within game studies. There seem to be several reasons for this. In related discussions it is not unusual to find various forms of textual analysis being conflated. For instance, examples of content analysis generated as part of social scientific “media effects” studies are labeled textual analysis (see Bryce and Rutter 2002) – and while this description is not actually incorrect, such work has little in common with the textual analysis generated by film and media scholars based in the humanities. The processes involved, the kinds of claims made and the theoretical underpinnings differ. Further problems arise when textual analysis is confused with structural analysis. Both are relevant to games, but confusing them (when considering the limitations of either) does create problems. Textual analysis does have limitations, but it is possible to theorise textual analysis in different ways, and the specifics of each variation (including its limitations) would depend on the model of textuality that is employed. Furthermore, questions about methodological applicability need to be considered in terms of the questions being asked during a specific investigation. The point is not that different methods might be more or less appropriate per se, but that different perspectives and tools support particular kinds of inquiries. In addition to questions of applicability, when theories and methods developed for one media form are applied to another, there are issues of adaptation.  In this interview at IGN.com, RE4 producer Hiroyuki Kobayashi explains that “They’re not villagers. They’re not zombies. They are these things”: http://uk.cube.ign.com/articles/515/ 515402p1.html.

Interpretation, representation and methodology

505

Adapting theory for games analysis involves considering how games differ from other media forms. Game theorists concerned with meaning and interpretation in games have struggled to reconcile rules, representational content, cultural resonance and historical contexts. The critique that has emerged in response to the Civilization series is a useful case in point (for a review see Carr 2007). As such work makes apparent, games are designed; they have rules, and they are actualised through various modes of play. Play is experiential and ephemeral, embodied and culturally situated. Theorising game textuality is not straightforward. It matters, however, because such work might usefully inform research into related issues such as meaning and interpretation. Theorising interpretation is important to those interested in the cultural politics of representation. It also has implications for those designing the research of related topics including, for example, learning and games. As conceptualised in this chapter, structural analysis relates to game design and form, while textual analysis relates to signification and to the game as actualised in play. Barthes’ work on structural analysis (Barthes 1997a: 79–124), and his later accounts of textual analysis (Barthes 1974; Barthes 1977b 125–141; Barthes 1997c: 1–12) inform this distinction. A great deal of existing games analysis that is described as textual analysis is actually structural analysis. Structural analysis makes sense in games studies. Consider, for instance, how useful structural narratology proved to theorists seeking to demonstrate the difference between games and narrative (see Frasca 2003 for an overview). In other work, such as Zagal, Mateas, Fernandez-Vara, Hochhalter and Lichti’s paper “Towards an Ontological Language for Game Analysis” (2007: 21–36) the importance of game structure and its relevance to game design, is recognised: “A deep reading of any one particular game would require an analysis of its representational conventions, allusions and connotations. Our ontology helps position the more formal or structural elements of the game within the game design space; other methods and techniques would be required to approach representational issues” (Zagal, Mateas, Fernandez-Vara, Hochhalter, Lichti 2007: 25). Barthes’ later theories of textuality suggest that the textual analysis of games does not necessarily involve a focus on “the game itself” as if divorced from considerations of play. Barthes’ analysis in S/Z is designed to reflect the act of reading. His focus is on the text “as read”. So, in the context of games studies, according to this particular conceptualization of textuality, thinking about textual analysis means thinking about the game “as played”. Dovey and Kennedy (2006) argue that “In order to study a computer game we cannot have recourse solely to its textual characteristics; we have to pay particular attention to the moment of its enactment as it is played” (Dovey and Kennedy 2006: 6). This is certainly true. However, if I use a model of textuality that incorporates the notion of practice, and that characterizes meaning and interpretation as emergent and situated, then emphatic distinctions between the “game as text” and the “game as played” might prove difficult to sustain.

506

Diane Carr

3 Methodology My analysis of Resident Evil 4 involves the use of three interlocking lenses: structure, textuality and inter-textuality. The version of structural analysis is derived from Barthes’ Introduction to the Structural Analysis of Narrative (1977a), the model of textual analysis is drawn from Barthes (1974, 1977b, 1977c) and the approach to inter-textuality (with a focus on reading formations) is borrowed from Bennett and Woollacott (1987). According to textual analysis as conceptualized here, meaning emerges when the text is actualized or practiced (“read” in the case of a novel; “played” in the case of a game). The concepts of reading formations and intertextuality offered by Bennett and Woollacott offer perspectives through which the overlaps between the text and the meaning-making resources of the situated subject may be considered, without moving into alternative methodological domains.

3.1 Structural analysis Computer game theorists and designers including Salen and Zimmerman (2004) have stressed the importance of understanding games as systems, as “a set of parts that interrelate to form a complex whole” (Salen and Zimmerman 2004: 55). In his 1968 essay titled “How Do We Recognize Structuralism” Gilles Deleuze (2004) writes of structuralism’s interest in systems. Systems require units, he explains, and it is the relative placement of units within a system – rather than the qualities of the individual units in isolation – that is the defining quality of these units. This, he writes – while making reference to Levi-Strauss’ work on playing cards and Lacan’s use of ludic metaphors – explains “structuralism’s inclination for certain games” (Deleuze 2004: 175). In Barthes’ essay Introduction to the Structural Analysis of Narratives (Barthes 1977a) he explores the relations between units in narrative discourse, and the consequences of these relationships for meaning. Faced with the pervasiveness and endless variation of narrative, he proposes that its study should involve the investigation of structure, rather than the cataloguing of individual manifestations. Barthes points to Saussure’s work on language, Levi-Strauss’ work on myth and Propp’s work on folklore as precedents – instances where patterns have been sought and identified, the aim being to extract “a principle of classification and a central focus for description from the apparent confusion of the individual messages.” (Barthes 1977: 80). For Barthes, looking for meaning in structure involves splitting a narrative discourse into various classes of function or unit, and considering the ways in which these units correlate and combine. Working from Barthes’ essay, then, it is possible to argue that the structural analysis of a computer game involves the organization of the game’s constituting units and the ways in which these units inter-relate in time and space. The units of the game-as-system are reconfigurable, to varying degrees, and the manner in

Interpretation, representation and methodology

507

which they inter-relate contributes towards meaning (just like Barthes’ units in discourse). The difference would be that the positions of units in a game, and the relationships between these units, might not be constant, stable or consistent. The scope for movement, alteration or manipulation is accommodated by the game – it is part of the game’s organization, and part of what the game-as-system is. Based on this, it might be proposed that aspects of game design, typologies (of rules or genre, etc.), temporality, spatial navigation or the study of event types (i.e. ludic or narrative) would fall within the ambit of structural analysis. Thus while structural analysis is important to the understanding of Resident Evil 4 as a game, perhaps textual analysis is more pertinent to issues of representation.

3.2 Textual analysis Barthes opens his essay “The Struggle with the Angel” (Barthes 1977b) by commenting that while he will be applying structural analysis, he will also be using a less “pure” approach (1977b: 126) – that of textual analysis, where “text” refers to a methodological field, or a site of meaning, rather than a material object. For Barthes, textual analysis “tries to say no longer from where the text comes (historical criticism), nor even how it is made (structural analysis), but how it is unmade, how it explodes, disseminates – by what coded paths it goes off” (Barthes 1977b: 127). Barthes suggests how to distinguish between structural and textual analysis, while acknowledging that such distinctions should be considered provisional or even tactical. He makes further comparisons later in the same essay, writing that “textual analysis is founded on reading rather than on the objective structure of the text, the latter being more the province of structural analysis” (Barthes 1977b: 131). Again, this suggests that textual analysis relates to practice, while structural analysis addresses the schematic that accommodates these practices. In the context of games analysis, the relevant practice is, of course, play. This emphasis on textual actualization is useful, but there are problems in adapting this work to games – hardly surprising given the differences between games and literature. For example, to demonstrate textual analysis, Barthes breaks the text into lexia. A lexia might be a word, a cluster of words, or a group of sentences – “It is what surgeons would call an operating field” (Barthes 1977c: 4). These lexia are then read, in “slow motion” (Barthes 1977c: 4) and the various “senses” classed and discussed in relation to a set of codes, which can be summarized as: Hermeneutic code – which relates to narrative enigma, questions posed, and truths revealed Proairetic code – the code of actions, logical sequence, causality. Semic code – connotations, themes Symbolic code – mythic antitheses and binaries Referential code – cultural codes, values

508

Diane Carr

Pertinent as these codes may be, applying them to computer games is not straightforward. For example, would the code of enigma relate to mysteries presented in the back-story, as well as the enigmas (challenges, puzzles, delays and obstructions) presented by the game’s ludic elements? What of the enigmas that are the result of player error or getting lost? Would the proairetic code refer to the actions in the back-story, the actions called for by the game, or the actions taken by the player – or all of these? Would accidental or unsuccessful actions differ from deliberate actions? Clearly there are issues when adapting theory from literature, to games. These are further explored in the analysis section of this chapter. Regarding the limits of textual analysis, Barthes writes that “we must not confuse connotation with association of ideas: the latter refers to the system of a subject; connotation is a correlation immanent in the text” (Barthes 1974: 8). Connotation is closely linked to association, but the former is an aspect of the text, the latter would be a property of the reader. No attempt is made in the analysis that follows, however, to maintain this distinction. Connotation is located in the text in so far as a specific feature (a particular action, image, colour, gesture) is evident in the text. However, I am not sure that I could consider the meaning of any such feature without employing association – in practice these appear inextricably entangled. So, rather than attempting to distinguish between connotation and association, it is acknowledged that culturally-situated association is part of analysis. For this reason it is necessary to add a third lens to this approach: a means by which it becomes possible to incorporate and acknowledge the interpretive framework of the player-analyst. Theories of inter-textuality and reading formations make this possible.

4 Inter-textuality and reading formations Bennett and Woollacott’s work on inter-textuality and reading formations offers a framework that makes it possible to think through the relationship between connotation and association. In their work in Bond and Beyond: the Political Career of a Popular Hero (1987) the authors argue that texts cannot be understood in isolation, but rather than proposing a straight switch to audience studies on that basis, they consider aspects of the relationship between text and user, to argue that the reading or viewing subject arrives at a text with a set of “reading formations” in place, and these will influence if (or which) aspects of that text will have resonance for that subject. Viewers and readers will respond to and interpret a text according in part to their reading formations – the social, cultural and historical make up of their interpretive perspective. For Bennett and Woollacott, the term inter-textuality refers to “the social organisation of the relations between texts within specific conditions of reading” (Bennett and Woollacott 1987: 45). They argue, for instance,

Interpretation, representation and methodology

509

that the publicity that surrounds the release of a James Bond film is part of what Bond means, because this material does not just “organise expectations in relation to a particular film but [it also reorganises] the signifying currency of Bond so as to inflect its ideological and cultural articulations” (Bennett and Woollacott 1987: 248). Publicity, for example, functions “alongside the other components of a reading formation (systems of inter-textuality, the institutional practices which bear on the formation of reading competencies, etc.), to organise the relations between texts and readers” (1987: 248, emphasis in original). Bennett and Woollacott intend to move past a focus on either the text or the reader, to highlight the “cultural and ideological forces which organise and reorganise the network of inter-textual relations within which texts are inserted as texts-to-be-read in certain ways by reading subjects organised to read in certain ways” (Bennett and Woollacott 1987: 249). Various models of inter-textuality or trans-textuality could be used as the starting point for a different discussion addressing, for instance, the relationship between Resident Evil 4 and other games in the franchise, or the spin-off films. The emphasis here is on the cultural shaping of interpretation. Reading formations impact on interpretation, and thus they will impact on analysis. Such considerations segue into issues of context, role and social practice. For example, to write this, I’m drawing on particular theoretical perspectives to produce an analysis while addressing a particular community. Game scholarship, like games fandom, involves a ‘particular set of critical and interpretive practices’ (Jenkins 1992: 277–278). Different interpretations will stress different aspects of game-play. Different interpretations will be accorded legitimacy in different contexts, and the interpretations performed or produced in different social contexts or communities of practice, may be incompatible. The actual methods used during analysis were suggested by the procedures Barthes demonstrates in S/Z. The game was played through for pleasure when it was released. It was then replayed in preparation for this analysis. Then, a particular section or episode was played, paused, and re-played by the author, and notes taken.

5 The analysis The section selected for analysis occurs very early in the game. Leon has already dispatched a resident. Leon’s advisor, Ingrid Hunnigan checks in via his headset device: HUNNIGAN: Is everything okay? LEON: There was a hostile local. I had no choice but to neutralize him. There are still others surrounding the area.

510

Diane Carr

HUNNIGAN: Get out of there and head toward the village. Take whatever measures necessary to save the subject.5

The player picks up the basics of play through a number of small-scale encounters. Houses, cupboards and shed interiors are worth searching for pick-ups (gold, medicine, ammunition). Crates are worth smashing (more pick-ups), and typewriters function as save points. Leon follows a dirt track to the village as instructed by Hunnigan and directed by sign posts. The track terminates at a large, metal gate. There is no way around it. Unlike the wooden, ramshackle buildings encountered thus far, the gate is new and imposing, it suggests institutional power and privacy. The gate is tall, made of studded metal, and marked with an unidentified symbol suggestive of secrets, or cultish ritual. Directed by an on-screen prompt, the player hits the action button. Leon moves through the gate and a new area is accessed. The gate marks a transition, in various ways. Leon proceeds down the path, and into the village, and the spatial/temporal framing of the action shifts, from a relatively linear, introductory set of events (as epitomised by the single dirt track) to a set of possible events subject to player prerogative or strategy, set in the villageas-arena. This is indicative of the game as a whole. Areas are accessed in turn, and each is a spatial puzzle: a particular arrangement of obstacles, enemies and potential damage, balanced by a certain number of resources (ammunition, medicine, etc.). Resident Evil 4 could be described as a set of inter-locking spatial puzzles linked by a narrative thread. As the above description of the gate indicates, it would be a mistake to categorise in-game elements as either structural or textual depending on their particular properties. It is more productive to regard each element in the game as potentially viewed through various analytical lenses. Their interpretation will depend on the analytical lens employed (structural, textual or inter-textual, for example). There is no reason to assume that of one of these meanings would be definitive or dominant, although particular interpretations might be more or less likely or appropriate in different contexts. Leon might carefully survey the village, or he might recklessly plunge into the fray. This variability is an outcome of the player’s capacity to manipulate the avatar, and this would be one problem for the analyst attempting to apply Barthes’ codes. There is another. Lexia, writes Barthes (1977c) are “sieves as fine as possible, thanks to which we shall “skim off” the senses, the connotations” (Barthes 1977: 4). Barthes fragments the narrative text to produce lexia, in order to facilitate analysis. The lexia will include sometimes a few words, sometimes several sentences; it will be a matter of convenience: it will suffice that the lexia be the best possible space in which we can observe

 From ‘Resident Evil 4 Game Script’, by The Headcrook (2005) online at http://uk.faqs.ign.com/ articles/674/674283p1.html accessed July 2009. Helpfully the protagonists suggest the importance of ‘neutralizing’ the hostile other in order to safeguard ‘the subject’.

Interpretation, representation and methodology

511

meanings; its dimension, empirically determined, estimated, will depend on the density of connotations […] all we require is that each lexia should have at most three or four meanings (Barthes 1974: 13).

It is unlikely that a game fragment could be limited to three or four meanings, once graphics and imagery, sound, movement, player action, commands, the interface and the ludic economy (rules, resources, damage, etc.) have been taken into account. In an attempt to limit the “operating field” (Barthes 1977c) attention is narrowed to the first village-based confrontation between Leon and the locals. The focus is on the clusters of meaning associated with specific elements; similar to that which Barthes describes in S/Z as “connotative signifiers”: a signifier which will occur in several places in the text; it is a shifting element which can combine with other similar elements to create characters, ambiences, shapes and symbols […] although every unit we mention here will be a signifier, this one is of a very special type: it is the signifier par excellence because of its connotation, in the usual meaning of the term (Barthes 1974: 17).

This will involve focusing on units of meaning, or semes, with a binding quality; that form clusters and continuities within, across and beyond this specific game (like threads that run through the game, and that connect into other media texts or culture contexts more generally). For the sake of the necessarily short discussion that follows, I have identified a particular seme on which to focus, and tagged it “work”. It would be possible to explore other semes in the game, such as ethnicity or masculinity (for example). A full analysis would involve the consideration of these various semes as they emerge and combine during play.

6 Work Prior to entering the village, Leon has an opportunity to observe the occupants. They are working: shoveling, pushing wheel barrows, carrying buckets of water. Once Leon enters the area, they lumber towards him armed by picks, pitchforks and axes. Now their job is to kill Leon. The villagers’ employment and their use of tools sets them apart from Romero’s zombies, who tend to be uninhibited, unemployed and autonomous (Paffenroth 2006). In a sense, the possessed farmers of Resident Evil 4 are throwbacks to an earlier era. These villagers are in service, and they recall the zombified workers of White Zombie, a film about control, sex, possession, drone-like labour, race and slavery. Deleuze and Guattari (1983) describe zombies as “good for work” (Deleuze and Guattari 1983: 335), while the 2004 British film Shaun of the Dead features an image of the “mobile deceased” working in supermarkets. The notion of the zombie as worker, and workers as zombies, has continuing cultural salience. Like the village itself, the consecutive zones that Leon encounters in Resident Evil 4 are frequently places of work. He fights zombies in grim offices, grimy labs,

512

Diane Carr

farm yards, quarries, sheds, barns and mines. Leon’s entire mission is work related. He is on the job. He works his way up, killing a series of bosses, each of increasing seniority within an organization. He carries an attaché case, rather than a backpack or satchel. Leon and the villagers carry the tools of their trades (weapons and binoculars in Leon’s case, while the villagers wield shovels, hoes and axes). Thanks to his headset, Leon is online, linked up with the formal and bespectacled Hannigan who communicates strategically valuable information in a brisk tone using military language. In contrast to Leon, the villagers might be regarded as either offline (the village does not have much in the way of modern technology), or as being online in the wrong way. They are linked-up via biology rather than technology, and directed by a feudal/religious order, rather than technologically advanced/military order. Neither Leon nor the villagers could be considered autonomous. Leon’s headset is an augmentation that slots him into a particular chain of command. As such, it mirrors the infestation that facilitates the arch villain’s control over the villagers. One form of augmentation is constructed as positive or advantageous. The other is constructed as a disease or deficit. In each case, augmentation has ramifications for role, work and identity.6 By definition, a horror game promises discomfort, fear and anxiety. In Resident Evil 4, these sensations are associated with the identities and bodies of the infected villagers. Theorists frequently discuss zombies in terms of ambiguity (un-alive, undead), contagion and inefficiency (Leverette 2008, Shaviro 1993). Despite a repeated focus on damaged bodies, the explicit associating of zombie-hood with disability is very rare in zombie literature. Paffenroth, for example, describes Romero’s zombies as fallible and uncoordinated (Paffenroth 2006: 15) and as “slow, clumsy imbeciles who can barely stand up” (Paffenroth 2006: 17). He does not discuss these points in relation to his perceptions of disability. Lauro and Embry (2008) however, note that “without consciousness and without speech, the zombie recalls the mentally ill or the language impaired, such as those with aphasia. Even the lumbering gait of the cinematic zombie, which probably is meant to reflect rigor mortis and advanced decay, looks like a muscular disorder” (Lauro and Embry 2008: 103). Furthermore, Lauro and Embry point out that, as a metaphor, the “zombie reveals much about the way we code inferior subjects as unworthy of life” (Lauro and Embry 2008: 87). Considering Leon’s and the villagers’ labouring bodies, their online/offline status, and their augmentations in relation to notions of ability and disability, suggests a way to think about the anxieties potentially evoked by Resident Evil 4, without resorting directly to the psychoanalytically-informed accounts of abjection that have proven so influential in horror film analysis (see Kristeva 1982 as applied by Creed 1986).  Later in the game this natural order is corrupted: Leon becomes infected with a parasite, and the villains highjack his headset to send him nasty messages.

Interpretation, representation and methodology

513

This alternative account of horror would still centre on issues of ambiguity, identity and the body. Instead of exploring this anxiety through psychoanalysis, this approach considers such anxieties as they relate to social policy and discourse. As Branson and Miller (2002) record in their history of the pathologizing of deafness, the 19th Century rise of bureaucracy (empire, army, education) saw the measuring and recording of difference (racial, physical, sexual, etc.) become increasingly prevalent. Measurement and institutionalization became instruments of control and order. Of course this had (and continues to have) profound ramifications for those classified as deviant, uneducatable or unemployable. But it also has implications for those subjects who are positioned to experience their identity as normal or able. For these subjects, it is possible that deinstitutionalization (or the collapse of categorization) potentially “threatens not only their identity, especially when faced with the everyday presence of those formerly hidden away, but also the disciplined and orderly nature of their environment” (Branson and Miller, 2002: 52). This suggests the possibility of thinking about anxiety within the game in terms of difference and control, and loss of control. From this perspective the zombie operates as a disabled escapee, trailing riot, impropriety and disorder. Zombies feed on and convert the “normals”. From this angle, the anxiety promised by the monstrous in Resident Evil 4, is not just about the monster’s ability to transgress boundaries – it’s about classification, and the potential for reclassification of any subject (including subjects positioned as “able”). It is an anxiety associated with differentiation as an incomplete or unstable process. In which case a zombie is not scary because it is a representation of a disabled individual. Zombies are scary because they are a representation of the idea of disability itself within ‘able’ culture. They hint at the temporary nature of able identity, and its conceptual reliance on the co-construction of disability, and the instability of both of these categories (see Linton 1998). In Resident Evil 4, work is connected to the body and identity. Work determines what is worn and the weapon that is wielded. Work and identity are associated with an augmentation that is either advantageous (in Leon’s case) or impairing (in the case of the mind-controlling parasites). Leon is online in the “right way” and he fits into a particular chain of command. Leon is occupied, whereas the villagers are possessed. This examination of the single seme “work” in Resident Evil 4 suggests that the game provokes, plays with and teases the player with anxieties of classification, ambiguity and differentiation. The game then soothes these anxieties by offering the pleasures of exactitude and precision. Precision – structured into the game as the efficient use of weapons and accurate targeting – is an absolutely central aspect of the game-play. Ammunition is rationed, but combat is a strategic necessity. Precision offers relief. It enables Leon’s survival, but the shooting in Resident Evil 4 also serves a function similar to filing: Leon does not kill the villagers so

514

Diane Carr

much as resolve their status. Once they are properly dead and emphatically classified, the neutralized villagers neatly exit the game-space.

7 Conclusion There may be a temptation to conclude from the above that Resident Evil 4 is a reactionary text that involves the emphatic dispatching of an abject other for the sake of comfort and neatness. However, to return to the discussion of methodology at the centre of this paper, producing a situated interpretation is not the same as producing an account of what the game means in any definitive or fixed sense. The analysis raises further questions. For example, consider that the villagers are possessed and remotely controlled, but so is Leon. He is an avatar. It might be assumed that my role as Leon positions me in a specific way and renders me more susceptible to the naturalised values of the game-as-text, but such assumptions are problematic. Crucially, any account of ideology in games that failed to conceptualise either interpellation or subject position (or some other mechanism to explain how it worked) would be incomplete. The trouble is that any single elements of the game (and particular moments in the game-play) might potentially position the player in different ways, and be interpreted according to different criteria, at the same time – including characterization, narrative, the setting, emotions such as fear or frustration, game goals and obstacles. The methodology discussed in this paper is adapted from literary studies. The theories used are not a perfect fit. What they offer, however, are tools that make it possible to articulate differences between game textuality and game structure. These theories offer a basis for exploring the relationship between text and play. They allow for an account of methodology that is explicit as to its theoretical underpinnings. These procedures allow the analyst to approach issues of representation in games, and they suggest a framework through which it becomes possible to theorise interpretation itself. These are some of the offers of textual analysis to game studies. Finally, this analysis has raised questions about the prevalence of fantasies and imagery that relates to disability in horror and science fiction themed games generally, including images of bodily alteration, augmentation and physical damage. It raises questions about the relationship between fantasies of disability in games and the depiction of ability as something that can be attained, proven and measured. It raises issues about the relationship between ability and disability as cultural constructions, and about the pervasive yet generally unexamined links between status and perceived ability within gaming culture. Acknowledgments: This chapter is based on a paper first presented at DiGRA 2009 and it appears here with DiGRA’s permission. Carr, Diane (2009), Textual Analysis,

Interpretation, representation and methodology

515

Digital Games, Zombies’, Atkins, B., Krzywinska, T. & Kennedy, H. (eds.) Proceedings of Breaking New Ground: Innovation in Games, Play, Practice and Theory (DiGRA 2009) London: Brunel University. Screenshot images courtesy of CAPCOM U.S.A., INC., © CAPCOM CO., LTD. ALL RIGHTS RESERVED

References Aarseth, Espen. 1997. Cybertext: Perspective on Ergodic Literature. Baltimore: John Hopkins University Press. Barthes, Roland. 1974. S/Z. Trans. by R. Miller. Oxford: Blackwell. Barthes, Roland. 1977a. Introduction to the Structural Analysis of Narrative. In Stephen Heath (trans.), Image Music Text, 79–124. London: Fontana Press. Barthes, Roland. 1977b. The Struggle with the Angel; Textual analysis of Genesis 32: 22–32. In Stephen Heath (trans.), Image Music Text, 125–141. London: Fontana Press. Barthes, Roland. 1977c. Textual Analysis of a Tale by Edgar Poe. Poe Studies 10(1): 1–12. Reprinted by the Poe Society of Baltimore. Available at: www.eapoe.org/pstudies/ps1970/ p1977101.htm. Bennett, Tony and Janet Woollacott. 1987. Bond and Beyond: The Political Career of a Popular Hero. New York: Methuen. Branson, Jan and Don Miller. 2002. Damned for their Difference: The Cultural Construction of Deaf People as Disabled. Washington: Gallaudet University Press. Bryce, Jo and Jason Rutter. 2002. Killing Like a Girl: Gendered Gaming and Girl Gamers’ Visibility. In: Frans Mäyrä (ed.), Proceedings of Computer Games and Digital Cultures Conference, June 2002, 243–255. Tampere: Tampere University Press. Carr, Diane. 2007. The Trouble with Civilization. In: Tanya Krzywinska and Barry Atkins (eds.), Videogame/Text/Player/Play, 222–236. Manchester: Manchester University Press. http:// playhouse.files.wordpress.com/2008/11/carrcivproof.pdf. Consalvo, Mia and Nathan Dutton. 2006. Game analysis: Developing a methodological toolkit for the qualitative study of games. Game Studies 6(1). Creed, Barbara. 1986. Horror and the Monstrous-Feminine: An Imaginary Abjection. Screen 27(1): 44–71. Deleuze, Gilles. 2004. Desert Islands and Other Texts. Paris/Cambridge: Semiotext(e) Foreign Agents Series distributed by The MIT Press. Deleuze, Gilles and Felix Guattari. 1983. Anti-Oedipus: Capitalism and Schizophrenia. Minneapolis: University of Minnesota Press. Dovey, Jon and Helen Kennedy. 2006. Game Cultures: Computer Games as New Media. Maidenhead: Open University Press. Dyer, Richard. 1993. The Matter of Images. London: Routledge. Frasca, Gonzalo. 2003. Ludologists Love Stories Too: Notes from a debate that never took place. In: Marinka Copier and Joost Raessens (eds.), Level Up: Digital Games Research Conference, 92–99. Utrecht: Universiteit Utrecht. Hall, Stuart. 1997. Representation: Cultural Representations and Signifying Practices. Milton Keynes/London: The Open University/Sage. Jenkins, Henry. 1992. Textual Poachers. New York: Routledge. Juul, Jesper. 2001. Games Telling Stories? A brief note on games and narratives. Game Studies 1(1).

516

Diane Carr

Kristeva, Julia. 1982. Powers of Horror. Trans. by Leon Roudiez. New York: Columbia University Press. Krzywinska, Tanya. 2002. Hands-On Horror. In: Tanya Krzywinska and Greg King (eds.), ScreenPlay: Cinema/Videogame Interfaces, 206–223. London: Wallflower. Krzywinska, Tanya. 2008. Zombies in Gamespace: Form, Context, and Meaning in Zombie-Based Video Games. In: Shawn McIntosh and Marc Leverette (eds.), Zombie Culture: Autopsies of the Living Dead, 153–168. Plymouth: The Scarecrow Press. Lauro, Sarah Juliet and Karen Embry. 2008. A Zombie Manifesto: The Nonhuman Condition in the Era of Advanced Capitalism. Boundary 2 35(1): 85–108. Available at: http://boundary2.dukejournals.org/cgi/reprint/35/1/85.pdf. Leverette, Marc. 2008. The Funk of Forty Thousand Years; or, How the (Un)Dead Get Their Groove On. In Shawn McIntosh and Marc Leverette (eds.), Zombie Culture: Autopsies of the Living Dead, 185–212. Plymouth: The Scarecrow Press. Linton, Simi. 1998. Claiming Disability. New York: New York University Press. McIntosh, Shawn. 2008. The Evolution of the Zombie: The Monster That Keeps Coming Back. In Shawn McIntosh and Marc Leverette (eds.), Zombie Culture: Autopsies of the Living Dead, 1–19. Plymouth: The Scarecrow Press. Paffenroth, Kim. 2006. Gospel of the Living Dead: George Romero’s Visions of Hell on Earth. Waco: Baylor University Press. Rogers, Martin. 2008. Hybridity and Post-Human Anxiety in 28 Days Later. In Shawn McIntosh and Marc Leverette (eds.), Zombie Culture: Autopsies of the Living Dead, 119–134. Plymouth: The Scarecrow Press. Salen, Katie and Eric Zimmerman. 2004. Rules of Play. Cambridge: MIT Press. Shaviro, Steven. 1993. The Cinematic Body. Minneapolis: University of Minnesota Press. Zagal, Jose P., Michael Mateas, Clara Fernandez-Vara, Brian Hochhalter and Nolan Lichti. 2007. Towards an Ontological Language for Game Analysis. In Suzanne de Castell and Jennifer Jenson (eds.), Worlds in Play, 21–36. New York: Peter Lang Press.

Susan Wright

23 The art of voice: The voice of art – understanding children’s graphicnarrative-enactive communication Abstract: This chapter illustrates how visual thinking and graphic representation constitute important and perhaps primary vehicles of understanding and communicating. Because art involves thinking in symbols, it is a significant conduit for the embodiment of thoughts and emotions and the surfacing of imaginative, expressive and creative capabilities. Using an analytical framework that features three modes – graphic, narrative and enactive – several examples of children’s work illustrate how drawing, talk and movement are parallel and mutually transformative processes. The textual features inherent in these multimodal representations are similar to that of film and theatre, involving: characters; objects, place, setting, time, scenery, décor; speech, actions, subtexts; and the flow of information, articulated in space and time (e.g., themes, plot, events and sequences). Examples of art-based play and aesthetic decision-making reveal children as philosophers, dramatists, technicians, aesthetes and fantasy creators and illustrate the significance of the semiotic act of meaning construction.

1 The significance of drawing in young children’s learning There is considerable evidence that visual thinking and graphic representation constitute an important and perhaps primary vehicle of understanding and communicating (Arnheim [1969] 1997; Gardner 1980). Indeed, for decades, art has been the focus of observation-based studies that have centred on children’s processes of symbolic representation (Arnheim [1969] 1997; Kress 1997; Vygotsky 1978), the act of creating (Golomb 2011; John-Steiner, Connery and Marjanovic-Shane 2010; Wright 2007a, 2007b, 2010) and problem-solving (Matthews 2003; Thomas and Silk 1990; Wright 2011). Increasingly, in-depth studies are documenting and describing how child art is equated with the general capacity to think and feel (Soundy 2012) and to cultivate dispositions for learning (Eisner 2002). Art is a significant conduit for children to illustrate their intellectual, emotional and aesthetic competences (Freeman 1980). Because the act of drawing is akin to play, it “lets in more meaning” (Pahl and Rowsell 2005: 43) for children than conventional methods of communicating. Indeed, drawing is a key precursor to learning sign-making in other forms of communication (Dyson 1993; Kress 1997). Art-making involves thinking in symbols, which Dewey ([1934] 1988) described as one of the most sophisticated modes of thought.

518

Susan Wright

Children are endowed with symbol-making propensities to go beyond reality as immediate experiences and to organise and make sense of their environment through art. Indeed, art goes hand in hand with children’s natural, early proclivities. As Egan (1999: 13) pointed out, the distinctive characteristics of young children’s thinking are actually superior to typical adults’ thinking, such as “imaginative skills attached to metaphor and image generation” and links to fantasy and fiction. However, contrary to traditional developmental theories, where children generally are seen to improve over time, these unique qualities tend to decrease as young children grow older, perhaps as a result of the “academic” curriculum taking precedence within schools (Anning and Ring 2004; Nichols and Berliner 2007). Yet these are qualities that should be preserved for as long as possible and, in fact, are highly valued by many professions and employers (Florida 2004; Pink 2006; Robinson 2009). Ironically, by depriving children of “non-academic” subjects, such as art, and focusing instead on “academic” subjects, we are underestimating children “intellectually”. Many researchers recognise the significant role of art in the development of early symbolic reasoning and have studied children’s drawings for at least 400 years (Fineberg 2006). Studies have focused on how the visual system of drawing is used by infants, toddlers and young children to communicate sophisticated concepts and feelings long before they begin to read or write (Gardner 1980; Golomb 2011; Kellogg 1969; Kolbe 2000; Kress 1997; Matthews 2003; Wright 2007b). Indeed, the things that children draw often represent highly sophisticated concepts – ideas that are more abstract than can be expressed in words alone. Drawing is a pictorial language that allows children to find concrete visual means of representing their thoughts and feelings through “a kind of child speech” (Vygotsky 1978: 112). Hence, drawing plays a fundamental communicative function and is an important semiotic tool for young children’s imaginative, creative, and expressive meaning-making (Vygotsky [1925] 1971). Vygotsky (1978) referred to drawing and speaking as “first-order” visual and verbal symbol systems and emphasised the important developmental link between the concrete representations of drawing and the more abstract one of writing. For Vygotsky, writing is the visual representation of the verbal system – using alphabetic notation – which he described as a “second-order” symbol system that serves as a handmaiden to drawing and speaking (Adoniou, 2012). When children draw, they depict things; when they write they are drawing disembedded language. Yet, the education system, which privileges the second-order symbol systems of literacy and numeracy, would be well advised to build upon children’s competence in the visual realm, to enrich and extend children’s knowledge and skills in other areas. The erudite nature of meaning-making and communication through drawing may be attributed to the integrated role of emotions and the body in relation to the creation of art. As Ross (2000) describes it, such integration involves exchanges between the psyche (mind) and the soma (body). Vygotsky ([1925] 1971:

The art of voice

519

215), similarly, argues that “art is based upon the union of feeling and imagination”. Like play, drawing is a dynamic and complex activity which, according to Vygotsky ([1933] 1976), represents an interactive form of embodied imagination – imagination in action – which “requires and leads to complex symbolic constructions” (John-Steiner, Connery, Marjanovic-Shane 2010: 11). The meaning-making efforts of children often “intertwine internal and external states” that appropriate “thought and signs together” (John-Steiner, Connery, Marjanovic-Shane 2010: 12). To Vygotsky, the process of creativity, which includes play, imagination and fantasy, is a “transformative activity” (John-Steiner, Connery, Marjanovic-Shane 2010: 12). Transformative characteristics of art making are similar those of child play. Levin and Bus (2003) and Wright (2010) describe how the open system of drawing allows for meaning-making to occur more easily than in more linear and rule-governed symbol systems such as writing. For instance, additions, deletions, substitutions and transformations can be easily made as the drawing is constructed. A human figure can become a cowboy by the addition of a hat; an object can be deleted by scribbling it out; this object can be substituted by another; and grass can be transformed into water to suit a shift in plot. Thus, as others have also noted (Adoniou 2012; Kress 1997), drawing allows for experimenting, evaluating, revising and integrating ideas. Yet, it must be acknowledged that a child’s drawing can fulfil a variety of purposes and functions. Hence, the artwork will be constructed quite differently depending on the child’s intentions. The young artist can choose different drawing tools, “just as a writer would choose different language structures when writing a description of a house or directions for getting around the house” (Adoniou 2012: 5). On one occasion, the child may decide to focus on aesthetic aspects, such as pattern, design or colour, with little regard for depicting people or events – similar to adult abstract art. Indeed, as with adults, children’s visual communication may also function as diagrams, maps, graphs, messages, pretty pictures, patterns, designs or other images. In this chapter, however, the focus will be on the multimodal, integrated texts of drawing in combination with narrative, created during playful interactions occurring between individual children and a trained, adult interlocutor.

2 Methodology – interlocutor-child creation of ZPD The methodology used in this research was similar to the approach espoused by Vygotsky in that a situation was created in which a new solution process was provoked (John-Steiner, Connery, Marjanovic-Shane 2010). Specifically, children were asked to draw “what they think the future might be like”. This provocation was intended to inspire intellectual challenges for children to imaginatively move beyond drawing their “best” schema (e.g., my house, family and garden).

520

Susan Wright

The children in the study were from two primary Catholic schools in a large town within a regional area of Queensland, Australia (more details about the design of the study can be found in Wright, 2007a). Neither of the schools employed an art specialist, although there was an art room which contained standard art materials and resources which were accessible to all classes; children participated in art-based activities for about 30 minutes per week. 108 five-to-eight year old children were randomly selected from three grade levels, as follows: 30 Preschool: 16 boys, Mean age 63.5, 14 girls, Mean age 63.3 38 Grade One: 19 boys, Mean age 75.0, 19 girls, Mean age 75.47 40 Grade Three: 20 boys, Mean age 98.7, 20 girls, Mean age 99.45

On a one-to-one basis (i.e. interlocutor-child), each child drew a picture using white A3 paper and a set of colored felt pens to represent their view of Futures. They were encouraged to discuss their drawing while they worked and, to elicit the key focus of the content, each child also was asked to give his/her work a title. Throughout the drawing-telling experience, the interlocutor asked open-ended questions, seeking clarification and extension of the child’s images, narratives and gestures. This was a challenging task as she simultaneously kept running records summarizing what was said, making rough sketches of the child’s images, and numbering these to note the sequence in which they were drawn. The sessions, which ranged between about 15–60 minutes in length, were video recorded to capture the children’s words, gestures, body language and facial expressions in relation to their unfolding visual narratives. Detailed transcriptions of parallel streams of graphic, verbal and non-verbal communication were then compiled to form the basis for analysis. The role of the interlocutor in this process warrants some discussion, as a great deal of sensitivity and empathy is required to grasp the content of the child’s work and the form in which this is communicated. When we search for both the reason and the meaning of the child’s communication, our attention centres on how the “features” of the artwork are ascribed aesthetic, intellectual and emotional significance. But the “context” surrounding the drawing may also influence the outcome (Anning and Ring 2004; Cox 2005) – as observers of a child’s processes, we become part of this context. Whether we are simply watching and listening, or actively engaging with the child, our presence serves as a form of facilitation (Wright 2010). Success in eliciting children’s self-directed drawings and encouraging related narratives depends on the researchers’ ability “to establish empathic and harmonious relationships in a non-threatening context, based on mutual trust” (Coates and Coates 2006: 226). Often simply observing, or perhaps commenting in general (e.g., “um-hum” or “I see”) without further elaboration can be adequate to sustain the experience. At other times, we may be interested in finding out more or obtaining a better understanding of what the child is doing. In this case, we assume the role of

The art of voice

521

“interlocutor” (Wright 2010), taking part in a dialogue with the child. Coates and Coates (2006: 226) reflect on their own research, noting that the researcher’s comments should encourage children “to extend their visual explorations and spoken discourse, and enhance their confidence in the worth of these by the means of approval and positive reinforcement”. Ahn and Filipenko (2007) also emphasize the importance of engaged discussion, so that children are able to define their realities and experiences. From a Vygotskian (1978) perspective, as an interlocutor, we are giving children experiences that are within their zones of proximal development (ZPD), thereby encouraging and advancing their individual learning. Newman and Holzman (1993) describe the ZPD as “a creative, improvisational activity” that people engage in collectively by taking risks and supporting each other. The ZPD is the “ever-emergent and continuously changing “distance” between being and becoming” (Holzman 2000: 5). Indeed, when ZPDs are created, people are performing – they are being who they are and who they are not, simultaneously (Lobman 2010) by pretending, playing, and imagining. So while the interlocutor predominantly assumes the role of audience, in many ways, he/she also serves as a type of playmate with the child, where his/her comments and questions may become a catalyst that enriches the child’s play. This may facilitate and protract the experience, and consequently, the play might be more recursive that if the child were playing alone (Wright 2010). In some ways, an interlocutor’s role is similar to dramatic improvisation. Hence, like with good jazz, the results of each episode will be unique and incapable of replication. By plunging into a mutual experience, the interaction becomes relational, interactive, negotiated. We should be truly curious and interested in how children approach the experience and ask them open-ended questions – the “kind that seek no specific answers but rather build a chain of ideas without the need for closure” (Paley 2007: 153). Sensitive comments usually will be open-ended and responsive, to surface the voice of the child rather than lead it. Hence, some strategies that might elicit a child’s elaboration may include (Wright 2010: 28): Clarification (“Can you give me an example of that? What did you mean when you said …?”) Mirroring or reflective probe (“What I hear you saying is … Have I understood you correctly?”) Nudging probes (“Really? So what happened then?”) Outloud thinking (“I wonder about … What do you think?”)

Such participation entails reciprocity not only between the “child and the materials” but also between the “child and the interlocutor”. Yet, it is worth noting that children’s visual communications tend to exist in a fragmented form (Wilson and Wilson 1979). As Labitsi (2007: 185) notes, “some children develop elaborate settings but do not people them with characters. Some concentrate on drawing actions devoid of any setting; others create characters that never go into action”.

522

Susan Wright

It is also noteworthy that the child’s ideas and depictions may shift in and out of an overall framework, where the complete structure may evolve in bits and pieces. At times the child may be one step ahead of us, explaining the meaning of what has just been drawn while simultaneously drawing a new concept (Wright 2007a, 2007b). Likewise, the child may return to previous images to elaborate on ideas with graphic or gestural detail, or extend the narrative in relation to these new concepts (Wright 2010). So, as interlocutors, we should see children’s graphic-narrative-enacted communication (Wright, 2007a, 2010) as “the depiction of content through graphic and body-based action, while talking about aspects of the artwork and the processes of its creation through a free-form type of narrative”. From a graphic focus, the examples featured later in this chapter illustrate how children carefully considered the “best” way to depict the content of their drawings by utilizing formal elements, such as spatial arrangements on the page, relationships of marks, lines and shapes, the use of colour, light and darkness, and the application of texture and shading. Because the provocation was for the children to talk about their drawings of the future, the narrative component, in particular, provided an avenue for what Van Manen (2001: 120) described as “practical theorizing”. For instance, in the Figure below, drawn by a girl (6 years 1 month), the young artist uses fantasy to depict how a mother protects her daughters (these daughters are described but not drawn). The narrative evolves through abstractions of binary concepts of aggression-defending and fighting-forgiving (Wright 2010: 56):

Fig. 1: Use of fantasy to depict how a mother protects her daughters. This is me and I am spreading out a picnic blanket for the children … My girls are over at the swings. Kayli and Sarah. One’s five and one’s ten. Their hair is really um good. When you brush it, it goes like silky. And Sarah, her hair she can think, and then she can … [magically] change her hair to go blue or green or red. She just likes different coloured hair. When someone has been naughty and she wants to strangle someone, she has got a magic hand and has got rings and just punches them in the head. When a baddie comes she [Kayli]

The art of voice

523

just tells her sister and her sister tells me, and then I have got blue eyes and stuff comes out [of my eyes], and it follows them and they go away. She doesn’t do this to her friends though, because she doesn’t want anything to happen to them. She got these magic powers when she was a little baby in her Mummy’s tummy. And she was powerful in her brain and neck and all that. Sometimes the girls fight, and they get a magic flower [left-hand side of the page] and they just stop the fight and are friends again. The magic flowers never grow, they are just as they are, they never die off.

Such rich tapestries of each child’s styles of thought were captured through video documentation of their graphic-narrative-enactive communication, and post-event transcriptions were made of these three modes of meaning-making. These videos and transcriptions provided records of children’s holistic communications of emotion, meaning and cognition. These artefacts illustrate how children built hypotheses about themselves and their world, and their real or imagined place within it. The next section describes how a semiotic orientation framed the analysis.

3 Semiotic analysis of graphic-narrative-enactive representations The analytical framework of the research was built predominantly upon the work of three key theorists – Vygotsky, Bruner and Peirce – and a range of authors who have extended on their theories. First, Vygotsky’s thesis surfaced how children’s art is a transformative activity ([1925] 1971). Art is the synthesis of thoughts, emotions and signs through embodied imagination (Vygotsky [1933] 1976), which he sees as a kind of “child speech” (1978: 112). Second, Peirce’s well-known trichotomy of three central signs – index, icon, and symbol – provided a framework for the analysis of how signs become meaningful through “semiosis”. Semiosis suggests that “sign functioning always involves an enlargement and expansion of meaning, not a simple substitution of one thing for another” (Siegel 1995: 1). Finally, Bruner’s (1986, 1996) three distinct types of reasoning – enactive, iconic and symbolic – provided a framework for describing how drawing is strongly associated with spatial and physical thinking through action. In addition, Bruner’s description of narrative (1986) provides a conception for describing how children construct their interior, psychological worlds. Fundamental aspects synthesized from these ideologies are summarized in an analytical framework which features three modes – graphic, narrative and enactive. Table 1 provides details of the forms that these modes may take, along with some examples of each for illustrative purposes. Examples of children’s communications have been selectively sampled to illustrate particular features and to highlight key terminology in relation to Table 1 (see Wright, 2010, for more in depth details and additional examples of children’s work).

Literal

Fictional

Metaphoric

through verbal

presentation)

Filmic textual features

(representing

Characters; objects, place, setting, time, scenery, décor; speech, actions and subtexts of depicted characters; structure and flow of information, articulated in space and time (e.g., themes, plot, events)

Spatial-temporal relationships

Non-fictional

In front/behind, close/distant, above/below, similar, proximal, surrounded, vertical/horizontal axes; shifting of scenes and timeframes within the context of the drawing (See Location and Time Indexes)

Visual icons

presentation on a surface)

Narrative

Speech/thought bubbles to “voice” the characters; whoosh lines to “move” the objects or figures; radiating lines to suggest vibration or “lighting up”; dotted lines to show movement trajectories; marks to connect or separate objects; dotted lines to represent “invisibility”; an arrow to show the location of some referent in spatial terms

Symbols

(representing through visual

Rhetorical, symbolic, allegorical, abstract. This may include: Figures of speech (e.g., heavy music); Repeating words for emphasis (e.g., fat, fat, fat sleeves) Vocalisms (i.e., emphasis through change in pitch, volume or speed of speech); Onomatopoeia (i.e., the use of a word or vocal imitation of the thing or action designated (e.g., grrrr to spin a chair; drrrr to play a guitar).

Imaginary, unreal, fantastic, illusory

Descriptive, factual, exact, unembellished (often delivered in third person; omniscient)

Real, true-life, personal (often presented in first person)

Letters, words or phrases to label characters, places, objects or events, or to title the work; numbers to quantify content or signify sequences; flags, traffic lights, emblems, badges, motifs, logos; prototypes to universally stand for a category (e.g., heroic human)

Marks, lines, shapes, colours, textures, shadings, proportions, composition, perspective; metaphor (e.g., warm colour, angry edges)

Art elements

Graphic

Example

Form

Mode

Tab. : Modes, Forms and Examples for the Analysis of Children’s Communication.

524 Susan Wright

Gestures, facial expression, body-language Action to accompany onomatopoeia (e.g., flicking fingers to go together with the word pooooofff) Repeated word to emphasise a graphic or dramatized action (e.g., colour, colour, colour; walk, walk,walk) Enacting as if being a character or an object (e.g., using fingers to “walk” up drawn stairs)

Expressiveness

Dramatisation

through motor presentation)

Pointing at something or moving the hand across or outside the page to indicate location, time, character or structure of content (e.g., the shape of the moon in relation to Earth’s revolutions)

Location: to indicate a physical or spatial context (this, that, here, there) or relative location (near, far) Time: to indicate a temporal context (before, after, now, then) Identification: to indicate “characters” in a specific context, perhaps relative to other characters (I, you, he, she, they or indefinite pronouns such as the one, or the other) Organization: to indicate a classification of things in relation to other things (e.g., alphabetical order)

Indexical (terms that clarify “with reference to” links)

Descriptive action

Example

Form

Enactive (representing

Mode

The art of voice

525

526

Susan Wright

The textual features inherent in children’s graphic-narrative-enactive representations, similar to that of film and theatre, include (Wright 2007b, 2010): – characters with signifying functions (e.g., postures, expressions, gestures), linked to their roles, personal qualities, behaviours and goals; – objects, place, setting, time, scenery and décor and associations with these; – speech, actions and subtexts of depicted characters; and – structure – the flow of information, articulated in space and time (e.g., themes, plot, events and sequences). During the children’s art-based play, aesthetic decisions are being made on the selection, execution, framing and reframing of their ideas in relation to these textual features. Such decisions allow the young artists to bring their works “alive”, similar to how a film director works with various elements to create a “movie”, or how an actor depicts a character in a “play”. The child’s plot and his/her authorial intentions often changes, as the child spontaneously “edits” the evolving visual narrative. Extending on Peirce’s trichotomy, the analysis of children’s texts, and the processes they used to generate these texts, was additionally informed by the visual semiotician Roland Barthes (1973, 1977), who focused on visual layers of meaning in terms of denotation (surface level) and connotation (deep level). Barthes (1977) describes denotation as the “literal message”, such as the people, places, objects and events depicted, and connotation as the “symbolic message”, such as the broader, abstract concepts, ideas and values being expressed. Abstract concepts, like “love”, “friendship” and “justice” are especially high in connotation. Such concepts, which Barthes sees as culturally shared meanings, are communicated in visual forms through associations with “what” the representations stand for. Examples might include a person with arms raised heavenwards, connoting exuberance, or scenery which features a bookcase, connoting an intellectual environment. Hence, connotation is not only about how content is being communicated, but also how the form of this content is expressed aesthetically, through techniques such as shading, bold lines, strong contrasts, angles, or framing. As an illustration, the philosophical representation in Figure 2 by a boy (8 years 1 month) contrasts two parks, one living and one dying, to depict concepts of pollution through two binaries: healthy-sick and preservation-destruction. He describes the causes of pollution and its affect on animals, people and the environment, and how peoples’ greed and lack of government policies have failed to address these concerns (Wright 2010: 142). There’ll be lots of cars, trucks and motorcycles … noise pollution. And a lot of our seas won’t be as nice as they are now. Because … of … us … because whenever we get more buildings, we start not caring about what we already had. ‘Cause all the smoke would cause a lot of acid rain – it’s rain with a lot of pollution and stuff in it … and chemicals. And these are electricity centres. And that’s all the smoke coming out.

The art of voice

527

Fig. 2: Portrayal of healthy and poluted parks.

These are trees from a far distance. And these little things are birds flying around. And a waterfall. I’m going to put a sign there to say that they’re going to be gone. And that’s going to be the other waterfall, and I’m going to draw some dead trees. We’re not looking after it. And it’s mainly our fault, not the animals’. Humans have been cutting down trees, and um … looting … taking other peoples’ treasures, ‘cause they’ve worked hard all their life to keep the parks a long time. They wanted that [non-polluted park] … but the government didn’t listen … and, now it’s that [polluted park]. There’s a dead bird over here. It won’t be a nice place to live, if this did happen. There’s two people walking here [healthy park]. But there’s none here [dead park]. Because they wouldn’t want to go here. On this side, they’d come back, but over here, that bird’s kicking out now that it’s nest’s gone. It can’t survive over here, ‘cause some of the trees here are getting cut down. There is a lot of air pollution, noise pollution and [water] pollution that is making us feel not that good about ourselves.

Metaphorically, the boy uses movement-based terms to describe the bird’s “kicking out” and refers to the humans cutting down of trees as “looting” other people’s “treasures”. The artist uses aesthetic form to show the parks in the far distance. Compositionally, the close proximity and similarity of these two parks suggest a conceptual link. He frames these parks using billowing, cloud-like shapes, similar to thought bubbles, implying a type of internal reflection. The healthy park’s frame is blue, connoting purity whereas the unhealthy side is black, suggesting death; likewise the two large birds in each of the parks are blue and black respectively. The trees in the healthy park are upright with foliage whereas those on the dying side are slumping and bare. Ideologically, the artist’s message is that it is the human’s fault (and not the animals’), in particular the government’s fault, that there is pollution and the death of living things. Prophetically, he is trying to send a message, and says he will “put a sign there” to warn people that it won’t be a nice place to live unless they begin to care for their environment.

528

Susan Wright

Fig. 3: Depiction and enactment of digging and walking.

In contrast to this philosophical message, the following example illustrates a younger boy’s (5 years 2 months) dramatization of his communication. His depiction is an embodied experience, involving vocalisms, descriptive gestures, facial expressions and body movements to clarify, enhance or enliven his communication. As in play, his drawing was enactive and his meaning was represented through motor response. Indeed, the word “backhoe” which he asked the interlocutor to write above the machine, was intended to function as a verb (“what I’m doing”) rather than as a label for an object. Reuban said, “this is a small machine because it’s got a digger at the front (puts hands out like a tray) and it’s got a little digger on the side”. His visual narrative included: action (e.g., circular lines in the ground to show the energetic results of the machine’s work on the dirt), and direction (e.g., the upward orientation of the stairs, with Reuban stepping up the treads).

Reuban was aware of proportion within the artwork and how the scale of the drawing was smaller than real objects, saying: “It’s going to be a big shed. It’s gotta be a little drawing it will.” In addition, the stairs represented only part of the house: “And I had to go off this thing [the page] because I didn’t have enough room”. The close-up depiction of Reuban walking up the stairs was dramatized by his use of his fingers on the page, as if they were the drawn figure’s legs taking each step. This action was accompanied with the repeated word, “step step step …” voiced in first person in synchrony with his finger movements. He then said, “I’m going to make a tunnel into the house too”, and pushed his thumb into the paper, as if it were the front digger of the backhoe, breaking up the dirt; he used repeated words, “dig dig dig” to accompany this dramatized action. Through these multimodal strategies, Reuban “entered” the drawing (Kress 1997), making the representation of movements more “real”. Soundy (2012: 49) describes this similarly in relation to children’s engagement with storybooks, stat-

The art of voice

529

ing that children virtually move into the action zone, whereby they become “agents with the characters and directly identify with events, images, or anything else that attracts their attention”. In Reuban’s communication, the integration of graphic, narrative, and enactive modes worked in a symbiotic way, each informing and enriching the other (Wright 2010). Suhor (1984) and Leland and Harste (1994) refer to this as transmediation – the “translation” of content from one sign system into another. Translation has the potential for enhancing leaning – moving across sign systems is a generative process where new meanings are produced (Siegel 1995: 3). The visual, graphic and embodied modes work in complementary ways, thus “offering additional (perhaps more accessible) avenues for children to represent their knowledge” (Binder 2011: 377). Cultural knowledge that has been coded into multiple semiotic means functions similarly to documentaries or movies in that they hold the potential of being more successfully mediated to a broader audience (Connery 2010: 88). Indeed, Wright (2007b) found analogies between children’s drawings and filmic texts, aspects of which are illustrated in the drawing below by a boy (8 years 6 months). The artist’s textual features included: (a) multiple scenes and timeframes within a single artwork and (b) iconic devices, such as whoosh lines accompanied with gesture, to animate the action. His artwork centres on events involving sci-fi technology, including a car that can fly (bottom, centre of page). The artist imaginatively applies concepts of solar-based power, gravity and daynight relationships in various parts of the world (Wright 2010: 148). Okay, I’m just going to draw the family car first, with special air bubbles for the kids, and an air bubble for the adults, so the kids don’t keep on yelling in your face. It’s a sound proof bubble … And there are little jet packs on the back, so you can go a bit faster … And it’s solar powered. But … you’ll still have fuel (adds red whoosh line behind the car). And … in case you want to go to San Francisco, and you’re in Australia … there is a little thing here that can take you there. (Draws a red box below the car). But you have to turn on to your solar power because, say if you went at day, it would be night there.

John applies scientific knowledge when describing cameras, lasers and force fields in relation to surveillance at the bank and the special technology that the police use to catch criminals (Wright 2010: 149). And the banks [building on the left] have more lasers, to track down the bad people that are coming in. The bank’s not that big because they can store the money in other banks. (Adds a surveillance camera to the right-hand side of the bank). And they have special lasers that come down, like a big wall that you can’t go through. It comes down to the very bottom of it.

(Adds a square to the left side of the bank, with a surveillance camera suspended from its roof). And, just in case they come in by the back, up here, there’s a little camera, and that laser there will go from the back to the front, and from the front to the back. And if there’s two

530

Susan Wright

Fig. 4: Drawing including sci-fi technology.

people coming from the back and the front, the lasers will split in half and cover both areas. And the bank also has a good advertisement. You see, the people get fooled. They have a safe in there. (Draws a safe on top of the bank). And if anyone tries to get in, it’ll have that [arch] over it, ‘cause they’ll fool everyone. Because the safe inside the bank isn’t filled with any money, only the cash registers are. But this [safe up here] is filled with money. So, they trick them, by putting their money on the top [of the bank]. But in case they’re a bit curious, they’ll go in there, and that’ll stop ’em. (Adds radiating lines to the arch over the safe). By the force field around there. It’ll stop ’em, say if they’re a only a metre close. A special ray’ll hit them and kind of knock them out, but not seriously … just … put them asleep … ‘til the police get there. ‘Cause the police have got this special dart. If noone sees the man, he’ll sleep forever. But the police have got this special dart that they can put in him and stop him from sleeping.

Then John draws a dome over the city, with associated concepts of temperature control (Wright 2010: 150). And … they’ve got a new technology. Here’s the sun and that’s all the clouds and everything. And there’s a special kind of a roof, over the city [undulating lines]. It’s in all different colours ‘cause people have spray painted it. Graffiti (gestures swirling motion across the page, following the line of the roof). So, in case you’re getting a bit cold (grabs the sides of the desk and shivers to dramatize this), it’ll … kind of [be like] one of them warmers in a house. Some cities have it, some cities don’t. Keeps it cool in summer and warm in winter. But say if it’s really hot in winter, it’ll get cool.

John draws a force field that people can pass through without injury. The artist applies principles of anthropomorphism when creating a human-like character – it has human qualities and abilities but is “geographic” [holographic]. If a kid wants to cross the road, and cars might be in his way … he’s kind of a geographic model. And, say if he’s in the bank, and he wants to walk over to the shop, but the truck’s too big, he can just run through cars and everything, and go to the shop.

The art of voice

531

This holographic quality is also applied to cars, enabled through a devise that drivers attach to the fronts and backs of their cars, so they can pass through each other in the event of an accident. If a car ran into another car … your car will turn into a geographic model. A special thing on its bonnet. So, the car will just hit that thing first. There’s a special thing at the back too that will hit them. And if two cars both have these they’ll just go through each other. And they can just keep on driving.

John’s visual narrative was played out on a seemingly impersonal plane of pictorial fantasy. His characters had no individual personality. Instead, they were embodiments of abstractions, presented in a prototypical fashion. The characters were narrated in third person, as in a commentary, where opinions and descriptions were presented from an omniscient point of view. Yet, in some ways, John could impersonally “be” everyone and no-one at once. The artist’s voicing of people, objects and events allowed him to assume some element of his fictional self. Characters were spoken about using identification indexes like “he”, “that person”, “someone” or “they”, and events and objects were spoken “through” the artist, described using location indexes such as “this” or “there”. John readily applied fantasy to create fictional characters that bore some resemblance to personal experience, but his sci-fi knowledge liberated him to morph reality, to create fictional characters in a fictional world. A final example by a girl (8 years 5 months) illustrates how the scenery of the artwork connotes peace. The artist drew herself at the age of 59, beside her lifelong, best friend. These two figures are in close proximity in a central position, standing with walking canes beneath a highly colourful sunrise. Although the narrative centres on growing old, the prominent sunrise, coupled with back-to-nature ideologies, create an atmospheric, spiritual context that features animals, fish, plants and flowers. The peacefulness of the setting and its positive associations with old age make it tempting to metaphorically call the artwork “The Sunrise of Life” (Wright 2010: 161): I’m drawing the grass in the garden and a rose bush [far right]. It’s got some roses on it. And I’m picking flowers and I’ve got some nice ferns. I’m going to draw myself when I get big. I’m old. It’s a walking stick. I’m about 59. And this is Brooke. She’s old like me. We’ve been friends forever. She’s got brown hair. Long curly hair … blowing out everywhere. Out to enjoy the world [metaphorically]. Brooke has got a blue tracksuit. Now I’ll draw a dog with a bone, because it likes its bone. The dog is gold. It’s a golden dog, so I’ll draw it yellow. And I’m going to draw a fishpond, because we like to keep fish. We’ve got some purple fish, some red fish and a multicoloured fish. And the cover [on the fish pond] is made out of clear glass so we can see through it, to see all the fishes. And the red fish is putting its head up to say hello. We’re in the country, and we’ve got some horses in the paddock. And my horse is called Star and Brooke’s is King. Winter’s almost over but there’s some blue sky, and it’s early in the morning. And it’s colder in the mornings I guess, so that’s why I need my tracksuit. There is some maroon sky. I think

532

Susan Wright

Fig. 5: “The Sunrise of Life”. I have to draw some red in the clouds so that it is the sun making them bright. The sun is making the colourful sky. We didn’t like the city because it was too noisy, so we moved from the city to the country. And we just wanted to go back there. ‘Cause that was the way our life was meant to be. We only get up early once a week, otherwise we don’t get enough sleep. I’ll call this ‘Peaceful Morning’ because there’s quiet in the country and it’s so beautiful in the peace and quiet.

The five examples of children’s graphic-narrative-embodied communications featured in this chapter are quite representative of the individualistic types of meaning-making that emerged from the overall group of children in the project. Within the scope of this chapter, these few examples can only be illustrative, rather than inclusive. However, such illustrations celebrate children as fantasy creators, philosophers, dramatists, technicians, and aesthetes (see Wright 2010) with their own practical theorizing about the future and their place within it. Some of the key modes and forms of communication used during the interlocutor-child interactions are discussed in the final section, and key principles featured in Table 1 are emphasized for illustrative purposes.

4 Discussion Given the topic “The Future”, it may have been anticipated that the children would have focused their drawings on prophesizing about what their lives and worlds would be like in some forthcoming period. A stereotypic notion of this might have been along the lines of the cartoon TV Series The Jetsons (1962–1988), with scenarios focusing on the misadventures of a futuristic family. Instead, the children approached the task sincerely and sensitively, and appeared to frame their work in relation to their current realities, envisaging where this reality might lead. The children’s representations seem to have been based on their familiar perceptions and experiences, and these most likely reflected the children’s own reali-

The art of voice

533

ties and the common realities of others. It is also likely that these realities were derived from layers of memories, emotions, impressions and desires. Often the children’s communications addressed normative realities, where they re-presented the culture’s standards using binary oppositions (Wright 2011), such as good/bad, right/wrong, just/unjust, which they played out through their imaginations. And as the futures-based task suggested, the children’s communications generally symbolised their forthcoming lives by generating images and models of their own prospective selves, actions and worlds. Such outcomes resonate with the reality classifications described by the Wilsons (1982), which were formulated based on their own studies of children’s drawings, namely: common, archaeological, normative and prophetic realities. The first example in this chapter, created by a six-year-old girl, shows a strong blend of emotion and thought in her fantasy-based representation of her future self, as a mother. The main themes might be seen as addressing a common reality in that they centre on the ideology of protecting her children from harm and on reducing sibling aggression, but the form in which this is communicated is through the use of telepathic mind power (blue stuff that comes out of the mother’s eyes and chases the baddies away) and through the giving of magic flowers (so that the daughters stop fighting). The second example, by an eight-year-old boy, addresses prophetic and normative realities by centring on binary oppositions: a polluted versus healthy world. In addition to depicting content that stood for pollution, such as smog, industrial waste and acid rain, the young artist connotatively captured the emotion of this message by using a number of body-based metaphors (Lakoff and Johnson 1980) such as drooping trees, falling acid rain, rising smoke and a bird “kicking out” of the polluted park. He enhanced the form of his message through the use of colourbased metaphors to connote the concepts of blue=living and black=dead. Third, the five-year-old boy’s dramatist projection of himself, as a backhoe driver in the future, reflects a common reality (the perception of machines/toys and their movement). The artwork is an embodiment of the machine digging in the dirt, with the artist using his thumb to simultaneously emphasising the enactment using the repeated words, “dig, dig, dig”. The boy “entered” the drawing by depicting himself climbing the staircase and waving goodbye, and used his fingers as if they were the legs of the drawn figure, while simultaneously chanting “walk, walk, walk” as he stepped up the treads. The fourth example, by an eight-and-a-half year old boy, is futures-based and might reflect a prophetic reality. The visual narrative is highly technical and includes many actions that are described but not drawn, such as lasers from surveillance cameras scanning the bank, and holographic people/objects going through each other without feeling anything. The content centres predominantly on futuristic resources (e.g., solar-fuel powered flying cars, a thermostatically controlled domed city and invisible force fields surrounding banks). These prototypes deal with ideologies of physical comfort and environmental security.

534

Susan Wright

The final drawing, by an eight-and-a-half year old girl, might be seen as reflecting an archaeological reality, and could possibly be derived from memories or impressions of older people’s desires for peace and tranquillity in the latter stages of their lives. The artist’s communication is a blend of emotion and aesthetics, which centres on sensorial matters of warmth and quietness. These are represented through visual metaphors of a colourful sunrise and described through body-based metaphors of freedom such as her life-long friend’s extensive hair “blowing out everywhere, out to enjoy the world”. These artefacts of children’s symbolic reasoning demonstrate their individual styles, and their sophisticated appropriation of thoughts, emotions and signs, which worked together in a symbiotic way. The examples demonstrate how the use of the visual realm extends and enriches children’s existing competencies. What can be expressed through the integrated graphic-narrative-embodied modes is more sophisticated and abstract than can be expressed through words alone. The multimodal act liberates children to communicate layers of realities and to surface, and sometimes discover anew or for the first time, relationships between their internal and external states. Graphic-narrative-enactive play is a transformative activity. It is an interactive form of embodied imagination, where body and mind unite, and where intellectual, emotional and aesthetic forms of meaning-making intertwine. The content that children denote at the surface level can be understood when we look deeper into the connotation of their meaning, which is revealed in the formal properties of their: (a) graphic representation and use of art elements, such as colour, line, composition, icons and spatial-temporal relationships, (b) narrative representation, such as their use of figures of speech, repeated words, vocalisms, and indexical terms and (c) enactive representation, such as descriptive action, expressive gestures and dramatizations. As interlocutors, we enter that creative, imaginary, expressive space with the child. By reflecting on the child’s use of textual features, such as their characters, objects, places, settings, speech, actions and structures, we come to understand and appreciate the imaginative, creative, expressive and metaphoric voices of young children. These are voices well worth hearing, respecting and sharing.

References Adoniou, Misty. 2012. Drawing to Support Writing Development in English Language Learners. Language and Education 27(3): 261–277. Ahn, Jiryung and Margo Filipenko. 2007. Narrative, Imaginary Play, Art, and Self: Intersecting worlds. Early Childhood Education Journal 34(4): 279–289. Anning, Angela and Kathy Ring. 2004. Making Sense of Children’s Drawings. Maidenhead: Open University Press. Arnheim, Rudolf. [1969] 1997. Visual Thinking. Berkeley: University of California Press.

The art of voice

535

Barthes, Roland. 1973. Mythologies. London: Fontana. Barthes, Roland. 1977. Image-Music-Text. London: Fontana. Binder, Marni. 2011. Contextual Worlds of Child Art: Experiencing multiple literacies through images. Contemporary Issues in Early Childhood 12(4): 367–380. Bruner, Jerome S. 1986. Actual Minds, Possible Worlds. Cambridge: Harvard University Press. Bruner, Jerome S. 1996. The Culture of Education. Cambridge: Harvard University Press. Coates, Elizabeth and Andrew Coates. 2006. Young Children Talking and Drawing. International Journal of Early Years Education 14(3): 221–241. Connery, M. Cathrene. 2010. The Social Construction of a Visual Language: On becoming a painter. In M. Cathrene Connery, Vera John-Steiner, and Ana Marjanovic-Shane (eds.), Vygotsky and Creativity: A Cultural-historical Approach to Play, Meaning Making and the Arts. 83–105. New York: Peter Lang. Cox, Sue. 2005. Intention and Meaning in Young Children’s Drawing. Journal of Art and Design Education 24(2): 115–125. Dewey, John. [1934]1988. Art as Eperience. New York: Perigee Books. Dyson, Anne Haas. 1993. From Prop to Mediator: The changing role of written language in children’s symbolic repertoires. In Bernard Spodek and Olivia N. Sarachio (eds.), Yearbook in Early Childhood Education: Language and Literacy in Early Childhood Education. Vol. 4, 21–41. New York: Teachers College Press. Egan, Kieran. 1999. Children’s Minds, Talking Rabbits and Clockwork Oranges: Essays on Education. New York: Teachers College Press. Eisner, Elliot W. 2002. What can Education Learn from the Arts about the Practice of Education? Journal of Curriculum and Supervision 18(1): 4–16. Fineberg, Jonathan (ed.). 2006. When We Were Young: New Perspectives on the Art of the Child. Berkley: University of California Press. Florida, Richard. 2004. The Rise of the Creative Class. New York: Basic Books. Freeman, Norman H. 1980. Strategies of Representation in Young Children. New York: Academic Press. Gardner, Howard. 1980. Artful Scribbles: The Significance of Children’s Drawings. New York: Basic Books. Golomb, Claire. 2011. The Creation of Imaginary Worlds: The Role of Art, Magic and Dreams in Child Development. London: Jessica Kingsley. Hozman, Lois. 2000. Performative Psychology: An untapped resource for educators, Educational and Child Psychology 17(3): 86–103. Hozman, Lois. 2010. Without creating ZPDs there is no creativity. In M. Cathrene Connery, Vera John-Steiner and Ana Marjanovic-Shane (eds.), Vygotsky and Creativity: A Cultural-historical Approach to Play, Meaning Making and the Arts, 28–39. New York: Peter Lang. John-Steiner, Vera, M. Cathrene Connery and Ana Marjanovic-Shane. 2010. Dancing with the Muses: A Cultural-historical Approach to Play, Meaning Making and Creativity. In Connery, M. Cathrene, Vera John-Steiner and Ana Marjanovic-Shane (eds.), Vygotsky and Creativity: A Cultural-historical Approach to Play, Meaning Making and the Arts, 4–15. New York: Peter Lang. Kellogg, Rhoda. 1969. Analysing Children’s Art. Palo Alto: National Press Books. Kolbe, Ursula. 2000. Seeing Beyond Marks and Forms: Appreciating children’s visual thinking. In: Wendy Schiller (ed.), Thinking through the Arts, 48–60. Amsterdam: Harwood Academic Publisher. Kress, Gunther. 1997. Before Writing: Rethinking the Paths to Literacy. London: Routledge. Labitsi, Visiliki. 2007. ‘Climbing to Reach the Sunset’: An inquiry into the representation of narrative structures in Greek children’s drawings. International Journal of Education through Art 3(3): 185–193.

536

Susan Wright

Lakoff, George and Mark Johnson. 1980. Metaphors we Live By. Chicago: University of Chicago Press. Levin, Iris and Adriana Bus. 2003. How is Emergent Writing Based on Drawing? Analyses of children’s products and their sorting by children and mothers. Developmental Psychology 30(5): 891–905. Lobman, Carrie. 2010. Creating Developmental Moments: Teaching and learning as creative activities. In M. Cathrene Connery, Vera John-Steiner and Ana Marjanovic-Shane (eds.), Vygotsky and Creativity: A Cultural-historical Approach to Play, Meaning Making and the Arts, 199–214. New York: Peter Lang. Matthews, John. 2003. Drawing and Painting: Children and Visual Representation, second edition. London: Paul Chapman. Newman, Fred and Lois Holzman. 1993. Lev Vygotsky: Revolutionary Scientist. New York: Routledge. Nichols, Sharon L. and David C. Berliner. 2007. Collateral Damage: How high stakes testing corrupts America’s schools. Cambridge: Harvard Educational Press. Pahl, Kate and Jennifer Rowsell. 2005. Literacy and Education: Understanding the New Literacy Studies in the Classroom. London: Paul Chapman. Paley, Vivian Gussin. 2007. HER Classic: On listening to what the children say. Harvard Educational Review 77(2): 152–163. Peirce, Charles Sanders. [1931–1958]. In Charles Hatshorne, Paul Weiss and Arthur Burks (eds.), The Collected Papers of Charles Sanders Peirce [in Eight Volumes]. Cambridge: Harvard University Press. Pink, Daniel. 2006. A Whole New Mind: Why right-brainers will rule the world. London: Penguin Books. Robinson, Ken, with Lou Aronica. 2009. The Element: How finding your passion changes everything. New York: Cambridge University Press. Ross, Janice. 2000. Arts Education in the Information Ages: A new place for somatic wisdom. Arts Education Review 101(6): 27–32. Siegel, Marjorie. 1995. More Than Words: The generative power of transmediation for learning. Canadian Journal of Education 20(4): 455–475. Soundy, Cathleen S. 2012. Searching for Deeper Meaning in Children’s Drawings. Childhood Education 88(1): 43–51. Suhor, Charles. 1984. Towards a Semiotics-based Curriculum. Journal of Curriculum Studies 16(3): 247–257. Thomas, Glen V. and Angèle M. J. Silk. 1990. An Introduction to the Psychology of Children’s Drawings. New York: New York University Press. Van Manen, Max. 2001. Researching Lived Experience. London, Ontario: Althouse Press. Vygotsky, Lev. 1971 [1925]. The Psychology of Art. Cambridge: MIT Press. Vygotsky, Lev. 1976 [1933]. Play and Its Role in the Mental Development of the Child. In Jerome S. Bruner, Alison Jolly and Kathy Sylva (eds.), Play – Its role in development and evolution, 537–554. New York: Penguin. Vygotsky, Lev. 1978. Mind in Society: The Development of Higher Psychological Processes. Ed. by Michael Cole, Vera John-Steiner, Sylvia Scribner and Ellen Souberman. Cambridge: Harvard University Press. Wilson, Brent and Marjorie Wilson. 1979. Children’s Story Drawings: Reinventing worlds, School Arts 78(8): 6–11. Wilson, Brent and Marjorie Wilson. 1982. Teaching Children to Draw: A Guide for Parents and Teachers. Englewood Cliffs: Prentice-Hall. Wright, Susan. 2007a. Graphic-narrative Play: Authoring through multiple texts and fluid structures. International Journal of Education and the Arts 8(8): 1–27.

The art of voice

Wright, Susan. 2007b. Young Children's Meaning-making through Drawing and ‘Telling’: Analogies to filmic textual features. Australian Journal of Early Childhood 32(4): 37–48. Wright, Susan. 2010. Understanding Creativity in Early Childhood: Meaning-Making and Children’s Drawings. London: Sage. Wright, Susan. 2011. Meaning, Mediation and Mythology. In Dorothy Faulkner and Elizabeth Coates (eds.), Exploring Children’s Creative Narratives, 157–176. London/New York: Routledge.

537

Danielle Almeida

24 The political values embedded in a child’s toy: The case of “Girl Power” in the Brazilian doll Susi Abstract: Advertisers draw on imagery and language to represent their products to specific markets. This is a process of loading these objects with meanings. For example, beer becomes loaded with the meanings of “friendship”, a car with “freedom”. These products become infused with particular ideas, values, identities and social practices. Of course we might argue that these are the identities that are in harmony with the values of corporate consumer capitalism. In this paper I am particularly interested in the marketing material for Brazilian doll made by the toy manufacturer Estrela. To best draw out the kinds of identities, values, ideas and social practices that are loaded onto this doll I draw on some of the concepts presented by Kress and Van Leeuwen (1996; 2006). This social semiotic approach allows us to look at the kinds of gender representations of women that present a particular passivized and aestheticized view of women that fuses with changing work and career roles in Brazilian society.

1 Introduction The toy industry is able to generate billions of dollars driven by strong worldwide publicity campaigns that launch a vast number of objects targeted at different agegroups and genders onto the market every year. Toys form powerful and ubiquitous representations in the world of children. Toys such as Barbie, Play Mobil, Power Rangers, Hello Kitty are merchandised not only as objects to be held and manipulated but also that form parts if interior designs, clothing, and entertainment media in the form of television programmes and computer games. (CaldasCoulthard and Van Leeuwen 2001). Arguably, this co-operation between the media and the toy industry plays a fundamental role in the construction of identities initiated during childhood, attesting to its role as a meaningful form of mass culture that helps to perpetuate the ideology of consumption through its fierce strategies of promotional and persuasive discourse (Kline 1993; Seiter 1993; Fleming 1996). Such semiotic representations have made some scholars (e.g. Machin and Van Leeuwen 2009; Caldas-Coulthard and Van Leeuwen 2001, 2002; Varney 1999, 2002; Fleming 1996; Kline 1993, Seiter 1993) attempt to demystify the common-sense belief that toys should be regarded as mere trivialities, conventionally incorporated to our day-by-day life as domestic things. What these scholars have tried to do is to bring to surface some of the social and cultural values that toys communicate

540

Danielle Almeida

and that are transmitted to children from early age. Toys therefore transmit models of kinds of ideas, values and actions that exist in the world and in doing so they can be seen as taking a fundamental role as cultural assets responsible for the construction of and fostering of particular kinds of social identities. In this chapter I take this approach to the Brazilian girl’s doll Susi. While toys have been given some attention by scholars (e.g. Machin and Van Leeuwen 2009; Caldas-Coulthard and Van Leeuwen 2001, 2002; Varney 1999, 2002; Fleming 1996; Kline 1993, Seiter 1993) there has been little research done specifically on girls’ toys as a subgenre. As Peers (2004) has pointed out in her seminal book The fashion doll – from Bébé Jumeau to Barbie, such an omission constitutes a scholarly lacuna given that girls’ toys stand for reliable sources of historical and social information. Considering toys as semiotic representations of gendered social actors (Caldas Coulthard and Van Leeuwen 2002), this paper takes a social semiotic approach to communication, which involves a closer analysis of the specific communicative resources used by visual designers with a view to drawing out the kinds of ideologies that they communicate. I draw on a number of concepts offered by Kress and Van Leeuwen’s (1996). They present three kinds of representational forms which derive from Halliday’s functional grammar (1994), namely, the representational, interactive and compositional meanings. These, as I will show, offer useful ways to pick apart the configurations of semiotic choices found in the advertisements for Susi. The analysis of visual structures viewed from the paradigm that pictorial structures do not merely reproduce structures of reality but “are bound up with the interests of social institutions within which the pictures are produced, circulated and read” (Kress and Van Leeuwen 1996: 45) sets the ground for the semantic realisation of pictures as indeed ideological. Having said that, an analysis of images like the ones presented here, supported by a structured mode of analysis such as the visual social semiotic approach, allows for the categorization of pictorial patterns as representatives of a broader domain of experience.

2 Fashion dolls: Artefacts of female representation In her seminal book The Fashion Doll: From Bébé Jumeau to Barbie, Peers (2004) tracks back the history of fashion dolls over the past four decades in a cultural, art-historical perspective. For her, the term “fashion doll” implies four different types of dolls. The first one refers to dolls prior to the 19th century which conveyed information about Parisian fashion. The second type of fashion doll can be exemplified by the collectable Parisian adult female dolls with porcelain heads produced from 1850’s to 1890’s. The third type relates to the adult female plastic dolls manufactured in the United States during the 1950’s, which also reflected “French

The political values embedded in a child’s toy

541

Haute Couture”. The fourth and last type of fashion doll, developed through the late 1950’s and used up to the present day, is linked to the contemporary usage of the term which includes “adult female-styled” dolls. This last category refers to dolls like Barbie and their strong relation with the various expressions of popular culture, such as “fashion […], entertainment, film, cult television and the music industry” (Peers 2004: 15). As it can be seen, dolls of different eras embody culture and identities in different ways, but it is the last type of fashion dolls that I will focus on this investigation. While promoting these toys, advertisers have been requested to specify “the particular strata they are targeting” (Kline 1993: 248), by picturing only boys as the reference group in advertisements of vehicles and male action dolls and only girls in advertisements for fashion, baby and female action dolls (1993). When criticised for acting in a “narrow and sexist” manner, toy marketers argue that what they promote is “the most popular and acceptable kinds of play” and that “sex-stereotypic play existed long before advertising” (Kline 1993: 252). However, before the Industrial Revolution the gender-specific symbolic design of most toys was not clearly determined as “most toys were home-made so that dolls would frequently be crudely fashioned lumps of clay or some other material which children felt could stand in for a doll” (Varney 1999: 16). It was only after the Industrial Revolution that self-made toys lost their potential to be created upon to give place to factory-made toys, which became increasingly available. That was the time when toys started to gather ideological concepts and acquire gender identities, marking the beginning of the “girlish pink world” (Varney 1999). From then on, dolls started to be perceived as girl-related toys which represented the social need to encourage the practice of nurturing and motherhood from early age and doll-playing as a prelude to the limited opportunities that girls would have to face thereafter. Within the restricted boundaries of their universe of family-gathering, child-rearing, shopping and household affairs, doll-playing was therefore believed to help acquaint little girls with the roles they would have to follow, according to the socially-dictated rules. As Fleming (1996) has noted, whichever the doll’s image is, the very activity of doll-playing has ever since been conceived of as the opportunity “where girls rehearse the traditional domestic, mothering role” (Fleming 1996: 43). For some writers (e.g. Caldas-Coulthard and Van Leeuwen 2002), even dolls like the grown-up, big-breasted and sexually-appealing Barbie – play a role in reinforcing the discourse of “domesticity” through her represented narrative universe which typically revolves around activities such as going to work, going shopping, etc. Drawing on Butler’s assertion (as mentioned in Caldas-Coulthard and Van Leeuwen 2002) that “gender is not something that we have or are but rather something we perform” (Caldas-Coulthard and Van Leeuwen 2002: 97), one may speculate that the toy industry does not allow for such performativity to take place since

542

Danielle Almeida

the gendered representations attached to toys are clearly pre-determined before children can have access to them. Take, for example, unchangeable features in toys like Barbie’s feet specially designed for high-heeled shoes. Features such as this embed toys with values that may otherwise be interpreted as “given gendered meanings” (Caldas-Coulthard and Van Leeuwen 2002: 97). Given gendered meanings like Barbie’s high-heeled feet corroborate Varney’s (1999) argument that the design of some contemporary toys constructs a visible gap between girls’ and boys’ types of play especially when toys’ kinetic features are used to convey specific gender representations. One of the ways this can also be done is through the way the toys are designed to move, that is, through their kinetic possibilities. While analysing texts available on the web pages of Barbie (http://www. barbie.com) and Action Man (http://www.actionman.com), Caldas-Coulthard and Van Leeuwen (2002) observed that some contemporary toys are designed for “action”, whilst others are designed for “posing” (Caldas-Coulthard and Van Leeuwen 2002: 98). This seems to be the particular case of boys’ dolls, usually located “out there” and endowed with flexibility to move sideways, open their legs, hold objects and stand by themselves without falling (Caldas-Coulthard and Van Leeuwen 2002: 98). In their multimodal semiotic analysis of contemporary war toys, Machin and Van Leeuwen (2009) observed that action figures like Action Man and GI Joe are endowed with a kinetic potential which allows children to arrange their “articulated bodies of action figures in heroic individual poses” (Machin and Van Leeuwen 2009: 54). Contrastively, girls’ dolls are commonly represented living “entirely in [their] own enclosed world” (Fleming 1996: 53) and are kinetically designed to be restricted in their action potentials. Toys’ given gendered meanings can also be seen in pictures found in toy catalogues and boxes, which commonly feature the inability of movement of girls’ dolls like Barbie when she is, for example, located in places related to domestic life. This evokes a sense of submission that clearly contrasts with boys’ dolls like GI Joe and the Action Man, often “placed in specific contexts involving exterior setting, danger and movement” (Caldas-Coulthard and Van Leeuwen 2002: 99). Just as dolls’ kinetic features, the choice of colours constitutes another relevant aspect of gender. Traditionally, the colour pink has commonly been associated to girl-related items whereas blue has been used to refer to the boys’ world. Along the years, these colours have been found stamped in toys’ packaging, advertisements, catalogues and the background of most web pages associated with children’s toys. Such colour distinction leads to the assumption that more than simply colour tones, the meaning that the “pinkish girl world” and “blueish boy world” embed, attach, in fact, values of femininity and masculinity, which are not “explicitly codified” in contemporary popular culture (Caldas-Coulthard and Van Leeuwen 2002: 101). In Barbie’s catalogues, for instance, different shades of pink have been used to denote ideas of childhood (light pink) and teenagehood (darker shades) or even

The political values embedded in a child’s toy

543

sexuality (Caldas-Coulthard and Van Leeuwen 2002). Other shades and colours have also attested to gender distinctions although their meaning potential may have not been so easily interpreted. This has been the case of certain shades of pink, mauve or purple which only “the context, and degree of darkness and intensity will […] narrow this meaning further […] in the direction of mystery, danger, or sexuality” (Caldas-Coulthard and Van Leeuwen 2002: 104). At the textual level, gender representations in these texts have been revealed by the ideational meanings and their concepts of classification and evaluation. In this respect, Caldas-Coulthard and Van Leeuwen, (2002) have pointed out to a high incidence of relational and mental structures, as in the case of Barbie’s texts, in opposition to a profusion of material structures verified in the Action Man’s texts. Also, as already stated, toys for boys have frequently been located in actionoriented settings and named after their super-natural powers (e.g. Ninja), which stresses the naturalised classification of their power relations. Constructed differently, the texts of girls’ dolls frequently locate them within the domestic space and their classifications, instead of containing “super human evaluations”, attach values connected to female roles and professions such as “the shopper, the cook, the mother, the ballerina,” etc, pointing to “romantic, idealised constructions of womanhood” (Caldas-Coulthard and Van Leeuwen 2002: 105). A closer look at these two sets of texts has also revealed that through linguistic devices such as long nominal groups that positively evaluate Barbie and her adorns in terms of their stunning and graceful “aesthetic values” (my italics), Barbie’s ideational representation is characterised by the use of mental and material processes such as is, wears or represents, which portray Barbie more statically, as if she is “not in control of what is happening to her body” (Caldas-Coulthard and Van Leeuwen 2002: 106). Comparatively, although the Action Man texts are also characterised by the positive evaluation of his super skills through epithets such as “greatest, fabulous” and “super-cool,” this is often done in terms of some pre-established social judgement (Caldas-Coulthard and Van Leeuwen 2002). Further, his ideational portrait usually occurs on the basis of his controlling position as the main actor in the material processes identified, such as “leaps, goes into action, opens” and “descends” (Caldas-Coulthard and Van Leeuwen 2002). By analogy, whereas Barbie’s text can be said to belong to a “catwalk genre”, whereby “models are described and adults are interested in attributes and features of the doll” (Caldas-Coulthard and Van Leeuwen 2002: 106), the Action Man texts can otherwise be said to “belong to a TV ad type of genre addressed to a child interested in imaginary play” (Caldas-Coulthard and Van Leeuwen 2002: 106–7). In a similar study, this time about the gender features of toys’ TV advertisements, Kline (1993) noticed that while boys’ advertisements tend to predominantly present boys as emotionally involved in forceful action, girls’ advertisements, on the other hand, tend to portray girls as involved in what he calls an act of “per-

544

Danielle Almeida

formative role enactment” (Kline 1993: 53), whereby instead of being actively engaged in creative make-believe, girls are depicted looking at their dolls in wonder or excitement. For Kline (1993), in such “toy-based dramas” more than simply spectators, boys actively engage in the activities portrayed “playing at being specific personalities rather than taking up social roles” (Kline 1993: 53) whereas girls, contrarily, are shown manipulating their dolls through “simple domestic rituals” (Kline 1993: 53) in which they rehearse their motherhood roles. That said, toys and their representations can be assumed to enact gender distinctions as it has been verified in the way male and female roles are constructed through toys’ “given gendered meanings”, viz., their kinetic features, colour distinctions and textual representations. In that regard, not only do toys represent and reproduce social relations and values but they are also “closely related to many larger cultural patterns in society” (Caldas-Coulthard and Van Leeuwen 2002: 107). In doing so, their representations continue to transmit a “damaging, gendered and unequal world” (Caldas-Coulthard and Van Leeuwen 2002: 107) in which dolls are kinetically limited and unquestionably placed in domestic environments whereas action toys for boys keep on heroically risking their lives to save the planet.

3 “Girl Power” in contemporary media representations For Kress and Van Leeuwen (1996), the discussion of the meanings of a given visual representation – media advertisements, for example – goes beyond the mere description of what linguists regard as the “lexicon” of images to offer a more systematic way of looking at visual structures through a set of formal rules that despite limiting “the relative freedom hitherto enjoyed by other visual analysts” (Kress and Van Leeuwen 1996: 3), turns their framework into a powerful linguistic tool capable of fully exploring images’ contextual dimension as well as drawing on their relationship with the grammar of language. Drawing on a terminology which may sound complex at first sight (Jewitt and Oyama 2001: 154), Kress and Van Leeuwen (1996) attempted to provide a more systematic exploratory method for pictorial analysis which added to the visual semiotics of Roland Barthes (1957) and to the field of iconography by proposing visual syntactic patterns as analytical categories. The work of Kress and Van Leeuwen (1996) draws on three “metafunctions,” adapted from Halliday’s functional grammar, namely representational (“ideational”); interactive (“interpersonal”); and compositional (“textual”). It assumes that, just like the verbal language, the visual language is endowed with a kind of syntax, in which elements are organized into visual structures to communicate a

The political values embedded in a child’s toy

545

coherent whole. These structures can include people, places or inanimate objects as represented participants, and be organized into different levels of complexity. Just like Halliday’s Systemic Functional Grammar (1994), Kress and Van Leeuwen’s visual perspective assumes that the analysis of the semiotic choices that realise a visual structure are directly related to its cultural and situational context, as it relies on contextual references to account for the discussion in a macroanalytical level. These analytical tools offered by Kress and Van Leeuwen’s framework (1996) will be presented, explained and applied in this order in sections as we move along the forthcoming analysis.

3.1 Representational meanings In this first section we are interested in the “representational” metafunction of visual elements. Kress and Van Leeuwen (1996) layout a number of observations or tools that we can systematically apply to compositions. These allow us to begin to draw out features that may not be immediately observable to the casual observer. This function conveys the relationship among the participants depicted in a visual structure through their “vectorial relations,” which correspond to the category of action verbs – or “processes” – in language (Kress and Van Leeuwen 1996). They way “participants” are connected in a pictorial structuring may be described as “narrative” or “conceptual” processes. The participants in a “narrative process” can be referred to as (1) “Actor/Reactor” and “Goal/Phenomenon”, (2) “Sayer” and “Utterance” or as (3) “Senser” and “Phenomenon”. If the image contains both an “Actor” and a “Goal” connected by means of a vector – or an “action line” – this image is said to be “transactional”, which means that it portrays “an action taking place between two parties” (Jewitt and Oyama 2001: 143). The opposite is also true: an image is said to be “non-transactional”, whenever the action only involves the “Actor” and “there is no apparent participant to whom the action is directed, [hence] there is no Goal”. Likewise, whenever the action performed by the participant in the image involves his/her “looking” at something, that is, whenever it involves a “vector” “formed from the eye line of one or more participants” (Unsworth 2001: 76), this process is said to be a “reaction” rather than an “action” and the thing s/he is looking at is referred to as a “Phenomenon” rather than a “Goal”. If in the image there is only the participant looking at something but no apparent “Phenomenon”, this image must be described as a “non-transactional” reaction. “Narrative” representations are also characterised by their “circumstances”, that is, the settings in which the participants are placed and their adjuncts, “such as the artefacts, tools and minor figures that complete the meanings to be found in an image” (Lewis 2001: 148). “Conceptual processes”, on the other hand, are visually characterised by the absence of vectors as they commonly “define, analyse or classify people, places

546

Danielle Almeida

and things” (Jewitt and Oyama 2001: 142), including abstract ones. “Conceptual processes” can be of three main types: “Classificational, Symbolic and Analytical”. “Classificational” visual structures arrange people, places or things together in a given visual space in order to show that they are somehow similar, that is, they belong to the same class, order or category. In “classificational” images, participants are represented in a sort of “tree structure” (Kress and Van Leeuwen 1996: 81), in which one or more superordinate, top-level participant is related to a number of other subordinate, lower-level participant in the hierarchy of the taxonomy. Whenever the superordinate participant is suppressed and only the subordinate one is represented, the taxonomy is said to be “covert” (Kress and Van Leeuwen 1996; Lewis 2001; Unsworth 2001). If, contrarily, the taxonomic display includes the superordinate participant in an explicit manner, this taxonomy is referred to as “overt” (Kress and Van Leeuwen 1996). “Symbolic” structures help to establish the participant’s identity in a visual representation by means of attributes made conspicuous through elements such as size, choice of colours, positioning, use of lighting, etc, thus constituting what Kress and Van Leeuwen (1996) regard as a carrier-attribute type of relation. “Analytical” structures portray participants in terms of a part-whole relation wherein the whole stands for ‘the carrier’ and the parts for its possessive attributes (Kress and Van Leeuwen 1996). When analysed in their “representational” semiotic meanings, Susi’s ads seem to generally fit into a “conceptual” type of “process” of visual structuring, wherein Susi is presented in rather static way, as an object to be defined, classified and even assessed by the viewer. Nevertheless, when looked at in a more careful, systematic basis, each of the ads presents itself as a rather complex source of information about the way Susi is visually depicted. One of the ways such complexity is created derives from what Kress and Van Leeuwen (1996) have addressed as “Circumstance of Setting” of the narrative visual structure. It accounts for the relationship established between the

Fig. 1: Susi Moda Praia.

The political values embedded in a child’s toy

547

represented participants (RPs) and the “setting” – or “background” – of the pictorial framework, which, in pictures like Susi’s advertisements can anchor the images and provide the viewer with a further comprehension of the nature of the activities performed by the depicted doll. This figure has been previously shown, which pictures Susi laid in a hammock against a tropical and sunny background, in a predominantly narrative visual structure characterized by its dynamic vectors: According to Kress and Van Leeuwen (1996), the required contrast between foreground and background of the image can be realised in four distinct ways: 1. the participants in the foreground overlap, and hence partially obscure the Setting; 2. the Setting is drawn or painted in less detail (or, in the case of the photography, has softer focus); 3. the Setting is more muted and desaturated in colour, with various colours tending towards the same hue, for instance the blue of distance; 4. the Setting is darker than the foreground, or lighter, so that it acquires an “overexposed” ethereal look (Kress and Van Leeuwen 1996: 71, my italics) The absence of essentially dynamic vectors and the de-contextualised background in which Beto and the three versions of Susi are placed point to a sort of passive portrait – or conceptual process – in which these toys assume the role not of actors or agents of the visual structuring but of “carriers” of the “possessive attributes” which are pictorially assigned to them by virtue of their outfit and their physical appearance.

Fig. 2: Estrela’s Beto Estilo.

Fig. 3: Estrela’s Cartela Susi Visual da Moda.

On the other hand, some ads are presented to the viewer as complex images in that they are embedded with a few narrative processes instantiated by the role that the background plays and the dynamic vectors which point to the sort of activities which Susi pictorially performs. This includes:

548

Danielle Almeida

1. sports:

Fig. 4

Fig. 5

Dynamic vectors portraying Susi’s sport activities.

2. leisure:

Fig. 6 Background portraying Susi’s leisure activities.

Fig. 7

The political values embedded in a child’s toy

549

3. professional activities:

Fig. 8

Fig. 9

Background portraying Susi ’s professional activities.

Notice that images like the following one can pose rather complex relationships regarding its visual structuring in that the plain, decontextualised background in which Susi is placed initially points to a conceptual type of visual structuring. Yet this is contradicted by the dynamic vector of her walking and the sense of movement that her sporty clothes convey, these being typical of a narrative image:

Fig. 10: Complex relations created by background.

The same applies to the figures whereby Susi is depicted swimming and rollerskating. The way Susi’s background – or her “Circumstance of Setting” – is pictorially constructed evokes in itself a sense of movement which signals to her narrative portrait:

550

Fig. 11

Danielle Almeida

Fig. 12

Complexity created by Circumstance of setting.

Constructed differently, some picture Susi against highly contextualised backgrounds such as a sunny beach, an airport, a recreation area or a ballroom all of which serve to add more credibility to her naturalistic depiction:

Fig. 13

Fig. 14

Fig. 15

Credibility through background: naturalistic representation.

In images like the ones below, Susi is presented within four different contexts – winter, summer, spring or autumn – whichever one she is expected to be appropriately dressed. As it can be seen below, the colour schemes of such advertisement produce some affective meanings linked to the social signification of each season of the year: whereas winter and autumn are enacted by darker tones which predominate – even in Susi’s outfit – such as a tone of bluish purple used in the first advertisement, summer and spring are represented by a hot mix of yellow and

The political values embedded in a child’s toy

551

Fig. 16: Different contexts.

pinkish tones which contribute to attach the idea of femininity related to these two seasons of the year. In the same vein, Figure 14 adopts more conventional, naturalistic colours to get closer to reality, as an attempt to come as close as possible to the real representation of an air hostess who walks along the aircraft path. Insofar as their degree of realism is concerned, Kress and Van Leeuwen (1996) have pointed out that girls’ toys like dolls, for example, are represented in far more naturalistic detail than are boys’ toys like the popular Playmobil figures or the “make-believe world of science-fiction vehicles and weapons, or [the] world of dinosaurs and other monsters” (Kress and Van Leeuwen 1996: 257). Conversely, girls’ toys are typically represented by “highly naturalistic versions of adult objects” (Kress and Van Leeuwen 1996: 256), like “realistic washing machines, beauty shops, vacuum cleaners, dolls’ cots” (Kress and Van Leeuwen 1996: 257). This, as both authors contend, is believed not only to emulate the adult universe but also to anticipate girls’ roles as both mothers and housewives (Kress and Van Leeuwen 1996: 257) Variations in the degree of realism of toys can also be verified according to the child’s age range. According to Kress and Van Leeuwen (1996), the younger the child, the more abstract and essentialised the representation of his/her toys. Just like the old Playmobil figures, whose eyes are generally represented by “two dots, the mouth [by] a curved line, the hair [by] an almost featureless helmet with a few indentations suggesting texture and the length signifying gender” (Kress and Van Leeuwen 1996: 257), young children’s toys are marked by featureless shapes and textures. However, as the child grows older, the level of specificity and details of his/her toys increases, as in the case of the Brazilian doll Susi, who has reached such a reasonable level of realism from the point of view of representational detail, that she is presented to the viewer under various models, different outfits, “with blushes on cheeks, shadows under eyes [and] a shine on the lips” (Kress and Van Leeuwen 1996: 258), traces which surely resemble a real teenage woman. This can be observed in the Figure displayed next, which shows Susi Baile de Formatura in magnified version so that its high level of representational detail becomes more visible:

552

Danielle Almeida

Fig. 17: Susi Baile de Formatura. High level of representational detail.

In Susi’s advertised images, the level of representational detail is conveyed not only by means of her naturalistic appearance but also by the way her activities are constructed through vectors which communicate her process of interaction with both her “Circumstance of Setting” and with other represented participants. Take these ads, for example:

Fig. 18: Pool setting.

Fig. 19: Prom setting.

In these advertisements, Susi is shown as an interactive participant connected to a second interactive one by means of a vector which expresses the dynamic relationship going on between them. In Figure 18, for example, Beto is presented as the main participant of the visual structuring, the “Actor” from whom the vector emanates towards Susi, the “Goal”, who, in turn, is looking away. In this particular case, both participants are involved in a “non-transactional action”, in which the vector does not convey a “reaction” from whom the action is done to. Here Susi assumes a more passive role in not having responded to Beto’s gaze probably because she had not yet taken notice of his arrival.

The political values embedded in a child’s toy

553

Studies of narrative processes and the nature of women and men’s participation in images have indicated that women are indeed chiefly presented as more passive and more involved in non-transactional actions than men, who are often presented as involved in transactional actions which portray them as seizing the initiative in, for example, starting sexual activities (Jewitt and Oyama: 2001: 143). In a similar vein, in her account on toy advertisements from the 1950’s and 1960’s, Seiter (1993) also draws attention to the passive representation of girls in these images, with the girl often “watching” while the boy plays or with “mother and daughter often [taking] the place of observers of the action, spectators to the male bonding of father and son at play” (Seiter 1993: 81). On this respect, Seiter (1993) claims that the inclusion of girls as active participants in images like toy advertisements is virtually irreconcilable with “traditional notions of maleness” (Seiter 1993: 93) albeit acknowledges a few incorporations of girls’ active appearances while featured alone in contemporary toy advertisements. This is particularly the case of Susi’s advertisements, which do include a number of images wherein Susi is portrayed as actively involved in varied activities, which, in a sense, serves to demystify the traditional passive role ascribed to women in representations such as toy advertisements. As narrative images embedded within “classificational” structures, pictures such as Figure 16 relates the participant(s) through a covert “taxonomy” which communicates to the viewer how, for example, Susi’s four identities promoted by her outfit possibilities are realised by being of the same kind (Kress and Van Leeuwen 1996: 83). The same happens to the instances of Susi’s graduate ball shown in Figure 20, whereby the viewer is presented with interrelated moments of what has been verbally described as “one of the happiest days of her life”.

Fig. 20: Moments of life.

554

Danielle Almeida

3.2 Interactive meanings The second dimension of analysis related to the “interpersonal” level of visual structures, aspects such gaze (or “contact”), “distance” (or “size of frame”), “perspective” (or “point of view”) and “modality” (or “reality value”) play a major role in creating a particular relation between viewers and the image within the pictorial frame. Whenever a represented participant in a visual semiotic structure is presented looking directly at the viewers’ eyes, s/he brings about an imaginary, pseudo relation with them. This type of gaze has been described as “demand”. Alternatively, if the represented participant is depicted not looking at the viewer’s eyes and offered as the subject of viewer’s look and object of his/her scrutiny, this image is said to be of an “offer” kind, for it impersonally “offers” the represented participant (s) to the invisible onlooker’s contemplation and/or inspection (Kress and Van Leeuwen 1996: 124). The “size of frame or distance” of a shot also alludes to different relations between represented participants and viewers. Depending on the distance between them, a different degree of intimacy is established, therefore suggesting a more intimate personal relationship or a more impersonal one. When participants are pictured from a close-up, every detail of their appearance is captured, including their facial expression, which helps to reveal traces of their personality and makes us feel more intimately acquainted with them. This does not happen when we see participants being pictured from a “long shot” which, reversibly, contributes to their portrait in a more unattached way, as if they were “types rather than individuals” (Jewitt and Oyama 2001: 146). The intermediate degree between a “close-up shot” and a “long” one, that is, between “the most intimate relations and the total absence of [one]” (Jewitt and Oyama 2001: 146) can be found in the “medium shot” whereby participants are cut off somewhere between the waist and the knees indicating that the relationship between the depicted participant (s) and viewer (s) is of a social kind. The concept of “perspective” or “point of view” applied to the analysis of a visual semiotic structure denotes the audience’s ‘subjective attitudes’ towards the represented participant (s), which is instantiated by means of “frontal, oblique” and “vertical angles”. The use of a “frontal angle”, for example, has been associated with an attitude of involvement whereby the viewer is invited to become part of the world depicted in the image. On the other hand, a sense of detachment is conveyed by the use of an “oblique angle” whereby the participant is presented in profile, thus implying that what the viewer sees is not part of his/her world (Kress and Van Leeuwen 1996: 143). The “vertical angle” and its variants (e.g. “high”, “low” or “eye-level”) signal power in different levels and relations. For example, whenever a represented participant is pictured from a “high angle”, s/he is seen from the perspective of the viewer’s power. Conversely, if the represented participant is shown from a “low

The political values embedded in a child’s toy

555

angle”, s/he is said to have power over the viewer. Finally, if the image is at the “eye-level”, it encodes a relation of equality in which the level of power of each part is equivalent. The term “modality” applied to the conceiving of a visual semiotic medium is linked to the notion of “reliability of message” (Caldas-Coulthard and Van Leeuwen 2002) or as Jewitt and Oyama (2001) prefer to put it, to its “reality value”, concepts used to talk about how truthfully a given image is represented. Broadly speaking, the ways to modulate reality in visual representations can be based on different truth criteria, viz., “naturalistic” (or real) or “sensory” (or fantastic) “modality”. The “naturalistic modality” is defined on the basis of how much congruence there is between the object of an image and the one you can see with the naked eye (Kress and Van Leeuwen 1996). Put simply, the greater the correspondence, the higher the modality of an image. As interpersonal relations can arouse from the establishment of interaction between the represented participants of a given image and/or between the represented participant (s) and the viewer of the image, one can say that this occurs in the ads under analysis in a number of ways. Firstly, by portraying Susi and Beto holding hands and dancing while their gaze is directed to the camera, the viewer is somehow “demanded” to establish an imaginary relationship of social affinity with them, as if invited to become part of the moment they are experiencing:

Fig. 21: Establishing a relationship.

This type of gaze categorised as “demand” and actualised in pictorial genres such as doll advertisements, foregrounds persuasion at the same time it backgrounds instruction and exposition, as Kress and Van Leeuwen (1996) assert. The sense of connection invoked by the demand picture is equally instantiated in the images whereby Susi addresses the viewer directly, as if explicitly demanding to be scrutinised.

556

Danielle Almeida

As said, the meaning expressed by a “demand” picture reinforces the bond between the viewer and the participant(s) depicted, although such an empathetic relation is not utterly fostered in that the doll is frequently depicted in a slightly “oblique” manner, which foregrounds a sense of detachment from the part of the represented participant. Whenever the participant is presented in profile, the viewer tends to conceive of him/her as not belonging to their world (Kress and Van Leeuwen 1996: 143). In further exploring the question of involvement of the audience with the depicted dolls, it can also be noticed that both Susi and Beto are predominantly shown from a “long shot” whereby their full body and the surrounding space can be observed. This is believed to entail an unattached view of them in outline as if they were “types to be analytically examined rather than individuals with their own specificity” (Jewitt and Oyama 2001: 148). Pictorially speaking, such a far social distance as Kress and Van Leeuwen (1996) put it, has a more formal and impersonal character, inasmuch as it allows the viewer to carefully scrutinise the depicted participant as if saying “stand away so that I can look at you” (Kress and Van Leeuwen 1996: 131).

Fig. 22

Fig. 23

Scrutiny through far social distance.

Identification is therefore fostered through the depiction of the dolls as whole figures and this can also be noticed through an examination on the way the viewer is aligned in perspective with the represented participant(s). The concept of perspective is linked to the type of subjective attitude (s) established between the represented participant (RP) and the viewer (Kress and Van Leeuwen 1996). In that respect, the use of a “frontal angle” is believed to produce an attitude of involvement from the part of the viewer, who is invited to become part of the world depicted in the images.

The political values embedded in a child’s toy

557

Also, by virtue of an “eye-level” gaze, a relationship of equality between Susi and the viewer is encoded, to the point that there is no power difference between them (Kress and Van Leeuwen 1996: 146). The use of this type of angle is particularly effective in advertisements such as toys’, for they help to invoke an attitude (s) of friendship and empathy with the viewer established through power equality, thus fostering consumption, the ultimate goal of advertising. Insofar as the reality value – or the degree of “modality” – of Susi’s advertisements is concerned, it can be said that the choice for differentiated tones on the bare, “de-contextualised” setting in most of the ads increases the “sensory modality” of the image by adding a “more-than-real” effect to the product being advertised, thus rendering it the impression of a “fantasy” or the “promise of bliss” (Kress and Van Leeuwen 1996). Indeed, the placement of represented participants against a void, unmodulated background assigns them the value of a more generic type, the status of a “typical example” (Kress and Van Leeuwen 1996: 166). By nature, naturalistic images are characterised by a contrast range of brightness values, “often in ways which have no explanation in terms of the logic of illumination” (Kress and Van Leeuwen 1996: 167). Here, again, the variation from a darker tone of pink to a lighter one, mixed with the brightness created by the yellowish lit areas and the dolls’ shadows brings about an ethereal sensation triggered by the dolls’ hyperreal appeal caused by a high modality value. Also, props such as the use of lighter tones in these advertisements suffice to lower the sensory modality of the image to the extent of giving them a sense of “photographic naturalism” (Kress and Van Leeuwen 1996: 166).

3.3 Compositional meanings The third visual metafunction encapsulates three main resources to convey the “compositional” meanings of visual structures, viz., “information value”, “framing” and “salience”. The “information value” of an image is mostly given through the placement of the elements of a composition. Depending on whether an element is placed within the three dichotomies of pictorial zones (Kress and Van Leeuwen 1996) (“left/right; top/bottom; centre/margin”), it is believed to endow the picture with a specific idea. In the words of Kress and Van Leeuwen (1996), “the right seems to be the side of the key information, of what the reader may pay particular attention to, of the message […]” whereas “the left is the side of the already given, something the reader is assumed to know already, as part of [his/her] culture” (Kress and Van Leeuwen 1996: 186–187). The information value of “top” and “bottom” placements also carries distinct features in that the upper section of a visual composition is believed to contain

558

Danielle Almeida

what is presented as “Ideal” whereas the lower section what is “Real”. In this respect, an “Ideal” element epitomises “the promise of a product”, the “idealised or generalised essence of information” (Kress and Van Leeuwen 1996: 193) therefore being shown as the most prominent part of the visual composition, one that calls for an emotive appeal by showing us “what might be” (Kress and Van Leeuwen 1996: 193). In opposition, a “Real” element tends to be presented to the viewer in a less ostensible way, including more practically-oriented information of “what a product is”: the details on how to obtain it, where to order it and/or how to request further information about it (Kress and Van Leeuwen 1996). Finally, the “central” and “marginal” spatial arrangements of elements of a given visual composition imply preponderance, as in the case of the “central” element or subservience to the nucleus of information, as in the case of the ancillary, “marginal” items. In visual terms, “connection” can be achieved through similar colours, forms, connective vectors, overlapping or superimposition of images, that is, through the absence of “framing” lines around the elements of an image, thus stressing their sense of group identity. “Disconnection”, on the other hand, can be created by the presence of “framing”, through contrasts of colours and forms and white space between the elements which visually signify individuality and differentiation (Kress and Van Leeuwen 1996: 215). Whenever the elements within the layout of an image are “disconnected” or marked off from each other, they are said to be “strongly framed” and therefore are presented to the viewer as a separate piece of information. If, however, these elements are presented to the viewer in a “connected,” joined together manner, they are described as “weakly framed”. Lastly, the compositional component of “salience” stands for the way some elements in a visual composition are arranged in order to look more eye-catching than others. Aspects such as the placement of an element in the foreground or background of a picture, their size, their colour contrasts and sharpness can reinforce or decrease its level of conspicuousness by creating a “hierarchy of importance among the elements, selecting some as more important, more worthy of attention than others” (Kress and Van Leeuwen 1996: 212). Compositionally, the visual configurations of Susi’s advertisements are organised around the placement of elements within the pictorial framework according to the following patterns: (1) top/bottom, left/right and centre/margin structures; (2) salience; and (3) framing. As aforementioned, Susi is mainly placed in shaper focus in the foreground of the image, mostly in a central position which posits her preponderant role in the image over other ancillary elements. In all four pictures displayed next, Susi is shown as the most important element within the images by placing her in central position. Differently, Figure 25 pictures Susi at the bottom, right-hand corner of the image, in a marginal pictorial zone which implies her subordinate, secondary posi-

The political values embedded in a child’s toy

559

Fig. 24: Susi foregrounded.

tion in relation to Beto, who is placed on top of the right-hand corner of the referring visual composition. Insofar as the “informational value” of such dichotomies of pictorial zones are concerned, Figure 25 is therefore believed to portray both Susi and Beto as the key information “at issue” within the image, the message to which the viewer must pay particular attention:

Fig. 25: Placing.

Likewise, the depiction of Susi in her swimming costume on the left of the picture implies that this is information with which the viewer is already acquainted, something s/he is assumed to know already. Nonetheless, her depiction at the bottom, right hand corner of the image conveys the “real” information of what this specific model of Susi can do, that is, she can get wet and perform her swimming abilities. The value of the colour schemes used in the advertisements under analysis also serves to question if the elements within the images are meant to be connected together as “one unit of information” (Kress and Van Leeuwen 1996: 215) or if, contrarily, their absence of connection – that is, of “framing” – denotes the individual character of each pictorial item (Kress and Van Leeuwen 1996: 215). On this matter, the strong framing formed by the repetitive use of bright, vibrant colours

560

Danielle Almeida

Fig. 26: Information provided.

on the background of many advertisements not only serves to catch the viewer’s attention to the most salient element(s) of the picture – Susi and Beto dolls – but also functions as a disconnective device that clearly demarcates the outlines of the dolls thus enhancing their “salience” and underpinning their individuality. An example of this is Figure 27, in which Susi’s presence is stressed at the foreground of the image by being placed against a bright green backdrop that contrasts with the vivid yellow and deep blue hues of her outfit.

Fig. 27: Contrasting background.

Fig. 28: Background colours.

Similarly, the next Figure makes Beto’s figure even more conspicuous through the use of strong, vibrant colours such as deep purple, dark pink and bright green at the background of the image and through the highlighting contours around Beto’s body. The subsequent section summarises the most relevant findings concerning the visual analysis of Susi’s advertisements.

The political values embedded in a child’s toy

561

4 Conclusions As demonstrated, there has been a contemporary tendency in visual advertised representations towards the depiction of women in passive, objectified positions, thus reinforcing their image as pretty fixtures, repositories of aesthetic attributes. The representation of women in doll advertisements draws predominantly on passive roles which identify them on the basis of the accoutrements that compose their outfit or the aesthetic features that make up their looks. The few narrative visual processes verified in the investigated advertisements and the prevailing number of “conceptual”, “classificational”, or “analytical” visual processes identified in both sets of doll advertisements have proved to this effect. On the other hand, considering the negotiation of gender roles throughout the decades, the traditional representation of women as “home”-related items seems to be not the only possibility conveyed by doll advertisements. Other “career” and “event”-related representations – such as Susi Aeromoça, which portrays Susi as an air hostess leaving an aircraft, Susi Seleção by Milene Domingues, which depicts Susi wearing a football uniform and Susi Baile de Formatura, which shows her taking part in her graduation ball – have come to the forefront suggesting that mass media is well aware of contemporary women’s active engagement in society. Indeed, there has been a verified revamp on the image of “blissful domesticity” (Gauntlett 2002) propagated by former doll advertisements which used to portray little girls playing the role of simpering young housewives within the domestic context of their kitchen or living room. Although such representations continue to be promulgated nowadays – maybe as a reflection that this is a role with which women still have to cope – they are not as expressive as they used to be in the past. Instead, the concept of traditional femininity has been revised and replaced by another concept which, in contemporary times, has been associated to female identity – the one of “girl power” (Gauntlett 2002). The notion, which embodies concepts like assertiveness, independence and autonomy, has been clearly incorporated into dolls’ representations and associated with the huge success these miniaturized representations of emancipated women have been reaching in the toy market. Concurring with that, Moorhead (mentioned in Gauntlett 2002), has called attention to the fact that the sales of dolls like Barbie “are reported to be falling because only youngest girls will accept such a “girly” toy nowadays, whilst young women have a range of assertive “girl power” role models to choose from in magazines, movies, and pop music” (Gauntlett 2002: 10). Alongside with the celebration of the independence women have been conquering throughout the years, also runs their fear of becoming victims of an objectified representation that over-stresses their aesthetic appearance. In the case of the analysed doll advertisements, Susi’s focus on fashion has been blended with career and leisure-related activities like “flying around the world” and “going roller-skating”.

562

Danielle Almeida

On the whole, what the analysed advertisements seemed to be portraying through their visual representations reflect the multiple, cumulative social roles that compose the agenda of contemporary women and which girls need to be familiar with from early age as a prelude of what they will face later in life. Some of these roles involve their “juggling” in old domestic activities like taking care of the house, caring for the children, caring for themselves, caring for their husbands and their assimilation of contemporary trends, fads and technologies that the new millennium awaits. Acknowledgments: I would like to thank Brazilian toy manufacturer Estrela for understanding the scientific nature of the present study and allowing me to reproduce the advertised toy images used in this article.

References Barthes, Roland. 1957. Mitologias. Paris: Edições 70. Caldas-Coulthard, Carmen Rosa and Theo van Leeuwen. 2002. Stunning, Shimmering, Iridescent: Toys as the Representation of Gendered Social Actors. In Lia Litosseliti and Jane Sunderland (eds.), Gender Identity and Discourse Analysis, 91–108. Amsterdam: John Benjamins. Caldas-Coulthard, Carmen Rosa and Theo van Leeuwen. 2001. Baby’s First Toys and the Discursive Constructions of Babyhood. Folia Linguistica 35: 1–2. Downie, Andrew. 2010. Brazilian girls turn to a girl more like them. Women Making History Today. Available at: http://www.csmonitor.com/atcsmonitor/specials/women/mirror/ mirror012000.html. Fleming, Dan. 1996. Powerplay: toys as popular culture. Manchester: Manchester. University Press. Gauntlett, David. 2002. Media, Gender and Identity. London: Routledge. Halliday, Michael and Alexander Kirkwood. 1994. An Introduction to Functional Grammar. London: Edward Arnold. Jewitt, Carey and Rumiko Oyama. 2001. Visual Meaning: a Social Semiotic Approach. In Theo van Leeuwen, and Carey Jewitt (eds.), Handbook of Visual Analysis, 134–156. London: Sage. Kline, Stephen. 1993. Out of the Garden: Toys and Children’s Culture in the Age of Marketing. London: Verso Press. Kress, Gunther. 1997. Visual and verbal modes of representation in electronically mediated communication: the potentials of new forms of text. In Ilana Snyder (ed.), Page to Screen: taking literacy into the electronic era, 53–79. Sydney: Allen and Unwin. Kress, Gunther and Theo van Leeuwen. 1996. Reading images: The Grammar of Visual Design. London: Routledge. Lewis, David. 2001. Reading Contemporary Picture Books. In Gunther Kress and Theo van Leeuwen (eds.), Grammar of Visual Design, 145–167. London: Falmer. Machin, David and Theo van Leeuwen. 2009. Toys as Discourse: Children’s war toys and the war on terror. Critical Discourse Studies 6(1): 51–64. Peers, Juliette. 2004. The Fashion Doll: from Bébé Jumeau to Barbie. Oxford: Berg. Seiter, Ellen. 1993. Sold Separately: Children and Parents in Consumer Culture. New Brunswick: Rutgers University Press.

The political values embedded in a child’s toy

563

Snyder, Ilana. 1997. Page to Screen: taking literacy into the electronic era. Sydney: Allen and Unwin. Unsworth, Len. 2001. Describing Visual Literacies. In Len Unsworth (ed.), Teaching Multiliteracies across the Curriculum – Changing contexts of texts and image in classroom practice, 71–112. Philadelphia: Open University Press. Van Leeuwen, Theo and Carey Jewitt. 2001. Handbook of Visual Analysis. London: Sage. Varney, Wendy. 1999. Toys, play and participation. In Brian Martin (ed.), Technology and Public participation, 15–36. Wollongong: Science and Technology Studies University of Wollogong. Varney, Wendy. 2002. Love in Toytown. M/C: A Journal of Media and Culture 5(6). Available at: http://www.media-culture.org.au/0211/loveintotytown.html. Wright, Lenore. 2003. The Wonder of Barbie: Popular culture and the making of female Identity. Essays in Philosophy – A Biannual Journal 4(1).

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

25 The role of images in social media analytics: A multimodal digital humanities approach Abstract: The analysis of social media data is an emerging research field that aims to study the dynamics of urban life. In this study, we adopt a multimodal digital humanities approach to combine the analysis of text-based social media data with visual social media data in an interactive map to investigate urban life in Singapore from a social semiotic perspective. Twitter is used as a source of user-defined localised textual data, Instagram as a source of localised user-generated images and Foursquare as a source of user-defined location-based information where is semantically organised according to Wikipedia’s classification tree. In this way, we track the multimodal content of social media according to semantically organised location-based sources. This study suggests that users of Twitter express emotion about their own lives and the world around them, but these linguistic resources are differentially deployed according to venue. However, there is less variation in the use of photos to construe personal relationships, suggesting that photos fulfill an intrinsic need to be observed which transcends the nature of the social practice which is taking place. It is envisaged that the role of the visual will continue to expand as digital technologies refashion and transform our semiotic world.

1 Introduction Every time we look at a photograph, we are aware, however slightly, of the photographer selecting that sight from an infinity of other possible sights (Berger 1972: 10).

The analysis of social media data to explore the dynamics of urban life is an emerging research field (e.g. Bawa-Cavia 2011; Braga et al. 2012; Cranshaw et al. 2012). For example, studies of messages posted on Twitter1, the online social networking service which permits users to send short text messages up to 140 characters long, have increased over the past few years, as the “texts are as timely as a pulse beat and, taken together, automatically compile the raw material of social history” (Hotz, 1 October 2011). In this study, we adopt a “multimodal digital humanities” approach (Svensson 2010; Podlasov and O’Halloran 2013) to combine the analysis of text-based social media data with visual social media data in an interactive map, in this case to investigate urban life in Singapore. As we demonstrate in this exploratory study, a multimodal digital humanities approach, where vast quantities of different social media data can be amassed, clustered, analysed and converted into interactive visualisations, provides rich resources for mapping

566

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

socio-cultural trends (e.g. see information visualisations in Lima 2011). Importantly, the multimodal digital humanities approach allows us to explore the role of images, both in social media and for social media analysis, collectively referred to as “social media analytics”. The approach developed in this study involves the linguistic analysis of Twitter texts and the visual analysis of photographs posted on Instagram2, the online photo-sharing and social networking service that enables users to take photographs, apply filters and post the resulting images on social networking sites. The results of the linguistic and visual analyses are displayed as an interactive map which is organised according to the venues where the text messages and photos are posted, as recorded by Foursquare3, the online location-based social networking service which permits users to post venues localised using Global Positioning System (GPS) in their mobile devices or network locations. The Foursquare venues are categorised using Wikipedia4 classification tree to provide semantic descriptions of the various types of locations. In this way, we develop a methodology to investigate what people are talking about and looking at using mobile social media technologies and we create interactive visualisations to map the resulting patterns and trends in different urban spaces in Singapore. Significantly, our use of socially-created content from Wikipedia to provide semantic descriptions of the Foursquare venues overcomes the limitations of traditional categorical systems (e.g. library systems, encyclopedias and dictionaries) which lack the necessary information about contemporary social activities. For example, visual comparisons of the major categories in Wikipedia and the Universal Decimal Classification (UDC) show that environment, culture, music, arts and entertainment form a substantial component of Wikipedia’s classification system, unlike UDC’s classifications which are dominated by the applied sciences (Salah et al. 2011; Salah et al. 2012)5. As we demonstrate, socially constructed knowledge systems like Wikipedia (versus “designed” knowledge systems like UDC) provide valuable semantic classifications for modeling and analysing social-cultural patterns and trends. In this study, language and images are conceptualised as social semiotic resources for making meaning (e.g. Halliday 1978; Kress and Van Leeuwen 2006; O’Toole 2011), both for users of social media and also for researchers interested in social behaviour. To illustrate the approach developed in the study, we focus on interpersonal meaning – that is, emotion and personal relations – to investigate the social aspect of the postings, as opposed to the experiential content. For this purpose, we use sentiment analysis algorithms (e.g. Wright 2009) to analyse the emotional content in the Twitter texts and computer vision techniques for face detection (e.g. Viola and Jones 2001) to analyse personal relationships in Instagram photos for postings made in Singapore over a six-week period from 22 August to 4 October 2012. The results are displayed as an interactive map according to venue type and geographical location in Singapore. In this way, we explore how

The role of images in social media analytics

567

emotion and personal relationships are related to social (and professional) activities in Singapore. In doing so, we show how algorithms and visualisation techniques applied to large-scale user-generated multimodal social media data have the potential to contribute valuable knowledge about the patterns and trends of contemporary social life, including the roles of linguistic and visual communication in social media networks. In what follows, we discuss the social media data sources used in the study, the methodology and algorithms which are employed, and the findings. We also discuss the limitations of this exploratory study and possible paths forward with regards to multimodal digital humanities research. Also, as we demonstrate in this study, both the role and scope of visual communication is increasing in our current age of mobile digital media technology.

2 Social data sources The rapidly expanding popularity of social networking services provides the scientific community with extensive sources of user-generated data. This data is not only limited to natural text, which is challenging to analyse, but also provides more accessible data like images, geographical co-ordinates and categorical descriptions of the data records. In the current study, we develop a technique to bring together multimodal sources of social data in order to investigate how people use linguistic and visual modes of meaning to communicate emotion and construct personal relations in different urban areas in Singapore. In order to investigate the potential of multimodal user-generated content, we use three different social networks, each with a different kind of media as their main focus of interest. That is, we use Twitter as a source of user-defined localised textual data, Instagram as a source of localised user-generated images and Foursquare as a source of user-defined location-based information. Figure 1a presents a geographic mapping of the collected social media data in the study, where each coloured point represents a data record that is referenced to a geographic location. The Twitter messages are coded in blue (i.e. the most dense area in the map), Instagram photos in yellow (i.e. the lighter areas in the most dense area in the map), and Foursquare venues are coded in pink (i.e. the points are spread across the entire map, including small islands and ships in the ocean). Figure 1a reveals that social media text messages and photos are most commonly posted in the main island of Singapore where the vast majority of people live and work. In order to cope with non-uniform density of data samples evident in Figure 1a, we define a uniform density grid over the geographical map of Singapore. The map consists of variable size cells calculated in such a way that each grid cell contains 500 or less data records. Therefore, regions with low density of data points are covered with larger size cells, and regions with high density are covered

568

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

a: Geographical Location of Social Media Data: Foursquare (pink), Twitter (blue) and Instagram (yellow).

b: Constant Density Grid. Fig. 1: Geographical Location of Social Media and Grid System.

The role of images in social media analytics

569

with smaller size cells. The grid used in the current study is illustrated in Figure 1b, where smaller cells denote higher data density regions, suggesting greater activity in comparison to their larger counterparts. For example, the grid shows locations of high activity areas in Singapore such as (A) Central Area (e.g. central business district, shopping outlets, hotels etc); (B) Pasir Ris residential area; (C) Chua Chu Kang residential area; (D) Jurong, which is a mixed-use area (e.g. industrial, residential and tourist areas); and (E) Serangoon residential area. The mapping in Figure 1b is intuitive to the perception of space in context of urban and rural constructs where the small, dense and busy urban spaces listed here may be compared to large expanses of open spaces which are sparsely populated with a relatively low rate of activity. In what follows, we describe the social media data collected from Foursquare, Twitter and Instagram postings in Singapore.

2.1 Foursquare data Foursquare is a location-based social network with a user-defined database of venues, where people use their mobile phones to “check-in” into a venue in order to update friends on their current location and share their experiences. This model motivates users to actively update the database with venues of interests and categorise venues according to a site’s predefined ontology, which results in a very comprehensive source of geographically localised social data with descriptions of millions of places around the world. In this study, we systematically parsed Foursquare’s data warehouse using the available Application Programming Interface (API), collecting information about 129,549 Foursquare venues in Singapore.

2.1.1 Venue Ontology Foursquare describes the venue types using a predefined ontology of hierarchically organised categories. For example, top level categories include: Arts & Entertainment; College & University; Food; Professional & Other Places; Nightlife Spots; Residences; Great Outdoors; Shops & Services; and Travel & Transport. The complete description can be retrieved from Foursquare Catalogue6 (requires Foursquare authentication) by using an API.

2.1.2 Mapping the venue ontology to Wikipedia categories The Foursquare categorisation covers a wide variety of venue types, so we use the venue ontology to identify higher-level semantics in location-based data by correlating Foursquare categories with corresponding Wikipedia articles. Wikipedia is an online encyclopedia entirely created by its users, and besides providing

570

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

readable explanations of different terms, it also organises pages into a hierarchical categorisation tree. That is, every page in Wikipedia is associated with a set of semantically-related categories, and every category is also categorised in the same way. In this way, the hierarchical categorisation structures in Wikipedia organise general concepts into corresponding categories. For example, the categorisation tree for “Restaurant” from Wikipedia is given in Table 1. Tab. : Sample -level categorisation tree for a keyword “Restaurant”. Keyword:

Restaurant

Categories: Level 

Restaurants

Categories: Level 

Food Retailers; Hospitality Industry; Buildings and Structures by Type; Gastronomy

Categories: Level 

Retailers; Food Retailing; Commerce; Living Arrangements; Tourism; Travel; Service Industries; Buildings and Structures; Categories by Type; Nutrition; Culture; Hobbies

The categorisation structures can be utilised to traverse from the actual instance, represented by the Wikipedia article, to more semantically general concepts, represented by the higher order of categorisation. Although not always precise, the categorisations in Wikipedia provide reasonable semantic approximations of actual instances. In order to utilise this approach to organise the Foursquare venues into semantic categories, we map the venue catalogue onto the corresponding Wikipedia article and then use the traversing approach outlined below.

2.1.3 Using DBPedia to produce the Wikipedia classification tree The most straightforward approach assumes querying the Wikipedia website for a known term over the Internet, and parsing the resulting raw HTML data in order to identify the categories it belongs to. This, however, requires substantial time for network communication, which may grow to unacceptable levels in cases where several levels of categorisation need to be retrieved. DBPedia7 is an online project aiming at automatic parsing and processing of Wikipedia data in order to organise a structured representation of various aspects of its enormous database. In the scope of the current study we need a robust (and preferably fast) way of mapping the terms in Foursquare’s venue catalogue to Wikipedia’s categorisation tree. Fortunately, DBPedia’s datasets include downloadable offline redirects, article categories and categories of the category mappings. Though still big in size, proper implementation handles the mappings and produces a categorisation tree for a given term in a reasonable time.

The role of images in social media analytics

571

2.1.4 Venue-based semantic map Using the above approach and concepts, we map Foursquare’s venue descriptions onto a set of higher-level semantic categories and aggregate the data into the grid which is displayed in Figure 1b. Every cell of the grid is associated with a vector of accumulated higher-level semantic categories, providing a conceptual description of the area coved by the grid cell. Table 2 presents an example of accumulated higher-level categories for a sample cell. The interactive map displays these conceptual descriptions for each cell in Singapore to provide a description of the social venues and associated activities undertaken in that specific area. Tab. : Example of Accumulated Semantic Categories for a Sample Cell. Category

Counter

Schools Students Education Youth Academic institutions Age and society Human development Universities and colleges Higher education

        

2.2 Twitter data Twitter is a micro-blogging service allowing users to share short text messages with friends. Although attributes can be attached to a single message by the user (e.g. images and geo tags), as well as by the service itself (e.g. user account metadata), the main resource used by Twitter is natural language; that is, text. Natural Language Processing (NLP), a well-established field of science, is concerned with automatic analysis and understanding of natural texts (e.g. Bates 1995; Jurafsky and Martin 2008; Manning and Schütze 1999). One of the techniques developed by NLP is “sentiment analysis” (e.g. Wright 2009) which aims to extract and evaluate the expression of emotion in natural texts. In the current study, we apply sentiment analysis (described below) to a set of geo-located Twitter messages and aggregate the results in the uniform density grid displayed in Figure 1b.

2.2.1 Accessing Twitter data Twitter provides a rich API8 for accessing and interacting with its database. Although it imposes some limits on accessing historical data, Twitter messages

572

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

may be collected in real time, subject to filtering criteria. In this study, we collected Twitter messages with geographical locations in Singapore for six weeks, resulting in 98,733 messages in total.

2.2.2 Sentiment analysis While many computational approaches to sentiment analysis exist (e.g. Liu 2010), including new techniques specifically designed for Twitter messages (e.g. Pak and Paroubek 2010; Agarwal et al. 2011), we favour established techniques which are readily available for use as implemented software due to the prototype nature of this study. Therefore, we use Synesketch9 which is a freely available sentiment analysis tool implemented in Java. The algorithm uses the WordNet10-based lexicon of words and their emotional weights together with NLP heuristics to evaluate scoring of basic emotions in a given text. The basic emotions that are evaluated (e.g. Ekman 1999) are happiness, sadness, fear, anger, disgust, and surprise. In our system, every captured Twitter message is passed to the sentiment analysis module, which produces a vector of scores assigned to each basic emotion. These vectors are aggregated in the uniform density grid described above and normalised. Table 3 represents the example of sentiment analysis vector for a sample grid. Tab. : Aggregated and normalised sentiment analysis vector for a grid cell. Surprise

Disgust

Happiness

Sadness

Fear

Anger

.

.

.

.

.

.

2.3 Instagram data Instagram is a social network which allows users to take photos, process them using a set of predefined filters, and share the results with friends. In cases where the user enables the geo-tagging functionality, every picture is associated with a pair of location coordinates. These features backed by an effective API make Instagram a valuable source for automated social data analysis. In these study we used Instagram database to download 301,865 images geographically located in Singapore.

2.3.1 Automated Face Detection Automated content analysis of a random image is a very challenging task and developing such techniques lies well beyond the scope of the current study.

The role of images in social media analytics

573

Instead, we use a robust technique, which firstly produces meaningful results for our task and, secondly, can be implemented and freely used for this study. OpenCV11 is a free open source computer vision library with numerous image and video processing techniques. One of the techniques available with OpenCV and directly applicable for this study is face detection. Face detection attempts to identify rectangles containing faces in a given image at variable scale. The current state-of-the-art techniques are quite reliable and have become a standard feature in modern digital cameras and smartphones. OpenCV implements face detection using a fast and reliable cascade-based detector originally proposed in Viola and Jones (2001). The processing is organised in a similar way as the processing of Twitter and Foursquare data. Firstly, the images are aggregated into cells in the constant density grid and then face detection is applied to every image and number of faces is calculated. Finally, face counters are aggregated and normalised over the total number of images in the cell.

2.4 Clustering Raw data projected on a grid is still difficult to interpret, especially in case of Wikipedia category vectors, which may contain many different counter values. In order to emphasise similarities between different grid cells we apply a “clustering” technique to group similar data entities together. Generally, clustering algorithms investigate data and assign cluster labels to similar (in some sense) raw data vectors. There are many different clustering approaches available, and given the nature of our data, we use “hierarchical clustering” (e.g. Gan, Ma and Wu 2007), which is a simple approach that groups data vectors (in this case, for the grid cells) with their closest match. The algorithm iterates through the data and sequentially merges vectors which are most similar to each other, and tracks the accumulated error. Then, the resulting hierarchy is organised according to a threshold with a desired error level to produce a final labeling. Clustering simplifies data analysis and visualisation by assigning the same label to similar data vectors to reveal the structure of the underlying database. Manual identification of a database of similar size would be very challenging and perhaps not possible.

2.5 Overall system The overall data aggregation and analysis system is illustrated in Figure 2. Different data sources are arranged in rows and different processing stages are organised in columns. The data is acquired at stage (A) and processed at stage (B). At processing stage (B), sentiment analysis algorithm is applied to Twitter messages, face detection is applied to Instagram photos and Foursquare venue categories are

574

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

Fig. 2: Overall data aggregation and analysis system

mapped onto Wikipeda’s categorisation tree. The resulting data structures (C) are represented by emotion score vectors for Twitter, relative face counters for Instagram photos, and relative category counters for Foursquare. The data is then clustered and visualised at the analysis and visualisation stage (D). The system is largely implemented using Python12 programming language, face detection is implemented in C++ and sentiment analysis in Java. The visualisation of the social media analysis is implemented as interactive graphical user interface (GUI) using Java and the “Processing” (Reas and Fry 2007) graphics library. The GUI renders a grid over the screenshot of Singapore’s geographical map and facilitates interactive navigation (i.e. panning and zooming) for exploration of the data. That is, the GUI has facilities for selecting a cell, highlighting the other cells in the same cluster and zooming to the street level. The overall data aggregation and analysis system in the GUI are illustrated in Figure 3a–b. The user can click on any cell to view the other cells which belong to the same cluster as calculated from the emotion vectors, face counters and category counters, as displayed in Figure 3a. The social media analysis for each cell (in terms of venue counters, emotion vectors, face counters, and cluster counters and averages) are displayed in the lists of aggregated data as shown in Figure 3b. In what follows, we use the GUI to explore the multimodal analysis for cells and cell clusters in different regions in Singapore.

The role of images in social media analytics

a: Data cells in Singapore.

b: Data cells in Singapore with analytical dimensions. Fig. 3: Overall data aggregation and analysis system.

575

576

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

3 Social media analytics in Singapore Singapore is a small highly urbanised city-state which has a history of comprehensive long-term urban planning to ensure that the country is “economically vibrant and one of the most liveable cities in Asia” (Urban Redevelopment Authority 2012)13. The Urban Redevelopment Authority (URA) in Singapore has two key plans, the “Concept Plan” for strategic land use and transportation for 40–50 year period (reviewed every 10 years), and the “Master Plan” for strategic land use plan for 10–15 year period. Such plans are deemed necessary, given limited land in Singapore and the high density of the population. URA’s most recent Master Plan 2008, for example, aims to make more creative use of land (e.g. use of underground spaces, mixed space usage), develop new growth centres outside of the city and enhance public transport and leisure options, while retaining Singapore’s built and natural heritage. Following the URA’s plans, Singapore is organised into five regions (Central Region, East Region, North Region, North-East Region and West Region) and 55 areas, each of which have their own detailed land-use plan, as described in the Master Plan 200814. In this study, cells from four regions and seven areas are chosen to cover the major types of land usage in Singapore, following the Master Plan 2008 (i.e. residential, commercial, hotel, business, educational, reserves and gardens). As displayed in Table 4, the areas are: (1) Chua Chu Kang – residential area; (2) Pasir Ris – residential area; (3) Jurong East – mixed-use with gardens, education centre, reserve and residential areas; (4) Serangoon Road – mixed-use in an Indian cultural area (i.e. ‘Little India’) which has commercial and residential areas; (5) Marina Bay – commercial area (luxury hotel, integrated resort); (6) Orchard Road – commercial area (retail, hotels, restaurants and other activities); and (7) Shenton Way – the central business district. Descriptions of these regions are summarised in Table 4. For illustrative purposes, the selected cell for Marina Bay is highlighted in Figure 4a, along with other cells in the grid that belong to the same cluster. In Tab. : Areas in Singapore. Region

Area

Classification

West Region East Region West Region

Chua Chu Kang Pasir Ris Jurong East

Central Area

Serangoon Road

Central Area

Marina Bay

Central Area

Orchard Road

Central Area

Shenton Way

Residential Residential Mixed-Use: Japanese Gardens, Science Centre, a reserve and residential areas Mixed-Use: “Little India” with Indian restaurants, retail outlets, and residences Commercial: luxury hotel, integrated resort and retail outlets Commercial: retail outlets, restaurants, hotels and other commercial activities) Business: central business district

The role of images in social media analytics

577

a: Marina Bay Sands (highlighted cell).

b: Marina Bay Sands (highlighted cell) with Aggregated Data. Fig. 4: Marina Bay Sands.

Figure 4b, the classifications for the venues are displayed in column (1), the results of the sentiment analysis and face counts are displayed in column (2), and the semantic classifications for venues in the cluster are given in column (3). Similarly, the selected cells for Orchard Road and Shenton Way and the results of the social media analyses are given in Figure 5a and 5b respectively.

578

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

Fig. 5a: Orchard Road.

We may note that the cells in the same cluster can be spread across the different regions in Singapore (see Figure 3). This is not surprising, as traditional methods of categorising urban spaces (e.g. Singapore’s “Concept Plan” and “Master Plan”) do not provide a semantic description of the different venues located in the areas. However, they do provide a description of the designated land use,

The role of images in social media analytics

579

Fig. 5b: Shenton Way.

and for this reason, we can check the results of our Wikipedia classifications for Foursquare venues against these descriptions. In what follows, we discuss the results of the social media data analyses for the cells described in Table 3.

580

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

3.1 Semantic descriptions of urban areas The mapping of Foursquare venues onto semantic categories derived from Wikipedia’s classification tree provides robust results, as ascertained from the land use descriptions of each area in the URA’s Master plan 2008. For example, the highest order semantic categories for the Chua Chu Kang cell (see Figure 6) are “Property”, with subcategories of “Building”, “Architecture”, “Service Industries”, “Physical Objects”, “Real Estate”, “Buildings & Structures”, “Construction”, “Structural Engineering” and so forth, which resonate with the recent development of this residential area under the Housing and Development Board in Singapore. That is, Chua Chu Kang is listed as one of the top ten planning areas (ranked by the size of resident population) in a government study of the Singapore resident population in June 2009 (Pang 2009). The semantic categories for Jurong are “Visitor Attractions”, “Parks”, “Scientific Buildings”, “Road Transport” and “Science Museum” (see Figure 6) for the cell that contains the Japanese Gardens, the Science Centre and some residential blocks. Similarly, the highest order semantic category for Serangoon Road (i.e. “Little India”) is “Cuisine” and “Architecture” with sub-classifications “Indian Cuisine”, “Indian Culture”, “Mughlia Cuisine”, “Pakistani Cuisine”, “South Asian Cuisine”, “Vegetarian Cuisine”, “Interior Design” and so forth, for an area which is famous for Indian culture and restaurants. These preliminary results suggest that Wikipedia’s classification tree provides useful semantic descriptions of venues and geographical locations. Using this technique, we explore the map to investigate the emotional pulse and social relations in urban areas in Singapore.

3.2 Emotion Vectors for Twitter Messages As displayed in Figure 7a the sentiments expressed in Twitter messages for the analysed cells in Singapore are generally related to happiness, followed by sadness. The weighting for anger is much lower, and the weightings for emotions of fear, surprise and disgust are even lower still. It is difficult, however, to make firm conclusions about these findings without comparable data from other countries. However, we can explore the sentiments expressed in Twitter messages across different areas in Singapore. The highest general weight for emotion is found in Twitter messages posted in the cell located in Serangoon Road, which also has the highest weightings for each emotion (i.e. happiness, sadness, fear, surprise, disgust and anger). Serangoon Road is quite different from other areas in Singapore (including Serangoon residential area in the North East Region) because it is an Indian cultural area with many traditional “shophouses” (two and three story buildings with a shop on the ground floor and residence above), Indian restaurants, hawker stalls and shopping centres. The area attracts tourists and locals alike throughout the week,

581

Fig. 6: Semantic Categorisations of Venues.

The role of images in social media analytics

Fig. 7: Residential, Mixed, Resort and Shopping and Business.

b: Face Detection for Different Areas.

a: Sentiment Analysis for Different Areas.

582 Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

The role of images in social media analytics

583

and it is extremely crowded on Sunday evenings when non-resident Indian workers congregate in this area on their day off. In contrast, the lowest general weight for emotion is found in the Twitter messages posted in the cell from Shenton Way, the primary business district in Singapore, which also has the lowest scores for all categories of emotions, except for surprise where it is ranked fifth. These preliminary results appear to fit demographics of the two areas with respect to the social and professional activities which take place there. That is, Serangoon Road is a colourful cultural area with many Indian restaurants and shopping outlets and some residential dwellings, while Shenton Way is located in the heart of the central business district. Given that Singapore is the fourth largest financial centre in the world, according to the Global Financial Centres Index (Z/Yen, September 2012) compiled by the London-based company Z/Yen and published by the City of London Corporation, one would expect that emotion is somewhat curtailed in social media messages posted in this area. The residential area of Chua Chu Kang has the second highest ratings for general emotional weight and happiness, sadness, fear disgust and anger, followed by the residential area of Pasir Ris which was rated third in these same categories. In terms of population, Chua Chu Kang, an inland residential area in the West Region, has a larger number of residents below 15 years of age in 2009 (Pang 2009) compared to Pasir Ris, which is an older and more established residential in the East Region. Again, it is difficult to make firm conclusions with this limited data, but it does appear that more emotion is expressed in Twitter messages which are posted in residential areas with a relatively younger population in Singapore. In summary, Twitter messages posted in cultural and tourist areas (i.e. Little India) and residential areas (Chua Chu Kang and Pasir Ris) appear to contain higher emotional content, compared to messages posted in the central business district (Shenton Way) and major shopping areas and hotels (e.g. Marina Bay, Jurong and Orchard Road). These results will be compared to the personal relations displayed in photos posted on Instagram.

3.3 Face Counts for Instagram The analysis of the face counts, that is, the percentage of Instagram photos which contain faces, shows less variation across the different areas of Singapore compared to sentiment analysis of the Twitter messages, as displayed in Figure 7b. That is, faces of people appear in approximately 20 % to 30 % of photos posted on Instagram, regardless of location. The largest number of faces appear in the images posted in the Jurong cell (32 %), which has local and tourist attractions, including the Japanese Gardens, Science Centre, Omni-Theatre and Snow City, which is an indoor snow centre which introduces the concept of snow and cold climates to visitors who can experience real snow and a sub-freezing temperature. Presum-

584

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

ably, people take photos of themselves, each other and other people at these venues in Jurong. With regards to the other areas, the relative percentage of photos with faces in descending frequency are Chua Chu Kang (31 %), Orchard Road (29 %), Serangoon Road (28 %), Marina Bay (26 %), Shenton Way (23 %) and Pasir Ris (22 %). It appears that Instagram users post photos of each other, themselves and other people at different venues and regions, with a slight tendency to post more photos of faces in areas which contain local and tourist attractions and educational centres as opposed to commercial, residential and business areas, although this observation is very tentative. Also, we may note that people appear in a larger number of images that indicated here, as the calculations in this study are derived from face detection algorithms. That is, photos of people whose faces are not directly in front of the camera are not included. In what follows, we discuss the implications of these preliminary results, where the analyses suggest that there is variation in the use of linguistic resources to express emotion in social media, compared to the use of photos of faces which function to establish personal relations for users of social media.

3.4 Language and image in social media According to Java et al. (2007), people mainly use Twitter to post messages about what their daily routine is and what they are doing (i.e. “chatter”), variously described as “pointless babble” (Kelly 2009) and “social grooming” and “peripheral awareness” via social contact (Boyd 2009). The next most frequent use of Twitter is conversations, followed by information/URL sharing, reporting news and the use of the link structure for information sources, friends and information seeking (Java et al. 2007). Kwak et al. (2010) note that Twitter differs from other human social network systems because messages are often forwarded (i.e. “retweeted”) so they reach a wider audience compared to the source’s immediate followers. Lee et al. (2010: 600) also note that Twitter messages have “a short effective diameter and low reciprocity, which all mark a deviation from known characteristics of human social networks”. This study suggests that users of Twitter express emotion about their own lives, their relationships and the world around them, but these linguistic resources are differentially deployed according to venue and social activity (e.g. leisure, home and business). However, we have observed that there is less variation in the use of photos to construe personal relationships in social media, given that people seem to take photographs of themselves, each other and others, and post them, regardless of where they are. These preliminary results could be artifacts derived from computational algorithms used in this study – that is, standard NLP algorithms which do not take into account the special characters used in Twitter messages, and face recognition which detects frontal views of faces, but not side and oblique views. However, regardless of the conclusiveness of these results, given

The role of images in social media analytics

585

that the face is the most direct marker of identity and a key resource for expressing emotion, it is not surprising that social media users post photos of faces, regardless of venue and social activity. Names can be shared, but not faces (unless digitally enhanced) which function to create an individual space in the social world for users of social media, a point worth discussing further. Photography imposes ways of seeing which alters reality (Sontag 1977; see also Berger 1972), an observation which is true of any semiotic act where specific choices function to create a particular view of the world and the social relations embodied in that world (Halliday 1978; Kress and Van Leeuwen 2006; O’Toole 2011). However, following Sontag (1977: 4), “[t]o photograph is to appropriate the thing photographed”. In the case of social media, the phenomena being appropriated include the identities, emotions and social relationships construed in the photos, along with the time and place of the event. Moreover, given the fundamental “reversibility” in vision (Lacan 1981 [1964]), where we both see and are seen (i.e. we are the subject and the object), the social media user who sees and photographs people, in turn is seen by others as a person who has some relation with those photographed, including of course the potential to be observed as a person in the photo as well. Lacan (1981) explains that the possibility of being observed is primary, in contrast to the possibility of seeing. In this context, the ubiquity of faces in social media photos is perhaps understandable, because photos fulfill an intrinsic need to be observed which transcends the nature of the social practice which is taking place. The photo, “a record of how X had seen Y” (Berger 1972: 10), becomes a record of how X can be observed by Z, in this case by social media users. This record extends beyond the semantic realm of Twitter messages, where emotion, identities and relationships are circulated in the same format by any number of users. However, a person’s visual profile is unique – it can be observed by others, but it cannot be recreated or circulated (without permission, at least). In this way, the photos, particularly of faces, create an exclusive niche for users of social media, and one that cannot be replicated.

4 Conclusion The results of this study, where social media users are seen to use linguistic and visual resources in different ways, need to be interpreted cautiously due the prototype nature of the approach and the shortcomings of the computational algorithms which have been used to analyse and cluster the multimodal data. That is, while the algorithms are robust approaches for natural language processing and face detection, they are not specifically designed for social media data. Also, the emotions in the faces are not analysed, although there is certainly potential for future research in this area (e.g. Fasela and Luettinb 2003). Moreover, the data collection

586

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

and analysis needs to be much more extensive, rather than sampling cells from different regions in one country, before definitive conclusions can be reached. Despite the limitations, the multimodal digital humanities approach adopted in this study yields promising results with regards to using socially-created content such as Wikipedia and social media networks for analysing and understanding different forms of communication and social life in urban spaces. That is, many studies of social media track the traffic of social networks, but not the multimodal content according to semantically organised location-based sources. In this study, this semantic content is integrated across text-based and image-based social networks, which in turn allows us to understand more about the roles of semiotic resources in social media, in this case, for language and photos and in the future, potentially for video data. Within this approach, the significance of images in social media analytics, both as the object of study and an interactive resource for investigating and understanding social behaviour, is apparent. The roles of visual resources (and others) will continue to expand as digital technologies refashion and transform our semiotic world. Acknowledgments: The research was supported by the research grant Mapping Asian Cultures: From Data to Knowledge (No. HSS-0901-P02) (Principal Investigator: Kay O’Halloran), funded by the National University of Singapore. This study is the result of interdisciplinary collaboration between Kay O’Halloran and former researchers in the Multimodal Analysis Lab in the Interactive & Digital Media Institute at the National University of Singapore. Alvin Chua developed the interactive visualisation map, and Alexey Podlasov developed the techniques for analysing the Twitter and Foursquare data and applied the clustering algorithms. We thank Yanpeng Cao for downloading and analysing the Instagram photos.

References Agarwal, Apoorv, Boyi Xie, Ilia Vovsha, Owen Rambow and Rebecca Passonneau. 2011. Sentiment Analysis of Twitter Data. Proceedings of the Workshop on Language in Social Media (LSM 2011), 23 June 2011, Portland, Oregon, USA, 30–38. Stroudsburg: The Association for Computational Linguistics. Bates, Madeleine. 1995. Models of Natural Language Understanding. Proceedings of the National Academy of Sciences of the United States of America 92(22): 9977–9982. Bawa-Cavia, Anil. 2011. Sensing the Urban: Using location based social network data in urban analysis. Presented at the First Workshop on Pervasive Urban Applications (PURBA), run in conjunction with the Ninth International Conference on Pervasive Computing in San Francisco on June 12–15, 2011. Available at: http://purba.mit.edu/content/sensing-urbanusing-location-based-social-network-data-urban-analysis. Berger, John. 1972. Ways of Seeing. London: BBC and Penguin Books. Boyd, Dana. 2009. Twitter: “Pointless babble” or peripheral awareness + social grooming? 16/ 08/09. Available at: http://www.zephoria.org/thoughts/archives/2009/08/16/twitter_ pointle.html.

The role of images in social media analytics

587

Braga, Reinaldo B., Ali Tahir, Michela Bertolotto and Hervé Martin. 2012. A Multi-layer Data Representation of Trajectories in Social Networks based on Points of Interest. Proceedings of the Twelfth International Workshop on Web information and Data Management (WIDM ’12), 19–26. New York: ACM. Cranshaw, Justin, Raz Schwartz, Jason I. Hong, and Norman Sadeh. 2012. The Livehoods Project: Utilizing social media to understand the dynamics of a city. Presented at The 6th International AAAI Conference on Weblogs and Social Media, Dublin, Ireland, 2012. Available at: http://justincranshaw.com/papers/cranshaw_livehoods_icwsm12.pdf. Ekman, Paul. 1999. Basic Emotions. In: Tim Dalgleish and Mick J. Power (eds.), Handbook of Cognition and Emotion, 45–60. Chichester: John Wiley & Sons. Fasela, Beat and Juergen Luettin. 2003. Automatic Facial Expression Analysis: A Survey. Pattern Recognition 36(1): 259–275. Gan, Guojun, Chaoqun Ma and Jianhong Wu. 2007. Data Clustering: Theory, Algorithms, and Applications. ASA-SIAM Series on Statistics and Applied Probability (No 20). Cambridge: Cambridge University Press. Hotz, Robert L. 1 October 2011. Decoding our Chatter. The Wall Street Journal: Technology. Available at: http://online.wsj.com/article/ SB10001424052970204138204576598942105167646.html. Java, Akshay, Tim Finin, Xiaodan Song and Belle Tseng. 2007. Why We Twitter: Understanding microblogging usage and communities. Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 Workshop on Web Mining and Social Network Analysis, 55–65. New York: ACM. Jurafsky, Daniel and James H. Martin. 2008. Speech and Language Processing. (2nd edition). Upper Saddle River: Pearson Prentice Hall. Kelly, Ryan (ed.). 2009. Twitter study – August 2009. San Antonio: Pear Analytics. Available at: http://www.pearanalytics.com/blog/2009/twitter-study-reveals-interesting-results-40percent-pointless-babble/. Kress, Gunther and Theo van Leeuwen. 2006. Reading Images: The Grammar of Visual Design. (2nd edition). London: Routledge. Kwak, Haewoon, Changhyun Lee, Hosung Park and Sue Moon. 2010. What is Twitter, a social network or a news media? Proceedings of the 19th international conference on World Wide Web WWW '10, 591–600. New York: ACM. Lacan, Jacques. 1981[1964]. The Split Between the Eye and the Gaze. In: Jacques-Alain Miller (ed.), The Four Fundamental Concepts of Psychoanalysis, 67–78. Trans. by Sheridan, Alan. New York: Norton. Lima, Manuel. 2011. Visual Complexity: Mapping Patterns of Information. New York: Princeton Architectural Press. Liu, Bing. 2010. Sentiment Analysis and Subjectivity. In: Nitin Indurkhya and Fred J. Damerau (eds.), Handbook of Natural Language Processing. (2nd edition), 627–666. Boca Raton: CRC Press. Manning, Christopher D. and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. Cambridge: MIT Press. O'Toole, Michael. 2011. The Language of Displayed Art. (2nd edition). London/New York: Routledge. Pak, Alexander and Patrick Paroubek. 2010. Twitter as a Corpus for Sentiment Analysis and Opinion Mining. In Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, Daniel Tapias (eds.), Proceedings of the International Conference on Language Resources and Evaluation, LREC 2010, 17–23 May 2010, Valletta, Malta, 1320–1326. Paris: European Language Resources Association. Pang, Chin Eng. 2009. Geographic distribution of the Singapore resident population. Statistics Singapore Newsletter September 2009. Available at: http://www.singstat.gov.sg/pubn/papers/people/ssnsep09-pg8–12.pdf.

588

Kay l. O’Halloran, Alvin Chua and Alexey Podlasov

Podlasov, Alexey and Kay L. O’Halloran. 2013 [in press]. Japanese Street Fashion for Young People: A multimodal digital humanities approach for identifying socio-cultural patterns and trends. In Emilia Djonov and Sumin Zhao (eds.), Critical Multimodal Studies of Popular Culture. London/New York: Routledge. Reas, Casey and Ben Fry. 2007. Processing: A Programming Handbook for Visual Designers and Artists. Cambridge: MIT Press. Salah, Alkim, Almila Akdag, Cheng Gao, Andrea Scharnhorst and Krzysztof Suchecki. 2011. Design vs. Emergence: Visualisation of knowledge orders. In Katy Börner and Michael J. Stampe (eds.), 7th Iteration (2011): Science Maps as Visual Interfaces to Digital Libraries, Places & Spaces: Mapping Science. Available at: http://scimaps.org. Salah, Alkim, Almila Akdag, Cheng Gao, Krzysztof Suchecki and Andrea Scharnhorst. 2012. Need to Categorise: A comparative look at the categories of Universal Decimal Classification System and Wikipedia. Leonardo 45(1): 84–85. Sontag, Susan. 1977. On Photography. London: Penguin. Svensson, Patrik. 2010. The Landscape of Digital Humanities. Digital Humanities Quarterly 4(1). Available at: http://digitalhumanities.org/dhq/vol/4/1/000080/000080.html. Wright, Alex. 2009. Mining the Web for Feelings, not Facts. New York Times, 23/08/09. Available at: http://www.nytimes.com/2009/08/24/technology/internet/24emotion.html?_r=1. Viola, Paul and Michael Jones. 2001. Rapid Object Detection Using a Boosted Cascade of Simple Features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001) Volume 1, 511–518. Los Alamitos: IEEE. Z/Yen Group. 2012. The Global Financial Centres Index 12. London: City of London Corporation. Available at: www.longfinance.net/Publications/GFCI%2012.pdf.

Websites . . . . . . . . . . . . . .

Twitter: https://twitter.com/. Instagram: http://instagram.com/. Foursquare: https://foursquare.com/. Wikipedia: http://wikipedia.org. http://scimaps.org/maps/map/design_vs_emergence__127/. Foursquare Catalogue:. https://developer.foursquare.com/docs/explore#req=venues/ categories. DPedia: http://dbpedia.org/. Twitter API: https://dev.twitter.com/. Synesketch: http://www.synesketch.krcadinac.com/. Wordnet: http://wordnet.princeton.edu/. OpenCV: http://opencv.willowgarage.com/wiki/. Python: http://www.python.org/. http://www.ura.gov.sg/MP2008/intro.htm. http://www.ura.gov.sg/MP2008/.

Nathaniel Dafydd Beard

26 From static to dynamic: The changing experience of fashion imagery Abstract: Fashion is experienced and understood as much through the visual as the tactile. Static photographs, in magazine or newspaper editorials and advertising, assist to assert and confirm fashion’s status as a playful, idealised, and glamorous world. For instance, in the plethora of glamorous images of top models and catwalk shows which appear in newspapers and their weekend supplements and especially in glossy magazines such as Harper’s Bazaar, L’Officiel or Vogue. Yet it is often forgotten that feature films, too, play their role in showcasing fashion, disseminating both clothing styles and fashionable lifestyles to a wider audience. Increasingly, especially since the late 1990s, the static and the moving image are becoming intertwined as a means to communicate fashion. As print media has sought to develop its own virtual platforms, fashion image makers are developing new ways to bring static editorial shoots to life. This has changed the audience experience, pulling fashion image-making away from the 2D into the 3D. This chapter traces these developments, looking at the different ways the fashion industry has sought to capitalise on the new opportunities provided through dynamic or moving image making.

1 Introduction Fashion, both in its abstract form as a conceptual ideal and as an important commercial economic entity, poses a curious dilemma. While fashion ostensibly concerns and, perhaps, manifests itself best through the production of its physical objects – that is clothes and accessories such as bags or shoes – it is, in many respects, primarily experienced purely visually, most often in 2D. For an entity that relies so much on its desirability through the tactile or the haptic, at its most prosaic in the touching of cloth and the feel of contrasting textures, perhaps such an exclusion adds to the enhancement of fashion’s attraction. Indeed, fashion is often perceived as an enclosed entity, which excludes as much as it includes. To be able to participate in fashion, or rather in being fashionable and to wear fashionable clothes, it is usually thought necessary to be able to understand its codes, to be able to decipher what is “in” from what is “out”. In attempting to navigate this, those within the fashion industry, and the public it seeks to entice, rely heavily on the use of visual devices to communicate such codes and assist in unpacking their message. Such devices take shape in the form of photographic images in magazine and newspaper editorials, and also in the advertisements placed by fashion brands in the same publications. A key component of these photographic

590

Nathaniel Dafydd Beard

images lies in their “magical” quality. This quality is perhaps also shared by other photographic images as Vilém Flusser (2007) explains: The magical nature of images must be taken into account when decoding them. Thus it is wrong to look for ‘frozen events’ in images. Rather they replace events by states of things and translate them into scenes. The magical power of images lies in their superficial nature, and the dialectic inherent in them – the contradiction peculiar to them – must be seen in the light of this magic (Flusser 2007 [1983]: 9).

The magical quality of images and the strength of their “superficial nature” is, perhaps, especially inherent in the communication of fashion through visual imagery. Often thought of as a powerful form of escapism, fashion is very much concerned with the evocation of fantasy, dreamscapes and magic. Yet, in communicating such messages, those involved in the production of fashion imagery – including photographers, stylists, designers, fashion editors, advertisers, art directors, hairdressers, and make-up artists – must often tread a fine line between creating something which indeed fuels the fantasy or dream of the viewer, with something that remains potentially obtainable. Within the general discourse on fashion, and especially given the important role of image-making that exists within fashion, it is striking how strongly the dichotomy between the commercial versus the creative remains. As Anandi Ranamurthy (2004) asserts: It is clear that there are tensions in the relationship between fashion photography and both advertising photography and art photography. The fashion image attempts to stand aloof from the undiluted commercial context of advertising, since most fashion spreads are commissioned by magazines which are not directly selling clothes. Yet the undeniable commercial angle has separated it from the art photograph, despite the inevitable commercial context of the latter (Ranamurthy 2004: 222).

This tension in the creation of imagery specific to fashion, and its primary role as a means to promote and sell clothes, is not, however, a contemporary facet of its current position. Rather it appears as if this tension is almost intrinsic within the creation of fashion imagery, in particular the communication of fashion through photography, and has been a cause of concern throughout its history. As Nancy Hall-Duncan (1979) comments in the introduction of her seminal book The History of Fashion Photography: It is the twofold stigma of commercialism and materialism that makes fashion one of the few types of photography whose very values are called into question. Fashion photographs are designed to be seducers, propaganda so potent it can beguile us into buying the most frivolous products. Fashion photography also commits the ‘sin’ of being produced not only for love but for money, implying creative manipulation and the sacrifice of photographic and artistic integrity. Fashion photographs are ostensibly as transitory as last year’s style or this month’s magazine issue (Hall-Duncan 1979: 9–10).

Yet developments in communicating fashion through the visual, and in particular the development of fashion photography, has perhaps been assisted through its

From static to dynamic: The changing experience of fashion imagery

591

own attempts to rationalise its position between the twin pillars of the commercial and the creative. Although fashion communicated through photography is often viewed as lacking in substance, it is perhaps this tension that has instead led to a far richer and deeper contribution to the wider field of visual communication than is generally perceived. For example, as Hall-Duncan further explains: Fashion photography has sometimes been called ephemeral, commercial, and frivolous, and its importance has been called into question. That fashion photography has a commercial intent implies to some that it lacks photographic and artistic integrity. In reality, it has produced some of the most creative, interesting, and socially revealing documents and revealed the attitudes, conventions, aspirations, and taste of the time. It also reflects women’s image of themselves, including their dreams and desires, self-image, values, sexuality, and interests (Hall-Duncan 2010: 300).

Far from communicating only the specific and contemporary interests of fashion editors or fashion brands, the images created by, and through, fashion photography have a far greater resonance and appeal. Yet it should not be forgotten, as it so often is, that fashion has been, and continues to be, communicated, referenced, and explored not only through the static reproductions that appear in magazine or newspaper editorials and in advertising or on billboards, but also through “moving images”, or film. The classic movies of Hollywood, and the cinematic output of many other film industries in Europe and elsewhere, such as “Bollywood”, have also played a significant role in the communication of fashion, and indeed, fashionable living. Surprisingly, however, the significance of film as a vehicle in communicating fashion, not only through feature films, but also through documentary or public information films, often goes unacknowledged. Within the contemporary 21st Century fashion industry, however, in parallel with new developments in technology, film as a device in communicating both fashion imagery and knowledge is experiencing a revived and renewed interest. In addition to the exploration of film as a method to document the experience of contemporary fashion, in the evolvement of a new genre referred to as “Fashion Film”, film-making itself is increasingly viewed as an extension of the practice of not only fashion photographers, stylists and art directors, but also that of fashion firms as diverse as Burberry and Gareth Pugh. In addressing these developments, the aim of this chapter is to explore and reveal the depth and diversity in today’s visual communication of fashion, considering the differing roles that static and dynamic imagery have come to play in an increasingly competitive and globalised fashion industry.

2 Communicating fashion It remains curious to note, here in the second decade of the 21st Century, that the general perception of fashion remains the image of an imperious model posed on the catwalks of Paris, London, Milan, New York, or Tokyo, the “Big Five” cities

592

Nathaniel Dafydd Beard

Fig. 1: Alexander McQueen, Spring/Summer 2013, Milan Fashion Week. Photograph courtesy of Catwalking.com.

often referred to as the “Fashion Capitals.” Usually this model is photographed wearing a quite incredible outfit, perhaps garish in colour, made of some unusual material, with too much make-up, too high shoes, and too thin a waist. Can she really be “real”? Yet in magazines, and more particularly in the pages of newspapers, this is how “fashion” is often communicated, as something not just playful or amusing, but actually as a joke, certainly for the general public. Yet, for those who work within the fashion industry, such images represent instead an important facet of publicity in developing awareness for the featured designer’s work. While you may not want to wear the outfit, it may well tempt you into finding out more about the featured designer, and even into considering buying their perfume, handbag or sunglasses (see Figures 1 and 2). Top catwalk photographers, such as Chris Moore or Peter Stigter, play a key role in both the creating and dissemination of such images, which are relayed today not only via the traditional print media of newspapers and magazines, but also via virtual, Internet-based platforms. Speed is of the essence in this process, where it is now possible to up-load photographic images from the international collections displayed in the Fashion Capitals within an hour of the catwalk show taking place. Yet within the firmament of international fashion photography, while the specialist catwalk photographer is relied upon and admired for their agility in

From static to dynamic: The changing experience of fashion imagery

593

Fig. 2: Prada, Spring/Summer 2012, Milan Fashion Week. Photograph courtesy of Catwalking.com.

obtaining high-quality reproductions of fashion collections at a fast turn-around, the photographers whose work appears in the high-end, glossy magazines, are considered to be fashion’s leading creative image-makers. Photographers in this category include Mert Alas and Marcus Piggott, Nick Knight, Inez van Lamsweerde and Vinoodh Matadin, Steven Meisel, Sølve Sundsbø, Mario Testino, and Tim Walker, amongst others. Where the catwalk photograph is considered an instantaneous “means-to-an-end”, the 2D spreads found in weekend supplements and especially glossy fashion magazines, are meant to be appreciated in an undisturbed and more opportune setting. Such images are also constructed with the ultimate aim of having a greater longevity, ideally outside the context of the magazine spread, as Anandi Ramamurthy (2004) states: […] at the same time as producing the typical, fashion photographers have aimed to construct a sense of what is original and unique within a particular fashion. They have also tried to produce images which stand their ground beyond the transitory space of the magazine and the transitory nature of fashion, and for example enter the gallery or the coffee-table book (Ramamurthy 2004: 222).

The work of many of the leading fashion photographers, past and present, can today be increasingly found not only on the pages of Purple, L’Officiel, Self Service,

594

Nathaniel Dafydd Beard

Tank, or Vogue but also on the walls of various public museums and commercial galleries. In many ways the museum or gallery space has become a further site for the showcasing of fashion imagery, reinforcing its primacy as an important cultural force within the minds of the general public. Exhibitions as diverse as Chic Clicks: Creativity and Commerce in Contemporary Fashion Photography (Lehmann 2002), held at the ICA Boston, Italian Eyes: Italian Fashion Photographs from 1951 to Today (Frisa et al. 2005), held at Fondazione Pitti Immagine, Florence, Face of Fashion (Bright and Aletti 2007), held at the National Portrait Gallery, London, Not in Fashion: Photography and Fashion in the 90s (von Olfers and Teixeira 2010), held at the MMK, Frankfurt am Main, Vanity: Mode/Fotographie aus der Sammlung F.C. Grundlach, (Matt and Genzmer 2011) and No Fashion, Please! Fotographie Zwischen Gender und Lifestyle, (Matt and Weiermair 2011), both held at the Kunsthalle Wien, Vienna; demonstrate the interest and vitality in exhibiting fashion photography that has occurred since the early 2000s.

3 From static to dynamic The display and communication of fashion through the form of static images fixed to the page of a magazine or newspaper, or perhaps gazing down upon us from a billboard on the street or a poster at the bus stop, is integral to the potency of fashion in permeating the environment around us. Yet the moving image or film has also played a part in communicating ideas around both fashion and fashionable living. Although the moving, or “dynamic” non-static image, has usually been considered peripheral in the visual communication of fashion, the relationship between the two has often been intimately connected. As Marketa Uhlirova and Christel Tsilibaris (2006), founders and curators of the first Fashion in Film Festival: Between Stigma and Enigma, state: Despite the relatively common use of film and later TV and video in recording new fashions, the moving image has never been as integral a part of the fashion industry as photography. However, the modern industries of film and fashion are unthinkable without each other: fashion has relied on film for its capacity to awaken desires in and entertain a wide audience. Film, in turn, has drawn heavily on fashion’s spectacular effects, design novelties and its characteristic environment. It has used fashion and fashionable clothing as a vehicle to elate and seduce, as a pretext to probe various socio-political and cultural issues or as a potent metaphor for them (Uhlirova and Tsilibaris 2006: 3).

While fashion and film appear to be the perfect partners, each concerned with the seduction of their audiences, the relationship between the two has often proved problematic. While the classic movies of 1930s, 1940s and 1950s Hollywood, for example, were often the settings for grand spectacles encompassing what appears to be fashionable clothing, the production processes of film-making were often too slow to meet the needs of the fashion industry during this time (Gaines 2011: 145);

From static to dynamic: The changing experience of fashion imagery

595

a situation which continues today. As Drake Stutesman (2011: 20) asserts “The cinema costume design world and the couture world are linked but also very separate.” While many film-costume designers, such as Gilbert Adrian, Edith Head, and Travis Banton became well known as designers for the many films they worked on, and Parisian designers of the calibre of Chanel, Jeanne Lanvin, Molyneaux, and Elsa Schiaparelli provided costumes for films and dressed film-stars off-duty, the long lead-times of film-making meant the openings of films rarely coincided with the staging of the bi-annual fashion collections in Paris. With films appearing a year or more after their completion, this often negated any real sense of “fashionability” in their portrayal of fashionable clothes, despite the large fan followings of the leading movie stars, such as Marlene Dietrich, Joan Crawford, Rita Hayworth or Ava Gardner, who copied or emulated the styles worn by their heroines. While there is some evidence that fashion photographers also worked with the medium of film on a sporadic basis between the 1950s and 1980s, notably Erwin Blumenfeld towards the end of his career in the 1960s, and Guy Bourdin in the 1970s, these were primarily experimental, private efforts not intended for the benefit of their commercial fashion clients. It was not until the mid-to-late 1990s that film, as a medium, began again to attract the serious attention of fashion photographers as an extension to their image-making practice. This renewed interest was very much driven in part by the development of new technology, and especially the role computers began to play as a “tool” in the making of fashion imagery. As Charlotte Cotton (2000) explains: The use of digital technology to create seamless visions of fashion also offered a timely vocabulary for the fashion industry and brought new image-makers to the fore. By the midnineties, the computer had become the creative tool behind the presentation of detached and ‘hyper-real’ fashion imagery. These images, in part assisted the re-establishment of glamorous fashion photography, albeit with a contemporary twist (Cotton 2000: 7).

Experimentation with early forms of digitisation and computer-assisted imagemaking led to many fashion photographers in collaboration with stylists and others to reconsider how they went about communicating fashion, and in particular the possibilities that adding a “moving” or “dynamic” dimension to their imagemaking may have. It was not until the very cusp of the 21st Century, however, before such experimental work could find a platform via the new developing codes and programmes becoming more accessible on the “World Wide Web” or “Internet”. At the time, many of these appeared rather clunky and amateurish, due in no small part to the low-band widths and “dial-up” Internet connections then available. In November 2000, however, a new start-up established in London in the form of the web-based platform SHOWstudio became perhaps the first official showcase for these new forms of dynamic fashion image-making. Founded by leading fashion photographer Nick Knight, who had come to prominence with advertising campaigns for Yohji Yamamoto, SHOWstudio began as a “not-for-profit” initiative to showcase both Knight’s and others work in this developing field and the

596

Nathaniel Dafydd Beard

first experiments in, and what was to become defined as, “Fashion Film”. A very early example of the kind of “hyper-real” work being made and displayed on the SHOWstudio website is Sweet, a collaboration between Nick Knight and stylist Jane How. Recreating fashion outfits from the Spring/Summer 2000 collections out of sweet wrappers, doilies and paper cake cases the results were filmed and animated through using a 3D scanner, set to a sound-track reminiscent of those played when opening a child-like music box and watching the small ballerina twirl around. The use of such primitive 3D techniques allowed the viewer to observe the garments “in the round” as on a real body, including in this case the internal structures of the clothes. Even viewed today, Sweet deftly juxtaposes technology with a naive charm as the heads of the models and their seemingly dismembered sweet-wrapper outfits spin around or float past. What was so different and radical about the work of those showcased on SHOWstudio then, and which continues today, was that as well as being an outlet for experimental fashion image-making which might not otherwise have been seen, was the fact that it allowed the viewer to see the clothes “in action”, worn on moving bodies, in a dynamic environment. As Penny Martin, founding Editor-In-Chief of SHOWstudio, explains in conversation with Eugénie Shinkle: Traditionally we’ve understood fashion by the definitive still, where you only see one angle of the garment. SHOWstudio explores the idea that you can see 360 degrees, you can hear the sound of a garment, you can watch it move – that’s how fashion was designed to behave. It’s in spite of fashion photography rather than because of it that SHOWstudio is progressing (Martin, cited in Shinkle 2008: 124).

While SHOWstudio today remains one of the leading platforms for the showcasing of dynamic fashion imagery, under the moniker “The Home of Fashion Film,” other platforms on the Internet have emerged to showcase the work of both Fashion Film-makers and fashion designers. These include YouTube and Vimeo,TEST MAG, DAZED Digital, and also the LVMH-owned Nowness; alongside the websites of fashion brands. Carrie Munden, designer of the label CassettePlaya best known for its Japanese Anime, rave, and skater, inspired aesthetic of psychedelic prints and worn by Nicki Minaj and M.I.A. amongst others, presents many of her collections through the medium of film or film-like technologies such as augmented reality. The most recent film K_A_R_M_A_G_E_D_D_O_N made for Autumn/Winter 2012/ 2013 is displayed on both the CassettePlaya website and on YouTube; showcasing Munden’s collection it focuses on the scenario of a boy and girl getting reading for a night of partying. The couple are shown in the film pulling on and off clothes, smoking or drinking beer, with the girl toying with her green hair or blue, glittery eyebrows, as if performing in front of a mirror (see Figures 3 and 4). Fashion Films such as these assist in affirming the designer’s vision of their collection and also provide an engaging ambient experience for viewers and potential clients. Such film’s accessibility via social networking sites also enhances their availability to a wider and global audience than the traditional catwalk show. At the

From static to dynamic: The changing experience of fashion imagery

597

Fig. 3: Film Still, K_A_R_M_A_G_E_D_D_O_N, CassettePlaya, Autumn/Winter 2012/13. Art Direction: Carrie Munden, Director: Grace Ladoja. Courtesy of Carrie Munden, CassettePlaya.

Fig. 4: Film Still, K_A_R_M_A_G_E_D_D_O_N, CassettePlaya, Autumn/Winter 2012/13. Art Direction: Carrie Munden, Director: Grace Ladoja. Courtesy of Carrie Munden, CassettePlaya.

same time, traditional print-based glossy magazines and newspapers, from the established, well-known media brands such as Vogue, Elle, Harper’s Bazaar, L’Officiel, Marie-Claire, The Times, The Telegraph, and The International Herald Tribune, through to the edgier and niche magazines such as Dazed & Confused, i-D, Tank, and The Business of Fashion, amongst others, are also looking to create their own Fashion Films or showcase dynamic fashion content on their web-based editions. With the development of fashion magazine content to be viewed specifically on smart-phones or iPads (Kansara 2011; Smyth 2011), such as the iPad only magazines JENESEQUA and Post, this offers further opportunities to fashion image-makers to develop and extend their practice. As the introduction to the recent book MasterMind: Art Direction, Fashion Styling and Visionary Photography asserts:

598

Nathaniel Dafydd Beard

The extension of fashion photography is not just a change of medium but a whole new sphere to evoke possibilities between gestures, emotions and their relationships to their surroundings. Additional sound elements and movements also imply a call for a hybrid thinking and technical excellence in atmospheric construction, plot design and filming, and to forge new ideas in fashion photography in return (Viction:ary 2011: 3).

In the planning of Fashion Films the notion of plot design remains rather vague and contentious as this touches on how such films are defined, both by the fashion industry itself and within the discourse on film-making and the study of visual communication. While SHOWstudio and the films showcased on its website are very much experimental projects aiming to push the boundaries of what constitutes a film, others have a more prescribed viewpoint. For instance, Diane Pernet who stages a travelling film festival A Shaded View On Fashion Film (ASVOFF), which has appeared in various guises in Arnhem, Barcelona, Cluj, Mexico City, New York, Paris, and Riga tends to prefer to showcase films with a distinct narrative that goes beyond being just a visually rich spectacle. In addition, the selected films shown as part of the festival are further distinguished between how much involvement the brand or designer has had in their making, although this is sometimes difficult to determine. These films are divided into those considered to be “advertorials” for fashion brands, for example Yang Fudong’s First Spring made for Prada in 2010, from those which are made from the film-makers own viewpoint free from specifically commercial constraints, such as Armageddon Coleur 1 & 2 by Frode and Marcus. In parallel with the photographers and film-makers creating dynamic fashion imagery as an extension to their practice, in some cases deciding to focus solely on producing such work, the emerging genre of Fashion Film has also brought about a re-consideration of how fashion designers perceive and decide to present their fashion collections. With the average cost and production of a catwalk show, at the cheaper end of the scale, now approximately £ 80,000 (A 100,634) to put on as part of London Fashion Week (Dacre 2012), the much cheaper costs of producing a Fashion Film to showcase a collection, starting from £ 3000–£ 4000 (A 3774– A 5031), certainly help to make the economic case of such films very persuasive. Equally, however, and aside from their practical or financial implications, the Fashion Film opens up new possibilities for the showcasing of a designers specific creative vision, often in co-operation with a film-maker. Perhaps one of the most successful examples of this came in the form of Gareth Pugh’s Autumn/Winter 2009 collection, presented to an audience of international fashion editors during Paris Fashion Week. Arriving to view the collection, many of the editors were shocked to find not a catwalk, but instead a blank screen. With everyone finally assembled and the lights dimmed, the editors were introduced to what has since become a legendary benchmark in collaborations between a designer and a film-maker, in this case Ruth Hogben, a former assistant of Nick Knight. Perhaps in part taking inspiration from the work of Hiro, an active fashion

From static to dynamic: The changing experience of fashion imagery

599

photographer in the 1960s and 1970s and a former assistant to Richard Avedon, Hogben directed Pugh’s collection to dramatic effect, the models in the film posed in silhouette strutting in static poses on a stark white background, the monochrome black fabric of the clothes billowing around their bodies in motion. Established fashion brands too, including Burberry, Dior, Givenchy, Lanvin, Alexander McQueen, Missoni, and Prada have also sought out the talents of leading filmmakers to create both promotional advertising films and dynamic catwalk displays. For example, a number of fashion image and film-makers including Poppy de Villeneuve, Kathryn Ferguson, Stéphanie Di Giusto, Julie Verhoeven, and Mary Clerté were recently commissioned by Chloé to make the films C/Counter-Couture, H/Horses, L/Light, O/The Roundness of ‘O’, and E/Embroidery. Spelling out the name of the brand Chloé, these films are amongst a series to coincide with their recent 60th Anniversary and exhibition Chloé. Attitudes at the Palais de Tokyo in Paris; showcased on Chloé’s website as part of its Alphabet digital heritage project these films are also viewable on YouTube and Vimeo. Increasingly, such brands are also becoming as much “multi-media” or “entertainment” brands as they are fashion brands, as they seek to compete with each other in a globalised marketplace, attempting to outpace each other with ever more dazzling and extravagant spectacles. Burberry, which recently topped New York University “think-tank” L2’s Digital IQ Fashion Index for the second year running (Strugatz 2012), is perhaps the most active and forward-thinking of all these brands with a dedicated digital team who oversee all the brands digital and dynamic output. Its activities include screening Burberry’s catwalk shows live online, managing its social network presence on Twitter and Facebook, developing digital marketing campaigns and creating original dynamic content for Burberry’s website and to enhance its flagship store environments (Cartner-Morley 2012 and Williams 2012). At the same time, such developments are changing the experiences, and expectations, of the audiences for fashion image-making. As Vicki Karaminas (2012) states: While the ostensible function of the virtual fashion image is to produce and construct desirable identities connected to collective fantasies and contemporary lives, it also opens up an experimental field where the manipulation of surface appearances allows for the construction of a new language, a discourse concerning the materiality of fashion and its relation to production and communication via the use of emergent technologies. Webcasts, blogs, and social networks such as YouTube and Twitter encourage viewers to engage with the production of fashion images and thereby take an active part in its creative process by contributing to online forums, submitting photographs, downloading patterns, and offering opinions on fashion and style. These technological interventions have fundamentally opened up the fashion system by granting access to a mass audience rather than by being regulated by elite groups and authorized media (Karaminas 2012: 181).

The newly evolving dynamic fashion image making, in combination with access to social media sites, assists in this changing interplay between who is in charge –

600

Nathaniel Dafydd Beard

the fashion brands and the image-makers they employ or the audiences they seek to address. While the participation in fashion amongst the general public has seemingly become increasingly “democratised” through such activities, it is still necessary to be in the possession of the knowledge and, to a certain extent, the skill-set to be able to access such digital technologies. Not only that, it is also necessary to be able to make judgements or decisions on the content provided and, in turn, to manipulate them, to further such experiences so they can become mutually beneficial.

4 Conclusion Within the context of the contemporary fashion industry the communication of fashion through visual image-making, whether through traditional static photography now enhanced through digitisation or through the dynamics of Fashion Film, still retains both its potency and relevance. It is almost as if fashion communicated through the visual is now more real than the actual experience of fashion obtainable through visiting stores or being present at live catwalk shows. As Vicki Karaminas (2012) asserts: In all permutations, the fashion image has become the predominate mode through which fashion is articulated and disseminated. It opens up a critical space where the discourses and practices of everyday life are reimagined and reinscribed in a new experimental field of limitless possibilities (Karaminas 2012: 185).

What is most striking, perhaps, is how the discourse and practices of communicating fashion visually has been taken out of the hands of fashion’s traditional “gatekeepers” – such as editors, buyers, photographers, stylists and art directors – and put into the hands of the audiences these gatekeepers traditionally serve. In an effort to retain their authority, the glossy fashion magazines seek to continue to entice their readership by engaging with the same technologies that their readerships have access to, providing dynamic fashion content on their own websites, staging both “real-life” and virtual events, such as Vogue magazine’s Fashion’s Night Out and Fashion’s Night In, and also providing guidance and editorial coverage of such developments in their editorials. For example, a recent edition of British Elle ran a story entitled “How to Look Good on Facebook” (Cronin August 2012: 98–99), advising its readers on how to pose for images that appear on their and their friend’s Facebook pages or via other social media websites or blogs. To return to Vilém Flusser (2007 [1983]: 30), “Power has moved from the owner of objects to the programmer and the operator.” While the audiences of fashion today may not be in control or have direct access to its physical objects, they can still indulge in the magic of fashion through their control and interaction with its imagery and image-making. This is not to say that the role of the professional

From static to dynamic: The changing experience of fashion imagery

601

fashion image-maker is over, far from it, since the photographer and Fashion Filmmaker has in many ways become a more professionalised, and indeed, serious role. With the developments of digitisation and the increasing use of computer software to assist in “taking” the fashion photograph or moving-image these professionals are today as much multi-media specialists as mere image-makers, commissioned to take a certain type of image for a commercial client. In many ways today’s professional fashion image-maker has access to a fluidity in their practice that fashion photographers of the past could only imagine about; no longer just relegated to taking static or dynamic images, instead they can choose either, or combine the two. Far from losing its magic, instead the visual communication of fashion is evolving to become ever more intriguing, emotive and fantastical, with innovations in “augmented reality” technologies perhaps an indicator of where such developments may be headed. Acknowledgments: With grateful thanks to Carrie Munden and Catwalking.com for their permission to use images to illustrate this chapter, to Marketa Uhlirova for the opportunity to work on the 2nd Fashion In Film Festival “If Looks Could Kill,” and from whom I learned so many great lessons, and also Dr. Jacque Lynn Foltyn, National University, California and Dr. Rob Fisher, Inter-disciplinary.net for the opportunity to present some initial research on the topic of Fashion Film at the 3rd Global Conference on Fashion at the University of Oxford in 2011. Further thanks also go to my PhD supervisors Dr. Claire Pajaczkowska and Dr. Barry Curtis for their guidance and support and also my good friend and colleague Kathryn Ferguson, film-maker and curator, for all the great discussions and with whom I had the opportunity to collaborate with in putting on several public talks on the subject of Fashion Film, including at the Royal College of Art and London College of Fashion.

References Bright, Susan and Vince Aletti (eds.). 2007. Face of Fashion. London: National Portrait Gallery Publications. [Exhibition Catalogue]. Cartner-Morley, Jess. 2012. Burberry designs flagship London shop to resemble its website. The Guardian, 12/09/12. Available at: http://www.guardian.co.uk/fashion/2012/sep/12/burberrylondon-shop-website. Cotton, Charlotte. 2000. Imperfect Beauty: The Making of Contemporary Fashion Photographs. London: V&A Publications. Cronin, Emily. 2012. How to Look Good on Facebook. Elle, UK Edition, August, 98–99. Dacre, Karen. 2012. Teams behind the scenes who make magic on the catwalk. Evening Standard, 17/09/12. Available at: http://www.standard.co.uk/news/london/teams-behindthe-scenes-who-make-magic-on-the-catwalk-8144452.html. Flusser, Vilém. 2007[1983]. Towards a Philosophy of Photography. Trans. by Anthony Mathews. London: Reaktion Books Ltd.

602

Nathaniel Dafydd Beard

Frisa, Maria Luisa, Francesco Bonami and Anna Mattirolo (eds.). 2005. Italian Eyes: Italian Fashion Photographs from 1951 to Today. Milan: Edizioni Charta and Florence: Fondazione Pitti Immagine Discovery. [Exhibition Catalogue]. Gaines, Jane. 2011. Wanting to Wear Seeing: Gilbert Adrian at MGM. In Adrienne Munich (ed.), Fashion in Film, 135–159. Bloomington: Indiana University Press. Kansara, Vikram Alexei. 2011. Fashion 2.0 The Trouble with iPad Magazines. The Business of Fashion 23/05/11. Available at: http://www.businessoffashion.com/2011/05/fashion-2-0-thetrouble-with-ipad-magazines.html. Karaminas, Vicki. 2012. Image: Fashionscapes – Notes Towards an Understanding of Media Technologies and Their Impact on Contemporary Fashion Imagery. In Geczy Adam, and Karaminas Vicki (eds.), Fashion and Art, 177–187. London/New York: Berg. Hall-Duncan, Nancy. 1979. The History of Fashion Photography. New York: Alpine Book Company. Hall-Duncan, Nancy. 2010. Fashion Photography. In Valerie Steele (ed.), The Berg Companion to Fashion, 300–304. Oxford/New York: Berg. Lehmann, Ulrich (ed.). 2002. Chic Clicks: Creativity and Commerce in Contemporary Fashion Photography. Ostfildern-Ruit: Hatje Cantz Publishers and Boston, MA: Institute of Contemporary Art, Boston [Exhibition Catalogue]. Matt, Gerald A. and Synne Genzmer (eds.). 2011. Vanity: Mode/Fotographie aus der Sammlung F.C. Grundlach. Vienna: Kunsthalle Wien and Nürnberg: Verlag für moderne Kunst Nürnberg GmbH. [Exhibition Catalogue]. Matt, Gerald A. and Peter Weiermair (eds.). 2011. No Fashion, Please! Fotographie Zwischen Gender und Lifestyle. Vienna: Kunsthalle Wien, and Nürnberg: Verlag für moderne Kunst Nürnberg GmbH [Exhibition Catalogue]. Ramamurthy, Anandi. 2004. Spectacles and Illusions: Photography and Commodity Culture. In Liz Wells (ed.), Photography: A Critical Introduction, 193–244. London/New York: Routledge. Shinkle, Eugénie. 2008. Interview with Penny Martin. In Eugénie Shinkle (ed.), Fashion as Photograph: Viewing and Reviewing Images of Fashion, 113–124. London/New York: I.B. Tauris & Co. Ltd. Strugatz, Rachel. 2012. Burberry again tops digital rankings. Women’s Wear Daily 4/10/12. Available at: http://www.wwd.com/media-news/digital/burberry-again-tops-digital-rankings6383987. Stutesman, Drake. 2011. Costume Design, or, What is Fashion in Film? In Adrienne Munich (ed.), Fashion in Film. Bloomington: Indiana University Press. Symth, Diane. 2011. The future of fashion magazines on the iPad. British Journal of Photography 28/09/11. Available at: http://www.bjp-online.com/british-journal-of-photography/report/ 2112485/future-fashion-magazines-ipad. Uhlirova, Marketa and Tsilibaris, Christel. 2006. Fashion in Film Festival: Between Stigma and Enigma. London: Fashion in Film Festival. Viction:ary. 2011. MasterMind: Art Direction, Fashion Styling and Visionary Photography. Hong Kong: viction:workshop Ltd. von Olfers, Sophie and Eugenia Teixeira (eds.). 2010. Not in Fashion: Photography and Fashion in the 90s. Frankfurt am Main: MMK Museum für Moderne Kunst Frankfurt am Main and Bielefeld: Kerber Verlag. [Exhibition Catalogue]. Williams, Eliza. 2012. Heritage meets digital in new Burberry flagship store. Creative Review 14/ 09/12. Available at: http://www.creativereview.co.uk/cr-blog/2012/september/burberrymixes-heritage-with-digital-high-tech-in-new-london-store.

Nurit Peled-Elhanan

27 The de-humanization of Palestinians in Israeli school books: a multimodal analysis of layout, intertextuality and reading paths Abstract: The paper examines texts from two Israeli Civic Studies textbooks, one for mainstream secular high schools (years 11-12): Going the Civilian Way (2011) and the other – Values, Democracy and Judaism (2008), for year 9, an experimental textbook with a liberal-religious orientation. Both texts discuss the law regarding human rights to dignity and liberty in Israel and use multimodal means to justify possible transgressions from it, especially when Palestinians are concerned. I shall argue that these textbooks legitimate the state of exception of Palestinians under Israeli regime of occupation. The analysis uses the social-semiotic approach to multimodal analysis.

1 Background In spite of Israel’s success to generate recognition of its regime as a Democracy, it is often defined by researchers either as an “Ethnocracy” (Yiftachel 2006) or as an “ethnic Democracy” (Smooha 2002), where ethnicity1 and not citizenship is the main determinant for the allocation of rights, power and resources in Israel. Israel is defined legally as the state of the Jews wherever they dwell and not as the state of its citizens. Jews from other countries and Jewish settlers who live beyond the official border of the state have full citizenship rights while Palestinian citizens don’t, and Palestinians from the occupied West Bank – being subjected to a military government – have no civil rights at all and can be therefore regarded as living in a constant state of exception, deprived of all human rights and treated as “bare life”. A state of exception is a situation in which, by sovereign decision, a whole population is placed officially outside the law, is deprived of all civil and human rights and is treated with violence sanctioned by the ruler. People who live in a state of exception are juridically unable to defend themselves.2 And indeed Palestinians cannot defend themselves in Israeli courts.

 The study adopts Yiftachel's (2006) definition of ethnicity as ‘cultural identity, based on belief in common ancestry at a specific place’, Planning Theory 5(3) p. 216.  Agamben: Homo Sacer 1987; State of Exception. 2005.

604

Nurit Peled-Elhanan

2 The role of schoolbooks in the process of legitimation Schoolbooks as a genre of communication can be defined by their function: they are produced in order to instruct. Yet while they are commonly thought to do this on the basis of factual knowledge based around sound pedagogical principles a number of scholars have pointed to the way that they in fact are one key site where dominant ideologies are disseminated (Kress, 1979; Van Leeuwen 1996). Such ideologies seek to legitimate the interests of the ruling groups in that society, for example by naturalizing processes of capitalist exploitation of non western societies. Wertch (2002) has also discussed the role of school-books of history in terms of the way they serve to reproduce the national narrative at the expense of accuracy. In this paper I am interested specifically in the way that Israeli school books naturalize and legitimate the Zionist collective memory and create a Jewish national-territorial identity, which in fact entails the exclusion of Palestinians from the Israeli “lifeworld”. This “state of exception” is presented in the books as normative (Peled-Elhanan 2012). The non-representation of Palestinians in Israeli textbooks can be summarized under the four categories offered by Thompson (1987): 1. Legitimation: representations designed to immortalize dominance through its presentation as legitimate. These representations in schoolbooks are mainly maps, photographs, diagrams graphs and paintings, colour and layout. Each of these modes serves legitimation according to its affordances. For instance all maps show the whole region as the Land of Israel and conceal the existence of Palestine (Peled-Elhanan 2009). 2. Dissimulation: presentation of events from a single point of view. Israeli textbooks foreground the military-Zionist perspective that dominates all interpretations. Other versions regarding social and geopolitical facts (such as settlement, land rights, water disputes, cultural and social issues) are hardly ever considered (Podeh 2002). The history of Palestinians for the last 1500 years is never mentioned. 3. Fragmentation: separating people from places, describing the land while ignoring or concealing the existence of its indigenous population. Textbooks neither mention nor show Palestinians in context where they live and work. The appropriation of the land and the effacement of its population are usually supported by biblical phrases reiterating the divine promise to Abraham, to give his (Jewish) “seeds” the entire area. 4. Reification: “Representing transitory, historical states of affairs as if they were permanent, natural, outside of time.” (Thompson 1987: 521). Contrary to what is regarded as the case by international governments, Israeli schoolbooks present the occupation of Palestine as permanent both in maps and in the verbal texts (Bar-Gal 1993, Peled-Elhanan 2009). Jewish cities and settlements

The de-humanization of Palestinians in Israeli school books

605

that were built across the international border are presented as the natural homeland of Israeli Jews since the dawn of history. When Palestinians are depicted they are represented as a homogeneous group (“Israel’s Arabs” or “the Arab population/sector”), through demeaning stereotypes, in actions that are held in low esteem, whether subservient or vile (Figure 5). This representation connotes “natural” and cultural negative characteristics such as backwardness, clannishness, over-multiplication and terror. (Peled-Elhanan 2009, 2010, 2012).

3 Methodology: Social Semiotic approach to multimodal composition Social semiotics investigates how semiotic resources are used in specific historical, cultural and institutional contexts. The basic assumption of Social Semiotics is that meanings are made in signs or sign-complexes in distinct ways in specific modes (Kress 1993). Signs reflect ideology and are affected by use, namely by their history in a given culture (Kress 2010). Every sign is treated as motivated by interests, perspectives, values and positions of the sign maker regarding the message and its recipients, because signs are made to function in communication (Kress 2003). Reading and interpretation also reflect the interests and beliefs of the interpreter within a certain culture and history. Social semiotic inquiry is convenient for the analysis of textbooks because it is: not a pure theory or a self contained field […] it does not offer ready-made answers but rather provides ideas to formulate questions about human meaning making through sign making. […] Social Semiotics only comes to its own when it is applied to specific instances and specific problems and it always requires immersing oneself not just in semiotic concepts and methods as such but also in some other field[…] interdisciplinarity is an absolutely essential feature for a social semiotic analysis (Van Leeuwen 2005: 2).

Kress (2010) calls social semiotics a rhetorical approach to communication. The rhetor, he says, makes an assessment of all the aspects of the communicational situation: his or her interest, the characteristics of the audience and the semiotic requirements of the issue at stake and the resources available for making an apt representation; together with establishing the best means for its dissemination. However, in all societies there are “semiotic regimes” (Van Leeuwen 2005: 53), differential kinds of rules for the use of semiotic resources. These rules may receive their legitimation from impersonal authority such as tradition, scriptures, laws or ideology, but they all need agents to implement them, and therefore there is always room for personal interpretation. What this means in practical terms, as I will illustrate, is that all kinds of content and design decisions in textbooks are indicative of ideological choices.

606

Nurit Peled-Elhanan

This can include what kinds of photographs are included and the way the participants are represented or who is not represented. It can also include things like where things are positioned on the page which suggests kinds of reading paths and hierarchies.

4 The relationships between words and image in the textbook Different semiotic modes such as image, writing or vocal modes, can realize the same roles by different means through different observable forms (Van Leeuwen 2005: 75).

The more recent tradition of social semiotic scholarship draws in the concept of “mode”. This refers simply to different modes of communication. Writing would be one mode. The photograph and page layout would be other modes. What is important for social semiotic analysis is the way that different modes can perform different kinds of communicative roles. Though words and images may be parts of the same message, “the world narrated is different from the world displayed” (Kress 2003: 2). The motivation to use one mode or another is their aptness, or their being “best fit” for the occasion (Kress 2010). Verbal language is typonomic while image is toponomyc (Lemke 2001). Writing or natural language can name, label, categorize but cannot express intensity, movement or degree as subtly as image can. The relationships between text and image were defined by R. Barthes as either “anchorage” or “relay” (1977: 39). In Anchorage words elucidate pictures. As Van Leeuwen puts it, “the text directs the reader through the signifieds of the image, causing the viewer to avoid some and receive others,” and “remote-controls him towards the meaning chosen in advance” (Van Leeuwen 2005: 229). In relay “text and image stand in complementary relationship and are both fragments of a more general syntagm to which each contributes its own distinct information” (Van Leeuwen 2005: 229). Images frequently serve as elaborations, adding more details, place, action and object-signs such as clothes or tools (Kress and Van Leeuwen 2006). Another possibility (Kress and Van Leeuwen 2006) is “Integration”: Text and picture can be integrated into one another or overlap. Some texts in Israeli school books, for example, are written over photographs such as soldiers coming or going to war, a historic event such as the declaration of the state, a symbolic image such as the flag, or against the background of Greater Israel. Such complex signs are “palimpsests” – two overlapping texts that create a new meaning. The overlapped photograph serves as a cohesive device that connects together all the elements of the page and gives them new meaning as one semiotic unit. Here are some examples:

The de-humanization of Palestinians in Israeli school books

607

“Anchorage”: In a history textbook The 20th century (1998: 195) there is a photograph of a flooded street in what seems like a poor neighbourhood, defined by its caption as “The Palestinian Problem”. Nothing in the photograph itself could have told the viewer he or she is watching “the Palestinian problem” for although it shows a problem – an environmental one at that – there is nothing “Palestinian” about the image – neither Palestinian people nor Palestinian landscape or houses are shown, only a flooded street in a poor neighbourhood that could be anywhere in the world. “Relay”: In the Geography textbook People In Space (1998) we find four pictures whose collective title is “traditional life in the non-Jewish population”. NonJewishness is exemplified by the image of a “primitive farmer” and a traditional Arab village. From a social semiotic point of view, the question is, whose interest was it to show “Non-Jewishness” in this manner and why? Integration will be discussed in the following section.

5 The multimodal legitimation of Palestinians’ state of exception 5.1 Values, Democracy and Judaism (VDJ) The first text book I analyze is Values, Democracy and Judaism (VDJ), published in 2008, and designed for years 11–12. It is defined on the cover as part of an “interdisciplinarian program in cultural Jewish heritage and Democracy studies” which is a part of a more comprehensive program of the Hartman Institute called “Revival of the teaching of Jewish studies in state high schools”. The program is sponsored by the ministry of education with the aim of enhancing Jewish studies in secular schools in order to “anchor” universal modern values of Democracy in Jewish thought. In this spirit, the book is composed of biblical and Talmudic extracts of laws and anecdotes that exemplify Jewish humanistic stance towards “others”, including slaves and foreigners, alongside extracts from modern Israeli law that correspond to these tenets. I will now give an overview of the section from this book specifically about human rights to dignity and liberty. In this section I show how the analysis of the layout itself allows us to reveal its deeply ideological nature. Layout is regarded by Kress and Van Leeuwen (1995) as text or rather as intertext, namely as one semiotic entity. By assembling different texts and visuals on the same page, double-spread, or even whole chapters and books, layout brings “the process of semiosis […] which is always multimodal […] to a temporary standstill in textual form” (Kress 2000: 141, 152). Structurally, elements in layout can be “disconnected, marked off from each other, or connected, joined together” (Van Leeuwen and Kress 1995: 33). But what may seem as collage or bricolage is actually

608

Nurit Peled-Elhanan

a transformation of ideologically motivated signs into new meaningful wholes or new “complex signs”. This transformation is achieved through the three signifying systems that are involved in Layout: information value, framing and salience, “all serving to structure the text, to bring the various multimodal elements of the multimodal text together into a coherent and meaningful whole.” (Van Leeuwen and Kress 1995: 26). I say a little more about each of these and then show how they allow us to look at the way they help us to read the text books.

5.2 Information value, salience and reading paths Placement of elements in a layout endows them with specific information values that are attached to the various zones of the visual space (Van Leeuwen and Kress 1995: 26). A given element does not have the same value and meaning when placed on the right or on the left, in the upper or in the lower section of the page, in the centre or in the margins. Each of these zones accords specific values to the element placed within it. When a layout polarizes left and right, placing one kind of element on the left and another, perhaps contrasting, element on the right, the elements on the left are presented as Given (in Hebrew the elements on the right are presented as given, since writing runs from right to left). For something to be “given” means it is presented as something the reader already knows, as a familiar and agreed upon departure point for the message. For something to be “New” (right in English, left in Hebrew) means it is presented as something not yet known to the reader, and hence it becomes “the crucial point of the message, the issue to which the reader must pay special attention. The New is therefore in principle problematic, contestable, the information at issue, while the Given is presented as commonsense and self-evident” (Van Leeuwen and Kress 1995: 27). Van Leeuwen and Kress explain that “Such structures are ideological in the sense that they may not correspond to what is the case either for the producer or for the consumer of the layout” (1995: 27). The information is presented as though it had that status or value for the readers, who have to read it within that structure, even if that valuation may then be rejected by a particular reader. When a layout polarizes top and bottom, placing different, perhaps contrasting elements in the upper or lower parts of the page, the elements placed on top are presented as the Ideal and those placed at the bottom as the Real. For something to be Ideal means that it is presented as the idealized or generalized essence of the information, and therefore as its ideologically most salient part. The Real is then opposed to this in that it presents more specific information (e.g. details) and/or more “down to earth information” (e.g. real data) and/or more practical information (e.g. practical consequences, directions for actions etc.) (Kress and Van Leeuwen, 1995: 28). The connections between the elements of Layout are expressed in vectors, or in repetitions of shapes and colours, which convey that the elements thus con-

The de-humanization of Palestinians in Israeli school books

609

nected are to be read as belonging together in some sense, as continuous or complementary. A prototypical vector would be an arrow, but other visual elements can fulfill the same function as we will see, such as the gaze of participants for example. The vectors may be oblique, and they may connect elements on one page, on a double-spread or on two sides of the page (Van Leeuwen and Kress 1995). Multimodal texts, such as schoolbooks, open the question of directionality and the question of “point” (Kress 2003: 157) namely of “reading path”. Whereas in verbal texts – typical of religious textbooks in Israel where the world is told in one version, and presents only one truth – there is a distinct and strict “reading path” students must follow, multimodal reading enables reading as design (Kress 2010). It cannot constraint the viewer to a pre-established reading path, rather the viewer is following a to-be-constructed path of reading. However, the page or double spread – and the whole textbook – is always organized according to the educational interests of the writers. The section I would like to discuss is about human dignity and liberty. It is more than 60 pages long. Every double-spread in this section is divided into two distinct part: a Talmudic story regarding human rights, appears usually on the right side of the double-spread which is the point where the message starts, or the “given” side in Hebrew, and is therefore preserved for things past and for known information. On the left side of the page or double spread – which contains the “new” information or the point where the message is going in Hebrew, we find the modern implications of these rights in democracies all over the world. This way the reader is constantly reminded that Jewish law preceded international laws and practices and served as their hypotext. This organizing principle may imply that since all universal moral values stem from “our” culture, a Jewish democracy cannot but uphold the highest ethical and moral standards. The first double spread starts with the idea of Man being born in the image of God, with quotes from the Bible on the right and then moves on to the implication of this principle in other democracies. On the left hand side of this page (New) there is a colorful column where we see at the top Leonardo Da Vinci’s drawing of the human body, followed by the hand of the statue of liberty, followed downwards by a photograph of a blond girl helping a black girl read, and again Da Vinci’s drawing. The headings between the images are “Human Dignity, Liberty, Equality”. This page, as one complex sign, creates a sense of universal enlightment that follows from the right page, namely from “our” Hebrew Bible. The next double spread discusses again (on the right “given” side) Man in the image of God, with many quotes from the Bible. In strongly framed margins we see on the right Michelangelo’s painting of God touching the hand of Man, and on the left margin quotes from Emanuel Kant. At the bottom of the page – where hell should usually be – we find the painting of Hieronymus Bosch: the garden of terrestrial pleasure. The biblical message is the unifying centre of all these

610

Nurit Peled-Elhanan

Fig. 1: VDJ p. 17.

representations of the western world. On the third double spread we find a story about Rabbi Akiva (1st–2nd centuries B.C.) who coined the phrase “love thy neighbour” – which in Hebrew reads: Love thy friend – and died as a martyr refusing to obey Roman command to stop studying the Torah. Another shorter story about another Rabbi who died as a martyr also follows. The main text deals with this rabbi’s rulings against offending one’s fellow man. Above this text one sees a photograph of an Israeli soldier in an army post, leaning over his gun to shake the hand of a little boy. The photograph seems like the modern realization of the “ancient” text. Finally the students are asked to compare between the rabbi’s teaching and Kant’s philosophy. The left (New) page is an example of the ideological meaning of “integration” in layout. A text written by Supreme judge Aharon Barak, asserting that human rights to dignity and liberty were prescribed in “our” Bible and had passed from the Hebrews on to the Christian philosophers and to the whole world, is overlapping a photograph of what seems like a rebellion of slaves. The book does not give the details of the picture,3 thus turning it into an icon – the struggle for human rights or the embodiment of the biblical laws regarding the rights of slaves. The photograph actually shows a rebellion of gold diggers in Brazil where a half naked black man grasps a rifle directed to him by a policeman and all the diggers around him look at them in amazement. Both Barak and the digger are strongly framed in the left (“New”) margin as if belonging to the same “order”. The gold digger is bursting out of the photo that is otherwise blurred by the text of Barak, so that he is the most salient part of the page and therefore the first item to attract readers’ attention and determine the reading path of the whole page. As  The picture is called: Gold diggers in Brazil. Copy rights Sebastiao Salgado Contact Press images 1998.

The de-humanization of Palestinians in Israeli school books

611

Van Leeuwen and Kress explain, “Salience does not only have an aesthetic function; it also plays a vital role in structuring the message” (1995: 34). Between the top of the page – the great judge (representing the “ideal” theory) and its bottom – the freedom fighter (the “real” action) lies a short biography of Barak. Nothing is said about the gold digger, he is but an illustration or the realization of Barak’s or rather of Israeli – Jewish creed. The following 30 pages cover stories of the struggle of minorities for liberty and equality, all over the globe. The struggle of the Palestinians is never mentioned. Humanity according to this book is made of Jews and other people – mainly Westerns, some Africans, Americans, but not Arabs. The 60 pages that deal with human rights to dignity and liberty seem like a long prelude for a sub-chapter titled: Basic laws are laws of compromise, in the second part of the book: relationships between majority and minority. This section asserts the moral right of Israel to violate human rights when “terrorists” or their supporters are concerned. It starts with another text by judge Aharon Barak who affirms the right of the state to violate the law of human dignity and liberty in order to protect itself: Human dignity mentioned in the basic Law refers to the dignity of man in an organized society which has national goals and in which the government is functioning. […] Human rights are not a stage for national extermination. A constitution is not a recipe for national suicide (Barak, VJD, p. 56).

Below this text, at the top of the next page, we see a photograph with a caption: “The two terrorists captured in the bus no. 300 affair.” The details of the affair are not given: on 12. 4. 1984 a bus was high-jacked by 4 Palestinians. Two died as the army attacked the bus, along with an Israeli soldier girl, the other two were murdered on the way to prison. The unit reported that they died in battle but this

Fig. 2: The two terrorists that were captured in the Bus 300 affair (JVD p. 57). Courtesy of the photographer Shmuel Rachmani.

612

Nurit Peled-Elhanan

picture, along with another one, published a few days later, contradicted the report. The paper that published the photograph was closed-down for bypassing censorship. When the “affair” was exposed, all the people involved received presidential amnesty without a trial. The picture shows General Itzhak Nordechai, then chief paratrooper commander, holding his gun in one hand and a young boy in another, running and shouting. Another young boy is next to the first one, head bowed, held by secret service man Ehud Yatom (who murdered the two boys later that night beating them with stones and an iron pole); a third soldier pushes this boy from behind. The photograph shows hectic movement, and if we follow the eyes of Mordechay he seems to be looking straight at us, or at the photographer, probably telling him not to takes photos. For Israeli readers the photograph contains object-signs that suggest a certain interpretation: the paratroopers – Israel’s greatest role models – and the figure of Mordechay who is an official hero – inspire confidence in the just cause, although it involves the violation of human rights. The caption leaves no room for empathy with the Palestinian teenagers; it doesn’t even mention the fact that they were very young. In Israel Palestinian children who disrupt the order are merely “terrorists” and are treated as such.4 Without this verbal and visual guiding and in another environment it is not certain that a photograph of fully armed big soldiers pushing two slender teenagers would inspire sympathy towards the capturers and not towards the captives. In another, less biased context, the text could have confronted the Israeli students’ motivation to serve in the army with that of the Palestinian youngsters, who saw themselves as freedom fighters equally motivated to serve their people and their cause. But for Israeli readers the small thin figure of the Palestinian boy, dressed in tight jeans and jacket is nothing keen to us, nothing but a “terrorist”. The reading path of this page is obviously from bottom up: as in the previous example, starting with the photo – a real shocking action – the reader looks upward, for justification or explanation and there she finds Barak’s words that serve as a legitimating theory to the picture. The photograph – like the gold diggers’ revolt – is but an example of such a confrontation and serves only as illustration to Barak’s statement – this is what a confrontation between human rights and the safety of the state looks like. The text plays “the ideological lead role, and the pictures play the subservient role, which however is important in its own right as specification, exemplification, evidence, practical consequence.” (Kress and Van Leeuwen 1995: 29). The division of labour between text and image on this page is clear: the intensity, the movement, the tension of the “affair”, along with the special “ambiance”

 See Breaking the Silence: Our Harsh Logic: Israeli Soldiers’ Testimonies from the Occupied Territories, 2000–2010. Metropolitan Books N.Y. New-York.

The de-humanization of Palestinians in Israeli school books

613

of this nightly operation are all expressed in the image which is the specification, the evidence, while the verbal text gives the theory, the legal and moral justification for the action, instructing us – along with the caption – how to read the image: who are the heroes and who are the villains. The photograph is black and white like photographs that are taken for internal use only, as evidence, to be put in secret files for later discussion. Hence, it is the utmost expression of “authenticity” and truth. In sum, the whole page as one complex sign seems to indicate that these boys deserve no justice. They are the exception to the law and provide the possibility to transgress it. As in the previous example, on top of the picture there are some questions that monitor and control students’ reading: “What is the condition to violate the law of human dignity and liberty? Give example from reality in which the power of the state is confronted with human dignity. How would you solve this confrontation?” The social semiotic question this layout poses is why was this photograph chosen? What does the Hartman Institute, that professes liberal Judaism, say with this page to its readers, young Zionist teenagers about to become fighters themselves, imbued with the same faith that they are going to protect their people? The motivation for choosing such a photograph could have been to let the students know, after 60 pages of examples from Jewish sources and Israeli laws that command respect for human rights that extreme cases call for extreme measures. The meaning of the layout of the double spread: on the left page of a double spread is the “new” information – the affair of bus 300. On the right page – the “given” part reserved for things past, we read a Talmudic anecdote about a 16th century dispute between the rabbis from Safed – who were the majority – and the minority of rabbis from Jerusalem; the minority rabbis persuaded the majority they should have consulted with them before reaching a certain decision. Under the story there is a colorful photograph of the city of Safed by a 19th century French photographer, Bonfil. one can see that the photograph was painted over according to the Orientalist European vision of the Middle East: huge saturated-green olive trees and in between them a figure of an Arab shepherd seen from behind, wearing a long red kaffiyah, and at the distance a blurred figure of a child at the foot of a tree. The difference between the two photographs is ideological and may be explained by Barthes’ observation: Color is an artifice, a cosmetic (like the kind used to paint corpses). What matters to me is not the photograph’s “life” (a purely ideological notion) but the certainty that the photographed body touches me with its own rays and not with a superadded light (Barthes 1980: 81).

The “authentic” true, uncoloured photo of the terrorists and the soldiers contrasts the long-lost pastoralia of the other photograph. The two parts of the double-spread differ on many levels: while the shepherd in the picture is going into the landscape, the “terrorist” boys are being pushed

614

Nurit Peled-Elhanan

Fig. 3: Things past.

The de-humanization of Palestinians in Israeli school books

615

out. The olives and shepherds are symbols of dignity and freedom and ancient roots, peace, stability, and the proper cycle of life – in other words the old world. The captive boys are the disruptive element of this old world as they are of our democracy that tries to follow ancient humanistic laws. Regarding top and bottom, the two texts – the victory of the minority rabbis over the majority and the text affirming the rights of the dominant authorities to transgress laws of human rights – occupy the upper parts of their respective pages but they differ from each other in mode. The biblical and Talmudic texts on the right page are reproduced in their original fonts, which entails the sanctification of the textbook, for no book containing holy script can be ever discarded or maimed in any way. The left page, in constrast, is written in unmarked regular font, contains modern legal truth. The meaning of the double spread is created by the dialectics between the two. The pictures, of the almost fictional orchard and of the authentic “operation” occupy the bottom centre of their respective pages. Therefore we can assume they are connected with strong vectors (Kress and Van Leeuwen 2006) that may suggest two contrastive stages of the story of the land: the same country now and then, old and new, stability and its wrecking; respect for human dignity and freedom versus its violation from all sides. This may also explain the fact that the “affair of bus 300” is not detailed in the text, because the photograph should function as an icon symbolizing present day “situation” and not a particular case.5

6 A defensive democracy My second example of the legitimation of Palestinians’ state of exception is from the civic studies textbook Going the Civilian Way (2012). This book, for years 11–12, dedicates 200 pages to the idea of democracy and 200 pages to Israel as a Jewish democratic state. Pages 183–189 deal with the idea of defensive democracy which is a milder variant of the German wehr-hafte Demokratie – a fighting democracy, and bring Israel as an example of such a regime. I will analyze the layout of the double spread 186–187 – which is part of the chapter titled: The right of Democracy to protect itself. Page 186 contains only a verbal text which imposes one reading path: a definition of Israel as a defensive democracy and the list of the licenses this definition entails. The text explains that a defensive democracy may use any measure it sees

 It is worth noting here that this version of the book was not authorized by the Ministry of Education and the new edition of VDJ omitted this sub-chapter altogether and replaced it with animal rights.

616

Nurit Peled-Elhanan

Fig. 4: A Check Point on the road from Modiin to Jerusalem.

The de-humanization of Palestinians in Israeli school books

617

Fig. 5: Kalandya checkpoint, courtesy of Tamar Fleishman.

fit in order to protect its citizens against potential terrorists and their supporters. The practices of Occupation such as Check-points, targeted assassinations, administrative detention (“detaining a person without trial, without the possibility to meet a lawyer for an indefinite span of time”), house demolition, confiscation of land, curfew and the use of mild physical and psychological pressure (“not torture”) are thereby legitimated. On the right margin the students are asked to look at newspapers and internet sites for more information about “security censorship” and find one advantage and one disadvantage in its use. Facing this theoretical background (on p. 187 – the “New” part of the double spread) there is a photograph of a checkpoint (Figure 4). It is a “flying check point” – placed at random without warning. But the book does not explain that. The checkpoint is situated in the middle of nowhere: several concrete blocks, loosely placed, against which two groups of people are leaning leisurely, talking. Some of them are police, some soldiers. One sees an open empty police van, the flag of Israel on top of a canteen and a bare slope of a hill in the distance. The absence of the people against whom this checkpoint was placed creates a “blind spot” in which their existence is only assumed (Barthes 1980: 855). The motivation for choosing this picture may be the wish to present the checkpoint as a nonthreatening preventive measure, to reassure the future soldiers that “doing checkpoints” is not that bad. The enemy is not to be seen, may be far away, hiding among the rocks, but we are alert. As in the previous example, there are leading questions above the picture: “What in your opinion is the purpose of the checkpoint? How does it represent the defensive democracy?” The reality of permanent checkpoints in Israel is quite different, but photos such as these are not shown for various reasons. One reason may be that it would be confusing for the students to see the “enemy” as PLU, (Person Like Us).

618

Nurit Peled-Elhanan

Fig. 6: Kalandya checkpoint, courtesy of Tamar Fleishman.

The students may realize that in spite of the unbearable crowdedness and stress in the tiny space between prison like cages and automatic iron gates, the people seem self-possessed “rational and educated, possessing the positive characteristics Israeli students tend to associate with their own collectives, a […] civilized, stereotype-busting native.” (Rabinowitch 2001: 76). The people represented above don’t look like people who are potentially “blood thirsty desperados, yearning for violent revenge, pushed to act against their own interests if they can only harm as many Israelis as they can” (ibid.). Such a picture can raise questions Israeli education would rather avoid: Is this what terrorists look like? How can you tell? Is everyone on the street may be a terrorist, children and babies included? Check points don’t have to be crowded in order to look as menacing as they really are. The following photograph (Figure 7) – which does not feature in any school book – arouses as much awe, fear, a sense of imprisonment and the imagined memory of a concentration camp which calls to mind not a defensive democracy but a brutal oppressive dictatorship. The absence of people – both of soldiers and of potential “terrorists” – creates a disturbing “blind spot” in which fear rules. Israeli children, being traumatized year after year by horrifying pictures of Nazi camps can again press some very disturbing questions in view of such photographs.

The de-humanization of Palestinians in Israeli school books

619

Fig. 7: http://www.machsomwatch.org/reports/checkpoints/27/07/2012/morning/21386.

Returning to the page in question, the book relates right under the checkpoint as part of an “exercise” the story of Mordechay Vaanunu who violated “security censorship” and sold the secrets of Israel’s nuclear weapons to the London Times back in 1986 for which crime he was sentenced to life imprisonment and then to house arrest, until today. The students are asked to enumerate the human rights that were denied Vaanunu and to define his crime in legal terms. Both pages have large colorful margins. As Kress and Van Leeuwen explain, “Just how marginal margins are will depend on size, and, more generally, the salience of the centre” (1995: 30).” The left margins, next to the checkpoint and Vaanunu’s story, seem very cheerful and childish. At the top, an icon that appears on every page: a bunch of Western – looking laughing children who seem very happy to learn about the defensive democracy they live in. A mischievous monkey and the wise owl trying to learn the facts of life. The owl learns about administrative detention and has to enumerate at least one advantage and one disadvantage of this practice. The monkey seems puzzled over the nuclear device, and doesn’t know which button to press, probably an allegory of Vaanunu himself. What is the meaning of this layout as one complex sign? Whose interest was it to present a regime of closure, check points, administrative detention, lifeimprisonment, torture and targeted assassinations in such a light, cheerful way? What is this page teaching? Since the audience of the book are the future soldiers of this regime, the interest of the writers may be to ease their way into military service and convince them that what they will be required to do is perfectly legal and necessary for Israel’s safety. Vaanunu reminds them that Loyalty to this regime will save them from trouble. Presenting the practices of occupation like an episode from Winnie the Pooh with Owl and a monkey (piglet could never feature in a Jewish textbook), laughing, happy, secure children and “cool” soldiers makes it seem alright.

620

Nurit Peled-Elhanan

7 Intertextuality and the importance of representation Every text, as a “fully functioning semiotic resource must have the potential to meet 3 demands: to represent states of affairs in the world (ideational function), to represent or establish relations between reader and writer, speaker and listener (interpersonal function) and to represent all this as a message-entity namely as text which is internally coherent and coheres with its environment (textual function).” (Kress 2010: 87). In these respects there are similarities between the two texts analyzed here. “Ideationally”: both texts represent general realities and use specific events and specific people either as examples to the rules or as “authority legitimation” (Van Leeuwen 2007). In both cases the authority of judge Barak legitimates the act of breaching human rights. “Interpersonally”: Both texts are authoritative. Supporting their teaching by sacred scriptures and legal documents, the texts seem to demand reader compliance; scriptures and Jewish “sources” are venerated even by Israeli secular people who accept them as the “ultima verba” on every subject. The assumed readers’ compliance is also evident in the leading questions that require students to reiterate and reinforce the arguments of the chapter. “Textually”: both texts are mixed genres or “blendings” (Kress 2010), where different legitimatory discourses of authority are woven (Van Leeuwen 2007, PeledElhanan 2010). These “blendings” contribute to the authenticity, the solemnity, the reliability and the validity of the political-educational message, which is complemented and reinforced by photographic “evidence”. Fairclough (1989) suggests that we should describe and interpret texts within the context of their site, and then explain them by placing them within their larger social context, through intertextuality. Both texts examined here seem to suggest that “the boundaries of the text are not the boundaries of the genre” (Kress 2003), for they rely for their full meaning on intertextuality with political speeches and journalistic reports. They both relate cases that demonstrate the crucial role of representation. The two stories – the unmentioned one of bus 300 and the story of Vaanunu – confirm what Kress (2010) mentions in relation to the scandal around the photos of prisoners from Abu Gharib. The outrage he says, was against the photographs and their publication and not against the practices of torture: Outrage at the breaking of representational convention was treated as more serious and overshadowed outrage at the breaking of humanitarian conventions on the treatment of prisoners (Kress, 2010: 25).

Although neither bus 300 affair nor Vaanunu’s report was fabricated, the outrage was at their publication, at the violation of the norm: never to show Palestinians’

The de-humanization of Palestinians in Israeli school books

621

suffering or Israeli crimes, and not at the criminal acts of the state and its army. In both cases the only party that was punished was the messenger – Vaanunu and the newspaper that published the photographs of the captured Palestinians in bus 300. As Vaanunu’s lawyer, Michael Sfard, insisted, “In Israel of 2007 a man was convicted for talking (to journalists), not for the content of the talk but for the very act of talking. This reality does not become a democratic state.” This reality is not discussed in the books. However, both texts are significantly intertextualized with relation to a current affair in which thousands of classified documents were given to the journalist Uri Blau by ex-soldier Anat Kam. The documents revealed that the Israeli Defense Forces had committed war crimes, suggesting that the military had defied a court ruling against assassinating wanted militants in the West Bank who could potentially be arrested safely, just like the boys in bus 300. According to Uri Blau (Haaretz, 28 November 2008), the former commander of the occupied territories, General Yair Naveh, was quoted saying: “Leave me alone with the Supreme Court, I want the “wanted” to be killed instantly without messing around”. The state decided to prosecute Kam and Blau for the very serious crimes of espionage: passing on classified information with the intent of harming state security, charges which carry a maximum sentence of life in prison. The criminal generals had not stood trial.6 This intertextuality suggests that as a rule, crimes of representation are more severe than crimes against humanity in Israel.

8 Conclusion Giorgio Agamben maintains that: “The state of Israel is a good example of how when the state of exception is prolonged then all democratic institutions collapse. This is also what happened in the Weimar Republic”.7 But Israeli books try to prove in a multimodal fashion that nothing collapses and that dictatorial practices can be justified in a democracy. The religious book does it by establishing that the democratic institutions of Israel are well based on biblical and Talmudic laws of human rights which became the laws of the enlightened world; the secular textbook by explaining that modern laws of human rights allow and even decree some transgression in times of necessity. In order to legitimate Palestinians’ state of exception in a digestible way none of the texts present Palestinians as Persons Like Us – or as an integral part of our democratic society. Unlike other oppressed people in the world they are nothing but “the enemy” visible or invisible, repre http://www.haaretz.com/news/anat-kam-i-stole-idf-documents-to-expose-west-bank-war-crimes1.284112.  Giorgio Agamben. The State of Exception – Der Ausnahmezustand. Lecture at European Graduate School. August 2003. Transcription by: Anton Pulvirenti. I borrowed this term from G. Agamben because he himself uses Israel as an example.

622

Nurit Peled-Elhanan

sented either by images that create “blind spots” where their existence is only guessed or by photographs that depict them as “evidence” to terrorism, not as human beings we can identify with or wish to be close to. This representation keeps them as far as possible from the Jewish students, about to be drafted to the army, where they will be required to commit actions that contradict all the values and norms they have been educated to respect, especially those regarding human rights of freedom and dignity. However, it seems that by raising these issues the authors mean not only to legitimate Palestinians’ state of exception but also to teach the future citizens how to defend it against accusations from human rights champions, to use political and military argumentation, exemplified in the words of Foreign minister Zipi Livni during the raid on Gaza in 2009. Livni spoke of the “legitimate rights of Israel with respect to international law”. Referring to world criticism of Israeli policy, the foreign minister said she was “ready to receive the criticism of the international community in every matter, but wants the evaluation to be legitimate”. Just like the school books, she claimed that “We share the values of the free world, but there is a tremendous gap between the true values of the state of Israel and its image in the world. [The] legitimacy of Israel is derived from its core principles as a Jewish and democratic state.”8 As Coffin has stated regarding Australian schoolbooks, History Israeli textbooks teach the students the discourse of power, of politicians and generals, and put at stake “the disciplinary politics of truth” (Coffin, 1997: 201).

References Agamben, Giorgio. 1987. Homo Sacer: Sovereign Power and Bare Life. Meridian: Crossing Aesthetics. Agamben, Giorgio. 2005. State of Exception. Chicago and London: Chicago University Press. Bar-Gal, Yoram. 1993a. Moledet and geography in a hundred years of Zionist Education. Tel-Aviv: Am Oved Publishers. Bar-Navi, Eli. 1998. The 20th Century: A History of the People of Israel in the Last Generations, for Grades 10–12. Tel Aviv: Sifrei Tel Aviv. Barthes, Roland. 1980. La chambre claire, Oeuvres Complètes, vol. V. Paris: Editions du Seuil. Coffin, Caroline. 1997. Constructing and Giving Value to the Past. In Frances Christie and Jim Martin (eds.), Genres and the Institutions, 196–230. London: Continuum. Fairclough, Norman. 1989. Language and Power: Meeting Papers, 1989 (British studies in applied linguistics). Centre for Information on Language Teaching & Research. UK. Geldi, Bina, Nisan Nave and Asaf Matzkin. 2012. Going the Civilian Way. Tel-Aviv: Reches Publishers (in Hebrew). Hefetz, Irit and Yochay Eden. 2009. VDJ (ADI) – Values, Democracy and Judaism. [for year 9. Experimental edition] Jerusalem: Hartman institute (in Hebrew).

 The Jerusalem Post 26 March 2008.

The de-humanization of Palestinians in Israeli school books

623

Halliday, Michael. 1985. An Introduction to Functional Grammar. (1st edition). London: Arnold. Kress, Gunther. 1993. Against Arbitrariness: The Social Production of the Sign as a Foundational Issue in Critical Discourse Analysis. Discourse in Society 4(2): 169–91. Kress, Gunther. 2003. Literacy in the New Media Age. London: Routledge. Kress, Gunther. 2010. Multimodality A Social Semiotic Approach to Contemporary Communication. London: Routledge. Kress, Gunther and Theo van Leeuwen. 1996/2006. Reading Images: The Grammar of Visual Design. London. Routledge. La Capra, Dominick. 2001. Writing Shoa, Writing Trauma. Baltimore: Johns Hopkins University Press. Lemke, Jay. 1998. Metamedia Literacy: Transforming Meanings and Media. In David Reinking, Linda Labbo, Michael McKenna and Ronald Kiefer (eds.), Handbook of Literacy and Technology: Transformations in a Post-Typographic World, 283–301. Hillsdale: Erlbaum. Peled-Elhanan, Nurit. 2009. The Geography of Hostility and Exclusion. Visual Literacy 27(2): 179– 208. Peled-Elhanan, Nurit. 2010. Legitimation of massacres in Israeli school history books. Discourse & Society 21(4): 377–404. Peled-Elhanan, Nurit. 2012. Palestine in Israeli School-books. Ideology and propaganda in education. London: I.B. Tauris. Rap, Esther and Tzvia Fine. 1996/1998. People in space. Tel Aviv: The Centre for Educational Technologies Pub. Rabinowitch, Dan. 2001. Natives with jackets and degrees. Othering, objectification and the role of Palestinians in the co-existence field in Israel. Social Anthropology 9(1): 65–80. Smooha, Sammy. 2002. The Model of Ethnic Democracy: Israel as a Jewish and Democratic State. Nations and Nationalism 8(4): 475–503. Thompson, John B. 1987. Language and Ideology: A Framework for Analysis. The Sociological Review 35: 516–535. Van Leeuwen, Theo. 1996. The representation of social actors. In Carmen Caldas-Coulthard and Malcolm Coulthard (eds.), Texts and Practices: Readings in Critical Discourse Analysis, 32– 70. London: Routledge. Van Leeuwen, Theo. 2000. Visual Racism. In M. Reisigl and Ruth Wodak (eds.), The Semiotics of Racism, 333–350. Vienna: Passagen Verlag. Van Leeuwen, Theo. 2005a. Introducing Social Semiotics. New York: Routledge. Van Leeuwen, Theo. 2005. Multimodality, Genre and Design. In Sigrid Norris and Rodney Jones (eds.), Discourse in Action, 73–95. London: Routledge. Van Leeuwen, Theo. 2007. Legitimation in Discourse and Communication’. Discourse & Communication 1(1): 91–112. Wertsch, James. 2002. Voices of Collective Remembering. Cambridge: Cambridge University Press. Yiftachel, Oren. 2006. Ethnocracy – Land and Identity Politics in Israel/Palestine. Philadelphia: University of Pennsylvania Press.

William Cannon Hunter

28 Visual communication in tourism research: Seoul destination image Abstract: Visual methods are belatedly gaining traction in tourism research. These methods, including semiotics, content analysis, drawing and mapping are being applied to decoding the highly visual narratives found in print and online tourism promotion materials and in tourists’ user generated content. These narratives reveal how tourism destination image is formed by the perpetual and cumulative circulation of photographic representations by multiple parties. In this chapter the formation of Seoul, South Korea destination image is discussed in terms of basic tourism theory, empirical methods and outcomes. This case study demonstrates the effective fit between tourism theory and visual methods that is increasingly recognized by tourism theorists, researchers and practitioners. The step by step presentation of the research makes this case study an ideal guide for students and researchers new to visual methods in communication.

1 Introduction Semiotics, content analysis, drawing and mapping, and other visual analysis methods are increasingly used in multidisciplinary tourism research for understanding the highly visual content in print and online tourism promotion materials and in tourists’ own narratives. The growing use of these methods parallels the social and technological developments in print and especially online communications that have made international tourism an increasingly visual discourse. In tourism research the representation of destinations and tourism experiences to actual and potential visitors has been a key point of discussion for some 30 years. Notions concerning the tourist gaze, arrangements and marking of attractions and the photograph have been discussed individually and in conjunction with a theory of destination image. In this chapter, theoretical perspectives in tourism research related to visual communication are outlined and an application of practical research techniques is illustrated via the analysis of Seoul, Korea destination image. This example demonstrates the effective fit between tourism theory and visual methods that is increasingly recognized by tourism theorists, researchers and practitioners.

626

William Cannon Hunter

2 Tourism theory and visual communication Visual communication in tourism is associated with the fundamental problem of how the tourism phenomenon alters or magnifies social reality through representations. Tourism as a cross-cultural, transitory and commercial exchange combined with its economic impact makes communication in any form simultaneously highly important and highly suspect. Early research in tourism focused on trying to describe the language of tourism as a discourse using verbal slogans and declarative statements (Dann 1996a) and visual representations (Edwards 1992). These were thought to be the products of destination marketers intended to capitalize on and direct the tourist’s gaze (Urry 1990) toward certain products and experiences. Tourism itself was described as part of a larger modern-industrial spectacle, filled with pseudo-events (Boorstin 1992; Debord 1995). Some researchers considered tourism to be a highly suspect phenomenon that transforms the reality of a destination’s most typical or recognizable features into goods or experiences that can be purchased by the visitor (Prideaux 2003). Problems concerning the arrangement of social spaces through tourism – using markers, signs and monuments (MacCannell 1973) – were eventually superseded by the problem of the photograph which gradually became understood as a sort of meta-representation of place. The photograph gained theoretical importance as a key indexical sign – light writing – of touristic reality. Its verisimilitude to reality made it a true description of the destination, touristic experience and commercial exchange. As a representation of the destination it could reduce the complexities of a way of life or a strange and foreign destination to a few visual cues, serving as the functional shorthand of touristic discourses (Geertz 1995). As a representation of touristic experience it could instruct visitors in how to construct their experiences, how to view a place and how to create their own photographs (Dann 1996b). And as a representation of commercial exchange, the photograph became the most important thing in tourism marketing (Holloway and Robinson 1995; McGregor 2000). Two interrelated problems arose from the increasingly visual communication in tourism via photographic representations. The first problem was that photographs seem true in the sense that they could accurately represent the material features of a destination but deceptive in that they might connote any number of other meanings (Edwards 1996; Hendry 2005). In the photograph, the message and its intentions are entwined to such an extent that it becomes ideologically saturated (Beerli and Martin 2004; Hollinshead 2007). The second problem is that it is difficult to identify the absolute origin point of a photograph. A photograph or photographic theme could be borrowed from the visitor for the purposes of destination marketing, or reproduced by the visitor during the travel experience as an exact copy of a photograph that appeared online or in a travel brochure (Dann 1996b). These two problems come together in a destination image theory for tourism that deals with 1) the social effects of visual communication, and 2) the problem of empirical measurement and causality.

Visual communication in tourism research: Seoul destination image

627

The social effects of visual communication in tourism are potentially positive or negative. In a positive sense photographic representations can be used to present destination culture in a way that effectively promotes sustainable economic activity. The results are increased creativity and interpersonal relations, cultural pride and autonomy (Ryan 2002). Many argue that the cultural, historical and commercial interests at a destination are compatible and mutually interdependent (Shinde 2010). Others argue that visual representations of destination culture are bound up in a commodity fetishism that drains culture of its symbolic meaning (Selwyn 1996). In this sense, the photograph has been blamed for exaggerating certain elements of ordinary life (Bruner 1996) and silencing others (Laxon 1991); reducing residents of a community into a unitary cultural entity (Andsager and Drzewiecka 2002); and implementing the commoditization of a destination to make it commercially interchangeable with others (Schellhorn 2010; Hunter 2011b). A large amount of criticism is directed at destination marketing which, intentionally or not, uses the photograph in ways that misrepresent local culture in order to maximize its desirability (Sternberg 1997). Misrepresentation through marketing or through the exaggerated curiosity of visitors over the mundane (KirshenblattGimblett 1998) was seen to cause residents to parody themselves for the tourist (Sweet 1989) or to otherwise lose a sense of their own collective identity (Conquergood 1992). The problem in measurement and causality in tourism research originated with problems associated with the definition of destination image. Originally the theory was used to describe an induced-organic image dualism (Gunn 1972) that contrasted destination marketing against the experiences and perceptions of visitors, respectively. Recognizing that induced and organic image perspectives might be mutually independent, the theory was refined into a set of three constructs, including: 1) destination imagery as the topography of the touristic landscape and its material objects and experiences (Wolcott 1995); 2) perceived destination image as the effect of individual or collective audiences’ exposure to destination imagery; and 3) projected destination image as the purposive generation and packaging of imagery in print and electronic forms for marketing or commentary (Pritchard and Morgan 2001). The earlier problems associated with trying to identify discreet components of destination image (MacKay and Fesenmaier 1997) have been largely overcome by conceptualizing the image as a whole collection of denotative and connotative messages (McGregor 2000) and reasoned or emotional responses (Walmsley and Young 1998; Beerli and Martin 2004) that work as a semiotic feedback loop of pre-conceptions, perceptions and impressions of revisionary synecdochical devices (Verschaffel 1999; Smith 2005) constantly recycled in a system of published photographs and stories. Researchers have moved toward more relativistic attitudes toward destination image as a “nebulous concept” (Hughes and Allen 2008: 30); as a multidimensional concept (Gallarza, Saura, and Garcia 2002); as a dynamic

628

William Cannon Hunter

and circular process of negotiation between promotion and experience (Gilbert and Hancock 2006); or as a hermeneutic circle (Ryan 2002: 965). Destination image is understood as “circular”, in the sense of a semiotic feedback loop or in the sense that representations “circulate” within and between cultures (Jenkins 2003). Tourism’s semiotic feedback loop of visual communication consists of print and online photographs, simulations, maps and diagrams and drawings that describe or depict tourism destinations and travel experiences. It is created and constantly recycled by destination marketers who use it as a marketing tool and by tourists who use it as a way to record and share their personal travel experiences.

3 Visual research methods in tourism research Visual methods in tourism research are used to identify and understand the visual culture of destination image negotiation (Crouch and Lubbren 2006). Visual methods are also used to facilitate the generation of destination image or analysis of its component features (van Leeuwen and Lewitt 2001; Pink 2007). Visual communication includes photographs, video, paintings, diagrams, maps or other visual forms (Harper 2005; Rakić and Chambers 2011) but extends beyond media into the built and natural landscapes they represent. They are environments of signs that convey any number of denotative (literal) and connotative (implied) meanings. In tourism research, the meanings of signs are explored using quantitative and interpretive approaches such as psychoanalysis, discourse analysis, content analysis, semiotic analysis or other visual methods (Russel and Ison 2000; Rose 2003) individually or in combination. In previous tourism research, a combination content analysis-semiotic analysis methodology has been used to identify types of photographs used for tourism destination promotion (Jenkins 2003; Hunter 2008; Hunter 2011a). Other destination image research has focused on the collection and analysis of online pictures and text (Govers and Go 2004). The full potential of visual methods in tourism is not fully realized (Smith 2005), however, in spite calls for more use of unstructured and semi-structured methods (Prebensen 2007). In the following section an analysis of Seoul, Korea destination image is presented. The study outlines an evaluation of the visual communication of destination marketers using a combination content analysis-semiotic analysis. The methodology is based on a synthesis of visual research techniques (van Leeuwen and Jewitt 2001; Rose 2003). This approach includes five steps: 1) collecting visual data; 2) sorting visual data; 3) generating categories; 4) identifying denotative elements, and; 5) identifying connotative elements. The study uses a basic semiotic view of reality through the lens of photographs published in tourism brochures and guidebooks. These photographic representations are interpreted to identify how the real denotative features of the city are pictured in quantities and ways that reveal deeper connotations associated with ideologically driven intentions.

Visual communication in tourism research: Seoul destination image

629

The task was to connect the material symbolism of the destination to the agendas or actions of marketers. It worked to identify the types of synecdochical (a part that stands for the whole) meta-signs in circulation and their role in the maintenance of the destination image. A synecdoche is a sign that identifies the destination as a whole and gives it symbolic significance (Smith 2005). It can inform researchers’ understandings of how a place is hard branded and visually consumed or experienced (Zukin 1998; Evans 2003). The synecdoche also reveals how certain privileged views, social conventions, power relations, metaphors or other collectively shared fields of reference determine how destination culture appears outside audiences (Crump 1999). As mentioned previously, measurement and causality are problematic issues in destination image research because imagery circulates as a semiotic feedback loop, blurring distinctions between visitors’ perceptions and experiences and marketers’ intentions. However, in the study described in the following sections, the emphasis is on the projected element of destination imagery, evidenced by the fact that the material analyzed was published by government and private for profit agencies for the purpose of destination tourism promotion. In practice, the theoretically nebulous destination image concept comes into focus through the strategic use of empirical research techniques, including good theoretical definitions and rigorous sampling procedures.

4 The Seoul destination image In this study on visual communication in tourism, projected destination image was conceptualized as the combined product of government authority and private commercial efforts which work to generate a singular and iconic representation of the city as a showcase of ideal features. This image is a simplified discourse or commercial product that focuses on the individual consumer rather than a wider context of social relations (Selby 2004: 81). Seoul uses the synecdoche – where a part can stand for the whole or a whole can stand for the part – as a mechanism for containing the complex reality of a large city with one or more highly visual elements of the landscape. In this sense, the synecdoche is a re-imaging tool for Seoul, a means to represent the social image of the city as a whole in the best possible way, to external audiences (Pike 1996). For tourism destination cities, a synecdoche is a newly constructed or restored building or monument, an urban waterway, a specially zoned district or a major event. It conveys a massive and solid appearance (Verschaffel 1999: 333). The synecdoche exists as a physical reality in time and space and also exists as a representation in the form of a photograph, map or simulation circulated in tourism promotional materials.

630

William Cannon Hunter

4.1 Seoul past and present Seoul was founded in 1394 on the principles of geomancy, between mountains and the Han River to provide for defense and inland navigation for commerce. The construction of city gates including Gwanghwamun, the Gyeongbok Palace gate and Namdaemun, the south city gate began immediately and completed in 1395 and 1398 respectively. These were the original monumental natural and built features of the city. These waterways and gates have survived as key representations of the city up to the present day in spite of foreign occupation, war and rapid industrial development. In the last century Japanese Forced Occupation (1910 to 1945), the Korean War (1950 to 1953) and rapid post-war industrialization have caused the dismantlement, destruction and removal, reconstruction and revision of the city’s waterways, monumental architecture and whole districts. New synecdochical event-projects since the implementation of the 1962 National Economic Development Plan have included: the 1970 “Expo 70” in Osaka and Korea’s Year of Tourism; the 1988 Olympics hosted in Seoul; and the 2002 World Cup co-hosted with Japan. Construction projects have included the rehabilitation of Seoul waterways and reconstruction of historic city gates. Shopping areas have been developed by theme and anchored by their proximity to monumental structures and spaces as are events hosted by the city. This is a strategic development approach that defines globalizing cities (Richards and Wilson 2004; Knox 2006). A long history, auspicious natural landscape features, a history of forced reconstruction and historical revisionism, and rapid modern tourist city development have worked to reinforce and secure the city’s synecdochical features. The production of tourism as a patriotic industry and hard branding through construction projects, marketing strategies and the hosting of mega events suggest a concentrated effort to project a destination image. These developments and conditions make Seoul a good example of how an urban destination image is formed or projected. The question arises, however, as to what highly visible features, exactly, make up the Seoul urban landscape (Weightman 1987) and how are they used in tourism promotion. A secondary question follows, as to whether the Seoul projected image reflects the full potential range of imagery available to its visitors (Crawshaw and Urry 1997).

4.2 Collecting and sorting visual data This study used data sampling and content analysis sorting procedures that, in tourism research, are relatively standardized. The aim of the study was to collect as many photographs of Seoul as possible. The photographs were defined as those appearing in print form in tourist brochures and guidebooks. A non-probability, non-random sampling approach (Noth 1990; Gobo 2009) was used to gather tourism promotion materials. This was accomplished by recruiting volunteers of vari-

Visual communication in tourism research: Seoul destination image

631

Tab. : Content analysis descriptive statistics. Tourism materials collected

Tourism materials publisher

Seoul city

Han River

Seoul, Seoul, River, gov. priv. gov.





Brochure Guidebook Magazine Postcards

   

   

Total:  pp.





Publication type: Map

Total & (%)  (.%)    

(.%) (.%) (%) (.%)

 (%)

River, priv.

Total











   

   

   

   

   











Frequency and Type of Representation: Seoul City ( photographic representations)

Han River ( photographic representations):

 cityscapes and tourist districts

 landscapes: in daylight (), at sunset (), at night (), at night with fireworks ();

 monumental structures: major city gates (), Seoul Tower (), built landmarks (), cultural memorials (), other buildings ();

 representations of recreational activities on the river or riverbanks;

 festivals, events and performances;

 maps of the river, the riverbanks and river parks;

 maps and simulations: maps (), simulations ();

 simulations of futuristic structures and landscape design;

 culture related; traditional culture (), museums (), historical photographs ();

 representations of boats () and bridges (), or transportation and tourism infrastructure found on or over the river, and;

 nature related: Han River (), Cheonggyecheon (), parks and mountains ().

 representations of landmarks near the river with river as background or foreground.

ous ages, genders and nationalities through a major Seoul university. They were briefed on the logic of destination image and representations, organized into twenty four groups of seven and instructed to visit tourist information booths and kiosks, tourism attractions and destination marketing organizations throughout Seoul to gather as many brochures, guidebooks, maps, shopping guides and postcards as they could find. Groups worked independently. After collection, focus groups met again for a de-briefing and discussion. In this manner the maximum quantity and variety of tourism information was collected with minimum researcher bias (Patton 1990; Miles and Huberman 1994).

632

William Cannon Hunter

Two sorts of the brochures and guidebooks were conducted using a team of three coders. In Sort 1, duplicates were eliminated and a page count was made. The focus groups collected 2008 tourism promotion brochures, maps, guidebooks, shopping guides and postcards of which 1615 were duplicates. The remaining sample included 393 unique brochure types with 9115 pages of material. Slightly more tourism materials were published by government than by private organizations but the same photographs, maps and simulations were found repeatedly in different pamphlets regardless of publisher. The 393 brochure types included 362 pamphlets with 951 visual representations of city tourism in general, and 31 pamphlets with 372 representations of the Han River. The river was the most prominent synecdochical element of the Seoul landscape featured in tourism promotion. There are more brochures promoting Seoul, but more representations of the Han River overall. In Sort 2, an inductive approach was used to sort the 1,323 photographs into initial categories or types of denotative sign elements. The two general categories of Han River and Seoul Cityscapes were derived and mutually exclusive subtypes were identified in each category. The team of coders worked together to cross check for the threats of hidden variables, micro-analysis and the confusion of visual elements and verbal forms (Collier and Collier 1999). Six subtypes associated with the Han River representation and six subtypes of the Seoul Cityscapes representation were derived. Details concerning sampling and sorting are shown in Table 1 and include a basic description of the tourism materials found and the 12 subtypes.

4.3 Analysis of denotative and connotative elements In this analysis, denotative sign elements of the photograph are analogical to reality and interpreted literally; coded according to their relative position or importance within the photograph (Bell 2001). The aim at this stage of the content analysis was to identify the 12 representational subtypes in the two major categories. The Han River representation (372 photographs) was the denotative element most frequently appearing in the photographs and the city’s most prevalent synecdoche. Within this denotative element are 6 representational subtypes, the most frequent being that depicting the River as a landscape (109) in daylight (57). The Seoul Cityscape representation (951 photographs) also included six subtypes, the most frequent being cityscapes and tourist districts (368) and the second being monumental structures (134). Thus, the city’s most frequent representation found in tourist brochures is cityscapes and tourist districts. However these photographs of cityscapes do not appear remarkably different from each other or from the cityscapes of other cities. They are more indicative of benchmarking or urban cloning. A more iconic feature is required as a synecdoche and the content analysis identifies monumental structures (134) as the second most synecdochical element of the

Visual communication in tourism research: Seoul destination image

633

Seoul destination image. Depictions of festivals events and performances (133) were also difficult to identify as specific to Seoul. Connotative elements of the image are supplementary to reality and are interpreted theoretically. In the content analysis denotative elements of photographs are used to identify types and frequency; but secondary interpretive analysis of connotative elements can generate more insight into the message of the destination image as depicted in photographs. By focusing on discursive elements of visual culture (Harper 2005) the logic and histories that form the destination image of Seoul can be interpreted as a semiotic system (Jenks 1993: 121; Metro-Roland 2009). Rather than being simply a random and accidental collection of photographs, the brochures published by public and private agencies might be purposefully conveying information concerning interests in furthering certain urban development goals or in influencing visitors’ experiences. The content analysis and findings generated by identifying denotative elements of the Seoul destination image provide a structure for identifying the messages conveyed by the city’s most synecdochical features, the Han River and icons of the cityscape. The interpretive analysis in the following discussion section investigates the results generated in the content analysis in light of historical perspectives and tourism theory surveyed in the literature review.

4.4 Understanding the Seoul synecdoche 4.4.1 Han River synecdoche and its connotations The Han River synecdoche, Version One, depicts landscapes and activities using a number of privileged or non-ordinary views from above or wide angles. At night fireworks, colored lights and fountains installed on bridges and bright city lights add to the impression of an extraordinary experience. There are 109 representations depicting the river as a geographical whole or a complete and continuous landscape. The river is also pictured 130 times as a whole using maps and simulations. Recreational activities are featured that take place on the water’s surface and on the banks with the river in the background or foreground. These representations highlight riverside parks and combine with 82 depictions of the river’s surface punctuated with windsurfers or Jet Ski riders against a backdrop of the cityscape. This imagery connotes a clean and accessible waterway that is highly developed. Adding to the connotation of accessibility are 47 photographs that feature infrastructure such as boats and bridges. Examples of the most frequent representations of the Han River are found in Figure 1 taken directly from published tourism materials. Non-ordinary views, a unitary whole, accessibility and development and recreation are key connotations. The Han River synecdoche, Version Two, depicts the river in the foreground of the city as an integrated feature of a future Seoul. In this science fiction view,

634

William Cannon Hunter

Fig. 1: Han River.

Fig. 2: Simulations.

simulations and maps are fully used to promote Seoul as a global city. There were 183 images that depicted the Han River and the city as a unitary whole, and 60 simulations of futuristic structures and landscape design. These are digital images that are at times indistinguishable from conventional photographs. These views connote an anticipated future Seoul that is highly developed and completely interconnected. Two most common simulations are the floating island and the envisioned mega sea-port station to be built on the bank of the river. These simulations, while depicting a future green, connected and global city, are laden with connotations of big budget spending and favored pet projects. This version of the Han River is pictured in Figure 2.

Visual communication in tourism research: Seoul destination image

635

Fig. 3: Waterways.

The Han River synecdoche, Version Three, depicts the River as transformed. The photographs waterways showcase the effects of three distinctly marked eras, spanning a period of 50 years. The Han River is shown as a recreational area in mid-1960. Beaches were major recreational areas prior to the growth oriented industrial development (1968–1986) that covered most of them with concrete. Later images show the river as polluted or crowded by highway development and highrise apartments that detract from or destroy the river’s recreational and environmental qualities. Current images show the early effects of a 23 year Han River Renaissance Project (since 2006) that plans to revive the Han River as a world-class attraction for citizens and tourists under the motto, “Vibrant Waterfront City”. The conjunction of past and present connote regeneration and progress and the accomplishment of an attractive and livable city that is worth visiting. Waterways are decorated with thematic spectacles such as fountains, colored lights and firework displays. They are representations of an eco-friendly leisure and tourism city that has caught up to the damage of compressed modernity. There are also accidental connotations associated with new cultural norms; people no longer go to the river to play in the water – now the river is a showcase and people are spectators. The three-stage transformation of the River is pictured in Figure 3.

4.4.2 Seoul Cityscapes synecdoche and its connotations The content analysis identified 368 photographs of cityscapes and tourist districts. They depict crowded streets and designer signs from non-ordinary viewpoints. They include urban waterways in the background, a few instances of monumental architecture such as Seoul Tower, Seoul Station and other landmarks. Cultural experiences are pictured and feature festivals, events and performances in the context of a cityscape. However, the majority of these depictions do not appear remarkably different from the cityscapes of other cities and the connotation is that Seoul is interested in benchmarking, urban cloning and an overall projection of a

636

William Cannon Hunter

Fig. 4: Gates.

globalizing city image. The most prominent synecdochical feature of the Seoul cityscape is the unique architecture of the city gates, Gwanghwamun and Namdaemun. These are highly valued cultural assets that are depicted as linking together adjoining tourism districts, attractions and waterways. The city gates were featured in 53 photographs and, like the Han River, the theme of transformation is evident. In Figure 4 Gwanghwamun (top) and Namdaemun (bottom) gates act as markers of historical events and connote the city’s long past. They are also representations of culture. Namdaemun was designated Korea’s National Treasure No. 1 in 1968. These gates might be the most iconic and original elements of the city, built at the founding of Seoul and revised continually ever since. Local residents are aware that Gwanghwamun was completed in 1395 and destroyed in 1592 (Hedeyoshi invasion); rebuilt in 1865 (Joseon Dynasty); moved in 1926 (construction of the Japanese Government Building); destroyed in 1950 (Korean War); rebuilt in the wrong place in 1968 and dismantled for repositioning from 2006 to 2010 at a cost of about $ 25.5 million USD. Namdaemun was completed in 1398, restored in the 1970’s and burned in 2008 and will take about 3 years and $ 21 million USD to rebuild. The circulation of historic photos and the conspicuous recent reconstruction of the gates connote their importance as city icons and their role in the telling of a story of cultural survival and improvement through restoration.

4.4.3 Discussion – more connotations By viewing the results of the visual analysis it can be seen that when sorted and arranged by frequency, literal or denotative categories emerge from a sample of photographs that can be interpreted as synecdochical elements of a city’s destination image. In addition, these images might include connotations of hidden agendas or accidental messages that show the city in a different light. They might

Visual communication in tourism research: Seoul destination image

637

suggest a tacit overarching promotional strategy shared by organizations to promote a clearly defined destination image based on the most prominent features of the city. Multiple representations of a tightly knit portfolio of highly synecdochical urban features suggest the purposive and edited hard marketing of an urban image, a global city (Baloglu and McCleary 1999: 869; Gilbert and Hancock 2006). The juxtaposition of past, present and future connotes continuity; that Seoul is a transforming and progressive city. The Han River is pictured in ways that indicate that it is an evolving place that reflects the cultural norms and values of the times. Simulations of a future city reflect aims to further certain urban development goals or projects. The conjunction of contemporary and historical photographs of monumental architecture suggest revisionary tactics in the marking or erasing of effects of violence, invasion, accident, wear and tear of time and the contestation of power. Taken as a whole, the Seoul projected destination image connotes historical authenticity, change and progress toward a better future. In Seoul, the projected destination image acts as a commercial product that obliterates alternative perceptions. Urban design efforts and expensive development projects such as the restoration of Seoul’s waterways and gates and the continual improvement of tourist shopping zones are mirrored almost seamlessly in the projected image, and strategic zoning makes a unitary whole out of the parts. The message seems to ensure that visitors are almost automatically conveyed to the sites or attractions intended for them to visit. Such a message may affect the travel experience by generating a simulation of reality that is so complete, any other competing discourse is completely disintegrated (Baudrillard 1996). The conspicuous absence of weedy species – unwanted populations and ugly developments – connotes the new and clean city, the final synecdoche identified in this research. This synecdoche is conspicuous in its invisibility, achieved by spatial tactics of erasure or through the strategic selection of photographs and simulations. Namdaemun as a gathering place for the city’s homeless is unseen as is City Hall which serves an important function as a gathering place for unsightly public demonstrations. The collective sharing of photographs between private operators and public organizations connotes the tacit recognition of an approved message concerning Seoul’s destination image. The message also seems to suggest an intention to influence the movement of tourists within the city. The projected destination image appears to be a normative construction of a unitary and coherent representation of the city. The scrapbook-like appearance of the brochure and its seemingly random availability camouflages the monolithic and singular message projected by a limited number of synecdochical representations of the city. A summary of the Seoul destination image, its synecdochical features and the denotative and connotative elements of photographs identified in the content analysis is provided in Table 2. Further discussion regarding other connotations is also summarized in the table.

638

William Cannon Hunter

Tab. : Seoul destination image content analysis findings.

Synecdoche

Denotative element

Connotative element

Han River, Version  (Present)

Landscape and its features Recreational activities Boats and bridges

Non-ordinary views of a unitary whole. Clean highly accessible and connected recreational attraction. A product of development and improvement.

Han River, Version  (Future)

Simulations and maps Floating island Mega sea-port

Non-ordinary views of a unitary whole. Idealized views and an anticipated future reality. Representation of public planning best policy.

Seoul, Version  (Past)

Waterways Historical representations

Transformation and historical revision and regeneration. Progress and development, conjunction of past and present. Attractive and livable global city, eco-friendly.

Seoul, Monuments Iconic Gates Monuments Urban context

Historical past and cultural survival through restoration. Heritage, revitalization and the value of culture. A global city, progress and connectivity.

Clean

No element

Weedy species conspicuously invisible. Strategic removal of undesirable elements.

Summary of connotative elements

Non-ordinary views of a unitary whole convey accessibility and connection. The juxtaposition of past, present and future to demonstrate transformation & development. Prescribed views and activities intended to influence visitors’ travel goals and choices. Sanitized image conveys a clean and new city image. A normative viewpoint and monolithic representation of the city. Clearly defined, shared destination image promotional strategy. Construction of a global city and the good management of public funds are conveyed.

5 Conclusion – visual communication and tourism research In this chapter an overview of the history of visual communication in tourism theory was introduced. An example of visual research on destination image was described to illustrate how tourism destination image is interpreted. Destination image is seen as a product of visual communication that circulates as photographic representations in print and online tourism promotion material. In some cases, like Seoul, the imagery is largely the product of hard branding, developed by government agencies and commercial operations. In other cases the imagery includes visitors’ own visual narratives – especially online – where they share depictions of their own experiences at the destination. Content and semiotic analy-

Visual communication in tourism research: Seoul destination image

639

sis methods are semi-quantitative, semi-structured and interpretive approaches to understanding visual representations of a tourism destination. The tone and focus of tourism research is affected by practical interests. As a multidisciplinary field, tourism research must mix theoretical concerns with practical applications. Most research in tourism works to produce findings that can inform decision making in policy development and destination marketing and management. In light of the study described in the previous section, typical policy and management implications might address: 1) the motives (if any) driving destination marketing strategy; 2) the effects of the destination image; and 3) the effectiveness of tourism research theory and practice in improving the management and operation of international tourism.

5.1 Motives The motives behind the development of a projected visual destination image on potential recipients, when there are any, are evidenced in the connotative messages conveyed by the Seoul projected destination image. In the Seoul case, it seems possible that planners may be interested in showcasing recently completed urban development projects or highlighting simulations of proposed projects. They may be interested in sending a message to visitors as well as residents that public funds have been well spent in the creation of a global standard city. The exclusion of less photogenic elements of the city (Balibrea 2001) might be a way to mitigate economic, political or social pressures. Perceptive residents or visitors might interpret the destination image as a selective framing of sights (Jackson 1992) and the virtual removal or dislocation of subjects and revision of social life or the privileging of certain interests over others. Government authorities and tourism developers should be sensitive to the connotations associated with the projection of any visual destination image to ensure that it does not seem contrived or manipulative. It is also quite probable, however, that no deliberate attempt has been made to develop a message using destination image. The positive identification of a visual destination image raises questions about whether it is an accidental collection of photographs endlessly circulating between agencies and organizations and haphazardly published or a purposeful and carefully organized destination marketing strategy.

5.2 Effects The effects of the destination image on visitors and tourists can vary depending upon the medium, the contents and its overall accessibility. Destination planners and marketers should work together to ensure that more attention should be given to the planning and development of the image. Destination image requires methodical analysis to ensure that it is purposeful, representative and readily

640

William Cannon Hunter

interpreted by its intended recipients. New destination promotion projects should include steps to analyze the existing image before publication of new material. By clearly identifying the existing projected destination image planners and developers can take steps to alter it in ways that will best conform to policy on the one hand and to the needs and interests of preferred visitors on the other. The denotative message should be a highly identifiable and potentially memorable image for visitors and potential visitors to Seoul and it should also clearly represent the interests of the destination and its residents. The destination synecdoche can represent the city as an easily recognizable whole by acting as a logo or brand. The synecdoche is a photographic miniature of the destination that can be transmitted by digital and print media to virtually anywhere simultaneously. But choosing a synecdochical representation of the destination includes the risks of selective exclusion of other attractive features, the misrepresentation of the reality of the destination in all its scope and the possibility that with growing recognition over time the representation will become outdated and difficult to replace. There will also be differing opinions on what the ideal synecdochical feature of the destination should be and how it should be portrayed. For these reasons a destination image should be composed of a limited but sufficient variety of synecdochical features that allow room for changes over time while retaining brand recognition. The destination image should be a portfolio of images that balances universal recognition with a message that can be accurately interpreted by preferred visitor markets. The destination image should be a tool that can help to transform and improve the city and increase tourism. Planners must recognize how residents and visitors perceive the existing destination image and set clear objectives defining what denotative and connotative messages are to be conveyed and their expected impact. Planners must also recognize their own culturally imbued conventions, social and political norms and path dependence that have contributed to the appearance and representation of the city (Nassauer 1995). The construction of a destination image should be accomplished in partnership with outside professional consultants, visitors and residents. Strategic sponsorship of creative projects and events give artists and designers outlets for expression. User created content and social networks online (web 2.0) also have valuable potential for destination image development as they are rich in travelers’ stories and photographs commemorating visits to Seoul.

5.3 Tourism research Tourism research is turning to new methodologies to deal with the increasingly visual communication used by both destinations and visitors to convey the features of the destination and travel experiences available there. More work needs to be done to improve the way researchers identify visual messages and interpret

Visual communication in tourism research: Seoul destination image

641

their impact. Tourism research can reveal the intended or accidental interests of developers, and the effects of a destination image message. The claim that modern global cities are the product of urban cloning can be addressed by identifying and comparing cities’ synecdochical features. In connection with urban cloning, a better understanding of visitors’ perceptions and the effect of destination image on tourist behavior can be developed. A more sincere and focused image can be developed that will leave visitors and residents with lasting positive impressions. Visual methods can be applied to user created content and online social networks to better understand the experiences and perceptions of visitors toward the destination. While the visual content of official tourism websites is an electronic simulacrum of printed material, user created content might convey a completely different image of the destination. Visual methods can inform future research that seeks connections between verbal and visual representations of the destination image. Other studies can expand the scope of visual research by focusing on interviewing destination planners and policy makers who are directly responsible for designing or implementing the projected destination image. Destination planners will be able to use this information to develop effective marketing that represents the destination and appeals to the interests of preferred visitor markets.

References Andsager, Julie L. and Jolanta A. Drzewiecka. 2002. Desirability of differences in destinations. Annals of Tourism Research 29(2): 401–421. Balibrea, Mari P. 2001. Urbanism, culture and the post-industrial city: challenging the ‘Barcelona Model’. Journal of Spanish Cultural Studies 2(2): 187–210. Baloglu, Seyhmus and Ken W. McCleary. 1999. A model of destination image formation. Annals of Tourism Research 26(4): 868–897. Baudrillard, Jean. 1996. The Perfect Crime. Transl. by Chris Turner. London: Verso. Beerli, Asunciòn and Josefa D. Martin. 2004. Factors influencing destination image. Annals of Tourism Research 31(3): 657–681. Bell, Philip. 2001. Content analysis of visual images. In Theo van Leeuwen and Carey Jewitt (eds.), Handbook of Visual Analysis, 10–34. London: Sage. Boorstin, Daniel. 1992. The Image: A Guide to Pseudo-events in America. New York: Vintage Books. Bruner, Edward M. 1996. Tourism in Ghana: The representation of slavery and the return of the black diaspora. American Anthropologist 99(2): 290–304. Collier, John and Malcolm Collier. 1999. Visual Anthropology: Photography as a Research Method. Albuquerque: University of New Mexico Press. Conquergood, Dwight. 1992. Performance theory, Hmong shamans, and cultural politics. In Janelle. G. Reinelt and Joseph. R. Roach (eds.), Critical Theory and Performance, 41–64. Ann Arbor: University of Michigan Press. Crawshaw, Carol and John Urry. 1997. Tourism and the photographic eye In Chris Rojek and John Urry (eds.), Touring Cultures: Transformations of Travel and Theory, 176–195. London: Routledge. Crouch, David and Nina Lubbren (eds.). 2006. Visual Culture and Tourism. New York: Berg.

642

William Cannon Hunter

Crump, Jeff R. 1999. What cannot be seen will not be heard: the production of landscape in Moline, Illinois, Ecumene 6(3): 295–317. Dann, Graham. 1996a. The Language of Tourism: A Sociolinguistic Perspective. Wallingford: CAB International. Dann, Graham. 1996b. The people of tourist brochures. In Tom Selwyn (ed.), The Tourist Image: Myths and Myth Making in Tourism, 61–82. New York: John Wiley & Sons. Debord, Guy. 1995. The Society of the Spectacle. Transl. by Donald Nicholson-Smith. New York: Zone Books. Edwards, Elizabeth. 1992. The tourist icon: Four Australian postcards. Tourism in Focus 6: 4–5. Edwards, Elizabeth. 1996. Postcards: Greetings from another world. In Tom Selwyn (ed.), The Tourist Image: Myths and Myth Making in Tourism, 197–222. New York: John Wiley & Sons. Evans, Graeme. 2003. Hard branding the cultural city – from Prado to Prada. International Journal of Urban and Regional Research 27(2): 417–440. Gallarza, Martina G., Irene G. Saura and Haydée Garcia. 2002. Destination image: Towards a conceptual framework. Annals of Tourism Research 29(1): 56–78. Geertz, Clifford. 1995. After the Fact: Two Countries, Four Decades, One Anthropologist. Cambridge, Massachusetts: Harvard University Press. Gilbert, David and Claire Hancock. 2006. New York City and the transatlantic imagination: French and English tourism and the spectacle of the modern metropolis, 1893–1939. Journal of Urban History 33: 77–107. Gobo, Giampietro. 2008. Re-conceptualizing generalization: Old issues in a new frame. In Pertti Alasuutari, Leonard Bickman and Julia Brannen (eds.), The Sage Handbook of Social Research Methods, 193–213. London: Sage. Govers, Robert and Frank M. Go. 2005. Projected destination image online: website content analysis of pictures and text. Information Technology and Tourism 7: 73–89. Gunn, Clare. 1972. Vacationscape: Designing Tourist Regions. Austin University of Texas. Harper, Douglas. 2005. What’s new visually? In Norman K. Denzin and Yvonna S. Lincoln (eds.), The Sage Handbook of Qualitative Research, 747–762. London: Sage Publications. Hendry, Joy. 2005. Reclaiming Culture: Indigenous People and Self-representation. New York: Palgrave Macmillan. Hollinshead, Keith. 2007. Indigenous Australia in the bittersweet world: The power of tourism in the projection of ‘old’ and ‘fresh’ visions of Aboriginality. In Richard Butler and Tom Hinch (eds.), Tourism and Indigenous Peoples: Issues and Implications, 281–304. New York: Elsevier. Holloway, J. Christopher and Chris Robinson. 1995. Marketing for Tourism. (3rd edition). London: Addison Wesley Longman. Hughes, Howard L. and Danielle Allen. 2008. Visitor and non-visitor images of Central and Eastern Europe: A qualitative analysis. International Journal of Tourism Research 10(1): 27– 40. Hunter, William C. 2008. A typology of photographic representations for tourism. Tourism Management 29(2): 354–365. Hunter, William C. 2011a. Rukai indigenous tourism: Representations, cultural identity and Q method. Tourism Management 32(2): 335–348. Hunter, William C. 2011b. The good souvenir: Representations of Okinawa and Kinmen Islands in Asia. Journal of Sustainable Tourism (Article available online, early view). Jackson, Peter. 1992. Constructions of culture, representations of race: Edward Curtis’s ‘Ways of Seeing’. In Kay Anderson and Fay Gale (eds.), Inventing Places: Studies in Cultural Geography, 89–106. London: Longman. Jenks, Chris. 1993. Culture. New York: Routledge. Jenkins, Olivia H. 2003. Photography and travel brochures: The circle of representation. Tourism Geographies 5(3): 305–328.

Visual communication in tourism research: Seoul destination image

643

Kirshenblatt-Gimblett, Barbara. 1998. Destination Culture: Tourism, Museums, and Heritage. Berkeley, CA: University of California Press. Knox, Dan. 2006. The sacralised landscapes of Glencoe: from massacre to mass tourism, and back again. International Journal of Tourism Research 8(3): 185–197. Laxon, Joan, D. 1991. How “we” see “them”: Tourism and Native Americans. Annals of Tourism Research 18(3): 365–391. MacCannell, Dean. 1973. Staged authenticity: Arrangements of social space in tourist settings. American Journal of Sociology 79(3): 589–603. MacKay, Kelly J. and Daniel R. Fesenmaier. 1997. Pictorial elements of destination image formation. Annals of Tourism Research 24(3): 537–565. McGregor, Andrew. 2000. Dynamic texts and tourist gaze. Annals of Tourism Research 27(1): 27– 50. Metro-Roland, Michelle. 2009. Interpreting meaning: An application of Peircean semiotics to tourism. Tourism Geographies 11(2): 270–279. Miles, Matthew B. and A. Michael Huberman. 1994. Qualitative Data Analysis. (2nd edition). Thousand Oaks: Sage Publications. Nassauer, Joan I. 1995. Culture and changing landscape structure. Landscape Ecology 10(4): 229– 237. Noth, Winfried. 1990. Handbook of Semiotics. Indianapolis: Indiana University Press. Patton, Michael Q. 1990. Qualitative Evaluation and Research Methods. (2nd edition). Newbury Park: Sage. Pike, Steve. 1996. The City as Image. In Richard Legates and Frederic Stout (eds.), The City Reader, 243–249. London: Routledge. Pink, Sarah. 2007. Doing Visual Ethnography. London: Sage. Prebensen, Nina K. 2007. Exploring tourists’ images of a distant destination. Tourism Management 28(3): 747–756. Prideaux, Bruce. 2003. Commodifying heritage: Loss of authenticity and meaning or an appropriate response to difficult circumstances? International Journal of Tourism Sciences 3(1): 1–15. Pritchard, Annette and Nigel J. Morgan. 2001. Culture, identity and tourism representation: Marketing Cymru or Wales? Tourism Management 22(2): 167–179. Rakić, Tijana and Donna Chambers (eds.). 2011. An Introduction to Visual Research in Tourism. London: Routledge. Richards, Greg and Julie Wilson. 2004. The impact of cultural events on city image: Rotterdam, cultural capital of Europe 2001. Urban Studies 41(10): 1931–1951. Rose, Gillian. 2007. Visual Methodologies: An Introduction to the Interpretation of Visual Materials. (2nd edition). London: Sage. Ryan, Chris. 2002. Tourism and cultural proximity: Examples from New Zealand. Annals of Tourism Research 29(4): 952–971. Schellhorn, Matthias. 2010. Development for whom? Social justice and the business of ecotourism. Journal of Sustainable Tourism 18(1): 115–135. Selby, Martin. 2004. Understanding Urban Tourism: Image, Culture and Experience. London: IB Tauris. Selwyn, Tom. 1996. Introduction. In Tom Selwyn (ed.), The Tourist Image: Myths and Myth Making in Tourism, 1–32. New York: John Wiley & Sons. Shinde, Kiran A. 2010. Entrepreneurship and indigenous entrepreneurs in religious tourism in India. International Journal of Tourism Research 12(5): 523–535. Smith, Andrew. 2005. Conceptualizing city image change: The ‘re-imaging’ of Barcelona. Tourism Geographies 7(4): 398–423. Sternberg, Ernest. 1997. The iconography of the tourism experience. Annals of Tourism Research 24(4): 951–969.

644

William Cannon Hunter

Sweet, Jill D. 1989. Burlesquing “The Other” in Pueblo performance. Annals of Tourism Research 16(1): 62–75. Urry, John. 1990. The Tourist Gaze: Leisure and Travel in Contemporary Society. London: Sage. Van Leeuwen, Theo and Carey Jewitt (eds.). 2001. Handbook of Visual Analysis. London: Sage. Verschaffel, Bart. 1999. The monumental: On the meaning of a form. The Journal of Architecture 4: 333–337. Walmsley, Jim D. and Martin Young. 1998. Evaluative images and tourism: The use of personal constructs to describe the structure of destination images. Journal of Travel Research 36(3): 65–69. Weightman, Barbara A. 1987. Third world tour landscapes. Annals of Tourism Research 14(2): 227–239. Wolcott, Harry F. 1995. Making a study “more ethnographic”. In John van Maanen (ed.), Representation in Ethnography, 79–111. London: Sage. Zukin, Sharon. 1998. Urban lifestyles: diversity and standardisation in spaces of consumption. Urban Studies 35(5–6): 825–839.

Randall Teal

29 Thinking visuals: What the challenges of architectural representation can tell us about visual communication Abstract: Architects generally do not build buildings, but describe future buildings using visuals. This gap between the built thing and its representation creates a complicated relationship for architecture and visual communication. That is to say, if the gap that arises between the material and the virtual is not acknowledged and accounted for during the design process, architectural visuals will tend to become ends in themselves. When this occurs, a false priority is placed on visual orientation to the world, which leads to existential thinness in our environments. Counteracting such a limiting use of representation requires a visual communication that aspires to reach beyond visual correspondence – seeking to not simply indicate – but to make one think.

1 Visual confusion In his iconic painting, Ceci n’est pas une pipe, Rene Magritte reminded us of something that should be self-evident – that a pipe and the representation of a pipe are not the same thing. Despite the seeming obviousness of such a comment, it is surprising how much the title of this painting rings true in architecture, where there is a constant need to be reminded that drawings of buildings are not, in fact, actually buildings; rather, they are mediating artifacts that point towards future buildings (Pérez-Gómez and Pelletier 1997: 7). Further, it is often the building designers themselves that need to be reminded the most about this difference. This is because their intense involvement with producing representations of buildings (i.e. not “building” buildings) creates a focus that can make it easy to lose sight of the fact that these representations are not ends in themselves. In short, architectural drawings, at their best, connect designers, clients, and users to the potential of a future “built” environment. It is because of this particular challenge that architectural theorist Robin Evans called the task of architectural composition a “peculiar enterprise”; and he explained that this challenge arises because architecture is a “metric organization that is judged optically, it mix(es) one kind of geometry with the other kind of assessment (Evans 1997: xxxiii)”. The metric corresponds to our haptic sense. That is, the metric is directly connected to our sense of touch and our bodily involvements in the world. As such, metric descriptions seek to discover and describe, not optical data; but rather sensorial, relative, and situated interactions between places and people. The projective, on the other hand, describes the world as visual

646

Randall Teal

and measurable, and provides objective understanding. Certainly, both metric and the projective descriptions have their place, as each provides very different feedback about the world. Unfortunately, as Evans points out, in architecture these two different modes of processing information are frequently conflated. When this occurs we get metric systems (buildings) that are driven by objective visual assessment, which severs connections to the haptic nature of a three-dimensional environment. This conflation of the metric and projective in architecture happens, in part, because the multisensory and temporal aspects of architecture resist direct transcription and communication. Simply, it is both easier and more rationally justifiable to communicate the look of a cohesive and well-composed building (projective) than to describe the “feel” of cohesive and well-composed building (metric). When we do the former – judge a building’s quality solely via its visual character – we tend to create architecture that feels thin and shallow, because the importance of visual appearances come to dominate the building design. Here, the designer has imagined a shorter distance between the object and its representation than is there in reality, by either overlooking the multiplicity of the architectural environment, or falling victim to the immediacy of direct graphic methods. Despite the difficulty of accessing and visually describing the multisensory aspects of architecture it is not any fault of our human capacities. In fact, humans have a remarkable aptitude for conjuring up the experiential content of places and events merely through recollection (think of the famous madeleine scene in Proust’s Remembrance of Things Past). We can imagine ourselves inside a particular auditorium, classroom, spa, restaurant and invoke not just the sights, but the sounds, the smells, and the specific mood of the encounter. Unfortunately, the regularity with which we encounter information that is packaged into crisp digestible visuals helps to deactivate this capacity, because such visuals tell us very explicitly what we should think, desire, feel; and thus we become conditioned to not think beyond the visual itself. In other words, visual communications such as advertisements, political propaganda and way-finding signage, function less effectively if they cause the viewer to think critically or imagine alternatives. For example, a stop sign that requires one to think about what it means is not an effective traffic regulating device; nor is an advertisement an effective sales device if it does not clearly communicate the singular desirability of the product or service that is being offered. Although such visual information is effective in these realms, it also tends to foster passivity in the way we engage visual communications. In short, because our world is filled with an abundance of direct communications, we can begin to believe that directness and immediacy are the hallmarks of effective visual communication. However, this is not true for all visual communications. Architecture has long benefitted from visuals that are both ambiguous and, in a way, incomplete. The benefit of these types of visuals is that they invite the viewer to not fixate on the drawings as such; but rather, to imagine the built reality

Thinking visuals

647

Fig. 1: Way-finding communication must be direct and unambiguous. Photo credit: Randall Teal.

that they describe. However, with a growing reliance upon digital technology like Google Sketchup and Building Information Modeling (BIM) softwares, architecture is being challenged to keep its visuals from becoming subsumed by the myth that directness, immediacy, and completeness is effective visual communication. This has always been a concern, but the threat has become more immediate because, these softwares convey the seductive notions that one has a miniature building before them. The danger of the “miniature building” is that it is a catalyst for falsely shortening the distance between object and representation. As such, the seeming reality of these digital representations can cover up a designer’s lack of intellectual and experiential understanding of the future built environment it represents: the material assemblies – such as windows, doors, countertops walls, and ceilings – that make up the building envelope and interior, the users and events that will occupy their proposed projects, and the spatial complexity of an inhabitable place. In short, far from being self-evident that a building and a drawing (or digital model) of a building are different in kind, it is often the case that one finds building designers doing precisely the opposite of what Rene Magritte suggested: treating drawings of buildings to be the same as the buildings they should be communicating; and this occurs because the designers assume a direct and immediate correspondence between their proposed buildings and their representations of them.

2 Between drawing and building Instead of the simple substitution that occurs in the situations described above, architectural drawings must be able to elicit a multitude of feelings and under-

648

Randall Teal

standings, and offer a portal through which one can project into number of possible realities. This is important because good design arises via an iterative search where one zeros in on the best solution by trying out possibilities and then adjusting the results accordingly (imagine trying out words and phrases to in a search engine to better refine your results). Achieving this kind of refinement requires visuals that ask viewers to participate, imagine, and think. For example, consider of all those photos you have taken of a building interior or landscape that ends up, upon later viewing, to be a mere fragment of your recollection of the actual place. As this limit of the photograph suggests, it is not necessarily the degree of visual reality that is communicated in a representation that is decisive, it is the degree to which it can trigger a more extensive vision. This is why Evans argues that the thing that makes design drawings most valuable “is their very distance from being architecture” (Evans 1997: 154). Unlike a building, which is a specific fixed culmination of a process of design and construction, a drawing is an open-ended framework that can change its potential with a re-orientation, an erasure, a scribble, an overlay, an added line, or any other number of operations. Coupling this open-ended quality of the drawing with a clear view of its role in mediating between viewer and built reality can transform any drawing into a potent stimulus. For example, at a basic level sketches are very good tools exactly because they only provide specific hints of a building. That is, sketches are ambiguous in specific ways. Because of this quality, sketches are rarely taken to be reflective of a finished work, thus are not seen as fully determined designs; rather, they prompt viewers to dream in a particular direction. They elicit possibility via the thinking that is spawned by their ambiguity. Counter to the ambiguity of the sketch is the preliminary drawing that looks designed. I remember a specific occurrence of this problem from my own practice where I had roughed out an idea for a client in a digital model. In order to aid the client’s visualization (so I thought), I proceeded to add some detail, materials, color and lighting to the model. However, to my surprise, when I showed the images to my client she got a sad look on her face and she said disappointedly, “oh, so I guess the design is done”. What had happened was that the visual communication of this technique said, “read these images as the photos of a real place”, rather than the rough approximations I had intended them to be. Interestingly, this kind of misunderstanding does not just happen with clients who are unfamiliar with design drawing conventions. It also happens to the designers themselves; and this occurs when the mediating role of the architectural drawing gets lost. The frequent result of this lapse is that designers will clarify, articulate, and refine compositional relationships and details “for the sake of the drawing”. In other words, certain marks and representational conventions help make drawings read well as drawings, but do not necessarily translate into good architecture. In fact, often when translated into architecture, these same marks and conventions can actually degrade the architectural composition. The detailing of brickwork

Thinking visuals

649

Fig. 2: A material mash-up. Photo credit: Randall Teal.

offers a great example of this. There are many beautiful architectural subtleties – like cornices, string courses, and articulated panels and reveals – that are executed in traditional brickwork that, when drawn as such, will lack the contrast and dynamics of a visually striking drawing. However, if one misconstrues such subtleties in the constructed whole to be visual details, then it is likely that an amplification of the details will occur in order for the drawing to read well. This is the way the metric (actual built details) comes to be evaluated and described projectively, and ultimately physical assemblies get refined by the criteria of a compelling drawing instead of those of a well-articulated building. When design media is not used to envision “built” refinements (i.e. drawing conventions are conflated with building assemblies), our built environment becomes choked with buildings characterized by either overblown detailing; or buildings that neglect detailing altogether. For a tangible example of what this confusion helps to create let us continue on with the example of architectural brickwork. Contemporary brickwork is frequently non-structural and thus has a freedom that traditional brickwork did not have. Unfortunately, this freedom plays right into the confusion of the projective and metric. Here, brick tends to be treated as an inert material, used by many as a mere colored exterior cladding. Such a limited deployment of brick misses the opportunities inherent to the material (utilized by traditional masonry) and thus often resorts to either treating the brick as boring sheets of weatherproofing, or makes these now flat and undeveloped walls more “interesting” through hollow gestures or material mash-ups. The material mash-up is a perfect example of the kind of overcompensation that occurs when a designer designs a building like a drawing. That is to say, the

650

Randall Teal

Fig. 3: Brickwork with little thought given to detailing (foreground) verses a more nuanced brick design. Photo credit: Randall Teal.

Fig. 4: Treating a wall as a material assembly (rather than a drawing) yields depth, variety and interest. Photo credit: Randall Teal.

material mash-up is generally a result of a projective clarification of the relationships of the different parts of the building within the drawing itself. So for example, in the included image of a material mash-up, instead of finding sophistication in the detailing of the surfaces and connections, we see clumsy details and largescale shifts in material and color used to create interest and demarcate the tripartite composition. That is, the bottom is designated by the concrete block, the middle is brick and/or clapboard siding, and the top corrugated aluminum. And unlike many traditional brick buildings, all of the materials present in the mash-up image

Thinking visuals

651

Fig. 5: Treating a wall as a material assembly (rather than a drawing) yields depth, variety and interest. Photo credit: Randall Teal.

are strongly contrasted against one another; and they all occur in virtually the same plane (i.e. there is no depth in the surfaces or connections), making a fairly superficial composition. Now it must be said that this discussion is not a call for a return to traditional brickwork – as I have said, construction parameters have changed. Rather, this is a call to rediscover the care that is evidenced in traditional brickwork so that we might cultivate parallel opportunities with our contemporary materials and techniques. Without this search, building assemblies will continue to be treated as equivalent to visual details, and our cities will be inundated with the corresponding flat, thin, and either aggressively graphic (to compensate for their lack of subtlety) or mind-numbingly banal buildings that go with them.

652

Randall Teal

In order to move closer to understanding how a drawing might be used to direct one toward the complexity of the architectural experience, let us return to the effectiveness of the sketch. In the sketch it is the openness of the different juxtaposed elements that pull the viewer into participation, and thus open a window on a number of potential realities. However, a sketch is ultimately limited because it is a highly preliminary representation; it can only hint at a complex environment because it does not contain a comparable depth of information (the kind of information required for constructing a building). However, it is possible to retain the open dynamic of the sketch, while ascending to the next level of refinement. Here, an increased level of detail extended across multiple drawings becomes critical. Of this strategy, graphics guru Edward Tufte has said, “multiples amplify, intensify and reinforce the meaning of images […] relying on parallelism, well crafted multiples provide high-resolution views of complex material (Tufte 1990: 105, 112)”. In other words, achieving more complex communications relies upon not upon exhaustively complete description, but like the use of montage in film, it is the way both the images are crafted and how they produce an evocation of a greater whole.

3 Architectural confection In architecture, the use of multiple drawings has been a staple of the discipline, because multiple drawings used in concert extend both the definite and indefinite elements of the representation and utilize the power of comparison, mutual reinforcement, and modification that occur when different slices of information are

Fig. 6: Actual building based on Figure 7.

Thinking visuals

653

Fig. 7: Here, an elevation drawing and a section drawing are used in parallel to describe the material reality of the building above. Photo credit: Randall Teal; drawing credit: Randall Teal.

presented simultaneously. This strategy is invaluable for those attempting to describe the sheer density of any built environment. A good example of this is the analytique drawing, which architectural theorist Marco Frascari describes as being comprised of various elements “composed in different scales in an attempt to single out the dialogue among the parts in the making of the building (Frascari 1996: 502).” In other words, the analytique sought the architectural whole through the rigorous investigation into a building’s specific parts and their relations therein.

Fig. 8: Analytique: “Construction details”, Giovanni Battista Piranesi. Photo credit: (public domain) http://www.wikipaintings.org/en/giovanni-battista-piranesi/construction-details.

654

Randall Teal

Fig. 9: Analytique: “Construction details of the Cloaca Maxima”, Giovanni Battista Piranesi. Photo credit: (public domain) http://www.wikipaintings.org/en/giovanni-battista-piranesi/constructiondetails-of-the-cloaca-maxima.

Fig. 10: Analytique: “The Roman antiquities, t. 4, Plate XXXVII. Vista of the second-largest share of the Theatre of Marcellus”, Giovanni Battista Piranesi. Photo credit: (public domain) http:// www.wikipaintings.org/en/giovanni-battista-piranesi/the-roman-antiquities-t-4-plate-xxxvii-vistaof-the-second-largest-share-of-the-theatre-of.

Thinking visuals

655

One might imagine the communication of the analytique drawing as being somewhat akin to providing navigational directions to someone based on specific prominent landmarks rather than precise measurements. The former is an approach that depends upon experiential and embodied understandings of the overall area to be traversed and is infinitely interpretable as the journey unfolds and one becomes familiar with the region. So it is too with the analytique; its focus is on describing the “region” of the building via its “landmarks.” That is, it describes particular pointers that make clear the overall affectual zone that a building is to (or does) occupy; in this way, the design is both specific but free to evolve in either one’s imagination or during the actual construction. It was a means of representation that relied upon mental and physical interpretations and thus encouraged thinking. For example, although the Theatre of Marcellus analytique is a drawing of an existing artifact, let us – for the sake of better understanding the workings of the analytique – imagine it as a design drawing. Unlike the images of architectural projects we sometimes see used for marketing building projects, this drawing does not provide a marquee (visual) image of the “finished” building. Further, it does not seek to be read strictly in terms of a drawing; rather, it seeks to articulate particular building assemblies – the key components that will most determinant of the architectural affect – and because of this, as a drawing it seems slightly esoteric. However, this is only if we approach it as a purely visual communication. Instead, if we understand it to describe, with precision, the depth and articulation of the interrelated parts of this structure we begin to see how the drawing evokes the building whole as a kind of spirit that inhabits the space around the details being described. In a way, this drawing says, “get this detail right and you get the building right”. That is, the focus on the critical construction details of the column and entablature assembly indicates that this detail is the driving force of this building’s overall character; and as such, requires that additional supporting details of this building must respond to and be built in relation to this critical assembly. Interestingly, the thinking reflected in analytique drawings shows almost an almost total inversion of practices today. That is to say, in the past there was no direct and total description of the building, instead the building was only communicated partially; the architect’s job was to masterfully design the critical affective parts and orchestrate harmonious relationships between them, not to execute a full set of drawings that detailed all aspects of construction. In this way, the sketchlike ambiguity of the analytique fostered relationships with the builders and craftsmen who were trusted with the interpretation of the drawings and execution of the building. However, unlike the sketch, which is by its nature incomplete, preliminary, and “sketchy”, the analytique drawing operated via a technique of visual communication that exhibited both well-crafted specificity “and” strategic ambiguity. Unlike modern practices, these two terms were seen as non-contradictory, and in fact, productive. Having distance between the object and its representation was

656

Randall Teal

not a liability but an asset, because they made the viewer’s thinking obligatory. Tufte refers to this strategy of visual communication more generally as a “visual confection”, and putting a fine point on the worth of this strategy, Tufte claims that, “confections make reading and seeing and thinking identical” (Tufte 1997: 151).

4 Conclusion: The killing effect of linearity Computer technology has increased our ability to collect, compile, analyze and recall vast amounts of data; yet this innovation does not necessarily make this data more easily communicated, nor guarantee a thoughtful exchange between data and designer. In fact, under the digital aegis it becomes increasingly important to resist tools that “offer” to “think” for us. For example, the “building information modeling” (BIM) software we discussed earlier has a tendency to eliminate the notion of detailing as an interactive craft. Instead of facilitating a more acute awareness of how to bring together the elements of a design so as to create the greatest aesthetic (living) impact, BIM software encourages designers not to “design” components and assemblies, but to “select” components and assemblies – like windows and guardrails – that are packaged with the software as standards. This often occurs in student projects and is characterized by “finished” details that have absolutely nothing to do with the rest of their design. When asked why these details are the way they are, the answer is almost universally, “that is what was given in the software”. In other words, these students have missed the fact that the elements in question are building assemblies that people can (and should) design. When this oversight happens, details move from the proper domain of design question to forgone conclusion; further, such automation of finish adds an appearance of professionalism to our work that covers-up the incompleteness of the thought that has gone into the design. Here, visualizations become impediments to creative thinking. Such non-thinking in design is not just an architectural problem. For instance, a similar situation to the BIM example arises through the use of PowerPoint’s “auto-content”, where one communicates just enough “finish”, and sense of competence to legitimate ideas of marginal worth (or sometimes, undermine ideas of immense value). Here, there is an appearance of communication, but one that is not particularly rhetorical. In such cases, there is often little craft employed to determine the what’s, how’s and why’s, of the visuals, and consequently of the ideas themselves. Further, unlike the example of the analytique drawing where suggestion and ambiguity are critical to connecting visuals to architectural realities, the typical PowerPoint-based presentation lives on the idea that visual communication means that the speaker’s content is duplicated in the slides and that

Thinking visuals

657

“visual communication” equals fully and directly illustrated, period. Unfortunately, such a belief misdirects one’s energy away from the heart of good communication – the kind that can discover and reveal understandings about complex systems. Here, visuals are created to “support” ideas and thus encourage reflection from their audience. Unlike a typical bullet-point presentation, where the communication of ideas and the conveyance of facts is frequently conflated, effective visual communication transforms mere looking into thinking (Tufte 2006). The conflict between the metric and projective in architectural communication is indicative of impediments to visual communication more generally, because it what happens when communications are not responsive and appropriate to the material they are trying to communicate. Like effective architectural communications, visual communications generally benefit when visual information is employed to provoke, reveal, and describe in a way that activates embodied understanding. The challenge to such communication is parallel to one of the obstacles to metric description in architecture: it is often easier to create optically oriented communication, because it is direct – it is what it is; an open sign for a restaurant needs to only communicate this fact, and a yard-sign for a political candidate needs to only make the candidate’s name memorable. However, a presentation that is intended to promote water conservation, for example, must not simply say this, but must help the audience imagine living through the manner in which a community gets its water, the daily actions and activities can either conserve or waste this resource, and the benefits/consequences of water conservation. In other words, in an example like this, visuals need to be employed in a way that asks the audience to mentally rehearse the information into their lives. Such a result, however, demands that the author is also performing this same rehearsal; and perhaps most importantly, in developing the communication strategy, that the author is using their evolving visualizations to help them facilitate their own thinking. It is such a process that initiates a thinking visual.

References Evans, Robin. 1997. Translations from Drawing to Building. In Robin Evans (ed.), Translations from Drawing to Building and Other Essays. Cambridge: MIT Press. Frascari, Marco. 1996. The Tell-the-Tale Detail. In Kate Nesbitt (ed.), Theorizing a New Agenda for Architecture, 28–31. Princeton: Princeton Architectural Press. Pérez-Gómez, Alberto and Louise Pelletier. 1997. Architectural Representation and the Perspective Hinge. Cambridge: MIT Press. Tufte, Edward. 1990. Envisioning Information. Cheshire: Graphics Press. Tufte, Edward. 1997. Visual Explanations. Cheshire: Graphics Press. Tufte, Edward. 2006. The Cognitive Style of Powerpoint in Beautiful Evidence. Cheshire: Graphics Press.

Karel Kleisner and Timo Maran

30 Visual communication in animals: Applying a Portmannian and Uexküllian biosemiotic approach Abstract: In modern biology, the appearance of organisms is largely understood as an adaptation serving the survival function. Here we advance a biosemiotic perspective inspired by the works of Adolf Portmann and Jakob von Uexküll. From this perspective, the visual dimension of every living being can be understood as a representation of the evolutionary experience of a species. In the study of animal displays, biosemiotics focuses on the qualitative perspective in both the logic behind the emergence of such signs as well as their perception and interpretation by perceiving animals. In this study, we claim that convergence of the animal surface patterns that stem developmentally from different ontogenetic precursors should be taken as evidence for congruence in biological meaning. As a specific example, the article discusses the development, perception, and evolution of semantic organs such as eyespots on butterfly wings and vertebrate eyes.

1 Diversity of visual communication in animals Studying visual communication in non-human animals always means taking a comparative perspective. Visual communication channels, i.e. organs of perception and expression, remain invariant in human visual communication; but between differing species, the specifics and effects of these can differ to a large extent. And so to focus on any singular trait, or even to build up a typology based on any single criterion, produces a simplified picture of animal visual communication. For this reason we claim in this chapter that the Portmannian and Uexküllian biosemiotic approach is potentially useful in studying animal visual communication as it provides a holistic understanding by incorporating several causal mechanisms, taking the specifics of living systems into account, and also including within it the animal’s subjective perspective. On a purely physical level, the range of perceivable light-waves differs between taxonomical groups: many birds and insects are able to perceive ultraviolet patterns whereas many reptiles sense infrared wavelengths. This means that human perception of the visual communication of other species may be biased due to our own physiological capacities. For instance, many differences or similarities that are significant to the members of other species remain outside our visual range. Thus, we do not sense the different coloring of the head cap of the male and female blue tits in UV, which is an important marker in their interspecific communication (Andersson et al. 1998), and it is easy for us to distinguish between

660

Karel Kleisner and Timo Maran

the colors of orchid red helleborine and bellflowers, which are undistinguishable for certain bees for whom these form a mimicry of resemblance (Nilsson 1983). In addition to the simple range of wavelength, visual perception may differ for different animal species in many other aspects. It appears that we humans, as well as many other primates, have exceptionally good color vision when compared to other mammals, whereas other mammalian taxa, cats in particular, may have sharper eyesight in low-light conditions. Also, the ways in which animal eyes organize the color information can be rather different. Our visual apparatus consists of three types of retinal cells that distinguish between base colors – green, blue, and red – and renders all hues as a combination of these; birds have four types of color receptors (one of which is suited to perceive UV wavelengths) and accordingly, for birds, four base colors exist. When the many evolutionary pathways and the variety of anatomies of visual perception organs from simple unicellular structures capable of distinguishing light from non-light to the compound eyes of insects which have a large field of vision but relatively low resolution (though even here there are exceptions, as for instance dragonflies) are added to all the varieties of eyes in vertebrates, one can see that taking the comparative perspective to animal visual communication is a real necessity. Looking at visual signals themselves, one can distinguish between several types based on a formal criterion. One possibility would be to pose the question: how do colors emerge? This would allow a distinction between the visual signals of emitted colors (as in fireflies Photuris) and reflected colors, whereas the latter type could be further divided between embodied visual signals, those that exist within an animal’s body structure, and detached signals, which would include territorial markings, tracks, nests, feathers, and other environmental signs of the animal’s presence. Based on the physical mechanisms of color generation, embodied visual signals could in turn be divided between physical colors (blue, green, and metallic tones that are caused by color diffraction in the microstructures of body integuments) and pigmental colors (mostly brown, black, yellow and red tones that are caused by selective light absorption in chemical compounds). Finally, specific body structures exist through which messages of visual communication are expressed in animals, and which vary greatly (including outgrowths, spikes, outer ears, horns, feather structures, tales, among others). In addition to such purely formal description, any analysis of specific cases of visual communication in animals should pay attention to the spatial or temporal organization of the signals of a given species. Depending on taxonomical group, colors in animals can form patterns and specific indexical signs, for example the pale “follow-me” sign below the tail of many ungulates. Visual signals can also form repertoires of behavioral acts with a communicative function, e.g., the facial gestures or body postures of many group-living birds and mammals that are depicted in ethology as ethograms. In temporal means, visual signals can form specific sequences of ritual behavior, e.g., “wedding dances” and other forms of

Visual communication in animals

661

courtship behavior. In their full species-specific complexity, animal signals can be dynamically employed in dialogic or polylogic encounters between the members of one and the same or different species, and they can be described as conveying meaning or function based on codes that organize specific acts of communication. A well-known example of spatially and temporally complex visual communication systems in animals is claw movements in signaler crabs, which are used in speciesspecific sexual and territorial communication. Another much studied case is the “dance language” honeybees use to communicate information about the position of a food source or other resource, relative to the bee-hive and the position of the sun. In its particulars, this appears to be a well-organized communication system that allows the conveyance of referential information about environmental objects (thus matching much of the criteria specific to human language e.g., Hockett 1960). Concerning the function of visual signals of animal communication, contemporary evolutionary biology mostly tries to establish that specific animal signals have selection value for the given animal species, thus contributing to its survival. Such outcome may relate to the effect of natural selection, for instance where the warning coloration of an insect species announces that it is unpalatable or poisonous. Alternatively, such an outcome can be related to sexual selection, as where vivid red and yellow colors in the plumage of the males of some passerine birds supposedly reveal their health and diet to the females. Given the variety of visual communication systems in animals, the function or meaning of a particular message can also be much more concrete. For instance, the German theoretical biologist Günter Tembrock (1971: 56) has described the basic categories of meaning that are transmitted in intraspecific communication as follows: (1) identity of the sender (species, group, age, sex, and individuality); (2) motivation (physiological status such as hunger and behavioral status such as intention to fly); (3) other living beings (dangerous and non-dangerous animals), things, territory, food, meteorological conditions. To present a typology does not mean to imply that all the colors in animal body surfaces necessarily have meaning or function. It is plausible to assume that there are also many aspects in animal body coloration that have no special meaning or function. In other cases, it is more adequate to talk about meaning complexes that can include several actualized or potential meanings, as we are about to show in the following semiotic excursion into animal appearances.

2 Introducing Portmannian and Uexküllian biosemiotics Biosemiotics generally studies the semiotic processes (e.g. recognition, communication, and interpretation) that are considered to exist in a variety of forms down to the simplest living organisms and to the lowest levels of biological organization.

662

Karel Kleisner and Timo Maran

Biosemiotics can focus on codes and coding in biological systems, biological functions of semiotic processes, typology and evolution of sign usage, etc. (see Kull et al. 2008). Compared to mainstream Darwinian biology, the biosemiotic and especially Portmannian-Uexküllian biosemiotic approach is closer to many treatments of visual communication in the humanities and social sciences. This is because Portmannian-Uexküllian biosemiotics combines structural descriptions of the ecologies, physiologies, and development of animals with a subject-centered focus. Many of Uexküll’s basic concepts such as building plan (Bauplan), form-shaping rule, functional cycle (Funktionskreis), and counterpoint (Kontrapunkt) can be interpreted as tools of structural description; and both Portmann and Uexküll emphasize the subjective perspective by treating animals as perceiving, expressing, and communicating creatures (e.g. Uexküll’s analysis of animal Umwelten and Portmann’s concepts of self-representation and inwardness). It can be assumed that Portmannian-Uexküllian biosemiotics is more capable at describing qualitative aspects of the specific appearances in animal species. The Swiss zoologist Adolf Portmann (1897–1982, born and died in Basel) belongs in any list of the most original thinkers of twentieth-century biology. His originality was not founded by any new radical invention, but rather by elaborating the classical ideas of German romantic science. Portmann’s ideas were highly influenced by a tradition of continental speculative philosophy; perhaps this is why, with a few exceptions, philosophers and social scientists have appreciated his work, rather than biologists (see Gould 1977: 349). Adolf Portmann understood organic form as something of value in itself which should be studied as such. In his thinking, the external surface of an organism has its own formal value and a certain kind of autonomy over other life-sustaining functions. He was convinced that this outermost aspect of an organism informs us about the innermost dimensions as these surface manifestations reflect the inner self-experience of every living being. He stated that the only way to get closer to understanding the existence of other living creatures is by a sensitive interpretation of the outermost organic surfaces (Portmann 1960a, 1960b, 1965, 1969: 315). Though Portmann’s views may seem part of German idealistic biology, his approach does take animal appearance into account as a whole, making it very suitable for studying the complex visual patterns of an animal.1  This tradition of biology was common in German intellectual space, i.e. besides Germany it was especially common in Switzerland, Holland, Austria, the former Czechoslovakia, Estonia, and St. Petersburg in Russia, but some of its adherents also lived in France and England. Idealistic biology crystallized in the works of German Naturphilosophen towards the end of the eighteenth and beginning of the nineteenth century. It stresses the concepts of form, metamorphosis, and different kinds of holistic organization. Individual development (ontogeny) was often taken as co-logical with evolution. The rapid development of idealistic biology in the first half of nineteenth century was impaired by various subsequences of Darwin's appearance (1859), though in continental Europe it survived in various diasporas until the second world war. The foremost representatives of this intellectual line were L. Oken, J. W. Goethe, K. E. v. Baer, E. G. Saint-Hilaire, R. Owen, W. Troll, and A. Naef.

Visual communication in animals

663

Jakob J. von Uexküll (born 1864 Estonia, died 1944 Italy) developed an original semiotic theory of nature that emphasizes meaning as the organizing force of animal physiology as well as individual relations with the living and non-living environment (Uexküll 1982). His work was inspired by the philosophies of Immanuel Kant and J. Wolfgang von Goethe and, in turn, influenced the latter ethological studies of Konrad Lorenz and Nikolaas Tinbergen. In the context of the present paper, we focus particularly on Uexküll’s notion that the specific form of an animal – its features and ontological development, the functional relations it has with others of its own species, especially its mates, but also with representatives of other species, physical forces etc – is interconnected by meanings.

3 Semantic organs The exposed surfaces of organisms play a crucial role in visual communication. It is largely through these that animals recognize each other’s species, sex and individuality, and these are the primary displays with which an animal can express its intentions and moods. Let organic appearances that may potentially enter the Umwelten of other living beings be called semantic organs (“semes” in short; Kleisner and Markoš 2005; Kleisner 2008b). In formal language, semantic organs can be defined as semiautonomous relational entities dependent on Umwelt-specific interpretation (Kleisner 2008a). It is remarkable that semantic organs can be defined neither by listing solely their anatomical, morphological, developmental or genetic components nor by unambiguous attribution of a particular signaling function. More precisely, every semantic organ exists at the interface between the expression of bodily features and their meaning within the Umwelt of a receiver. Semantic organs come into existence through semiotic cooption and semiotic selection. Semiotic selection occurs because semiosis literally operates through perception, interpretation and feedback, being thus an evolutionary derivative of Uexküll’s Funktionskreis (Maran 2008: 177; Maran and Kleisner 2010). Semiotic cooption happens when a trait expressed by an organism is recognized as meaningful by another organism (Kleisner 2011). It does not matter whether the co-opted trait was shaped to serve some specific function or had no function at all. In this respect, semiotic cooption is a kind of exaptation operated by the sensory channels and cognitive sphere of an organic subject (on exaptation, see Gould and Vrba 1982; Weible 2012). In our following exposition, we take eyespots as an example of semantic organs par excellence. Though eyespots take on immense diversity of form and color, still they bear something in common: the meaning of an eye. Eyespots are found in almost all kind of ecosystems whether underwater, in the air, or on terra firma. These iconic signs (i.e. signs based on resemblance) occupy the bodies of organism living nowadays as well as of those that were fossilized long before mammals first appeared.

664

Karel Kleisner and Timo Maran

By focusing on this specific case, we hope to demonstrate the methodology and argumentation characteristic to the Portmannian-Uexküllian biosemiotic approach. Throughout the following exploration we highlight the complex nature of the visual appearances in animals that needs to be maintained to the final conclusions of our study. Our methodological approach is to analyze and finally synthesize the following aspects of animal appearance: 1. distribution and diversity across species and groups; 2. array of the possible functions of the phenomenon (eyespots in the present study); 3. evolutionary history of the particular appearance and the species involved; 4. developmental background of the individual; 5. behavioral activities of the animal (i.e. which behaviors are needed to make the appearance function as a message in the given communicative situation); 6. interpretations of appearances by other species (e.g. predatory species), their feedback and further effects. We address these methodological points in pairs in the following sub-chapters: 1 and 2 in “Eyespots – their distribution, morphological diversity and functions”; 3 and 4 in “Eyespots in evolution and development”; 5 and 6 in “Behavioral components do matter”. The following survey should lead us to a complex model of animal visual communication systems that is qualitative (e.g. by preferring the concept of appearance over signals, inter alia), sees the semiotic faculty (meanings and signification) as a main organizing force of communication, and highlights the interplay between meanings in communication and evolutionary history.

4 Eyespots – their distribution, morphological diversity and function Different spots and eyespots may be found in a number of animals of different phylogenetic origin such as squids, octopuses, turtles, sharks, rays, fishes, amphibians, reptiles, birds, and especially insects. The various eyespots on the surfaces of animals were traditionally considered representations of eyes. One may suggest this is only imaginative, resulting from subjective “anthropomorphic” projections that have nothing to do with modern science. Here we do not follow such an ultimate purification of the meaning of these iconic signs. We would rather ask whether the similarity between eyespots and eyes reflects a common principle that exceeds the range of human cognition and touches the other living beings in a similar way. In the most general sense, eyespots are mostly considered to be survival devices that deter enemies. Eyespots on the wings of butterflies especially represent the most intensively studied semantic system. The experimental approach is con-

Visual communication in animals

665

tinuously perfected to bring more reliable results. There are two major explanations for the protective function of eyespots. The first represents an “intimidation” hypothesis according to which the function of the eyes is to deter predators, and prevent an attack. The usual reasoning here is that eyespots intimidate predators because they represent an imitation of vertebrate eyes: this is the “eye mimicry” hypothesis (Blest 1958). Recently, this hypothesis has been strongly criticized, especially by authors who perform field behavioral experiments (Stevens et al. 2007, 2009). In this view, eyespots serve an antipredation function, not because they are representations of eyes but because they are conspicuous. The fact that many of them resemble a vertebrate eye to a human observer arguably has nothing to do with their warning function. Eyespots effectively intimidate the predator’s attack because they are simply conspicuous and highly contrasting, not because they imitate vertebrate eyes. This understanding led Stevens et al. to abandon the term “eyespots” as anthropocentric while proposing the term “wingspots” for these structures in butterflies, “finspots” in fishes, etc. This hypothesis is based on the deflection of a predator’s attack away from the vital body parts, e.g. the head, to less vulnerable ones, e.g. the wing margins. Current support for the deflection hypothesis remains equivocal (Lyytinen et al. 2003; Hill and Vaca 2006; Vlieger and Brakefield 2007). Eyespots have yet other functions such as playing a signaling role in sexual selection. In the butterfly Bicyclus anynana (Nymphalidae), for instance, the dorsal wing pattern is supposed to partake in female mate choice whereas the ventral pattern serves a camouflaging and/or predator-deflecting role (Brakefield and Reitsma 1991; Lyytinen et al. 2004; Stevens 2005). Despite the current research progress in the behavioral ecology of antipredation signals, there are still some questions that transcend the focus of the mainstream adaptationist agenda. Our concern with eyespots occurs prior to formulating and testing a hypothesis. We assume that some pre-understanding to “what is an eye?” is always present when we approach objects that are conspicuous, circular, and concentric. One should ask: do eyespots enter the Umwelten of nonhuman animals as representations of an eye? May eyespots represent vertebrate eye mimicry? How has it come about that different circular objects are experienced as eyes? And, how have various representations of the eye come into existence?2 In the following, we will argue that neither the “eye mimicry” hypothesis nor the “conspicuous signal” hypothesis represents an appropriate explanation of the evolution of eyespots. Here, we propose a third perspective which differs from the previous “either … or” scenario and, at the same time, remains compatible with recent findings.

 For a more detailed application of the semiotic methodology to the study of animal communication, see Maran 2010.

666

Karel Kleisner and Timo Maran

5 Eyespots in evolution and development There are two major hypotheses on the evolution of eyespots in butterflies. The first proposes that eyespots originated from the undifferentiated row of elements (border ocelli) – each occupying a single wing compartment (or wing cell) – that became further individuated and diversified in number and morphology. According to the second scenario, eyespots originally appeared as single autonomized compartmental elements that were duplicated and repeatedly co-opted by other wing compartments (Monteiro 2008). Morphologically, the wings of satyrid and nymphalid butterflies often bear a row of eyespots that are homologous to the border ocelli within the generalized wing pattern of the nymphalid ground plan (Süffert 1927; Nijhout 1991); see Figure 1. By contrast, the eyespots on the hindwings of some sphingid moths originate from elements of the central symmetry system, that is, from a different part of the nymphalid groundplan. Inasmuch as the element of border ocelli consists of an array of eyespots, the whole system has a modular character similar to that of the vertebrate backbone. In the ancestral state, the eyespots were presumably highly coupled, both genetically and developmentally. However, during the course of evolution, acting adaptive forces led to their genetic decoupling, favoring individuation and independent diversification of single eyespots (Beldade and Brakefield 2003). Artificial selection

Fig. 1: Generalized ground plan of wing patterns of family Nymphalidae according to Süffert (1927).

Visual communication in animals

667

applied to eyespots had a rapid effect on size, location and color composition of the eyespots, whereas much lower heritability was reported for eyespot shape during selection for elliptical eyespots in the anteroposterior and proximodistal axes (Monteiro et al. 1997a; Monteiro et al. 1997b). As a result, it seems evident that when compared to other parameters, the circular shape of the eyespots is strongly constrained. This is probably because of developmental-mechanistic reasons during the formation of the eyespot such as the response of the surrounding epidermis to radial diffusion of a signal from a central focus point (Nijhout 1980, 1991; French and Monteiro 1994). However the same explanation can hardly be applied to instances where the eyespots do not originate from material homologous to border ocelli. The eyespots on the forewings of peacocks (Inachis io, Nymphalidae) represent a composite of neighboring elements of color pattern. The eyespots originated by fusion; integrations of different parts of a centrally symmetric system are frequent, for instance within sphinx moths, but also some mantids bear eyespots formed from spiralized stripes, etc. The suggestion of Stevens (Stevens et al. 2007: 526) that the high frequency of circles compared to the other shapes may be explained by “[…] the radial diffusion of a morphogen outward forming a concentration gradient, with the epidermal cells producing specific pigments depending on the morphogen concentration” is thus valid only for a limited number of eyespots, namely those derived from the marginal ocelli. But eyespots tend to be circular even when generated from noncircular precursors. Alternatively, “the circularity of these elements are favored because they increase the probability that the eyespots will be perceived as “eyelike” by a majority of possible receivers and thus attract the sight of predators” (Kleisner 2008a: 215). In other words, specific developmental mechanisms do not appear to fully explain the existence of generalized forms that are present in many species and groups. Within most studies, eyespots serve as a model system of development. Particular phenotypic changes in eyespot parameters provide information on the functioning of developmental mechanisms and genetic underpinnings. For such approaches, eyespots are an excellent experimental model, but this does not help us understand their close resemblance to vertebrate eyes. Again, one wonder how these eyespots became “eye-like” and how are they maintained as such during evolution.

5.1 Eyespots in fossils The role eyespots play in communication between and within species has a long evolutionary history. Eyespots are well evidenced from insect fossils. For example, members of the Eurasian family Kalligrammatidae were large insects from the Jurassic period that belonged to the order Neuroptera. Their conspicuously patterned wings, often with large eyespots on hind- and fore-wings, are well preserved in the fossil record. As nicely stated by Grimaldi and Engel:

668

Karel Kleisner and Timo Maran

Kalligrammatids were the “butterflies” of the Jurassic. Their densely setose and patterned wings and bodies and their long palpi gave them the superficial appearance of large moths or butterflies, despite occurring 60–90 MY before the first papilionoid fluttered […] It is tantalizing to speculate how the eyespots of kalligrammatids served a similar function of startle defense. It is equally interesting as to what peculiar Jurassic vertebrates kalligrammatids might have been mimicking, perhaps some small, insectivorous reptile or even an archaeopterygid bird? (Grimaldi and Engel 2005: 347).

Edmunds’ (1976) mention of eyespots found on the wings of the carboniferous insects from the order Protodiamphipnoa represents another example from an even older geological period. If these eyespots functioned as recent ones, the only possible vertebrate receivers of these signals would have been amphibians and/or reptiles (Komárek 1989). One should ask, however, whether or not these eyespots represented an imitation of the eyes of some particular animal group. The question whether there were any eyespots on the surfaces of animals from the period that preceded the evolutionary origin of the ventricular eye cannot be answered easily. Nevertheless, the eyespots found within the fossil record seem to be of the same forms and shapes as recent ones. The simplest explanation is that the eyespots probably did not represent an imitation of eyes, neither of a particular vertebrate species nor a vertebrate class. For now, we would rather say that eyespots generally do not imitate the eyes of any particular kind of vertebrate.

5.2 Eyespots underwater The different physical and physiological constraints of an aquatic environment do not prevent the communicational function of eyespots in inhabitants of marine and freshwater ecosystems. Various eyespots (or just spots) occur in many fish species. The most striking examples involve the presentation of the eyespots (finspot) on the caudal part of the body, while a dark band conceals the real eye, as seen in some species of butterfly fish (Chaetodontidae). As a result, the caudal part of the body fulfils the role of a false head. The entire effect is sometimes supported by movement opposite to body direction (backward in relation to real head). The coloration of the whole fish body, the concealed eyes and caudal fin-spots, together with associated behavior, together generates a context within which the spots carry the meaning of an eye and the caudal region that of a false head. The location of the eyespot and false head may be supposed to misdirect the attack of predator to less vital parts of the animal body as described by the misdirection hypothesis (Cott 1957; Neudecker 1989; Meadows 1993; Winemiller 1990). This hypothesis assumes that the eyespots should imitate the real eye in size and structural composition, while differing in position. Nevertheless, the caudal eyespots of many fish, for instance the four-eye butterfly fish (Chaetodon capistratus), markedly exceed the size of natural eyes. The larger size of the eyespots appears to refer to the presence of a larger animal, thus they would be more effective in

Visual communication in animals

669

intimidating a predator (the intimidation hypothesis). The occurrence of dislocated and large sized eyespots may also be explained by the “reaction-distance hypothesis” which assumes that the larger size of eyespots may induce predators to initiate an attack from too great a distance, thus giving a temporal advantage to the prey and potentially thwarting the attack. According to this hypothesis, the predators use the angular extent of the eyelike pattern on their retina as a cue to trigger the attack. Moreover, the oval or elliptic shape of the eyespots may confuse the predator about its actual angle of approaching trajectory and thus misdirect the attack (Meadows 1993). These examples show that explaining the particular appearance of the eyespots may require taking into account the interpreter’s perceptual specifics and behavioral preferences.3 It would be extremely interesting to test these ideas in the context of insect eyespots, as we often find therein either a series or single eyespots of different sizes and sometimes spatially warped shapes. It is worth noting that the eyes themselves appeared to be a very conspicuous signal within the newly discovered red fluorescence of reef fishes. Red fluorescent eye rings were reported in 30 species from four families (Gobiidae, Tripterygiidae, Blenniidae, Syngnathidae). The frequent occurrence of this feature indicates that fluorescent eyes may potentially function as a presence indicator or deceptive gaze signal (Michiels et al. 2008). We return to the importance of the conspicuousness of the animal eyes in the final part of our discussion.

6 Behavioral components do matter In addition to the structural diversity and the developmental and evolutionary history of animal appearance, its dynamical and behavioral aspects must also be taken into account. The comparison of effectiveness of deimatic coloration in two models of butterflies shows that the behavioral component can play a crucial role.4 Vallin et al. (2007) performed an experiment in which hawkmoths (Smerinthus ocellatus) and peacocks (Inachis io) were subjected to attack by birds (great and blue tits). As a result, the birds killed more hawkmoths than peacocks. This is surprising because both species have similarly sized eyespots, which were suspected of having a similar intimidating effect on bird predators. Both peacocks and hawkmoths are palatable and non-poisonous. It is worth noting, that hawkmoths are a much larger prey item, which should be easier to attack than the smaller peacocks. However, when peacocks are disturbed not only do they open their wings with the eyelike ornaments, they also start to flick their wings and  The recent research on coral reef fish Pomacentrus amboinensis also indicates the possible role of eyespots in social interactions (Gagliano 2008). The eyespots may take a role in rapid recognition of conspecifics, thus bringing a selective advantage to the species (Zaret 1977).  Edmunds (1974) used the term deimatic behaviors for all forms of displays, postures and sounds taken up by prey with the apparent purpose of intimidating an attacking predator.

670

Karel Kleisner and Timo Maran

allegedly turn around to follow the movement of the attacking bird (Vallin et al. 2007). The protective behavior of hawkmoths also involves “eyespots flickering” but this may seem especially impressive to human (mammalian) observers, while seen as more static and less frightening to birds. This is supposedly related to differences between the perceptual worlds (Umwelten) of humans and birds, the reaction times and temporal perception being generally much faster in the latter. A different species of hawkmoth, the tobacco hornworm (Manduca sexta), which has dorsal black and yellow bands on its abdomen, gives other example of correspondence between behavior and warning coloration. The abdominal pattern of Manduca somewhat imitates a wasp-like pattern. When Manduca sexta is irritated (attacked), it starts to move with its abdomen in a way that simulates the stinging behavior of aculeate hymenopterans (Evans 1983). The wasp-like coloration followed by an imitation of aculeate abdominal movements makes an overall expression quite intimidating for a potential receiver. The intimidating effectiveness of such deimatic displays thus depends on the behavioral repertoire of a bearer. Quite probably it could be the behavioral component that makes the eyespots a more effective anti-predation signal than the other conspicuous forms examined by the experimental design of these British behavioral ecologists. In other words, eyes and eye-marks should not be isolated as merely static displays, but considered in a way that emphasizes their dynamical and functional being – that is, within the context of activities such as looking, eye-blinking, head-turning, etc. However difficult it would be to test this possibility in the field, further research that integrates behavioral components with deimatic coloration is needed.

7 How a spot becomes an eyespot? Both theoretical arguments and experimental evidence have challenged the “vertebrate eye mimicry” hypothesis (e.g. Stevens et al. 2007). When we discard the concept of “eyespot” for the more or less neutral concept of “wingspot” – as suggested by Stevens and co-authors – we simultaneously abolish the question as to why some wingspots resemble eyes to us and perhaps also to other non-human receivers. Therefore, we should ask how wingspots become eyespots, or how a circular structure becomes an iconic sign that stand for an eye. Another source of criticism may point to the fact that experiments use an artificial design to examine the functioning of the anti-predatory role of eyespots – not butterflies themselves. This means that the possible active role of the prey in this prey-predator interaction is abandoned a priori. We have already shown that the behavior of the potential prey does influence a signal’s effectiveness, as probably do the specific features and capacities of the receiver, its response and co-action with the prey. We learn that different eyespots (wingspots, finspots) function because they are conspicuous, not because of their imitation of vertebrate eyes. It is also pos-

Visual communication in animals

671

sible, however, that they refer to an eye as a general model, since the eyes themselves are usually conspicuous structures (as shown in the previous examples in fish).5 Therefore, we suggest that if the conspicuousness hypothesis attempts to explain the functioning of eyespots it should consider the question of why vertebrate eyes themselves are often conspicuous. Very likely, vertebrate eyes are mimicked because they are conspicuous structures marking the position of the animals’ heads. Vertebrate eyes are in fact headspots, to use Stevens’ (2005) vocabulary. But also the eyes of some vertebrate species became conspicuous secondarily, only after they have adopted a signaling function. Usually only conspicuous parts of other animals deserve to be imitated, as one can see in the frequent cases of partial mimicry (e.g. false heads, eyespots). It is in principle impossible to describe the evolutionary origin of complex eyespots on the wings of butterflies without any reference to the interpretative act (the signification) on the part of predator that gives a meaning of “vertebrate eye” to the primarily raw structures on butterfly wings. Only after such prospectively eye-like structures acquire the meaning of the eye in the Umwelt of the animal interpreter, can selection act to make them more and more similar to the representation of a vertebrate eye in a predator’s Umwelt, and thus ultimately become closely similar to real vertebrate eyes (Kleisner 2008b). Moreover, the morphological diversity of complex eyespots on the wings of insects may hypothetically be explained by differing Umwelt-specific interpretations of predators that have acted as selective agents. In this sense, one can also think about the attempt to infer the predator’s inner representation of an “eye” from eye-like ornaments exposed on the wings of his potential insect prey (Hinton 1977). In other words, if one wants to discover the meaning of an eye in the Umwelt of a particular interpreter it will potentially be helpful to look at the eyespots on the wings of its potential prey. We conclude this section with a couple of questions. Can conspicuousness partly explain the evolution of eyespots’ aversive function? Yes. Does conspicuousness suffice to elucidate the evolution of eyespots? No. Eventually we suggest that only wing spots, fin spots and other spots that are semiotically co-opted as representing eyes in the Umwelt of a particular animal interpreter should be named eyespots. Spots that are not so involved in any animal Umwelten should rather be considered as structures with an existing but yet-unrevealed semiotic potential.

8 Theoretical implication of semantic morphology One may question the need for new concepts such as semantic organs, appearance, co-option and others when contemporary biology already has such terms as signal  “Mimicry of snakes by insects appears to be analogous to mimicry of vertebrate eyes by many species of insects. Many of these false eyes are very like the eyes of vertebrates: the incorporate

672

Karel Kleisner and Timo Maran

and cue by which to label organic appearances. We insist, however, that the vocabulary of signals and cues, and the biological rationale underlying these terms, does not fit many of the situations that we find within the animal world. A sender emits a signal to directly affect a particular receiver, i.e. a signal is a sign that serves manipulative purposes. Cues, on the other hand, are not specific to living organisms and could also be present and perceived within inanimate objects. What is wrong with these terms is that the concepts of cues and signals are rather used to denote specific features of an animal body or an environment, and not the animal body as a whole. By contrast, when considering the whole surface of an organism as derived meaningfully in the function of a living body, and self-display as a revelation of the otherwise inaccessible innermost potential of an organism (i.e. as representation of organic inwardness, in Portmann’s terms), a new terminology is needed. These concepts should, on the one hand, describe animal appearances as meaningfully organized wholes and, on the other, make a clear distinction between living and non-living surfaces. The apparent feature of all exposed organic surfaces is their uniqueness in relation to other bodily structures. Everything that comes into being could be potentially perceived by someone, therefore being existent means being somewhat apparent. Being existent as animate supposes that appearance has inner causes, for organisms build themselves. If you remove the surface layer of a rock, you will usually find the same pattern beneath. This is because the appearance of a rock does not represent the result of activity stemming from the rock’s centrality, i.e. from the rock itself. Looking under the coat of a mammal produces a quite different experience. Although this basic fact is evident even to children, most of today’s biologists seems behave as if they are unaware of it. These differences between inner and outer lie at the core of the division between life/non-life. Living beings have an inwardness that can be interpreted in evolutionary terms as an evolutionary experience gathered by a lineage during the temporally vast period of evolutionary history. This deeply anchored dimension of living beings simply cannot be erased, forgotten, or overwritten. For the same reason, neither can it be wholly imitated. When thinking in terms of semantic organs, we simultaneously refer to these innermost dimensions of living. While a sign is a representation of something other, the semantic organs are self-representations of organisms (thus coming close to “proper names” in a semiotic typology of the signs). In identifiably particular ways, they reveal the evolutionary experience of a lineage and thus also their specificity and individuality. To talk about organisms in terms of semantic organs means to take their ontological wholeness into consideration. There is one important point stemming from the logic of semantic organs. One could admit that the exposed surfaces of an organism are often imitated, and

eccentric circles and an apparently reflective high-light from a moist cornea. They can be astonishingly realistic, but they do not resemble the eyes of any particular vertebrate.” (Pough 1988: 81).

Visual communication in animals

673

because many organic surfaces can be conceived as semantic organs, so too semantic organs must be subjects of imitation. Imitation is indeed ubiquitous in the living world, but semantic organs cannot be imitated easily. What we mean by this is that such an imitation would never be safe for the mimic (the imitating organism). As semantic organs are tightly related to organic inwardness (and living experience), changing a semantic organ will also change that being’s inwardness. We should now correct the former statement that “semantic organs cannot be imitated easily” to suggest that “semantic organs cannot be imitated ‘superficially’”. For example, take numerous cases of mimicry where a model is imitated by one or more mimic species. In the case of Batesian mimicry, the model is protected (unpalatability, sting, poison etc.), whereas the mimics have no protection. It is apparent that mimicking organisms gain a selective advantage by adopting the semantic organs (semes) of a model. In classical neo-Darwinian exposition, the story ends at this point. We suggest that advantage in survival is necessarily connected with a less apparent disadvantage in terms of self-representation. This is the loss of species-specific semes. This is what a mimic must pay for increased reproductive success, or in other words, for hiding behind a model. The most obvious example of this is the necessity for new types of intraspecific communicative mechanisms to arise, as imitating other species may cause previous speciesspecific communicative mechanisms to collapse. But this lead to crucial consequences as: […] self-representation of mimic does not longer stand for the presentation of the self, but it is the presentation of the semes of a model on the body of the mimic. In other words, bodies of the mimics serve as a projecting screen for the semes of the model. So, in fact, the display of a mimic represents the “self”-representation of the model (Kleisner and Markoš 2009: 305–306).

By analogy from human society, losing face is also conceived as a dramatic event with non-trivial consequences for an individual. The other example can be seen when someone aims to ‘clear’ his or her family name; simply adopting a new “false” family name is usually not considered a satisfying solution. Some of the ideas presented in this chapter remain unacceptable in mainstream biological thought. However, without taking the “self” of living beings seriously as a representation of evolutionary experience gathered into a lineage by its unique evolutionary history, we will be in danger of erroneous reasoning about evolution. For the humanities and social sciences, biological treatment of communication may appear predominantly focused on evolutionary history and functionality in message conveyance. In this context, the Portmannian-Uexküllian biosemiotic approach attempts to introduce greater awareness about the historical and communicative contexts of meanings and the roles of the sender and the interpreter in bringing forth and shaping the messages in communication.

674

Karel Kleisner and Timo Maran

Acknowledgments: We thank David Machin for the kind invitation to contribute to this volume. Karel Kleisner was supported by Czech Grant Agency project GACR P505/11/1459, and by Charles University in Prague project UNCE 204004, Timo Maran’s research was supported by European Union through the European Regional Development Fund (Centre of Excellence CECT, Estonia) and by Estonian Science Foundation Grant No 7790.

References Andersson, Staffan, Jonas Örnborg and Malte Andersson. 1998. Ultraviolet Sexual Dimorphism and Assortative Mating in Blue Tits. Proceedings of the Royal Society of London B 265(1395): 445–450. Beldade, Patrícia and Paul M. Brakefield. 2003. Concerted Evolution and Developmental Integration in Modular Butterfly Wing Patterns. Evolution & Development 5(2): 169–179. Blest, A. D. 1957. The Function of Eyespot Patterns in the Lepidoptera. Behavior 11(2/3): 209–256. Brakefield, Paul M. and Nico Reitsma. 1991. Phenotypic Plasticity, Seasonal Climate and the Population Biology of Bicyclus Butterflies (Satyridae) in Malawi. Ecological Entomology 16: 291–303. Cott, Hugh B. 1957. Adaptive Coloration in Animals. London: Methuen Press. Edmunds, Malcolm. 1974. Defence in Animals. A survey in Antipredator Defences. Harlow: Longman. Edmunds, Malcolm. 1976. The Defensive Behaviour of Ghanaian Praying Mantids with a Discussion of Territoriality. Zoological Journal Of the Linnean Society 58(1): 1–37. Evans, David L. 1983. Relative Defensive Behavior of Some Moths and the Implications to Predator-prey Interactions. Entomologia Experimentalis et Applicata 33(1): 103–111. French, Vernon and Atónia Monteiro. 1994. Butterfly Wings: Colour patterns and now gene expression patterns. Bioessays 16(11): 789–791. Gagliano, Monica. 2008. On the Spot: The absence of predators reveals eyespot plasticity in a marine fish. Behavioral Ecology 19(4):733–739. Gould, Stephen Jay and Elisabeth S. Vrba. 1982. Exaptation – A missing term in the science of form. Paleobiology 8(1): 4–15. Gould, Stephen J. 1977. Ontogeny and Phylogeny. Cambridge, MA: Harvard University Press. Grimaldi, David and Michael S. Engel. 2005. Evolution of the Insects. Cambridge: Cambridge University Press. Hill, Ryan I. and Jarol F. Vaca. 2006. Differential Wing Strength in Pierella Butterflies (Nymphalidae, Satyrinae) Supports the Deflection Hypothesis. Biotropica 36(3): 362–370. Hinton, Howard. 1977. Mimicry Provides Information about the Perceptual Capacities of Predators. Folia Entomológica Mexicana. 37: 19–29. Hockett, Charles F. 1960. Logical Considerations in the Study of Animal Communication. In Wesley E. Lanyon, and William N. Tavolga (eds.), Animal Sounds and Communication, 392–430. Washington: American Institute of Biological Sciences. Kleisner, Karel. 2011. Perceive, Co-opt, Modify, and Live! Organism as a centre of experience. Biosemiotics 4(2): 223–241. Kleisner, Karel. 2008a. The Semantic Morphology of Adolf Portmann: A starting point for the biosemiotics of organic form? Biosemiotics 1(2): 207–219. Kleisner, Karel. 2008b. Homosemiosis, Mimicry, and Superficial Similarity: Notes on the conceptualization of independent emergence of similarity in biology. Theory in Biosciences 127(1): 15–21.

Visual communication in animals

675

Kleisner, Karel and Anton Markoš. 2005. Semetic Rings: Towards the new concept of mimetic resemblances. Theory in Biosciences 123(3): 209–222. Kleisner, Karel and Anton Markoš. 2009. Mutual understanding and misunderstanding in Biological Systems Mediated by Self-representational Meaning of Organisms. Sign Systems Studies 37(1/2): 299–310. Komárek, Stanislav. 1989. Vorkommen, Morphologie und Evolution der Augenmuster in der Flügelzeichnung der Familie Sphingidae. Zoologische Jahrbücher. Abteilung für Systematik, Ökologie und Geographie der Tiere 116: 217–254. Kull, Kalevi, Claus Emmeche and Donald Favareau. 2008. Biosemiotic Questions. Biosemiotics 1(1): 41–55. Lyytinen, Anne, Paul M. Brakefield and Johanna Mappes. 2003. Significance of Butterfly Eyespots as an Anti-predator Device in Ground-based and Aerial Attacks. Oikos 100: 373–379. Lyytinen, Anne, Paul M. Brakefield, Leena Lindström and Johanna Mappes. 2004. Does Predation Maintain Eyespot Plasticity in Bicyclus Anynana? Proceedings of the Royal Society of London B, 271: 279–283. Maran, Timo. 2008. Mimikri Semiootika. [Semiotics of mimicry, Tartu Ülikooli doktoritöid]. Tartu: Tartu University Press. Maran, Timo. 2010. Semiotic modeling of mimicry with reference to brood parasitism. Sign Systems Studies 38(1/4): 349–377. Maran, Timo and Karel Kleisner. 2010. Towards an Evolutionary Biosemiotics: Semiotic selection and semiotic co-option. Biosemiotics 3(2): 189–200. Meadows, Dwayne W. 1993. Morphological Variation in Eyespots of the Foureye Butterflyfish (Chaetodon capistratus): Implications for eyespot function. Copeia 1993(1): 235–240. Michiels, Nico K., Nils Anthes, Nathan S. Hart, Jürgen Herler, Alfred J. Meixner, Frank Schleifenbaum, Gregor Schulte, Ulrike E. Siebeck, Dennis Sprenger, and Matthias F. Wucherer. 2008. Red Fluorescence in Reef Fish: A novel signalling mechanism? BMC Ecology 8: 16. Monteiro, Antónia, Paul M. Brakefield and Vernon French. 1997a. Butterfly Eyespots: The genetics and development of the color rings. Evolution 51(4): 1207–1216. Monteiro, Antónia, Paul M. Brakefield and Vernon French. 1997b. The Genetics and Development of an Eyespot Pattern in the Butterfly Bicyclus Anynana: Response to selection for eyespot shape. Genetics 146(1): 287–294. Monteiro, Antónia. 2008. Alternative Models for the Evolution of Eyespots and of Serial Homology on Lepidopteran Wings. Bioessays 30(4): 358–366. Neudecker, Stephen. 1989. Eye Camouflage and False Eyespots: Chaetodontid responses to predators. Environmental Biology of Fishes 25(1–3): 143–157. Nijhout, H. Frederik. 1980. Pattern Formation on Lepidopteran Wings: Determination of an eyespot. Developmental Biology 80(2): 267–274. Nijhout, H. Frederik. 1991. The Development and Evolution of Butterfly Wing Patterns. Washington: Smithsonian Institution Press. Nilsson, Anders L. 1983. Mimesis of Bellflower (Campanula) by the Red Helleborine Orchid Cephalanthera Rubra. Nature 305: 799–800. Portmann, Adolf. 1960a. Neue Wege der Biologie. München: Piper. Portmann, Adolf. 1960b. Die Tiergestalt. Studien über die Bedeutung der tierischen Ercheinung. Basel: Friedrich Reinhardt. Portmann, Adolf. 1965. Neue Fronten der biologischen Arbeit. In Günter Schulz (ed.), Transparente Welt. Festschrift zum 60. Geburtstag von Jean Gebser, 23–37. Bern: Huber. Portmann, Adolf. 1969. Einführung in die vergleichende Morphologie der Wirbeltiere. Basel: Schwabe & Co. Pough, F. Harvey. 1988. Mimicry of Vertebrates: Are the rules different? The American Naturalist 131: S67–102.

676

Karel Kleisner and Timo Maran

Stevens, Martin. 2005. The Role of Eyespots as Anti-predator Mechanisms, Principally Demonstrated in the Lepidoptera. Biological Reviews 80(4): 573–588. Stevens, Martin, Elinor Hopkins, William Hinde, Amabel Adcock, Yvonne Connelly, Tom Troscianko and Innes C. Cuthill. 2007. Field Experiments on the Effectiveness of ‘Eyespots’ as Predator Deterrents. Animal Behaviour 74(4): 1215–1227. Stevens, Martin, Abi Cantor, Julia Graham and Isabel S. Winney. 2009. The Function of Animal ‘Eyespots’: Conspicuousness but not eye mimicry is key. Current Zoology 55(5): 319–326. Süffert, Fritz. 1927. Zur vergleichenden Analyse der Schmetterlingszeichnung. Biologisches Zentralblatt 47(7): 385–413. Tembrock, Günter. 1971. Biokommunikation: Informationsübertragung im biologischen Bereich I. Berlin: Akademie–Verlag. Uexküll, Jakob von. 1982. The Theory of Meaning. Semiotica 42(1): 25–82. Vallin, Adrian, Sven Jakobsson and Crister Wiklund. 2007. An Eye for an Eye? – On the generality of the intimidating quality of eyespots in a butterfly and a hawkmoth. Behavioral Ecology and Sociobiology 61: 1419–1424. Vlieger, Leon and Paul M. Brakefield. 2007. The Deflection Hypothesis: Eyespots on the margins of butterfly wings do not influence predation by lizards. Biological Journal of the Linnean Society 92(4): 661–667. Weible, Davide. 2012. The Concept of Exaptation Between Biology and Semiotics. International Journal of Signs and Semiotic Systems 2(1): 73–88. Winemiller, Kirk O. 1990. Caudal Eyespots as Deterrents against Fin Predation in the Neoptropical Cichlid Astronotus Ocellatus. Copeia 1990(3): 665–673. Zaret, Thomas M. 1977. Inhibition of Cannibalism in Cichla Ocellaris and Hypothesis of Predator Mimicry among South American Fishes. Evolution 31(2): 421–437.

Maximilian Rapp and Markus Rhomberg

31 The importance of Murals during the Troubles: Analyzing the republican use of wall paintings in Northern Ireland Abstract: The Northern Ireland republican movement was visually emphasized by images on the gable ends of houses and walls in Belfast and (London-)Derry. These murals were used by the nationalist working class to get political messages across in order to mobilize people and to generate awareness as well as an iconographic expression of the ongoing social injustice. Soon the free space on the walls was exploited by paramilitary groups and was transformed into a visual medium for their political demands. Therefore, republicans developed different strategies over the years to use the paintings as a communication tool sui generis. In this paper we seek to analyze the political use of visual images on the nationalist movement during the ‘Troubles’ in Northern Ireland. We argue that (1) nationalists took over the use of murals from loyalism to initiate a well-planned mobilization campaign and (2) the IRA used murals to convey a strong ideological message to cement support for the republican movement.

1 Introduction Since more than 800 years republican Catholics, following the Celtic and Irish tradition, and Protestant loyalists, favoring the Union to Great Britain, are fighting in the North of Ireland in order to clarify the question of who should govern the territory. This cumulated in a civil-war like struggle in 1968, when the so-called “Troubles” began. Paramilitary organizations like the republican IRA (Irish Republican Army) or the loyalist UVF (Ulster Volunteer Force) spread fear and terror for many decades and were responsible for more than 3.000 deaths. However, this armed struggle was visually emphasized by images on the walls and gable ends of houses in the urban territories. These murals were used by both sides to get political messages across in order to mobilize people and to generate awareness for their terrorist and political campaigns. Soon the free space on the walls was exploited by paramilitary groups and was transformed into a visual medium for their political demands. This became especially important for the Catholic population, as their right to use traditional communication channels was restricted by the Unionists. Therefore, political wall paintings developed to a communication channel sui generis helping republicans to spread their propaganda and to document the political change in Northern Ireland during the times of the “Troubles”. In the context of this research we are seeking to find out how murals can be seen as a visual mirror for the republican movement in Northern Ireland’s context.

678

Maximilian Rapp and Markus Rhomberg

Furthermore we want to find out how the IRA used the images to articulate their goals and spread propaganda in order to boost recruitment for the republican movement. Our visual analysis is supported by evidence we have drawn from qualitative interviews and correspondence with leading figures from Northern Ireland’s politics, economy, science, institutions, together with citizens and former terrorists, as well as the painters themselves – the muralists. First, we want to give an overview on how the “Troubles” began and how visual images can be seen as a powerful tool in a political context. This is followed by outlining a theoretical framework for a historiographical analysis of murals. In particular, we examine how murals have been used since the death of the ten republican hunger strikers in 1981 to project the republican message and to mobilize new recruits for the fight for a united Ireland.

2 The “Troubles” in Northern Ireland Northern Ireland has been in the spotlight of world’s media coverage now for more than forty years, as images of hooded gunmen and threatening terrorists were spread all over the news channels. While many reporters focused on the religious division between Protestants and Catholics after the outbreak of the “Troubles”, academics soon pointed out that Northern Ireland suffers an armed struggle, which is based on the question of identity and territoriality (McGarry and O’Leary 1997). After the partition of the country in 1922, Catholics were treated as second class people in Northern Ireland, as they were deprived especially in the sector of housing and jobs (Gurr 1976). The different traditions and heritage as well as a mixture of socio-economic, political and geographical disequilibrium against Catholic citizens led to tremendous desperation on the republican side and later on to many civil right marches in the 1960s, which were violently separated by provoked loyalists and the Northern Irish police, the Royal Ulster Constabulary (RUC). The republicans stroke back in retaliation and soon the aggression and violence escalated on the streets of Northern Ireland’s urban territories for a period of time, until the peace talks intensified in the scope of the Good Friday Agreement in 1998 (Maloney 2002). Major bombings as well as dreadful killings like the one on “Bloody Sunday” 1972, where 14 innocent Catholics were shot down during a civil right march by British soldiers, developed Belfast and (London-)Derry into operational zones for decades to come. In addition, thousands of IRA guerrilla fighters used the Catholic society as a cover to bomb specific British or loyalist targets, while loyalist groups like the UVF, the Ulster Freedom Fighters (UFF) or the Ulster Defense Association (UDA) countered with targeted massacres and killings.

2.1 Visual reflection of the Northern Ireland conflict However, the intercultural hate between republicans and loyalists led to an uncontrolled and war-like struggle, which was also displayed on almost every wall and

The importance of Murals during the Troubles

679

on the gable ends of houses through aggressive, threatening and violent images, showing hooded gunmen or heavy armed terrorists. So, murals have, besides other public displays like curbstones (painted pavements in the colors of the British Union Jack or the Irish Tricolor) as well as flags, always enlarged the cultural gap between republicans and loyalists by mirroring pictures of sectarian hate (Rolston 2004), as they were predominantly used as propagandistic weapons to strengthen the cultural phalanx against the enemy identity (Bryan et al. 2009; McCormick and Jarman 2005). Republican wall-paintings in particular can be seen as a visual display of a social movement, which was radicalized after the IRA took over again in the early 1970s to fight for greater political voice and a reunited Ireland. While social movements are politically described as a group of people with a common ideology who try to achieve a certain goal, they are visualized in the murals, which depicted the goals of nationalism and helped to literally paint a common vision of a shared ideology. Interestingly, republicans did not paint murals before 1980 – nearly 70 years after loyalists began to emphasize their annual celebrations for the Protestant King William III by illustrating his victory at the Battle of the Boyne in 1689. Soon after adapting loyalist traditions, the IRA discovered the range of possibilities linked to the expression of opinion through visual wall art; not only could murals help convey a republican message, they also had the effect of marking nationalist territory and thus effectively creating no-go areas for British security forces. Multiple assaults by British soldiers or officers from the RUC with dye bombs to censor republican opinion only served to confirm to republicans the strategically importance of murals in a war of propaganda. Decades of social and political disequilibrium strengthened republican desire to articulate nationalist identity and culture, as well as to express outrage at social injustice. Virulent republican ideology was censored in much of the mainstream media and murals were one means of expressing republican goals and identity and ideology to a worldwide media with cameras rolling on the streets of Northern Ireland.

2.2 Thematic frame: a picture is worth a thousand words Visual symbols and icons play a central role in constructing political images, but political communication scholars have largely ignored this topic. As stated by Schill (2008: 4): “Communication scholars rarely examine visual symbols in their research, instead focusing on other elements, such as rhetorical strategies, issue framing, and tactical use of linguistic symbols”. It is our aim to go beyond these shortfalls. First, we seek to identify why visual images are described as a powerful tool and second we will introduce the role of wall paintings as a recruiting tool to strengthen the nationalist spirit by common frames of references, relying on Vatikiotis’ (2005: 4) notion that communication processes are not only addressed in

680

Maximilian Rapp and Markus Rhomberg

“representative terms (‘for the people’), but in participatory terms as well (‘from the people’)”. Peteet (1996: 139–140) states in an article on the role of graffiti during the Intifadas in Palestine that these are “a form of cultural production during a sustained political contest” and “a cultural device in a communicative process”. Although we are well aware that the techniques of graffiti are different to murals, we can abstract these remarks for all kinds of wall paintings, graffiti as well as murals. All these iconic forms are symbols or representations and as Farrell (2009: 1) states, with reference to Berger and Luckmann (1966), they are “powerful communicative devices because they are easily recognizable and instantly meaningful to a large group of people”. A second aspect of his notions refers directly to the case of Northern Ireland. Icons are “particularly important when a leader or a group requires a rallying call around a political cause that aims to radicalize a people, initiate revolution, or instantiate drastic social change” (Farrell 2009: 1). By fulfilling these functions, icons (depicted in murals) allow a community to identify with a goal or an issue, and they allow a particular territorial area to be identified with a community. In general, certain effects are attributed to visual images in contrast to textual forms. Schweiger and Adami (1997) state that they deploy a higher potential than texts to activate and remember. Other scholars refer to the notion that images quickly communicate the full spectrum of emotions, have rhetorical impacts and make persuasive arguments to viewers. Furthermore, they document and authenticate that an event occurred or that something happened and take their audiences to different times and places by showing remarkable scenes from the past (Messaris 1997). This can especially be useful in political contexts to “tap into iconic, societal symbols and draw on the emotional power associated with those symbols” (Schill 2008: 23). To sum it up, using visual symbols helps to build associations, causal connections, contrasts, analogies and generalizations by their viewers. Before entering Northern Ireland’s case, it is useful to discuss the question as to how these mentioned single phenomena affect communities and therefore build a copious medium for a recruiting strategy for the republican movement. In the democratic-participant theory, media are seen in small-scale terms, “favouring horizontal patterns of interaction and facilitating the expression of citizens’ need” (Vatikiotis 2005: 5). McQuail (1994) argues that one of these participative mediachannels are wall posters (others are the underground or alternative press, community cable television, micro-media in rural settings, neighbourhood media and media for ethnic minorities). These media channels have also developed due to the failure of the mass media to meet: the needs that arise from the daily experience of citizens, to offer space to individual and minority expressions. […] The geographical locality and/or a community of interest constitute an essential context of communication since participants, both senders and receivers, share the same concerns on community issues; in this context, a ‘community’ realizes the reproduction and representation of its shared interests (Vatikiotis 2005: 6–7).

The importance of Murals during the Troubles

681

2.3 Social movements and nationalism in Northern Ireland Recent studies in political science discuss the function and political influence of social movements being in relation to their capacity for agenda setting (Goldstone 2003). Given the scope of the research question addressed here, it is important to highlight the possibility of murals as a tool for visualizing goals of a specific movements and communicating certain messages to change the political status quo. While social movements in the late 19th century were often a prototype for social gatherings leading to the formation of communist and socialist groups, the period after World War II showed that the call for civil rights dominated the development sustainable (McAdam 1982). According to Aberle (1966), who categorized the types of social movements, the republican movement can be seen as redemptive social movement, being radical and initiated by a specific group within the divided society in Northern Ireland. Referring to a more sociological approach towards categorizing social movements (Snow et al. 1969), it becomes clear that because of the desire to change the value system even in a violent way, the republican movement has to be seen as a new, radical and violent movement, which was group-focused and highly organized by the IRA and its political wing Sinn Féin. Moreover, the resource mobilization theory from Jenkins (1983), which emphasizes the importance of resources in social movement development, shows that the republican armed struggle for more civil rights can be seen besides the political and cultural approach from an important sociological point of view (Kendall 2005). According to Jenkins (1983) social movement needs financial, knowledge, media and labor support as well as solidarity, and they develop when individuals with grievances are able to mobilize the masses. In Northern Ireland, financial support for the republican movement came overwhelmingly from Irish-America and institutions like Noraid (Northern Ireland Aid), while armament supply was inter alia given by Libyan dictator Colonel Gaddafi (Moloney 2002: 25–33). In contrast, media support can be identified as a crucial issue after different “Broadcasting Acts” were initiated to ban the circulation of republican content. Hence murals were used increasingly to demonstrate republican ideals and visualize their struggle against the British occupation and social injustice, not least because the wall-paintings were difficult for British security forces and the RUC to censor and control. Furthermore, murals were an important mirror for expressing identity and culture as well as a medium to strengthen the republican ethos by reflecting images of nationalist ideals. Ethno-symbolists highlight that nationalism “derives its force from ‘inner’ sources like history and culture” (Isiksal 2002: 2), which can be visualized by a variety of symbols and iconographies (Githens-Mazer 2007: 3). The images in republican strongholds on historical events, nationalists’ iconographies like the Irish Tricolor, the Easter lily, commemorating the rising from 1916, etc. can consequently be described as a visual expression of republican social movement. Hence, Githens-Mazer highlights in his essay on ethno-nationalism that “symbols, ranging from flags to anthems, from paintings to passports,

682

Maximilian Rapp and Markus Rhomberg

from saints to martyrs have the ability to set agendas, add legitimacy to elites and institutions, and inspire powerful popular protests against colonial powers” (Githens-Mazer 2007: 3).

3 Methodological approach and scope of the analysis Our goal is to contextualize the political and social capacity of murals in a conflictual context and to ask what the significance of murals in this conflict is. Hence, we combine the visual depiction of the murals with qualitative interviews to illustrate our arguments. By using a historiographical revue, we can give an overview on the impact of murals on the republican movement during the decades of the armed struggle between 1968 and 1998. This helps to understand the meaning of murals in a better way as well as to analyze the synthesis of wall paintings and social movements in the context of the “Troubles”. Qualitative research methods are flexible and adapt to the object of investigation and hence they are an optimal basis to deepen the research. Moreover, the combination of static historical analysis and the open procedure of qualitative research methods contribute to explore circumstances in the context of scientific questions (Flick 1995). In the scope of our research we used qualitative interviews and corresponded with leading figures on Northern Ireland’s politics, economy, science, institutions as well as with muralists, former terrorists and several citizens. While the decision makers were chosen specifically to cover the political party system in Northern Ireland and to deepen the research-work of scientists studying the “Troubles” and murals, we tried to reach a balanced proportion (Catholics-Protestants) by also asking local citizens (Catholics living in Catholic areas and Protestants living in Protestant areas). In addition, it is very important to gather qualitative insights about the wall paintings and how the artists themselves analyze their impact on the Catholic society. Consequently, as in this research the experiences and perceptions of the interviewed persons are of primary concern, the qualitative narrative interview research is applied. The interviews were held from March 2009 until September 2011. The interviews last from 20 minutes (many politicians with lack of time) to an hour (especially muralists and citizens were open to a long time interview) and were partly recorded. Many interviewees (former terrorists and politicians as well as muralists), however, declined to have the interview recorded on the grounds it made them feel uncomfortable. In addition, we found it useful to get insights from the people who experienced best the impact of murals during the “Troubles”, the citizens of the working class areas, especially in republican strongholds like Falls Road, Ballymurphy, Andersonstown and Short Strand as well as in loyalist areas like Shankill Road, Sandy Row or Newtonards Road. The use of qualitative inter-

The importance of Murals during the Troubles

683

views enables to “reach areas of reality that would otherwise remain inaccessible such as people’s subjective experiences and attitudes” (Peräkylä 2005: 869).

4 Murals as a propagandistic weapon in the republican movement 4.1 Starting a republican tradition The birthdate of political images on walls in Northern Ireland can be referred to the year 1908 as the Protestant John McClean painted a mural at Beersbridge Road in Belfast to remember the historical victory of King William of Orange (King Billy) over the last Catholic King of England, James II. This event is yearly imprinted in Loyalists mind on July the Twelfth, when parades, bonfires and celebrations are taking place, but also lead partly through Catholic areas, which is why the day is always overshadowed by a violent hassle between the two parts of the society (Abshire 2003). Loyalists then used this method for commemorating different historical events for the Protestant community. The Battle of the Somme, where the Irish 36th division (constituted predominantly of UVF militants) fought in World War I, as well as the destiny of the Titanic, which was built by Belfast shipyard Harland & Wolff, were appealing motifs for loyalist muralists. This tradition and ritual was anchored through permanent repaintings and new impressions emerging on gable ends of houses in loyalist areas over the next decades, even though the Unionist spirit was split in the 1960s and 1970s, because of the rising displeasure of London’s handling with the Northern Ireland Affair (Rolston 1998). After the outbreak of the “Troubles” the images became more and more militaristic, showing mostly hooded gunmen with machine guns and rifles. While many researchers argue that this reflects the “no surrender” mentality of loyalists (Rolston 1998), many more emphasizes the assertion that the Protestants are still an unsettled people with a lack of identity (McKay 2000; Vannais 2001). Nevertheless it becomes clear that the tradition of mural paintings in Northern Ireland was at first exclusively a loyalist phenomenon, not least because republicans were simply not allowed to express their opinion publicly, as republican muralist and former IRA member Danny Devenney points out: “We were living in a state dominated by loyalists. There was a riot in 1964 after republicans put up a tricolor in Divis Street. It was insane. What do you think the reaction would have been to a mural painted by republicans?” (Danny Devenney, personal communication, August 16th 2010). The Catholic minority in Northern Ireland considered itself to be exposed to intense disequilibrium, which cumulated in segregation, minor civil rights, gerrymandering and socio-political injustice (Bew and Gillespie 1999). Therefore Catholics decided to protest against the status quo, which is why the Northern Ireland Civil Rights Association (NICRA) was founded in the mid-1960s. This organization

684

Maximilian Rapp and Markus Rhomberg

Fig. 1: Anti-RUC mural at Whiterock Road, Ballymurphy, Belfast (photo: Author 2008).

focused on their goal to strengthen Catholics rights and therefore initiated the Derry march in 1968, where republican demonstrators were beaten up by the RUC and the “Troubles” officially began. One of the demonstrators, Bernadette Devlin McAliskey, described the reaction of the police forces as following: “A few policemen were at least trying to stop us from being killed, but the others were quite delighted that we were getting what, in their terms, we deserved” (Devlin 1969). The situation escalated in the following years after the Battle of the Bogside in (London-)Derry 1969, when Catholics barricaded themselves in the working class area, created extempore “no-go areas” for loyalists and police forces as well as carried out street battles, in which the RUC used CS Gas against the demonstrators (Gillespie 2008). A second catalyst for the escalation has been the decision to send out British troops to Northern Ireland and the incident of Bloody Sunday, where 14 innocent people got killed by firearms in the scope of a civil march in 1972. But it took until 1981, when ten IRA and INLA imprisoners from Long Kesh jail starved to death, that republicans started to use the paintbrush as a weapon against the colonialization of Ireland (Multhaupt 1988) and the military suppression carried out by the police service, namely the RUC (see Figure 1). Hence anti-police murals were painted to demonstrate visually against the status quo. The following was painted in Ballymurphy, Belfast and emphasizes the Catholic animosity against

The importance of Murals during the Troubles

685

the RUC and their use of plastic-bullets to break up demonstrations. The red color and circle, where the police-officer in the central position is crossed out shows the opinion of the majority of Catholics and nationalists, who started to revolt publicly against the suppression of their kind. The use of more than 42,000 plastic bullets between 1973 and 1981, which are originally designed as non-lethal weapon, lead to fourteen people killed, including nine children and to the strong aversion of republicans towards police shown by the muralists in this painting (Curtis 1982). The reason was the existing injustice against Catholic imprisoners and the circumstances of the trials. In order to support the hunger strikers, who demonstrated against the status quo, the people in the republican areas of Belfast started to paint slogans on the walls. What at first seemed to be more graffiti than art, changed immediately after the death of Bobby Sands, the leading figure of the hunger strike, who was elected MP during his starvation. Hundreds of murals came into being during the nights of grieve and lead to a new tradition in republican culture. International media focused on Northern Ireland and the IRA, who declared its Long War against their enemies (O’Brien 1995) and took over to use the images on the walls to express republican culture and to illustrate that they have been the innocent part during the conflict. Jarman (Jarman 1998) recounts that in this period many television reporters from international broadcasting stations were filmed in front of republican murals to present the situation as vividly as possible. This gave republicans the chance to get their message out of boundaries like Falls Road or Ballymurphy. Even Protestant novelist Col Buchanan emphasizes the strategically orientation of republicans by focusing on identity and therefore hid the militaristic propaganda in cultural expressions: In relation to the ‘Nationalist’ murals that I saw within Nationalist areas of Belfast […] while many of them were militaristic just like their Protestant counterparts, others expressed- with some quality of craft- Irish National culture, and its sentiments of freedom and equality. The Bobby Sands and other hunger-striker murals in particular I’m thinking of. These appealed to me much more than those of my Protestant brethren. I found that they also, often, expressed sentiments of hope, whereas Loyalist murals seemed based more on fear and hatred and violence (C. Buchanan, personal communication, December 12th 2010).

Ethno-symbols like the republican iconographic of the Phoenix, which symbolized the reincarnation of the nationalist movement in the Provisional IRA (PIRA) (after it split from a more Marxist-socialist wing, the Official IRA), or the sunburst, the iconographic symbol of the Fenians, a Celtic warrior clan from the myths. These symbols used by the muralists to strengthen the republican ethos, but were combined with topics like the commemoration of the fallen hunger strikers or aggressive images like IRA gunmen or masked freedom fighters (see Figure 2). Figure 2 shows a mural, which was drawn right after the death of the tenth hunger striker at Rockville Street in the republican “murder line”, Falls Road, on a black background to associate death. The muralists wanted to commemorate the martyrs by illustrating their forenames on a ribbon right under the central

686

Maximilian Rapp and Markus Rhomberg

Fig. 2: Republican Phoenix and sunburst at Rockville Street, Falls Road, Belfast (Rolston 2010).

iconographies, the yellow sunburst and the phoenix with a green, white and orange wing, symbolizing the Irish tricolor. “The Final Salute” is still one of the IRA’s trademarks, honoring the fallen with a last volley from a rifle. This military impact is also supported in the mural by the parallelism of the freedom fighters and their gun barrels. Furthermore the crossed flags of the Irish tricolor and the Starry Plough Flag of the Irish National Liberation Army (INLA), a republican terrorist group, emphasize the desire of the muralists to share their message of nationalism and social injustice against the Catholic population.

4.2 Using murals as political and Celtic propaganda Moreover, the core of ethnicity “diffuses the myths, memories and symbols of the group contemporarily through the ethnic group and across generations, preserving and maintaining the form of the group, and the content of its identity over the longue-durée- i.e. those myths, symbols, values and memories that make the ethnie distinct and separate” (Githens-Mazer 2007). The potential of resistance through a widespread myth-symbol complex was a strategy developed by the IRA to increase the support of the people, knowing as Margaret Thatcher once pointed out that “publicity is the oxygen of terrorism”. Therefore a variety of symbols, which are

The importance of Murals during the Troubles

687

anchored in nationalist history or in Celtic roots, like the lily flower symbolizing the Easter Rising of 1916 or the Celtic Cross commemorating the Catholic victims of the conflict, were realized on the walls to remind people of their common heritage. Especially the commemoration murals were designed with an additional message to influence the subconsciousness and make clear that the Protestants are guilty of various war crimes. The sense of powerlessness against the socio-political injustice, the military suppression of Catholics and the visualization of those images on the walls lead to an admission of new recruits for the IRA, the loss of forbearance for loyalist and Protestant citizens as well as to an intense control of the Catholic society by the Irish Republican Army (O’Duffy 2007). As the Deputy leader of the Alliance Party and MP, Naomi Long puts it: “Many murals during the Troubles became very paramilitary in style and as such were both offensive and intimidating to those from outside the areas, […] where they reinforced a sense of control of those organisations” (N. Long, personal communication, February 8th 2011). This view is also shared by the famous Irish artist, Brian McCarthy, who states: I don’t believe that everybody in Belfast on the streets likes once the murals particularly. You know what happens is you have the tough guys, the IRA or whatever and people are afraid. They decide that they paint them into your house, because you live at the end of the street and there is nothing you can do about it. […] It’s control, it’s territory, it’s Public Relations, that’s all that is. They talk about art, something new and impatiently interesting, but they are not. It’s propaganda (B. McCarthy, personal communication, September 15th 2011).

The IRA more and more used the potential of visual displays after they nearly collapsed at the end of the 1960s when they couldn’t protect their communities from being violently suppressed by the RUC, because they dumped their weapons soon before (Mallie and Bishop 1988). People in charge, for example Jim Gibney, who came up with the idea of listing Bobby Sands for the election, saw how rich and effective the visual language on the walls can be: “There was a time when Sinn Féin was not heard and Republican voices were suppressed. The murals were a medium for the nationalists to get heard. It is a symbol for resistance by the Catholics to express their feelings and their Irish culture” (J. Gibney, personal communication, August 15th, 2010). One important impact of the hunger strike was that Sinn Féin was officially on the political board (McKittrick and McVea 2000), even though it was obvious that they acted as a political wing of the IRA. Soon the murals were used as a medium for political propaganda (see Figure 3), knowing that they have a much more impressive impact than posters or flyers, even though, Jonathan McCormick, the founder of the CAIN-website (www.cain.ulst.ak.uk), a search engine for murals in Northern Ireland, highlights: They have the potential for greater longevity than election posters, but not always. They are more timely and costly than election posters certainly. They can be taken more seriously or have a greater impact but a billboard campaign can reach a wider audience and can have a mainstream location (J. McCormick, personal communication, June 27th, 2010).

688

Maximilian Rapp and Markus Rhomberg

Fig. 3: Election mural from Sinn Féin, Beechmount Avenue, Belfast (Rolston 2010).

The figure of political propaganda is one of the key differences between republican and loyalist images as Neil Jarman, head of the Conflict Research Institute in Belfast shows: Murals for elections were just common in republican areas when Sinn Féin was gearing up. They (republicans) were also using more posters, because an election campaign is for short terms, rather than murals, which are for long terms. The Protestant murals are however mostly apolitical (N. Jarman, personal communication, August 15th 2010).

The election-mural from Sinn Féin shows a man in front of an Irish tricolor, symbolizing nationalist ideals in the scope of an armed struggle. Next to him there is a group of people demonstrating for more social rights with signs showing “Houses” or “Jobs” in big characters, referring to the massive social injustice Catholics faced in Northern Ireland like housing and work segregation as well as political marginalization. Hence, while the IRA itself used the paintings to mobilize new recruits for their armed struggle, which in total led to more than 2,000 deaths, the political wing of the IRA tried to use the murals and their potential coverage to mobilize voters for their listed president Gerry Adams and a “New Ireland”, as it is said on the wall. Smith points out that myths are an essential assumption for a common identity and a strengthened nationalist spirit: “Myths of origin and

The importance of Murals during the Troubles

689

Fig. 4: Mural of King Núada Airgetlám at Ballymurphy, Belfast, repainted from origin Gerald Kelly (photo: Author 2009).

descent constitute the primary definers of the separate existence and character of particular ethnies” (Smith 1999: 15). The republicans used to integrate the stories and tales of old to their everyday life since the founding of the Fenians and later of the Irish Republican Brotherhood in the 19th century, the official processors of the IRA. The ancestry of Catholics in Northern Ireland was always referred to the myths and heroes, who defended the country against invaders and traitors and identified with the Celtic rituals and roots (Cotterell 2002). A key role in nationalist myths played Cú Chulainn, the defending hound of Ulster. The memory of heroes fighting for the land is an important part of a common heritage and distinct groups of people from an identified rest in a divided society. During the late 1980s the former IRA prisoner Gerald Kelly influenced mural paintings sustainable after realizing images with artist depth, structure and detailed proportionality (Rolston 2010). One of his first murals showed King Núada Airgetlám (see figure 4), who was according to Celtic mythology one of the most powerful warrior kings with an artificial hand and a magical sword. The painting was in fact so detailed and colorful that the majesty of the king was transferred as a message to nearly every person crossing the mural. It showed self-confidence and power of Celtic tradition and therefore became a shrine of national identity. The IRA used such displays to fight against the occupation of British forces and

690

Maximilian Rapp and Markus Rhomberg

loyalist paramilitaries. This is why a new version of the old image was repainted in Ballymurphy, Belfast, several years ago, showing the King in the center with Celtic knots surrounding him, symbolizing Celtic tradition and nationalist force and pertinacity. However to emphasize the identical expression and nationalist raison d’être, more and more illustrations about Celtic music, dance, language and sports came into being. Regarding Andersons’ theory of language as the key factor of nationalism (Anderson 1998), it is important to single out the fact that nearly every mural in republican areas have a Gaelic headline (often combined Gaelic-English) and hence emphasize the differentiation of nationalists in Northern Ireland’s society. Former IRA member Rab Kerr reflects the positive sides of republican murals by marking out that “they (the murals) show what we are proud of; should that be our culture in the form of our music, language, sport, mythology etc. or aspects of our community, history, our struggle or our patriots. Many murals are based around the concept of justice and this cuts across so many different murals, murals of international solidarity as well as local campaigns and issues” (R. Kerr, personal communication, September 24th, 2010). Furthermore, the Deputy First Minister of Northern Ireland, Martin McGuinness, states that: murals have been a part of community expression in many areas of the North evolving through the years, reflecting contemporary events during the conflict to the present time where muralists have widened the scope of the message to include cultural, sporting, and international events and individuals (M. McGuinness, personal communication, January 22th 2011).

International murals have always been a key difference between nationalist and loyalist images on the walls. Through the above stated media coverage of Northern Ireland following the Hunger Strikes, many cultural groups around Europe identified with the Catholic struggle. However, this circumstance was an important propagandistic benefit for the IRA, knowing that many other cultures supported their movement, and having more arguments to convince Catholics to join their military campaign. Moreover Smith marks out that historical events or a common ethno-history constitutes grounding for nationalism: “By ethno-history is meant the ethnic members’ memories and understanding of their communal past or pasts, rather than any more objective and dispassionate analysis by professional historians” (Smith 1999). Researchers have shown that the more disastrous an historical event for an identity, the more enduring its effects. Githens-Mazer emphasizes the importance of memories by stating: “In this way, myths, memories and symbols are not static, but rather dynamic and constantly being re-cast or re-invoked to maintain contemporary pertinence and meaning and so as to keep their popular resonance” (Githens-Mazer 2007). Therefore incisive events, like the Easter Rising 1916, when Irish nationalists (Irish Republican Brotherhood and Irish Volunteers) tried to end British rule by announcing an Irish Republic played an important part in republicans’ development. The rebellion, which was led by teacher Pádraig Pearse, failed totally.

The importance of Murals during the Troubles

691

Fig. 5: Mural about “The Great Famine” (Antocras Mór), New Lodge Road, Belfast, (photo: Author 2010).

But in the following the British executed the leading figures of the revolt and released unintended the nationalist spirit. Even though republican organizations fought for more rights before, this event is known as the birthdate of modern Irish nationalism and is described by one of the most famous poet, William Butler Yeats, in one of his poems (Easter 1916) as “terrible beauty”. Since then Eastern-time was always combined with nationalist events and quite similar to the loyalists July Twelfth, republican murals mostly were created in this time of the year as a ritual. Moreover nationalists around the island are wearing Easter lily pins in those holidays and are showing publicly 1) their identity and heritage, but also 2) that nationalists always had to fight for their rights and were suppressed by the British. This type of symbolic propaganda can also be seen in a variety of other historical events like the “Great Famine” between 1845 and 1852. Muralists, who were mainly instructed by the IRA, painted those murals not only to remember the 150th centenary of this crisis, but also to emphasize Irish innocence in the Northern Ireland conflict and to show the desire to have a northern province without British attendance. Such illustrations, like the following painted at New Lodge Road in Belfast, still polarize the society and push nationalist ideas through Catholic working-class areas. Figure 5 shows four emaciated farmers desperately seeking for potatoes on their farm. The background mirrors the green

692

Maximilian Rapp and Markus Rhomberg

Fig. 6: The “Petrol Bomber”, Rossville Street, (London-)Derry, (photo: Author 2010).

Irish coast and blue sea, demonstrating the beauty of the land, while more and more derelict stone-cottages symbolize the poverty and misery of Catholics during the Great Famine. The muralists’ ability to paint true-to-detail has to be accentuated as well as the fact that they emphasize the Irish-nationalist association by naming the mural bilingual, “An tocras mór”, Gaelic for “The Great Hunger”. Expressing Celtic culture, combined with subliminal nationalist slogans has also been a key difference between the two societies. While the IRA understood to hide its propaganda under various identical and cultural messages, loyalist murals mirrored their military strategy straight forward by visualizing hooded gunman threatening everybody who passes its way. The nationalist renunciation of clear militaristic images was also favored by the work of the Bogside Artists, a group of three muralists under the lead of Tom Kelly. They started to paint their first mural in 1994, showing a young kid with a gas mask fighting against the police forces and their CS gas during the Battle of the Bogside. In his hand he is holding a Molotov cocktail, which is symbolic for the fight against Catholic suppression in Northern Ireland. While the Bogside Artists in their later works wanted to focus on historical images (a reason why they are all painted in black and white) and avoid propaganda in their paintings, their first mural (the “Petrol Bomber”, see Figure 6) is seen by republicans as an expression of nationalist values and as a

The importance of Murals during the Troubles

693

Fig. 7: Anti-British mural, Whiterock Road, Ballymurphy, Belfast, (photo: Rolston 2004).

Statue of David referring to their armed struggle against suppression and colonialization (Guildhall Press 2008). Even Tom Kelly highlights that the mural: conveys most accurately the reality of the struggle in which children and adults got involved. The child is both a combatant and a victim. That contradiction makes him special. He is also prepared to stand up for his rights. Last but not least people in Derry at that time were innocent as children when it came to street warfare. The idea that soldiers would fire live rounds occurred to nobody (T. Kelly, personal communication, July 23th 2010).

In the run-up to the Good Friday Agreement 1998 nationalists tried again to express their innocence through a variety of new motifs, but coincidental make clear that as long as the British security forces and the RUC continue their existence on the streets, there will be never peace in Northern Ireland. Hence images against the use of plastic-bullets from the RUC as well as murals about collusion, the composition of security forces and loyalist paramilitary organizations, were painted and should address all Catholics not to be content with the milestone on Good Friday, but to demand the disorganization of the RUC and the drawback of British troops from Northern Ireland. On the basis of a poster from the Irish artist Robert Ballagh, republican muralists started to paint a series of wall paintings, to broach the issue of the drawback of British soldiers. While one of them shows how British troops are going on a street back home to England under the headline

694

Maximilian Rapp and Markus Rhomberg

“Slan Abhaile” (Safe Home), another illustrates a dove of peace carrying home a soldier while saying “Time For Peace”, the soldier is answering “Time To Go” (see Figure 7). These murals emphasize nationalists’ conviction that the Troubles was a conflict rooted in British colonialism.

5 Conclusion In the scope of this research we analyzed the impact of murals on nationalists’ movement during the “Troubles” in Northern Ireland. Our first key finding is that murals were used as a medium by the republican movement to project a message within and beyond the boundaries of nationalist communities. The armed struggle against political disequilibrium and social injustice developed a visual and iconographic component, which lead to a new style of ritual in the republican movement. Painting walls gave nationalists the chance of being heard by a broad audience and getting in contact with other identities, who share their struggle for more civil rights. Furthermore, republican organizations like the Provisional IRA identified the various possibilities to get their anti-British messages across and used the murals as a medium to convince people in Catholic areas to join the armed fight. According to the historiographical analyses and various interviews with leading decision makers in Northern Ireland, this research shows that the republican movement in Northern Ireland was supported by the visual images on the walls and gable ends of houses. Moreover, this paper shows that nationalists changed their strategy of propaganda through visual images during the “Troubles”. Mirroring first masked gunmen and IRA symbols like loyalists still do, the offensive propaganda strategy changed continuously to messages, which were absorbed by the recipients more subconsciously. Murals about Celtic tradition, Irish heritage or Gaelic history were painted in broad variety but should transport nationalist ideals and future goals beyond the impressive artworks. However this changed in the forefront of the Good Friday Agreement, which was signed by all key players in Northern Ireland’s politics, excluding the Democratic Unionist Party (DUP) under the lead of Rev. Ian Paisley. Anti-RUC images as well as collusion murals and “Brits Out” slogans made clear that the republican movement continues to reach their goals like a troop free Ulster (British forces) or the decrease of socio-political and economic segregation, even though the IRA ended their armed campaign later on in 2005, when they officially dumped their arsenal of weapons. Nevertheless other nationalist splinter groups like the Real IRA, the Continuity IRA or Óglaigh na hÉireann still try to bomb the country back into its violent past. But the mainstream of republican movement flows today in a much more peaceful way, hence murals have changed dramatically in style and now project a new image for the post-Agreement republican ethos.

The importance of Murals during the Troubles

695

References Aberle, David. 1966. The Peyote Religion among the Navaho. Chicago: Aldine. Abshire, Jean. [2003]. Northern Ireland’s Politics in Paint. Peace Review 15: 149–161. Anderson, Benedict. 1998. The Spectre of Comparisons: Nationalism, Southeast Asia, and the World. London/New York: Verso books. Berger, Peter and Thomas Luckmann. 1996. The Social Construction of Reality. Garden City: Anchor Books. Bew, Paul and Gordon Gillespie. 1999. Northern Ireland: A Chronology of the Troubles 1968– 1999. Dublin: Gill & Macmillan. Bryan, Dominic, Stevenson, Clifford, Gordon Gillespie and John Bell. 2009. Public Displays of Flags and Emblems in Northern Ireland: Survey 2006–2009. Belfast: University Press. Cotterell, Arthur. 2002. Die Enzyklopädie der Mythologie. Reichelsheim: Edition XXL. Curtis, William. 1982. Modern Architecture Since 1900. Oxford: Phaidon Press. Devlin, Bernadette. 1969. The Price Of My Soul. London: Random House. Diana, Kendall. 2005. Sociology In Our Times: the essentials. Thomson/Wadsworth. Farrell, Jewell. 2009. Is this Soil Irish or British? How Iconic Symbols in Mural Paintings Shape Reality in Northern Ireland. Paper presented at the annual meeting of the NCA 95th Annual Convention, Chicago Hilton & Towers, Chicago. Available at: http://www.allacademic.com/ meta/p368062_index.html. Flick, Uwe. 1995. Qualitative Forschung. Theorie, Methoden, Anwendung in Psychologie und Sozialwissenschaften. Reinbeck: Rowohlt. Gillespie, Gordon. 2008. Years of Darkness: The Troubles remembered. Dublin: Gill & Macmillan Ltd. Githens-Mazer, Jonathan. 2007. Ethno Symbolism and the Everyday Ressonance of Myths, Memories and Symbols of the Nation. Paper for the International Conference of Everyday Life in World Politics and Economics, LSE, London, May 11. Goldstone, Jack. 2003. States, Parties, and Social Movements. Cambridge: Cambridge University Press. Gurr, Ted. 1970. Why Men Rebel. Princeton: Princeton University Press. Isiksal, Husevin. 2002. Two Perspective on the Relationship of ethnicity to nationalism: comparing Gellner and Smith. Alternatives: Turkish Journal of International Relations 1(1). Jarman, Neil. 1998. Painting landscape – the place of murals in the symbolic construction of urban space. In: Anthony Buckley (ed.), Symbols in Northern Ireland. 81–98. Belfast: The Institute of Irish Studies. Jenkins, Jennifer. 1983. Resource Mobilization Theory and the Study of Social Movements. Annual Review of Sociology 9: 527–553. Mallie, Eamon and Patrick Bishop. 1988. The Provisional IRA. London: Corgi. McAdam, Doug. 1982. Political Process and the Development of Black Insurgency, 1930–1970. Chicago: University of Chicago Press. McCormick, Jonathan and Neil Jarman. 2005. Death of a Mural. Journal of Material Culture 10(1): 49–71. McGarry, John and Brendan O’Leary. 1997. Explaining Northern Ireland. In: Katleen Cavanaugh (ed.), Interpretations of political violence in ethnically divided societies. London: Routledge. McKay, Susan. 2000. Northern Protestants: An Unsettled People. Belfast: Blackstaff Press. McKittrick, David and David McVea. 2000. Making Sense of the Troubles. Belfast: Blackstaff Press. McQuail, Dennis. 1994. Mass Communication Theory: An Introduction. London: Sage. Messaris, Paul. 1997. Visual persuasion: The role of images in advertising. Thousand Oaks: Sage. Moloney, Ed. 2002. A Secret History of The IRA. London: Penguin Books.

696

Maximilian Rapp and Markus Rhomberg

Multhaupt, W. 1988. Die Irisch Republikanische Armee. Aus Politik und Zeitgeschichte 45: 88. O’Brien, Brendan. 1995. The Long War: The IRA & Sinn Féin from Armed Struggle to Peace Talks. Dublin: The O’Brien Press. O’Duffy, Brendan. 2007. British- Irish Relations and Northern Ireland: From Violent Politics to Conflict Regulation. Dublin: Irish Academic Press. Peräkylä, Anssi. 2005. Analyzing Talk and Text. In: Norman Denzin and Yvonna Lincoln (eds.), The Sage Handbook of Qualitative Research, 869–886. Thousand Oaks/London/New Delhi: Sage. Peteet, Julie. 1996. The Writing in the Walls: The Graffiti of the Intifada. Cultural Anthropology 11(2): 139–159. Rolston, Bill. 1998. Drawing Support 2: Murals of War and Peace. Belfast: Beyond the Pale. Rolston, Bill. 2004. The War of the Walls: political murals in Northern Ireland. Museum International 223: 38–45. Rolston, Bill. 2010. Trying to reach the future through the past: Murals and memory in Northern Ireland. Crime, Media, Culture 6: 285–307. Schill, D. K. 2008. The Visual Image and the Political Image: Finding a Place for Visual Communication in the Study of Political Communication. Paper presented at the annual meeting of the International Communication Association, TBA, Montreal/Quebec, Canada Available at: http://www.allacademic.com/meta/p233469_index.html. Schweiger, G. and M. Adami. 1999. The nonverbal image of politicians and political parties. In Bruce Newman (ed.), Handbook of political marketing, 347–364. Thousand Oaks: Sage. Smith, Anthony D. 1991. Myths and Memories of the Nation. Oxford: Oxford University Press. Snow, David, Sarah Soule and Hanspeter Kriesi. 1969. Social Movements. Oxford: Blackwell Press. Vannais, Julie. 2001. Postcards From the Edge: Reading Political Murals in the North of Ireland. Irish Political Studies 16: 133–160. Vatikiotis, Pantelis. 2005. Communication Theory and Alternative Media. Communication and Culture 2(1): 4–29.

Rupert Cox

32 Transforming art and visual anthropology: Imitation, innovation and inspiration in two Japanese art museums Abstract: The idea that art-works are reflective and constitutive of social relations has been a distinguishing focus of anthropological approaches to the art museum, characterised in recent years by the questions raised by the anthropologist Alfred Gell about art and agency. Debates about this approach have drawn on visual studies in asking ‘what do pictures really want? and on the sensory turn in the social sciences, in asking what it means to talk about ‘visual sense’. In this essay, I analyse the strategies and technologies of display used to present copies of Western art-works in a Japanese art museum, so as to show how ways of looking and touching are intertwined and mutually constitutive of forms of aestheticism that are rooted as much in the mimetic capacities of materials as in the expression of social relations. I argue that this mode of representing cultural knowledge through the act of material representation rather than the reading of its visible form may be a paradigm for visual anthropology because it does not confuse the capacity to visualise images of other cultural worlds with their meaning.

1 Just looking In the book The Object Stares Back (1996) the art historian and critic James Elkins critically explores the proposition that “there is no such thing as just looking”. He argues that this proposition is a foundational proposition for art history based on an assumption about the singularity of vision and that when “The Observer Looks at the Object” it is a directly communicative act. This assumption is also central to a certain kind of visual anthropology, which has until relatively recently1 not begun to explore the implications of the suggestion that looking through the camera may involve a kind of corporeality that is different than the operation of the eye or an extension of camera technology2 when conceived in modern terms as what Crary has called “techniques of the observer” (1990). Elkins’ argument explores this notion of the trained and technologically concentrated gaze and proposes that even in a designed viewing mechanism such as the art museum there are many kinds of looking going on. His work shows

 See the article Blurred Visions, Cox and Wright (2012) for more on this.  This is an argument developed by the visual anthropologist David MacDougall in his book Corporeal Vision and discussed in Cox and Wright (2012).

698

Rupert Cox

how looking may be fragmented, partial and inconclusive and suggests that it is inseparable from the way that the affective energies of the museum act upon the bodies of observers. In respect of the development of ethnographic museums and displays these points are supported by Alison Griffith’s works (2002, 2008) which have shown how the dramatic, cinematic nature of exhibition design and technology was developed with an appreciation of a modern observer ready to be stimulated by the excitement and wonder of a vicarious contact with other cultures. These arguments about the sensory engagements of visual technology dovetail with recent work in the anthropology of the senses (Classen 1993, 1997; Howes 1991, 2003, 2005; Stoller 1997; Taussig 1993) revising the primacy accorded to vision in the museum and in anthropology, and demonstrating that other senses are engaged synesthetically alongside looking. In the context of art-works, this is the position explored by Janet Brodsky (2002), who proposes that a multi-sensory “whole body” experience is engaged in both the making and the viewing of artworks and that this is a perceptual relationship with a work of art not readily available to the museum observer because it has been ground out by the particular sensory environment of the museum and the ocular-centric epistemologies of art history. In this chapter I want to examine how these institutional and professional modes of observation implicate the senses in distinct relationships to each other and may include a kind of “haptic visuality” (Marks 2000) or “tactile vision” (Taussig 1993). This is a way of seeing that is circumscribed by the sensory environment of the institution and also by the “tactility” of the artwork itself. I am referring to the senses in two ways here: in relation to the visual technologies of the museum and in relation to the pictorial image. In the first sense I mean that the control of the environment in an art museum setting may be understood as a necessary condition for a work so that its mimetic value as a formal equivalence in composition and material structure to the object it represents can be appreciated. In the second sense, I mean to recognise the subjectivity of the sensory observer who may experience the work not as an original but as a reproduction that borrows from the original not only by virtue of the fidelity of its likeness but through the manner of its reproduction and by its presentation. Then visual similitude between an original and a copy may be perceived and made meaningful through an arrangement of vision and sense that is part of the way an art-work has been re-made to be seen. Understanding how this works requires an anthropological sensitivity to the whole environment of art display and to what happens behind the scenes which is not available to the trained eye of the connoisseur but part of the way value is subjectively created. The recognition of the subjective observer may then help us to appreciate the sensory elements in a work that are not simply dependent on a mimetic understanding of its representational value. It is an approach that moves us beyond modes of interpreting images exclusively as representations to be seen and read like linguistic signs and addresses

Transforming art and visual anthropology

699

the crucial question of depiction: that is do pictures cause us to see things they represent because they are likenesses of them or do we call them likenesses because they cause us to see them? It therefore draws attention to the question WJT Mitchell has posed “What do pictures really want” (1996)? This is a question which has obvious implications for the weight that is given in visual anthropology to the determinative relationship of meaning making, between the filmic or photographic image and the object of representation, once we acknowledge that there is a powerful, embodied relation between the process of the creation of the image and those who view it. The acknowledgement of the artifice of the anthropological image, like the recognition of a crisis in written representations of the other,3 has led to the development of a search for authenticity in the originality of the view through the camera as it is used by native others. It is a development that presents a significant challenge to the idea of the anthropologist and particularly the visual anthropologist as author and of the capacity of an arrangement of images, such as in an archive, to show patterns of culture in comparison, contrast and change. In this chapter I want to develop this argument about visual anthropology through an investigation of a Japanese art museum devoted to the display of copies of Western art and to think through how an understanding of what gives the objects in this museum value for its creators is a faith in the power of the copy that is similar to the attitudes adopted in the image making endeavours of visual anthropologists.

2 Copy and contact In Japanese art museums, it is often the case that objects designated items of cultural property and “national treasures” (Kokuhô.) are copies of originals. “Wallhanging scrolls” (e-makimono) for example, are often considered obsolete after 50 years and need to be repaired or duplicated. The use of copies by museums allows for the display of art-works whose originals are considered to be in need of conservation. In the practice of conserving the original, the Japanese museum conservation system admits the partial or total replacement of old materials by new ones. Thus, the reproduced and displayed copy may be valued for itself as a cultural property and the original, but deteriorated object may acquire characteristics that are accepted as a new and important aesthetic value of “perishability”. In the conservation of Japanese art-works, time marks are even emphasised, so that in some cases cracks in objects are not mended but highlighted.4  This crisis was precipitated by the debates following the publication of the book ‘Writing Culture’ edited by Marcus and Fischer (1986).  The Japanese attitudes to copies and copying are described from a variety of critical perspectives in my book The Culture of Copying in Japan: critical and historical perspectives (2006).

700

Rupert Cox

The practical knowledge, which is needed to reproduce or conserve these art objects, is also considered a cultural property in Japan, what is called an “intangible cultural property”. The technical procedures and art skills of an artist/conservator form a sizeable body of “traditional knowledge” that is passed on from master to apprentice over generations by acts of copying and known as denshô. A so called “life treasure” is just such a person, who through years of initially imitating their master has accumulated a specific knowledge of denshô in the field of the traditional arts and crafts in Japan. The official recognition of those craftsmen expert in reproduction of certain materials is regarded as a way of preserving the knowledge of the ancestral masters as a form of cultural property. The tension between acts of creation and of preservation in the aesthetic treatment of objects of cultural particularity is also an underlying factor in visual anthropology where knowledge of others is regarded as being emergent from the visible practices of life, mediated through the visual techniques of the ethnographic camera. The inherent capacity of the camera to copy and to create can be understood in terms of theories of mimesis which Michael Taussig has explored through the work of James Frazer and Walter Benjamin in his work Mimesis and Alterity (1993). In Frazer’s theory of magic and Benjamin’s argument about “art in the age of mechanical reproduction” Taussig finds that the notion of mimesis is transformative, such that every copy contains within it traces of and the potential for “contact”. He conceives of mimesis as two layers, firstly emphasising mimesis as a form of representation, an expression that involves copying. The other layer of mimesis is that of perception and experience which involves palpable, sensuous contact (1993: 21). As an act that does not separate out or privilege vision as discrete from the other senses, mimesis and the copies that result from it engage with the whole sensorium, becoming “tactile vision” (Taussig 1993). Like Taussig, I am interested in investigating “the sticky webs of copy and contact, image and bodily involvement of the perceiver in the image” (Taussig 1993: 21). This network of associations is more elaborate, uncanny, and powerful than the normative use of terms of visual communication like “representation” and “expression” can suggest and we find such a network in evidence at a Japanese art museum where ideas about the power of the copy as a form of contact, cultural knowledge and property have been transferred to the display of simulated western art objects.

3 Technology of vision The Ohtsuka Museum of Art, situated on a hill in a corner of the Seto Naika Inland sea national park overlooking the Naruto straits in Japan’s Tokushima prefecture, houses the world’s first collection of ceramic-board reproductions of Western art masterpieces. It was built in 1998 at a cost of some US $ 400 million to commemo-

Transforming art and visual anthropology

701

rate the 75th anniversary of the founding of the Ohtsuka Chemical group, a company commonly known in Japan for dietary aids such as Calorie Mate and the sports drinks Pocari Sweat. The museum is housed in an eight-story building (five floors below ground and three above) and is the first of its kind to exhibit full-size reproductions of famous artworks baked into ceramic boards at a temperature of 1,300 °C so as to fix the image. Masahito Ohtsuka, Director of the Otsuka Museum, has said that his group started to make tiles out of the sand of the Naruto straits using an innovative method that allowed them to reach the size of one square meter without blemishes or cracks. However, after the world oil crisis in 1973 there was no market for its tiles and it was only after a visit to a Moscow cemetery in 1975 that he found a new purpose for these tiles. He noticed that attached to many of the tombstones were name-card sized photographs of the deceased which had faded in the sun. It occurred to Masahito Otsuka that baking the photographs into ceramics would preserve the original color forever. That was how the Otsuka group began making ceramic reproductions of artworks, concentrating at first on Japanese paintings and scenes – specially commissioned and that was how the idea for the Otsuka Museum of Art emerged. It was a hugely ambitious project, for through four km of corridors (the biggest museum space in Japan) and over one thousand individual objects, it attempts to tell the whole history of western art using photographically accurate facsimiles, made from the ceramic boards for which the company is famous. The process involved taking photographs of the originals, scanning them digitally and baking the image into the ceramic board that could then be touched up at the end of the process using a palette of 30,000 available colors. The lay-out of the museum with objects arranged according to a conservative, formalist chronology of the development of styles and of the discrete spatial separation of each individual artist’s own corpus of works indicates pathways of learning for the visitor. The modernist scopic regime that is at work in this exhibition strategy seems out of place in Japan as it is a regime that came to realization as far as art museums are concerned with the application of Johann Winckelmann’s 19th century aesthetic theories about viewing and appreciating art-works in orderly fashion and from a clear distance. In this manner the value and significance of art-works are communicated by their placement in a progressive sequence that would give the viewer, walking at a measured pace and stopping for long contemplative periods before each artwork, an appreciation of the stylistic development within art forms and ultimately of the Platonic ideals of beauty. Besides this contemplative attitude, the museum is also designed to inspire awe through its scale and size as the visitor enters the museum5 via a long ascent by escalator into the

 This visit was in the year 2000 when I was conducting fieldwork into the history and practices of copying in Japan through an postdoctoral ESRC award.

702

Rupert Cox

middle of the hillside and emerges into a vastly impressive spectacle, a full-scale reproduction of the Sistene Chapel in Rome. When I visited the museum my guide suggested to me that its attraction was due to the size of the reproductions and the sheer number of exhibits but also because of the educational benefit. The museum told the whole history of Western art in one place and could stimulate in the visitor an interest in the original. However, unlike the originals, which were displayed inconveniently or inaccessibly in over two hundred different locations ranging from museums, galleries and churches to private collections and artist’s own studios, here the visitor could get right up close to them and even make personal contact with the pictures by actually touching them. My guide felt that this made the museum a unique experience of the original work of art, because you were not required to show the solemn awe and respect normally due to originals.6 The attitudes behind this form of “tactile knowing” raise a number of questions which complicate the expectation behind the design of this museum display of a distanced, aloof observer, looking with an increasingly educated and detached eye. For the gallery space is in spatial terms designed for undistorted vision underpinned by three themes: the ideal of beauty, the theme of stylistic development through chronological development and the cult of genius. This application of a systematic, comprehensive ordering mechanism for making art available for visible scrutiny and at the same time disrupting the value of observational distance by encouraging touch has its parallels in the ambivalence over the use of still and moving camera technologies in visual anthropology. The anthropologist’s early use of the still camera in attempting to find patterns of cultural difference through the anthropometrical piecing out of the native’s naked body is like the faith placed later on the unblinking stare of the observational film method in that both are dependent on the ability to control the power of the copy to represent. In Taussig’s terms, touch or “contact”, whether felt through the skin or through the imagination of extramission disrupts this detached visualism and suggests that it is more productive to think of the viewing technologies of the museum and of the camera in terms of their capacity to engage sensory vision. The point here is that the ethnographic subject like the art object is not pacified and dulled by its containment within technologies of viewing but able to “look back” and disrupt the authority of the professional and institutional gaze. As Rony (1996) has said in writing about the ethnographic spectacle of photographing native others in the age of international exhibitions and Elkins has argued about the attraction of looking at art in museums, the return gaze, the looking back of the object of  When I spoke with the company director of Ohtsuka pharmaceuticals, at his factory in Shigaraki prefecture, he said that the scale and number of exhibits in the museum were necessary so as to create a comprehensive history of Western art. This made the museum a unique educational experience for Japanese students who would otherwise have to travel widely (136 museums in 25 countries) to see the original works themselves.

Transforming art and visual anthropology

703

vision, complicates our notion of visual communication and gives agency to the object as part of a network of relations between people and things, between looking and touching and between copying and contact.

4 Sensory vision A relationship between viewer and object that relies on the haptic rather than the optic sense in a contemporary museum displaying archetypes of Western art is not just different. In this context it is also an act of re-enculturation. As a means to knowledge, and cultural capital, it may be seen as challenging the very idea of the art museum, with its deliberate notion of seeing at a distance and seeing singularly (as an “observer”) through an uninterrupted line of vision. Touch introduces the body’s surface and reflexivity into knowing, making us, in the context of an art museum display, aware of ourselves as more than just a vehicle for the eye. For while the hand cannot see, it may inscribe a kind of viewpoint. In the Ohtsuka museum the haptic sense of touch is encouraged in a way that suggests it were a kind of optical prosthesis filling in for the visual impairments of a person who cannot see well, or well enough. Indeed, an interesting parallel with this haptic strategy are the policies and procedures for the visually impaired and sightless people in museums. Kevin Heatherington has described such a case in the British museum where a pop up book was used to get around the problem of making the Parthenon frieze accessible to the visually impaired visitor by providing a simplified visual version of the frieze that can also be touched (2002). This kind of innovation is part of a contemporary discourse of access that goes someway to re-dressing the injunction “do not touch” in the museum that Classen has traced as a modern invention (2005). Classen describes how in early museums, touch was expected and even encouraged as a way of knowing and linked to the aesthetic value of objects. It was the public nature of museums, the introduction of glass cabinets and the elevation of sight in scientific investigation in the 19th century7 that put paid to the tactile enjoyments of the early museum visitor. It would be a mistake to think of the visitors’ interactions with art-work copies in the Ohtsuka museum as a looking back to the touching relationship of visitors to objects in early museums in the West or as playful subversions of the proscription to touch, because these are objects made with a serious purpose, precisely so as to encourage this kind of contact. The touch of the visitor is a way of establishing their veracity in a different, Japanese sensory order of cultural relations that is haptic as opposed to scopic. This difference is manifested through the act of

 The importance of sight in the scientific practice influenced its privileged position as the primary sensory faculty of the emerging discipline of anthropology.

704

Rupert Cox

copying rather than the feel of the touch because the ceramic board surfaces of all the reproductions feel the same. That is to say that what the touching hand makes contact with is not with the difference between say Picasso’s Guernica or Leonardo’s Last Supper, but rather with the particular Japanese skill in creating and applying the technology that created these copies. So as a method of representation and interaction, these ceramic board copies flatten out the differences of art-works for the sake of establishing a model of aesthetic communication that integrates a Western epistemology with a Japanese relational ontology. In this way the technology of copying and the tactility of contact at work in the museum is an expression of the traditional knowledge of denshô and it is this knowledge – the skill manifested through the body of the craftsmen and their tools – that the visitor may apprehend through their touch. This mode of representing cultural knowledge through the act of representation rather than the reading of its visible form is a paradigm for visual anthropology because it does not confuse the capacity to visualise images of other cultural worlds with their meaning. Rather, it creates a productive analogy between the act of looking at images (and in this case touching) with the act of cultural perception that went into its creation. This is what we might call “sensory vision” and it may draw the visitor into a relationship with the Japanese nature of the works on display. This sensory vision is distinct from the ways that the museum otherwise tries to embellish a sense of contact with the original work of art by engaging the other senses as in the case of the reproduction of the Scrovegni Chapel from Padua where my guide related that it was possible to close the doors of the chapel, turn on appropriate music and imagine you were not in Japan at all; something which he confessed that he liked to do. The addition of music extended the idea of touch into a flight of fancy. In order to understand the ontological issue as stake here, about the material nature of the copy, we need to look more closely at the process of its creation.

5 Surface and depth The knowledge of art history depends in important ways on the technological practices of photography and the realm of digital reproduction, for it is through these means that works of art can be perused and discussed separately from the mostly public and sometimes inaccessible situations of the original work. Debates8 about the comparability of the textural qualities of the original and the digital resolution of its photographic copy help to focus in on the question of what may

 The most high profile of these debates has been between the scholars Lev Manovich and WJT Mitchell about the degrees of abstraction and complexity in the digital domain and whether it is even sensible to think of “visual media” as purely “visual” anyway.

Transforming art and visual anthropology

705

be gained from high resolution digital reproductions. Such reproductions carry with them the inherent possibility of a certain kind of concentrated looking that may bring the observer “inside” the composition of the work so as to understand the techniques by which it was composed. What may be lost through this incisive gaze is the “auratic” sense of contact the original that Benjamin (1968) has described as its unique if not sacred quality. The detail offered by the digital domain suggests the promise of going beyond a surface description and creating a deeper level of perception. This is what visual anthropologists strive for as a quality of their image making, but are all too often accused of failing9 to achieve, because the detail of a photographic image is never quite the same thing as the meaning of the scene it reproduces. In the Ohtsuka museum art-work copies, the same problem was in evidence with their desire to use digital photography techniques to make what they described as “real” reproductions without any creative interpretation. It was for this reason that the photography of the original art works presented many problems. At issue was the enlargement of the photographs, to the size of the original art-work that often resulted in a loss of resolution in the quality of the image.10 This was the point at which digitization most critically intercepted the path from the original to copy in two ways. Firstly because in the making of the ceramic boards, the “original” was the photograph, so its trajectory was subject to the limitations of photography and the digital domain. Secondly the properties of the image could now be changed in accord with the shape, colours and textural features of the ceramic plates. To ensure the fidelity of the finished copy, the museum hired a panel of fifteen European experts11 in the copying of well-known art-works. Following their advice the museum arranged in some cases for the facsimile to be hung temporarily alongside the original for comparison. The paintings were then assessed to be qualified as sufficient in ways that we might normally consider as contrary and even regard as proof in a negative sense of their fakery. These contrary criteria for evaluation were termed “comprehensive non-literal similarity” meaning whether the fundamental essence or structure of the original had been duplicated in the copy and “fragmented literal similarity” which referred to the literal similitude of the visible aspect of a copy of a small segment of the original on one of the porcelain boards. The aesthetic integrity of the structural essence of the original taken as a whole, was then held in check by the visible similitude of any one of the

 The accusation that the images made through photography and visual anthropology is only a surface rendering of the world of experience and meaning has been most famously advanced by Hastrup (1992) and countered by Taylor (1996).  This problem was in part a function of the available technology in 1998 when the photographs were made.  The names of these “experts” were never disclosed to me during my interviews with the director of the museum and in my guided visit to the gallery space.

706

Rupert Cox

uniformly sized composite sections of ceramic plates. Ideally the composite part and the structural whole would together convey a productive “realism”. Further concerns for a form of realism were evident in the picture frames. These were specially commissioned for every single piece in the collection from workshops in France and Italy and cost in total fifteen million pounds with the most expensive single frame being five hundred thousand pounds.12 Surprisingly, the reproductions only took one year to complete and when the museum was finally opened, it was this technical feat, along with the sheer scale of the enterprise in terms of its size and the amount of money spent, which convinced the president that the reproductions could not be regarded as mere “copies”. He argued that the skill and artistry required in the act of copying the photograph of the original onto the porcelain surface made the copy a new work of art in its own right, separate from the original. For the director, the museum’s art-works may be the arrangement of uniformly composite parts or what was called “point-by-point tracings” (môsha) of originals, but the special “technologies of allure” that he had devised, that allowed for the creation of a new porcelain based aesthetic with a distinctive brightness and clearness of color. Such efforts were partly necessary he claimed, because of the “pressure” he felt in the presence of pictures with a Christian theme, to make reproductions of a very high quality. In spite of being a self-confessed atheist, he said it was only when he had made a presentation of one of his reproductions to St Ignatius cathedral in Tokyo that he felt some “calm in his mind” about the whole project. It is interesting in a historical as well as a personal sense that the director ascribes to these foreign art-works, the means for aesthetic experience to be a means of personal transformation and in doing so refers to Christian art as a source for the motivation to make copies. It was schools of Western style painting created by Jesuit missionaries in the 16th century that were used to communicate the tenets of the Catholic faith through a method where the adepts at these schools were required to copy paintings with religious themes (Curvelo 2007). The motivations behind the Ohtsuka museum project were also local and pragmatic: to overcome the technical challenge of creating a new art form (the first innovation in porcelain manufacture since its introduction from Korea in 1616) and to construct a museum that would provide a complete education in the history of Western Art. Official recognition for these efforts was achieved when three visitors from the Katsura Rikyû museum in Kyoto claimed that the quality of the works in the museum were equal to any of the “national treasures” they had in their own collection. If we think in terms of these personal, practical and historical factors, the copying capacities of photographic digital technology used by the Ohtsuka are not  These were figures given to me by the director of the Ohtsuka company in the course of our interview in 2000.

Transforming art and visual anthropology

707

merely re-producing the surface appearances of originals and re-presenting them in a new context against the white walls and corridors of the museums space.13 They are also entering into a deep engagement with the original, which becomes apparent through their concern for “copyright”. Copyright permission was obtained for all the works in the museum because the reproductions were always five percent bigger or smaller than the originals. Further recognition for the fidelity of the Ohtsuka museum copies was sought through personal letters from living artists and a celebrated visit from Jan Miro. The law of copyright on photographs of works of art at the time of the museum’s creation in 1998 was far from clear. It was generally understood that the copyright resided in the photograph or its negative, not in work of art itself and it was the commissioner of the photographs – in this case the Ohtsuka museum – not the owner of the original work being photographed who had the original rights. This was a position complicated by the storage and replication of images on computers but the museum proceeded on the basis that while most copyrights on artistic and literary productions expire, the copyright on photographs of art images were deemed to be perpetual.

6 Copyright/rites It was this notion that the director reiterated to me, emphasizing that the acts of copying on display in the museum were necessary because the object which is not copied is under threat from deterioration, and here in the museum, through photography, computer technology, and their innovative methodology of ceramic board manufacture they could protect and conserve the fidelity of the original for all time. The real value of the Ohtsuka collection would therefore only become evident in one hundred years or so from now. The director’s discourse about the importance of salvage, with its desire to freeze and preserve the condition of the original at a certain point in time would appear to work against the proposition that these copies are also new works of art in their own right. How are these copies both modes of preservation and of creation? I have argued here that these notions productively co-exist because underpinning them is the recognition of the Japanese traditional skill – denshô – which goes into their (re)making. It is this knowledge that makes the copies that are produced by the technologies and skills of the Ohtsuka craftsmen acts of creation and allows visitors to touch art-objects and feel the nature of this Japanese originality for themselves at the same time that they may see the visible arrangement of art-works as pathways to cultural knowledge of the West. It is also what brings us back to Taussig’s notion of the creativity and

 The reference point and inspiration for the director in thinking about the concept of a museum of copies as was the surrealist Andre Mulraux’s Museum without Walls.

708

Rupert Cox

sticky-ness of contact in the copy, because we are dealing here with a form of “tactile vision” For visual anthropology, the compulsion to use photography and film to capture and fix the images of others was and to a degree still is driven by the desire to preserve and protect traditional forms of knowledge and ways of life. The contemporary ubiquity of these kinds of images as they circulate and are reproduced in new media networks has made the inherent if not inevitable capacity of the camera to create as well as to record a complex and problematic technology to apply. At issue is the politics of representation and ethical criteria which have turned what were copy-rites – practices of image making based on intimate relationships between people and between people and things – into copyrights – the legalistic framework for protecting and restricting the image of the indigenous “other”. What the Ohtsuka museum reveals for visual anthropology is how the copy-rites expressed in the “skilled vision” (Grasseni 2007) of technologies of image and material production can add value to a copy, not as a mere function of the technology and the reproduction of the visible surface of things, but as a deep, touching engagement with the material nature of images, conveying feeling as well as insight. Ultimately what sense are we to make of the Ohtsuka museum? When considering the value of the collection as a repository of Western cultural knowledge our distinctions between the authentic original and the artificial copy are not very useful, based as they are on an understanding of the aesthetic primacy of vision as a direct and discrete mode of communication. For post-modern commentators, the Otsuka museum may confirm that there is no distinction in Japanese culture between the authentic and the artificial and that the mechanical reproduction of copies and their contemporary proliferation confirms that the act of copying in Japan, mechanically aided or by hand, is above all a cultural act and that to copy is to acquire (in this case) Western cultural knowledge and at the same time to subvert it by self-consciously affirming the Japanese qualities inherent in the acts of reproduction. However, my interview with the president of Ohtsuka suggested that rather than there being any playful ironies at work in the creation of the museum collection it was the equation of the enterprise with a matrix of Japanese national-cultural values of technological innovation, traditional skills in copying as an act of creation (denshô), the preservation of art and its educational value in being presented all in one place and on such a scale, that drove the initiative. For the visitors to the museum whom I spoke with, the artificiality, artistry and sheer ambition of the project were apprehended through moments of tactile interaction alongside their path-finding along networks of art-history. For these museum goers as for the agents and subjects of visual anthropology, just as there is no such thing as “just looking” there is no such thing as just copying in the sense we make of visual communication.

Transforming art and visual anthropology

709

References Banks, Marcus and Howard Morphy (eds.). 1997. Rethinking Visual Anthropology. New Haven: Yale University Press. Benjamin, Walter. 1968. The Work of Art in the Age of Mechanical Reproduction. In: Hannah Arendt (ed.), Illuminations, 217–252. New York: Schocken Books. Benjamin Walter. 1999. The Arcades Project. Edited by Susan Buck-Morss. Cambridge: MIT Press. Brennan, Teresa and Martin Jay. 1996. Vision in Context: Historical and Contemporary Perspectives on Sight. New York/London: Routledge. Brodsky, Joyce. 2002. How to “see” with the whole body. Visual Studies 17(2): 99–112. Classen, Constance. 1993. Worlds of Sense: Exploring the Senses in History and Across Cultures. London: Routledge. Classen, Constance. 1997. Foundations for an anthropology of the senses. International Social Science Journal 153: 401–12. Classen, Constance. 2005. Touch in the Museum. In Constance Classen (ed.), The Book of Touch, 275–286. Oxford: Berg Press. Clifford, James and George E. Marcus (eds.). 1986. Writing Culture: The Poetics and Politics of Ethnography. Berkeley: University of California Press. Corsin-Jimenez, Alberto and Rane Willerslev. 2007. An anthropological concept of the concept: reversibility among the Siberian Yukaghirs. Journal of the Royal Anthropological Institute (N.S.) 13: 527–544. Cox, Rupert. 2007. The Culture of Copying in Japan: critical and historical perspectives. London: Routledge/Curzon Press. Cox, Rupert and Chris Wright. 2012. Blurred Visions: Reflecting Visual Anthropology. In Richard Fardon and John Gledhill (eds.), The SAGE Handbook of Social Anthropology, 1424–1463. London: Sage Press. Crary, Jonathan. 1990. Techniques of the Observer: On Vision and Modernity in the Nineteenth Century. Cambridge: MIT Press. Curvelo, Alexandra. 2007. Copy to Convert: Jesuit’s missionary practice in Japan. In Rupert Cox (ed.), The Culture of Copying in Japan: critical and historical perspectives, 111–128. London: Routledge/Curzon Press. Duncan, Carol. 1991. Art Museums and the ritual of citizenship. In Ivan Karp and Steven P Lavine (eds.), Exhibiting Cultures, the politics and poetics of museums, 88–103. Washington: Smithsonian Institution Press. Edwards, Elizabeth. 2001. Raw Histories: Photographs, Anthropology and Museums. Oxford, England: Berg Press. Edwards, Elizabeth (ed.). 1992. Anthropology and Photography, 1860–1920. New Haven: Yale University Press. Elkins, James. 1996. The Object Stares Back: on the nature of seeing. New York/London: Harcourt Inc. Press. Fabian James. 1983. Time and the Other: How Anthropology Makes its Object. New York: Columbia University Press. Grasseni, Cristina (ed.). 2007. Skilled Visions: Between apprenticeship and standards. New York: Berghahn Books. Griffiths, Alison. 2002. Wondrous Difference: Cinema, Anthropology, and the Turn of the Century Visual Culture. New York: Columbia University Press. Griffiths, Alison. 2008. Shivers Down Your Spine. Cinema, Museums and the Immersive View. New York: Columbia University Press. Hastrup, Kirsten. 1992. Anthropological Visions: Some notes on visual and textual authority’. In Peter Crawford and David Turton (eds.), Film as ethnography, 8–26. Manchester: Manchester University Press.

710

Rupert Cox

Hetherington, Kevin. 2002. The Unsightly: Visual Impairment, Touch and the Parthenon Frieze. Theory, Culture and Society 19(5/6): 187–205. Howes, David (ed.). 1991. The Varieties of Sensory Experience: A Sourcebook in the Anthropology of the Senses. Toronto: University of Toronto Press. Howes David. 2003. Sensual Relations: Engaging the Senses in Culture and Social Theory. Ann Arbor: University of Michigan Press. Howes David. 2005. Empire of the Senses: The Sensual Culture Reader. Oxford/New York: Berg Press. Ingold, Tim. 2000. The Perception of the Environment: essays on livelihood, dwelling and skill. London: Routledge. Ingold, Tim. 2007. Lines: A brief history. London: Routledge. Martin, Jay. 1988. ‘Scopic Regimes of Modernity’. In Martin Jay (ed.), Force-Fields: Between Intellectual History and Cultural Critique, 115–33. New York/London: Routledge. MacDougall, David. 2006. The Corporeal Image: Film, ethnography and the senses. Princeton: Princeton University Press. Marks, Laura U. 2000. The Skin of Film: Intercultural Cinema, Embodiment and the Senses. Durham: Duke University Press. Mitchell, WJT. 1996. What do Pictures Really Want? October 77: 71–82. Mitchell, WJT. 1994. Picture Theory: Essays on Verbal and Visual Representation. Chicago: University Chicago Press. Okely, Judith. 2001. Visualism and Landscape: Looking and seeing in Normandy. Ethnos 66: 99– 120. Pink, Sarah, László Kürti and Ana Isabel Afonso (eds.). 2004. Working Images, visual research and representation in ethnography. New York/London: Routledge. Rony, Fatimah. T. 1996. The Third Eye: Race, Cinema, and Ethnographic Spectacle. Durham: Duke University Press. Taussig, Michael. 1993. Mimesis and Alterity: A Particular History of the Senses. New York/ London: Routledge. Taylor, Lucien. 1994. Visualizing Theory: Selected Essays from Visual Anthropology Review 1990– 1994. New York/London: Routledge. Taylor, Lucien. 1996. Iconophobia. Transition 69: 64–88. Wright, Christopher. 1998. The Third Subject: perspectives on visual anthropology. Anthropology Today 14: 4.

Maria Nikolajeva

33 Emotion ekphrasis: representation of emotions in children’s picturebooks Abstract: The chapter considers how various emotions (happiness, sadness, anger, fear) are conveyed through the interaction of word and image in multimedial texts addressed to young readers. The theoretical framework employed in the chapter develops ideas from cognitive literary theory adapting it to the specific conditions in which there is a significant difference between the sender’s and the recipient’s cognitive and affective level. The concept of emotion ekphrasis is used to demonstrate the various ways of representing emotions. Special attention is paid to the issues of mind-reading, empathy, identification, projection, simulation, misattribution and other aspects of recipients’ engagement. The predominantly theoretical argument is illustrated by a number of classic and contemporary picturebooks.

1 Introduction A picturebook is a narrative art form that creates meaning through the synergy of two media, the verbal and the visual. The common misconception about picturebooks is that they simply are books for very young children, with few words and with images that complement the words and have predominantly a decorative function, all based on a loose and unscientific claim that “children like pictures”. There are indeed good reasons why children like pictures, and there is vast research within psychology and neuroscience about why and how children and adults engage with visual stimuli; however this is beyond the scope of the present chapter. For my argument it is sufficient to agree that picturebooks, unlike “books with pictures”, are a complex multimedial phenomenon that addresses its audience at different levels of sophistication. Although picturebook studies have relied heavily on visual semiotics and communication (e.g. Mitchell 1994; Kress and Van Leeuwen 1996), it is the interaction of text and image that is highlighted as the foremost characteristic of picturebooks (Nodelman 1988; Nikolajeva and Scott 2001; Lewis 2001). The narrative co-created by words and images is referred to as iconotext, and the various types of dynamic relationships between words and images range from symmetric (words and images duplicate each other) and complementary (words and images compensate each other’s insufficiencies) to counterpointing and contradictory (words and images tell different stories to create a complex whole). There is a tendency in recent picturebook research to focus on counterpointing picturebooks, frequently labelled “postmodern” to reflect their playfulness, selfreferentiality, metafiction, fragmentation, frame-breaking and other elements asso-

712

Maria Nikolajeva

ciated with postmodern philosophy (e.g. Sipe and Pantaleo 2008). While such books undoubtedly present a challenge for scholars, they are questioned by educators as suitable for young audience. There is, however, considerable empirical research convincingly demonstrating that even very young children are capable of making sense of complex picturebooks (e.g. Arizpe and Styles 2003). A possible explanation of educators’ scepticism is, once again, the common belief that picturebooks are intended for young children and that school-based literacy should primarily involve the mastery of verbal, written language. The supremacy of the verbal over the visual in formal Western education regrettably results in children losing their innate ability to engage with images – the ability to a high extent connected to affective responses. In other words, images evoke our emotions stronger than language. There are intricate neuropsychological reasons for this which I will not delve into, beyond stating, with support from cognitive science (e.g. Evans 2001), that while visual stimuli are hardwired, verbal ones are not; that affective response to visual stimuli is evolutionarily dependent; and that visual stimuli are processed by the brain fractions of seconds quicker than the verbal ones and through different paths. All this has far-reaching consequences for picturebook studies, but there is so far no reliable research. Therefore the following argument is purely exploratory and has no solid empirical evidence. In this chapter I consider how various emotions can be conveyed through the interaction of word and image in picturebooks addressed to young readers. To do this, I borrow and adapt some ideas from cognitive literary theory that combines recent achievements in brain research with the most profound issues of literary criticism, including why reading offers aesthetic pleasure (e.g. Tsur 1992; Stockwell 2002; Hogan 2003). The particular area of cognitive criticism I employ (Zunshine 2006; Vermeule 2010; Hogan 2011; Oates 2011) examines the issues of readers’ cognitive and affective engagement with fiction, or, to shift the focus of attention from readers to texts, how fiction, through various narrative strategies, can alert readers to get engaged. Fiction, as cognitive criticism claims, creates situations in which emotions are simulated, and because of certain mechanisms in our brains, we can respond to fictional emotions as if they were real. There is so far very little research within cognitive criticism that takes into consideration young readers, who not only lack the experience of a full range of human emotions, but who have not yet fully developed their theory of mind and empathy skills (both emerge around the age of four). Theory of mind, a concept from cognitive psychology, refers to the capacity to understand how other individuals think. Empathy refers more specifically to the capacity to understand how other people feel. It is intriguing for a picturebook theorist to consider how engagement with picturebooks can endorse theory of mind, which is an indispensable social skill. I will refer to the verbal or visual representation of emotions as emotion ekphrasis. The term ekphrasis is used in intermedial studies to denote a representa-

Emotion ekphrasis: representation of emotions in children’s picturebooks

713

tion of an art form by means of a different art form or medium, for instance, a description of a building in a novel, or a musical representation of a painting. Emotion ekphrasis implies a representation of an emotional state by verbal, visual or multimedial means. In prose fiction, emotion ekphrasis is dependent on language. It can be simple and straightforward, for instance, a narrative statement: “He was happy” or direct speech: “I am happy”. It can be further expanded into complex depictions of mental states, involving a variety of narrative devices such as direct thought, reported thought, internal focalisation, free direct discourse, free indirect discourse and psychonarration, all minutely examined by narratologists (e.g. Cohn 1978; Banfield 1982). Narrative theory of the novel rightfully claims that prose fiction is the only art form which allows the receivers direct access to other people’s mind. However, emotions are, unlike language, non-linear, imprecise and diffuse; they are non-verbal by definition. There are degrees of joy and distress; there are nuances between joyful, glad, happy, and exhilarated, between sad and upset, angry and furious. There is a distinction between a momentary emotion, such as joy or fright, and a more permanent mood, such as happiness or anxiety. Verbal language is therefore essentially inadequate to convey “subverbal states” (Cohn 1978). Figurative language, especially metaphors, is a powerful device to approximate the nebulosity of emotion. Yet visual images carry a more powerful potential. As already noted, visual stimuli are faster and stronger than verbal ones. Thus viewing a person’s facial expression or bodily posture sends a stronger signal to the brain than reading the verbal statement “This person is happy, sad, angry or frightened”. Many picturebooks utilise wordless doublespreads to convey strong emotions for which words would be insufficient and inadequate. The best known are perhaps the three wordless spreads in Where the Wild Things Are, following the protagonist’s outcry: “Let the wild rumpus begin”; and the wordless spread in The Tunnel, expanding the mini-ekphrasis: “She was frightened” (see Nikolajeva 2012). Picturebooks offer perfect training in mind-reading even for very young, pre-literate children. (Needless to say, this process continues well into adulthood, which explains why most adults enjoy and engage with visual media). Moreover, reading visual emotion ekphrasis is significantly closer to the ways we employ theory of mind in real life. Unlike novel characters that can be transparent (cf. Cohn 1978), characters depicted visually are always opaque. In a multimedial narrative, such as a picturebook, images substantially enhance the meaning expressed by words, emulating the vague and indefinable emotion. A visual image can potentially evoke a wide range of emotions circumventing the relative precision of words. Multimedial narratives frequently make use of ambiguity created in the interaction between media when conveying a character’s emotional state.

714

Maria Nikolajeva

2 Reading basic emotions Examining a sufficiently large corpus of picturebooks, whether they directly focus on emotions or deal with them as an integral part of the plot, it becomes obvious that classic and modern books, Western and from other parts of the world, contain images that, alone or in combination with words, express a wide range of emotions, both basic emotions and social, or higher-order emotions. Basic emotions, that include joy, distress, fear, anger, surprise and disgust (Oatley 1992) are universal, for instance, independent of the verbal language; physical manifestation for them, notably facial expression, but also body posture and gestures, normally do not require any special training (Evans 2001). Empirical research shows that even babies as young as two weeks old respond adequately not only to their caregivers’ faces, but also to visual representations of facial expressions; moreover, babies can produce these expressions themselves; apparently even babies who are born blind can do so. The implication is that although children cannot articulate a basic emotion, they can still experience it and recognise it in others. Unlike basic emotions, social, or higher cognitive emotions, such as love, guilt, shame, pride, envy, and jealousy, are not innate, or at least considerably less innate than basic emotions; and they may be culturally dependent. The concept of social emotions emphasises that they involve more than one individual and are thus subject to negotiations. Love is a social emotion requiring that two individuals’ ultimate goals, to be happy, become equally valuable for both. Social emotions are at least partly connected to basic emotions. For instance, love can only lead to happiness if it is reciprocal. Unrequited love, on the contrary, leads to distress. Guilt comes from a contradiction between the basic emotion of joy and an action that disturbs the joy. Physical disgust corresponds to the social emotion of hatred. Envy and jealousy originate in frustration on observing somebody else’s happiness. This background from cognitive science is of overall importance for our understanding of how picturebooks communicate with readers and elicit their affective responses. Young children may not know what the verbal statements: “He was sad” or “She was frightened” mean. Yet they will presumably respond to a visual ekphrasis of sadness or fear. Social emotions are not directly connected to external expressions and thus more difficult to communicate. While there are emoticons for all shades and degrees of joy, sadness or anger, it is problematic to create a universal facial expression for envy or pride. If we need unequivocal signifiers for social emotions, we have to use symbols rather than icons – such as stylised heart for love or skull for hatred – which are culturally dependent and generally need insider knowledge. We can also infer, based on our real-life experience or previous exposure to stories, that two people in close embrace love each other and are happy together. Thus social emotions are more complex and composite; they can also be self-contradictory and lead to positive as well as negative outcomes. Young

Emotion ekphrasis: representation of emotions in children’s picturebooks

715

children are initially solipsistic, but gradually become emotionally socialised, that is, trained to both express their emotions in a manner comprehensible for other people and understand other people’s emotions and empathise with them. It is frequently social emotions that feature in art and fiction; most of world literature is centred around at least one of them. Picturebooks are no exception. But how exactly do we read emotion ekphrasis? What are we looking at? How do we apply theory of mind to fictional characters? How does representation of emotions work in a multimedial narrative, in which words and images can collaborate or contradict each other? And not least, why do we care about fictional characters at all? To begin with, let us distinguish between diegetic, or textual emotion and extradiegetic, or extratextual emotion. Extradiegetic emotions, which are predominantly basic, are the ones we experience independently of fictional characters; for instance, we become scared of an image or happy, sad or disgusted, seeing an image. Such affective responses are hard-wired in the brain and are evolutionarily conditioned. It is reasonable to ask why we would engage with pleasant, disgusting or scary images at all. They are not real things, but representations of things, and as representations, are neither dangerous nor satisfactory; they can seemingly never elicit the same emotional response as the real things. However, against common sense, most of us have indeed laughed at funny images or become upset about disturbing ones. Cognitive criticism explains this by the complex mechanism of mirror neurons, which enables our brain to simulate responses to representations as if they were real visual stimuli. The same mechanism allows us to engage vicariously with fictive characters emotions, diegetic emotions; and these will be the focus of the present argument. Diegetic emotions are thus those within the narrative, within the diegesis; the emotions that the characters experience. In our involvement with multimedial narratives, these two kinds of emotions inevitably interact. We can both be scared of an image and empathise with the character who is scared. Moreover, we should be able to read the character’s fear even if we do not experience the same emotion when looking at the image; or understand that the character is sad without feeling distressed ourselves. This is where theory of mind comes in, Our emotional response to visual ekphrasis is possible because we have stored, albeit inaccurate and fragmented memories of the represented emotion, either from real-life experience or from an earlier experience of fiction, whether verbal, visual or multimedial. We may not have a direct experience of extreme distress or extreme fear, but the little experience we have is sufficient to trigger the memory. When reading visual emotion ekphrasis, we are looking for recognisable external tokens of emotions, because this is how we employ theory of mind in real life. Cognitive studies claim that the most prominent features that reflect human emotions are the eyes and the mouth, and that these features are universal. This is probably why emoticons work for universal visual communication. The shape of

716

Maria Nikolajeva

Fig. 1: The Lost Thing.

the mouth is arguably the most salient trait, and therefore used excessively in picturebooks. Happiness and sadness are clearly recognisable by the up-turned and down-turned corners of the mouth. A wide-open mouth may signify fear or anger. Wide-open eyes signal either surprise or fear. A combination of closed eyes

Emotion ekphrasis: representation of emotions in children’s picturebooks

717

and open mouth suggests distress. Raised eyebrows may convey anger. Facial expression is, however, not the only way of communicating emotions. Cognitive science emphasises that emotions are embodied, that is, connected to body movements and spatial position. Indeed, we can recognise emotions, in real life as well as in visual representations, through body language. Thus, outstretched arms signal joy, while limp arms hanging on the sides of the body signal distress. In reading images, we project our own embodied emotions onto represented figures. In addition, when reading picturebook images, which are part of fictional narratives, we also apply our knowledge of artistic conventions. We read characters foregrounded and placed in the centre of an image as happy and content; while we read characters crammed in corners or pushed into the background as unhappy, lonely and scared. We attribute happiness to characters positioned high on the page. We connect certain moods with certain colours: red with aggression, yellow and green with joy, gray and black with distress; brown perhaps with disgust since it reminds us of faeces. Not least, we may be familiar with graphic conventions such as motion lines or thought balloons. Motions lines, for instance, apart from conveying movement, may signify strong emotions such as fear and anger. Thought balloons can contain further images reflecting the character’s state of mind. They can also include images of pleasant, sad or scary memories. The convention of emoticons makes a wide register of emotions easily understandable when we encounter them in multimedial narratives. Although it does not happen in real life, by convention, a zigzagged mouth implies anger or fear. Hair on end suggests extreme fear. In semiotic terms, basic emotion ekphrasis is an indexical sign: it does not directly represent the signified, but points at it. In the following, I offer some examples of how insights from cognitive criticism can be employed to analyse our engagement with diegetic emotions in a number of classic and modern picturebooks, with a particular focus on Curious George, by H. A. Rey, Little Blue and Little Yellow, by Leo Lionni, and The Lost Thing, by Shaun Tan (Figure 1). My selection is based, firstly, on the fact that these are well-known award-winning books which, however, have not been considered from the cognitive point of view. Secondly, from hundreds of picturebooks that I have worked with and taught, these felt the most suitable to illustrate the current argument.

3 Reading symmetrical ekphrasis Although there are wordless picturebooks where the title may be the only guidance toward interpretation of emotions, typically, picturebooks are based on the interaction of words and images, the iconotext. This interaction can be more or less symmetrical: that is, the words say: “He was happy”, and the image supports the verbal statement. In picturebooks for young children, verbal statements are frequently used to guide our interpretation of the image, for instance, through the

718

Maria Nikolajeva

title, such as Frog is Sad. Without the title, our interpretation of the cover image would arguably be less precise; we could read it as “The character is sad”, “The character is meditative” or “The character is bored”. Yet the most likely choice is “sad” because sadness is a basic emotion. Nuances of the emotion can be conveyed verbally with the range of sad, upset, melancholy, pensive, miserable, gloomy, unhappy, anxious, and so on. From images, we cannot exactly decide on the nuance. This is both an advantage and disadvantage. The advantage is a wider range of interpretation and subsequently response. The disadvantage is the steering of interpretation. One might argue that explicit verbal ekphrasis in picturebooks is redundant because images evoke emotional response more immediately and directly. Most probably it is employed as a pedagogical device that helps young children to verbalise the emotion, in order to recognise it later; but perhaps also as a step in language acquisition: from the basic “sad” toward a wider range of emotions. On the other hand, the interaction of word and image makes a connection between the vicarious emotional experience and its verbal description. When words are symmetrical with images, mind-reading is restricted. However, to assume that a young child might not be able to read basic emotions without verbal support is dubious. As already mentioned, even very young children respond adequately to visual representation of basic emotions. Empirical research confirms that children respond stronger to visual emotion ekphrasis when it is not supported by words. Let us consider how we respond to a relatively simple ekphrasis. The establishing page of H. A. Rey’s Curious George is seemingly a good example of symmetry: George the monkey is smiling as he swings in a tree, and the words confirm our immediate interpretation of the character’s mental state: “This is George. He lived in Africa. He was very happy”. There are, however, some communicative worries with this iconotext. Firstly, we can only engage with the protagonist’s emotions because we anthropomorphise him, that is, ascribe him human consciousness. In real life, monkeys do not smile, and when they bare their teeth, it is typically a sign of aggression. Yet, even without the prompt from the words, we instantly recognise the character’s emotion as happiness. This is possible as long as we acknowledge the iconotext as fiction (for instance, we would be less certain with a photograph of a monkey with bared teeth) and the animal character as representing a human being. For a number of reasons anthropomorphic animals are abundant in children’s literature, especially in picturebooks. As readers we are therefore likely to engage with animal characters’ emotions as if they were humans. Secondly, the verbal statement “He was very happy” comes from an omniscient narrator who has strong authority. The communication chain of the statement can be expanded to: “The narrator wants me to believe that George is happy”, rather than the immediate mind-reading through the image: “Based on the external signs, I recognise the character’s state of mind as happiness”. The verbal ekphrasis is not a direct representation (as we can argue the image is), but a meta-

Emotion ekphrasis: representation of emotions in children’s picturebooks

719

representation, originating in a narrative agent (cf. Zunshine 2006). Further, there is a difference between a permanent state of happiness and a momentous state of joy. Since the image is static and can only indirectly convey duration, we tend to interpret the character’s emotion as a temporary mood (“He feels happy right now”), while the verbal statement, not least through the past tense, emphasises a permanent state or even a character trait, equal with joyous, cheerful or carefree. Thus the ostensible symmetry of word and image in fact allows substantial complexity as soon as we consider exactly how we engage with the iconotext. For instance, the final, rather cynical statement as the happy monkey is put into a zoo – “What a nice place for George to live!” – is accompanied by the image of the character smiling as broadly as in the beginning. It is easy for the reader to succumb to the interpretation imposed by the iconotext. The verbal narrator wants us to believe that George is happy. The image suggests, based on the external indicators, that George feels happy. However, captivity is usually incompatible with happiness. We know that the character feels happy, but we also know that the happiness is false. However, to make this inference, we need to bring in our previous experience, either from real life or from other fictional texts. In addition, although we empathise with the character, we need to liberate ourselves from his subjectivity in order to assess his misjudgement of his own situation. Cognitive criticism would maintain that the fictional character and the real reader – in this case me as an adult, professional reader – have different goals and therefore respond differently to the outcome. For me, the outcome is unsatisfactory, although I also can understand that George feels happy. Empirical studies show that young children cannot synthesise contradictory information. They are therefore likely to “identify” with the character, that is, uncritically share his explicit state of mind, rather than pose a more far-reaching question of “how would I feel if I were placed in a cage”. Interestingly enough, a much later book, Zoo by Anthony Browne, encourages exactly the child’s empathy with incarcerated animals that Curious George lacks. Curious George is a simple narrative in its affective engagement. Throughout the narrative, the mental state of the character alternates rapidly between joy and distress, which is conveyed exclusively through images and in most cases through facial expression. Likewise, Frog is Sad, solely focused on the character’s change of mood, represents emotions more or less symmetrically through words and images. Other picturebooks in the same series, Frog is Frightened, Frog is Frog and Frog in Love, demonstrate more complexity. Fear, for instance, is a directed emotion: we are scared of something. In Frog is Frightened, the reader is encouraged to disengage from the character, without losing empathy. A very young reader may experience extradiegetic fear; it is, however, more likely that the reader understands that Frog is scared, but that there is no real danger: empathy without identification. Further, while the cover of Frog is Sad is symmetrical in word and image, the cover of Frog is Frightened shows three animals sleeping peacefully in a bed,

720

Maria Nikolajeva

evoking a sense of security rather than danger. Frog is Frog offers a slightly more complex iconotext, since the title effectively equals “Frog is happy”, while using a verbal metaphor rather than a literal statement. Finally, Frog in Love involves, in addition to the basic emotions of distress and joy, the social, or higher-order emotion of love. These very simple books exemplify the wide range of possibilities even within relatively symmetrical iconotexts (for a detailed discussion of the Frog books, see Nikolajeva 2012).

4 Reading absent faces While we have no problems understanding George’s or Frog’s emotion ekphrasis, it is almost impossible to recognise an emotion when there is no external expression of it. Non-figurative art can elicit extradiegetic emotions, but if there are no characters, human or non-human, to whom we can ascribe emotional states, our empathy will remain switched off. A classic picturebook, Leo Lionni’s Little Blue and Little Yellow, features characters who are blots of colour. They have no faces and no limbs to express their emotional states. The inferences we make from the visual representation of the main character, Little Blue, are based on conventions. Bright blue is, at least in the Western culture, a “happy” colour. Little Blue is depicted in the middle of the page, which suggests confidence. The background is white negative space, which also emphasises Blue’s independent position. Generally, the establishing page conveys harmony and therefore joy. In the subsequent pages Blue is presented in various situations that confirm the character’s happiness, further enhanced by words: Blue has a safe home with two parents; he has friends, one of whom is his special friend, Little Yellow. (Interestingly enough, there is no personal pronoun in the verbal text that would suggest Yellow’s gender; while we know that Blue is a boy, Yellow can be either a boy or a girl. For the sake of gender balance, I will refer to Yellow as a girl). Yellow is likewise a bright, joyful colour, suggesting that Little Yellow is just as happy as Blue, and she also has a home with two parents. When Blue one day finds Yellow’s house empty, the words do not explicitly say that Blue is upset. The absence of face provides no clues. The reader may naturally infer that you get upset when you want to meet a friend but cannot find them. However, the iconotext also employs visual means to convey the character’s state of mind. The sentence “He looked here/and there/and everywhere […]”, divided between three pages, is accompanied by images of Little Blue shifted from his initial position in the centre of the page toward the edges. Moreover, while the first of the three pages features the usual white negative space, the second page has a black background and the third page, red. Black has negative connotations and is likely to evoke fear, probably because it is associated with darkness and therefore with danger. Images by themselves cannot convey temporal duration, and there is no indication of any

Emotion ekphrasis: representation of emotions in children’s picturebooks

721

longer span of time (for instance: “He looked until it got dark”). Therefore darkness is not an objective external representation, but the character’s emotion ekphrasis. Blue is scared. The red background on the next page may be viewed as reflecting anger. It affects the viewer’s extradiegetic engagement since our brain ostensibly reacts to red by aggression. But is it also Blue’s emotion discourse: he is now angry because his friend has abandoned him and is nowhere to be found. Conversely, we may decide that red, in contract to the black of the previous page, suggests hope, and the words may prompt this interpretation by adding: “[…] until suddenly, around the corner […]”. In fact, Blue is in this page closer to the centre and thus to the initial balance and harmony. Indeed, around the corner, that is, on the next page, Blue finds Yellow. A reunion of friends or lovers is the most satisfactory ending of certain narrative genres, not least children’s books. The reason offered by cognitive criticism is that such a reunion indicates that two individual goals, happiness, have been successfully reconciled. Our goal as readers coincides with the characters’ goals, and we experience vicarious happiness through the happiness of the characters. In case we have missed the point, the words provide the label: “Happily” and expands it into action: “they hugged each other”. As already mentioned, cognitive criticism highlights that emotions are typically embodied, that is, connected to movement, as well as to visual, tactile and olfactory perception. The words “hugged each other”, repeated on the next page: “and hugged each other”, accompanied by the images of the two colour blots successively merging into a single blot of green, evokes the sense of pleasurable physical proximity. (It should be noted that the interpretation of ekphrasis is contextually determined. Detached from the plot, the merging of the blots could possibly be viewed as an act of aggression, one character devouring the other. For anyone who wishes to extract sexual connotation from this scene, the possibility is open). The narrative, however, does not stop on this happy note, but continues, involving further emotional complications. Neither the blue nor the yellow parents recognise their child. Once again, the absence of faces and body language provides no clues to the characters’ emotions, but there are some visual signs that indicate exclusion, for instance, in both cases, the green blot is placed half-way through the edge of the brown surface indicating home, not allowed to enter further. The sense of being rejected by someone we love (if we assume that children love their parents) leads to distress. Even before we turn the page to read: “Little blue and little yellow were very sad”, we have no doubt about how the characters feel. In doing this, we employ projection: relying on our own real-life experience or alternatively our previous experience of narratives. The iconotext now emphasises how we frequently react to distress: “They cried big blue and yellow tears. / They cried and cried until they were all tears.” The images show a clever expansion of a linguistic metaphor. Cognitive criticism claims that metaphors precede everyday language, both evolutionarily and

722

Maria Nikolajeva

within individual cognitive and affective development (Lakoff 1980; Gibbs 1994; Turner 1996). It also claims that metaphors are embodied. Indeed, the metaphoric ekphrasis “they were all tears” is stronger than the plain statement “they were upset”, and it is directly connected to a physical response to the emotion. The image, however, deconstructs the metaphor back to its literal form: the characters literally fall apart into small bits of blue and yellow. Moreover, the metaphoric/ literal play continues on the following page, as the verbal metaphor “they […] pulled themselves together” (normally meaning “making a mental effort”) is depicted visually as the characters physically reassemble themselves from a pile of coloured tears. Their emotions are not expressed directly, but we infer that, since the reason for distress is removed, happiness is restored. In the next page, the child-parent reunion amplifies the achieved goal: “Mama blue and papa blue were very happy to see their little blue”, and as the misunderstanding is sorted out, everybody “hugged each other with joy”, the images showing the inevitable merging into green in the process. This very simple book demonstrates how we ascribe strong and complex emotions even to characters who lack external expressions of interiority, which also enables us to engage with them as if they were human. However, it is primarily possible within a coherent narrative with a temporal and causal structure. Separate images from the book may arguably evoke viewers’ extradiegetic emotions, but hardly suggest employing theory of mind to explore what the characters feel. The verbal narrative supports the visual one, but is more or less symmetrical. As a thought experiment, we could remove the words and most probably still be able to understand both the plot and the emotion ekphrasis.

5 Reading artificial intelligence Shaun Tan’s The Lost Thing is an endlessly more complex iconotext than Little Blue and Little Yellow, both in terms of elaborate imagery, narrative perspective, and word/image interaction. Yet it also offers an interesting example of engaging with a non-human character without explicit external tokens of emotion. The word in the title, “lost”, prompts us to perceive the bizarre title character as distressed, since, based on our real-life experience or previous experience of fiction, we associate the state of being lost with distress. Otherwise, there is nothing in the appearance of the Thing that suggests any emotion. (The animated version of the book amplifies our theory of mind through motion and music). It has no face and no decisive front and back. It has a number of orifices, but they are definitely nonorganic. It has some sort of limbs, the movement of which is hard to decipher. It obviously expresses something through small puffs of smoke, but we are not given any key to its language. The reason we ascribe the Thing any emotions at all is again the strong tendency we have, in real life as well as in fiction, to anthropo-

Emotion ekphrasis: representation of emotions in children’s picturebooks

723

morphise animals and inanimate objects, endowing them with consciousness and thus emotions. Even as adult readers of The Lost Thing we engage with the character with the premise that it is intelligent, since this is a common convention in fiction. In fact, the Thing is a kind of artificial intelligence debated within philosophy and cognitive science, a hybrid of animal and machine; however, the fictional narrative, especially one associated with young audience, makes suspension of disbelief less arduous. There are several ways in which we engage emotionally with the iconotext. Firstly, we try to understand the Thing’s emotions, which is problematic, since there is nothing in the image directly recognisable as an expression of an emotion. The verbal narrative states that the Thing “had a really weird look about it – a sad, lost sort of look”. We infer that the Thing is happy when the narrator plays with it since we know from our own experience that playing is usually joyful. One of the images shows the Thing in an attitude reminiscent of dancing, which may or may not be a sign of happiness (dancing can in certain cultures express aggression). Likewise, it is happy when provided with food, or rather fuel. We infer that the Thing feels unhappy when left alone in a tool shed. We infer that it gets scared of the dark, intimidating interior of the Federal Department of Odds & Ends. There is no doubt that it is happy seeing the paradise-like place of similarly bizarre creatures, partly because the image is bright and has soft edges, as opposed to the dismal scene of the Department with its straight lines; partly because we recognise the emotion of being united with someone like ourselves after being among strangers. The words confirm the inference: “the lost thing made an approving sort of noise”. Thus we can tentatively read the Thing’s mind on the simple scale of happiness – distress. However, we also need to engage with the nameless protagonist/narrator’s emotions: directly, through reading his face and body posture, indirectly, through his comments and behaviour, and implicitly, through our previous knowledge. The visual style of the narrative is not helpful in reading his mind. He is wearing glasses disguising his eyes, and except for a couple of occasions we have no eye contact with him. His mouth is either hidden or absent. His usual body posture is a crooked back, suggesting if not distress then absence of joy. His gestures are sparse. He waves at the Thing to attract its attention and holds up his hands to say goodbye. The verbal narrative itself is also brief. The story is narrated in the first person, which contradicts the visual narrative that shows the character in the images. A visual first-person perspective implies that we share the narrator’s literal point of view and therefore cannot see him (a device referred to in film studies as subjective camera). Yet we also know that the story is told in retrospect, since it “happened a few summers ago”. This initial information, together with the final comments constitute a narrative frame, which separates the narrating agent from the experiencing agent. At the time of narration, the narrator already knows the outcome of the

724

Maria Nikolajeva

story. The visual images are those seen by the temporally external narrator, therefore they are in fact in accordance with the verbal text rather than in contradiction. The story is thus a flashback, a memory. As cognitive science reminds us, memories are fragmentary, imprecise and frequently inaccurate, since they are stored randomly in our brain. The fragmentary nature of memory is emulated in the iconotext, especially in page layout with panels of variable size against a collage background of what may be old schoolbooks. The text is handwritten on small sheets of ruled notebook paper and also fragmented, placed all over the doublespread. Most of the text is a straightforward account of external events: “I […] saw the thing”, “I started talking to it”, “I played with the thing”, “I hid the thing in our back shed”. There are, however, short statements that the narrator makes in retrospect about his own state of mind: “I was intrigued”, “I was baffled”. He is perplexed meeting the Thing, and we share this emotion since it is also extradiegetic, affecting us independently of the protagonist. Surprise is a basic emotion that is evolutionarily wired, giving the brain a warning signal through “low paths” fractions of a second before the higher cerebral areas have time to process the information. Indeed, together with the protagonist, we need to decide quickly whether the Thing is dangerous. The narrator states that the Thing “was friendly” and that it was “great fun” playing with it. Sharing the protagonist’s point of view, we agree that the Thing is benevolent. There is a possibility, not pursued in this particular story, that the protagonist and the reader make different inferences. This is possible because in interpreting the images, supported by the words, we apply highorder mind-reading, that is, infer what the protagonist thinks and feels about the Thing’s thoughts and feelings (as suggested above, we act on the premise that the Things is a conscious being). The narrator’s comment is merely that the Thing “looked lost” and later “I couldn’t help feeling that something wasn’t quite right”. We are encouraged to infer that the protagonist feels sorry for the Thing, which is confirmed several pages later through a direct comment: “I felt kind of sorry for it”. Pity is a complex social emotion that demands understanding that someone else is distressed. If we like someone who is distressed, the normal reaction is to offer consolation or try to solve the problem. The protagonist acts on his emotion, asking other people whether they know anything about the Thing, consults a friend, takes the Thing home, gives it food, “once I found out what it liked”; and finally takes it to a place where it belongs. The resolution of the story involves a number of contradictory emotions. We know that the protagonist knows that the Thing is happy because it has found its paradise. We also know that the protagonist is sad because he has to part from the Thing. We know he is sad because we can infer that he likes the Thing, and parting from someone we like causes sorrow. We also know that he is sad from his face and body posture. However, his empathy with the Thing seems to be stronger than sadness based on his own failed goal (to be happy). Altruism is a

Emotion ekphrasis: representation of emotions in children’s picturebooks

725

type of social behaviour that enables us to give up our own goals for someone else. We know that the Thing is happy because it has found his friends. We cannot make inferences about its emotions on the basis of exterior indicators since it has no face and no explicit body language. We have two options in our theory of mind. Since the Thing is not human we cannot be quite sure what degree of consciousness it possesses and whether it can feel empathy. If it is egocentric, like a very young child or an autistic person, it does not understand that the protagonist is sad. If it, on the contrary, is fully capable of empathy, it knows that the protagonist is sad to part from it, but also happy for it to be happy. In the farewell scene, the Thing possibly displays body language that suggests some kind of emotion. At the end of the story, we are reminded that it is told in retrospect. The narrator’s emotional engagement during narration is flavoured by the knowledge of the positive outcome, but also sorrow about the final parting from the Thing. The sparsity of the verbal narrative in The Lost Thing makes images the foremost channel of emotional communication between the text and the reader. The Thing itself and the rich and grotesque background imagery create defamiliarisation, which is a powerful narrative device inviting empathy. As a mental exercise, we could test whether the verbal narrative alone would encourage any emotional engagement at all. Even if it would, the images expand the verbal narrative to the infinity.

6 So what is new? I have deliberately chosen for discussion picturebooks that have been in the focus of critical attention. Curious George has been analysed from many viewpoints, including postcolonial. My cognitive reading reveals the manipulative interaction of word and image that creates a false sense of the character’s affective state, which confirms and amplifies the postcolonial interpretation. Little Blue and Little Yellow has likewise been scrutinised through ideological lens, as propagating diversity. My cognitive reading demonstrates that there is a strong emotional charge in this superficially simple story which engages the reader affectively and therefore makes the ideological implications more tangible. The Lost Thing and other books by Shaun Tan have been primarily discussed as examples of postmodern narratives. The postmodern philosophy of art claims, among many other things, that a postmodern text deliberately alienates the reader to expose the complexity of the world we live in. The Lost Thing does this effectively through page design. At the same time, it provides a subtle balance between alienation and engagement, which a cognitive approach unveils. For all of these books I would claim that the toolkit of cognitive criticism has opened dimensions so far unexplored. While Curious

726

Maria Nikolajeva

George and Little Blue and Little Yellow have the reputation of being simple, my reading points at complexity hidden behind the words and within the images. In contrast, The Lost Thing has been questioned as too complicated and enigmatic to be enjoyable. My reading explains why readers, as empirical evidence shows, get deeply involved with it. Notably, I have also, quite unintentionally, chosen texts with non-human characters, which implies an additional effort in mind-reading but obviously is particularly gratifying, since it shows that anthropomorphism is an innate skill we employ to make sense of the world. In other words, cognitive approaches to visual and multimedial texts offer a radical re-evaluation of how and why we engage with them, which should reasonably be a central line in future scholarship. Fictional narratives, including picturebooks, are about interpersonal communication, both within and outside the text. Multimedial ekphrasis enables communication where simple verbal description is insufficient. If, as cognitive criticism claims, we read fiction because we want to learn more about ourselves and about other people, picturebooks are an excellent first step toward emotional intelligence.

References Arizpe, Evelyn and Morag Styles. 2003. Children Reading Pictures: Interpreting visual texts, London: Routledge Falmer. Banfield, Ann. 1982. Unspeakable Sentences: Narration and Representation in the Language of Fiction. Boston: Routledge and Kegan Paul. Browne, Anthony. 1987. The Tunnel. London: Julia McRae. Browne, Anthony. 1992. Zoo. London: Julia MacRae. Cohn, Dorrit. 1978. Transparent Minds. Narrative Modes for Presenting Consciousness in Fiction. Princeton: Princeton University Press. Evans, Dylan. 2001. Emotions. A Very Short Introduction. Oxford: Oxford University Press. Gibbs, Raymond W. 1994. The Poetics of Mind. Figurative Thought, Language and Understanding. Cambridge: Cambridge University Press. Hogan, Patrick Colm. 2003. The Mind and Its Stories. Narrative Universals and Human Emotion. Cambridge: Cambridge University Press. Hogan, Patrick Colm. 2011. What Literature Teaches Us about Emotions. Cambridge: Cambridge University Press. Kress, Gunther and Theo van Leeuwen. 1996. Reading Images. The Grammar of Visual Design. London: Routledge. Lakoff, George and Mark Johnson. 1980. Metaphors We Live By. Chicago: University of Chicago Press. Lewis, David. 2001. Reading Comtemporary Picturebooks. Picturing Text. London: Routledge. Lionni, Leo. 1959. Little Blue and Little Yellow. New York: McDowell. Mitchell, William J. Thomas. 1994. Picture Theory. Essays on Verbal and Visual Representation. Chicago: University of Chicago Press. Nikolajeva, Maria. 2012. Reading Other People’s Minds Through Word and Image. Children’s Literature in Education 43(3): 271–291.

Emotion ekphrasis: representation of emotions in children’s picturebooks

727

Nikolajeva, Maria and Carole Scott. 2001. How Picturebooks Work. New York: Garland. Nodelman, Perry. 1988. Words About Pictures. The Narrative Art of Children’s Picture Books. Athens: The University of Georgia Press. Oatley, Keith. 1992. Best Laid Schemes. The Psychology of Emotions. Cambridge: Cambridge University Press. Oatley, Keith. 2011. Such Stuff as Dreams: The Psychology of Fiction. London: Wiley Blackwell. Rey, Hans Augusto. 1941. Curious George. Boston: Houghton Mifflin. Sipe, Lawrence and Sylvia Pantaleo (eds.). 2008. Postmodern Picturebooks: Play, Parody, and Self-referentiality. New York: Routledge. Sendak, Maurice. 1963. Where the Wild Things Are. New York: Harper. Stockwell, Peter. 2002. Cognitive Poetics: An Introduction. London: Routledge. Tan, Shaun. 2000. The Lost Thing. Sydney: Lothian. Tsur, Reuven. 1992. Towards a Theory of Cognitive Poetics. Amsterdam: North Holland. Turner, Mark. 1996. The Literary Mind: The Origins of Thoughs and Language. New York: Oxford University Press. Velthuijs, Max. 1989. Frog In Love. London: Andersen. Velthuijs, Max. 1994. Frog Is Frightened. London: Andersen. Velthuijs, Max. 1996. Frog Is Frog. London: Andersen. Velthuijs, Max. 2003. Frog Is Sad. London: Andersen. Vermeule, Blakey. 2010. Why Do We Care about Literary Characters? Baltimore: The Johns Hopkins University Press. Zunshine, Lisa. 2006. Why We Read Fiction: Theory of Mind and the Novel. Columbus: The Ohio State University Press.

Paul Brighton

34 ‘ The Ocular Proof ?’: Television news and the pursuit of reality Abstract: The chapter explores the decision-making of television journalists in finding the most effective moving images to cover major news stories. It will take as its basis the autobiographical accounts of prominent BBC and ITV television reporters published in the last thirty years. The chapter will examine the accounts given of the balance between a desire to “illustrate reality” and provide telling framings of significance on the one hand, and the demands of deadlines, logistics, and operational/technical difficulties on the other. The chapter looks at how some of the more enduring images of television news were actually achieved, and explores the balance between the crafted television “package” report, and its partial displacement by extended live footage on rolling news channels. It will also explore the debates between practitioners (in particular, between the field reporter and the news staff back in the studio) as to how effects should best be achieved, and the competing elements and pressures in their achievement.

1 Introduction How is “the news” represented on television? How and why are the images we see on our screens chosen to present a visual companion to the spoken words (and occasional written text) that we see and hear? Do the visual representations reflect an approach to an ideal vision of the reporter/producer/editor involved with the story? What competing pressures militate against such “ideal” representations being achieved? Do these competing pressures occur primarily as a result of logistical factors? Are television images also the results of editorial and journalistic decisions as to what should be the visual style of news programmes as a whole, and individual story treatments within them? This chapter will examine and analyse the composition of television news not so much from the perspective of the selection and non-selection of stories and their journalistic “worth” (Galtung and Ruge 1965; Brighton and Foy 2007), but more from the starting-point of how visual representation relates to such decisions. Indeed, it will argue that visual representation is an increasingly influential determinant of such decisions. It will analyse the factors involved, for practitioners, in the construction of the grammar of television news, taking account of the presence of a number of different conventions or tropes of representation. This will involve detailed examination of the structuring not only of individual story treatments, but of bulletins and programmes as a whole. This is necessary because decisions as to the construction and visualisation of individual television news stories may

730

Paul Brighton

not be made solely on the basis of what best suits that particular item of news. It may also be – indeed, almost certainly is – affected by the overall context of the bulletin or programme as a whole. Adopting the practitioner perspective, we shall argue that the aesthetics of television news have been, and still are, gaining in importance in the way terrestrial news is presented, and that this also has consequences for continuous or rolling news, as well as for terrestrial news. We shall argue that new trends towards the literalisation of visual communication are occurring, and we shall seek to explore the assumptions that are made about viewer reception of, and response to, such images. We shall also explore how individual field reporters adopt different approaches to filming stories, and the effects and consequences of this from the viewer’s perspective. This is more than simply a debate about, or attempted critique of, the importance of “framing” images, so important in responses to “still” photography, and sometimes neglected in television news analysis. It goes to the heart of the debate about the difference between imaginative or creative visual arts and many other visual media. This, in turn, raises questions about authenticity, trust, and what viewers expect from television news footage.

2 Television news and visual art When one looks at, for example, a Poussin or a Constable painting, there is an implicit understanding on the part of most observers that they expect something in addition to literal verisimilitude. Indeed, in looking at modernist paintings, literal verisimilitude is not expected at all, and indeed may have no relevance whatsoever. We are consciously investing in the creative power of the artist to give us a transformed and transformative reality. One is almost saying: “If all I wanted was a literal rendition, of the scene, I would have bought a photograph of it: or, perhaps, taken my own photograph”. We expect the exercise of transformative creativity to be part of the experience of painting or sculpture. We do not, for instance, mind that the painting may be the result of many different days’ observation, and that some elements may be entirely imaginary or even symbolic. In older paintings, we understand that rules of perspective will be applied differently from the way our own eye would record, process and take in the scene. Of course, in earlier periods of art, before the Renaissance, the self-conscious aesthetic of the appreciation of creative beauty and artistic skill was not the primary or ostensible purpose of art or visual representation at all: art was, at least officially, a route towards, or conduit for, piety and worship than a consciously sensuous experience. The concern was not with whether the Virgin Mary actually looked like the face in the painting or the sculpture. Presumably, at some level of consciousness, fourteenth and fifteenth century observers might have sensed that the historical Mary did not dress in the fashion of their own century: but it was

The Ocular Proof?’: Television news and the pursuit of reality

731

not a problem if the work of art allowed thoughts of the Christian faith to be appropriately channelled. Even with still photography, we consciously or subconsciously cede to the photographer the right to apply his or her own rules of framing, of inclusion and exclusion, in order that the image should be especially powerful or pleasing.

3 True or false? Television news pictures, however, are subject to different expectations: we might almost say different imperatives. It is precisely authenticity and representational reality, which we demand, like Othello demanding the “ocular proof” of his wife Desdemona’s infidelity. Creative licence in rendering reality may destroy faith in the product; it certainly does not enhance it. Even in a documentary, the knowledge that First World War combat footage was actually filmed in the UK using film of military training is unlikely to enhance trust in the medium, even if it does not unilaterally eliminate it. The history of the newsreel industry abounds with stories of faked footage: The French cinematographer Georges Melies faked the entire coronation of Edward VII for the Warwick Trading Company in Birmingham using actors and cardboard props, in a Warwickshire barn. Most of the footage of the San Francisco earthquake and fire was faked in a warehouse, where the studio staff set fire to a scale model; a photograph exists of a smiling figure standing over the burned-out model, holding a watering can. Yet the mayor of San Francisco, who watched the resulting film, was convinced it was true (Simpson 2002: 63).

Simpson (2002) argues that this heritage has led to a greater degree of viewer scepticism towards television news images than towards newspapers (and, by implication, radio). The idea that “you can fake the pictures, can’t you?” has left a need for transparency to convince not merely conspiracy theorists who believe we never went to the moon, but media-literate viewers more broadly. The cutaway shot, where we are supposed to imagine two cameras – one pointing at the interviewee, the other at the interviewer, who is periodically pictured nodding thoughtfully at points where the interviewee’s responses need editing for brevity or sense – is as much a deception as Edward VII’s “coronation” of 1902. An alternative involves visible “jumps” in the film, of the sort (often parodied) in 1930s newsreel films. Less intrusive versions of this are in fact now sometimes used. Another is a very brief fade to black or what is known as a ”dissolve” edit. And, of course, television news is now much more assiduous in flagging up its use of “library pictures” – older footage not shot as part of current coverage of the story. Nicholas Jones recalls a case from the miners’ strike of 1984–5 in which complaints followed an inept “cutaway”, in which the same speaker was shown with his jacket on and then with his jacket off only seconds later. (Jones 2012). Simpson (2002) himself argued that visible cuts or “dissolves” should be used,

732

Paul Brighton

and the cutaway shot dispensed with; although he acknowledges that there is some viewer resistance to excessive labelling of “reconstructions”, library footage, and obvious and visible edits. The latter, he claims, are acceptable in documentaries and current affairs, but still rather less so in television news images: [W]hat is needed is an acceptance by the broadcasters that any kinds of falsity, even the harmless, purely cosmetic type, are best avoided. Perhaps, too, we need an acceptance by the audience that although they are seeing something that is different and perhaps more awkward than they are used to on the news, it does tell them frankly that something has been done to the words they are hearing. That way, perhaps, we can ditch one of the last vestiges of the undeniably dodgy past of television news (Simpson 2002: 72).

4 Real quality? Simpson (2002) also explains how ineffective visual communication of a news story can undermine its news value and journalistic potency. He describes his own 1979 report on the return of Ayatollah Khomeini to Iran. (It makes an interesting contrast, incidentally, with Gavin Hewitt’s footage of Pope Paul VI’s final journey, which we will examine later): It consisted of the following. No proper beginning, just a few dull shots captured during the flight. A glimpse of Khomeini’s profile by a porthole, and a grumpy word or two from him, entirely unintelligible. More shots of someone else looking out of a window. Then, suddenly, we were on the ground and I appeared, wearing a sports jacket that should never have been seen on television. I talked to camera for an absurdly long time […] Finally the old gentleman made it down to the ground and bowed to someone, and that was it […] I hadn’t seen the pictures when I wrote the script, so I couldn’t use words to underline whatever qualities they possessed. Altogether, it was a screw-up. Worse, it actually detracted from the superb pictures of Khomeini driving in triumph through the streets of Teheran (Simpson 2002: 98).

Another vastly experienced television journalist complained about the distortions inherent in visual communication via the television screen. Michael Nicholson was particularly despairing of how television news had coped with the relative absence of “real” footage from the Falklands: [T]here were strip cartoons on all channels showing air attacks, land assaults and sinking ships, lantern-jawed marines, slim moustachioed paratroopers, the bugler playing an heroic Last Post over San Carlos. It was “Boy’s Own” without the POWs! and WHAMs! But it was not war. Not as I knew it (Nicholson 1992: xiii).

At one level, this is a clear departure from the norms of television communication. Insofar as the journalists involved had rationalised the issue, these devices were not intended to be direct and literal representations or even mediations of reality. They were conscious surrogates, in a sense different even from the contortions of reporters illustrating their stories by means of devices such as ascents and

The Ocular Proof?’: Television news and the pursuit of reality

733

descents in lifts. They were a stark, if tacit, admission of the media’s inability to circumvent the particular modes of censorship imposed by the government and the military in the Falklands conflict. Perhaps more worrying, however, is Nicholson’s verdict on the televisual representation of real violence and injury. Describing a scene of violence from the Vietnam War, Nicholson recalls: [O]n the coloured screen, blood is a very sobering red and television will not transmit such horrors. That is its Catch 22. Fascinated by it themselves, programme editors filter out what they consider viewers should not see. They contrive a surrogate war, where much of the suffering is deleted. They help to sanitise war; they almost make it acceptable (Nicholson 1992: xiii).

5 Keep it real? Can we compare the experience of viewing events without the prism of a television screen with the more obviously mediated version it gives us? Marina Warner, then of the “Spectator”, recalls seeing civilian casualties in Vietnam for the first time: I saw this old woman coming down the road with a child in her arms. The child’s flesh was falling off. I said to myself, ‘My God, I’ve seen this all before.’ I had. On television. Somehow seeing it before on television took away some of the reality and I wasn’t as shocked as I expected to be (Warner, cited in Knightley 2003: 407).

Rather than emphasising the difference between the television experience of a visual image and its physical reality, Warner regarded television’s mediation as a desensitising agent, for which, she tells us, she felt a guilt which required expiation in the form of exposing herself to unnecessary risk in the combat zone: to “see” in a non-televisual and unmediated way what television had rendered too anodyne and too unthreatening (paradoxically) for comfort.

6 The grammar of television news So, what elements combine to produce the images we actually view as we consume television news? Among them surely are: decisions of programme producers and editors as to which stories are to be covered and included; decisions as to the form(s) of the treatment of the stories; the elements constituting the coverage of the story (interview/clip, actuality, piece to camera, videowall, reporter-presenter two-way, etc.); the approach to the filming of the story taken by the individual reporter; and the subsequent changes to story length and format taken by production staff between successive bulletins, programmes, or news cycles (Brighton and Foy 2007; Machin and Niblock 2006). There is sometimes a tension between the original intention as to how a story will be treated or presented when the decision is initially taken to cover it, and

734

Paul Brighton

the form in which it is eventually presented in broadcast form. This may be because other stories, deemed more important, break between the moment of decision and eventual transmission. Other incidental factors may also affect the final outcome: technical issues, last-minute cancellations of interviewees, etc. (although these are less likely to affect a predetermined running order than might be imagined), and other unforeseen factors. Another important variable is the impact of what came to be called “citizen journalism”. While professional television (or multimedia) reporters may well have individual views on or approaches towards how stories should be filmed and framed, such considerations are less likely to influence non-professionals who witness extraordinary events and record them on mobile phones. And yet news values frequently dictate the inclusion of such elements in otherwise carefully crafted packages or illustrated two-ways. One of the striking features of coverage of the 2011 Libyan revolution, and the overthrow of the Gaddafi regime, was how some television news broadcasters – particularly Alex Crawford on Sky News – were keen to use mobile phone footage, even (perhaps especially) when shot by their own staff, to drive coverage of the key moments of the conflict. One thinks of the occupation of Tripoli’s central square in late August, 2011, when the phone footage was left to run live on Sky News for many minutes at a time, with little or no commentary or other form of “professionalised” mediation. Mobile phone footage of the capture of Gaddafi, and the subsequent display of his corpse after he was killed, were broadcast in October, 2011; but this was not live: the footage emerged almost in instalments (as happened with the execution of Saddam Hussein at the end of 2006), and was therefore able to be incorporated in more conventional formats such as packages and two-ways on all the main UK news channels and programmes (Sky News: August and October 2011).

7 Artful or artless? There is thus inevitably a tension not only between the “ideal”, aesthetically satisfying, perfectly shot and framed television news package, and the raw actuality which sometimes has to be preferred in the race to be first with the news. Indeed, there is a sense in which the use of raw, poor-quality visual material, often from viewers’ own mobile phones, is itself used as a signifier of immediacy and of exclusivity in newsgathering: it conveys the sense of a scoop; and it also serves as part of the increasingly important subtext sent out by news and other broadcast output: that the programme is “yours”: that it is part of (at least the illusion of) interaction between broadcaster and consumer. The very lack of professionalism of the visual material can, on occasion, serve to reinforce that implied message. It is important, of course, to distinguish between the demands of rolling news and the criteria involved in story and treatment selection for terrestrial television

The Ocular Proof?’: Television news and the pursuit of reality

735

news programmes. One of the contentions of this chapter is that the word “programme” is increasingly applicable in relation to terrestrial television news. Several factors, but above all the demands of ratings, ensure that terrestrial output has to be constructed with a view not merely to conventional journalistic notions of news values but also to the need to ensure broad conformity with the surrounding programming and the “feel” or ethos of the channel itself.

8 Bulletin or programme? The move from “bulletin” to “programme” has been a growing trend within terrestrial television news in the UK in recent years. Indeed, as the UK completed the switchover from analogue to digital television reception, it may increasingly be asked whether there is a long-term future for terrestrial television news at all. It may also fairly be inquired whether the effort involved in preserving a space for terrestrial television news is also distorting (or at least influencing) previously fairly generally accepted news values. (Brighton 2011) The argument, briefly, runs thus. Commercial pressure on ITV and Channels Four and Five entails the need for the maximum possible viewing audience: especially on ITV. Political pressure to justify the licence fee settlement obliges the BBC to maximise its ratings on the flagship channel, BBC One. These commercial and political pressures require that terrestrial television news bulletins should not, if possible, be significant audience losers. This, in turn, involves a major effort to ensure that the style and feel of the news programmes should be as similar to, or as little unlike, the surrounding programming as possible. These trends have clear implications both for the way news is filmed and shot, and for the decisions made as to how to present and showcase the material once acquired and edited. In addition, it affects the yet more fundamental question of which stories are included and which excluded – the news values of the programme. Assumptions as to the likely audience and its tastes, interests and predilections, guided by extensive use of focus group research, contribute strongly to these decisions. If such groups indicate a preference for “human interest”, crime, sport and celebrity stories over political and economic ones, this will inevitably be reflected in running orders. Where economic news is so important that it cannot reasonably be excluded, as in the years since 2007, the coverage will be addressed, as far as possible, in “individualised” terms. How will these developments affect you? Here are some people who (we contend) are quite a bit like you: this is what they think and feel. Even more noticeably, television news struggles to visualise economic data. This is why the trend of literalising economic concepts has been so evident in the years since 2007/2008.

736

Paul Brighton

9 Show me what you mean Let us look at some examples of these trends. An increasingly familiar sight on terrestrial news programmes (often replicated on rolling news) has been the employment of devices such as a reporter travelling upwards in a lift to illustrate rising indicators (higher unemployment, inflation, fiscal debt and so on); and a corresponding descent to showcase falling ones. BBC Economic journalist Hugh Pym was filmed thus in late 2011. Former BBC Political Correspondent Nicholas Jones sees it as a recent example (one of many) of “more time being spent thinking where to film and activate the piece to camera than concentrating on what to say.” It is, says Jones, “very dispiriting to be criticised for not being able to think in pictures.” (Jones 2012). When there is even more problematic material to be communicated – percentages, amounts in millions or billions and the like – recourse is had to increasingly imaginative (or bizarre) uses of videowalls: surfaces against which data can be visually represented. These might be as varied as: a dried-up river bed (in a story on global warming); the front or side of an urban building such as a boarded-up shop front in an item on recession and economic decay; and even the back of a white van parked on the street from which a piece to camera is recorded (perhaps simply because it is there, or for reasons of story signification which are not always apparent to viewers!). We can examine these issues at individual story level – how individual reporters approach the task of filming stories, the overall approach to visualisation, plus aesthetic and framing considerations. At the same time, we are mindful that individual news packages are sometimes compiled out of footage shot by more than one person. A local reporter may conduct an interview far away from the main reporter: although we may not hear or see them, the local reporter and camera person (if it is not a solo effort) may film in a slightly different style. One can also explore the issues further at bulletin or programme level – decisions as to formats and treatments, as well as running order and programme balance considerations. This does not, of course, refer to political or editorial balance, but to the desire for internal aesthetic balance within the piece of output. It involves, for instance, the desire to avoid too many examples (and especially back-to-back examples) of the same “genre”: not too many presenter-reporter two-ways, or videowalls in studio or as part of a package, and other format duplications. These considerations are somewhat less important for rolling news, which is subject to different imperatives and contrasting assumptions about viewer behaviour. Few apart from professional journalists are likely to sit in conscious or even unconscious judgement on the internal programming balance of sections of rolling news; and, although it is clearly divided into hourly, and to some extent halfhourly and even quarter-hourly sections, many viewers are unlikely to watch it regularly in symmetrical, undisturbed segments. For terrestrial news, however, these elements are now regarded as increasingly important.

The Ocular Proof?’: Television news and the pursuit of reality

737

Efforts are made to use visual material as well as verbal/audio narrative to induce in viewers a sense, if not of continuity (almost a contradiction in terms for a news programme), then at least of a programme “feel” or identity. ITN, for example, will precede its opening recital of forthcoming stories with a gallery of photographs of the day’s “newsmakers”: the very use of which, incidentally, reinforces the increasing personalisation of stories. A picture of, let us say, Christine Lagarde of the IMF personifying a forthcoming story on global financial developments; even, sometimes, a picture of an anonymous member of the public, minus his or her head, embodying obesity or social deprivation. Stories are trailed ahead; and visual as well as verbal “teasers” used to illustrate the “and finally” story: now, more than ever, an attempt to hold the audience until the end of the programme as well as to leave the viewer with a sense of amusement or well-being. Also important is the tension between the priorities of the individual story reporter and the programme production and editorial team. There are accounts of the public accounts of tensions and editorial disputes between war reporters in the field and newsroom teams back home as to how much of the grim realities of war, terrorism, famine and other forms of death and destruction can realistically be expected to be broadcast on viewers’ screens (Brighton 2013). Here, however, we can broaden it into a more general discussion of the process of resolving debates between reporter and production staff as to what form of visual representation best suits the story – and, crucially, its wider place within the programme.

10 Form and substance An inevitable part of the play of news values in programme construction is the possibility of a story being broadcast in a form different from that initially envisaged. In the event of a major breaking news story, there is likely to be little dissent: a prior story will be reformated, maybe reduced to a mere News In Brief (or NIB) item; or dropped altogether. Just occasionally, a more timeless or less sharply “pegged” story could even he held back for another day. In more nuanced cases, it is more probable that running orders for terrestrial news programmes agreed at earlier news planning meetings will be adhered to. Terrestrial television news, more sharply conceived as a programme in its own right, will sometimes omit or postpone late-breaking news or new lines which may render some of the alreadypackaged material slightly out of date. Such additional information will probably be running on rolling news channels (except where, for instance, BBC News also carries the main terrestrial news programme; and, even then, it may well be running as part of the text “banner” at the bottom of the screen). Reporters are aware that what was intended as a full package could end up in another format – as part of an illustrated two-way, or even as a News In Brief. The extent of negotiation between production/editorial team and reporter as to the

738

Paul Brighton

outcome of such cases may well depend on a range of factors: the seniority of the reporter or correspondent, his or her physical location, and the urgency of the decision. This is an aspect of news presentation that is relatively rarely the subject of public discussion, though the recent proliferation of journalistic autobiographies and memoirs has begun to shed a little more light on some instances. The increase of personality-led reporting may also play a part in this. Some experienced broadcasters argue that the choice of visual material relates to the nature of the journalist’s profile. Former BBC Political Correspondent Nicholas Jones says that his area is increasingly personality-led: The best location for a piece to camera, – e.g. at a party conference – would be kept for the political editor. Producers are determined to make the most of their star […] and would go to great trouble to get the best possible background. Currently [with international stories] John Simpson would get the most graphic setting and lesser correspondents [would] be relegated to more mundane settings (Jones 2012).

Many of the examples mentioned in the memoir accounts by senior journalists refer to the period before the start of rolling news. The ability to broadcast multiple treatments of the same story in a fairly short space of time was not available, for example, on BBC television news until the start of BBC News 24 in 1997. (Just occasionally, there would be multiple treatments of the same story on national and then on regional television news; but, more often than not, with different reporters, albeit often shared footage). However, although there is so much more broadcast news space available now, it is still possible that, on a day of a major breaking story, a lesser story will be dropped altogether, since the emphasis then shifts from imaginative visual presentation to the attempt to find as many live interviewees as possible, without regard to the aesthetic, visual considerations that would normally be much more important.

11 Library footage Less often noted in critical accounts of television news are the elements that are not part of that day’s filming. For example, stock library footage; or more personal images. The home video clips of Millie Dowler and Madeleine McCann, and a still photograph of Stephen Lawrence, have been repeated countless times over a number of years. Others are less poignant, but part of the same phenomenon: film of shoppers (often in reports on the economy or pre-Christmas retail items), anonymised footage of people in the street to illustrate items on obesity or binge drinking and of motorway traffic in items on transport issues are all frequent devices on television news. Slightly different, but also ubiquitous, are visual images deployed at times where there is no actual footage of what is being described. Again, there are many examples. Old film was repeatedly used, often slowed down and sometimes visually altered, of missing dictators (again, one thinks of Saddam Hussein

The Ocular Proof?’: Television news and the pursuit of reality

739

between April and December 2003, and Gaddafi from August to October 2011) or terrorists (Osama bin Laden from 2001 to 2011). The same occurred in late 2012 during the so-called “Gategate” story involving Andrew Mitchell, his bicycle, and the Downing Street gates. There are occasions when an old still photograph is all that is available: Mandela before his release in February, 1990; the murderers Brady and Hindley for several decades after the 1960s. The repeated use of stock footage of Saddam, Gaddafi and bin Laden, by contrast, was deployed either because those shots were seen as somehow epitomising the individuals involved; or, more simply and pragmatically, because it was logistically easier to continue using the same piece of footage, even at the risk of boring or irritating regular viewers: especially viewers loyal to the same channel or programme. The balance between deciding on what visual material should be sought and used “before” the story is covered and “after” it has been filmed shifts all the time. Former BBC cameraman and director Paul Williams says: If you can’t get the story together, then visuals [or graphics] are fine. Also, sometimes they do actually make the content clearer as to the point of the piece. So you would then work out how you can make ‘what you’ve got in the can’ work with the graphics. With overseas reporting they become a lifeline to clarity. […] Few people understand the geography […] Words or pictures can’t do this. Good graphics (at the head of a story) can ease the viewer into the context and make the content more relevant (Williams 2012).

So how do individual reporters and camera staff view the logistical and aesthetic challenges they faced in attempting to make television news visually arresting while, at the same time, providing a suitable partner for the verbal and other audio-based components of the story? How much is filming undertaken with an awareness of how viewers receive visual images with widely differing levels of engagement? And how much is research into the relative amounts of viewer concentration focused on visual and non-visual elements of television news taken into account both in programme and in individual story construction?

12 This is how I did it One useful source of information in this regard, as we have noted, is the series of published accounts of careers by experienced and prominent journalists. There has been a plethora of such accounts in the last two decades or so – perhaps a reflection of the changes in the publishing industry and the rise of the celebrity culture and the resulting “celebrity” autobiographies; along with the increasing celebritisation of some journalists. Because newsreaders in particular, but also eye-catching war correspondents, now come firmly within this category, books have followed in considerable number. Because such books are aimed at the general reader, much of the focus, entirely understandably, is on “adventure stories”: of difficulties encountered in

740

Paul Brighton

harsh, inhospitable landscapes, and logistical nightmares in returning material to base after filming. Some accounts, however, do shed light – sometimes almost incidentally – on the thinking behind the construction of particular images and pieces of film. BBC journalist Gavin Hewitt (2005), in his autobiography A Soul on Ice, writes of the way he tried, as a young reporter, to differentiate his images of the events surrounding the death and funeral of Pope Paul VI in August 1978. He describes how he was able to film the journey of Pope Paul’s body from Castelgandolfo, where he died, back to the Vatican, for his funeral. He and his crew managed to drive alongside the cortege for part of the journey – despite police attempts to block their way. The result: “A wonderful tracking shot.” (Hewitt 2005: 23). Later, having run out of film, and having stopped to replace the magazine and lost the distinctive vantage point, the crew filmed some more footage: but, for Hewitt, it was the filming alongside the cortege and the hearse which made the piece special. “That evening, we edited the piece and, of course, included one brief shot that no one else had.” (Hewitt 2005: 23–24). Hewitt frames the story within the context of his own self-questioning about how far one should go in pursuit of a story and a distinctive image. He recalls praise from the late Cardinal Hume for the sensitivity of the coverage of the deaths of Pope Paul and of Pope John Paul I (who died in late September in the same year). “It crossed my mind what he would have said or written if he had seen that Fiat alongside the Pope’s hearse.” (Hewitt 2005: 24).

13 Artists and technicians Many of the journalistic autobiographies stress the importance of cameramen (almost invariably male, especially in accounts of the 1960s and 70s, and predominantly so afterwards). Of course, with more recent changes to broadcast technologies, and the style and quantity of output, the dynamic has changed somewhat. There have been phases of greater reliance on agency pictures, and sometimes on self-filmed footage. Earlier on, however, says Hewitt, “[…] the cameraman has to understand the story almost as well as the reporter to know the action, the closeup, the cut-away that captures the essence of it.” (Hewitt 2005: 118). When former BBC reporter and newsreader Michael Buerk started filming the reports in Ethiopia in October 1984 which not only increased his own renown but also led to Band Aid and Live Aid, he was working with the late Mo Amin, already a cameraman highly esteemed by his peers. Crucially, as Buerk notes, Amin was a “superb” stills photographer as well as a television cameraman. Intriguingly, it was probably Amin’s stills photographer’s instinct which led him to capture what became the most iconic and abiding image of Buerk’s Ethiopia reports: the starving Tigrayan boy with striking eyes, whose face, interwoven with the live music, was for millions the most enduring and moving aspect of the Live Aid event in 1985:

The Ocular Proof?’: Television news and the pursuit of reality

741

I looked around for Mo to point the little boy out to him. I needn’t have bothered. He already had him locked into close-up. He was a superb stills photographer and his work that morning would make him the most famous television news cameraman on earth (Buerk 2004: 283).

Buerk expresses much of the self-doubt about the pursuit of such images that Hewitt touches on. He speaks of how they “watched people dying, and filmed them, until we could take no more.” (Buerk 2004: 283). Again, slightly later, Buerk touches, without further comment, on the paradox that still images are often more potent and more poignant than moving ones: A mother and her two-month-old baby were wrapped together in a grey blanket, still on the poles used to carry them from where they had died together in the night. Two pairs of dusty bare feet, adult and baby, stuck out of the end of the bundle. I don’t think anybody could see those feet and not cry (Buerk 2004: 283).

When it came to getting the film reports to air, there was the dilemma of choosing between the potential hazards of the satellite feed from Nairobi and flying back to London – with the added dimension of the fear of being scooped by ITN. The decision to make the flight ensured that this most visual of all stories could be edited with extra care – and, more than ever, the pictures came first. No sign here of the “text first, pictures to fit” ethos that later came, perhaps rather simplistically, to be associated with John Birt’s journalistic regime. Mac [Mclean, tape editor] and I ran and reran the Korem tapes, putting the story together entirely in pictures: I would find the words later. It was obvious where to start, on the tracking shot I had told Mo I would never use. It set the scene like no other sequence could […] We worked through the sequences, trying to show the scale of it all and then paint a picture of horror with all those faces etched deep by loss and despair. We left the bringing out of the dead to near the end. I wanted people to think it could not get any worse and then hit them with that terrible sequence that showed so clearly the daily cost was in human lives and uncontrollable grief (Buerk 2004: 292).

Buerk’s account is worth quoting not least for its rare glimpse into the thought processes involved in the deployment of images, as well as for the precision about the respective timing of the visual and the verbal elements. In addition, it reminds us of the retrospective influence of subsequent visual and editorial considerations in assessing the worth of a shot or a set of images. The tracking shot, previously rejected, becomes the image which, says Buerk, gave the report the quality of a tragic biblical tableau. “It looked exactly like one of the colour plates in the Bible I had at school”, says Buerk. “This is why it had such strange resonances for me and so many others from my generation and my background.” (Buerk 2004: 292) In more general terms, this is confirmed by Paul Williams: [A]ctually perceiving the shots is an art and I think that like many parts of media [work] it is (if you can do it well) a gift. You are part artist and part technician and the accountants love you because you save them money … [W]hile the reporter/presenter is getting the ‘core’ facts

742

Paul Brighton

they [the cameraman/recordist/operator] are usually already putting the material together. Cut-aways, establishers, atmos – they can usually put 50 % – 60 % of the ‘pictures’ away before the main shooting begins (Williams 2012).

14 That’s the shot In the introduction to his history of the Frontline news agency, David Loyn enumerates some of the most memorable images gathered by the Frontline team: Remember the night shots of rockets in the first Gulf War, the first massacre victims in Kosovo, life under Serbian siege in Bosnia, the US bomb that was misdirected onto a bunker killing women and children in Baghdad, Mullah Omar wrapping himself in the cloak of the Prophet Mohammed and declaring holy war for the Taliban – the only time he was ever filmed? They were all Frontline pictures. Then there was the uncertain expression on the face of Ceausescu when he realised the game was up in Romania (Loyn 2006: xiv).

The question of which visual images we recall is, of course, a complex one. It may depend on a number of personal factors (strength of memory, interest in the story) as well as more obvious issues such as the number of times an image has been broadcast. The likelihood of images of war and terror being widely remembered relies on different criteria. It may be the skill of editing, or a particularly evocative unification of sound and vision. Loyn writes later of the Baghdad bombing: It was a horrific scene, with details that could not be reported on television […] The rescuers carried charred bodies and body parts out for three days, praying all the time, a mesmerising soundtrack to Rory [Peck]’s pictures (Loyn 2006: 139).

15 That could be me There is also the question of whether visual images are in conformity with viewer expectations. Indeed, there is something of a paradox. The news value of deaths and injuries will obviously depend to some extent on factors such as the identities, backgrounds and ethnicities of the victims. Several reporter accounts confirm that, other things being equal, terrible incidents that happen to people who do not “look like us” are generally deemed less shocking (and less newsworthy) than those that happen to people we identify with more readily. On the other hand, Kate Adie claims that we have certain expectations of the visual material relating to scenes of death and injury in conflict. She describes the Lasva valley in Bosnia – a scene of relative middle class affluence: large new houses, satellite dishes (in the early and mid-1990s), and little trace of the Ottoman heritage. She quotes a soldier asking his commanding officer: “How come we’re saving the lives of a bunch of people who live in bloody mansions, sir?” She continues:

The Ocular Proof?’: Television news and the pursuit of reality

743

so many of the pictures which subsequently came to symbolise Bosnia were of rural village people, many of them elderly, the women in headscarves and their baggy divided skirts, the men gnarled and toothless under their dome-like hats. The impression given was of mediaeval folk cut off from western [sic] Europe […] The cameras loved these scenes: the depressed peasants herded along country roads behind clip-clopping skinny beasts. For the received (but not much acknowledged) TV wisdom is that it is not quite so embarrassing to journalist or viewer when the suffering group looks a bit different from us. The camera can zoom in and look at the miserable faces under thick headscarves, in funny flowered trouser-skirts, they’re not the people you’re going to meet in Marks and Spencer. There’s no doubt that the less the resemblance to the life we know, the easier it is to intrude and inspect (Adie 2002: 256–7).

Other factors also come into play. Filming overweight British people for a report on obesity may, as mentioned, deliberately omit their heads partly for legal reasons, and partly out of a genuine concern that they should not become targets for bullying and abuse. But it is also, at least in part, because it may also prove offensive to some viewers who believe in personal privacy (and perhaps even provide a sense of “there, but for the grace of God, go I”). Adie (2002) is right that these considerations would apply less to footage shot in Bosnia, Iraq or Afghanistan than in, say, London, Wolverhampton or Glasgow.

16 Illusion or reality? Some reporters have also written of the effects of the television screen on mediating reality. Although writing in an age when just about everyone watched on an actual television (as opposed to a computer screen, mobile telephone or other digital platform), Martin Bell’s account of the effects of television framing are, if anything, even more relevant now as they were in 1995: Television by its very nature has an aptitude for illusion anyway. Reality is somehow diminished by being framed in the modest rectangle in the corner of the living room. Images from the front line especially lose something in the transfer, and take on the quality almost of travelogue, as if in a scene from one of P. J. O’Rourke’s ‘Holidays from Hell’ […] Having viewed [my field cassettes], I would ask myself, is that really all there was? Was that it? Something was missing, something that will always be missing in the compressed world of the tube: the sense of the surrounding reality, the sharper perceptions of the eye as against the camera, the sights and sounds and smells of actually being there. For this, there is no video substitute, not even the ‘virtual reality’ of TV news (Bell 1995: 215).

Thoughtful journalists continue to worry that an increasing concentration on refining visual techniques simply as a way of staving off inattention, or the impulse to press the remote and switch channel or turn off, operates as a potential bias against understanding. Nicholas Jones says: “The danger is that visuals become a distraction and [are] disconnected from the storyline.” (Jones 2012). The extent to which that continues to become an issue will depend, in turn, on the survival of

744

Paul Brighton

terrestrial television news as a distinct genre of programming, and on the very future of television news itself.

References Adie, Kate. 2002. The Kindness of Strangers. London: Headline. Bell, Martin. 1995. In Harm’s Way. London: Hamish Hamilton. Brighton, Paul. 2013. A Bad Taste Business? In Feona Attwood, and Vincent Campbell (eds.), Controversial Images, 269–285. Basingstoke: Palgrave. Brighton, Paul. 2011. The Future of Terrestrial Television News. Conference paper. What is Newsworthy? Reexamining News Criteria Conference, 23–26 October 2011. Haifa, Israel. Brighton, Paul and Dennis Foy. 2007. News Values. London: Sage. Buerk, Michael. 2004. The Road Taken. London: Hutchinson. Hewitt, Gavin. 2005. A Soul on Ice. London: Macmillan. Knightley, Philip. 2003. The First Casualty. London: Deutsch. Loyn, David. 2006. Frontline. London: Penguin. Machin, David and Sarah Niblock. 2006. News Production. London: Routledge. Nicholson, Michael. 1992. A Measure of Danger. London: Fontana. Simpson, John. 2002. News from No Man’s Land. London: Macmillan.

Interviews Jones, Nicholas. 2012. Several interviews in May. Williams, Paul. 2012. Interview 3 June. Television news output examples gathered from BBC, ITV and Sky News broadcasts, August to November, 2011.

Biographical sketches Danielle Almeida: Dr. Danielle Barbosa Lins de Almeida holds a PhD from the Federal University of Santa Catarina (UFSC), with training programme on Visual Communication at the University of New South Wales (UNSW) in Sydney, Australia. She currently works as a Professor in the Department of Modern Foreign Languages (DLEM) and in the Post-Graduate Course in Linguistics (PROLING) at the Federal University of Paraíba (UFPB), Brazil. Her academic interests are focused on Discourse Analysis, Visual Grammar, Multimodality, Systemic Functional Linguistics, Cultural and Media Studies. Her publications include the book Perspectives on Visual Analysis: From Photojournalism to Blog (2008), as well as scientific articles in national and international journals. She also coordinates the Visual Semiotics and Multimodality Research Group (GPSM). John Bateman is professor of Applied Linguistics at Bremen University where he has been researching issues of multimodality in several media for several years. He has published widely in the area, with recent books focusing on annotation methods for static page-based documents and the application of functional semiotics to the audiovisual moving image. He is currently head of the Bremen Institute for transmedialTextuality Research (BItT). Address for correspondence: Faculty of Linguistics and Literary Sciences, Bremen University, Bremen 28334, Germany Nathaniel Dafydd Beard is a graduate of the London College of Fashion and ArtEZHogeschoolvoor de kunsten Arnhem; and currently a PhD Candidate (Fashion Womenswear), Department of Fashion and Textiles, School of Material, Royal College of Art. His work has been published in Fashion Theory: Journal of Dress, Body and Culture, Address – Journal of Fashion Writingand Criticism, and Sexymachinery and presented in papers at the universities of Brighton, Helsinki, Oxford, and Sheffield, Università Cattolica del Sacro Cuore, Milan, London College of Fashion, and Institut Français de la Mode (IFM), Paris. In September 2012 he convened Fashioning the City: Exploring Fashion Cultures, Structures, and Systems, the RCA’s first fullscale international, inter-disciplinary conference on fashion. www.fashioningthe city.wordpress.com. Gwen Bouvier is a Lecturer of Communication, Cultural and Media Studies at the University of Glamorgan, Cardiff, UK. She has taught courses across journalism, international media, national identity, and representation theory. Her research has focused on 9/11 and representation, discourse analysis, and social media and identity. Her publications include: ‘How Facebook users select identity categories for self presentation’ and ‘Breaking News: the First Hours of the BBC Coverage of 9/11 as a Media Event’. She is presently carrying out research into news representations of current military conflicts.

746

Biographical sketches

Paul Bowman teaches media and cultural studies at Cardiff University, UK. He is author of Post-Marxism versus Cultural Studies (2007), Deconstructing Popular Culture (2008), Theorizing Bruce Lee (2010), Studiculturali e cultura pop (2011), Culture and the Media (2012), Beyond Bruce Lee (2013) and Reading Rey Chow (2013), as well as the official biography of Bruce Lee, The Treasures of Bruce Lee (2013). He has edited many books including Interrogating Cultural Studies (2003), The Truth of Zizek (2006), The Rey Chow Reader (2010), Reading Rancière (2011) and Rancière and Film (2013), as well as issues of such journals as Parallax, Social Semiotics, Postcolonial Studies and Educational Philosophy and Theory. He is the founding editor of JOMEC Journal, and is currently researching a book on the relations between media representations and martial arts practices. Paul Brighton grew up in Wolverhampton. He was educated at Wolverhampton Grammar School, from where he won an Open Scholarship to Trinity Hall, Cambridge. After a First in English, and research for a Ph.D., Paul worked in broadcasting for over twenty years, initially in commercial radio, and then as a BBC radio presenter for many years. In 2003, Paul joined University of Wolverhampton as Head of Broadcast Journalism. He is now Executive Principal Lecturer and Head of Media and Film. His books include “News Values”, published by SAGE in 2007, and “Original Spin”, I.B. Tauris, 2013. Giles Brougère Professor in sciences of education at University Paris 13 – Sorbonne Paris Cité, Gilles Brougère is specialised in research on play and games, toys, childhood culture, childhood sociology, comparison and intercultural aspect of early childhood education, informal learning. He is responsible for a master’s degree on play and toys, and for the Ph. D program on sciences of education. He published Jouetsetcompagnie (Stock, 2003), Jouer/Apprendre (Economica, 2005), La ronde des jeux et des jouets. Harry, Pikachu, Superman et les autres (Autrement, 2008) Vincent Campbell is a Lecturer in Media and Communication in the Department of Media and Communication at the University of Leicester. As well as writing on the representation of science in documentary and factual entertainment, he has recently co-edited a book on controversial images in the media (Atwood, Campbell, Hunter and Lockyer (eds) (2013) Controversial Images: Media Representations on the Edge (Palgrave Macmillan). This chapter was written with support from the University of Leicester’s Study Leave Scheme. Marvin Carlson is the Sidney E, Cohn Distinguished Professor of Theatre, Comparative Literature, and Middle Eastern Studies at the Graduate Center of the City University of New York. He has authored many books and articles on theatre and performance history and theory including Theories of the Theatre and Performance: A Critical Introduction. His honors and awards include a Guggenheim Fellowship, the Bernard Hewitt and George Freedley Awards, the George Jean Nathan Award, the Calloway Prize, the ATHE Career Achievement Award, the ASTR Distin-

Biographical sketches

747

guished Scholarship Award, and an Honorary Doctorate from the University of Athens. Diane Carr is a Senior Lecturer in Media and Cultural Studies at the Institute of Education, University of London. She has published research on games, virtual worlds and gaming cultures. Her current research (2013–2014, funded by the ARHC) focuses on representations of ability and disability within digital games that feature science fiction and horror content. Information about previous research and publications is online at http://playhouse.wordpress.com/ Alvin Chua is a PhD Scholar at the KU Leuven, ASRO, Research x Design Group where he works in the field of urban data visualisation and analytics. His interests also include generative art and design. He works with computational tools and techniques to create physical representations of data as well as visualisations of systemic processes. Alvin was previously a research assistant at the Multimodal Analysis Lab, Interactive Digital Media Institute, National University of Singapore. He graduated from LASALLE, College of the Arts, Singapore in 2010 with a Master of Arts in Design. Martin Conboy is Professor of Journalism History and Director of Graduate Studies in the Department of Journalism Studies. He is also the co-director of the Centre for the Study of Journalism and History based in Sheffield. His research interests include the representation of national identity, popular journalism and celebrity culture. He is the author of seven single-authored books on the language and history of journalism and is on the editorial boards of Journalism Studies: Media History; Journalism: Theory, Practice and Criticism; and Memory Studies. He is also co-editor of the book series Journalism Studies: Key Texts. Rupert Cox My background is in anthropology and my regional orientation is Japan. I began my research investigating the Zen Arts, meaning practices such as the tea ceremony and the martial arts, work which has since been published with Routledge Press. I then undertook a multi-sited fieldwork project which investigated the history and practices associated with the idea of Japan as a ‘copying culture’. In these two projects and, while working as co-director of an independent documentary film company, Native Voice Films, I have developed extensive interests in the intersections between art and anthropology. In a new project that aims to develop soundscape studies and the use of sound recording within anthropology, I am investigating the perception and significance of aircraft noise in the vicinity of US military bases in Okinawa. Mary Dyson is a Senior Lecturer in the Department of Typography & Graphic Communication, University of Reading, UK. She studied experimental psychology leading to a PhD in perception, and now teaches theoretical and empirical approaches to typography and graphic communication. Her most recent research interests are driven by a desire to bridge the gap between scientists and designers and find

748

Biographical sketches

commonalities. She is therefore committed to interdisciplinary research and enjoys collaborating with colleagues from various disciplines to explore areas of common ground. Sarah Edge is a Professor of Photography and Cultural Studies at the University of Ulster, UK. Her articles on early photography include: ‘The Great Famine: absence, memory and photography’ in Cultural Studies Vol. 24 No. 6, with Gail Baylis, ‘Photographic History and the Visual Appearance of An Irish Nationalist Discourse 1840–1870’, in Victorian Literature and Culture Vol. 23 issue 01 and ‘The Power to Fix the Gaze: Gender and Class in Victorian Photography of Pit-Brow Women’, in Visual Sociology Vol. 12(2). She is currently writing, ‘The Extraordinary Archive of Arthur J. Munby: Photographing Class and Gender in the 19th Century’ for publication by IB Tauris in 2014. She is also an art photographer and regularly exhibits her work. Charles Forceville is associate professor in the Media Studies department at University of Amsterdam. He wrote Pictorial Metaphor in Advertising (Routledge 1996), and co-edited Multimodal Metaphor (with Eduardo Urios-Aparisi, Mouton de Gruyter 2009) and The Agile Mind, a volume presenting cognitivist perspectives on creativity (with Tony Veale and Kurt Feyaerts, Mouton de Gruyter 2013). Committed to cognitivist, evolutionary, and relevance-theoretical approaches, Forceville’s work is branching out from visual/multimodal metaphor to visual/multimodal rhetoric and narrative more generally. Genres and media he finds pertinent include fiction film, documentary, animation, comics & cartoons, and advertising. See “Adventures in Multimodality” at http://muldisc.wordpress.com/ Ross P. Garner is a Lecturer in Media and Cultural Studies in the School of Journalism, Media and Cultural Studies at Cardiff University. He teaches a variety of undergraduate modules concerning television, textuality and industry and has recently completed his PhD on Nostalgia and Post-2005 British Time Travel Dramas. Jana Holsanova is Associate Professor in Cognitive Science and Senior Researcher in Linnaeus environment “Cognition, Communication, and Learning” at Lund University, Sweden. Her research focuses on language, images, cognition and visual communication. Jana uses eye movement measurement to study image perception, the interplay between language and images, the role of images for learning, visual thinking (mental images), and interaction with various media. Her books include "Discourse, vision and cognition" (2008, Benjamins), "Myths and facts about reading. On the interplay between language and pictures in various media" (2010, Nordstedts), and "Methodologies for multimodal research" (2012, Special issue of Visual communication, Sage). Jana is Vice chair/ Chair elect of the Visual communication division, International Communication Association (2011-2015) and
Chairman of the Swedish Braille Authority at the Swedish Agency for Accessible Media.

Biographical sketches

749

Dr. William Cannon Hunter is an Associate Professor in the Department of Convention Management, College of Hotel and Tourism Management at Kyung Hee University in Seoul, Korea. He has lived as a world citizen in Asia for almost 20 years. His research focuses on destination imagery and problems related to cultural and touristic representations using visual research methodologies and Q method. Carey Jewitt is Professor of Learning and Technology and Head of the Culture, Communication and Media Department at the Institute of Education, University of London. Her research interests are the development of visual and multimodal research methods, video-based research, and researching technology-mediated interaction in the school classroom. She is Director of MODE – Multimodal Methods for Researching Digital Data and Environments, a NCRM Node, funded by the ESRC (Mode.ioe.ac.uk) and Crossing Boundaries, an ESRC funded project that explores the synergies between research methods in the arts and social sciences. Carey is a founding editor of the Sage journal Visual Communication, and her recent publications include The Sage Handbook of Researching Digital Technologies (2013) with Sara Price and Barry Brown, The Routledge Handbook of Multimodal Analysis, Second edition (2013) and Technology, Literacy and Learning: A multimodal approach (Routledge, 2008). Helen Kelly-Holmes lectures in Sociolinguistics and New Media at the University of Limerick, Ireland. She has published widely in a range of journals on the economic aspects of multilingualism, and media and multilingualism, particularly in relation to minority languages. Her publications include Advertising as Multilingual Communication (Palgrave, 2005), Language and the Market (with GerlindeMautner, Palgrave, 2010) and Peripheral Multilingualism (with Sari Pietikäinen, Oxford University Press 2013). She is currently co-editor (with Sue Wright) of the Language and Globalisation series published by Palgrave and is a partner in the Peripheral Multilingualism project, funded by the Finnish Academy and led by Sari Pietikäinen. Karel Kleisner is an Assistant Professor at the Department of Philosophy and History of Science, Charles University in Prague, Czech Republic. His main interest focuses on evolution of semantic organs; evolutionary psychology, development of comparative concepts in biology, theoretical biology, biosemiotics, and philosophy and history of biology in general. Christina Konstantinidou is an assistant professor in the department of Sociology, University of Crete, Greece. She studied Sociology at MirailUniversité de Toulouse, France and holds a PhD in Sociology of Panteion University of Athens. Her research and teaching interests include media studies, gender, immigration, discourse analysis methods and visual culture. David Machin is Professor of Media and Communication at ÖrebroUniversity, Sweden. He has published numerous books and journal papers, mostly drawing

750

Biographical sketches

on linguistic methods to look at other modes of communication such as Introduction to Multimodal Analysis (2007), Analysing Popular Music (2010), Language of Crime and Deviance (2012) and How to do Critical Discourse Analysis: A Multimodal Introduction (2012). He is Editor of the international peer reviewed journal Social Semiotics. Timo Maran is a Senior Research Fellow at the Department of Semiotics, University of Tartu, Estonia. His research interests include theory and history of zoosemiotics; ecocriticism and semiotic relations of nature and culture; and theory and semiotics of biological mimicry. His publications include Mimikrisemiootika [Semiotics of mimicry] (2008), Readings in Zoosemiotics (ed., with D. Martinelli and A. Turovski, 2011), Semiotics in the Wild. Essays in Honour of Kalevi Kull on the Occasion of His 60th Birthday (ed., with K. Lindström, R. Magnus and M. Toennessen 2012). Martha Michailidou is a lecturer in the department of Communication, Media and Culture, Panteion University, Athens. She studied Sociology and Philosophy at City University, London, and went on to do an MA in Media and Communications and a PhD in Sociology and Media and Communications at Goldsmiths College, University of London. She has taught at the Department of Sociology, University of Crete and at the department of Media and Communications, Goldsmiths College. Her research and teaching interests include qualitative methods of media and communications research, the relations between the production of culture, cultural consumption and the government of populations, and work in the creative industries. Maria Nikolajeva is a Professor of Education at the University of Cambridge, UK. She is the author and editor of numerous books, including one of the most established theoretical studies of multimedial texts for young children, How Picturebooks Work (2001), in collaboration with Carole Scott. Her most recent book publication is Power, Voice and Subjectivity in Literature for Young Readers (2010). She was one of the senior editors for The Oxford Encyclopedia of Children’s Literature and received the International Grimm Award in 2005 for a lifetime achievement in children’s literature research. Prof. Nurit Peled-Elhanan teaches Language Education at the Hebrew University in Jerusalem. Her research concerns all aspects of educational discourse: classroom discourse, racism and multiculturalism in Israeli schools, writing development and the multimodal analysis of text books. Her book: Palestine in Israeli Schoolbooks – Ideology and Propaganda in Education, was published by I. B. Tauris, London, April 2012. Kay O’Halloran is Associate Professor in the School of Education at Curtin University in Perth Western Australia and founding Director of the Multimodal Analysis Lab in the Interactive Digital Media Institute at the National University of Singapore. Her areas of research include multimodal analysis, social semiotics, mathematics discourse, multimodal literacy and the development of interactive digital

Biographical sketches

751

media technologies and mathematical and scientific visualization techniques for multimodal and socio-cultural analytics. Alexey Podlasov worked as a Post-Doc Research Fellow at the Multimodal Analysis Lab in the Interactive and Digital Media Institute at the National University of Singapore from 2008 to 2012. His background is in computer science and applied mathematics with research interests in data modeling, visualization and analytics, computer vision, image and video processing, as well as with software proto-typing and development. He is currently working with photorealistic rendering techniques for mobile 3D graphics. Dennis Puhalla is a Professor Emeritus of Design at the University of Cincinnati in Cincinnati, Ohio, USA. He teaches courses in color theory; principles of two and three-dimensional design; visual language design and motion design. He holds a B.S. in Design and an MFA from the University of Cincinnati, and a Ph.D. in Design from North Carolina State University, Raleigh, North Carolina, USA. Doctor Puhalla served ten years as Director of the School of Design, College of Design, Architecture, Art, and Planning at the University of Cincinnati. Doctor Puhalla’s professional work has been exhibited internationally and is included in private and corporate collections throughout the United States. He serves as a consultant in graphic communication design. He is the author of Design Elements: Form & Space, Rockport Publishers. Also, Doctor Puhalla is published in articles on visual organization strategies. Prior to his appointment at the University of Cincinnati, he taught in the College of Fine Arts at Miami University in Oxford, Ohio, USA Maximilian Rapp (*1987) is a PhD student at the University of Augsburg, Germany. His dissertation is about the political wall paintings in Northern Ireland, in particular about the functions of murals, how they developed during the ‘Troubles’ and if they can be used as unifying symbols. Besides his work as a consultant in the field of public management and open innovation, he is a lecturer at the University of Innsbruck, Austria. Furthermore, his current research interests are in the field of public management, open government and political communication. Markus Rhomberg (*1979) is Professor for Political Communication at Zeppelin University, Germany. From 2009 to 2010 he worked as Visiting Professor at the University of Hamburg, Germany. Rhomberg studied Political Studies, Communication Studies and Sociology at the University of Vienna, Austria, and received his PhD in 2006 with a book on Agenda-Setting and Media Democracy. His current research interests are in the field of public communication on climate change issues and, more generally, in the field of public opinion and political communication. Ian Roderick is an Associate Professor in Communication Studies at Wilfrid Laurier University, Canada. His research interests include cultural studies, technology and culture, visual communication and social semiotics.

752

Biographical sketches

Roger Sabin is Reader in Popular Culture at Central Saint Martins College of Arts and Design, University of the Arts London. His books include Adult Comics: An Introduction (Routledge), and Comics, Comix and Graphic Novels (Phaidon). He serves on the editorial boards of many of the key academic journals in the field of comics scholarship, and reviews graphic novels for the press and radio. The Daily Telegraph called him ‘The F. R. Leavis of the graphic novel’, and he’s not sure if that’s a compliment or an insult. Inna Semetsky has a PhD in philosophy of education from Columbia University in New York. Among her appointments are: Postdoctoral Fellow in Monash University and Research Academic in the University of Newcastle (Australia). She has over a hundred publications including 7 books. Her titles include “Deleuze, Education and Becoming” (2006), “Re-Symbolization of the Self: Human Development and Tarot Hermeneutic” (2011), and “The Edusemiotics of Images: Essays on the Art~Science of Tarot” (2013). Her latest appointment was Adjunct Professor, Centre for Global Studies in Education at the University of Waikato (New Zealand). She has also been a practicing Tarot reader for more than two decades. Camilla Smith is lecturer in Art History at the University of Birmingham. She is the author of several articles on the Anglo-Swiss artist Henry Fuseli and on Swiss travel writing. Her monograph on the importance of Switzerland in shaping Fuseli’s artistic production is forthcoming. Her current research projects include subversive travel guides in Weimar Germany, as well as an examination of the importance of Magnus Hirschfeld’s use of imagery in understanding ‘aberrational’ sexualities in early twentieth-century Germany. Göran Sonesson is Head of the Centre for Cognitive Semiotics at Lund University. He has written abundantly on the themes of pictorial and cultural semiotics, and recently has been at the forefront in the creation of the new subject matter cognitive semiotics. Within pictorial semiotics, he has notably demonstrated the errors of Eco, Goodman, and others in the study of iconicity, and he has suggested an extended model of visual rhetoric, which takes into account our knowledge of perceptual psychology. Sonesson is Secretary General of the International Association for Visual Semiotics and President of the World Association for Theoretical Semiotics. He is a member of the board of many international journals, notably Cognitive Semiotics,VISIO, Signata, and The Public Journal of Semiotics. Randall Teal, an Associate Professor of Architecture at the University of Idaho, is a writer, teacher, painter and designer. His pedagogical and research interests are in design fundamentals and architectural theory with a significant influence from Continental thought. His writing focuses primarily on understanding and promoting situated dialogue between creative processes and the built environment. Professor Teal’s most recent publication is the introductory design text, Preparing the Ground: Discovering the Everyday Practices of Design. In this book, he highlights the importance of a-rational faculties in extending our reach beyond the limits of

Biographical sketches

753

the rational mind; and, how design thinking – which utilizes these capacities – can be instrumental in creating a more attuned and ethical relation to our planet. Professor Teal was recently featured on Boise Community Radio (89.9 KRBX) discussing the future of sustainability via his article, “The Process of Place: A Temporal View of Sustainability in the Built Environment”. Other recent publications include: “Design Process as a Hermeneutic Practice”, In Form: The Journal of Architecture, Design, and Material Culture, Volume 11: Design Process; “An Inquiry into an Architecture of the Ungrounded”, Architecture and Ideas; “Foundational History: An Integrated Approach to Basic Design, History, and Theory”, JAE: The Journal of Architectural Education 64, No. 2; and “Developing a (Nonlinear) Practice of Design Thinking”, The International Journal of Art & Design Education 29.3. Professor Susan Wright is the Chair of Arts Education and Director of the Melbourne UNESCO Observatory of Arts Education at The University of Melbourne, Australia. Formerly, she was Professor and Head of Early Childhood and Special Needs Education at the National Institute of Education, Nanyang Technological University, Singapore and Director of the Centre for Applied Studies in Early Childhood at the Queensland University of Technology, Brisbane, Australia. The focus of Professor Wright’s research, teaching and publishing has been on children’s and teachers’ learning and development through artistic symbol systems, with a particular focus on embodied cognition/feeling, meaning-making and communication through multi-modal, creative and somatic forms. Her most recent books are Children, Meaning-Making and the Arts (2012, Pearson), Understanding Creativity in Early Childhood: Meaning-Making and Children’s Drawings (2010, SAGE), Special Education: Perspectives and Practices (2008, Pearson) and The Arts, Young Children and Learning (2003, Allyn and Bacon).

Index Action Man 542–543 Advertisements 138, 149 Aesthetic pleasures 712 Affect 75 Angle of interaction 83, 84, 304, 313, 319, 320 Anthropology 245, 697–708 Architecture 645–654 Archetypal images 175, 176 Art 173, 174, 206, 357–383, 697–708 Artists studio 357–383 Barbie 244–251, 540–543 Barthes 24–27, 173, 174 BBC 405–422 Built environment 647 Caricatures 435, 442 Cartoons 429–443 CGI 464–481 Children’s drawings 37, 517–534 Children’s picture books 711–726 Classroom layout 392–395 Cognitive semiotics 23 cognitive psychology 34–37, 248, 337, 722, 723 Colour 26, 51, 86, 195–212, 332–338, 367, 543, 621 Colour harmonies 195–212 Colour wheel 198 Comics 61–68, 253, 429–443 Compositional meaning 557–560 Computer games 501–515 Content analysis 289–291 Crime drama 405–422 Critical discourse analysis 73 Cutaway shots 731 Darwin 54 Demand images 555–556 Denotation and connotation 632–629 Design 195–197, 206, 215–240, 332–347, 349–351 Digital technology 396–398 Dinosaurs 464–479 Discourse 92, 94, 129, 301–326, 388 Documentary 464–481 Dolls 248–253, 255, 539–562

Education 254, 253–255 Ekphrasis 715–716 Emotion and images 711–726 Encoding images 55, 60 Engravers 432 Ethnicity 485 Ethnography 698 Eye tracking 331–352 Exhibitions 360–363, 373–376 Face detection 566–574 Fashion 589–601 Film 151–159, 301–326 Foucault 91, 94 Gay men 417 Gaze 337, 340, 347, 554–556, 697–698 Gender 153, 157, 171, 244, 248, 252, 407– 408, 417, 544, 545 Graphics 466, 469 History 407–412, 422, 430–431 Iconography 3, 30, 685–686 Image–text relations 606–607 Immigrants 93–129 Indexicality 40, 42 Intertextuality 508, 509 Interviews 682 Jung 173–191 Legitimation 604–605 Library footage 738–739 Linguistic landscapes 136, 139 Oil painting 33, 357–383 Magazines 365, 373, 380 Maps 196, 199, 211 Mental images 340 Metric projection 646 Militarism 71–89 Mimetic value 698 Modalities 67, 86 Multimodality 7, 138, 244, 301–326, 387, 539–562, 603–622 Mulvey 157–161

756

Index

Murals 677–694 Museums 697–708 Narrative 301–326, 405, 409, 411, 422, 433, 501–506, 514 Narrative processes 552–553 News bulletins 730–744 News grammar 733–734 Newspapers 93–97, 129, 261–277, 281–297 Nostalgia 405–422 Peirce 40 Perception 303, 311, 321, 331, 352 Photographs 96–97, 103, 124–126, 264, 278, 701 – Analyzing participants 291–293 – Analyzing settings 293–294 – As records 483–498 – Realism 286–288 Photojournalism 92–129, 281–297 Pictoral semiotics 27–29 Point of view 554–555 Political campaigns 677 Political posters 687 Popular culture 261–277 Posture 52 Propaganda 683, 686 Prototypes 38, 39, 57 Psychoanalysis 176–188 Psychology 23, 218, 220, 247, 711–726 – Experimental 221 – Perception 215–240 Reading paths 603–622 Relevance theory 51–54 Representational meaning 545–547 Resemiotisation 78 Rhetoric 251–257 Salience 611 Saussure 24 School books 603–622 Science representation 389–398, 464–481

Semiotics 2, 4, 92, 99, 115, 128, 173–191, 267, 276, 332, 337–341, 367, 543, 621 Semiosis 73 Sensory environment 698 Social actor analysis 82, 85 Social media 565–586 Social semiotics 73, 79 Sociolinguistics 135–138, 148 Social semiotics 605–607 Spectacle 463, 471, 480, 489–490 Street signs 647 Structural analysis 506–507 Surveillance 470, 476 Symbolism 9, 181–189, 196–198, 203–211 Synechdoche 633–638 Tabloid 261–277 Tactility 698, 700 Teaching 387–403 Teddy bears 248 Television 71–89, 155, 254, 405–422, 464– 481, 730–744 – Drama 405–422 Theatre 445–461 – Illusions 450, 452, 459 – Use of live video 459, 460 – Visual effects 446–450 Thematic frames 679–680 Tourism and images 625–641 Toys 243–257, 539–562 Twitter 565–586 Typography 215–240, 267 Video 153, 171 Viewer position 83, 338 Visual confusion 645–647 Visual crowding 348 Visual Studies 4–8 Voyeurism 470 War photography 281–297 Watercolour 358–382 Whiteboards 396