Shaping the North Through Multimodal and Intermedial Interaction (Arctic Encounters) 3030991032, 9783030991036

Citation preview

ARCTIC ENCOUNTERS

Shaping the North Through Multimodal and Intermedial Interaction

Edited by Juha-Pekka Alarauhio Tiina Räisänen Jarkko Toikkanen Riikka Tumelius

Arctic Encounters

Series Editor Roger Norum, Cultural Anthropology, University of Oulu, Oulu, Finland

This series brings together cutting-edge scholarship across the social sciences and humanities focusing on a critically important region of the world, the Arctic. Books in the series present high-calibre, critical insights in an approachable form as a means of unpacking and drawing attention to the multiple meanings and messages embedded in contemporary and historical Arctic social, cultural, and environmental changes.

More information about this series at https://link.springer.com/bookseries/16629

Juha-Pekka Alarauhio · Tiina Räisänen · Jarkko Toikkanen · Riikka Tumelius Editors

Shaping the North Through Multimodal and Intermedial Interaction

Editors Juha-Pekka Alarauhio Research Unit for Languages and Literature University of Oulu Oulu, Finland

Tiina Räisänen Research Unit for Languages and Literature University of Oulu Oulu, Finland

Jarkko Toikkanen Research Unit for Languages and Literature University of Oulu Oulu, Finland

Riikka Tumelius Research Unit for Languages and Literature University of Oulu Oulu, Finland

ISSN 2730-6488 ISSN 2730-6496 (electronic) Arctic Encounters ISBN 978-3-030-99103-6 ISBN 978-3-030-99104-3 (eBook) https://doi.org/10.1007/978-3-030-99104-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover illustration: Toni Faint/Getty Images This Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Series Editor’s Foreword

The Arctic. The central importance of this region to today’s world, and to our planet itself, cannot be overstated. The Arctic holds key relevance not merely as a poster child of global environmental change—and of human action for and against such change—but as a geophysical bellwether and scientific laboratory for planetary climate effects. The Arctic is fundamental to many contemporary concerns: energy transitions, cultural heritage, mobility and the influence of indigenous peoples on local and global political consciousnesses, to name but a few. The Arctic plays a key role in debates about the pasts, presents and futures of water, land and space. The Arctic has long been integral to transnational economic and geopolitical considerations, to various pan-European, North American and Asian affairs, and to multiple anxieties over the planetary centres and peripheries. All this Arctic epistemic industriousness has led to a surge in interest and need for Arctic science, and subsequently in the number of publications, programmes and institutions which have taken the region as their core concern. Arctic Encounters is a book series that seeks to encourage, support and disseminate cutting-edge scholarship across the social sciences and humanities, focusing on this vast region spanning the edges of the planet. The books in Arctic Encounters examine with critical insight the multiple meanings and messages embedded within the contemporary and historical environmental, social and cultural contexts of the Arctic. The series takes the overarching approach of transdisciplinarity, which v

vi

SERIES EDITOR’S FOREWORD

is to say that it encourages scientific research and creative communication of science that is aimed not merely across and between disciplines, but beyond and outside the academy as well. A core objective of transdisciplinarity is to understand the world in all of its complexity, instead of focusing on any one part of it, or from any one point of view. This can stimulate new perspectives in theory, method, framework and approach, and can generate novel and out-of-the-box thinking for innovating science and advancing knowledge. The series comprises both single- and multi-authored monographs and edited collections, while also encouraging less strictly curated forms of expression. These include artistic and multi-modal interventions, as well as publications which arise out of project partnerships and which are co-created with community members outside academe’s framework. By drawing on multiple disciplines and multiple traditions of writing, Arctic Encounters hopes to attract not just a diverse authorship but also a diverse readership, comprised of Arctic-focused scholars as well as those who come from outside the field—and outside the ivory tower. Happy reading! Arctic Encounters the series has its origins in Arctic Encounters the project, a HERA-funded collaborative research initiative led by Prof. Graham Huggan at the University of Leeds that sought to bring together distinct perspectives and institutions in the Nordic countries and beyond to address the increasingly important role of cultural mobilities in fashioning understandings of the European Arctic. With this project’s impactful work as a point of departure, then, this series aims to further epistemologically “unscramble” the broader Arctic from the regionalist and colonial roots that once characterized knowledge creation about this region, once thought by Europeans to be remote and peripheral. Holding the Arctic as an implicit focus, the series’ thematic scope maintains a wide frame in order to champion novel and inventive submissions that speak across multiple perspectives and approaches to generate environmentally, societally and planetarily influential science. Oulu, Finland

Roger Norum

Acknowledgements

The editors wish to thank the many individuals shaping the interactive multimodal and intermedial process of producing the volume at hand. You have made all the difference. Firstly, we want to acknowledge our local environment with the Research Unit for Languages and Literature, and the Faculty of Humanities, at the University of Oulu, Finland. Our faculty colleague Roger Norum was our first contact in lifting the book proposal off the ground and profiling it for the Arctic Encounters series at Palgrave Macmillan. Secondly, we extend our gratitude to the staff at our esteemed publisher, Palgrave Macmillan. From day one we have felt confident about providing the best and most professional end result for the international audience, and we excitedly look forward to releasing the book with you. Thirdly, we give a loud cheer to our article writers as we hope to have guided the whole process with fairness and gusto, while we have also done justice to your contributions as editors. As you know, there is plenty of further research to conduct—for which this step is only the start. Finally, we offer our sincerest thanks to those members of the academic community who, willingly and without pay, took part in the open peer review of this volume in the autumn of 2021. We are all well aware of the pro bono duties that come with the academic profession to place demands on the use of personal resources but which are nonetheless crucial to science.

vii

viii

ACKNOWLEDGEMENTS

In this spirit, the editors tip their collective hat to Nassim Balestrini (University of Graz, Austria), Coppelie Cocq (Umeå University, Sweden), Caroline Debray (University of Basel, Switzerland), Sabine Grasz (University of Oulu, Finland), Samira Ibnelkaïd (University of Oulu), Teppo Jakonen (University of Turku, Finland), Antti Kamunen (University of Oulu), Lydia Kokkola (University of Oulu), Malene Charlotte Larsen (Aalborg University, Denmark), Liisa-Maria Lehto (University of Oulu), Stefan Lundström (Luleå University of Technology, Sweden), Jakub Mlynar (University of Applied Sciences Western Switzerland), Johannes Riquet (Tampere University, Finland), Pajari Räsänen (University of Helsinki, Finland), Ingrid de Saint-Georges (University of Luxembourg, Luxembourg), Markku Salmela (Tampere University), Niklas Salmose (Linnaeus University, Sweden), Kasimir Sandbacka (University of Oulu), Stephanie Schnurr (University of Warwick, UK), Laura Siragusa (University of Oulu), and Christian Dyrlund Wåhlin-Jacobsen (Copenhagen Business School, Denmark). A special commemoration is dedicated to Lars Elleström, the head of the Linnaeus University Centre for Intermedial and Multimodal Studies at Växjö, Sweden. Lars was a leading light and eminent influence in this field of research and his recent passing was a terrible blow from which we shudder to recover. As our final contact, he took part in the open peer review too. Each article, with the exception of the Introduction chapter, was non-anonymously peer reviewed by at least two readers according to the guidelines of the Federation of Finnish Learned Societies (TSV) at https://tsv.fi/en/services/label-for-peer-reviewed-schola rly-publications. The process has been designed to meet the peer review criteria of international academic publishers. Your editors, December 2021 in Oulu

Contents

1

Introduction: Multimodality and Intermediality in the North Juha-Pekka Alarauhio, Tiina Räisänen, Jarkko Toikkanen, and Riikka Tumelius

1

Part I Mediating Work and Education 2

3

4

5

A Design-Driven Approach to Language Teacher Education in the Era of Digitalization Riikka Tumelius, Leena Kuure, and Maritta Riekki Bad News Delivery as an Interactional Context for Constructing Professional Identities and Social Relations: Multimodal Approach Tiina Räisänen and Tuire Oittinen Multimodal Negotiation for the Right to Access Digital Devices Among Elderly Users and Teachers Joonas Råman Zooming in on a Frame: Collectively Focusing on a Co-participant’s Person or Surroundings in Video-Mediated Interaction Mari Holmström, Mirka Rauniomaa, and Maarit Siromaa

19

41

67

95

ix

x

CONTENTS

Part II 6

7

8

9

10

11

Mediating Arts and Culture

Voicing a Northern Minority Culture on a Global and Digital Arena: Sami Music Videos on YouTube Annbritt Palo, Lena Manderstedt, and Outi Toropainen Global Participation in the North: Exploring the Issues of Silent Participation and Building a Zone of Identification in a Hostile Digital Environment Matti Nikkilä Light and Darkness: Transmediality in the New Provincial Self-Identification and Construction of the Finnish North Katja-Maria Miettunen and Jussi Jalonen Transmediality and Multimodality in the Artistic Work of Nils-Aslak Valkeapää Kuisma Korhonen and Veli-Pekka Lehtola Imaginations in the North: Cross-Modal Communication in Johan Ludvig Runeberg’s The Moose Hunters and Matthew Arnold’s Balder Dead Juha-Pekka Alarauhio Endless North: Intermedial Experience of Motion and Balance in H. P. Lovecraft’s The Dream-Quest of Unknown Kadath Jarkko Toikkanen

Index

123

147

167

187

213

233

247

Notes on Contributors

Juha-Pekka Alarauhio is Lecturer in English at the University of Oulu, Finland. His research work focuses on literary narrative structures and traditions as facilitators in literary communications. He has previously published on Matthew Arnold’s writings. Mari Holmström is Doctoral Researcher at the University of Oulu, Finland. She explores break-taking at work from different perspectives, utilizing ethnography, dramaturgy, and conversation analysis to form a comprehensive understanding of social (inter)action in break rooms. Jussi Jalonen is Finnish historian living in Rauma and Adjunct Professor at the University of Oulu, Finland. His interests include nationalism and cultural history of warfare. Kuisma Korhonen is Professor of Literature at the University of Oulu, Finland. He has authored and edited several books in English and Finnish on literature, philosophy, and cultural memory. Currently, he is interested in creative writing and multimodal narration. Leena Kuure is Senior Lecturer Emerita at the University of Oulu, Finland, and Adjunct Professor at the University of Eastern Finland. Her research interests include children’s participation in the transformation of the future school, as well as the changing scene of language pedagogies and language teacher education in the technology-rich world.

xi

xii

NOTES ON CONTRIBUTORS

Veli-Pekka Lehtola is Professor of Sámi Culture at the University of Oulu, Finland. He is specialized in the history of the Sámi and Lapland, in modern Sámi art, and in the development of the Sámi representations. His work The Sámi People—Traditions in Transition (University Press of Alaska) has been a popular textbook in Indigenous and Scandinavian studies worldwide. Lena Manderstedt is Associate Professor in Swedish and Education at the Luleå University of Technology, Sweden. Her main research interests include teacher education, YA literature, literature and education in relation to online cultures, as well as ideology and power. Katja-Maria Miettunen is Finnish historian and Ph.D. from the University of Tampere, Finland. She has worked at the Finnish Centre of Excellence in the History of Society. Her research interests include collective memory, identity-building, and images of the past. Matti Nikkilä is Doctoral Researcher at the University of Oulu, Finland. His research concentrates on harmful discourses online. Previously he has worked in journalism for more than a decade. Tuire Oittinen is Postdoctoral Researcher at the University of Oulu, Finland. She investigates social interaction in multinational work settings, including educational and business contexts. Annbritt Palo is Associate Professor in Swedish and Education at the Luleå University of Technology, Sweden. She is primarily interested in issues of literature, literature and education, gender, intersectionality, and media literacy. Tiina Räisänen is Senior Lecturer in English at the University of Oulu, Finland. She has published in IEEE Transactions on Professional Communication and European Journal of International Management on professional communication in global settings. Joonas Råman is linguist and Ph.D. from the University of Oulu, Finland. His research focuses on the topics of embodied interaction, participation and educational settings. Mirka Rauniomaa is Senior Lecturer in English at the University of Oulu, Finland. She is interested in language and social interaction as they take form in various contexts of everyday life.

NOTES ON CONTRIBUTORS

xiii

Maritta Riekki is Language Teacher and Language Teacher Educator, and Ph.D. from the University of Oulu, Finland. Her research interests include the complexity of change in the field of language education, the development of language teacher identity, and language pedagogy at large. Maarit Siromaa is Senior Lecturer in English, and Postdoctoral Researcher at the University of Oulu, Finland. She works in the Break project and studies human interaction and cultural processes across diverse cultural contexts and configurations. Jarkko Toikkanen is Senior Lecturer in English at the University of Oulu, Finland, and Adjunct Professor in English at Tampere University, Finland. He has launched a three-tier model of mediality to study the intermedial experience of medial environments including literature and television. Outi Toropainen is Senior Lecturer in Swedish and Education at the Luleå University of Technology, Sweden. Her Ph.D. dealt with communicative utterances in L2 Finnish. Most recently, she has studied language learners’ use of metalinguistic knowledge and pragmatic language skills. Riikka Tumelius is Doctoral Researcher at the Research Unit for Languages and Literature, University of Oulu, Finland. She has published on designing for language learning in technology-mediated environments.

List of Figures

Fig. Fig. Fig. Fig. Fig. Fig. Fig.

4.1 4.2 4.3 4.4 4.5 4.6 6.1

Image 9.1 Image 9.2 Image 9.3

Excerpt 1, part 1 Excerpt 1, part 2 Excerpt 2, part 1 Excerpt 2, part 2 Excerpt 3, part 1 Excerpt 3, part 2 Elleström’s model for communication (2018, p. 218) From Trekways of the Wind (1994; Ruoktu Váimmus, 1985) From Beaivi, Áhˇcážan (1988) From The Earth, My Mother (2017; Eanni, eannážan, 2001)

75 76 80 81 84 85 131

198 200 203

xv

List of Tables

Table Table Table Table

2.1 2.2 4.1 6.1

Research materials The design process on the course Transcription conventions Analyzed videos

27 28 89 128

xvii

CHAPTER 1

Introduction: Multimodality and Intermediality in the North Juha-Pekka Alarauhio, Tiina Räisänen, Jarkko Toikkanen, and Riikka Tumelius

Studies on the north have often observed it as an exotic geographic space that, within some disciplines, produces an artistic imaginary with peculiar affective potential (see Chartier, 2018; Hennig et al., 2018; Dodds & Nuttall, 2019; Hansson et al., 2020), and, within others, a geocultural economy with special identities and environmental resources

J.-P. Alarauhio · T. Räisänen · J. Toikkanen (B) · R. Tumelius Research Unit for Languages and Literature, University of Oulu, Oulu, Finland e-mail: [email protected] J.-P. Alarauhio e-mail: [email protected] T. Räisänen e-mail: [email protected] R. Tumelius e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_1

1

2

J.-P. ALARAUHIO ET AL.

(see Körber et al., 2017; Markides & Forsythe, 2019; Steinberg et al., 2015). In the volume at hand, from now on referred to as MIN, these orientations are combined with one another as multimodal and intermedial interaction in shaping understandings of the north is brought into focus. Our idea of the north serves as a thematic framework that is not bound to any single region but whose reach is understood widely, as a notion with globally shared fascination. In what ways are ideas on the singularity of the north imagined and analyzed with varying functions from the perspectives of language, literary, cultural, and media studies? MIN collects ten articles that employ innovative theories and methods in the fields of multimodality and intermediality research to tackle the north as a theme in continuous change. Multimodality refers to communicating information, often simultaneously, in several modes such as the verbal and the visual, whereas intermediality refers to the phenomena that occur between media, focusing on what it is possible to present in a specific medium or media technology (see Bruhn & Schirrmacher, 2022; Elleström, Lars, 2021a, 2021b; Forceville, 2020; Herzogenrath, 2012; Jewitt, 2014; Moschini & Sindoni, 2021; O’Halloran & Smith, 2011; Petersson et al., 2018; Rippl, 2015). Both approaches emphasize humans interacting and taking part in making meaning with multimodal resources and relating experience via intermedial means. In MIN, the key question is how this kind of interactive participation involves experiences of the north either as a physical setting or a more abstract cultural condition that shapes the activity. Our contributors study old and new ways of multimodal and intermedial interaction, non-digital and digital, in the north to demonstrate their use in practice, paving way for collaboration across the humanities and research in media and technology. In such work, many of the difficulties in finding a common speech can be due to differences in disciplinary jargon, and indeed one of our main objectives is to develop the conceptual language into the direction of an interdisciplinary dialogue. While multimodality has been particularly valuable in linguistic research and translation studies—including studies on multimodality in social interaction based on sociological and ethnomethodological theory—intermediality has drawn on a host of traditions in art and media research. As deceptively similar sounding terms such as transmediality and cross-modality (see Elleström, 2017, 2019; Salmose & Elleström, 2020) have recently become popular in the field, their comparative theoretical nuances and methodological applications must be recognized too for their role in the developing language.

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

3

MIN aims at bringing the separate disciplinary ways of talking and writing about the north closer to one another. We include articles from teaching language and the use of digital devices in the physical north to representing the cultural environment of the north across the media and producing experiences on imagining the north in literature and the other arts. All the individual contributions are described in detail in the main section introductions. The challenge, as recognized in the title, will be to create a space in which the multimodal and intermedial approaches of quite different directions may come together and generate new knowledge in the field through their interaction. Within this joint remit, our contributors burrow into their case studies to analyze empirical and linguistic data, as well as media products and literary texts, for knowledge in concepts and results in practical application. The showcased technologies range from the latest digital devices, tools of communication, and environments to more traditional, non-digital ways of presenting ideas and imaginations in language and the arts. With its unique profile, the benefits of the volume are manifold as the articles participate in topical theoretical debate, put novel methodology to test, and offer insight into the versatile role of multimodal and intermedial interaction in shaping the north. To position MIN in the field of multimodality and intermediality research, we the editors will next embark on a dialogue on our key theoretical concepts—“mode,” “medium,” and “interaction.” Toikkanen: Joshua Meyrowitz (1999) has recognized three recurring metaphors of “medium” on the concept being employed in theoretical discussion. While Meyrowitz’s article is from two decades ago, and leans on the mass media studies perspective, it provides a highly useful starting point for a shared dialogue on the concept, how it adds up to an understanding of “intermediality,” or what occurs between media, and how this process is interactively engaged with. The first metaphor is that of a vessel or conduit—the medium is blank matter such as the physical television set that conveys information without affecting it. The second metaphor is that of a language—the medium has its own kind of grammar that systemically shapes the content coming through. The idea of television expands from the physical item to a specific language afforded by the rules and conventions of audiovisual expression. The third metaphor is that of an environment—the medium endows a space or practice for both individual and collective use. The idea of television expands further from a physical item and systemic language toward television as humans interacting and

4

J.-P. ALARAUHIO ET AL.

taking part in how the medium is effectively utilized. In describing these three metaphors of medium, and how they may be integrated, Meyrowitz does not talk about intermediality as such but, in my perspective, any relations that occur between the metaphors—conduit, language, or environment—can be analyzed as intermedial relations. These relations are the focus of intermediality research, within any disciplinary framework. The way I see it, they can be connected with multimodality research within the particular framework of semiotics where the linguistic concept of “mode” belongs, in both “rather coarse” and “more fine-grained” theoretical applications, as Lars Elleström put it (2019, pp. 57–58). Alarauhio: These metaphors of medium seem to be highly resilient and can be seen on the background in some of the recent scholarship in intermediality as well. In Elleström’s (2021a) model of communication, for instance, the notion of “transfer” of message by means of a media product effectively resurrects the “conduit” metaphor yet refines it by filtering it through an advanced, embodied view of interaction and communication. This also raises a question: How does the placement of the interdisciplinary work in the relations between these metaphors help in creating a common ground for a multidisciplinary effort? Toikkanen’s suggestion above—bridging between research into intermediality and that of multimodality—identifies a crucial divergence in disciplinary emphases. At this point, what seems highly appealing is the opportunity to see a range of disciplinary approaches in practical work with these intermedial relations. This brings about a degree of specificity that is often lacking when the discussion focuses on terminology. Furthermore, the shared theme, the north, may function as a kick starter on the level of praxis, where shared interests can be noted as they emerge in concrete and detailed case studies. The concept of “mode” and that of multimodality can certainly be harnessed for work in the contexts of literary and cultural scholarship, but will there remain enough resemblance between this type of usage and those in the other fields to validate a genuinely interdisciplinary effort? Räisänen: Multimodality is a pervasive characteristic of any communicative situation. Within linguistics, which is a broad field of study, multimodality has been one of the key focus points for quite a long time especially in the study of social interaction (see, e.g., Goodwin, 2000; Mondada, 2007). Along with an increasing interest in multimodality in language and communication studies more broadly, research

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

5

on social interaction that originates from ethnomethodology and conversation analysis (CA, see Garfinkel, 1967) has investigated embodiment, such as gestures, and materiality, including the use of tools and other artifacts, along with language as among the resources mobilized by participants to build accountable social actions (e.g., Mondada, 2016). In this regard, verbal language (including prosody, phonetics, syntax, lexis) is not seen as the most important communicative resource for human participants a priori, but rather as equally available for them among a range of resources in producing and engaging in social action. In CA, multimodality is related to human action, and resources are mobilized to format action and to make it intelligible for others (see Drew & Heritage, 1992; Mondada, 2016; Sacks et al., 1974; Sidnell & Stivers, 2013). While CA focuses on social action, multimodality in social semiotics is related to meaning-making and to multiple semiotic forms, or modes, that contribute to meaning (Kress & van Leeuwen, 2001), including not only speech and embodied resources, but also moving images or music (see Bateman et al., 2017). These modes are semiotic resources which allow the simultaneous realization of discourses and types of (inter)action (Kress & van Leeuwen, 2001). Social actors produce and learn modes in and for their use. Mode is a socially shaped and culturally given resource for making meaning. Image, writing, layout, gesture, speech, moving image, soundtrack are examples of modes used in representation and communication (Kress, 2009, p. 79). Hence, modes are not “out there” a priori, but they have to be made accountable by social actors (CA approach), learned and produced by users, and defined and determined in each study (Norris, 2004). Tumelius: As described above, studies in multimodality do not view modes as a set category, but what is considered as a mode depends on what is perceived relevant in interaction to communicate meanings (Kress & van Leeuwen, 2001). Socio-semiotic approaches to language— multimodal discourse analysis building on critical discourse analysis in particular—derive from Halliday’s work on systemic functional linguistics (Halliday, 1973, 1978). Halliday’s influence can also be seen in nexus analysis (Scollon & Scollon, 2004) and multimodal interaction analysis (Norris, 2004), which are methodological frameworks studying social interaction from the perspective of mediated discourse theory (see, e.g., Norris & Jones, 2005; Scollon, 2001; Wertsch, 1998). Drawing on the work of Vygotsky and Bakhtin, Wertsch (1998) takes mediated action as the unit of analysis for sociocultural research where social action, i.e.,

6

J.-P. ALARAUHIO ET AL.

mediated action, is seen to be produced by various mediational means or cultural tools. Consequently, mediated discourse theory views social interaction as inherently multimodal where language is one, although important, resource of meaning-making. From the mediated perspective, modes can be defined as complex systems of mediated action with regularities, which can be seen as an interplay of the social actor(s) and mediational means involved in action (Norris, 2013). We’re jointly examining how to make the difference between mode and medium. Kress and van Leeuwen (2001) describe the distinction as being a question content (mode) and expression (media). They continue to define mode as “semiotic resources which allow the simultaneous realisation of discourses and types of (inter) action,” and media as “the material resources used in the production of semiotic products and events, including both the tools and the materials used” (2001, pp. 21–22). For example, when exploring embodied interaction, we can use facial expressions or hand gestures as modes that carry certain meanings, expressed via the medium of the face or hands. The medium can take the form of various cultural tools, such as books, handheld mobile devices, or online learning platforms, through which the social action is mediated. According to Kress and van Leeuwen (2001), modes carry meaning that can be expressed via different media and have an inner grammar or, as Norris (2013) described, are used with regularities. For example, comedy can be considered as a mode, or color, e.g., red to signal danger or raise alert. The relationship between mode and medium is complex, and what is a mode or a medium is dependent on the situated action with its social actors (see Norris [2013] for a discussion on problematizing what is a mode). This complexity becomes visible in the studies of the current volume as they take on different approaches to utilize the conceptual tools of multimodality and intermediality, and thus, foreground the interplay between modes and media, and showcase the variation in their theoretical premises. Toikkanen: The challenge, then, in how ideas and experiences of the north are shaped through multimodal and intermedial interaction is about defining one’s key concepts from the perspective of how the distinction between modes and media can add layers and depth to analyzing case studies. It might not be very productive simply to place articles next to one another in which some apply exclusively the semiotic concept of mode with a particular methodology and others exclusively a conceptualization of medium rooted in a particular tradition in art and media

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

7

research. For this reason, we have encouraged our writers to discuss explicitly what the value of the distinction between modes and media is to their contribution—what the questions and results are that might go unnoticed without it. As noted, another aspect in recent work in the field of multimodality and intermediality research is that scholars have distinguished between terms that may sound deceptively similar—including transmediality. (For comparative intermedial methodologies, see Balestrini & Bergmann, 2018; Bruhn, [2016]; Jensen, [2016]; Wolf, [2011]; Rajewsky, [2005], and, for transmedial methodologies, Thon, [2016]; Ryan & Thon, [2014]; and Jenkins, [2008].) Indeed, the stakes for sustaining a clear terminology have been raised if “intermedial relations are everywhere in all forms of human communication,” and they appear in the form of “coexisting media types, media products and more specifically media modes” (Elleström, 2019) whose multimodal information may be transmedially shared across media platforms and “remediated,” to use a millennial term, through new media technologies (see Bolter & Grusin, 2000). The language is a lot to digest. MIN accepts the terminological challenge with its selected focus and strives to turn it into the strength of our interdisciplinary effort. In my chapter, for instance, I put the distinction between intermediality and transmediality so that, for practical purposes, intermedial analysis focuses on the differences between presentations in various media whereas transmedial analysis observes the similarities across them. As I study the intermedial experience of reading Lovecraft, there is only the single medium of the verbal literary text but, since ideas arise that are mediated by the imagined sensory perceptions mediated by the story’s words, intermedial analysis can be used to study the experience of the three medial tiers interacting in specific ways. What is more, transmedial analysis can help in recognizing the elements of reading shared across the tiers, such as the rhetorical descriptions of imaginary places that have the potential to engage several senses for their different effects. Reading thus produces intermedial experiences of the north as a physical space and cultural imaginary to affect both the senses and ideas about the north that transmedially appears as a central or peripheral element in the material at hand. Literary texts achieve this result through the rhetorical devices that create literary worlds and realities, functioning as kinds of literary technologies. Tumelius: Centrality and peripherality become focal topics in the discussion of the north within the chapters in the book. Geographically

8

J.-P. ALARAUHIO ET AL.

speaking, northern areas can carry the connotation of being peripheral, having distance from the central areas of the society with economic power and decision-making (Paasi, 1995). However, living in the socalled periphery does not necessarily create a peripheral experience of the everyday life. Place can be defined as a social construction, a center of meaning for an individual as well as for shared experiences of groups (Cresswell, 2004; Tuan, 1975). Thus, studying the everyday life of people living in the north does not automatically include explicit descriptions of being specifically Northern or leading a Northern life as is seen in the chapters in this volume, as well. The north is a relational concept, depending on the individual perspective, and accordingly, this volume exemplifies a variety of conceptions of what is north. The discourses defining north as peripheral and the individual experience of centrality create tensions which are amplified by the modern digital technologies which bring the global in the local experiences. From a perspective of the inhabitants in Oulu, Finland, to which many of the authors in this volume are affiliated, the capital area of Finland is often described as “the south” even if the capital city Helsinki is very far north from a global perspective. Furthermore, Oulu can be considered as belonging to the south when taking the perspective of people residing in the Finnish Lapland. In this context, Oulu may serve as a gateway between the south and the “real” north. Given the complex and relational nature of north as a social construct, the volume refrains from defining the north as a distinct place or area. The chapters rather show us different perspectives on how the north both affects social interaction as well as becomes shaped by the interaction. Räisänen: Some of the examples introduced so far confirm the value of placing empirical research from the north and on the north into dialogue in our attempt to understand how different interactions shape the north. Both temporal and spatial aspects of the context (see Blommaert, 2015) need to be considered in capturing how the meaning, and/or accountability, of a mode or a resource gains its value in the social context of the human experience and construction of north. This also means drawing on ethnographic perspectives. As Bateman et al. (2017, p. 16) note, “we also need to see how such meanings can function collectively, combining in the service of unified activities of communication.” They continue to point out that “one such challenge is that the various forms will generally have very different properties from each other—using forms together then requires that ways are found of making sense of heterogeneous ways

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

9

of making meaning” (p. 16). They further argue that “multimodality […] is inherently and intrinsically an interdisciplinary cooperative enterprise. Making this work is itself a major aspect of, and motivation for, the account” (p. 20, italics in original). As we’ve taken up this interdisciplinary challenge in this volume, the question arises what are the multimodal and intermedial ensembles (Bezemer & Kress, 2015) used to create and shape the north in and across the different contexts of arts, media, work, and education? Alarauhio: As the discussion so far attests, the theme of situatedness— in the north of some description—can be viewed pluralistically, and we can see the multivarious conceptions of the north at least in the three ways suggested by Lefebvre (1991, pp. 11–12): space as partly physical, partly mental, and partly social. Turning then to Elleström’s notion of the virtual sphere, defined as “a mental sphere, created by communicative semiosis and consisting of cognitive import formed by represented objects” (2021a, p. 29), we can observe how communications about the north form such virtual spheres informed in each chapter by a different configuration of the physically, mentally and socially constructed notions of space. Since any communication involving situatedness in the north—explicitly or implicitly—may without much conscious effort create a virtual sphere that bears the stamp of the north on it, we can see how a virtual space thus produced connects physical situatedness with mental and social dimensions of place almost inevitably, as the connotations regarding centrality and peripherality illustrate. At the same time, as the chapters in this volume amply demonstrate, human beings seem to be highly capable of readjusting themselves into the variant spatial orientations when switching between different virtual spheres and the media used to create them. In other words, most people will have no trouble differentiating between hypothetical norths brought to us in the fiction of Lovecraft, Arnold, and Runeberg, and the north held as a source of identity for inhabitants of the Finnish Lapland, or the north as the geographic and cultural complex effecting the professional premises of teachers in Northern Finland. Whether a conceived space or a lived space—represented space or space of representation to use Lefebvre’s (1991, p. 36) terms—it is through multimodal and intermedial means by which the virtual spheres evoking the north are deployed in interaction. In this way, we can see the north as a theme that can further support interdisciplinary work. One sign of this happening is how the dichotomy between central

10

J.-P. ALARAUHIO ET AL.

and peripheral manifests under the topic of the north so conspicuously in so many of the chapters of this volume. The book is structured so that the volume is divided into two main sections—Mediating Work & Education, and Mediating Arts & Culture—according to the specific media environments in which multimodal and intermedial interaction in shaping understandings of the north takes place. Since our idea of the north serves as a thematic framework for ways of engaging with and conceptualizing the north from the perspectives of language, literary, cultural, and media studies, in the main section introductions we describe the individual contributions in detail. Our writers have set out to research, within their respective theoretical and methodological frameworks, how the kind of interactive participation they explore in their materials and case studies involves experiences of the north either as a physical setting or a more abstract cultural condition that shapes the activity.

Mediating Work and Education In this main section, the focus is on multimodal and intermedial analysis specifically in language studies. The chapters draw on multimodal conversation analysis and nexus analysis, and apply ethnography in different ways. The authors in this section demonstrate how the meaning of specific modes emerge in the complex web of resources drawn on by interactants in their different medial environments (face-to-face, virtual, digital). The authors show how interactants need knowledge about their coparticipants, interactional rules, and the surrounding community and how to use suitable resources and modes in the given media in order to create meaningful actions, activities, and practices for work and learning. In particular, the chapters contribute to our understanding of multimodal interaction shaping the north as a work and a learning environment and as an existing as well as emergent dimension of identities and discourses. “North” emerges in the empirical studies as a complex theme: on the one hand, as a physical setting with specific affordances for social action, and on the other hand as a more abstract cultural condition through underlying discourses shaping actions and identities of people residing in different localities. The chapters in this section illustrate how the local and global experiences of north blend in social interaction (see also Nikkilä, and Palo, Manderstedt & Toropainen in the other main section), at the

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

11

same time blurring the boundaries of material space manifested in the pervasive digital environments of our everyday life. While research physically located in northern settings can be more straightforward in illustrating northern interactions, in settings that are not geographically in the north, exploring the theme of north may be more elusive. The chapters show how the theme can be put under scrutinization with the help of ethnographic perspectives. The concepts of multimodality and intermediality further provide tools to investigate the complexities involved and shed light on the role of language in the phenomena under study. Tumelius , Kuure, & Riekki investigate discourses that circulate the nexus of practice of becoming a language teacher in the north in the context of a design-based university course focusing on language learning. They show how future language teachers need to balance between traditional and new discourses of language learning in the digitalizing society when designing language learning and accomplishing their situated activities through different modes and drawing on various resources of diverse media. Räisänen & Oittinen examine how professional identities and social relations are co-constructed in multimodal ways during bad news delivery in a business context. They illustrate how a Finnish manager, constructing a Nordic manager identity, enacts egalitarian leadership in the interactions with his subordinates in a multinational company located in China. The study provides further understanding of the complex, multimodal ensemble emerging when people with diverse linguacultural backgrounds encounter in the global space of business interactions. Råman examines how digital skills are taught on a course aimed at elderly users of digital devices in the Finnish context. He focuses on multimodal negotiation for the right to access digital devices and how the pedagogical aims affect how the situations unfold. The study provides a perspective on how two global megatrends, digitalization of services and rapidly aging population, are manifested in the everyday life in the Nordic countries. Holmström, Rauniomaa, & Siromaa examine how members of work communities engage in virtual breaks through the medium of the Zoom platform. The authors show how the participants create a sense of copresence and social intimacy by drawing on various multimodal resources afforded by the intermedial environment.

12

J.-P. ALARAUHIO ET AL.

Mediating Arts and Culture In this main section, the focus is on multimodal and intermedial analysis specifically in literary, cultural, and media studies. The concept of the “north” becomes topical in the chapters in presenting approaches to the north as both geographic and symbolic ground for cultural and artistic production. Through investigations into the processes of northern meaning-making, the authors show how specific multimodal and intermedial interactions are not only representative of this cultural condition but generate its new manifestations. The analyses of the cultural production of the north highlight phenomena such as online participation, transmediality, cross-modal description, and intermedial experience. Each chapter examines a specific set or chain of interactions in which experiences of and assumptions about the north become relevant. The role of the north in these interactions varies, remaining in some cases in the background as a given communicative context, or sometimes becoming more topical as a contested ground on which different identities (national, local, peripheral) gain meaning or are challenged. Finally, the north also appears as a realm for imaginative representations sometimes produced based on lived experience, at other times as a more symbolic landscape affectively imbued with otherworldly experience. While language is, at least initially, a central semiotic resource in most of the chapters, the authors deploy the concepts of multimodality and intermediality to explore more complex and refined views of experiences of the north. The outcome of this approach is that it brings into view the plurality of the images and imaginings of the north. In other words, the chapters illustrate the complex ways in which experiences of the north are informed by multimodal and intermedial meaning-making and thus difficult to do justice to by focusing solely on linguistic discourses detached from the embodied experience of the north. Palo, Manderstedt, & Toropainen study how a social media platform becomes a medium to voice a Northern minority culture in various multimodal means. They examine Sami music videos published in the global and digital arena, and the role of language choice in reaching toward different audiences. Their analysis sheds light on the chains of communication that emerge as the observers participate in multilingual interaction with each other and with the artist in the comments of the music video.

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

13

Nikkilä draws attention to how, even in online interaction, geographic location—and the cultural assumptions and perceptions related to it— informs communicative situations and contexts in various ways. Through case examples, Nikkilä shows how situatedness in the north may manifest in online participation of different types and across the boundaries of various media platforms. Miettunen & Jalonen take the regional diversity in Finland as their starting point for studying the new provincial identity of the Finnish North. The region, and the province of Lapland in particular, is conditioned as a frontier of its own kind whose works of popular literature, cinema, and music the authors analyze as resources for a transmedial process that is both representative and generative of Northern exceptionalism in art. Korhonen & Lehtola explore the multimodally rich productions of the famous Sámi artist Nils-Aslak Valkeapää as a career-long project by which the works in different media transmedially converge. One of the key concepts studied is that of “voice,” indicating the possibility of identifying a common element of expression across the boundaries of specific artistic techniques and philosophical views for a holistic vision of art and life. Alarauhio argues how literary texts consisting only of words can be productively analyzed with multimodal and intermedial methods. Employing concepts from either field, the study provides a comparative reading of two epic poems by observing their connections to traditional narrative structures, which elucidates the communicative process of making meaning through multimodal resources and intermedial means in the works of Johan Ludvig Runeberg and Matthew Arnold. Toikkanen focuses his analysis on the intermedial experience of reading H. P. Lovecraft’s “The Dream-Quest of Unknown Kadath,” specifically involving motion and balance. He demonstrates how the story produces an experience of this kind by engaging the reader’s senses in mediating sensory perceptions of an imagined north. While interacting with the media in this way, the reader is brought to the north of Lovecraft’s imagination—a sublime rhetorical space without limits. In further research on shaping the north through multimodal and intermedial interaction, as suggested in the last article, the collective results of the volume might be employed to reach out from studies in the humanities to relevant interests in technology and information studies, among other disciplines. Advancing a research infrastructure with shared

14

J.-P. ALARAUHIO ET AL.

affinities in topical issues and trends such as sustainable user-based experience design and content production will require a joint effort from a variety of academic fields for improved theoretical grounding and extensive support for practical development and implementation. With MIN, we hope to have provided some initial scaffolding for precisely this kind of vision and mission across the disciplines.

References Balestrini, N. W., & Bergmann, I. (Eds.). (2018). Intermediality, life writing, and American studies: Interdisciplinary perspectives. de Gruyter. Bateman, J., Wildfeuer, J., & Hiippala, T. (2017). Multimodality: Foundations, research and analysis–A problem-oriented introduction. Walter de Gruyter GmbH & Co KG. Bezemer, J., & Kress, G. (2015). Multimodality, learning and communication: A social semiotic frame. Routledge. Blommaert, J. (2015). Chronotopes, scales, and complexity in the study of language in society. Annual Review of Anthropology, 44, 105–116. Bolter, J. D., & Grusin, R. (2000). Remediation: Understanding new media. The MIT Press. Bruhn, J. (2016). The intermediality of narrative literature: Medialities matter. Palgrave Pivot. Springer. Bruhn, J., & Schirrmacher, B. (Eds.). (2022). Intermedial studies: An introduction to meaning across media. Routledge. Chartier, D. (2018). What is the imagined north? Presses de l’Université du Québec. Cresswell, T. (2004). Place: An introduction. Blackwell. Drew, P., & Heritage, J. (Eds.). (1992). Talk at work. Cambridge University Press. Dodds, K., & Nuttall, M. (2019). The Arctic: What everyone needs to know. Oxford University Press. Elleström, L. (2017). Bridging the gap between image and metaphor through cross-modal iconicity: An interdisciplinary model. In A. Zirker, M. Bauer, O. Fischer, & C. Ljungberg (Eds.), Dimensions of Iconicity (pp. 167–190). John Benjamins. Elleström, L. (2019). Transmedial narration: Narratives and stories in different media. Palgrave Pivot. Elleström, L. (Ed.). (2021a). Beyond media borders, volume 1: Intermedial relations among multimodal media. Palgrave Macmillan. Elleström, L. (Ed.). (2021b). Beyond media borders, volume 2: Intermedial relations among multimodal media. Palgrave Macmillan.

1

INTRODUCTION: MULTIMODALITY AND INTERMEDIALITY …

15

Forceville, C. (2020). Visual and multimodal communication: Applying the relevance principle. Oxford University Press. Garfinkel, H. (1967). Studies in ethnomethodology. Prentice-Hall. Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32, 1489–1522. Halliday, M. A. K. (1973). Explorations in the functions of language. Arnold. Halliday, M. A. K. (1978). Language as social semiotic. The social interpretation of language and Meaning. Arnold. Hansson, H., Leavenworth, M. L., & Ryall, A. (Eds.). (2020). The Arctic in literature for children and young adults. Routledge. Hennig, R., Jonasson, A.-K., & Degerman, P. (Eds.). (2018). Nordic narratives of nature and the environment: Ecocritical approaches to Northern European literatures and cultures. Rowman & Littlefield. Herzogenrath, B. (Ed.). (2012). Travels in intermedia[lity]: ReBlurring the boundaries. Dartmouth College Press. Jenkins, H. (2008). Convergence culture: Where old and new media collide. NYU Press. Jensen, K. B. (2016). Intermediality. In B. J. Klaus et al. (Eds.), The international encyclopedia of communication theory and philosophy (pp. 3–12). Wiley. Jewitt, C. (Ed.). (2014). The Routledge handbook of multimodal analysis. Routledge. Körber, L.-A., MacKenzie, S., & Stenport A. W. (Eds.). (2017). Arctic environmental modernities: From the age of polar exploration to the era of the anthropocene. Palgrave Macmillan. Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication. Routledge. Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse. The modes and media of contemporary communication. Cappelen. Lefebvre, H. (1991). The production of space. Blackwell. Markides, J., & Forsythe, L. (Eds.). (2019). Research journeys in/to multiple ways of knowing. DIO Press. Meyrowitz, J. (1999). Understandings of Media. Et Cetera, 56(1), 44–52. Mondada, L. (2007). Multimodal resources for turn-taking: Pointing and the emergence of possible next speakers. Discourse Studies, 9(2), 194–225. Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3), 336–366. Moschini, I., & Sindoni, M. G. (Eds.). (2021). Mediation and meaning making in digital environments. Routledge. Norris, S. (2004). Analyzing multimodal interaction. Routledge. Norris, S. (2013). What is a mode? Smell, olfactory perception, and the notion of mode in multimodal mediated theory. Journal Multimodal Communication, 2(2), 155–170.

16

J.-P. ALARAUHIO ET AL.

Norris, S., & Jones R. H. (Eds.). (2005). Discourse in action: Introducing mediated discourse analysis. Routledge. O’Halloran, K., & Smith, B. A. (Eds.). (2011). Multimodal studies: Exploring issues and domains. Routledge. Paasi, A. (1995). The social construction of peripherality: The case of Finland and the Finnish-Russian border area. In H. Eskelinen & F. Snickars (Eds.), Competitive European peripheries (pp. 235–258). Springer. Petersson, S., Johansson C., Holdar, M., & Callahan S. (Eds.). (2018). The power of the in-between: Intermediality as a tool for aesthetic analysis and critical reflection. Stockholm University Press. Rajewsky, I. O. (2005). Intermediality, intertextuality, and remediation: A literary perspective on intermediality. Intermédialités, 6, 43–64. Rippl, G. (Ed.). (2015). Handbook of intermediality: Literature—Image— Sound—Music. de Gruyter. Ryan, M.-L., & Thon, J.-N. (Eds.). (2014). Storyworlds across media: Toward a media-conscious narratology. University of Nebraska Press. Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking for conversation. Language, 50(4), 696–735. https://doi.org/10.1353/lan.1974.0010 Salmose, N., & Elleström, L. (Eds.). (2020). Transmediations: Communication across media borders. Routledge studies in multimodality. Routledge. Scollon, R. (2001). Action and text. Toward an integrated understanding of the place of text in social (inter)action. In R. Wodak & M. Meyer (Eds.), Methods in critical discourse analysis (pp. 139–183). Sage. Scollon, R., & Scollon, S. W. (2004). Nexus analysis: Discourse and the emerging Internet. Routledge. Steinberg, P. E., Tasch, J., & Gerhardt, H. (Eds.). (2015). Contesting the Arctic: Politics and imaginaries in the circumpolar north. Bloomsbury. Sidnell, J., & Stivers, T. (Eds.). (2013). The handbook of conversation analysis. Blackwell. Thon, J.-N. (2016). Transmedial narratology and contemporary media culture. University of Nebraska Press. Tuan, Y.-F. (1975). Place: An experiential perspective. Geographical Review, 65(2), 151–165. https://doi.org/10.2307/213970 Wertsch, J. V. (1998). Mind as action. Oxford University Press. Wolf, W. (2011). (Inter)mediality and the study of literature. CLCWeb: Comparative Literature and Culture, 13(3). https://docs.lib.purdue.edu/clcweb/ vol13/iss3/2/

PART I

Mediating Work and Education

CHAPTER 2

A Design-Driven Approach to Language Teacher Education in the Era of Digitalization Riikka Tumelius, Leena Kuure, and Maritta Riekki

Considering schools as sites for learning, the digital landscape has changed fundamentally. Today’s language learners need to be seen in the light of superdiversity, with their heterogeneous histories in terms of languages and technologies (Blommaert & Rampton, 2011; Chik, 2021). Digital media have contributed to the blurring of boundaries that have traditionally defined our literacy practices (Jones & Hafner, 2012). Broader

R. Tumelius (B) · L. Kuure · M. Riekki Research Unit for Languages and Literature, University of Oulu, Oulu, Finland e-mail: [email protected] L. Kuure e-mail: [email protected] M. Riekki e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_2

19

20

R. TUMELIUS ET AL.

opportunities are available for working with language by utilizing diverse media and modalities. Ways of collaboration and interaction have been changing, allowing diverse forms of participation. From the perspectives of language pedagogies and language teacher education, it is important to reconsider how to approach language learning and teaching in the era of digitalization. The sociocultural ecological perspective views language learning as meaningful interaction and allows the complex relationships between the learner and the environment to be the focus, i.e. social action (Van Lier, 2007). Social action and interaction are seen as multimodal and intermedial (Kress & Van Leeuwen, 2001; Rippl, 2015). Social actors accomplish situated activities multimodally, using a range of modes sequentially or simultaneously in interaction, drawing on the resources of diverse media that contribute in an intermedial way to communicating meanings as an assemblage (Elleström, 2010). Concerning our study’s northern location, northern Finland is often described as peripheral, bearing meanings of distance and dependence, as well as difference from the centre (Paasi, 1995). More “central” areas of economic power to which the periphery is often compared are the capital area in southern Finland or Central Europe. Drawing on the conceptualization of place as a socially constructed centre of meaning, emerging from both individual and socially shared experience (Cresswell, 2004; Tuan, 1975), one can claim that the digitalization of societies and access to global discourses and interaction create tensions in the individual’s experience of the core and periphery as places. Transforming the role of geographical location in social interaction, digitalization gives new affordances for creating new meanings and a sense of place for the university students aiming to become language teachers in the future. As Helles (2013) points out, smartphones and mobile communication technologies more broadly free users’ choice of media from stationary location and follow them across all contexts of daily life, enhancing possibilities of intermedial practices. Digital tools provide affordances for modifying, mixing, and merging tools for adapting to unique circumstances and unique goals, as Jones and Hafner (2012, p. 14) put it. Computer-supported language pedagogy has been offered as a solution for organising formal learning across long distances, e.g., when sharing teaching resources for language classes. It has provided environments for inquiry learning, project-based pedagogy, and multidisciplinary collaboration either online or in hybrid configurations joining participants together

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

21

across time and space. Although considerable effort has been made to reshape language pedagogy, language teacher education does not necessarily prepare future teachers sufficiently to meet the challenges of a digitalized world in flux (Education Policy Outlook in Finland, 2020). There seems to be a general will to promote language education in today’s complex environments. However, this does not easily translate to practice. This study’s rationale arises from these transformations of society due to digitalization, and the consequent changes envisioned in educational practice and social interaction. We will explore the use and productivity of the concepts of multimodality (Kress & Van Leeuwen, 2001) and intermediality in our case (Elleström, 2010; Rippl, 2015), because they are at the core of the sociocultural ecological perspectives of language learning (e.g., Van Lier, 2007) which has yet to become fully established in language education despite the longstanding push of the national curriculum (Finnish National Board of Education, 2016). The chapter examines more closely a course concept referred to as LLT (Language Learning and Technologies), applied annually for two decades in variable forms. The concept draws on a design-driven approach, which involves pre-service teachers of languages designing scenarios and concepts for the distributed online learning of English, and applying one in practice with real participants. The approach provides a setting for preservice teachers to explore how to assume agency as language teachers in the digitalizing world. The designs are expected to align with a sociocultural understanding of language learning that views language and language learning in the frame of interaction and the multimodal ensemble (Bezemer & Jewitt, 2018), because the view considers the affordances of our digitalized environment for language learning (Blommaert & Rampton, 2011; Kramsch & Whiteside, 2008; Kress & Van Leeuwen, 2001; Van Lier, 2007). Working with the sociocultural framework challenges pre-service teachers to reconsider their conceptions of language learning and teaching, which often bear a history of being accustomed to the classroom-based and materials-driven language instruction. Nexus analysis is used as a research approach and theoretical perspective to social action in this study, drawing on the long-term observation and analysis of diverse research materials, exploring both situated and wider scales of discourses and action (Scollon & Scollon, 2004). The study focuses on how pre-service teachers were able to draw on sociocultural perspectives of language learning when advancing their designs.

22

R. TUMELIUS ET AL.

The multimodal and intermedial aspects of the designs were also a focus in analysis because the concepts provide a further angle to the situated meaningful participation on which the sociocultural understanding of language learning relies.

Background to the Study Digitalization at School Digitalization as a development to be considered in education has been included in the Finnish curricula since the knowledge society strategies of research and education in the 1990s. The policy texts such as strategy papers emphasized twenty-first-century skills, multidisciplinarity, and the role of pedagogically informed technology use. Nevertheless, the digitalization of school has been slow, and teacher education approaches vary greatly, entailing very different views about the role of technology in language pedagogy (Starkey, 2020). It seems that pedagogically driven technology use is not yet established on a broad scale in (language) teacher education (Colpaert & Gijsen, 2017; Jalkanen, 2015; Karamifar et al., 2019). Challenges of Change In the Finnish context of this research, the high digitalization of society has not been reflected as a significant transition towards digitality in learning situations (Blin & Jalkanen, 2014; Kaarakainen & Saikkonen, 2021). In the field of language education, possible reasons include narrow understandings of language and language learning (Blin & Jalkanen, 2014; Jalkanen, 2015). Pre-service teachers’ assumptions about their future teaching practices often start from text- and textbook-driven activities that echo traditional approaches to language and language learning rather than approaches acknowledging participatory practices (ibid.). Textbooks have their place in education, creating continuity and support for diverse learners (Moate, 2021). When pedagogical activities are transferred from one environment to another, intermedial relationships in the designs should be reconsidered carefully so that the potential of the new settings for language learning can be recruited fruitfully (Elleström, 2010; Rippl, 2015). For example, gamification in language pedagogy often

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

23

repeats practices originally based in classrooms, thus overlooking affordances for promoting learners’ agency (Zheng & Newgarden, 2012). Widening pedagogical attention from classroom interactions to hybrid spaces integrating material and digital dimensions requires language teachers to reconsider their teacher identities and practices (Guichon & Wigham, 2016; Tumelius & Kuure, 2020). To understand the challenges of digitalization in pedagogical change, dimensions beyond actual teaching situations such as the support and maintenance of equipment also need consideration. Pre-service teachers already need experience of teaching in complex situations that require simultaneous orchestration of diverse activities, which is not included in initial teacher education (Tumelius & Kuure, 2020). Pedagogical Renewal, Professional Vision, and Agency Pedagogical renewal involves a complex interplay of institutional, professional, and informal discourses, which may either hinder or enable pedagogical change (Ensor et al., 2017). Opportunities for real-life experimentation with learning approaches and guiding activities with learners online allow pre-service teachers to explore and question being language teachers in new kinds of settings (Anderson & Shattuck, 2012; Booth et al., 2017; Tumelius & Kuure, 2020). There is a need to strengthen language teachers’ and pre-service teachers’ agency in changing prevailing practices (Ahearn, 2001; Edwards, 2007; Guichon & Wigham, 2016; Kubanyiova, 2020). Agency involves responsiveness to others’ perspectives in offering and asking for support, which advances collaboration towards a shared goal (Edwards, 2007; Kubanyiova, 2020). This involves assuming professional responsibility, a disposition to act with agency (Barahona, 2020), which means switching perspectives from being students to “real” teachers in charge. Design as a Tool for Change Design-driven approaches have been found fruitful in envisioning the transition from current to future practices, providing opportunities for strengthening pre-service teachers’ agency (Edwards, 2007; Kubanyiova, 2020). In the design process, the personal and design goals being negotiated are often in conflict, and they should be integrated, not only with the learners in mind but also all the participants in question (Colpaert, 2010).

24

R. TUMELIUS ET AL.

In discussing different approaches to design in language pedagogy, Blin (2016) considers alternatives based on cultural-historical activity theory (Engeström, 2001) as fruitful in advancing professional growth as they trigger negotiation for meanings concerning teachers’ reconceptualization of their approach to language, language learning, and language teaching in technology-rich language learning environments (see also Blin & Jalkanen, 2014). Expanding and deepening the understanding of the object of action at hand requires special attention and provides opportunities for agency growth (Edwards, 2007; Kuure et al., 2016; Tumelius & Kuure, 2020). Although research on design-driven courses in the context of language teacher education is scarce, a body of earlier research highlights the complex entanglement of aspects contributing to pre-service teachers’ sensemaking about language pedagogy and their professional future. Among other things, challenges related to change can be attributed to the wide network of actors that affect how learning is perceived, and how people behave in a learning situation. These actors include parents, teachers, peers, curriculum designers, technology developers, and societal decision-makers whose voices may echo in pedagogical situations through various policies, strategies, and belief systems grown across a long-term timespan (Koivistoinen, 2016; Riekki, 2016; Tumelius & Kuure, 2020). Language teachers’ situated actions when dealing with multiple overlapping activities in hybrid spaces have been conceptualized as orchestration (Dillenbourg, 2008; Guichon, 2010; Tumelius & Kuure, 2020). Design-oriented collaboration in orchestrating schoolchildren’s online work supports the growth of pre-service teachers’ professional vision (Goodwin, 1994), relational agency (Edwards, 2007), and professional responsibility (Barahona, 2020). During design and orchestration, preservice teachers balance familiar and new pedagogies, while managing diverse attention spaces, to appropriate new practices in language teaching (Koivistoinen et al., 2016; Kuure, 2018; Tapio, 2013; Tumelius & Kuure, 2020). Mediated Discourse Perspective to Multimodality and Intermediality A mediated discourse perspective binds the perspectives of participants’ experiences and accustomed practices (historical bodies), their mutual relations (interaction order), and discourses circulating in place (Scollon

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

25

& Scollon, 2004). Thus, situated action is viewed in its historical perspective reaching broader, even societal spheres as in design approaches drawing on cultural-historical activity theory (see Blin, 2016, above). The approach entails a mediated view of social action as accomplished through mediational means (Scollon & Scollon, 2004; Wertsch, 1998). Diverse social semiotic resources, modes, can be used multimodally (sequentially or simultaneously) as mediational means in (inter)action and discourse (Kress & Van Leeuwen, 2001). Modes are not a priori categories but are produced, learned, and acquired in use by social actors for their use (Norris, 2013). Different media and material resources such as tools may be used in the production of semiotic products and events (Kress & Van Leeuwen, 2001). Media are understood here as both a language and an environment for people to engage in meaning making, because media are also products of human action (Meyrowitz, 1999; Norris, 2013). The interest in the intermedial perspective is to examine the relationship between different media in mediating (inter)action (Rippl, 2015). Rippl (2015) points out that conceptions about the nature of the linkage vary from being distinct to merging. Rippl (2015) refers to Elleström’s (2010) interpretation of all media as mixed and complementary in different ways (see also Jones & Hafner, 2012).

Research Approach, Methodology, and Materials Nexus Analysis as a Research Strategy The study asks how the pre-service teachers on the design-driven university course were able to build their language learning designs on sociocultural perspectives (Kramsch & Whiteside, 2008; Van Lier, 2007). The intermedial and multimodal aspects of the designs were also a focus. However, these were only relevant in this analysis, because the terms were not introduced to the pre-service teachers. The research approach was based on nexus analysis. It relies on the conception of social action as an intersection of historical bodies of participants, their mutual interaction orders , and discourses in place (Scollon & Scollon, 2004). People bring to encounters their own experiences, beliefs, and understandings, and they position themselves and others reciprocally while drawing on diverse discourses from the past, but also projecting new discourses and actions (Scollon & Scollon, 2004).

26

R. TUMELIUS ET AL.

Nexus analysis involves three main tasks or activities: engaging, navigating, and changing the nexus of practice (Scollon & Scollon, 2004). In the engaging phase, researchers begin the study by considering their own zones of identification within the nexus of practice under study (Scollon & Scollon, 2004). We authors have our backgrounds in language teaching and language teacher education in the north of Finland, which also provides the setting for the study. Moreover, we have conducted research on the changing scenarios of language pedagogy and language teacher education by utilizing research materials from various iterations of the university course under scrutiny in this chapter. One of us has led all the iterations as the teacher responsible. In the engaging phase, the researchers explore the nexus of practice of interest with open eyes, asking what is going on, examining how it is talked about in public, and who the key participants are, and gathering research materials of different kinds (Scollon & Scollon, 2004). After the engaging phase, nexus analysis entails navigating the diverse data more deeply, using methods suitable for the topic and materials to hand. Change can be an intentional aspect of research, but in considering the researchers as participants, their presence already somehow affects the interaction order of the nexus of practice (Scollon & Scollon, 2004). Research Participants and Materials The participants in the study were 16 university students with English language as their major subject at master’s level. Most were studying to become language teachers, although some were orienting towards careers in translation or other fields of language work. The students were taking the course because they were expecting to gain hands-on experience of the pedagogically informed use of current technologies. As the language teacher profession was a possible career path, all the participating students are called pre-service teachers in this study. Of the several annual course iterations of Language Learning and Technologies (LLT), one was chosen for closer examination, because its design process provided particularly rich resources for the analysis of multimodality and intermediality in the pre-service teachers’ designs. Materials (Table 2.1) were gathered from the course workspace (LLT) and the school workspace created by the pre-service teachers (Coffee Shop). The materials foreground participants’ understandings elaborated individually and in teams, the evolving designs from the ideation phase to the

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

Table 2.1 Research materials

27

Course workspace online (LLT) Scope of materials Source of materials 5 text documents Project management memos 144 entries Discussions 6 slideshows, 4 texts Background packages 48 design ideas Planning documents 8 jpg images Illustrations 6 team logs Team process 6 packages Coffee Shop designs 13 texts Reflection papers School project workspace online (Coffee Shop) Source of materials Scope of materials Visual interface 21 images Activities 33 pages Teacher’s guide 8 documents Learner’s guide 1 document Discussion list 4 starting entries Chat 1 object

final prototype and post-project reflections. The research complies with the national ethical guidelines for research integrity, as well as data privacy and protection (Finnish National Board on Research Integrity, 2019). The research participants gave their informed consent, and the research materials were anonymized (e.g., pseudonyms for names). The Design Process—The Emerging Coffee Shop The LLT course advanced through a design process (see Table 2.2) that proceeded from background work and ideating to creating a specific design concept and prototype or example project, and finally to evaluating the product and design process (Henriksen et al., 2020). The work started with an orientation about what was coming through background research on five topics on which the pre-service teachers focused to explore the participants’ lifeworld (see Table 2.2). The work aimed to give the course participants tools for critically considering their own experiences and understandings of language learning and teaching, as well as for appropriating the sociocultural perspective on learning (e.g., Kramsch & Whiteside, 2008; Van Lier, 2007). The concrete result of the orientation phase was an information package about each topic (e.g., links, short texts, articles, pictures, and video) that provided the pre-service teachers with resources for designing the language learning

28

R. TUMELIUS ET AL.

Table 2.2 The design process on the course Design stage

Themes in focus

Orientation

Background research (language and literacy learning; technologies of today and tomorrow; pedagogical approaches; different learners; playfulness) Delineating design values Possible language learning activities and projects Project shortlist Final decision (Coffee Shop) Fine-tuning the Coffee Shop concept (age-free, based on shared interests) Table topics (Books; Video CV; Future; Games and leisure; Travel; Restaurant) Synchronous and asynchronous activities with participants from school and home Playfulness in action Project management and teamwork Lessons learnt from designing the online project Post-project reflection papers

Developing ideas

Getting ready

Online project

Evaluation and wrap up

project(s). The design values were defined based on the background research: playfulness; learner-centredness; and scalability in terms of age, language skills, and interests. The orientation stage was followed by the development of ideas. Using the background work and orientation discussions as a basis, the preservice teachers started brainstorming possible language learning projects. From 48 suggestions, a shortlist was elaborated, and the participants arrived at the concept of Coffee Shop for further development. In the next stage, getting ready, the pre-service teachers were split into teams, each responsible for the activities of one Coffee Shop table. When the concept had been advanced into a prototype built on a learning platform as an online project, the participants used their own contacts to find people— children or adults—to join the Coffee Shop to experiment with the design. The evaluation of the activities based on real-life testing already led to fine-tuning the designs, but at the end of the course, an evaluation of the whole process took place as a wrap-up for the course.

Findings In the following, the findings are illustrated in more detail in response to the question of how the pre-service teachers were able to draw

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

29

on sociocultural perspectives on language learning during the handson design-driven university course (Kramsch & Whiteside, 2008; Van Lier, 2007). Moreover, the designs are examined considering intermediality (Elleström, 2010; Rippl, 2015) and multimodality (Kress & Van Leeuwen, 2001), because they provide a further angle to the situated meaningful participation on which the sociocultural understanding of language learning relies. In the analysis, the three-dimensional nature of social action is introduced as a heuristic lens through the concepts of historical body, interaction order, and discourses in place (Scollon & Scollon, 2004). Pedagogical, Intermedial, and Multimodal Aspects in the Pre-Service Teachers’ Designs for Learning The Coffee Shop project included six tables with different activities: Books; Video CV; Future; Games and Leisure; Travel; and Restaurant. The pre-service teachers worked in small teams to plan and carry out the activities. The activities in the tables included creating an online magazine (Travel), a structured online chat discussion (Restaurant), a word quiz via online chat (Games and Leisure), creating a CV and a video CV (Video CV), discussion in an online discussion forum (Books), and compiling a selection of digital artefacts and documents (Future). Pre-service teachers’ designs and their discussions and reflections revealed some tensions arising from their historical bodies in two ways. On the one hand, the pre-service teachers’ strong expertise in the use of digital resources in their daily life gave them ground for ideating fruitful scenarios based on a situated view of language use. On the other hand, their own past experiences of language learning in traditional classroom settings posed challenges in detaching from the textbook-driven practices that have endured until today. Being aware of the diversity of digital affordances, they considered technology use important for a teacher’s professionalism. However, their experiences of their own schooldays or pedagogical studies did not seem to provide a positive picture in terms of student-driven pedagogy or technology use for pedagogical purposes (see Example 1). (1) I’m rather interested in the new technology aspects of learning and teaching English as I’ve learned the language mainly from sources outside the school system and the classes didn’t really have anything to offer me, going through the boring assignments and other stuff that seemed very old fashioned and pointless. (Ville, orientation discussions)

30

R. TUMELIUS ET AL.

Ville’s comment (1) illustrates the current situation of people being able to use and learn English in their interest networks beyond school, which may reduce their motivation for engaging in classroom work planned on different assumptions of learners’ knowledge and skills. For example, the teams drew on the use of literature, television series, films, the Internet, blogs, and newspaper platforms as resources for the table activities that involved or supported intermedial practices. An imaginary coffee shop environment was thus created for the learners to enter and act in, drawing on diverse media. Although intermediality was visible in the designs, their final versions, however, showed a lack of understanding of the intermedial relationships in question: which modes were central for the medium, and how to best utilize them in creating affordances for language learning. In most cases, the practical executions of creative and pedagogically viable ideas were toned down towards familiar classroom-based activities assigning agency to the teacher rather than the learners. The typical interaction orders were foregrounded—the instructions were directed to the teacher, and team members’ exchanges indicated assumptions about the teacher keeping tight control of learners’ activities and the time required. This also indicates how the pre-service teachers used the typical 45-minute language lesson and its flow as a reference point for their assessment of how their designs fitted their imagined scene of language teaching. Writing, texts, and the ideal of correct language were often highlighted, while a more complex view of distributed interaction and supporting learners’ digital practices was less visible. However, the team creating the activities for the Travel table developed a design concept that provided a wide range of affordances for multimodal and intermedial practices. They created a Travel magazine on an international platform for e-magazines. The learners were able to write stories and create photo albums on their trips, combining the narratives with images, applying the genre of the e-magazine. The design of the team was thus authentic and meaningful for the participants and potential users, approximating the real-life situations of language use. The pre-service teachers’ reflections generally suggested that in their teacher education studies, sufficient knowledge and hands-on experience of technology for the service of pedagogy was not provided—rather, the focus was technology-driven. At the onset of the course, the pre-service teachers explicitly expressed their will to move away from the tradition of textbook-based and teacher-led language learning, and wished that digital technologies could provide them with new perspectives to achieve the

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

31

aim. However, they struggled to make sense of the modal affordances of different media, which modes become foregrounded in a particular medium, and how the change of medium affects the array of affordances for language learning. The challenges of change were visible in the preservice teachers’ post-project reflections (see Examples 2–4). (2) The project that I ended up taking part in was a fairly simple question/answer (in a fixed environment) pair work, but the trick was that it would happen in a chat room […] It is important that the teacher actively supervises the exercise in the chat room and gives guidance when needed. (Sophia, reflections on the Restaurant table) Sophia (2) describes her project from a non-agentive stance, portraying it as something pre-existing in which she has played no role of any kind (I ended up taking part ). Her reflection about the role of the teacher presents a traditional view of the teacher as overseeing the situation (the teacher actively supervises ). The example also illustrates that the team had not considered the chat as a medium for interaction of its own but transposed face-to-face interaction to the chat itself. The task was a structured dialogue of ordering food at a restaurant with translations to follow, reproducing a teacher-led activity familiar from many textbooks. Thus, the design did not utilize the potential of the online medium, but the task remained detached from a communicative situation in real life. Intermediality was here rather a matter of doing separate things through separate activities than drawing on different media as complementary resources in aiming for a goal (Elleström, 2010). Examples (3) and (4) illustrate two different perspectives on the pedagogical situations and the teacher’s role. (3) Still, as I said before, the most important thing is that the pupils got something useful and fun to do while studying as a chance of pace for the usual print paper studying. (Jyrki, reflections on the Future table) (4) The […] project that I participated in had the advantage of encouraging independent work of the pupils. We tried to create the environment so that the pupils would not need much teacher assistance to engage in the activities. It also had tasks for both advanced and less skilled learners, and activities for different age groups and interests,

32

R. TUMELIUS ET AL.

which makes it valuable from the point of view of differentiation. For pupils interested in literature it should be quite motivating, and there are a lot of pictures to make it more visual and fun to use. (Anna, reflections on the Books table) Jyrki (3) portrays language learning as usually print-oriented, while “fun” is only an occasional aspect of study provided by the teacher. Anna (4), on the other hand, sees the learner as having more agency in the learning situation—having an opportunity to engage in the activities without a teacher’s assistance. She highlights flexibility in planning a learning scenario suitable for different kinds of people. She also mentions resources that would add to the multimodality of the materials, although the example does not reveal how the task would be designed, and whether it would involve dealing with intermedial relationships or multimodality as a focus of learning. The pre-service teachers had a strong interest in technology as a source of change in language pedagogy. However, in realising their design ideas, their narrow understandings of language and language learning came to the surface together with their innovative ideas based on socioculturally driven language pedagogies. Some pre-service teachers on the course claimed not to be technologically oriented, which they mentioned as a rationale for taking the course in preparation for their future careers as teachers. Again, some participants were technology-savvy but had no aspiration of a teaching career. Generally, all believed that language teaching needed to reflect the current world and people’s ways of interacting. Different voices about learning and teaching prevailed among the pre-service teachers. Nevertheless, in aligning with the course aims, the pre-service teachers highlighted their need to consider language learning in a new way to be able to use new technologies in a pedagogically sound way. Many of the teams had considered multimodality in their presentations of their activities and instructions: different typefaces, backgrounds, font colours, and images were used, and the texts were accessibly organized, e.g., creating step-by-step procedures. On the basis of participatory observation, we can say that the pre-service teachers also discussed alternatives for presentation from the perspectives of their users and considered intermedial relationships in making choices about the media to be used. However, from the pedagogical perspective, these ponderings were minimally utilized.

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

33

Finding a New Perspective for Language Learning and Teaching The pre-service teachers expressed a need to learn to ideate and even innovate new kinds of approach towards language learning. In their reflections, the pre-service teachers wrote that the strength of the course was the inventive participants, and that they appreciated the design-driven collaborative work on the course. Many courses can involve group work, which does not automatically mean collaborative teamwork. Although the course set-up advanced such a collaboration-supporting and balanced interaction order, which was also the experience of most participants, there were also some differing voices. Jyrki, for example, writes in his final reflections about his experiences of confusion in the working project (see Example 5). (5) Looking back, it seemed quite promising at first during the miniprojects, but when we started to work on the product of the course, everything fell apart in my opinion. I, personally, failed to see the main connection between the mini-projects and the main project, other than to give an idea about how we were going to be working during the course. And when we were brainstorming for project ideas, I was slightly confused when the purpose was to come up with a couple of ideas for the project and ended up with seven separate miniprojects. I know the intention was good, not to exclude any ideas that seemed doable to most people, but I think the potential of the group could have been utilized better with just a couple of projects. (Jyrki, reflections on the Future table and the Coffee Shop as a whole) The concept of the Coffee Shop (the main project in Jyrki’s account) with different tables was an idea developed with all the participants in a workshop session. This allowed people of different ages and language competences to engage in the activities. In other words, the aim was not to prepare activities for schoolchildren at a particular class-level only. This goal seems to have escaped Jyrki, which then caused confusion. His final comment about his preference for doing only a couple of projects refers to his initial understanding voiced at the beginning of the course about the hands-on project being directed at a group of same-age pupils in the same class. The course set-up afforded for collaboration and pre-service teachers was able to assume a stronger agency when the pedagogical designs were

34

R. TUMELIUS ET AL.

created. Nevertheless, their design work showed that they were not necessarily able to consider their designs from the learner’s perspective (see Example 6). (6) On higher level the exercise can be more abstract and perhaps then slightly more meaningful as the students have to improvise and use the Internet for finding and applying information in their discussion, but for lower grades the amount of trouble a teacher has to go through in order to prepare the exercise does not seem worth it (Ville, reflections on the Restaurant table) The pre-service teachers had a strong focus on what the teacher’s role would be in the activities, and how the teacher would guide the learners. In Example 6, Ville ponders on the teacher’s role in a way which demonstrates a narrow understanding of the learners’ agency. In the case of older pupils, he considers more demanding language learning activities in the light of information search and use heavily structured by the teacher, while he does not have great expectations about the younger learners’ potential as independent language users. The teacher-centred way of assessing the designs shows how much the pre-service teachers relied on their past experiences of language learning as a print-oriented and language-focused activity, instead of drawing on the sociocultural view supporting learners’ meaningful interaction, and trusting their agency as the starting point for their designs. A strong focus in thinking about how the language was taught instead of how it was learned was visible. Nevertheless, in their post-project reflections, the pre-service teachers showed their appreciation of the hands-on collaborative experience of designing language learning. They were able to critically evaluate the projects and envision future development in the light of the sociocultural framework. They also drew on multimodal meaning making and considered intermediality in their pedagogical designs, though not with conscious reflection using those terms. Their reflections suggested that they were able to see themselves as active and independent agents of language pedagogy, which points more to their empowering stance towards their teacherhood than their submission to accustomed practices. The design-driven course proved a fruitful framework for continuous negotiations for meanings among the pre-service teachers and course teachers. These negotiations concerned the participants’ understandings

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

35

of language, language learning, pedagogical approaches, and the role of technology in teaching and this project. The design process allowed these topics to be rendered accessible for sensemaking, but it will be important to develop this aspect of the design-driven course for use as a conscious tool in language teacher education in the future.

Discussion and Conclusion This study explored through a nexus analytical research strategy (Scollon & Scollon, 2004) how pre-service teachers on a master’s-level designdriven course produced their design of the Coffee Shop concept and prototype. The analysis considered intermediality and multimodality in the context of sociocultural-ecological perspectives on language pedagogies (Elleström, 2010; Kramsch & Whiteside, 2008; Kress & Van Leeuwen, 2001; Rippl, 2015). The topic of the study is important as preservice teachers becoming language teachers in northern Finland need to activate their potential to act as change agents in reshaping language pedagogies that draw on the current flows in society in the development of technology. The study demonstrated how the varied conceptions and understandings of pre-service teachers shaped by their historical bodies came into play when designing language learning in digitalizing environments. The university course on which the design project was advanced provided an important, safe, and experiential site of engagement for the participants to ideate, innovate, design, and evaluate their work. This was afforded by a mutually collaborative atmosphere through a balanced interaction order. The pre-service teachers were able to become “project owners” and gain relational agency through joint meaning making and reflections throughout the course (Barahona, 2020; Edwards, 2007). The study shed light on discourses in place in the public forums, on the one hand, and in the context of the university course in the hands-on design project, on the other. These discourses expressed tensions between the traditional and the new in terms of how language learning was viewed in the context of the digitalizing society, and how the tradition persisted even in the flux of modern practices and discourses. Considering digitalization and changing environments in designing language learning requires sensitivity to the complexity of interaction and the fluctuating role of language. Viewing the research materials

36

R. TUMELIUS ET AL.

from the perspectives of multimodality and intermediality showed that the pre-service teachers’ pedagogical designs were balanced between traditional and modern conceptions of language learning. The course design entailing a design-driven approach promoted the students’ sense of ownership, which became visible in the students’ actions and interactions in advancing the project. Moreover, the students’ reflections indicated that the university course had potential as a forum for promoting change. The pre-service teachers insightfully analysed their designs and their potential for future use, and showed emerging understandings of language learning as a multimodal and intermedial phenomenon. This portrays an important perspective switch of seeing language and language learning in terms of dealing with superdiversity (Blommaert & Rampton, 2011). Nevertheless, change is slow, and as varied understandings and practices compete in the field, newly graduated teachers will have to make sense of the circulating discourses in place and shape their own approaches as they start working. Our findings show that pre-service teachers recognize a need for change in the accustomed practices, and expect teacher education to provide the newest tools and ideas for language teaching. However, their expectations may not be met. Pre-service teachers would like more guidance and experiences in learning to use modern technologies in the service of language pedagogy. The course aims to strengthen pre-service teachers’ agency so that they can make use of the affordances of the changing technological landscape in their future work. The study highlights the importance of language professionals’ agency in detaching themselves from familiar practices and dealing with uncertainty, given that renewing language pedagogies as a design-driven stance to one’s work entails an envisioning of the future. The pre-service teachers in the study become language teachers in a northern locality with its global flows through people’s mobility, which digitalization enhances even further. From the perspective of our northern circumstances, with long distances and the fragility of school networks due to great differences in population density, it is particularly important to educate (language) teachers who have the competence and agency to be creative designers of their everyday pedagogical solutions on a long-term basis. Language teacher education can benefit from applying the hands-on, design-driven approach of the course concept, because it provides a platform to explicitly discuss the principles of designing language learning

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

37

and to consider what kinds of issues affect the practice of language education. The iterative process involves constant reflection on and evaluation of the designs as the pre-service teachers proceed with the work, which promotes a growing awareness of the choices made to create the designs. The analysis showed that the concepts of multimodality and intermediality were useful in evaluating designs for language learning. As the pre-service teachers gain awareness of multimodality and intermediality in interaction, their sensitivity to recognizing superdiversity and complexity as a natural part of our technology-rich life also develops. This strengthens ground for their emerging professionalism in becoming designers of language pedagogies for the future.

Funding This work was supported by the Finnish Cultural Foundation’s North Ostrobothnia Regional Fund, The Scholarship Fund of the University of Oulu, and Oskar Öflunds Stiftelse.

References Ahearn, L. M. (2001). Language and agency. Annual Review of Anthropology, 30, 109–137. https://doi.org/10.1146/annurev.anthro.30.1.109 Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. https://doi.org/10.3102/0013189X11428813 Barahona, M. (2020). Developing and enacting professional pedagogical responsibility: A CHAT perspective. The European Journal of Applied Linguistics and TEFL, 9(2), 131–152. Bezemer, J., & Jewitt, C. (2018). Multimodality: A guide for linguists. In L. Litosseliti (Ed.), Research methods in linguistics (2nd ed., pp. 281–304). Bloomsbury. Blin, F. (2016). Towards an ‘ecological’ CALL theory: Theoretical perspectives and their instantiation in CALL research and practice. In F. Farr & L. Murray (Eds.), The Routledge handbook of language learning and technology (pp. 39– 54). Routledge. Blin, F., & Jalkanen, J. (2014). Designing for language learning: Agency and languaging in hybrid environments. Apples—Journal of Applied Language Studies, 8(1), 140–170. https://apples.journal.fi/article/view/97861 Blommaert, J., & Rampton, B. (2011). Language and superdiversity. Diversities, 13(2), 1–22. http://www.unesco.org/shs/diversities/vol13/issue2/art1

38

R. TUMELIUS ET AL.

Booth, P., Guinmard, I., & Lloyd, E. (2017). The perceptions of a situated learning experience mediated by novice teachers’ autonomy. The EUROCALL Review, 25(1), 76–91. https://doi.org/10.4995/eurocall.2017.7081 Chik, A. (2021, August 15–21). Sidelining learner autonomy: An Australian case study [Conference presentation]. AILA 2021 World Congress, Groningen, the Netherlands. Colpaert, J. (2010). Elicitation of language learners’ personal goals as design concepts. Innovation in Language Learning and Teaching, 4(3), 259–274. https://doi.org/10.1080/17501229.2010.513447 Colpaert, J., & Gijsen, L. (2017). Ontological specification of telecollaborative tasks in language teaching. In C. Ludwig & K. Van de Poel (Eds.), Collaborative learning and new media. New insights into an evolving field (pp. 23–39). Peter Lang. Cresswell, T. (2004). Place: An introduction. Blackwell. Dillenbourg, P. (2008). Integrating technologies into educational ecosystems. Distance Education, 29(2), 127–140. https://doi.org/10.1080/015879108 02154939 Education Policy Outlook in Finland. (2020). OECD education policy perspectives, 14. OECD Publishing. https://doi.org/10.1787/f162c72b-en Edwards, A. (2007). Relational agency in professional practice: A CHAT analysis. An International Journal of Human Activity Theory, 1(1), 1–17. Elleström, L. (2010). The modalities of media: A model for understanding intermedial relations. In L. Elleström (Ed.), Media borders, multimodality and intermediality (pp. 11–48). Palgrave Macmillan. Engeström, Y. (2001). Expansive learning at work: Toward an activity theoretical reconceptualization. Journal of Education and Work, 14(1), 133–156. https://doi.org/10.1080/13639080020028747 Ensor, S., Kleban, M., & Rodriguez, C. (2017). Telecollaboration: Foreign language teachers (re)defining their role. Alsic—Language learning and information and communication systems, 20(2). https://doi.org/10.4000/alsic. 3140 Finnish National Board of Education. (2016). National core curriculum for basic education 2014. Finnish National Board of Education. Finnish National Board on Research Integrity. (2019). The ethical principles of research with human participants and ethical review in the human sciences in Finland, 3. Publications of the Finnish National Board on Research Integrity. Goodwin, C. (1994). Professional vision. American Anthropologist, 96(3), 606– 633. https://doi.org/10.1525/aa.1994.96.3.02a00100 Guichon, N. (2010). Preparatory study for the design of a desktop videoconferencing platform for synchronous language teaching. Computer Assisted Language Learning, 23(2), 169–182. https://doi.org/10.1080/095882210 03666255

2

A DESIGN-DRIVEN APPROACH TO LANGUAGE TEACHER …

39

Guichon, N., & Wigham, C. (2016). A semiotic perspective on webconferencingsupported language teaching. ReCALL, 28(1), 62–82. https://doi.org/10. 1017/S0958344015000178 Helles, R. (2013). Mobile communication and intermediality. Mobile Media & Communication, 1(1), 14–19. https://doi.org/10.1177/205015791245 9496 Henriksen, D., Gretter, S., & Richardson, C. (2020). Design thinking and the practicing teacher: Addressing problems of practice in teacher education. Teaching Education, 31(2), 209–229. https://doi.org/10.1080/10476210. 2018.1531841 Jalkanen, J. (2015). Future language teachers’ pedagogical landscapes during their subject studies. Nordic Journal of Digital Literacy, 10(2), 84–101. https://doi.org/10.18261/ISSN1891-943X-2015-02-03 Jones, R. H., & Hafner, C. A. (2012). Understanding digital literacies: A practical introduction. Routledge. Kaarakainen, M.-T., & Saikkonen, L. (2021). Multilevel analysis of the educational use of technology: Quantity and versatility of digital technology usage in Finnish basic education schools. Journal of Computer Assisted Learning, 37 (4), 953–965. https://doi.org/10.1111/jcal.12534 Karamifar, B., Germain-Rutherford, A., Heiser, S., Emke, S., Hopkins, J., Ernest, P., Stickler, U., & Hampel, R. (2019). Language teachers and their trajectories across technology-enhanced language teaching: Needs and beliefs of ESL/EFL teachers. TESL Canada Journal, 36(3), 55–81. https://doi.org/ 10.18806/tesl.v36i3.1321 Koivistoinen, H. (2016). Changing understandings of language learning and teaching: The perspectives of pupils, parents and future language teachers. Acta Universitatis Ouluensis, B Humaniora, 147 [Doctoral dissertation, University of Oulu] http://urn.fi/urn:isbn:9789526214122 Koivistoinen, H., Kuure, L., & Tapio, E. (2016). Appropriating a new language learning approach: Processes of resemiotisation. Apples—Journal for Applied Language Studies, 10(2), 105–117. https://doi.org/10.17011/apples/urn. 201612145091 Kramsch, C., & Whiteside, A. (2008). Language ecology in multilingual settings. Towards a theory of symbolic competence. Applied Linguistics, 29(4), 645– 671. https://doi.org/10.1093/applin/amn022 Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. Arnold. Kubanyiova, M. (2020). Language teacher education in the age of ambiguity: Educating responsive meaning makers in the world. Language Teaching Research, 24(1), 49–59. https://doi.org/10.1177/1362168818777533 Kuure, L. (2018). Language students designing a learning project for children: A matter of managing multiple attention spaces. In G. Murray & T.

40

R. TUMELIUS ET AL.

Lamb (Eds.), Space, place and autonomy in language learning (pp. 113–127). Routledge. Kuure, L., Molin-Juustila, T., Keisanen, T., Riekki, M., Iivari, N., & Kinnula, M. (2016). Switching perspectives: From a language teacher to a designer of language learning with new technologies. Computer Assisted Language Learning, 29(5), 925–941. https://doi.org/10.1080/09588221.2015.106 8815 Meyrowitz, J. (1999). Understandings of media. ETC: A Review of General Semantics, 56(1), 44–52. http://www.jstor.org/stable/42579860 Norris, S. (2013). What is a mode? Smell, olfactory perception, and the notion of mode in multimodal mediated theory. Multimodal Communication, 2(2), 155–169. https://doi.org/10.1515/mc-2013-0008 Moate, J. (2021). Seeking understanding of the textbook-based character of Finnish education. Journal of Education for Teaching, 47 (3), 353–365. https://doi.org/10.1080/02607476.2021.1896341 Paasi, A. (1995). The social construction of peripherality: The case of Finland and the Finnish-Russian border area. In H. Eskelinen & F. Snickars (Eds.), Competitive European peripheries (pp. 235–258). Springer. Riekki, M. (2016). Navigating change: Nexus-analytic explorations in the field of foreign language education. Acta Universitatis Ouluensis, B Humaniora, 146 [Doctoral dissertation, University of Oulu]. http://urn.fi/urn:isbn:978952 6214108 Rippl, G. (2015). Introduction. In G. Rippl (Ed.), Handbook of intermediality: Literature – Image – Sound – Music (pp. 1–31). de Gruyter. Scollon, R., & Scollon, S. W. (2004). Nexus analysis: Discourse and the emerging Internet. Routledge. Starkey, L. (2020). A review of research exploring teacher preparation for the digital age. Cambridge Journal of Education, 50(1), 37–56. https://doi.org/ 10.1080/0305764X.2019.1625867 Tapio, E. (2013). A nexus analysis of English in the everyday life of FinSL signers— a multimodal view on interaction [Doctoral dissertation, University of Oulu]. https://www.finna.fi/Record/jykdok.1437269 Tuan, Y.-F. (1975). Place: An experiential perspective. Geographical Review, 65(2), 151–165. https://doi.org/10.2307/213970 Tumelius, R., & Kuure, L. (2020). Towards a shared vision of language, language learning and a school project in emergence. Classroom Discourse. https://doi. org/10.1080/19463014.2020.1808495 Van Lier, L. (2007). Action-based teaching, autonomy and identity. Innovation in Language Learning and Teaching, 1(1), 46–65. https://doi.org/10.2167/ illt42.0 Wertsch, J. V. (1998). Mind as action. Oxford University Press. Zheng, D., & Newgarden, K. (2012). Rethinking language learning: Virtual worlds as a catalyst for change. International Journal of Learning and Media, 3(2), 13–36. https://doi.org/10.1162/IJLM_a_00067

CHAPTER 3

Bad News Delivery as an Interactional Context for Constructing Professional Identities and Social Relations: Multimodal Approach Tiina Räisänen and Tuire Oittinen

Introduction News delivery sequences (NDS) are collaborative interactional events that involve the speaker informing the recipient(s) about something that is oriented to by her or him as news (Maynard, 1997, 2003). News deliveries are delicate moments in which knowledge discrepancies and situated identities come into play and need to be negotiated. Having to deliver

T. Räisänen (B) · T. Oittinen Research Unit for Languages and Literature, University of Oulu, Oulu, Finland e-mail: [email protected] T. Oittinen e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_3

41

42

T. RÄISÄNEN AND T. OITTINEN

bad news in particular constitutes a practical problem because it is disaffiliative by nature and requires orienting to the subtleties of the situation via additional interactional work. In workplace contexts, involving professionals from different organizational levels who enact their job-specific roles with certain entitlements, bad news delivery makes a particularly relevant context for negotiating professional identities and relationships. However, it is yet unknown how these moments of NDS are organized in situ, and how they shape and are shaped by the contingencies of the ongoing interaction, and participants’ verbal and embodied conduct in the sociomaterial environment. Furthermore, while earlier research exists on the construction of professional identities at work, there are still only a few studies on its multimodal nature (e.g., Svennevig, 2011; Van de Mieroop et al., 2020). This chapter investigates the emergence of bad news delivery, and how it is negotiated as part of an unplanned (i.e., ad hoc) business meeting. Our data come from a corpus of video-recordings of business interactions in a large multinational company collected as part of a longitudinal ethnographic research project. We adopt a multimodal conversation analytic (CA) approach complemented with ethnographic knowledge and investigate a situation involving a Finnish manager and a Chinese subordinate who use English as a business lingua franca (BELF). Our analysis focuses on their mutually coordinated social actions, showing how the context of bad news delivery and the organization of verbal and embodied conduct in specific ways affords the participants with a place to co-construct and (re)negotiate their professional identities and relationship. We thereby view professional identities as in situ accomplishments and illustrate how their situated nature is anchored in the space of global business. By analyzing a manager’s and a subordinate’s multimodal actions during bad news delivery in the unplanned meeting, we respond to a research gap on the ways social relations at work are established and professional identities constructed outside the formal meeting format (see, e.g., Clifton et al., 2020).

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

43

Interactional Approach to News Delivery and Professional Identity News Deliveries and Their Organizational Features News deliveries can be characterized by their social and structural dimensions, which means that they are always connected to the surrounding sequential and sociomaterial context and the moment-by-moment organization of both the speaker and recipient’s conduct. There are two important points to this. First, for an utterance to constitute news means that the interlocutors mutually orient to its newsworthiness in the unfolding talk (Maynard, 1997, 2003). In other words, no substance or information counts as news if it is not treated as such. Second, news deliveries typically include and invoke evaluative stance-taking and reciprocal work; they can be seen as moments when “participants achieve a mutual sense of some event-in-the-world as good or bad news” (Maynard, 2003, p. 116). Hence, the way news becomes recognized and organized involves ascribing to both its meaning and valence, manifesting NDS as interlocutors’ practical activity that makes relevant attending to their discrepancies related to knowledge and epistemics (i.e., what one knows, and the others do not know) and moral obligations (i.e., to align with the news delivery and to either affirm or modify the first evaluation). What is thus at stake and needs to be negotiated in news deliveries is the interlocutors’ differing epistemic stances, that is, reaching epistemic congruence, and ways to come to an agreement about the content of the news. News deliveries constitute an important research topic in conversation analysis (CA). Studies have focused on mundane and medical settings, illustrating situation-specific organizations of news and how good news delivery might be different from that of bad news (see Freese & Maynard, 1997; Stivers & Timmermans, 2017). In his seminal work, Maynard (1997) identifies the basic structure of news deliveries, highlighting specific actions that are produced at the onset of launching the news, such as preannouncements, and others that have relevance for its overall progression, such as assessments and elaborations. Other studies have also emphasized how the valence of news and its nuances are accomplished by the interlocutors’ moment-by-moment verbal contributions (e.g., Terasaki, 2004) and prosodic features (Freese & Maynard, 1997). Furthermore, Stivers and Timmermans (2017) illustrate how bright-side sequences as part of bad news deliveries in clinical consultations facilitate

44

T. RÄISÄNEN AND T. OITTINEN

achieving “bivalent equilibrium,” that is, augmenting the bad news with a positive remark or a potential solution. In addition, according to studies on mundane interactions, the degree to which the recipient is expected to agree and affiliate with the evaluative stance and experience conveyed by the news deliverer depends on the intensity of the news produced. Heritage (2011, p. 161) describes moments at which affective responses to news deliveries lead to not only congruent epistemic stances but also congruent emotional stances as “empathic communions,” illustrating how intensely communicated experiences obligate the recipient to join and affiliate with the speaker’s stance. We extend these notions of NDS to business contexts and show how, in an institutional environment that typically involves the expectation of emotional neutralism (see Drew & Heritage, 1992), the organization of bad news and the negotiation of its nuances may include elements that manifest the interlocutors’ orientation to the normativity of the encounter and to their interpersonal relationship (i.e., both their epistemic and emotional stances). Constructing Professional Identities at Work Previous studies have highlighted the emergent nature of professional identities (e.g., Antaki & Widdicombe, 1998; Clifton, 2019). The focus has been on the ways in which professional roles and identities can be constituted and enacted at the interactional level in different types of workplace encounters, such as formal meetings (e.g., Du-Babcock & Tanaka, 2017; Schnurr & Zayts, 2011; Van De Mieroop et al., 2020; Vine et al., 2008). In general, meetings are among the core activities in the workplace and their interactional characteristics, such as a turntaking system in which turns are allocated by the chair, are well known. Other features of formal meetings include pre-set agendas and timeframes (e.g., Asmuβ & Svennevig, 2009; Boden, 1994) and having a focus on reporting past events, dealing with the present, and/or orienting to the future (Holmes & Stubbe, 2015; see also Clifton et al., 2020). Studies of meeting interactions have focused on, for example, feedback and leadership practices (Asmuβ, 2008; Svennevig, 2011) and negotiation and decision-making (Barnes, 2007; Du-Babcock & Tanaka, 2017). So far, there are very few studies beyond formal meetings (e.g., Clifton

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

45

et al., 2020) which constitute places for more spontaneous interaction, for example, prompted by bad news deliveries, and where identity construction may be different compared to pre-arranged settings. While professionals have certain entitlements based on their job tasks and roles at work, their identities are practical accomplishments that are oriented to in interaction (Clifton et al., 2020; Larsson et al., 2021). In his study on Scandinavian managers’ practices in feedback meetings in ELF, Svennevig (2011, p. 19) describes the establishment of social relations at work along three dimensions: the epistemic, the normative, and the emotional. The epistemic dimension relates to the way knowledge is displayed and claimed in interaction, highlighting the interlocutors’ expertise by which they can be associated with certain groups at work or business domain. The normative dimension is connected to professionals’ understanding of the surrounding norms and structures and how this becomes visible in the ways entitlement (i.e., rights and obligations to say and do something) is enacted. Finally, the emotional dimension concerns the way interlocutors display affect in interaction and may convey a certain level of interpersonal closeness (Svennevig, 2011). These dimensions are visible in professionals’ work-oriented (i.e., transactional) and relationship-oriented (i.e., relational) behaviors (e.g., Schnurr & Zayts, 2017). For example, leadership can be indexed by practices of problemsolving, moving discussion along, and reaching consensus (Räisänen, 2020; Schnurr & Zayts, 2017). Managers can ascribe specific characteristics to themselves and their subordinates through verbal strategies which manifest their deontic rights (i.e., to determine the actions of self and others; see Clifton, 2019, p. 347) and contribute to the construction of professional identities. With certain feedback practices managers display their role as persons responsible for the company operations, and thereby their leadership style (Asmuβ, 2008; Svennevig, 2011), whereas with other verbal techniques, such as using subjective modal markers, they may reduce their epistemic authority and thus work toward constructing a collaborative and egalitarian leadership style. While various earlier studies on professional identities focus on talk only (e.g., Clifton, 2019; Larsson et al., 2021; Schnurr & Zayts, 2017; Svennevig 2011), recently more multimodal studies have been conducted. For instance, studies on meeting openings and closings (e.g., Oittinen, 2021) have shown how not only talk but also the sociomaterial practices are intertwined with leadership identity. This

46

T. RÄISÄNEN AND T. OITTINEN

body of work further demonstrates how meeting participants, through their deontic claims, enact identities as leaders. With these starting points, we aim to contribute to a better understanding of professional identities, roles, and relationships as multimodally and collaboratively constructed in situ in an unplanned meeting setting in a business context. We focus on a specific interactional context, namely bad news, with a dual focus: we investigate its sequential and temporal organization as accomplished via the manager and the subordinate’s actions. Second, we analyze how the aligning and affiliating behaviors produced in the medial environment of the office space contribute to achieving epistemic and emotional congruence and (re)negotiating the identities relevant for the meeting.

Multimodal Conversation Analysis of Meeting Data from an Ethnographic Study This chapter is based on an ethnographic study of a group of Finnish business professionals (e.g., Räisänen, 2019). The business professional (pseudonym Oskari) has been followed in 2003–2020, from their student days in mechanical engineering through career advancement toward management positions. We focus here on Oskari’s work as an Operations Manager (OM) at a multinational company (Service MNC), being based in China and zoom into one specific type of event as part of his management tasks, that is, dealing with news. This serves as a practical example at our focus and is illustrative of the spontaneous encounters between the manager and his team. Oskari’s first language (L1) is Finnish, and he has studied English as a foreign language for over ten years. As a manager, Oskari has used both English and Finnish and worked in China for several years. The Service MNC employs almost 4000 people and provides customers with services and solutions. Its headquarters are in Finland and offices in nearly a dozen countries on three continents. Oskari’s OM position entailed managing the operations in one of the China offices, including team management in their customer collaboration. Oskari’s work practices were collected and recorded in 2016–2018 with the researcher present in various face-to-face interactions and observing his work practices at work. The data comprise fieldwork during 13 working days during three fieldwork periods (October 2016, March 2018, and October 2018). Oskari has also been interviewed twice in 2016 and 2018, and once in 2020.

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

47

Our analytical focus is on an ad hoc meeting (total length 37 minutes) between Oskari and a Department Manager (DM), Mark, who is Oskari’s subordinate, occurring during the first fieldwork period in October 2016. Mark’s first language is Chinese and his work as a DM included leading and managing a team of junior engineers working on several customer projects. Oskari and Mark used English as a lingua franca. The meeting at focus occurred in the middle of a busy work week. Before the meeting, Oskari is in his office with a visitor, Kalle, his superordinate (L1 Finnish) stationed at another China office. Oskari and Kalle had just finished discussing various confidential issues related to company operations and Kalle was packing his belongings and preparing to leave. The ad hoc meeting started with Mark knocking on the door with a request to enter. This was a typical start for ad hoc meetings as Oskari’s subordinates were encouraged to visit him with any immediate issues. At this point, an audio-recorder was on (at the discretion of the confidential discussion between Oskari and Kalle video camera was not used), while the videorecording was started after Kalle had left and Oskari and Mark had exchanged a few words. We use multimodal conversation analysis (CA) to study the unfolding of the news delivery sequence. Focusing on the detailed examination of the interactional order, that is, how social actions are sequentially and temporally organized in the sociomaterial environment (Hazel et al., 2014; Schegloff et al., 1977), CA allows us to show how the participants themselves engage in joint sense-making of the situation and construct their situated roles. We bring CA together with ethnography in a “limited affinity” which is crucial for being able to understand the structuring of the participants’ spoken and embodied sociality that is impacted by past events (Maynard, 2006, p. 70). The extracts from the meeting have been transcribed according to Jeffersonian (2004) conventions and applying Mondada (2001, 2006) for multimodal conduct and embodied displays (see Appendix: Transcription Conventions).

Multimodal and Collaborative Organization of Bad News Delivery Our starting point in the analysis is the beginning of the meeting and the activity it makes relevant, namely that something has happened in the company that must be communicated to Oskari. The analysis shows the moment-by-moment unfolding of a multiunit bad news delivery sequence

48

T. RÄISÄNEN AND T. OITTINEN

in which the valence of the news is negotiated throughout the encounter: it is a case of an accumulatively organized news. We illustrate how the participants use talk, embodied resources, including gaze, body movement and gestures, and the office space to advance the NDS at its different stages and negotiate their situated identities and differing epistemic and emotional stances (see Drew & Heritage, 1992). The analysis follows the chronological order of the NDS, focusing on moments of (1) establishing the conditions for bad news delivery, (2) achieving epistemic congruence, and (3) achieving emotional congruence, that is, emphatic communion. Establishing the Conditions for Bad News Delivery and Enacting Manager-Subordinate Identities Prior to news deliveries, the interlocutors need to establish the frames and conditions that make launching them possible (Maynard, 2003). It can involve physical actions with which a suitable bodily formation is reached (e.g., Kendon, 1990), a verbal orientation shift, and an explicit or a subtle entry into a new topic. The news delivery under scrutiny starts from the initial moment of Mark and Oskari’s encounter: Mark has knocked on Oskari’s door and enters the office, with which he also accomplishes the pre-beginning of their ad hoc meeting. Although a third person, Kalle, is also in the room for the first part of their discussion, he is not the primary recipient of the news. Moreover, he is about to leave and does so after a while. The extract foremost shows how, through the first words spoken that modify the participation framework, (1) launching the news becomes a relevant next step and (2) Oskari and Mark enact their manager-subordinate identities especially via a question–answer sequence relating to the news (Lines 2–10). Key for the initial (re)configuration of the context for the news delivery is Mark’s preannouncement (Line 1) and the physical action of closing the door that follows it.

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

49

Extract 1 1

Mark

2 3 4 5 6 7 8 9 10

Oskari Mark Oskari Mark Oskari Mark Oskari Mark

11 12

Kalle Oskari

what happened resignations

Kalle Oskari Kalle Mark Oskari

how many two today? yesterday. yesterday

13 14 15 16 17 18

(2.0)

19 20 21

I should close the door. ((Mark closes the door)) what is it resignations. ok when yesterday who were they aa: one is this aa: one is from [customer1] ok and the other one is aa: is aa (.) one guy working for the aa: [customer2]

(4.0) Oskari

22

electrical engineering and hhhhh.[customer2] well(4.0)

23 24

Oskari Kalle

ok mä voin mä voin soittaa sulle kun hyppään autoon [I can I can call you when I’m in the car]

25

Oskari

26

Kalle

hyvää matkaa [have a good trip] kiitos [thank you] ((Kalle leaves the room and closes the door))

27 28 29

Oskari Mark

ok and today I received Megan’s aa: (1.5

30 31 32

Oskari Mark Oskari

official yeah yeah yeah (.) ok.

33 34 35

Mark Oskari Mark

so that makes four in this month already who is the fourth aa fourth is this Hanna

The extract begins when Mark enters Oskari’s office and initiates a conversation with an evaluative moral account, “I should close the door” (Line 1). The turn has a multifaceted function. First, it not only prefaces

50

T. RÄISÄNEN AND T. OITTINEN

Mark’s next action of closing the door, but also makes it a prerequisite for him to continue. Second, it configures the context and audience for a very specific kind of an upcoming turn: a targeted news delivery (see Maynard & Freese, 1997). The first turn here is a conversation opening and a preannouncement of something that follows, making it an important initial step toward establishing the conditions for its actual delivery. Once the door is closed, Oskari responds to Marks’ turn with a news inquiry, “what is it?” (Line 2), by which he orients to the newsworthiness of what is to follow. By acknowledging Mark’s single-word answer with an agreement token, “ok,” and another question, “when?”, instead of displaying any form of “change-of-state” (i.e., indicating a “now knowing” stance, Heritage, 1984), Oskari aligns with the prior turn and thereby fashions the news as something that was fairly expected. This stance is further ratified in the subsequent Lines (6–9), in which the topic is elaborated by Oskari’s initiative. The initial moments of the news delivery, and the question–answer sequence in particular, make Oskari’s manager identity relevant in the ongoing interaction. Kalle’s behavior manifests a similar reaction, as he enters the conversation after packing his belongings. He first produces a news inquiry, “what happened?”, and then two elaborative questions (Lines 14 and 16), by which he accomplishes the change of his status from a “hearer” to a “ratified participant” (Goffman, 1981). This way the participation framework becomes adjusted, but only for a while. After Oskari has provided more details about the resignations, regarding their time and the identities of the people, he continues with a discourse marker, cut-off “well” (Line 21). After Oskari’s acknowledgment turn (“ok”), Kalle continues with an action projecting utterance, “I can call you when I’m in the car” (in Finnish in Line 24), which manifests his close working relationship with Oskari and support toward Oskari as his subordinate. The language shift excludes Mark from understanding the exchange. Once Oskari has wished Kalle a safe trip back home, also in Finnish, Kalle thanks him and leaves the room (Line 26). The use of Finnish signals their shared linguistic background and Finnish manager identities. After this, Oskari reorientates himself toward Mark and the just delivered news by producing a boundary marker, “ok,” now in English. This prompts Mark to continue, which he does by producing a conjunctive “and” that is followed by a sequence expansion involving a relevant topic: another person’s official notice. As he fails to recall the last word, Oskari co-participates in the word search and completes it (Line 30, see e.g., Hayashi, 2003). After Mark’s ratifying

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

51

turn and Oskari’ sequence-closing token, “ok” (Line 32), Mark uses a summarizing formulation to close the initial phase in the NDS. This section has shown the multimodal and collaborative establishment for the conditions of news delivery, involving embodied preparatory actions, such as Mark walking in and closing the door, and some adjustments in the participation framework, namely Kalle leaving the room. All three participants have contributed to building a mutual understanding about the delicacy of the situation and about Oskari’s leadership identity. As the manager of the present office and someone who knows his personnel and team structure in different customer projects, Oskari has the right and responsibility to handle the content of the news (cf. Clifton, 2019). Furthermore, although Oskari and Kalle have both oriented to the newsworthiness of Mark’s deliverable, the news itself has not been treated as surprising due to the participants’ shared history. At this point, there is an incipient agreement about the valence of the news, that is as something not pleasant, although it has not been explicitly mentioned. Achieving Epistemic Congruence and Egalitarian Leadership Before the sequence in Extract 2, Mark has provided the reasons behind a specific person’s resignation (family business) and Oskari his evaluation of the situation, saying how several resignations are worrying, but not all of them major loss workwise. In Extract 2, Mark is made accountable for elaborating on the news and giving more reasons for the resignations that reach beyond the single case. This becomes relevant when Oskari asks for Mark’s opinion (Lines 36–37). By doing so, Oskari promotes Mark’s individual responsibility and enacts a leadership identity that is now more collaborative and egalitarian (see Svennevig, 2011). Prior to the excerpt’s beginning, Oskari has produced a multiunit turn, during which he has engaged in solitary sense-making and merely speculated the reasons, but he is now asking for a perspective of the news deliverer, Mark, who is potentially an insider.

52

T. RÄISÄNEN AND T. OITTINEN

Extract 2 36 37 38

Oskari

oskari 39 40

Oskari mark fig

what is your opinion= =wha- what do you think? how is the- is there any any other +problems behind this= +turns gaze to M =is it is it the workload or is there any any kind of a: (0.3)*#(0.7) *turns gaze forward #1

Figure 3.1. Mark turns gaze.

41 42

Mark mark

I think workload is one thing and then aa: (0.8) okay:, =*it de pends, actually *turns gaze and upper body to O

43 44

Oskari Mark mark

mm *so if we are talking about these uh *junior engineer *turns gaze to front *to O

45 46

Oskari Mark mark

[yeah ] [or *like] uh two or three years of working experience *gaze to front, starts to move hands

47

Mark mark

*engineer like that* then *u:h its kind of *hand--------------* *glances at O

mark

natural *that u:h if they are *(.) not uh (0.5) *turns gaze to O *gaze to front

mark

(.) with the *nature of the work, *turns gaze to O

50 51 52

Mark Oskari Mark mark

then u:h (0.4) they decide to leave. yeah? that is *u:h (.) *actually that is quite common, *gaze to front *to O

53

Mark oskari mark

+(1.0) I’m not sure if *uh it’s worldwide +nods *turns gaze to front

54

Mark mark

but anyway in *China, *turns gaze to O

55 56 57 58

Mark Oskari Mark Mark mark

so you basically work in a company, m[m. [two or three years, especially you are (.) actually graduate *from university *gaze down

48 49

3

59

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

Mark mark

*then u:h two or three years its kind of u:h (.) *turns gaze to O

mark

*it’s kind of u:h (0.4) let’s say *uses hands, gaze away from O

oskari fig

(0.4)+#(0.3) +starts shifting to an upright position...... #2

60 61

53

((coughs))

Figure 3.2. Oskari initiates body shift.

62 63

Mark Oskari mark oskari

[it’s + u-] [yeah I *know] (.) that they are quite eager .hh *turns gaze to O .................................................

64

Oskari oskari Mark

to +change or the- they leave quite easily +starts sorting paper documents yeah.

65

The extract begins with Oskari’s open-ended question to Mark regarding his opinion about the reasons behind the resignations. He rephrases it to narrow down the focus, and two explanatory factors become juxtaposed: the workload and “other problems” (Lines 38–39). With his question, that solicits an account from Mark regarding the situation, Oskari indicates an epistemic stance of not knowing and plays down his deontic status (i.e., as the one responsible for holding the floor and solving the issue; see also Clifton, 2019, p. 351). After the conclusion of the turn and a pause, Mark turns his gaze away from Oskari (Fig. 3.1). As the question is formulated in a way that calls for making a decision, that is, selecting one of the two alternatives, Mark produces his response accordingly. By still not looking at Oskari, he produces an assessment about the workload, pointing to its relevance as “one” contributing factor (Line 41). After this, he continues with a progressively oriented expression “and then” but suddenly hesitates. The averted gaze and this behavior indicate some difficulty in answering the

54

T. RÄISÄNEN AND T. OITTINEN

question and in explaining the reasons behind resignations, which are not as straightforward as workload which both Oskari and Mark know. However, Mark suddenly reorients to launching his response with a boundary marker “okay,” straightening his posture, and turning to look at Oskari. He continues with an assessment that makes it clear that no simple answer exists. The adverb “actually” further indicates the change of direction in the turn underway (Line 42), which Oskari also verbally acknowledges (Line 43). This multimodally constructed and rephrased sequence beginning indicates Mark’s slight resistance with accepting the suddenly assigned role as the one with more knowledge, that is, someone with epistemic authority (see Heritage & Raymond, 2005), and his potential discomfort in being the intermediary. It also functions as a bridge to the longer account that follows, as Mark here clearly orientates to delivering a further elaboration of reasons behind the news. Mark prefaces his upcoming elaboration with a conditional clause that shapes the context of his utterance: his explanation relates foremost to the resignations of the “junior engineers” (Line 44), which he also defines in the subsequent lines (46–47). Toward the end of his utterance, he turns his gaze down. When he begins to formulate his account regarding that specific group of employees, he still gazes at his hands and takes his time to choose the words. He uses a hesitation marker and a softened assessment “it’s kind of natural” as a way to preface another conditional clause, one that presents a potential reason for the younger employees leaving, that is, the nature of the work not being satisfying (Lines 48–50). Before uttering the most crucial information, the predicative “satisfied,” Mark takes a long pause, and once he proceeds, he maintains his gaze away from Oskari. With these verbal and embodied actions (Lines 44– 50), he indicates being aware of the potential face threat his utterance poses and reluctance to having to produce it (cf. Heritage & Raymond, 2005). After Mark’s summarizing formulation, Oskari acknowledges the turn with a go-ahead token, “yeah,” uttered with a slightly rising intonation (Line 51). Mark continues with a hesitative assessment, saying that it is “common,” which he then softens retrospectively; he produces an epistemic account that indicates uncertainty about whether this is similar elsewhere in the world. Mark continues his turn with an adverbial phrase, “but anyway in China” (Line 54), to further emphasize how reasons for resigning might be different from what Oskari is used to. This moment makes Oskari’s knowledge and background as a non-Chinese manager visible. When Mark begins to reformulate his point, he adds the category

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

55

“a graduate from university” as an important factor that may help explain the reason (Line 58). He struggles to find the words to complete the turn, which is indicated by repeating the first part of a potential assessment twice and bodily orienting to a solitary word search (Goodwin & Goodwin, 1986). As he initiates the same verbal phrase, “it’s,” for the third time, Oskari starts to move to a more upright position (Fig. 3.2), which projects his verbal turn-taking that becomes launched in overlap with Mark’s incipient assessment. In the subsequent lines (63–64), Oskari produces a summarizing formulation of what Mark has just said and begins to organize the paper documents on the table. He thereby accepts the explanation for the resignations without questioning it or asking for more information and indicates the achievement of epistemic congruence regarding the situation. Mark uses different techniques to address Oskari’s inquiry regarding the resignations and to handle the delicate situation. His utterance “these junior engineers” functions as a membership categorization device (Sacks, 1992) which establishes a dividing line between the specific group of engineers and other engineers and employees and ascribes them with specific identities. Mark connects the category, identifiable in China (Line 54), with attributes like “with two to three years of work experience,” “engineer like that,” “graduate from university,” and the explanation “so it’s natural if they are not satisfied they decide to leave.” As became apparent during the ethnographic fieldwork, in the China office, the junior engineers were seen to exercise “job shopping,” which means testing a job for a while and easily changing if the work and the salary were unsatisfactory. In a recruitment meeting Oskari and HR personnel talked about the need for skilled junior engineers with good knowledge of English. These engineers were seen to hold power in the job market in being able to choose the most appealing jobs which resulted in frequent salary discussions and these types of resignations discussions. Mark’s categorization work thus functions as a resource in the news delivery sequence and a mitigation device for producing potentially face threatening turns. Mark also provides Oskari with cultural information that may be otherwise inaccessible to him and thereby make their cultural backgrounds relevant. This second section has illustrated the elaboration of the news during which additional interactional work is done toward achieving epistemic congruence. Oskari initiates the sequence specifically to reduce his authority and gain more knowledge from an insider. By inviting Mark to fill in the gaps, Oskari gives him responsibility and enacts the identity

56

T. RÄISÄNEN AND T. OITTINEN

of an egalitarian leader. However, Mark’s behavior indicates some degree of discomfort regarding this change in epistemic and deontic status, as shown by his orientation to deference. After the extract, the meeting continues with the participants discussing the reasons for resignations and seeking solutions to manage with the high workload. Achieving Emotional Congruence and Interpersonal Affect In the third extract that occurs later during the encounter, Mark produces another piece of news about a pressing and urgent matter that he already knows is not good news either. Although it is no longer about resignations, but about a customer needing the company’s service earlier than expected, it has an accumulative effect because of the news he has already delivered. Mark produces many hesitative displays prior to launching the news, but the situation yet results in Oskari’s strong emotional evaluation (Line 106) that calls for a similar affective stance from Mark (cf. Heritage, 2011, p. 160). In the end, Oskari and Mark both produce verbal and embodied affiliative displays by which they create an emphatic communion that also momentarily promotes their interpersonal relationship instead of their organizational identities. There has been a long pause before the extract during which Oskari has been engaged with using his phone. While he is still oriented to it, Mark closes the previous phase with a sequence-closing formulation, “that was it,” after which he produces a boundary marker, “then” to project the beginning of a new sequence (Line 96). His hesitation, gaze aversion, and the pause that follow indicate again some difficulty in producing the next item. However, instead of marking a trouble in producing a specific language item in the turn-in-progress, it is about delaying the utterance for another reason: What is to follow is another piece of bad news. He prefaces the news with a preannouncement, stating what the topic concerns (Line 98). It is noteworthy that although he does not use the word “news” but a “thing,” he yet frames the upcoming utterance as something newsworthy. Concurrently, he turns gaze to Oskari, who ceases from using his phone and places his right hand under his chin, indicating heightened attentiveness and waiting stance (Fig. 3.3). As Mark continues to a “so”-prefaced account about having received an email from the customer that the news concerns, he again turns his gaze away: now downwards. After this, he repeats the preposition, “them,” once more, produces yet another hesitation marker, and then pauses (Lines 99–100).

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

57

While still not looking at Oskari, Mark picks up his pen from the table simultaneously starting to deliver the first part of the news, “they want to have” (Line 101). The gist of his bad news formulation is preceded by another pause and a hesitation marker along with which he finally restores his gaze to Oskari. He then continues hesitatively toward uttering the person’s name needed to deliver service to a customer already on the first of November (Line 101). The person (Katie) is a junior engineer who had been agreed to provide the service in a set schedule with the customer earlier; however, Mark explains here that the customer needs the service, and thus Katie, a month earlier than originally planned. As the workload is already difficult to manage due to resignations, this schedule change challenges the operational management and thus Oskari. When Mark completes the news delivery with a verbal contextualization by mentioning the date when the person was supposed to come, Oskari reacts to it both visibly and audibly. He first produces a laughing token, puts his phone on the table, and then removes his glasses and, concurrently with loud breaths, begins to rub his eyes. Simultaneously, Mark turns his gaze away. After this, Oskari produces a cut-off expletive (Line 106 “voi vit-” “oh fu-”), which functions not only as a “response cry” (Heritage, 2011, p. 173), but also as an indication of his orientation to the news as crushing. During the following silence, Mark repositions himself slightly and aligns with Oskari’s bodily behavior by beginning to rest his face on his hands (Fig. 3.4). This builds up a moment when Mark affiliates with Oskari’s experience and evaluation, and the two construct an emotionally congruent stance, that is, emphatic communion (Heritage, 2011; Line 109). Once Oskari attempts to produce an assessment regarding the news, Mark suddenly initiates a new turn in overlap in which he orients to solving the trouble; his turn concerns the solution (Line 109). However, Oskari inhales strongly during this and does not align with the turn but makes a statement about the just delivered news, it seems that he is still processing it.

58

T. RÄISÄNEN AND T. OITTINEN

Extract 3 93

Mark mark oskari

94

Mark

95 96 97 98

Mark mark Mark mark oskari oskari fig

*+ o:::kay. *gazes to front +gazes to and manipulates phone yeah

al right,

(2.1) that was *it (.) then *uh *glances to right (1.3) anoth- another *thing is +regarding +the (customer)= *turns gaze to O +puts right hand on chin #+turns gaze to M #3

Figure 3.3. Mark turns gaze to Oskari; Oskari puts hand on chin.

99

Mark mark

=>so we got email< fro*m (.) from them that aa: *gaze down

100

(1.1)

101 Mark mark

*they want to have, (0.4) *uh (Katie): (.) *picks up pen *gaze back to O

102 Mark

the first of (.) November (.) already.

103 Oskari oskari

+ ehe +starts turning gaze to left--->

104

+(1.2) +puts phone on table, removes eyeglasses .hh*hh+h *gaze down +starts rubbing eyes--->+

Oskari 105 Oskari mark oskari 106 Oskari

voi vit[oh fu-]

3 107 mark fig

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

59

*(1.0) *#(0.2) *adjusts position *glances at O; puts palms together and begins to lean on hands --->* #4

Figure 3.4. Oskari rubs eyes; Mark leans on hands.

108 Oskari mark 109 Mark mark 110 Oskari

so *it’s [a --->*turns gaze to O [*got question from [another department *turns gaze and head to hands [.hh

111 Oskari oskari mark

so +you saved the best for *the last --->+turns gaze to M *turns gaze to Oskari

112 Oskari oskari 113 Mark

heh+[heh] +gaze down--->+ [heh]

114 Mark mark oskari

yeah *but we have we +have some proposals already so *turns gaze to hands *--->> --->+turns gaze to M

115 Oskari oskari

okay? +(.) I really hope you can +somehow solve it +glances downwards +begins to move the hand with glasses

116

+(1.0) +puts glasses on

oskari

Mark continues by shifting away from the bad news and reassuring that even though this was “best for the last,” they have some proposals to remedy the situation (Line 114). This type of self-initiative taken by the subordinates illustrates the successful implementation of Oskari’s management style (described in an interview with Oskari) to promote subordinates’ initiative-taking and solution-finding at first by themselves before bringing them on to his table. It also functions as a transition to examine the “bright side” of the topic (Stivers & Timmermans, 2017). After this, Mark gets another chance to explain the possible solution

60

T. RÄISÄNEN AND T. OITTINEN

which includes a replacement for Katie. Mark’s problem-solving indexes renegotiation of his identity as not only a subordinate telling the news but also a manager with leadership responsibility. Hence, leadership emerges as a collaborative effort between Oskari and Mark. The news delivery sequence ends after six minutes with Oskari’s conclusion that, despite the bad news delivered, the work can be managed with adjustments, and his question to Mark if there is “anything else,” projecting the possibility of further news. After Mark’s negative answer, Oskari concludes the meeting with an announcement about lunch time.

Discussion and Conclusions This chapter investigated the emergence of bad news delivery in an ad hoc BELF meeting and the in situ co-construction of professional identities and social relations. Using multimodal conversation analysis and ethnographic background information, we focused on the sequential and temporal organization of a multiunit news delivery sequence and how its valence and meaning were negotiated by the two professionals. Our findings highlighted the role of multimodal resources and sociocultural knowledge drawn on by the participants while establishing and adjusting the conditions for the news delivery and ensuring its progression. We also showed the relevance of aligning and affiliating behaviors coordinated throughout the encounter. These were seen to contribute to achieving epistemic and emotional congruence regarding mutual knowledge and understanding of the situation but also in relation to claiming and enacting the identities relevant in the bad news delivery and the meeting. Our study contributes to a better understanding of the interactional context of bad news delivery and the way it can afford a special place for collaborative construction and (re)negotiation of professional identities (e.g., Clifton, 2019). The analysis illustrated the progressive and reciprocal nature of bad news delivery in the business context, in which the underlying structures, norms, and roles may require a delicate approach and recipient-designed ways by all parties to handle the situation. In our case, the two professionals’ conduct in the medial environment of the office space included their orientation to each other’s rights and entitlements and the utilization of verbal, embodied, and material resources in specific ways. Certain verbal tokens, such as preannouncements, hesitations, and categorization; and embodied displays, such as alternating gaze direction, functioned as

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

61

important means for the subordinate to initiate and frame the news in a respective and empathetic manner, and they also indicated his difficulties in delivering the news to his superordinate. The superordinate, in turn, used news inquiries, questions, and mutual gaze (Extracts 1 & 2) to display recipiency and seek understanding of the news and as ways to enact his manager identity. Extract 3 showed how verbal and embodied resources were also important in displaying interpersonal affect and achieving emotional congruence as part of handling the difficult situation. Our study contributes to earlier findings on NDS and increases knowledge about the way these moments are multimodally organized (cf. Maynard, 1997, 2003; Stivers & Timmermans, 2017). More specifically, it shows bad news delivery in the workplace as an emergent situated accomplishment, which may require the negotiation of not only epistemic discrepancies but also participants’ emotional stances. With a detailed analysis of the extracts, we were able to show how the co-construction of identities during the bad news delivery is a situated accomplishment manifested in the participants’ orientation to each other’s epistemic stances and entitlements (i.e., to perform the given actions) as well as in their ways to achieve emotional congruence. Our analysis illustrated tasks central for the manager and the subordinate (cf. Svennevig, 2011, p. 34): delivering the news about personnel resignations and changes in customer schedules was central for the Department Manager as the subordinate, while for the Operations Manager as the leader it was important to hear about the news. During the episode, the superordinate reduced his deontic and epistemic authority by asking for the subordinate’s opinion regarding the news and inviting him to align in a moment of emphatic communion (Heritage, 2011). The participants’ use of the embodied configuration vis-á-vis the ongoing activity to display affiliation made relevant their interpersonal relationship and collegiality: aspects that are not always visible in data from institutional settings with the normative expectations of emotional neutralism. Our analysis also showed the moment-by-moment negotiation of the Finnish manager’s leadership identity, as he moved from an authoritative leadership style to a more egalitarian one. This depicts the Finnish manager as someone who gives his subordinates the possibility to enter the office with bad news, urging them to seek solutions to problems themselves first, engaging in discussion, and during difficult situations showing emotions. Not imposing on his point of view and not claiming superior knowledge resonates with

62

T. RÄISÄNEN AND T. OITTINEN

earlier findings on managers claiming mutual obligations with their subordinates (e.g., Svennevig, 2011). Our study thus contributes to research on professional identities and leadership as multimodally realized in and through people’s actions outside the formal meeting format. The findings further contribute to our understanding of business interactions in the global space where people from different linguacultural backgrounds come to together for work, using various resources available to them.

Appendix: Transcription Conventions , . ? ↑ ↓ = [] thawhat what < < what °what° WHAT £what£ wh(h)a(h)t (what) (x) (.) (0.5) ((gazes)) # * * * l.9

intonation is continuing intonation is final rising intonation rising intonation/high pitch falling intonation latched utterances overlapping talk a cut-off word word emphasis speech pace that is quicker than the surrounding talk speech pace that is slower than the surrounding talk speech that is quieter than the surrounding talk speech that is louder than the surrounding talk smiley voice laughingly uttered word uncertain hearings unrecognizable or confidential item micro pause, less than 0.2 seconds silences timed in tenths of a second transcriber’s comments location of the figure in relation to talk and non-verbal action gesture or action described continue across subsequent lines gesture or action described continue until the same symbol is reached gesture or action described continue until and after excerpt’s end gesture or action described continue until the line mention

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

63

References Antaki, C., & Widdicombe, S. (Eds.). (1998). Identities in talk. Sage. Asmuβ, B., & Svennevig, J. (2009). Meeting talk: An introduction. Journal of Business Communication, 46, 3–22. Asmuβ, B. (2008). Performance appraisal interviews: Preference organization in assessment sequences. Journal of Business Communication, 45(4), 408–429. Barnes, R. (2007). Formulations and the facilitation of common agreement in meetings talk. Text & Talk, 27 , 273–296. Boden, D. (1994). The business of talk: Organizations in action. Polity Press. Clifton, J. (2019). Using conversation analysis for organizational research: A case study of leadership-in-interaction. Communication Research and Practice, 5(4), 342–357. Clifton, J., Larsson, M., & Schnurr, S. (2020). Leadership in interaction. An introduction to the Special Issue. Leadership, 16(5), 511–521. Drew, P., & Heritage, J. (1992). Talk at work. Cambridge University Press. Du-Babcock, B., & Tanaka, H. (2017). Leadership construction in intra-Asian English as lingua Franca decision-making meetings. International Journal of Business Communication, 54(1), 83–98. https://doi.org/10.1177/232948 8416675451 Freese, J., & Maynard, D. W. (1997). Prosodic features of bad news and good news in conversation. Language in Society, 27 , 195–219. Goffman, E. (1981). Forms of talk. University of Pennsylvania Press. Goodwin, C., & Goodwin, M. (1986). Gesture and coparticipation in the activity of searching for a word. Semiotica, 62(1/2), 51–75. Hazel, S., Mortensen, K., & Rasmussen, G. (2014). A body of resources—CA studies of social conduct. Journal of Pragmatics, 65, 1–9. Hayashi, M. (2003). Language and the body as resources for collaborative action: A study of word searches in Japanese conversation. Research of Language and Social Interaction, 36(2), 109–141. Heritage, J. (2011). Territories of knowledge, territories of experience: Empathic moments in interaction. In T. Stivers, L. Mondada, & J. Steensig (Eds.), The morality of knowledge in conversation (pp. 159–183). Cambridge University Press. Heritage, J., & Raymond, G. (2005). The terms of agreement: Indexing epistemic authority and subordination in assessment sequences. Social Psychology Quarterly, 68(1), 15–38. Heritage, J. (1984). Change-of-state token and aspects of its sequential placement. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action: Studies in conversation analysis (pp. 299–344). Cambridge University Press. Holmes, J., & M. Stubbe (2015). Power and politeness in the workplace: A sociolinguistic analysis of talk at work. Routledge.

64

T. RÄISÄNEN AND T. OITTINEN

Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. H. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp. 13–31). John Benjamins. Kendon, A. (1990). Conducting interaction: Patterns of behavior in focused encounters. Cambridge University Press. Larsson, M., Clifton, J., & Schnurr, S. (2021). The fallacy of discrete authentic leader behaviours: Locating authentic leadership in interaction. Leadership, 17 (4), 421–440. Maynard, D. (1997). The news delivery sequence: Bad news and good news in conversational interaction. Research on Language and Social Interaction, 30(2), 93–130. Maynard, D. W. (2003). Bad news, good news: Conversational order in everyday talk and clinical settings. University of Chicago Press. Maynard, D. W. (2006). Ethnography and conversation analysis: What is the context of an utterance? In S. Hesse-Biber & P. Leavy (Eds.), Emergent methods in social research (pp. 55–94). Sage. Maynard, D. W., & Freese, J. (1997). Prosodic features of bad news and good news in conversation. Language in Society, 27 , 195–219. Mondada, L. (2001/2016). Conventions for multimodal transcription. https:// franz.unibas.ch/fileadmin/franz/user_upload/redaktion/Mondada_conv_ multimodality Oittinen, T. (2021). Material and embodied resources in the accomplishment of closings in technology-mediated business meetings. Pragmatics. https://doi. org/10.1075/prag.19045.oit Räisänen, T. (2020). The use of multimodal resources by technical managers and their peers in meetings using English as the business lingua franca. IEEE Transactions on Professional Communication, 63(2), 172–187. Räisänen, T. (2019). Cultural knowledge as a resource in BELF interactions: A longitudinal ethnographic study of two managers in global business. Iperstoria, 13, 34–46. Sacks, H. (1992). Lectures on conversation (2 vol., G. Jefferson, Ed. & E. Schegloff, Intro.). Blackwell. Schegloff, E., Jefferson, G., & Sacks, H. (1977). The preference for selfcorrection in the organization of repair in conversation. Language, 53, 361–382. Schnurr, S., & O. Zayts (2011). Be (com)ing a leader: A case study of coconstructing professional identities at work. In J. Angouri & M. Marra (Eds.), Constructing identities at work (pp. 40–60). Palgrave Macmillan. Schnurr, S., & Zayts, O. (2017). Language and culture at work. Routledge. Stivers, T., & Timmermans, S. (2017). Always look on the bright side of life: Making bad news bivalent. Research on Language and Social Interaction, 50(4), 404–418.

3

BAD NEWS DELIVERY AS AN INTERACTIONAL CONTEXT …

65

Svennevig, J. (2011). Leadership style in managers’ feedback in meetings. In J. Angouri & M. Marra (Eds.), Constructing identities at work (pp. 17–39). Palgrave. Terasaki, A. K. (2004). Pre-announcement sequences in conversation. In G. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp. 171– 223). John Benjamins. Van De Mieroop, D., Clifton, J., & Verhelst, A. (2020). Investigating the interplay between formal and informal leaders in a shared leadership configuration: A multimodal conversation analytical study. Human Relations, 73(4), 490–515. Vine, B., Holmes, J., Marra, M., Pfeifer, D., & Jackson, B. (2008). Exploring coleadership talk through interactional sociolinguistics. Leadership, 4(3), 339– 360.

CHAPTER 4

Multimodal Negotiation for the Right to Access Digital Devices Among Elderly Users and Teachers Joonas Råman

In the recent years, two global megatrends have become particularly prevalent in the Nordic countries: the digitalization of services and everyday interactions and the rapidly aging population. At the intersection of these developments are the elderly users of digital devices, an often overlooked or inherently problematized group (e.g., Alexopoulou, 2020; Cutler, 2005). While not necessarily the earliest adopters of novel technologies, the elderly in Nordic societies are nonetheless a growing user group of ICTs (Oksman, 2006). This fact is recognized in Finland, where several state-funded initiatives such as Kansalaisen digitaidot (Citizens digital skills , see https://kansalaisopistot.fi/kansalaisen-digitaidot/)

J. Råman (B) Research Unit for Languages and Literature, University of Oulu, Oulu, Finland e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_4

67

68

J. RÅMAN

have been launched with the goal of ensuring that the citizenry has the skills required to participate in the discourses of a digitalizing society. This chapter provides one micro-level examination of how exactly such skills are taught. The setting examined is a state-funded course in an adult education center focusing on the use of digital devices (smartphones, laptops, and tablets), and aimed specifically at the elderly. In this course, the students have the opportunity to either learn to use their own digital devices or borrow devices from the classroom and practice using them. This chapter focuses on moments of trouble, where the students have encountered a problem with their device(s) that they cannot immediately solve themselves and where assistance from the teacher is recruited (Kendrick & Drew, 2016). This chapter seeks to uncover how haptic, intermedial, access to the digital devices is multimodally negotiated between the student and the teacher. Digital devices, such as smartphone and laptop computers, are not simply physical items, but rather objects which grant access to another, digital, medium (Elleström, 2018; Rippl, 2015; Thurlow, 2015). As such, using them is an intermedial affair which requires the objects to be treated both as physical and as digital artifacts. Bridging these two mediums is the user of the device, who needs both visual and haptic access to the device to successfully operate them. As might be expected when dealing with intermedial objects, any issues in their use may stem from either medium, the physical or the digital. An example of a physical problem would be a student struggling to locate the correct place on their laptop to plug in their mouse. Conversely, failing to locate Google image search would be a problem stemming from the digital medium. Several studies have reported on the variety of physical problems or limitations the elderly encounter when using digital devices (McLeod, 2009; Roupa et al., 2010). Similarly, the digital medium can also present challenges for the elderly (Czaja & Lee, 2002; Leung et al, 2010; Lin et al., 2009; Mallenius et al., 2007). The medium of the problem dictates the medium (and modality) of the solution. When the problem arises from the physical nature of digital devices the solution is presented in the physical medium. When the problem encountered stems from the digital nature of the devices, the solution requires both digital and physical mediums to be engaged. As Tuncer and Haddington (2019, p. 2) note in their discussion on object-transfers, there is a growing interested among scholars in the field of multimodal interaction to examine objects in interactions (see e.g.,

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

69

Fox & Heinemann, 2015; Horlacher, 2019; Rossi, 2014; Takada & Endo, 2015). The role objects, and their transfers, play in various social actions such as requests (e.g., Dixon, 2015; Rauniomaa & Keisanen, 2012), directives, and offers (e.g., Rossi, 2014) is a topic which has recently been examined in both every-day (e.g., Zinken & Ogiermann, 2013) and institutional settings (e.g., Heath et al., 2018). When examining objects in interaction, it is important to discern whether the interactions are object-centered (Tuncer et al., 2019) or object-implementing (Weilenmann & Lymer, 2014). In the latter, objects are present and utilized as resources in the interactions. In the former, objects form the very core of the interaction, becoming the topic of the discussion, and, especially in institutional setting, shaping the course and structure of the interaction, as is the case in the data set examined for this study. When engaged in resolving the student’s problems, the teacher’s institutional role as an educator becomes particularly pronounced. Many of the problems encountered by the students might quite easily be resolved by the teacher simply taking control of the device and performing the necessary steps themselves. However, in the data, this does not often happen. Instead, most of the problems are resolved via a relatively lengthy sequence where multiple modalities such as spoken instructions and directives, physical gestures, and finally, demonstrations are employed to reach a solution. This is because some of the problems are treated by the teacher as learnables (Majlesi & Broth, 2012), meaning that understanding the ways of achieving the solution is considered by the teachers to form an important pedagogical goal for the student. Here, the journey is more important than the destination. By contrast, some problems are, indeed, resolved in an expedient manner by the teacher personally taking control of the device. In such instances the problem encountered was deemed by the teacher to be solvable. In such instances, arriving at the destination is considered by the teacher to be more important than understanding the journey. A further consideration which may influence the teacher’s approach to the problem is their institutional responsibility for ensuring the progressivity of activity. They need to make sure that the class advances, which oftentimes requires more expedient ways of resolving individual students’ issues. While the data also features more traditional teacher-led lecture phases, especially in the beginning part of the sessions, a large majority of the time spent in the class is dedicated to the students practicing using their devices

70

J. RÅMAN

“hands on.” The teachers walk around the class and respond to any questions or issues raised by the students (see e.g., Greiffenhagen, 2012; Jakonen, 2020). In doing so, the teachers typically engage in two types of activities: problem-solving and instructional demonstration. Both activity frameworks (e.g., Broth & Mondada, 2013) require that the teacher has, at least visual, but ideally, haptic access to the digital device being operated by the student(s). Whichever activity the teacher engages in, they first need to access the student’s device, a process, which presents some deontic considerations.

Data Set and Methodology The course examined in this study took place in an adult education center located in a city in Northern Finland during the spring of 2020. All together the course comprised of 12 three-hour sessions. The sessions were structured around specific topics, such as “social media,” “word editors,” and “Google services.” The sessions began with the teacher giving a brief introduction to the topic of the day and handing out tasks which the students were expected to finish during the class. However, in every session, the students were encouraged to ask the teachers questions and help on any topics or problems they might encounter when using their devices, regardless of whether they were related to the topic. Indeed, many of the students utilized these sessions to ask questions on problems they encountered during the week between the sessions. The course had two teachers per session as well as one or two “digital volunteers” (volunteer workers trained by the adult education center to assist with issues related to digital devices), serving as teaching assistants. Furthermore, in addition to filming the sessions, the researcher also assisted the students when necessary. Altogether, the number of students in the class ranged between 15 and 22. The interactions were predominantly in Finnish, though one student was not a native speaker (being a native speaker of Russian). In total, three sessions (9 hours) were recorded with multiple cameras. The recordings were done with 2–3 GoPro cameras, which were attached to shoulder or chest straps and carried by the teachers (and the researcher in two of the recorded sessions), one hand-held video camera carried by the research during one of the sessions, and one tri-pod mounted video camera stationed at the back of the class. To gain access to the setting, the researcher first volunteered to assist the teachers during the class,

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

71

presented their research interests to the participants, and asked for the participants written consent to be recorded. Cameras were only introduced to the classroom on the sixth weekly session. For this study, the interactions between the researcher and the students have been excluded from the data set. The images included in the data excerpts provide the “point of view” of the teacher, acquired from the shoulder or chestmounted GoPro cameras. The chosen camera angle does not fully reveal the teachers’ embodied actions. For this purpose, the camera stationed at the back of the class was utilized where necessary. An external audio recorder was also placed at a central location to ensure adequate sound quality of the recordings. Altogether, the recordings yielded roughly 33 hours of video data and 9 hours of audio data. The primary methodology utilized in this study is multimodal conversation analysis (e.g., Mondada, 2016), which is a data-driven, microanalytic, and qualitative research approach focusing on the empirical analysis of recordings made from naturally occurring interactions. Conversation analysis is, in essence, the study of social actions through the examination of the micro-level features of interaction. At the core of CA is the notion of the turn-taking organization (Sacks et al., 1974) and the various ways in which social meaning, institutions and roles are talked in to being (Heritage, 1984) within interaction. Conversation analysis examines the ways in which turns at talk are produced and organized sequentially to achieve a variety of purposes such as requests, offers, questions, answers, etc. In this way, the microanalytic perspective of conversation analysis can, in fact, produce macro-level observation of the social world. The inclusion of the multimodal aspects of interaction, in the wake of what Nevile (2015) calls the “embodied turn” of the last twenty to thirty years of conversation analytic research, allowed for a re-examination of what already were thought to be well-established systematics of interaction. Examination of modalities beyond talk revealed novel ways to “build and interpret the public intelligibility and accountability” of situated actions (Mondada, 2018, p. 86). Indeed, as this study will show, problem-solving can be an inherently multimodal process, in which talk does play an important, though certainly not dominant role. Typically, conversation analysts approach data without presuppositions in mind (Sacks, 1984; Schegloff, 1996), engaging rather in what is known as unmotivated viewings of the data. In this study, what drew the researcher’s initial interest during these viewings were the moments of teacher touching or manipulating the student’s digital device. There

72

J. RÅMAN

seemed to be two very distinct ways of gaining access to the device: a more straightforward and a more complicated way. Having built a collection of 40 such moments, with detailed multimodal transcripts, the initial analytic goal was to discern the potential reasons for the teachers touching the students’ devices in the classroom. As the language spoken in the class was Finnish, the transcripts include an English translation below the line indicating the original Finnish spoken turns in italics. In addition, this study utilizes the multimodal transcription format pioneered by Mondada (e.g., 2018) in which the onsets and endings of embodied actions are marked in relation to the spoken turns via symbols dedicated to each participant in interaction (see Appendix: Transcription Conventions). In the transcripts, TEA is an abbreviation of teacher and ST1 and ST2 of student 1 and student 2.

Rendering Assistance Before examining the actual transfer of haptic access to the device, it is important to understand the history of the interaction preceding it, namely how, why, and by whom, it is initiated. The data set revealed that in every recorded instance, the actual physical manipulation of the device by the teacher is preceded by interactional sequence where assistance is recruited from the teacher. Kendrick and Drew (2016) introduced the concept of “recruitment” of assistance as a framework encompassing. […] the linguistic and embodied ways in which assistance may be sought— requested or solicited—or in which we come to perceive another’s need and offer or volunteer assistance. (p. 2)

In the majority of the cases observed, the recruitment is self-initiated, meaning that assistance is directly requested (e.g., Drew & CouperKuhlen, 2014; Enfield, 2014) by the students or offered by the teacher following the students reporting trouble. A few rare cases in the data can be classified as other-initiated, meaning that they comprise of the teacher either responding to students’ displayed trouble or projecting potential future trouble and volunteering assistance. In keeping with the basic principles of adjacency pairs (Sacks et al., 1974) both requests and trouble reports by the students are directly followed by the teacher rendering assistance in the data. This is done via two potential paths: (1) the path of instructional demonstration; or (2) the path of problem-solving. The

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

73

two pathways differ in the way access to the device is negotiated. The following sections will examine both modalities of rendering assistance as well as the multimodal ways in which the negotiation sequences unfold within them. Instructional Demonstrations Goffman (1974, p. 66) defines demonstrations as “performances of an activity outside its typical context in order to allow someone who is not the performer to obtain a close picture of the doing of the activity.” By “typical context” Goffman refers to the natural setting in which the demonstrated activity appears in its original, unaltered form. In the context of this study, the original context could be thought to be the everyday moments where the digital devices are used by the students at home. When performed in a setting detached from the original context, this use of digital devices is transformed into a “keying” (Goffman, 1974) and is thought to be altered. Once an original activity becomes a keying, it receives additional, context-sensitive elements. The teaching context, for instance, introduces supportive and annotative aspects to the activity of handling digital devices (e.g., Clark & Gerrig, 1990). The first immediate observation stemming from the data regarding the embodied negotiation sequences preceding instructional demonstrations is that they nearly always follow either the teacher issuing instructions (e.g., Stefani & Gazin, 2014) in a variety of formats ranging from directives (Craven & Potter, 2010; West, 1990) to deictic and multimodal locating statements (“se on tuolla” it’s in there, accompanied by a pointing gesture) or the student directly offering the device to the teacher. Even in the latter case the teacher may at first decline to accept the device and provide the student with verbal instructions. Direct physical manipulation of the device appears, evidently, to be something of a dis-preferred modality when instructing the use of digital devices to the elderly. In the following example the teacher (carrying a GoPro camera on a shoulder strap), standing in front of a screen at the front of the class (see img1), has given the students a task of creating a new folder on the desktop of their laptop computers. The theme of the session is resource management in a windows computer, and the students have just prior been instructed to locate and open “This PC” (fin. Oma tietokone), from where they must locate and open the “Desktop” (fin. työpöytä)”

74

J. RÅMAN

folder under “Quick access” (fin. pikakäyttö ). The excerpt begins with the teacher instructing the whole class where to locate the Desktop folder. One of the students (ST1, sitting on the extreme top left corner of img1) reports trouble by stating that the computer “won’t let me” (Figs. 4.1 and 4.2). Excerpt 1 The student’s initial trouble report on lines 5–6 is produced in a lower volume and it is not necessarily directed at the teacher. In fact, as can be seen from the two small screencaps in img2, it is one of the nearby students who leans toward ST1’s (sitting right-most in the two images) computer, possibly to assist her. On line 3 the teacher walks over to the screen and points at the desktop folder (img2.), providing visual context for her upcoming question. She then asks the entire class in a raised volume whether they have managed to open the desktop folder on line7, providing the interactional space for reporting trouble or asking for assistance. Following this, ST1 re-verbalizes her trouble report on line 8, prompting the teacher to approach her (lines 9–11, img3) and ask her whether she managed to find the Desktop folder on line 10. On line 11, the student, partly in overlap with the teacher’s question, states that she had indeed found the folder but is unable to open it. Having reached the student and her computer, the teacher brings her left hand in a pointing gesture toward the screen (img4). At first, she brings the index finger close to the left of the screen where the quick access bar can be found (img5). After, this she brings the pointing gesture toward the Desktop symbol (img6). In this way, she establishes that ST1 has indeed managed to locate the correct object on the screen. On line 11, the student invites the teacher to witness as she re-enacts the issue (Evans & Reynolds, 2016; Tutt & Hindmarsh, 2011) and clicks on the Desktop. Following, and partly overlapping with this reenactment, the teacher issues the first instructions in a directive form on line 12 (öö:: (.)kaksoisklikkaa—umm:: (.) double click). During this instructional turn the teacher also performs a clicking gesture with her left index finger and returns the hand to home position (Cibulka, 2015; Sacks & Schegloff, 2002) (img7–8). During the ensuing 2.9 second silence, the student proceeds to click on the trackpad two times, failing to pace her clicks fast enough for the computer to register them as a double-click

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

Fig. 4.1 Excerpt 1, part 1

75

76

J. RÅMAN

Fig. 4.2 Excerpt 1, part 2

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

77

command. During ST1’s failed attempt to follow the teacher’s instructions, the teacher gradually brings her right hand closer to the trackpad with her index finger extended (img9). During lines 13–14 she performs another more pronounced clicking gesture with her index finger (img10). Whether or not this gesture is produced specifically for-witness by ST1 is somewhat unclear. It is, however, certainly witnessable (Nevile, 2007) and functions as an embodied demonstration of the necessary movement. Following the third failed attempt to open the Desktop folder, and the teacher’s clicking gesture, the student retracts her hands from the trackpad (line 14, image 11), thus relinquishing her control of the device and providing the physical space required for the teacher to resolve the issue. Following this relinquishing, the teacher brings her right hand to the trackpad and successfully double clicks on the Desktop (lines 14–15, img12), performing another embodied demonstration of double clicking and accompanying it with an annotation (Clark & Gerrig, 1990) on lines 16–17. While she double clicks on the icon, she produces a turn at talk on line 14 whose function is twofold: it accounts for the teacher taking control of the device, and it builds rapport between ST1 and TEA. In the data, the teacher always produces some variation of this “controlassuming turn” just prior to, or during, the moment when she handles the device. In this excerpt, the rapport is built on the premise of the computer refusing to obey ST1’s (and TEA’s) commands, utilizing the negative (semi)auxiliary verb construction ei anna mennä (won’t let [you] go). Of note, is the fact that the ditransitive verb antaa in this construction is lacking the indirect object, the person who is not “let go” (e.g., “you”), resulting in a connotation of the computer refusing entry to the Desktop regardless of the person using it. In this way, the turn at talk on line 14 also has a face-saving function. Other common modalities of this controlassuming turn encountered in the data include: use of the plural first person form of the verb “look”—“katsoa” in indicative form (katsotaan— let’s look); imperative form of the verb “show”—“näyttää” combined with softening tone particles “-pä,” “-päs,” or “-s” (näytäpäs, näytäs — show me)1 ; and straightforward requests such as “Saanko katsoa?—Can I take a look?.” In summary, the grammatical formulations of the controlassuming turn seem to indicate that the teacher strives to establish a joint cause shared by two deontically equal parties. It is the student who shows the device to the teacher, and it is the student and the teacher who both

78

J. RÅMAN

look at it. Most importantly, it is the device that seeks to thwart them both. In this excerpt, the teacher treats the reported trouble primarily as a learnable (Majlesi & Broth, 2012). This becomes evident through the variety of instructions she produces prior to handling the device herself in an instructional demonstration. At first, she seeks to locate the correct object on the screen through a pointing gesture. Having done this, she provides instructions in both verbal and embodied form. Seeing that her instructions have not resolved the issue, she slowly brings her right hand closer to the device. This gradual, phase-by-phase approach to the device allows the student the opportunity to resolve the issue on her own before the teacher must resort to an embodied demonstration on the device itself in order to secure the progressivity of the activity. The data suggests that once a problem is deemed “learnable,” the teachers in their embodied and verbal interaction display a marked preference for the student to remain in control of the device. Indeed, it is easy to argue that the learnable in this excerpt—double clicking with the trackpad—is something that is best learned via doing as acquiring the correct timing and precision requires haptic access to the device. Often, when instructional demonstrations are utilized, they seem to be something of a last resort. Problem-Solving At times, the teachers treat the students’ problems with the digital devices as “solvables” rather than as “learnables.” In such instances, the negotiation for the right to access the device seems to be much more straightforward. The interactions preceding the actual handling of the device by the teacher do not, to the same extent as in the previous excerpt, build rapport between the two. Instead, the teacher’s deontic authority over the student becomes more pronounced in these problemsolving sequences and the student’s agency (Ahearn, 2001) seems to be diminished. Two distinct categories of “solvables” can be identified from the data. Certain problems may be treated as too simple to be worthy of pedagogic focus. Conversely, certain problems may be treated as too complex or time consuming to be resolved via step-by-step instruction. In addition, with over 15 participants in each session, the teachers conduct can certainly be influenced by the fact that they are pressed for time. Often, simply resolving the issue themselves is the most efficient way of securing the progressivity of the activity.

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

79

The next excerpt takes place at the very beginning of the session. The students are engaged in taking their seats and turning on their laptops or tablets. Just prior to the beginning of the segment, the teacher, wearing a chest-mounted GoPro camera and standing in front of the class, has finished going through a list of attendees for the session. Sitting leftmost on img1 is ST1, who is struggling with her laptop (a device she has borrowed from the classroom). She turns the closed laptop around, simultaneously moving two cords around: evidently searching for the correct ports for the power cord and the mouse. Whether or not the teacher pays attention to ST1’s issues with the laptop is unclear, but after the student turns the laptop the wrong way (the screen facing the teacher, see img2), the teacher begins to walk towards the student (img3) (Figs. 4.3 and 4.4). Excerpt 2 Unlike in the first excerpt, here the recruitment of assistance is otherinitiated (Kendrick & Drew, 2016). The student does not shift her gaze toward the teacher as she turns the laptop in front of her. Here then, the teacher responds to perceived trouble by the student and volunteers her assistance. As the teacher is walking around the table separating her from ST1, she begins to resolve the first issue the student faces: ST1’s desk is cluttered with different cords. While assuming control of the device on lines 4–6 she produces the spoken turns sen voi oikeastaa jättää pois tämän- tämän johdon—tätä ei tarvita tässä vaiheessa—(that can actually be removed- this cord- this is not needed at this point) and takes hold of the power cord, which, as can be seen in the last frame of image 4, the student is still holding, and places it aside. The turn establishes to the student that the power cord is unnecessary. The laptops have been fully charged prior to the class. The student, in overlap with the teacher, confirms this on line 7 (not needed at thi-), to which the teacher provides a response on line 8. After the cord is removed from the student’s hands, ST1 takes a hold of the laptop with both her hands (line 7, img5). After setting aside the cord, the teacher also takes hold of the laptop (img5). The student immediately relinquishes her hold of the device, and the teacher turns it so that the screen will be facing the student once the device is opened (lines 9– 10, img6). The teacher then begins to resolve the next issue: locating the USB-port for the mouse cord. While setting the device down after

80

J. RÅMAN

Fig. 4.3 Excerpt 2, part 1

turning it, the teacher produces a pointing gesture with her right index finger toward the USB-port and produces a locating statement on line 12 (img7). While syntactically formulated as a question and potentially produced as self-talk (for discussion on self-talk in educational settings, see Hall & Looney, 2021), the accompanying pointing gesture marks this turn as directive (Craven & Potter, 2010; Ervin-Tripp, 1976), prompting the student to provide an embodied response by plugging in the mouse cord on lines 13–17. Directly after the pointing gesture, the teacher brings her right hand on top of the laptop (lines 12–13), to a position

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

81

Fig. 4.4 Excerpt 2, part 2

from which the screen can be lifted. After the student plugs in her mouse (first frame of img8), the teacher pulls open the laptop screen (last two frames of img8), annotating her actions on line 18. At the end of the segment, the teacher produces a directive to turn the device on, after which she walks back to the front of the class, taking the power cord with her. Compared to the learnable of the first example—the act of double clicking on the mouse—the issues encountered by the student on Excerpt 2 are primarily treated as a solvables and subsequently resolved in a more

82

J. RÅMAN

direct and rapid fashion. The teacher does not hedge her hands in phase by phase toward the device. She does not initiate the problem-solving by providing verbal or embodied instructions, but rather immediately takes control of the power cord and device, in two occasions taking an object directly from the student’s hands. In Excerpt 1, the student first relinquishes control of the laptop by retracting her hands prior to the teacher handling the device. In Excerpt 2, the student’s and the teacher’s hands are both touching the power cord and the laptop before the student lets go of the objects. The teacher’s turns at talk do not build rapport between ST1 and TEA and are often produced in overlap with the ST1’s turns. The student also produces a truncated turn at talk on line 11 which prompts no response from the teacher. Excerpt 2 is an instance where the problem to be solved is treated as a solvable likely because teaching it via talk and demonstration is both unpractical and needlessly complex. Furthermore, verbally walking the student through the problem might be considered a face-threatening act, potentially coming across as patronizing or condescending. The acts of setting a power cord aside, turning a laptop the other way, and opening the screen are easier done than said. Rather than going through a set of instructions and directives the teacher most likely finds it more expedient to resolve the issues herself. She does still verbally account for her actions on lines 17 and 20 in an annotative way which both suit and reinforce the institutional and instructional nature of teacher-student interactions. Nonetheless, the way the teacher approaches the student’s problem in excerpt, 2 and in the data in general, indicates that the handling of digital devices and attached paraphernalia (such as power cords) as physical objects do not form the pedagogical core content of the sessions, and thus tend to not be established as learnables in interaction. Rather, to ensure the progressivity of the class, the teachers often resolve such practical and physical problems themselves. Excerpts 1 and 2 both feature problems which the teachers manage to resolve relatively fast because in both instances the source of students’ trouble is immediately evident, as is the solution. This is largely due to the way in which the problem is made visible, either intentionally or unintentionally, by the students. In the first excerpt, the teacher can witness the student’s slow double-click on the trackpad and observe the resulting non-response from the device. In the second excerpt, the student is seated close to the teacher who can directly observe her moving the computer

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

83

and related paraphernalia around. The teachers do not need to dedicate time to understanding the problem themselves prior to assisting the student. At times, however, the issues the students encounter are not as easily identified or resolved. Excerpt 3 features a situation where the teacher at first must handle the device personally before the problem can be pinpointed and resolved. In the following excerpt, ST1 (handling the tablet in img1) is attempting to show ST2, who is sitting next to her, a game she used to enjoy playing on her tablet. The teacher, wearing a shoulder-mounted GoPro camera, is leaning on the table to the right of the two students. The game ST1 attempts to show is evidently an instructional game which teaches coding, and the student has not played it, according to her own words, in two or three years. As she tries to open the app, found under the My Playgrounds sub-folder, the game fails to start. Not included in the transcript is a preceding two-minute segment where ST1, ST2, and TEA discuss different games they, their children, or grandchildren have played. It is during this discussion that ST1 first brings up the coding game, which she then proceeds to try to demonstrate to her co-participants. Prior to the segment included here, she has unsuccessfully attempted to open the app one time. Her second failed attempt (lines 1–2, img2) and the accompanying trouble report prompt the teacher to engage in solving the issue (Figs. 4.5 and 4.6). Excerpt 3 In the very beginning of the segment, ST1 can be seen tapping on the game icon (first frame of img2) which moves the icon on the screen to the top left corner (second frame of img2). There, the icon changes shape and color. The student taps on the icon again (third frame of img2), which removes the icon from the screen (final frame of img2). While engaged in tapping on the icon, the student first produces a trouble report (line 2), a trouble alert (line 3), followed by another trouble report (line 4). Following this combination of recruitment modalities (Kendrick & Drew, 2016), the teacher moves (lines 5–7) to a position between ST1 and ST2, from where she will have a clearer visual, but more importantly, haptic access to the device (img3). As she moves closer, she produces a turn at talk on line 5 which indicates that she will seek to acquire visual access to the device (mitä siellä on—what’s in there). Coinciding with the teacher

84

J. RÅMAN

Fig. 4.5 Excerpt 3, part 1

arriving on the new location, ST1 leans back on her seat, thus providing the teacher more room and facilitating easier haptic and visual access to the tablet. The student’s turns at talk on lines 6–7, ST1 provide a potential account on why she is struggling to open the game: she has not played it in two or three years. They also function to establish the student’s epistemic status regarding the game as “knowing (K+)” (Heritage, 2012). Lines 9 and 10 feature overlap where ST1 produces an assessment of the game and TEA reads aloud the name of the sub-folder in which the game icon can be found. Turn 10, produced with a lower volume of voice while assuming control of the device, does not build rapport between

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

85

Fig. 4.6 Excerpt 3, part 2

ST1 and TEA. The student’s assessment on line 9 prompts no response from the teacher, who proceeds to tap on the game icon with her right index finger (line 11, img4). At the moment of touching the screen, she says the name of the game “Learn to Code” again with a lower volume of voice (line 11). The game icon can be seen moving to the top left corner (img5). The teacher responds to this with a rising intonation on line 13, after which she says the name of another game found in the My Playgrounds sub-folder Hello byte/bite (once more in lower volume of voice) and proceeds to tap that icon as well (line 15, img6). By verbalizing the different icons, she sees on the screen (on lines 10, 11, and 15), the teacher most likely seeks to signpost to the student her own process of problem-solving. The icon of the second game is also seen to move to the top left corner of the screen (img7). Following this, ST1 tells the teacher more about the game on line 16, a turn at talk which again asserts her

86

J. RÅMAN

K+ status but prompts no response from the teacher. Having witnessed both icons move to the top left corner of the screen and change shape, the teacher evidently discovers the source of the problem, and provides an account and a solution (on line 17) on why the game fails to start: it is being downloaded to the tablet. It is likely that the long hiatus (two to three years) the student has taken in playing the game has forced the device to install updates to the game as the app is being opened. As the student was tapping and re-tapping the icons, she was in fact starting and canceling the download process. To be able to assist the student, the teacher requires haptic access to the device. She must tap on the icon on the tablet herself to understand the process which follows. It is important to also note that unlike in Excerpts 1 and 2, the problem encountered by the student in Excerpt 3 is also shared by the teacher, who at the beginning of the segment is equally unaware of the solution. Access to the device is not explicitly negotiated, as the teacher simply brings her index finger to the screen. The student facilitates this by not resisting the teacher’s actions. Teacher’s quick access to the device is further facilitated by the student who leans backwards as the teacher takes her position between ST1 and ST2. The student’s ignored turns at talk on lines 9 and 16, and the quiet production of the annotations of the teacher’s actions on lines 10, 11, and 15, indicate that TEA is engaged on discovering a solution and not on demonstrating. The teacher’s turns at talk do not pinpoint the trouble or provide instructions prior to turn 17, where she can clearly state to the student the source of the issue. Excerpt 3 suggests that the way in which control of the device is transferred to the teacher is dependent not only on the teacher’s institutional role as an expert, but also on her actual epistemic status regarding the encountered problem. To treat an issue as a learnable, the teachers must understand it themselves. If this is not the case, the issue is treated as a solvable and control of the device is transferred in a more straightforward manner. Excerpts 1 and 3 show that teachers seem to treat learning in the context of digital device use as a primarily haptic process. In Excerpt 1, the teacher exhausts a selection of modalities which facilitate for the student to remain in control of the device for as long as possible. Arguably, the best way to learn the timing of the double-click is repeated attempts. In Excerpt 3, it is the teacher who at first must make sense of the trouble, and subsequently requires immediate haptic access to the device before

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

87

being able to teach the student. For the teacher to acquire an understanding of the trouble in Excerpt 3, verbal instructions issued to the student would likely have proved to be a needlessly complicated method. Conversely, when there is nothing to learn, as is the case in Excerpt 2, student’s haptic access to the digital device, can be discarded to resolve the issue in a more expedient fashion.

Concluding Remarks This study set out to examine the activity contexts where teachers handle the students’ devices. Examination of the data revealed that this is done to render assistance to the students in moments where they have encountered a problem. When engaged in responding to students’ self-initiated recruitment of assistance (Kendrick & Drew, 2016), that is, requests of assistance or reports of trouble, the modality of the rendered assistance is not dependent on the modality of the recruitment, but on the nature of the trouble as well as on the epistemic status of the teacher. When issues encountered by the students in the classroom are “interactionally established as relevant and developed into a shared pedagogical focus” (Majlesi & Broth, 2012, p. 1), they become learnables. Learning to use digital devices requires haptic access to the devices in question. As shown in the three excerpts, this holds true both for the students and the teachers. In Excerpt 1, the teacher provided the student with ample opportunity to resolve the issue herself prior to resorting to an embodied demonstration. In Excerpt 3, the teacher first must handle the device herself to familiarize herself with the problem, before she can resolve it. The multimodal negotiation sequences preceding embodied demonstrations reflect this preference for hands-on-learning. The teachers tend to use verbal and gestural instructions prior to assuming control of the device. When they do, they seek to build rapport between themselves and their students. It is interesting to note that whether the digital device is actually owned by the student has no impact on the way transfer of access is conducted by the participants. The laptop computer featured in Excerpt 2 was the property of the adult education center, while in Excerpt 3 the featured tablet belonged to the student. In both cases, however, the transfer of access followed a similar, straightforward path. Deontic considerations pertaining to the ownership, or indeed perceived epistemic authority regarding the details and use, of said device seem to play no role when

88

J. RÅMAN

the teacher engages in problem-solving. Though not discernible from the included excerpts of this chapter, the data set revealed that this holds true for instructional demonstrations as well. There were no clear differences in the way transfer of access is negotiated for the purpose of instructional demonstration depending on whether the devices were borrowed from the classroom or brought by the students themselves. It is, however, undeniable that digital devices such as tablets and smartphones are personal objects and gaining access to another’s device would normally require a negotiation sequence where a reason for transferring the object is given. Evidently then, the classroom, as an institutionally laden space, influences the participatory roles of not only people, but of objects. The nature of these devices changes the very moment they are brought to the classroom by the student. They turn from personal devices to shared pedagogical tools, and as such can be accessed by the teacher more readily. Conversely, when examining the ways in which the teachers deploy multiple other modalities prior to physically demonstrating the solution to a problem, and the way in which they seek to build rapport with the students, it becomes clear that any devices the teacher brought to the class have also, to some degree, become a shared property of the student and the teacher. Traditionally, classroom settings see the teachers endowed with a certain level of epistemic (e.g., Heritage & Raymond, 2005) and deontic (Stevanovic & Peräkylä, 2012) authority over the students. However, the teachers’ conduct in the examined negotiations seems to suggest that the students also exert certain authority over the teacher in relation to the ownership and right to access their digital devices (regardless of who actually owns said devices). The setting examined in this chapter seems to be so clearly objectcentered (Tuncer et al., 2019) that instead of object-transfers or transfers of access, it might even be justifiable to discuss of transfer of usership. After all, as the excerpts show, the devices themselves do not physically move a great deal, nor does the goal that the participants seek to achieve with devices change. It is, rather, the people using them that re-configure their participation in relation to these devices. The findings discussed in this chapter suggest that the participants in their interactions actively seek to resolve issues in collaboration, striving for joint problem-solving. The interactions examined here show a marked preference for shared access to the device, a preference which is displayed in the conduct of both participants.

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

89

Appendix: Transcription Conventions See Table 4.1.

Note 1. It is interesting to note that the data does not include a single instance of the teacher utilizing the imperative form of the verb “give”—“antaa” in its literal meaning (anna—give me) when assuming control of the device. The verb “show”—“näyttää” certainly assigns the student more agency (Ahearn,

Table 4.1 Transcription conventions

Italics

English translation of the Finnish speech turn

CAPITAL LETTERS IN Embodied conduct GRAY FONT TEA: / ST1:

Uppercase speaker tags denote spoken turns

tea: /st1

Lowercase speaker tags denote embodied actions

* #£

The beginning of embodied conduct in relation to speech

-----

Continuation of embodied conduct

->*

The end of embodied conduct

-># £>>

Embodied conduct began prior to the transcribed segment

::

Prolonged pronunciation

(img.1)

Indicates the location of the accompanied illustration in the transcript in relation to speech

(.)

Short pause (under 0.3 seconds)

(2.0)

Longer pause (length in brackets)

[]

Overlapping speech

(speech)

Transcriber uncertain of what is said

°

Lower volume of voice

(hh)

Audible outbreath

90

J. RÅMAN

2001) than “antaa”—the student allows the teacher a glimpse of the device even when the device is held and controlled by the teacher.

References Ahearn, L. (2001). Language and agency. Annual Review of Anthropology, 30, 109–137. Alexopoulou, S. (2020). The portrait of older people as (non) users of digital technologies: A scoping literature review and a typology of digital older (non) users. Gerontechnology, 19(3), 1–15. https://doi.org/10.4017/gt.2020.19. 003.11 Broth, M., & Mondada, L. (2013). Walking away. The embodied achievement of activity closings in mobile interaction. Journal of Pragmatics, 47 (1), 41–58. Cibulka, P. (2015). When the hands do not go home: A micro-study of the role of gesture phases in sequence suspension and closure. Discourse Studies, 17 (1), 3–24. Clark, H., & Gerrig, R. (1990). Quotations as demonstrations. Language, 66(4), 764–805. Craven, A., & Potter, J. (2010). Directives: Entitlement and contingency in action. Discourse Studies, 12(4), 419–422. Cutler, S. J. (2005). Ageism and technology. Generations, 29(3), 67–72. Czaja, S., & Lee, C. (2002). Designing computer systems for older adults. In J. Jacko & A. Sears (Eds.), Handbook of human-computer interaction (pp. 413– 427). Lawrence Erlbaum and Associates. De Stefani, E., & Gazin, A.-D. (2014). Instructional sequences in driving lessons: Mobile participants and the temporal and sequential organization of actions. Journal of Pragmatics, 65, 63–79. Dixon, S. (2015). Gimme! Gimme! Gimme!: Object requests, ownership and entitlement in a children’s play session. Journal of Pragmatics, 82, 39–51. Drew, P., & Couper-Kuhlen, E. (2014). Requesting—From speech act to recruitment. In P. Drew & E. Couper-Kuhlen (Eds.), Requesting in social interaction (pp. 1–34). John Benjamins. Elleström, L. (2018). A medium-centered model of communication. Semiotica, 224, 269–293. Evans, B., & Reynolds, E. (2016). The Organization of corrective demonstrations using embodied action in sports coaching feedback. Symbolic Interaction, 39, 525–556. Enfield, N. (2014). Human agency and the infrastructure for requests. In P. Drew & E. Couper-Kuhlen (Eds.), Requesting in social interaction (pp. 35– 54). John Benjamins. Ervin-Tripp, S. (1976). Is Sybil there? The structure of some American English directives. Language in society, 5(1), 25–66.

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

91

Fox, B., & Heinemann, T. (2015). The alignment of manual and verbal displays in requests for the repair of an object. Research on Language and Social Interaction, 48(3), 342–362. Goffman, E. (1974). Frame analysis: An essay on the organization of experience. Northeastern University Press. Greiffenhagen, C. (2012). Making rounds: The routine work of the teacher during collaborative learning with computers. International Journal of Computer-Supported Collaborative Learning, 7 (1), 11–42. https://doi.org/ 10.1007/s11412-011-9134-8 Hall, J. K., & Looney, S. D. (2021). The role of self-talk in downgrading a teacher’s certainty about grammar matters. TESOL Quarterly, 55(1), 185– 218. Heritage, J. (1984). Garfinkel and ethnomethodology. Polity Press. Heath, C., Luff, P., Sanchez-Svensson, M., & Nicholls, M. (2018). Exchanging implements: The micro-materialities of multidisciplinary work in the operating theatre. Sociology of Health & Illness, 40(2), 297–313. Heritage, J. (2012). Epistemics in action: Action formation and territories of knowledge. Research on Language and Social Interaction, 45(1), 1–29. Heritage, J., & Raymond, G. (2005). The terms of agreement: Indexing epistemic authority and subordination in talk-in-interaction. Social Psychology Quarterly, 68(1), 15–38. https://doi.org/10.1177/019027250506800103 Horlacher, A.-S. (2019). Workplace asymmetries and object-passing in hair salons. In D. Day & J. Wagner (Eds.), Objects, bodies and work practice (pp. 33–60). Multilingual Matters. Jakonen, T. (2020). Professional embodiment: Walking, reengagement of desk interactions, and provision of instruction during classroom rounds. Applied Linguistics, 41(2), 161–184. https://doi.org/10.1093/applin/amy034 Kendrick, K., & Drew, P. (2016). Recruitment: Offers, requests, and the organization of assistance in interaction. Research on Language and Social Interaction, 49(1), 1–19. https://doi.org/10.1080/08351813.2016.1126436 Leung, R., Findlater, L., McGrenere, J., Graf, P., & Yang, J. (2010). Multilayered interfaces to improve older adults initial learnability of mobile applications. ACM Transactions on Accessible Computing (TACCESS), 3(1), 1–30. Lin, C., Hsieh, T., & Shiang, W. (2009). Exploring the interface design of mobile phone for the elderly. In M. Kurosu (Ed.), Human centered design, HCII 2009, LNCS 5719 (pp. 476–481). Springer-Verlag. Majlesi, A., & Broth, A. (2012). Emergent learnables in second language classroom interaction. Learning, Culture and Social Interaction, 14(3–4), 193–207.

92

J. RÅMAN

Mallenius, S., Rossi, M., & Tuunainen, V. (2007). Factors affecting the adoption and use of mobile devices and services by elderly people–results from a pilot study. Proceedings of 6th Annual Global Mobility Roundtable, Los Angeles. McLeod, E. (2009). The use (and disuse) of mobile phones by baby boomers. International Journal of Emerging Technologies and Society, 7 (1), 28–38. Mondada, L. (2016). Multimodal resources and the organization of social interaction. In A. Rocci & L. Saussure (Eds.), Verbal communication (pp. 329–350). De Gruyter. Mondada, L. (2018). Multiple temporalities of language and body in interaction: Challenges for transcribing multimodality. Research on Language and Social Interaction, 51(1), 85–106. Nevile, M. (2007). Seeing the point: Attention and participation in the airline cockpit. In L. Mondada & V. Markaki (Eds.), Interacting bodies: Online proceedings of the 2 nd international conference of the International Society for Gesture Studies. ENS LSH & ICAR Research, Lyons, France. Nevile, M. (2015). The embodied turn in research on language and social interaction. Research on Language and Social Interaction, 48(2), 121–151. Oksman, V. (2006). Young people and seniors in Finnish ‘Mobile Information Society’. Journal of Interactive Media in Education, 20(2). https://doi.org/ 10.5334/2006-3 Rauniomaa, M., & Keisanen, T. (2012). Two multimodal formats for responding to requests. Journal of Pragmatics, 44(6–7), 829–842. Rippl, G. (Ed.). (2015). Handbook of intermediality: Literature—Image— Sound—Music. de Gruyter. Rossi, G. (2014). When do people not use language to make requests? In P. Drew & E. Couper-Kuhlen (Eds.), Requesting in social interaction (pp. 303– 334). John Benjamins. Roupa, Z., Nikas, M., Gerasimou, E., Zafeiri, V., Giasyrani, L., Kazitori, E., & Sotiropoulou, P. (2010). The use of technology by the elderly. Health Sciences Journal, 4(2), 118–126. Sacks, H. (1984). Notes on methodology. In J. Heritage & J. Atkinson (Eds.), Structures of social action: Studies in conversation analysis (pp. 2–27). Cambridge University Press. Sacks, H., & Schegloff, E. (2002). Home position. Gesture, 2(2), 133–146. Sacks, H., Schegloff, E., & Jefferson, G. (1974). A Simplest systematics for the organization of turn-taking for conversation. Language, 50(4), 696–735. Schegloff, E. (1996). Confirming allusions: Toward an empirical account of action. American Journal of Sociology, 102(1), 161–216. Stevanovic, M., & Peräkylä, A. (2012). Deontic authority in interaction: The right to announce, propose and decide. Research on Language and Social Interaction, 45(3), 297–321.

4

MULTIMODAL NEGOTIATION FOR THE RIGHT …

93

Takada, A., & Endo, T. (2015). Object transfer in request-accept sequence in Japanese caregiver-child interactions. Journal of Pragmatics, 82, 52–66. Thurlow, C. (2015). Multimodality, materiality and everyday textualities: The sensuous stuff of status. In G. Rippl (Ed.), Handbook of intermediality: Literature—Image—Sound—Music (pp. 619–636). de Gruyter. Tuncer, S., & Haddington, P. (2019). Object transfers: An embodied resource to progress joint activities and build relative agency. Language in Society, 49(1), 61–87. Tuncer, S., Licoppe, C., & Haddington, P. (2019). When objects become the focus of human action and activity: Object-centred sequences in social interaction. Gesprächsforschung: Online-Zeitschrift zur verbalen Interaktion, 20, 384–398. Tutt, D., & Hindmarsh, J. (2011). Reenactments at work: Demonstrating conduct in data sessions. Research on Language and Social Interaction, 44(3), 211–236. Weilenmann, A., & Lymer, G. (2014). Incidental and essential objects in interaction: Paper documents in journalistic work. In M. Nevile, M. Rauniomaa, P. Haddington, & T. Heinemann (Eds.), Interacting with objects: Language, materiality, and social activity (pp. 319–338). John Benjamins. West, C. (1990). Not just ‘Doctors’ Orders’: Directive-response sequences in patients’ visits to women and men physicians. Discourse & Society, 1, 85–112. Zinken, J., & Ogiermann, E. (2013). Responsibility and action: Invariants and diversity in requests for objects in British English and Polish interaction. Research on Language and Social Interaction, 46(3), 256–276.

CHAPTER 5

Zooming in on a Frame: Collectively Focusing on a Co-participant’s Person or Surroundings in Video-Mediated Interaction Mari Holmström, Mirka Rauniomaa, and Maarit Siromaa

Workplace breaks have a long tradition in Finland, and depending on the length of the working day, employees are legally granted a certain number of breaks (The Finnish Working Time Act TAL 872/2019:24 §). Finns are also known for their enthusiasm for coffee, and Finnish workplaces often feature break rooms of varied proportions and sizes that are

M. Holmström (B) · M. Rauniomaa · M. Siromaa Research Unit for Languages and Literature, University of Oulu, Oulu, Finland e-mail: [email protected] M. Rauniomaa e-mail: [email protected] M. Siromaa e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_5

95

96

M. HOLMSTRÖM ET AL.

also equipped to varying degrees but that, at the bare minimum, provide facilities for preparing a cup of hot beverage (e.g., Clausnitzer, 2021; ‘Coffee Culture in Finland’, 2021). In many workplaces, regular coffee breaks seem to be a fixed practice with contacts and communities formed around it, and they may aid in setting the pace for a routine working day. Indeed, much in the way of the Swedish ‘fika’, which refers to a custom of taking a break from an activity to drink coffee, eat sweet or savoury snacks and chat with others, Finnish coffee breaks have developed into highly ritualized ways of spending time together. Shared coffee breaks provide an opportunity of getting to know one’s workmates and maintaining social relationships as well as sharing knowledge and information at the workplace. With the COVID-19 pandemic and the subsequent exile to home offices in Finland and around the globe in early 2020, members of work communities needed to find new ways of connecting with each other when break-taking at the office was no longer possible. Some adopted video-mediated break-taking and began to meet on virtual platforms, such as Zoom, Skype and Teams. Virtual breaks enable social interaction from afar, as well as a certain kind of intermission from work activities, even though they may not allow leaving the computer and physically moving away from a workstation to a separate break room. During virtual breaks, even if filters or background images are applied, the video connection grants participants partial visual and auditory access to others and their respective environments. Because these environments are often personal, in addition to views of others’ (upper) bodies, participants are privy to a view of various self-extensions, like homes and others that participants are responsible for, such as pets and kids (Pillet-Shore, 2017, 2018). This constitutes a new kind of a break experience that might somewhat paradoxically be perceived in some respects more personal but at the same time more distanced due to the lack of physical co-presence and the essentially two-dimensional rendering of the virtual platform. Bearing this general context in mind, we study recordings of virtual breaks held among members of relatively long-standing work communities in Finland, most of which have established shared practices of break-taking in person already before the pandemic. We focus on interactional sequences in which a participant registers or notices (see e.g., Kesselheim et al., 2021; Pillet-Shore, 2020) something about another’s person, background or activity, and verbally topicalizes it, making it relevant for the ongoing interaction, and thus in a way zooms in on a frame.

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

97

In examining these sequences, we begin with the notion of ‘social curiosity’ (Hartung, 2010; Renner, 2006) that refers to the way in which people actively take steps to acquire social information and knowledge of others to form interpersonal relations and manage their social environment. That is, such sequences are seen as situated instances of creating and maintaining social intimacy and co-presence, a sense of ‘being there’ for and with others in a shared virtual space. While the video-mediated nature of the interactions can require the use of additional resources and practices, the participants nevertheless draw on recognizable verbal and embodied means. We adopt a multimodal approach which allows us to take into consideration the ‘diversity of resources that participants mobilize to produce and understand social interaction as publicly intelligible actions’ (Mondada, 2019a, p. 1). That is, we look into the ways in which participants interact with each other through language, gaze, gestures, body posture and movement, and manipulation of artefacts and objects in their immediate environment, among others. At the same time, we explore the data from an intermedial point of view, considering the role of the virtual medium, and how meanings and social actions are produced through and across the different resources at the participants’ disposal (see Elleström, 2021; Rippl, 2015). Further, we take note of the participants’ displays of their sensory experiences (Mondada, 2021), and how these may be established, in a way, as a common focus of attention in interaction.

Data and Methods In this study, we draw on approximately 10 hours of recorded virtual breaks, collected between April 2020 and April 2021 in two work communities in Finland. In both communities, shared break-taking is a routinized practice that, at least among some members, was established before the period of distance working. Most of the participants are familiar with each other and have, over time, accumulated knowledge about others and their personal lives. The participants used video-conferencing software for their shared breaks, and the breaks were recorded using the software’s built-in function to capture both video and audio. Most of the breaks were recorded by one participant. Due to the use of the recording function, no general assumptions can be made about what kind of views the participants have (e.g. speaker view, gallery view), how the frames are arranged in their

98

M. HOLMSTRÖM ET AL.

respective views, or indeed if they are looking at a view of other participants at all. It also appears that participants use various devices for attending breaks, such as phones, laptops or the combination of a laptop and a screen. Some inferences can be made by examining participants’ gaze (for example, do they appear to be focused on something else than the ongoing break), but it is not possible to make any claims about what exactly they may be looking at. However, participants can interact during virtual breaks even if they do not have access to the exact gaze direction of others. From a conversation-analytic perspective, then, these kinds of data are not considered problematic as the participants and analysts equally have access to a screen view and audio. In the data, we found 27 sequences of social curiosity in which participants direct joint attention to the person, background or activity of another participant. We have transcribed the selected examples multimodally, to allow for a holistic view on interaction and an examination of the various vocal, embodied, material, semiotic and spatial resources drawn on by participants to carry out social actions (see e.g., Goodwin, 2000; Mondada, 2013, 2016). We examine the means employed by participants to create and maintain co-presence and social intimacy by referring to and thus bringing aspects of life from beyond the screen to virtual breaks. To this end, we adopt a perspective that allows us to produce a rich description of the phenomenon and to show how it unfolds in interaction. We apply the multimodal conversation-analytic method in discussing the data examples and analyzing them on a turn-by-turn basis. Furthermore, we explore the complex multimodal and spatial dimensions of virtual breaks and the characteristics related to sensorial experiences and intermediality as they appear and are made salient in video-mediated encounters.

Studying Shared, Lived Experiences in Virtual Environments While break-taking may centre around consuming beverages and having a moment of rest away from one’s workstation, it facilitates recovery (Hunter & Wu, 2016; Kim et al., 2017) and aids in establishing and maintaining social relationships in the workplace (see Siitonen & Siromaa, 2021), which can be beneficial for work as well (Barmeyer et al., 2019; Liberati et al., 2019). Indeed, in a Danish workplace setting, for instance, coffee break encounters were connected to the formation of ‘coping

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

99

communities’ at work (Stroebaek, 2013; see also Korczynski, 2003). For work communities that regularly take breaks together, and to whom the break rooms as places themselves had become significant (see e.g., Relph, 1976; Seamon & Sowers, 2008), the move to distance working in the spring of 2020 due to the global COVID-19 pandemic brought about a new kind of reliance on video-mediated interaction, moving break-taking online as well. Video-mediated interaction relies on specific technology, which has the potential to restrict and facilitate interaction in various ways (Due & Licoppe, 2021, p. 6; see also contributions to the special issue). For example, when engaging in interaction on a video-mediated platform, participants have asymmetrical access to others and their visible surroundings (for a comprehensive overview, see Mlynáˇr et al., 2018), and additional multimodal and semiotic resources may be needed to deal with technical challenges. In a video-mediated setting, the ‘perceptual range’ (see Hutchby, 2001, 2014) is limited, as participants see only a portion of their co-participants and their surroundings, and auditory cues might be more difficult to locate. Indeed, a video-mediated setting can be characterized as comprising of a compilation of ‘talking heads’ (see Licoppe & Morel, 2012) and presenting ‘fractured ecologies’ (see Luff et al., 2003). It offers users a two-dimensional view, and depending on their settings and preferences, they see a certain constellation of participants. Furthermore, in video-mediated interaction, participants may see themselves participating in interaction, which is usually not the case in in-person interaction. What is more, because participants of video-mediated interaction have some form of access not to one, but potentially several spaces, they may orient to something in any of those spaces as relevant for interaction (see Licoppe, 2015; Licoppe & Veyrier, 2017). Instead of focusing on technology as a certain kind of context, studies on interaction on virtual platforms have focused on what participants themselves make salient during mediated interaction (see e.g., Arminen et al., 2016; Fornel, 1996; Oittinen, 2020). Further, technologies have been shown to provide participants with affordances (Hutchby, 2001, 2014; Oittinen, 2020; on ‘affordances’, see Gibson, 1982, as cited in Hutchby, 2014): depending on what participants want to accomplish (see Rintel, 2015), they can employ, for example, virtual backgrounds, mute audio or disable the camera. That is, participants can shape the content of their individual

100

M. HOLMSTRÖM ET AL.

frames and, for instance, block sensorial access or enhance it. Virtual backgrounds are an additional means of creating and enhancing an online presence; for example, they can be used to hide personal spaces behind the screen, or to add an extra, perhaps humorous, layer to the meeting. Oittinen (2020, p. 23) discusses three, intertwined and multi-layered, interactional spaces that participants in video-mediated interaction orient to: the local space of individual parties or participants, the overall meeting space in which the participants come together, and possible adjoining spaces which the participants may evoke, for example, through the use of additional digital devices. This chapter illustrates how, in virtual breaktaking, the participants make relevant both the overall meeting space, as well as their personal local spaces. They also have access to additional adjoining spaces (such as mobile phones) that sometimes occupy their attention. Through the overall meeting space, participants have partial access to others’ local spaces, and they have the opportunity to topicalize their sensorial (here, visual and aural) experiences of others’ spaces and direct shared attention to those. Nevertheless, among participants, interaction still unfolds on a turn-by-turn basis, and participants configure their embodied and verbal actions in situ. Even though visual or auditory access to others might be limited, blocked or delayed, participants appear to pay attention to others’ use of such cues anyway. In short, virtual breaks provide a possibility to be social with one’s co-workers, and members of a community can maintain a sense of togetherness and social intimacy. What is more, in times of a global pandemic, virtual platforms offer a chance for ‘caring from afar’ (Ibnelkaïd, 2021). They allow individuals to create co-presence that is built on shared histories and ‘technobodily literacies’, defined as ‘technical, cognitive, sensory, socio-affective and bodily skills’ that are moulded over time and that enable participants to interact on the multiple levels that are possible in the socio-digital world (Ibnelkaïd, 2022, p. 33). As the examples analyzed in the following sections illustrate, participants draw both on their previous knowledge of each other and on their current sensorial experiences to display closeness and strengthen existing social ties in video-mediated interaction.

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

101

Responding to Visual and Auditory Cues in Participants’ Local Spaces The following analyses support the view that when participants comment on a co-participant’s person, extensions of self or dependent others, those comments can be interpreted not as preambles to lengthier conversations but rather as displays of social curiosity that act as social glue, and as tools to involve people in interaction and co-construct the lived break-taking experience. Putting these sensorial experiences into words could also be a way of bringing the group together. The analyses also reveal embodied interactional practices through which participants respond to instances of social curiosity. More specifically, participants may orient to the concrete aspect that is brought to focus in an embodied way, for example, by looking elsewhere away from the screen. The participants may also display openness to and invite others’ social curiosity by orienting to something in their own immediate environment. In our data, participants’ orientation to ‘signs of life’ shows us that people do reach beyond the talking heads and the screens, as they orient to activities in others’ backgrounds. As people orient to others in virtual environments, they also open the floor for other participants to join in, even as the virtual platform may support only one conversation at a time. In example 1, a dog has previously been seen in the background of one participant, and when a previous topic comes to a close and participants’ local spaces become topicalized, another participant inquires about the whereabouts of the dog. There are ten participants in the conversation, and they may have their microphones muted when they are not talking.

102

M. HOLMSTRÖM ET AL.

Example 1: A Dog’s Business 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

(4.0) ((children's voices in the background; Continue intermittently throughout)) RII: (mm other people on the line) (0.9) RII: that- (.) eh he he he ULL: mhe mhe (0.8) RII?: mhe (2.4) LIN: it’s nice to hear:, (.) signs of life. (0.6) RII: (ye[ah)] ULL: [mm.] mhm, (0.6) RII?: .hh (0.9) ???: hmh, ULL: the dog uh (.) disappeared. RII: .nff yeah? (0.4) LIN: yeah, h[e went somewhere.] ULL: [joo- Joona disapp]eared. he he he LIN: yeah. RII: mmh, LIN: he went away on, (.) on *some business.# *gazes to r.-->> fig #1 ((audible laughter by some participants; smiles by others))

At the beginning of the excerpt, just after a lapse in the conversation, there is a background noise that could be coming from Riitta’s local space. Indeed, Riitta seems to comment on the noise jokingly, topicalizing the presence of others in her house (line 2). Linnea responds by stating that it is nice to hear signs of life, showing sympathy over the situation (line 9). The shared sensorial experience highlights the intermedial nature of the background noise and complex spatial dimensions of the virtual break: the noise originates in one local, physical space but is sensed and also topicalized in another.

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

103

In what follows, Ulla comments on the disappearance of the dog, Joona, that has been seen in Linnea’s background a few times during the break. Following the observation of the dog’s disappearance, Linnea comments that he went away on some business (line 24), a humorous answer to which several people respond by laughing, perhaps because dogs are not usually associated with ‘going away on business’. After her answer, Linnea also shifts her body and gaze away from the screen, as if looking for the dog, and thus potentially attempting to display a connection and engage with the referent, which in this case is a dog, a sentient being that moves around in Linnea’s immediate surroundings. The participant whose local space is made relevant in the overall meeting space is seen to be ‘looking elsewhere’, away from the screen, and shifting focus into the local environment in response to a display of social curiosity by another participant. Later (data not shown), Linnea mentions ‘the other one’, referring to another dog in the house, and recounts his whereabouts and sleeping preferences to the other participants, while looking away from her screen to her left. Linnea thus, in a manner of speaking, volunteers further personal information by extending her orientation to her immediate local space. In sum, the participants in example 1 are able to experience signs of life and others’ personal environments through both audible and visual cues, and even though Linnea mentions hearing signs of life, in this example, other participants can also see them. At least some people on this break are aware, based on previous interactions, that Linnea indeed has two dogs. Example 2 further illustrates how participants may orient to auditory cues from one local space and others respond to it. Just before the excerpt, Ulla has been telling a story, towards the end of which Anisha has entered the virtual break space. Some of the other participants say hello and wave to Anisha, while Ulla is still telling her story. After the telling comes to an end, Ulla orients to Anisha.

104

M. HOLMSTRÖM ET AL.

Example 2: There’s a Bit of an Echo There 01

ULL:

02

ANI:

03 04 05 06 07 08

RII:

09 10 11 12 13 14 15 16 17 18

ANI: RII: ???: ANI: RII: RII: ANI: ANI: RII: ANI: fig

19 20 21 22 23

hi *Anisha he he* he he *finger-wiggle wave* §hei§ he he he §lifts hand, small wave§ how are you::. (.) goo:d and you::. (yeah) >good good¶ RII: [yeah, yeah.]

Ulla delivers a greeting, hi, and identifies Anisha by her name accompanied by a finger-wiggle wave (line 1). Anisha responds with a small wave and a greeting, hei (line 2), and as she speaks for the first time, an echo in her voice is clearly detectable. After the initial exchange of pleasantries (lines 1–8), Riitta orients to the echo and directs the group’s

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

105

attention to her own sensorial experience of it, for which she seems to seek validation by asking whether Anisha is in a new apartment (lines 9– 10). Anisha indeed provides an affirmation, yes yes I’m at my new place now in ((suburb)) (line 13), which confirms, on one hand, Riitta’s sensorial experience and, on the other hand, the assumption that Anisha is in a new apartment. Riitta’s display of social curiosity in a form of an inquiry not only taps into an everyday experience that there often is an echo in a relatively empty room but also to her prior knowledge of Anisha’s current living arrangements. Riitta is seen actively taking steps to acquire social information and knowledge about Anisha to promote interpersonal relations within this group of colleagues. In this example, it is Anisha whose local space is made relevant in the overall meeting space. In the multimodal unfolding of interaction, Riitta’s inquiry not only prompts a verbal confirmation of the sensorial experience from Anisha: it sounds a bit empty (line 18) but also an embodied response in that Anisha simultaneously looks away from her screen, possibly orienting to her surroundings and connecting with the referent, namely the empty apartment. In much the same way as Linnea in example 1, Anisha is here seen to be ‘looking elsewhere’, away from the screen, and shifting focus into the local space in response to another participant’s inquiry that functions as a display of social curiosity. The examples suggest that such shift in orientation by looking elsewhere may be a recurrent interactional practice in sequences of social curiosity in virtual encounters. Unlike in the two previous examples, in example 3, a participant responds to an auditory cue in her own local space by looking away and smiling, and this response tips one of the co-participants off to draw attention to the referent in question.

106

M. HOLMSTRÖM ET AL.

Example 3: Is It Mimmi Coming Over 01

ULL:

02 03

tar mim NOO:

04 05

ULL:

06

NOO:

07 noo fig

08

ULL:

09

TAR: mim

10 11

TAR:

12

ULL:

13

NOO:

14

ULL: noo

näyttäs että se olis täsä. seems like it’s here (.)&*(0.8) &briefly nods *nails clicking--> joo. yeah (0.9) kat[sotta]vissa. viewable [noni.] alright (1.0)%#(1.0)% %looks to r. and smiles% #3

pittääpä kat[t*oa ], I’ll have a look [£hm hm] -->* (.) Mimmikö sieltä tuli,£ is it Mimmi coming over saa [jotain ] muuta muuta tämän vastapainoksi I could get something to counterbalance this [£jooh.£] yeah %minun tämän lenkki-% lenkillä kuuntelemani, one that I listen to on my walks %looks to r. and smiles%

In this example, five participants are on a break, and some of them are actively discussing a TV show and whether it can be watched for free. Noora and Ulla are the primary participants talking about the matter; however, Taru is also listening in and nodding to Ulla’s remark (line 2).

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

107

The topic seems to be coming to an end or moving to a new direction. During this sequence, something can be heard clicking on a participant’s background, and shortly after, Noora looks to her right and smiles. Unlike in examples 1 and 2, the participant whose local space is brought to focus is first seen ‘looking elsewhere’, away from the screen, and shifting attention into her local space in response to an auditory cue. A display of social curiosity by another participant follows immediately after on line 11 when, with the current conversation still ongoing, Taru briefly directs attention to the auditory cue and asks in a smiley voice whether it is Mimmi that is coming. By doing this, Taru also makes it known that the noise originating from one local space simultaneously occupies the overall meeting space. It is noteworthy that Taru uses a proper noun to refer to the assumed source of the noise, Mimmi being the name of Noora’s dog (as we have learnt from other recordings). Noora smiles, confirms that this is the case, partly in overlap with Ulla’s turn, and Ulla continues to talk about what to watch and listen to. In contrast with examples 1 and 2, here the participant whose local space is made the focus of attention displays being open to social curiosity through orienting to her immediate environment—possibly a specific referent such as the dog in this case—by looking away from the screen and smiling (line 7). And while there is only an auditory cue in the overall meeting space, Taru displays knowledge of Noora’s personal life by seeking confirmation to having identified the clicking noise to originate from Noora’s dog, Mimmi. Like in example 2, participants here orient to complex spatial dimensions when a noise is heard in the overall meeting space: potentially due to Noora’s initial orientation to her local space, Taru is able to locate the noise. Further, Taru is then able to topicalize the issue in the overall meeting space, by verbally proffering a candidate identification of the source of the noise. The short interlude to an ongoing talk topic in example 3 is a response to an auditory cue, but rather than introducing a lengthier discussion on the whereabouts of Mimmi, it merely directs attention to Noora’s local space. It displays social curiosity and a sense of familiarity between Taru and Noora. Noora’s embodied orientation to Mimmi walking past could be considered an opening for others to comment or make note of the audible cue. It seems that only Taru orients to Mimmi; of the five participants, Matthew and Sonja appear to be focused elsewhere and Ulla is in the middle of a turn. Taru’s turn (line 11) is formulated as an interrogative, implying Mimmi is not visible, yet Taru possesses the kind of

108

M. HOLMSTRÖM ET AL.

personal knowledge that indicates Mimmi is the most likely option. This fleeting moment in the middle of another conversation shows that participants’ local spaces can, in a way, act as extensions to the overall meeting space: depending on their pre-existing knowledge about co-participants, they can make assumptions or further inquiries into what is currently available to them.

Responding to Cues Visible on Screen The next two examples will further explore how participants in one overall meeting space orient to individual local spaces, responding to cues visible on screen and in this way initiating commentary that nurtures social curiosity. Example 4 displays a situation where five participants (Riitta, Noora, Ulla, Matthew and Martti) are taking a virtual break together, when Ulla directs attention to Martti’s appearance. Prior to distance working, these participants had seen each other on a regular basis at the office, and some of them have a long history of working in the same unit. Martti has joined the others a moment earlier and has just turned on the microphone.

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

109

Example 4: Do I Have a Moustache 01

MAR:

02

RII:

03 04 05 06

ULL: NOO: RII: ULL: mar fig

07 08 09 10

MAR: mar ull mat MAR:

12

rii RII: mat NOO:

13 14 15 16

noo rii ull ULL: RII: ULL: NOO:

11

itekseni puhelen täälä ko mikki on pois päältä. I'm talking to myself here ’cause the mic is off aij(h)aah. [hah hah] [nii justiin.] I see that's right [heh heh] [heh heh heh heh] heh heh he [.hhhhh] [onko sulla] ennen ollu v*iiksiä.# have you earlier had a moustache *scratches lip/side of nose--> #4

(2.1) onko mulla* viikset. Hh do I have a moustache -->* (0.8)+(0.5)&(0.2) +brings laptop closer--> &leans fwd--> tota, ei: mulla varm‡aan. erm no I probably don't/haven't ‡leans fwd--> mheh HEH HEH HEh heh& heh [hhh .hh] -->& [heh¤ £kaik]‡[ki vaan] [katt¤oo+(-),£] everyone's just looking (-) ¤leans fwd--------------¤ -->‡ -->+ [kHEh] [he::h heh] heh [heh] [heh] heh heh

After the participants have laughingly addressed Martti’s evident problems with unmuting his microphone, Ulla enquires about Martti’s facial hair: onko sulla ennen ollu viiksiä ‘have you earlier had a moustache’ (line 6). The turn is designed so that it implies, firstly, that Martti currently has a moustache and, secondly, that Ulla knows that Martti has not always had a moustache. Here Ulla’s display of social curiosity takes the form of an inquiry concerning Martti’s appearance. While Ulla’s turn is still ongoing, Martti brings his right hand up to his face and begins to scratch

110

M. HOLMSTRÖM ET AL.

his upper lip and the side of his nose. He continues the scratching through the 2.1-second gap and until he has initiated his response. In the previous examples, the participants oriented to referents that were brought to focus by looking elsewhere in their local spaces but here Martti orients and physically connects to the suggested referent, his own face, by touching it. Martti’s response is a partial repeat of Ulla’s question, with a deictic change from second person to first person, and it seeks confirmation of the implications of the question (onko mulla viikset ‘do I have a moustache’, line 8). In lieu of a verbal confirmation or disconfirmation, Ulla brings her laptop closer, so as to be able to scrutinize Martti’s appearance, and Matthew also leans forward towards the camera and assumably the computer screen. That is, the participants now treat Martti’s suggested moustache as not entirely apparent but in need of further inspection. When Martti finally provides an answer, it is hesitant and ambiguous: tota ei mulla varmaan ‘erm no I probably don’t/haven’t’ (line 10) can be heard to negate both the suggestion that he may earlier have had a moustache and the implication that he currently has one. Apart from scratching his upper lip and the side of his nose, Martti holds still while the others in an increasingly exaggerated manner lean forward to get a better view. After Ulla and Matthew, Riitta also leans forward closer to the camera and screen and, finally, Noora does the same, while commenting on how ‘everyone’s just looking’ (line 12). Indeed, laughter is evoked not only by the general unfolding of the sequence (i.e., Ulla asking about Martti’s facial hair in the first place and Martti’s hesitant responses), but also by the participants’ visible embodied conduct, which towards the end is produced mainly to a comical effect. Social curiosity here takes the form of close scrutiny of a co-participant’s person, which is made possible and salient by the video-mediated nature of the break and the ‘distant proximity’ that it entails: participants may inspect others from close range, and have themselves inspected by others, in the comfort of the overall meeting space while remaining in the safety of their respective local, personal and private spaces. Example 4 also highlights the challenges of the intermedial dimensions of the virtual break: while the platform provides, depending on the settings of individual users, a view of all co-participants, the video images are not life-sized and issues of audio and video quality as well as camera angle and lighting, for instance, may occur. Due to changes in light and shade, or indeed poor video, the participants may think that

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

111

they are seeing a moustache on Martti. At the same time, however, the virtual platform provides participants, and here especially Martti, continuous access to their own video image, a mirror of a kind, which they typically would not have in in-person interaction (see e.g., McIlvenny, 2002). Moreover, posing a question that reflects one’s current visual access to a co-participant’s person and displays social curiosity towards their physical appearance here leads to a sequence of interaction in which familiarity and social intimacy are maintained. That is, in bringing the growth of a workmate’s facial hair to joint focus of attention and discussion, the participants also orient to the overall phenomenon of being apart for a long time and people having taken a more relaxed stance to their appearance. In the following example, eleven participants are discussing the various virtual background images that people have selected for themselves, and Ulla directs her attention to the image displayed behind Noora and then Matthew. At the beginning of the excerpt, five participants are displaying virtual images that represent, for example, a deck of a sailing boat, a nightly sky with Northern lights and a wrecked room.

112

M. HOLMSTRÖM ET AL.

Example 5. Working from Detroit 01

ULL:

02 03

NOO: ULL:

04 05

NOO: ull SAI:

06 07

NOO: SAI:

08 09 10 11 12 13

OLI: SAI: OLI: ULL: NOO: mat fig

*but where is Noora (.) working from. *leans fwd--> ehm the Tar[dis ]. [public] is it a public toilet or a pub.§ -->§ it’s a: it’*s a:: a police box (0.3) it’s the it’s the Tardis. -->* +mhm +nodding head--> erh[m: ] (.) from Dr Who? [nice].+ -->+ ye[ah so] you’ve just come back from saving the world again [oh ]. haven't you. yeah [right]. [yeah ] yeah (.) you know the usual, (.)#(1.0) #background image of destroyed building appears #5

((1 min 45 sec omitted)) 14 15 16 17 18 19 20 21

ULL: is m- is Matthew also working from Detroit(h)? LOT: ◦he he [he he he◦] MAT: [.h ye:ah ] I’m in a- [I’m in an- in an] abandoned ehm ULL: [eh heh heh heh ] MAT: [mental ] facility. LOT: [.h ◦nice◦]. ULL: eh heh heh heh ((joint audible laughter by most participants))

The interaction flows multimodally in that Ulla begins to lean closer to the camera and appears to be carefully inspecting something. Such embodied craning motion is potentially visible to all participants and may be interpreted as foreshadowing a comment on some detail on her screen. Ulla indeed goes on to ask Noora but where is Noora working from (line 1), making a reference to her virtual background that illustrates the insides of the time machine Tardis from Dr Who. During the discussion, Matthew starts showing a background image of a destroyed building, and

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

113

the graffiti on its wall reads God has left Detroit. By doing so, Matthew signs up for the ongoing merriment of showing virtual images but treating them jokingly as actual working environments. This can be treated as a display of being open, perhaps, to a specific form of social curiosity, the playful commentary concerning one’s virtual background image. No one immediately reacts to the change but less than two minutes on in the conversation, Ulla directs attention to Matthew’s virtual background, is Matthew also working from Detroit (line 14). Matthew responds to Ulla’s inquiry by confirming that he is in an abandoned mental facility (lines 16 and 18), which is followed by shared laughter and further joking. Unlike in all previous examples, Matthew does not physically orient to anything off-screen, or in his local space, likely because his background image is virtual. He does, however, verbally orient to his new virtual environment and plays into the narrative of it being ‘real’. Virtual backgrounds are one semiotic resource that the participants can make use of in modern video-mediated interactions, and as example 5 shows, they may be used as a playful feature that not only enables but also in effect invites social curiosity. What is more, the fictitious Tardis and the abandoned mental facility in Detroit are jokingly oriented to and treated as participants’ local workspaces, which is telling of the complex multimodal and spatial dimension and intermedial nature of video-mediated interactions. People can participate from almost anywhere as long as they have the appropriate technology at their disposal.

Conclusion This chapter has explored the complex multimodal and spatial dimensions of virtual breaks and the characteristics related to sensorial experiences and intermediality as these appear and are made salient in interaction. We have shed light on how participants zoom in and bring depth to the two-dimensional video-mediated break-taking and how participants create co-presence and inclusion on a virtual platform during a global pandemic. We have examined instances in which participants’ sensorial experiences prompt them to comment on others’ looks, living arrangements, happenings in their homes as well as their virtual background images. Through such comments, the participants are seen tapping into life beyond the screen to bring depth into their encounters, as they move beyond the two-dimensional ‘face wall’ (Hochuli, 2021), and orient to

114

M. HOLMSTRÖM ET AL.

those spatial extensions that others’ local environments grant access to. We have analyzed these sequences as displays of social curiosity through which people actively take steps to acquire social information and knowledge of others to promote interpersonal relations and manage their social connections among members of the work community. In our data, the participants may know each other from before, and so it could be argued that the displays of social curiosity play a part in strengthening existing social ties. That is, such sequences are seen as situated instances of creating and maintaining social intimacy and co-presence, a sense of ‘being there’ for and with others on a virtual platform. Furthermore, during these sequences, participants show interest in others, display their knowledge of others’ personal lives, and in the case of virtual background images, jokingly treat them as real. The notion of North permeates the study as the data were collected in Finland and as taking breaks and having coffee together at the workplace is a common practice not only in Finland but also throughout the Nordic region. During a time of a pandemic, work communities are seen to use the medium of their choice to meet the need of being social and to continue taking breaks and drinking coffee together, which for many has meant the maintenance of a long-standing tradition. Although this tradition may have lost some of its material dimensions in the process of being transferred online, the affordances provided by the virtual medium offer new kinds of resources for enjoying breaks together and building shared lived experiences in different ways. This can be seen, among other things, in how the participants orient to the overall meeting space, and both their own and others’ local spaces. Access to workmates’ homes is seen to provide a novel dimension to breaks, as people are able to witness ‘in the flesh’ those from home who have been talked about during breaks. Participants can choose to display parts of their personal lives to others, but sometimes for example family members or pets can make an appearance on their own, too. Sometimes they are not even visible, but other participants can hear them. By showing that participants are familiar with different aspects of each other’s lives, togetherness and social intimacy can be created on shared breaks. Some of the data collection for this study took place during the early days of the COVID-19 pandemic (in spring 2020), and some were collected later, in the spring of 2021. The uncertain situation with COVID-19 also manifested in the recorded interactions, as people discussed new working arrangements, what to do to pass the time in

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

115

physical distancing and speculated how long the situation might last. The notion of ‘signs of life’ came up in the data, a befitting description for the pandemic times as well as orientations to what goes on beyond the screen. First, when during the pandemic many people started working from home and it was highly recommended that people reduce the number of social contacts, depending on personal circumstances, the amount of interaction in people’s lives was possibly reduced. Furthermore, because dealing with the pandemic and the so-called new normal may at times have felt surreal, seeing and hearing ‘signs of life’ may have been reassuring, proof that life does indeed happen and go on. Due to the fluctuating nature of the pandemic, we cannot tell what the situation will be when the book ‘Multimodality and Intermediality in the North’ is published. Perhaps we have returned to a situation where people spend most of their time at the office again and take breaks together in a break room. Perhaps we have adopted a hybrid strategy where people do take in-person breaks at the office, and sometimes participate in virtual breaks. Time will tell. The data presented here, then, reflects the time when people were working remotely for the most part, and the unusual state of the world is reflected in the interaction as well. Acknowledgements We would like to extend our warmest gratitude to Jakub Mlynáˇr, Samira Ibnelkaïd, Caroline Debray and Stephanie Schnurr, who kindly acted as reviewers and helped us to clarify the focus and main argument of the chapter. We would also like to thank those who participated in the Langnet Multimodality group’s text seminar, as well as the panel ‘Practices of inclusion in workplace interaction’ at the IPrA conference in summer 2021 and provided helpful comments in early stages of the writing process. Furthermore, a special thanks to Kenan Hochuli for insightful comments about the data, as well as helping in specifying descriptive terminology. Distribution of Work All of the three authors actively participated in preparing this chapter. However, the corresponding author (Holmström), was primarily responsible for writing the chapter, as well as for initially inspecting the data and identifying the phenomenon in question.

116

M. HOLMSTRÖM ET AL.

Appendix: Transcription Conventions The transcription conventions in this chapter have been adopted from Jefferson (2004) regarding talk, and Mondada (2019b) for the multimodal representation of interaction. The following is not an exhaustive list of all possible symbols but presents those utilized in this chapter. Talk is presented in regular black font, possible English translations in italics and descriptions of embodied conduct in regular grey font. [word] (0.5) (.) . , ? wo::rd word °word° >word< wo(word) £word£ hhh .hhh (( )) ** ++ && *–> −>* −>> noo fig #

Overlapping talk Numbers in parentheses indicate silence, represented in tenths of a second ‘Micropause’, ordinarily less than 0.2 of a second Falling intonation Level intonation Rising intonation A colon indicates prolongation or stretching of the preceding sound. The number of colons indicates the length of the prolongation Underlining indicates emphasis The degree signs indicate that talk between them is markedly quiet or soft Increased speaking rate A hyphen after a word or part of a word indicates a cut-off Uncertain hearing, in case of empty parentheses there is no likely candidate Smiley voice Outbreath Inbreath Transcriber’s descriptions of events, rather than representations of them Descriptions of embodied actions are delimited between two identical symbols (one symbol per participant and per type of action) that are synchronized with correspondent stretches of talk or time indications The action described continues across subsequent lines until the same symbol is reached The action described continues after the excerpt’s end Participant doing the embodied action is identified in small caps in the participant column, unless the same as current speaker The exact moment at which a screen shot has been taken is indicated with a sign (#) showing its position within the turn/a time measure

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

117

References Arminen, I., Licoppe, C., & Spagnolli, A. (2016). Respecifying mediated interaction. Research on Language and Social Interaction, 49(4), 290–309. https:// doi.org/10.1080/08351813.2016.1234614 Barmeyer, C., Mayrhofer, U., & Würfl, K. (2019). Informal information flows in organizations: The role of the Italian coffee break. International Business Review, 28(4), 796–801. https://doiorg.pc124152.oulu.fi:9443/10.1016/j. ibusrev.2019.04.001 Clausnitzer, J. (2021, July 5). Consumption of roasted coffee per capita in Finland 2020–2020. Statista. https://www.statista.com/statistics/708603/coffee-con sumption-per-capita-in-finland/ Coffee Culture in Finland: A Story of Togetherness. (2021, October 25). All things Nordic. https://allthingsnordic.eu/coffee-culture-in-finland-a-story-oftogetherness/ Due, B. L., & Licoppe, C. (2021). Video-Mediated Interaction (VMI): Introduction to a special issue on the multimodal accomplishment of VMI institutional activities. Social Interaction. Video-Based Studies of Human Sociality, 3(3). https://doi.org/10.7146/si.v3i3.123836 Elleström, L. (2021). The modalities of media II: An expanded model for understanding intermedial relations. In L. Elleström (Ed.), Beyond media borders, Volume 1. Palgrave Macmillan. https://doi.org/10.1007/978-3030-49679-1_1 Fornel, M. de. (1996). The interactional frame of videophonic exchange. Réseaux. Communication, Technologie, Société, 4(1), 47–72. https://doi.org/ 10.3406/reso.1996.3305 Gibson, J. (1982). Reasons for realism: Selected essays. Lawrence Erlbaum Associates. Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10), 1489–1522. https://doi.org/10.1016/ S0378-2166(99)00096-X Hartung, F. (2010). Social curiosity and its functions [Doctoral Dissertation]. University of Konstanz. Hochuli, K. (2021, June 29). Visuality and co-presence in “face wall”videoconferences: How participants organize attention and reference when taking turns in digital university lectures. Presentation at IPrA 2021 conference. Hunter, E. M., & Wu, C. (2016). Give me a better break: Choosing workday break activities to maximize resource recovery. Journal of Applied Psychology, 101(2), 302–311. Hutchby, I. (2001). Conversation and technology: From the telephone to the Internet. Polity; Blackwell Publishing.

118

M. HOLMSTRÖM ET AL.

Hutchby, I. (2014). Communicative affordances and participation frameworks in mediated interaction. Journal of Pragmatics, 72, 86–89. Ibnelkaïd, S. (2021). Caring from afar: Video interaction as a safe heterotopia? 71st ICA (International Communication Association) Conference. Ibnelkaïd, S. (2022). Technobodily literacy in video interaction. In A. Duranti, R. George, & R. Conley Riner (Eds.), New Wiley Blackwell companion to linguistic anthropology. Wiley. Jefferson, G. (2004). Glossary of transcript symbols with an introduction. In G. H. Lerner (Ed.), Conversation analysis: Studies from the first generation (pp. 13–34). John Benjamins Publishing Company. Kesselheim, W., Branderberger, C., & Hottiger, C. (2021). How to notice a tsunami in a water tank: Joint discoveries in a science center. Gesprächsforschung, 22, 87–113. Kim, S., Park, Y., & Niu, Q. (2017). Micro-break activities at work to recover from daily work demands. Journal of Organizational Behavior, 38(1), 28–44. Korczynski, M. (2003). Communities of coping: Collective emotional labour in service work. Organization, 10(1), 55–79. https://doi.org/10.1177/135 0508403010001479 Liberati, E. G., Tarrant, C., Willars, J., Draycott, T., Winter, C., Chew, S., & Dixon-Woods, M. (2019). How to be a very safe maternity unit: An ethnographic study. Social Science & Medicine, 223, 64–72. Licoppe, C. (2015). Video communication and ‘camera actions’: The production of wide video shots in courtrooms with remote defendants. Journal of Pragmatics, 76, 117–134. https://doi.org/10.1016/j.pragma.2014.11.008 Licoppe, C., & Morel, J. (2012). Video-in-interaction: “Talking heads” and the multimodal organization of mobile and Skype video calls. Research on Language & Social Interaction, 45(4). https://doi.org/10.1080/08351813. 2012.724996 Licoppe, C., & Veyrier, C.-A. (2017). How to show the interpreter on screen? The normative organization of visual ecologies in multilingual courtrooms with video links. Journal of Pragmatics, 107 , 147–164. https://doi.org/10. 1016/j.pragma.2016.09.012 Luff, P., Heath, C., Kuzuoka, H., Hindmarsh, J., Yamazaki, K., & Oyama, S. (2003). Fractured ecologies: Creating environments for collaboration. Human-Computer Interaction, 18(1–2), 51–84. https://doi.org/10.1207/ S15327051HCI1812_3 McIlvenny, P. (2002). Here’s me looking at me looking at me talking: Communicating in graphical cyberspace. In J. Haines, E. Kärkkäinen, & T. Lauttamus (Eds.), Studia linguistica et litteraria septentrionalia: Studies presented to Heikki Nyyssönen (pp. 127–144). Department of English, University of Oulu.

5

ZOOMING IN ON A FRAME: COLLECTIVELY FOCUSING …

119

Mlynáˇr, J., González-Martínez, E., & Lalanne, D. (2018). Situated organization of video-mediated interaction: A review of ethnomethodological and conversation analytic studies. Interacting with Computers, 30. https://doi.org/10. 1093/iwc/iwx019 Mondada, L. (2013). Embodied and spatial resources for turn-taking in institutional multi-party interactions: Participatory democracy debates. Journal of Pragmatics, 46(1), 39–68. Mondada, L. (2016). Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3), 336–366. Mondada, L. (2019a). Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics, 145, 47–62. Mondada, L. (2019b). Conventions for multimodal transcription, version 5.0.1. https://www.lorenzamondada.net/multimodal-transcription Mondada, L. (2021). Orchestrating multisensoriality in tasting sessions: Sensing bodies, normativity, and language. Symbolic Interaction, 44(1), 63–86. Oittinen, T. (2020). Coordinating actions in and across interactional spaces in technology-mediated business meetings. JYU Dissertations 225. University of Jyväskylä [Doctoral Dissertation]. http://urn.fi/URN:ISBN:978-951-398176-1 Pillet-Shore, D. (2017). Preference organization. In J. Nussbaum (Ed.), Oxford research encyclopedia of communication. Oxford University Press. https://doi. org/10.1093/acrefore/9780190228613.013.132 Pillet-Shore, D. (2018). How to begin. Research on Language and Social Interaction, 51(3), 213–231. Pillet-Shore, D. (2020). When to make the sensory social: Registering in faceto-face openings. Symbolic Interaction, 44(1), 10–39. Relph, E. (1976). Place and placelessness. Pion. Renner, B. (2006). Curiosity about people: The development of a social curiosity measure in adults. Journal of Personality Assessment, 87 , 305–316. https:// doi.org/10.1207/s15327752jpa8703_11 Rintel, S. (2015). Omnirelevance in technologised interaction: Couples coping with video calling distortions. In R. Fitzgerald & W. Housley (Eds.), Membership categorization analysis: Studies of social knowledge in action (pp. 123–150). Sage. Rippl, G. (2015). Handbook of intermediality: Literature—Image—Sound— Music. De Gruyter. Seamon, D., & Sowers, J. (2008). Place and Placelessness (1976): Edward Relph. http://dx.doi.org/10.4135/9781446213742.n6 Siitonen, P., & Siromaa, M. (2021). Accounting for leaving the break room: Work obligations as a resource in transitions from one activity to another

120

M. HOLMSTRÖM ET AL.

at the workplace. Journal of Pragmatics, 171, 178–199. https://doi.org/10. 1016/j.pragma.2020.09.032 Stroebaek, P. S. (2013). Let’s have a cup of coffee! Coffee and coping communities at work. Symbolic Interaction, 36(4), 381–397. https://doi-org.pc1 24152.oulu.fi:9443/10.1002/symb.76 Working Time Act (Työaikalaki) 5.7.2019/872.

PART II

Mediating Arts and Culture

CHAPTER 6

Voicing a Northern Minority Culture on a Global and Digital Arena: Sami Music Videos on YouTube Annbritt Palo, Lena Manderstedt, and Outi Toropainen

Introduction This study focuses on the voicing of Sami culture in six music videos published on YouTube by Sami musicians. Sápmi is a geographically defined area, and the historical and cultural home of indigenous people in Northern Europe, mainly spread over three nation states—Norway,

A. Palo (B) · L. Manderstedt · O. Toropainen Department of Health, Education and Technology, Luleå University of Technology, Luleå, Sweden e-mail: [email protected] L. Manderstedt e-mail: [email protected] O. Toropainen e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J.-P. Alarauhio et al. (eds.), Shaping the North Through Multimodal and Intermedial Interaction, Arctic Encounters, https://doi.org/10.1007/978-3-030-99104-3_6

123

124

A. PALO ET AL.

Sweden, and Finland (with some in Russia). To date, few studies have focused on multimodal analyses of music videos (cf. Helland, 2018), even though multimodality is crucial to music videos which use several semiotic modes in meaning-making, communication, and representation. The multimodal analyses consider verbal, nonverbal, and contextual communication. Our interest is not only the published videos but also the comments and interaction occurring in connection to these videos. The multilingual discussions in connection to the videos create complex chains of communication. These in turn promote communication and enable communities to establish, interact, and flourish regardless of the geographical location. For ethnic minorities, social media offer possibilities to strengthen the offline communities and to form closer connections between community members. Simultaneously, if needed and wanted, participatory online digital media, such as YouTube, provide opportunities for supporting the feeling of belonging to a global community by bringing to the fore issues central to them (cf. Cocq & DuBois, 2020). Online communities proffer opportunities for socializing and sharing experiences and values. Digitalization is particularly important in the Arctic as it allows people in this sparsely populated region to form global and local communities. Lindgren and Cocq (2017) state that the use of social media enables communication irrespective of geography and “makes visible common interests and goals on a global scale” (p. 131) but point out that indigenous communication online includes both inreach and outreach communication practices (see also Landzelius, 2006). The inreach practices aim at the establishment and negotiation of meaning and community building, whereas the outreach practices concern “connecting with people from outside the group, to increase visibility or to correct stereotypes and other false representations” (p. 135). The choice of language can be an inreach communication practice. Lindgren and Cocq (2017) discuss “the potential and effects of social media for emic discourses—that is, from within the group to reach a broader audience” (p. 132). YouTube is a global online video-sharing platform for content which “offers participants a way to garner wide exposure” (Burgess & Green, 2009, p. 5). It is also an archive for contemporary culture, and a social network, where participation is central. Participation online can include affiliations, expressions, collaborative problem-solving, and circulations (Jenkins et al., 2015).

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

125

By posting or commenting videos, participants can exchange selfrepresentations of cultural identities and promote their cultural heritage while simultaneously establishing their “cosmopolitan cultural citizenship” (Burgess & Green, 2009, p. 81). YouTube offers its users two modes of participation: as observers, and as participants through uploading content or writing comments. People can take part in the cultural heritage of the person uploading material, and the act of uploading material, in a language not spoken by the majority of users, offers an intracultural arena. It addresses both the audience having the linguistic competence, and people who will have to make meaning through other modalities than the spoken word. Consequently, participation can be seen as an expression of voice. Participation is a way to reach out, connect and voice one’s experiences of cultural pride but also of oppression and injustice. Thus, YouTube is a potential site of political power, an arena for stating an opinion and conveying a message.

Aim The aim of this article is to contribute to the knowledge of the multimodal voicing of minority culture on a global and digital arena, in a social media communication, namely YouTube. We interpret the voicing in six music videos by Sami musicians, the promotion of Sami culture, and the chains of communication shaped by the multilingual communication in the comments.

Ethnicity and Social Media Communication Ethnicity is to be understood as an aspect of social organization (Barth, 1969; Gil-White, 2001) and social relationships. In the Nordic countries, ethnicity is a question of self-affiliation, only allowing data containing self-affiliation when individuals request services in a minority language or voting rights for the Sami parliament. If a common language variety coincides with an affiliation with a social group, it is possible to refer to an ethnolinguistic identity (Liebkind & Henning-Lindblom, 2015). To be identified, or self-identify as Sami, it is not necessary to speak a Sami language. Individuals, or social groups, may however consider cultural differences between themselves and others socially relevant and communicate the difference through ethnic markers: language, clothes, and lifestyle. By performing a particular lifestyle, an individual can strengthen

126

A. PALO ET AL.

his or her subjective feeling of being part of a cultural community. Traditional Sami music, such as yoik and leu’dd, can be understood as an important ethnic marker to Sami people alongside with traditional clothes, the language, and the reindeer since these ethnic markers are not shared with the majority society (Stoor, 2007). However, Ridanpää and Pasanen (2009) point out that “yoik music, large-scale reindeer herding and a colourful national costume” (p. 214) have come to be seen as characteristics of all Sami people, not just the North Samis, even though only a minority of the Samis earn a living through reindeer husbandry. For example, reindeer husbandry is not characteristics of the Inari Samis, and not all Samis understand or speak any of the existing nine Sami languages. Out of these Sami languages, North Sami has around 30,000 speakers. Other Sami languages—Inari, Skolt, Ume, Lule, and South Sami—have only a few hundred users, but all nine languages are classified as endangered (UNESCO, n.d.). North Sami usually serves as the lingua franca between people who have skills in Sami languages. Language can be considered part of an individual’s self-affiliation. Voice can also be perceived as an expression of the individual’s “inner self” (Lawy, 2017, p. 196), as well as the experience of a group, and unite the members of the group in question. Lawy (2017) points out that “a theory on voice is not only about speaking, participating or making yourself heard but also must consider the using a voice that relies upon dominant structures to legitimize it” (p. 192). YouTube is an important digital arena where minorities’ voices can be expressed and heard. However, artists posting videos on YouTube are not only representatives for their communities, but also individuals and artists.

Methodological Perspectives In this qualitative study, netnography is used to approach cultural phenomena in their context. Netnography allows access to a diverse and large amount of online data (Kozinets, 2020, p. 19; see also virtual ethnography, Hine, 2015), such as YouTube videos and posted comments. The Association of Internet Researchers recommends that informed consent should be obtained, and particular is care taken when using semi-public or private data (franzke et al., 2020). No such data were used for this study. For this article, only data requiring no membership, subscription, or password to access the music videos, or comments have been used. In

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

127

addition, many comments can only be connected to a moniker, not to a particular individual. The unit of analysis of the comments is the social acts and utterances, and the strategies used to evaluate, embrace, endorse, or explain matter (cf. Kozinets, 2020), not the identities of the commenters. The data consist of six music videos by Sami musicians in different genres, from Finland, Norway, and Sweden (see Table 6.1). All videos and comments were sampled on February 14, 2020, to ensure that the videos and the comments were available on YouTube on that date. The choice to restrict the data collection to the Nordic countries is based on the volume available, and the netnographic search did not yield Sami music videos from Russia fulfilling the criteria established for the search. The criteria were music videos by Sami musicians, published by the musicians or their official company, published no earlier than 2010 on YouTube, and with a minimum of five comments. Furthermore, the language used in the music videos was not a selection criterion. Instead, it was understood as an ethnic marker with consequences for the use of other ethnic markers (cf. Ridanpää & Pasanen, 2009). The selection criteria led to the production of a primary corpus of more than 30 videos, which were then sorted to fulfil the further criteria: equal representation from the three countries, equal representation between male and female artists, and films rather than still photography. The search words used in a search engine were “Sami music” and “Sami musicians”, mainly in English and Swedish. The list of names was then used for the netnographic search (see Method). The total number of comments was approximately 600, of which slightly more than 100 were analyzed in detail for this study. Initially, the comments were copied into a word file to allow for close reading (e.g. Brummett, 2019). In particular, comments focusing on Northern minority culture have been included. Thus, for example comments concerning pure music or the quality of the video were analyzed but not discussed in this article (cf. ritual appreciation, Georgakopoulou, 2017). One exception was made; the texts written in some of the Sami languages have been included regardless of the subject matter. Moreover, no comments consisting of sole likes ( ) or dislikes ( ) or other emojis were included in the discussed data. For comments on these music videos, the use of emojis is considered part of the ritual appreciation.

Year of posting on YouTube

2010

Mari Boine

Analyzed videos

Artist

Table 6.1

Kautokeino (North Sámi) https://www. youtube.com/ watch?v=4jr GmT5gZNE (Link no longer valid)

Norway

Title of the song Nation and language state of of the lyrics origin 4:05 Video unavailable but available when netnographic sweep was carried out

Length (min:sec) Additional language in music video

Number of views 131,708

Number of comments 41

128 A. PALO ET AL.

Year of posting on YouTube

2013

2016

Artist

Niillas Holmberg & Roope Mäenpää (feat. Ánne Mágga Wigelius)

Sofia Jannok (feat. Anders Sunna)

Gállok (North Sámi) https://www. youtube.com/ watch?v=pbh iXT0z0RA We are still here (English) https://www. youtube.com/ watch?v=EVH 0jvnaIqU Sweden

Finland (Norway)

Title of the song Nation and language state of of the lyrics origin

5:14

2:54

Swedish North Sámi

English Swedish

Length (min:sec) Additional language in music video

53

5

109,511

12,591

Number of views

(continued)

Number of comments

6 VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

129

Year of posting on YouTube

2016

2019

2019

Amoc (feat. Ailu Valle)

Duolva Duottar

Jon Henrik Fjällgren (feat. Elin Oskal)

(continued)

Artist

Table 6.1

Kiälláseh (Inari Sámi) https://www. youtube.com/ watch?v= 39wwX_pdXPQ Márkanstállu (North Sámi) https://www. youtube.com/ watch?v=abJ LpOlR3cw The Way You Make Me Feel (English) https://www. youtube.com/ watch?v=EF1 s5XSOFDQ Sweden (Norway)

Norway

Finland

Title of the song Nation and language state of of the lyrics origin

4:14

4:23

5:05

North Sámi Finnish

Length (min:sec) Additional language in music video

379

81

67

Number of comments

359,155

26,824

No data

Number of views

130 A. PALO ET AL.

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

131

Analytical Framework As an analytical tool, Elleström’s medium-centred model of communication (2018) is applied for the analysis of the communication, both in the videos themselves and in the written comments. The visual diagram (Fig. 6.1) contains the three entities of communication: 1. Something being transferred. In the analyzed data including both music videos and written comments relating to them, the observers perceive, in the videos, images of Saminess conveyed by the musicians, and/or music producers. This transfer is a musical, visual, and verbal voicing of the minority culture, of which the visual and verbal voicing were focused for this article. Furthermore, the written comments enable the observers to actively take part in the communication, for example, by a comment in the language of their preference. 2. The producer’s mind and the perceiver’s mind. What the observer will perceive or see and hear in the videos, and read in the comments, depends on the prior knowledge and ability to interpret, and connect the key elements of the multimodal communication in the actual data. 3. An intermediate stage enabling the transfer. The media product represents something and prompts a specific interpretation (Elleström, 2019). As a material product, the music video is a relatively speaking, stable artefact, whereas what is perceived depends on the observer. The written comments relating to the videos are understood as the observers’ reactions to the musical, verbal, and visual elements in the video (cf. ritual appreciation) or to some theme or issue on more abstract level that awakes the observer. The

Fig 6.1 Elleström’s model for communication (2018, p. 218)

132

A. PALO ET AL.

latter alternative might then be a new media product arisen from the explicit video. Drawing on Elleström (2018), we understand music videos as material products with presemiotic and semiotic qualities, which are necessary in communication at large (Elleström, 2019). The presemiotic modes are bound up in “media traits that are involved in signification” (Elleström, 2019, p. 48) and concern the mediation of the media product. The presemiotic qualities consist of (i) material traits, or resources, that is, the characteristics that make the observers perceive the actual multimodal product as a music video (and not a feature film, a newsreel, etc.), (ii) the spatiotemporal traits, such as the structure of the narrative, the cuts and juxtaposition of key elements, and (iii) the sensorial traits, for example artefacts, colours, setting, and sounds. In addition, the music videos also have semiotic qualities, which concern the representation of the media product. These qualities are connected to the physical objects and phenomena that are comprehended in the observer’s mind to signs hence all presemiotic resources are meaningless without the observer’s cognitive comprehension of them (Elleström, 2019). Therefore, the analysis of the music videos was carried out in two stages. In order to analyze how the presemiotic resources (or nonverbal and contextual communication) contribute to voicing Sami minority culture, the videos were analyzed frame by frame. The presemiotic analysis clarifies some key elements and visual artefacts used, and, finally, how the semiotic resources (verbal communication consisting of language, signs, etc.) relate to the presemiotic ways of voicing the culture. As Fig. 6.1 shows, Elleström’s model of communication depicts a chain containing a producer initiating the chain of communication, a media product, and an observer, or as Elleström calls it, a perceiver. The music videos can be seen as the primary product in such a chain, and in this study, this is also the primary chain. On YouTube, this chain is then followed by a secondary chain consisting of commenters who become producers of comments to be read—perceived—by others. These secondary chains of communication can then generate tertiary chains, and so on. Therefore, the analyses include the secondary chains of communication shaped by the posted comments. The mark […] used in excerpts is a sign of omitted comment or part of a comment that is not in focus in the actual context, whereas *** indicates separate communicative chains. The translations of the comments (in square brackets) are our own.

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

133

The producers, in Elleström’s terminology, are the six artists who, presemiotically and semiotically, voice their minority culture in and through these publicly shared music videos. As YouTube is a public forum, the observer of the videos can be anyone. What the observer perceives depends on his or her cultural and linguistic background. Some of the observers might express themselves by leaving a comment to the video or reacting to another comment. Comments can be understood as signs of interpretation of, and mental activities caused by the video in the observer’s mind. These activities are meaningful signs to the observer. As other observers or the artists themselves are reacting to them—or, as Elleström (2018) puts it, “the perceiver becomes a producer in creating another media product (of the same or another kind) that reaches another perceiver’s mind” (p. 291), these verbal comments create a new communicative chain.

Voicing a Minority Culture Through Multimodal Communication The choice of language in the videos can be seen as a tool for inreach communication practices, in particular if no translation is provided. Four of the six videos are sung in Sami languages, in a few cases with some elements in a national majority language or English: Kautokeino, Gállok, Kiälláseh, and Márkanstállu. Two videos are in English, one with some elements of the national majority language and English: The Way You Make Me Feel, and We Are Still Here (see Table 6.1). Comments can be in any language, but the most common language used in the interaction connected to the selected six music videos is English. Especially The Way You Make Me Feel has been repeatedly commented on in English as the artist (Jon Henrik Fjällgren) is, or seems to be, known widely, due to his participation in the national finals in the Eurovision Song Contest. This is also respected by a North Sami speaker: “Fiinna lavlun ja luohti, dakkariin Eurovision lavlungilvvuide. Giitu Jon ja Elin”. [Excellent song and melody for the Eurovision Song Contest. Thanks Jon and Elin.] (Toivo West). Not everybody is positive regarding the use of English. Fjällgren is criticized when Timberjack4995 requires for more music in Sami instead of English in North Sami: “Eambo musihkka davvisámegillii, eai eaŋgalasgiella!” [More music in North Sámi, not in English!]. The national languages Swedish and Finnish are also used in comments.

134

A. PALO ET AL.

warezguy4ever Elsa Laula ser jag på bildn, hon ha ju samma släktingar som mig från Vilhelmina [I see Elsa Laula in the picture, she has the same relatives as I in Vilhelmina] ∗ ∗ ∗ panzer krieg fin mä oon pienestä astin käynyt kesäsin ja talvisin utsjoella koska mummu ja pappa asuivat siellä ja asuvat vieläkin [...] ja tällä hetkellä mun sylissä on saamen koulu kirja [Since I was a little kid, I have visited Utsjoki during the summer and winter time because my grandma and grandpa live there [...] and at the moment I have my school book in Sámi language on my knees]

The Way You Make Me Feel features the South Sami singer Jon Henrik Fjällgren from Sweden and the North Sami singer Elin Oskal from Norway. The setting of the video is a snow-clad, mountainous landscape with birch trees and conifers, open plateaus, and deep river valleys. Apart from the two singers, no other human beings can be detected, nor any buildings. Visually and verbally, the music video speaks of nature barely touched by man. The male and female figures are dressed in what could be perceived as Sami clothing, in particular the female dressed in a Sami dress and a risku (a brooch). As the song is in English and there is no explicit reference to Sami people, the sensorial, presemiotic resources become vital in visually and, through the yoik, musically voicing the Sami culture. Without the yoik, the clothes, and the jewellery, it could be perceived as a mere love song and not particularly connected to Sami culture. We Are Still Here constitutes a reply by Sofia Jannok and Anders Sunna to the voiceover argument presented by the Swedish state’s legal representative in the court case between the state and the Sami community of Girjas, between 2009 and 2020 with the ruling in favour of the Sami community. Jannok’s first lines of the lyrics connect the treatment of the Sami people with Euro-American colonizers’ treatment of indigenous people by connecting the Native Americans and the Sami. The references to the attempts to exterminate the bison and the burning down of tipis—neither of which exist in Sápmi—make it clear that the video wants the listeners to make the connection between the oppression of indigenous people then and now, here, and there. Furthermore,

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

135

the choice of language underscores that it is an outreach communication. As English is not the language of the colonizer in Sweden, singing in English cannot be seen as a corroboration of the colonization, whereas singing in the national majority language might be perceived in that way. However, without the nonverbal and contextual communication, through the yoik, the artwork, the setting, and the voiceover, the verbal communication—Jannok using the Sami word lávvu (tent) and mentioning reindeer—might not be enough to convey Saminess. The comparison between the colonization of the indigenous people in different parts of the world can be seen as both inreach and outreach communication, by showing that the Sami communities share their experience with other indigenous groups. This is noted in comments such as these ones: “Beautiful Singer, I am from Canada and for me this is a universal song most of us indigenous around the world go through these issues” (Cynthia Jacobs), and “The battle the Sami people have is not unique, sadly governments around the world seem to be bent on destroying the indigenous people of their own countries” (Robert McKenzie). Here, the commenters position themselves as inside or outside an indigenous community, respectively. Verbally, the beginning of We Are Still Here sets the narrative in an international context. As a contrast, the visual references then projected are northern: snow, and reindeer. A man dressed like an urban guerrilla soldier in black with a balaclava, sprays words—YOU HAVE NOT BEEN IN THIS AREA, WE ARE STILL HERE and the derogatory and outdated term for Sami: LAPP. He also sprays images of Sami females like the activist Elsa Laula (1877–1931), and graffiti-like male characters: a wolf in Western clothes and a white reindeer in a gákti. The words and images of the graffiti as well as the audible lyrics challenge the political correctness of the Swedish society by underscoring the notion that the colonizer, the Swedish national majority, has stolen the mother of the Sami people, that is, the land. The lyrics point out that “thieves are not to blame/ (…) when laws are written by the same”. This is an intertextual reference, that is an inreach communication, to epic yoiks recorded in the nineteenth century in Fellman’s The Thief and the Shaman (Swe. Tjuven och schamanen, Sa. Suola ja noaidi). According to Gaski (1987) the epic yoiks from the nineteenth century include explicit political opinions about the colonization process in Sápmi. Thus, We Are Still Here can—and must—be considered part of a literary Sami tradition, depicting the colonizer as a thief intent on stealing the land of the

136

A. PALO ET AL.

indigenous Sami people. The video challenges the colonizing power by connecting with the experiences of other indigenous people worldwide and by declaring that the Sami people prevail. By combining presemiotic resources, such as the cuts and juxtaposition of elements, and the use of artefacts, colours, setting, and sounds, and by combining these with the semiotic resources of the English language, Sami, and the voiceover and the derogatory word in Swedish, Jannok and Sunna contradict the colonizers’ narrative by voicing and visualizing the Sami perspective on the Swedish colonization of Sápmi, and on the colonization of indigenous land and people worldwide. However, singing in English is not a prerequisite for outreach communication, as shown in the music video Gállok by Niillas Holmberg, Roope Mäenpää, and Ánne Mágga Wigelius. In 2006, a British mining company was granted rights to prospect for iron ore at Gállok in Swedish Sápmi. The video Gállok portrays the exploitation of Sápmi, focusing on the Sami protests. The Sami flag raised at the site of the prospect clarifies the perspective, as does the picture of a young man wearing a gákti. The young man is photographed next to a t-shirt proclaiming “Homeland Security. Fighting Terrorism since 1492”, and a picture of Geronimo, a prominent Native American leader who in the nineteenth century fought the American colonization. The outreach communication is established visually as questioning the ownership of the land and promoting the resistance to the colonization, while the inreach communication is established both visually, that is nonverbally, and verbally, as the song is in North Sami, and no translation is provided. The choice of language, artefacts such as the Sami flag, and Sami clothes (gákti) or jewellery (risku) are the elements distinguishing this resistance from other protests. As in the songs, the choice of language in the comments can be a way of positioning oneself, culturally, or in a chain of communication. Commenting in a Sami language, or in Finnish, Norwegian, and Swedish, can potentially limit the interaction with others. Comments in Swedish and Norwegian are possibly understood also in Finland, while the comments in Finnish or Sami languages are mostly incomprehensible for those who do not have basic language skills. Nowadays, many languages can be translated by using online services, but the use of languages in the comments indicates whether the chain of communication is intended to be outreach or inreach. By writing “SÁPMI POWER WE ARE STILL HERE” (Liv-Anita A N), the commenter positions herself as belonging

6

VOICING A NORTHERN MINORITY CULTURE ON A GLOBAL …

137

to the Sami community but choosing to express this in English makes her statement part of an outreach chain of communication. The use of English in the comments makes it possible to reflect on the observers’ backgrounds. As the next example shows, several observers reflect on their Sami backgrounds as kietsuhime and ButterFly1970 do in English. By contrast, Kate Brown does it in North Sami. By this choice of language, she restricts her communication to other Sami speakers, in this case explicitly to the artist. Thus, these comments show that outand inreach communication can take place in the very same chain of communication, as shown in these excerpts: kietsuhime I’m part Saami myself and my great grandparents fled to America to escape Saami persecution. ∗ ∗ ∗ ButterFly1970 I was born in Finland, my family is from Savonia and Rovaniemi, we are Sami people. ∗ ∗ ∗ Kate Brown Bures, Sofia. Mon lean Kaatjaa (…) Mu ákku eadni lei sámi. [Hi Sofia. My name is Kate (…) My great-grandmother was Sami.] ∗ ∗ ∗ Nilas Lejonhjärta Tack så mycket Jon för att du har inspirerat mig till att utforska mitt samiska ursprung