Task-Based Language Learning in a Real-World Digital Environment 9781474264075, 9781474264044, 9781474264068

Citation preview

Task-Based Language Learning in a Real-World Digital Environment

Advances in Digital Language Learning and Teaching Series Editors Michael Thomas, University of Central Lancashire, UK Mark Peterson, Kyoto University, Japan Mark Warschauer, University of California, Irvine, USA Today’s language educators need support to understand how their learners are changing and the ways technology can be used to aid their teaching and learning strategies. The movement towards different modes of language learning – from presence-based to autonomous as well as blended and fully online modes – requires different skill sets such as e-moderation and new ways of designing and developing language learning tasks in the digital age. Theoretical studies that include practical case studies and highquality empirical studies incorporating critical perspectives are necessary to move the field further. This new series is committed to providing such an outlet for high-quality work on digital language learning and teaching. Volumes in the series will focus on a number of areas including but not limited to:            

task-based learning and teaching approaches utilizing technology language learner creativity e-moderation and teaching languages online blended language learning designing courses for online and distance language learning mobile-assisted language learning autonomous language learning, both in and outside of formal educational contexts the use of web 2.0/social media technologies immersive and virtual language learning environments digital game–based language learning language educator professional development with digital technologies teaching language skills with technologies

Enquiries about the series can be made by contacting the series editors:  Michael Thomas ([email protected]), Mark Peterson ([email protected]) and Mark Warschauer ([email protected]). Titles in the Series Autonomy and Foreign Language Learning in a Virtual Learning Environment, Miranda Hamilton Online Teaching and Learning: Sociocultural Perspectives, edited by Carla Meskill Teaching Languages with Technology: Communicative Approaches to Interactive Whiteboard Use, edited by Euline Cutrim Schmid and Shona Whyte WorldCall, edited by Ana Gimeno, Mike Levy, Françoise Blin and David Barr

Task-Based Language Learning in a Real-World Digital Environment The European Digital Kitchen Edited by Paul Seedhouse

BLOOMSBURY ACADEMIC Bloomsbury Publishing Plc 50 Bedford Square, London, WC1B 3DP, UK 1385 Broadway, New York, NY 10018, USA BLOOMSBURY, BLOOMSBURY ACADEMIC and the Diana logo are trademarks of Bloomsbury Publishing Plc First published in Great Britain 2017 Paperback edition first published 2018 Copyright © Paul Seedhouse and Contributors, 2017 Paul Seedhouse has asserted his right under the Copyright, Designs and Patents Act, 1988, to be identified as the Editor of this work. For legal purposes the Acknowledgements on p. xiii constitute an extension of this copyright page. Cover design by James Watson Cover image © Shutterstock All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage or retrieval system, without prior permission in writing from the publishers. Bloomsbury Publishing Plc does not have any control over, or responsibility for, any third-party websites referred to or in this book. All internet addresses given in this book were correct at the time of going to press. The author and publisher regret any inconvenience caused if addresses have changed or sites have ceased to exist, but can accept no responsibility for any such changes. A catalogue record for this book is available from the British Library. Library of Congress Cataloging-in-Publication Data Names: Seedhouse, Paul, editor. Title: Task-based language learning in a real-world digital environment : the European digital kitchen / edited by Paul Seedhouse. Description: London ; New York : Bloomsbury Academic, an imprint of Bloomsbury Publishing Plc, [2017] | Series: Advances in Digital Language Learning and Teaching; 4 | Includes bibliographical references and index. Identifiers: LCCN 2016038820| ISBN 9781474264075 (hb) |ISBN 9781474264051 (epub) Subjects: LCSH: Language and languages--Computer-assisted instruction--Europe. | Language and languages--Computer network resources--Europe. | Web-based instruction--Europe. | Task analysis in education--Europe. | Interaction analysis in education--Europe. | Web-based instruction--Europe. Classification: LCC P53.28 .T27 2017 | DDC418.00285--dc23 LC record available at https://lccn.loc.gov/2016038820 ISBN: HB: 978-1-4742-6407-5 PB: 978-1-3500-8212-0 ePDF: 978-1-4742-6406-8 ePub: 978-1-4742-6405-1 Series: Advances in Digital Language Learning and Teaching Typeset by Newgen Knowledge Works (P) Ltd., Chennai, India. Companion Website: www.europeandigitalkitchen.com To find out more about our authors and books visit www.bloomsbury.com and sign up for our newsletters.

Contents List of Illustrations Notes on Contributors Acknowledgements Transcription Conventions

vii x xiii xiv

Part 1 Background 1 Introduction Paul Seedhouse 2 Locating the European Digital Kitchen in Its Research Context Sandra Morales

3 19

Part 2 Design 3 The Pedagogical Design of the Digital Kitchen Paul Seedhouse 4 The Technology behind the European Digital Kitchen for Language Learning Paul Seedhouse 5 The Human Viewpoint and the System’s Viewpoint Natacha Niemants and Gabriele Pallotti

45 69 99

Part 3 Implementation 6 Assessing and Promoting Language Development in an Interactive Learning Environment Jana Roos, Nina Reshöft, Lea Hartung and Johanna Bußwinkel 7 Cooking, Interaction and Learning: The Finnish Digital Kitchen as a Language Learning Environment Salla Kurhila and Lari Kotilainen 8 ‘The More I Cook, the More I Learn’: Tracing Ava’s Learning Itinerary through Her Participation in Four Cooking Sessions Dolors Masats, Marta Juanhuix and Javier Albines 9 Vocabulary Learning in a Real-World Digital Environment Gabriele Pallotti, Natacha Niemants and Paul Seedhouse 10 Sight and Touch in Vocabulary Learning: The Korean Digital Kitchen Jaeuk Park and Paul Seedhouse

137 157

181 207 231

vi

Contents

Part 4 Conclusions 11 Conclusions and Future Developments Paul Seedhouse

261

Index

283

Illustrations Figures 2.1 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 5.1 5.2 5.3 5.4 6.1 6.2 6.3 6.4 7.1

Development of the EDK Example of slide in audio-visual vocabulary slideshow (screenshot) This image relates to line 10 Example of cooking instructions with specific cooking terms Example of prompts for the learners This image relates to line 500 Example exercise from the post-task The graphical user interface (GUI) Mixing the ingredients Wireless accelerometer sensors Sensors attached to objects The tablet in its casing Interaction tools Overview of the system The authoring tool interface Selecting sensors for the success conditions Pre-task activity outline with supports During-task activity outline with supports Help 2 multimedia gloss in the pre-task phase Help 3 multimedia gloss video in the during-task phase Summary of materials design activities Pair 20 (Recipe 2, Step 25) Help 1 Available Pair 20 (Recipe 2, Step 25) Help 1 Taken Pair 20 (Recipe 2, step 25) Help 2 Available Pair 20 (Recipe 2, Step 25) Help 2 Taken Pair 24 - Making scones Pair 11 - Making “Milchreis” German pre-test/post-test for Milchreis (prepositions) German pre-test/post-test for Milchreis (verbs) Pointing gesture, extract 2b, line 7

31 51 52 54 54 56 59 60 64 71 72 73 76 76 82 83 91 93 94 94 96 122 126 127 128 145 147 151 151 163

viii

7.2 7.3 7.4 8.1 8.2 8.3 9.1 9.2 9.3 9.4 9.5 9.6 9.7

Illustrations

Dana gazes at the tablet, extract 4b, lines 1–2 Dana and Alex look at each other, extract 4b, lines 4–5 Walking towards the cooker, extract 4b, line 6 Utensils and ingredients have inserted or attached sensors Interacting with the kitchen Measuring gesture The test cycle and the task cycle Red and green interaction tools Help 2: multimedia gloss in the pre-task phase Help 3: multimedia gloss video in the during-task phase Lexical production scoring protocol (spoken) Study 1: mean score on 16 items in pre, post and delayed-post test Study 1: scores for ‘sieve’ and ‘milk’ in pre, post and delayed post-test 9.8 Study 1: activities undertaken by learners between the post and delayed-post test 9.9 Study 2: pre- and post-test scores for individual lexical items in Recipe 1 9.10 Study 2: pre- and post-test scores for individual lexical items in Recipe 2 10.1 Mixed methods approach 10.2 Mixed methods data sequencing 10.3 Original experimental design to be adapted 10.4 Groups in relation to variables 10.5 A representation of the final adapted quasi-experimental design 10.6 Carrying out the task 10.7 Pre-task in the classroom 10.8 The during-task in the classroom 10.9 The post-task in the classroom 10.10 Task and test procedures 10.11 Productive pre-test, post-test and delayed-post-test in the KDK: name the object 10.12 Receptive immediate and delayed post-tests in KDK: Matching labels to objects 10.13 Productive pre-test, post-test and delayed-post-test in the classroom: Name the object 10.14 Receptive immediate and delayed post-tests in the classroom: Matching labels to objects

171 171 172 185 190 194 212 212 213 213 218 220 221 221 225 226 233 234 235 236 236 238 239 239 240 242 243 244 245 245

Illustrations

10.15 A comparison of overall vocabulary gains in the KDK and classroom 10.16 Vocabulary gains in KDK and classroom settings 11.1 The authoring tool interface 11.2 Techniques for attaching sensors to equipment 11.3 The task and test cycles 11.4 A sensor log 11.5 The ELAN multi-tier transcription format combining human talk and action with the system’s actions 11.6 Enjoying the kitchen task

ix

246 252 265 269 275 277 278 279

Tables 2.1 3.1 4.1 6.1 6.2 6.3 9.1 9.2 9.3 9.4 9.5 10.1 10.2 10.3 10.4

TBLT in the EDK Task design features of the FDK EDK task design components LanCook descriptors for listening comprehension based on the CEFR Results for Learners 3 and 4 (prepositions) Results for Learners 3 and 4 (verbs) Study 1, mean and standard deviation scores Study 1, statistical significance in relation to the tests Study 2, recipe 1, mean and standard deviation scores Study 2, recipe 2, mean and standard deviation scores Study 2, all participants, mean and standard deviation scores The number of international languages used by participants Vocabulary test results (whisk) Mean differences between pre-test, post-tests and delayed post-tests Languages and scripts used in the transcripts

24 48 90 140 152 154 220 220 224 224 224 237 241 247 248

Notes on Contributors Editor Paul Seedhouse is professor of Educational and Applied Linguistics and School Research Director in the School of Education, Communication and Language Sciences, Newcastle University, UK. His monograph The Interactional Architecture of the Language Classroom was published in 2004 and won the Modern Languages Association of America Mildenberger Prize. He has worked with colleagues in Computer Science at Newcastle to develop iLab:Learn, a centre to develop digital educational technology. He led the French Digital Kitchen project, which won the EU Language Label prize in 2011.

Contributors Javier Albines is a PhD student at Universitat Autònoma de Barcelona and is currently writing his dissertation on the Spanish cuisine. He has a BA in Education from the Universidad Nacional Mayor de San Marcos, Lima (Perú), a MA in Educational Research and ample experience as a Spanish secondary teacher. Johanna Bußwinkel is a teacher of English and Spanish at a German secondary school. Lea Hartung is research assistant in English language education at the University of Paderborn, Germany. She teaches English and French at Goethe Gymnasium Bensheim, Hesse. Marta Juanhuix has a MA in Catalan Language and Linguistic Variation and is currently a part-time lecturer and researcher at Universitat Autònoma de Barcelona and Head of the Catalan Language Service at Universitat Internacional de Catalunya. She conducts her research in the fields of Pragmatics, Sociolinguistics and Ethnography of Communication. Lari Kotilainen is a postdoctoral researcher at the Department of Finnish, Finno-Ugric and Scandinavian Studies, University of Helsinki, Finland. His

Notes on Contributors

xi

earlier work has mainly focused on cognitive and construction grammar, but his teaching career as a Finnish as a foreign language teacher at the universities in Finland, Germany, Sweden and Russia has led his interests to turn more to second language learning. Salla Kurhila is university lecturer at the Department of Finnish, Finno-Ugric and Scandinavian Studies, University of Helsinki, Finland. Her monograph Second Language Interaction was published in 2006. Her publications include studies on different phenomena in second language conversations, language learning in interaction and correction and intersubjectivity in conversation. Dolors Masats is a full-time lecturer and researcher at Universitat Autònoma de Barcelona. Her research interests include (but are not limited to) CA for SLA, language awareness, language across the curriculum, project-based learning and the use of video and ICT in language learning. Sandra Morales is an experienced English teacher and teacher trainer in higher education. At present, she is a PhD researcher and research assistant in Ilab:Learn (School of Education, Communication and Language Sciences) at Newcastle University, UK. Her current research includes the observation of how L2 teachers develop technological and pedagogical competence to teach languages using technology. Natacha Niemants received her PhD in Comparative Languages and Cultures from the University of Modena and Reggio Emilia, where she works as research assistant. She previously taught translation and interpreting at the University of Macerata, and is currently teaching interpreting at the University of Bologna. Her research interests include interaction and task-based language teaching. Gabriele Pallotti is professor of language teaching methodology at the University of Modena and Reggio Emilia. His research focusses on L2 interaction and socialization, cross-cultural discourse analysis, methodology and epistemology in applied linguistics. He is the editor, with J. Wagner, of L2 Learning as Social Practice (2011). Jaeuk Park is a PhD candidate at Newcastle University, UK, and is currently writing his thesis on the Korean Digital Kitchen. His research interests include foreign language teaching and learning, CALL, Multimodality, interactional competence and intercultural competence. Nina Reshöft is a lecturer and researcher at the University of Paderborn, Germany. She studied English and Romance languages and linguistics. She works

xii

Notes on Contributors

in the field of English linguistics (teaching and research), and her research interests include second language acquisition, language and cognition and the expression of space and motion in language. Jana Roos is professor of Applied Linguistics and English Language Education at the University of Paderborn, Germany. Her research focuses on Second Language Acquisition and (Early) Foreign Language Teaching and Learning in Institutional Contexts.

Acknowledgements The French Digital Kitchen project was funded by the Digital Economy Programme, which is a Research Councils UK cross-council initiative led by the Engineering & Physical Sciences Research Council. The European Digital Kitchen was part of an EU project funded with the support of the Lifelong Learning Programme (Key Activity 2) of the European Union. This project was funded with support from the European Commission. This publication reflects the views only of the author, and the commission cannot be held responsible for any use which may be made of the information contained therein. Enormous thanks to project collaborators at Open Lab, Newcastle University, for design, construction and development of the technology and materials for all versions of the digital kitchens: Patrick Olivier, Anne Preston, Dan Jackson, Phil Heslop, Madeline Balaam, Thomas Ploetz, Saandia Ali, Ashur Rafiev, Clare Hooper, Rob Comber and the late Jürgen Wagner. The Lancook partners would like to thank everyone who has taken part in trials of the digital kitchens over the years, and for their feedback, which has contributed to the development of the environment, the technology, the recipes and teaching materials.

Transcription Conventions Punctuation marks are used to capture characteristics of speech delivery, not to mark grammatical units. [ ] =

(3.2) (.) word e:r the::: ? ! , . CAPITALS ◦◦ ↑↓ >< () (guess) [gibee] [æ]

S1/S2 RA KIT ((text)) ((text)) (xxx)

indicates the point of overlap onset indicates the point of overlap termination if inserted at the end of one speaker’s turn and at the beginning of the next speaker’s adjacent turn, it indicates that there is no gap at all between the two turns an interval between utterances (3 seconds and 2 tenths in this case) a very short untimed pause underlining indicates speaker emphasis indicates lengthening of the preceding sound a single dash indicates an abrupt cut-off rising intonation, not necessarily a question an animated or emphatic tone a comma indicates low rising intonation, suggesting continuation a full stop (period) indicates falling (final) intonation especially loud sounds relative to surrounding talk utterances between degree signs are noticeably quieter than surrounding talk indicate marked shifts into higher or lower pitch in the following utterance indicate that the talk they surround is produced more quickly than neighbouring talk a stretch of unclear or unintelligible speech. indicates transcriber doubt about a word in the case of inaccurate pronunciation of an L1/L2 word, an approximation of the sound is given in square brackets phonetic transcriptions of sounds are given in square brackets indicate that the talk they surround is produced slowly Student No. 1 or 2 Research Assistant The Kitchen indicate a turn performed as a gesture Transcribers’ comments indicate an uncomprehensible fragment

Transcription Conventions

xv

The Kitchen (KIT) is treated as one of the participants. Hence, its verbal turns are represented as plain speech (including pauses, emphasis etc.): KIT:

no (.) this is the knife

The kitchen’s non-verbal actions (such as help availability, success sounds or the display of pictures) which are designed to be overtly available to participants are encoded in double brackets. KIT: KIT: *H1*

((help 2 picture)) ((success sound)) The system is indicating that help level 1 is available

Part One

Background

1

Introduction Paul Seedhouse

This chapter provides information about the background to the European Digital Kitchen (EDK) project, its motivation, rationale and the real-world challenges which it seeks to tackle. It introduces the central argument of the book. Although individual projects may be ephemeral because of the pace of technological change, the analyses in this collection draw out general principles and a model in relation to real-world, pervasive environments for language learning employing digital sensor technology and a task-based approach. These principles may then continue to be applied to other such environments for language learning (and other areas of learning) which may be built in the future. An overview of the sections and chapters which form the collection is provided, introducing the reader to how pedagogical and technological designs complement each other, as well as to areas of research interest.

Aims of this book This book explains how a real-world, pervasive digital environment for language learning was conceived, designed, built, trialled and researched. How can the latest digital technology be used to create an environment in which people can learn European languages while performing a meaningful real-world task and experiencing the cultural aspect of learning to cook European dishes? This book explains how to do this from A to Z, covering how a digital environment for language learning was designed, built and researched, as well as revealing what actually happens in practice in terms of the learning experiences of users in five European countries. The project aimed to make language learning motivating by tapping into people’s interest in both cooking and technology – you can

4

Paul Seedhouse

learn a language while cooking and interacting with a speaking digital kitchen. The book explains the design and implementation principles and procedures involved, enabling readers to design and implement a real-world digital learning environment themselves in the same way. In-depth research studies show how the system was actually implemented in five different European countries with seven different languages (plus Korean) and with different types of learners. There are numerous photographs of the system in use and evidence of how and what 250 users actually experienced and learnt throughout the project. The research studies illustrate both the process of language learning in this environment as well as the product in terms of measurable gains in language proficiency. We demonstrate how a framework for researching language learning must be integrated into the design of the digital environment itself. The book is written for readers with a background and interest in applied linguistics, as well as educational technology or pervasive computing. We intend it to be particularly useful for postgraduate students and researchers who are planning to create a digital environment for language learning themselves. To make the study relevant, we draw out general principles and a model in relation to real-world, pervasive environments for language learning employing digital sensor technology and a task-based approach. These principles may then prove useful for application to other environments for language learning (and other disciplines) in the future.

Origins and rationale of the project In this section we explain how the project (or rather, three related projects) started and developed, as well as our reasons for choosing a kitchen environment and cooking as an appropriate task for language learning. Before we do so, we will unpack the terms ‘pervasive’, ‘real-world’ and ‘digital’. By digital we mean that a digital computer system communicates with learners and tracks their physical actions through digital sensors embedded in the utensils and ingredients which they manipulate. By real-world, we mean that users are actually in a real kitchen cooking real food, as opposed to virtual digital environments which simulate reality. By pervasive we refer to an application of digital technology which is familiar to the vast majority of people in developed countries, but which has not yet had much impact on language learning, namely pervasive computing. In this field, the technology is intended to retreat into the background, providing timely advice, help and feedback whenever necessary to enable users to perform a task. Readers will be familiar for some years now with

Introduction

5

the experience of having a computer giving them verbal instructions while they are performing the important task of driving a car (satellite navigation). The digital system, aided by satellite signal data, tracks their progress in the task and provides timely, context-specific feedback. So although pervasive computing has become firmly embedded in our everyday life in the case of some of our daily tasks, its potential has so far not been fully realized in the field of language learning. Weiser (1991) introduced ubiquitous computing (Ubicomp) as an approach ‘that takes into account the natural human environment and allows the computers themselves to vanish into the background’. There are a number of similar terms used to describe similar approaches, including ‘pervasive’, ‘ambient’, ‘everyware’. In this book we follow Ogata’s (2008) classification of digital learning environments and use the term pervasive as one particular type of ubiquitous environment. The digital kitchen is a computer-supported pervasive learning environment in that it has a high level of embeddedness in a context, but a low level of mobility. In other words, the technology is designed to function in an ordinary kitchen. What is the point of trying to create a pervasive, real-world digital environment for language learning? First, a number of well-known problems relating to classroom foreign language teaching are addressed by this project. The universal problem of classroom language teaching is that student rehearse using the language, rather than actually using the language to carry out real-world actions. In the digital kitchen, languages are learnt by physically carrying out a cooking task which engages all of the five senses. Second, there is the difficulty of bringing a foreign culture to life in the classroom, and cuisine offers a window into this culture: ‘The relationship among language, food and culture in a society is an inextricable one’ (Ayeomoni 2011: 51). In the digital kitchen environment, we intend that learners will be able to learn aspects of the language multimodally while performing a meaningful and enjoyable real-world task and will simultaneously experience the cultural aspect of learning to cook a foreign dish. Third, there is the issue of how to motivate people to learn languages. One solution is to tap into existing motivations, and one such is cookery. Currently there is a huge interest throughout the European Union in cooking, as can be seen in the number of cookbooks sold and the number of cooking programmes on TV. Many adult learners are motivated to learn European languages through their interest in cuisine and culture, and this project taps into this motivation. Also, many people find technology an inherently motivating tool for learning, as evidenced by the plethora of digital materials available for learning via a variety of platforms. Multimodal technology involving physical activity (such as Nintendo Wii

6

Paul Seedhouse

™) is particularly popular, and this project employs similar accelerometer sensor technology to track physical movements in three dimensions. User feedback indicates that many found the technology inherently motivating:  ‘Wonderful technology’; ‘The sensors were cool’; ‘I said that it was fun to do and that it’s great that you can do something practical whilst learning languages and different recipes and I really like the idea and the technology.’ Fourth, the pedagogical design employs task-based language teaching (TBLT), a well-researched approach to language learning, which prompts learners to achieve a goal or complete a task. So far TBLT has predominantly been based on tasks to be undertaken within the classrooms which simulate real-world tasks. Some innovations in TBLT have combined language learning with other, non-linguistic skills in a similar way to this project. Paterson and Willis’s (2008) English through Music, for example, aims to help children to absorb English naturally as they enjoy making music together. There have been few attempts to employ TBLT in naturalistic settings outside the classroom; the project described here is innovative in combining TBLT and digital technology in a naturalistic kitchen setting outside the classroom. Given the emphasis of the authentic task within TBLT, we have used the kitchen environment as a learning context since the act of cooking a meal is an authentic task with a clear goal and end product. Finally, it means that language learning can be normalized or integrated into everyday activities, such as cooking. One does not necessarily need to take time out of everyday life in order to learn some aspects of a foreign language. Pervasive computing specialists believe that digital sensors will be embedded throughout the homes of the future. We will therefore be able to tell our home what we want to do and it will be able to provide us with help and guidance for everyday tasks such as cooking food, carrying out a repair, travelling and so on. The technology to achieve this is already available and can be adapted to integrate language learning into the task. So, one aim of the project is to establish whether language learning can in principle be normalized by integrating it into everyday activities such as cooking at home. This project also engages with challenges at national and international levels. A significant challenge for the UK is how to improve the declining foreign language proficiency of its workforce. The number of pupils gaining a qualification in a foreign language has decreased significantly, while a recent British Academy report discussed concerns that the future of the UK’s world-class research base might be threatened because of the decline in foreign language learning. At the same time, the EU has recognized the problems that Europe faces in increasing foreign language proficiency, as the number of languages increases with new member states. The EU therefore promotes projects which try to increase social

Introduction

7

and professional mobility between member states and the integration of the large number of migrants currently entering the EU. At the international level, globalization of world economy, mass international travel and conflicts place a premium on projects which promote greater understanding and tolerance of other languages and cultures. We believe that pervasive digital computing, in combination with the realworld task of cooking, offers an appropriate means of addressing these problems and challenges. We therefore developed a series of digital kitchens, projects which involved taking a normal kitchen and adapting it by using digital sensor technology. In the Ambient Kitchen project (2007–2014), a host of pervasive sensors (including RFID, floor pressure and embedded accelerometers) were designed to monitor activity in the kitchen to assist dementia sufferers (Olivier et al. 2009). The system analysed the sensor data to attempt to identify breaks in task progression, so that assistive prompts could be given. When Paul Seedhouse and Patrick Olivier discussed the Ambient Kitchen, it became clear that the technology could be adapted to language learning, since cooking was both an appropriate task for the implementation of a TBLT approach and an interesting way of exploring foreign culture in tandem with a foreign language. In the French Digital Kitchen project (2010–2012), the Ambient Kitchen was therefore re-specified to create a language learning environment for British learners of French (Seedhouse et al. 2013). We used objects instrumented with accelerometers and activity recognition of the way objects were manipulated to inform the system of user actions, together with pre-recorded audio prompts and a user interface for interaction. In the EDK project (2011–2014), we expanded the system to seven languages and five countries and employed photo and video tools to explain unknown words to users. We simplified ‘activity recognition’ to object movement, as focus on specific cooking techniques was not required. We also developed an authoring tool to enable users to create their own recipes in different languages. We constructed a real-world, purpose-built kitchen (figure 1.1) that communicates with learners in a European language and gives them stepby-step instructions on how to prepare cuisine.. Digital sensors are inserted in or attached to all the kitchen equipment and ingredients, so the digital kitchen detects what learners are doing and gives them feedback, rather like sat-nav in cars. Learners are also able to communicate with the kitchens and can ask for help (photos, videos) if they don’t understand any foreign language words. Chapter 4 gives a more detailed account of the evolution of this technology. We chose to use a kitchen because cooking is a task which has considerable resonance with both language and culture. Cooking allows learners to

8

Paul Seedhouse

immerse themselves in an everyday real-world task with a tangible (and edible) product; the technology should operate facilitatively in the background. We designed the kitchen to support independent, autonomous, collaborative learning – that is, to be used by pairs of learners without the presence of a teacher. The requirement for users to cooperate in the cooking task should generate spoken interaction. The kitchen as a setting provides a tangible connection to what Skehan referred to as ‘real-world activities’ (1998: 95), where authentic language is used for communicative purposes and a real-world task outcome is seen in terms of a European dish, which is eaten at the end of the task. The notion of cooking as pedagogy or experiential learning has been specifically explored by cultural anthropological approaches as an activity involving ‘multisensory experiential learning’ (Trubek and Belliveau 2009: 16). The kitchen is seen to ‘engage students at an almost instinctive level; the smells, sounds, sights, textures and tastes excite senses and intellects’ (2009: 16). One strand of our research has been to investigate how well students can learn new vocabulary when physically touching and manipulating utensils and ingredients and this is reported in chapter 9. The very clear evidence is that being able to touch and manipulate objects in a meaningful task makes a significant difference to vocabulary learning of those objects. The physical nature of the kitchen task provides immediacy relating to all of the senses: sight, sound, smell, touch and (in the post-task) taste. A  kitchen setting is therefore eminently suitable for multimodal language learning. The digital kitchen is intended to offer a language learning environment which is not only multimodal but multisensory as well as a part of learners experiencing aspects of the foreign culture and cuisine. The physicality of the experience is intended to enhance learner motivation and interest. TBLT has in the past sometimes been criticized for failing to produce enjoyable, motivating and engaging classroom tasks for learners. The EDK project started the design process by identifying a real-world task which people in all cultures find engaging and enjoyable; the evidence for this is the popularity of cooking programmes on TV around the world. The cooking task is enjoyable and engaging because it involves all five senses, because it is an act of creation in which you are actively involved and because you can consume what you produce. Cooking is not only a universal task which is important in all cultures, but also provides a window into that culture:  ‘Food is a central activity of mankind and one of the single most significant trademarks of a culture’ (Kurlansky 2004: 11). In the EDK, users have the novelty of learning a new cuisine, culture and language all at the same time as a form of integrated learning.

Introduction

9

Figure 1.1

From a technological viewpoint, our main focus has been on how the situated nature of language instruction (timeliness and in context of the tasks) could be supported by technology. In broader terms, the project explores how technology can be used to perform real-world and culturally engaging tasks via the medium of a foreign language and also provides an example of how two rather different sets of skills (language and cooking) may be acquired at the same time by use of appropriate technology. Although some of the chapters focus on the technological development of the system, the EDK needs to be understood as a real-world environment for language learning in which learners make active choices as to which aspects of that environment they are going to employ to help them complete their task. The system is of course a key component of that environment. However, the evidence detailed in this book is that learners make use of a number of other environmental supports, depending on their own learning styles, strategies and prior knowledge of L2 and of cooking. In chapters 3, 7 and 8, we see how participants make use of their partners to collaborate in both the cooking task and the language learning task. Chapter 3 demonstrates how learners are able to exchange relevant skills with each other. Chapter 7 examines in detail how

10

Paul Seedhouse

learners collaborate to manage language learning, while chapter 8 shows how the performance of an individual learner over four sessions varies according to the learner she is partnered with. As seen in figure 1.1, the real-world kitchen environment provides a rich, multisensory, multimodal and stimulating learning environment of ingredients and equipment. For example, learners who do not know the French word ‘gousse de vanille’ and had never seen a vanilla pod before, were nonetheless able to jointly find and identify a ‘pod of vanilla’. In a post-task interview a participant pointed out that ‘unfamiliar words were generally easy enough to work out from the context’.

The main argument The main argument of the book is this: Individual projects may be transient due to the rapidity of technological innovation. Nonetheless, the studies in this book delineate general principles and a model for the development of real-world, pervasive environments for language learning which use digital sensor technology and TBLT. These principles are then applicable to the development of other such environments for language learning (and other disciplines) in the future. The key features of any pervasive, real-world digital environment for language learning are: 











Participants physically carry out real-world tasks (using real-world equipment) which are embedded in everyday, real-world contexts such as a kitchen, an office or a shop. The task can be broken down into a series of specifiable physical actions. Participants should receive some L2 input from some source and be able to learn some aspects of the L2 by performing the task. Participants physically touch and manipulate real-world objects while carrying out the task, have the opportunity to learn the L2 names of these objects (as well as the processes involved) and may be evaluated on their learning if required. The digital system can track how participants are carrying out the series of physical actions which constitute the task via a number of digital sensors embedded in the environment. The technology is designed to facilitate performance of the task but is not the focus of the activity – it remains in the background. The system provides timely instructions, feedback, help and tips to users to enable them to perform the task. The feedback facilitates multimodal and multisensory learning by use of audio, photos and videos.

Introduction 



















11

The learning environment provides a range of possible supports or scaffolds to cater for a variety of learning styles and L2 proficiency levels, and learners can decide for themselves which to make use of. Participants can ask the system for help or for explanations, but are not obliged to. Participants may work in pairs or groups and tasks may be designed to promote interaction between them in L1 or L2. There should also be a real-world outcome, for example, drawing a picture, singing a song. A task cycle should be planned, consisting of pre-task, during-task and post-task. If testing of language learning is required, there should be careful planning of how the testing cycle relates to the task cycle (see Chapter 9, figure 9.1). Evaluation of language learning is integrated into the experience in some way so that participants (and others) can see what has been learnt. Consideration should be given to how evaluation of learner performance can be undertaken and related to the Common European Framework of Reference for Languages (CEFR). Chapter 6 provides examples of how to relate performance to Can Do descriptors and CEFR bands. It is a good practice to develop an authoring tool so that materials can easily be developed for other languages. The task employed should be a motivating and engaging one.

It is essential that a framework for researching both the process and the product or the effectiveness of language learning is built into the structure of the environment itself from the beginning rather than as an afterthought. These principles are intended to guard against obsolescence in the future; the tasks that people wish to learn will change, as will the technology, but the principles may be adapted to these. During the course of this book, we explain how the EDK project developed procedures, systems and materials based on the above principles. We will return to these key features again in the concluding chapter in order to evaluate how these features have been put into practice.

The design and structure of the book In this section I explain how the book is structured, how pedagogical and technological designs complement each other and how the diverse component chapters are intended to cohere to form a whole. The whole book is pedagogically

12

Paul Seedhouse

conceived as a case study which will enable others to design and implement a real-world pervasive digital learning environment using a task-based approach. It is intended to be particularly useful as a model for students of pervasive computing who are required to develop real-world applications as part of their studies. At Newcastle University, for example, year 1 Computing Science undergraduate students produce language learning apps for their end-of-year projects. Conversely, students of applied linguistics or education who are interested in developing applications of digital technology should also find this model useful. Moreover, the book serves as an example of interdisciplinary collaborative work on the development of digital learning environments. In their chapter conclusions, all chapter authors relate their findings to the bigger picture such as: What can their chapter tell others about how to design and implement a real-world digital learning environment? What general lessons can be drawn and what advice might be given to others? The analyses of data and the discussions in this collection draw out general principles and procedures in relation to real-world digital environments for language learning employing digital sensor technology and a task-based approach. Chapter 11 summarizes these principles and procedures, and condenses them into an explicit model, which may then be used to design and implement future real-world pervasive environments for learning using digital sensor technologies. There is also a discussion of other real-world learning settings that might be suitable for approaches employing digital sensor technology and a task-based pedagogy. The research infrastructure for both the process and product or effectiveness of language learning is integral to the structure of the environment. We review the lessons learnt from the empirical research presented in this collection and arrive at a methodological model for researching language learning in realworld digital environments. The emphasis throughout the book is on the principles and procedures of designing, implementing, researching and evaluating a real-world pervasive digital learning environment employing a task-based approach. The EDK project illustrates a successful, functioning environment, so that there is a continuous interplay between theory and practice. This book comprises four parts. Part  1 consists of the ‘Introduction’ (chapter 1) and Chapter 2, which is a literature review chapter entitled ‘Locating the European Digital Kitchen in its Research Context’. This chapter sets the project in the context of the research literature on language learning pedagogy and technological developments. It does so by reviewing the literature on ComputerAssisted Language Learning, Human-Computer Interaction and language

Introduction

13

learning and teaching. The project is portrayed as drawing on a number of research traditions and methodologies, which are adapted to develop project aims. The chapter identifies the areas in which the project is innovative, as well as the ways in which it builds on previous work. Part 2 discusses the design of the environment and consists of three chapters which cover pedagogical, interactional and technological aspects of the design process, as well as how these can be analysed and researched. Chapter 3 covers pedagogical design and explains the pedagogical principles underlying the system, materials and procedures. The project employs the principles of TBLT. Tasks are divided into three phases: pre-task, during-task and post-task, providing a clear design structure for materials, for conduct of sessions and for evaluation of performance. It is argued that the project attempts to realize some of the advantages of TBLT using digital technology in a real-world setting outside the classroom. The chapter explains and exemplifies how the concepts and principles of TBLT have been operationalized in the digital kitchen setting, from the pedagogical perspective. Chapter 4 explains the technology used in the EDK project, assuming a nonspecialist readership. We look at how the technology was designed and developed to deliver specific environmental features for the EDK, and how materials were designed to work with the technology. There is particular emphasis on two technological aspects: How does the system recognize what is going on in the kitchen? How can the system present appropriate materials to the users at the right time, thereby promoting language learning, interaction and effective cooking? The technological components of the pervasive system consist of the sensors, the tablet with its graphical user interface and the interaction tools. The authoring tool allows materials to be uploaded onto the system. The principles behind the technology are described, and the TBLT principles underlying materials design for individual languages and recipes are elucidated. Chapter  5 compares the human viewpoint and the system’s viewpoint, describing how the same real-world language learning session is ‘seen’ from two different points of view; the human users’ and the digital systems. Two transcripts are placed side by side, one containing the users’ verbal and nonverbal actions, the other showing sensor logs from the system, that is, evidence of who did what and when. This contributes to an understanding of what type of ‘communication’ takes place in an EDK cooking session, with implications for redesign and technology development. We also see how a task is implemented from both perspectives. This chapter demonstrates how a session in a real-world

14

Paul Seedhouse

digital environment can be seen as a case of communication between computers and humans, which is best understood if the reader is aware of the perspectives of both sides. Part  3 provides a detailed research picture of how the digital environment was implemented in five countries with 250 learners and presents a range of data and perspectives which explain how it works from a user’s perspective. The digital kitchen environment is a complex one, involving users across a range of experiences. Therefore, the contributing authors have employed a range of research methodologies and both quantitative and qualitative approaches to investigate both the product of learning (in terms of gains) and the process of learning. The studies have focused on a range of phenomena in order to do justice to the multiplexity of experiences. The chapters investigate listening skills, the learning experience, the progress of an individual, vocabulary learning and comparing the human and system viewpoints. Chapter  6 investigates the assessment and development of listening skills. One of the key competences promoted through the EDK is listening comprehension. By listening to the digitized cooking instructions in the target language, learners at different competence levels are guided in the preparation of one food item and achieve an outcome, namely the preparation of a dish typical of a European country related to that language. As they implement the procedure, learners demonstrate how well they can understand and follow the instructions. Examples taken from audio-visual data collected during cooking sessions in the German Digital Kitchen are presented. They are analysed in relation to specific descriptors for listening and receptive skills developed for the EDK. The descriptors are related to the scale for overall listening comprehension in the CEFR. The data illustrate that assessing learner data in this way can serve two different purposes. First, it is possible to rate the learners’ linguistic performances during the cooking sessions. Second, it becomes possible to determine which future language learning goals can be identified for individual learners in order to support them in their language learning process. In the last section of the chapter, the results of a vocabulary pre-test and a post-test are presented, which exemplify how cooking in the EDK can support learners in understanding instructions and learning topic-related vocabulary. Chapter  7 covers cooking, interaction and learning in the Finnish Digital Kitchen. The authors discuss and demonstrate how language is learned in and through interaction during the real-life activity of cooking. The data consist of video recordings from EDK cooking sessions by exchange students learning Finnish in different Finnish universities. In particular, the focus is on how

Introduction

15

the authentic environment supports learning. Given that the participants are involved in a goal-oriented real life activity, our examples illustrate that they have great motivation to understand the language used by the computer system and perform the cooking activities in the manner requested. Thus, when confronted with a linguistic problem, students will not let the problem pass. Instead, they actively seek affordances and use both the interactional and technological resources that are available to them in the situation. For example, when the computer uses an unknown word, learners can get support from each other, or they can deduce the meaning of the word on the basis of their former knowledge and experience of cooking. It will be argued that the interactional setting and taskbased nature of the EDK offers the learners resources to construct meanings and thus scaffold their understanding process. Chapter  8 is entitled “The More I  cook, the More I  Learn” and traces one student’s learning itinerary through her participation in four cooking sessions. The Catalan and Spanish materials for the EDK offer a variety of four dishes, two in each language. Most often, each recipe was trialled with different pairs of students. However, a few individual learners volunteered to try another one. This is the case of Ava, a French learner of Catalan, who had some knowledge in Spanish as well. As language learning is a social activity, Ava’s case is an interesting one because her performance in the EDK varied, depending on who she was cooking with, and on the familiarity she gained with the system. In this chapter, we follow Ava’s individual progress as a learner in the EDK. We will chronologically examine how, in each trial, she interacts with the system; her discourse is mediated both by the task instructions and by the relationship she establishes with her cooking partner; they both interpret and conduct the cooking task; she activates and puts into play her linguistic resources and repertoires, as well as the knowledge she possesses on how the kitchen works, in order to negotiate meaning and to produce the recipe. Chapter 9 reports on how learning of vocabulary items was assessed in the English and Italian Digital Kitchens, the methods being different but complementary. In Italy the results of various questionnaires and forms of lexicon assessment were reported and supplemented with micro-genetic analyses of learning of new vocabulary items using transcript data. In England a quantitative approach was adopted, involving pre-test, post-test and delayed post-test of sixteen vocabulary items (utensils and ingredients) learnt by individuals. The results showed significant gains resulting from the learning sessions in the EDK. A combination of the two studies provides a rich and detailed perspective on one single aspect of the overall learning experience in the EDK.

16

Paul Seedhouse

Chapter 10 reports on the Korean Digital Kitchen (KDK) research project, which investigated the relative importance of sight and touch in vocabulary learning. The project involved teaching a non-European language with a different script, culture and cuisine. We attempted to disaggregate ‘seeing’ the objects and ingredients from the other supports in the KDK environment and took two parallel groups in a quasi-experimental design. One group carried out the standard digital kitchen cooking task as described in this book, whereas the other group carried out a parallel task in the classroom, learning the same items by looking at photographs of the objects only. This aimed to see whether there was any significant difference between vocabulary learning, which involved seeing the learning items only, and learning which involved touching and using the items in a meaningful task in the KDK environment. Learners were able to learn Korean vocabulary items significantly better in the KDK than in the classroom for both reception and production, for both spoken and written media and in both the post-test and delayed-post-test. Being able to touch and manipulate physical objects as part of a meaningful task in the KDK involving all of the senses as well as self-organized learning helped students learn vocabulary more effectively than merely being able to see them in photographic format in a classroom. This was suggested by the qualitative analysis of the interaction and learning processes in each environment. In Part 4, Chapter 11 draws together all of the points which emerged from the empirical evidence and discussions in the previous chapters. The analyses of data and discussions in this collection have drawn out general principles and procedures in relation to pervasive environments for language learning employing digital sensor technology and a task-based approach. These principles and procedures are summarized in this chapter as an explicit model for the future design and implementation of other real-world pervasive environments for learning using digital technologies and TBLT. The model delineates the stages and procedures involved and suggests which other real-world learning settings might be suitable for similar projects. Finally, we review the lessons learnt from the empirical research presented in this collection and present a methodological model for researching language learning in real-world digital environments. One advantage of the EDK project is that all of its aspects have been conceived, designed, implemented and researched from scratch by the same team, including technology, pedagogy, materials, assessment procedures, research design and data analysis. This means we can be confident that learning is due to the designed interventions. Another key aspect to this project has been the

Introduction

17

European dimension. The EDK was an EU-funded language learning project with the short name LanCook, meaning ‘Learning languages, cultures and cuisines in digital interactive kitchens’. This project developed language learning materials for European languages and cuisines: English, French, German, Spanish, Catalan, Italian and Finnish. It involved partners throughout Europe: Newcastle University (UK), Università degli Studi di Modena e Reggio Emilia (Italy), Helsingin yliopisto (Finland), Universität Paderborn (Germany) and Universitat Autònoma de Barcelona (Spain). Websites in all these languages form part of the main project website www.europeandigitalkitchen.com. The five different partners involved developed and trialled new materials with a range of users linked to CEFR levels A to C; adult, higher education and vocational students as well as migrants and overseas students. This widened the range of groups for whom the materials were relevant. Furthermore, working as a trans-European consortium led to cross-fertilization of ideas concerning the relationships between language, cuisines and cultures, as well as different working practices. LanCook also engaged with many European agendas by promoting language learning, as well as linguistic and cultural diversity, in that our project provided materials in seven different European languages.  The project supports development of innovative ICT for language learning, mobility across EU countries and the integration of migrant language learning. In Finnish and Catalan, our project promotes the acquisition of less-used languages. With the development of the authoring tool, materials for the EDK can now be produced for any language in the world; Chapter 10 reports on the follow-up Korean Digital Kitchen project. A wealth of materials about the project, including videos of the EDK in use, can be found on www.europeandigitalkitchen.com.

References Ayeomoni, M. O. (2011), ‘Language, food and culture: Implications for language development and expansion in Nigeria’, International Journal of Educational Research and Technology, 2 (2): 50–55. Kurlansky, M. (2004), Choice Cuts: A Savory Selection of Food Writing from Around the World and Throughout History, Michigan: SOS Free Stock. Ogata, H. (2008), ‘Computer supported ubiquitous learning: Augmenting learning experiences in the real world’, Fifth IEEE International Conference on Wireless, Mobile and Ubiquitous Technology in Education. Olivier, P., X. Guangyou, M. Monk and J. Hoey (2009), ‘Ambient kitchen: Designing situated services using a high fidelity prototyping environment’, IPETRA

18

Paul Seedhouse

09: Proceedings of the 2nd International Conference on Pervasive Technologies Related to Assistive Environments: 1–7, New York: ACM Press. Paterson, A., and J. Wills (2008), English through music, Oxford: Oxford University Press. Seedhouse, P., A. Preston, P. Olivier, D. Jackson, P. Heslop, T. Plötz, M. Balaam and S. Ali (2013), ‘The French digital kitchen: Implementing task-based language teaching beyond the classroom’, International Journal of Computer Assisted Language Learning and Teaching, 3 (1): 50–72. Skehan, P. (1998), A Cognitive Approach to Language Learning. Oxford: Oxford University Press. Trubek, A. B., and C. Belliveau (2009), ‘Cooking as pedagogy: Engaging the senses through experiential learning’, Anthropology News 50(16), doi: 10.1111/ j.1556-3502.2009.50416.x. Weiser, M. (1991), ‘The computer for the 21st century’. Available online: http://www. ubiq.com/hypertext/weiser/SciAmDraft3.html. (accessed 19 December 2013).

2

Locating the European Digital Kitchen in Its Research Context Sandra Morales

Introduction Languages are essential for communication and the development of individuals, societies and cultures. Therefore, to promote second language acquisition (SLA), investigations in second language teaching and learning become relevant so as to contribute theories and practical strategies to facilitate language education. Through decades, there have been a series of debates around issues regarding what to teach (e.g. whether to prioritize linguistic features and/ or communication, productive and/or receptive language skills) and how to teach languages more effectively. To this end, language teaching and learning approaches (e.g. grammar-translation, audio-lingual, communicative language teaching) have been proposed according to different theoretical perspectives (e.g. cognitive, interactional) to meet the learners’ needs (Cook 2008; Hall 2011; Larsen-Freeman and Anderson 2011). Due to this variety of language teaching and learning views, the current reality of language education is that of a ‘methodological eclecticism’ (Hall 2011: 60), which allows educators and researchers to combine theoretical and pedagogical principles from different methods and areas of research to improve SLA. Bearing this in mind, the European Digital Kitchen (EDK) is a multimodal innovation in language teaching and learning due to its interactive, effective and sustainable framework (Olivier et al. 2009; Hooper et al. 2012; Seedhouse et al. 2013; Seedhouse et al. 2014). The design, implementation and evaluation of the EDK incorporate principles or ‘ingredients’ from second language teaching and learning methods which engage learners in communicative real-world tasks through an everyday life activity such as cooking. The EDK also applies criteria

20

Sandra Morales

from computer-assisted language learning (CALL) in its user-friendly authoring tool, interactive interface, help options and feedback cues to scaffold learners. Furthermore, foundations from human-computer interaction (HCI), ambient technology, in particular, inform the technological choices of the EDK’s software and hardware. In addition, cultural aspects of language (e.g. authentic recipes) are also considered as a means to provide learners with a genuine learning situation so as to improve their understanding of pragmatic elements of SLA. This novel combination of pedagogical and technological properties allows learners to use the EDK outside the classroom to a more realistic context. Here, learners can work collaboratively and independently without direct guidance of a teacher. Instead, they are at the centre of their learning experience and teachers take the role of material developers (i.e. supported by the authoring tool) and facilitators of the learning process (i.e. lesson planning). In general, language learning and teaching innovations such as the EDK, involving ubiquitous computing and multimodal high tech resources, have been under-researched mainly due to their interdisciplinary nature. The aim of this chapter is therefore to locate the EDK in its research context. To do so, we explore how languages are taught considering language teaching approaches such as communicative language teaching (CLT) and task-based language teaching (TBLT), in particular, as the main methods that support the pedagogical foundations of the EDK. The role of technology in SLA is also examined, as well as the main concepts of CALL and HCI. The principles regarding language teaching and their importance in learning with technology are discussed in order to show their contribution in SLA. They are further explored as they build up the foundation of the EDK’s unique framework as an innovative resource for language teaching and learning.

Language learning and teaching approaches For decades, language teaching and learning approaches have been a reflection of the theoretical views of certain eras (Richards and Rodgers 2001; Cook 2008; Hall 2011; Cutrim Schmid and Whyte 2014). For instance, in the 1930s, behaviouristic tendencies predominated in language teaching and learning and hence approaches such as the direct method (e.g. language association) and audio-lingual method (e.g. repetition, memorization of structures) appeared. In the 1940s, linguistic knowledge became the main indication of language proficiency, thus grammar-translation techniques (e.g. direct L1/L2 translation of

Locating the EDK in Its Research Context

21

forms) were used to foster language learning and teaching. However, regardless of the grammatical accuracy that was observed, and demanded at that time, learners could mainly use linguistic features in isolation and lacked the proper knowledge to apply their competence in different communicative situations. As a result, in the 1970s, language educators shifted their teaching focus towards communication and fluency. In this sense, the learners’ communicative competence (i.e. awareness of language rules and appropriate use, Hymes 1972) started to develop in language classes under the assumption that conveying messages in the target language (TL) correctly seemed to be more important than proper use of grammatical forms. In this sense, SLA was theoretically linked to how people learn their first language: immersed in an authentic environment where the language is used in order to obtain input naturally. Based upon these assumptions, the CLT approach appeared as a significant strategy to recreate contexts of ‘real’ communication in the second language.

Communicative language teaching (CLT) In general terms, ‘CLT is an approach to L2 instruction which is primarily meaning-based and includes attention to both fluency and accuracy’ (Spada 2007:  272). To promote second language communication, teachers started to incorporate CLT into their language lessons so that their learners could be exposed to opportunities where they had to interact to know the meaning and, in turn, the forms of the language (Spada 2007; Cook 2008). Strategies such as role plays, information gap activities and situations that resembled real life were therefore applied with language learners to foster the development of language skills (Cook 2008; Hall 2011; Larsen-Freeman and Anderson 2011). At the same time, the role of teachers in CLT changed to that of a facilitator who engaged students into using the language rather than that of a knowledge authority and feedback provider. The use of CLT in language classrooms has predominated since the 1970s (Nunan 1987; Richards and Rodgers 2001). However, its popularity has not been spared from challenges and ‘misconceptions’ (Spada 2007: 269). For example, debates regarding whether or not learners should use their native language (L1) as a resource to complete CLT tasks have arisen, or, more importantly, what ‘judicious use’ of L1 (Larsen-Freeman and Anderson 2011: 125) means and its impact in the language classroom. Also, it has been difficult for learners to getting used to the idea that feedback, something that is relevant for SLA (Hattie

22

Sandra Morales

and Timperlay 2007), is not given by the teachers explicitly. As CLT is learnercentred, the students can work in pairs or groups, which highlights even more the role of the teacher as a guide and not the (only) language ‘expert’, as students can also contribute to each others’ learning. Nevertheless, this poses questions as well. How can learners use the language if they do not receive useful input that they can actually understand and use? (see notion of comprehensible input in Long 1998). Since learners in CLT should pay attention mainly to the message they want to communicate rather than the form to use to achieve that goal, a lack of linguistic knowledge might undermine their performance, causing issues in terms of, for example, language evaluation. As the definition and use of CLT can be wide and therefore interpreted in different ways, it has been suggested (Cook 2008; Larsen-Freeman and Anderson 2011) that it can be manifested in either ‘weak’ or ‘strong’ forms. The former suggests incorporating controlled practices (e.g. grammar exercises) into CLT to provide comprehensible input that students can later use in their interactive tasks. The latter, on the other hand, suggests a more open CLT approach where learners can use the language naturally and focus mainly (or only) on the meaning of the task to achieve a communicative goal (Savignon 2007; Spada 2007; Cook 2008; Hall 2011; Larsen-Freeman and Anderson 2011). This strong version of CLT has been commonly denominated TBLT. Thus, it is possible to link the principles of CLT in the strong form with the development of the EDK. In this sense, the different cooking tasks that the users perform foster collaborative negotiation and language interaction that make language learning a more natural process.

Task-based language teaching (TBLT) As explained earlier, TBLT derives from CLT and has gradually become popular in second language teaching and learning mainly due to its use of ‘authentic tasks’ to promote second language interaction and negotiation. It also fosters learner-centred, collaborative and autonomous learning as, for instance, the teacher acts as a facilitator of tasks and students take greater responsibility on their learning to complete the tasks. Although there is no clear consensus about a standard definition of ‘task’, TBLT researchers agree that a task is a means which would systematically help the students to improve their productive and receptive language skills (Long 1985; Willis and Willis 2001; Nunan 2004). Ellis (2003:  3)  differentiates tasks

Locating the EDK in Its Research Context

23

from exercises by stating that ‘ “Tasks” are activities that call for primarily meaning-focused language use. In contrast, “exercises” are activities that call for primarily form-focused language use’ and points out that the common goal of both of these activities is promoting language learning. To complete tasks, students use the language and incorporate cognitive and communicational processes to communicate effectively (Skehan 2003). Additionally, the meaning-focused feature of real-world tasks activate learners’ prior knowledge (e.g. daily life situations) that they can develop in the second language (Long 1985; Richards, Webber and Platt 1985; Nunan 1989, 2004; Bygate, Skehan and Swain 2001; Willis and Willis 2001; Ellis 2003; Larsen-Freeman and Anderson 2011). In this sense, the meaning focus of TBLT allows students to use their language freely to achieve communicative goals, and their performance is evaluated accordingly. As Willis and Willis (2001: 5) explain, ‘the success of the procedure is judged on whether or not learners communicate successfully’. In TBLT students also have the opportunity to reflect, thus becoming aware of the language, as a means of making choices about which grammatical structures to use according to a particular language context. Bearing this in mind, TBLT can be linked to psycholinguistic, cognitive and sociocultural theories of language learning. Regarding psycholinguistic perspectives, Long (1985) focuses on interactions during negotiation of meaning processes when students encounter difficulties in language and solve them through interactive cues (e.g. clarification requests, feedback etc.). In relation to cognitive views, Skehan (2003) bases his work on how tasks can support language complexity, fluency and the correct use of linguistic forms. From the sociocultural perspective (Vygostsky 1978; Lantolf 1994), the ‘scaffolding’ aspect of discussions during task performance is perceived to increase language learning, as students use and share their knowledge collaboratively to complete the tasks. TBLT is the main pedagogical foundation for the EDK as the tasks in the foreign language (i.e. the action of cooking in this case) provide real-world experiential learning, which prepares students for similar interactive situations in the future. In the EDK, TBLT not only promotes SLA by cognitive views of individual learning (Swan 2005), but also a complete language experience that becomes relevant for L2 learning. Table 2.1 shows the links between TBLT and the EDK. Foster and Skehan (1999) suggest that the effective use of TBLT for L2 learning has to do with how tasks are designed, organized and implemented. To design meaningful tasks, therefore, language educators should inform their pedagogical choices on (1) how well tasks promote communication (2) whether they will focus on form, meaning or both (3)  task coherence (4)  amount of

24

Sandra Morales

Table 2.1 TBLT in the EDK TBLT

The European Digital Kitchen

Promotes autonomous learning

Students work independently (i.e. the teacher is not directly involved in the cooking session) Students cook authentic dishes (i.e. using recipes in different languages) Students apply their knowledge and experiences in cooking Students discuss the language and work in collaboration to complete the task Students negotiate feedback together. Also, they are provided with help options. The main goal of the EDK is to promote language learning in an authentic context.

Uses real-world related tasks Fosters the use of prior/shared knowledge Reflection is usually involved There is usually some mechanism of negotiation and feedback There is a task process and outcome

comprehensible input and (5)  the opportunities tasks provide for students to notice and become aware of language structures (Skehan 1998; Ellis 2003; Nunan 2004). Willis and Willis (2001: 157) provide seven parameters that can be considered and adapted when designing TBLT session(s): (1) outcome (open or closed) (2) starting points for task (a text) (3) pre-task preparation (4) control of agenda and task structure (5) interaction patters and participant roles (6) pressure on language production and (7) post-task activities. In relation to the methodological principles of TBLT, a task-mediated lesson should consider three stages. In the pre-task stage, the task topic is presented to students and their prior knowledge is activated. Ellis (2003: 245) states that in order to prepare students for the main task, they could (1) conduct a similar task to the main task (2) be presented with a model on how to conduct a task (3)  develop activities that are not related to the task to elicit their knowledge and (4) plan how to conduct the main task. These strategies can help activate the necessary cognitive and linguistic knowledge students need to complete the main task effectively (Skehan 1998; Ellis 2003). The ‘during task’ stage involves the type of task students are going to perform. In this sense, teachers should decide whether the task is implemented using ‘task performance options’ or ‘process options’ (Ellis 2003: 249). The main difference between these alternatives is the degree of ‘freedom’ students have to conduct the task. For example, if the TBLT lesson is planned based on task performance

Locating the EDK in Its Research Context

25

options, students have time and/or input restrictions (i.e. being given specific information to do the task). On the other hand, if process options are implemented, decisions are made when the task is in progress. For instance, students use their experiences and prior knowledge to finish the task as the teacher facilitates interaction, negotiation and collaboration through pairing or group work. The post-task stage, Ellis (2003) explains, fosters noticing and awareness of language structures. In this follow-up phase, students are able to (1) repeat the task (2) explain how they conducted the task (3) practice linguistic forms and (4)  receive explicit feedback. The decisions on what techniques to use in the post-task will depend on the focus of the main task and objectives to support the experiential learning students went through while doing the task. In this sense, the post-task helps confirm grammatical competence and appropriate language use. Like CLT, the use of TBLT presents some challenges. For instance, the lack of a standard definition of task might lead to a misunderstanding regarding both their use and usefulness as the design and implementation of a task-driven lesson will depend on how tasks are perceived. In addition, although tasks have been widely recognized for their benefits to increase interaction as they recreate ‘natural communication situations’, issues regarding how effective a TBLToriented syllabus can be in the long term have been discussed (Richards and Rodgers 2001; Seedhouse 2005; Hall 2011). As Cook (2008: 260) states, ‘Taskbased learning concentrates on what can work in the classroom. Its expressed goal is short-term fluency. It does not appear concerned with overall teaching goals, which are hardly ever mentioned.’ It has also been suggested that TBLT could not work properly in cultures where, for example, learning is predominantly teacher-centred or the syllabus is test-oriented (Cook 2008; Hall 2011; Larsen-Freeman and Anderson 2011; Willis and Willis 2001). Due to the guiding role of the teacher, students might feel they are not receiving enough corrective feedback or even feel uncomfortable by having to increase their talk time during the lesson. Also, if students needed to prepare themselves for an evaluation, including tasks that are not related to this objective might not meet this purpose. As discussed earlier, TBLT can pose some difficulties – like most of language teaching approaches – however, it provides a valuable student-centred framework which allows the flexible design of meaningful tasks to develop language skills. Due to this flexibility, TBLT can be combined with other L2 teaching approaches and learning resources to tackle some of the issues explained previously as a means to improve SLA. For example, in his discussion about the link between tasks and technology Skehan states:

26

Sandra Morales ‘What is really exciting about the use of technology is its potential as a source of language learning materials and input. And in that respect, the major change in the last 5–10 years has been the emergence of the web as a colossal languagematerials resource. On occasions, this may consist of resources which have been put together specifically for language learners. But the vast majority of the materials exist for other purposes, and are simply there with potential to be exploited.’ (2003: 403)

Thus, digital resources can be used to support TBLT. Technological tools and materials are part of the real-world, and technology-mediated tasks can be designed and implemented to promote authentic language education. The development of the EDK therefore includes innovative multimodal technology (e.g. sensors, audio-visual materials, feedback) which facilitate task completion and SLA.

Technology uses in the digital age As stated earlier, technology plays an essential role in today’s globalized world. It is used to facilitate everyday activities (e.g. online shopping), to communicate in real time (e.g. chat), delayed time (e.g. email) and to build up personal and professional relationships (e.g. social networks). Considering this notion of technology as ‘ambient’ or ‘ubiquitous’, the field of HCI has widely contributed to designing, implementing and evaluating digital resources (e.g. software, computational systems, mobile devices) for the benefit of technology users. Kim (2015:  1)  states that ‘aside from merely making the necessary computational functionalities available, the early focus of HCI has been in how to design interaction and implement interfaces for high usability. The term high usability means that the resulting interfaces are easy to use, efficient for the task, ensure safety, and lead to a correct completion of the task’. For instance, ambient technology has been used to improve people’s quality of life and to promote autonomy and social collaboration. One example of this is the Ambient Kitchen (Olivier et al. 2009), a situated design developed using ubiquitous computing (i.e. sensory technology) to provide cognitive support to people with dementia. Olivier et al. suggest that kitchens in homes provide an appropriate environment to exploit pervasive technologies; thus the aim of their design was to implement a computer system to help these individuals to be more independent and transform their lives. Another example is the ‘Kitchen stories’ project (Terrenghi, Hilliges and Butz 2007), where researchers brought technology

Locating the EDK in Its Research Context

27

to the kitchen to foster communication. The project sought to engage a group of people with different backgrounds in live cooking sessions which could be recorded and shared through a tablet PC. The findings show that participants found the experience motivating, in particular, when the cooking was done in pairs. However, there were factors, such as the level of difficulty of the recipes, the participants’ prior cooking knowledge and technological skills to manage the tablet, that need to be further researched. In terms of education, and language learning and teaching particularly, computer-mediated communication (i.e. CMC, human interaction that is supported by digital tools, according to Murray 2000)  provides communication resources (i.e. virtual learning environments, blogs, discussion forums, chatrooms) which can be beneficial and motivating for SLA. In this sense, CALL has played a significant role in L2 education.

The role of technology in language teaching and learning Technological resources have been considered enablers for innovation and interaction in language teaching and learning. Since the advent of the digital era, computers and CMC tools have been included in the language classroom as a means to support teachers and students. However, as technology develops at a rapid pace, its uses in language education change as well. For example, Warschauer (2002: 453) explains that ‘the role of the computer in education has gradually been transformed from that of a tutor to that of tool’. That is, the computer is no longer a provider of information but a device with multiple resources used to get access to and create new knowledge. Therefore, in order to make use of technology effectively, educators and learners need to develop digital skills (Warschauer 2003). Also, the use of technology in SLA has not only been supported by technological developments but also by the evolution of approaches for language teaching and learning. For example, computers have been included in language classrooms in different ways according to language learning theories and methodologies. Initially, computers were used to promote SLA through repetition and explicit grammatical exercises (e.g. Warschauer’s ‘tutor’ role), techniques strongly linked to behaviourist learning. Later on, with the arrival of multimedia tools (e.g. sound, images), computers were used in combination with CLT principles as a means to encourage learners to communicate (e.g. Warschauer’s ‘tool’ role). As a result, language learning and teaching became more interactive

28

Sandra Morales

and multimodal (e.g. use of subtitles), thus communication and negotiation in the target language was highly improved. Additionally, as technology becomes more social, CMC resources are used with other language teaching approaches, such as TBLT, to promote authentic communication and cooperative learning to foster collaborative work. Moreover, technology nowadays allows users to have access to real communicative situations in the target language. This not only facilitates language learning but also the presentation of necessary cultural and pragmatic aspects of the language. Such features are important to understand how to use the language in different contexts. Consequently, the EDK incorporates technology interactively by providing students with lexical and grammatical input and feedback and by implementing real-world materials and tasks as catalysts for interactive communication.

Language teachers and learners’ role This technological evolution has also had an influence on the profile of students and teachers. Nowadays, technology is embedded in learners’ academic and personal lives. Today, technology is used inside and outside the language classroom, not only to learn and construct knowledge, but also helps learners to communicate, create their own identity and relate with their peers. These learners have been denominated ‘digital natives’ as they ‘have spent their entire lives surrounded by and using videogames, digital music players, video cams, cell phones, and all the other toys and tools of the digital age’ (Prensky 2001:  1). Prensky also indicates that ‘digital natives are used to receiving information really fast. They like to parallel process and multi-task’ (Prensky 2001: 3), therefore, digital natives have become more technologically active and demanding to meet their learning and personal needs. This label, however, has been cause for debate. Bennett, Maton and Kervin (2008) question the digital native concept, as they suggest not all learners have the same learning opportunities and access to technology. Warschauer (2003) also agrees that there is a technological gap (i.e. he calls it ‘digital divide’) in some parts of the world, particularly regarding the contact students have with technology and whether and how it is used for learning purposes. Thus, this disparity may surely affect the students’ educational context. Furthermore, the fact that learners are born surrounded by technology does not necessarily mean that they actually know how to use it for academic purposes. Despite this ongoing debate, it is clear that students nowadays are more exposed to

Locating the EDK in Its Research Context

29

technology than in the past. As a result, Prensky (2001) also suggests that language educators should acknowledge these characteristics and be prepared to engage students with this new profile. In L2 education, teachers, on the one hand, are aware of the students’ digital role, the benefits that technology brings to their language courses and the large variety of resources available for language teaching (Egbert, Paulus and Nakamichi 2002; Hubbard and Levy 2006). On the other hand, however, they might find it difficult to apply technological resources with language learners without the appropriate digital competence (Hampel and Stickler 2005; Hubbard and Levy 2006; Compton 2009). Also, they have to be prepared to promote their students’ computer literacy so that they can take better advantage of technology to learn a new language (Hubbard 2004). Therefore, teacher education in technology (i.e. either as part of a continuing professional development course or self-taught experience), becomes essential for the sustainability of digitally mediated language teaching and learning. Thus, it is possible to say that language instruction in general has been transformed with the incorporation of technology in education. For instance, nowadays with the prolific and flexible use of mobile resources, language learning and teaching can be taken outside the limits of the classroom (Kukulska-Hulme and Shield 2008). Incorporation of mobile tools into L2 learning might help reduce not only the sometimes restrictive utilization of technological resources in educational contexts, but also the opportunities students have to practice the language. With mobile technology, for example, informal learning (e.g. gaming) can also be implemented to promote student-centred learning, language skills, learner autonomy and motivation. The EDK design, therefore, uses mobile technology resources (e.g. tablet and sensors) and principles (i.e. learning beyond the classroom) as a means to support and improve students’ L2 learning in an authentic task-based environment.

Computer-Assisted Language Learning (CALL) According to Levy (1997: 1), CALL is defined as ‘the search for and study of applications of the computer in language teaching and learning’. Currently, however, as technology rapidly evolves, the term CALL has expanded and includes a wider range of resources (e.g. internet, mobile devices), which can be used for language instruction (Kern 2006; Chapelle 2010). Garret (2009: 719) states that CALL is the ‘full integration of technology into language learning’ and explains

30

Sandra Morales

that the proper use of CALL not only has to do with the sole application of technology but also with the informed decisions teachers make to promote language teaching. In this sense, the use of CALL has to be supported by pedagogical goals and theoretical perspectives to foster an effective use of technological resources in the language classroom. Also, to ensure the efficiency of CALL activities, their research and evaluation are essential for the progress and proper implementation of technology for language teaching. In general, CALL research has been informed by interactionist theories of SLA (Chapelle 2001); however, with the advent of new technologies (i.e. online learning, social networks) and the role they play in communication, social-constructivist (Dewey 1933) and sociocultural (Vygostsky 1978) approaches have also served as foundation for CALL studies. CALL’s flexible methodological framework (Hubbard 1996) provides the appropriate pedagogical and technological principles to promote SLA, as they are strongly linked to the communicative language teaching (Richards and Rodgers 2001). Hubbard’s framework includes three base modules (1) development (2) implementation and (3) evaluation which can be adapted according to the syllabus, teaching objectives and learners’ needs. In the CALL framework, technology, learners and teachers, who can act as developers and researchers, play an equally significant role. According to Hubbard (1996), the development module includes design of CALL applications (e.g. software) or tasks taking into consideration teaching objectives, learner profile, the skill(s) to develop and the language structures to use. In addition, the design has to be consistent with the language teaching approach or theory (e.g. TBLT, interaction theory) to use with CALL. In terms of the procedure, this has to do with the type of activity designed and how it will be presented (e.g. with/without feedback options). The implementation module deals with planning. Here, teachers have to consider the context where the CALL application or task will be applied, for instance, the access to technological resources the students will have, how they will use the application and whether and how the teacher is involved. Careful attention should be paid to the implementation module as principles and guidelines here must be clear to be effective (Hubbard 1996). In the evaluation module, the effectiveness of a CALL application or task is tested. Therefore, the design, approach and procedure are observed to determine the success of a CALL application or task. To do this, surveys, checklists and peer evaluation can be applied (Hubbard 1996). Also, Chapelle suggests (2001) a set of criteria based on the interactionist theory for SLA for CALL evaluation.

Locating the EDK in Its Research Context

Development

31

Evaluation

Technology Language Teaching Approaches

Trials Users’ Experiences

Implementation

Authentic Materials Interaction Feedback

Figure 2.1 Development of the EDK

In figure 2.1,1 the modules of the CALL framework are presented in relation to the EDK’s principles. For instance, in the development phase, ambient technology and software have been produced and TBLT has served as the main teaching approach to support the design. In terms of the implementation, authentic materials (e.g. recipes) with high cultural background have been produced to promote interaction and collaboration. Help options and feedback have been introduced as well. The EDK’s design has been greatly evaluated as we present later in this book through cooking sessions with language learners from different contexts. Garret (2009: 722) suggests that CALL should be categorized into (1) tutorial (2) authentic materials engagement and (3) communication. The first category has to do with the initial, and mainly traditional, use of CALL to practice grammar, pronunciation, reading and writing through iterative drills. In tutorial CALL, the computer acts as a ‘teacher’ as it can also provide feedback and help options. Corrective feedback is produced by the computer system to scaffold the learners’ language learning performance. Feedback strategies (Lyster and Ranta 1997) and the implementation of computer-generated feedback to promote language skills have been researched in CALL (Pujola 2001; Heift 2004; Sauro 2009; Lavolette, Polio and Kahng 2015) and are considered effective for language learning. Regarding help options, Cardenas-Claros and Gruba (2009: 69) define

32

Sandra Morales

them as ‘embedded application resources that assist learners in performing computing operations and/or support language learning’. They state that these can be, for example, annotations, sounds or images which can promote different language skills when learners encounter difficulties conducting CALL tasks. The second category uses the computer as a tool and incorporates authentic materials (e.g. newspaper articles, TV shows) in the language classroom. Garret (2009: 272) states that they ‘characterize materials created by and for native speakers, in contrast to those created for pedagogical purposes’. The use of authentic materials for language learning can be tremendously beneficial if they are used with a pedagogical purpose. If they are applied just for the sake of using genuine communicative messages in the target language, their effectiveness may not be guaranteed. Today, the internet provides easy access to authentic resources; however, they should be chosen and implemented properly to foster SLA. In this sense, this category is related to the first one because in order to make authentic materials useful for learners, features from tutorial CALL (i.e. annotations for lexical items) might be incorporated in CALL activities to facilitate learning. In the third category – communication CALL – technological resources from CMC are widely used to promote SLA through communicative experiences. Here, elements from social networks and mobile tools can be applied for L2 learning and teaching not only to develop language skills, but also to present the cultural aspects of the language. For example, online learning (i.e. purposeful education with technological resources) can help build communities of practice for students and teachers around the globe. This can be of great help to support language learning outside established instruction time in schools. In addition, telecollaboration applies technological resources to connect students beyond the boundaries of the language classroom as a means to promote language and cultural learning (Hauck and Youngs 2008; O’Dowd 2013). As seen, the use of communication CALL can be diverse and is not restricted to specific physical places such as the computer laboratory, school or university. This CALL taxonomy is helpful to know which technological resources to use and how to use them accordingly. The discussion of which of these categories should be prioritized when using CALL is debatable as they are not mutually exclusive. Nowadays, it is possible to take elements from each category to improve language teaching with CALL. In this sense, the EDK applies the concept of a computer as a tutor and a tool, as the design incorporates learning resources (e.g. real L2 recipes as authentic materials) and options (e.g. feedback) provided by the software to promote collaboration and communication.

Locating the EDK in Its Research Context

33

TBLT and technology for second language learning and teaching As discussed earlier, the CALL framework can be adapted for teaching contexts and learners’ needs considering the different language-teaching approaches available for SLA. In this regard, the use of TBLT in CALL has been regarded positively, mainly due to the fact that authentic tasks can be enhanced by technological resources for a more genuine learning experience. To increase the effectiveness of tasks in CALL, Doughty and Long (2003) suggest a set of principles and procedures teachers may consider when designing technologymediated tasks. The authors focus particularly on online tasks and explain that these principles are not only based on TBLT criteria but also on general theories of learning. Doughty and Long (2003: 52) state that regarding activities, tasks should be implemented as ‘units of analysis’ where learners experience language instead of only receiving information on how to use it, as may happen, for example, with ready-made materials. Also, this promotes the principle of ‘learning by doing’, which is related to experiential learning and how learners use their own language resources to complete a real-world task. In the EDK, users apply their prior knowledge (i.e. the ‘cooking’ task) as they encounter new linguistic, cultural and communicational features (i.e. recipes in different languages), which promote SLA. Regarding input, Doughty and Long suggest that tasks should provide great amounts of inputs and that, when possible, input from authentic materials should be adapted to meet the learners’ needs to promote learning. Learning processes with TBLT and CALL should foster implicit instructions (the authors call it ‘chunk’ learning), so learners include complex language structures in their discourse while conducting the task. The idea is that students manage the challenging language first and then become aware of its linguistic norms. TBLT criteria suggest that tasks should be meaning-focused (Ellis 2003); however, Doughty and Long indicate that this might not be sufficient to develop language accuracy. Therefore they propose that a combination of meaning-focused and form-focused (i.e. grammatical features) tasks should be used to enhance language learning. Additionally, they state that providing negative feedback is beneficial for SLA in technology-mediated tasks. The problem with feedback in technological learning contexts is that in asynchronous CMC (i.e. blog, email), as interaction does not take place in real time, learners might not receive immediate feedback, which could in turn undermine the learning process. As part of

34

Sandra Morales

its framework, the EDK provides learners not only with immediate feedback (e.g. positive/negative), but also with options such as captions and audio-visual materials which will support the users’ L2 acquisition. Doughty and Long argue that the students’ internal learning processes (i.e. interlanguage) should be paid attention to in SLA. They indicate that TBLT promotes the students processes by ‘providing input that is at least roughly tuned to learners’ current processing capacity by virtue of having been negotiated by them during collaborative work on pedagogic tasks’ (2003:  66). Therefore, they strongly recommend collaborative and cooperative work in technologymediated tasks. Particularly, as such tasks can be completed more productively in collaboration. This is relevant especially nowadays with the ubiquitous nature of technology and the proliferation of mobile devices and social networks. Finally the authors state that the learners’ individual differences should be considered in TBLT with technology to increase SLA. In the EDK, the use of genuine cooking tasks in collaboration increase L2 learners’ interaction so as to foster the use of their prior language and the input provided in the multilingual recipes. Thus, teachers should take advantage of technological resources, as they can cater for factors such as learning styles, lesson purpose and motivation, which might have an effect on the students’ learning processes. Gonzalez-Lloret (2003) implemented Doughty and Long’s TBLT principles in the design of a web-based CALL tool for L2 Spanish learners. To evaluate the application, the author observed and analysed the interaction and negotiation instances among twelve English- speaking learners of Spanish. Findings show that the tool fostered negotiation of meaning and engaged the students in completing the task successfully. Language researchers have studied the use of TBLT and technology and have made useful contributions to an increasingly growing area of study (Salaberry 1996; Collentine 2009; Hampel 2010; Sarre 2013; Ferreira et al. 2014). In 2010, Thomas and Reinders edited a selection of papers which dealt with a variety of relevant aspects of TBLT and technological resources. For example, they presented investigations regarding the design and application of tasks in technology-mediated learning environments, such as telecollaboration and virtual worlds, and how teachers can implement tasks with technological resources for L2 lessons. More recently, Gonzalez-Lloret and Ortega (2014) discuss TBLT mainly in online or blended learning contexts with CMC resources. The epistemological tendency of TBLT and CALL research has been based on interactionist theories for SLA and, more recently, on sociocultural perspectives of language teaching and learning. However, the implementation of TBLT in CALL-mediated and face-to-face environments has been under-researched.

Locating the EDK in Its Research Context

35

Therefore, there is a clear need to provide evidence of the effectiveness of the application of tasks in such learning contexts. Bearing this in mind, the EDK fulfils a current gap in the task-based instruction scene in CALL. The design principles of the EDK take advantage of the pervasive presence of technology to combine multiple technological resources with real-world materials and tasks to support interactive language learning.

Multimodal CALL As technology evolves and becomes natural in people’s lives, new resources that can be applied for language teaching also appear. These multimedia affordances of the technological environment allow CALL practitioners to choose from a variety of tools and also combine them to improve SLA. In this sense, language learning is no longer supported by a single knowledge transmission channel (e.g. text) but by many other media such as images, symbols and/or sound. These different ways of communicating messages, or ‘modes’, can be defined as the ‘semiotic resources which allow the simultaneous realisation of discourses and types of (inter) action’ (Kress and Van Leuween 2001: 21). Yet multimodality is not only created with technological resources (as in non-virtual communication, for instance) but is also supported by body language and gestures. Lamy and Hampel (2007: 37) suggest that nowadays ‘we communicate using a complex system made up of written, spoken and visual resources, each with its own modes and affordances’, so this reality can create an appropriate environment for multimodal language learning. Here, learners are able to simultaneously use different tools from synchronous and asynchronous CMC (e.g. blog post with embedded video clip) to develop linguistic and communicative competence. For example, Guichon and McLornan (2008) studied the use of subtitles in video recordings for second language comprehension. They conducted an investigation with forty graduate students with intermediate level of English proficiency. The participants were divided into four groups, where group 1 used sound only; group 2, sound and image; group 3, sound, image and subtitles in the learners’ native language and group 4, sound, image and subtitles in the L2. The authors used the learners’ written texts (i.e. notes and an essay) to analyse the data, considering lexical items related to the contents of the video reported by the students. Issues regarding the usefulness of images, sounds and the influence of L1 were explored. Findings show that the use of visuals (e.g. subtitles and images) in the L2 was beneficial to foster the students’ receptive language skills.

36

Sandra Morales

Nevertheless, Guichon and McLornan (2008) state that when designing multimodal resources for language learning, attention should be paid to the amount of input students will be exposed to, the level of difficulty and length of the materials. These could have an impact on the learners’ cognitive load, which might, in turn, undermine the process of learning. Hampel and Hauck (2006) also advocate the use of multimodality in CALL. Their argument is based on the fact that the incorporation of different modes of information that technology provides for language learning will promote more meaningful instruction. They highlight the importance of using multimodal CALL in virtual environments (e.g. online learning) so the input the students receive through multimedia resources is a replica of the traditional face-to-face classroom. Stockwell (2010: 100) discusses the role of multimodality in CALL tasks and states that ‘the use of multiple modes of CMC with language-learning tasks has the potential to develop different aspects of their second language (L2)’. Stockwell explains that this supports multimodal CALL, particularly considering the influence it can have on L2 learners’ interactional dynamic to improve their language proficiency. Multimodal communication becomes relevant in the current digital era especially due to today’s learner’s profile. From early days, students are constantly exposed to advanced technology for everyday activities. They use it to communicate, learn and build relationships and identities among themselves. For example, video gaming today has become a powerful industry as it increasingly develops high-tech systems that can (re)create images and gestures and movements in the online world (e.g. virtual reality, Wii technology). These advanced tools have also been used for language learning. For instance, Rama et al. (2012) examined the benefits of the online game ‘World of Warcraft’ for Spanish language learners. They analysed students’ interactions during game sessions and concluded that game-based learning fosters communication, collaboration and promotes multimodal environments for SLA. Also, elements from 3D worlds (e.g. Second Life) have been applied with language learners not only to foster language learning but also as an access to cultural information. Multimodality in CALL can be highly beneficial as seen earlier; however, it also presents challenges that need attention. Hampel and Hauck (2006) explain that learners’ cognitive load might increase if they are exposed to too many modes, causing confusion and difficulty to remain on task. Students’ technological skills can also play a role in multimodal tasks, as it has been shown that learners switch modes to complete tasks (see Guichon and McLornan 2008).

Locating the EDK in Its Research Context

37

Furthermore, if they do not have the abilities or the training to deal with multimodal environment, the learning experience can be more daunting rather than motivating. Bearing this discussion in mind, the EDK is a revolutionary design for multimodality in CALL. The EDK not only includes a variety of modes to promote SLA (e.g. subtitles, sound, image) but also tackles the challenges that virtual contexts and CMC resources pose by incorporating face-to-face collaboration among learners. This allows them, for example, to solve both technological and language issues that may rise during a cooking session. Tasks and interactions in the EDK also promote experiential learning by activating users’ senses such as touch or smell when in contact with ingredients and utensils. This certainly provides rich soil for research. As seen previously, most multimodal CALL investigations have been conducted mainly in online environments, thus the study of the EDK aims to shed light on mixed-context multimodal designs. This will certainly contribute to the sustainability of pedagogical innovations in language teaching and learning with technology.

Research context of the European Digital Kitchen As seen throughout this chapter, the use of communicative language approaches, HCI resources and CALL in the language classroom play a significant role in promoting innovative ways of teaching and learning languages. Technology allows teachers to create knowledge, use authentic materials and multimodal resources as a means to foster SLA. It also promotes independent learning, interactivity, communication and cultural awareness among learners. Furthermore, technology-mediated language education provides teachers with strategies and resources to improve teaching and also tools to follow up students’ performance. For example, many of the computer systems available for teaching and learning with technology can register learners’ actions and provide data regarding track records, which can help improve language development. In addition, this data can be used to conduct research and support language developers in the enhancement or creation of new effective technology-mediated tools for SLA. Nevertheless, as technological tools rapidly evolve, evidence of their efficiency on language learning is essential to provide teachers and learners not only with beneficial but also sustainable resources and strategies to maximize technologymediated language learning. Taking this reality into account, the design, evaluation and implementation of the EDK aimed to seek for solutions to language

38

Sandra Morales

learning with technology (i.e. interaction in real-world tasks) and provide sustainable solutions for the benefit of SLA. To do so, the EDK uses tasks as the main tool for communication in a collaborative CALL environment and takes language learning outside the classroom to a genuine communicative situation. This can further support instruction that takes place inside the L2 classroom by increasing the opportunities learners have to use the language, particularly in collaboration and through experience. Furthermore, the flexible design of the EDK (later in this book) allows language teachers to adapt and create their own tasks according to their students’ needs and proficiency levels. In addition, cultural language input included in the design makes the learning experience more authentic by providing the learners with background information, which will help them improve their use of language within a specific context. The computational system of the EDK further supports SLA by incorporating multimodal communication (i.e. video, sound) and help options. These give students the opportunity to control their learning and facilitate communicative interaction which, in turn, can support language learning in the long term. For these reasons, the EDK provides sustainable and innovative properties which help to enhance language teaching and learning with technological resources. Additionally, the multimodality of the EDK in terms of language learning methods (i.e. TBLT), technology (HCI, CALL) and the collaborative application of real-world materials that activate senses (e.g. taste) make it a novel and unique object of learning and research. Therefore, this book presents a collection of studies that shows the sustainable use, effectiveness and challenges of designing, implementing and evaluating the EDK. Following chapters will discuss its inception, development and application with language teachers and learners across five European countries in seven different languages.

Note All figures and tables were produced by the Newcastle University team.

References Bennett, S., K. Maton and L. Kervin (2008), ‘The “digital natives” debate: A critical review of the evidence’, British Journal of Educational Technology, 39 (5): 775–786.

Locating the EDK in Its Research Context

39

Bygate, M., P. Skehan and M. Swain (eds) (2001), Researching Pedagogic Tasks, Second Language Learning, Teaching and Testing. Harlow : Longman. Chapelle, C. (2010), ‘The spread of Computer Assisted Language Learning’, Language Teaching, 43 (1): 66–74. Chapelle, C. (2001), Computer Applications in Second Language Acquisition, Cambridge: Cambridge University Press. Cárdenas-Claros, M. S., and P. Gruba (2009), ‘Help options in CALL: A systematic review’, CALICO Journal, 27 (1): 69–90. Collentine, K. (2009), ‘Learner use of holistic language units in multimodal, task-based synchronous computer-mediated communication’, Language Learning & Technology, 13 (2): 68–87. Compton, L. K. L. (2009), ‘Preparing language teachers to teach language online: a look at skills, roles and responsibilities’, Computer Assisted Language Learning, 22 (1): 73–99. Cook, V. (2008), Second Language Learning and Language Teaching, London: Hodder Education. Cutrim Schmid, E., and S. Whyte (eds) (2014), Teaching Languages with Technology: Communicative Approaches to Interactive Whiteboard Use. A Resource Book for Teacher Development. Advances in Digital Language Learning and Teaching (Series Editors: Michael Thomas, Mark Warschauer & Mark Peterson), London: Bloomsbury. Dewey, J. (1933). How We Think: A Restatement of Reflective Thinking to the Educative Process, Boston: D. C. Heath (Original work published in 1910). Doughty, C., and M. Long (2003), ‘Optimal psycholinguistic environments for distance foreign language learning’, Language Learning & Technology, 7: 50–80. Egbert, J., T. Paulus and Y. Nakamichi (2002), ‘The impact of CALL instruction on classroom computer use: A foundation for rethinking technology in teacher education’, Language Learning & Technology, 6 (3): 108–126. Ellis, R. (2003), Task-Based Learning and Teaching, Oxford: Oxford University Press. Ferreira, A. F., J. G. Salinas and S. Morales (2014), ‘Using a task-based approach for supporting a blended learning model for English as a foreign language’, International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 4 (1): 44–62. Foster, P., and P. Skehan (1999), ‘The influence of source of planning and focus of planning on task-based performance’, Language Teaching Research, 3 (3): 215–247. Garrett, N. (2009), ‘Computer‐assisted language learning trends and issues revisited: Integrating innovation’, The Modern Language Journal, 93 (s1): 719–740. Gonzalez-Lloret, M. (2003), ‘Designing task-based CALL to promote interaction: En busca de esmeraldas’, Language Learning & Technology, 7 (1): 86–104. González-Lloret, M., and L. Ortega (eds) (2014), Technology-Mediated TBLT: Researching Technology and Tasks (vol. 6), Amsterdam/Philadelphia: John Benjamins.

40

Sandra Morales

Guichon, N., and S. McLornan (2008), ‘The effects of multimodality on L2 learners: Implications for CALL resource design’, System, 36 (1): 85–93. Hall, G. (2011), Exploring English Language Teaching: Language in Action (Routledge Introductions to Applied Linguistics). Oxon: Routledge. Hampel, R. (2010), ‘Task design for a virtual learning environment in a distance language course’, in M. Thomas and H. Reinders (eds), Task-Based Language Learning and Teaching with Technology, 131–153, London: Continuum. Hampel, R., and M. Hauck (2006), ‘Computer-mediated language learning: Making meaning in multimodal virtual learning spaces’, The JALT CALL Journal, 2 (2): 3–18. Hampel, R., and U. Stickler (2005), ‘New skills for new classrooms: Training tutors to teach languages online’, Computer Assisted Language Learning, 18 (4): 311–326. Hattie, J., and H. Timperley (2007), ‘The power of feedback’, Review of Educational Research, 77 (1): 81–112. Hauck, M., and B. L. Youngs (2008), ‘Telecollaboration in multimodal environments: The impact on task design and learner interaction’, Computer Assisted Language Learning, 21 (2): 87–124. Heift, T. (2004), ‘Corrective feedback and learner uptake in CALL’, ReCALL, 16 (2): 416–431. Hooper, C. J., A. Preston, M. Balaam, P. Seedhouse, D. Jackson, C. Pham, C. Ladha, K. Ladha, T. Ploetz and P. Olivier (2012), ‘The French Kitchen: Task-Based Learning in an Instrumented Kitchen’. Paper presented at the Ubiquitous Computing 2012 Conference, September 2012, Pittsburgh, USA. Hubbard, P. (1996), ‘A methodological framework for call courseware development’, in M. C. Pennington (ed.), The Power of CALL, 15–32, Houston: Athlestan Publications. Hubbard, P. (2004), ‘Learner training for effective use of CALL’, in S. Fotos and C. Browne (eds), New Perspectives on CALL for Second Language Classrooms, 45–88, Mahwah, NJ: Lawrence Erlbaum. Hubbard, P., and M. Levy (eds) (2006), Teacher Education in CALL. Amsterdam: Philadelphia, PA: John Benjamins. Hymes, D. H. (1972), ‘On communicative competence’, in J. B. Pride and J. Holmes (eds), Sociolinguistics. Selected Readings, 269–293, Harmondsworth: Penguin. Kern, R. (2006), ‘Perspectives on technology in learning and teaching languages’, TESOL Quarterly, 40 (1): 183–210. Kim, G. (2015), Human Computer Interaction: Fundamentals and Practice. Boca Raton, FL: Taylor and Francis. Kress, G. R., and T. Van Leeuwen (2001), Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold. Kukulska-Hulme, A., and L. Shield (2008), ‘An overview of mobile assisted language learning: From content delivery to supported collaboration and interaction’, ReCALL, 20 (3): 271–289.

Locating the EDK in Its Research Context

41

Lamy, M. N., and R. Hampel (2007), Online Communication in Language Teaching and Learning. New York: Palgrave Macmillan. Lantolf, J. P. (1994), ‘Sociocultural theory and second language learning: Introduction to the special issue’, The Modern Language Journal, 78 (4): 418–420. Larsen-Freeman, D., and M. Anderson (2011), Techniques and Principles in Language Teaching: Practical, Step-By-Step Guidance for ESL Teachers, and ThoughtProvoking Questions to Stimulate Further Exploration, 3rd edn, Oxford: Oxford University Press. Lavolette, E., C. Polio and J. Kahng (2015), ‘The accuracy of computer-assisted feedback and students’ responses to it’, Language, Learning & Technology, 19 (2): 50–68. Levy, M. (1997), Computer-Assisted Language Learning: Context and Conceptualization, Oxford: Clarendon Press. Long, M. H. (1985), ‘A role for instruction in second language acquisition: Taskbased language teaching’, in K. Hyltenstam and M. Pienemann (eds), Modelling and Assessing Second Language Acquisition, 77–99, Clevedon, Avon: Multilingual Matters. Long, M. H. (1998), ‘Focus on form: Theory, research and practice’, in C. J. Doughty and J. Williams (eds), Focus on Form in Second Language Acquisition, 15–41, Cambridge: Cambridge University Press. Reprinted in L. Ortega (ed.), Second Language Acquisition: Critical Concepts in Linguistics. London: Routledge. Lyster, R., and L, Ranta (1997), ‘Corrective feedback and learner uptake’, Studies in Second Language Acquisition, 19 (1):37–66 Nunan, D. (1987), ‘Communicative language teaching: Making it work’, ELT Journal, 41 (2): 136–145. Nunan, D. (1989), Designing Tasks for the Communicative Classroom, Cambridge: Cambridge University Press. Nunan, D. (2004), Task-Based Language Teaching. Cambridge: Cambridge University Press. O’Dowd, R. (2013), ‘The competences of the telecollaborative teacher’, The Language Learning Journal, 43 (2): 194–207. Olivier, P., G. Xu, A. Monk and J. Hoey (2009), ‘Ambient kitchen: Designing situated services using a high fidelity prototyping environment’, Workshop on affect and behaviour related assistance in the support of the elderly, The 2nd International Conference on PErvasive Technologies Related to Assistive Environments, University of Texas at Arlington, 2009. Prensky, M. (2001), ‘Digital natives, digital immigrants’, On the Horizon, 9 (5): 1–6. Pujolà, J. T. (2001), ‘Did CALL feedback feed back? Researching learners’ use of feedback’, ReCALL, 13 (1): 79–98. Rama, P. S., R. W. Black, E. van Es and M. Warschauer (2012), ‘Affordances for second language learning in World of Warcraft’, ReCALL, 24 (3): 322–338. Richards, J. C., J. Platt and H. Weber (1985), Longman Dictionary of Applied Linguistics. London: Longman.

42

Sandra Morales

Richards, J. C., and T. Rodgers (2001), Approaches and Methods in Language Teaching. Cambridge: Cambridge University Press. Salaberry, M. R. (1996), ‘A theoretical foundation for the development of pedagogical tasks in computer mediated communication’, Calico Journal, 14 (1): 5–34. Sarré, C. (2013), ‘Technology-mediated tasks in English for Specific Purposes (ESP): Design, implementation and learner perception’, International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 3 (2): 1–16. Sauro, S. (2009), ‘Computer-mediated corrective feedback and the development of L2 grammar’, Language Learning & Technology, 13 (1): 96–120. Savignon, S. J. (2007), ‘Beyond communicative language teaching: What’s ahead?’ Journal of Pragmatics, 39 (1): 207–220. Seedhouse, P. (2005), ‘“Task” as research construct’, Language Learning, 55 (3): 533–570. Seedhouse, P., A. Preston, P. Olivier, D. Jackson, P. Heslop, M. Balaam, A. Rafiev and M. Kipling (2014), ‘The European Digital Kitchen project’, Bellaterra Journal of Teaching & Learning Language & Literature, 7 (1): 1–16. Seedhouse, P., A. Preston, P. Olivier, D. Jackson, P. Heslop, T. Plötz, M. Balaam and S. Ali (2013), ‘The French Digital Kitchen: Implementing task-based language teaching beyond the classroom’. International Journal of Computer Assisted Language Learning and Teaching, 3(1): 50–72. Skehan, P. (2003), ‘Task-based instruction’, Language teaching, 36 (1): 1–14. Spada, N. (2007), ‘Communicative language teaching: Current status and future prospects’, in J. Cummins and C. Davison (eds), International Handbook of English Language Teaching, 271–288, New York: Springer. Stockwell, G. (2010), ‘Effects of multimodality in computer-mediated communication tasks’, in M. Thomas and H. Reinders (eds), Task-Based Language Learning and Teaching with Technology, 83–104, London: Continuum. Swan, M. (2005), ‘Legislating by hypothesis: the case of task-based instruction’, Applied Linguistics, 26: 376–401. Terrenghi, L., O. Hilliges and A. Butz (2007), ‘Kitchen stories: Sharing recipes with the Living Cookbook’, Personal and Ubiquitous Computing, 11 (5): 409–14. Thomas, M., and H. Reinders (eds) (2010), Task-Based Language Learning and Teaching with Technology. London and New York: Continuum. Vygostsky, L. S. (1978), Mind in Society: The Development of Higher Mental Processes. Cambridge, MA: Harvard University Press. Warschauer, M. (2002), ‘A developmental perspective on technology in language education’, TESOL Quarterly, 36 (3): 453–475. Warschauer, M. (2003), ‘Dissecting the “digital divide”: A case study in Egypt’, The Information Society, 19 (4): 297–304. Willis, D., and J. Willis (2001), ‘Task-based language learning’, in R. Carter and D. Nunan (eds), The Cambridge Guide to Teaching English to Speakers of other Languages, 173–179, Cambridge: Cambridge University Press.

Part Two

Design

3

The Pedagogical Design of the Digital Kitchen Paul Seedhouse

This chapter explains the pedagogical principles underlying the digital kitchen systems, materials and procedures. The pedagogical design is based on the principles of task-based language teaching (TBLT). Tasks are divided into three phases: pre-task, during-task and post-task, providing a clear design structure for materials, for conduct of sessions and for evaluation of performance. We demonstrate how the phases are implemented in practice by analysing extracts of learners working through the cycle. It is argued that the project realizes some of the advantages of TBLT using digital technology in a real-world setting outside the classroom. The chapter explains how the concepts of TBLT were operationalized in the digital kitchen setting by reference to the French Digital Kitchen (FDK) project, as a prelude to the presentation of the design of the European Digital Kitchen (EDK) in the following chapters.

What is task-based language learning and teaching (TBLT)? The pedagogical design of the FDK (Seedhouse et al. 2013) employs TBLT, a wellestablished approach to language learning which prompts learners to achieve a goal or complete a task (Skehan 1998, 2003). TBLT seeks to develop students’ language through providing a task (such as asking for directions) and then using language to solve it. According to Ellis (2003: 9) the criterial features of a task are that: a task is a workplan; meaning is primary (language use rather than form); a classroom task relates directly to real-world activities; a task can involve any of the four language skills; tasks engage cognitive processes; task completion is a priority and assessment is done in terms of outcomes. Willis (1996: 1) defines the aim of tasks as ‘to create a real purpose for language use and to provide a natural

46

Paul Seedhouse

context for language study’. Van den Branden (2006: 6) suggests that in TBLT, people not only learn languages to make functional use of it, but also by making functional use of it, and proposes that tasks invite learners to act more like language users than learners. Samuda and Bygate (2008: 7) see TBLT as involving holistic activity in that all sub-areas of language are employed to make meaning. They argue that it is in such holistic language work that key language learning processes take place. It is generally assumed (Ellis 2003: 263) that tasks are carried out in pairs or in small groups to maximize interaction and autonomy. There has been a substantial programme of research in relation to TBLT, summarized in Skehan (2003). Ellis (2003:  320)  suggests that ‘there is a clear psycholinguistic rationale (and substantial empirical support) for choosing “task” as the basis for language pedagogy’. Skehan (1998:  95)  argues that transacting tasks ‘will engage naturalistic acquisitional mechanisms, cause the underlying interlanguage to be stretched, and drive development forward’. Long (2015) outlines seven criteria which any approach to language teaching should meet:  (1)  consistency with theory and research findings; (2)  embodying progressive social values; (3) accountability; (4) relevance; (5) avoidance of known problems; (6)  learner-centredness and (7)  functionality. Long concludes that TBLT meets all of these criteria. From the perspective of the FDK project, the major advantages of TBLT as pedagogy were these. There was a natural match with the chosen activity of cooking, which could be easily conceptualized as a task. TBLT has well-developed procedures and principles for task design which could be followed and which blended well with HCI design principles. Johnson (2003:  96)  stresses the importance of an iterative development cycle when designing language learning tasks. He examines the cyclic episodes that task designers actually go through, listing actions such as ‘compare’; ‘evaluate’; ‘reject’; ‘modify’ and ‘review’. This iterative cycle is very much in harmony with the iterative development approach employed in pervasive computing and HCI, and so it proved easy to integrate pedagogical and technological design from this perspective. Tasks form a useful basis for designing research as well as pedagogy. As the FDK and EDK were both research projects, this proved a distinct advantage. A description of a task specifies the type of input learners are expected to receive, if operating within an input-output model of learning. This means that it can be planned to include certain variables in the input but to exclude others. As Ellis (2003: 34) suggests, the relationship between research and pedagogy is strengthened when practitioners of both work with shared constructs. TBLT has so far predominantly been based on tasks to be undertaken within the classroom which simulate real-world tasks. Some innovations in TBLT have

The Pedagogical Design of the Digital Kitchen

47

combined language learning with other, non-linguistic skills in a way similar to this project. Paterson and Willis’s (2008) English through Music, for example, aims to help children to absorb English naturally as they enjoy making music together. However, there have been few attempts to employ TBLT in naturalistic settings outside the classroom. The project described here is innovative in combining TBLT and digital technology in a naturalistic kitchen setting outside the classroom. Whereas classroom-based TBLT may engage the learners’ senses in terms of sight, sound and touch, the digital kitchen engages the senses of smell and taste as well, delivering a vivid, kinesic language learning experience. A further innovation is that vocabulary teaching is not normally associated with tasks (Samuda and Bygate 2008: 87) and this is an interesting but relatively unresearched area. In Chapter  9 we show how vocabulary learning was integrated into the task cycle and systematically tested. In relation to TBLT and digital technology, Thomas and Reinders (2010: 7) refer to the relative dearth and ‘marginalization’ of CALL research on tasks. Their collection tackles this issue by identifying and developing a range of areas involving technology-mediated tasks; these are reviewed in Chapter 2. The FDK and EDK projects therefore contribute to the research agendas of both TBLT and technology-mediated TBLT.

The principles of TBLT and digital kitchen design The overarching main task in the kitchen was designed according to Ellis’s (2003) criterial features quoted earlier in the following ways: we designed it to encourage learners to focus on meaning rather than purely language – that is, they use the language to complete a culinary task, rather than focusing primarily on the language itself. Second, learners must employ all four language skills in a holistic manner to achieve the task. Third, the task is situated in an authentic real-world context. The task is goal-oriented, involving the production of a dish. Fourth, kitchen tasks are carried out in pairs. In some cases, this generated interaction in L2. In the UK context, for example, we paired learners of English who did not share an L1, compelling them to communicate in English L2. Finally, learners can measure their own success by goal completion, through cooking and consumption of the food. A further characteristic of the FDK task is that it is a focused task in that it is necessary for learners to recognize the spoken form of named L2 vocabulary items to carry out the task. Learners are pushed to use these items in L2 talk with each other, but are not compelled; Ellis (17) notes that learners can always use communication strategies to avoid using the target

48

Paul Seedhouse

Table 3.1 Task design features of the FDK Design feature

Description in relation to the FDK Task

Goal

To cook a meal following L2 instructions; to learn a vocabulary set related to utensils, ingredients and cooking processes. L2 spoken, written and graphical input provided by the FDK system; contextual information is provided by the kitchen environment. This is a convergent task in that users must agree on how to cook the meal and a single outcome is targeted. All users receive the same basic information, but receive individualized feedback according to their choices and task progress. The task is intended for pairwork and for users to collaborate and produce some L2 talk related to cooking procedures. A meal from the L2 cuisine which can be eaten. Linguistically, it is predicted that some specific L2 vocabulary items will be learnt. Specifically, there will be concrete items (e.g. utensils) manipulated during the task.

Input

Conditions

Procedures

Predicted outcome

feature. Ellis (142) suggests that focused tasks are of value because they involve both reception and production and provide a means of teaching language items communicatively, under real operating conditions. Ellis (2003:  21)  provides a systematic framework for describing the design features of tasks, in which one must specify the goal, input, conditions, procedures and predicted outcomes. These are applied to the FDK task as shown in table 3.1.

Phase framework In order to operationalize TBLT in this setting, we adopted the cyclical pedagogic TBLT framework put forward by Skehan (1998) and Ellis (2003), which divides activity related to the completion of a task into three phases: pre-task, duringtask and post-task. This provided both a clear design structure for materials and a guide to implementation. The pre-task prepares the stage for the main task in the during-task phase. This might involve presenting new language, mobilizing existing language knowledge and clarifying the type of knowledge required (Skehan 1998: 138), framing the task and motivating learners (Ellis 2003: 244).

The Pedagogical Design of the Digital Kitchen

49

The during-task phase delivers the main task set. Skehan claimed that in this phase learners’ attention can be specifically manipulated through features such as time pressure, support and surprise. Ellis (2003: 257) suggests that a range of implicit and explicit techniques can be employed to focus attention on linguistc form during the task, for example, clarification requests, recasts, explicit correction, metalinguistic comment, query and advice. While in most classrooms these options can be provided by the teacher, in the FDK we see in Chapter 4 that some of these are provided by the system as help options. However, we also found that the learners often scaffolded each other’s learning in practice, as we will see in the extracts later. The post-task phase is designed to manipulate attention through reflection on and analysis of during-task performance, identification of what has been learnt and as a period of evaluation of the task outcomes. This recalls the ‘plenary’ section of a school lesson in which a teacher typically introduces the learning objectives of a lesson and pupils specify ‘what they have learned’.

Operationalizing the phase framework In this section we explain how the three-phase cycle was operationalized in relation to the experience of a cooking session in the FDK. Given the large amount of interactional data and the limitations of space, we can present only a small portion of the data. We illustrate what happens in the FDK by presenting representative episodes from a complete task cycle. We believe that this approach will best elucidate how the principles of TBLT are implemented in this real-world setting, and bring to life the experience of participants using the kitchen. Furthermore, the presentation illustrates how the three-stage task cycle is implemented in practice. The FDK and the EDK projects handled the pre-task phase in slightly different ways. This was because the FDK dealt exclusively with English learners of French, which meant that translation was a possibility and was not feasible with the number of language configurations operating in the EDK project. So two key supports which featured in the FDK but not in the EDK were (1) spoken and written translations of French words into English (2) labels in French placed on the kitchen utensils and ingredients. By explaining the TBLT cycle with reference to the FDK, we show the further range of possible supports which can be used in digital pervasive environments for language learning. Furthermore, readers can also follow the evolution in design from the French to the European Digital Kitchen, the design of which is outlined in the following chapters.

50

Paul Seedhouse

Pre-task The pre-task functions as a preparation stage for the activity to be carried out in the during-task phase. This may include the presentation of new language, the mobilization of existing language knowledge and clarification of the type of knowledge that would be required (Skehan 1998: 138). All three features directly relate to preparing or priming the learners’ attentional resources and are based on the operations involved in processing information in the short-term and working memory. The learners should get an indication of the purpose of the task and the kind of task it is. The pre-task in the FDK involved a dual focus on cooking and French skills and was divided into presentation and preparation of French and cooking. First, learners could watch a purpose-made video recording with optional subtitles of a native-French speaker making the chosen dish for the project, Clafoutis aux poires. This familiarized them with both the cooking procedures required and with the French language to be employed. They were able to choose to watch without subtitles, with French subtitles or with English subtitles, depending on the level of support they required. This facility enabled individualization of learning. In TBLT terms, this aspect of the pre-task framed the main task, motivated the learners and focused their attention on the L2 words which they would encounter during the main task. It introduced them to the process by which they would generate the task output, namely the dish. Second, the learners were able to watch an audio-visual slideshow of the different utensils and ingredients they would need to make the dish, in order to familiarize them with the specific French vocabulary required for the task. Figure  3.1 shows how each slide contained a photo of the kitchen utensil or ingredient, the corresponding word written in French and the option to listen to an audio file of the word being spoken. This introduced new language and mobilized existing resources. These first two activities were displayed on specially designed ‘ambient’ display screens on the walls of the kitchen (see Chapter  1 figure  1.1). Third, the final stage of the preparation involved listening to the kitchen, which verbally communicated the ingredients and quantities required via the speakers. At this stage in the task, the learners also had the opportunity to use the interactive screen to request help, such as a translation in English or the repetition of a phrase. So whereas the first two pre-task elements were passive and involved listening, the third element involved learners in actively noting down information and transferring it from spoken to written format. This element focusses them both on the language required and on the physical materials required for the cooking.

The Pedagogical Design of the Digital Kitchen

51

Figure 3.1 Example of slide in audio-visual vocabulary slideshow (screenshot)

The role of the pre-task in the overall cooking session is to prepare the users for the cooking activity. Its pedagogical aim is to provide input about cooking and language through the notions of preparation and presentation. In TBLT, these introduce learners to the linguistic and procedural knowledge required to complete the task. In the FDK we re-specified the notions of presentation and preparation to a dual focus on language and cooking. Extract 1 demonstrates the pre-task in action at the ingredient listing stage. Extract 1 KIT:

S1: S2: 1 2 3 4 5 6

KIT:

S1: S2:

Recorded Instruction

Student 1 (beginner L2 level) Student 2 (advanced L2 level) pour préparer votre clafouti (.) vous aurez besoin des ingrédients suivants (S2 picks up a pen and prepares to write) (2.2) what’s that mean? you need the following ingredients

52 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Paul Seedhouse S1:

right (.) ok (3.8) KIT: prenez un stylo et une feuille pour noter les ingrédients S1: [is that a pen? HA HA HE S2: [a pen HA HA HE (1.8) S1: I understand the odd word (3.8) KIT: cent grammes de farine= (S2 writes down the ingredient) S2: =a hundred grams of flour (.) S1: ok (3.2) S1: should we write – KIT: trente grammes (.) de poudre d’amande (.) (S2 writes down the ingredient) S2: thirty grams of almond powder

Prior to the extract 1, the learners had been introduced to the target vocabulary. In the extract, we see that the learners are required to listen and make a written note of the quantity of ingredients required. In the FDK, we paired a learner with culinary skills with a learner with French skills in order to create an information gap to promote interaction. So in extract 1, we see an interactional

Figure 3.2 This image relates to line 10

The Pedagogical Design of the Digital Kitchen

53

sequence which is a consequence of the pairing, namely S2 translating what the system says from French to English for S1’s benefit. The extract begins in line 1 with an audio message from the kitchen that the users are to be provided with the ingredient list to prepare the flan. The orientations of the users to the two opening messages from the kitchen at this early stage exemplify how the pre-task encourages the activation and initiation of existing and new language knowledge. In line 5, S1 self-initiates repair, which is hearably addressed to S2, who repairs S1’s trouble by providing a translation of the message. In line 7, S1 confirms that the trouble is resolved. In line 9, the kitchen asks them to get a pen and paper to write the ingredients down. This time S1 identifies a target-language item ‘un stylo’ and in line 13 displays a specific orientation to the state of her L2 knowledge. In line 15, the listing activity begins in earnest with the first type and amount of ingredient as one hundred grams of flour. A series of paired turns then follows (lines 15–23) where, throughout the listing of ingredients, S2 demonstrates an orientation to their own existing knowledge while at the same time providing substitute translations in English for S1. This process can be seen as a collaborative task where S2 provides S1 with opportunities for the activation of new language knowledge while at the same time displaying a dual orientation to cooking preparation and existing language knowledge. Throughout the pre-task, the collaborative activation and initiation of existing and new language knowledge is supported by the affordances of the technological design and real-world context of the kitchen. The users are supported interactionally through the process of cooking preparation:  after each item is presented as an audio message, the interactional space is offered for a user-initiated language focus on the ongoing activity. In TBLT terms, the pre-task obliges users to notice and process specific vocabulary items in the input. The content of the feature is provided by the requirement to locate and move the object itself and the linguistic form is salient as it is supplied by the system several times in both spoken and written forms. We can also see that the task involved learners using all four skills in a holistic fashion. The learners listen to instructions, write down ingredients, read text on the screen and speak to each other.

During-task The during-task phase involves the performance of the main task set. Skehan (1998) claimed that learners’ attention can be specifically manipulated through

54

Paul Seedhouse

Épluchez, dénoyautez et coupez les poires en quartiers et en tranches (Peel, core and cut the pears in quarters and in slices) Figure 3.3 Example of cooking instructions with specific cooking terms

Il vous faut un peu de farine (You need a little flour) Figure 3.4 Example of prompts for the learners

a range of features in this phase, such as time pressure, support and surprise. Here Skehan also refers to extended task procedures where during-task phases are designed to foster attention through scaffolding, where no specific input is given in the pre-task and what becomes the focus of learners’ attention is based on their needs in the during-task (Samuda et al. 1996). The during-task phase of course entailed cooking the dish. It involved step-by-step instructions on how to prepare the dish, together with a range of relevant feedback. The instructions and the feedback were verbally communicated by the kitchen as and when required, in response to the learners’ actions. The cooking task instructions were formulated in such a way as to include cooking-specific vocabulary on which we expected learners would focus most of their attention, having been introduced to the items in the pre-task. Some examples of these instructions are shown in figures 3.3 and 3.4 and cooking-specific vocabulary is highlighted in bold. Feedback included creating alternative versions of instructions, often reformulated in terms of ‘tips’ about cooking technique, which acted as prompts; figure  3.4 shows a sample of these. English translations were also created using cooking specific vocabulary. The final design of the cooking task instructions and feedback came about after a series of trials of different task instructions. The during-task phase of the cooking session, where users are engaged in cooking, promotes the activation and initiation of existing and new L2 knowledge by offering ways for users to notice and use the L2 to complete the cooking task, as we see in the extract 2. Extract 2 KIT:

Recorded Instruction

S1: S2:

Student 1 (beginner L2 level) Student 2 (advanced L2 level)

The Pedagogical Design of the Digital Kitchen 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494

KIT:

495 496 497 498 S2: 499

épluchez les poires (S1 continues with prior task of whisking the batter mix) (3.0) i don’t know what that means (S2 takes the pears out of a container) (3.6) how many pears? (.) [four] [four] (3.0) one (.) two (.) three (.) four (2.2) right tell me again lady (S2 takes the pears over to the chopping board) (2.9) how do I make her [tell me ] [n’oubliez pas] d’éplucher les poires i’m not sure what that means (1.9) (S2 uses GUI to select an English translation) don’t forget to peel [the pears] [aaaaah] (.) i swear that could be Eugene that’s talking (S2 takes a large knife and goes to peel the pears) er there’s a (reading off the label) do you want one of those? (S1 moves towards the peeler on the counter at the side of him whilst holding the whisk over the mixing bowl) (3.7) (S2 puts large knife on counter and picks up peeler) (reading off the label on the peeler) (2.9)

500 501 502 503 504 505 506

very helpful (S2 peels a pear using the peeler) what does that mean? [a knife to- ] [it’s a peeler] a knife to peel? [is that what it literally means? [yeah yeah (S1 continues to whisk the batter mixture)

S2:

S2: S1: S2: S2:

S2: KIT: S2:

KIT: S2:

S1:

S2: S1: S2: S1: S2:

55

Extract 2 demonstrates how the pedagogical and technical design features support the autonomous learning processes engaged in by the users. The episode begins half way through the whole cooking task with the audio message from the kitchen, ‘épluchez les poires’ (peel the pears). There follows an extended

56

Paul Seedhouse

Figure 3.5 This image relates to line 500

sequence of talk where a trouble source is identified by S2, namely that S2 does not know the meaning of épluchez (line 472). The focus on trouble is momentarily suspended while the pair returns to a collaborative focus on cooking activity only, where they are performing a check about the number of pears required (lines 475–478). In line 480, S2 reinitiates repair of the existing trouble source and in doing so, makes relevant the centrality of the linguistic focus to the task. Line 480 is particularly interesting from a human-computer interaction perspective as S2 directly addresses the kitchen as if it were an interactant. In line 483, S1 displays a further orientation to the technological context of the setting: S2 is trying to work out how to make the kitchen repeat the previous utterance. In line 484, an automated reminder is initiated. This feature is designed to support the users if the intended activity has not been detected after a certain time. In this case, the system supplied the repetition the users were looking for. However, in line 485, we see that the users still do not know the meaning of épluchez.

The Pedagogical Design of the Digital Kitchen

57

In line 487, S2 draws on one of the user-controlled supports available in the kitchen by pressing the GUI interface (a tablet, graphical user interface:  see figure 3.7) to get a translation of the last audio message. In line 489, S2 confirms that the trouble has been resolved. It is now possible for the users to progress with the cooking activity as they know what action is required. The real-world nature of the task means that the users need to access the right equipment to carry out the right actions for the task; it is not just a question of understanding the input provided by the audio messages. From line 492, we can see how, in this task-based learning environment, knowledge of language and cooking is interdependent. S1 in line 492 applies the recently noticed verb éplucher to identify the adjective éplucheur in the noun phrase couteau éplucheur (peeler, or directly translated, peeling knife). S1 does this by reading the label attached to the peeler with its name in French. S1’s turn demonstrates a dual focus on the ongoing cooking action, where a peeler is now needed, and a selfinitiated focus on language. In lines 500 to 501, S2 accepts the peeler from S1, reads the label aloud, evaluates it as helpful and starts using it. In lines 502 to 505, we see an example of a difference in learning styles being negotiated by the learners. Whereas S2 is happy with the functional translation into English as ‘peeler’, S1 wishes to understand what a literal translation of couteau éplucheur would be. Extract 2 shows how the pedagogical and technological design allows for the initiation and application of a new language, which can be made potentially relevant at any point in the during-task phase. The learning environment provides a range of possible supports or scaffolds to cater to a variety of learning styles and L2 proficiency levels, and learners can decide for themselves which to make use of. Here, the users draw on the GUI interface for translation, sensorrecognition-generated messages from the kitchen, as well as drawing on objects in the physical context, labels on the utensils and each other. We can see from the extracts in this chapter that the environment provides a multimodal and multisensory learning experience and that learners can choose the mode which most suits their own individual learning style and strategy. For example, learners can choose to find the meaning of a spoken instruction which they do not understand in a variety of ways. If they do not understand couteau éplucheur, they may request a translation. During the trials, for example, we found that some students of French whose written French was better than their spoken French would stand by the GUI. When they received the spoken French instruction from the system and did not understand it, they would obtain the instruction in written French displayed on the GUI and would generally understand

58

Paul Seedhouse

that. Some users were not interested in translation and preferred to look at the labels on the objects, while others preferred to try out actions to see how the system responded.

Post-task The post-task phase is designed to manipulate attention through the analysis of during-task performance and reflection, as a period of evaluation and consolidation after the completion of the task. It can also involve identification of what has been learnt and evaluation of the task outcomes. Skehan presents the post-task as an alternative to what he calls ‘within-task interference’, that is, the disruption that might be caused to the preservation of the communicative purpose if learners were too focused on attention to language features in the performance of the during-task phase (1998: 148). This is similar to the ‘plenary’ section of a school lesson where a teacher goes through the learning objectives of a lesson and pupils identify ‘what they have learned’. The post-task in the FDK focused on evaluation of what the users had learnt, as well as sampling of the task outcome, namely the dish produced. Targeted vocabulary was reintroduced to the learners through reflection and a series of short exercises. The exercises were designed to be completed orally and presented on the ambient display screens in the kitchen. So while the focus during-task was on meaning and task completion, the focus post-task was partly on linguistic form and on the language items learners had acquired through the task. An example of one of these exercises is given in figure 3.6. The pedagogical aim of the post-task focuses on assessment of and reflection about language. Assessment of language knowledge in the post-task is via the identification of language items by the users. Extract 3 provides an illustration of the post-task phase. 1 2 3 4 5 6 7 8 9

RA

S2: S1: S2: S1: S2:

that one is just about vocabulary see if you can match the English words with the French words mainly the ingredients and a few cooking terms that you may have heard ok (1.2) have you learnt any words (name)? une gousse de vanille? yeah! yes we’ve got butter (.) beurre (.) pear poire (.) milk lait (.)

The Pedagogical Design of the Digital Kitchen

59

Figure 3.6 Example exercise from the post-task 10 11 12 13 14 15 16

RA: S2: RA: S2: RA:

flour farine (.) vanilla pod gousse de vanilla (.) to peel is: (.) éplucher!= =oui! c’est ça éplucher to core is dénoyauter dénoyauter oui dénoyauter c’est ça dénoyauter ouais and slice is trancher excellent!

In this episode, the same users as in extract 1 are involved. The researcher is pointing them to a slide displayed on the GUI in the digital kitchen which involves matching target language items from the cooking activity with English translations (see figure  3.6). S1 identifies gousse de vanille (vanilla pod) as an item learnt and S2 reciprocates enthusiastically. This can be traced back to the pair’s first meeting with this item in the pre-task. In lines 9 to 16, S2 matches target language items with their English equivalents. The reflective nature of the activity is demonstrated through S1’s initiation of new knowledge which is oriented to more specifically as a relevant activity for S2. As the episode progresses, the correct matching of language items by S2 also demonstrates consolidated knowledge from the cooking task. The data provided in the post-task were combined with interactional data from the pre-task and during-task phases to provide evidence for learning of specific language items. An important element of the post-task was evaluation of the product, which involved eating the

60

Paul Seedhouse

Figure 3.7 The graphical user interface (GUI)

food produced and evaluating it. We also asked the learners to reflect on the experience, on what they had learnt, and to give us feedback on the experience. However, a limitation of the evaluation procedures in the FDK was that there was no systematic means of establishing which French vocabulary items the learners knew before the start, and therefore which items has been learnt. Therefore, as we see in Chapter 9, in the subsequent EDK project, a test cycle was ‘wrapped around’ the task cycle, to generate clear evidence of learning. How did the technological design support the three-phase TBLT framework? The ambient displays (Chapter 1 figure 1.1) provided a location from which to show the recipe-preparation video and the vocabulary slideshow, and the in-built speakers streamed the audio information attached to this visual information. The speaker system also communicated the list of ingredients in the pre-task. The GUI interface was specifically designed to support learning processes (figure 3.7). In the pre-task, the GUI offered three types of scaffolding: translation request, repetition request and the option of moving back and forth through the list of ingredients. In the during-task phase, the GUI provided learners with translations, repetitions and the option to move around the cooking instructions. In classroom-based TBLT, the learners carry out tasks themselves in pairs or groups, but can call on the teacher as a resource if they require some kind of help or support. In a similar way, the GUI provides scaffolding for learners if and when they require it.

The Pedagogical Design of the Digital Kitchen

61

Learning was also supported through the activity recognition sensor technology, which was designed to take the students through the different steps of cooking instructions in a timely manner, that is, as and when learners were ready in terms of how they were progressing through the recipe operations. Further scaffolding was provided in terms of prompts, consisting of alternative versions of instructions, often reformulated in terms of ‘tips’ about cooking technique. The prompts were designed in such a way as to occur in response to two alternatives: (1) after a period of non-activity where the sensor technology was able to detect that an operation had not been carried out even though a cooking instruction had been communicated or (2) if the incorrect food item or kitchen equipment had been moved as a result of miscomprehension. A  final part of the physical design of the kitchen environment was the inclusion of labelling in French on all items associated with the cooking task (tub of flour, oven door etc.) as well as the kitchen itself (cupboards, chopping board, tap etc.). Similar to the technological affordances for the pre-task, the post-task exercises were also embedded in the kitchen using the ambient displays. This technology offers the advantage over some classroom-based TBLT that learners receive immediate feedback on whether they have carried out task-relevant actions correctly or not.

Relating the principles of TBLT to the FDK tasks Ellis (2003: 276) introduces eight principles of TBLT (italicized) which can be used to guide implementation and design of participation. In this section we see how these were implemented in relation to the FDK. 1. Ensure an appropriate level of task difficulty. This was implemented by having a wide range of available resources and an introductory video with a range of options (e.g. subtitles) so users could tackle the task by choosing the resources suitable to their own level. 2. Establish clear goals for each task-based lesson. The main goal of cooking a dish was implemented by showing the video of the dish being prepared, including the final result. Goals for vocabulary learning were established in the pre-task by introducing the target items in both the video and the slides. 3. Develop an appropriate orientation to performing the task in the students. This was developed by supplying information about the task in advance of the session to users, by preparing them for the task in the pre-task and reflecting on it in the post-task.

62

Paul Seedhouse

4. Ensure that students adopt an active role in task-based lessons. The system was designed to require the users to take decisions and perform physical actions on their own initiative. There is no teacher present. 5. Encourage students to take risks. Users are told that they can make their own decisions as to which resources to make use of in order to complete the task. 6. Ensure that students are primarily focused on meaning when they perform a task.Users must focus primarily on carrying out the physical task by manipulating utensils and ingredients. 7. Provide opportunities for focusing on form. Users are able to summon help when they have problems in understanding L2 instructions. The help facility provides help in the linguistic form of the L2 target item in both spoken and written forms. The labels in French provide an alternative opportunity. We saw in extracts 1 and 2 that learners focus on form as and when required. 8. Require students to evaluate their performance and progress. In the post-task we require users to reflect on and evaluate their task completion and their learning. So it has been possible to implement TBLT principles and procedures in the design and implementation of tasks for the FDK, as demonstrated by the data.

Researching interaction and learning in a pervasive digital environment Samuda and Bygate (2008:  87)  suggest that TBLT process research studies concentrate on tracking learners’ individual and joint behaviours moment by moment, in order to derive a picture of how tasks as accomplished. There have been relatively few process studies of how TBLT is actually implemented in the classroom, and none of how it is implemented outside the classroom. In this section, we explain how we researched the task implementation process. The research element of the project had a dual function:  first, to provide data for iterative system redesign and second, to portray the process of users engaging with the task and the digital environment, as well as revealing the nature of the learning experience. We investigated whether any aspects of French language were acquired by users of the kitchen by analysing transcribed video and audio data with their transcripts, post-test and interview data. The task targeted specific vocabulary items and these were tested via the digital display afterwards. A  post-task interview questionnaire established what they had learnt. So we

The Pedagogical Design of the Digital Kitchen

63

were able to check for learning of specific lexical items by combining data from the interaction with post-test and report data. In TBLT, the accomplishment of a (non-linguistic) task is of key value, and we checked that participants were able to successfully cook meals as planned, by observation and by eating the dishes! The data sessions lasted 60–90 minutes and we paired participants so that one was more skilled at French and the other at cooking. According to TBLT principles, this might create an information gap, thus promoting information transfer between the partners. Thirty-six audio- and video-recorded sessions of paired adult learners with L1 English (totalling 72) cooking in the kitchen were carried out over a period of eight weeks. Each learner wore a microphone, and two cameras recorded task-related actions of the learners. French levels ranged from advanced to absolute beginners. Many participants were British undergraduates studying French and other subjects, while others were college students of catering. As noted above, we wished to portray the process of learners engaging with the digital learning environment in great detail. These data were analysed using Conversation Analysis (CA), a multidisciplinary methodology for the analysis of naturally occurring spoken interaction which is now applied in a very wide range of professional and academic areas. CA is used to describe and analyse the process of task-based learning, providing insight into the processes of learning through interaction and showing which of the many resources in the digital environment users are making use of for their learning. For an introduction to CA methodology in language learning, see Seedhouse (2004). There were two reasons for analysing the interaction between kitchen users in such detail. First, to provide evidence of the learning process of French and catering skills, as illustrated in extract 4. Second, the data fed into the process of iterative redesign, as shown in Chapter 4. To illustrate what actually happens in terms of task-based interaction and learning when users carry out the cooking task in the FDK, we will now examine some interactional data. It is generally assumed (Ellis 2003: 263) that tasks are carried out in pairs or small groups in order to maximize interaction and autonomy. Van den Branden (2006:  175)  sees the classroom teacher’s role in the task-in-process as (1) motivating the learners to complete the task (2) supporting task performance so as to promote processes such as negotiation of meaning and content, focus on form and production of output, to facilitate acquisition. There is designedly no teacher in the FDK and the task scaffolding function is assumed by both the system and the human co-participant, as we shall see. We paired students with higher level of catering skills with students with higher levels of French in the expectation that they might be able

64

Paul Seedhouse

to transfer skills to some extent, for example, S1, a catering expert with little knowledge of French, with S2, an upper intermediate learner of French with little knowledge of catering. Extract 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14

KIT: S2: KIT: S2: S1: S2: S1: S2: S1:

S1:

mélangez ces ingredients (mix these ingredients) °mix them together° et réalisez un puits dans la farine (and make a well in the flour) when you mix them you make a hole in the centre= =a well in [the cen]tre [yeah ] yeah (2.1) need to break them up (2.1) is that yeah they’ll break up (.) if you just give em a (1.0) .hh mix around (1.2) what’s mix (.) in F-

Figure 3.8 Mixing the ingredients

The Pedagogical Design of the Digital Kitchen 15 16 17 18 19 20 21 22 23

S2: S1: S2: S1: S2: S1: S2:

65

mélanger mélanger (.) mélanger oeufs (1.5) and then to make (0.4) faire un trou au milieu (.) is another way of saying what do you do to them. u::m un trou (.) a [hole] [a ] well a kind of (.) hole

In this extract we see how the task involves a dual orientation to the cooking task and to language (Seedhouse and Knight 2016). In this case there is a tension between the two orientations. In lines 1 and 3 we hear the system giving the students instructions in French on how to proceed with the cooking, which S2 translates into English for S1’s benefit. It is important to understand that the technical catering term to use in the case of making a hole in a quantity of flour in which eggs will be poured is ‘puits’ in French and ‘well’ in English, which is a literal translation. So although S1 does not understand the instruction in French in line 3, he knows from the context of the cooking operation that they are making a well and uses that technical term in line 6, and again in line 22. S2 does not appear to know the technical catering term in French or English and refers to it as ‘hole’ in lines 5, 21 and 23 and as ‘trou’ (hole) in lines 18 and 21. S2 appears to be trying to teach S1 the French word ‘trou’ in 18 and 21, although this was not a word spoken by the digital kitchen. However, the point is that the participants display such an orientation to completing the cooking task that some confusion over linguistic terms is not a problem, given the context of the hole/well in the flour which they have created. In lines 8–13 S1 gives cooking advice to S2 on how to do the mixing to get the best results. Then in line 16 S1 asks for a translation of ‘mix’ into French, which S2 provides (line 15). S2 helps S1 with language and S1 helps S2 with cooking skills. In this episode, we can see users following the instructions provided by the system, engaging with both the linguistic and culinary levels of the task, and providing help to their partner. We can see evidence of incidental focus on form in lines 14–16, as well as of the information gap and information transfer processes targeted by TBLT. Heritage (2012) demonstrates that information imbalances act as drivers for talk in a wide range of contexts. The extract also illustrates one of the two ways in which the sociocultural SLA concept of ‘scaffolding’ was designed to work in the FDK context. Ellis (2003:  180)  defines scaffolding as the dialogic process by which one speaker assists another in performing a function that he or she cannot perform alone. In

66

Paul Seedhouse

this extract, we see that scaffolding can be a two-way process for the FDK participants, involving different skills. The second way in which scaffolding takes place is that the system is able to offer help to the participants, if they choose to use it, to enable them to carry out the task. This was illustrated in extract 2. The Making a Hole pair said in a post-task interview: 

 

‘I was able to understand the task instructions with assistance from [partner] and deep thinking of my own French knowledge’ ‘I found the cooking task easy because [partner] explained it well’ ‘It was interesting learning the recipe and the most effective techniques’

Discussion and conclusions This chapter has introduced the TBLT principles which underlie the pedagogical design of the FDK, shown how these were operationalized and illustrated the interactional and learning processes in which learners are engaged. We can conclude that it is indeed possible to employ TBLT principles outside the classroom, and that these provide a suitable basis for designing a pervasive digital environment for language learning. Motteram and Thomas (2010: 228) report on a number of common criticisms of TBLT in relation to CALL. The first is that it is functional as opposed to pleasurable and creative. One important aspect of this particular TBLT task is that participants typically find it very enjoyable as well as creative in that they have produced something which can be admired and eaten. The evidence for this is contained in participant feedback on the project website, as well as in interview and questionnaire data. Motteram and Thomas (2010: 229) argue that the members of the current ‘net generation’ have a number of distinguishing criteria which should be taken into account when evaluating the suitability of learning materials. We comment on these here in relation to the FDK experience. 





They are independent, wanting to search for information themselves: The FDK allows them to decide which learning resources to employ. They have an emotional and intellectual openness to others: The FDK exposes them to other cuisines, cultures and languages, but not directly to people from other cultures. They are interested in social inclusion: The idea of inclusion of people from other cultures is at the heart of FDK design.

The Pedagogical Design of the Digital Kitchen 











67

They demonstrate free expression and strong views: They are able to evaluate the experience in the post-task. They like innovation: The technology and overall experience are clearly innovative. They have a mature attitude to life and learning: They are able to take control of their use of learning resources and can make their own decisions. They are investigators: They are able to take an active role in investigating the language they require, how the system works and how to cook the dish. They have a sense of immediacy: the physical nature of the kitchen task provides immediacy relating to all of the senses: sight, sound, smell, touch and (in the post-task) taste. They need to do everything at high speed: The cooking task can be done at the users’ own pace.

Motteram and Thomas (2010: 230) further report that another typical criticism of TBLT is cultural relativity, that it reflects Western educational values. One way in which the EDK works against this criticism is to actively engage users with a foreign culture through engaging with its cuisine and language, thus promoting cultural exchange. The EDK has been trialled with students from all over the world who are learning European languages. However, we have also developed and trialled the Korean Digital Kitchen at Newcastle University, and there is no evidence that it is any less suitable for learning Korean than for Western languages. Resistance to TBLT has not been reported by participants from other cultures during EDK trials. However, the EDK is not presented to users as a TBLT project as such. Rather, they are told they will be cooking a meal from a foreign culture in a digital kitchen and will learn aspects of the L2 at the same time. Feedback from learners (which can be read on the project website) typically focuses on the experiences of cooking and of using new technology rather than anything specifically related to TBLT methodology. Since cooking is such a universal task and so closely related to culture, it may be that it occupies a special position in the array of possible tasks which could be used for TBLT. Therefore, we should be wary of generalizing about TBLT and culture from the FDK and EDK projects. In Chapter 4, we will see how the pedagogical principles of TBLT were combined with technological design principles to develop the EDK. In a number of the following chapters, reference will be made to the TBLT principles which we have encountered in this chapter.

68

Paul Seedhouse

Note All figures and tables were produced by the Newcastle University team.

References Ellis, R. (2003), Task-based Language Learning and Teaching. Oxford: Oxford University Press. Heritage, J. (2012), ‘The epistemic engine: Sequence organization and territories of knowledge’, Research on Language and Social Interaction, 45 (1): 30–52. Johnson, K. (2003), Designing Language Teaching Tasks, Basingstoke: Palgrave Macmillan. Long, M. (2015), Second Language Acquisition and Task-Based Language Teaching. Oxford: Wiley-Blackwell. Motteram, G., and M. Thomas (2010), ‘Afterword: Future directions for technologymediated tasks’, in M. Thomas and H. Reinders (eds), Task-Based Language Learning and Teaching with Technology, 218–235, London: Continuum. Paterson, A., and J. Willis (2008), English through Music. Oxford: Oxford University Press. Samuda, V. and M. Bygate (2008), Tasks in Second Language Learning. Basingstoke: Palgrave Macmillan. Samuda, V., S. Gass, and P. Rounds (1996), ‘Two types of task in communicative language teaching’. Paper presented at the TESOL convention, Chicago. Seedhouse, P. (2004), The Interactional Architecture of the Language Classroom: A Conversation Analysis Perspective. Malden, MA: Blackwell. Seedhouse, P., and D. Knight (2016), ‘Applying digital sensor technology: A problemsolving approach’, Applied Linguistics, 37 (1): 7–32, doi: 10.1093/applin/amv065. Seedhouse, P., A. Preston, P. Olivier, D. Jackson, P. Heslop, T. Plötz, M. Balaam and S. Ali (2013), ‘The French Digital Kitchen: Implementing task-based language teaching beyond the classroom’, International Journal of Computer Assisted Language Learning and Teaching, 3 (1): 50–72. Skehan, P. (1998), A Cognitive Approach to Language Learning. Oxford: Oxford University Press. Skehan, P. (2003), ‘Task-based instruction’, Language Teaching, 36: 1–14. Willis, J. (1996), A Framework for Task-Based Learning. Harlow, UK: Longman. Thomas, M., and H. Reinders (eds) (2010), Task-Based Language Learning and Teaching with Technology. London, New York: Continuum International. Van den Branden, K. (ed.) (2006), Task-Based Language Education: From Theory to Practice. Cambridge: Cambridge University Press.

4

The Technology behind the European Digital Kitchen for Language Learning Paul Seedhouse

Introduction In Chapter 1 we noted the key features of any pervasive, real-world digital environment for language learning. The features which specifically require some kind of technological support are as follows: 













Participants physically carry out real-world tasks (using real-world equipment) which are embedded in everyday, real-world contexts such as a kitchen, an office or a shop. The task can be broken down into a series of specifiable physical actions. Participants should receive some L2 input from some source and be able to learn some aspects of the L2 by performing the task. Participants physically touch and manipulate real-world objects while carrying out the task and have the opportunity to learn the L2 names of these objects. The digital system can track how participants are carrying out the series of physical actions which constitute the task via a number of digital sensors embedded in the environment. The technology is designed to facilitate performance of the task, but is not the focus of the activity – it remains in the background. The system provides timely instructions, feedback, help and tips to users to enable them to perform the task. The feedback facilitates multimodal and multisensory learning by use of audio, photos and videos. The learning environment provides a range of possible supports or scaffolds to cater for a variety of learning styles and L2 proficiency levels, and learners can decide for themselves which to make use of.

70 



Paul Seedhouse

Participants can ask the system for help or for explanations, but are not obliged to. It is best practice to develop an authoring tool so that materials can easily be developed for other languages.

In this chapter, we look at how the technology was designed and developed to deliver these environmental features for the EDK, and how materials were designed to work with the third-generation EDK technology. The firstgeneration technologically enhanced kitchen (The Ambient Kitchen) was developed at Newcastle University to support older people and those with dementia in their everyday kitchen activities (Olivier et al. 2009). The technology of the second-generation French Digital Kitchen (FDK) (Seedhouse et al. 2013) was described in Chapter 3.

How does the European Digital Kitchen work? A technological overview In this section we examine the technological components of the pervasive system, namely the sensors, the tablet with its graphical user interface, and the interaction tools.

Sensors The system talks to the users in a foreign language, using pre-recorded audio files, and gives instructions which are intended to lead them step by step through the task of cooking a European dish associated culturally with that language. Encased wireless digital sensors (figure 4.1) are inserted in or attached to all of the cooking equipment and ingredients, allowing the system to detect activity and transmit data to the system as users progress through the stages of their cooking task. The system can tell whether users are carrying out the intended actions or not and give appropriate feedback. The EDK tracks progress through recipes by using sensors called wireless accelerometers or WAXs, which measure proper acceleration, or 3-dimensional movement. These are commonly used in applications such as Nintendo Wii™, which detect and send information about the 3-dimensional movement of users so this can virtually simulate playing tennis, golf and the like. The sensors have wireless capability so they can transmit

Technology behind Digital Kitchen

71

Figure 4.1 Wireless accelerometer sensors

information to a receiver located in the system, which is run from a specially adapted tablet (figure 4.3). In the new third-generation EDK project, the team designed and built a very secure and robust red plastic casing for the digital sensors. In the second-generation FDK, we found that users would forget about the sensors attached to the equipment and immerse them in a sink full of water, bake them in the oven or let them melt on the hotplate! We budgeted for 30 sensors per EDK kitchen, 5 of which are spare sensors. Of the remaining 25 sensors, 1 acts as a receiver (passing information from the other sensors to the tablet system), and 24 can be paired with utensils and food items. Sensors are charged by plugging them into a USB port, using the cable provided and a full charge takes around one hour. Once fully charged, a sensor will last for 10 hours of active usage; when not in use, the sensors go into a ‘sleep’ mode that lasts a month. Testing that sensors are working before sessions is vital, which can be done by running through the motions of the recipe – that is, shaking the appropriate kitchen items at each step. The sensors then had to be attached to kitchen equipment and food. How could this be done with flour? As can be seen in figure 4.2, sensors can be integrated into the handles of cooking utensils. In the case of the peeler and spatula shown, these had recesses into which the sensors could be inserted. In the case of the knife, a 3-D printed design was necessary so the sensor could fit inside the handle. In the case of flour or other foodstuffs, these are placed in plastic or metal containers and sensors can be attached by Velcro, or a cradle with a magnet. In the case of the spoon, it has a hole in the handle and a ring can attach the

72

Paul Seedhouse

Figure 4.2 Sensors attached to objects

sensor. Similarly, Velcro or magnets can attach sensors directly to kitchen appliances, for example, oven knob. So various devices are commercially available to attach sensors to virtually any object which could be instrumented for use in a digital learning environment.

The tablet and graphical user interface The system runs from a commercially available touch screen tablet. This has a 3-D printed outer casing to protect it in the messy kitchen environment (figure 4.3). The wireless receiver is housed in the casing in a space next to the tablet. The graphical user interface or GUI is the touch screen interface component specially developed for the EDK project. The GUI is the main interface through which users interact with the EDK system, with the interaction tools (see figure 4.4) offering an alternative. The system provides information to the users in terms of sounds (playing audio files), displaying photographs and videos of objects and actions as well as written text. The users input information to the system by pressing the GUI buttons as follows:

Technology behind Digital Kitchen

73

Figure 4.3 The tablet in its casing

This allows the user to navigate around the recipe and go back to hear the last step.

74

Paul Seedhouse

This allows the user to repeat the last action played by the kitchen, which could be a repetition of an instruction or an image file of a video. This will only play the last action and not the step; for the step, the user should use the navigation back button.

This appears when the kitchen detects a problem and allows the user to request the help available with a particular step. It reappears when the step has been completed.

This function allows the user to pause the system. This button should not be used to pause the system between steps but in case the user needs to stop the cooking activity (e.g. to go to the lavatory).

This allows the user to indicate to the system that they have completed a specific action. The user will only need to do this when they are carrying out an action which does not use a sensor (e.g. kneading dough).

Technology behind Digital Kitchen

75

This allows the user to navigate around the recipe and go forward to the next step. This button should not be used to move the system to the next step as the sensors are designed to pick up on when the user is ready to move on.

This allows the user to check where they are in the recipe, how many actions they have completed and how many they have left.

This button closes the EDK player

Interaction tools The team developed two tangible hand-held interaction tools for use within the kitchen environment, namely a green and a red tool (figure 4.4). These aimed to offer users choice and control over the cooking process and soliciting help. Each interaction tool contains a wireless sensor. When moved and then placed at rest again, the red tool sent a request for help, while the green tool sent a ‘step complete’ message to the system. This gave users the choice of using either the interaction tools or the GUI interface for the two most common commands. This meant that, rather than going over to the GUI all of the time, users could give instructions without breaking off from their task. The green tool also enabled learners to alert the system that a particular cooking step was complete in which sensors could not be involved. For example, when kneading dough, it is not desirable to insert the sensor in the dough.

Technical design modules In this section we look at the overall organization of the system. From a technical perspective the system consists of three main parts: a sensing and recognition

76

Paul Seedhouse

Figure 4.4 Interaction tools

Sensors

Sensing & Recognition Simple AR DTW AR

Inference

Prompting & Interaction

Automaton

User Interface

- Recipe

- Audio

- State

- Text

- Prompt

- User input

Figure 4.5 Overview of the system

module (S&R) for tracking the learners’ activities and the state of the system; an inference module (INF), which infers what progress users are making through the stages of the recipe; and a prompting and interaction system (P&I) for providing situated support related to the language learning task. Figure 4.5 gives an overview of the system. The team attached acceleration sensors (see figure  4.1) to equipment and ingredients. When a sensor detects movement, it starts transmitting the raw acceleration data to a nearby receiver, which is housed in the case of the tablet from which the system runs. In order to provide timely prompts and context-sensitive instructions to the learners, the system must keep track of the progress they are making with respect to the steps of the recipe they are cooking. Since recipes are typically linear, the

Technology behind Digital Kitchen

77

team used simple state automatons for recipe modelling that are easy to author for a human system designer. The automaton model consists of a sequence of major states. A major state specifies the overall task being carried out at that stage of the recipe. Each major state has a sequence of sets of one or more actions. Each set must be completed before moving to the next, but the actions within a set can be completed in any order. Each action has a condition that must be met for it to complete, for example, an activity is recognized; a fixed time period is reached; or a set of activities has been absent for a certain length of time. If an action’s condition is not met within a defined interval, the next prompt for that action is read by the system. Therefore, at any given stage of the recipe, the system would await a combination of signals showing that the learners were engaged in the current step. For example, after an instruction to ‘grease the parchment paper’, the system would wait for three things to happen before providing the next instruction: sensor data show movement of the butter dish; sensor data show use of the pastry brush; then a subsequent lack of activity implies that the learners had finished using the butter dish and pastry brush. As well as the automated prompts based on the learners’ progress in the recipe, communication to and from the system can be controlled by the learners via two channels: a GUI (figure 4.3) available via a touch screen tablet and two interaction tools (figure 4.4).

High-level outline of the EDK system In this section an overview is provided of how an entire cooking session is organized from the system’s perspective. Section 1 shows the basic structure of the session, for which the specific recipe content is then specified by means of the authoring tool, whose basic structure is described in section 2.

Section 1: Playback tool Part 1: Pre-intro Interface for recipe selection. The GUI displays the range of recipes in different European languages and cuisines and the user selects a particular recipe.

Part 2: Pre-intro This is an optional interface to play video. For specific recipes, videos may be available for users to watch the whole process of cooking the recipe. Options

78

Paul Seedhouse

are available to switch on subtitles in L1 or L2. This means that users have the option to select suitable support for their own level of L2 proficiency. Typically, advanced level users skip this stage.

Part 3: Intro Pre-task  – Ingredient and equipment collection activity supported by userinteraction interface. Displays:    

Back and forward buttons Help request Pause Photos with voice-over and written text

The pre-task covered presentation and preparation of language and cooking knowledge. Learners were prompted to collect the utensils and ingredients needed to prepare a dish. They thereby became familiar with the L2 target language and also with the way in which they could communicate with the EDK.

Part 4: Main Cooking activity (during task) – Leading users step by step through the activity supported by communication via GUI interface and interaction tools. Displays:      

Back and forward buttons Tips Help request Pause Photos with voice over and written text Videos with voice over and written text

For the ‘during task’, the main cooking activity, the system prompted learners to move step by step through a set of cooking actions. After an initial instruction in L2, a range of supports were available to them, if they chose.

Part 5: End Post-task – optional ingredient and equipment collection activity which mirrors pre-task support by user-interaction interface

Technology behind Digital Kitchen

79

Displays:  

Back and forward buttons Pause

Section 2: Authoring tool Part 1: Pre-intro Authoring for recipe selection: Record and upload audio message 

‘please select your recipe’ from the range available

Once selected, play the message 

‘great, you’ve chosen to make …..’

Part 2 (optional): Pre-intro Authoring for video: 

upload video for background film about the recipe selected, with subtitle options

Part 3: Intro Authoring for pre-task: Record and upload audio messages in L2 to be played during the pre-task ‘for this recipe you will need’ individual recordings with the pronunciation of ingredients and equipment which must be located during the pre-task a negative feedback message to be played when the wrong item is selected a positive feedback message to be played when the correct item is selected Record and upload audio-visual materials to be used when help is selected  

 



photo with voice-over and written text of ingredients and equipment

80

Paul Seedhouse

Part 4: Main Authoring for main cooking activity (during task): Record and upload L2 audio messages     

instructions for individual steps in the recipe a negative feedback message to be played when the wrong item is moved a positive feedback message to be played when the correct item is moved tips and hints ‘do you need help?’ statement

Record and upload audio-visual materials to be used when help is selected  

picture with voice-over and written text of ingredients or equipment video with voice-over and written text of ingredients or equipment

Part 5: End Authoring for post-task:  

individual recordings of ingredients and equipment success statement or sound

The EDK authoring tool The authoring tool allows users to upload their materials to the EDK system so that they can be used as recipes by anyone using the digital kitchens. An authoring tool is a computer-based system that allows technical and non-technical specialists to programme content for use in software applications. It is designed in a user-friendly format and does not require knowledge of programming. The features that make the content (audio instructions, photo and video) work in the system software are built in, but hidden behind buttons and other tools. Users therefore simply ‘upload’ their materials in any language to the authoring tool. The authoring tool can accept input and produce output in any language in spoken and written formats, which is achieved by loading video and audio files with writing in any script as subtitles. An excellent way of understanding how the EDK system interacts with users is by examining in detail how materials are loaded onto the authoring tool.

Technology behind Digital Kitchen

81

First of all, though, we must be clear about the work which needs to be done to a recipe before it can be loaded onto the authoring tool. The system must ensure that learners understand what is required of them at a particular stage. In terms of pre-prepared input, this involved ensuring that appropriate step-bystep recipes were developed in a coherent and logical format and that steps were suitably designed and pre-trialled for use with associated sensors. During the cooking activity the system provided situated prompting through the following procedures. Users in the kitchen follow the recipe step-by-step. However, as opposed to recipes which are followed through a conventional route such as a recipe book, every single action of the user needs to be presented separately. For example, consider the following instruction: ‘Pre-heat the oven to 220 degrees and line a baking sheet with parchment paper and grease it’ (Extract from an existing scone recipe)

This instruction presents three steps in one instruction, which is not appropriate for the materials for EDK. Rather, this kind of activity should be presented as: Step 1: Preheat the oven to 220 degrees (user then carries out the step) Step 2: Line the baking sheet with parchment paper (user then carries out the step) Step 3: Take a small knob of butter and put it in the microwave for 5 seconds Step 4: Grease the paper with melted butter (user carries out the step) Here the instructions from a conventional recipe have been broken down into smaller steps and relevant additional information has been provided to make any potentially ambiguous steps as clear as possible to the user (as in the case of greasing the parchment paper).

The Setup tab Authoring a new recipe starts with selecting the Setup tab. In this section, write the title of the recipe and upload a photo of the completed dish. A video can be uploaded to show the preparation of the dish with optional L1 and L2 subtitles. In the Global Settings option select the word ‘no’ for negative feedback in the L2, the language of the recipe. There is an option to upload success messages (e.g. well done!) which play when users complete a required action. An option to upload messages which tell the user when help is available (i.e. do you need help?) is also available.

82

Paul Seedhouse

Figure 4.6 The authoring tool interface

The Sensors tab For the programme to work correctly, the system must be able to relate a specific sensor movement with a specific instruction. For example, if the instruction is ‘stir the mixture with the spoon’, then the system must detect movement from the spoon’s sensor only to know that the correct action has been performed. In this stage, sensors are matched to instructions and actions. First, add the name of the sensor, which corresponds to the object to which the sensor is attached. Then upload a photo of the object to which the sensor is attached. Upload a correction phrase for the object to which the sensor is attached (e.g. you need butter). Repeat this process for each sensor by selecting Add Sensor.

The Pre-task tab This stage involves ingredient and equipment collection activity for familiarization with both language and recipe tools. Choose Add Step and add the name of the step, for example, ‘fetch the pastry brush’. Upload an audio file in L2 giving the name of the object involved in the step, for example, ‘fetch the pastry brush’.

Technology behind Digital Kitchen

83

Figure 4.7 Selecting sensors for the success conditions

Next, under Step1, select ‘Success’. You then tell the system what the success condition for this step is by selecting the photo associated with the object. If the system receives movement data from that sensor, it registers success and moves on to the next step. Each sensor must be connected or ‘paired’ with an object name and photo on the system; it must then be attached to the real-life ingredient or utensil in the kitchen. With some cooking actions, it may not be possible to have a sensor associated with it. For example, if the step involves collecting a baking sheet for the oven, it is not possible to attach a sensor to it as the sensor will get baked! In this case you should still select the baking sheet in setting your success condition but use a tick box option. This means that the user must tell the system when they have finished the action by using either the tick option on the GUI or by shaking the green interaction tool.

The Task tab In this tab you input instructions and help for the main cooking task. Select Add Step and add the name of the step, for example, pre-heat the oven to 220ºC. Then upload an audio file which gives those instructions. Then select the Help 1 tab 1. Add an audio file with a slower, more emphasized version of the instruction. Select the time period you wish to make this help available (e.g. 10 seconds)

84

Paul Seedhouse

Next, staying under the Task tab, select the Help 2 tab. Add a photo file to accompany the audio instruction, which can include the name of the object in written form in L2. Select the time period you wish to make this help available (e.g. 5 seconds). If relevant to the step, staying under the Task tab, select the Help 3 tab. Add a video file to accompany the audio instruction to illustrate a particular process, which can include the name of the object in written form in L2. Select the time period you wish to make this help available. If relevant to the instruction, staying under the Task tab, select the Tip tab. Here, add an audio file for a tip which accompanies the step. Next, staying under the Task tab, select the Success tab. As with the Pre-task tab, you tell the system what the success condition for this step is by selecting the photo associated with the object. If the system receives movement data from that sensor, it registers success and moves on to the next step. Repeat these actions (Instruction, Help 1, 2, 3, Tip and Success) for each step in the Task tab by selecting Add Step. When complete, the interface should look like figure 4.6.

The Post-task tab Next, select the Post-task tab and select Add step. Add the name of the step (e.g. ‘you will need butter’). Add an audio file associated with the step (‘you need butter’). Next, staying under the Post-task tab, select the Success tab. Select the object or objects with sensor attachments which are associated with the step (butter).

The Publish tab Finally, check that you have filled in all the relevant sections and select the Publish tab. If there are no errors in your recipe, the Authoring Tool will publish your recipe. If there are errors in your recipe, you will be given a list of issues to resolve under the Set-up, Sensors, Pre-task, Task or Post-task tabs. When the recipe is published, it can be loaded onto an individual tablet ready for use with the sensors which have been associated with the specific recipe, ingredients and utensils. Specific named sensors need to be attached to the matching ingredients and utensils.

Technology behind Digital Kitchen

85

This overview of the system has shown how materials are created and how the users are intended to interact with the system. Next, Chapter 5 provides examples of how the users and the system actually interact in practice and also develops a transcription system which enables analysis of this two-way interaction.

Technological design principles The principles underlying the technological design derive from pervasive or ambient computing, information processing (IP) theory and interaction design, which are outlined in this section. The technological design of the kitchen, drawing on activity recognition and sensor technology, is what helps to support independent language learning in the real-world environment of a kitchen. The technology is a form of pervasive or ambient technology, which is defined by Weiser as the ‘third wave in computing’ and ‘roughly the opposite of virtual reality. Where virtual reality puts people inside a computer-generated world, ubiquitous computing forces the computer to live out here in the world with people’ (Weiser 1996). Pervasive or ambient learning environments offer contextualized and situated learning experiences in everyday settings where users are guided and supported through learning tasks by ambient intelligence. The notion that the ambient technology is embedded in these everyday settings not only relates to the idea that it is hidden or fully integrated into the learning environment but also that it is able to adapt and support the everyday learning-based activities that occur there. Therefore, there is an important difference which distinguishes the use of ambient technology for language learning from the different learning goals and purposes of traditional approaches to CALL. This difference originates in the relationship between human and machine where the focus of ambient learning is on human-computer interaction (HCI), where technology is designed not to play a mediating role but to interact directly with humans. As Mazur puts it, in HCI, ‘humans’ interactions with machine can be characterised as “conversations” by denoting the sequencing of “give and take” involved in the use of expert systems as “turns” ’ (2004: 1080, emphasis in original). The affordances of HCI are linked to how all possibilities for talk-ininteraction support real-world face-to-face interaction. For language learning, this means that ambient technology can not only provide linguistic input, but can also afford how and when that input is provided in response to interaction

86

Paul Seedhouse

with people performing real-world tasks. In HCI environments, ambient technology is designed to provide timely linguistic input and real-time linguistic and procedural support based on its ability to anticipate and respond to users’ actions. Importantly, this input can only ever be pre-defined, pre-programmed feedback rather than completely tailored to users’ actual productions or actions in a real-world environment. A further practical component of EDK’s integrated technological and pedagogical design is the notion of interaction design. This concerns how the learners actually interact with the system and the materials, both as a collaborator in their cooking experience and as a source of assistance. The computing science EDK team spent a considerable time investigating and considering how the learners should practically request the help on offer if it is needed and how the learner is told about the help available. Encompassed in this design activity is the notion of ‘accompaniment’ or how the learners work in conjunction with the kitchen as partner in their activity. Giving the learners control over their learning experience is an important aspect to facilitating autonomous learning processes in the kitchen. However, there could be pedagogical tensions around this design choice. As we found in the previous FDK project, certain learners repeatedly used certain ‘easy option’ supports (such as translation) to complete the task, rather than engaging with existing or new language knowledge. Some learners chose to focus solely on the outcome-related purpose, rather than the language-learning goals. The EDK therefore offers learners choice over when and how they wish to receive this support but is balanced with some kitchencontrolled input. Learners themselves have to select, through communication via the GUI or interaction tool, whether they would like to receive this help or continue with the task without further guidance from the system. It also ensures that learners spend some time engaging with the language before receiving help from the system. In either case, the supports are provided in an escalating fashion ranging from less (audio repetition) to more support (video). Preston et  al. (2015) applied IP theory to create a learning framework for the EDK, as opposed to a learning theory. This provides the bridging stage between the pedagogical framework of TBLT and technological implementation. IP informed the interaction design in the EDK in terms of how the system presented information (L2 talk) to learners to promote learning. This information had the dual function of supporting the cooking task and providing feedback, which encouraged learners to notice the formal properties of the L2 they

Technology behind Digital Kitchen

87

encountered. The team aimed to maximize opportunities for attentional manipulation (Skehan 1998) and ‘noticing’ (Schmidt 1990), on the assumption that the more frequent and salient a language form is, the more likely it is to enter the working memory. We applied the IP perspective proposed by MacLaughlin et  al. (1983) to strike a balance to the interaction design between case 3 (performance based on implicit learning) and case 4 (performance in communicative situations) in. For case 3, design decisions balanced a focus on content (the cooking instruction) and linguistic form (the language contained in the cooking instruction). For case 4, our priority was to allow communication to flow. In the following section we show how the IP framework relates to the three-phase task cycle which was introduced in Chapter 3.

Task-based materials design for the EDK So far in this chapter, we have looked at how the technology works, as well as at the principles underlying its design. In this section (based on Preston 2012), we examine how the pedagogical principles of TBLT, which we encountered in Chapter 3, have been realized in the pedagogical design of the learning environment of the EDK. First, we need to understand the components of the EDK learning environment as a whole. We have seen earlier that the basic technological components of the environment are the wireless sensors, the system housed in the tablet and accessed by users via the GUI, and the interaction tools. From the user’s perspective, there are other significant components to the learning environment which are not technology based, namely the food ingredients, the utensils, the physical kitchen (cooker, surfaces) and their learning partners. Another vital component of the learning environment is the materials for learning cooking and foreign languages, which is loaded onto the system as software. This then communicates with the users and guides them through the cooking and language learning task. The interface between the system and the materials is the authoring tool (figure 4.6). Recalling the TBLT principles introduced in Chapter 3, Willis (1996) defines the aim of tasks as ‘to create a real purpose for language use and to provide a natural context for language study’ (1996:  1). Central to the real-world use of language is the notion of using the target language to achieve an outcome. Nunan (1989) defines a communicative task as ‘a piece of classroom work which

88

Paul Seedhouse

involves learners in comprehending, manipulating, producing or interacting in the target language while their attention is principally focused on meaning rather than form’ (10). Nunan’s definition is particularly helpful for task-based learning materials design in that it makes relevant the notion of tasks as representing a unit of organization for language teaching which can be manipulated as such. Taken together, Willis’s and Nunan’s definitions of task-based learning reflect the dual focus of EDK materials development:  to create a learning experience where learners cook a culturally relevant dish in the target language and to learn/acquire aspects of the target language. This dual focus reflects how EDK materials should support both a learner’s sense of achievement in reaching their goal, to cook the dish, and our pedagogic goal as designers to develop the kitchen as a language learning tool. Task design for EDK can be broken down further into the following components:   



Content (linguistic and procedural input in the recipes) Materials (in terms of the physical world of the kitchen) Activities (pre-during-post task activities to support provision of input and initiation of output) Learners (ability, needs, motivation which can be supported through the pedagogical design of the activities) (adapted from Shavelson and Stern 1981)

These components can be specifically adapted according to the nature of the task through the manipulation of specific features in different phases of a task:   

Task complexity Salience (of language forms) Procedure (in terms of achieving the outcome)

The materials are designed to support learners in achieving this aim through the following objectives, which are embodied in the technological and pedagogical design: For cooking:  

To guide learners in cooking a dish by providing step-by-step instructions To guide learners in identifying a range of cooking actions and techniques

For language learning: 

To provide learners with opportunities for noticing of and attention to linguistic form through a focus on meaning and subsequently, form

Technology behind Digital Kitchen 

89

To provide learners with opportunities for the negotiation of meaning by providing modified input

The pre-task is generally understood as an introduction or priming stage where the focus is on meaning-based activities and a focus on language (or focus on linguistic form). The linguistic input provided and explored through a range of activities at this stage resembles that which will be used later on in the post-task. The during-task involves a further focus on language within the communicative context of planning and rehearsing for the outcome of the task in the post-task. The post or ‘production’ stage involves drawing together the work of the preand during-task stages. Table 4.1 shows how the design and manipulation of all the above components of the materials is carried out in EDK to support second language learning: In the next section we examine how task-based materials design is implemented in the three phases of the task cycle.

Pre-task Learners were prompted to collect the utensils and ingredients needed to prepare a dish. They thereby became familiar with the L2 terms and also with the way in which they could communicate with the EDK. The pre-task is essential in the overall cooking task to provide a dual focus on cooking and language, which can be divided into presentation and preparation of language and cooking. As such it is a priming stage for (1) the introductory activation of existing linguistic knowledge (2) the introduction of new linguistic knowledge (3 making salient of linguistic forms in the overall activity (4) the practical preparation of learners for the cooking activity and (5) the introduction of a cultural focus of the activity. From a learner-centred perspective, the pre-task provides the opportunity for the learners to decide for themselves what was important for them in terms of language knowledge. From the learner’s perspective, it provided them the opportunity to decide for themselves (to notice) what was important for them in terms of attention to language knowledge. The dual focus on cooking and language in the pre-task keeps the outcome of the task central to the learner. The pre-task is an activity in its own right in which there is ‘primary concern for message content (although this does not preclude attention to form), the participants must be able to choose the linguistic and nonlinguistic resources needed, and there must be a clearly defined outcome’ (Ellis 2003: 141).

Table 4.1 EDK task design components Component

Description

Variable to be manipulated Complexity

Content

Physical materials

Salience

Presentation of discrete Linguistic input is made presentation of linguistic available to provide for items comprehensible input/ output for different level of learner The physical world of the Physical world is introduced Labelling of ingredients, kitchen (kitchen, utensils, presented in the pre-task utensils, equipment equipment, ingredients) and in the during task through visual supports Linguistic and procedural input in the recipes

Activity

Pre-during-post task activity

Learners

Ability, needs, motivation

Procedure Presentation of appropriate procedural input in order to cook the dish (logic of recipe) Learners are supported through the recipe in terms of what physical items are needed and when Pre-during-post task provide a specific structure to the procedure

Pre-task: supports Pre and during task presentation of linguistic make linguistic items and procedural salient post-task allows preparation for the for reflection and during task consolidation During-task: Supports are provided to enable comprehensible input/ output Support for different ability Technological design makes Support for different ability level of learner to attend salient that support is level of learner to attend to learner need in terms of available for different to learner need in terms language analysis for the learners and immediate of cooking completion of completion of the task feedback is available from the task the kitchen

Technology behind Digital Kitchen

91

Video presentation The first activity of the pre-task is to engage with preparation for cooking, at which stage the notion of culture could also be introduced. The materials can include a video presentation in L2 which includes information about the background to the recipe to be made, including its cultural/linguistic origin. The language to be used in the clip can reflect the exact nature of the instructions to be given by the audio instructions in the during-task. This can be combined with a practical demonstration of the dish to be made (similar to a cooking programme on television).

Collection of ingredients and equipment Based on findings from the previous FDK project, the pre-task for EDK makes more use of the technological and real-world context of the kitchen to involve an interactive collection activity where pairs work together to gather the ingredients and equipment needed for the cooking task. As such, this also becomes the first time the learners engage with the kitchen as a ‘participant’ in the task, where the kitchen provides audio instructions, appropriate feedback and support to the learners. Figure 4.8 shows the underlying structure of this activity:

Figure 4.8 Pre-task activity outline with supports

92

Paul Seedhouse

During-task The aims and rationale for this phase are: presentation of step-by-step cooking instructions for the preparation of a culturally relevant dish; provision of opportunities for the meeting of new and existing language knowledge (noticing); provision of opportunities for the analysis of new and existing language knowledge (processing). The rationale behind these sub-aims has been outlined earlier. It was also described how the manipulation of task components (table  4.1) could facilitate these opportunities. This section shows how the discussion of components, principles in TBLT and materials design have been ‘translated’ into the practical integrated technological-pedagogical design of EDK’s materials. In the during-task, opportunities for meeting and analysing new and existing language knowledge are supported through the integrated technologicalpedagogical design. In the EDK project, the system prompted learners to move step by step through a set of cooking actions. After an initial instruction in L2, a range of supports were available to them, if they chose. This meant that the system could adapt to the needs of learners from beginners to advanced level in L2, with the beginners using the most help facilities and advanced learners using the fewest. Learners were therefore in control of how to resolve any language trouble themselves and select the level of help they needed. The level of support moved from a slower audio repetition with emphasis, to a photo of the object with written text (figure 4.10) through to a video of the action being carried out, again with the audio (figure 4.11). This upwards escalation means that the EDK can adapt to the real-world self-determined needs of the learners at a specific moment. Figure  4.9 offers a visual representation of how the during- task has been designed in the materials: After the provision of an initial instruction in the L2, a range of supports are available to support the learner in their completion of the step, should they need them. The help facilities were made available in three different ways: 1. User-controlled ‘back’ and ‘forward’ functions allowing learners to move back and forward through the recipe stages if required, as well as requesting repeats of an instruction (both accessed via the GUI). 2. Help available indicators (‘do you need any help?’) prompted by timeouts within the EDK system. 3. After the indication that help is available, user-controlled help requests were made available in gradation. Help 1 provided repetition with stress;

Technology behind Digital Kitchen

93

Figure 4.9 During-task activity outline with supports

Help 2 showed photo with text (figure 4.10); Help 3 showed video plus text (figure 4.11). Each of these cumulatively decreased linguistic complexity and could be accessed by either the GUI or the red tool. A final language support is the automated provision of tips or phrases about cooking techniques to provide advanced learners with more complex language along with information relevant to cooking. These supports were designed to assist learners both with previously unknown language and with cooking-relevant actions, reflecting a balance between controlled (explicit focus on language) and automatic (focus on task meaning) performance. So the escalation of support offered different levels of control in their attention to language. The audio-visual supports were designed to assist the learner when they meet new items and analyse them to ensure the materials provided comprehensible input and encouraged noticing activity. They helped scaffold learners when instructions activated existing linguistic knowledge. The real-world nature of the task meant that audio-visuals were particularly suited to scaffolding learning of both linguistic knowledge and where relevant, cooking actions and technique. The audio-visuals were intended as multimedia glosses (Bowles 2004; Mohsen and Balakumar 2011). For the language learner, these glosses functioned like a

94

Paul Seedhouse

Figure 4.10 Help 2 multimedia gloss in the pre-task phase

Figure 4.11 Help 3 multimedia gloss video in the during-task phase

Technology behind Digital Kitchen

95

picture dictionary and provided different modalities (textual, visual and auditory) and modes (video, picture and text) to suit their individual learning styles and strategies. The use of multimedia glosses can initiate information processing through the twin systems of ‘text and sounds’ and ‘pictures or objects’. Instead of increasing cognitive load, information accessed via these two systems activates the other and therefore eases the cognitive demands (Mohsen and Balakumar 2011).

Post-task The sub-aims and rationale for this phase are:  provision of opportunities to reflect on language knowledge (noticing and uptake); provision of opportunities to reflect on the task outcome (prepared dish); provision of opportunities to evaluate the task and the technology. In TBLT methodology, the post-task phase can be used in many different ways to fulfil the teaching and learning objectives. Learners are invited to reflect and report on the linguistic items they noticed and learnt during the pre- and during- phases of the task. As the task outcome is the prepared dish, the post-task involves an evaluation of the dish produced by the learners – put simply, a ‘tasting session’. Task evaluation would involve asking how the learners experienced the kitchen technology, the activities and their general learning experiences. This kind of information is invaluable to future materials design, particularly from the point of view of human-computer interaction.

Conclusion In this chapter, we have seen how the technology to deliver the EDK learning environment was designed. The technological components of the pervasive system consist of the sensors, the tablet with its graphical user interface and the interaction tools. The authoring tool allows materials to be uploaded onto the system. The principles behind the technology were described, and the TBLT principles underlying materials design for individual languages and recipes were elucidated. Figure 4.12 below summarizes the materials design process for the EDK.

96

Paul Seedhouse

Figure 4.12 Summary of materials design activities

Note All figures and tables were produced by the Newcastle University team. The chapter is based on materials produced by all members of the Lancook team – see acknowledgements.

Technology behind Digital Kitchen

97

References Bowles, M. A. (2004), ‘L2 glossing: To CALL or not to CALL’, Hispania, 87 (3): 541–552. Ellis, R. (2003), Task-Based Language Learning and Teaching. Oxford: Oxford University Press. Maclaughlin, B., T. Rossman and B. McLeod (1983), ‘Second language learning: An information processing perspective’, Language Learning, 33: 135–158. Mazur, J. (2004), ‘Conversation analysis for educational technologists: Theoretical and methodological issues for researching the structures, processes and meaning of on-line talk’, in D. H. Jonassen (ed), Handbook for Research in Educational Communications and Technology, 1073–1098, Mahwah, NJ: Lawrence Erlbaum. Mohsen, M., and M. Balakumar (2011), ‘A review of multimedia glosses and their effects on L2 vocabulary acquisition in CALL literature’, ReCALL, 23 (2): 135–159. Nunan, D. (1989), Designing Tasks for the Communicative Classroom. Cambridge: Cambridge University Press. Olivier, P., X. Guangyou, M. Monk and J. Hoey (2009), ‘Ambient kitchen: Designing situated services using a high fidelity prototyping environment’, in PETRA 09: Proceedings of the 2nd International Conference on Pervasive Technologies Related to Assistive Environments, 1–7, New York: ACM Press. Preston, A. (2012), ‘Step-by-step guide to Lancook materials production (template)’, Unpublished manuscript, Newcastle University. Preston, A., M. Balaam, P. Seedhouse, S. Kurhila, L. Kotilainen, A. Rafiev, D. Jackson and P. Olivier (2015), ‘Can a kitchen teach languages? Linking theory and practice in the design of context-aware language learning environments’, Smart Learning Environments, 2 (9) doi:10.1186/s40561-015-0016-9. Schmidt, R. W. (1990), ‘The role of consciousness in second language learning’, Applied Linguistics, 11 (2): 17–46. Seedhouse, P., A. Preston, P. Olivier, D. Jackson, P. Heslop, T. Plötz, M. Balaam and S. Ali (2013), ‘The French Digital Kitchen: Implementing task-based language teaching beyond the classroom’, International Journal of Computer Assisted Language Learning and Teaching, 3 (1): 50–72. Shavelson, R., and P. Stern (1981), ‘Research on teacher’s pedagogical thoughts, judgements, decisions and behaviour’, Review of Educational Research, 51 (4): 455–498. Skehan, P. (1998), A Cognitive Approach to Language Learning. Oxford: Oxford University Press. Weiser, M. (1996), ‘Internet Rock’n Roller charts digital future at internationally heralded research center’, Xerox, 14 August. Available online: http://www.ubiq.com/weiser/weiserannc.htm (accessed 24 May 2016). Willis, J. (1996), A Framework for Task-Based Learning. Harlow, UK: Longman.

5

The Human Viewpoint and the System’s Viewpoint Natacha Niemants and Gabriele Pallotti

Introduction This chapter describes how the same immersive language learning session is ‘seen’ from two different points of view: the human users’ and the digital system’s. Given the complexity of human-computer interaction data, whose transcription requires integrating verbal and non-verbal activities by quite different sorts of ‘participants’, the chapter will start by problematizing such a secondary source of data, reflecting on why, how and what to transcribe. Different transcription formats will be evaluated, including parallel columns and score-like representations, as are made possible with the software ELAN. Such integrated transcripts can yield interesting insights into the system’s working and contribute to an understanding of what type of ‘communication’ takes place in a EDK cooking session, with implications for redesign and technology development. This chapter demonstrates that a session in an immersive digital environment can be best represented by reporting how both humans and the system access the ongoing situation.

Conversation analysis and human-computer interaction Several chapters in this book use Conversation Analysis (CA) to analyse the interactions taking place in the European Digital Kitchen’s (EDK) cooking sessions. This chapter addresses some general methodological issues bearing on transcription and analysis, and on future research on human-computer interaction from a CA perspective, by taking some examples from the Italian Digital Kitchen (IDK).

100

Natacha Niemants and Gabriele Pallotti

Conversation Analysis was initially developed as a radical form of microsociology, drawing on Garfinkel’s ethnomethodology and on Goffman’s analysis of everyday encounters (for an overview of CA, see Sidnell and Stivers 2013). Its approach is radically antimentalistic, as it aims to describe the social, public methods, procedures and practices that people follow when interacting with one another. The main focus is on concrete episodes, situated in their particular social contexts: more general phenomena are discovered and described at a later stage, and in any case always inductively, beginning with the observation of concrete cases and never through the lenses of some a priori theory on human action. This approach is often deemed ‘emic’, using a term first introduced by Pike (1967) to generalize to human sciences as a whole the distinction between phonetics, whose aim is to develop general categories for distinguishing sounds across all human languages, and phonemics, which is concerned with describing the phonological contrasts that are peculiar to each individual linguistic system. Emic has thus taken the meaning of local, particular, internal to a given sociocultural reality. For example, in ethnography emic is often used as a way of describing how ethnographers strive to reconstruct people’s points of view, attitudes and interpretive categories, in order to achieve ‘sufficient identification with or empathy for the members of the group’ (Duranti 1997: 85). In CA the term takes a different meaning, while keeping the original idea of orienting to the local contexts rather than using abstract and generalized interpretive categories. According to Markee and Kasper (2004: 495, emphasis added), ‘CA establishes an emic perspective not by interviewing research participants, but by examining the details of the “procedural infrastructure of situated action” (ten Have 1999: 37), specifically, the orientations and relevancies that participants display to each other through their interactional conduct (Schegloff 1992)’. Emic thus means paying attention to how participants publicly display to each other their orientations and relevancies, and how all this belongs to the realm of social action rather than to private mental worlds. Many conversationanalytic works consist in micro-analyses of how people locally achieve coordination by publicly exhibiting how they are interpreting ongoing actions. Thus, according to Mori (2007: 853), CA ‘attempts to reconstruct how the participants themselves are analysing their co-participants’ contributions in a moment-bymoment fashion and reflects their analysis in the ways in which they design their own contributions’. This passage raises one of CA’s central methodological issues, regarding the criteria whereby one can assess the validity of analyses by a researcher who is

Human Viewpoint and the System's Viewpoint

101

outside the action and who ‘attempts to reconstruct’ and ‘reflect’ the participants’ own analyses. Seedhouse (2004: 195, emphasis added) makes this even more explicit, when he writes that, in analysing pedagogic interactions, ‘the analyst follows exactly the same procedure as the participants and traces the evolving relationship between pedagogy and interaction, using as evidence the analyses of this relationship which the participants display to each other in their turns’. The basic idea comes from ethnomethodology’s notion of reflexivity, implying that participants and analyst share the same set of ethno-methods for describing social actions, with the latter having no privileged status. However, in our case, several questions arise. How can we reconstruct how a machine analyses a situation? And can the machine be considered to be one of the participants? If so, can its actions be interpreted through the same categories we use to interpret human behaviour? And can all these interpretations be interwoven in the analysis of a single, integrated course of action? On a purely descriptive level, this does not seem to pose any special challenges: the machine’s turns and actions can be reported like everything else happening on the scene, including turns and actions produced by human beings. It is immaterial whether some verbal utterances coming out of a loudspeaker or some pictures displayed on a screen were produced by a computer or by a person speaking behind a microphone or typing on a keyboard:  they are just there as ‘facts’. Problems arise when one moves on to interpreting these manifest actions. One of CA’s key questions is in fact ‘why that now?’ (Schegloff and Sacks 1973), where the ‘why’ implies an explanation, an interpretation, no matter whether mentalistic or not, but in any case going beyond the simple description of what is happening. CA’s answer, as we have seen, would be that human behaviour can be interpreted based on social practices and ethno-methods that the analyst shares with the participants, having both undergone similar learning and socialization processes. But how is the question ‘why that now?’ to be answered when the behaviour was produced by a machine? In the case of a machine with a deterministic programming logic, like the EDK, the answer is rather simple: the machine has received a given input which, according to an action scheme written in the programme, has led to producing a given output. Knowing the input and the programme, the machine’s behaviour, or output, is entirely predictable and explainable. Things would be more complex in the case of a machine based on a self-learning neural network:  in these cases the computational mechanisms occurring between input and output are not entirely transparent, even for the developer, and the production of a given output is never completely

102

Natacha Niemants and Gabriele Pallotti

predetermined, but depends on probabilistic logics introducing an element of unpredictability. But the fundamental question remains the same:  both with deterministic programmes and with neural networks, what is the nature of the difference between human and machine behaviour? Is the difference merely quantitative, so that human behaviour would just be produced by extremely complex circuits, much more complex than those of any existing computer, but not qualitatively different from those of any simple circuit with three cables and two switches? If so, once the underlying programming circuits were discovered, it would be possible in principle to reduce all human behaviours to deterministic inputoutput processes. This is the assumption behind all reductionist programmes, including some forms of behaviourism, whose goal is to reduce human behaviour to a mechanics of actions and reactions. Alternatively, one might postulate a qualitative, ontological difference between human and machine behaviour. Four centuries ago, Descartes already maintained that there are two fundamental substances, the res extensa, that is, physical matter, of which machines (and animals!) are made, and res cogitans, a thinking matter that pertains to human beings alone. As one can see, the question is centuries old and it is still far from being settled. Among the many reformulations of this old question, the one most concerning us here is whether one can really talk of interaction between humans and machines, and whether the latter can be considered to be ‘participants’ in interactions. Button and Sharrock (1995), for instance, strongly doubt that it is possible to have a conversation with a computer, but that between humans and machines there can only be ‘simulacrums of conversation’. For Suchman (2007:  23), too, ‘interactivity as engaged participation with others cannot be stipulated in advance but requires an autobiography, a presence, and a projected future. In this strong sense, I would argue, we have yet to realize the creation of an interactive machine’. However, Suchman (2007: 38) herself notes that if a machine exhibits minimal linguistic behaviours, human beings tend to attribute to it much higher linguistic competences. Furthermore, ‘insofar as the machine is somewhat predictable [ . . . ] and yet is also both internally opaque and liable to unanticipated behavior, we are more likely to view ourselves as engaged in interaction with it than as just performing operations on it or using it as a tool to perform operations upon the world’ (42). Thus, leaving aside philosophical reflections about whether a machine exhibiting not completely predictable linguistic behaviours can ‘really’ interact and can be considered to be a ‘true’ participant, it is a fact that human

Human Viewpoint and the System's Viewpoint

103

beings tend to treat it as another participant and see it as capable of interacting with them. If one adopts an emic viewpoint reconstructing the participants’ perspective, this fact cannot be ignored. From a strictly practical point of view concerned with improving the machine design, too, the programmer must take into consideration the users’ attitudes towards the machine, which often involve treating it as an interlocutor, talking to it, trying to interpret its behaviours and making oneself understood, and getting angry if it does not understand. If it thus seems legitimate – at least on a practical level of technical design and on a methodological level of analysis – to consider the machine as a participant, the question arises as to how this participant’s behaviour can be described and interpreted. In order to provide a situated account of interactions, based on local contingencies that emerge moment-by-moment, one needs to describe ‘situations’ as they present themselves to participants. Clearly, humans and machines have different representations of the same situation. Quoting once again Suchman (2007: 12), in her pioneering work analysing how users interacted with an expert system giving instructions on how to use a photocopier, she wrote that ‘the machine could only “perceive” that small subset of the users’ actions that actually changed its state. [ . . . ] It was as if the machine were tracking the user’s actions through a very small keyhole and then mapping what it saw back onto a prespecified template of possible interpretations’. Such a characterization fits very well with what happens in the EDK and can be taken as a starting point for this chapter, which aims to describe in a parallel way the situations as they are accessible to humans and the machine, in order to then analyse how their actions deploy moment-by-moment. This description has to rely on a representation format which, as in all CA studies, consists in transcribing the primary data sources, which in our case are video recordings. In the next sections we will see that transcription, too, is not a neutral activity and that it requires to make choices with theoretical and methodological consequences.

Problematizing transcription As detailed in Chapter  4 and elsewhere in the book, the main data for the research presented in this book consists of sensor logs and video recordings of cooking sessions. Clearly neither the sensor logs nor the video recording are the past communicative event one aims to analyse, but they are both primary sources of data, which could theoretically be matched to get a clearer picture

104

Natacha Niemants and Gabriele Pallotti

of the real object of analysis. Unfortunately, however, that is hardly the case in practice, since video recordings are complex and difficult to analyse per se, so that written transcripts are generally required to facilitate analysis and dissemination, as well as the verification and replication of theoretical statements. But just as recording selects elements from the observable event, transcription selects elements from the recording, being the result of interpretive choices (i.e. what to transcribe) and representational ones (i.e. how to transcribe), and ultimately depending on the research objectives (Niemants 2012; 2015). Given the complexity of our human-computer interaction data, where transcribing is complicated by the need to represent overlapping verbal (humans’ and system’s words) and non-verbal (humans’ and system’s actions) activities, it may be worth problematizing such a secondary source of data, reflecting on what could be ‘lost in transcription’ (Rosenthal 2009) and on what could alternatively be preserved to be observed through specific analytical lenses. The literature on human-computer interaction offers little guidance on transcribing, as researchers often deal with written sources of data (e.g. logs and chats) and they seldom wonder whether – and how – to integrate different data sources into one single written record. In this chapter, we would like to argue that integrating different data sources into a single transcript is truly desirable and actually feasible: we will start by explaining why such an integrated transcription could be worthwhile, how that may practically be done and what could be transcribed. The answer to the first question (why transcribe?) is straightforward: transcription is worth doing because it is a way of ‘doing data’ (Ayaß 2015) and of ‘resurrecting’ things that would otherwise go unnoticed and that could hardly be studied (Cencini and Aston 2002). While acknowledging that there is no one-to-one correspondence between the events that unfold during humancomputer interaction and what a researcher hears/sees and transcribes, a written transcript ‘preserves the data in a more permanent, retrievable, examinable, and flexible manner’ (Lapadat and Lindsay 1999: 80) and it is ‘not simply a way for a researcher to capture, represent, or “re-present” talk, but a constructive and interpretive act in which the researcher positions him/herself ’ (Lapadat 2000: 209). Although time- and energy-consuming, transcribing is thus worth the effort, and practical examples of what can be noticed (and hence studied) with the aid of transcripts provide a further argument in favour of this written approach, which also promotes familiarity with the data and fosters the methodological and theoretical thinking that is at the basis of data interpretation.

Human Viewpoint and the System's Viewpoint

105

The answer to the second question (how to transcribe?) first has to do with the choice of spatial arrangement for data display, be it a vertical format as in playscripts, a column format where each column corresponds to one speaker, or a multi-tier format like a musical score (Edwards and Lampert 1993: 11). While vertical formats have probably been the most widely used in orthographic (i.e. non-phonetic) transcriptions of audio data over the past decades, especially in CA, multi-tier formats seem better equipped to represent video-recorded data, since multiple tiers can be used to capture the simultaneous dimensions of communication (Antinoro-Pizzuto et al. 2010). In the next section, we will discuss these different options in relation to our specific type of data. After choosing the spatial arrangement, one also has to decide whether to use a specially developed transcription tool, and if so, which one. Although some researchers still prefer to rely on separate software tools to manage the recordings, produce transcripts and analyse logs, there is a case for working in a single interface where the transcript of verbal and non-verbal features acts as a dynamic index of the recording, and is displayed in synchrony with the video, as this gives instant access to both the secondary and the primary data source. For some of the analyses in this chapter we illustrate one such interface called ELAN. The answer to the third question (what to transcribe?) is based on an interpretive process, where selection appears to be the first issue involved. Just like a good map maker, the researcher should determine ‘what to miss out rather than what to include’ (Cook 1995: 45) and, as will become apparent in the next section, criteria for exclusion generally reflect the research ideology and methodology. It shall be stressed here that the transcription conventions developed within CA (e.g. Jefferson 2004), which were born in the pre-digital era, aimed to account for all aspects of oral communicative behaviour that can cast light on what participants are constructing with their talk-in-interaction, although this representation can never be exhaustive. Various levels of granularity have been proposed, depending on the research aims and on the availability of the primary data:  these may range from narrative accounts of the communicative event to the representation of (para)linguistic, interactional, prosodic and multimodal features (Jenks 2011). More precisely, linguistic and paralinguistic features concern the words pronounced by different participants, as well as nonverbal vocalizations – laughter, coughs, audible inhalations, exhalations and the like. Interactional features concern when and by whom things are uttered: turnsat-talk may be produced one after the other or they may overlap, or there may be pauses between (or within) turns. Prosodic features include tempo (faster/ slower), volume (louder/softer), pitch (higher/lower), duration (lengthening/

106

Natacha Niemants and Gabriele Pallotti

truncation) and intonation (rising/falling). Finally, multimodality subsumes gaze, gesture and body language, which may sometimes be largely ancillary to utterances in speech, but at other times may play a primary role, as for instance, in second and foreign language talk, where speakers may make use of a number of non-verbal articulators, such as hands, head, face and body, to compensate for their language deficits. In the case of the system’s sensor logs, the selection process is two-fold:  a first selection occurs when the system is programmed to track certain pieces of information only, while a second selection occurs when the transcriber decides which of all these pieces to represent in his/her transcript by integrating different data sources. As we will see in the following sections, this second selection is always theory-laden, since the analytical lens one opts for inevitably determines what to miss out.

Describing interactions in/with the European Digital Kitchen: A simple example As the previous section made clear, interactions can be described in many ways, and one of the aims of this chapter is to discuss several options.

Describing the event’s macro-structure A first level of description is that of the events’ macro-structure. From this point of view, the Italian cooking sessions broadly followed the same protocol as the English ones with a few differences (some of which will be further discussed in Chapter 9). The first difference that is worth mentioning here is that while in England cooking sessions took place in a real kitchen, in Italy they were organized in a variety of settings using a portable hotplate, which made the kitchen completely mobile. A second difference is that cooking sessions in the UK mainly involved one single recipe, scones, while Italian participants were divided into two groups following two different recipes, pasta with olives and capers (Recipe 1) and meat rolls in tomato sauce (Recipe 2). Interactions in the IDK took place during the pre-task and the main-task phases: as is often the case on cooking-themed programmes on TV, the users started by collecting the ingredients needed to make the recipe (pre-task), then they moved step by step through the recipe and prepared the dish (main task). Both phases were divided into steps and programmed as such in the authoring

Human Viewpoint and the System's Viewpoint

107

tool, but while the number of steps in the pre-task was the same across recipes (10), their number in the main task depended on the complexity of the chosen recipe, and amounted to 30 steps in Recipe 1 and to 21 steps in Recipe 2.

Describing the interaction in CA terms A second level of description concerns the micro-analysis of the actions unfolding in each cooking session, reporting ‘what is it that’s going on here’ (Goffman 1974), as is the standard practice in CA and some forms of Discourse Analysis. Using some analytic categories from these approaches, one may say that the basic interactional sequence in the EDK’s cooking sessions was the triadic format initiation – response – feedback (or follow-up) (IRF), which typically appears in educational contexts (Seedhouse 2004). In our case, the system takes the teacher’s role, who initiates instructional sequences by uttering a command or instruction. Human participants take the pupils’ role, by following the instruction. When this is the case, the kitchen provides a positive feedback, in the form of a characteristic success sound followed by a verbal utterance randomly chosen among a repertoire of sentences like well done, good job, that’s ok. If users do not perform the expected action, the kitchen starts a repair sequence, which in CA terms can be seen as a form of self-initiated self-repair (Schegloff et al. 1977). The system first asks whether help is needed, with a characteristic sound associated to this action. Learners can ignore the help offer, and the kitchen will not take any further action. If learners decide that they do need some help, they can request it by shaking the red tool or pressing the Help button on the GUI interface. Help is provided in three forms with increasing explicitness: 1. Help 1: a slower verbal repetition of the kitchen’s instruction; 2. Help 2: a verbal repetition (at normal speed) of that same instruction with an accompanying picture and caption; 3. Help 3: a verbal repetition (at normal speed) of that same instruction with an accompanying video. If the first help message is not enough, learners can ask for the second and third one, by shaking the red tool or pushing the Help button. In a sense, what they are doing might be called, in CA terms, other-initiated self-repair, in that the system repairs its problematic utterances upon request by the users. In sum, the whole repair sequence is a hybrid between self-initiated and other-initiated selfrepair: it is self-initiated by the kitchen with its first offer for help, but then it

108

Natacha Niemants and Gabriele Pallotti

is in the users’ hands, who take the initiative of asking the system to provide more explicit formulations when they feel the need. We might call this peculiar sequence self-initiated + other-initiated self-repair. These repair sequences could also be described in terms of negotiation of meaning, following the terminology of cognitive-interactionist SLA research (e.g. Mackey et al. 2012). When users have trouble in understanding the meaning of words uttered by the kitchen, they can make some ‘clarification requests’ by shaking the red tool or pressing the Help button. Perhaps ‘negotiation of action’ would be a more appropriate characterization: kitchen and users negotiate what action is to be performed, and the sequence ends when it is correctly understood and realized. The project’s aim was to create a system that would be perceived by users as being able to interact in an ‘intelligent’ manner, similar to the way human beings do, that is, based on the production and negotiation of meanings and not on a simple action-reaction mechanics. However, anticipating potential difficulties by the learners (or technical defaults in the sensor system, as shown in extract 3), a further possibility was introduced, which allowed direct control by the users through two buttons on the touch screen that made the programme advance to the following step or return to the previous one. In such cases, the (simulation of) interaction between humans and system is lost and one falls back on a more traditional modality where the user directly controls the programme by simply pressing buttons. Extract 1a shows a very simple episode containing the triadic IRF sequence, initiated by the kitchen (KIT) and brought to an end by one of the participants, Learner 1 (S1) and Learner 2 (S2). The video of this and the following extracts can be retrieved from the LanCook website at http://www.europeandigitalkitchen.com. Extract 1a: Pair 04, Recipe 1, Step 14 01

03

KIT poi coprite then cover coperchio e lid and S1

04 05 06 07

S2 okay (4.0) ((L1 takes lid and puts it on pot)) S2 °oh° KIT ((success sound))

02

la pentola con il the pot with the [aspettate ]che l’acqua bolla wait for the water to boil [°coperchio°] lid

Human Viewpoint and the System's Viewpoint 08 09

109

(2) ((audience laughs)) KIT ben fatto. andiamo al passo successivo well done. let’s move on to next step

At line 1 the kitchen requests the pot to be covered with the lid and at line 4, S2 produces a minimal acknowledgment token with her okay. At line 5, S1 performs the required action and at line 07, the kitchen produces positive feedback; first with a success sound, which the audience receives with laughter, then with some words announcing the ensuing step (line 9). This description reproduces an analytic approach typical of Conversation and Discourse Analysis, the only difference being that one of the ‘participants’ here is a system. This account represents what is perceived by human beings watching the video (and, presumably, those on the scene as well), or, in other terms, what is accessible to them.

Reporting the system’s sensor logs A third level of analysis is also possible, based on what the system ‘perceives’ and is thus accessible to it. Two sources of information can be used to this end. On the one hand, we have a log of the sensors that are being activated every time the objects they are connected to move. We might call this, metaphorically, a record of the system’s perceptual system. On the other hand, we also know the programme making the kitchen work, which in a sense corresponds to its processing mechanisms. Given that the technology behind the EDK has already been thoroughly dealt with in Chapter 4, we will only briefly recall here some details on the sensor logs. Sensor logs record what happens by detailing when sensors are active (Sensor Moving) and inactive (Sensor Not Moving) in the different steps. Extract 1b reproduces extract 1a, but this time from the system’s point of view. Extract 1b: Pair 04, Recipe 1, Step 14 1. 2. 3. 4. 5. 6. 7. 8. 9.

22/05/2013 15:51:17.351|STEP 14|START STEP 22/05/2013 15:51:28.768|STEP 14|SENSOR MOVING|bigger lid 22/05/2013 15:51:29.543|STEP 14|SENSOR NOT MOVING|bigger lid 22/05/2013 15:51:29.803|STEP 14|SENSOR MOVING|bigger lid 22/05/2013 15:51:30.063|STEP 14|SENSOR MOVING|bigger pot 22/05/2013 15:51:31.093|STEP 14|SENSOR NOT MOVING|bigger lid 22/05/2013 15:51:31.353|STEP 14|SENSOR NOT MOVING|bigger pot 22/05/2013 15:51:34.168|STEP 14|SUCCESS|SENSORS 22/05/2013 15:51:42.360|STEP 14|NEXT STEP|SUCCESS

110

Natacha Niemants and Gabriele Pallotti

Line 1 (Start Step) corresponds to the production of the instruction ‘then cover the pot with the lid and wait for the water to boil’. The programme’s instructions detail how the kitchen will react to various combinations of sensor movement/ non-movement. Normally, the kitchen considers an action to be correctly performed when the object mentioned in the instruction is moved and then stops moving. In our case, at line 2 the lid sensor starts moving and stops after about one second, at line 3; the lid moves again at line 4 and stops at line 6, after about 1.3 seconds. This sequence of movement/non-movement is interpreted by the programme as a correct performance of the action, which leads to producing the success sound and the congratulation sentence (line 8). After a few seconds, the system records that the sequence for step 14 has been successfully accomplished, which makes it ready to move on to step 15 (line 9). It is important to note that the kitchen also records the pot’s sensor movement (lines 5 and 7), which however does not produce any consequence: in other words, only the movement of the ‘expected’ sensor is taken to be relevant for triggering the success sequence, while the movement of unexpected sensors is ignored. If the kitchen systematically reacted to the movement of all sensors, for example, by giving negative feedback like ‘don’t move the pot’ or ‘you’re moving the wrong object’, this would produce a number of false positives, given that sensors often move because of accidental shocks or because learners are still engaged in performing actions from previous steps (e.g. see extract 4). Kitchens are places where many unexpected strong vibrations and noises occur. In addition to sensor movements, the logs also record the help offered by the kitchen, detailing (1)  the moment when some help becomes available (2) the moment when the users request that help (in this case by shaking and placing down the red help tool) and (3)  the moment when that same help is actually offered by the system (and presumably taken by the user), as can be seen in extract 2 where (1) occurs at line 1 (2) at lines 2 and 3, and (3) at line 4. Extract 2: Pair 20, Recipe 2, Step 2 1. 2. 3. 4.

17/10/2013 09:14:38.346|STEP 2|HELP 1 AVAILABLE 17/10/2013 09:14:59.556|STEP 2|SENSOR MOVING|*HELP* 17/10/2013 09:15:01.596|STEP 2|SENSOR NOT MOVING|*HELP* 17/10/2013 09:15:01.631|STEP 2|HELP 1 TAKEN|TOOL

Human Viewpoint and the System's Viewpoint

111

Developing an integrated transcription format The previous pages have shown how the same very short episode (extract 1) can be represented both from the human participants’ and the system’s point of view. How can these representations be combined in a unified transcription? As discussed earlier, transcription choices have consequences that are not merely practical, but also theoretical and methodological. An intuitive option would be placing the sensor log next to the standard ‘vertical’ transcription, which would produce something like extract 1c (in the interest of space, in the sensor log’s column we won’t report step number and date and minute, but just seconds and milliseconds). Extract 1c: Pair 04, Recipe 1, Step 14

17.351|START STEP 01

04 05

KIT poi copr ite la pentola con il then cover the pot with the coperchio e [aspettate ]che l’acqua bolla lid and wait for the water to boil L1 [°coperchio°] lid L2 okay (4.0) ((L1 takes lid and puts it on pot))

06

L2 °oh°

07 08 09

KIT

02 03

KIT

((success sound)) (2.0) ((audience laughs)) ben fatto andiamo al passo successivo well done let’s move on to next step

28.768|SENSOR MOVING|bigger lid 29.543|SENSOR NOT MOVING|bigger lid 29.803|SENSOR MOVING|bigger lid 30.063|SENSOR MOVING|bigger pot 31.093|SENSOR NOT MOVING|bigger lid 31.353|SENSOR NOT MOVING|bigger pot 34.168|SUCCESS|SENSORS

42.360|NEXT STEP|SUCCESS

This transcription format has some advantages. First, the two viewpoints are put side by side, which allows a more intuitive reading of what happens in parallel. Second, the left column preserves the standard transcription format similar to a theatre script, which is easily readable. Finally, this form of representation is rather compact from a typographical point of view. This format, however, has some disadvantages, too. The vertical format of speech representation works rather well with theatre or cinema scripts,

112

Natacha Niemants and Gabriele Pallotti

where usually people speak one at a time and where few indications are given about gestures accompanying speech. When transcribing real interactions, it is suitable for telephone conversations, which involve just two speakers and where it is not necessary to report the non-verbal channel; simultaneity can be easily represented by using square brackets to indicate overlap beginning and end, as is done here at lines 2 and 3. However, when there are more than two speakers, or when one wants to also report non-verbal actions occurring alongside speech, this format soon proves inadequate. In fact, consecutive lines may represent both subsequent and simultaneous phenomena, and square brackets may become three or more in multi-party interactions, which substantially complicates reading. Furthermore, it is not clear where non-verbal actions should be represented, whether in a separate line (which however multiplies the problems just mentioned), or next to verbal turns, for example, in double brackets, as is the case in the Jeffersonian transcription format commonly used in CA (Sidnell and Stivers 2013). The latter solution, albeit practical for reducing line numbers, represents non-verbal communication as secondary with respect to linguistic production, which many find highly questionable from a theoretical point of view. Moreover, the simultaneity of verbal and non-verbal actions is represented approximately, with no indication about the start and end points of gestures, and about how they are temporally related to words. It should be emphasized that these criticisms have been raised several times with respect to human interaction and are thus not limited to humancomputer communication. A possible solution would be to use parallel columns, each representing a different speaker and a different communication level, verbal and non-verbal, adding in our case a column for the system’s sensor logs and another one for its ‘actions’. The result would be a table with many columns, whose representation on a printed page would be almost impossible. As we have seen in the previous section, students of multimodal interaction have addressed these issues by developing alternative transcription formats looking like musical scores, with a horizontal timeline on (or under) which various tiers can be added, each representing a participant and his/her verbal and non-verbal actions. Our example would thus be represented as in Extract 1d. Here too, to save space, we will only report the objects whose movement was recorded in the sensor log, with the abbreviations SM  =  sensor moving; SNM = sensor not moving).

Human Viewpoint and the System's Viewpoint

113

Extract 1d: Pair 04, Recipe 1, Step 14 KITwords/ actions KITactions

poi la coprite pentola

con il e l’acqua coperch aspetta bolla io te che

START STEP

KITlog S1words

[°coper chio°]

S1actions S2words S2actions

nods

CLASSwords time

17

18 19

20

21

22

23

KITwords/ actions KITactions KITlog

SM big lid

SNM big lid SM big lid

SM big pot

S1words S1actions

nods

S2words

°okay°

L1 takes

big

lid and

puts it on °oh°

S2actions

looks at L1

CLASSwords time

24

25

26

27

KITwords/ actions KITactions KITlog

28

29

30

success sound SUCCESS SNM big lid SNM big pot

S1words S1actions

big pot resting position

smiles

smiles smiles

114

Natacha Niemants and Gabriele Pallotti

S2words S2actions

resting position

CLASSwords time

31

32

KITwords/ actions KITactions

ben fatto

33

an

34

diamo al passo suc

smiles

smiles smiles

laugh

laugh

laugh

35

36

37

cessivo NEXT STEP SUCCESS

KITlog S1words S1actions S2words S2actions CLASSwords time

38

39

40

41

42

The first three lines concern the kitchen, which was placed first to represent its initiating role in the cooking sessions. The first line reports the words, sounds and images (‘actions’) externally produced by the kitchen and thus perceivable by human beings. The second line contains the kitchen’s actions in the language of its log: some correspond to hearable and visible actions, others to steps occurring in the programme with no corresponding manifest action, as for example Next Step| Success which corresponds to the completion of the Step 14 sequence, which triggers the advancement to the following Step 15. The third line finally includes the system’s perceptions of sensors moving and resting. The following lines concern human participants, one for each participant containing words and another with non-verbal actions, plus a line for the audience (that in our recordings could generally be heard but not seen). The last line displays time and represents an objective reference point to which all other events can be collocated. Sensor logs report events with a precision of milliseconds, and it was often the case that human words and actions fell halfway between one second and another. Here, for simplicity’s sake, we will refer to whole seconds, unless more precision is needed. Such a format has the advantage of representing the simultaneity of different courses of action, be they verbal, non-verbal or digital, exactly placing each of them with respect to the others on a common timeline. The main

Human Viewpoint and the System's Viewpoint

115

disadvantages are that it occupies a large space on the page, that reading is difficult, especially when there are several empty lines between actions of different participants, and that it is not easy to visualize what is happening.

Combining the system’s and the human points of view when analysing interactions A simple case After discussing various ways of transcribing the simple episode in extract 1, let us now try to analyse it integrating the human and the kitchen’s point of view, following the score-like transcription, which in our opinion is the most complete and detailed one (extract 1d). On second 17, the kitchen launches the first phase of Step 14, by playing the audio file containing a verbal instruction. However, as one can see, this does not coincide with the utterance of words: in fact, recorded audio files normally began with a short silence, so that the part that can be heard by a human ear only begins at second 19. Human participants produce several receipt tokens towards the end of the kitchen’s sentence, like S1 repeating coperchio (lid) at second 22 and S2’s okay at second 24. It should be noted that the system has no way of perceiving these productions and therefore only records inactivity during this whole period of time. The beginning of S1’s gesture (moving her hand towards the lid) is not perceived by the system either, nor is the oh with which S2 displays, at second 29, acknowledgement that S1’s gesture can be an appropriate answer to the instruction received. For the kitchen, the first reaction token is recorded at second 28, when the lid’s sensor starts moving. The kitchen, on the other hand, notes some micro-details in the objects’ movements that a human observer would probably miss, like for instance the fact that the lid stops its movement for 0.25 seconds (second 29) or that the pot slightly moves when it is touched by the lid (second 30). In seconds 32 and 33 apparently nothing happens, both as regards sensors, and as regards humans, at least in a traditional, simplified transcription format, that only records a silent pause (extracts 1a and 1b). Actually, the cooks move to reach a waiting position, one with hands on her lap (S1) and the other with crossed arms (S2). It is clear that the audience here plays a role: S1 and S2 turn to the class, looking at them, thereby publicly displaying that they are waiting for a

116

Natacha Niemants and Gabriele Pallotti

feedback sign from the kitchen. In CA’s terms this might be called ‘doing waiting’, underlying the fact that even so called non-actions are actions anyway, as communicative behaviours that are potentially relevant for interaction (Sacks 1984). On second 34 the system produces the success sound, a sort of musical tadah! which prompts smiles in the main participants and clearly audible laughter by the audience. Following the programme’s instructions, the kitchen then plays an audio file containing a randomly chosen congratulation message, in this instance, ‘Well done. Let’s move to next step.’ When this is over, at second 42 the kitchen records in its log that the procedure was successful and that it can now proceed to the next step (Next Step | Success). The analysis of this short extract allows us to make some theoretical and methodological reflections. First, it is clear that the kitchen’s perception of what is happening on the scene is extremely narrow, being limited to the sensors’ movement or to buttons being pressed on the touch screen. The biggest difficulty for human beings lies precisely in understanding the kitchen’s way of perceiving the world, and that its actions follow a programme that is not accessible or transparent to them. In the next sections we will see some cases in which lack of understanding of these mechanisms produces various forms of misalignment. For human participants, the kitchen thus represents a sort of hardly interpretable black box, whose perceptual and cognitive mechanisms must be gradually discovered. On the other hand, if we take the external analyst’s perspective, the situation is completely reversed. The analyst has in fact an absolutely exhaustive record and understanding of what the kitchen does – both as externally hearable and visible actions, and as internal events and procedures. The sensor log, in fact, faithfully reproduces what the system ‘senses’ moving and thus introduces a level of representation (perception) that is usually not recorded and transcribed in CA studies. As stated earlier, one of CA’s key methodological principles is that its focus are overt behaviours, publicly displayed and available to participants and, to some extent, to the analyst, who grounds his/her analysis precisely on what is accessible to participants, trying to reconstruct their point of view from an emic perspective. This approach deliberately bans any consideration or speculation about participants’ psychological states, like their perceptual and cognitive processes. One could thus conclude that, for the analyst, participants are the real black box – whose overt behaviours can be interpreted based on a socially shared and thus public interpretive logic, but with no access to their private psychological processes – while the EDK is perfectly transparent in relation to both what it does publicly and to what happens inside it.

Human Viewpoint and the System's Viewpoint

117

More complex cases In this section, we will analyse three cases of growing complexity, where problems arise from the system’s or the human point of view, or both.

When the system does not ‘sense’ The first extract comes from a cooking session that took place in a hotel offering weekly courses of Italian as a foreign language. The session involved a pair of women with German and English as L1s, who had been studying Italian for ten years and three months, respectively. The excerpt chosen starts halfway through Recipe 2 (meat rolls) when the kitchen instructs them to take some toothpicks, and is short and simple enough to be displayed in a vertical format with two columns. Indeed, the three speakers mainly speak one at a time, non-verbal actions are limited and can be easily inserted within double parentheses, and the few logs available are shown in parallel, thereby allowing a more intuitive reading of what happens. Extract 3: Pair 12, Recipe 2, Step 21 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19

37:54.983|STEP 21|START STEP KIT adesso andate a prendere gli stuzzicadenti now go and take the toothpicks (1.0) S1 ah ((S1 looks at toothpicks while moving her right arms towards them)) ((S1 takes and opens box)) ((S1 takes out one toothpick)) S1 prendiamo quattro (.) per essere [sicuro] ((keeps on taking them)) we take four to be sure S2 [okay] questi altro, these more ((S2 looks around and indicates)) S1 ehm voilà this is it ((S1 tries closing toothpicks box)) 38:14.768|STEP 21|HELP 1 AVAILABLE KIT ((help sound)) ((S1 keeps on trying to close box)) KIT serve una mano ? do you need a hand? S1: no: ((while putting down box)) S1: ((presses touchscreen with right 38:23.310|STEP 21|NEXTSTEP| hand)) TOUCHSCREEN

In this case, what went wrong is simply that the system stopped working properly. Because of a technical fault, from step 13 onwards the central unit was not receiving any input from the sensors, but only from the touch screen. This is also apparent from the log in the right column, which only includes the system’s actions but no information about sensors moving or resting. From the

118

Natacha Niemants and Gabriele Pallotti

system’s viewpoint, thus, nothing has happened after the instruction was given, which is why help becomes available after the time span programmed in the authoring tool. If we look at the transcript, however, many things happen in those 20 seconds: not only does S1, the more advanced learner, acknowledge receipt of the instruction with the token ah and take the right object (lines 4–5), but she also takes the initiative in extracting four toothpicks (line 8), thereby showing a certain degree of autonomy with respect to the kitchen’s instruction adesso andate a prendere gli stuzzicadenti (now go and take the toothpicks). It then takes her a few seconds to close the toothpicks box, and while she is still trying to do so, the kitchen proffers its help serve una mano? (do you need a hand?), to which S1 immediately replies no. Once she has put down the box, she uses the same hand to touch the tablet and move on to the following step. This short extract allows us to make some remarks on both the system and the humans who are using it. To begin with the system, we have seen that although it is programmed to always log all the selected pieces of information, the necessary condition for that to be true is that the receiving sensor is working properly. If, for whatever reason, that sensor stops working, the system is unable to ‘sense’ anything and the logs only include things that happen in the tablet, both on the system’s and the humans’ initiative, and that can also be observed by looking at/listening to the participants’ actions. In a sense, these abridged logs are close to what a traditional CA analyst would select: all the system’s ‘perceptions’ are missed out, and only the visible/audible behaviour is included. From the humans’ viewpoint, the faulty sensor does not impede a successful cooking session: the two learners keep on responding to the system’s offers and, while partially unaware of what is going on, they easily find a way to move on to the following step anyway.

When the action is okay, but the request for help itself becomes problematic The second case also comes from a cooking session that took place in the abovementioned hotel, and is also presented using the vertical two-column format. It involves a pair of beginners, one stating she had studied some Italian ‘for a long time’ and the other declaring she had been studying it for one week only, with Norwegian and Czech as L1s. The excerpt chosen starts halfway through Recipe 1 (pasta with capers and olives) when the kitchen tells them to take the smaller pot – and displays the whole sensor log in order to make the system’s viewpoint more transparent.

Human Viewpoint and the System's Viewpoint

119

Extract 4: Pair 13, Recipe 1, Step 17

01 02 03 04 05 06 07 08 09 10 11

12 13 14 15

[KIT

ora (.) prendete la pentola più piccola (.) e appoggiatela sul fornello libero] now take the smaller pot and put it on the free burner [((while KIT talks S2 keeps on cutting basil leaves))] [((S1 looks at S2 while she is cutting leaves and touches the table spoon))] S1 ah ((looks at pot and [heads towards it))] S2 [io non capi:sco] I don’t understand

16 17 18

[((help sound))] [((takes object] and puts it in the right place)) KIT [posso essere utile ?] can I be useful ? S1 [((listens to KIT while placing the pot))] (2)

19

S1

20 21 22 23 24

KIT S1

posso essere utcan I be usS2 you can use [it] KIT [((success sound))] S1 come come ha detto ? what what did it say ? S2 you can use it S1 ah.

51.777|STEP 17|START STEP 52.570|STEP 17|SENSOR NOT MOVING|big knife 54.976|STEP 17|SENSOR MOVING|big knife 55.231|STEP 17|SENSOR NOT MOVING|big knife 59.785|STEP 17|SENSOR MOVING|big knife 02.412|STEP 17|SENSOR NOT MOVING|big knife 04.267|STEP 17|SENSOR MOVING|table spoon 04.797|STEP 17|SENSOR MOVING|table knife 05.057|STEP 17|SENSOR MOVING|big knife 05.592|STEP 17|SENSOR NOT MOVING|table knife 07.167|STEP 17|SENSOR NOT MOVING|table spoon 07.437|STEP 17|SENSOR NOT MOVING|big knife 07.977|STEP 17|SENSOR MOVING|basil 09.037|STEP 17|SENSOR NOT MOVING|basil 10.357|STEP 17|SENSOR MOVING|big knife 10.402|STEP 17|HELP 1 AVAILABLE 12.277|STEP 17|SENSOR MOVING|smaller pot 12.287|STEP 17|SENSOR NOT MOVING|big knife 12.832|STEP 17|SENSOR MOVING|big knife 13.369|STEP 17|SENSOR NOT MOVING|smaller pot 13.961|STEP 17|SENSOR NOT MOVING|big knife 14.743|STEP 17|SENSOR MOVING|smaller pot 16.085|STEP 17|SENSOR NOT MOVING|smaller pot 16.360|STEP 17|SENSOR MOVING|basil 16.875|STEP 17|SENSOR MOVING|big knife 17.405|STEP 17|SENSOR NOT MOVING|basil 17.930|STEP 17|SENSOR NOT MOVING|big knife 18.975|STEP 17|SENSOR MOVING|big knife 20.020|STEP 17|SENSOR NOT MOVING|big knife 20.190|STEP 17|SUCCESS|SENSORS 22.141|STEP 17|SENSOR MOVING|big knife 22.381|STEP 17|NEXT STEP|SUCCESS

120

Natacha Niemants and Gabriele Pallotti

In extract 4, many things happen from both the human and the system viewpoint, and the limitations of the vertical/column arrangement for data display start to become apparent, as too many actions happen simultaneously and it may be hard to understand what is overlapping with what by simply using square brackets. Having said so, the logs clearly show that in addition to making some visible/audible actions (here in bold), the system ‘senses’ many other movements. These are not necessarily those we would expect following its instruction (i.e. the smaller pot one is required to take), but also, and exclusively in the first 19 seconds, those that S2 is making while cutting basil leaves; that is while finishing the action required by the preceding step (using the big knife, chop the basil and leave it there, on the chopping board). The system is correctly detecting the movement of the two associated sensors (big knife and basil), plus that of a tablespoon touched by S1 before acknowledging receipt of the kitchen’s instruction (ah, line 8). From the system’s viewpoint, at second 10.402 nobody has made the required action yet, which is why KIT makes Help 1 available for use. If one looks at the second part of the log, the smaller pot starts moving and it stops a few seconds later. The system thus acknowledges success and prepares to move one to the following step, totally unconcerned by the other movements it senses – big knife and basil – which are again due to S2 cutting the basil leaves. From the system’s viewpoint, the action has been concluded and Step 17 is successful. But if we consider the learners’ point of view, we get a more complex  – although apparently less detailed – picture of what happens in Step 17. The first thing we notice is that while dictating the actions, the system is not necessarily dictating the timings: it may well be the case (and it often is) that the kitchen moves on to the following step while one or both learners are still carrying out the action required by the previous one. This does not prevent them, however, from listening to the instruction and acknowledging receipt, and what happens here shows a clear division of work. While S2, the weaker learner, continues the previous action, S1, who has understood what is to be done next, follows the kitchen’s instruction. At the moment when S1 takes the right object, S2 says io non capisco (I don’t understand, lines 10–11), thereby making her comprehension problem ‘public’ (Heritage 1985). Interestingly enough, a few lines later, it is S1 who makes it public that she has problems in understanding the system’s offer, which she does by echoing part of KIT’s posso essere utile? (can I be useful?, line 19), and S2 offers her (wrong) translation into English (you can use it, line 20). But since this translation is partially overlapped by the kitchen’s success sound, S1 makes a clarification request come come ha detto? (what what did it say?, line

Human Viewpoint and the System's Viewpoint

121

22) that projects S2’s response – once again the same (wrong) translation – and S1 uses the receipt token ah (lines 23–24). So, from the humans’ viewpoint, the repair sequence initiated by the kitchen itself contains a trouble source. More precisely, it can be the case that learners have trouble understanding the utterance (randomly) chosen by the system to make help available, and in the IDK this frequently happened with posso essere utile? When redesigning the system, we initially thought of removing that utterance, but then decided to leave it anyway, because it appeared that learners gradually got to understand it as long as the session moved on, as can be seen in the following extract involving the same two learners, a few minutes later. Extract 5: Pair 13, Recipe 1, Step 23 01

KIT:

02 03

S1:

04

S2:

posso essere utile ? can I be useful ? (2.0) ((S1 appears to think and makes a face)) sì:: [hahaha] ((and heads towards the tablet to ask for help)) yes [hahaha] (1.0) posso essere utile. can I be useful.

This short extract shows that while S1 responds to the kitchen’s help offer with the positive answer sì (yes, line 3) and a meaningful action (she accepts that offer by using the touch screen), S2 is now able to perfectly repeat the instruction she had wrongly translated before (line 4). This is of course not proof that she now understands its meaning, but it shows that even troublesome repair initiations by the kitchen can stimulate talk and interaction, ultimately leading to learning opportunities.

The ELAN transcription format The vertical transcription format is severely challenged when a wide range of speakers are involved, such as in a classroom interaction. This is the main reason why our third case (extract 6) was transcribed using a multi-tier format, which is more suitable to capture the simultaneous dimensions of verbal and nonverbal communication by a wide range of speakers, kitchen included. In order to do so, the specialized transcription tool called ELAN (https://tla.mpi.nl/tools/ tla-tools/elan/) was employed, where the transcript is displayed in synchrony with the video, thus allowing instant access to the primary and secondary data source, and where different export options are available. For the purpose of this chapter, we have chosen to export using the ‘Traditional Transcript Text’, which

122

Natacha Niemants and Gabriele Pallotti

is close to the turn-by-turn CA transcript, and the ‘Image from ELAN window’, which allows one to ‘re-present’ data as if the reader and the analyst were sharing the same visual angle.

Figure 5.1 Pair 20 (Recipe 2, Step 25) Help 1 Available

As can be seen in figure 5.1, the ELAN window enables one to simultaneously view the transcribed texts in two different areas. In the bottom part of the screen, called ‘Timeline viewer’, transcription segments are organized in tiers corresponding to participants (with multiple annotation levels, e.g. for words and actions). Simultaneous events are placed at the same horizontal position, for example, the system and the humans’ words and actions, and the whole time span one can visualize on a single line is generally very short (less than 15 seconds, depending on font size). In the top part of the screen, transcription segments can be viewed  – along with the video  – in different formats and time frames. In our case we chose the ‘Grid viewer’, which displays the content as well as the begin time, end time and duration of all segments from a single tier (here KITwords), enabling one to observe what happens in a wider time span, in this case the whole duration of the video. ELAN provides a holistic perspective on users’ EDK experience, portraying the human and system’s viewpoints and the relationship between talk and actions. By clicking on a transcription segment in the Timeline viewer, all simultaneous events occurring at that moment in time get selected (and highlighted with a darker background), and a small triangle appears in the Grid viewer, showing where that specific segment is occurring with respect to the kitchen’s turns. For example, in figure 5.1, the selected

Human Viewpoint and the System's Viewpoint

123

segment occurs after the kitchen has given its troublesome instruction usando il coltello da tavola pelate uno spicchio d’aglio e mettete anche quello nella padella (using the big knife peel a garlic clove and put it into the pan too, line 2) and before it produces the audible sound, meaning that help is available (help sound, line 3). That same sound is also displayed in the kitchen’s tiers of the Timeline viewer {KITwords}. We will now see what that entails, and curly brackets {} will conventionally be used to refer to specific tiers.

The case of the class negotiating help The third and last case comes from a different setting, and will be presented using the ELAN transcription format. While the other examples involved pairs of learners, at least as active participants (which was the default setting for most trials of the EDK), this extract comes from a cooking session that took place in a classroom: two learners of Italian prepared Recipe 2, with the hotplate, utensils and ingredients placed on the teacher’s desk, while the rest of the class, including the teacher, were sitting around that table and looking at them. Given the classroom setting, the tablet was connected to an overhead projector, and while the pair could choose to either look at the small or the wide screen, the audience could only see the latter. Although out of the camera frame, this audience visibly affected the pair’s behaviour and they audibly participated in the cooking session with their comments and feedbacks, which is the main reason why this extract was transcribed using the ELAN multi-tier format. The chosen extract involves a pair of true beginners (they are both migrants who have just arrived in Italy), with Tagalog (the woman) and Armenian (the man) as L1s. The excerpt starts towards the end of Recipe 2, at step 25, when the kitchen tells them to peel a garlic clove and put it into the pan, and ends 1 minute and 20 seconds later, when the kitchen moves on to the following step. Given its length, we will first provide an overview of the words uttered in the entire extract using the vertical format, with interlinear translations where relevant. We will then focus on two repair sequences using the ELAN display, without including a transcription tier to make representation more compact. Extract 6: Pair 20, Recipe 2, Step 25 01 S2words 02 KITwords 03 CLASSwords

è poco it’s few molto bene (.) andiamo avanti very good (.) let’s move on ((laughs))

124 04 KITwords

05 S1words 06 S2words

07 CLASSwords

08 S1words 09 KITwords 10 CLASSwords 11 KITwords

12 CLASSwords

13 TEAwords 14 CLASSwords 15 TEAwords

16 CLASSwords

17 KITwords

18 CLASSwords 19 TEAwords 20 CLASSwords

Natacha Niemants and Gabriele Pallotti usando il coltello da tavola pelate uno spicchio d’aglio e mettete anche quello nella padella using the big knife, peel a garlic clove and put it into the pan too ((laughs)) (0.64) la padella the pan (0.04) vale vale vale go go go ((in Spanish)) (6.14) poco few help sound (1.42) oh [troppo troppo] ((laughs)) too much too much [posso essere utile ?] can I be useful ? (0.67) che cos’è quello what is that (1.11) fritti:: saranno fritti gli fried they will be fried the [oddio] oh god [involtini] rolls (1.57) ((laughs)) oddio oh god (0.61)

using the big knife, peel a garlic clove and put it into the pan, too (2.02) ((uncomprehensible)) mmmmmm chiedete aiuto ask for help no sì sì sì sì yes yes yes yes

Human Viewpoint and the System's Viewpoint 21 TEAwords 22 TEAwords 23 KITwords 24 S1words 25 S2words 26 CLASSwords 27 TEAwords

28 KITwords

29 CLASSwords

125

prendi il coso rosso take the red thing co:sa mettete in padella ? what do you put into the pan ? help sound aglio ? garlic ? aglio garlic l’aglio chiedi aiuto chiedi aiuto ask for help ask for help (4.32) usando il coltello da tavola pelate uno spicchio d’aglio e mettete anche quello nella padella using the big knife, peel a garlic clove and put it into the pan too ((incomprehensible comments)) l’aglio ((incomprehensible comments)) ((laughs))

If we just consider the words spoken, very few events happen simultaneously: at lines 10 and 11, where the kitchen’s words partially overlap with the class’ comment; at lines 14 and 15, where the class partially overlaps with the teacher; and many silences occur between one segment and the other. But if we add other layers of representation, such as the participants’ actions, the complexity of the recorded event grows and many things happen during the same time span, so that the above silences are filled with a variety of non-verbal activities. We now examine the above sequence using the ELAN format.

Figure 5.1 (repeated)

126

Natacha Niemants and Gabriele Pallotti

Figure 5.2 Pair 20 (Recipe 2, Step 25) Help 1 Taken

Looking again at figure 5.1, it appears that the first repair sequence is otherinitiated by the female learner (S1), who shakes the red tool and puts it down {S1actions}. The system perceives that movement {KITlog}, and also senses that the red tool has stopped moving while S1 is rubbing her hands and touching the pan {S1actions, KITlog}, but the first level of help is not provided because it only becomes available two seconds later {KITwords, KITactions}. This example thus shows that, while a strict adherence to CA’s anti-mentalistic stance would proscribe any account based on human cognition or ‘perceptions’, there is a case for considering sensor logs (the system’s ‘perceptions’) in an integrated transcript, as this enables one to better understand why that help is not provided now. A few seconds later, after she has autonomously added more oil to the pan, causing the audience’s feedback and laughter (see {CLASSwords} in figure 5.1), S1 tries again to shake the red tool (see {S1actions} in figure 5.2). As can be observed by looking at the triangle in the Grid viewer, this time the selected segment (i.e. S1’s movement) happens after the kitchen’s help sound and offer have been produced. It is consequently treated as meaningful by the system, which senses it {KITlog} and provides a slow repetition of the troublesome instruction {KITwords}. Slow tempo is here indicated according to CA conventions using the signs, which are visible in both the Timeline and the Grid viewers.

Human Viewpoint and the System's Viewpoint

127

Figure 5.3 Pair 20 (Recipe 2, step 25) Help 2 Available

A few seconds later (Figure 5.3), despite this slow repetition of the kitchen’s instruction, the learners still do not know what to do and wrongly move the meat towards the pan {S1actions}, while looking at the audience as if asking for help {S2actions}. Help comes from the teacher herself {TEAwords}, who comments on S1’s action and tells the pair to ask for help, mmmmmm chiedete aiuto (mmmmmm ask for help, see also line 19 in the vertical extract 6), then specifying they shall do so by taking the red tool, prendete il coso rosso (take the red thing, line 21). S1 therefore shakes the red tool {S1actions}, and the system senses its movement {KITlog}, but once again that happens immediately before Help 2 becomes available {KITwords, KITactions}. In other words, this second repair sequence too is other-initiated by the learner, following the teacher’s suggestion, before the kitchen is programmed to provide more help. While coping with the kitchen, both learners utter the name of the ingredient they are required to peel, aglio (garlic). S1 does so with a rising intonation and S2 does so while pointing to it with his right finger and asking for audience confirmation, which comes right after {CLASSwords}. Although the pair and the class acknowledge the receipt of that specific ingredient (see also lines 24–26 in the extract 6), the teacher invites one of them to ask for help {TEAwords}, as they still do not know what to do with garlic and the kitchen instruction is likely to contain other trouble sources.

128

Natacha Niemants and Gabriele Pallotti

Figure 5.4 Pair 20 (Recipe 2, Step 25) Help 2 Taken

In figure 5.4, following the teacher’s invitation, S1 shakes the red tool again to initiate the repair sequence {S1actions} and this time, given that it happens after the help sound (see Grid viewer, triangle between lines 6 and 7), the kitchen utters a repetition (at normal speed) of that same troublesome instruction, with an accompanying picture and caption {KITwords, KITactions}. As can be deducted from the human actions tiers {S1actions, S2actions}, neither S1 nor S2 are looking at the tablet, since the system had been plugged into an overhead projector and the pair ended up looking at the wide instead of the small screen. In spite of this second level of help, the two learners were not able to carry out the instruction and Help 3 was requested, in some turns omitted from this discussion, after the research assistant’s suggestion. It was only towards the end of the cooking session that the pair, the class and the teacher got to understand the ‘machinery’ of human-computer interaction in the IDK, and especially the fact that repair sequences could only be initiated after the kitchen had made its first move, that is, produced the help sound and thereby made help available. In the end, this extract shows all the limitations of a purportedly ‘interactive’ system. Users’ ‘initiatives’, for example, asking for help by shaking the red help tool, cannot occur anywhere, but only at some specific points where the programme makes help available. In other words, the system is programmed to react to requests for help only if they are made in predetermined time frames. However, the extract also shows that, regardless of what the kitchen says and ‘perceives’, humans exert a certain autonomy of action, and manipulate objects according to what they ‘sense’ to be right, although we obviously do not have

Human Viewpoint and the System's Viewpoint

129

access to this level of detail and can only transcribe what they do (e.g. adding more oil to the pan). From the system’s viewpoint, the extract shows that the EDK is much more ‘sensitive’ than many other expert systems, such as that discussed by Suchman (2007), as it can ‘perceive’ and track quite a number of the users’ actions. The fact remains, however, that it only reacts to the few sensors that are expected to be moving/not moving at a particular moment, thus disregarding, in addition to the movements that humans do accidentally (e.g. the pot moved in the first example), many actions that humans do on purpose, at precise moments. It is clear that a system like the EDK cannot answer the basic question why that now, in that it is limited to sensing that now, and reacts to it in predetermined ways, with no room for what one could call ‘interpretation’. Before concluding the analysis, we shall further elaborate on the advantage of working with the ELAN integrated transcript, where different data sources can be put together in the transcription phase and then selected according to specific analytical lenses. Although transcription invariably entails a certain degree of analysis, such an integrated environment appears to limit subjective interpretations of turn boundaries by linking words and actions to a common timeline and by forcing the analyst to make decisions about actions and their boundaries public and accountable, based on their intended uses. Being pre-analytical, such an integrated transcript can yield interesting insights into the system’s working, enabling one to better understand why that repair sequence is not happening now, and to possibly think of future system re-designs, while at the same time allowing one to export and disseminate data in a more classical CA turn-by-turn script, where the analyst will decide whether sensor logs (the system’s ‘perceptions’) should be included or not. Such a decision requires being explicit as regards the aims of the analysis, for example, whether it is at the service of a theoretical discussion on the nature of human-computer interaction or of a more practical endeavour like producing an efficient system with certain characteristics. The ELAN format certainly permits the analysis of EDK sessions from both the human and system’s viewpoints. This is vital in uncovering how and why exactly human-computer communication problems arise, which may then feed into re-design.

Conclusions The present study has undertaken analysis of the multimodal interactions occurring between pairs and classes of L2 learners and the Italian Digital Kitchen. We

130

Natacha Niemants and Gabriele Pallotti

have discussed several transcription options that can be used to represent this complex interactive digital environment, each with its pros and cons. The classic, vertical CA format allows one to save space on the page and to see at a glance how rather long sequences develop over time. Despite its including a much broader level of detail vis-à-vis other transcription approaches – which simply report spoken words and sometimes even edit them to make the final product look ‘cleaner’ – compared with other micro-analytic formats it looks like a good telescope. The score-like format looks more like a microscope, as it can display actions by several participants with the greatest precision as regards their collocation on a common timeline. The downside is that, besides requiring a large space on the printed page, it also makes it difficult for the analyst to perceive the interaction in its entirety, given that this ends up being broken down into chunks representing a few seconds each. The format with two or three vertical columns that we used in sections ‘Developing an integrated transcription format’ and ‘When the system does not ‘sense” seems to lie somewhere in between. Every format thus has its strengths and weaknesses, and choosing one or another depends on practical and analytical reasons. This is also why in this chapter we have used all three of them, to meet different needs. It seems to us that the multitier format produced by ELAN is the most complete and detailed one, and that it is almost indispensable when having to do with more complex interactions with several participants and lots of events taking place in the system. On a more theoretical level, we have problematized the idea that the EDK can be considered a ‘participant’ and that human beings can have real ‘interactions’ with it. It is quite obvious that its action and reaction space is dramatically limited and strongly conditioned by the underlying programme. One may wonder whether this can be seen as an extreme case of a terribly dull participant with an awfully limited interactional competence, but that differs only quantitatively from much more sophisticated human beings, or whether all electronic systems are qualitatively different from people, regardless of whether they are cuttingedge computers or very simple circuits. As applied linguists, we remain agnostic with respect to such a question, which we are happy to leave to philosophers. Surely, if the EDK had been a simple touch screen with three buttons (forward, back, help), few would even have wondered whether it could have been called a ‘participant to interaction’. When sensors stop responding, as in extract 3, the system works in exactly this way, and some learners chose to use it this way all the time, even when sensors were working. Some did so because the procedure seemed to flow more swiftly, everything was clear, predictable and under control, just like when one turns on a TV set or a blender.

Human Viewpoint and the System's Viewpoint

131

On the other hand, those who tried to ‘interact’ with the system through the sensors were able to experience something like a simulacrum of an interaction, which made one feel like he/she was interacting with the system. This feeling is given by the fact that, between the sensor moved by the user and the system’s reaction, there is a space that for human beings is opaque, undetermined:  it is neither clear nor easily predictable what reaction will follow a given action, which is what makes the EDK appear to some extent like another human participant. Paradoxically, thus, a certain degree of complication in the interactions makes them perhaps slower and less effective, but at the same time more similar to those we have with other human beings. Furthermore, although the kitchen’s behaviours are rigidly programmed, the programme does not contemplate everything that can occur on the scene, especially by human beings, who have a considerable freedom space, for example, when they choose how many toothpicks to use for closing the meat rolls or how much oil to add in the frying pan. In such cases, the system just feels that something is being moved, but never blocks with its ‘nos’ the creativity that all cooks, regardless of their linguistic or culinary experience, bring to the task. The EDK thus on the one hand is extremely limited and limiting, on the other it also leaves users a large margin of freedom. For many participants the cooking session has been, among other things, a gradual discovery of what they could do with the kitchen, of how it reacted to their actions, whether they were holding a utensil or asking for help with the red help tool. In most cases, by the end of the cooking session, interaction between humans and the system flowed more smoothly than at the start. Given that the system was programmed deterministically, with no learning algorithm, one cannot say that it was the system that learned how to interact with humans. On the contrary, it was the humans who gradually learned how to trigger the system’s reactions, when they needed them at a particular point, so that one might well speak of ‘configuring the user’ (Woolgar 1990). In order to represent this complex process of actions and reactions (interactions?), access to sensor logs is key. True, it violates one of CA’s fundamental assumptions, as it implies reporting what happens ‘inside’ a (simulacrum of a) participant, instead of just describing its publicly accessible actions. On the other hand, looking at the sensor logs and the programme is the only resource available to the analyst to develop an emic approach to the kitchen’s moves. While moves by humans can be interpreted on the basis of socially shared procedures, the only way to answer the question why that now? as regards the system is by examining its programme and sensor log.

132

Natacha Niemants and Gabriele Pallotti

Reading human actions next to sensor logs thus allows one to interpret what happens, moment by moment, in these particular interactions, which can contribute to a better understanding of how the system works in view of a possible redesign, but also of how human beings communicate, to the extent that behavioural models, albeit partial and limited, can shed light on behaviour itself.

Notes This chapter is the outcome of a highly cooperative effort by both authors, Natacha Niemants and Gabriele Pallotti. However, for the specific concerns of the Italian academia, we specify that Natacha Niemants wrote sections ‘Problematizing transcription’ and ‘Combining the system’s and the human points of view when analysing interactions’ and Gabriele Pallotti sections ‘Conversation analysis and human-computer interaction’, ‘Describing interactions in/with the European Digital Kitchen: A simple example’ and ‘Conclusions’. All figures were produced by the University of Modena and Reggio Emilia team.

References Antinoro Pizzuto, E., I. Chiari and P. Rossini (2010), ‘Representing signed languages: Theoretical, methodological and practical issues’, in M. Pettorino, A. Giannini, I. Chiari and F. Dovetto (eds), Spoken Communication, 205–240, Cambridge: Cambridge Scholars Publishing. Ayaß, R. (2015), ‘Doing data: The status of transcripts in conversation analysis’, Discourse Studies 17 (5): 505–528. Button, G., and W. Sharrock (1995), ‘On simulacrums of conversation: Toward a clarification of the relevance of conversation analysis for human-computer interaction’, in P. Thomas (ed), The Social and Interactional Dimensions of HumanComputer Interfaces, 107–125, Cambridge: Cambridge University Press. Cencini, M. and G. Aston (2002), ‘Resurrecting the corp(us|se): Towards an encoding standard for interpreting data’, in G. Garzone and M. Viezzi (eds), Interpreting in the 21st Century: Challenges and Opportunities, 47–62, Amsterdam/Philadelphia: John Benjamins. Cook, G. (1995), ‘Theoretical issues: Transcribing the untranscribable’, in G. Leech, G. Myers and J. Thomas (eds), Spoken English on Computer, 35–53, New York: Longman. Duranti, A. (1997), Linguistic Anthropology. Cambridge: Cambridge University Press.

Human Viewpoint and the System's Viewpoint

133

Edwards, J. A., and M. D. Lampert (1993), ‘Principles and contrasting systems of discourse transcription’, in J. A. Edwards and M. D. Lampert (eds), Talking Data: Transcription and Coding in Discourse Research, 3–32, Hillsdale: Lawrence Erlbaum. Goffman, E. (1974), Frame Analysis. New York: Harper & Row. Heritage, J. (1985), ‘Recent developments in conversation analysis’, Sociolinguistics, 15: 1–19. Jefferson, G. (2004), ‘Glossary of transcript symbols with an introduction’, in G. H. Lerner (ed), Conversation Analysis: Studies from the First Generation, 13–31, Amsterdam/Philadelphia: John Benjamins. Jenks, C. J. (2011), Transcribing Talk and Interaction. Amsterdam/Philadelphia: John Benjamins. Lapadat, J. C. (2000), ‘Problematizing transcription: Purpose, paradigm and quality’, International Journal of Social Research Methodology, 3 (3): 203–219. Lapadat, J. C., and A. C. Lindsay (1999), ‘Transcription in research and practice: From standardization of technique to interpretive positioning’, Qualitative Inquiry, 5: 64–86. Mackey, A., R. Abbuhl and S. M. Gass (2012), ‘Interactionist approaches’, in S. M. Gass and A. Mackery (eds), The Routledge Handbook of Second Language Acquisition, 7– 23, New York: Routledge. Markee, N., and G. Kasper (2004), ‘Classroom talks: An introduction’, The Modern Language Journal, 88 (4): 491–500. Mori, J. (2007), ‘Border crossings? Exploring the intersection of second language acquisition, conversation analysis, and foreign language pedagogy’, The Modern Language Journal, 91 (1): 849–862. Niemants, N. (2012), ‘The transcription of interpreting data’, Interpreting, 14 (2): 165–191. Niemants, N. (2015), ‘Transcription’, in F. Pöchhacker (ed), Routledge Encyclopedia of Interpreting Studies, 421–423, London: Taylor & Francis. Pike, K. (1967), Language in Relation to a Unified Theory of the Structure of Human Behavior, 2nd edn, The Hague: Mouton. Rosenthal, A. (2009), ‘Lost in transcription: The problematic of commensurability in academic representations of American Sign Language’, Text and Talk, 29 (5): 595–614. Sacks, H. (1984), ‘On doing being ordinary’, in J. M. Atkinson and J. Heritage (eds), Structures of Social Action, 413–429, Cambridge: Cambridge University Press. Schegloff, E. A. (1992), ‘On talk and its institutional occasions’, in P. Drew and J. Heritage (eds), Talk at Work, 101–134, Cambridge: Cambridge University Press. Schegloff, E. A., and H. Sacks (1973), ‘Opening up closings’, Semiotica, 8: 289–327. Schegloff, E. A., G. Jefferson and H. Sacks (1977), ‘The preference for self-correction in the organization of repair in conversation’, Language, 53 (2): 361–382.

134

Natacha Niemants and Gabriele Pallotti

Seedhouse, P. (2004), The Interactional Architecture of the Second Language Classroom. Oxford: Wiley-Blackwell. Sidnell, J., and T. Stivers (eds) (2013), The Handbook on Conversation Analysis. Oxford: Wiley-Blackwell. Suchman, L. (2007), Human-Machine Reconfigurations: Plans and Situated Actions. Cambridge: Cambridge University Press. Ten Have, P. (1999), Doing Conversation Analysis: A Practical Guide. London: Sage. Woolgar, S. (1990), ‘Configuring the user: The case of usability trials’, The Sociological Review, 38 (1): 58–99.

Part Three

Implementation

6

Assessing and Promoting Language Development in an Interactive Learning Environment Jana Roos, Nina Reshöft, Lea Hartung and Johanna Bußwinkel

Introduction LanCook was a European initiative designed to support the learning of additional languages. Central to this project is the European Digital Kitchen (EDK), which communicates with language learners in the target language and gives step-by-step instructions on how to prepare a dish typical of a European country related to that language. The human-computer interaction that develops in this learning environment forms the basis for language learning in LanCook: see Chapter 4. Learners at different competence levels are guided in the preparation of one food item, and as they implement the procedure, they demonstrate how well they can listen, understand and follow the instructions given by the EDK. This involves learning L2 vocabulary items. Therefore, it can be said that two key areas in which language learning is supported during the cooking sessions are listening comprehension and vocabulary learning. This chapter focuses on how language development in these two areas can be assessed and promoted in the EDK’s interactive learning environment. In the first part of the chapter, a link between LanCook and the Common European Framework of Reference for Languages (CEFR) (Council of Europe 2001) is created in order to be able to rate learners’ performance during a cooking session. Then, descriptors for listening and receptive skills that we developed specifically for LanCook on the basis of the corresponding illustrative scales in the CEFR will be presented. Following this, we present extracts taken from audio-visual data collected during cooking sessions in English and German in the EDK. These extracts illustrate how the descriptors for listening and receptive skills that were

138

Roos, Reshöft, Hartung and Bußwinkel

developed for use in the EDK can be related to the interaction during a cooking session, so that it becomes possible to describe learners’ language-related actions at different proficiency levels. In the last section of the chapter, we take a look at some results of a pre-test and a post-test that the learners performed before and after the cooking sessions. They exemplify how cooking in the EDK can support learners in understanding instructions and learning topic-related vocabulary.

LanCook and the Common European Framework of Reference for Languages The CEFR has become an influential instrument in the area of language education in Europe, and its descriptor scales that ‘indicate what learners at given “levels” are believed to be able to do’ (Hulstijn et al. 2010: 13) are widely used as a ‘major point of reference for language in education’ (ibid: 15). As pointed out in the CEFR, a key idea related to the descriptor scales is that [t]he development of definitions of learner proficiency related to categories used in the Framework may assist in making more concrete what it may be appropriate to expect at different levels of achievement in terms of those categories. This in turn may aid the development of transparent and realistic statements of overall learning objectives. (Council of Europe 2001: 16)

The EDK as an environment for language learning and the learner’s L2 performance in that environment can be related to the CEFR in two main ways. First, CEFR descriptors can be used by teachers or researchers to rate the learners’ performance during the cooking sessions. Alternatively, they can be used as a basis for self-assessment by the learners themselves. Second, it becomes possible to set up learning goals that are related to the cooking sessions in the EDK and that can take the form of CEFR ‘Can Do’ descriptors. Thus, the EDK can provide an environment for language learners in which language learning can be assessed and promoted at all levels of competence. As mentioned, when relating LanCook to the CEFR, we should take into account that one of the key competences promoted in the cooking sessions is listening comprehension. To carry out the task of cooking in the EDK, ‘the language user as listener receives and processes’ (Council of Europe 2001: 66) the instructions given by the EDK. Therefore, to be able to rate learners’ performance, specific descriptors for LanCook have been developed on the basis of the descriptors for ‘overall listening comprehension’ and ‘listening to

Assessing and Promoting Language Development

139

announcements and instructions’ in the CEFR (Council of Europe 2001: 66f). The scale in table 6.1 illustrates how the levels of learner performance outlined in the CEFR (left column) (Council of Europe 2001: 66) can be combined with actions that the learners perform in the EDK (right column). In both cases, ‘Can Do’ terms (North 2014: 108) describe what a learner at each level is able to do in terms of listening comprehension. The left column combines the CEFR descriptors for overall listening comprehension and listening to announcements and instructions (printed in bold) that most closely reflect the competences required by learners in the EDK. As can be seen, the descriptors already include reference to the ability to “follow detailed directions” (see descriptor in B1). This is also of crucial importance during the cooking sessions, where the learners follow the instructions given by the EDK and are thus able to perform the actions required to cook a specific dish. The right column shows the descriptors that we have developed specifically for LanCook and form an illustrative scale that can be applied to the cooking sessions. A feature that plays an important role in these descriptors is the help function. The EDK is designed to guide and support its users, so in case they do not understand the instructions during the cooking process, they can use the interactive screen (GUI) to request help. By doing so, the learners have the opportunity to listen to the instructions again or to make additional use of photos or short video clips that illustrate the requested items or actions. In accordance with the idea reflected in the CEFR descriptors that more advanced learners gain increased control of the target language, it is assumed here that learners use the help function less frequently as they move towards higher levels of competence. This is reflected in the descriptors developed for LanCook. These descriptors can be used to rate the learners’ performance during the cooking sessions or to formulate learning goals for individual learners. If the aim is to rate learner performance, a possible way of doing this is to look at how many times learners make use of the help function which the tablet computer provides. For instance, if they never use the help function (like native speakers), an appropriate descriptor could be the one at level C1. If the learners need to listen again and regularly make use of the help function in order to understand the instructions given in the recipes, their listening skills could be said to be at level A2. Here, an indicator could also be whether the instructions are short and clear (cf. A2) rather than long and possibly more complex. The latter could be identified by analysing how often the help function is used in connection with a certain instruction. If the aim is to formulate learning goals, a possible way of doing this is to specify the ‘language activities’ in which language users

140

Roos, Reshöft, Hartung and Bußwinkel

Table 6.1 LanCook descriptors for listening comprehension based on the CEFR Overall listening comprehension Listening to announcements and instructions

Level

Application to EDK cooking sessions1

Has no difficulty in understanding any kind of spoken language, whether live or broadcast, delivered at fast native speed. As C1 Can understand enough to follow extended speech on abstract and complex topics beyond his/her own field, though he/she may need to confirm occasional details, especially if the accent is unfamiliar. Can understand complex technical information, such as operating instructions […]. Can understand standard spoken language, live or broadcast, on both familiar and unfamiliar topics normally encountered in personal […] life. […] Can understand the main ideas of propositionally and linguistically complex speech on […] concrete […] topics delivered in a standard dialect […]. Can understand announcements and messages on concrete and abstract topics spoken in standard dialect at normal speed. Can understand straightforward factual information about common everyday or job related topics, identifying both general messages and specific details, provided speech is clearly articulated in a generally familiar accent. Can understand the main points of clear standard speech on familiar matters […]. Can understand simple technical information, such as operating instructions for everyday equipment. Can follow detailed directions. Can understand enough to be able to meet needs of a concrete type provided speech is clearly and slowly articulated. Can understand phrases and expressions related to areas of most immediate priority […] provided speech is clearly and slowly articulated. Can catch the main point in short, clear, simple messages and announcements.

C2

Hardly uses help function.

C1

Occasionally makes use of help function during cooking, for example, for more complex instructions or where accent is unfamiliar.

B2

Can understand simple instructions but may have difficulties with more complex ones. Can understand main ideas of more complex instructions with occasional use of help function.

B1

Can understand instructions that he/she is familiar with. May understand less familiar instructions when using help function.

A2

Can understand main ideas when using help function.

Assessing and Promoting Language Development

141

Overall listening comprehension Listening to announcements and instructions

Level

Application to EDK cooking sessions1

Can follow speech which is very slow and carefully articulated, with long pauses for him/her to assimilate meaning. Can understand instructions addressed carefully and slowly to him/her.

A1

Can understand instructions when speech is slow and he/ she can make use of the help function.

Source: CEFR (Council of Europe 2001: 66f)

engage at different levels. For example, in the area of listening comprehension, the specification might be: ‘understanding some words, phrases or more complex sentences (instructions) related to cooking’. Thus, language learning goals can be identified for individual learners to support them in their language learning process. Assessing learners in this way can serve two different purposes: first, it is possible to rate their performance during the cooking sessions; second, it becomes possible to determine which future language learning goals can be identified for individual learners to scaffold them in their language learning process. Examples of how the descriptors developed for LanCook can be used to assess L2 learner data will be presented in the next section.

Listening comprehension in an EDK cooking session: Examining learner performance In the course of the LanCook project, the German and English recipes that had been developed for the EDK were trialled in cooking sessions with exchange students learning German at a German university as well as with German learners of English from different instructional contexts. Learners from both groups were at different levels of proficiency. In the cooking sessions, they worked in pairs and prepared dishes that were associated with the food culture in Germany and the UK, namely Milchreis (rice pudding) and scones (a type of pastry). The cooking sessions were video recorded, and the audio-visual data collected made it possible to examine in what ways the learners interacted with the kitchen and with one another and how often they made use of the different kinds of support that were provided by the help function. The extracts taken from data collected during the cooking sessions that are presented in the following are analysed in

142

Roos, Reshöft, Hartung and Bußwinkel

relation to the specific descriptors for listening and receptive skills developed for LanCook (see table 6.1). In extracts 1a and 1b, two German learners of English are making scones, following the English recipe and the instructions given by the EDK. Extract 1a: Two German learners of English are making scones KIT: *H1* L1: L2: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Recorded Instruction spoken by the EDK The system is indicating that help level 1 is available Learner 1 Learner 2 KIT:

Line (.) the backing sheet (.) with parchment paper. *H2* L1: Ja, das haben wir ja. Yes we did that. ((L2 presses ‘forward’ on the GUI interface)) KIT: Weigh eight ounces of flour. (10.0) ((System shows that help is available)) ((L2 selects help)) KIT: Weigh (.) eight ounces (.) of flour. *H1* L2: Eight ounces of flour. L1: (xxx) ((System shows that help is available)) ((L2 selects help) [((a help level 2 photo is displayed)) KIT: Weigh eight ounces of flour. *H2*] ((L1 takes the scale)) L2: (xxx) (3.5) L2: Flour. L1: Was sind denn acht ounces? What are eight ounces? L2: (xxx) ((L2 takes flour)) L1: (xxx) ((L1 puts the bowl on the scale)) ((L2 weighs eight ounces of flour)) ((L1 takes the bowl from the scale)) ((L1 presses ‘forward’ on the GUI interface)) ((System shows that help is available)) L1: Das hat noch gar nichts gesagt. It did not say anything. L1/2: (xxx)

Assessing and Promoting Language Development 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68

KIT: L1: L2: KIT: L1:

L1: L2: L1: KIT:

KIT: L2: L1: KIT:

L1: KIT:

L1:

143

((L1 presses ‘forward’ on the GUI interface)) Next (.) sift the flour into the mixing bowl. Das haben wir doch gerade! We did that already! Ja Yes. ((L1 presses ‘forward’ on the GUI interface)) Using the measuring spoons, add two tablespoons of baking powder (.) to the flour. (xxx) ((L2 points to the tablet’s GUI interface)) ((System shows that help is available)) Ich würde da einen Teelöffel reintun. I would add one teaspoon. Ja Yes. ((L2 selects help)) So was haben wir denn? So, what do we have? ((L1 takes the measuring spoons)) Using the measuring spoons (.) add two tablespoons (.) of baking powder (.) to the flour. *H1* ((L2 points to a spoon)) ((System shows that help is available)) Do you need any ((L2 selects help)) Was? What? Ja? Yes? ((a help level 2 photo is displayed)) Using the measuring spoons, add two tablespoons of baking powder to the flour. *H2* ((L2 selects a spoon and is measuring the baking powder)) ((system plays a success sound)) Hey, hör auf! Nicht so schnell! Hey, stop! Not so fast! Next using the measuring spoons, add half a teaspoon of salt to the flour and baking powder in the mixing bowl. ((L1 is still measuring the baking powder)) (xxx) ((System shows that help is available))

As the transcript shows, the learners need to make use of the help function to be able to follow the instructions and frequently do so during the interaction. At different points in the cooking process (lines 7, 12, 45, 54), the opportunity to repeatedly listen to the instructions and to make additional use of photos

144

Roos, Reshöft, Hartung and Bußwinkel

that illustrate the required item or action enables the learners to complete a step successfully and to move on to the next step. The learners’ difficulties in understanding what they are supposed to do become especially obvious when the instructions are more complex. In line 48, the instruction given by the kitchen (KIT) is ‘Using the measuring spoons, add two tablespoons of baking powder to the flour.’ Here, the learners’ difficulties in understanding the instruction become obvious in comments such as ‘What?’ or ‘Yes?’ (lines 55, 56), which accompany their actions. But even with shorter and presumably less complex instructions, we can see that the two learners have comprehension difficulties. For example, after the instruction ‘Weigh eight ounces of flour’ (cf. line 5), one of the learners selects the help function (Help 1 level) as soon as it is made available by the system (lines 7–8). Although after a slower and more careful repetition of the instruction provided by the help function (cf. line 8), the learner is able to repeat part of the instruction (‘Eight ounces of flour’, cf. line 9), both learners decide to use the help function again, now at the Help 2 level. This time, it provides a photo of the action in addition to the spoken instruction (cf. line 14). The picture, displaying the action of weighing flour, helps the learners identify the scales and the flour which they still need in order to perform the action. However, the words ‘eight ounces’ still pose a problem for them (cf. line 20) because this is the only information that they need to extract from the spoken instruction. At this point, the learners rely on one another and collaboratively perform the action by reading the scales, which measure in ounces. As a result, the learners in the example can be assigned to level A1 and the respective descriptor developed for LanCook. It states that the learner ‘can understand instructions when speech is slow and he/she can make use of the help function’. This descriptor refers directly to the descriptors which the CEFR provides for overall listening comprehension and listening to announcements and instructions at the same level (A1), namely:  ‘Can follow speech which is very slow and carefully articulated, with long pauses for him/her to assimilate meaning.’ and ‘Can understand instructions addressed carefully and slowly to him/her’ (Council of Europe 2001: 66f). Another interesting factor that can be observed in the data of the same pair of learners is that in order to perform the task successfully, they not only draw on their language competences, but also on their world knowledge, or, more specifically, their prior experience in the area of cooking. This is illustrated in extract 1b, taken from the same cooking session as extract 1a:

Assessing and Promoting Language Development

145

Extract 1b: Two German learners of English are making scones 1 2 3 4 5 6

KIT: L1: L2:

Using your hands, put the dough onto the surface and knead for a few seconds. ((uses help function)) *H1* ((kneads the dough)) Wir machen das jetzt. Was soll denn da jetzt sonst kommen? Let’s do it like that. What else should come next?

After the instruction ‘Using your hands, put the dough onto the surface and knead for a few seconds’ (cf. line 1), one of the learners selects the help function (H1). Even though this indicates that the learners have difficulties in understanding and following the instruction, they nevertheless proceed to the next step. The comment ‘Wir machen das jetzt. Was soll denn da jetzt sonst kommen?’ (cf. line 5) clearly shows that their cooking experience is an advantage here and makes them confident that they perform the required action.

Figure 6.1 Pair 24 - Making scones

146

Roos, Reshöft, Hartung and Bußwinkel

As the CEFR is not language-specific but relevant to all languages, the descriptors which capture learner performance in the EDK (cf. table  6.1) are equally applicable to the six different EDK languages and to the recipes involved in LanCook. This is demonstrated in extracts 2a, 2b and 2c in which two African learners of German are cooking Milchreis (rice pudding). Since they do not share the same mother tongue (one speaks Bansoa; the other speaks another language spoken in Cameroon), they use French as a common language to talk to each other while following the German recipe and the instructions given by the EDK: Extract 2a:  Two African learners of German are making Milchreis (rice pudding) 1 2 3 4 5 6 7 8

KIT: L3: L4: L3: KIT: L3:

Sie brauchen den Messbecher. You need the measuring jug. Messbecher Measuring jug Non, c’est pas ça. No, that’s not it. Messbecher ((Takes the measuring jug)) Hervorragend. Well done. Dankeschön.Thank you.

The transcript illustrates that even though one of the learners does not immediately understand the word ‘measuring jug’ and is corrected by his partner, the learners do not need the help function to be able to find the right equipment, and they receive positive feedback from the system (cf. line 6). With regard to the competence levels established in accordance with the CEFR, the learners thus show that they ‘can understand simple technical information, such as operating instructions for everyday equipment’ as well as ‘follow detailed directions’ (see table  6.1). This means that they can be assigned to level B1. What this transcript also shows is that the learners not only interact with one another during the cooking session, but one of them also engages in verbal interaction with the EDK and reacts to the system’s positive feedback (‘Hervorragend’, cf. line 6), which he seems to take as a compliment (‘Dankeschön’, cf. line 7). As the learners proceed in their preparation of Milchreis, the instructions given by the EDK become longer and increasingly more complex. At first, the learners continue to be able to perform the required actions. For instance, in extract 2b, they do not have any comprehension difficulties and can follow the instructions.

Assessing and Promoting Language Development

147

Figure 6.2 Pair 11 - Making “Milchreis”

Extract 2b:  Two African learners of German are making Milchreis (rice pudding) 1 2 3 4 5

KIT:

L3:

Waschen Sie die Zitrone gründlich unter fließendem Wasser ab. Wash the lemon thoroughly under running water. ((takes the lemon and washes it)) Gießen Wasser ab. Pour away the water.

However, as extract 2c shows, the subsequent instruction is linguistically more demanding and includes unfamiliar vocabulary, so that the learners have to use the help function several times. Extract 2c:  Two African learners of German are making Milchreis (rice pudding) 1 2 3 4 5 6 7 8 9 10

KIT:

L3:

L4:

Nehmen Sie sich das Messer und schneiden Sie ein großes Stück von der Zitronenschale ab. Take a knife and cut off a big piece of lemon peel. Großes Stück. Big piece. Großes Stück (.) ((Learners look at each other, holding the lemon and the knife)) Quelque chose(.) French: Something. (xxx) Großes Stück (.) Großes Stück.

148 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

Roos, Reshöft, Hartung and Bußwinkel L3: L4: KIT: L3: L4: L3: L4:

L3:

L4: KIT: L3: L4:

L3: L4: L3:

L4: L3: L4: L3: L4:

Schneiden? Cut off? ((L2 selects help)) Nehmen Sie das Messer und schneiden Sie ein großes Stück von der Zitronenschale ab *H2* Abschneiden (.) Großes Stück. Stück abschneiden. Comme ça. Découper. French: Like that. Cut off. ((shows the cutting movement with his hands)) ((L4 takes over the knife and the lemon and shows the movement with the equipment)) (xxx) ((cuts lemon in half)) Macht kein Zeichen. There is no sign. [the kitchen does not give positive feedback] ((selects help function again)) *H3* Nehmen Sie das Messer und schneiden Sie ein großes Stück von der Zitronenschale ab. Ja (.) Und dann? Yes. And then? ((L4 selects help video)) *H4* ((Learners watch help video and see the correct demonstration of the action)) Ah. Großes Stück. ((L3 takes the chopping board)) Großes Stück. ((Puts the lemon on the chopping board and shows movement with the knife)) ((Selects help function / video again)) Weiter. Go on. Non. French: No. Qu’est-ce que tu fais là ? French: What are you doing? (xxx) ((takes over the equipment and performs the action))

In line l, the instruction given by the KIT is ‘Nehmen Sie sich das Messer und schneiden Sie ein großes Stück von der Zitronenschale ab’. Here, the learners’ difficulties in understanding the instruction seem to be related mainly to the fact that it contains unknown vocabulary. This becomes obvious when they repeat the words that they do not understand several times, as in ‘Großes Stück’ or ‘Abschneiden’ (lines 4, 5, 10, 15, 16). They collaboratively try to work out what they are supposed to do with the lemon and the knife and make use of the help function several times (lines 14, 25, 29). Having decided to cut the lemon in half, they are still insecure because the system does not give positive feedback (cf.

Assessing and Promoting Language Development

149

line 23). Only after they have watched the video provided by the help function twice (lines 29, 38), do they understand what they were supposed to do, namely to cut off a large piece of lemon peel, and they finally perform the correct action. This confirms that the learners are at an intermediate level of German and can be assigned to level B1 and the respective descriptor developed for LanCook. It states that the learners ‘may understand less familiar instructions when using help function’ (see table 6.1). The transcript also shows that the learners not only use French when repeating the words that they did not understand, but they also speak some German in their interaction with each other and with the EDK (lines 10–11, 23, 28). Here, the potential of the EDK to initiate communication and interaction in the target language between the participants becomes clearly evident. This shows that even though the main focus in the cooking sessions is on listening comprehension, the EDK also provides a context for the development of the learners’ spoken skills. To summarize, when comparing the extracts of the learners who were preparing scones (extracts 1a and 1b) with the extracts of the learners who were making Milchreis (extracts 2a, 2b and 2c), it can be seen that the EDK creates a learning environment in which both beginning and intermediate learners can complete cooking instructions in the target language successfully. The EDK is designed so it can be used by learners at any level of competence, as it offers a variety of support for learning. It caters to the learners’ needs by providing the support that is necessary at a certain level and can thus promote individual language learning processes as outlined earlier. While low-level students will spend a lot of time on pre-learning of vocabulary and using the photos and videos provided for the instructions, students at higher levels will make less use of the help function and are likely to use more active learning strategies such as working things out from context or negotiation of meaning. As all learners work in pairs, they may also use each other as a resource for learning. It should also be pointed out here that the language used in the instructions is closely interwoven with the activity of cooking. This means that previous cooking experience and genre-specific language skills may also have an influence on a learner’s performance, and that expert chefs may apply non-linguistic culinary knowledge. However, this also reflects how language is used in everyday life, where language skills often consist of genre-specific knowledge of language use. It can be concluded that different perspectives can be taken when relating the CEFR to LanCook. While cooking, learners at different levels of competence

150

Roos, Reshöft, Hartung and Bußwinkel

demonstrate how well they can understand and follow the instructions. The data presented has illustrated that assessing learner data on the basis of Can Do descriptors such as the ones developed for LanCook can serve two different purposes: first, it is possible to rate the learners’ performance during the cooking sessions; second, it becomes possible to determine which future languagelearning goals can be identified for individual learners. The statements made in table 6.1 could also be interpreted as predictions about the development of the learners’ competences, which can be helpful to scaffold them in their language learning process.

Opportunities for language learning: Cooking instructions and topic-related vocabulary The examples of learner performance in the digital kitchen that were presented in the earlier pages underline the importance of (receptive) knowledge of key vocabulary for the successful completion of cooking sessions. A question that arises in this context is whether and how the learners’ comprehension of key words and expressions that play a role in the instructions improves during the course of the cooking sessions. Therefore, in addition to the German recipe for Milchreis that was developed for LanCook, a receptive vocabulary test was developed, which the learners completed before and after the cooking session (pre-test and post-test). For a more detailed account on vocabulary acquisition in LanCook see Chapter 9. The pre-test and the post-test consist of the same set of reading comprehension tasks but differ in the order in which the tasks are presented to avoid a memory effect. The focus was on verbs and prepositions that played a role in the recipe for Milchreis. In the following, we take a look at the results of the two learners (Learner 3 and Learner 4) who prepared Milchreis in the transcripts detailed in extracts 2a, 2b and 2c, to see if there are any parallels between the listening comprehension skills they demonstrated in the cooking sessions and the reading comprehension skills required to complete the test and if their comprehension of certain verb-preposition combinations improved. In addition, we analysed whether expressions that were difficult to comprehend in the tests also caused comprehension problems in the video data. In pre-test and post-test, learners were asked to complete cooking instructions by filling gaps with the correct prepositions (see figure 6.3) or verbs (see

Assessing and Promoting Language Development

151

Waschen Sie die Zitrone gründlich __________ flieβendem Wasser ab. Halbieren Sie die Zitrone __________ dem Messer. Füllen Sie die Milch __________ den Topf Stellen Sie den Topf __________ den Herd. Stellen Sie den Herd __________ die höchste Stufe. Streuen Sie den Puderzucker __________ die Kirschen. Bereiten Sie die Kirschsoβe zu __________ der Reis kocht. Nehmen Sie die Zitronenschale __________ dem Topf. Gieβen Sie den Zitronensaft __________ die Kirschen.

Figure 6.3 German pre-test/post-test for Milchreis (prepositions)

___________ Sie die kleine Schüssel beiseite. ___________ Sie den Esslöffel und ___________ Sie einen Esslöffel Zucker in den Topf. ___________ Sie den Reis mit der Kirschsoβe. ___________ Sie den Herd auf die mittlere Stufe. Lassen Sie den Reis für 20 Minuten ___________.

Figure 6.4 German pre-test/post-test for Milchreis (verbs)

figure 6.4). A list of the missing prepositions and verbs was given, and it was mentioned that each item could be used more than once. Tables 6.2 and 6.3 show the solutions for the pre-test and the post-test and the results that Learner 3 and Learner 4 obtained in each test. As can be seen in table  6.2, both learners filled the gap in the instruction ‘Waschen Sie die Zitrone gründlich unter fließendem Wasser ab’ with a preposition that was not used in the EDK’s instructions. Both of them chose mit (with) as a preposition, which would also be a correct preposition in this context. Similarly, in the instruction ‘Streuen Sie den Puderzucker über die Kirschen’, Learner 4 used the preposition auf (on, onto), which is another possible and correct preposition in this context, since both über and auf express the same concept. The video data clearly show that the learners understand both instructions without any difficulties and carry out the steps immediately (see transcript extract 2b). The same is true for the instruction ‘Gießen Sie den Zitronensaft über die Kirschen’, where the prepositions in (in) (Learner 3, pre-test) and auf (Learner 4, pre-test and post-test) would be correct in the respective context. These examples seem to indicate that the two learners are flexible enough in their understanding of German to use other possible prepositions in these contexts. Moreover, in his post-test, Learner 3 uses the same preposition that is used

Table 6.2 Results for Learners 3 and 4 (prepositions) Solutions prepositions Pre-test/post-test for ‘Milchreis’ (see figure 6.3)

Results Learner 3 pre-test

Post-test

Results Learner 4 pre-test

Post-test

Waschen Sie die Zitrone gründlich unter fließendem Wasser ab. Halbieren Sie die Zitrone mit dem Messer. Füllen Sie die Milch in den Topf. Stellen Sie den Topf auf den Herd. Stellen Sie den Herd auf die höchste Stufe. Streuen Sie den Puderzucker über die Kirschen. Bereiten Sie die Kirschsoße zu während der Reis kocht. Nehmen Sie die Zitronenschale aus dem Topf. Gießen Sie den Zitronensaft über die Kirschen.

unter ‘under’

mit ‘with’

mit ‘with’

mit ‘with’

mit ‘with’ in ‘in’ über ‘over’ auf ‘on, onto’ über ‘over’ während ‘while’ aus ‘out of ’ in ‘in’

mit ‘with’ in ‘in’ auf ‘on, onto’ auf ‘on, onto’ über ‘over’ während ‘while’ aus ‘out of ’ über ‘over’

mit ‘with’ in ‘in’ unter ‘under’ auf ‘on, onto’ über ‘over’ während ‘while’ --auf ‘on, onto’

mit ‘with’ in ‘in’ auf ‘on, onto’ auf ‘on, onto’ auf ‘on, onto’ während ‘while’ in ‘in’ auf ‘on, onto’

Assessing and Promoting Language Development

153

in the instruction given by the kitchen. Thus, it seems that his knowledge has improved during the cooking session. An instruction for which learners did not use the correct preposition in the pretest was ‘Stellen Sie den Topf auf den Herd’. Learner 3 used über, and Learner 4 used unter (under), but both learners used the correct preposition auf in the post-test. The video data show that in the cooking session, the learners were insecure when the system presented the instruction, because they had already put the cooking pot on the stove in one of the previous steps. However, the test results show that both learners are able to use the correct preposition in the post-test, which could be interpreted as evidence that they learned its correct use during the cooking session. Another interesting aspect is that directional verb-preposition combinations seem to pose some problems for the learners, even after the cooking session. As mentioned earlier, both learners speak French in addition to different African languages. French uses more general prepositions like de (from) and à (to) to express directions in these contexts, whereas in German, additional concepts (such as the notion of containment in the preposition aus [out of]) are expressed. The difficulties that the learners have with such prepositions become obvious in instructions like ‘Nehmen Sie die Zitronenschale aus dem Topf ’, although the verb nehmen (take) gives a clue to the correct preposition. Here Learner 4 was insecure about the correct preposition and did not fill the gap in the pre-test and chose the incorrect preposition in in the post-test. A similar problem can be observed with directional verbs like stellen (put), geben (give) and nehmen (take), as shown in table  6.3. For example, for the instruction ‘Stellen Sie die kleine Schüssel beiseite’, Learner 3 used incorrect verbs in both the pre-test and the post-test, and Learner 4 was insecure in the pre-test, leaving the gap empty, but chose the correct verb in the post-test. Similarly, for the instruction ‘Geben Sie einen Esslöffel Zucker in den Topf ’, Learner 3 used the verb nehmen, which denotes the opposite direction or deixis of the correct verb geben. Learner 4 did not fill the gap with any of the available verbs. However, both learners used the correct verb in their post-tests. The video data, however, show that the learners carried out these steps immediately and without any comprehension problems. In sum, most verbs and prepositions that the learners had problems with prior to the cooking session were filled in correctly in the post-test. Despite the learners’ lack of knowledge of some verbs and prepositions, the video data show that the two learners carried out most of the steps in the recipe without major difficulties. A reason could be that their knowledge of vocabulary for ingredients and utensils was often sufficient to understand the instruction in

154

Roos, Reshöft, Hartung and Bußwinkel

Table 6.3 Results for Learners 3 and 4 (verbs) Solutions verbs Results Learner 3 Pre-test/post-test for ‘Milchreis’ pre-test post-test (see figure 6.4)

Results Learner 4 pre-test post-test

Stellen Sie die kleine Schüssel beiseite. (‘put’) Nehmen Sie den Esslöffel. (‘take’)

---

Geben Sie einen Esslöffel Zucker in den Topf. (‘give’) Garnieren Sie den Reis mit der Kirschsoße. (‘garnish’) Stellen Sie den Herd auf die mittlere Stufe. (‘put’) Lassen Sie den Reis für 20 Minuten köcheln. (‘simmer’)

geben ‘give’ nehmen ‘take’ nehmen ‘take’ garnieren ‘garnish’ stellen ‘put’ köcheln ‘simmer’

nehmen ‘take’ nehmen ‘take’ geben ‘give’ garnieren ‘garnish’ stellen ‘put’ köcheln ‘simmer’

------stellen ‘put’ köcheln ‘simmer’

stellen ‘put’ nehmen ‘take’ geben ‘give’ garnieren ‘garnish’ stellen ‘put’ köcheln ‘simmer’

the context of cooking the recipe in a kitchen. Listening comprehension skills, even of more complex instructions (i.e. different verb-preposition combinations), could therefore be ranked higher on the descriptor scales than reading comprehension. The following example that was found in the video data of Learner 3 and Learner 4, further illustrates how another expression that is repeatedly used in the instructions improves during the course of the cooking session. The instruction ‘Rühren Sie gut mit der Gabel um’ includes the German separable verb umrühren composed of the verb rühren (stir) and the particle um (around). The German language has a variety of such verbs, whereas French does not. In the video data we can see that Learner 3 starts stirring the mixture, he then repeats the particle um and starts moving the fork around in a circle. By repeating this part of the instruction and carrying out the action, we can see that the learner learns to understand the instruction. However, no parallel can be drawn between the performance observed here and the results of the tests, because this expression was not part of the tests. Our aim was to find out whether learners’ comprehension of key words and expressions used in the instructions could improve during the course of a cooking session. Moreover, we wanted to find out if parallels can be drawn between the reading comprehension skills required to complete the test and the listening comprehension skills observed in the video data. The results of the pretest and the post-test exemplified how learners used some forms incorrectly in

Assessing and Promoting Language Development

155

the pre-test but correctly in the post-test. This indicates that the learners have improved their language skills during the cooking session. As for the parallels between reading and listening comprehension skills, at least for the example presented, no clear relationship between the two types of data can be discerned. However, both types of data are valuable sources to determine learners’ competence levels.

Conclusion The overall aim of this chapter was to demonstrate how LanCook and the EDK can be related to the CEFR. In this context, we presented Can Do descriptors that we developed for the EDK and that are based on the descriptor scales for listening and receptive skills in the CEFR. Our data provided evidence that these descriptors can be used successfully to rate learner performance in the EDK at different proficiency levels and, focussing on individual language skills, can be used as a basis to set future language learning goals. However, the data also provided insights into the complex learning processes that learners are engaged in when language learning takes place in a real-world digital learning environment and is combined with cooking. They show that the listening and receptive skills that the learners demonstrated in the transcripts are just one part of their overall communicative competence, which comprises linguistic, pragmatic and sociolinguistic competences (Council of Europe 2001: 4, 9f). As learners prepare a dish in the EDK, they listen to and use the target language and they also make use of different strategies, for example, when relying on world knowledge or using the help function, to follow the instructions and to work in a goal- and product-oriented way. In this sense, the descriptors presented here could also be modified according to the learners’ needs or serve as a model to develop additional descriptors for other aspects of language use. For instance, the CEFR also provides illustrative scales for the use of reception strategies (e.g. ‘Identifying cues and inferring from them’) (European Commission 2001:  72f) that could be related to LanCook, because such strategies are involved in the understanding of cooking instructions and can be potentially developed in the EDK. We have shown that assessment through the use of Can Do descriptors related to the CEFR is feasible and can make learning processes and their outcomes more transparent. It should therefore be regarded as an integral part of designing a digital learning environment.

156

Roos, Reshöft, Hartung and Bußwinkel

Every cooking session is different and is determined by the learners’ level of language proficiency and the dynamics of the interaction that develops both between them and the system. Thus, the EDK provides opportunities for learners to develop their communicative competence in multiple and highly individual ways.

Notes All figures and tables were produced by the University of Paderborn team. 1. The CEFR-based descriptors for LanCook were developed by the authors as part of the learning materials designed for the German Digital Kitchen.

References Council of Europe. (2001), Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Hulstijn, J. H., J. C. Alderson and R. Schoonen (2010), ‘Developmental stages in second-language acquisition and levels of second-language proficiency: Are there links between them? ’ in I. Bartning, M. Martin and I. Vedder (eds), Communicative Proficiency and Linguistic Development: Intersections between SLA and Language Testing Research, 11–20, Eurosla monographs. Available online: http://eurosla.org/monographs/EM01/EM01home.html (accessed 28 March 16). North, B. (2014), The CEFR in Practice, Cambridge: Cambridge University Press.

7

Cooking, Interaction and Learning: The Finnish Digital Kitchen as a Language Learning Environment Salla Kurhila and Lari Kotilainen

This chapter examines different types of learning opportunities that have been created by the EDK. By adopting a Conversation Analysis (CA) approach to the data that we collected in the Finnish Digital Kitchen, we demonstrate how language is learned in and through interaction during cooking sessions. In particular, we focus on how the authentic and interactional nature of the EDK supports learning, with further emphasis on the question of how learners use the opportunities provided by the EDK. Many chapters in this volume conclude that the EDK is an authentic learning environment: cooking is a real-life activity that leads to a real – and tasty – goal. Students using this system have a concrete need to understand, because they strive to achieve the goal and taste the dish. Our examples illustrate that this underlying logic of striving for an authentic goal motivates students to perform the cooking activities in the manner requested. Thus, when confronted by a linguistic problem, the students will not let the problem pass. Instead, they actively seek affordances and use both the interactional and technological resources that the cooking situation offers them. In addition to authenticity, we emphasize another key feature: peer interaction. We argue that a central component of the EDK learning process is the interaction between two equal language learners. The lack of an authoritative teacher encourages the students to assume a more active role in their own learning process. Three types of learner activity will be discussed: (1) repeating of linguistic items provided by the computer (2) modifying and further developing linguistic material and (3) introducing new, contextually relevant learning objects.

158

Salla Kurhila and Lari Kotilainen

We argue that the interactional setting and the task-based nature of the EDK offers various opportunities for learners to construct meanings and scaffold their understanding process. The students in our data strongly orient to language learning and therefore take full advantage of the language-learning opportunities that are provided.

Background: Conversation analysis, learning and interaction As our interests lie in situated language learning as it unfolds moment by moment in interaction, our main methodological approach is CA, a method that has been specifically developed to investigate interaction on a detailed level (on CA, see Sidnell and Stivers 2013). CA is based on the interactional or dialogical perspective on language, and it sets out to study members’ practices in ordinary social activities. It is important to emphasize that learning was not an original point of interest in CA studies. However a rapidly growing body of research adopts CA methods to address questions related to second language acquisition (Seedhouse 2005; Hall et al. 2011; Kasper and Wagner 2011; Pallotti and Wagner 2011). Within the field of learning studies, CA-influenced research could be positioned as an approach that focusses on the sociocultural dimension of language and learning, intersecting with orientations such as the ecological theory of learning (van Lier 2000), Vygotsky’s sociocultural theory, and Bakhtin’s dialogicality. The CA-based language learning studies are in line with what is referred to as the ‘social turn’ of second language acquisition (Firth and Wagner 1997, Block 2003). This means that learning is envisioned as an inherently social accomplishment, an increased ability to participate in different social actions. According to this perspective, language is itself a resource for constructing intersubjective meaning between people. Because of the social turn, the scope of the study within second language acquisition research has broadened from the cognitive processes and abilities of individuals to learning in interaction (see also van Lier 2000). For example, Seedhouse (2005) lists several areas in language learning research that have benefited from a CA perspective, such as teaching languages for specific purposes, language proficiency assessment and language classroom interaction. Eskildsen and Theodórsdóttir (2015) list a number of similarly beneficial research findings, such as a deepened knowledge of the role gestures play in L2 learning. As the

Cooking, Interaction and Learning

159

body of work has grown, different lines of research have also emerged within CA studies. For example, Pallotti and Wagner (2011) suggest the following distinction: the first trend is interested in the longitudinal change in the communicative competence of the learners (e.g. see Brouwer and Wagner 2004, Hall, Hellerman and Pekarek Doehler 2011; for criticism see Lee 2010). The second trend, often referred to as ‘doing learning’, aims to determine and describe instances of local learning in interaction (Lee 2010, Sahlström 2011, Koshik and Seo 2012). The detailed CA analysis is used to identify sequences and practises where the interactional environment offers learners learning opportunities. In this chapter, we apply the approach of ‘doing learning’ to our data. The former (first trend) approach would not be possible for us, as our data have not been collected longitudinally. Our data consist of 29 video recordings of approximately 45-minute cooking sessions in the Finnish Digital Kitchen. The speakers are predominately university students, and their skill levels in Finnish vary from A1 to C1.1 It should be noted, however, that our data have a very short-term temporal dimension because the recipe includes the same words in different phases of the cooking. The participants are occasionally able to approximate and integrate unknown linguistic items into their linguistic repertoire during the session when the item emerges several times in different contexts. We will expand on this issue later in the analysis. In order to determine how the EDK supports learning, we explore how the learners orient to their learning. The aim is to locate those interactional moments when the participants encounter linguistic problems and we investigate how the participants manage to overcome them. During these sequences, the participants orient to language learning. Thus, from our perspective, learning manifests itself in the interaction when the learner discovers the object of learning, and acts on it (Lee 2010). We demonstrate that in the EDK, the interlocutors are able to resolve linguistic problems and learn from them. In that regard, our data displays clearly how ‘L2 learning is not something that happens to people; L2 learning is something people make happen through intentional social interaction and co-construction of reflected-upon knowledge’ (Ortega 2014). The intentionality of our participants is illustrated in their often undertaking more than would be necessary to resolve the problem. This additional effort manifests itself in their orientation to language learning. Our analysis of learning could be described as a learning process tracking in the sense proposed by Markee (2008). In other words, through detailed analysis of interaction, we attempt to determine when and how participants orient to potential learning objects in the discourse.

160

Salla Kurhila and Lari Kotilainen

Many CA and learning studies have investigated language classroom data that have the teacher assuming the central role. For example, Gardner (2008) specifies four environments that can be potential learning ‘sites’ in classrooms that are all teacher-dependent, such as starting sentences that students can complete or providing opportunities for words to be repeated. Recently, an increasing interest has evolved in language learning ‘beyond’ the classroom, or ‘in the wild’ (Theodórsdóttir 2011; Seedhouse et al. 2013; Eskildsen and Theodórsdóttir 2015 with comparisons to different environments). In addition, some recent classroom studies have focused more clearly on peer interaction (Mori and Hasegawa 2009; Jakonen 2014). We find this to be a welcome development because from a purely interactional point of view, interaction (that supports) learning can also occur anywhere outside of the traditional classroom. When the language learners are talking to each other, they follow – despite their possible lexical or grammatical shortages – the same basic principles of interaction as first-language speakers (van Lier 2000; Kurhila 2006). From the perspective of learning, however, a lack of linguistic authority may even lead to more talk, and more affordances, as will be shown in our analysis.

From receptive to productive: Repetitions In the following analysis, we proceed from the simplest instances of orientation to language learning to the more advanced. We discuss three phenomena, each of which is related to repetition. We proceed from straightforward word or phrase repetitions to more complicated cases involving participants who recycle linguistic elements that are produced either by the EDK or by the co-participant. One simple yet revealing aspect of the EDK instructions is the high number of repetitions by the students during their cooking. These repetitions can be best observed in the pre-task. The design of the EDK adheres to the principles of Task-Based Language Learning and Teaching (TBLT) by organizing the task so that it involves different phases: pre-task, during-task, and post-task phases (see Chapter 3). During these different phases, the learner’s attention is guided toward language resources that are relevant to complete the task. In the pre-task, lexical knowledge is most relevant to task completion, whereas the physical action is relatively unimportant. The participants are asked to identify a series of lexical items, but in each case, this can be done in a similar way – simply by moving the correct utensil or ingredient. In other words, if the students can identify the lexical items, they can complete the task with minimal effort – by shaking the correct item – without having to negotiate the details of the task with their interactional partner.

Cooking, Interaction and Learning

161

Considering the straightforward nature of the pre-task phase, it is interesting to note that even when the participants recognize words and perform the desired action by shaking the right item, they often explicitly pronounce the word to themselves. The students in our data often repeat the word produced by the computer, both when they recognize the word, and particularly when they do not recognize it. Thus, even though the design of the pre-task specifically promotes receptive language skills, the students actively transfer their receptive skills into the productive use of language. This is illustrated in the following extract from the pre-task: Extract 1 01 KIT: 02 03 Ling: 04 Chen: 05 KIT: 06 Ling:

porkkana, carrot (0.4) porkkanat, carrots mm, (.) (--). ((Ling lifts jar with carrots)) ((success so[und))] [ ok]ay?

While Ling recognises the word porkkana, she produces the word (line 3) before performing the expected action. By exposing the word in the conversation, Ling shares this resource with her co-participant. Chen acknowledges Ling’s repeat with the particle mm. The word ‘carrot’ is one of the nine lexical items that need to be recognized in the pre-task. Ling and Chen know six of the items without needing to request help, but (with one exception) they repeat even the items they are able to recognize. Extract 1 illustrates a straightforward instance of word recognition that nonetheless includes a repetition. When participants do not recognize the words the EDK offers, they most often produce the word themselves. Extract 2a is a case in point: Extract 2a 01 KIT: 02 03 Vera: 04 Lena: 05 06 Lena:

tilli. dill (0.6) °mitä,° what tilli, hh ha ha (.) .hhh dill (2.4) ((Lena and Vera look at the items on the table)) tämä? mhihh ((points at the dill pack)) this

162

Salla Kurhila and Lari Kotilainen

07 08 Lena:

(1.3) ((Vera shrugs her shoulders)) voi↑daan ↓kokeill(h)a. ((lifts the dill pack)) we can try mm hy, (1.0)

09 Vera: 10

Neither of the students in extract 2a, Vera or Lena, recognize the meaning of the word tilli (dill). Vera has not even been able to detect the word from the computer’s turn, given that she only produces an open-class repair initiation (Drew 1997) that indicates a problem in hearing or understanding the previous turn. However, Lena is able to repeat the item (line 4) and thus shares her recognition of the phonetic form of the word with Vera (on cooperation in word searches, see also Mori and Hasegawa 2009). Yet she is unsure what the word means, as is evident from her facial gestures and embodied behaviour. When repeating the word tilli, Lena raises her eyebrows and makes a gesture with her mouth that indicates uncertainty or lack of knowledge (rounding her lips so that her teeth show). Her bodily movements also display uncertainty: she shifts her gaze from one item to another on the table and laughs briefly (line 5). In line 6, Lena then tentatively points to the correct item and suggests testing it (line 8).2 Even though the item is correct, the system does not immediately acknowledge the movement, and the result is that the students need to ask the computer for help by touching the screen and also by consulting the research assistant (who is also present) about the options the computer offers for help. After they have established that they have performed the necessary steps and that the computer is sometimes slow to respond (this discussion takes 16 turns), extract 2b takes place: Extract 2b 01 KIT : 02 RA2 : 03 Lena: 04 05 KIT : 06 Lena: 07

((success sou[nd))] [ hyv]ä, good mhh (1.4) hy vä. (.) ny t voimme a[loittaa kokkaamisen,] good now we can start cooking [.mff tilli.] dill ((Lena looks at and points to the dill package and nods))

After the students have identified dill as the final ingredient, they have completed the entire pre-task successfully and receive positive feedback both from the EDK (line 1)  and the research assistant (line 2). The EDK then explicitly

Cooking, Interaction and Learning

163

verbalizes moving on to the next phase – from the pre-task to the ‘real’ task of cooking (line 5). In an overlap with this transition turn, Lena again utters the word tilli while nodding and making a pointing gesture towards the correct item (lines 6–7). This activity is not prompted by the EDK, which is already initiating the next phase. This suggests that Lena is actively creating a learning opportunity for herself and her co-participant by producing the new word, ‘populating it with her accent and her intentions’ (Bakhtin 1981: 293) and in a Bakhtinian sense, Lena is making it her own. From the point of view of second language learning research, this repetition can be viewed as ‘uptake’ which may contribute to acquisition (Ellis 2003: 199). A question arises as to whether Lena’s turn (line 6) should even be labelled as a repetition, since it is a long time (25 turns) since the EDK has produced the word, and the participants have been engaged in other activities, such as discovering the help functions . Lena’s turn could be interpreted as her recycling an element (Anward 2004) as well as repeating it. Rather than merely echoing something said by the EDK, Lena uses the word as an affordance to contribute to the learning process of this particular lexical item. Furthermore, she uses both verbal and gestural modality; she combines a pointing gesture with the pronunciation of the word and its meaning (figure 7.1) and nods while producing the word to confirm the connection between the new word and its referent.

Figure 7.1 Pointing gesture, extract 2b, line 7

164

Salla Kurhila and Lari Kotilainen

By actively repeating new words and exposing their references, Lena (and Ling in extract 1) create learning opportunities not only for themselves but for their co-participants as well. By using repetition they establish joint ownership of the task (Ellis 2003: 189). As has been noted in previous research, repetition is a common resource for language learning (DiCamilla and Anton 1997; Piirainen-Marsh and Tainio 2009; Reichert and Liebscher 2012). These studies point out to the same direction as our data: a very simple turn of repetition actually makes multiple contributions on multiple levels simultaneously and helps learning in a number of ways. The repetitions in our data commonly occur in three different environments: immediately after the instruction, even when the instruction is understood (extract 1), after the unclear instruction to highlight the problematic part of the instruction (extract 2a), and after the successful realization of the instruction (extract 2b), as ‘memorizing repeats’. However, these verbatim repetitions are by no means the only ways that the students use the linguistic material given by the computer. In the following analysis, we focus on instances of students taking items produced by the EDK as a starting point and subsequently modifying them according to their own needs.

Focusing on language: Approximating and sharing linguistic items As became evident in the previous section, there are different types of repetitions of linguistic items, ranging from immediately echoing what the EDK has said, to actively using the word for the speaker’s own purposes in new contexts. However, even in extract 1, the participants do more than merely echo the words uttered by the EDK. Extract 1 (repeated) 01 KIT: 02 03 Ling: 04 Chen:

porkkana, carrot (0.4) porkkanat, carrots mm, (.) (--).

It should be noted that Ling’s turn is more than a verbatim repeat because it also modifies the reference to be more accurate in relation to the immediate physical

Cooking, Interaction and Learning

165

context. As there are several carrots in the jar on the table, Ling uses the plural form instead of the singular. Thus, Ling uses the resources produced by the EDK, but modifies those to better serve her communicative needs. Furthermore, by revealing her reasoning, she shares the resource with her co-participant. From an interactional and learning point of view, however, extract 1 can be interpreted as a fairly simple instance of modification. Ling repeats the word porkkana in a slightly altered form, but her turn does not evoke negotiation between the participants; Chen acknowledges Ling’s turn rather minimally (line 4). Several more complicated instances occur in the data, which involve the repetitions and modifications of the linguistic material produced by the EDK and which result in extensive negotiations. Extract 3a illustrates the negotiation that is triggered by the verb kiehua (to boil): Extract 3a 01 KIT: 02 03 04 KIT: 05 Anna: 06 Beth: 07 08 Anna: 09 Beth: 10 11 Anna: 12 Beth:

varmista (.) että vesi kiehuu. (0.5) make sure that the water is boiling kun vesi kiehuu? (0.4) laita levy pienemmälle. when water boils turn the heat down (11.2) ((Anna and Beth put spices into the pot)) ((help sound)) (--) (-) ves- (.) kun: vesi: (.) ääm mikä? wate- when water mm what (1.0) ää ki- kiehuu? boils kiehiä kiehuu? (.) sitten: em, (mispronounced) boils then mm (0.6) me me täyty[y.] we we must [ p]ienempi. smaller

The EDK gives an instruction that includes the verb form kiehuu (boils) twice (lines 1–2). This word presents a problem for Beth, which is reflected in her repair initiation in line 6: she produces a ‘trouble-framing repeat’ (Dingemanse et al. 2014: 22) by locating the trouble-source with a partial repetition and a question word. Anna is able to help and provides the target item, albeit with some hesitation and a restart (line 8). It should be mentioned that although Anna’s repair turn (second pair part) would interactionally be a sufficient response to

166

Salla Kurhila and Lari Kotilainen

the repair initiation (first pair part), the sequence does not end there. Beth also repeats the verb, first in a modified form and then in the standard form (line 9). In doing this, she displays both agency and active orientation to learning the word. As is evident in the excerpt, the affordance offered by the computer – the two occurrences of the verb form kiehuu in the instruction – is not enough for Beth to approximate this word in her linguistic repertoire, but when scaffolded by her co-participant, she is able to actively produce the word. Again, the speaker (Beth) can be perceived as making use of another’s words in a Bakhtinian sense, appropriating someone else’s word and making it her own by adapting it to her own semantic and expressive intention (Bakhtin 1981: 293). Another point worth noting is that while Beth does not immediately comprehend the verb in the instruction, she understands the action that they are expected to perform. After she has produced the verb (line 9), she indicates continuation by using a temporal adverb (sitten). Anna continues the utterance by filling in the subject and the verb (me täytyy [we must]), and Beth subsequently produces the key adjective (pienempi).3 The participants collaboratively complete each other’s utterances, producing the key elements of the desired action (water – boils – we need – smaller). No further negotiation is needed to clarify the meaning of the verb form kiehuu; the authentic environment offers contextual support for the learners to deduce the verb’s meaning. After this excerpt, the participants proceed to the (ultra-modern) cooker and begin figuring out how to turn the heat down and determine what the right temperature would be (this takes 11 turns). Subsequently, the following conversation takes place: Extract 3b 01 Anna: 02 Beth: 03 04 Beth: 05 Anna: 06 Beth: 07 Anna: 08 Anna:

(tämä [tässä).] this here [a:ijaa ] tämä on ihan hyvä. (.) ehkä, I see this is quite good perhaps (.) ko[ska ] (.) se kie- ↑kiehbecause it boi- boi[joo.] yes [, to boil [>(kiehu)