Investigating Content and Language Integrated Learning: Insights from Swedish High Schools 9781788922425

Citation preview

Investigating Content and Language Integrated Learning

BILINGUAL EDUCATION & BILINGUALISM Series Editors: Nancy H. Hornberger (University of Pennsylvania, USA) and Wayne E. Wright (Purdue University, USA) Bilingual Education and Bilingualism is an international, multidisciplinary series publishing research on the philosophy, politics, policy, provision and practice of language planning, Indigenous and minority language education, multilingualism, multiculturalism, biliteracy, bilingualism and bilingual education. The series aims to mirror current debates and discussions. New proposals for single-authored, multiple-authored, or edited books in the series are warmly welcomed, in any of the following categories or others authors may propose: overview or introductory texts; course readers or general reference texts; focus books on particular multilingual education program types; school-based case studies; national case studies; collected cases with a clear programmatic or conceptual theme; and professional education manuals. All books in this series are externally peer-reviewed. Full details of all the books in this series and of all our other publications can be found on http://www.multilingual-matters.com, or by writing to Multilingual Matters, St Nicholas House, 31–34 High Street, Bristol BS1 2AW, UK.

BILINGUAL EDUCATION & BILINGUALISM: 116

Investigating Content and Language Integrated Learning Insights from Swedish High Schools Edited by

Liss Kerstin Sylvén

MULTILINGUAL MATTERS Bristol • Blue Ridge Summit

DOI https://doi.org/10.21832/SYLVEN2418 Library of Congress Cataloging in Publication Data A catalog record for this book is available from the Library of Congress. Names: Sylvén, Liss Kerstin - editor. Title: Investigating Content and Language Integrated Learning: Insights from Swedish High Schools/Edited by Liss Kerstin Sylvén. Description: Bristol; Blue Ridge Summit, PA: Multilingual Matters, 2019. | Series: Bilingual Education & Bilingualism: 116 | Includes bibliographical references and index. Identifiers: LCCN 2018038737| ISBN 9781788922418 (hbk : alk. paper) | ISBN 9781788922401 (pbk : alk. paper) | ISBN 9781788922449 (kindle) Subjects: LCSH: English language—Study and teaching (Secondary)—Sweden. | English language—Study and teaching (Secondary)—Swedish speakers. | Language arts (Secondary)—Correlation with content subjects—Sweden. | English language—Study and teaching (Secondary)—Immersion method. | Language and education—Sweden. Classification: LCC PE1068.S9 I58 2019 | DDC 428.0071/2485—dc23 LC record available at https://lccn.loc.gov/2018038737 British Library Cataloguing in Publication Data A catalogue entry for this book is available from the British Library. ISBN-13: 978-1-78892-241-8 (hbk) ISBN-13: 978-1-78892-240-1 (pbk) Multilingual Matters UK: St Nicholas House, 31–34 High Street, Bristol BS1 2AW, UK. USA: NBN, Blue Ridge Summit, PA, USA. Website: www.multilingual-matters.com Twitter: Multi_Ling_Mat Facebook: https://www.facebook.com/multilingualmatters Blog: www.channelviewpublications.wordpress.com Copyright © 2019 Liss Kerstin Sylvén and the authors of individual chapters. All rights reserved. No part of this work may be reproduced in any form or by any means without permission in writing from the publisher. The policy of Multilingual Matters/Channel View Publications is to use papers that are natural, renewable and recyclable products, made from wood grown in sustainable forests. In the manufacturing process of our books, and to further support our policy, preference is given to printers that have FSC and PEFC Chain of Custody certification. The FSC and/or PEFC logos will appear on those books where full certification has been granted to the printer concerned. Typeset by Nova Techset Private Limited, Bengaluru and Chennai, India. Printed and bound in the UK by Short Run Press Ltd. Printed and bound in the US by Thomson-Shore, Inc.

Contents

Contributors Acknowledgements

vii xi

Part 1: The Context Introduction to the Volume and to Part 1: The Context

2

1

CLIL, CLISS and the Swedish Context: An Overview Liss Kerstin Sylvén

3

2

Mapping CLIL in Sweden BethAnne Paulsrud

3

The CLISS Student: Some Background Factors of the Participating Students in the CLISS Project Britt-Marie Apelgren

19

35

Part 2: Assessment and Motivation Introduction to Part 2: Assessment and Motivation

56

4

Assessment in CLIL Helena Reierstam and Liss Kerstin Sylvén

59

5

CLIL and Motivation Revisited: A Longitudinal Perspective Amy S. Thompson and Liss Kerstin Sylvén

76

Part 3: English Introduction to Part 3: English

98

6

English Receptive Vocabulary Liss Kerstin Sylvén and Sölve Ohlander

101

7

English Productive Proficiency Eva Olsson and Liss Kerstin Sylvén

117

8

English Reading Comprehension Liss Kerstin Sylvén and Sölve Ohlander

136

9

Extramural English Liss Kerstin Sylvén

152

v

vi

Investigating Content and Language Integrated Learning

Part 4: Swedish Introduction to Part 4: Swedish 10 The Development of Academic Vocabulary in Swedish Per Holmberg

170 173

11 The Development of Linguistic Correctness in CLIL and NonCLIL Students’ Writing in the L1 at Upper Secondary School Maria Lim Falk

187

12 Visualizing Vocabulary: An Investigation into Student Assignments in CLIL and Non-CLIL Contexts Sofie Johansson and Elisabeth Ohlsson

216

13 The Development of Swedish Receptive Vocabulary in CLIL: A Multilingual Perspective Inger Lindberg and Sofie Johansson

236

Part 5: Students and Teachers Introduction to Part 5: Students and Teachers 14 Multilingual Students in a CLIL School: Possibilities and Perspectives Tore Otterup 15 Just a Little Plus: The CLIL Student Perspective BethAnne Paulsrud 16 Teaching and Learning Content through Two Languages: The Biology and History Teacher Perspective Ylva Sandberg

260 263 282

298

Epilogue Liss Kerstin Sylvén

315

Index

321

Contributors

Britt-Marie Apelgren ([email protected]) is Associate Professor of Language Education at the University of Gothenburg (Sweden). She obtained her PhD in language education at the University of Reading (UK), and her main research interests concern teacher cognition, language teaching and learning, language assessment, and content and language integrated learning (CLIL). Britt-Marie is the co-editor for the Brill/Sense book series Critical Issues in the Future of Learning and Teaching and a member of the editorial board of the journal Personal Construct Theory & Practice (PCTP). Maria Lim Falk ([email protected]) is a researcher in the Department of Swedish Language and Multilingualism, Stockholm University (Sweden). She works within the field of educational linguistics, specializing in first language (Swedish) and the development of academic writing in content and language integrated learning in Swedish bilingual school contexts. Per Holmberg ([email protected]) is Professor of Swedish Language at the University of Gothenburg (Sweden). He specializes in research on writing and written texts in school contexts, and has been particularly interested in developing sociosemiotic models for this research, as well as in introducing such models into teaching practice for the scaffolding of student learning. Furthermore, Per is currently engaged in the study of the emergent writing of Iron Age Scandinavia. Sofie Johansson (sofi[email protected]) is Associate Professor in Swedish as a Second Language, senior researcher and teacher in Swedish, Swedish as a Second Language and Computational Linguistics at the University of Gothenburg (Sweden). She obtained her PhD in Computational Linguistics and Natural Language Processing, and her research interests include Swedish L2 learning, vocabulary acquisition, text comprehension, language learning and teaching in science education and social sciences, language in academic settings, political language and using corpus linguistics in research and education. She has published in journals such as Language Resources and Evaluation

vii

viii

Investigating Content and Language Integrated Learning

Journal, Oslo Studies in Language, US-China, Foreign Language, Utbildning & Demokrati and Nordiska studier i lexikografi. Inger Lindberg ([email protected]) is Professor Emerita of Bilingualism at Stockholm University (Sweden). Her research has mainly addressed sociopolitical and sociocultural aspects of second language education. In recent publications Inger has shown a special interest in the lexical aspects of second language literacy development in multilingual school contexts. Sölve Ohlander ([email protected]) is a Professor (Emeritus) of English Language at the University of Gothenburg (Sweden). His publications range from core linguistic areas, e.g. syntax and lexicology, to more ‘applied’ issues, such as reading comprehension and the crosslinguistic influence of different fi rst languages on the acquisition of English as a second/foreign language, stressing the importance of ‘multi-contrastive awareness’ for teachers in today’s multilingual classrooms. Sölve’s research interests further include the interface of linguistics and popular culture in a wide sense, especially football language. He has co-authored grammars for upper secondary school as well as for students at university level. Elisabeth Ohlsson ([email protected]) is a Lecturer in Swedish in the Department of Pedagogical, Curricular and Professional Studies at the University of Gothenburg (Sweden). She obtained her licentiate in education in 2018 and now continues as a PhD student. Elisabeth’s research is focused on productive writing skills in Swedish as a first language. She has presented her research at various conferences such as AILA and ARLE. Eva Olsson ([email protected]) obtained her PhD in language education in 2016. She is Assistant Professor of Pedagogy at the University of Gothenburg (Sweden), where she is engaged in national test development (English proficiency), in teacher training and in academic language courses. Her research interests include language assessment, content and language integrated teaching and learning, and second language learning in and outside school, in particular students’ development of academic vocabulary and academic writing proficiency. Eva has published articles in journals such as Oslo Studies in Language and Apples – Journal of Applied Language Studies. Tore Otterup ([email protected]) has a PhD in Swedish as a second language and is a former Senior Lecturer in Swedish Language at the University of Gothenburg (Sweden). His particular field of research interest is second language acquisition and development inside and outside school as well as multilingualism and identity formation among young people in multi-ethnic and multilingual settings.

Contributors ix

BethAnne Paulsrud ([email protected]) is Senior Lecturer in English at Dalarna University (Sweden) and is affi liated with the Centre for Research on Bilingualism at Stockholm University. She holds a doctoral degree in educational sciences in languages and language development. Her research interests include English-medium instruction, multilingualism and language policy in education, family language policy, translanguaging and linguistic ethnography. BethAnne has recently co-edited two books on translanguaging in education, one for Multilingual Matters (2017) and one in Swedish for Studentlitteratur (2018). She has also published her research on young children’s translanguaging, language ideologies, teacher education and national curricula. Helena Reierstam ([email protected]) fi nished her licentiate thesis on assessment in CLIL at the University of Gothenburg (Sweden) in 2015, and now works as Director of Education at the Swedish National Agency for Education. While working with a national framework for the national tests, she currently also holds a PhD position at Stockholm University, where her research focuses on subject matter assessment in the multilingual context among newly arrived immigrant students. Helena is investigating teachers’ beliefs about the role of language and their assessment practices when assessing subject content. She has a background as a foreign language teacher in Sweden and in the United States. Ylva Sandberg ([email protected]) is a Lecturer in Language Education at Stockholm University (Sweden). With 15 years of experience in language teaching at upper secondary school, Ylva started working in teacher education in 2008. She completed her licentiate thesis, a sub-study of the CLISS project, in 2018. Her research interests are professional development, teacher and learner cognition, multilingual education and multimodal learning and teaching. Liss Kerstin Sylvén ([email protected]) is a Professor of Language Education at the University of Gothenburg (Sweden). She obtained her PhD in English linguistics, and her research interests include various perspectives of content and language integrated learning (CLIL), second language vocabulary acquisition, language learning motivation, individual differences and extramural English language learning. Liss Kerstin has recently co-authored a book about extramural English (2016), and examples of her publications can be found in journals such as the International Journal of Bilingual Education and Bilingualism, Journal of Immersion and Content-Based Education, Studies in Second Language Learning and Teaching, CALICO Journal and ReCALL. Amy S. Thompson ([email protected]) obtained her PhD at Michigan State University in 2009. She is a Professor of Applied Linguistics

x

Investigating Content and Language Integrated Learning

and currently the Department Chair in the Department of World Languages, Literatures, and Linguistics at West Virginia University. Her primary research interests involve individual differences in SLA and the interactions of these individual differences and multilingualism. Amy teaches a range of theoretical and methodological courses in applied linguistics. Examples of her research can be found in journals such as the Modern Language Journal, TESOL Quarterly, Foreign Language Annals and the International Journal of Bilingual Education and Bilingualism, among others.

Acknowledgements

The CLISS-project had not been possible without external funding. We are therefore grateful to the Swedish Research Council, who provided financial means (project number 721-2010-5376) to enable this unique longitudinal study to be realized. When funding was secured, the next step was to fi nd informants willing to participate in the project, which is easier said than done for such long-term commitments. We are deeply grateful for the three schools, their teachers and students, who in the end decided to accept our invitation to be part of this cutting-edge research into CLIL in Sweden. We would also like to show our appreciation for our publisher, Multilingual Matters. First, for their immediate and positive response to our initial request concerning this publication. Second, for the efficient, professional, and always friendly handling of the process, from start to end by everybody involved. In particular, I am grateful to Laura Longworth, who, with amazing patience, knowledge, and humor always responded to any queries along the way. During the many years of this project, separate studies have been reported on at conferences and seminars. We are happy about all the insightful perspectives, comments, and questions provided by discussants, readers, and audience. Our gratitude also goes out to the anonymous reviewer(s), who really helped in bringing out the best in each chapter, and, not least, in structuring the volume in a reader-friendly and logical way. As the editor of this book, I fi nally would like to thank all my CLISS colleagues for their hard work and dedication to the project. We did not choose the easiest route, but together we strove forward and managed to pull the project together. With this book in my hands, I can defi nitely say that it was worth all the effort! Liss Kerstin Sylvén Göteborg, November 2018

xi

Part 1 The Context

Introduction to the Volume and to Part 1: The Context

This is a book about content and language integrated learning, CLIL, in the Swedish context. More specifically, it is about the large-scale, longitudinal research project Content and Language Integration in Swedish Schools, CLISS, in which various aspects and dimensions of CLIL in Sweden were investigated. In the following chapters, our fi ndings and insights regarding such CLIL concerns as language learning outcomes and motivation are reported; however, other less frequently focused upon topics, such as assessment and participating students’ thoughts in relation to CLIL, are also covered. The volume in its entirety comprises five parts, each devoted to an overarching theme with individual chapters focusing on different aspects of that theme. By way of introduction, the fi rst part covers the context of the project. The second addresses assessment and motivation in relation to CLIL. In the third part, various aspects of students’ proficiency and progress in as well as contact with English are investigated; in the following part, similar perspectives in relation to Swedish are covered. Finally, Part 5 focuses on the views and experiences of students and teachers. In order for our results and findings to make sense, however, an understanding of the overall setting in which the research has been conducted is fundamental. In this introductory part, therefore, the aim is to give the reader some insight into the general context as well as the layout and design of the CLISS project itself. It contains three chapters. In the fi rst chapter, written by Liss Kerstin Sylvén, the Swedish educational system is presented, followed by an outline of the CLISS project, its aims and methods. In Chapter 2, BethAnne Paulsrud offers an overview of the types and extent of CLIL in present-day Sweden. Part 1 concludes with Chapter 3, by Britt-Marie Apelgren, in which background facts about the students involved in the CLISS project are addressed.

2

1 CLIL, CLISS and the Swedish Context: An Overview Liss Kerstin Sylvén

1 Introduction

Content and language integrated learning (CLIL) offers an immersive second or foreign language (L2/FL) context, using the L2/FL as the medium of instruction in one or several non-language subjects, while also studying the L2/FL as a subject in its own right. The acronym CLIL was introduced in the late 1990s (Nikula, 1997) as a distinctly European version of L2 teaching and learning, based on the Canadian immersion method (Swain & Lapkin, 1981), even though the use of a language other than the students’ fi rst language (L1) to convey content instruction had been implemented in schools around Europe long before then (Sylvén, 2013). CLIL has been widely studied in Europe, Asia and Latin America during the last decade (Dalton-Puffer, 2011; Li, 2002; Lim & Low, 2009; Lorenzo et al., 2005; Maljers et al., 2007; Tsui & Tollefson, 2007). Today, CLIL is not limited to the European context, but is becoming increasingly popular in other regions such as Australia, New Zealand, Asia and South America (Banegas, 2011; Cross, 2016; Lin & Man, 2010; Smala, 2012). This volume focuses on CLIL, and in particular CLIL in Swedish schools investigated in the large-scale longitudinal research project Content and Language Integration in Swedish Schools (CLISS). In the Swedish context, the setting of the work presented here, a typical example of CLIL would be a Swedish teacher using L2/FL English to teach history to Swedish students in a Swedish school following the Swedish curriculum. This introductory chapter starts with a summary description of the Swedish education system, followed by an account of the role of English in this context. After that, CLIL is presented along with our understanding of the term, and an overview of previous research, with a particular focus on the Swedish context, is provided. The chapter concludes with a 3

4

Part 1: The Context

description of the CLISS project and a brief introduction to the overall structure and plan of the volume. 2 The Swedish Education System

Sweden is a country with approximately 10 million inhabitants. While Swedish is the major official language, spoken by approximately 80% of the population (Parkvall, 2016), there are five officially recognized minority languages: Finnish, Meänkieli (i.e. Tornedal Finnish), Sami, Romani Chib and Yiddish. In addition, a large number of other languages are represented, such as Arabic (spoken by some 155,000), Kurdish (84,000) and Polish (76,000). Since the mid-1960s, the influx of immigrants has been substantial, and approximately 16% of the population are of nonSwedish background (https://www.migrationsinfo.se/migration/sverige/). In Sweden, 5–5½ years of study at university are required in order to qualify as a teacher at senior high school level. In addition to the educational core (pedagogy and didactics) and practice periods in schools, two subjects are studied. All in all, approximately 76% of all senior high school teachers are appropriately qualified, leaving some 24% without the required level of education (Swedish National Agency for Education, 1). To date, no CLIL teacher education is available at pre-service level, and only a handful of in-service courses are offered. School is mandatory from the age of seven in Sweden. The present-day reality, however, is that most children attend preschool starting at the age of one. In 2015, 83% of all one- to five-year-olds (and 94% of the fourand five-year-olds) in Sweden attended preschool (Skolverket, 2016). At the age of six, all children are offered the opportunity to attend preschool class before entering the compulsory school system. Statistics from 2015 show that 97% of all six-year-olds opted for preschool class (Swedish National Agency for Education, 2). School is then compulsory from Year 1 through Year 9. School Years 10 through 12 – senior high (upper secondary) school – are the years focused on in the CLISS project. They are not mandatory, but 98% of all students go on to senior high school after Year 9. However, statistics show that only 65% of those registered for senior high school fi nish their studies within the stipulated three years (Swedish National Agency for Education, 3). Senior high school in Sweden thus consists of three years of school study. There are 18 national programs to choose from, 12 of which are vocational programs (for instance, the building and construction program, the hotel and tourism program and the vehicle and transport program), and six are preparatory for higher education (for instance, the natural sciences program, the social sciences program and the economics program). In addition, there are five introductory programs for students not yet qualified for national programs. The CLISS project focuses on the preparatory programs for higher education, namely natural and social

CLIL, CLISS and the Swedish Context: An Overview

5

sciences, and economics. Common to all the programs preparatory for higher education are the nine subjects of English, history, mathematics, natural sciences (biology, physics and chemistry), religion, social sciences, sports and health, and Swedish or Swedish as a second language. Even though all these subjects are mandatory in all these programs, they are studied to various degrees, and for some of them the syllabi differ depending on the program. Of special interest to the CLISS project is the subject of English. It is studied from an early age in Swedish schools and, although not compulsory until Grade 3, it is often introduced in Grade 1 or as early as preschool or preschool class. All in all, Swedish students study English for 480 hours throughout the compulsory school system (Swedish National Agency for Education, 4). In the preparatory programs for higher education, English can be studied for a total of 300 credits, divided into three courses: English 5, 6 and 7. The number of hours allotted to each credit is decided by the individual school. While most students in these programs study English 5 and 6, English 7 is optional and thus is not taken by all students. English is one of the subjects covered by the National Tests (see http://nafs.gu.se/english), the other two subjects being Swedish (L1 or L2) and mathematics. As should be clear from the above, English plays an important role in the Swedish school system. But it does so in Swedish society as well, which is the focus of the next section. 3 English in Sweden

English abounds in Swedish society. English is heard on TV and in movie theatres, as all shows and fi lms are subtitled rather than dubbed. Music with English lyrics makes up the majority of the top charts; English words and phrases are used in advertising to an ever-increasing degree. The use of English as the medium of instruction in higher education is widespread, with many of the larger universities offering courses from Master’s level and upwards in English as well as individual courses at lower levels. A great deal of the literature used in higher education is in English. Sweden hosts a number of international companies, such as ABB, AB Volvo and Ericsson. In these and many other companies, English is the corporate language. The influx of English during the last several decades has sparked a discussion as to whether English should be considered a second language, rather than a foreign one, in Sweden (Hyltenstam, 2004; Josephson, 2004; Norén, 2006). The jury is still out, but on an individual level English can certainly be considered as an L2. For young people, who communicate in English when connecting with international friends on the internet, or who study all or part of their education through English and see English as a natural part of their future career, English may well be seen as an L2.

6

Part 1: The Context

However, our view is that, in general, English should still be considered a foreign language, simply because Swedish remains sufficient and adequate for anybody to function in most aspects of Swedish society – and also because the overall competence in English, at all registers and levels of formality, is hardly sufficient for it to be considered a second language in the conventional sense of the term. In sum, English is a very important language making its presence felt at many levels of Swedish society. Consequently, in contrast to many other languages, it is also seen as vital to know by the majority of students (Erickson, 2012; Sundqvist, 2009). The somewhat special status of English in Sweden needs to be taken into account when investigating L2 English proficiency and progress among Swedish students. In the CLISS project this was done by, among other things, collecting data on students’ extramural exposure to English and how such exposure correlates with some of the results obtained (see Chapter 9). 4 Content and Language Integrated Learning (CLIL)

When CLIL started to appear in various European countries it was, in many cases, based on the immersion method introduced in Canada in the mid-1960s (Swain & Lapkin, 1981). However, there were additional reasons for the use of an L2/FL as a language of instruction in educational contexts. A case in point is Germany, where bilingual education was introduced in the regions bordering France in the early 1960s, as a result of the French-German treaty and as a means to reconcile the French and the Germans after WWII (Zydatiss, 2007). In Sweden, however, the fi rst experiments using English as the medium of instruction (later to be referred to as CLIL) in the 1970s were clearly inspired by the very good results reported from immersion in Canada (Åseskog, 1982). Whatever the initial reason for implementing CLIL, the aim has always been to improve learners’ proficiency in the language of instruction. In our globalized world, knowing languages other than one’s L1 is a necessity, and in CLIL, exposure to the target language, in most cases English, is increased by combining its use with the simultaneous learning of specific subject knowledge (Council of Europe, 1995). In other words, CLIL extends the time the learner is exposed to the language in comparison with what is possible in regular language arts class. In addition, ‘real’ and ‘authentic’ language is used to convey content in non-language school subjects. The terms real and authentic are written within quotation marks as, of course, content in language arts class is also real and authentic, but in that case the main goal is to learn the language itself. In non-language subjects, the explicit goal is to learn subject content which thus, in a CLIL situation, needs to be learned and understood even though conveyed through a language other than the students’ L1. Therefore, motivation to

CLIL, CLISS and the Swedish Context: An Overview

7

grasp the content is perceived as high, and understanding the language through which it is communicated is crucial (Sylvén, 2017). There is an ongoing debate as to whether CLIL is a method in its own right (see, for example, Tedick & Cammarata, 2012) or whether it should rather be viewed as an umbrella term covering all sorts of educational practices where a language other than the students’ L1 is used to convey subject content (see, for example, Cenoz et al., 2014). We adhere to the latter view, acknowledging the lack of a defi ning framework and contextdriven implementations of CLIL in Europe and elsewhere, and noting the many common denominators of, for instance, immersion, content-based instruction and task-based learning and teaching. 4.1 CLIL in Sweden

As already hinted, the fi rst case of CLIL in Sweden, inspired by the Canadian immersion method, was the use of English as the medium of instruction in the subject of electrical engineering at high school level (Åseskog, 1982). This turned out to be a successful attempt to increase L2 English learning motivation among engineering students, soon to be followed by several other examples. In the mid-1990s there was an explosion of schools around the country implementing CLIL in one way or another. Thus, at the time of writing this volume, CLIL has been implemented in Sweden for more than 30 years, yet research into its use and effects has been conspicuously scarce. In the late 1990s, the first PhD thesis devoted to CLIL was published, concluding that CLIL students’ proficiency in English had not increased more than that of their non-CLIL peers, and that their content knowledge seemed to have suffered from the use of English as the language of instruction (Washburn, 1997). The second PhD thesis appeared in 2004, showing that the L2 English superiority of the CLIL students could be explained by external factors, such as exposure to English outside of school, rather than by CLIL per se (Sylvén, 2004). Four years later, another PhD study, concerned with the CLIL classroom and its effects on learners’ L1, showed that while non-CLIL classrooms were interactive, involving teachers and students, CLIL classrooms were more quiet and monologic, with the teacher working more from a script which was difficult to adapt and the students more reluctant to make their voices heard (Lim Falk, 2008). It was further shown that using English as the medium of instruction had detrimental effects on students’ use of Swedish subject-related terminology. In a similar vein, Alvtörn (2000) found that CLIL students were less accurate in their use of written Swedish than their non-CLIL peers. With a slightly different focus, Kjellén Simes (2008) focused on English-medium instruction in the International Baccalaureate program. The study looked into students’ development of low-frequency vocabulary and motivated tense shift in their L2 English writing, fi nding positive

8

Part 1: The Context

effects over time, not least among motivated but less proficient students. Terlevic Johansson (2011) investigated CLIL where German was used as the target language. Unlike most studies into CLIL in English, Terlevic Johansson found that the students involved gained in German vocabulary profi ciency, also using a more varied repertoire of communicative strategies. More recently, both Olsson (2016) and Kontio (2016), at least in part, devoted their respective PhD theses to CLIL. Olsson compared the effects of extramural English (EE; see Chapter 9) and CLIL on students’ productive L2 English proficiency, finding that EE seems to have a greater effect among younger learners than among older ones, and that CLIL students used more academic vocabulary from the very start of CLIL. In other words, they were more proficient than their non-CLIL peers even before embarking on their CLIL experience. Even though CLIL students were still ahead of their non-CLIL peers after three years of CLIL, the gap between the two groups did not widen. Kontio’s (2016) study took place in a rarely studied CLIL context, namely a vocational high school program. His findings showed that, in many CLIL situations, the students played around with language to a great extent, making jokes and translanguaging or code-switching, in order to make themselves understood and to understand what was communicated to them. It was concluded that, in the relaxed atmosphere of the CLIL lessons, these students produced larger amounts of L2 English than in English class (Kontio & Sylvén, 2015). In addition to these PhD theses, there have been some minor studies on CLIL published in Sweden. However, apart from Terlevic Johansson (2011) whose focus was CLIL in German, and Kjellén Simes (2008) where the very specific type of English-medium instruction found in the International Baccalaureate program was the target of investigation, there is as yet no convincing evidence for the improvement of target language proficiency in CLIL contexts. And, as is the case elsewhere (Dalton-Puffer, 2011), English is used as the medium of instruction in the vast majority of cases in Sweden (see, for example, Yoxsimer Paulsrud, 2014). There are several hypotheses as to why the results obtained in Sweden differ from some obtained elsewhere. First of all, the lack of framework or policies surrounding CLIL makes for completely idiosyncratic implementations at the individual CLIL schools (cf. the description of the three schools involved in the CLISS project as a case in point; see below). This, in turn, makes it difficult, and sometimes impossible, to make relevant comparisons. Thus, there may certainly be schools where the implementation of CLIL is a success from a number of perspectives, just as there are schools where virtually no benefits can be seen. Secondly, as a consequence of the non-existence of policies, there is no teacher education specifically targeting CLIL teachers. Needless to say, non-language teachers should not be expected to be able to include language aspects in their teaching of subject content unless specifically trained for the task. Thirdly,

CLIL, CLISS and the Swedish Context: An Overview

9

CLIL is normally implemented in senior high schools in Sweden, whereas in many other countries (e.g. Spain) much younger children are involved. It may be that at such a relatively late stage, CLIL instruction is too little and comes too late in order for it to have an effect. Fourthly, as touched on above, in Sweden exposure to English outside of school is enormous (see Chapter 9, this volume). Such exposure, in addition to the early start of English as a mandatory subject in school, makes Swedish children and adolescents highly proficient in English, scoring at the top level in international comparative studies (e.g. European Commission/SurveyLang, 2012; Skolverket, 2012). These factors are further discussed in an international perspective in Sylvén (2013). We now turn to the CLISS project itself. 5 Content and Language Integration in Swedish Schools: The CLISS Project1

As shown above, research into CLIL in the Swedish context has been relatively scarce (Lim Falk, 2002, 2008; Sylvén, 2004, 2007a, 2007b; Washburn, 1997). Wesche (1993: 74) stresses the need for carrying out more longitudinal studies related to content-based learning in language in order ‘to confi rm linguistic, academic and attitudinal outcomes of content-based approaches’. This call for longitudinal studies has since been heard from other scholars in the field as well (e.g. Dalton-Puffer, 2011). Consequently, funding from the Swedish National Research Agency in 2010 for the research project Content and Language Integration in Swedish Schools (CLISS) offered a unique opportunity to delve into the effects of CLIL in Sweden from a plethora of perspectives relevant to L2 learning. CLISS is a large-scale, longitudinal and multi-perspective investigation into CLIL as implemented in Sweden. Below follows a detailed account of the project. 5.1 Overall description

The CLISS project as a whole aimed at illuminating CLIL in the Swedish context from a multitude of perspectives, especially investigating the role of the language of instruction, English, in the development of different academic language competencies among senior high school students. The informants were, on the one hand, students in CLIL programs (henceforth: CLIL students), where English, apart from being a separate subject, was also the medium of instruction in several or all non-language subjects, such as biology and history. On the other hand, the CLISS project also included students in programs where Swedish was used as the medium of instruction throughout the school day, English being studied as a separate subject (henceforth: non-CLIL students). The main focus was on the students’ proficiency and progress in written academic

10

Part 1: The Context

language, in both English and Swedish. Thus, our target was to investigate the type of language used in the school context, which in many ways deviates from more personal, oral, everyday communication (Cummins, 1979; Schleppegrell, 2004). The focus on academic language is by no means unique to this project. Recent years have seen a growing interest in this particular field of English, in terms of vocabulary as well as grammar in a wide sense, and also from an L2/FL perspective, as evidenced by, for example, the Longman Exams Dictionary (2006) (see Ohlander, 2007) and the Cambridge Grammar of English (Carter & McCarthy, 2006), where academic English is accorded a chapter of its own as a special feature (cf. Ohlander, 2008: 19). The increased attention paid to academic English can be seen as related to the spread of English as a global language, the world’s foremost lingua franca (Crystal, 2003; Graddol, 1997), not least within European institutions of higher education in the wake of the Bologna Process (http://www.ehea. info/), making proficiency in the academic registers of English a necessity for students intending to continue their education at tertiary level. Of special interest in the discussion about academic versus everyday language is the distinction made by Cummins (1979) between basic interpersonal communication skills (BICS) and cognitively advanced language proficiency (CALP). As mentioned above, the students involved in the CLISS project are all enrolled in theoretical, academically oriented programs aiming at preparing students for higher education. It is therefore of great interest to investigate possible progress in the CALP register among the informants. The overarching research question for the CLISS project, involving comparisons between CLIL and non-CLIL students as well as between English and Swedish writing proficiency, was: •

How do productive and receptive competencies in academic language develop? Subsidiary questions were:

• •

To what extent, if any, are gender differences in evidence? To what extent, if any, are there differences between students with Swedish as their L1 and students who speak Swedish as an L2? And, further:

• •

What are students’ views on CLIL? What are teachers’ views on CLIL? The following sections present the CLISS project in some more detail.

5.2 Method and material

Since the overall purpose of the CLISS project was to provide as multifaceted a picture of CLIL in Sweden as possible, several different methods have been employed. First of all, the study was longitudinal, spanning

CLIL, CLISS and the Swedish Context: An Overview

11

three years. Thus, students were followed from Year 10, i.e. the start of senior high school, throughout the three years making up this educational level in the Swedish school system. This enabled comparisons at any given point during the study, concerning both group and individual progression. Table 1.1 gives an overview of the types of data collected within the CLISS project. As the table illustrates, both quantitative and qualitative data are included in the study. Examples of quantitative data are the questionnaires and the vocabulary tests. Qualitative data are, among other sources, to be found in the interviews. It should be noted, however, that most of the quantitative data have also been analyzed qualitatively. As the focus of the project was on students’ written proficiency, all empirical data specifically relating to language skills were in writing. The vocabulary tests used were the Vocabulary Levels Test (VLT; Laufer & Nation, 1999; Nation, 2001) for English and a similar test for Swedish. Further, the tests concerning synonyms and collocations were, for English, the Depth of Vocabulary Knowledge Measure and the TOEFL Vocabulary Measure, as described in Qian and Schedl (2004), and corresponding tests for Swedish (see Chapter 6 for English and Chapter 13 for Swedish). The reading comprehension tests in English were tests that were originally designed for use in the English Reading Comprehension (ERC) part of the Swedish Scholastic Assessment Test for higher education (SweSAT) and which, when tried out on relatively large numbers of test takers, have proved slightly too easy for inclusion in the high-stakes SweSAT (Ohlander, 1996; Reuterberg & Ohlander, 1999; see Chapter 8). The tests of free written production were carefully designed by members of the project team, based on curriculum goals, teacher Table 1.1 Overview of data collection Year 1 Term 1

Year 2 Term 2

Term 3

Year 3 Term 4

Term 5

Term 6

Student background questionnaire

X

X

Motivation questionnaire

X

X

Vocabulary tests

Sw/Eng

Synonyms and collocations Reading comprehension

Sw/Eng Sw/Eng

Eng

Argumentative text

Sw/Eng

Sw/Eng

Exploratory text Classroom observations

X

Student interviews

X

Teacher interviews

Sw/Eng

Sw

Sw/Eng Sw/Eng

Sw/Eng

X

X

X

X

X

X

X

X

X

X

X X X

12

Part 1: The Context

experience and pilot testing, mirroring the design of the written component of the obligatory national tests, in both English and Swedish (see Chapter 7 for English and Chapters 11 and 12 for Swedish). In addition, a questionnaire was administered covering students’ language background and experience, parents’ educational level, students’ self-assessment of language proficiency, and other relevant aspects. The background questionnaire was administered twice, once at the outset and once at the end of the three-year period (see Chapter 3). A questionnaire tapping into students’ motivation, language anxiety and willingness to communicate (Dörnyei, 2009) was also administered to the CLISS informants (see Chapter 5). Finally, data on students’ exposure to English outside of school were collected twice through a so-called language diary (see Chapter 9). Apparently unique to the CLISS project is the fact that we followed students throughout their entire senior high school experience, for three full school years, and also that we collected similar and simultaneous data in both English and Swedish from all our informants, both CLIL and nonCLIL students. 5.3 Informants

At the outset of the study, in 2011, the informants in the CLISS project were 15–16 years old. They had just fi nished their nine compulsory years of schooling and started their three years at senior high school level. As mentioned previously, while not obligatory by law, this is opted for by approximately 98% of all students in Sweden (Skolverket, 2012). In all, 221 students in eight groups at three different schools were invited to take part in the study. A letter of consent, in accordance with the guidelines set up and authorized by the regional ethical review board at the University of Gothenburg (http://www.epn.se/en/start/), was sent to all students. A total of 203 positive replies were returned, resulting in a participation rate of 92% among the initial students in the project. As is always the case in longitudinal, classroom-based studies, things happened during the course of the study. As a result, some of the initial students involved in the project left for other schools while others joined in at a later date. In Year 3 of the data collection, a total of 245 students were involved in the study (see Table 1.2). 5.4 The schools

The three schools differ in several respects. One of the schools, School A, is located in a large city, with students displaying a variety of L1s. In this school, Swedish L1 students are in the minority, with other students’ L1s including Arabic, Hebrew, Macedonian, Mandarin, Syrian and Tagalog (see Chapter 3). This is representative of the population of the

CLIL, CLISS and the Swedish Context: An Overview

13

city where the school is located, and also of the fact that many international companies reside there, with families staying for extended periods needing an English-speaking school for their children. Schools B and C are located in medium-sized cities and have a relatively homogeneous body of L1 Swedish students, although students with other L1s are found in all groups. At School A, English is spoken throughout the school day and is often also heard among students during breaks. It is used as the sole language of instruction, except for language arts. There are several native speakers of English among the staff and no non-CLIL classes are found at School A. At School B, the use of English and Swedish is carefully planned and implemented in the CLIL classes. The teachers have devised a strategy that they consider to work well where, for instance, whole-class instruction is done in English, whereas group or individual tasks are done in Swedish and English, depending on students’ preferences. The staff consists entirely of non-native speakers of English, and there are parallel nonCLIL classes (i.e. the same program is offered in both a CLIL and a non-CLIL version). Finally, at School C, the use of English varies across the school day, depending on subject and current topic of interest. Many contextual factors, such as scheduling and teacher availability – which in many subjects necessitate a merge of CLIL and non-CLIL students – make it difficult for School C to implement English-medium instruction to as high a degree as they would have liked in the CLIL classes. As at School B, there are parallel non-CLIL classes at School C. All students attend academically oriented study programs, aimed at preparing them for higher education. At School A two CLIL groups (and no non-CLIL group) took part in the project. At School B one CLIL and one non-CLIL group participated in parallel programs, and at School C there were two parallel groups of each kind (see Table 1.2). Table 1.2 shows that there were about twice as many females as males among the informants. As the CLISS project was carried out in intact classes, the distribution of students could not be controlled for. Furthermore, there were more CLIL groups (N = 5) than non-CLIL (N = 3), and CLIL groups in general tend to have a majority of female Table 1.2 Overview of group distribution: CLIL versus non-CLIL; male versus female students CLIL groups

CLIL students

Non-CLIL groups

Non-CLIL students

Female students

Male students

Total

School A

2

75

0

0

50

25

75

School B

1

33

1

33

41

25

66

School C

2

47

2

57

65

39

104

Total

5

155

3

90

156

89

245

14

Part 1: The Context

students (Lasagabaster & Ruiz de Zarobe, 2010; San Isidro, 2010; Sylvén, 2010). It should also be noted that students with an L1 other than Swedish seem to prefer the CLIL strand as opposed to the non-CLIL one. In the CLISS project, students with another L1 made up approximately 23% of the total number of students in the CLIL group, compared to approximately 4% in the non-CLIL groups (Sylvén & Thompson, 2015). 5.5 Research team and research perspectives

Just as the student body was somewhat heterogeneous, so was the project team. The composition of the research team is multifaceted as regards the theoretical perspectives and subject domains represented. In addition to ten senior researchers, five PhD students have been connected to the project. It should be emphasized that, although various perspectives and domains are represented, all of them are language driven, focusing on the language side of CLIL, none representing the content side. The senior researchers on the team represent various fields of expertise and illustrate the variety of perspectives taken in the project. English as an L2/FL was an area of primary interest. Large amounts of data were collected to cover this aspect, more specifically in the form of vocabulary and reading comprehension tests, texts written by the students, classroom interaction and interviews. Further, motivation is of great importance in any learning environment, and in the CLISS project it was of particular interest. The majority of the informants in the project had Swedish as their L1; consequently, L1 Swedish received special attention. To investigate this, analyses of student texts were made, as well as analyses of vocabulary and reading comprehension tests. In this connection, classroom observations and interviews have also been essential. As a substantial proportion of the participants had L1s other than Swedish (see Chapter 3), L2 Swedish was another relevant field of study. Data similar to those for L1 Swedish were collected to cover this perspective; in addition, focus groups of L2 Swedish students were formed in which group discussions and interviews took place. Classroom interaction in both CLIL and nonCLIL groups was another area meriting further study. Accordingly, classroom observations were made throughout the period of investigation, documented through field notes as well as audio- and video-recordings. The PhD students also covered distinctly separate areas. One of them focused on the assessment practices among CLIL and non-CLIL teachers. By collecting tests from teachers, differences were identified between CLIL and non-CLIL classes in this respect, while teacher interviews were used to give an insight into teacher thinking as regards assessment. Another PhD student investigated the effects of extramural English (EE) on CLIL and non-CLIL students’ written English, using a questionnaire tapping into students’ EE activities and analyzing texts produced by the students. A third PhD student focused on teachers’ views on and

CLIL, CLISS and the Swedish Context: An Overview

15

understanding of CLIL, by closely following them both in class and in qualitative, personal interviews. Classroom observations and semistructured interviews were used to this end. A fourth PhD student was concerned with giving an overview of CLIL in Sweden at large – there were no updated figures available since Nixon (2000) on the spread of CLIL in Sweden – while at the same time taking a student perspective by, for example, shadowing individual students during entire school days. Finally, the fi fth PhD student took a special interest in the development of  students’ writing proficiency in Swedish, including both CLIL and non-CLIL students in her study. 6 Plan and Structure of the Book

This volume is divided into five main parts. This part consists of two more chapters which present details about CLIL in Sweden and the participants in the CLISS project. Part 2 consists of two chapters dealing with two aspects of great relevance to language learning in general and of specific interest in relation to CLIL, namely assessment and motivation. In Part 3, comprising Chapters 6–9, our fi ndings as regards English are accounted for, and in Part 4, Chapters 10–13, those for Swedish. The fi nal part, Chapters 14–16, is focused on student and teacher voices. Each part is introduced separately. The book concludes with a brief epilogue, where the major findings are pulled together and areas in need of further research are identified. Note (1) This section builds on previous publications, primarily Sylvén and Ohlander (2014).

References Alvtörn, L. (2000) Språk- och innehållsintegrerad undervisning under gymnasiet – en fallstudie. Lund: Department of Nordic Languages, Lund University. Åseskog, T. (1982) Att undervisa el-lära på engelska. Gothenburg: University of Gothenburg. Banegas, D.L. (2011) Content and language integrated learning in Argentina 2008–2011. Latin American Journal of Content and Language Integrated Learning 4 (2), 33–50; doi:10.5294/laclil.2011.4.2.4. Carter, R. and McCarthy, M. (2006) Cambridge Grammar of English: A Comprehensive Guide. Spoken and Written English. Grammar and Usage. Cambridge: Cambridge University Press. Cenoz, J., Genesee, F. and Gorter, D. (2014) Critical analysis of CLIL: Taking stock and looking forward. Applied Linguistics 35 (3), 243–262; doi:10.1093/applin/amt011. Council of Europe (1995) White Paper on Education and Training. Teaching and Learning: Towards the Learning Society. Brussels: European Commission. Cross, R. (2016) Language and content ‘integration’: The affordances of additional languages as a tool within a single curriculum space. Journal of Curriculum Studies 48 (3), 388–408.

16

Part 1: The Context

Crystal, D. (2003) The Cambridge Encyclopedia of the English Language (2nd edn). Cambridge: Cambridge University Press. Cummins, J. (1979) Cognitive/academic language proficiency, linguistic interdependence, the optimum age question and some other matters. Working Papers on Bilingualism 19, 198–203. Dalton-Puffer, C. (2011) Content-and-language integrated learning: From practice to principles? Annual Review of Applied Linguistics 31, 182–204. Dörnyei, Z. (2009) The L2 motivational self system. In Z. Dörnyei and E. Ushioda (eds) Motivation, Language Identity and the L2 Self (pp. 9–42). Bristol: Multilingual Matters. Erickson, G. (2012) Internationella språkstudien 2011 (375/2012). Stockholm: Skolverket. European Commission/SurveyLang (2012) First European Survey on Language Competences. Final Report, 12 January. See http://ec.europa.eu/languages/eslc/ index.html. Graddol, D. (1997) The Future of English. London: British Council. Hyltenstam, K. (2004) Engelskan, skolans språkundervisning och svensk språkpolitik. In B. Lindgren and O. Josephson (eds) Engelskan i Sverige (pp. 36–107). Stockholm: Svenska Språknämnden. Josephson, O. (2004) Engelskan i 2000-talets Sverige. In B. Lindgren and O. Josephson (eds) Engelskan i Sverige (pp. 7–24). Stockholm: Svenska Språknämnden. Kjellén Simes, M. (2008) Room for improvement? A comparative study of Swedish learners’ free written production in English in the foreign language classroom and in immersion education. Karlstad University Studies No. 2008:32, University of Karlstad. See http://urn.kb.se/resolve?urn=urn:nbn:se:kau:diva-2760. Kontio, J. (2016) Auto mechanics in English: Language use and classroom identity work. PhD thesis, Uppsala University. Kontio, J. and Sylvén, L.K. (2015) Language alternation and language norm in vocational content and language integrated learning. The Language Learning Journal 43 (3), 271–285; doi:10.1080/09571736.2015.1053279. Lasagabaster, D. and Ruiz de Zarobe, Y. (eds) (2010) CLIL in Spain: Implementation, Results and Teacher Training. Newcastle upon Tyne: Cambridge Scholars. Laufer, B. and Nation, P. (1999) A vocabulary-size test of controlled productive ability. Language Testing 16 (1), 33–51. Li, D.C.S. (2002) Hong Kong parents’ preference for English-medium education: Passive victims of imperialism or active agents of pragmatism? In A. Kirkpatrick (ed.) Englishes in Asia: Communication, Identity, Power and Education (pp. 29–62). Melbourne: Language Australia. Lim, L. and Low, E.-L. (2009) Multilingual, globalizing Asia: Implications for policy and education. AILA Review 22. Lim Falk, M. (2002) Språk- och innehållsintegrerad inlärning och undervisning i praktiken: meningsfull målspråksundervisning. Stockholm: Institutionen för nordiska språk. Lim Falk, M. (2008) Svenska i engelskspråkig skolmiljö: Ämnesrelaterat språkbruk i två gymnasieklasser. PhD thesis, Stockholm University. Lin, A.M.Y. and Man, E.Y.F. (2010) Bilingual Education: Southeast Asian Perspectives. Hong Kong: Hong Kong University Press. Lorenzo, F., Casal, S. and Moore, P. (2005) Orientaciones para la elaboracíon del currículo integrado de las lengaus en los centros bilingües. Seville: Consejería de Educacíon (Junta de Andalucía). Maljers, A., Marsh, D. and Wolff , D. (eds) (2007) Windows on CLIL. Content and Language Integrated Learning in the Spotlight. The Hague: European Platform for Dutch Education. Nation, P. (2001) Learning Vocabulary in Another Language. Cambridge: Cambridge University Press.

CLIL, CLISS and the Swedish Context: An Overview

17

Nikula, T. (1997) Terminological considerations in teaching content through a foreign language. In D. Marsh, B. Marsland and T. Nikula (eds) Aspects of Implementing Plurilingual Education (pp. 5–8). Jyväskylä: University of Jyväskylä. Nixon, J. (2000) Content and Language Integrated Learning and Teaching in Sweden. Stockholm: Swedish National Agency for Education [Skolverket]. Norén, K. (2006) Universiteten väljer språk. Språkvård 1, 26–29. Ohlander, S. (1996) Forskning kring ordförrådet i det engelska läsförståelseprovet. In: Högskoleprovet: Genom elva forskares ögon (pp. 27–31). Stockholm: Högskoleverket. Ohlander, S. (2007) Longman Exams Dictionary – en ordhandbok av akademiskt intresse. LexicoNordica 14, 275–292. Ohlander, S. (2008) A ‘huge book’. Spoken and written English: A grammar for advanced learners. Moderna Språk 102 (1), 16–34. Olsson, E. (2016) On the impact of extramural English and CLIL on productive vocabulary. PhD thesis, University of Gothenburg. See https://gupea.ub.gu.se/ handle/2077/41359. Parkvall, M. (2016) Sveriges språk i siff ror. Stockholm: Morfem och Språkrådet. Reuterberg, S.-E. and Ohlander, S. (1999) Engelsk läsförståelse i högskoleprovet. Göteborg: Göteborgs universitet. Qian, D.D. and Schedl, M. (2004) Evaluation of an in-depth vocabulary knowledge measure for assessing reading performance. Language Testing 21 (1), 28–52. San Isidro, X. (2010) An insight into Galician CLIL: Provision and results. In D. Lasagabaster and Y. Ruiz de Zarobe (eds) CLIL in Spain. Implementation, Results and Teacher Training (pp. 55–78). Newcastle upon Tyne: Cambridge Scholars. Schleppegrell, M. (2004) The Language of Schooling: A Functional Linguistics Perspective. London: Lawrence Erlbaum. Skolverket (2012) Internationella språkstudien 2012, Vol. 375. Stockholm: Fritzes. Skolverket (2016) Barn och personal i förskolan (Dnr 5.1.1-2016.526). See http://www. skolverket.se/om-skolverket/publikationer/sok?_xurl_=http%3A%2F%2Fwww5. skolverket.se%2Fwtpub%2Fws%2Fskolbok%2Fwpubext%2Ftrycksak%2FBlob%2 Fpdf3638.pdf%3Fk%3D3638. Smala, S. (2012) CLIL Down Under: External support structures to overcome the ‘tyranny of distance’. In D. Marsh and O. Meyer (eds) Quality Interfaces: Examining Evidence and Exploring Solutions in CLIL (pp. 212–224). Eichstaett: Eichstaett Academic Press. Sundqvist, P. (2009) Extramural English matters: Out-of-school English and its impact on Swedish ninth graders’ oral proficiency and vocabulary. Karlstad University Studies No. 2009:55. PhD thesis, University of Karlstad. See http://kau.diva-portal.org/ smash/record.jsf?pid=diva2:275141. Swain, M. and Lapkin, S. (1981) Bilingual Education in Ontario. A Decade of Research. Toronto: Ontario Institute of Studies in Education. Sylvén, L.K. (2004) Teaching in English or English teaching? On the effects of content and language integrated learning on Swedish learners’ incidental vocabulary acquisition. PhD thesis, University of Gothenberg. Sylvén, L.K. (2007a) Are The Simpsons welcome in the CLIL classroom? VIEWS – Vienna English Working Papers 16 (3), 53–59. Sylvén, L.K. (2007b) Swedish CLIL students’ extracurricular contact with English and its relation to classroom activities. In D. Marsh and D. Wolff (eds) Diverse Contexts – Converging Goals. CLIL in Europe (pp. 237–252). Frankfurt-am-Main: Peter Lang. Sylvén, L.K. (2010) Teaching in English or English Teaching? On the Effects of Content and Language Integrated Learning on Swedish Learners’ Incidental Vocabulary Acquisition. Vol. 97. Gothenburg: ACTA Universitatis Gothoburgensis, University of Gothenburg. Sylvén, L.K. (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320.

18

Part 1: The Context

Sylvén, L.K. (2017) Motivation, second language learning and CLIL. In A. Llinares and T. Morton (eds) Applied Linguistics Perspectives on CLIL (pp. 51–65). Amsterdam: John Benjamins. Sylvén, L.K. and Ohlander, S. (2014) The CLISS project: Receptive vocabulary proficiency in CLIL and non-CLIL groups. Moderna Språk (2), 81–119. Sylvén, L.K. and Thompson, A.S. (2015) Language learning motivation and CLIL: Is there a connection? Journal of Immersion and Content-Based Language Education 3 (1), 28–50. Tedick, D.J. and Cammarata, L. (2012) Content and language integration in K-12 contexts: Student outcomes, teacher practices, and stakeholder perspectives. Foreign Language Annals 45 (S1), S28–S53. Terlevic Johansson, K. (2011) Erfolgreiches Deutschlernen durch CLIL? PhD thesis, University of Gothenburg. See http://hdl.handle.net/2077/26560. The Swedish National Agency for Education 1. https://www.skolverket.se/statistik-ochutvardering/statistik-i-tabeller/gymnasieskola/personal/personal-i-gymnasieskolanlasar-2016-17-1.258565 The Swedish National Agency for Education 2. http://www.skolverket.se/statistik-ochutvardering/statistik-i-tabeller/forskoleklass The Swedish National Agency for Education 3. http://www.skolverket.se/statistik-ochutvardering/statistik-i-tabeller/gymnasieskola/betyg-och-studieresultat/betyg-ochstudieresultat-i-gymnasieskolan-lasar-2014-15-1.243900 The Swedish National Agency for Education 4. http://www.skolverket.se/laroplaneramnen-och-kurser/grundskoleutbildning/grundskola/timplan/timplan-for-grundsko lan-1.159242 Tsui, A.B.M. and Tollefson, J.W. (eds) (2007) Language Policy, Culture and Identity in Asian Contexts. Mahwah, NJ: Lawrence Erlbaum. Washburn, L. (1997) English immersion in Sweden: A case study of Röllingby High School 1987–1989. PhD thesis, Stockholm University. Wesche, M. (1993) Approaches to language study: Research issues and outcomes. In M. Krueger and F. Ryan (eds) Language and Content: Discipline-content Approaches to Language Study. Lexington, MA: D.C. Heath. Yoxsimer Paulsrud, B. (2014) English-medium instruction in Sweden. Perspectives and practices in two upper secondary schools. PhD thesis, Stockholm University. Zydatiss, W. (2007) Deutsch-Englische Züge in Berlin (DEZIBEL). Eine Evaluation des bilingualen Sachfachunterrichts in Gymnasien: Kontext, Kompetenzen, Konsequenzen. Frankfurt-am-Main: Peter Lang.

2 Mapping CLIL in Sweden BethAnne Paulsrud

1 Introduction

This chapter presents an investigation into the scope and extent of CLIL in the Swedish context. Although researchers have often suggested that CLIL (especially in English) has increased in Swedish schools in recent years (e.g. Edlund, 2011; Hyltenstam, 2004; Lim Falk, 2008; Stålhammar, 2010; Sylvén, 2004), there were no current official national statistics on the number of schools offering content instruction through the medium of a language other than Swedish when the CLISS project commenced in 2011. The last official statistics on this educational option were compiled by the Swedish National Agency for Education (henceforth, the Agency) in 1999 in a survey study, followed by a small-scale qualitative study in 2001. In 2012 a new survey study was completed in connection to the CLISS project (Yoxsimer Paulsrud, 2014). While the data collection for CLISS focused mainly on the students, this survey was deemed necessary to support an understanding of the context in which the three participating schools exist. In this chapter, the original national survey and subsequent report produced by the Agency are described, followed by a presentation of the CLISS survey and analysis. The development of CLIL in the Swedish context is then explored through a comparison of the official statistics from 1999/2000 with the 2012 study, thus allowing for an understanding of the status of CLIL as well as offering a view on the knowledge of the form of education in Sweden at the time. 2 CLIL in Sweden: Official Statistics

Until the early 1990s, few schools in Sweden offered CLIL (Nixon, 2000: 8), with only 10 senior high schools off ering CLIL in 1992 (Hyltenstam, 2004: 69). However, a school reform that year allowed individual schools more autonomy to decide how to follow the national curriculum and how to reach the course goals for the specific national programs (Dentler, 2007: 167; Nixon, 2000: 3). While all senior high schools are tuition-free in Sweden and continued to be so after the

19

20

Part 1: The Context

reform, a market for independent schools was opened. Thus, the number of independent schools at the senior high school level run by private operators increased. These new independent schools, as well as the existing municipal schools wishing to maintain their student populations, employed new marketing techniques to attract students. One method was offering CLIL programs, mainly through the medium of English, which resulted in a ‘CLIL boom’ (Dentler, 2007: 167), as up to 15 new CLIL programs were started every year for a period of time. Although more senior high schools were reportedly offering CLIL, the Agency neither maintained statistics on the option nor offered guidelines for the schools, partly due to the fact that senior high school education in the Swedish system is not compulsory schooling (see Chapter 1, this volume). At the turn of the century, however, the Agency began investigating CLIL in the Swedish school system. 2.1 The 1999 Agency report

In 1999 the Agency commissioned the fi rst study on the prevalence of teaching content subjects in languages other than Swedish across the school system, aiming to produce statistics for the nine years of compulsory schooling as well as the non-obligatory senior high school programs, both vocational programs and higher education preparatory programs. A total of 6320 schools in Sweden received the questionnaire, including 724 senior high schools, the focus of this chapter. Of those senior high schools, 76% responded, with results indicating that 23% of them self-reported that they offered some content instruction in a language other than Swedish (in most cases English) or that they planned to start with CLIL in the following school year, 1999/2000. Thus, according to the report, 122 schools or 17% already had CLIL and 44 schools or 6% were planning for CLIL. However, this combined and oft-quoted prevalence of 23% is based on the number of questionnaires sent to the schools and includes the non-responding schools in the statistics as well, making the actual number unclear. Of the 554 actual replies, 122 senior high schools answered yes, or 22%. Also, 44 of 554 senior high schools reported plans to start CLIL during the coming school year, or 8%. The actual totals based on the official report and the percentage of questionnaire responses is presented in Table 2.1. As seen in Table 2.1, the officially reported results indicating that ‘over 20%’ of all Swedish schools offered or planned to offer some form of teaching in other languages (Nixon, 2000: 3) may be misleading. Of those responding, 30% of senior high schools either offered or were planning to offer some form of CLIL. It is unknown if those not responding offered or planned to offer CLIL. In the questionnaire, the schools were also asked to indicate the language of content instruction, although there were no categories referring

Mapping CLIL in Sweden 21

Table 2.1 Results from the 1999 survey

Numbers in the final report

Yes, we have CLIL.

Yes, we plan to offer CLIL in the coming school year

Totals

122 replies

44 replies

166 positive replies 388 negative replies

Percentage of CLIL schools in the final report

17% of 724 schools

6% of 724 schools

23% of 724 schools

Percentage of actual responses

22% of 554 schools responding

8% of 554 schools responding

30% of 554 schools

to the amount of CLIL (i.e. if CLIL encompassed an entire program, entire subject courses or single lessons). However, as stated in the report, the scope (referring to the range of subjects and/or programs) and extent (referring to the actual amount of CLIL teaching in those subjects) of what was offered ranged from entire courses taught through the medium of another language to occasional theme work or projects with minimal teaching time in another language. Thus, another study was conducted by the Agency, this time focusing on a more in-depth description of a selection of the schools that had indicated that they offered CLIL in the initial study. This second, more detailed survey was sent specifically to 166 senior high schools that had indicated that they offered or planned to offer CLIL, and the response rate was 98 schools. From this additional questionnaire, it became clear that 8% of the senior high schools actually offered the International Baccalaureate (IB) (see below for a discussion on the difference between IB and CLIL). The results also indicated that municipal schools were more likely to offer CLIL than independent schools, that CLIL schools were more common in bigger cities and municipalities than in smaller, and that the most commonly offered national programs of study were the natural sciences and social sciences. 2.2 The 2000 Agency report

Following the original survey and subsequent additional targeted questionnaire from 1999, the same Agency researcher, John Nixon, conducted a fi eld study in 2001 at 11 schools, including seven senior high schools. Of the seven schools, four offered higher education preparatory programs (two each of the natural sciences and social sciences) and three were vocational. The study focused on municipal schools as no independent schools were visited. One day was spent at each school, for interviews with teachers and students (mainly in English) and the administration of questionnaires. In two of the senior high schools (one vocational and one higher education preparatory program), limited

22

Part 1: The Context

observations were also conducted on the same day. Thus, very little time was actually spent in CLIL classrooms (cf. Lim Falk, 2008), although the study aimed to determine if the schools offered ‘limited’ or ‘extensive’ CLIL, according to the hours and subjects taught in another language (Nixon, 2001: 12). These specifications were instead determined based on the questionnaires and interviews conducted at the schools. This study was presented in the official report, Quality in SPRINT: Towards Quality Assessment and Assurance in Content and Language Integrated Education (Nixon, 2001). The Agency noted that both solid evaluation and documentation of CLIL in the schools were lacking. 2.3 The 2010 Agency report on compulsory schools

The Agency has, to some degree, monitored English-medium CLIL in obligatory schooling (Grades 1–9). At the compulsory school level, teaching subjects through English was initially considered a trial introduced by the Agency in 2003, with CLIL schools requiring the approval of the Agency. Regulations at this level originally limited the teaching of any one subject in a language other than Swedish to 50% or less of all teaching, meaning that Math, for example, could not be taught wholly in English. However, this requirement was waived in 2009, and the 50% requirement currently involves the entire school time. Thus a single subject, such as Math, may be taught entirely in English at the compulsory school level. In 2010 the Agency investigated Englishmedium CLIL in compulsory schools, presenting an assessment of those municipal and independent compulsory schools (from preschool through Grade 9, ages six through 16) offering this option. In the 2008/2009 school year, 42 schools, or approximately 1% of all compulsory schools in Sweden, off ered some CLIL in English, usually in the courses within  social sciences (such as civics) and natural sciences (such as biology) in Grades 6–9. The schools indicated that many of the teachers were native speakers of English. However, as seen above in the earlier studies, evaluation and documentation of CLIL at this level were also lacking. 2.4 Conclusions on the official reports

Reviewing the official statistics and reports presented by the Agency, it is clear that: (1) the actual scope and extent of CLIL in Sweden is difficult to determine; and (2) most CLIL surveys are based on self-reports on a form of instruction that has no official or widely accepted defi nition in the Swedish school context. The primary researcher, Nixon, himself noted that CLIL could ‘take a wide variety of forms’, although this wide variety is never really addressed in any of the Agency’s

Mapping CLIL in Sweden

23

studies. This lack of a clear defi nition has also been noted in previous research on CLIL in Sweden (e.g. Edlund, 2011; Lim Falk & Strand, 2009; Sylvén, 2013). However, it is clear that regulations do exist for compulsory schools, although they may not be followed or monitored, and that the range of what was considered CLIL at the time of the offi cial studies on the senior high school level was very broad, from single occasional lessons to entire programs of study. In conclusion, an analysis of the official reports above indicated that a new survey was needed in order to place the CLISS studies in the wider context of the current status of CLIL in Sweden. 3 The 2012 CLISS Survey

A new survey of all senior high schools in Sweden was planned in order both to determine the status of content teaching in a foreign language and to make comparisons with the earlier official statistics. The aim was to generally map the current status of CLIL in Swedish senior high schools in order to set the scene for the CLISS studies in the schools. The survey was created using an electronic questionnaire in Google Docs (docs.google.com) with only one main multiple-choice question: Does your school currently offer content and language integrated teaching (i.e. subject teaching in a language other than Swedish)? The multiple choice alternatives were intended to be open enough to include any target foreign language and to provide a range of alternatives for several different specific arrangements of CLIL. A short text presented the aim of the survey and also requested the following: the school’s name, the municipality, and the name and email address of a contact person. Additionally, space was available for open responses. Only senior high schools offering the 18 nationally recognized vocational degree programs and higher education preparatory programs were included in the survey (note that due to a school reform in 2011, fewer national programs were offered than when Nixon’s studies were undertaken, although the total number of senior high schools had increased). Schools offering special education or individual instruction in a remedial program were not included. In total, 891 schools received the electronic questionnaire via email. The survey commenced in April 2011, and the fi fth and fi nal round was conducted in February 2012, with a total of 523 responses. However, after controlling the responses for duplicates and for replies from schools offering remedial programs or special education, the fi nal response rate was determined to be 56%, with 502 answers from 891 schools. These results were analyzed using descriptive statistics and the 502 responses from the returned questionnaires are presented in Table 2.2. Each response was designated a category. Below, a description of each category is first presented, followed by a discussion of the general observations about the responses.

24

Part 1: The Context

Table 2.2 Number of schools offering CLIL in the 2012 survey Response (1)

Yes: An entire program is taught mostly in another language (not IB).

(2)

Number of schools

Percentage

Category

5 schools

1%

CLIL

Yes: Some courses are taught mostly in another language.

40 schools

8%

Partial CLIL

(3)

Yes: We have a few lessons or days when certain subjects are taught in another language.

86 schools

17%

(4)

No: But we plan to offer instruction in another language in the near future (within 2 years).

15 schools

3%

(5)

No: We never have teaching in another language.

336 schools

67%

20 schools

4%

Occasional CLIL

Planned CLIL

No CLIL

In addition: (6)

Yes: We have the International Baccalaureate Program.a

IB

Note: aThis was not a separate alternative on the questionnaire; see the text below for further explanation of this category.

3.1 Categories of responses (1) Yes: An entire program is taught mostly in another language

Five schools indicated that they offered one or more entire CLIL programs in English and are categorized as simply CLIL. Table 2.3 presents the basic demographics of these schools, with information provided in the questionnaires and gleaned from individual school homepages. Note that here, the size of the school refers only to the number of senior high school students, even if the school includes other school forms, such as compulsory education, in the same school. The municipality sizes are indicated according to official categories as the following: a large city has 50,000– 200,000 inhabitants; a suburban municipality has at least 50% of the inhabitants commuting daily to a metropolitan municipality; and a metropolitan municipality is one of the three largest municipalities in Sweden, with over 200,000 inhabitants (www.kolada.se). As indicated in Table 2.3, all five schools offer the social sciences program, a reflection of the general popularity of this program across all schools in Sweden. The natural sciences program is offered as a CLIL program in three schools. Two of the schools offer the IB program in addition to the CLIL programs. Each CLIL school is located in a highly populated municipality, although they represent a geographic spread across Sweden.

Mapping CLIL in Sweden 25

Table 2.3 Overview of the five schools reported to offer an entire program through CLIL in 2011 School School type

Greater region

Size of municipality

Size of school

CLIL programs

1

Municipal

Norrland Large city

≈1700

Social sciences + natural English sciences

2

Independent Svealand Suburban municipality

≈ 400

Social sciences

3

Municipal

Svealand Metropolitan ≈1350 municipality

Social sciences + natural English sciences (also IB)

4

Municipal

Götaland Metropolitan ≈350 municipality

Social sciences + natural English sciences (also IB)

5

Independent Götaland Metropolitan ≈200 municipality

Social sciences

Language of instruction

English

English

(2) Yes: Some courses are taught mostly in another language

The second category is Partial CLIL. Although few schools offered entire CLIL programs, 40 schools reported that they offer some courses taught mostly in another language. Of the 40, one also offered an entire social sciences program taught in English and another one offered an IB degree program. In the open responses, a number of schools offered information on the subjects taught, for example automotive electronics on a vocational program and economics on a university preparatory program. The language of instruction, whenever indicated, was always English. Several schools reported offering more than 30% of instruction through the medium of English. One school wrote that English was the ‘working language’. (3) Yes: We have a few lessons or days when certain subjects are taught in another language

The third category, Occasional CLIL, was also the most common response from schools offering CLIL, with 86 schools reporting some form of integration, some of the time. For example, schools offered single lessons or days of integrated content and language learning. However, this was not always intentional, as one school reported that their integration of music and media courses with English was ‘not actually something we strive for’. In others, though, the instruction through English was reported to be an intentional part of the program, for example in courses on the hotel and restaurant program as well as the auto mechanics program. Many schools regularly work thematically, according to the responses, which included the integration of language and content. As with the categories above, those schools mentioning a language of instruction indicated English. Finally, in some schools, visitors, exchange students or international contacts encouraged the integration.

26

Part 1: The Context

(4) No: But we plan to offer instruction in another language in the near future (within two years)

In the responses, 13 schools indicated plans to offer CLIL within the next two years, making this category, Planned CLIL, nearly three times the size of the number currently offering entire programs. While the exact extent and scope of the Planned CLIL was not clear, two of these schools specified that they planned to start IB programs, while another school reported the plan to offer ‘more instruction’ through the medium of English. Although one school did not clearly have plans to initiate a CLIL program, their response was an enthusiastic, ‘Maybe something to consider!’. (5) No: We never have teaching in another language

The fi nal category is also the largest: No CLIL. 336 schools indicated that they never taught content through a language other than Swedish. In the open responses, many schools explained their choice. For example, one school wrote that they focused on international exchanges instead. Several wrote that they did not ‘need’ CLIL or that there had not been an expressed interest. Twelve schools also responded that modern foreign languages such as French and Spanish were usually taught through the target languages. Thus, languages other than Swedish may be used regularly even in senior high schools not offering CLIL. (6) Yes: We have the International Baccalaureate Program

At the time of the 2012 questionnaire, 31 senior high schools in Sweden offered the English-medium IB degree program. Twenty of these schools responded to the questionnaire, choosing the option Yes: An entire program is taught mostly in another language. As these schools indicated in the open comments that they offered the IB degree program, the responses could be recategorized to indicate the difference between the IB programs and other Swedish schools responding that they offer CLIL. The IB program does not follow the Swedish national curriculum – rendering the IB schools outside of the target group of the survey. While IB schools do  cooperate with local schools, they are obliged by the international organization to follow the IB curriculum and assessment routines (Kjellén Simes, 2008: 16). However, the number of IB schools in Sweden at the time  of the survey is included here to complete the picture of how many  Swedish schools are involved in content instruction through an additional language. 3.2 Discussion of the 2012 CLISS survey

The analysis of this survey indicated that many respondents to the questionnaire, usually a teacher or principal from the school, did not understand what ‘content and language integrated learning’ meant,

Mapping CLIL in Sweden 27

despite an explicit, brief explanation in the accompanying text and the fact that the one question asked emphasized subject instruction in another language other than Swedish. Thus, in the analysis, four main ‘problem areas’ were identified in the following areas: (1) modern languages; (2) mother tongue instruction; (3) Swedish language support; and (4) missing answers from schools known to offer CLIL. A discussion of the problem areas follows. First, several schools reported yes to CLIL, indicating that they offered some integration, but in the open responses it was clear that the respondent meant that modern foreign languages and/or English language as a subject were taught through the target language. While this may be considered to be ‘Spanish-medium’ or ‘French-medium’ language instruction, it cannot be considered CLIL if the only subject being taught is the language. Thus there appears to be a lack of understanding of the difference between content instruction and language instruction. As there was not a clear integration, these schools were not included in the CLIL category. Secondly, many survey answers also indicated information about an individual school’s mother tongue tuition. In Swedish schools, mother tongue tuition is offered as extra language instruction to students who actively use another language other than Swedish at home. Mother tongue tuition is thus a language subject and will likely be taught solely in the target mother tongue (see Hedman & Ganuza, 2017, for more on mother tongue support). However, mother tongue is not a foreign language for the students. As the purpose of the survey was to determine if subject content was being taught in a foreign language other than the mainstream Swedish, responses about mother tongue tuition were not included in the CLIL category. A third problem area related to the respondents’ perception of the ‘need’ for CLIL. Some reported that, because all or the majority of the students in their school have Swedish as their mother tongue, the students did not need language support. For example, they indicated that they would naturally offer Swedish language support if they had newly arrived students not proficient in Swedish, but that the need did not currently exist. Others reported that as they were a very small school they did not need to offer CLIL. Thus, there is a lack of understanding that the CLIL option to offer instruction through a foreign or additional language for any and all students exists regardless of student population or needs. Also, CLIL in Sweden would not entail instruction through Swedish. Finally, the fourth problem was that at least two more schools offer entire CLIL programs with subject instruction through the medium of English, but these schools declined to respond to the questionnaire. Although the information on their CLIL programs is readily available online, as these schools did not answer the survey they are not included in the results. Thus, there remains the chance that other schools with Partial, Occasional or Planned CLIL also did not respond to the survey. The lack

28

Part 1: The Context

of participation of these two schools also highlights the fact that the results are only an indication of the status of CLIL in the Swedish senior high school as it is impossible to know how many of the other schools that did not respond actually do offer CLIL. To conclude, the problematic responses above suggest that the concept of CLIL is not widely understood by many school staff such as principals, administrators and teachers who answered the questionnaire. As noted above, there is no official defi nition of CLIL in the Swedish context. The responses may also suggest that colleagues may be unaware of others’ teaching practices, perhaps due to a high turnover rate of school heads and staff. The arduous attempts to track down principals and teachers for email addresses for the questionnaire made this high turnover rate evident, as many emails were returned, indicating that the original recipient had left the school or was on leave of absence. These responses, together with the four main problem areas outlined above, also suggest the possibility that some schools indicating that they offer Partial or Occasional CLIL may have actually referred to modern foreign languages and/or mother tongue provision. 3.3 Limitations of the 2012 CLISS survey

The 2012 CLISS survey has limitations. First, as the survey was restricted by the response rate of 56%, the results cannot be used for greater claims about the development of CLIL in Sweden since the last official statistics. Rather, they can only offer some indications about the current status as well as possible changes over the last decade. Secondly, the 2012 survey did not request details of the content instruction through another language (e.g. which subjects, which languages or the student demographics) of the responding schools. Thus, the results offer only a general status of CLIL. Thirdly, the survey relied mainly on self-reports on whether or not content and language integration was offered at the schools. Without interviews or observations, the results may lack reliability. Despite these weaknesses, the responses from the schools, especially in the open comments section, provided a wealth of new information that facilitates the understanding of CLIL in Swedish senior high schools today. In the next section, these results and insights from the 2012 survey are compared to the 1999 survey conducted by the Agency. 4 Comparison of the 1999 Survey and the 2012 Survey

In order to understand possible developments in CLIL as well as current interpretations of CLIL in the Swedish senior high school context, a comparison was made between the original 1999 study commissioned by the Agency and the 2012 CLISS survey. The original 1999 survey was chosen for comparison as the subsequent 2001 study on quality in CLIL

Mapping CLIL in Sweden

29

Table 2.4 Overview of 1999 survey and 2012 survey

Number of schools

1999 survey

2012 survey

724

891 502

Responses

554

Response rate

76%

56%

Target group

All senior high schools

Only senior high schools offering the 18 national programs of study

was limited to 11 focus schools. Both surveys aimed to include all senior high schools in Sweden, although the 1999 survey also included other levels of education. Also, the 1999 survey included all programs of study at senior high school level while the 2012 survey focused only on vocational and higher education preparatory programs, excluding individual study and special education programs. Also, as noted above, the programs offered in 1999 and 2011 were not exactly the same. An overview of the two surveys is presented in Table 2.4 (in this comparison, only the 1999 results for the senior high schools are used). The official 1999 survey and the 2012 CLISS survey each involved simple questions intended both to be answered quickly and to determine the status of CLIL in individual Swedish schools. Although the 2012 survey did not include a question about the target language used for instruction in another language, which the 1999 survey did, this information was offered by many respondents in the open responses. Neither survey aimed to determine the exact extent or scope of CLIL. However, the two reports present the results differently: in the 1999 survey, the reported results were based on the total number of schools contacted rather than the total number of actual responses. The results from the 2012 survey only indicate the analysis of the actual responses. Figure 2.1 presents the comparison of results based on actual responses to the two questionnaires. While the 2012 survey did ask informants to indicate the 450 400 350 300 250 200 150 100 50 0 1999 2012

CLIL 122 151

Figure 2.1 Comparison of results

Planned CLIL 44 15

No CLIL 388 336

30

Part 1: The Context

general extent of CLIL (subsequently categorized as CLIL, Partial CLIL, or Occasional CLIL), for the purposes of comparison in this section, all schools offering some form of CLIL have been compiled into one category of CLIL and the Planned CLIL category remains. Although the response rate in 2012 was also lower than in 1999, there are some interesting comparisons to be made. For example, even though there were 167 more senior high schools in Sweden in 2012, the number of schools with no CLIL was lower than in 1999 (336 versus 388). Another curious note is the fact that the total number of schools offering and/or planning to implement CLIL is the same in both 1999 and 2012: 166 schools. In the category Planned CLIL, 44 schools planned to offer CLIL within the coming school year in the 1999 survey, while in 2012 only 15 schools planned to offer CLIL within two years. The difference, 29 schools, is the same number of schools actually offering some form of CLIL in 2012. However, it is not possible to claim that these are the same schools, as the names of the individual schools in the 1999 survey are unknown. Still, such a comparison would be a very interesting follow-up study. Two specific similarities between the two surveys can also be noted, concerning the type and location of schools as well as the programs offered. In both surveys, more municipal schools offered CLIL than independent schools, although in the 2012 survey the difference is minor (three versus two). In both surveys, CLIL schools were more prevalent in larger municipalities. Finally, the two most commonly offered national programs of study were natural sciences and social sciences programs in 1999 as well as 2012. Figure 2.2 illustrates the changes from 1999 to 2012, as indicated by the number of actual replies to the surveys. In the analysis of the two surveys, some questions about differences demand attention. First, in 1999, 43 schools were described as offering

80% 70% 60% 50% 40% 30% 20% 10% 0% CLIL Planned CLIL No CLIL

1999 22% 8% 70%

Figure 2.2 Changes from 1999 to 2012

2012 27% 3% 67%

Mapping CLIL in Sweden 31

the CLIL option for entire programs, while only five were identified in 2012 (although note that at least two more existed; see above). However, the 43 schools from the 1999 survey may have actually offered only Occasional or Partial CLIL, as the scope and extent of each term is not identified in the 1999 study. For example, one school described CLIL as theme work with content and language integrated learning (Nixon, 2000: 35). Secondly, in 1999 most senior high schools reportedly started with CLIL in Grade 2 (Nixon, 2000: 17), while in in 2012 the five CLIL schools started with CLIL in Grade 1. Therefore, a clear comparison and a complete understanding of the development are both difficult. Finally, as the 2012 analysis revealed that some schools offering CLIL did not respond  to  the questionnaire, the same may be assumed of the 1999 survey. Thus, the results from each survey cannot be seen as an exact representation of the status of CLIL at the time, but rather an indication of the prevalence. 5 Conclusion: A Clear Map of CLIL in Sweden Today?

Is CLIL on the rise in Sweden? The answer to this is not straightforward. The results of the 2012 survey suggest that the number of schools offering instruction through a medium other than Swedish is not actually increasing. In conclusion, the analysis of the 2012 survey together with a comparative analysis with the 1999 survey suggest three issues concerning the scope of CLIL, the language of CLIL and the defi nition of CLIL in the Swedish context. First, according to one broad study (Eurydice Network, 2006), the most common form of CLIL in the European context is what has been categorized as Partial or Occasional CLIL in this chapter: that is, CLIL provision encompassing everything from an hour a week to 50% or more of instruction time. The 2012 study also suggests that Partial or Occasional CLIL may be increasing in the Swedish context, although further studies into exactly which courses are taught partly or occasionally in a foreign language as well as how languages are used in that teaching are needed for a more in-depth understanding of the status of CLIL today. Secondly, in both surveys it was clear that those schools that do choose to offer this option also choose to do so almost exclusively in English, as was also seen in the Agency study on compulsory school education (Swedish National Agency for Education, 2010). Thus, further studies into the role of English as well as the lack of other languages in the CLIL context in Sweden are warranted. Thirdly, the analysis of the 2012 survey indicates that the concept of CLIL, the teaching of non-language content subjects through the medium of a language other than Swedish, is not generally well known or understood by school personnel. This may be partly due to a lack of an official or even commonly accepted defi nition and practice of the educational option offered by the Agency or any other school authorities,

32

Part 1: The Context

although recent research has called for just that (Edlund, 2011; Sylvén, 2013; Yoxsimer Paulsrud, 2014). Instead, CLIL can mean ‘anything’, as Nixon stated already in 2000. For that reason, the present CLISS studies may offer an important and necessary contribution towards a defi nition of CLIL today in Sweden. This mapping of the development of CLIL in Sweden since the official statistics presented by the Agency in 1999 allows us to better understand the context in which the CLISS studies have taken place. As noted in earlier research in the Swedish context, all aspects of CLIL in senior high school are under-researched and the actual extent and scope of subject teaching in another language are difficult to estimate (e.g. Yoxsimer Paulsrud, 2014). Hence is the importance of the CLISS project. In response to the need for more research and a better understanding of Englishmedium instruction in the Swedish context, the Agency (2018) recently completed a new official survey of CLIL programs in senior high schools in 2016. The conclusion is that English-medium instruction has modest or no effects on the English proficiency of the students, and rather may even pose a risk to both their Swedish language development and their acquisition of subject-specific content knowledge. The Agency noted that Englishmedium instruction is unusually unregulated in Sweden. Additionally, teaching content through English in Swedish schools is in discord with both the current Swedish Language Act (Språklag, 2009) and the national curriculum. The Agency therefore proposes that the government should further investigate and review both regulations and practices. The Agency’s most recent results reflect those of the study presented in this chapter. One lesson learned in this survey study is that requesting information from schools is difficult when there is no standard, accepted defi nition of CLIL. This was also noted by the Agency in their 2018 report. Their results indicate that far fewer schools (4%) reported offering Englishmedium instruction in 2016 than in the survey conducted for this study (Swedish National Agency for Education, 2018: 32). This discrepancy may to a large degree be due to the lack of clear defi nitions and understanding of CLIL in the Swedish context. As more research is published and disseminated (for example, by CLISS project members) and the Agency strives for more reliable official statistics (for example, with their 2018 survey report), more schools may be aware of what the defi nition of CLIL is in the Swedish context. These developments, in turn, will allow us to map the spread of CLIL more accurately in the future. Acknowledgements

An earlier version of this chapter was previously presented in the doctoral thesis ‘English-medium instruction in Sweden: Perspectives and practices in two upper secondary schools’. The author wishes to thank Dr Joacim Ramberg, Stockholm University, for assistance with the list of schools.

Mapping CLIL in Sweden

33

References Dentler, S. (2007) Sweden. In A. Maljers, D. Marsh and D. Wolff (eds) Windows on CLIL (pp. 166–171). Alkmaar, The Netherlands: European Platform for Education. Edlund, A. (2011) Undervisning på engelska i den svenska gymnasieskolan – ett experiment med potential? En studie av tre elevgruppers engelska texter i två register [Teaching in English in the Swedish upper secondary school – an experiment with potential? A study of three student groups’ texts in two registers]. Licentiate dissertation, Stockholm University. Eurydice Network (2006) Content and Language Integrated Learning at School in Europe. Brussels: Eurydice European Unit. See http://www.indire.it/lucabas/lkmw_ fi le/eurydice/CLIL_EN.pdf Hedman, C. and Ganuza, N. (2017) Ideology vs. practice: Is there a space for pedagogical translanguaging in mother tongue instruction? In B. Paulsrud, J. Rosén, B. Straszer and Å. Wedin (eds) New Perspectives on Translanguaging and Education (pp. 208– 226). Bristol: Multilingual Matters. Hyltenstam, K. (2004) Engelskan, skolans språkundervisning och svensk språkpolitik [English, language education in school and Swedish language policy]. In B. Lindgren and O. Josephson (eds) Engelskan i Sverige. Språkval i utbildning, arbete och kulturliv [English in Sweden. Language Choice in Education, Work and Culture] (pp. 36–110). Stockholm: Norstedts Förlag. Kjellén Simes, M. (2008) Room for improvement? A comparative study of Swedish learners’ free written production in English in the foreign language classroom and in immersion education. PhD thesis, Karlstad University. Kommun- och landstingsdatabasen (n.d.) [Municipality and county statistical database]. See https://www.kolada.se/. Lim Falk, M. (2008) Svenska i engelskspråkig skolmiljö: Ämnesrelaterat språkbruk i två gymnasieklasser [Swedish in an English-speaking school environment: Contentrelated language use in two high school classes]. PhD thesis, Stockholm University. Lim Falk, M. and Strand, H. (2009) Sprint och svenskan i skolan: Om språkets betydelse för lärande och *språkutveckling* (kunskapsutveckling) [CLIL and Swedish in school: On the role of language in learning and language development (knowledge development)]. Resultatdialog 2009: Aktuell forskning om lärande. Konferensbidrag. Stockholm: Vetenskapsrådets rapportserie. Nixon, J. (2000) SPRINT: Språk- och innehållsintegrerad inlärning och undervisning. Rapport på uppdrag av Skolverket [CLIL: Content and language integrated learning. Report for the National Agency for Education]. Stockholm: Skolverket. Nixon, J. (2001) Quality in SPRINT. Towards Quality Assessment and Assurance in Content and Language Integrated Education. A Field Study Report. Stockholm: Skolverket. SFS (2009) Language Act. Swedish Code of Statutes No. 2009:600. Stockholm: Swedish Ministry of Culture. Stålhammar, M. (2010) Engelskan i svenskan: åttahundra år av lånade ord och språkliga influenser [English in Swedish: 800 Years of Loanwords and Linguistic Influences.] Stockholm: Norstedt. Swedish National Agency for Education [Skolverket] (2010) Undervisning på engelska. Utvärdering av en försöksverksamhet i grundskolan [Instruction in English. Evaluation of a trial implementation in compulsory school]. Report No. 351. Stockholm: Skolverket. See http://www.skolverket.se/publikationer?id=2450. Swedish National Agency for Education [Skolverket] (2018) Engelskspråkig undervisning. En kartläggning och utvärdering av engelskspråkig undervisning i svensk gymnasieskola [English-medium instruction. Mapping and assessment of English-medium instruction in the Swedish upper secondary school]. Report No. 465. Stockholm: Skolverket. See https://www.skolverket.se/publikationer?id=3897.

34

Part 1: The Context

Sylvén, L.K. (2004) Teaching in English or English teaching? On the effects of content and language integrated learning on Swedish learners’ incidental vocabulary acquisition. PhD thesis, University of Gothenburg. Sylvén, L.K. (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320. Yoxsimer Paulsrud, B. (2014) English-medium instruction in Sweden: Perspectives and practices in two upper secondary schools. PhD thesis, Stockholm University.

3 The CLISS Student: Some Background Factors of the Participating Students in the CLISS Project Britt-Marie Apelgren

1 Introduction

The focus in this chapter is on presenting results from an extensive questionnaire study on students’ reported language background, home background, attitudes and experiences of school and classroom work activities and extramural language activities in Swedish and English. As the entire CLISS project is described in Chapter 1, only the factors and issues relevant to this study will be presented here. The aim of this chapter is twofold: first, to present the results from the above-mentioned questionnaire study as part of the CLISS project; and secondly to provide a context for understanding the results in the other chapters in this volume. 2 Background

It is interesting to note that studies suggest that CLIL students in different countries exhibit similar background patterns. In optional CLIL streams, the fi ndings report that students are more motivated and come from higher socio-economic family backgrounds (e.g. Bruton, 2013; Doiz et al., 2014; Gablasova, 2014; Martínez Adrián & Gutiérrez Mangado, 2015; Olsson & Sylvén, 2015; Verspoor et al., 2015). Even without aptitude tests and with open selection, CLIL is regarded by some researchers as discriminatory and elitist and as ‘disguised streaming’ (Bruton, 2013: 593). In addition, CLIL students have been found to be more positive towards school than their non-CLIL peers (Sylvén & Ohlander, 2014; Sylvén & Thompson, 2015; Yoxsimer Paulsrud, 2014; see also Chapter 5, this volume). For example, Yoxsimer Paulsrud (2014: 87) found in her study of Swedish CLIL that stakeholders (parents, head teachers and teachers) believed that the students were ‘highly academically motivated’ 35

36

Part 1: The Context

and that they had been successful in their previous studies and therefore would be expected to have ‘strong chances of continuing to excel in school’. Further, she noted that many of the students came from academic and supportive homes where parents took great interest in their children’s academic progress. Earlier studies of CLIL in the Swedish context also show that CLIL seems to be more popular among girls than boys (e.g. Sylvén, 2004; Yoxsimer Paulsrud, 2014). Hence, it may not come as a surprise that both national and international studies fi nd that CLIL programmes in general have high status and attract motivated and high-achieving students (Dalton-Puffer, 2011; Sylvén, 2004; Yoxsimer Paulsrud, 2014). International studies have shown that the intended aim of increasing language proficiency in CLIL classes is being met (e.g. Admiraal et al., 2006; Lasgabaster & Sierra, 2009; Lo & Murphy, 2010; Ruiz de Zarobe, 2008). However, Bruton (2011, 2013) questions CLIL as ‘2-for-the-priceof-1’ (Bruton, 2011: 588) in his criticism of some research on CLIL and of the lack of a critical stance in CLIL outcomes reported in European studies, as student selection and socio-economic issues in relation to achievement and progress may be absent. In Sweden, though, the picture is somewhat different from most of the European studies. Some research suggests that CLIL has not led to increased English proficiency to the same extent in the Swedish context (Olsson, 2015; Sylvén, 2004; Sylvén & Ohlander, 2014). Internationally, research has also established that student aspirations are a fundamental part of a student’s decision-making process and have an impact on the choices made, for example in school choice and attending higher education (Bowden & Doughney, 2010). Likewise, it has been found that socio-economic status (defined as parental education level, ethnic background, students’ first language (L1) and language spoken at home) has an impact on the aspirations of secondary students from multilingual backgrounds. As Marks (2006) points out, higher status parents have the determination and, in places where education is not for free, the resources to place their children in more academic school environments than lower status parents with equally able students. In Sweden, however, all schooling is tax funded and without fees for the individual, but there is a free and open choice of upper secondary schools where the intake is based on lower secondary grades. Concerning academic achievement, a correlation between the socio-economic status of the family and academic achievement is likewise known (e.g. Reynolds, 1992; Schlee et al., 2009). In several Finnish research studies, it has been noted that parents in CLIL education have a higher educational background and income than average (Nikula, 2005). Aro and Mikkilä-Erdmann (2015) suggest that possible reasons as to why students do well in CLIL are their parents’ background, students’ active and conscious choice of CLIL classes, and a future-oriented view on education. Another issue of importance for CLIL is self-efficacy. One recurrent question in Swedish national language surveys and evaluations is ‘How

The CLISS Student: Background Factors of Participating Students

37

good do you think you are at English?’ (e.g. Erickson, 2004; Oscarson & Apelgren, 2005) and the most frequent answer has been ‘good’. Also in international comparative studies, Swedish students have performed well, especially in a European context (Bonnet, 2004; European Commission/ SurveyLang, 2012). In the Swedish national survey of compulsory school (NU-03), conducted in 2003 by the Swedish National Agency for Education, 87% of the pupils in Grade 9 of compulsory school viewed English as an important subject, even more essential than Swedish. Likewise, over 80% of the parents in the same survey considered English as a very important school subject (Oscarson & Apelgren, 2005). In addition, numerous studies have shown that extramural use of English (English outside school) is common for Swedish students (see, for example, Olsson & Sylvén, 2015; Oscarson & Apelgren, 2005; Sundqvist, 2009; Sundqvist & Wikström, 2015; Sylvén, 2004; Chapter 9, this volume). Regardless of choice of programme at the upper secondary level, young students in Sweden, as elsewhere in today’s global society, need a good command of English for their future studies and careers. English, being a core subject in Swedish schools, is often introduced as early as the fi rst grade (Swedish National Agency for Education, 2012) and is regarded as a very popular and important subject among students (Erickson, 2004; Oscarson & Apelgren, 2005; Sundqvist, 2009). For example, in the earlier mentioned Swedish national survey NU-03, 86% of the Grade 9 students (N = 10,000) said that they liked English (Oscarson & Apelgren, 2005). By offering CLIL classes, the learners’ exposure to English will increase while simultaneously learning subject content. The huge popularity of English among Swedish students in combination with an increased awareness of the significance of English as a language for communication in a global society is one probable reason for the choice of attending CLIL classes. 3 The Study

In this chapter the focus is on some of the important student background factors as found in an initial student questionnaire in the CLISS project in 2011. The data derive from an extensive questionnaire administered in the fi rst term of the students’ three-year upper secondary education. 3.1 The participants

The students (N = 218) were 15–16 years old at the time and came from three different Swedish schools, representing three types of local language policy (see Chapter 1, this volume). Earlier studies on CLIL in the Swedish context show that CLIL seems to be more popular among girls than boys (e.g. Sylvén, 2004; Yoxsimer Paulsrud, 2014), which is in line with the gender differences in the study.

38

Part 1: The Context

Table 3.1 Number of female and male in CLIL and non-CLIL classes CLIL

Non-CLIL

Total number of students

Male students

41 within CLIL: 30%

39 within non-CLIL: 48%

80

Female students

95 within CLIL: 70%

43 within non-CLIL: 52%

138

Total

within CLIL: 100%

within non-CLIL: 100%

218 (100%)

Of those students taking part in the questionnaire, 63% are female and 37% male. A majority of the students in the questionnaire study are CLIL students (62%) compared to non-CLIL students (38%). Table 3.1 shows the percentage of male and female students in CLIL and non-CLIL, respectively. 3.2 The schools

As the participating schools differ (see Chapter 1), it is important to present results on both the school level and class level, which may show differences and similarities between CLIL and non-CLIL students. Table 3.2 summarizes the participating classes and schools. School A is a CLIL school where all subjects except languages are taught in English; both School B and School C are bilingual Swedish and English schools with CLIL and non-CLIL classes. Hence, the amount of English spoken and used in the three CLIL classes differs. Table 3.2 Number of students in different schools and classes CLIL (N = 136)

Non-CLIL (N = 82)

School A (N = 61)

Natural science class (N = 29) Social science class (N = 32)

School B (N = 59)

Natural science class (N = 30)

Natural science class (N = 29)

School C (N = 99)

Social science class (N = 27) Business management and economics class (N = 19)

Business management and economics class (N = 26) Business management and economics class (N = 27)

3.3 The questionnaire

The questionnaire was written in Swedish and comprised 38 questions divided into four parts: (1) language background (13 questions); (2) home background (3 questions);

The CLISS Student: Background Factors of Participating Students

39

(3) extramural language activities in Swedish and English (12 questions); (4) attitudes and experiences of school and classroom work and activities (10 questions). Many of the questions had previously been part of other research projects and the Swedish national evaluation of compulsory school in 2003 (NU-03). This allows for a number of comparisons of the current data with earlier research and evaluations (e.g. Oscarson & Apelgren, 2005; Sundqvist, 2009; Sylvén, 2004). For information about language and family background we used closed questions in combination with open questions. For questions where students needed to express their opinion, attitudes and experiences, closed questions with a 4-point Likert scale were employed. All the items in the closed questions were coded for statistical analysis. The open questions were categorized and in some cases also coded to allow for statistical analysis. This chapter will present results from selected questions in the questionnaire, not the concise data. Instead, issues related to and particularly important for the CLISS study and for the following chapters in this volume will be analyzed, that is: why the students chose to attend CLIL/ non-CLIL classes; their linguistic background; socio-economic factors that may influence the students’ attitudes and linguistic performance in Swedish and English; language experiences; and attitudes to languages and school. The focus will be on the differences and similarities between CLIL and non-CLIL classes and gender. 3.4 The analysis

The data have been analyzed using the statistical social science tool, SPSS (Version 22), and through content analyses of open questions. Statistical methods were used for investigating relationships and comparisons between CLIL and non-CLIL students, between genders and between schools. Descriptive analyses as well as more exploratory analyses, correlations and correspondence analysis were used. 4 Results

In the following, the results from the analyses are presented in four parts: (1) the choice of attending a CLIL programme; (2) family and socioeconomic factors; (3) linguistic background; and (4) English learning and school factors. 4.1 Why CLIL?

Although more girls than boys chose to study at CLIL programmes, boys demonstrate a weak positive correlation to choosing CLIL compared

40

Part 1: The Context

to girls (p = 0.032). This indicates that, for the boys who took part in the questionnaire, there seems to be a considered choice to attend CLIL classes compared to the girls, which may in turn affect their motivation and results. This is one reason to look deeper into the rationale for choosing CLIL programmes. The quotation below from one of the 218 students who took part in the questionnaire summarizes well the reasons why students opted to attend CLIL classes: I have chosen to study at the CLIL programme because it is very broad and I can decide later what to do with my life. That fact that we study in English enables me to study or work abroad after high school, and I will be more prepared to study at universities in Sweden since much of the literature is in English. (First year upper secondary CLIL female student)

One of the open questions asked in the survey was why the students had made the choice of attending a CLIL programme instead of one of the traditional Swedish-taught ones (Q38: Why did you choose to study at this particular programme?). Their answers have fi rst been placed in seven thematic categories: (1) to be able to improve their English proficiency; (2) to prepare for higher education; (3) because they liked English; (4) because they liked their content subjects and wanted to study them in English; (5) because they believed that they would get a broader education; (6) because they believed it would generally be good to know English in the future; and (7) because they believed that they would get a better job in the future (see Table 3.3). (For a deeper understanding of the multicultural students’ particular choices and experiences in one of the CLIL schools, see Chapter 14, this volume). When comparing the CLISS students’ answers with the cohort of the Swedish national survey of compulsory school (NU-03) we can see similar reasons and motivations for learning English: 87% said that English was Table 3.3 Reasons for choice of CLIL Reasons for choice of CLIL

Instances

Examples

1 Better English proficiency

33

‘… to develop my English and I get so much for free when surrounded by English all day.’

2 For higher education

32

‘… because the course literature and many university studies are in English.’

3 Like English

27

‘I just love to speak English.’

4 Like social/natural sciences

25

‘I like Social Sciences and human right issues.’ ‘I like Biology, Maths and languages.’

5 Broad education

23

‘I want a broad education with lots of options later on.’

6 For future need

16

‘… good for the future.’

7 Better job opportunities

16

‘… to get a good and well paid job later on.’

The CLISS Student: Background Factors of Participating Students

41

important to know; 79% said that it was important for further studies; and 69% said that it was important for future work. Only 25% stated that they only worked for the tests and not for the knowledge per se (Oscarson & Apelgren, 2005). As indicated, the students in the CLISS sample mentioned both intrinsic motives and more pragmatic/external reasons for their choices. Those reasons are in line with the results in Yoxsimer Paulsrud (2014). In order for deeper statistical analyses, the students’ answers were further categorized into three different variables: ‘better English proficiency’ (Category 1); ‘intrinsic reasons’ (Categories 3 and 4); and ‘pragmatic/external reasons’ (Categories 2, 5, 6 and 7). By doing this, a clear pattern of reasons emerged between different groups. In a bivariate correspondence analysis, the relationship between gender, choice of CLIL and studies of different languages were tested. The results showed that boys and girls who had not opted for CLIL classes had chosen their programmes for pragmatic reasons. Among the CLIL students, girls chose CLIL on the basis of ‘better English proficiency’ and for ‘intrinsic reasons’, whereas the boys showed a slightly different pattern for choosing CLIL. Boys were more inclined to choose CLIL because of ‘better English proficiency’ and for ‘pragmatic/external reasons’ (p = 0.001). The different profiles are shown in Figure 3.1. Row and Column Points Symmetrical Normalization 1.5

Why CLIL Gender_CLIL

1.0

Male_CLIL

0.5 Dimension 2

Better English proficiency

Female_nonCLIL External reasons

Female_CLIL

0.0

Male_nonCLIL

Intrinsic reasons

–0.5

–1.0

–1.5 –1.5

–1.0

–0.5

0.0

0.5

1.0

1.5

Dimension 1

Figure 3.1 Bivariate correspondence analysis: correspondence between gender, choice of CLIL and studied languages (p = 0.001)

42

Part 1: The Context

In addition, the girls who chose CLIL also studied languages to a greater extent. The boys who chose CLIL differed from the corresponding girls in that they tended to study only Swedish and English. However, there was a reverse choice of languages in the non-CLIL classes, where the boys chose to study Swedish-English plus two modern languages, while the corresponding girls studied mostly Swedish-English plus one modern language, often German (p = 0.001). Further, there was also a clear pattern in which the Swedish-born students who chose CLIL did this due to intrinsic reasons or motivation as well as to improve their English proficiency levels. Students born abroad chose CLIL due to both intrinsic and pragmatic reasons. There were very few students of foreign descent who chose CLIL primarily to improve their English. These results correspond well to Otterup’s results (see Chapter 14, this volume). To summarize, we can establish that there is a significant relationship between language studies, choice of CLIL and gender (p = 0.001). 4.2 Family matters and socio-economic factors

Considering the strong fi ndings concerning the impact of parental and socio-economic factors on the choice of CLIL, it is crucial to explore these factors in the CLISS data. Thus, in addition to language-related questions, the questionnaire also provided questions about socio-economic character, such as housing and parents’ educational level, which are known as essential socio-economic factors. However, in the present study it has proved difficult to account for housing as a reliable socio-economic factor, mainly due to the different geographical locations of the three schools – one inner-city school with a broad uptake (School A), and the other two schools in middle-sized towns, one in mid-Sweden (School B) and one in the south (School C). In addition, the descriptive data show that the CLISS students do not correspond to the average figures for households’ housing in Sweden. On average, 50% of all Swedes live in privately owned houses, 15% in privately owned apartments, 25% in rented apartments and 10% in special housing such as students’ homes (SCB statistics, 2017). In our data, the figures for housing for School A are 36% privately owned houses, 21% privately owned apartments and 42% rented apartments. The equivalent figures for School B are 84% privately owned houses, 3% privately owned apartments and 13% rented apartments, and for School C 77% privately owned houses, 7% privately owned apartments and 16% rented apartments. Despite the general importance of housing as a socioeconomic factor, ‘housing’ has been excluded as a socio-economic factor in the following due to geographical differences. Concerning parental educational level, questions were asked about the mothers’ and fathers’ education and occupation (Q15a: What is your mother’s and father’s occupation? Q15b: What educational level does

The CLISS Student: Background Factors of Participating Students

43

Figure 3.2 CLIL and non-CLIL parents’ educational level

your mother and your father have: primary, secondary or tertiary?). At one end of the educational scale, the CLIL students’ mothers are the best educated with 62% having a higher education degree compared to, at the other end, the non-CLIL fathers with 37% having a higher education degree. The CLIL students’ parents on the whole seem to have higher educational levels (Figure 3.2). Looking more closely at the three schools, we can see that there is a great difference between the mothers’ and fathers’ educational levels at the different schools and between genders (Table 3.4). At School B 77% of the mothers have tertiary education compared to, for example, only 29% of the fathers in School C. Sixty percent of all of the students’ mothers have higher education degrees and 43% of their fathers. These figures should be compared to the average percentage for people with higher education degrees in Sweden, which is 35%. Table 3.4 Parents’ educational level at the three schools Mothers

School A

School B

School C

Total – Mother

Compulsory

14%

5%

3%

6%

Secondary

29%

18%

45%

34%

Tertiary

57%

77%

52%

60%

Fathers

School A

School B

School C

Total – Father

Compulsory

18%

10%

16%

15%

Secondary

23%

39%

55%

42%

Tertiary

59%

51%

29%

43%

44

Part 1: The Context

There are clear and significant correlations between (1) parents’ highest educational level, (2) CLIL and (3) parentage origin. We can see a pattern in which the Swedish-born students who had not chosen CLIL have parents with no more than secondary school education, while the Swedish-born students who chose CLIL have parents with university education to a greater extent. For immigrant students who chose CLIL, the parents’ educational levels are varied, but essentially those parents have primary school education. To establish the effect of parental background, a multivariable logistic regression analysis was performed looking at the effect of gender, nationality, motivation for CLIL and parents’ highest level of education in relation to the choice of CLIL (Mplus, robust standard error, non-standardized and standardized odds ratios; Table 3.5). Significant effects were found between being female, choosing CLIL to become better in English in relation to external motivation, as well as having a parent with secondary or university education as opposed to having a parent with primary education. The highest correlations exist for the educational background, where students with parents with upper secondary education and/or university education are 12 times more likely (the log odds) to choose CLIL compared to students with parents having lower secondary education. The standardized correlation located is more than 0.84 between the highest educational level and choice of CLIL. Furthermore, there is a significant negative correlation between choosing CLIL from the aspect ‘to be better

Table 3.5 The effect of gender, nationality and motivation to CLIL and parents’ highest educational level for the choice of CLIL classes (multivariable logistic regression analysis; non-standardized and standardized odds quotas) Beta

Intercept

B

Lower 95% C.I. odds ratio

Upper 95% C.I. odds ratio

S.E.

p

13.39

1.93

11.13

12.80

3.61

0.000

−0.87

−0.06

−0.13

−0.02

−2.14

0.032

−12.67

−0.74

−0.83

−0.64

−12.71

0.000

1.20

0.08

0.04

0.53

1.69

0.092

−2.02

−0.10

−0.40

−0.04

−0.76

0.449

University

12.44

0.84

0.46

0.95

7.83

0.000

Upper secondary

13.29

0.85

0.50

1.00

7.23

0.000

Gender Female Motivation Better Prof.Eng Intrinsic Nationality Swedish Parental highest education level

The CLISS Student: Background Factors of Participating Students

45

in English’ compared to the aspect ‘external causes’. Ethnic background showed no statistical significance in the analysis. Yet another indicator of parental support is engagement in helping with students’ homework (Q34: Do you receive help with your homework? If ‘yes’, indicate from whom: mother, father, sibling, friend or other). In our study, two-thirds of the students answered that they received help with their homework. Of those who received help, more than 50% did so from their mothers, 20% from their fathers and the rest from siblings or friends, respectively. There is a slight difference between CLIL and non-CLIL students, as the CLIL students’ mothers helped with their homework to a greater extent than the non-CLIL mothers (55% compared to 45%). The variation is not significant; still, it is interesting to note that the mothers whose education is higher on average are the ones who are more active in helping their children with homework. This is in line with international research where, for example, the mother’s education has been found to be an important predictor for school success (Zhan, 2006). In summary, the current survey revealed significant correlations between parents’ education and choice of CLIL (p = 0.001). 4.3 The CLISS students’ linguistic background

Of specific interest for the CLISS study has been to map the students’ linguistic backgrounds. Closely related to L1s is the question of place of birth. One of the fi rst questions concerned where they were born (Q2: Were you born in Sweden? If not, how old were you when you moved to Sweden?). In addition, we posed several questions related to the students’ linguistic background. The students’ answers to the questions below are the focus of this section: Q3a: Q3b: Q4: Q5: Q6: Q7: Q9: Q10: Q11: Q12: Q13:

What languages do you speak every day? What languages have you studied in school? What language(s) did you learn first? When did you start learning English? If you speak other languages, how old where you when you started to speak those languages? What languages are used most at home? What language(s) do you use together with your friends? Have you attended mother tongue classes at primary level, lower secondary level or upper secondary level? Have you attended Swedish as a second language classes at primary level, lower secondary level or upper secondary level? What language do you know best? What language do you mostly use?

There is a difference between the CLIL and the non-CLIL classes as regards how many of them were born in Sweden or in another country: 98%

46

Part 1: The Context

Figure 3.3 Percentage of students at different schools born in Sweden or in another country (Q2)

of the students in the non-CLIL classes were born in Sweden compared to 83% in the CLIL classes. When comparing our different schools, we can see that School A, the CLIL-only school, diverges from the other two schools, where as many as 25% of the students were born outside Sweden (Figure 3.3). Other questions we asked concerned the students’ L1 and when, where and with whom they spoke their L1. Eighty-five percent of the students in the questionnaire answered that Swedish was their L1, which is in line with the national average percentage of L1s other than Swedish. However, 24% of upper secondary students speak mother tongues other than Swedish according to official data (Swedish National Agency for Education, 2015). The discrepancy may be due to students’ viewing more languages than one as their mother tongue. The most common other L1s in the CLISS cohort are the following: Arabic, Kurdish, English, Albanian, followed by Bosnian, Greek, Hebrew, Chinese, Macedonian, Persian, Punjabi, Russian, Syrian, Swahili, Tagalog, Tigre, Turkish, German and Wolof. The list is impressive and shows that many of the participating students have a bi- or multilingual background, that is, Swedish and another language or Swedish and two other languages (for an in-depth picture of some of the multilingual CLISS students, see Chapter 14, this volume). When looking at CLIL and non-CLIL students there is a substantial difference between the two groups: almost all (98%) of the nonCLIL students claimed that Swedish is their L1 compared to 66% of the CLIL students as illustrated in Figure 3.4. In spite of the fact that almost a third of the CLIL students indicate that they do not have Swedish as their L1 and there is a national free option of attending Swedish as a second language (L2) class instead of Swedish class, only 11% of the students stated that they had attended Swedish L2 classes, mainly in primary school. Most of those students come from School A. However, the results are consistent with the national proportion of all students who studied L2 Swedish in the 2011/2012 school year, which was just under 8% (Swedish National Agency for Education,

The CLISS Student: Background Factors of Participating Students

47

Figure 3.4 Percentage of first language for CLIL and non-CLIL students (Q4)

2013). As with the option of L2 Swedish classes, there is an obligation for local educational authorities in Sweden to offer mother tongue classes to students with mother tongues other than Swedish. Again, the CLIL students from School A were those attending the most mother tongue classes, although all three schools have students who have attended or are attending such classes. In addition, we wanted to fi nd out which and to what extent different languages were used in communication. Several questions were therefore asked about the students’ language(s) of communication with family and peers (see above). Figure 3.5 shows that among CLIL students combinations of (or mixed) ‘Swedish and English’ (62%), ‘English and other languages’ as well as ‘other languages only’ were used in communication with peers more often than among the non-CLIL students. As seen in Figure 3.5, many languages are at play in the students’ daily interaction and are used for communication with friends, especially among the CLIL students. By comparing the linguistic factors in the three schools, we can see that Swedish is rarely used alone in peer communication at School A, whereas School C is the school where Swedish is mostly used and where ‘mixed Swedish and English’ is less often used than at the

Figure 3.5 Language of communication with peers, CLIL and non-CLIL students (Q9)

48

Part 1: The Context

other two schools. Regardless of linguistic background, English is the dominant language for communication, both along with and as a complement to Swedish. 4.4 English, self-efficacy and schooling

In the present study the students indicated high self-efficacy and beliefs in being good at English (Q28: How good are you at English?). However, the CLIL and non-CLIL students showed somewhat different patterns as regards their self-assessment of their abilities in English. The CLIL students believed they were ‘very good’ at English to a greater extent than the non-CLIL students; in School A as many as 43% of the CLIL students believed that they were ‘very good’ at English, compared to 25% in the non-CLIL classes at School C. One self-efficacy factor might be if the students had visited and spent some time in English speaking countries. To the question Q25: Have you visited a country where English is spoken?, 63% answered ‘yes’, with no real difference between the CLIL and nonCLIL students. However, when mentioning the visited country almost 20% of the non-CLIL students stated non-English speaking common tourist countries such as Greece and Spain. One or two comments suggested that the students regarded using English as a lingua franca as equivalent to ‘visiting an English speaking country’. Another important factor is that trips abroad could also be viewed as a socio-economic variable, insofar as not all parents can be expected to afford holidays abroad (Sundqvist, 2009). A frequent research question concerns how much English and Swedish the students believe they learn in school as compared to outside school (Q24b: Where do you believe you learn English?). It is interesting to note that, in spite of the many hours students spend using English in their spare time, the fi rst year students in the CLISS study still believed that most of their learning in English takes place in school (Table 3.6). A comparison with both the national NU-03 sample (Oscarson & Apelgren, 2005) and the sample in Sundqvist’s (2009) study reveals some Table 3.6 Percentage of ‘Learning English and Swedish in or outside school’ (Q23) CLIL or non-CLIL

Extramural

Almost all extramural

Almost all in school

In school

11

36

45

8

8

24

55

13

English CLIL Non-CLIL

Swedish CLIL

25

35

30

10

Non-CLIL

16

43

29

12

The CLISS Student: Background Factors of Participating Students

49

interesting patterns. In the NU-03, 18% said that they had learned most of their English in school and 82% responded outside school. Almost the reverse was found in Sundqvist’s study: approximately 69% of the students said that they had learned most of their English in school and 31% that they had learned most outside school. In our data, the equivalent figures differ between CLIL and non-CLIL students: 47% of CLIL students said that they learned most of their English outside school compared to 32% in the non-CLIL group. This means that the non-CLIL students showed similar results to the students in Sundqvist’s (2009) study. The CLIL students believed they learned more English in school to a much greater extent than the students in the NU-03 evaluation. When asked about learning Swedish (Q24a: Where do you believe you learn Swedish?), the CLIL students and the non-CLIL students showed similar results: 60% believed that they learned Swedish mostly outside school and 40% in school. Several questions in the background survey were related to extramural English, such as reading and writing activities. One question concerned how much the computer was used for various activities in Swedish and in English, e.g. doing homework, listening to music, watching films, playing computer games and chatting/using Facebook (Q23: Do you use the computer in your spare time? If yes, when do you use it?). Olsson and Sylvén (2015) found in one of the CLISS sub-studies that the CLIL students used English in their spare time to a significantly greater extent than the nonCLIL students (for further results on extramural English, see Chapter 9, this volume). Similar questions on extramural English were asked in the two national evaluations of English in 2003 and 1992: NU-03 and NU-92. A comparison may thus indicate how extramural activities have changed during the last 20 years. The most significant change during this time is the use of the internet: 98% of the students in the CLISS study indicated that they used the internet for chatting, MSN, Facebook and YouTube compared to 30% in the NU-03 study from 2003. Concerning how much English is spoken in class, research has found that this differs considerably even in CLIL classes, which are supposed to be taught in English (Lim Falk, 2008). In our study, which comprises both CLIL classes and non-CLIL classes, there is a difference in the answers between the two categories of classes already at the start of the fi rst term (Q31: How much of the English lesson time do teacher and students alike use English as the medium of instruction?). In Table 3.7, CLIL and nonCLIL students’ self-reported answers are compared with those of the NU-03 sample (all categories aged 15–16 years old). We can note that, although the students’ reports of English used in the English classroom seem to have increased, the non-CLIL students show a similar pattern to the students in the national evaluation 2003 (Oscarson & Apelgren, 2005). One would expect no difference between CLIL and nonCLIL students’ reported experiences at the very beginning of upper

50

Part 1: The Context

Table 3.7 Percentage of lesson time in English class using English as the medium of instruction (CLISS and NU-03) (Q31) CLIL or non-CLIL

All the time

Almost all the time

Half the time

Less than half the time

Hardly any time

Never

23

58

10

3

5

1

10

41

37

10

2

0

36

26

15

10

4

CLISS CLIL Non-CLIL

NU-03 Non-CLIL

10

secondary schooling. Compared to the 1992 national evaluation (NU-92), however, the amount of English has indeed increased. Nevertheless, one would expect that more than 50% of the lesson time would be spent speaking English, especially as the students report extensive use of English outside the classroom (Olsson & Sylvén, 2015). As indicated in Table 3.7, there are differences between the reported use of English in CLIL and non-CLIL classes. At School A with only CLIL classes, as much as 80% of the English class time is spent communicating in English. In the other CLIL classes at Schools B and C, only between 25% and 30% of the English lesson time is spent speaking English, and as little as 20–23% in the non-CLIL classes. There might be several reasons for this difference. At School A, where English is the language of instruction for all subjects except languages other than English, the students are surrounded by English to a greater extent. The effect of this may also increase the use of English in the English classrooms and foster a more English speaking culture. Another plausible explanation is that the students did not specifically indicate ‘lesson time in English using English as a medium of instruction’, but rather reported ‘lesson time using English’, including all other subjects being taught in English. Whichever explanation, overall the results indicate that less time is spent on using English in the classrooms and an increasing amount of time is spent on using English in a variety of activities outside school. 5 Conclusions

In this chapter background facts and figures from the initial student questionnaire in the CLISS study have been reported and discussed. The aim of this chapter has been to provide information that may help in understanding the results reported in the following chapters. The survey was conducted early in the project, when the students had just started their upper secondary education – most of them in CLIL classes and around a third of the students in non-CLIL classes. Three years in a young person’s life may seem like a lifetime, and their cognitive development can be remarkable, as many upper secondary

The CLISS Student: Background Factors of Participating Students

51

teachers can testify. The results are based on the information, descriptions, experiences and attitudes of students who were then 15–16 years old. Information on parental issues and language background do not alter, but attitudes and experiences may have changed during the three years. However, a description of the CLISS students has emerged, as discussed below. The students’ language background corresponds well to average figures in Sweden, both concerning L1s and participation in L2 Swedish lessons and mother tongue tuition. Still, there are differences, sometimes large, between the CLIL and non-CLIL students. For example, the CLIL students are more multilingual than the non-CLIL students and use a variety of languages in communication with both family and peers. There are also indications that CLIL students use English more often in their spare time. In addition, the students believe that they are ‘good’ or ‘very good’ at English, although the CLIL students believe so to a greater extent. Exposure to languages and language use outside school, as well as learning several languages, are factors that have a high probability of correlating with better and more advanced language skills and proficiency according to SurveyLang (Swedish National Agency for Education, 2012). If we add the students’ positive perceptions of the benefits of learning the language and their high estimation or positive perceptions of their language skills, the CLISS student should excel language-wise and academically. However, as the results in the remainder of this volume reveal, this is not always the case. All the students’ parents in the CLISS study have higher educational levels than average Swedes – even higher for the CLIL students’ parents than the non-CLIL students’ parents. Socio-economic factors are found to be significant for the choice of attending CLIL classes; in particular, parents’ education is influential. We could see that the likelihood of choosing CLIL increased by a factor of 12 if the parents had higher education. The international discussion of CLIL being selective and students being drawn from academic homes where parents take great interest in their children’s academic progress (Aro & Mikkilä-Erdmann, 2015; Bruton, 2013; Doiz et al., 2014; Gablasova, 2014; Martínez Adrián & Gutiérrez Mangado, 2015; Olsson & Sylvén, 2015; Verspoor et al., 2015) can thus be supported in this study. These background factors are important to keep in mind when exploring the CLISS students’ results in this volume. References Admiraal, W., Westhoff, G. and de Bot, K. (2006) Evaluation of bilingual secondary education in the Netherlands: Students’ language proficiency in English. Educational Research and Evaluation 12 (1), 75–93. Aro, S. and Mikkilä-Erdmann, M. (2015) School-external factors in Finnish content and language integrated learning (CLIL) Programs. Scandinavian Journal of Educational Research 59 (2), 127–142.

52

Part 1: The Context

Bonnet, G. (ed.) (2004) The Assessment of Pupils’ Skills in English in Eight European Countries. Thessaloniki: European Network of Policy Makers for the Evaluation of Education Systems. See http://cisad.adc.education.fr/reva/english/publicationsurvey. htm. Bowden, M. and Doughney, J. (2010) Socio-economic status, cultural diversity and the aspirations of secondary students in the Western Suburbs of Melbourne, Australia. Higher Education 59, 115–129. Bruton, A. (2011) Is CLIL so benefi cial, or just selective? Re-evaluating some of the research. System 39 (4), 523–532. Bruton, A. (2013) CLIL: Some of the reasons why … and why not. System 41, 587–597. Dalton-Puffer, C. (2011) Content-and-learning integrated learning: From practice to principles. Annual Review of Applied Linguistics 31, 182–204. Doiz, A., Lasagabaster, D. and Sierra, J.M. (2014) CLIL and motivation: The effect of individual and contextual variables. The Language Learning Journal 42 (2), 209–224. Erickson, G. (2004) Engelska i åtta europeiska länder – en undersökning av ungdomars kunskaper och uppfattningar. Stockholm: Skolverket. European Commission/SurveyLang (2012) First European Survey on Language Competences. Final Report, 12 January. See http://ec.europa.eu/languages/eslc/ index.html. Gablasova, D. (2014) Issues in the assessment of bilingually educated students: Expressing subject knowledge through L1 and L2. The Language Learning Journal 42 (2), 151–164. Lasagabaster, D. and Sierra, J.M. (2009) Language attitudes in CLIL and traditional EFL classes. International Journal of CLIL Research 1, 4–17. Lim Falk, M. (2008) Svenska i engelskspråkig skolmiljö. Ämnesrelaterat språkbruk i två gymnasieklasser. (Acta Universitatis Stockholmiensis. Stockholm Studies in Scandinavian Philology. 46.) Stockholm: Eddy. Lo, Y.Y. and Murphy, V.A. (2010) Vocabulary knowledge and growth in immersion and regular language-learning programmes in Hong Kong. Language and Education 24 (3), 215–238. Marks, G.N. (2006) Are between- and within-school differences in student performance largely due to socio-economic background? Evidence from 30 countries. Educational Research 48 (1), 21–40. Martínez Adrián, M. and Gutiérrez Mangado, M.J. (2015) Is CLIL instruction beneficial in terms of general proficiency and specific areas of grammar? Journal of Immersion and Content-Based Language Education 3 (1), 51–76. Nikula, T. (2005) English as object and tool of study in classrooms: Interactional effects and pragmatic implications. Linguistics and Education 16, 27–58. Olsson, E. (2015) Progress in English academic vocabulary use in writing among CLIL and non-CLIL students in Sweden. Moderna Språk 109 (2), 51–74. Olsson, E. and Sylvén, L.K. (2015) Extramural English and academic vocabulary: A longitudinal study of CLIL and non-CLIL students in Sweden. Apples – Journal of Applied Language Studies 9 (2), 77–103. Oscarson, M. and Apelgren, B.-M. (2005) Nationella utvärderingen av grundskolan 2003. Engelska. Stockholm: Skolverket. Reynolds, A.J. (1992) Comparing measures of parental involvement and their effects on academic achievement. Early Childhood Research Quarterly 7, 441–462. Ruiz de Zarobe, Y. (2008) CLIL and foreign language learning: A longitudinal study in the Basque country. International CLIL Research Journal 1, 60–73. SCB statistics (2017) See http://www.scb.se/he0111 (accessed 22 September 2017). Schlee, B.M., Mullis, A.K. and Shriner, M. (2009) Parents’ social and resource capital: Prediction of academic achievement during early childhood. Children and Youth Services Review 31, 227–234.

The CLISS Student: Background Factors of Participating Students

53

Sundqvist, P. (2009) Extramural English matters: Out-of-school English and its impact on Swedish ninth graders’ oral profi ciency and vocabulary. PhD thesis, Karlstad University. Sundqvist, P. and Wikström, P. (2015) Out-of-school digital gameplay and in-school L2 English vocabulary outcomes. System 51, 65–76. Swedish National Agency for Education [Skolverket] (2011) Curriculum for the Compulsory School System, the Pre-school Class and the Recreation Centre (Lgy11). Stockholm: Skolverket. Swedish National Agency for Education [Skolverket] (2012) Internationella språkstudien 2011 [The International Language Survey 2011]. Report No. 375. Stockholm: Skolverket. Swedish National Agency for Education [Skolverket] (2013) Facts and Figures 2012. Report No. 383. Stockholm: Skolverket. Swedish National Agency for Education [Skolverket] (2015) Sveriges offi ciella statistik 2014/15. Skolor och elever i gymnasieskolan läsår 2014/15. Stockholm: Skolverket. Sylvén, L.K. (2004) Teaching in English or English teaching? On the effects of content and language integrated learning on Swedish learners’ incidental vocabulary acquisition. PhD thesis, University of Gothenburg. Sylvén, L.K (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320. Sylvén, L.K. and Ohlander, S. (2014) The CLISS project: Receptive vocabulary in CLIL versus non-CLIL groups. Moderna Språk 108 (2), 80–114. Sylvén, L.K. and Thompson, A. (2015) Language learning motivation and CLIL. Is there a connection? Journal of Immersion and Content-Based Language Education 3 (1), 28–50. Verspoor, M., de Bot, K. and Xu, X. (2015) The effects of English bilingual education in the Netherlands. Journal of Immersion and Content-Based Language Education 3 (1), 4–27. Yoxsimer Paulsrud, B. (2014) English-medium instruction in Sweden: Perspectives and practices in two upper secondary schools. PhD thesis, Stockholm University. Zhan, M. (2006) Assets, parental expectations and involvement, and children’s educational performance. Children and Youth Services Review 28, 961–975.

Part 2 Assessment and Motivation

Introduction to Part 2: Assessment and Motivation

In Part 1, details about CLIL in Sweden and the CLISS project in general were explored. In this part, two areas of great importance for learners – and language learners in particular – come into focus: assessment and motivation. Assessment is at the heart of learning, and can be of the utmost importance for the individual learner. It is crucial that issues in connection with assessment practices are carefully analyzed and that transparent guidelines are available, as they are a necessary tool for teachers in their endeavors to apply fair and appropriate assessment procedures. Tests and other types of assessment are used both of and for learning; thus, they should also provide learning potential for the students (Erickson & Gustafsson, 2005). With a CLIL approach, assessment practices become more challenging than in regular teaching. In addition to evaluating the students’ content knowledge and progress, aspects of language use and understanding should also be included. In most CLIL contexts, however, guidelines on how to assess both content and language are non-existent. The lack of a general awareness of as well as a consensus on this important aspect of CLIL is brought up in Chapter 4, where Helena Reierstam and Liss Kerstin Sylvén give an overview of the literature on CLIL and assessment, and present details about existing assessment practices found within the CLISS project. The other area of concern in this section is motivation. Numerous studies demonstrate the significant role of motivation for a successful (language) learning process (Dörnyei, 2001; Dörnyei et al., 2014; Gardner, 2010; Gardner & Lambert, 1959). Several studies have investigated the role of motivation in connection with CLIL, many of which conclude that CLIL students appear to be more motivated than their non-CLIL peers (Abendroth-Timmer, 2007; Banegas, 2013; Doiz et  al., 2014; Lasagabaster  & Sierra, 2009). However, most of these studies do not account for the students’ pre-CLIL level of motivation, making it difficult to draw conclusions about effects of CLIL per se. Furthermore, several of the above studies implemented a cross-sectional design, while in fact

56

Introduction to Part 2: Assessment and Motivation

57

longitudinal studies would provide a more nuanced view of the motivation of CLIL students. Chapter 5, by Amy S. Thompson and Liss Kerstin Sylvén, deals with this important aspect of motivation within CLIL. The chapter is intended to fi ll a gap in the literature on CLIL and motivation, following up on a previous study on initial levels of motivation among the CLISS participants, and thus providing a longitudinal perspective on motivation, CLIL and gender. References Abendroth-Timmer, D. (2007) Akzeptanz und Motivation: Empirische Ansätze zur Erforschung des Einsatzes von bilingualen und mehrsprachigen Modulen. Frankfurt am Main: Peter Lang. Banegas, D.L. (2013) The integration of content and language as a driving force in the EFS lesson. In E. Ushioda (ed.) International Perspectives on Motivation: Language Learning and Professional Challenges (pp. 82–97). New York: Palgrave Macmillan. Doiz, A., Lasagabaster, D. and Sierra, J.M. (2014) CLIL and motivation: The effect of individual and contextual variables. The Language Learning Journal 42 (2), 209–224. Dörnyei, Z. (2001) Teaching and Researching Motivation. Harlow: Longman. Dörnyei, Z., MacIntyre, P.D. and Henry, A. (eds) (2014) Motivational Dynamics in Language Learning. Bristol: Multilingual Matters. Erickson, G. and Gustafsson, J.-E. (2005) Some European Students’ and Teachers’ Views on Language Testing and Assessment. Lancaster: European Association for Language Testing and Assessment. See http://www.ealta.eu.org/resources.htm. Gardner, R.C. (2010) Motivation and Second Language Acquisition. New York: Peter Lang. Gardner, R.C. and Lambert, W.E. (1959) Motivational variables in second language acquisition. Canadian Journal of Psychology 13, 266–272. Lasagabaster, D. and Sierra, J.M. (2009) Language attitudes in CLIL and traditional EFL classes. International CLIL Research Journal 1 (2), 3–17.

4 Assessment in CLIL Helena Reierstam and Liss Kerstin Sylvén

1 Introduction

At the core of pedagogical issues in CLIL is the relationship between language and subject content. All subjects have their own kind of literacy, including specific linguistic and discourse features. At the same time, the language of schooling implies shared cross-disciplinary academic registers, depending on genre. Learners need to master the subject literacies – the ‘languages’ of biology and history, for example – as well as more general academic language in order to successfully communicate subject content and what has been learnt. A CLIL classroom offers a bilingual learning environment in which a second or a foreign language (L2/FL) is used as the language of instruction in subject content courses. In assessment situations, students’ language may either enhance or limit their ability to present knowledge. The question is how the role and impact of language as a medium both for learning and for communicating knowledge are dealt with in teachers’ assessment practices in a CLIL context. This chapter relates to a specific study within the CLISS project, focusing on senior high school teachers’ attitudes to assessment. The teachers represent two subject content courses, in biology and history. The study seeks to provide insights into the teachers’ assessment practices and the data consist of teacher interviews, questionnaires and an analysis of teachers’ written assessment design. In order to fi nd out if, and in that case how, the assessment practices differ depending on the language of instruction, both CLIL and non-CLIL subject teachers were included. Additionally, English as a foreign language (EFL) teachers at the same schools were interviewed. The goal was to fi nd out whether course content and assessment practices in the EFL classes were affected as a result of English being used as a medium of instruction in the other courses. Finally, a cross-disciplinary comparison was made in order to distinguish possible common features concerning the language, content and design of the written assessment samples. The results of the study indicate the need to defi ne target language learning outcomes, not only for subject-specific terminology but also regarding general academic written language, in order to establish more conscious language-oriented assessment practices in CLIL instruction. 59

60 Part 2: Assessment and Motivation

2 Background

In the CLISS project, the main focus is on the possible effects the language of instruction has on students’ development of written academic registers in English as well as Swedish. Another issue of interest concerns students’ mastery of subject-specifi c concepts in the relevant disciplines, the languages of biology and history. Whether language is systematically and consciously taught in the subject content courses or whether the CLIL classroom is seen as an environment providing opportunities for learning language through immersion rather than explicit teaching becomes a matter of teachers’ pedagogical choice. In a context where no formal guidelines are articulated for CLIL teachers as regards the teaching of language (Sylvén, 2013; see also Chapter 1), the how and what of assessment depend on teacher cognition and teachers’ interpretation of national course goals. Validity in assessment is defi ned in terms of whether a test measures what it is intended to measure (Messick, 1989). In content courses, where a test aims at assessing curricular goals associated with subject content rather than language (if, indeed, they are separable), test items should not require language skills which may limit students’ responses. If students may be expected to perform less well on a test in a foreign language compared to doing it in their fi rst language (L1), validity may be at risk. Therefore, an important issue involves the role of language, and how language is perceived, when assessing content knowledge. There are several reasons for highlighting assessment in connection with CLIL. First of all, assessment is an integral part of education, and so of CLIL. Secondly, issues of validity, as already mentioned, come to the fore when the language of instruction is not the students’ L1. Thirdly, the reliability of students’ results may be in doubt if assessment practices are not adapted to the specific CLIL educational context. The following subsections will consider these aspects somewhat further, providing the backdrop against which the rest of the chapter should be seen. 2.1 Assessment as an integral part of education

Education in all forms includes aspects of assessment. There are highstakes tests on the basis of which decisions are made on, for instance, entrance into higher education, and there are low-stakes tests where the main purpose is to give an indication of the level of proficiency within a certain domain at a given point in time. Further, there are summative and formative tests (Scriven, 1996). Summative tests aim at measuring an individual’s level of proficiency, forming the basis on which grades are given, whereas the purpose of formative tests is to gain an understanding of an individual’s position in the learning process, from there taking the necessary steps to move forward. Formative assessment is also referred to as

Assessment in CLIL 61

assessment for learning (Black et al., 2002). In other words, formative assessment is seen as a tool to facilitate the attainment of learning goals. Thus, assessment in some form is needed in order for teachers to know what their students have learnt and what they need to improve, and for learners to know what their strengths and weaknesses are. Assessment is also an important way for parents to be informed about the progress of their child. At a societal level, too, assessment is a crucial source of information about the quality of the entire educational system (see, for instance, the regular evaluations of European school results as carried out in the PISA tests, https://www.oecd.org/pisa/aboutpisa/). All of the above perspectives are, of course, as vital in CLIL education as in any other educational context. 2.2 Validity

For tests and assessment to be regarded as valid and reliable, a number of factors need to be taken into account. To start with the broader concept of validity, a test should measure what it is intended to measure, as already noted (Messick, 1989). Taking a test in biology as an example, it can measure the level of knowledge within a specific area of biology covered in the classroom up to the point of the test. A test can also purport to measure the level of knowledge attained at the end of a course for summative purposes. Either way, the test should ideally be relevant in both content and coverage in relation to its intended purpose(s), and it should also have predictive utility (Bachman & Palmer, 1996; Wiliam & Thompson, 2008). Hence, construct underrepresentation, i.e. ‘the assessment doesn’t assess the things it should’ (Wiliam & Thompson, 2008), as well as construct-irrelevant variance, i.e. ‘the assessment assesses things it shouldn’t’ (Wiliam & Thompson, 2008), should be avoided. In other words, the biology test should assess what has been dealt with in class and what is described in the syllabus, and nothing else. These concepts, first used by Messick (1989), have great relevance in a CLIL context. In particular, if the subject has been taught in a language other than the students’ L1, in what language should the tests be given? If they are given in the language of instruction, the level of construct-irrelevant variance may be high, as students may have difficulty understanding the wording of test items. In a similar vein, if the language used in the test is the students’ L1, students may not be able to correctly convey the content knowledge taught in an L2/FL as they may lack the specific terminology required. In this case, then, lack of productive language proficiency may be viewed as construct-irrelevant variance. 2.3 Reliability

The reliability of assessment is defi ned as the extent to which tests are consistent. It may be seriously questioned when there is uncertainty

62 Part 2: Assessment and Motivation

regarding an individual’s possibility of showing his or her knowledge of a topic in a certain language, at a certain time, regardless of whether it is the L1 or an L2/FL. Reliability refers to stability within a test as well as between tests, whether the same test on different occasions or different tests on the same subject content. Alternate item types, as well as forms of assessment, should measure the same thing. This means that, for test results to be reliable, they also need to be valid. Students taking the same course, regardless of school context, should be measured along the same scale and be given a proper chance to show the same knowledge. Furthermore, when test scores are used for grading, it is crucial that students have been given a fair opportunity to learn what the test is intended to cover. Thus, no matter how a CLIL test is administered, it is difficult to ensure that it actually measures what it aims to measure, and that students are provided with the opportunity to convey their full knowledge potential on any given topic at any given time. In view of the aspects brought up above about the significant role of assessment and the importance it should be accorded, it is somewhat surprising that assessment in CLIL is a strikingly underexplored area. Massler et al. (2014) refer to assessment in CLIL as a blind spot. Only limited empirical studies have been conducted in the field. However, as will become evident in the following section, there have indeed been a few investigations reported recently. 3 Studies on Assessment in CLIL

Issues in CLIL and assessment concern the relationship between language and content. Kiely (2012: 4) discusses the purpose of assessment in CLIL as well as the issues of language versus content: ‘[H]ow do we use assessment to manage an appropriate balance in CLIL practice between content and language, such that there is no fear that children [students] achieve less where the learning is in L2/FL?’ Students’ possible barriers to understanding as well as presenting their knowledge, due to insufficient language ability, are sometimes accommodated for by giving them a choice of language in assessment situations. This, as touched on above, raises the question as to whether students should be assessed in the language of instruction, most often an L2/FL, in their L1, or using a mixture of languages (Gablasova, 2014). In an attempt to consider the specific language skills required for CLIL students, Morgan (2007) interviewed and observed teachers and students in Grades 3–10 in two CLIL schools, one in Austria and one in the UK. While there were differences between the two schools on a number of levels, many similarities were also found. One of them was the apparent lack of adequate assessment tools for CLIL. Morgan (2007) suggests new benchmarks for CLIL, where descriptors not found in non-CLIL curricula are included, seen as pivotal in the CLIL classroom. In particular,

Assessment in CLIL 63

descriptors for spoken language were found not to correspond to the needs in CLIL, where involvement in serious discussion and debate is crucial, as is the understanding of subject-specific information and concepts. Using a qualitative approach, Hönig (2009) investigated assessment practices at a senior high CLIL school in Austria. Four history teachers were observed and interviewed. Interestingly, even though the teachers claimed not to pay attention to language in their assessment, it turned out that they did so, to a very high degree. In oral examinations, less proficient students, seen from a language perspective, were graded lower than those exhibiting better language skills, even though their content proficiency was equal. Hönig argues that CLIL students are disadvantaged on these grounds, concluding that specific assessment criteria for CLIL need to be worked out, taking both language and content, as well as the combination of the two, into account. Tsagari and Michaeloudes (accepted) focus on CLIL in primary schools, looking into the ways content and language are assessed, and whether the focus is on language and/or content. Through questionnaires to teachers and classroom observations, it was found that a variety of formative assessment methods were used, and also that questioning was among the most common of these methods. Further, the InitiationResponse-Feedback (IRF) pattern was very frequent. In fact, the IRF pattern was often noted to be expanded into an IRFRF pattern, where a response from another student was given in direct connection to the IRF sequence, the teacher giving feedback to that response before moving on to the next sequence. Finally, Tsagari and Michaeloudes conclude that when content was in focus the target language was used, and when language issues were delved into they were closely connected to content. Wewer (2014) also studied CLIL students in primary schools. Using both qualitative and quantitative methods, involving parents and students at several schools in Finland, the fi ndings reveal a challenging situation. While it was clear that language was rarely assessed in these CLIL schools, it was also shown that both parents and students wished to learn more about the progress being made in the target language. Wewer (2014: 233) argues that ‘the assessment of primary CLIL learners’ English proficiency is often infrequent, incidental, implicit and based on impressions rather than evidence or the curriculum’. This, in turn, can be explained by the lack of a specific CLIL curriculum as well as any language objectives and common principles agreed on regarding assessment. High school students were in focus in Gablasova’s (2014) study into the assessment of CLIL and non-CLIL students in Slovakia. It was found that CLIL students were at a disadvantage when using the L2/FL in assessment situations. Furthermore, when using their L1, their speech was faster; consequently, more content could be conveyed in time-limited situations such as, for instance, during a test. This in turn means that when the L2/FL is expected to be used in such situations, students are not

64 Part 2: Assessment and Motivation

allowed to communicate their content knowledge to the same extent as would have been the case if the L1 had been used. In addition, the academic language expected was less prevalent in the L2/FL than in the L1. Finally, some disfluency and use of inaccurate terms were observed among the CLIL students when conveying content knowledge in their L1. Gablasova (2014) concludes that the assessment of content knowledge is complex, especially so when an L2/FL is used as the medium of content instruction, as is the case in CLIL. Two case studies were reported by O’Dwyer and de Boer (2015), looking into learner involvement and collaboration in assessment in CLIL contexts. The theoretical frameworks used were dynamic assessment and learning-oriented assessment, both of which, albeit in different forms, focus on formative assessment. In dynamic assessment, learner involvement is at the core (Lantolf & Poehner, 2008). Learners themselves mediate content, and the role of the teacher is to move around among learner groups in the classroom to facilitate learning. In the case of dynamic assessment in a CLIL context, the target language is used to discuss content, and the learning of the language takes place through social interaction. Learning-oriented assessment aims to develop learners’ ability to self-assess. O’Dwyer and de Boer (2015) argue that both approaches are relevant for CLIL contexts, and that they ‘facilitate learner-learner reflective gap-closing and informative feedback in various ways’ (O’Dwyer & de Boer, 2015: 418, italics in original). The studies referred to above have been carried out in different countries – the UK, Austria, Cyprus, Finland, Slovakia and Japan, respectively. Yet they all seem to share the view that assessment is particularly demanding in a CLIL context; further, they all address the need for guidelines and criteria specifically adjusted to this popular, and seemingly growing, teaching approach. In the following, we will present the fi rst study thus far into CLIL and assessment in the Swedish context. 4 Assessment in the CLISS Project

In view of the insufficient amount of research so far on assessment in CLIL, one of the areas focused on in the CLISS project is precisely that. In Reierstam (2015), a full account of the particular study on assessment in the schools participating in the CLISS project is found. Below, the main results from Reierstam (2015) are presented, while at the same time adding some new perspectives. To date, no comparative analysis has been made between CLIL and non-CLIL assessment practices. Therefore, some conceptual and terminological decisions had to be made to fit the purposes of the present study. Given that different disciplines and subjects were involved and compared, the terms interdisciplinary and cross-disciplinary are used. Interdisciplinary here refers to integrative aspects, such as collaboration

Assessment in CLIL 65

between school subjects; cross-disciplinary is used about features shared across disciplines in relation to, for example, linguistic registers spanning several disciplines. The term academic function word is used for the descriptors marking academic language functions1 in the test items. As regards the various test and item types, standardized concepts are used, except for the distinction here made between question tests and production tests. The fi rst category represents standardized test formats involving multiple test items, whereas the second involves various kinds of written assignments such as laboratory reports or essays. Many overlapping concepts are to be found in the literature to describe linguistic registers and to distinguish between everyday language, subject-specific language and general academic language. One such distinction is that between content-obligatory (CO) language and content-compatible (CC) language, where CO represents subject-specific terminology and CC the ‘fi ller’, or more general, language (CoBaLTT Project, http://www.carla. umn.edu/cobaltt/). In the present study the terms CO and CC are used, and the notion of content-compatible language was adapted to include general academic language. Since no formal policy decisions exist for CLIL in a Swedish context, the application of CLIL varies according to individual teachers’ beliefs and their aim to align instruction with national course goals (see Chapter 1). As accounted for above, other studies report the same lack of guidelines and policy decisions for CLIL and assessment (Hönig, 2009; Massler et al., 2014; Wewer, 2014). In what follows, an overview is first given of the material and the participants in the investigation into assessment carried out at the schools involved in the CLISS project. Some further terminological considerations are also presented, followed by a description of the data analysis procedures. Finally, the fi ndings of the study are discussed from a validity perspective, taking possible pedagogical implications into account. 4.1 Method and material

This study compares the views of 12 senior high school teachers. The participants in the study all came from the three schools involved in the CLISS project (see Chapter 1). The variation in CLIL teaching experience between the subject content teachers ranged from four to seven years. Out of the 12 participating teachers, six were subject content teachers (three in history and three in biology), and six were EFL teachers. Two of the subject content teachers were non-CLIL teachers, one each in history and biology. All of the subject content teachers were native Swedish speakers, whereas the EFL teachers at the international school were native English speakers. As stated in the introduction to this chapter, the aim was to compare the assessment practices of CLIL versus non-CLIL teachers, as well as those of content versus language teachers.

66

Part 2: Assessment and Motivation

A qualitative perspective was used for this study, consisting of descriptive data. The data analysis involved three data collection methods. The data collection started in February 2013 and ended one year later. The material included paired semi-structured interviews, one interviewee at a time with the interviewer. The interviews, which were recorded, lasted 16–58 minutes, depending on how much time the teachers were able to spend on them. Since the aim was to identify what influences teachers’ assessment practices in a CLIL context, the central topics covered in the interviews were teachers’ views on CLIL, assessment, course content, and disciplinary and/or interdisciplinary collaboration. The interviews were supplemented by a questionnaire, consisting of 26 questions, relating to the same themes. Contrasting the views and stated assessment practices of CLIL teachers with those of their non-CLIL colleagues, in both the interviews and the questionnaire the goal was to fi nd out if different or specific theories had an effect on the assessment procedures. Differing practices due to the use of English as a medium of instruction were of particular interest. Here, the EFL teachers’ beliefs were considered important, especially in relation to the possible integration of content themes in the EFL courses or interdisciplinary teacher collaboration, since none of the content teachers was a language teacher. The participating teachers were asked during the interviews to present written assessment samples from one of their courses. Thus the focus was on written assessment. The subject content teachers submitted some material, whereas only one of the participating EFL teachers presented any assessment samples. For the document analysis, a combination of methods was used to describe the features of the tests and the interplay between subject content and language. The aim, again, was to compare assessment practices between CLIL, non-CLIL, content and EFL courses, to see if – and how – teachers’ awareness of students’ CLIL context affected their choice of format and content. Table 4.1 provides a summary of the different layers and features used for the document analysis of the assessment samples, starting with a broad description of all assessment tasks in a course, systematically looking at more detailed features of the individual tests. The features in question were selected after consulting relevant literature on assessment design (Brown & Hudson, 2002; Levin & Marton, 1973; Wedman, 1988; Wikström, 2014) as well as previous research within the CLIL framework (e.g. Hönig, 2009; Wewer, 2014). Assessment in the respective subject content disciplines was also taken into account (Lindmark, 2013; Odenstad, 2010; Rosenlund, 2011). The analysis was performed on written assessment samples. National course goals, the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001) and Bloom’s revised taxonomy (Anderson & Krathwool, 2001) were used as points of reference when comparing the features in the assessment items, as listed in Table 4.1.

Assessment in CLIL 67

Table 4.1 Areas and features considered in relation to the assessment samples Areas

Features

Course layout

Number of tests Assessment types

Assessment design/layout ‘How’

Number of questions Test/question type: (a) Question test – Selected response – Constructed response (b) Production test Modalities Scoring/grading Time on task

Assessment content ‘What’

Subject theme: In relation to course goals Language in the test items: Academic function words Question words Context embeddedness Required linguistic skills, target language: Discipline specific/content obligatory language Communicative function/content compatible language Required cognitive skills: Higher versus lower order thinking skills

The tests were categorized according to type, i.e. question tests versus production tests, and a fi rst general description focused on layout features regarding test design. Next, the samples were analyzed in more detail in relation to what the test items contained, as well as what they could be expected to demand from the test takers. The language was described in terms of academic function words, adapted from a list made by DaltonPuffer (2007) in combination with Bloom’s taxonomy. An adapted version of Cummins’ matrix (Coyle, 1999) served as a tool to combine comparison of the cognitive difficulty of content, lower order versus higher order thinking skills, and the demands on language proficiency made in the tasks. Academic language functions can be hierarchically organized as in Blooms’ revised taxonomy (Anderson & Krathwool, 2001) or Cummins’ language proficiency levels (Cummins, 1984). Consequently, a frequency count was made of function and question words used in the test items to fi nd a measure to compare the level of difficulty in the test items. The question words and the phrasing of the test items were used to determine whether they involved a low or high cognitive demand. In Bloom’s revised taxonomy, the basic levels represent lower order thinking skills (LOTS), as exemplified by the question words What, When or Which. Higher order thinking skills (HOTS) include How and Why questions. Similarly, lower cognitive as well as lower linguistic demands are involved when using academic function words to identify, name or describe, for example. Higher cognitive and linguistic demands are made when verbs such as

68 Part 2: Assessment and Motivation

compare, analyze or interpret are used. The assessment samples in the study were analyzed and CLIL test items were compared with non-CLIL items according to the demands made by the questions on students’ linguistic and cognitive skills. 4.2 Results

Analysis of the interview and questionnaire data, as well as the assessment samples submitted, suggested that no apparent differences in assessment practices could be identified between CLIL and non-CLIL teachers due to the different languages of instruction. The biology teachers used almost identical written tests and assignments, due to the fact, according to the teachers themselves, that they depend on the same national course goals, regardless of the language of instruction. The non-CLIL biology teacher used a slightly higher number of longer essay questions, demanding longer answers. However, both the CLIL and the non-CLIL teacher used similar writing assignments. By contrast, the history teachers showed greater variation in their written assessment designs, due to differences in teacher cognition concerning their views on the subject discipline and personal preferences. However, the history CLIL teachers favored essays, whereas the non-CLIL teacher preferred question tests. In Sweden, teachers are free to choose their own teaching and assessment methods, and so teachers’ practices vary. In the EFL courses, where national, standardized high-stakes tests play an important role, teachers seem to rely on these tests to a large extent, whereas the rest of the assessment material varies a great deal. In the question tests, a frequency count was made of the most commonly used function and question words in the test items. The limited corpus of tests does not allow for a quantitative analysis, yet some findings are interesting to note. The instances counted represent six biology tests (three CLIL and three non-CLIL) and four history tests (two CLIL and two non-CLIL). The distribution of the words in the frequency count is presented in Tables 4.2 and 4.3. Words requiring lower cognitive and linguistic skills are to be found at the top, and those involving higher cognitive and linguistic skills at the bottom of the tables. The difference between more or less cognitively demanding words is not always self-evident, since it also relies on the wording of the test items, as seen below. The fi ndings as regards the distribution of various function words (Table 4.2) reveal that explain is the most frequently used word in biology, for both CLIL and non-CLIL. Concerning the total number of function words used, it is worth noting that in several of the test items in the biology tests, multiple function words are used in the same item, as in: ‘Describe and explain how nitrogen can transform into forms for living organisms to use.’ For history, the high frequency of the question word What (Table 4.3: 43 instances) is offset by a lower frequency in the use of

Assessment in CLIL 69

Table 4.2 Function words used in the test items Function words

Encircle

CLIL biology

Non-CLIL biology

CLIL history

Non-CLIL history

1

Match

1

Put in order

1

Write in the right place

1

Name

1

2

State

6

6

Mark

1

1

Give an example

1

1

Mention

4

5

Define

1

2

Describe

2

1

Discuss

1

1

Explain

12

14

Draw

1 1 4

1

2

4

2

2

Show

4

2

1

Compare

2

1

1

Motivate

1

2

Analyze

1

Give arguments

1 1

other question words. The cognitive and linguistic demands of the test items containing What varies a great deal, as demonstrated in the two extracts below, found in two different history CLIL tests by two different teachers: (1) What was the popular name for the young Macedonian king who was in charge of the Macedonian troops and personalized this development? He also had a very famous teacher, almost as famous as himself. What was his name? (4) Answer: The king: His famous teacher:

A Ar

(2) More explanatory answers needed. Points in brackets. What does the concept appeasement mean? Also give an example of  when one can say that the UK and France used that way of acting. (4)

Whereas the fi rst item implies factual recall, the second apparently requires a lengthier response and thus a cognitively and linguistically more demanding answer, while retaining the same question word. It is

70 Part 2: Assessment and Motivation

Table 4.3 Question words used in the test items Question words

CLIL biology

Non-CLIL biology

CLIL history

Non-CLIL history

43

2

What

7

3

When

2

1

Where

1

Who

2

Which

3

5

1

2

How

6

3

2

1

Why

2

Yes/no question

1

1 2

worth noting that the maximum score possible in both these items is 4, as specified in brackets, giving an indication of teachers’ varying use of scores in relation to items’ demands on test takers. The total scores of the two tests are similar, as well as the internal consistency of the tests by means of similar item types. Reliability and validity in the outcome of the second item may be at risk due to the importance of sufficient language skills and diffi culty in scoring the constructed response format, whereas a potential threat to the validity of the fi rst item depends on whether it actually measures the stated learning objectives of the course. The test items in the non-CLIL question tests typically contained one entry with many sub-questions, requiring the explanation of different concepts or events, which explains the low number of function and question words. The non-CLIL question tests also contained test items requiring lengthier answers, similar to writing prompts used in the production tests in the CLIL courses. Table 4.3 shows a higher frequency in the use of more cognitively demanding question words, such as How and Why in the biology CLIL tests. However, the analysis of the test items revealed that the non-CLIL tests contained a slightly higher number of longer constructed response items. The same content was covered in the CLIL and non-CLIL tests. The fi ndings indicate that where the extent of collaboration between teachers within the same subject discipline is higher, teachers seem to use the same tests, translating the items from Swedish to English. Subjectspecific features may also have an effect, where history has been defined as a ‘fuzzy discipline’ (Pace, 2011) regarding what and how to teach and assess, possibly resulting in less standardized assessment procedures. All of the participating teachers preferred written assessment over oral, claiming that validity and reliability are higher for written assessment tools. One of the CLIL teachers also mentioned that it is hard for students to achieve a certain level of complexity in oral discussion in a

Assessment in CLIL 71

class of 30 students. Written assessments, however, enable students to use more of their language proficiency and so present more knowledge, thus underlining the importance of language skills. The CLIL teachers noted that question tests are easier to grade, but do not leave enough time or room for students to show their analytical skills, as writing assignments and production tests do. The CLIL content teachers acknowledged a need to get more training in how to assess written skills. In the questionnaire, all content teachers considered grading to be difficult. The CLIL teachers expressed the opinion that language proficiency should not be part of the assessment: ‘That is up to the English language teachers.’ However, the biology CLIL teacher noted that subject-specific concepts have to be explicitly taught. Similarly, the history CLIL teachers reported that instruction should include discussion about the meaning of historical concepts and related vocabulary, both in English and in Swedish. Students were informed by one of the CLIL teachers about the possible risk of attaining lower grades in content courses due to the use of English as a medium of instruction. In the same vein, the non-CLIL history teacher claimed that the grade is based on how articulate students are. The teachers showed different strategies regarding how to deal with language and assessment in their courses. English was claimed to be used for the most part in classroom instruction, but in some cases the students could choose which language to use in assessment tasks. The EFL teachers generally seemed reluctant to include any subject content in the EFL courses. Lack of time, both for interdisciplinary collaboration and within the course frame, was one stated reason in all of the participating schools. Another was lack of competence in assessing the appropriate use of subject-specific terminology. Three of the EFL teachers claimed to include both language and content in their assessment, which raises a question about what is meant by content. Two of the EFL teachers were the only ones among all the participating teachers to state that the use of general, CC language was among the most important things when assessing students’ skills; however, none of the subject content teachers did. 5 Discussion and Pedagogical Implications

Traditional defi nitions of validity refer to whether a test measures what it was designed to measure. Consequential validity, according to Messick (1989), is not a property of the test itself; rather, it concerns the general question of whether test use and the interpretation of test scores can be justified. Thus, construct validity entails questions which have to be addressed to justify test use. Among such questions are, for example (Gipps, 1994: 60): Are relevant matters evaluated in an appropriate balance? Are there any side-effects of scoring? In CLIL contexts, construct validity becomes problematic since there are no stated benchmarks for the

72

Part 2: Assessment and Motivation

assessment of language. Enhanced language skills are described as a desired learning goal in CLIL (Dalton-Puffer, 2007), yet language is not perceived as part of the assessment among most teachers in the study. However, in subject content courses, teachers regard subject-specific terminology as an integral part of instruction, providing students with disciplinary concepts. Construct validation requires an explicit conceptual framework, including studies of performance differences over time, across groups and settings (Gipps, 1994: 61). In a CLIL context, construct validation becomes problematic since the question of what should be included in the construct has not yet been addressed, i.e. whether language is part of the construct and, if so, how it should be assessed, or at least considered. In this study, the data have shown that, when developing their assessment practices, CLIL teachers tend to ignore the fact that content is taught through a language other than the students’ L1. Some content teachers argued that language proficiency should not be part of the assessment, arguing that language is a matter for language teachers to deal with. Views like these are in line with Hönig (2009), who found that teachers claimed not to pay any attention at all to language in their assessment practices. However, in Hönig’s study, the opposite turned out to be true when explored in the reality of classroom activities. Whether this also applies to the Swedish context remains to be investigated. Based on the fi ndings of the present study, though, content teachers, regardless of subject, clearly seem uncertain or confused about how to deal with language in assessment. True, some teachers report paying special attention to subject-specific terminology but, apparently, they do not seem to regard such vocabulary as ‘language’, but rather as ‘subject content’. Yet, while not paying attention to language, the teachers prefer written assessment over oral, since writing provides an opportunity for students to use more language, enabling them to present more of their content knowledge. This somewhat paradoxical state of affairs needs to be addressed. Unless students are given appropriate exposure to and information about subject-specific language and genre, they are seriously hampered in their potential to express themselves in the relevant subject domains, not least in writing. The inclusion of EFL teachers in the present study adds an interesting dimension. Just as the content teachers seem to focus only on content, not bothering about language issues, the EFL teachers seem to stick wholeheartedly to language, not bringing in any of the content areas where English is used as the medium of instruction. The reason for this, they argue, is that there is a general lack of time to include anything more than what is stated in the curriculum. Thus, the subject of EFL also seems to be in need of adjustment in a CLIL context. Possibly, the basic purpose of CLIL – the integration of content and language – cannot be met unless a specific CLIL curriculum with defi ned target language learning outcomes

Assessment in CLIL 73

is in place, including both CLIL subjects and the L2/FL through which content is communicated (cf., for example, Sylvén, 2013). The pedagogical implications of these results are many. First of all, this study adds further fuel to the fi ndings of other studies (cf. above) as regards the need for a specifi c CLIL agenda and curriculum; this would also facilitate assessment procedures. Without such a frame, relevant differences between CLIL and non-CLIL outcomes are difficult to establish. Hence, the results of the present study are seen, where the variation in assessment between CLIL and non-CLIL is due to teachers’ personal preferences rather than anything else. Secondly, CLIL teachers should be off ered courses in CLIL methodology where, among other things, the relationship between language and content is highlighted. Needless to say, it is a tall order for teachers to be expected to adapt their pedagogy to a cross-disciplinary approach without having been given any kind of relevant training or preparation for such a challenging task. 6 Concluding Remarks

In sum, CLIL teachers seem to view themselves as content teachers rather than content and language teachers. What is especially important, then, is to prepare CLIL teachers so that they become aware of the important role language plays in their respective subjects and, further, that they, rather than the EFL teachers, are the experts in the subject-specific language area. In CLIL contexts there is a need to move beyond viewing language as a system only, towards acknowledging the crucial importance of language as a carrier of subject-specific content. Also, this is the primary area where steps need to be taken to create benchmarks for assessment in CLIL (cf. Morgan, 2007). In this way, language – content-compatible as well as content-obligatory – and content would be seen as equally important; this, after all, is what CLIL is all about. Note (1) The concepts academic language functions and cognitive discourse functions (CDF) are both used in the literature to denote the communicative acts involved in externalizing internal cognitive processes (e.g. Dalton-Puffer, 2007; Nikula et  al., 2016). Dalton-Puffer (2016)uses the terms function type and CDF verbs where we talk about function word(s).

References Anderson, L. and Krathwool, D. (2001) A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman. Bachman, L.F. and Palmer, A.S. (1996) Language Testing in Practice: Designing and Developing Useful Language Tests. Oxford: Oxford University Press.

74 Part 2: Assessment and Motivation

Black, P., Harrison, C., Lee, C., Marshall, B. and Wiliam, D. (2002) Working inside the Black Box: Assessment for Learning in the Classroom. London: GL Assessment. Brown, J.D. and Hudson, T. (2002) Criterion-referenced Language Testing. Cambridge: Cambridge University Press. Council of Europe (2001) Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Coyle, D. (1999) Theory and planning for effective classrooms: Supporting students in content and language integrated learning contexts. In J. Masih (ed.) Learning through a Foreign Language (pp. 46–62). London: CILT. Cummins, J. (1984) Bilingualism and Special Education: Issues in Assessment and Pedagogy. Clevedon: Multilingual Matters. Dalton-Puffer, C. (2007) Discourse in Content and Language Integrated Learning (CLIL) Classrooms. Amsterdam: John Benjamins. Dalton-Puffer, C. (2016) Cognitive discourse functions: Specifying an integrative interdisciplinary construct. In T. Nikula, E. Dafouz, P. Moore and U. Smit (eds) Conceptualising Integration in CLIL and Multilingual Education. Bristol: Multilingual Matters. Gablasova, D. (2014) Issues in the assessment of bilingually educated students: Expressing subject knowledge through L1 and L2. The Language Learning Journal 42 (2), 151–164. Gipps, C. (1994) Beyond Testing: Towards a Theory of Educational Assessment. London and Washington, DC: Falmer Press. Hönig, I. (2009) Assessment in CLIL. A case study. MPhil thesis, University of Vienna. Kiely, R. (2012) CLIL – the question of assessment. DevelopingTeachers.com. See http:// www.developingteachers.com/articles_tchtraining/clilpf_richard.htm. Lantolf, J.P. and Poehner, M.E. (eds) (2008) Sociocultural Theory and the Teaching of Second Languages London: Equinox. Levin, L. and Marton, F. (1973) Provteori och provkonstruktion. Stockholm: Almqvist & Wiksell. Lindmark, T. (2013) Samhällskunskapslärares ämneskonceptioner. PhD thesis, Umeå University. Massler, U., Stotz, D. and Queisser, C. (2014) Assessment instruments for primary CLIL: The conceptualisation and evaluation of test tasks. The Language Learning Journal 42 (2), 137–150. Messick, S. (1989) Validity. In R.L. Linn (ed.) Educational Measurement (7th edn). New York: Macmillan. Morgan, C. (2007) Appropriate language assessment in content and language integrated learning. The Language Learning Journal 33, 59–67. Nikula, T., Dafouz, E., Moore, P. and Smit, U. (2016) Conceptualising Integration in CLIL and Multilingual Education. Bristol: Multilingual Matters. Odenstad, C. (2010) Prov och bedömning i samhällskunskap. En analys av gymnasielärares skriftliga prov. Licentiate thesis, Karlstad University. O’Dwyer, F. and de Boer, M. (2015) Approaches to assessment in CLIL classrooms: Two case studies. Language Learning in Higher Education (CercleS) 5 (2), 397–421; doi:10.1515/cercles-2015-0019. Pace, D. (2011) Assessment in history: The case for decoding the discipline. Journal of the Scholarship of Teaching and Learning 11 (3), 107–119. Reierstam, H. (2015) Assessing content or language? A comparative study of the assessment practices in three Swedish upper secondary schools. Licentiate thesis, University of Gothenburg. Rosenlund, D. (2011) Att hantera historia med ett öga stängt: Samstämmighet mellan historia A och lärares prov och uppgifter. Licentiate thesis, Lund University. Scriven, M. (1996) Types of evaluation and types of evaluator. American Journal of Evaluation 17 (2), 151–161.

Assessment in CLIL 75

Sylvén, L.K. (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320. Tsagari, D. and Michaeloudes, G. (accepted) Provision of feedback in L2 exam classes in Cyprus. In Y. Bayyurt and N. Sifakis (eds) English Language Education Policies and Practices: A Mediterranean Perspective. London: Pearson. Wedman, I. (1988) Prov och provkonstruktion. Stockholm: Utbildningsförlaget. Wewer, T. (2014) Assessment of young learners’ English proficiency in bilingual content instruction CLIL. PhD thesis, University of Turku. Wikström, C. (2014) Konsten att göra bra prov. Stockholm: Natur och Kultur. Wiliam, D. and Thompson, M. (2008) Integrating assessment with learning: What will it take to make it work? In C.A. Dwyer (ed.) The Future of Assessment: Shaping Teaching and Learning (pp. 53–82). New York: Lawrence Erlbaum.

5 CLIL and Motivation Revisited: A Longitudinal Perspective Amy S. Thompson and Liss Kerstin Sylvén

1 Introduction

One of the benefits commonly attributed to CLIL is that it increases learner motivation. However, the majority of studies making such claims do not provide a baseline motivational level prior to the CLIL experience; thus, it is difficult to ascertain whether the high levels of motivation are pre-existing among CLIL students or are, in fact, a result of CLIL. As part of the CLISS project, this chapter presents the results of a longitudinal study investigating the change in motivation of CLIL (N = 109) and non-CLIL (N = 68) students during their three years in Swedish senior high school. The Motivational Factors Questionnaire (MFQ) (Ryan, 2009) was administered twice during the three years of the project – the first time at the beginning of senior high school, coinciding with the start of CLIL for the CLIL students, and the second, three school years later, at the very end of high school. Using one-way ANOVAs with CLIL or non-CLIL status as the independent variable and the factor scores of the MFQ as the dependent variables, the baseline data indicate that CLIL students are significantly more motivated on a number of factors compared to non-CLIL students at the onset of CLIL instruction (Sylvén & Thompson, 2015). The results presented in this chapter compare the baseline data collected in 2011, and reported in Sylvén and Thompson (2015), and the data collected at the end of high school (2014). Other than the CLIL status comparison, the baseline data also indicate a number of statistically significant differences between males and females, making gender an additional point of longitudinal inquiry. Filling the gap in the literature about the effect of CLIL on motivation, these longitudinal results reveal whether the relationship between motivation and CLIL versus non-CLIL students remains the same or changes over the course of the three-year senior high school period. Furthermore, the inclusion of the gender aspect makes it a three-dimensional 76

CLIL and Motivation Revisited: A Longitudinal Perspective

77

investigation of motivation, CLIL and gender. As this is one of very few studies of its kind, providing baseline data and taking a longitudinal perspective, the results will widen the understanding of any potential motivational effects of CLIL. 2 CLIL: Language Learning Motivation and Gender 2.1 CLIL and motivation

One of the underlying assumptions of CLIL is that, by using a second or foreign language (L2/FL) in non-language classrooms, students who are eager to learn the subject content are highly motivated to acquire the L2/FL through which such content is being communicated (e.g. Eurydice Network, 2006). Gardner (2010: 199) suggests that ‘the foreignness of the other language may well be less formidable’ when using it as a communicative tool in the classroom. CLIL classrooms are ‘widely considered as motivating’ (Dalton-Puffer, 2011: 191), and Marsh (2000: 2) argues that CLIL should ‘nurture a “can-do” attitude towards L2/FL learning’. While statements such as these may have some face validity (but see Bruton, 2013, for a strong counter-argument), the fact is that research into the motivational effects of CLIL is still scarce. In the UK context, with the CLIL classes in languages other than English, Coyle (2011) found CLIL students to exhibit higher levels of motivation regarding the learning environment, learner engagement and learner identity. In addition, the CLIL students indicated that learning through CLIL is ‘more fun’ (Coyle, 2011: 3). In a similar context (i.e. with the language of focus being a language other than English), Abendroth-Timmer (2007) found that German CLIL students with Spanish as the language of instruction increased their levels of motivation during the period of the study. Unfortunately, however, without pre-CLIL data it is difficult to know if the obtained results indeed can be attributed to CLIL, or if these CLIL students were already highly motivated at the onset of CLIL. In a Spanish context where English was the medium of CLIL instruction, using a cross-sectional design with no pre-CLIL data reported, Lasagabaster and Sierra (2009) studied language attitudes among CLIL versus non-CLIL students (i.e. those who studied the L2/FL only as a separate subject). The tool used was a semantic differential questionnaire, which consisted of a number of antonyms (e.g. necessary-unnecessary, appealing-unappealing). The participants were students in their third (age 14–15) and fourth (age 15–16) years of secondary level education. In the third-year group, positive adjectives were chosen more frequently among the CLIL students as compared to the non-CLIL ones. In the fourth-year group, the CLIL students still chose the positive adjectives to a larger extent than the non-CLIL students, but the difference between the two groups was smaller than among the third-year students. The results for

78 Part 2: Assessment and Motivation

the CLIL group in Year 3 indicate that the level of motivation of the CLIL students, in comparison with non-CLIL, slightly decreased with age in this context. Lasagabaster (2011) investigated motivation among students (age 15) in English as a foreign language (EFL) and CLIL settings, respectively, and found that CLIL students were significantly more enthusiastic about L2/FL English learning than were EFL students. The conclusion drawn was that not only is there a strong correlation between CLIL and motivation, but that CLIL also seems to sustain, and even improve, students’ level of motivation over time. These fi ndings are in line with Lorenzo et al. (2010), who similarly found CLIL students’ levels of motivation and self-concept to be higher than those of non-CLIL students. However, baseline data were not included in either of these studies. An additional study in the Spanish context was that of Doiz et al. (2014), who investigated CLIL and non-CLIL students (N = 393) in their fi rst and third years, respectively, of secondary school in a cross-sectional analysis. Their fi ndings showed that the CLIL students’ levels of motivation were higher and levels of anxiety were lower than those of the nonCLIL students in Year 1, and that these differences were maintained in Year 3. The authors’ conclusion is that CLIL ‘appears to have an effect on motivation’ (Doiz et al., 2014: 222) when comparing this cross-sectional data. As this is the first publication as part of a longitudinal study, subsequent publications from this project will provide more information about the change in motivation over time for CLIL versus non-CLIL students. More uncommon research methods and contexts, such as action research with CLIL classes in Argentina (Banegas, 2013) and CLIL in vocational education in the Netherlands (Denman et al., 2013), also led to insights into the relationship of CLIL and motivation. Banegas (2013) implemented action research in an Argentinian CLIL context to examine how co-developed and negotiated course content with colleagues and students would affect motivation. Indeed, these activities led to higher levels of motivation among students, as well as teachers. Banegas (2013: 94) concludes that in CLIL ‘it is vital that content, sources and activities emerge from the intersection between the school curriculum and students’ interests and cognitive development’, emphasizing the potential of a bottom-up process in the creation of CLIL programs. In the context of CLIL in vocational education, Denman et al. (2013: 298) found that motivation appears to increase among vocational students ‘who enjoy a challenge’. Additionally, the results indicate that the vast majority of students would recommend the CLIL option to friends and family ‘because they feel it is fun and motivating and it helps them develop their skills in English’. Declines in levels of motivation have been described in other studies involving immersion students (e.g. MacIntyre et  al., 2002), but several studies indicate that motivation is regained in the later years of the language learning process (Genesee, 1978; Lambert & Tucker, 1972; Swain &

CLIL and Motivation Revisited: A Longitudinal Perspective

79

Lapkin, 1982). Hood (2006) suggests that ‘enjoyment, motivation and selfesteem’ are factors at risk for students embarking on CLIL, as initial challenges are involved before becoming adapted to the use of a language other than the learner’s first language (L1) as the medium of instruction. Sallili and Tsui (2005) found similar results in their study set in Hong Kong, where a decline in self-efficacy scores was found among lower ability EMI students. Similarly, Seikkula-Leino (2007) found lower levels of selfconcept regarding L2/FL learning among early adolescent CLIL students in Finland. Likewise, with immersion students, Wesely (2009) found that his participants were very motivated for L2/FL learning, but that factors such as the relationship between peers and teachers were important for their motivation to grow. Thus, external factors, and not only the immersion experience itself, are involved in learners’ levels of motivation. Other affective constructs potentially related to motivation are also important to the study of CLIL and motivation. For example, Rumlich (2014), in a comprehensive study (N = 1152), found that CLIL students were significantly more interested in English than their non-CLIL peers before the start of the CLIL program. Anxiety is another affective construct that could affect language learning motivation. For example, several studies link anxiety and L2 achievement (Abu-Rabia, 2004; Horwitz et al., 1986; Kim, 2009), which is undoubtedly related in some way to language learning motivation. Furthermore, while anxiety per se is not the focus of Maillat (2010) and Nikula (2007), these studies indicate that CLIL students seem to have less speaking anxiety than other students. Although studies with a specific focus on language learning anxiety in CLIL contexts are less common than those with a language learning motivation focus, our own study shows that CLIL students were significantly less anxious than their non-CLIL peers on a number of anxiety-related factors before CLIL started (Thompson & Sylvén, 2015). 2.2 CLIL and gender

The present chapter investigates motivation, CLIL, and gender, and even though there are, to date, no known studies with this threedimensional focus, there are indeed a number of relevant studies investigating motivation and gender. In a meta-analysis of studies into motivation and gender in general, Meece et al. (2006: 367) found that, while boys’ achievement-related beliefs regarding math, science and, sports were positive, girls ‘show more favorable motivation patterns in language arts and reading’. Thus, the results of this study indicate that girls are more inclined to study languages from the start. These results are corroborated by Mori and Gobel (2006); Japanese EFL students were investigated, and the results indicate that the female students scored significantly higher on the items loading onto the integrativeness factor. In other words, the female students were more interested in travelling to, studying in, and getting to know the

80 Part 2: Assessment and Motivation

target culture than were the males. Likewise, Rumlich (2014) found that female students showed a greater interest in English than did the males. When Kissau (2006) explored gender differences in L2 French learning, he saw the same trend, namely that girls were highly motivated to study language, whereas boys rapidly lost interest as they grew older. However, when he tried to look beyond the mere results of the questionnaire by interviewing students, other important factors to explain this state of affairs emerged. In fact, what deterred young men from pursuing their L2 French studies were societal norms about what ‘real’ men should be like, and a fear of not fitting into this picture. In addition, Kissau et al. (2010) highlights the negative attitude among boys towards teacher-centered education and repetitive classroom tasks. Pavy (2006) and Carr (2002) also suggest learner-centered and task-based teaching and learning in order to heighten boys’ L2 motivation. Based on the literature, it is clear that longitudinal studies looking into the relationship between motivation and CLIL, as well as motivation and gender, are scarce, and that a focus on all three simultaneously are non-existent. Therefore, the purpose of this study is to help fill this void in the literature. 3 The Study

The impetus of the current study is a longitudinal examination of the effect of CLIL status on language learning motivation. This study is unique because of the two data points used to examine CLIL and motivation; most of the other research done on this topic only gives a snapshot in time of the motivation of CLIL and non-CLIL students, as opposed to examining how the students’ motivation changes from the beginning to the end of the CLIL program. As such, the primary research question is •

What effects does CLIL have on students’ language learning motivation?

Also of interest to the researchers is the idea of gender, as well as the interaction of time, CLIL status, and gender. As such, there is a secondary research question to this study, namely •

Is there a significant change in the male and female students’ motivation from the beginning to the end of their senior high school English studies?

More information on the context of the current study, as well as details about the participants, is presented in the following sections. 3.1 Context

English plays an important role in Sweden. Not only is it the only mandatory language to study at school, often introduced as early as the first grade,

CLIL and Motivation Revisited: A Longitudinal Perspective

81

but it is also encountered on a daily basis in everyday life (Hyltenstam, 2004; Josephson, 2004; Medierådet, 2010; Chapter 9, this volume). English-medium TV shows and films are subtitled rather than dubbed, English is used to a great extent in advertising and English is only a click away on the internet. As demonstrated in Chapter 1, each of the three schools involved in the CLISS project represents an idiosyncratic model of CLIL, which is typical for the Swedish context where no official guidelines on CLIL exist (see Sylvén, 2013, for an overview of CLIL in Sweden). Importantly, and as pointed out in several of the other chapters in this volume, CLIL is not mandatory, but rather a voluntary choice made by the individual student and the student’s family. 3.2 Participants

The participants in the present study were 15–16 years old (Grade level 10) at the time of the first data collection, and 18–19 (Grade level 12) at the time of the second. The study was carried out with intact groups of CLIL versus non-CLIL students. Although the group distributions are not perfectly balanced, the groups represent the demographic realities of Swedish high schools (Migrationsinfo.se, 2016). Table 5.1 illustrates the distribution of participants across group and gender. 3.3 Instruments and data analysis

As this is a longitudinal study on the effects of CLIL on language learning motivation, all of the participants in this study completed the background questionnaire and the motivation questionnaire twice: once in October 2011 (when the CLIL classes had just been formed) and once in April 2014 (just before high school graduation). Both the extensive background questionnaire and the MFQ (Ryan, 2009) were presented in both English and Swedish for ease of completion for both the CLIL and non-CLIL students and were constructed and completed online with SurveyMonkey (http://www.surveymonkey.com/). The translation into Swedish was done by one of the authors, checked for accuracy by a native Swedish university EFL teacher, and piloted with an L1 Swedish speaker. Table 5.1 Participant demographics T1

T2

Total

177

111

CLIL

109

60

Non-CLIL

68

51

Females

119

74

58

37

Males

82 Part 2: Assessment and Motivation

The MFQ was influenced by Dörnyei’s (2009) L2 Motivational Self System, as well as the study by Dörnyei et al. (2006), and includes different types of motivation found in diverse frameworks (i.e. the ideal L2 self and instrumentality). There are also several sections in the MFQ that investigate other constructs related to motivation, such as travel orientation, lack of self-confidence, and anxiety, among others. The MFQ’s 18 predetermined factors are as follows: • • • • • • • • • • • • • • • • •

cultural interest (6 questions) attitudes towards the L2 community (8 questions) instrumentality (10 questions) international contact (4 questions) interest in foreign languages (5 questions) international empathy (3 questions) fear of assimilation (4 questions) ethnocentrism (5 questions) travel orientation (4 questions) English anxiety (6 questions) attitudes towards learning English (6 questions) milieu (6 questions) parental encouragement (4 questions) the ideal L2 self (6 questions) L2 self-confidence (5 questions) willingness to communicate (WTC, in the L1 and English, 8 questions each) intended learning effort (8 questions).

The MFQ consists of three separate 6-point Likert scales: the first 20 questions with the scale of not at all (1) to tremendously (6); seven situations for which the participants had to choose how often they would initiate a conversation in their L1 or in English with the scale of I never initiate conversation at all (1) to I always initiate conversation (6); and the remaining questions with the scale of strongly disagree (1) to strongly agree (6). While it is best practice to perform an exploratory factor analysis (EFA) with questionnaire responses each time the questionnaire is used in a new context (see Thompson & Lee, 2013), there were not enough participants in this sample to result in reliable and theoretically consistent results. As data could only be collected from the students in the schools participating in the CLISS project, no further data could be collected to improve the sample size. The data were analyzed with SPSS Version 22.0 using a repeatedmeasure ANOVA for the Time 1 (T1) and Time 2 (T2) results for each of the 18 predetermined factors. CLIL status and gender were the independent variables and the MFQ factors were the dependent variables. To get a score for each participant for each of the factors, the items grouped in each factor were averaged. For example, the factor ‘international

CLIL and Motivation Revisited: A Longitudinal Perspective

83

empathy’ is composed of questions 46, 50 and 87. If a participant answered 4 for Question 46, 2 for Question 50 and 3 for Question 87, that participant’s score for ‘international empathy’ would be 3. To reduce the possibility of Type I error (i.e. a false positive due to too many t-tests), t-tests were only performed on those factors for CLIL and gender for which interactions were found; the main effects results are interpreted descriptively. For CLIL, an interaction with time was found for English anxiety, and main effects were found for international contact, interest in foreign languages, ethnocentrism, English anxiety, attitudes towards English, ideal L2 self, lack of L2 self-confidence, WTC in English, and intended learning effort. For gender, an interaction with time was found for English anxiety and lack of L2 self-confidence, and main effects were only found for cultural interest, travel orientation, English anxiety and lack of L2 self-confidence. There was also a main effect for time for attitudes towards English and intended learning effort, with the ideal L2 self approaching significance. 4 Results 4.1 CLIL status

RQ1 addresses the issue of the potential effect of CLIL on language learning motivation. To examine this question, a repeated-measures ANOVA was performed with time (at the start and fi nish of CLIL) as the within-subject variable and CLIL status and gender as the two betweensubject variables. The descriptive statistics of the CLIL and non-CLIL students at T1 and T2 are presented in Table 5.2.

Table 5.2 Descriptive statistics for factors with significant interactions and main effects: CLIL status at T1 and T2 T1

T2

CLIL

Non-CLIL

CLIL

Non-CLIL

M/SD

M/SD

M/SD

M/SD

F4: International contact

5.20/0.54

4.99/0.86

5.24/0.61

4.87/1.03

F5: Interest in foreign languages

4.86/0.72

4.32/0.98

4.78/0.86

4.55/1.06

F8: Ethnocentrism

2.21/0.64

2.71/0.89

2.03/0.68

2.65/0.94

F10: English anxiety

2.85/1.11

3.37/1.23

2.56/0.99

3.49/1.28

F11: Attitudes towards learning English

4.86/0.71

4.19/0.96

4.58/0.80

4.09/1.16

F14: Ideal L2 self

5.33/0.63

4.69/0.91

5.17/0.63

4.55/1.19

F15: Lack of L2 self-confidence

2.55/0.76

2.98/1.10

2.47/0.71

3.05/1.12

F17: WTC in English

3.96/0.88

3.27/1.05

4.05/0.88

3.35/1.08

F18: Intended learning effort

4.68/0.74

4.42/0.83

4.38/0.84

4.09/1.08

84 Part 2: Assessment and Motivation

There was one interaction between CLIL status and time (F10: English anxiety), indicating that the change over time was dependent on the group (CLIL versus non-CLIL). As can be seen in Figure 5.1, there was a significant group difference at T1 (p = 0.006; Sylvén & Thompson, 2015), a difference that became even larger at T2. The CLIL group became significantly less anxious (p = 0.026), whereas the non-CLIL group became more anxious (although not significantly so, p = 0.378). Table 5.3 shows a summary of the results of the repeated-measure ANOVAs with the nine factors for which the main effects of CLIL were found. When a significant interaction is found, interpretation of the interaction takes precedence over the interpretations of the main effects (Field, 2013); thus, the main discussion of English anxiety is above and is illustrated in Figure 5.1 below. In order to understand the results presented in Tables 5.2 and 5.3, CLIL/non-CLIL motivational differences at T1 as reported by Sylvén and Thompson (2015) are included, as well as the differences between the groups at T2. At T1 there were significant differences in the following factors: • • • • • • •

F5: Interest in foreign languages (p = 0.008) F8: Ethnocentrism (p < 0.001) F10: English anxiety (p = 0.006) F11: Attitudes towards learning English (p < 0.001) F14: Ideal L2 self (p < 0.001) F15: Lack of L2 self-confidence (p = 0.005) F17: WTC in English (p < 0.001).

Figure 5.1 CLIL/non-CLIL anxiety interactions for T1 to T2

CLIL and Motivation Revisited: A Longitudinal Perspective

85

Table 5.3 Repeated measures ANOVA results: Main effects for CLIL status df

F

Partial η2

Observed power

p

F4: International contact

1, 109

4.88

0.044

0.591

0.029

F5: Interest in foreign languages

1, 109

6.02

0.053

0.681

0.016

F8: Ethnocentrism

1, 109

17.21

0.144

0.988

0.000

F10: English anxiety

1, 109

21.00

0.194

0.995

0.000

F11: Attitudes towards learning English

1, 109

13.64

0.113

0.955

0.000

F14: Ideal L2 self

1, 109

18.51

0.147

0.989

0.000 0.000

F15: Lack of L2 self-confidence

1, 109

14.92

0.118

0.963

F17: WTC in English

1, 109

19.02

0.151

0.991

0.000

F18: Intended learning effort

1, 109

4.61

0.041

0.566

0.034

In other words, ‘CLIL students have a greater interest in foreign languages, more positive attitudes towards learning English, a stronger ideal L2 self, more English self-confidence, and a higher willingness to communicate (WTC) in English. Non-CLIL students are more ethnocentric and have higher English anxiety’ (Sylvén & Thompson, 2015: 36). As there was not an interaction effect for CLIL status and time for factors others than English anxiety, this indicates that the CLIL and non-CLIL groups are acting differently, irrespective of time. That being said, looking at the mean scores, we can look at the trends of the data to discuss the results descriptively. Using the mean scores from Table 5.2, we can see that the results at T2 looked similar to those at T1, with two notable differences. F5: Interest in foreign languages (indicated by the grey arrows in Figures 5.2 and 5.3) showed a significant group difference at T1; however, at T2 the mean scores are much closer. The opposite trend can be seen for F4: International contact (indicated by the black arrows in Figures 5.2 and 5.3), which was not significantly different at T1. At T2, however, the mean scores are further apart. The mean score differences for the other factors did not show much change from T1 to T2. The crucial aspect of RQ1 is whether or not a significant change could be seen over the three-year period, depending on whether or not the students were enrolled in CLIL or traditional EFL (non-CLIL) classes. In other words, does CLIL change aspects of students’ motivational profiles? As discussed above, the only factor with an interaction effect for group and time was anxiety; however, looking at the group means in Table 5.2, there are three additional factors with large mean difference changes from T1 to T2 for the CLIL group only (Figure 5.4, indicated by the black arrows): ethnocentrism, attitudes towards English and intended learning effort. These results show that, after three years of CLIL curriculum, the students are less ethnocentric, are less anxious, like English less, and put

86 Part 2: Assessment and Motivation

6 5.5 5 4.5 4 3.5 3 2.5 2 1.5 1 Intended learning effort

WTC in English

Lack of L2 self confidence

Ideal L2 self

Attitudes towards learning English

Ethnocentrism

Interest in FLs

International contact

CLIL NCLIL

T1: CLIL vs. NCLIL

Figure 5.2 T1 differences with the CLIL and non-CLIL students

less effort towards learning English. As for the non-CLIL students, there were no large mean difference changes from T1 to T2, indicating that standard EFL classes (which CLIL and non-CLIL students attended throughout senior high school) affect students less in terms of motivational profiles. The implications of these results are more closely 6 5.5 5 4.5 4 3.5 3 2.5 2 1.5 1

T2: CLIL vs. NCLIL

Figure 5.3 T2 differences with the CLIL and non-CLIL students

Intended learning effort

WTC in English

Lack of L2 self confidence

Ideal L2 self

Attitudes towards learning English

Ethnocentrism

Interest in FLs

International contact

CLIL NCLIL

CLIL and Motivation Revisited: A Longitudinal Perspective

87

6 5.5 5 4.5 4 3.5

T1

3

T2

2.5 2 1.5 1 CLIL

NCLIL

Ethnocentrism

CLIL

NCLIL

Anxiety

CLIL

NCLIL

Attitudes towards English

CLIL

NCLIL

Intended Learning Effort

Figure 5.4 T1 to T2 differences with the CLIL and non-CLIL students

examined in the discussion section, as these fi ndings provide insight into the affective changes that a CLIL curriculum can instigate with regards to English language learning. 4.2 Gender

RQ2 concerns the potential change in motivation of the male and female (both CLIL and non-CLIL) students over the course of their senior high school English education. As mentioned above, to reduce the possibility for Type I error, t-tests were only performed on those factors for CLIL and gender for which interactions were found, with main effects being interpreted descriptively. For gender, main effects were found for four of the factors: cultural interest, travel orientation, English anxiety and lack of L2 self-confidence. The descriptive statistics of the male and female students at T1 and T2 are presented in Table 5.4. There were two interactions between gender and time (F10: English anxiety and F15: Lack of L2 self-confidence). For English anxiety, at both T1 and T2, males were significantly less anxious than females, although the difference became smaller at T2 (T1, p < 0.001; T2, p = 0.01). The females became significantly less anxious from T1 to T2 (p = 0.033); the males became more anxious, although not significantly so (p = 0.16; Figure 5.5). As can be seen in Figure 5.6, at T1 there was a significant difference in lack of L2 self-confidence between males and females (p < 0.001), although

88 Part 2: Assessment and Motivation

Table 5.4 Descriptive statistics for factors with main effects: Gender at T1 and T2 T1

T2

Male

Female

Male

Female

M/SD

M/SD

M/SD

M/SD

F1: Cultural interest

4.39/0.69

4.77/0.65

4.40/0.72

4.79/0.67

F9: Travel orientation

5.30/0.74

5.59/0.61

5.20/0.65

5.48/0.72

F10: English anxiety

2.36/1.01

3.45/1.11

2.57/1.01

3.19/1.27

F15: Lack of L2 self-confidence

2.26/0.78

2.99/0.94

2.52/0.88

2.84/0.99

this difference disappeared at T2 (p = 0.10). Males had significantly higher scores in lack of self-confidence (i.e. less self-confidence) at T2 (p = 0.041), and although females had less lack of self-confidence (i.e. more selfconfidence) at the end of the CLIL curriculum, there was no significant difference for females between T1 and T2 (p = 0.096). Table 5.5 shows a summary of the results of the repeated-measure ANOVAs with the four factors for which main effects of gender were found. Baseline results from Sylvén and Thompson (2015) are included, as well as the differences between the groups at T2, in order to be better able to understand Tables 5.4 and 5.5. At T1 there were eight significant differences with the following factors: • • • • •

F1: Cultural interest (p < 0.001) F5: Interest in foreign languages (p = 0.001) F6: International empathy (p = 0.023) F9: Travel orientation (p = 0.001) F10: English anxiety (p < 0.001)

Figure 5.5 Male and female anxiety interactions for T1 to T2

CLIL and Motivation Revisited: A Longitudinal Perspective

89

Figure 5.6 Male and female lack of self-confidence interactions for T1 to T2 Table 5.5 Repeated measures ANOVA results: Main effects for gender df

F1: Cultural interest

1, 109

F

9.84

Partial η2

0.084

Observed power 0.875

p

0.002

F9: Travel orientation

1, 109

5.00

0.045

0.601

0.027

F10: English anxiety

1, 109

24.82

0.188

0.999

0.000

F15: Lack of L2 self-confidence

1, 109

13.92

0.115

0.959

0.000

• • •

F13: Parental encouragement (p = 0.016) F15: Lack of L2 self-confidence (p < 0.001) F18: Intended learning effort (p = 0.002)

As stated in Sylvén and Thompson (2015: 37): ‘females have more English cultural interest, more interest in foreign languages, more international empathy, a stronger travel orientation, higher English anxiety, more intended learning effort, and they perceive more parental encouragement. However, males have higher L2 self-confidence.’ With two of these factors, English anxiety and lack of L2 self-confidence, there was an interaction between group and time. Other than these two factors, there were only two additional factors with a main effect for gender: cultural interest and travel orientation. In both cases, females had a higher mean score at T2 than did the males. 5 Discussion

This chapter has accounted for a longitudinal investigation over three years looking into possible effects of CLIL on language learning

90 Part 2: Assessment and Motivation

motivation. As illustrated in Sylvén and Thompson (2015), these CLIL students were significantly more motivated than non-CLIL students before starting the CLIL curriculum, fi ndings which are corroborated in studies in other contexts (Rumlich, 2014). In the analyses of the longitudinal data, further insights regarding the motivation of CLIL and nonCLIL students were found. The only significant interaction for group and time with the CLIL analyses was with anxiety (see Tables 5.2 and 5.3, and Figure 5.1), indicating that the change over time with regards to English anxiety depended on the context (CLIL versus non-CLIL). CLIL students are less anxious at the end of their CLIL experience than at the start. As mentioned above, their levels of anxiety were already lower than those of the non-CLIL students from the beginning (Thompson & Sylvén, 2015). Exhibiting even less anxiety after three years can be explained by the fact that the use of the L2/FL has become an everyday occurrence among CLIL students. While the nonCLIL students still may view the L2/FL as a school subject and indeed show a greater degree of anxiety at T2, CLIL students use it as a necessary tool to communicate (cf. Sylvén, 2015), and the anxiousness they may have experienced pre-CLIL has worn off during the three-year period. In addition to anxiety, CLIL and non-CLIL students were found to pattern differently (irrespective of time) for eight more factors: international contact, interest in foreign languages, ethnocentrism, attitudes towards learning English, ideal L2 self, lack of L2 self confidence, WTC in English and intended learning effort. In other words, there was a main effect for group for those factors. With the exception of anxiety, there were no interactions between group and time; however, in order to understand the nuanced results, we can interpret the findings of the main effects for the CLIL versus non-CLIL groups descriptively, using the differences in the mean scores (see Figure 5.4). After the three-year period of senior high school, CLIL students show signs of having become less ethnocentric. Among the statements addressing this factor are ‘I don’t trust people with different customs and values to myself’ and ‘I’m not very interested in the customs and values of other people’. Speculating about the reasons behind these lowered levels of ethnocentricity, they may be explained by the more situated international outlook that the CLIL students acquire as compared to the non-CLIL students, as a result of CLIL. At one of the schools, where English is used as the medium of instruction in all subjects except language arts, the student body is rather heterogeneous with regards to student L1s, and in many cases English is the only language through which classmates can successfully communicate. Thus, students are classmates with people from, or with backgrounds in, other parts of the world, which is a good starting point to achieve an understanding of other cultures. While the student bodies at the other two schools in the study are more homogeneous, both schools include a trip to England as part of the CLIL

CLIL and Motivation Revisited: A Longitudinal Perspective

91

program. During this trip, students live in English host families and are intensely involved in English society for an entire week. The trip, at both schools, entails careful planning before departure, and the experiences are incorporated in various school tasks upon return. Thus, this trip forms an important part of the CLIL students’ experience, and is a good basis for cultural understanding across national contexts. Also potentially related to the amount of English use on a daily basis for CLIL students is the result that after a three-year period, CLIL students have significantly lower scores in intended learning effort. With increased use of English, it could have become integrated into their daily routine, as opposed to being a separate subject on which to focus (cf. Gardner, 2010). Another interpretation could be that by the end of CLIL the students no longer perceived that they were ‘learning English’ per se in their CLIL classes. Instead, they could have re-conceptualized their CLIL classes as learning content, rather than focusing as much on the language learning aspect as they perhaps did at the beginning. Interestingly, the CLIL students like English less at the end of high school than they did at the beginning. The explanation for this development may very well be similar to the ones discussed above for levels of ethnocentrism and anxiousness. Students who opt for CLIL are those who a priori like English, and in most cases are quite proficient in comparison to those who prefer the non-CLIL option (cf. Rumlich, 2014; Sylvén & Ohlander, 2014). During the three years when these students encounter and use English daily, however, it is only to be expected that their sense of novelty and excitement about English diminishes. It should be noted, though, that the trend for the non-CLIL students is exactly the same. They, too, exhibit lower levels of interest in English at T2 than at T1. Having said this, though, our results show that CLIL students still like English more than do the nonCLIL students at the end of the three years in senior high school. In discussions about the pros and cons regarding CLIL, the argument that positive attitudes towards other languages may suffer as a result of English being used as the medium of instruction in most CLIL contexts is often heard (e.g. Bruton, 2013). Our results indicate that CLIL students have a greater interest in foreign languages at the start of senior high school than do non-CLIL students. At T2, the mean score for the CLIL students went down slightly and the mean score for the non-CLIL students went up slightly, making the difference of the mean scores at T2 much smaller. Thus, while it does not seem that the CLIL classes are spurring interest in foreign languages other than English, the CLIL students still have a greater interest in foreign languages at the end of high school than do those students who were not enrolled in CLIL classes. Regarding the results for gender, the females display a higher level of English cultural interest, interest in foreign languages, international empathy, travel orientation, intended learning effort and parental encouragement compared to the males, which supports the trends found in

92 Part 2: Assessment and Motivation

previous research (Meece et al., 2006; Mori & Gobel, 2006). All of these aspects illustrate a greater interest in and motivation for languages. However, seemingly paradoxically, the females also display a higher level of English anxiety and lower levels of L2 self-confidence. This indicates that, even though the females are more motivated and make a greater effort to learn L2 English, at the same time they worry about their proficiency and are not as confident as the male students are, especially at the beginning of senior high school. There were significant interactions between group and time for English anxiety and lack of L2 self-confidence. While females still have significantly more anxiety than males at T2, it should be noted that the females have significantly lower anxiety than they did at the start of senior high school. Additionally, the significant difference for lack of self-confidence no longer exists at T2 for males and females, as it did at T1. Thus, while the social pressures for females to perform well never completely disappear, the gender differences are not as stark at the end of high school. 6 Concluding Remarks

This study has shown that those who chose to participate in CLIL were significantly more motivated on a number of factors before the start of CLIL, suggesting that the most motivated students are those who opt for CLIL. There was more variation in the mean scores from T1 to T2 for the CLIL students, whereas this was not the case for the non-CLIL students, indicating more dynamicity in the motivational profiles of the CLIL students. For example, CLIL students were less ethnocentric and had less anxiety at the end of high school, which are positive changes, and the less positive attitudes towards English and the reduced learning effort by the end of high school also have plausible explanations. As a majority of Swedish high school students study more than one foreign language, a direction of further research would be to investigate the motivational and related variables for other languages with CLIL and non-CLIL students. It would be of great interest for future studies to look into the relationship between CLIL, with English as the medium of instruction, and study of and progress in other foreign languages. One possibility is that the intensive English study in CLIL could make those students more motivated in their other foreign language classes as well; alternatively, the focus on English could reduce their motivation for the study of additional foreign languages. As has been shown in a multitude of studies, motivated students tend to perform better than those who are less motivated; therefore, this is crucial information when evaluating educational contexts such as CLIL, as well as with L2/FL study in general. Finally, as English is used substantially in Swedish society, it is important to conduct longitudinal research in contexts where exposure to English is limited outside of school to investigate the potential effect of context and motivation.

CLIL and Motivation Revisited: A Longitudinal Perspective 93

References Abendroth-Timmer, D. (2007) Akzeptanz und Motivation: Empirische Ansätze zur Erforschung des Einsatzes von bilingualen und mehrsprachigen Modulen. Frankfurt am Main: Peter Lang. Abu-Rabia, S. (2004) Teacher’s role, learner’s gender differences, and FL anxiety among seventh-grade students studying English as a FL. Educational Psychology 24 (5), 711–721. Banegas, D.L. (2013) The integration of content and language as a driving force in the EFS lesson. In E. Ushioda (ed.) International Perspectives on Motivation: Language Learning and Professional Challenges (pp. 82–97). New York: Palgrave Macmillan. Bruton, A. (2013) CLIL: Some of the reasons why … and why not. System 41 (3), 587–597. Carr, J. (2002) Why boys into languages won’t go: The problematic agenda in language education. Babel 37, 1–5. Coyle, D. (2011) ITALIC Research Report. Investigating Student Gains: Content and Language Integrated Learning. Aberdeen: University of Aberdeen. See http://www. abdn.ac.uk/italic. Dalton-Puffer, C. (2011) Content-and-language integrated learning: From practice to principles? Annual Review of Applied Linguistics 31, 182–204. Denman, J., Tanner, R. and de Graaff , R. (2013) CLIL in junior vocational secondary education: Challenges and opportunities for teaching and learning. International Journal of Bilingual Education and Bilingualism 16 (3), 285–300. Doiz, A., Lasagabaster, D. and Sierra, J.M. (2014) CLIL and motivation: The effect of individual and contextual variables. The Language Learning Journal 42 (2), 209–224. Dörnyei, Z. (2009) The L2 motivational self system. In Z. Dörnyei and E. Ushioda (eds) Motivation, Language Identity and the L2 Self (pp. 9–42). Bristol: Multilingual Matters. Dörnyei, Z., Csizér, K. and Németh, N. (2006) Motivational Dynamics, Language Attitudes and Language Globalisation: A Hungarian Perspective. Clevedon: Multilingual Matters. Eurydice Network (2006) Content and Language Integrated Learning (CLIL) at School in Europe. Brussels: Eurydice European Unit. Field, A. (2013) Discovering Statistics Using IBM SPSS Statistics. London: Sage Publications Ltd. Gardner, R.C. (2010) Motivation and Second Language Acquisition. New York: Peter Lang. Genesee, F. (1978) Second language learning and language attitudes. Working Papers on Bilingualism 16, 19–42. Hood, P. (2006) Unpublished data from CLIL research interviews with students at Tile Hill Wood Language College, Coventry, UK. In D. Coyle, P. Hood and D. Marsh (eds) CLIL: Content and Language Integrated Learning (p. 142). Cambridge: Cambridge University Press. Horwitz, E.K., Horwitz, M.B. and Cope, J. (1986) Foreign language classroom anxiety. The Modern Language Journal 70 (2), 125–132. Hyltenstam, K. (2004) Engelskan, skolans språkundervisning och svensk språkpolitik. In B. Lindgren and O. Josephson (eds) Engelskan i Sverige (pp. 36–107). Stockholm: Svenska Språknämnden. Josephson, O. (2004) Engelskan i 2000-talets Sverige. In B. Lindgren and O. Josephson (eds) Engelskan i Sverige (pp. 7–24). Stockholm: Svenska Språknämnden. Kim, S.Y. (2009) Questioning the stability of foreign language classroom anxiety and motivation across different classroom contexts. Foreign Language Annals 42 (1), 138–157.

94 Part 2: Assessment and Motivation

Kissau, S. (2006) Gender differences in motivation to learn French. The Canadian Modern Language Review 62 (3), 401–422. Kissau, S., Kolano, L.Q. and Wang, C. (2010) Perceptions of gender differences in high school students’ motivation to learn Spanish. Foreign Language Annals 43 (4), 703–721. Lambert, W.E. and Tucker, G.R. (1972) Bilingual Education of Children: The St. Lambert Experiment. Rowley, MA: Newbury House. Lasagabaster, D. (2011) English achievement and student motivation in CLIL and EFL settings. Innovation in Language Learning and Teaching 5 (1), 3–18; doi:10.1080/17 501229.2010.519030. Lasagabaster, D. and Sierra, J.M. (2009) Language attitudes in CLIL and traditional EFL classes. International CLIL Research Journal 1 (2), 3–17. Lorenzo, F., Casal, S. and Moore, P. (2010) The effects of content and language integrated learning in European education: Key fi ndings from the Andalusian bilingual sections evaluation project. Applied Linguistics 31 (3), 418–442. MacIntyre, P.D., Baker, S.C., Clément, R. and Donovan, L.A. (2002) Sex and age effects on willingness to communicate, anxiety, perceived competence and L2 motivation among junior high school French immersion students. Language Learning 52 (3), 537–564. Maillat, D. (2010) The pragmatics of L2 in CLIL. In C. Dalton-Puffer, T. Nikula and U. Smit (eds) Language Use and Language Learning in CLIL Classrooms (pp. 39–60). Amsterdam: John Benjamins. Marsh, D. (2000) Using Languages to Learn and Learning to Use Languages. Jyväskylä: University of Jyväskylä. Medierådet (2010) Ungar & Medier 2010. Stockholm: Medierådet. Meece, J.L., Glienke, B.B. and Burg, S. (2006) Gender and motivation. Journal of School Psychology 44 (5), 351–373; doi:10.1016/j.jsp.2006.04.004. Migrationsinfo.se. (2016) Sweden. See http://www.migrationsinfo.se/migration/sverige/. Mori, S. and Gobel, P. (2006) Motivation and gender in the Japanese EFL classroom. System 34 (2), 194–210; doi:10.1016/j.system.2005.11.002. Nikula, T. (2007) Speaking English in Finnish content-based classrooms. World Englishes 26 (2), 206–223. Pavy, S. (2006) Boys learning languages – The myth busted. Babel 41, 2–9. Rumlich, D. (2014) Prospective CLIL and non-CLIL students’ interest in English (classes): A quasi-experimental study on German sixth-graders. In R. Breeze, C. Martinez Pasamar, C. Llamas Saíz and C. Tabernero Sala (eds) Integration of Theory and Practice in CLIL (pp. 75–95). Amsterdam: Rodopi. Ryan, S. (2009) Self and identity in L2 motivation in Japan: The ideal L2 self and Japanese learners of English. In Z. Dörnyei and E. Ushioda (eds) Motivation, Language Identity and the L2 Self (pp. 120–143). Bristol: Multilingual Matters. Sallili, F. and Tsui, A.B.M. (2005) The eff ects of medium of instruction on students’ motivation and learning. In R. Hoosain and F. Sallili (eds) Language in Multicultural Education, Vol. 4 (pp. 135–156). Greenwich, CT: Information Age. Seikkula-Leino, J. (2007) CLIL learning: Achievement levels and aff ective factors. Language and Education 21 (4), 328–341. Swain, M. and Lapkin, S. (1982) Evaluating Bilingual Education: A Canadian Case Study. Clevedon: Multilingual Matters. Sylvén, L.K. (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320. Sylvén, L.K. (2015) CLIL and non-CLIL students’ beliefs about language. Studies in Second Language Learning and Teaching 5 (2), 251–272; doi:10.14746/ssllt. 2015.5.2.4.

CLIL and Motivation Revisited: A Longitudinal Perspective

95

Sylvén, L.K. and Ohlander, S. (2014) The CLISS project: Receptive Vocabulary Proficiency in CLIL and non-CLIL groups. Moderna Språk (2), 81–119. Sylvén, L.K. and Thompson, A.S. (2015) Language learning motivation and CLIL: Is there a connection? Journal of Immersion and Content-Based Language Education 3 (1), 28–50. Thompson, A. and Lee, J. (2013) Anxiety and EFL: Does multilingualism matter? International Journal of Bilingual Education and Bilingualism 16 (6), 730–749. Thompson, A.S. and Sylvén, L.K. (2015) Does English make you nervous? Apples – Journal of Applied Language Studies 9 (2), 1–23. Wesely, P.M. (2009) The language learning motivation of early adolescent French immersion graduates. Foreign Language Annals 42 (2), 270–286.

Part 3 English

Introduction to Part 3: English

This part presents fi ndings on learners’ proficiency and progress in, as well as their exposure to, the target language, English. It consists of four chapters, the first two concerning vocabulary, the third reading comprehension and the fourth extramural English. Vocabulary, or lexis, is often considered the most important aspect of a language, especially from a communicative perspective, being a vital part of all language learning (Carter, 1987; Nation, 1990; Schmitt et al., 2001). Of course, syntax and other linguistic levels such as discourse and pragmatics are also essential for learners, but without lexical units it is not possible to convey a verbal message. Somewhat simplistically, words in isolation, out of context, can be seen as an inventory of core meanings, as accounted for in ordinary dictionaries. Whereas the meaning of some words is inherently vague, e.g. thing and go, getting more well-defi ned contours in specific contexts, the meaning of others may be very specific, almost regardless of context, such as phoneme and aneurysm (Read, 2004: 213). There are also lexical units that have different meanings depending on whether they are used in an everyday conversation or in a more specialized and technical context, e.g. solution and parameter. Academic language is one example of a specialized context, and may here be broadly defi ned as features of vocabulary as well as grammar that occur more often in academic contexts than in others. Several attempts have been made to create academic wordlists. Within the CLISS project, the lists compiled by Coxhead (2000) and Gardner and Davies (2014) have been used in the analyses of learners’ proficiency and progress in English academic vocabulary. From a second and foreign language (L2/FL) perspective, vocabulary proficiency and acquisition are of great interest to investigate in language learning contexts because, as already noted, words form one of the basic building blocks of a language. As CLIL is a specific L2/FL language learning context, vocabulary provides an area of obvious relevance within the CLISS project. In Chapter 6, Liss Kerstin Sylvén and Sölve Ohlander account for the assessment tools used to investigate learners’ proficiency

98

Introduction to Part 3: English

99

and progress in receptive vocabulary. By contrast, Chapter 7 by Eva Olsson and Liss Kerstin Sylvén explores learners’ productive vocabulary. Closely linked to lexical proficiency is reading comprehension (Alderson, 2000). In the CLISS project, the participants were enrolled in high school preparatory programs for higher education, the common aim for the CLISS students. At this level, requirements of understanding written texts are high, and in many higher educational contexts those texts are in English, the world’s lingua franca par excellence and the fi rst L2/FL for most students in a country like Sweden. Consequently, it is of great relevance to investigate the extent to which CLIL prepares students for their future academic careers in this regard. This is the focus of Chapter 8 by Liss Kerstin Sylvén and Sölve Ohlander. Another venue than the educational context for the learning of L2/ FL English is exposure to English beyond the formal education of schooling. Such extramural English (EE) has been proven to be particularly beneficial for the learning of vocabulary (Hannibal Jensen, 2017; Olsson, 2016; Sundqvist & Sylvén, 2012; Sylvén, 2006; Sylvén & Sundqvist, 2012). EE involves all the contact language learners have with English in non-educational settings not primarily aimed at learning the language. The defi nition used here of EE is ‘English-related activities that learners come in contact with or are engaged in outside of the walls of the English classroom, generally on a voluntary basis’ (Sundqvist & Sylvén, 2014: 4). Some examples of EE are reading books or magazines, watching TV, listening to music and playing digital games. EE plays a significant role in sustaining global Englishes throughout the world, but especially in Europe’s non-English speaking countries, not least in Scandinavia. We know, for instance, that Swedish ten-year-olds average around seven hours per week on various EE activities, 12-year-olds around nine hours per week and 15-year-olds around 18 hours (Sundqvist, 2011; Sundqvist & Sylvén, 2014; Sylvén & Sundqvist, 2012). The exposure to English outside of school among participants in the CLISS project and its impact on certain learning outcomes are dealt with in Chapter 9 by Liss Kerstin Sylvén, which concludes the section on English as a target language. References Alderson, J.C. (2000) Assessing Reading. Cambridge: Cambridge University Press. Carter, R. (1987) Vocabulary – Applied Linguistic Perspectives. London: Allen & Unwin. Coxhead, A. (2000) A new academic word list. TESOL 34 (2), 213–238. Gardner, D. and Davies, M. (2014) A New Academic Vocabulary List. Applied Linguistics 35 (3), 305–327; doi:10.1093/applin/amt015. Hannibal Jensen, S. (2017) Gaming as an English language learning resource among young children in Denmark. CALICO Journal 1 (34), 1–19; doi:10.1558/cj.29519. Nation, P. (1990) Teaching and Learning Vocabulary. New York: Newbury House.

100

Part 3: English

Olsson, E. (2016) On the impact of extramural English and CLIL on productive vocabulary. PhD thesis, University of Gothenburg. See https://gupea.ub.gu.se/ handle/2077/41359. Read, J. (2004) Plumbing the depths: How should the construct of vocabulary knowledge be defi ned? In P. Boogards and B. Laufer (eds) Vocabulary in a Second Language (pp. 209–227). Amsterdam: John Benjamins. Schmitt, N., Schmitt, D. and Clapham, C. (2001) Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing 18 (1), 55–88. Sundqvist, P. (2011) A possible path to progress: Out-of-school English language learners in Sweden. In P. Benson and H. Reinders (eds) Beyond the Language Classroom (pp. 106–118). Basingstoke: Palgrave Macmillan. Sundqvist, P. and Sylvén, L.K. (2012) World of VocCraft: Computer games and Swedish learners’ L2 vocabulary. In H. Reinders (ed.) Digital Games in Language Learning and Teaching (pp. 189–208). Basingstoke: Palgrave Macmillan. Sundqvist, P. and Sylvén, L.K. (2014) Language-related computer use: Focus on young L2 English learners in Sweden. ReCALL 26 (1), 3–20; doi:10.1017/S0958344013000232. Sylvén, L.K. (2006) Extramural exposure to English among Swedish school students. In J. Einarsson, E. Larsson Ringqvist and M. Lindgren (eds) Approaches to Teaching and Learning in Linguistic Research: Papers from the ASLA Symposium in Växjö, 10–11 November 2005 (pp. 252–260). Uppsala: ASLA, Svenska föreningen för tillämpad språkvetenskap. Sylvén, L.K. and Sundqvist, P. (2012) Gaming as extramural English L2 learning and L2 proficiency among young learners. ReCALL 24 (3), 302–321; doi:10.1017/ S095834401200016X.

6 English Receptive Vocabulary Liss Kerstin Sylvén and Sölve Ohlander

1 Introduction

This chapter is focused on what is referred to as receptive vocabulary. The chapter starts with a brief discussion of what the term means and a presentation of some of the most commonly used types of receptive vocabulary tests. Then follows a survey of previous literature on receptive vocabulary in CLIL learning contexts. The chapter then describes the methods and material used in the present study. After that, the results are presented, followed by a discussion. The chapter ends with some concluding remarks, including some pedagogical implications of the main findings. 2 Vocabulary

Vocabulary may at first sight seem like a fairly straightforward part of language to investigate in a second or foreign language (L2/FL) learning context. However, vocabulary is a multifaceted building block of language, and so it is crucial to establish exactly what facet is being investigated. Defi nitions of what a word is differ among scholars. Here, we adhere to the defi nition of the term lexical unit by Cruse (1986), with two criteria stating that it has to be (1) at least one word, and (2) a semantic constituent. Bogaards (2001) explains as follows: A semantic constituent is a part of a whole (word, phrase, or sentence) that contributes in an independent way to the overall meaning of that whole. This means that John and sleeps in John sleeps as well as the prefi x dis- in disobey are semantic constituents. However, dis- is not a lexical unit because it is not a word (see criterion 1). In a sentence like Arthur pulled a fast one, the verb form pulled is not a lexical unit because, although it is a word, it is not a semantic constituent (see criterion 2). (Bogaards, 2001: 325)

(In Bogaards’ second example, pulled is part of an idiom, to pull a fast one [‘to trick somebody’], thus not an independent semantic constituent.)

101

102

Part 3: English

Further, the ultimate description of what it means to know a word, or a lexical unit, has yet to be formulated. Several scholars have made attempts at listing various aspects that need to be included in such a definition (e.g. Carter, 1987; Meara, 1996). The most complete so far, involving a variety of different levels or dimensions, is shown in Table 6.1 (see also Chapter 12). Partially overlapping, and partially complementing, the above list are the three parameters of word knowledge set out by Henriksen (1999), who argues that one of the dimensions inherent in vocabulary proficiency is the continuum ranging from partial to precise knowledge of a word, i.e. how much of the meaning of a word is known. Another dimension is the depth and range of word knowledge, which includes aspects of semantics, syntax and morphology. The third dimension is the continuum of receptive and productive knowledge where, at the receptive end, knowledge of the meaning of the word is emphasized. At the other end, the focus is on the productive ability of using the word in speech and/or writing. Often, the terms ‘passive’ and ‘active’ are used instead of receptive and productive, but as we regard all use of language as active we prefer the latter terminology; however, see, for example, Laufer (1998) for a different view. For our present limited purposes, we defi ne receptive vocabulary proficiency as knowing (one or several of) the meaning(s) of a specific word. Table 6.1 What does it mean to know a word? Form

Spoken

Receptive: what does a word sound like? Productive: how is it pronounced?

Written

Receptive: what does a word look like? Productive: how is a word written and spelled?

Word parts

Receptive: what parts are recognizable in a word? Productive: what word parts are needed to express the meaning?

Meaning Form and meaning

Use

Receptive: what meaning does a word form signal? Productive: what word form can be used to express a meaning?

Concept and referents

Receptive: what is included in a concept? Productive: what items can a concept refer to?

Associations

Receptive: what other words does a word make you think of? Productive: what other words could be used instead of this one?

Grammatical functions Receptive: in what pattern does a word occur? Productive: in what pattern must this word be used? Collocations

Receptive: what other words or types of words occur with a word? Productive: what other words or types of words must/ should/could be used with a word?

Constraints on use

Receptive: where, when and how often could a word be expected to occur? Productive: where, when and how often can a word be used?

Source: Adapted from Nation (2001: 27).

English Receptive Vocabulary

103

From another angle, instead of using the metaphor of a continuum, Laufer (1998) argues for three stages of word knowledge: passive knowledge, controlled active knowledge and free active knowledge. The fi rst stage coincides with the above defi nition of receptive knowledge, and the difference between the two latter notions is that, while controlled active knowledge means the ability to produce a word when required to do so, free active knowledge entails the ability to use words without an explicit stimulus, as in normal conversation. Taking the above perspectives on vocabulary knowledge into account, it can be concluded that testing the proficiency and progress of learners’ L2/FL vocabulary is no trivial task; establishing the full picture requires a large number of different tests and test formats. To prevent the test fatigue likely to occur if every aspect of vocabulary knowledge were to be taken into account, certain limitations are needed. In the present study, the investigation is restricted to students’ receptive and productive L2/FL English vocabulary. This chapter focuses on receptive lexical knowledge within the CLISS project, the next one on productive vocabulary. 3 Receptive Vocabulary and How to Test It

As already stressed, receptive knowledge of a word means at the very least understanding its meaning, though not necessarily its full contextually determined range. How, then, can such proficiency be tested and measured? There are several test formats available. Below we account for some of the most frequently used test types. Multiple choice is possibly the most common type of test for measuring receptive vocabulary (see, for example, Read, 2000). One advantage of using such tests is that they are easy for the test taker to understand and do, and are easily administered. A drawback is that, in order to obtain good reliability and validity on such tests, a large number of words need to be included, and the tests tend to become long and time-consuming. The test format illustrated in Example 6.1, taken from Schmitt (2000), is often used, where the target word and the answer alternatives are given without context, and only one correct meaning (B: hard) is found among the alternatives. This means that should the test taker know another meaning of the target item – e.g. business in this case – such knowledge is not captured by this type of test. Example 6.1

Firm A deep B hard C ward D clean Another type of multiple choice test is where the test item is given in isolation, but the answer alternatives are provided in context. In Example

104

Part 3: English

6.2, taken from vocabtest.com, alternative E is the correct one, where two of the target item’s meanings are captured: Example 6.2

The word LITANY most nearly means: A Worn out; Wearied B Something added to a building, an attachment; To add at the end, to attach C Perceived by touch or capable of being touched D Going straight to the point, frank, direct E A recital of prayers in response to a leader; A long or drawn-out account

Yet another variant of multiple choice is given in Example 6.3, where the target item is given in context and the answer alternatives stand in isolation (correct alternative: E), a format especially suitable when a specific meaning of a word is tested. Example 6.3

It was an example of a fast-moving narrative A car B career C lie D song E story

The Vocabulary Size Test (VST; Nation & Beglar, 2007), is a further format of multiple choice test, the original aim of which was to test, as the name suggests, the size of the test taker’s vocabulary. In the VST, the target item is first given in isolation and then repeated in a full but simple sentence, in order to reveal nothing more than the word class to which the word belongs, as in Example 6.4 (correct alternative: A): Example 6.4

Miniature: It is a miniature A A very small thing of its kind B An instrument for looking at very small objects C A very small living creature D A small line to join letters in handwriting Another test type is the so-called yes/no test, developed by Meara and colleagues (Meara & Jones, 1990). The test consists of a large number of words, where the test taker indicates whether or not a given explanation of a word is correct. In the test are also found nonsense words, i.e. words that do not exist, e.g. nonagrate, lannery and aistrope. The yes/no test is available for various frequency levels and is available on the internet (http://www.lextutor.ca). As with the VST, the aim of the yes/no test is to

English Receptive Vocabulary

105

measure test takers’ vocabulary size. One advantage of this test is the ease with which it can be administered. A disadvantage, however, is that in order to correctly reflect the test taker’s level of proficiency, the nonsense words, and how each individual has responded to them, need to be meticulously accounted for and results calculated by means of special formulas (Mochida & Harrington, 2006). A test-type based on the multiple choice idea, yet unique in its design, is the Vocabulary Levels Test (VLT; Nation, 2001). The VLT has been widely used for a variety of purposes. Originally it was intended as a diagnostic test, but its use has since been much extended (Beglar, 2010). In the VLT, six words are given, three of which are to be matched with the given synonyms, definitions or paraphrases, as shown in Example 6.5. A clear advantage of this test is that, in effect, all six words are tested, even though only three of them (in Example 6.5: horse, pencil, wall) match up with a counterpart. Example 6.5

1 2 3 4 5 6

business clock horse pencil shoe wall

_____ part of a house _____ something used for writing _____ animal with four legs

The VLT is frequency based, and the version used in the present study consists of words – in order of decreasing frequency – from the 2000, 3000, 5000 and 10,000 levels, i.e. lexical items found among the 2000 most common English words (e.g. debt and introduce), among the 3000 most common ones (e.g. dignity and prospect), among the 5000 most common ones (e.g. mortgage and provoke), up to items found among the 10,000 most common words (e.g. tier and obsolete), representing the most infrequent ones. This means that the test becomes more difficult the lower the frequency level of the target words. In addition, there is a section consisting of academic words (e.g. evidence and emerge) included in the present version of the VLT, where the words are taken from the Academic Word List (AWL; Coxhead, 2000). Among the various multiple choice formats available, with their respective mixes of advantages and disadvantages, the VLT was chosen as the instrument best suited to test receptive vocabulary proficiency in the CLISS project. This decision was primarily made due to the special design of the VLT and the fact that it is frequency based, thus providing insights into learners’ vocabulary range and possible progress over the course of the three years of the project. Another no less specific reason for choosing the VLT was that it had already been used in previous studies on L2/FL learning effects of CLIL (Merikivi & Pietilä, 2014; Sylvén, 2004), enabling comparison with other cohorts and contexts. A further circumstance in

106

Part 3: English

its favor is that the VLT has been validated as a reliable test by experts in the field of vocabulary acquisition (Beglar & Hunt, 1999; Cobb, 1997; Schmitt et al., 2001). 4 CLIL and Receptive Vocabulary Learning

Given that receptive vocabulary is a fundamental aspect of language proficiency, it is surprising that there are very few such studies to be found in connection with CLIL. One of them is Jiménez Catalán and Ruiz de Zarobe (2009), where it was shown that female CLIL students in the sixth grade outperform their female non-CLIL peers on receptive vocabulary tests. At the time of the testing, the CLIL students’ total exposure to English in school was 960 hours, compared to 629 for the non-CLIL group. Thus, the CLIL group had been exposed to English significantly longer than the non-CLIL group. Therefore, any other outcome than the one reported would have been highly unlikely. However, little is revealed in this study about the effects of CLIL, as no information about pre-CLIL levels of proficiency is presented. Another study looking into CLIL and receptive vocabulary – which accounts for baseline, pre-CLIL vocabulary proficiency – is Admiraal et al. (2006). In their study, CLIL students (N = 1305) were shown to initially score higher than their non-CLIL peers on receptive vocabulary tests. The CLIL students’ advantage remained after four years of CLIL, but the difference between the CLIL and non-CLIL students did not increase, as would have been the expected outcome. In a study specifically addressing the gender perspective, Jiménez Catalán (2010) used the VLT (see above) to test 12-year-old boys and girls in a Spanish context. Her results show no gender differences. Merikivi and Pietilä (2014), in their study of sixth and ninth grade CLIL and non-CLIL students, also used the VLT to investigate students’ receptive vocabulary skills. The CLIL students involved in this cross-sectional study – different groups of students being tested at the same time, rather than one and the same group being tested twice, with three years between the two tests – had attended CLIL from Grade 1. Thus, their exposure to English in school had been considerably larger than that of the non-CLIL group already in Grade 6. Statistically, as expected, the CLIL students significantly outperformed their non-CLIL peers in both Grades 6 and 9 (p < 0.001). The authors conclude, however, that even though CLIL seems to be conducive to vocabulary learning, its full potential was not taken advantage of, and that even better progress could be expected among CLIL students. Studies in the Swedish context on CLIL and receptive vocabulary have shown results which parallel those found in Admiraal et al. (2006), i.e. that CLIL students do not progress more than non-CLIL ones. In a study from 1997, Washburn investigated the proficiency and progress among

English Receptive Vocabulary

107

CLIL and non-CLIL students. While the CLIL students were significantly more proficient at the outset, two years later the results between the two groups were barely distinguishable. In a similar vein, Sylvén (2004) showed that CLIL students scored significantly higher both pre-CLIL and after two years of CLIL, as compared to their non-CLIL peers, who improved their results in general but not significantly so. However, in the study, measures of students’ exposure to English outside of school, socalled extramural English (EE), were also taken into account (Sundqvist & Sylvén, 2016). It was shown that, rather than being an effect of CLIL, the superiority of the CLIL students was explained by their significantly larger amount of EE. These somewhat negative results as regards CLIL in the Swedish context have so far only been found when English is the target language. Both Dentler (2002) and Terlevic Johansson (2011), in their studies of CLIL with German as the target language, show significant vocabulary progress among the CLIL students. There may be several explanations for this; see Sylvén (2013) for a full discussion. Having looked at what some other studies have found as regards CLIL and receptive vocabulary, we now proceed to a detailed account of the present study. 5 The Present Study: Method and Materials

As stated previously, students’ proficiency and progress in receptive English vocabulary were measured by the VLT. The test was administered twice: fi rst, at the outset of Grade level 10, coinciding with the start of CLIL for the CLIL students, referred to as T1; and secondly, towards the end of Grade level 12, i.e. close to three school years later, referred to as T2. The participants in these tests are accounted for in Table 6.2. As illustrated in Table 6.2, the number of CLIL students is larger than that of the non-CLIL group (118 versus 76 at T1), which is explained by the fact that five CLIL and only three non-CLIL classes were involved in the CLISS project. In addition, the number of participants is significantly reduced between T1 and T2, for several reasons. In the first place, only those students who took part in both tests are involved in the present analyses. Further, almost three years elapsed between T1 and T2, a time span when life events impact: for instance, several students changed classes, some moved to other schools and others were unable to attend at both T1 Table 6.2 Participants in the Vocabulary Levels Test, T1 and T2 CLIL

Non-CLIL

Total

Female

Male

Total

Female

Male

Total

Female

Male

Total

T1

87

31

118

42

34

76

129

65

194

T2

44

13

57

23

29

52

67

42

109

108

Part 3: English

and T2. Such variation in numbers is only to be expected in a project like CLISS, when intact classes participate and tests are administered in school. The tests were conducted during regular school hours. As pointed out elsewhere in this volume, participation was voluntary. Further, students were informed that the tests would not have any influence on their grades, which may have had an impact on who actually attended to take the test. Wherever feasible, a member of the CLISS research team was in the classroom to administer the tests and give information about how to fi ll it out. In those cases where, due to the physical distance between the research team and the school, it was not possible for CLISS researchers to attend, detailed information about the test procedure was conveyed to the teacher in charge of the class, who then administered the tests, collected them and sent them to the project leader. 6 Results

In this section, results from the two test rounds (T1 and T2) will be accounted for and comparisons made. An in-depth analysis of the results obtained from T1 can be found in Sylvén and Ohlander (2014). First, CLIL and non-CLIL are in focus; then gender is added to the analyses. The total results on the VLT are presented in Figure 6.1. On the Y-axis, the mean results are found, and on the X-axis are the two groups at T1 and T2, respectively. As illustrated in Figure 6.1, the CLIL group is ahead of the non-CLIL group at both T1 and T2. An independent samples t-test shows that on both occasions the CLIL students were statistically significantly ahead of their non-CLIL peers (p < 0.001). As the VLT builds on frequency levels, its design allows for detailed analyses concerning the various levels of the test. The results are given in Tables 6.3 and 6.4.

140 120

129 112 99

100

99

80

CLIL

60

non-CLIL

40 20 0 T1

T2

Figure 6.1 VLT results, T1 and T2, CLIL versus non-CLIL

English Receptive Vocabulary

109

Table 6.3 VLT results per frequency level at T1, CLIL versus non-CLIL VLT T1 2000 level (max = 30)

CLIL 29

SD 2.9

Non-CLIL 26

SD 4.6

**

3000 level (max = 33)

28

4.7

24

6.0

**

5000 level (max = 27)

18

6.3

16

6.3

*

9

4.3

8

3.5

ns

29

7.0

25

7.6

**

112

22.8

99

25.1

**

10,000 level (max = 18) AWL (max = 42) TOTAL (max = 150)

Note: * = significant at 0.05; ** = significant at 0.001.

It emerges from Table 6.3 that the CLIL students are significantly more proficient at all frequency levels, except for the 10,000 level, than their nonCLIL peers at T1. In other words, CLIL students know considerably more of the more frequent words and the words in the AWL section, but their level of proficiency for the most difficult words is equal to that of the nonCLIL students at T1. It is also evident that the standard deviation for the 2000 and 3000 levels is higher in the non-CLIL group, indicating that there is a spread, with individuals above as well as below the general mean. Table 6.4 shows that, at T2, CLIL students outperform their nonCLIL peers significantly at the 0.001 level for all frequency levels, including those with the most infrequent words. Again, the standard deviation for the non-CLIL group is much higher than for the CLIL group, roughly 40 compared to 14 in the total result. This shows that the level of proficiency in the non-CLIL group is much more heterogeneous than that in the CLIL group. Table 6.4 VLT results per frequency level at T2, CLIL versus non-CLIL VLT T2

CLIL

SD

Non-CLIL

SD

2000 level (max = 30)

29

1.5

25

8.4

**

3000 level (max = 33)

31

2.4

24

10.0

**

5000 level (max = 27)

22

4.5

17

8.1

**

10,000 level (max = 18)

12

4.7

9

5.4

** **

AWL (max = 42) TOTAL (max = 150)

35

3.2

24

12.4

129

13.8

99

40.4

Note: * = significant at 0.05; ** = significant at 0.001.

In Tables 6.5 and 6.6, the two groups are looked at individually to establish what progress may have taken place. Table 6.5 shows that a significant increase in proficiency has occurred among the CLIL students for all the frequency levels, except the 2000 level, covered by the VLT between T1 and T2. From Table 6.6, it can be seen that the non-CLIL students’

110

Part 3: English

Table 6.5 VLT results per frequency level for CLIL at T1 and T2 CLIL (N = 57)

T1

T2

2000 level (max = 30)

29

29

ns

3000 level (max = 33)

28

31

**

5000 level (max = 27)

18

22

**

9

12

**

30

35

**

113

129

**

10,000 level (max = 18) AWL (max = 42) TOTAL

Note: * = significant at 0.05; ** = significant at 0.001.

Table 6.6 VLT results per frequency level for non-CLIL at T1 and T2 Non-CLIL (N = 52)

T1

T2

2000 level (max = 30)

27

25

ns

3000 level (max = 33)

25

24

ns

5000 level (max = 27)

17

17

ns ns

10,000 level (max = 18) AWL (max = 42) TOTAL

8

9

26

24

ns

103

99

ns

receptive mastery of L2 English vocabulary has stagnated between T1 and T2 for all frequency levels measured. To get a more complete picture of the differences between T1 and T2, the gender aspect has also been considered in the analyses. In Table 6.7, the results for the CLIL group are shown per gender, and in Table 6.8 those for the non-CLIL group. Table 6.7 shows that there is a gender difference at T1, with CLIL males performing better than female CLIL students, both for the total result (p < 0.001), and for the respective frequency levels: the 2000 level Table 6.7 Results for the CLIL group divided by gender CLIL females

CLIL males

T1 (N = 87) SD

T2 (N = 44) SD

T1 (N = 31) SD

T2 (N = 13) SD

2000 level (max = 30)

28

2.9

29

1.6

30

2.8

29

1.0

3000 level (max = 33)

27

4.1

30

2.4

30

5.6

32

1.8

5000 level (max = 27)

16

6.0

21

4.6

22

5.4

25

2.2

8

3.3

11

3.9

12

5.1

16

4.7

7.0

35

3.1

33

5.5

37

3.0

22.4 140

11.0

10,000 level (max = 18) AWL (max = 42) Total (max = 150)

28 107

21.0 126

13.0 126

English Receptive Vocabulary

111

Table 6.8 Results for the non-CLIL group divided by gender Non-CLIL females T1 (N = 42) SD

Non-CLIL males T2 (N = 23) SD

T1 (N = 34) SD

T2 (N = 29) SD

2000 level (max = 30)

25

4.7 25

8.6

27

4.5

25

8.3

3000 level (max = 33)

23

5.3 24

9.9

26

6.4

25

10.2

5000 level (max = 27)

13

5.7 15

7.4

19

5.5

19

8.3

10,000 level (max = 18)

7

4.1

9

3.9

10

5.9

AWL (max = 42)

23

7

3.1

6.8 21

13.1

28

7.8

26

11.4

Total (max = 150)

91

22.8 92

25.1 106

41.2

38.8 108

(p = 0.033), the 3000 level (p = 0.009), the 5000 level (p < 0.001), the 10,000 level (p < 0.001) and the AWL segment (p < 0.001). In other words, the gender difference is statistically significant throughout the VLT. The results for T2 still show an advantage for the males in the total score (p = 0.001), but some changes have occurred at the various frequency levels: the 2000 level (p = 0.387), the 3000 level (p = 0.002), the 5000 level (p = 0.003), the 10,000 level (p = 0.001) and the AWL segment (p = 0.045). The females have more or less eliminated the gender gap at the 2000 level, where the difference is no longer statistically significant. For the other levels, the difference remains significant, but for the AWL segment the p-value is getting close to being non-significant. Table 6.7 also illustrates the drop in standard deviation, both for the females (SD at T1 = 21.0 and at T2 = 13.0) and the males (SD at T1 = 22.4 and at T2 = 11.0), indicating that the spread of vocabulary proficiency found at T1 has diminished at T2. Table 6.8 illustrates that among the non-CLIL students, too, there is a significant gender difference (p = 0.003) at T1. Some variation is found when looking more closely at the figures for the respective frequency levels: the 2000 level (p = 0.235), the 3000 level (p = 0.020), the 5000 level (p < 0.001), the 10,000 level (p = 0.058) and the AWL segment (p = 0.006). At T2, however, the gender difference in the total results has been eliminated among the non-CLIL students (p = 0.217), as well as for all the different frequency levels. As regards the standard deviation among the non-CLIL students, Table 6.8 indicates an increase for the females (SD at T1 = 22.8 and at T2 = 38.8) as well as for the males (SD at T1 = 25.1 and at T2 = 41.2). We now proceed to some further discussion of the results on the VLT accounted for in this section. 7 Discussion

To recap, the instrument used in this study was the Vocabulary Levels Test (VLT; Nation, 2001). It was administered twice: at the onset of senior

112

Part 3: English

high school, coinciding with the start of CLIL for the CLIL students, and almost three years later during the final term of high school. Consequently, the fi rst test round (T1) provided baseline data with which the data obtained in the second test round (T2) were compared. The results indicate that there is no doubt that CLIL attracts students whose receptive proficiency in English is significantly better than that of those opting for the non-CLIL strand. This correlates to a high degree with the results obtained for productive proficiency in the CLISS project (see Chapter 7), in line with previous studies in Sweden (Sylvén, 2004; Washburn, 1997) and elsewhere (Admiraal et al., 2006; Rumlich, 2013). There is nothing sensational about these results given the Swedish context, where CLIL is optional. What is important to note, however, is the difference in language proficiency levels between CLIL and non-CLIL students at the start of the program. Unless these pre-CLIL variations are taken into account, the results obtained will risk presenting an overly positive picture of CLIL. Thus, it is imperative that longitudinal studies, such as those in the CLISS project, are carried out, establishing students’ proficiency levels before they embark on CLIL. It is especially interesting to note that while the CLIL students make significant progress at all frequency levels covered by the VLT, the nonCLIL students appear to make little progress. The CLIL students make statistically significant progress throughout the VLT between T1 and T2, except for the 2000 level, which comprises the most common English words. The reason for the non-significant progress here is that they score close to the maximum already at T1, and a ceiling effect may be at work. The male students in the CLIL group do not progress significantly at the 3000 level either (p = 0.138), but they improve their score from 31 at T1 to 32 at T2, out of a possible maximum of 33. The female CLIL students, by contrast, improve their results significantly also at the 3000 level. These fi ndings clearly show that the receptive proficiency of English vocabulary increases during the three years of CLIL for these students, in accordance with fi ndings in other studies, e.g. Merikivi and Pietilä (2014). One caveat to be kept in mind, however, involves the good results already at T1 for the CLIL students. As is well known, the more words one knows, the easier it is to learn new ones (e.g. Nation & Waring, 1997), so this may be another effect of the high level of proficiency these students had from the start. Another complication is that, in these analyses, the students’ amount of extramural English (see Chapter 9) has not been accounted for. As shown in Sylvén (2004), CLIL students’ progress was to a very large extent linked to the time they spent on activities in English outside of school, rather than to the input of and exposure to English they received through CLIL in school. Finally, the standard deviation values shown in the non-CLIL group indicate that there are also non-CLIL students increasing their receptive vocabulary during the period of the study.

English Receptive Vocabulary

113

Another result worthy of consideration is the overall decrease in the values for standard deviation in the CLIL group. From being, overall, 21 for the CLIL females at T1, the standard deviation decreased to 13 at T2, the corresponding figures for the males being 22.4 at T1 and 11 at T2. This shows that the group as such became more homogeneous as regards their levels of receptive vocabulary knowledge, in turn an indication that CLIL, in the present study, has worked favorably in evening out proficiency differences, bringing the lower-scoring students up to a level almost equal to those who score the best. A fi nal point worth commenting on in connection with the CLIL results is that both females and males improve their scores in the AWL segment of the test. As academic vocabulary is of special interest in the CLISS project, it is relevant that CLIL students indeed increase their receptive command of such vocabulary. As has been pointed out, the participants in the project were enrolled in preparatory programs for higher education. It thus seems as though CLIL prepares students at least for some aspects of studies in higher education, such as reading, where receptive vocabulary knowledge is essential (see also Chapter 8). As regards the students in the non-CLIL group, their results are notable as they do not make any significant progress in their results on the VLT. In fact, while the females improve their results slightly from 91 at T1 to 92 at T2, the males’ results drop from a total of 108 at T1 to 106 at T2. Thus, the seemingly positive elimination of gender differences from T1 to T2 is due both to the females’ improved results and to the males’ decrease in scores. Interestingly, these results resemble in part those obtained in Sylvén (2004), where three test rounds aimed at vocabulary learning were administered over a two-year period and the non-CLIL students at one of the three schools involved scored lower in the final test round than in the second. There may be several explanations for this somewhat unexpected outcome. First of all, it may be that, at group level, the non-CLIL students actually do not increase their proficiency level of English vocabulary during their three years in senior high school. After all, they only take English as a separate subject. In reality, some of them do not even do that in their fi nal year, as the last year’s English course is optional (www. skolverket.se), which means that not all students undertake this advanced course (see Chapter 1). Another possible explanation is that the level of motivation among the non-CLIL students was in general lower for performing well on the tests administered here. A third likely explanation, closely linked to the one just mentioned, may be that the non-CLIL students focused more on actual school work than spending time on tests for the CLISS project. From another angle, it is abundantly clear that the male students outperform their female peers, in the CLIL as well as in the non-CLIL group. These fi ndings are in line with those found in other studies in the Swedish context (Herriman, 1997; Sylvén, 2004; Sylvén & Sundqvist, 2014;

114

Part 3: English

Washburn, 1997), where male students consistently score better than females with regard to lexical proficiency. Several hypotheses concerning this have been put forward, the most plausible being the amount and types of extramural English that male students in Sweden are exposed to as compared to female ones (see also Chapter 9). While virtually everybody in the younger generations comes into constant contact with English in their daily lives, males in general seem to benefit more – at least linguistically – from such contacts; girls seem to prefer social networks, whereas boys tend to be more involved in activities like interactive computer gaming (Sundqvist & Sylvén, 2012b; Sylvén & Sundqvist, 2012). Any contact with English outside of school is indeed conducive to language learning, but interactive computer games seem to offer close to ideal circumstances, not least as regards certain kinds of vocabulary, e.g. academic words (Gee, 2007; Peterson, 2012; Reinders, 2012; Sundqvist & Sylvén, 2016). However, even though the non-CLIL males are still in the lead over their female peers at T2, the gender difference is no longer significant, suggesting that the female students have benefited more from the English input in school, and that the male student’s extramural exposure at this level is no longer sufficient to increase their range of vocabulary. A fi nal note in regard to the gender differences in the results reported here concerns the fact that the mean result for CLIL females at T2 exactly coincides with that of the CLIL males at T1. 8 Concluding Remarks

CLIL is intended as a way of increasing L2/FL learners’ proficiency in the target language. In this chapter we have seen that students opting for CLIL are mostly those who already have a relatively high proficiency in English vocabulary. Clearly, students who are neither interested nor proficient in English are hardly likely to select programs where English is used as the medium of instruction. However, the CLIL students’ level of proficiency invites more challenging teaching and learning than presently seems to be the case. In other words, these already very proficient students would probably benefit from even more demanding L2/FL English input and use. As indicated, a certain ceiling effect may be noticed in the overall results for the CLIL students and, as always in a limited study such as the present one, caution is needed when interpreting the findings. Nevertheless, even though CLIL students progress in their receptive vocabulary proficiency over the course of the three years in senior high school, it seems clear that there is room for improvement, both for CLIL and non-CLIL students, not least concerning the more difficult frequency levels and academic words. As other studies indicate, Swedish learners of L2/FL English benefit greatly from extramural exposure (Olsson, 2016; Sundqvist & Sylvén, 2012a). Therefore, language education in school might well focus more on such relatively advanced, academically-oriented vocabulary.

English Receptive Vocabulary

115

Thus, the most important pedagogical conclusion to be drawn from the results presented in this chapter is that the teaching in non-language subjects where English is used as the medium of instruction, as well as in English as a subject of its own, should focus a great deal more on academic vocabulary proficiency. References Admiraal, W., Westhoff, G. and de Bot, K. (2006) Evaluation of bilingual secondary education in the Netherlands: Students’ language proficiency in English. Educational Research and Evaluation 12 (1), 75–93. Beglar, D. (2010) A Rasch-based validation of the Vocabulary Size Test. Language Testing 27 (1), 101–118. Beglar, D. and Hunt, A. (1999) Revising and validating the 2000 Word Level and University Word Level Vocabulary Tests. Language Testing 16 (2), 131–162. Bogaards, P. (2001) Lexical units and the learning of foreign language vocabulary. Studies in Second Language Acquisition 23, 321–343. Carter, R. (1987) Vocabulary – Applied Linguistic Perspectives. London: Allen & Unwin. Cobb, T. (1997) Is there any measurable learning from hands-on concordancing? System 25, 301–315. Coxhead, A. (2000) A new academic word list. TESOL 34 (2), 213–238. Cruse, A. (1986) Lexical Semantics. Cambridge: Cambridge University Press. Dentler, S. (2002) Case 17: Mixed ability – secondary. In D. Marsh (ed.) CLIL/EMILE – The European Dimension (pp. 167–171). Jyväskylä: University of Jyväskylä. Gee, J.P. (2007) What Video Games Have to Teach us about Learning and Literacy (Revised and updated edn). New York: Palgrave Macmillan. Henriksen, B. (1999) Three dimensions of vocabulary development. Studies in Second Language Acquisition 21 (2), 303–317. Herriman, J. (1997) Vocabulary and Reading Comprehension. Gothenburg: University of Gothenburg. Jiménez Catalán, R.M. (2010) Gender tendencies in EFL across vocabulary tests. In R.M. Jiménez Catalán (ed.) Gender Perspectives on Vocabulary in Foreign and Second Languages (pp. 117–138). Basingstoke: Palgrave Macmillan. Jiménez Catalán, R.M. and Ruiz de Zarobe, Y. (2009) The receptive vocabulary of EFL learners in two instructional contexts: CLIL versus non-CLIL instruction. In Y. Ruiz de Zarobe and R.M. Jimenéz Catalán (eds) Content and Language Integrated Learning: Evidence from Research in Europe (pp. 81–92). Bristol: Multilingual Matters. Laufer, B. (1998) The development of passive and active vocabulary in a second language: Same or different? Applied Linguistics 19 (2), 255–271. Meara, P. (1996) The dimensions of lexical competence. In G. Brown, K. Malmkjaer and J. Williams (eds) Performance and Competence in Second Language Acquisition (pp. 35–53). Cambridge: Cambridge University Press. Meara, P. and Jones, G. (1990) The Eurocentres 10K Vocabulary Size Test. Zurich: Eurocentres. Merikivi, R. and Pietilä, P. (2014) Vocabulary in CLIL and mainstream education. Journal of Language Teaching and Research 5 (3), 487–497. Mochida, A. and Harrington, M. (2006) The Yes/No test as a measure of receptive vocabulary knowledge. Language Testing 23 (1), 73–98. Nation, P. (2001) Learning Vocabulary in Another Language. Cambridge: Cambridge University Press. Nation, P. and Beglar, D. (2007) A vocabulary size test. The Language Teacher 31 (7), 9–13.

116

Part 3: English

Nation, P. and Waring, R. (1997) Vocabulary size, text coverage, and word lists. In N. Schmitt and M. McCarthy (eds) Vocabulary: Description, Acquisition, Pedagogy. Cambridge: Cambridge University Press. Olsson, E. (2016) On the impact of extramural English and CLIL on productive vocabulary. PhD thesis, University of Gothenburg. See https://gupea.ub.gu.se/ handle/2077/41359. Peterson, M. (2012) Learner interaction in a massively multiplayer online role playing game (MMORPG): A sociocultural discourse analysis. ReCALL 24 (3), 3561–3381; doi:10.1017/S0958344012000195. Read, J. (2000) Assessing Vocabulary. Cambridge: Cambridge University Press. Reinders, H. (ed.) (2012) Digital Games in Language Learning and Teaching. Basingstoke: Palgrave Macmillan. Rumlich, D. (2013) Students’ general English proficiency prior to CLIL. In S. Breidbach and B. Viebrock (eds) Content and Language Integrated Learning (CLIL) in Europe: Research Perspectives on Policy and Practice (pp. 181–202). Frankfurt: Peter Lang. Schmitt, N. (2000) Vocabulary in Language Teaching. Cambridge: Cambridge University Press. Schmitt, N., Schmitt, D. and Clapham, C. (2001) Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing 18 (1), 55–88. Sundqvist, P. and Sylvén, L.K. (2012a) Computer-assisted L2 English language-related activities among Swedish 10-year-olds. Paper presented at EuroCALL2012, University of Gothenburg. Sundqvist, P. and Sylvén, L.K. (2012b) World of VocCraft: Computer games and Swedish learners’ L2 vocabulary. In H. Reinders (ed.) Digital Games in Language Learning and Teaching (pp. 189–208). Basingstoke: Palgrave Macmillan. Sundqvist, P. and Sylvén, L.K. (2016) Extramural English in Teaching and Learning: From Theory and Research to Practice. London: Palgrave Macmillan. Sylvén, L.K. (2004) Teaching in English or English teaching? On the effects of content and language integrated learning on Swedish learners’ incidental vocabulary acquisition. PhD thesis, University of Gothenburg. Sylvén, L.K. (2013) CLIL in Sweden – why does it not work? A metaperspective on CLIL across contexts in Europe. International Journal of Bilingual Education and Bilingualism 16 (3), 301–320. Sylvén, L.K. and Ohlander, S. (2014) The CLISS project: Receptive vocabulary in CLIL versus non-CLIL groups. Moderna Språk 108 (2), 80–114; doi:http://ojs.ub.gu.se/ojs/ index.php/modernasprak/article/view/2789/2614. Sylvén, L.K. and Sundqvist, P. (2012) Gaming as extramural English L2 learning and L2 proficiency among young learners. ReCALL 24 (3), 302–321; doi:10.1017/ S095834401200016X. Sylvén, L.K. and Sundqvist, P. (2014) Needed: A vocabulary test for young learners of English. Paper presented at Early Language Learning: Theory and Practice, Umeå. Terlevic Johansson, K. (2011) Erfolgreiches Deutschlernen durch CLIL? PhD thesis, University of Gothenburg. See http://hdl.handle.net/2077/26560. Washburn, L. (1997) English immersion in Sweden: A case study of Röllingby high school 1987–1989. PhD thesis, Stockholm University.

7 English Productive Proficiency Eva Olsson and Liss Kerstin Sylvén

1 Introduction

The focus of this chapter is students’ English writing proficiency and, in particular, their progress in productive English vocabulary. By productive vocabulary, we mean the words students use in their own writing. Even though students do not necessarily use English to any great extent in writing outside the English classroom, CLIL students can be expected to encounter more English vocabulary, including academic vocabulary, than non-CLIL students, as English is used as the language of instruction in several school subjects. The ability to write texts in English is vital for most students aiming at higher education. Expectations are high for writing proficiency in English at tertiary level, even in countries such as Sweden where English is not normally the first language (L1) (Airey, 2009; cf. Nunan, 2003). Therefore, it is highly relevant to study the possible effects of CLIL on the development of productive second/foreign language (L2/FL) English proficiency. The objective of the study described in this chapter is to investigate whether CLIL education seems to have any impact on students’ English writing proficiency, with particular regard to their development of vocabulary. The chapter starts with a brief background of CLIL in connection to the development of writing proficiency and productive vocabulary, followed by an overview of the literature published on this topic. It then describes the present study into productive L2/FL English use in the CLISS project, and presents results based on analyses of writing assignments on topics related to the natural sciences and social sciences regarding general features, i.e. text length, word length and variation of vocabulary, as well as the development of general and academic productive vocabulary. The chapter ends with a discussion of the findings and some concluding remarks. The specific research questions addressed in this chapter are as follows: •

What differences are there, if any, between texts written by CLIL and non-CLIL students with regard to some general features, i.e. text length, word length and variation of vocabulary? 117

118

•

Part 3: English

What differences are there, if any, between CLIL and non-CLIL students in the development of general as well as academic productive vocabulary?

2 Background

Since one of the aims of CLIL is to increase students’ exposure to and contact with the target language, thus enhancing the students’ L2/FL acquisition and proficiency more than in traditional language instruction, it is of great relevance to investigate the effect of CLIL on the development of the L2/FL, in this case, English. A main argument for CLIL is that the content of lessons might be more substantial than in traditional EFL instruction where trivial topics are sometimes used (Genesee & LindholmLeary, 2013). As non-language subjects such as physics and history are taught in the L2/FL, students encounter the L2/FL in subject-related, often linguistically challenging contexts with the potential to enhance their L2/FL proficiency. Another important argument for CLIL is the time aspect; in traditional EFL instruction students often practice the L2/FL during a limited number of lessons per week, whereas in CLIL the L2/FL is used during a greater number of lessons a week, often even every day. As the focus of the CLISS project is on students at upper secondary level following preparatory programmes for higher education, students’ proficiency in using English productively is of obvious interest. Do CLIL students develop such proficiency more than students who follow regular non-CLIL education? And do they develop their proficiency in using academic English more than other students? To investigate these questions, four writing assignments covering topics related to the natural sciences and social sciences were completed by the students over a three-year period. Since words provide the building blocks of texts, analyses over time of the vocabulary in students’ writing may indicate how certain aspects of their writing proficiency develop – hence, the major focus of this study on vocabulary (cf. Hinkel, 2011). There are, of course, other aspects of writing proficiency that may be of great importance for writing proficiency as well, such as knowledge of grammar and text structure, which are not covered in this study, since a single study cannot cover all aspects. Here, the focus is, as already mentioned, on vocabulary. In research on vocabulary, a distinction is often made between receptive and productive vocabulary. Receptive knowledge of vocabulary (as pointed out in Chapter 6) implies that vocabulary items are understood when heard or read, whereas productive knowledge is the proficiency to use the words in speech and writing (e.g. Waring, 1997). Receptive vocabulary is larger and often precedes proficiency to use vocabulary productively (Laufer, 1998); it is cognitively less demanding to understand a word than to use it in language production. However, it is important to

English Productive Proficiency

119

acknowledge the fact that there is no strict dividing line between receptive and productive vocabulary. Rather, vocabulary knowledge should be seen as a continuum, where receptive proficiency is found at one end and productive at the other (see also Schmitt, 2010; Chapter 6, this volume). 3 Literature Review 3.1 International overview

Both CLIL and non-CLIL education can, of course, provide good opportunities for vocabulary learning and development of productive proficiency. In particular, this is possible if formal instruction is combined with authentic input and interaction (cf. Laufer, 1998). Thus, CLIL should offer an advantage in this respect, as it is built on the idea that a target language is used to teach and learn ‘real’, i.e. authentic, subject content (cf. Merikivi & Pietilä, 2014). However, research findings have indicated that there is often a limited focus on language in CLIL classrooms; instead, attention seems mostly to be paid to subject content (Genesee & LindholmLeary, 2013; Lyster, 2007). Nevertheless, international research on the effects of CLIL has generally shown that L2/FL proficiency appears to develop more among CLIL students than among students following traditional language education. CLIL students’ receptive and productive L2/FL vocabulary is often larger, and includes low-frequency words to a greater extent (Dalton-Puffer, 2011; Merikivi & Pietilä, 2014). In their study among students in Grades 6 and 9 in Finland, Merikivi and Pietilä (2014) found that the difference between receptive and productive scores was larger in frequency bands beyond the 3000 most frequent words in both CLIL and non-CLIL groups (as measured by the Vocabulary Levels Test; see Chapter 6). They conclude that the 3000 most frequent words seem to be available for production more easily than more infrequent words. CLIL students’ vocabulary use also tends to be more varied and accurate. For example, Ruiz de Zarobe (2008, 2010) found that CLIL students in upper secondary school in Spain scored more highly than non-CLIL students with regard to choice and use of English vocabulary in speech and writing. Moreover, the results indicated a more positive development among CLIL students over time, and also with regard to other aspects of writing proficiency, e.g. in the way texts were structured and in the use of grammar. In a similar vein, Jexenfl icker and DaltonPuffer (2010) found that CLIL students’ vocabulary range and accuracy were greater than those of non-CLIL students in a study among 16-yearold students in Austria. Further, Lo and Murphy (2010) reported that receptive vocabulary knowledge as well as productive vocabulary use in writing increased significantly more among English immersion students (aged 11–15) than among those who studied English in a traditional language class in Hong Kong.

120

Part 3: English

However, it has been pointed out that some studies have not included baseline data, i.e. information about the students’ level of proficiency when starting CLIL education. In such cases it is difficult to draw any conclusions about the effects of CLIL. Bruton (2011), for instance, is critical of some CLIL studies. He argues that the positive results for CLIL students’ reported achievements are not surprising as, fi rst, the students are often in one way or another selected for CLIL programmes, and secondly, CLIL students are exposed to the target language to a greater extent than students in non-CLIL strands. Thus, higher levels of proficiency can be attributed to the already superior target language knowledge of CLIL students and/or to the fact that they have greater access to the target language than non-CLIL students, rather than to CLIL per se. Moreover, some of the positive results just mentioned have been contested by fi ndings from other studies. In a longitudinal study of CLIL and non-CLIL students at upper secondary level in the Netherlands, Admiraal et al. (2006) found that initial differences in English receptive vocabulary knowledge remained at the same level rather than increasing; CLIL students scored higher from the start but the gap between CLIL and nonCLIL groups did not widen. Speculatively, then, it is possible to assume that similar results would have been obtained if productive proficiency had been in focus. 3.2 Previous research in the Swedish context

In a Swedish context, research has not convincingly shown that CLIL increases the progress of L2/FL proficiency (Hyltenstam, 2004; Sylvén, 2013). Even though Sylvén (2004) found that Swedish CLIL students scored higher on English vocabulary tests than non-CLIL students, certain background factors, especially the parents’ level of education and the students’ use of extramural English (see Chapter 9, this volume), also seemed to influence vocabulary knowledge; hence, no conclusions about the effect of CLIL could be drawn. Sylvén (2004) also found that male students had a larger vocabulary than female students. Of special interest here are the results in the CLISS project from the study based on the Vocabulary Levels Test (VLT; Nation, 2001; see also Sylvén & Ohlander, 2014; Chapter 6, this volume). It was found that the CLIL students already had a significantly larger receptive vocabulary in English than the non-CLIL students at the start of their CLIL education. This was also true for the section covering academic vocabulary. As was seen in Chapter 6, the CLIL students also outperformed their non-CLIL peers at the end of their three-year CLIL programme as regards receptive vocabulary. However, the development in both the non-CLIL and CLIL groups was very similar, indicating that the CLIL students did not make significantly more progress than the non-CLIL students.

English Productive Proficiency

121

3.3 Section summary

Thus, to sum up the research fi ndings with regard to the development of L2/FL proficiency in CLIL education, CLIL students tend to reach higher proficiency levels than non-CLIL students. However, the absence of pre-tests in many studies should be noted. Further, in research on CLIL in Sweden, the effects of CLIL have not been conclusive since background factors, e.g. the presence and use of English in students’ spare time, seem to be as important for L2/FL proficiency as CLIL itself (Olsson & Sylvén, 2015; Sylvén, 2013). 4 Method and Materials 4.1 Writing assignments

In the CLISS project, four writing assignments in English were given to the students during their three years in upper secondary school (see Chapter 1, this volume). The topics of the assignments are presented in Table 7.1. The fi rst assignment was given at the beginning of their fi rst year at upper secondary level, i.e. coinciding with the start of CLIL, and thus provided baseline data. The second and third assignments were both administered in the second year and the last assignment in the third year. All four assignments were designed based on the curriculum for upper secondary school with regard to topics and text types (cf. Swedish National Agency for Education, 2013). Accompanying each assignment were facts and figures offering some background information in order to enable those who knew very little about the specific topic to fi nd something to write about. The assignments were administered in full classes during regular school hours and, whenever possible, in the presence of a representative from the CLISS project. The students used computers for writing the essays, with spelling and grammar checkers turned off. All teachers were offered the opportunity to use the texts for their own purposes if they so wished, and in several classes the essays were therefore sent both to the CLISS project and to a teacher. The texts were then anonymized and each was given a number representing the individual student.

Table 7.1 The topics of the CLISS writing assignments Assignment number

Topic

Text type

1

For or against nuclear power

Argumentative

2

Matters of gender and equality

Expository

3

Ways to political change – violence or non-violence

Argumentative

4

Biodiversity for a sustainable society

Expository

122

Part 3: English

4.2 Text analyses

To investigate what differences there are, if any, in the development of writing proficiency, with regard to productive vocabulary in particular, between CLIL and non-CLIL students, different analyses were conducted of the students’ texts, based on the four assignments. 4.2.1 General features

Some general features known to be relevant to the general quality of texts were analyzed in the students’ texts: text length, word length and variation of vocabulary (cf. Grant & Ginther, 2000). More proficient writers tend to write longer texts, to use longer and thus more complex words and to vary their vocabulary to a greater extent than struggling writers. This means that there is often a certain correlation between the scores for these features and holistic judgements of the quality of texts. Wordsmith Tools 5 was used for these analyses. Text length is indicated in number of words, word length in number of characters, and variation of vocabulary in a standardized type/token ratio, where the number of different words (types) is divided by the total number of words (tokens) in a text. Statistical comparisons were made between texts written by CLIL and non-CLIL students, using SPSS Version 21. 4.2.2 General vocabulary

In order to investigate the students’ skills in the use of general vocabulary, texts written for Assignments 1 and 4 were analyzed using the New General Service List (NGSL; Browne, 2014) as a benchmark. In 1953, Michael West compiled the General Service List (GSL; West, 1953). The GSL consisted of about 2000 high-frequency words deemed especially important for learners of English to know. Over the years, the GSL has been used for both learning and research purposes. However, word lists grow old and, in time for the 60th anniversary of the GSL, Browne and colleagues released an updated version, the NGSL (Browne, 2013). The NGSL has since its fi rst launch been updated continuously, and it now consists of some 2800 words drawn from the Cambridge English Corpus, a corpus of approximately 2 billion words. Thus, the updated version is based on substantially more extensive material than was the case for the original GSL. The words in the NGSL are chosen in order to give ‘the highest possible coverage of English texts with the fewest words possible’ (Browne, 2014: 2). The NGSL is frequency based and consists of three levels: in the fi rst level the 1000 most frequent words are found; in the second level are the following 1000 words; and in the third are the 800 least frequent words. In other words, the lower the NGSL level, the easier the words, and vice versa so that the most difficult words are found in Level 3. The list is based on lexemes, with a headword and its inflected forms (e.g. the headword know includes the inflected forms knows, knew,

English Productive Proficiency

123

knowing, known), where variations in spelling are included in the same lexeme (e.g. realize/realise). Here, the number of types covered by the three frequency bands was noted for each student essay. Hence, a word was only counted once in a text even if it occurred several times. 4.2.3 Academic vocabulary

Turning to the study of students’ use of academic vocabulary, it should be noted that the term ‘academic vocabulary’ is broad and not altogether a clear-cut concept (see also Chapter 12 on academic vocabulary in students’ Swedish texts). Generally, a distinction is made between domain-specific vocabulary and general academic vocabulary (Baumann & Graves, 2010; Nation, 2001). In the present study, only the students’ use of general academic vocabulary is investigated as such vocabulary can be used in many different contexts and not only in highly specific contexts. General academic words are words such as reliable, source and specific – clearly vocabulary that can be used across different domains. To define general academic vocabulary, a corpus-based academic vocabulary list was used: the Academic Vocabulary List (AVL; Gardner & Davies, 2014). In the AVL, the 3000 most commonly occurring lemmas, i.e. individual words and inflected forms, of the academic part of the Corpus of Academic Contemporary American English (COCA; Davies, 2012) are listed. The COCA, a 425-million-word corpus, includes texts from a large variety of contexts, both formal and informal. The academic part consists of more than 120 million words from academic texts such as published research articles. The 3000 words in the AVL are words occurring across domains, i.e. general academic words as opposed to domain specific. Here, the vocabulary covered by the AVL in each of the students’ texts was analyzed using an interface available at http://www.wordandphrase.info/academic/. Comparisons of the proportion of academic words, i.e. the proportion of academic tokens of all tokens, in CLIL and non-CLIL students’ texts, were made. Further, statistical analyses of development over time were conducted. 5 Results

In Table 7.2 the number of students who completed each of the four writing assignments is shown. First, the results of the analyses of some general features of the students’ texts are reported, followed by the results of the analyses of general and academic vocabulary. Comparisons are made between the CLIL and non-CLIL groups. Whenever statistical significance is shown in the tables, ** corresponds to p < 0.001 and * to p < 0.05. 5.1 General features of the students’ texts

In Table 7.3 the length of texts (in number of words) written by the CLIL and non-CLIL students is compared. The table shows that the CLIL

124

Part 3: English

Table 7.2 Number of CLIL and non-CLIL students who completed the four assignments CLIL

Non-CLIL

N

N

Text 1

94

52

Text 2

80

46

Text 3

83

55

Text 4

80

35

Table 7.3 Average text length in number of words in texts by CLIL and non-CLIL students CLIL

Non-CLIL

Mean no. of words

SD

Mean no. of words

SD

t-value

p-value

Text 1

570

207

364

164

6.2