Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings: Educative Issues and Empirical Approaches 1032011092, 9781032011097

Citation preview

Routledge Research in Language Education

ASSESSMENT OF PLURILINGUAL COMPETENCE AND PLURILINGUAL LEARNERS IN EDUCATIONAL SETTINGS EDUCATIVE ISSUES AND EMPIRICAL APPROACHES Edited by Sílvia Melo-Pfeifer and Christian Ollivier

Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings

This book addresses contemporary issues in the assessment of plurilingual competence and plurilingual learners. Offering theoretical and practical lenses, it contributes towards an integrated and holistic assessment of plurilingual competence and plurilingual learners. The book provides both theoretical considerations and empirical approaches around how the specificities of plurilingual learners can be considered when assessing their various competences. It covers topics relating to learners in a variety of plurilingual settings: from the education of adult immigrants, assessment of young refugees and assessment of students in school and university to the assessment of plurilingual competence in foreign language education. Showcasing a wide range of international authors, the book provides cuttingedge research in the domain of multilingual foreign, second and heritage language assessment and assessment of content knowledge of plurilingual students. It bridges the gap between the fields of language policies and practices, research on plurilingual competence and assessment in language education. Providing new insights into a crucial and contentious issue, this volume will be an essential reading for researchers and postgraduate students in the fields of educational language policies, applied linguistics and multilingualism, in particular those involved in the assessment of plurilingual competence. Sílvia Melo-Pfeifer is Professor of Foreign Language Teacher Education, University of Hamburg, Germany. Christian Ollivier is Professor of Linguistics and Language Education, University of Reunion Island, Research group Icare, France.

Routledge Research in Language Education

The Routledge Research in Language Education series provides a platform for established and emerging scholars to present their latest research and discuss key issues in Language Education. This series welcomes books on all areas of language teaching and learning, including but not limited to language education policy and politics, multilingualism, literacy, L1, L2 or foreign language acquisition, curriculum, classroom practice, pedagogy, teaching materials, and language teacher education and development. Books in the series are not limited to the discussion of the teaching and learning of English only. Books in the series include: Sublating Second Language Research and Practices Contribution from the Hegelian Perspective Manfred Man-fat Wu EAL Research for the Classroom Practical and Pedagogical Implications Edited by Gavin Brooks, Jon Clenton and Simon Fraser Team Teachers in Japan Beliefs, Identities and Emotions Edited by Takaaki Hiratsuka Typical and Atypical Language Development in Cultural and Linguistic Diversity Edited by Weifeng Han and Chris Brebner Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings Educative Issues and Empirical Approaches Edited by Sílvia Melo-Pfeifer and Christian Ollivier For more information about the series, please visit www.routledge.com/ Routledge-Research-in-Language-Education/book-series/RRLE

Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings

Educative Issues and Empirical Approaches Edited by Sílvia Melo-Pfeifer and Christian Ollivier

First published 2024 by Routledge 4 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an informa business © 2024 selection and editorial matter, Sílvia Melo-Pfeifer & Christian Ollivier; individual chapters, the contributors The right of Sílvia Melo-Pfeifer & Christian Ollivier to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloging-in-Publication Data Names: Melo-Pfeifer, Sílvia, editor. | Ollivier, Christian, editor. Title: Assessment of plurilingual competence and plurilingual learners in educational settings : educative issues and empirical approaches / edited by Sílvia Melo-Pfeifer & Christian Ollivier. Description: Abingdon, Oxon ; New York, NY : Routledge, 2024. | Series: Routledge research in language education | Includes bibliographical references and index. | Identifiers: LCCN 2023015190 (print) | LCCN 2023015191 (ebook) | ISBN 9781032011097 (hardback) | ISBN 9781032011103 (paperback) | ISBN 9781003177197 (ebook) Subjects: LCSH: Multilingual education--Evaluation. | Heritage language speakers--Education. | Educational tests and measurements. | Language and education. Classification: LCC LC3719 .A77 2024 (print) | LCC LC3719 (ebook) | DDC 370.117/5--dc23/eng/20230524 LC record available at https://lccn.loc.gov/2023015190 LC ebook record available at https://lccn.loc.gov/2023015191 ISBN: 978-1-032-01109-7 (hbk) ISBN: 978-1-032-01110-3 (pbk) ISBN: 978-1-003-17719-7 (ebk) DOI: 10.4324/9781003177197 Typeset in Galliard by KnowledgeWorks Global Ltd.

Contents

List of contributors Foreword Introduction: On the unbearable lightness of monolingual assessment practices in education

viii xi

1

SÍLVIA MELO-PFEIFER AND CHRISTIAN OLLIVIER

PART I

Considering the specificities of plurilinguals in assessment: Theoretical framework, policy and educative issues29 1 Assessing multilinguals: Critical “wh-” questions about testing multiple language speakers in (non)educational contexts

31

GESSICA DE ANGELIS AND ANAT STAVANS

2 Latinx bilingual students’ translanguaging and assessment: A unitary approach

48

LAURA ASCENZI-MORENO, OFELIA GARCÍA, AND ALEXIS A. LÓPEZ

3 Examining plurilingual repertoires: A focus on policy, practice, and assessment in the Australian context

62

JACQUELINE D’WARTE AND YVETTE SLAUGHTER

4 Evaluating the effectiveness of digital tools for the assessment of young plurilingual students EMMANUELLE LE PICHON, MAI NAJI, DANIA WATTAR, AND VIVIAN LEE

76

vi Contents 5 Functional multilingualism in educational assessment

92

FAUVE DE BACKER, WENDELIEN VANTIEGHEM, AND PIET VAN AVERMAET

6 Assessing language competences of multilingual speakers – a brief review of two test approaches: C-tests and TBLA (task-based language assessment)

106

BARBARA HINGER

7 Assessing the first language skills of plurilingual secondary school students: An analysis of students’ and evaluators’ perspectives

116

ISABELLE AUDRAS

8 Developing a multilingual perspective in language assessment: Multilinguality and interculturality

131

ANTHONY J. LIDDICOAT AND ANGELA SCARINO

PART II

Assessing plurilingual competence: Principles and empirical approaches145 9 Assessing the multiple acquisitions from bi-/plurilingual learning – the questions to ask

147

MARISA CAVALLI

10 Crosslinguistic teacher education: How self-assessment and reflection tools can support plurilingual student teachers

163

LUKAS BLEICHENBACHER, ANNA SCHRÖDER-SURA, CHRISTOF CHESINI, MARA DE ZANET, CORNELIA GANTENBEIN, AND ROBERT HILBE

11 Plurilingualism and assessment: Some issues and options

178

ENRICA PICCARDO AND BRIAN NORTH

12 EVAL-IC: An integrated approach to plurilingual competences ELISABETTA BONVINO, FILOMENA CAPUCHO AND MARGARETA STRASSER

194

Contents vii 13 Towards an assessment of intercomprehension competences in coherence with plurilingual approaches

204

MADDALENA DE CARLO AND ANA ISABEL ANDRADE

14 Using the European Language Portfolio for the assessment of plurilingual students in the foreign language classroom 

216

LISA MARIE BRINKMANN AND SÍLVIA MELO-PFEIFER

15 Evaluation and plurilingual competence: Between epistemological consistency and ethical vigilance

230

EMMANUELLE HUVER

Conclusion: Somewhere over the rainbow ... a place for multilingual assessment?

245

SÍLVIA MELO-PFEIFER AND CHRISTIAN OLLIVIER

Index

250

Contributors

Ana Isabel Andrade is Full Professor of didactics at the University of Aveiro (Portugal). Laura Ascenzi-Moreno is Assistant Professor of bilingual education at City University of New York (CUNY) at Brooklyn College (USA). Isabelle Audras is Researcher in foreign language education at the University of Le Mans (France). Lukas Bleichenbacher is Professor at the Institute of Language Teaching, at the St. Gallen University of Teacher Education (Switzerland). Elisabetta Bonvino is Full Professor of educational linguistics in the Department of Languages, Literature and Culture at the Roma Tre University (Italy). Lisa Marie Brinkmann is PhD student and research assistant at the Faculty of Education, at the University of Hamburg (Germany). Filomena Capucho is Professor at the Universidade Católica Portuguesa in Viseu (Portugal) and a researcher at the Centre for Studies in Communication and Culture (CECC) of the same university. Marisa Cavalli is Consultant to the European Centre for Modern Languages of the Council of Europe, in Graz (Austria). Jasone Cenoz is Professor at the University of the Basque Country UPV/ EHU (Spain) and member of the Advisory Board of the Organization of Ibero-American States (OEI). Christof Chesini is Co-director and research associate at the Institute of Language Teaching, at the St. Gallen University of Teacher Education (Switzerland). Gessica De Angelis is Professor at the Department of Teacher Education at the NTNU in Trondheim (Norway). Fauve De Backer is Doctoral candidate at the Centre for Diversity and Learning, Linguistics Department, Ghent University (Belgium).

Contributors ix Maddalena De Carlo is Associate Professor in language teaching education at University of Cassino (Italy). Mara De Zanet is Lecturer and research assistant at the Institute of Language Teaching, at the St. Gallen University of Teacher Education (Switzerland). Jacqueline D’warte is Senior Lecturer in English language and literacy curriculum and pedagogy, in the School of Education at Western Sydney University (Australia). Cornelia Gantenbein is Student assistant at the Institute of Language Teaching at the St. Gallen University of Teacher Education (Switzerland). Ofelia García is Professor Emerita in the PhD programmes in Urban Education and Latin American, Iberian and Latino Cultures at the Graduate Center of the City University of New York (USA). Robert Hilbe is Research assistant at the Institute of Language Teaching at the St. Gallen University of Teacher Education (Switzerland). Barbara Hinger is Full Professor of foreign language education at the University of Graz (Austria). Emmanuelle Huver is Professor at Tours University (France) in the research team Dynadiv – Dynamics and stakes of linguistic and cultural diversity. Vivian Lee is Doctoral student in Educational Leadership of the Ontario Institute for Studies in Education and Policy (Canada). Emmanuelle Le Pichon is Assistant Professor and Head of the Centre de recherches en éducation Franco-Ontarienne at the University of Toronto (Canada). Anthony J. Liddicoat is Professor in the Centre for Applied Linguistics at the University of Warwick (England). Alexis A. López is Research Scientist in the English Language Learning and Assessment Center at Educational Testing Service (ETS) in Princeton, New Jersey (USA). Sílvia Melo-Pfeifer is Full Professor in the field of French and Spanish teacher education at the Faculty of Education at the University of Hamburg (Germany). Mai Naji is Doctor of Education Candidate in Educational Leadership and Policy, Ontario Institute for Studies in Education, University of Toronto (Canada). Brian North is Independent researcher, has been a teacher, project coordinator, course director, development manager and researcher and has collaborated extensively with the Council of Europe (Switzerland).

x Contributors Christian Ollivier is Professor of Linguistics and Language Education at the University of Reunion Island (France). Enrica Piccardo is Professor of Applied Linguistics and Language Education at OISE – University of Toronto (Canada) and the Head of the Centre for Educational research in Languages and Literacies (Canada). Angela Scarino is Associate Professor in Applied Linguistics and the Director of the Research Centre for Languages and Cultures (Australia). Anna Schröder-Sura is Lecturer and teacher trainer in the Didactics of Romance Languages at the University of Rostock (Germany) and St. Gallen University of Teacher Education (Switzerland). Yvette Slaughter is Senior Lecturer in language and literacy education within the Melbourne Graduate School of Education (Australia). Stef Slembrouck is Senior Professor in English linguistics at Ghent University and Director of the University Language Centre (Belgium). Anat Stavans is Associate Professor in applied linguistics in the English Department and former Director of the Research Authority at Beit Berl Academic College (Israel). Margareta Strasser is Researcher at the University of Salzburg and head of the Language Centre of the University of Salzburg (Austria). Piet Van Avermaet is Professor in ‘Language and Diversity’ in the Linguistics Department of Ghent University (Belgium) and Director of the Centre for Diversity & Learning. Wendelien Vantieghem is Research coordinator at the Department of Educational Sciences, Vrije Universiteit Brussel (Belgium). Dania Wattar is Research associate and lecturer at the Ontario Institute for Studies in Education, University of Toronto (Canada).

Foreword

“Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings” is a collection of 15 chapters on different aspects of assessment in plurilingual contexts. The volume focuses mainly on European learning situations where plurilingualism is increasing and assessment of plurilingual competence is a challenge. The mobility of the population, the spread of English as a language of international communication, the use of autochthonous languages in education and the increase of online communication in different languages are some of the factors that have influenced the use of different home and school languages not only in Europe but also in other parts of the world. Plurilingualism or individual multilingualism refers to the ability individuals have to communicate in two or more languages and they can be considered synonyms (see De Angelis & Stavans, this volume). This volume examines assessment of plurilingual/multilingual competence and it is really important because it takes a holistic view of the languages used and taught in school settings. Holistic views look at the multilingual repertoire and analyze the learner’s resources not only when learning languages but also in assessment. Shohamy (2011) made an important contribution offering critical insights on the assessment of multilinguals and the language policies related to assessment. A special issue of “Language Education” edited by Gorter (2017) included alternatives to traditional assessment in plurilingual contexts. Other important contributions are the volume by De Angelis (2021) and some articles (see for example Gorter & Cenoz, 2017; López et al., 2016; Schissel et al., 2019). It is worth mentioning as well the Toronto online conference on “Multilingual Approaches to Assessment and Education Symposium” that took place in May 2021. In spite of these important contributions, plurilingual/multilingual assessment has not received enough attention, and this volume is really necessary not only because of monolingual assessments are pervasive but also, as the editors of this volume explain in the introduction, because advances in plurilingual/multilingual assessment will contribute to the development of learners’ competence and also to social justice in education. Assessment is complex, powerful and difficult. Assessment, in general, and plurilingual assessment, in particular, are complex because there are different

xii Foreword types of assessment that have different aims, such as diagnostic, formative, summative, norm-referenced and criterion-reference. There are also many possible oral and written instruments and scoring procedures. In the case of plurilingual assessment, there are different possibilities regarding the languages used as it will be seen later. Assessment is powerful because it can have important implications for learners’ lives regarding their future studies, jobs, residence permits or nationality. National and international assessment are also used for rankings of different educational systems as in the OECD “Programme for International Student Assessment” (PISA) and can have an important influence on teaching (Shohamy, 2001). Plurilingual/multilingual assessment is difficult and faces many challenges as it can be seen in most chapters in this volume. One of the challenges is that there are different epistemological conceptions. This is not surprising because the study of plurilingualism/multilingualism is related to a number of disciplines (education, sociolinguistics, applied linguistics, linguistics, etc.) and the theories and models linked to different schools of thought influence researchers’ conceptions. There is a basic agreement among the authors of this volume about breaking with the monolingual traditional approaches, but there are still multiple possibilities and different positions can be found. Another important challenge in plurilingual/multilingual assessment, highlighted in the chapter by De Angelis and Stavans, is the great diversity regarding multilingual speakers. There are important differences regarding situations such as the home languages, school languages, the role of languages in society, level of competence in the different languages, age and intensity of exposure to the different languages. Another difficulty of adopting a holistic plurilingual/multilingual approach in assessment is related to the school curriculum that has traditionally linked languages to specific subjects and grades. This organization can be regarded as barrier when developing alternative views on assessment. Language assessment is important for all school subjects, including content subjects, because problems with plurilingual competence can seriously affect the learning of content in all school subjects. In many educational contexts, it is often a real challenge to make teachers of content subjects aware of language and of the possibilities of using resources from the learners’ multilingual repertoire. An additional challenge of plurilingual/multilingual assessment is linked to the use of minority languages in education. In some contexts, such as the Basque Country and Wales, students can have Basque or Welsh as languages of instruction, but the position of these languages in society is quite weak and teachers and stakeholders feel the need to protect these languages so that they are not mixed and become still weaker (see Cenoz & Gorter, 2017). A plurilingual/multilingual approach to assessment can take different shapes. Shohamy (2011) proposes a continuum. On one side of the continuum, several languages can be used but are clearly differentiated and, on the other end, “tests are based on the approach in which a mixture of languages

Foreword xiii

Figure 0.1 Continuum of plurilingual assessment.

and open borders among them is a recognized, accepted, and encouraged variety” (Shohamy, 2011, p. 427). There are also intermediate positions. This book provides examples of different positions along the continuum. All chapters in the volume have a plurilingual/multilingual approach, but they show different positions that are linked to their theoretical conceptions and the educational contexts where scholars work. In this volume, we can distinguish at least four positions that can be represented in this continuum (Figure 0.1). Apart from the four positions that have been identified in the chapters of this volume, the continuum could also accommodate other intermediate positions. The ends of the continuum correspond to Shohamy’s (2011) proposal. The left end of the continuum allows for the use of languages and the right end of the continuum is more far-reaching accepting and legitimizing translanguaging in assessment. There are also two intermediate positions: making accommodations and combining competences. The four positions represent different strategies in plurilingual/multilingual assessment. Using several languages. This position refers to assessment strategies that use different languages but keep the languages differentiated. The aim is to acquire competencies in different languages. One example is the European project EVAL-IC presented in Chapter 12 by Elisabetta Bonvino, Filomena Capucho and Margareta Strasser and in Chapter 13 by Maddalena De Carlo and Ana Isabel Andrade. The project aims at the definition and assessment of intercomprehension in five Romance languages: Spanish, French, Italian, Portuguese and Romanian. Learners carry out tasks that require oral and written intercomprehension of these languages. The strategies used to complete the tasks are in a way related to the original concept of translanguaging developed in Wales that proposed the alternation of languages for the input and the output (Cenoz & Gorter, 2021). Making accommodations. The idea of making accommodations is often reflected in providing support by giving students the possibility to read the instructions or questions and complete the tasks in tests not only in

xiv Foreword the main language of instruction but also in other languages. It goes a step further than the former position because the languages can be next to each other in the same oral or written text. Shohamy (2011) explained that in the context of assessing mathematics in Israel Russian immigrants obtained better results when they were given bilingual Hebrew-Russian tests rather than Hebrew monolingual tests. Other studies have confirmed these results and have also found that students find adjustments useful (De Backer et al., 2019; Heugh et al., 2017; López et al., 2019). In this volume, Emmanuelle Le Pichon shows how digital tools can be used when making accommodations. Laura Ascenzi-Moreno, Ofelia García and Alexis López also refer to accommodations when teachers help students by translating, using visual supports and dictionaries. Fauve De Backer and her colleagues report in this volume that accommodations do not necessarily work in all cases. Their study was carried out in Flanders, and students who had languages other than Dutch as home languages did not benefit from read-aloud accommodation and bilingual tests in science achievement. The authors consider that this could be due to their limited experience with accommodations but also to the fact that accommodations have a monolingual point of departure. Combining competences. This position is less developed than making accommodation, and it has gone in different directions. The idea is that plurilingual/multilingual assessment results from the combination of assessments in different languages. Unlike making accommodations, it is mainly focused on languages rather than the comprehension of content. An important contribution in this volume is the discussion of the European Language Portfolio (ELP) by Lisa Marie Brinkmann and Sílvia Melo-Pfeifer who highlight that the ELP addresses the whole plurilingual repertoire. A different approach to combine competences is to combine scores of different languages so as to add to a bilingual and multilingual total score. Cenoz et al. (2013) examined learners’ competences by creating two indexes, one for the combined measures of Basque and Spanish (bilingual index) and one for these two languages plus English (multilingual index). The global scores resulting from the combination of two or three languages provide a different perspective from the individual scores in each of the languages because they indicate the degree of bilingualism or multilingualism and not competencies in specific languages. The Common European Framework of Reference for Languages (Council of Europe, 2001) has a clear plurilingual/multilingual approach. It is widely used both to establish specific aims for different levels of language teaching and for assessing those levels. However, it usually focuses on only one language at the time. The Companion Volume (Council of Europe, 2020) has developed descriptors that focus on the use of resources from different languages (see chapter by Piccardo & North, this volume). This is an important step regarding a plurilingual/multilingual approach. There is an attempt to link the descriptors “Building on plurilingual repertoire” and “Plurilingual comprehension” to

Foreword xv the CEFR levels (A1 to C2), but the descriptors will need further development so as to achieve a better fit. Accepting translanguaging. This far-reaching approach to assessment is not common in education (see chapter by Anthony J. Liddicoat & Angela Scarino, this volume). Laura Ascenzi-Moreno, Ofelia García and Alexis López explain in this volume that, when translanguaging is legitimized in education, students are free to use resources from their full linguistic repertoire. Scoring, referred to as “conceptual scoring”, would be based on the responses given by the student without regarding the language that has been used. This is a farreaching approach, but there are still many obstacles to accept translanguaging, particularly in the case of official examinations. Only some of the chapters in “Assessment of Plurilingual Competence and Plurilingual Learners in Educational Settings” have been mentioned in this Foreword, but all the chapters offer important insights in plurilingual/multilingual assessment. The chapters in this book provide theoretical reflections, research data and teaching strategies that break new ground in assessment. The chapters shed light on different aspects of assessment, and the authors are aware of the difficulties associated with the challenge of moving away from traditional approaches into plurilingual/multilingual assessment. This volume is certainly an impressive achievement that will be of great interest for all those interested in assessment and education. Jasone Cenoz References Cenoz, J., Arocena, E., & Gorter, D. (2013). Multilingual students and their writing skills in Basque, Spanish and English. In V. Mueller Gathercole (Ed.), Bilingual assessment: Issues (pp 186–205). Bristol: Multilingual Matters. Cenoz, J., & Gorter, D. (2017). Sustainable translanguaging and minority languages: Threat or opportunity? Journal of Multilingual and Multicultural Development, 38, 901–912. Cenoz, J., & Gorter, D. (2021). Pedagogical translanguaging. Cambridge: Cambridge University Press. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press. http:// www.coe.int/en/web/common-european-framework-reference-languages. Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume. Strasbourg, France: Council of Europe. https://rm.coe.int/common-european-framework-of-reference-for-languageslearning-teaching/16809ea0d4. De Angelis, G. (2021). Multilingual testing and assessment. Bristol: Multilingual Matters. De Backer, F., Slembrouck, S., & Van Avermaet, P. (2019). Assessment accommodations for multilingual learners: pupils’ perceptions of fairness. Journal of Multilingual and Multicultural Development, 40, 833–846. Gorter, D. (2017) Education and multilingualism: Navigating policy and assessment. Language and Education, 31 (3) (special issue), 191–282.

xvi Foreword Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31, 231–248. Heugh, K., Prinsloo, C., Makgamatha, M., Diedericks, G., & Winnaar, L. (2017). Multilingualism(s) and system-wide assessment: A southern perspective. Language and Education, 31, 197–216. López, A., Guzmán-Orth, D., & Turkan, S. (2019). Exploring the use of translanguaging to measure the mathematics knowledge of emergent bilingual students. Translation and Translanguaging in Multilingual Contexts, 5, 143–164. López, A. A., Turkan, S., & Guzmán-Orth, D. (2016). Assessing multilingual competence. In E. Shohamy, I. Or, & S. May (Eds.) Language testing and assessment (pp. 91–102). New York: Springer. Schissel, J. L., Leung, C., & Chalhoub-Deville, M. (2019). The construct of multilingualism in language testing. Language Assessment Quarterly, 16(4–5), 373–378. Shohamy, E. (2001). The power of tests: A critical perspective on the uses of language tests. London: Pearson. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95, 418–429.

Introduction On the unbearable lightness of monolingual assessment practices in education Sílvia Melo-Pfeifer and Christian Ollivier The pervasiveness of the monolingual mindset in assessment Developments in the definition of what a language is (and is not), in the description of plurilingual subjects and functioning of plurilingual repertoires, have shown the unbearable lightness of mainstream assessment practices having currency in most educational contexts nowadays. Researchers have shown that plurilinguals communicate in a specific way, drawing on all their resources and often mixing elements from what linguists would consider different languages (Canagarajah, 2013; Grosjean, 2008, 2010; Lüdi & Py, 2009, 2013). Since the 1980s, following the work of Cummins (1981, 1991), scholars in the field have postulated the existence of a unique plurilingual competence and an equally unique repertoire (Cook, 1992, 2016; Coste et al., 2009; Grosjean, 1982, 1989, 2008, 2010), rejecting the idea of languages as distinct entities in the minds of plurilinguals. As a result, languages are increasingly seen as socially and even politically – not linguistically – defined (García & Wei, 2014). García and Lin (2017) go further by pointing out that the concept of individual languages as socially and politically defined entities have nothing to do with the repertoire of individuals: “named languages, imposed and regulated by schools, have nothing to do with individuals and the linguistic repertoire they use” (p. 126). The boundaries between languages thus become blurred or even disappear. Since the publication of the European Framework of Reference for Languages in 2001, the development of plurilingual and intercultural competences has been placed at the heart of language education and teacher training in Europe. In a preliminary study to that document, in 1997, plurilingual competence was defined as […] the ability to use languages for the purposes of communication and to take part in intercultural interaction, where a person, viewed as a social actor, has proficiency, of varying degrees, in several languages and experience of several cultures. This is not seen as the superposition or juxtaposition of distinct competences, but rather as the existence of a complex or even composite competence on which the social actor may draw (Coste et al., 2009, p. 12) DOI: 10.4324/9781003177197-1

2  Sílvia Melo-Pfeifer and Christian Ollivier In short, there are growing calls to take the specificity of plurilinguals into account in assessing their non-linguistic competences, also fed by contemporary changing demographics. Despite this visibility, the assessment of plurilingual learners has drawn little attention in education policies (Dendrinos, 2020; Gorter & Cenoz, 2017; Saville & Seed, 2022; Shohamy, 2011, 2022; Shohamy & Menken, 2015). And when it comes to assessing plurilingual competence itself, practice and even research often focus on the separate assessment of linguistic skills in different languages, taught separately – the high number of international language certifications is an obvious evidence of this trend. Such practices lead to a mismatch between acknowledging that a linguistic repertoire is a heterogeneous ensemble and the (formative and summative) assessment of discrete languages based on “business as usual” in the curricular design and classroom practices (Lopez et al., 2016). This contradiction is already present in the European Framework of Reference for Languages, as the notion of plurilingual and pluricultural competence put forward does not match the assessment practices promoted, and in which different languages are assessed through homogeneous scales. We had to wait until 2018 to see assessment scales for the plurilingual competence, for example (Council of Europe, Language Policy Programme, 2018). Notions such as the assessment of partial competences (for example, receptive competences) or the assessment of specific plurilingual skills, such as cross-linguistic mediation (Dendrinos, 2006; Stathoupoulou et al., forthcoming, 2022) have so far been neglected, despite the shy introduction of some innovative practices in mainstream language education (Kirsch & Duarte, 2020). Indeed, many educational experts emphasise the need for educational institutions to break down the barriers between languages and their learning (Candelier et al., 2012; Coste et al., 2009; García & Wei, 2014), and corresponding pedagogical theories and approaches have been developed, implemented, and analysed. This is the case of translanguaging (Cenoz, 2017; García & Wei, 2014), intercomprehension (Blanche-Benveniste et al., 1997; Bonvino, 2011; Caddéo & Jamet, 2013; Ollivier & Strasser, 2013), language awareness (Candelier, 2003; Candelier & Dabène, 2003; Tupin, 2002) or integrated language teaching (Hufeisen & Lindemann, 1998; Hufeisen & Neuner, 2003; Neuner, 2003). Similarly, aspects such as the integrated and holistic assessment of plurilingual competence have known little conceptual and empirical evolution, despite numerous publications in the field of communicative and task-based language assessment (Bachman, 2002; Morrow, 1977, 2012; Norris, 2016). Furthermore, while much has been said and proposed in terms of compensation of assessment inequalities (such as allowing for translanguaging in assessment tasks or providing external resources to students or more time to answer exams, see Lengyel, 2010 and Shohamy, 2011), little has been proposed regarding:

Introduction 3

• how plurilingual learners such as young migrants and refugees could be assessed in different school subjects (Gottlieb, 2020; Lopez et al., 2016; Yang, 2019), including in the foreign language classroom (Melo-Pfeifer & Thölkes, 2021); • how plurilingual competence can be assessed (Cenoz et al., 2013; Huver, 2010; Ollivier & Strasser, 2021), even in foreign language education.

Despite new concepts and conceptualisations referring to plurilinguals as different from the sum of several monolinguals, assessment practices have so far contributed to fossilising that same idea. It comes not as a surprise that a monolingual and monoglossic habitus (Gogolin, 1994) has prevailed: monolingual because students are most of the time assessed in and through one language at a time, in a foreign language or subject content classroom, respectively; monoglossic because assessment practices keep languages separate and do not allow the plurilingual repertoire to be mobilised holistically (Shohamy, 2022). The foundations for a language education aimed to develop and assess plurilingual competence, and plurilinguals seem to be laid out on the political and pedagogical levels, yet they often remain unimplemented in teaching and assessing practice or seen with scepticism (Huver, 2010). Indeed, languages are often still perceived as distinct systems with clear borders, and plurilingualism is often understood as the addition of several separate languages. In such approaches, translanguaging practices are largely depreciated (Schissel et al., 2021) if not prohibited. In practice, we are still far from the multilingual turn (May, 2013). Schissel et al. (2021) have pointed out that traditional assessment generates a “pressure to produce monolingual-like language”. As a result, translanguaging productions are frequently perceived as marks of failure and lack of competence (Huver, 2010) and remain banned in exams. Understandably, then, they are seen as problematic even for teachers who are aware of plurilingualism and the importance of accepting and promoting translanguaging. Assessment thus continues to reflect the traditional views of language separation, and plurilingualism is still considered a pluri-monolingualism (Dendrinos, 2020, particularly chapter 14; Lopez et al., 2016; Stavans & Hoffmann, 2015). In education institutions, languages remain separate and assessment focuses on competences in each of them. As Ascenzi-Moreno (2018, p. 358) observes, “when emergent bilinguals are required to answer in only one language during an assessment, they are placed at a disadvantage because they use only a portion of their language abilities”, in turn “keeping them silent and unengaged in […] assessment activities”, as García and Lin (2017, p. 123) point out. Nevertheless, there are glimmers of hope as attempts to assess plurilingual competence exist, and descriptions of plurilingual competence are available on which assessment can be based. Put shortly: the multilingual turn in education (despite its fallacies, Melo-Pfeifer, 2018) is still struggling to meet a

4  Sílvia Melo-Pfeifer and Christian Ollivier multilingual turn in assessment to support a coherent curriculum design and implementation of plurilingual pedagogies. Following the rationale outlined, this book addresses contemporary issues on the assessment of plurilingual competence and plurilinguals, which could be understood under the umbrella term “plurilingual assessment” (Saville & Seed, 2022) or “multilingual testing and assessment” (De Angelis, 2021), as a path to embrace “critical language testing” practices (Shohamy, 2022, p. 1448). In the scope of this book, we understand plurilingual assessment as a form and/or process of assessment that makes use of test takers’ plurilingual repertoire or adjusts to their (sometimes emergent) plurilingual repertoires. Such plurilingual repertoires were developed at school or through different mobility journeys, and the plurilingual assessment might occur either in (foreign) language education or in content-related subjects. The need for this approach to assessment comes from the acknowledgement that individuals are plurilingual because of their biographies, either growing up in bilingual regions or going through one or cumulative mobilities, or becoming plurilingual at school, or both. Five scenarios of plurilingual assessment can therefore be conceptualised within the school context, even if there may be some overlap depending on students’ linguistic biographies:

• assessment in and of a specific target language resorting to plurilingual

competence principally acquired in an educational context, for example while taking a test in a target language the individual might make use of the linguistic competences acquired across the language curriculum, following the concept of “integrated foreign language education” or the most current format of cross-linguistic mediation tasks (Dendrinos, 2006; Stathopoulou, 2015); • assessment of school content in subjects other than language resorting to plurilingual competence principally acquired in an educational context: this is close to the CLIL scenario, in which students make use of additional languages in specific subjects; in this case, multiple languages present in the school curriculum can be used to retrieve content; • assessment in the language classroom using plurilingualism acquired through mobility, for example migrant children integrating the language classroom and making use of their linguistic repertoires to progress in the language of schooling or a curricular foreign language (Levine & Mallow, 2021); • assessment of school content in subjects other than language resorting to plurilingual competence acquired through mobility, for example newcomer students have to prove their knowledge of school content by using their plurilingual competence (see chapters by Ascenzi-Moreno, García, & Lopez, De Backer, Vantieghem, & Piet Van Avermaet and by Le Pichon et al., this volume, for example); • holistic assessment of plurilingual competence, even if partial (as in the case of receptive competences in multiple languages), by making use of all the

Introduction 5 acquired languages at school and beyond (see chapters by Bonvino, Capucho & Strasser, De Carlo & Andrade and by Piccardo & North, in this volume). This book focuses especially on the latter two scenarios. It intends to match the multilingual turn in education with a much-needed multilingual turn in assessment by engaging with theories, philosophies, policies, and practices paving the way to considering plurilingual repertoires in assessment of language and other subject content, in mainstream education. After a section dedicated to language policy, we will distinguish between issues pertaining to the assessment of plurilingual students (which may include assessment practices across the curriculum) and issues related to the assessment of plurilingual competence (mainly in language subjects). We then provide an overview of the volume before we finish with some perspectives for further development of the field. Political will in Europe Educational policy – especially in Europe, but not exclusively – is often framed as a challenge for plurilingual assessment (Saville & Seed, 2022). Nevertheless, in Europe, the emergence of policy interest in plurilingual assessment is linked to the development of pluralistic approaches for teaching and learning languages and the increased voluntary or involuntary mobility of populations, inter-, and extra- European. As a result, in parallel to developments in research, a political will for plurilingualism and plurilingual assessment has emerged. In Europe, the context the co-editors are most familiar with, two reference frameworks have strongly shaped language education: the Common European Framework of Reference for Languages (CEFR) and its Companion Volume (Council of Europe, 2001; Council of Europe, Language Policy Programme, 2018) as well as the Cadre de référence pour les approches plurielles des langues et des cultures (CARAP)/Framework of reference for pluralistic approaches to languages and cultures: competences and resources (FREPA) (Candelier et al., 2007, 2012). The former calls for a significant shift in the purpose of language education, whereby the primary objective of education would become the development of plurilingual competence: The aim of language education is profoundly modified. It is no longer seen as simply to achieve “mastery” of one or two, or even three languages, each taken in isolation, with the “ideal native speaker” as the ultimate model. Instead, the aim is to develop a linguistic repertory, in which all linguistic abilities have a place. This implies, of course, that the languages offered in educational institutions should be diversified and students given the opportunity to develop a plurilingual competence. (Council of Europe, 2001, p. 5)

6  Sílvia Melo-Pfeifer and Christian Ollivier An ambitious objective for language education policy was thus set, but which struggles to gain acceptance. The obstacles have caused the Council of Europe to publish – and re-publish – guides for implementing plurilingual education. The foreword to the first edition of the Guide for the development and implementation of curricula for plurilingual and intercultural education states that The uses made of it [CEFR] tapped only part of its considerable potential and even, in some cases, disregarded certain values which the Council of Europe’s member states promote, and which underlie the approaches it describes. This obvious imbalance in implementation of the CEFR’s provisions chiefly affects plurilingual and intercultural education, although this is one of the CEFR’s main emphases. (Beacco et al., 2016, p. 5) The Guide is complemented by satellite studies. One of them is dedicated to assessing plurilingual competence (Lenz & Berthele, 2010). It is devoted to “assessment needs regarding competences that help learners to participate successfully in social life” and focuses on the “ability [of the learners] to use (and thereby improve) their plurilingual repertoire”, “assessment needs regarding competences that help learners to take responsibility and develop as language users and learners” and “assessment needs regarding competences that help learners to acquire new knowledge in their studies in plurilingual settings”. The political will to promote plurilingualism has been widely disseminated throughout the world, notably through the numerous translations of the CEFR. This will is also strong in certain countries or regions of the world, especially plurilingual ones. This is the case, for example, in the Spanish Basque Country through the Heziberri policy plan which aims to “afianzar el bilingüismo dentro de un marco de enseñanza plurilingüe” and advocates the “tratamiento integrado e integral de las lenguas para conseguir una competencia lingüística plurilingüe”. Another is French overseas island La Réunion, where translanguaging practices are widespread (and where one of the authors of this text works), the school authorities have recently published a language plan which promotes plurilingualism and are committed to training and awareness-raising for educational actors. Plurilingual education policies are complemented by a considerable amount of research work in the field of plurilingualism and plurilingual assessment (Candelier et al., 2012; García & Lin, 2017; García & Wei, 2014) and are primarily based on the “new” ways to define plurilingualism and plurilinguals (presented in the section The pervasiveness of the monolingual mindset in assessment), and/or on descriptions of plurilingual competence (developed further in the section Descriptions and assessment of plurilingual competence). Taking the specificity of plurilinguals into the acquisition and assessment process is also a priority of international educational organisations. Even if the satellite study to the CEFR dedicated to Assessment in Plurilingual and

Introduction 7 Intercultural Education explicitly does not make any attempt to take account of the linguistic dimension into the assessment of plurilingual individuals, the authors mention and list the “assessment needs regarding competences that help learners to acquire new knowledge in their studies in plurilingual settings” and underscore that “‘language in other subjects’ [than languages] – is relevant to the increasingly frequent situations where several languages of schooling coexist” (Lenz & Berthele, 2010, p. 8). Further, the Guide for the development and implementation of curricula for plurilingual and intercultural education (Beacco et al., 2016, p. 98) points out the increasing cultural and linguistic diversity of school communities and the need for teachers and learners to be “aware of the language dimensions of any subject studied – with a view not just to speaking and writing correctly, and managing communication in the class, but also to successful knowledge building and competence acquisition”. The Recommendation CM/Rec(2014)5 of the Committee of Ministers to member states on the importance of competences in the language(s) of schooling for equity and quality in education and for educational success (Committee of Ministers, 2014) states that it would be “desirable” that “those responsible for initial and further training and school administrative staff foster genuine consideration of the importance of the linguistic dimensions in teaching and assessment practices” and promotes “a diversity of approaches to assessment”. The publication of a policy paper by the UNESCO (Global Education Monitoring Report Team, 2018) on the need to ensure the recognition of migrants’ and refugees’ qualifications and prior learning (though it does not mention plurilingual assessment) is also evidence of international policy interest in the specificity of plurilingual migrants and refugees and the assessment of their non-linguistic skills. The assessment of plurilingual individuals As mentioned in the section The pervasiveness of the monolingual mindset in assessment, there are at least four ways to become plurilingual and develop a plurilingual linguistic biography. These can be through growing up in a multilingual context, through mobility (one’s own or family mobility, forced or not), through language learning in educational contexts, and through any full or partial combination of the three previous. For this reason, Cenoz and Gorter (2015) have said that being and becoming plurilingual should be seen as the norm. Hence, researchers have stressed the need to develop assessment formats that are more responsive to contemporary demographic and linguistic dynamics. However, not all these developments have impacted the discourses around “plurilingual assessment” in the same way. Indeed, De Angelis (2021) acknowledges that […] the field of multilingual testing and assessment has grown rapidly in recent years due to the widespread need to integrate immigrant

8  Sílvia Melo-Pfeifer and Christian Ollivier populations into mainstream education and provide fair and equitable forms of assessment for all students, regardless of prior language background, educational context and geographical location. (2021, p. 1) As also discussed in the section The pervasiveness of the monolingual mindset in assessment, (emergent) plurilinguals are not merely a sum of monolinguals: they are multicompetent speakers, different from so-called native speakers. Thus, if repertoires are integrated in almost a rhizomatically way, assessment should foster a more flexible use of linguistic – and other semiotic – repertoires, allowing individuals to be perceived holistically. Plurilingual individuals, particularly those who developed their plurilingual competence through mobility, should therefore be assessed considering their individual funds of academic, linguistic, and otherwise socially relevant knowledge that cannot be properly assessed in one language at a time or even through the use of languages only. This emphasis on assessment of and through (one) language alone could be called a logocentric perspective to assessment. In this section, we will problematise the assessment of plurilingual individuals who acquired their linguistic repertoires through geographical mobility processes and are therefore considered “minorities” in their host country. As we will see, in this particular context, issues of linguistic and educational equity, linguistic rights, and inclusion are at the core of the discourses on plurilingual assessment. As Shohamy puts it, “multilingual tests need to be practiced in schools to avoid language rights violations” ( 2022, p. 1445). Plurilingual lives matter! On the politics and practices of assessing plurilingual individuals

Language tests have a profound role in the organisation of societies marked by mobility, being used as instruments of selection (Extra et al., 2008; McNamara et al., 2015) and, in a mirror effect, as instruments serving the maintenance of the linguistic – and therefore social, economic, and political – status quo. As stressed by McNamara and Shohamy, “in most societies tests have been constructed as symbols of success, achievement and mobility, and reinforced by dominant social and educational institutions as major criteria of worth, quality and value” (2008, p. 89; Shohamy, 2001, 2022). This pervasive “power of the tests” (Shohamy, 2001) is also enforced in education, a field where linguistic ideologies are produced and reproduced, and where tests are used to ensure conformity and the illusion of equity, fairness, and justice. Tests are therefore also used to reproduce ideals of what it means to be plurilingual and what it means to use linguistic resources properly. Discrimination in education has been acknowledged as intersectional, following biased logics at the intersection of racial, linguistic, ethnic, and gender issues, among others (Flores & Rosa, 2015; Rosa, 2019; Schissel, 2019; Steele et al., 2022). One form in which discrimination at school is made visible is by

Introduction 9 making test-takers from different linguistic and cultural backgrounds pass the same tests, in the same formats, in the same languages (Altakhaineh & MeloPfeifer, 2022; Gottlieb, 2020), without considering their familiarity with the test culture or the test language. Another form is making the achievement of a random degree of competence in the majority language mandatory to access certain educational services, careers, and paths (Shohamy, 2022). By doing this, tests become mechanisms of linguistic profiling (Baugh, 2017) leading to discrimination based on language towards those test-takers perceived as not having sufficient command of the majority language(s). Such a “threshold” level that must be passed would be a way of “inventing homogeneity” (Piller, 2016, p. 24) and “testing against linguistic diversity” (Piller, 2016, p. 113). When this illusion of homogeneity is not achieved (which is often the case because of the overreliance on standardised assessment), students are framed in terms of deficit of skills, insufficient linguistic resources, and/or insufficient content knowledge. As stressed by Piller (2016), “underperformance in such tests is seen not as evidence of the failure of policymakers and test designers, but as evidence that casts aspersions on the academic abilities of disadvantaged groups” (p. 117). This analysis of the current state of the art makes it clear that issues related to social integration, social and political participation, equity in education, and fight against institutional (and institutionalised) discrimination, as well as the right to education, should also mean the right to be assessed on fair terms. Language testing is closely related to (linguistic) human rights (Shohamy, 2022) as taking a test in a language one does not master (yet) might prevent an individual from accessing education or at least some educational paths (McNamara & Shohamy, 2008; Phillipson et al., 1995). In other words: the right to education is not assured if students do not have the right to be appropriately assessed. Some traits that signal inequity in the assessment of migrant plurilingual students are the following:

• the use of multiple languages while performing assessment tasks is per-

ceived more as a problem, error, or sign of incompetence than a resource or strategy; • the fact that students might be developing literacies in multiple languages at a time (in the language of schooling and a curricular foreign language, for example) is overlooked; • some languages are allowed into the classroom while others are not (in a preliminary study by Melo-Pfeifer, 2021, in Portuguese as a second language in Brazil for refugee students, Spanish was more likely to be accepted in interaction and assessment than Arabic); • in the foreign language classroom, “teacher-initiated inter or cross-linguistic comparisons generally occur between the foreign language and the schooling language, leaving newcomers confronted with two new languages; explanations are generally provided in the schooling language, in which

10  Sílvia Melo-Pfeifer and Christian Ollivier the newcomers sometimes lack linguistic and metalinguistic knowledge” (Melo-Pfeifer & Thölkes, 2021, p. 221); • students have different schooling paths and school cultures, including different cultures of participation in the classroom, with such participation being differently valued and assessed by teachers (Thölkes, 2018); • inappropriate design of tests and other assessment materials/situations that may add a cultural bias to the linguistic one, as they usually include items on themes with which the learners have little or no background knowledge or experience (Flores, 2021; Lengyel, 2010); • plurilingual students with a migrant background are assessed in the same way as plurilingual students who developed their plurilingual competence through foreign language learning at school, the latter being more positively valued in terms of plurilingualism hierarchies. A way to counter this kind of linguistic discrimination has been developed around test accommodation strategies: […] Test accommodations are changes to the test administration (e.g. to small groups), test response (e.g. transcription) or the test itself (e.g. bilingual test forms) that aim to reduce construct-irrelevant variance due to language proficiency or the cultural background of languageminoritized bilinguals while not changing the construct that is intended to be measured. (Schissel, 2019, p. 27) Despite their goodwill, such test accommodation strategies may contribute to linguistic profiling of the student(s) and feelings of being involuntarily “outed”. As far as we know, there is a lack of students’ voices in research about test taking in general (a claim already made by Shohamy, 2001; an exception would be Altakhaineh & Melo-Pfeifer, 2022 or some of the studies reported by Shohamy, 2022), and about test-taking that is “accommodated” in particular (an exception would be De Backer et al., 2019). Another possibility would be to conceive a plurilingual test format from the beginning, that is, starting from a plurilingual dimension in the test design. Indeed, if accommodation strategies start from a planned and accepted monolingual perspective (the test language), in which some students would benefit from adjustments in terms of the test format, a plurilingual test design would normalise plurilingual testtaking and plurilingual performance for all students. We will return to this topic in the section Descriptions and assessment of plurilingual competence. What do we know, and what can we already do?

Gottlieb (2020) stresses that using monolingual practices to rank and rate plurilingual learners through top-down imposed assessments is one of the biggest impediments to social justice and equity in education. She therefore

Introduction 11 suggests the development of participative assessment formats, co-developed by stakeholders, families, and students themselves, and engaging in transformative assessment practices of learning, for learning, and as learning, using multiple languages. She urges school communities to embrace the linguistic and cultural resources of plurilingual learners as a way to counter the exclusive use of the schooling language as a measure of achievement. Such a move would substitute a deficit-oriented assessment with an assets-based orientation that sees plurilingual competence as a tool to show achievement, thus closing the achievement gap between so-called migrant and “autochthonous” learners. Resonating Shohamy (2011), who places multilingual assessment on a continuum, Gottlieb (2020) argues that introducing multiple languages in the assessment of plurilingual speakers would be an affirmative action serving to change attitudes towards plurilingualism and plurilinguals (namely those who are minoritised). Having said this, the introduction of multiple languages could be done both in the questions (plurilingual formulation of the questions and test items) and in students’ answers (Ascenzi-Moreno, 2018; Melo-Pfeifer & Thölkes, 2021; Steele et al., 2022). In Shohami’s conceptualisation (2011), grading could go from scoring answers only in the target language (other languages being used as a cognitive support) to scoring answers given in multiple languages at the same time. The literature has called out multiple biases in the assessment of plurilingual individuals, which are sometimes interwoven with intercultural bias (Altakhaineh & Melo-Pfeifer, 2022) or assumptions about literacy and multimodality in test design (Flores, 2021). When plurilinguals use their multiple resources, those are seen as mistakes and signs of deficit, particularly when they are not part of the languages promoted by the school curriculum. In more recent approaches to the assessment of plurilingual students still coping with difficulties in using the language of schooling, different sorts of test adaptations were already conceptualised and put into practice, such as external adaptation of the testing conditions, but the use of learners’ plurilingual repertoires as a resource is rarely foreseen. This perpetuates the use of the plurilingual repertoire in education as a remediation strategy rather than a strength students and teachers should capitalise on. In what follows, we offer a tour de table of strategies that can be used in educational settings to support plurilingual learners, at the planning, valuation, and follow-up phases of assessment (Melo-Pfeifer & Thölkes, 2021). At the planning level, teachers can create test formats and items collaboratively with other stakeholders (Gottlieb, 2020). They can integrate themes with which plurilingual migrant and refugee students are more at ease than using culturally loaded ones, increasing the validity of the test instrument. Another possibility might be to use linguistic accommodation strategies to formulate assessment items, when possible, also in the learners’ home language(s). To increase transparency in the assessment process, teachers might want to clarify expectations and the assessment format by providing examples of successful performance in similar assessments. At the valuation phase, that is, during

12  Sílvia Melo-Pfeifer and Christian Ollivier the assessment, teachers could facilitate the use of language instruments to support understanding and production or allow the use of translanguaging practices, namely the use of different languages and other semiotic resources such as drawings and visual materials, to provide as much support as possible for the assessment. Furthermore, teachers may want to value different semiotic resources as signs of commitment and engagement to the assessment process, use think-aloud techniques to ensure comprehension of assessment items, and engage with students in the assessment or grading process. Finally, in the follow-up phase, teachers can provide students with positive rather than deficitoriented feedback, negotiate with them the areas of acquisition that could come next, according to their goals and needs, and collaboratively engage in more integrative linguistic practices. This list of possibilities can also be read as a research agenda, where their effectiveness could be tested, and plurilingual students’ beliefs about them could be studied. Descriptions and assessment of plurilingual competence As we saw in the section The pervasiveness of the monolingual mindset in assessment, the assessment of plurilingual competence and assessment of plurilinguals are still relatively young areas in both language didactics and language education policy. The interest emerged largely from two reasons: in Europe, because of the introduction of linguistic policies promoting foreign language learning from a more integrated perspective; and worldwide, from the increasing diversity of societies and individuals, and especially from the diversity of learner groups and learners themselves. This superdiversity (Blommaert, 2013; Blommaert & Rampton, 2011; Vertovec, 2007) is the result of the intensification of mobility, especially of voluntary or forced migration that our societies have been experiencing over the past years. The need to take into account the particularity of each individual, and in particular their linguistic specificities, has become a priority in educational policies, with the aim of inclusion and integrating new populations. For several years now, an increasing number of practitioners and researchers have called for assessment processes to take learners’ plurilingualism into account, while looking for ways to assess plurilingual competence. In this section, we will focus on the latter. Plurilingual competence has been defined in terms of descriptors by the authors of CARAP-FREPA (Candelier et al., 2007, 2012), who carried out pioneering work proposing an impressive list of “resources” that compose plurilingual and intercultural competence. Most recently, the Companion Volume (Council of Europe, Language Policy Programme, 2018) of the Common European Framework of Reference published descriptors in the field of mediation and “plurilingual/pluricultural competence”, “innovative areas of the CEFR for which no descriptor scales had been provided in the 2001 set of descriptors” (Council of Europe, Language Policy Programme, 2018, p. 23). These descriptors concern “Building on pluricultural repertoire”, “Plurilingual comprehension”, and “Building on plurilingual repertoire”. In

Introduction 13 the field of intercomprehension, two European projects have carried out ambitious work proposing three-level scales (Project Miriadi, n.d.) and sixlevel scales (similar to the CEFR scales) (Garbarino & Melo-Pfeifer, 2020; Project EVAL-IC, 2019). Based on this research work, the assessment of plurilingual competence remains an area of ongoing development and experiments. In this section, we attempt to categorise the different types of assessment that we have identified in the scientific literature. We focus on the assessment of plurilingual competence of people who have grown up or spent some time in a plurilingual context or have developed or are developing plurilingualism through learning (in an institutional or non-institutional context). From this perspective, most of the plurilingual assessment formats we will present may be suited for assessing plurilingual competence and multilingual subjects. We propose to distinguish between:

• hybrid assessment, including profile assessment (in different languages) and calculated assessment, and

• holistic assessment. Hybrid assessment

Under hybrid assessment forms we include forms of assessment concerned with the multiple separated assessment of (partial) competences and skills in an individual’s different languages. Hybrid profile assessments

This is what Jamet (2010) proposes, for example, at the end of a course in Romance intercomprehension. She assesses reception skills with the help of exams designed to assess or even certify competences in the different Romance languages covered by the course. This makes it possible to assess a level in the different languages and to get a profile in Romance intercomprehension. This is also the procedure which is indirectly promoted by the authors of the CEFR (Council of Europe, 2001) and implemented in the European Language Portfolio. The descriptors allow for the separate assessment and self-assessment of competences in different languages in order to establish an individual plurilingual competence profile. In the case of self-assessment, the advantage is that an assessment by descriptors can be complemented by reflection and self-assessment on learning processes, experiences, tasks, and outcomes. Theses assessment formats show the strengths and limitations in different languages. They draw a multilingual profile of the individual without any scoring or calculation. In the assessment types, which we will now present, learners’ competences are also assessed separately in different languages. However, the assessment of skills in one language considers competences in one or more

14  Sílvia Melo-Pfeifer and Christian Ollivier other languages and can lead to what Gorter and Cenoz (2017) call “multilingual scoring”. Hybrid calculated assessment

We give here two examples of scoring. Mueller Gathercole et al. (2013) point out that “one cannot assess children’s performance in one language to gain insight into their performance in the other, nor into their overall linguistic abilities” (p. 51). They therefore propose to develop specific tests for bilinguals in which, on the one hand, “children’s language experience is taken into consideration” (p. 48), especially the language of the home. On the other hand, they propose a test procedure in one language with two calculated scores taking into consideration the plurilingual profile of the learners: “child receives two normed scores, one comparing him/her to the performance of all children at the given age, the other comparing him/her to the performance of children with similar linguistic experience” (p. 48). In a more holistic view of plurilingual competence – but using separate assessment procedures – Cenoz et al. (2013) defend the same idea. They state that it is essential to “take the students’ linguistic repertoire into account, including the differences in their L1s” (p. 15). In a study carried out in the Spanish Basque Country, they assess competences in three languages (English, Basque, and Spanish) separately and then add up the scores of Basque, Spanish, and English to obtain a multilingualism index and the scores of Basque and Spanish to obtain an index for bilingualism. In these examples, the perspective is holistic, but the tests are monolingual. The holistic point of view is actualised by considering the specificities of the plurilingual learners in the calculation procedure of the scores. We will now consider assessment protocols, which integrate many languages or allow the use of translanguaging. Holistic assessment

Under integrated holistic assessment, we report on forms of assessment that make integrated use of all of the individual’s languages. In these cases, as we will see, languages can be both the means of assessment (assessment through translanguaging) and the object of assessment (integrated assessment of plurilingual communication competence). Assessment through translanguaging

If we consider that plurilingual competence includes the ability to manage one’s entire repertoire and thus deal with several languages at the same time, integrated assessments seem to us to be an interesting avenue. We mean forms of assessment which integrate different languages into the tests, based on actual plurilingual practices (Gorter & Cenoz, 2017) or accept or even promote

Introduction 15 the use of several languages and their mix in learners’ responses. This form of assessment is therefore suited to both the assessment of plurilingual competence and the plurilingual individual. This category, which could be called “translanguaging assessment”, includes all forms of assessment that accept or promote translanguaging, that is the free and fluid use of one’s entire language repertoire without being restricted by socially established language boundaries (García & Wei, 2014). In this kind of assessment, “multilingual students are encouraged to express what they know by deploying their entire semiotic repertoire, regardless of whether features used are the ones legitimated in school” (García & Kleifgen, 2019). This is very close to what Shohamy (2011, p. 427) advocates: “multilingual tasks for which it is understood by the test-takers that mixing languages is a legitimate act that does not result in penalties but rather is an effective means of expressing and communicating ideas that cannot be transmitted in one language”. It is about assessing the competence of individuals to use their entire repertoire, to “express complex thoughts effectively, to explain things, to persuade, to argue, to give directions, to recount events, etc.” and to be “virtuous language users, rather than just careful and restrained language choosers” (García & Lin, 2017, p. 127). Integrated assessment of plurilingual communication competence

In recent years, there has been an increasing amount of work on the design of tests for assessment of plurilingual communication competence, especially in the pioneering field of intercomprehension. As mentioned above, competence descriptors have been developed in this field. In addition, intercomprehension training courses have been offered for several years, and assessment forms have been designed for these programmes. Several projects have therefore proposed models for the assessment of intercomprehension competences. Some use the forms of assessment presented above; others, such as EVAL-IC, have developed integrated assessment models based on the work of specialists in taskbased assessment (Norris, 2016) and plurilingualism. The EVAL-IC model is presented in the contribution by Bonvino et al. in this volume. It considers that it is possible to develop communicative competence within a group of languages that are linguistically and historically connected. This model suggests that pan-Romance (or pan-Germanic, or pan-Slavic …) communication competence is articulated in language activities (reception, production, interaction, and mediation) and can be assessed through plurilingual real-life tasks (Ollivier, 2022; Ollivier et al., 2019). The test designed by the EVAL-IC project for Romance languages invites students to imagine that they are applying to participate in an international (and multilingual) conference on sustainable development. Candidates are asked to prepare a presentation and a discussion with a multilingual jury. Therefore, they have (first) to inform themselves about the topic using various documents in various languages and formats (written, video). Then they have

16  Sílvia Melo-Pfeifer and Christian Ollivier to prepare a presentation with slides that the multilingual jury can understand. The exchange then takes place in a mix of languages, with each jury member using their own language and the candidate trying to make themself understood by all. The assessment is based on the descriptors for pan-Romance communication competence developed by the project. The multilingual dimension is thus present in each phase of the test and at the very heart of the assessment process. We believe that translanguaging assessment and tests like the one designed by the EVAL-IC project have a large potential for assessing plurilingual competence. Unfortunately, the two still often largely ignore each other. As Ollivier (2022) has shown, in the EVAL-IC test, candidates are asked to produce their slides and express themselves in one language, thus reducing the possibilities of plurilingual expression. Translanguaging practices are excluded. They also have little space in the descriptors of competence developed by the EVAL-IC project. We believe that mixing the potential of translanguaging and integrated assessment of plurilingual communication competence could lead to an emancipated and free form of integrated assessment, which could integrate different (socially defined) languages and allow students to show how they can manage their whole plurilingual repertoire in real-life tasks. This volume in a nutshell In this book, we intend to bring together researchers and practitioners to discuss theoretical and practical issues related to assessment of plurilingual competence and plurilingual individuals as a field of inquiry, potentially leading to more social justice and equity in education. Aiming at uncovering and problematising theories, discourses and ideologies attached to plurilingualism, plurilingual competence and plurilingual assessment, the book covers issues from the education of young immigrants and the assessment of young refugees to the assessment of plurilingual competence and plurilingual students in educational settings. As the structure of the introduction already makes clear, this book is organised around two parts. One section considers the specificities of plurilingual speakers in assessment practices (either in language subjects or other content subjects); another part focuses on the assessment of plurilingual competence. Despite the apparent separation of the two parts, multiple commonalities can be perceived, starting with the fact that plurilingual learners have by definition developed some form of plurilingual competence. But while all plurilingual speakers may have developed a plurilingual competence, not all have a background marked by mobility. Part I, called “Considering the specificities of plurilinguals in assessment: Theoretical framework, policy and educative issues”, is composed of eight chapters dealing with theoretical and empirical issues related to the assessment of individual plurilingualism, either developed at school or through mobility paths throughout life. The authors are primarily concerned with issues of

Introduction 17 linguistic equity in education and the need to disrupt monolithic assessment formats that are not equally valid for all individuals. Such disruption starts already with the need to deconstruct the notion of plurilingualism as sum of named languages and to embrace a more holistic and integrated view, under the concept of repertoire. The first part starts with a chapter by Jessica de Angelis and Anat Stavans called “Assessing multilinguals: Critical “wh-” questions about testing multiple language speakers in (non)educational contexts”. The authors lay the foundations to consider the assessment of plurilingual subjects and plurilingual competence. They start with a discussion of some basic terminological differences in the field, focusing on the distinctions between bilingualism/multilingualism and plurilingualism/individual multilingualism, and then describe the diversity of plurilingual profiles of plurilinguals. The following chapter, called “Latinx bilingual students’ translanguaging and assessment: A unitary approach”, is informed by language and raciolinguistic ideologies as well as translanguaging theory. Laura Ascenzi-Moreno, Ofélia García, and Alexis A. López discuss how translanguaging questions the validity of assessments grounded in a monoglossic perspective and even of bilingual assessments, which maintain the power hierarchy between languages and consider bilingualism as additive. The authors open avenues for other forms of literacy and content assessments that take actual bilingual students’ (trans)languaging practices into consideration and make assessments just and appropriate for them. These new forms of assessment allow students to fully use their unitary linguistic/semiotic repertoire, allowing them to use all their multilingual and multimodal resources to respond to the questions/test items (see Shohamy, 2022, on multimodal testing). They also allow a “conceptual scoring” regardless of the language or mode in which the response is given. Jacqueline D’warte and Yvette Slaughter, in a contribution called “Examining plurilingual repertoires: A focus on policy, practice and assessment in the Australian context”, discuss policy, practice, and assessment research pertaining to plurilingualism, with a focus on mainstream classrooms. They consider the significant linguistic, cognitive, and social benefits derived from recognising and harnessing students’ plurilingual repertoires alongside the competing tensions of predominantly monolingual, monoglossic educational policies, curricula and assessment frameworks in Australia. The authors argue that assessment practices are failing to keep pace with conceptual and pedagogical progress in the education system and that the effectiveness of any pedagogical change will be limited if assessment systems continue to perpetuate reductive interpretations of language and literacy. Emmanuelle Le Pichon, Mai Naji, Dania Wattar, and Vivian Lee, in their chapter titled “Evaluating the effectiveness of digital tools for the assessment of young plurilingual students”, address the complexity and difficulties of assessing the competences of plurilingual newcomer students who don’t yet master the language of schooling to show their actual capacities in academic subjects. They underline the risk of underestimating their academic skills by

18  Sílvia Melo-Pfeifer and Christian Ollivier using monolingual assessment tools. As a possible response to this challenge, they show how digital technologies, which can integrate different languages (especially a game-like multimodal platform used in Ontario: Binogi), can be a powerful tool to assess students (in STEM contents) in languages they know and to allow them to demonstrate their actual knowledge and skills in these subjects. The paper shows that the linguistic, cultural, and political settings influence the implementation and use of the platform. In “Functional multilingualism in educational assessment”, Fauve De Backer, Wendelien Vantieghem, and Piet Van Avermaet investigate how the content knowledge of plurilingual students with limited competences in the language of schooling can be assessed in a fair and valid way. They specifically address the assessment of science in Flanders and explore the impact of readaloud accommodations in the language of schooling and in the students’ L1 as well as written bilingual tests on the results of plurilingual learners. The authors demonstrate that none of the accommodations lead to better results for the plurilingual persons or to a fairer assessment. In their conclusion, the authors point out that accommodation is only one possible avenue to take the students’ plurilingualism into account and to provide fairer assessment. They propose a “Functional Multilingual Assessment” which includes two shifts: “from a testing culture towards an assessment culture and from monolingual towards multilingual assessments”. In “Assessing language competences of multilingual speakers – a brief review of two test approaches: C-tests and TBLA (task-based language assessment)”, Barbara Hinger argues that multilingual assessment approaches for content tests may need to challenge international standardised tests if they are to contribute to fairer assessment procedures. The author shares insights into recent studies focusing on language testing from a multilingual perspective by scrutinising existing language test constructs and expanding them towards multilingual purposes. Isabelle Audras, in a contribution called “Assessing the first language skills of plurilingual secondary school students: An analysis of students’ and evaluators’ perspectives”, addresses the assessment of home languages of plurilingual children and gives insights into a programme run in schools by an association in France promoting plurilingual education and diversity. Her study shows the positive impact on students, teachers, and evaluators. From a methodological perspective, the paper highlights the difficulties to translating a single test designed for the assessment of competences in a specific language into different languages and cultures, and the limitations of using a single assessment grid based on the descriptors of the CEFR. The first part ends with a chapter by Anthony J. Liddicoat and Angela Scarino called “Developing a multilingual perspective in language assessment: Multilinguality and interculturality”. In their contribution, the authors claim that, in additional language learning, the educational goal is to expand learners’ multilingual repertoires and develop multilinguality and interculturality. Following this premise, they examine how multilinguality in assessment can

Introduction 19 be conceptualised and what constitutes multilinguality in assessment. They critically discuss examples of assessment practices and examine issues such as the need to incorporate the assessment of content/knowledge and language use and to acknowledge the embodied nature of multilinguality. The authors conclude that “a multilingual orientation to assessment is a pluralistic endeavour that attends to the contextualisation of students, their languages, and their language learning”. The texts in Part II, entitled “Assessing plurilingual competence: Principles and empirical approaches”, share an emphasis in proposing, describing, and critically discussing specific assessment formats to assess plurilingual competence from what could be perceived as a European stance. The seven chapters report on theoretical and ethical issues related to the assessment of plurilingual competence, namely considering the added-value of assessing partial competences or, as E. Huver, even challenging the very possibility of doing so. Authors present empirical studies or discuss practical issues related to assessing plurilingual competence, namely in formal contexts covering second, foreign and heritage language education and contexts. The part opens with a chapter called “Assessing the multiple acquisitions from bi-/plurilingual learning – The questions to ask”. Marisa Cavalli explains that assessment is important in improving teaching and learning, and providing a fair education for all, though not all types of assessment are equal for this purpose. She therefore highlights a number of blind spots in the assessment of plurilingual skills, above all by exploring the educational added-value in terms of multiple acquisitions. The author stresses the importance of assessing the plurilingual individual’s linguistic and communicative competences as a process, and not only as a product of teaching and learning, thus taking a developmental stance. Lukas Bleichenbacher, Anna Schröder-Sura, Christoph Chesini, Mara De Zanet, Cornelia Gantenbein, and Robert Hilbe present a cross-linguistic teacher education curriculum for language teachers that was recently introduced at the St. Gallen University of Teacher Education, in Switzerland, and focus on self-assessment of plurilingual competences. In a chapter called “Crosslinguistic teacher education: How self-assessment and reflection tools can support plurilingual student teachers”, the authors describe the theoretical concept and scenarios of implementation for that programme, in which students with different target languages are introduced to the methodology of cross-linguistic and plurilingual education. The contribution presents the findings of a study based on the general self-assessment of an optional plurilingual project week and the specific self-assessment of an intercomprehension task. The results show that students exploited the diversity of languages and cultures for language learning, developed their listening and reading strategies in multilingual contexts, and increased their plurilingual language (learning) awareness. Enrica Piccardo and Brian North, in a chapter titled “Plurilingualism and assessment: some issues and options”, reflect upon the role of assessment in a coherent school curriculum informed by an action-oriented approach. After

20  Sílvia Melo-Pfeifer and Christian Ollivier discussing key points related to the assessment of plurilingual competence, such as the object and the subject, as well as modalities of assessment, the authors provide examples of formal and informal plurilingual assessment with secondary school students, including plurilingual comprehension and a plurilingual oral exam (implemented in Austria and called “The Austrian certificate of plurilingualism”). The authors stress that the examples discussed in the paper show that a shift towards assessment of plurilingualism in the language classroom is possible if it goes hand-in-hand with adopting plurilingual pedagogies. The chapter by Elisabetta Bonvino, Filomena Capucho, and Margareta Strasser adresses the evaluation of intercomprehension skills. With the title “EVAL-IC: An integrated approach to plurilingual competences”, the chapter gives a concrete example of a framework of reference with six levels of intercomprehension competence and an assessment protocol for plurilingual competence, more specifically for pan-romance communication competence. The contribution introduces a task-based assessment scenario with different tasks linked together by a macro-situation designed by the project EVAL-IC participants, in a holistic approach of assessment based on the idea of “intercomprehension”. Task and assessment are focused on meaning and the capacity to understand and be understandable. The contribution ends with information about the future modification of the assessment protocol due to the COVID pandemic and changes in educational practices. In a continuation of the previous chapter, Maddalena De Carlo, and Ana Isabel Andrade focus on intercomprehension as a modality of plurilingual communication and on the assessment of intercomprehension competences. In a chapter called “Towards an assessment of intercomprehension competences in coherence with plurilingual approaches”, the authors analyse the potential of the plurilingual assessment scenario designed by the EVAL-IC project, which simulates the application process for participating in an international conference on sustainable development. The study’s findings show that the subjects used different languages to acquire knowledge on the topic of the conference and that the test provided the students with new learning opportunities and contributed to the extension of their plurilingual profile and their linguisticcommunicative repertoire. Lisa Marie Brinkmann and Sílvia Melo-Pfeifer, in a chapter entitled “Using the European Language Portfolio for the assessment of plurilingual students in the foreign language classroom”, look at the state of the art on the use of portfolios as a metacognitive and process-oriented tool for assessing plurilingual competences. The authors show various benefits of the use of the European Language Portfolio – especially by migrants – in terms of learning agency and awareness, identity, autonomy, metacognitive, intercultural, and communicative skills. They conclude that the tool’s main strengths lie in the use of concrete “can do” descriptors for assessment and its ability to foster students’ reflection on their plurilingual identity and language learning strategies, including cross-linguistic strategies.

Introduction 21 In her chapter on “Evaluation and plurilingual competence: Between epistemological consistency and ethical vigilance”, Emmanuelle Huver reflects upon the dilemma between needing to assess plurilingual competence and the (technical and/or ethical) impossibility of actually doing so. She argues that research on plurilingual competence assessment has essentially consisted of a reflection on assessment tools and approaches and their design and implementation. By doing this, the author claims, as already described above in this introduction, that research has neglected a more fundamental theoretical/ epistemological reflection on the tensions between assessment and plurilingualism, which she discusses in terms of two perspectives of plurilingualism: a quantitative (plurilingualism as addition and juxtaposition of languages) and a qualitative one (plurilingualism as an interconnected repertoire). Conclusion and perspectives To promote the development of plurilingual competence (as advocated by the Common framework of reference and numerous researchers working on language education), ways to assess this competence must be explored. However, as far as we know, little research has been carried out in the field of assessment of plurilingual competence, and there is a need for more theoretical, scientific, and practical work on this key issue in the domain of language education. In addition, the development of migratory flows has resulted in educational institutions receiving more and more learners who are taught and assessed in languages other than their first languages without taking into account their specific linguistic profile. Assessment of plurilingual competence and plurilingual individuals is therefore not only an educative but also a politically sensitive issue. This book acknowledges these goals and the challenges that go with them. It was our intention, as editors, to bring together papers written by researchers from different backgrounds, responding to the strong scientific and societal need for assessment of plurilingual competence and plurilingual individuals. We intended to provide theoretical reflections and new insights on an important issue and to provide a significant reference for scholars and practitioners working in the domain of plurilingual education. This book, we hope, thus offers readers theoretical considerations and empirical approaches on how to take the specificities of plurilingualism into account while assessing various competences and how to assess plurilingual competence. With this book, we intend to start bridging the gap between the fields of language policies and research on plurilingual competence and the field of assessment in language education and in migrant/refugee integration policies. Despite some advancements, several issues remain open to pursue a research agenda:

• bringing together various approaches from different geographical and theoretical backgrounds, as assessment cultures might be situated in the contexts they are designed to;

22  Sílvia Melo-Pfeifer and Christian Ollivier

• developing original theories and designing test scenarios to assess plurilingual competence, that include different languages and modes;

• designing and researching language assessment formats in a plurilingual

and a multimodal way, accounting for a dynamic plurilingualism acquired both in and out of school; • as CLIL tends to expand in mainstream schools, designing and researching assessment formats in a plurilingual and multimodal way, accounting for plurilingualism and subject-specific knowledge acquired at school and beyond; • designing content assessment scenarios that are more personalised and respectful of individual repertoires by exploiting digital resources; • moving towards more realistic assessment formats that mirror real communicative practices, where the affordances of space and artefacts as well as non-verbal clues are used as meaning-makers in situated interaction, also exploiting the possibilities of immersive digital environments; • tracing intersectionality in the assessment of minoritised plurilingual students across the curriculum and the lifespan and developing formats and tools of assessment that counter homogeneous and imposed narratives of deficit, incompetence, and underperformance. References Altakhaineh, A., & Melo-Pfeifer, S. (2022). “This topic was inconsiderate of our culture”: Jordanian Students’ perceptions of intercultural clashes in IELTS writing tests. Applied Linguistics Review. https://doi.org/10.1515/applirev-2021-0183. Ascenzi-Moreno, L. (2018). Translanguaging and responsive assessment adaptations: Emergent bilingual readers through the Lens of possibility. Language Arts, 95(6), 355–369. https://www.jstor.org/stable/26779020. Bachman, Lyle F. (2002). “Some Reflections on Task-Based Language Performance Assessment”. Language Testing 19(4), 453-476. https://doi.org/10.1191/02655 32202lt240oa. Baugh, J. (2017). Linguistic profiling and discrimination. In O. García, N. Flores, & M. Spotti (Eds.), The Oxford handbook of language and society (pp 349–367). Oxford: Oxford University Press. Beacco, J.-C., Byram, M., Cavalli, M., Coste, D., Cuenat, M. E., Goullier, F., & Panthier, J. (2016). Guide for the development and implementation of curricula for plurilingual and intercultural education. Strasbourg: Council of Europe Publishing. Blanche-Benveniste, C., Valli, A., Mota, M. A., Simone, R., Bonvino, E., & Uzcanga Vivar, I. (1997). EuRom4. Método do ensino simultâneo das línguas românicas. Metodo para la enseñanza simultánea de las lenguas románicas. Metodo di insegnamento simultaneo della lingue romanze. Méthode d’enseignement simultané des langues romanes. Firenze: La nuova Italia. Blommaert, J. (2013). Ethnography, superdiversity and linguistic landscapes: Chronicles of complexity. Clevedon: Multilingual Matters. Blommaert, J., & Rampton, B. (2011). Language and superdiversity. Diversities, 13(2), 1–22. https://newdiversities.mmg.mpg.de/fileadmin/user_upload/2011_13-02_ art1.pdf

Introduction 23 Bonvino, E. (2011). EuRom5: Ler e compreender 5 línguas românicas = leer y entender 5 lenguas románicas. Milano: Hoepli. Caddéo, S., & Jamet, M.-C. (2013). L’intercompréhension, une autre approche pour l’enseignement des langues. Paris: Hachette français langue étrangère. Canagarajah, S. (2013). Translingual practice: Global Englishes and cosmopolitan relations. London: Routledge. Candelier, M. (Ed.). (2003). Janua Linguarum - la porte des langues: L’introduction de l’éveil aux langues dans le curriculum. Council of Europe; https://www.ecml.at/Por tals/1/documents/ECML-resources/JAN-FR.pdf?ver=2018-04-03-120408-473. Candelier, M., Camilleri-Grima, A., Castellotti, V., de Pietro, J.-F., Lörincz, I., Meißner, F.-J., Noguerol, A., & Schröder-Sura, A. (2007). CARAP : cadre de référence pour les approches plurielles des langues et des cultures. Graz: Centre européen pour les langues vivantes. Candelier, M., Camilleri-Grima, A., Castellotti, V., De Pietro, J.-F., Lörincz, I., Meißner, F.-J., Noguerol, A., & Schröder-Sura, A. (2012). FREPA. A framework of reference for pluralistic approaches to languages and cultures: Competences and resources. Graz: Conseil de l’Europe, Centre européen pour les langues vivantes CELV, Strasbourg. Candelier, M., & Dabène, L. (2003). L’Éveil aux langues à l’école primaire: Evlang, bilan d’une innovation européenne. Brussels: De Boeck. Cenoz, J. (2017). Translanguaging in school contexts: International perspectives. Journal of Language, Identity & Education, 16(4), 193–198. Cenoz, J., Arozena, E., & Gorter, D. (2013). Multilingual students and their writing skills in Basque, Spanish and English. In V. M. Gathercole (Ed.), Bilingual assessment: Issues and solutions (pp 186–205). Bristol: Multilingual Matters. Cenoz, J., & Gorter, D. (Eds.) (2015). Multilingual education. Between language learning and translanguaging. Cambridge: Cambridge University Press. Committee of Ministers. (2014). Recommendation CM/Rec(2014)5 of the Committee of Ministers to member states on the importance of competences in the language(s) of schooling for equity and quality in education and for educational success. Council of Europe. https://search.coe.int/cm/Pages/result_details.aspx? ObjectID=09000016805c6105. Cook, V. (1992). Evidence for multi-competence. Language Learning, 44(4), 557–591. http://www.viviancook.uk/Writings/Papers/LL92.htm. Cook, V. (2016). Working definition of multi-competence. Vivian Cook. http://www. viviancook.uk/Writings/Papers/MCentry.htm. Coste, D., Moore, D., & Zarate, G. (2009). Plurilingual and pluricultural competence. Council of Europe, Language Policy Division. https://rm.coe.int/168069d29b. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Press Syndicate of the University of Cambridge. Council for Cultural Co-operation. Education Committee. Modern Languages Division (Strasbourg). Council of Europe, Language Policy Programme. (2018). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume with new descriptors. Council of Europe; https://rm.coe.int/cefr-companion-volumewith-new-descriptors-2018/1680787989. Cummins, J. (1981). The role of primary language development in promoting educational success for language minority students. In California State Department of Education (Ed.), Schooling and language minority students: A theoretical framework (pp 16–62). Los Angeles: California State University.

24  Sílvia Melo-Pfeifer and Christian Ollivier Cummins, J. (1991). Language learning and bilingualism. Tokyo: Graduate School of Languages and Linguistics, Sophia University. De Angelis, G. (2021). Multilingual testing and assessment. Bristol: Multilingual Matters. De Backer, F., Slembrouck, S., & Van Avermaet, P. (2019). Assessment accommodations for multilingual learners: Pupils’ perceptions of fairness. Journal of Multilingual and Multicultural Development, 40(9), 833–846. https://doi.org/10.1080/ 01434632.2019.1571596. Dendrinos, B. (2006). Mediation in communication, language teaching and testing. Journal of Applied Linguistics, 22, 9–35. Dendrinos, B. (2020). The politics of foreign language policies, teaching and testing. Athens: Pedio. Extra, G., Spotti, M., & Van Avermaet, P. (Eds.) (2008). Language testing, migration and citizenship: Cross-national perspectives on integration regimes. London: Continuum. Flores, J. (2021). Assessing refugee-background adult second language learners with emerging literacy: How a social semiotic analysis reveals hidden assumptions of test design. In D. Warriner (Ed.), Refugee education across the lifespan. Mapping experiences of language learning and use (pp 139–159). Cham: Springer. Flores, N., & Rosa, J. (2015). Undoing appropriateness: Raciolinguistic ideologies and language diversity in education. Harvard Educational Review, 25(2), 149–171. Garbarino, S., & Melo-Pfeifer, S. (2020). Décrire et évaluer les compétences en intercompréhension: Du référentiel de compétences REFIC (MIRIADI) aux descripteurs de compétences en intercompréhension (EVAL-IC). In C. Hülsmann, C. Ollivier, & M. Strasser (Eds.), Lehr- und Lernkompetenzen für die Interkomprehension. Perspektiven für die mehrsprachige Bildung (pp 103–124). Münster: Waxmann. García, O., & Kleifgen, J. A. (2019). Translanguaging and literacies. Reading Research Quarterly, 55(4), 553–571. https://doi.org/10.1002/rrq.286. García, O., & Lin, A. M. Y. (2017). Translanguaging in bilingual education. In O. García, A. M. Y. Lin, & S. May (Eds.), Bilingual and multilingual education (pp 117–130). Cham: Springer. García, O., & Wei, L. (2014). Translanguaging: Language, bilingualism and education. Hampshire: Palgrave Macmillan. Global Education Monitoring Report Team. (2018). What a waste: Ensure migrants and refugees’ qualifications and prior learning are recognized. UNESCO. https:// unesdoc.unesco.org/ark:/48223/pf0000366312. Gogolin, I. (1994). Der monolinguale Habitus der multilingualen Schule. Münster: Waxmann. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31(3), 231–248. https://doi.org/10.1080/0950 0782.2016.1261892. Gottlieb, M. (2020). Classroom assessment in multiple languages. A handbook for teachers. Thousand Oaks: Corwin Press. Grosjean, F. (1982). Life with two languages: An introduction to bilingualism. Cambridge: Harvard University Press. Grosjean, F. (1989). Neurolinguists, beware! The bilingual is not two monolinguals in one person. Brain and Language, 36(1), 3–15. https://doi.org/10.1016/0093934X(89)90048-5. Grosjean, F. (2008). Studying bilinguals. Oxford: Oxford University Press.

Introduction 25 Grosjean, F. (2010). Bilingual: Life and reality. Harvard University Press. https://doi. org/10.4159/9780674056459. Hufeisen, B., & Lindemann, B. (Eds.). (1998). Tertiärsprachen: Theorien, Modelle, Methoden. Tübingen: Stauffenburg. Hufeisen, B., & Neuner, G. (2003). Mehrsprachigkeitskonzept, Tertiärsprachen, Deutsch nach Englisch. Strasbourg: Council of Europe. Huver, E. (2010). “J’ai pas pu corriger parce que c’est le mélange de langues”. L’évaluation de la compétence plurilingue entre résistances et tensions. Les Cahiers de l’ACEDLE – Recherches en didactique des langues et des cultures, 7(1). https:// journals.openedition.org/rdlc/2144. Jamet, M.-C. (2010). Intercomprensione, Quadro comune europeo di riferimento per le lingue, Quadro di riferimento per gli approcci plurilingui e valutazione. Synergies Europe, 5, 75–98. https://gerflint.fr/Base/Europe5/marie_christine_jamet.pdf Kirsch, C., & Duarte, J. (Eds.). (2020). Multilingual approaches for teaching and learning. From acknowledging to capitalising on multilingualism in European mainstream education. London: Routledge. Lengyel, D. (2010). Language diagnostics in multilingual settings with respect to continuous procedures as accompaniment of individualized learning and teaching. Reference study. Strasbourg: Council of Europe. https://rm.coe.int/16805a1cac. Lenz, P., & Berthele, R. (2010). Assessment in plurilingual and intercultural education. Guide for the development and implementation of curricula for plurilingual and intercultural education. Council of Europe. https://www.coe.int/t/dg4/ linguistic/Source/Source2010_ForumGeneva/Assessment2010_Lenz_ENrev.pdf. Levine, G. S., & Mallows, D. (Eds.) (2021), Language learning of migrants in Europe: Theoretical, empirical, policy, and pedagogical issues. Cham: Springer Lopez, A. A., Turkan, S., & Guzman-Orth, D. (2016). Assessing multilingual competence. In E. Shohamy et al. (Eds.), Language testing and assessment, encyclopedia of language and education. Cham: Springer. https://doi.org/10.1007/ 978-3-319-02326-7_6-1. Lüdi, G., & Py, B. (2009). To be not to be…. A plurilingual speaker. International Journal of Multilingualism, 6(2). https://doi.org/10.1080/14790710902846715. Lüdi, G., & Py, B. (2013). Etre bilingue (4e édition ajoutée d’une postface). Bern; Berlin; Brussels: Peter Lang. May, S. (2013). The multilingual turn: Implications for SLA, TESOL, and bilingual education. London: Routledge. McNamara, T., Khan, K., & Frost, K. (2015). Language tests for residency and citizenship and the conferring of individuality. In B. Spolsky, O. Inbar-Lourie, & M. Tannenbaum (Eds.), Challenges for language education and policy: Making space for people (pp 11–22). New York: Routledge. McNamara, T., & Shohamy, E. (2008). Language tests and human rights. International Journal of Applied Linguistics, 18(1), 89–95. Melo-Pfeifer, S. (2018). The multilingual turn in language education: Facts and fallacies. In A. Bonnet & P. Siemund (Eds.), Foreign language education in multilingual classrooms (pp 191–212). New York, Amsterdam: Benjamins. Melo-Pfeifer, S. (2021). “Distância linguística e perceção de integração escolar de famílias refugiadas: um estudo exploratório e comparativo das famílias hispanofalantes e arabofalantes”. Curitiba: Universidade Federal do Paraná, 23 de junho de 2021. Melo-Pfeifer, S., & Thölkes, M. (2021). “Wie soll ich das LKind bewerten?”: Between standardization and differentiation in the assessment of refugee students.

26  Sílvia Melo-Pfeifer and Christian Ollivier A qualitative study of foreign language teachers’ representations in Germany. In G. S. Levine & D. Mallows (Eds.), Language learning of migrants in Europe: Theoretical, empirical, policy, and pedagogical issues (pp 219–243). Cham: Springer. Morrow, K. (1977). Techniques of evaluation for a notional syllabus. Reading: Centre for Applied Language Studies. Morrow, K. (2012). Communicative language testing. In C. Coombe, P. Davidson, B. O’Sullivan, & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp 140–146). Cambridge: Cambridge University Press. Mueller Gathercole, V. C., Thomas, E. M., Roberts, E. J., Hughes, C. O., & Hughes, E. K. (2013). Why assessment needs to take exposure into account: Vocabulary and grammatical abilities in bilingual children. In V. C. Mueller Gathercole (Ed.), Issues in the assessment of bilinguals (pp 20–55). Bristol: Multilingual Matters. Neuner, G. (2003). Le concept de plurilinguisme et la didactique de langue tertiaire. In G. Neuner & B. Hufeisen (Eds.), Mehrsprachigkeitskonzept, Tertiärsprachen, Deutsch nach Englisch (pp 13–35). Strasbourg: Conseil de l’Europe. Norris, J. M. (2016). Current uses for task-based language assessment. Cambridge: Cambridge University Press. Ollivier, C. (2022). Vers une compétence de communication panromane et une intercompréhension libérée. In D. C. Koch & M. Rückl (Eds.), Au carrefour de langues et de cultures. Mehrsprachigkeit und Mehrkulturalität im Französischunterricht (pp 67–81). Stuttgart: ibidem-Verlag. Ollivier, C., Capucho, F., & Araújo e Sá, M. H. (2019). Defining intercomprehension competencies as prerequisites for their assessment. Rivista di psicolonguistica applicata. Journal of Applied Psycholinguistics, XIX(2), 15–30. https://www.torrossa. com/en/resources/an/4612743. Ollivier, C., & Strasser, M. (2013). Interkomprehension in Theorie und Praxis. Vienna: Praesens. Ollivier, C., & Strasser, M. (2021). Mehrsprachige kompetenz evaluieren – Der Fall Der Interkomprehension. In C. Helmchen, S. Melo-Pfeifer, & J. Von Rosen (Eds.), Mehrsprachigkeit in der Schule (pp 53–72). Tübingen: Narr Verlag. Phillipson, R., Rannut, M., & Skutnabb-Kangas, T. (1995). Introduction. In T. Skutnabb-Kangas & R. Phillipson (Eds.), Linguistic human rights. Overcoming linguistic discrimination (pp 1–22). Berlin: De Gruyter. Piller, I. (2016). Linguistic diversity and social justice. An introduction to applied sociolinguistics. Oxford: Oxford University Press. Project EVAL-IC. (2019). EVAL-IC: produits. EVAL-IC. https://evalic.eu/productions/ produits/. Projet Miriadi. (n.d.). Refic—Niveaux. Miriadi – Mutualisation et Innovation Pour Un Réseau de l’Intercompréhension à Distance. https://www.miriadi.net/niveaux. Rosa, J. (2019). Looking like a language, sounding like a race. Oxford: Oxford University Press. Saville, N., & Seed, G. (2022). Language assessment in the context of plurilingualism. In E. Piccardo, A. Germain-Rutherford, & G. Lawrence (Eds.), The Routledge handbook of plurilingual language education (pp 360–376). London: Routledge. Schissel, J. L. (2019). Social consequences of testing for language-minoritized bilinguals in the United States. Bristol: Multilingual Matters. Schissel, J. L., De Korne, H., & López-Gopar, M. (2021). Grappling with translanguaging for teaching and assessment in culturally and linguistically diverse contexts: Teacher perspectives from Oaxaca, Mexico. International Journal of Bilingual

Introduction 27 Education and Bilingualism, 24(3), 340–356. https://doi.org/10.1080/136700 50.2018.1463965. Shohamy, E. (2001). The power of the tests. A critical perspective on the uses of language tests. Harlow: Longman.. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418–429. https://www. jstor.org/stable/41262376 Shohamy, E. (2022). Critical language testing, multilingualism and social justice. The Modern Language Journal, 56(4), 1445–1457. https://doi.org/10.1002/tesq. 3185. Shohamy, E., & Menken, K. (2015). Language assessment: Past to present misuses and future possibilities. In W. E. Wright, S. Boun, & O. García (Eds.), The Handbook of Bilingual and Multilingual Education (pp 253–269). Oxford: Wiley Blackwell. Stathopoulou, M. (2015). Cross-language mediation in foreign language teaching and testing. Bristol: Multilingual Matters. Stathopoulou, M., Gauci, P., Liontou, M., & Melo-Pfeifer, S. (forthcoming, 2023). On the role of assessment in cross-linguistic mediation: Insights from the METLA project. In K. Vogt & B. Antia (Eds.), Multilingual assessment: Finding the nexus. Peter Lang. Stavans, A., & Hoffmann, C. (2015). Multilingualism. Cambridge: Cambridge University Press. Steele, C., Dovchin, S., & Oliver, R.. (2022). ‘Stop measuring Black Kids with a White Stick’: Translanguaging for classroom assessment. RELC Journal, 53(2). https:// doi.org/10.1177/00336882221086307. Thölkes, M. (2018). “Schaffen die das?”. Die Leistungsbeurteilung geflüchteter Kinder und Jugendlicher in der Regelklasse aus der Perspektive von Fremdsprachenlehrkräften. Master Thesis. Hamburg: University of Hamburg. Tupin, F. (2002). L’éveil aux langues: Entre recherche des effets et reflets de l’interculturalité des approches scientifiques. In Un état des savoirs à La Réunion (pp. 259–264). Saint André: ADSFOI. Vertovec, S. (2007). Super-diversity and its implications. Ethnic and Racial Studies, 30(6), 1024–1054. https://doi.org/10.1080/01419870701599465 Yang, X. (2019). Assessment accommodations for emergent bilinguals in mainstream classroom assessments: A targeted literature review. International Multilingual Research Journal. https://doi.org/10.1080/19313152.2019.1681615.

Part I

Considering the specificities of plurilinguals in assessment Theoretical framework, policy and educative issues

1

Assessing multilinguals Critical “wh-” questions about testing multiple language speakers in (non)educational contexts Gessica De Angelis and Anat Stavans

Introduction The aim of this chapter is to discuss a number of issues related to the testing and assessment of multiple language speakers in educational and non-educational contexts. The discussion is organized around guiding “wh-” questions designed to illustrate the main challenges we are currently facing when testing multilingual student populations. We begin with a discussion of some basic terminological differences in the field, focusing on the distinction between bilingualism and multilingualism, and plurilingualism and individual multilingualism. We then move on to consider the range of profiles multilingual test takers might have, the contexts in which they might be tested and who the test writer would normally be. This is followed by further considerations of the main reasons for testing multilinguals in education and society, what kind of evaluations are already in use and what we are willing to accept as possible testing contexts beyond traditional educational settings. Terminological issues The field of multilingual testing and assessment has been a pivotal concern in research and implementation practices in psychology, sociology, language policy, education and economics. As such, the terminology deployed in relation to multilingualism has been inconsistent. In this section, we wish to clarify the key terms we use throughout the chapter and why. The first distinction concerns the use of the terms bilingualism and multilingualism. Multilingual testing and assessment practices have been mostly based on the literature on bilingual testing and assessment, which was later adapted to include speakers of more than two languages. It is therefore essential for us to draw a distinction between the two terms. In this chapter, the terms bilingual and bilingualism are used to refer to speakers of two languages while multilingual and multilingualism to speakers of three or more languages. The distinction is not only quantitative, but also qualitative, as will result from the issues raised throughout the chapter.

DOI: 10.4324/9781003177197-3

32  Gessica De Angelis and Anat Stavans A second terminological distinction relates to the difference between individual multilingualism and plurilingualism and what these terms convey to educators and researchers in the field. In the multilingualism research literature, multilingualism is widely understood as a social and individual phenomenon and the difference is typically conveyed through the use of the terms societal and individual multilingualism. Initially, plurilingualism was equated to multilingualism when discussing plurilingual communities, where multiple languages were spoken. Later, plurilingualism, especially in the educational and sociolinguistic literature, was used to describe an individual as the agent of the interaction between the languages. In other words, plurilingualism unlike multilingualism allowed greater tolerance in the performance of the speakers. Plurilinguals do not necessarily have to be fluent in multiple languages. A competent plurilingual can speak in one language while understanding another and can switch between languages when appropriate and/or necessary. In her well-crafted paper, El Euch (2011) discusses the distinction between multilingualism and plurilingualism and emphasizes the value of seeing plurilingualism holistically and seeing the multilingual speaker as a complex psycholinguistic system. She states that: “… for the Council of Europe (2007), multilingualism refers to the presence of several linguistic varieties in a given geographical area (social phenomenon); and plurilingualism refers to the repertoire of languages or linguistic varieties an individual may use (individual phenomenon)” (El Euch, 2011, p. 1392). According to the Council of Europe, multilingualism refers “to the presence of several languages in a given space, independently of those who use them” (Council of Europe, 2007, p. 18) while plurilingualism refers to the “the intrinsic capacity of all speakers to use and learn, alone or through teaching, more than one language” (Council of Europe, 2007, p. 17). Multilingualism is thus interpreted as including a social dimension, and plurilingualism an individual one. At the heart of the plurilingualism definition is the speakers’ language repertoire and in a more recent Common European Framework of Reference (CEFR) publication we find plurilingual competence explained as: the ability to call flexibly upon an inter-related, uneven, plurilinguistic repertoire to: • switch from one language or dialect (or variety) to another; • express oneself in one language (or dialect, or variety) and understand a person speaking another; • call upon the knowledge of a number of languages (or dialects, or varieties) to make sense of a text; • recognise words from a common international store in a new guise; • mediate between individuals with no common language (or dialect, or variety), even with only a slight knowledge oneself;

Assessing multilinguals 33 • bring the whole of one’s linguistic equipment into play, experimenting with alternative forms of expression; • exploit paralinguistics (mime, gesture, facial expression, etc.). (Council of Europe, 2020, p. 28) This description implies that plurilingual competence is the ability to call upon one’s plurilingual repertoire. Such plurilingual repertoire, unlike the multilingual repertoire, does not regard the “languages” of the plurilingual as unique and contained linguistic systems associated with a “named language” (Otheguy et al., 2015), but rather a repository containing linguistic features, communicative needs, cultural practices, skills and abilities to produce and receive linguistically coded messages. What is missing from this perspective, which is instead expressed by the term individual multilingualism, is the attention to how these abilities arise and influence cognition. For instance, if plurilingualism is concerned with the ability to “switch from one language or dialect (or variety) to another,” individual multilingualism is concerned with the same ability, but also with how the repertoire comes into being, when and why it is used, what overt and covert processes are involved and how this ability influences overall cognitive functioning and, ultimately, performance (see De Angelis, 2021; Maher, 2017). As a development closest to the more traditional multilingualism concept, plurilingualism grew out of a specifically European or Eurocentric tradition. Modern languages scholarship and the European Union policy work, both foregrounded the importance of knowing and working across multiple languages in speech and writing. The ground on which the “pluri” has stood, then, is a multilingual ground – a view of languages as discrete entities and learners as acquiring each language independently even if simultaneously (Donahue, 2021; Kirsch, 2021). But more recently, “pluri” has begun representing other dynamic linguistic movements which value the way multilinguals “operate between languages” (Kramsch, 2009, p. 249) rather than the degree to which they master a given language. We no longer look at competence as associated with each language and the goal as “additionist” (Molina, 2011) but rather to multiplying linguistic abilities that draw across languages, even as one or the other is not perfected. Kramsch (2009) argues that plurilingualism forces us to draw on “… more than the mere coexistence of languages […] the transcultural circulation of values across borders, the negotiation of identities, the inversion, even inventions, of meanings, often concealed by a common illusion of effective communication” (Zarate et al., 2008 cited in Cook, 2009, p. 249). Such a view poses great challenges to testing and assessment especially within the educational context. Testing and assessment have been traditionally discussed from an educational perspective, hence the strong presence of the term plurilingualism in the educational literature, including that of the Council of Europe. In recent decades, however, academic discussions about testing and assessment have begun to focus on multilinguals’ learning processes and cognition (De Angelis,

34  Gessica De Angelis and Anat Stavans 2021; De Backer et al., 2017; Gorter & Cenoz, 2017; Szabo et al., 2021), which explains why the term individual multilingualism or more simply multilingualism are so widespread among expert literature. We are now at the stage where contributions from various fields of enquiry converge onto the debates about testing and assessment and multilingualism, bringing with them divergent terminological definitions such as the ones discussed above. The terms individual multilingualism and plurilingualism are clearly synonyms and share many similar concerns. The question, however, is not so much what the best term is, or which school of thought a term represents, but what questions we want to ask about testing and assessment and multiple language speakers. Asking a question as simple as “how many years does an immigrant need to catch up with native peers” is no longer a question that can be answered by focusing on school performance and language repertories, as this question also concerns speed of learning and how the mind functions when multiple languages are known and used in different contexts. For this reason, throughout this chapter, we will refer to the more traditional individual multilingualism and multilingual testing and assessment rather than plurilingualism and plurilingual testing and assessment as the former are broader in scope and intent and allows us to bring into the discussion reference to mental processes and their effect on the way a multilingual mind functions. Who are the multilinguals we are testing? We know relatively little about testing multilinguals because much of the existing research has focused on bilinguals mostly with English as one of the languages, often in homogeneous contexts (Abedi et al., 2005; Escamilla et al., 2018; Lopez et al., 2017; Ortiz et al., 2012; Rhodes et al., 2005). Testing multilinguals is naturally more complex and challenging than testing bilinguals because of the number of languages that can potentially be used to write a test and that students can use to answer test questions. The two populations are therefore not easily comparable when it comes to testing. The first question to ask is then: Who are the multilinguals we are testing and what is their profile? The potential multilingual test takers are not of a single type and most multilinguals of school age would be exposed to at least one foreign language from primary school onwards. A multilingual then may be: (a) a second or additional language speaker fluent in the L1 and less fluent in the additional languages, such as a first-generation immigrant learning the host language and an additional language at school, (b) a functional multilingual with a good level of proficiency in all of his or her languages, like a multilingual individual born and raised in a multilingual community contexts like South Tyrol, Italy, the Basque Country, Spain or mixed cities in Israel, (c) a functional multilingual who is also a dialect speaker like a child born in Italy who is learning English in school and was raised speaking Italian as well as a local dialect, like Neapolitan, Sicilian or Venetian, (d) a school multilingual, that is an individual raised monolingual who became multilingual through education or travel,

Assessing multilinguals 35 (e) a multilingual from birth who is exposed to different languages as his or her first languages and (f) a temporary worker who is not an immigrant and only lives in the country for a limited period of time. The wide variety of possible profiles means that classifying multilinguals for testing purposes can be very challenging, particularly for standardized testing agencies. The typical classifications found in standardized tests are no longer appropriate for the range of multilingual students we find in modern societies who are not easily classifiable using the few categories that standardized tests use such as native speakers, first- and second-generation immigrants. Considering the above profiles and the three core categories used most frequently in standardized testing, most functional multilinguals in (b) and (c) would not be classifiable at all, as most of them would be neither native speakers of the language of testing nor first- or second-generation immigrants. Classifying test takers using profiles that do not represent them can introduce a significant bias in the testing process, hence the need to reflect on who the test takers are and introduce classifications based on test-takers’ actual profiles. One possible solution is that testing agencies introduce the whole new category of functional multilingual, as De Angelis (2021) proposed. Who is assessing multilinguals and who are the test writers? Multilingual assessment is normally associated with formal education in primary and secondary schools or higher education contexts. The purpose of an assessment is thus typically educational in nature, but multilinguals may also be assessed for a range of other non-educational reasons such as employment or citizenship purposes (Milani, 2008; Shohamy, 2007). The decision as to who, when and why multilinguals are tested rests with test planners and policymakers. Test writers are those who interpret and translate policies and directives into an actual test, and test administrators are the ones who score the tests and interpret the results. Who are the test writers and what is their training and expertise? How aware are they of the linguistic and cultural diversity of the student/test taker populations? Are test writers influenced by their understanding and beliefs about testing, multilingualism or even language ability? Test writers can be professionals employed to fulfil this function, for instance those working for standardized testing agencies, or they can be school staff, most typically teachers. Their background, experience and training may therefore differ substantially. Standardized testing agencies usually train their professional test writers to write tests according to the agency’s aims and objectives, in a specific language, and following specific templates. Professional test writers, however, are often unaware of the challenges that second/foreign language learners encounter when taking a test in their non-dominant languages (Abedi, 2006; Abedi et al., 2001; Abedi & Gándara, 2006; Martiniello, 2008). Whether the test writer is a professional working for a standardized testing agency or a teacher working in a school, they will share the common practice

36  Gessica De Angelis and Anat Stavans of developing tests using only one of the languages of the multilingual’s repertoire, which may be a language in which the individual has not reached a sufficient level of literacy. Test writers may also have expertise in non-linguistic areas, such as mathematics, economics, literature or history, and may therefore write tests that are linguistically too complex for speakers of multiple languages whose knowledge of the language of testing may not be sufficiently advanced for them to perform effectively. Whenever there is little awareness of second language learning, test writers may underestimate or overestimate what students can do with the language of testing when taking a test. At the same time, language teachers and school staff, especially those with some experience with second language speakers and immigrants, are usually more aware of the challenges multilingual students face when taking a test in a language they do not fully master. These teachers tend to simplify the language they use, the sentence structure, the vocabulary or the grammar instinctively, as they are already used to doing so in their teaching (Bristol & Brett, 2015; Kim & Elder, 2005; Rix, 2009; Takahashi-Breines, 2002). Less experienced teachers may have more difficulty with simplifying a test and are more likely to need to learn through some awareness training sessions. Beyond the scholastic test writers, there are agencies that are commissioned to develop tests for employment, citizenship, security scanning or classification. Such tests also run across two vortices like the ones in education: content/purpose and language. Writers of these tests are often informed by language policymakers and other agencies and institutions that lack language expertise or awareness for linguistic diverse populations. Why are we assessing multilinguals? In addition to the variability among test takers in terms of their linguistic trajectory in being or becoming multilingual, as discussed in the previous section we also find great variability in the capacity, training and expertise of test writers and test administrators, whether they are testing language ability (usually one language) or using language to test content (Math and Literature at school or the ability to write a letter of application using one language). The motivation for testing is driven by “why” institutions, companies, countries or schools assess multilinguals using one language and excluding others from their repertoire. Multilinguals can be tested for educational, political or economic reasons, and in modern societies we can distinguish three main levels of testing associated with purposes that can be formal and overt, or informal and covert (Dendrinos, 2019; Lopez et al., 2017; Milani, 2008; Shohamy, 2007, 2011; Stavans & Hoffmann, 2015). These are the micro, meso and macro levels. At the micro level, testing is associated with the “economics” of the individual, for instance when testing for employment purposes, education opportunities or personal growth. At the meso level, testing is associated with a range of activities that can benefit a community in its entirety such as maintenance of a

Assessing multilinguals 37 language for religious purposes or rituals, inclusion into an ethnic or minority communitarian institution, complying with a family language policy, etc. And at the macro level, testing is concerned with the country itself, the employability of its residents, the denial of citizenship and participatory rights in the country management (such as elections). Hence, the motivation of testing multilinguals may be oriented, restricted or determined by internal and external forces exerted on the multilingual. What are we assessing? In the previous sections, we have briefly raised the distinction that exists in the tools and expertise as well as motivation for assessing multilingualism in formal and non-formal educational contexts. In fact, when thinking of assessing knowledge, especially that of language and communicative ability, the most prominent practices are associated with educational contours and those assessments outside of the educational institutions often draw on knowledge or techniques for assessment in the education systems. Hence, when pondering on what is assessed in multilingualism, the natural inclination is to focus on language structure, vocabulary, comprehension and production. Therefore, the range of tests that can be administered in educational contexts is very wide, and tests can have very different contents depending on the overarching objectives (usually formative, summative and diagnostic) of the test or assessment tool. A test can assess language knowledge or knowledge of a particular content, such as mathematics, history or geography. In the case of Content and Language Integrated Learning (CLIL), a test can assess language as well as content, and the question is often whether priority should be given to one or the other (Hönig, 2010; Lo & Fung, 2018; Massler et al., 2014; Otto & Estrada, 2019). Typically, language assessment isolates the linguistic forms of a specific language from content (as in an essay on a given topic or an argumentative text), function (whether the pragmatic and discursive functions of the language are met) and multilingual competences (whether the multilingual used different resources from all the languages in the repertoire). Moreover, language tests may assess procedural knowledge separating receptive (reading and listening) from productive (writing and speaking) skills disregarding the intertwined relation between these skills (as the relation between reading and writing). Although content is widely understood to be the priority in the school system (typically monolingually oriented) and content is communicated through language, separating the two is not always as straightforward as one would like. A test can also assess specific skills which are not exclusively language oriented, such as problem solving, critical analysis or creative thinking (Anderson & Krathwohl, 2001; Krathwohl, 2002). The traditionally grounded monolingual approach to testing and assessment has been widely criticized, in particular for not being sufficiently inclusive of learners who are linguistically diverse and do not perform linguistically

38  Gessica De Angelis and Anat Stavans as well as a native speaker (Gorter & Cenoz, 2017; Lopez et al., 2017; Poza, 2017; Shohamy, 2011). The main criticism is that multilingual speakers may not necessarily be able to complete a test in a language that is not dominant for them, as they may not understand the test questions or may not know the language well enough to be able to answer as expected. We can already see a general shift in the way tests for multilinguals are now being discussed, and the shift from a purely monolingual conceptualization of testing and assessment to a conceptualization of language and linguistic diversity mediated by translanguaging practices in formal education. This shift sees the multilingual as the agent of all his/her languages, the contexts and situations in which they are summoned and the linguistic negotiations that transpire in this extended network of abilities, knowledge, considerations and decision that participate in the individual multilingualism. In this sense, our uptake of individual multilingualism is not devoid from social multilingualism and hence is similar to plurilingualism particularly in contexts where several languages are spoken in the family and community environment, which inevitably impinge and affect the individual multilingualism. The very notion of language proficiency has undergone a profound change in just 20 years. Since the publication of the CEFR (Council of Europe, 2001), the concept of proficiency has largely shifted from being language-specific and based on grammatical accuracy and ideal L1 norms to being identified as an individual’s ability to communicate in a given language. With a series of “can do” statements, the CEFR identifies what people are able to do at each level of proficiency. For example, a speaker of any language at the A1 level (basic user): Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help. (Council of Europe, 2001, p. 24) With more and more bilingual and multilingual students in schools all over the world, it is now imperative that we begin to raise questions about what are we testing and in particular: what exactly is the standard in standardized testing? The whole concept of a standard is highly contested when applied to multilingualism, multiculturalism and multilingual learners. Let us take a classic standardized vocabulary test for early language development as an example: PPVT (Peabody Picture Vocabulary Test) which is an untimed test of receptive vocabulary for Standard American English and is intended to provide a quick estimate of the examinee’s receptive vocabulary ability. It can be used with the Expressive Vocabulary Test to make a direct comparison between the examinee’s receptive and expressive vocabulary skills. Often used as baseline information on lexicon in monolinguals, it has been

Assessing multilinguals 39 widely used in multilingual research, especially in studies that isolate the languages of the multilinguals, test them separately and compare them to the monolingual norms. There are norms that have been adjusted to the populations of a specific language, but the picture-based prompt has typically been kept. The problem is that a term that makes sense and has a certain frequency in American English may not make sense or have a similar frequency in another language or context, despite having been translated and adapted to the local language and geographical context. For instance, one of the pictures depicts an old man with a beard and a cane. This picture targets the word “old” in the PPVT. In applying this test with the Israeli norms, the frequency of “old” was similar to that of the original American English test, but the children in a religious neighbourhood interpreted the picture to mean “Rabbi” (i.e., the Jewish religious authority in their community). Moreover, in such testing assuming the norm for a particular item is the same in all the languages we can confer that the item is known but can we attribute it to a specific language? Can we know if “called upon” for production purposes in all the languages the same item will be used? Standardized testing agencies tend to address these issues by providing solutions that focus on the test itself, such as literal translations, linguistic simplification and cultural adaptation, but never consider introducing changes based on who the test takers are, their multilingual profiles and for which purposes they use different languages or a combination of languages (codeswitches, borrowings and other unique sophisticated multilingual capacities). In fact, testing the natural capability of a multilingual in a way that assesses not only the ability and skills within each language but also across the languages and abilities is a weakness in multilingual assessment. The reluctance to deviate from the monolingual paradigm is very clear, as is the lack of acceptance that the so-called standard is linguistically and culturally linked to ideal norms that are increasingly becoming obsolete in today’s world, or at the very least questionable. How are we assessing multilinguals? In recent decades, testing has mostly conformed to the monolingual paradigm, but some forms of bilingual testing have begun to emerge with increasing frequency in schools with a large proportion of bilingual students in class. With the spread of trilingual programmes in multilingual community contexts and the increasing demand for the inclusion of English in the curriculum from primary school onwards, there has also been increasing discussions of extending bilingual testing practices to multilingual learners (Cenoz et al., 2013). Research in this area is still very limited but what is available shows a clear transition from a traditional monolingual approach to more hybrid approaches that allow the use of multiple languages in the same test. Today we can find two main types of multilingual tests being used: the multilingual-by-translation and the multilingual-by-design tests (De Angelis,

40  Gessica De Angelis and Anat Stavans 2021). The multilingual-by-translation test is essentially a monolingual test translated into other languages. This is the typical format used by large, standardized tests such as the Program for International Student Assessment (PISA) (OECD, 2010, 2012), but translations for testing purposes are also used in bilingual programmes where needed. Standardized testing agencies tend to prefer this type of test as it allows them to maintain a monolingual approach to testing while combining the simplification and adaptation of content for culturally and linguistically diverse populations. It is more manageable but also more lucrative, and the adaptations beyond translation take on localized norms. The multilingual-by-design test is a type of test that allows the use of several languages within the same test. Being a more flexible format, there is no real standard that can be followed and language choices rest on the teacher, the student or both. This format implies that one needs to think in advance about the function and role of languages within the test. For instance, a test for a trilingual school may present a reading text in one language, instructions in another and answers in yet another, but the same test may also present a text written in several languages and give the student the choice of answering in any of the languages of instruction. In the first case, it is the teacher or the test writer who chooses the language; in the second, it is the student who decides in which language he or she wants to answer. A multilingual-bydesign test therefore does not have a fixed format as both students and teachers can make language decisions to some degree. This form of hybrid testing is already used in some trilingual programmes where integrated pedagogical practices are employed, that is where teaching involves the use of several languages in the same lesson. These approaches are certainly a step forward for multilingual testing or assessment, but when the choice remains the teacher’s prerogative, it reflects what the teacher considers linguistically appropriate and also the general assumption that the students’ knowledge of the three languages is well balanced. Stemming from traditional monolingual paradigms, multilingual testing and assessment needs to be reconceptualized to reflect the complexities associated with multilingual individuals and their variegated language backgrounds (the individual), and the languages spoken in the community in which they live (the social). In order to ensure a fair and equitable testing process, these two dimensions must be examined simultaneously, that is they must be viewed as integrated into the testing process. This level of flexibility is at the heart of the integrated approach to multilingual testing and assessment discussed in De Angelis (2021), which is a flexible approach that “refers to the process of gathering information about the knowledge, skills and abilities of multilingual learners, using tools designed for linguistically and culturally diverse populations that may be administered in multiple modalities, scored by multilingual examiners and interpreted using data about the test takers that includes information about their language background and their living environment.” (De Angelis, 2021, p. 2)

Assessing multilinguals 41 So far, we have raised several issues that tap into the essence of multilingual assessment by way of “wh-” questions. Our next move to multilingual assessment is the grassroot realization that flexibility and understanding are key to our future conceptualization of testing and assessment. Multilinguals do not see their languages as scholastic disciplines but rather as tools of expression and communication and therefore they do not “pile” languages but perfect their multilingual “machine” with multiple languages. Such is the necessary outlook on multilingual assessment we believe is yet to come. What kind of testing contexts are we prepared to accept? Testing contexts are mostly associated with educational settings, but there are also many other contexts in which multilinguals are formally tested or asked to perform in a language that is not dominant for them, such as a driving test (in the local language), a job interview, application for citizenship, etc. (Shohamy, 2007). Some pivotal questions concerning the context of multilingual testing are: How far does our concept of testing context go, and how far are we willing to extend our traditional conceptualization of non-educational contexts? Are assessments that involve multilinguals in other language contexts crafted differently and appropriately to that linguistic situation? or are they drawn from different contexts? In other words, are tests used to assess English as a native language appropriate for the learner of English as a second language or as a foreign language? as the language for academic or professional purposes? The formal tools for assessment in the educational tradition have been tests, quizzes and exams that contain different types of questions and target different abilities, knowledge and levels. However, there is a whole range of assessments that are not formally administered, but still have value for teachers who need to assess their students’ progress, and for students as feedback on their achievements and accomplishments. A classic example is homework, which students do on a regular basis but which is not formally credited as an assessment tool. Individual work such as homework is crucial for the teacher to understand if students are learning, or if some topics need to be revisited, reinforced or further elaborated in class. Concomitantly students receive continuous feedback on their performance that helps both them and their families to understand their progress and whether they need further help. This kind of feedback is particularly important for immigrant and multilingual learners whose central formal learning resources are the school, the teacher and their peers; and their informal learning resources are the family and friends. There are also contexts in which formal testing is not appropriate, such as in a kindergarten context. Yet, assessment is conducted by alternative means, albeit non-diagnostic assessments that pertain to disabilities in learning that are mediated through language. For example, if the kindergarten teacher tells the child to fetch a green glass (in any language) and the child follows the instructions, the teacher is implicitly assessing the child’s ability as reflecting the understanding of that language (its vocabulary, the sentence structure and

42  Gessica De Angelis and Anat Stavans even pragmatic knowledge). This type of assessment, while not particular to multilinguals, is important for multilingual children, especially immigrant children, who may speak other languages at home (Stavans, 2015). In addition to these less traditional contexts, it is also possible to introduce forms of testing that are not reliant on the traditional pen-and-paper/ computer mediated task model. For example, a student may be given a manual (written) or asked to follow instructions (orally) to put together an appliance/ to build an object with construction pieces, to plan a trip or to simulate an election process. In this case, the student is tested not only on language knowledge but also on the ability to follow instructions, to plan and execute a task, to solve a problem or to seek information, all of which are soft and hard skills and abilities that are particularly important in a modern society. These skills reflect not only language but also its use and production especially if and when such tasks may be collaborative and cooperative peer learning (Blum-Kulka & Snow, 2004; Cekaite et al., 2014) in the educational context. As such, these collaborative language-mediated activities assess the level of literacy, discourse ability, pragmatic and linguistic skills and enable spaces for different languages to be manifested and assessed. Not only among children one can find these alternative assessments but also among adults in different language-related activities such as reading an assembly manual or Covid-19 vaccination procedures provided by social agencies to citizens on the basis of their ability to understand a text and follow instructions. When we talk about testing and multilingualism, we must therefore think of the various testing possibilities that exist and that we use (as well as those we do not use or are not adequate for our purposes) to assess multilinguals, both by formal and informal means. Testing does not have to be based only on the classic pen-and-paper format but collaborative, participatory and productive means that examine individual performance in different and alternative contexts for different and authentic communication needs that are often fundamental for multilingual speakers and are also not constrained by a highly structured and rigid testing framework limited to one language. Where are we administering assessment? While formal tests are typically administered in school settings, some forms of testing can be conducted in spaces outside schools where the test format is appropriately and contextually geared to benefit from the particular environment, such as the home (home exams, projects that require the home environment or the people in those environments) or in less conventional places such as during a visit to a museum (where pupils may have to comment or inquire about an exhibited object) or a trip to the zoo. All these contexts will require a carefully crafted task/tool and both process and product can be used to assess language, content, skills and abilities. There are also contexts in which both explicit and implicit assessment of a multilingual’s language skills takes place, such as in professional contexts

Assessing multilinguals 43 among adults. If we take the example of a hospital, we often find foreign nurses who have to follow instructions from medical staff given in the local language and who have to understand and interpret the dosage of a medicine, the symptoms of a patient or follow a treatment procedure. Foreign hospital staff are usually asked to provide the results of a standardized language proficiency test, but this does not provide any information about the person’s actual competence in their particular area of work. Even such proficiency tests may be limited to specific and limited linguistic aspects of the language but not to the ability to use it in authentic or natural conversation. Multilingual patients, on the other hand, may have to fill in forms to obtain services such as surgery where they are called to indicate previous illnesses, health status, allergies and so on, all in a language that may not be dominant for them. Often terms and concepts that are within the medical realm may not be identical and this may result in inaccuracies that put the patient’s health if not their lives at risk. Are foreign hospital staff able to assist a multilingual patient in filling in a form? A standardized language test cannot necessarily assess the linguistic competence required in such a complex context. However, doctors implicitly evaluate the work of foreign staff, including their ability to follow instructions and react if necessary. It would certainly be a step forward to introduce tests that can assess the overall language competence of hospital staff and evaluate their actual knowledge of the field. This context is not the only one where professional services and attention are dependent on language and communication. Contexts such as the court, the classroom, the office of immigration, the bank and a store may also be considered along the same or similar vein. An obstacle this type of assessment poses is not only embedded in the type of test, the test taker and the test evaluator but in its monolingually oriented conceptual grounding in which multilingual competence would be ignored or evaluated negatively because it does not conform to the traditional monolingual paradigm. Multilinguals routinely function in a society that does not take their language abilities into account and what is needed is a shift from a traditional monolingual view to one that allows for the assessment of both monolingual competence and translingual communicative practices. This shift is natural not only in hospitals between patients and caregivers, but also through staff professional communication. For instance, in hospitals in Israel there is a wide variety of linguistic backgrounds among doctors, nurses, therapists, sanitation workers and administrative staff. There are professionals among all these hospital staff who are at least trilingual (i.e., Arabic-Hebrew-English, or Russian-Hebrew-English, and more) who not only meet the patient, but must communicate among and between all these professional agents. In their communication there are different aspects and interactions that may enable the use of more than one language depending on the interlocutors and the interaction situation. Their language assessment must be viewed then in a broader rather than limited perspective. Another context that has permeated our lives in the past decades and has become more pronounced in our daily communication is the assessment of

44  Gessica De Angelis and Anat Stavans language and more specifically multilingual behaviour in the use of language(s) in cyberspace, especially that that involves messaging in computer mediated communication (Stavans et al., 2021). Today we can write an e-mail in Mandarin Chinese and end it with “LOL” (laughing out loud) and use the exclamation OMG (Oh my God) or include a visual icon that is universally understood and may take on different linguistic representations such as a heart  (to indicate love, like, in the form of a verb or a noun), all of which are universally understood by most people independent of language, culture, age, time or space. Multilinguals are already communicating globally in a way that shows greater metalinguistic awareness and the ability to adapt written notations or oral messages to multilingual and multicultural audiences. We still know very little about this type of adaptation and the types of production that can result from it. Future tests will have to follow these new forms of communication, especially since they are now so widespread among the younger generation of learners and will eventually become part of classroom and workplace communication. Multilinguals are already integrating these forms of communication into their everyday language use and some of them are already sought after by employers who, for example, hire young professionals to moderate social websites that reach millions of users worldwide. We still know very little about this adaptability that multilinguals are already displaying, and in time we will also have to think about how such knowledge and skills can be tested. References Abedi, J. (2006). Language issues in item development. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp 377–398). Mahwah, NJ: Lawrence Erlbaum Associates Publishers. Abedi, J., Courtney, M., Mirocha, J., Leon, S., & Goldberg, J. (2005). Language accommodations for English language learners in large-scale assessments: Bilingual dictionaries and linguistic modification (CSE Report 666). Los Angeles, CA: The National Center for Research on Evaluation, Standards, and Student Testing. Abedi, J., & Gándara, P. (2006). Performance of English language learners as a subgroup in large-scale assessment: Interaction of research and policy. Educational Measurement Issues and Practice, 25(4), 36–46. Abedi, J., Hofstetter, C., Baker, E., & Lord, C. (2001). NAEP math performance and test accommodations: Interactions with student language background (CSE Technical Report 536). Los Angeles, CA: The National Center for Research on Evaluation, Standards, and Student Testing. Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman Publishing. Blum-Kulka, S., & Snow, C. E. (2004). Introduction: The potential of Peer talk. Discourse Studies, 6(3), 291–306. https://doi.org/10.1177/1461445604044290. Bristol, T. & Brett, A. L. (2015) Test item writing: 3Cs for successful tests. Teaching and Learning in Nursing, 10(2), 100–103. https://doi.org/10.1016/j.teln.2015.01.004. Cekaite, A., Blum-Kulka, S., Grøver, V., & Teubal, E. (Eds.). (2014). Children’s peer talk: Learning from each other. Cambridge: Cambridge University Press.

Assessing multilinguals 45 Cenoz, J., Arocena, E., & Gorter, D. (2013). Multilingual students and their writing skills in Basque, Spanish and English. In V. Gathercole (Ed.), Bilingual assessment: Issues (pp 186–205). Bristol: Multilingual Matters. Cook, V. (2009). Contemporary applied linguistics: Language teaching and learning. London: Continuum Publishing group. Council of Europe (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Press Syndicate of the University of Cambridge. Council of Europe (2007). From Linguistic Diversity to Plurilingual Education. Retrieved 8 September 2021 from https://rm.coe.int/16802fc1c4. Council of Europe (2020). Common European Framework of Reference for Languages. Learning, Teaching, Assessment. Companion volume. Retrieved 8 September 2021 from https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989. De Angelis, G. (2021). Multilingual testing and assessment. Bristol: Multilingual Matters. De Backer, F., Van Avermaet, P., & Slembrouck, S. (2017). Schools as laboratories for exploring multilingual assessment policies and practices, Language and Education, 31:3, 217–230. https://doi.org/10.1080/09500782.2016.1261896). Dendrinos, B. (2019). Multilingual testing and assessment for plurilingual education. “MultiTest” ECSPM position paper. Retrieved 8 June 2020 from http://ecspm. org/wp-content/uploads/2019/03/MultiTest.pdf. Donahue, T. (2021). Linguistic and social diversity, literacy and access to higher education. In E. Breuer, E. Lindgren, A. Stavans, & E. Van Steendam (Eds.), Multilingual literacy (pp 21–39). Bristol: Multilingual Matters. El Euch, S. (2011). Plurilingualism: What is known and what is still to be known. In Proceedings of INTED2011 Conference (pp 1391–1400). Escamilla, K., Butvilofsky, S., & Hopewell, S. (2018). What gets lost when English-only writing assessment is used to assess writing proficiency in Spanish-English emerging bilingual learners? International Multilingual Research Journal, 12(4), 221–236. https://doi.org/10.1080/19313152.2016.1273740. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31(3), 231–248. https://doi.org/10.1080/0950 0782.2016.1261892. Hönig, I. (2010). Assessment in CLIL: Theoretical and empirical research. Saarbrücken, Germany: VDM Verlag Dr. Müller. https://doi.org/10.1191/1362168805lr173oa. Kim, S. H. O., & Elder, C. (2005). Language choices and pedagogic functions in the foreign language classroom: A cross-linguistic functional analysis of teacher talk. Language Teaching Research, 9(4), 355–380. Kirsch, C. (2021). Promoting multilingualism and multiliteracies through storytelling: A case study on the use of the app iTEO in preschools in Luxembourg. In E. Breuer, E. Lindgren, A. Stavans, & E. Van Steendam (Eds.), Multilingual literacy (pp 187–210). Bristol: Multilingual Matters. Kramsch, C. (2009). The multilingual subject. Oxford: Oxford University Press. Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2. Lo, Y. Y., & Fung, D. (2018). Assessments in CLIL: The interplay between cognitive and linguistic demands and their progression in secondary education. International Journal of Bilingual Education and Bilingualism, 1–19. https://doi.org/10.1080/ 13670050.2018.1436519.

46  Gessica De Angelis and Anat Stavans Lopez, A. A., Turkan, S., & Guzman-Orth, D. (2017). Assessing multilingual competence. In Language Testing and Assessment. Encyclopedia of Language and Education (pp 91–102). Cham: Springer. Maher, J. C. (2017). Multilingualism: A very short introduction. Oxford: Oxford University Press. Martiniello, M. (2008). Language and the performance of English-language learners in math word problems. Harvard Education Review, 78(2), 333–68. Massler, U., Stotz, D., & Queisser, C. (2014). Assessment instruments for primary CLIL: The conceptualisation and evaluation of test tasks. The Language Learning Journal, 42(2), 137–150. https://doi.org/10.1080/09571736.2014. 891371. Milani, T. (2008). Language testing and citizenship: A language ideological debate in Sweden. Language. in Society, 37(1), 27–59. Doi:10.1017/S0047404508080020 Molina, C. (2011). Curricular insights into translingualism as a communicative competence. Journal of Language Teaching and Research, 2(6), 1244–1251. OECD (2010). TALIS 2008 technical report, TALIS, Paris: OECD Publishing. https://doi.org/10.1787/9789264079861-en. OECD (2012). PISA 2009 technical report, PISA, Paris: OECD Publishing. https:// doi.org/10.1787/9789264167872-en. Ortiz, S. O., Ochoa, S. H., & Dynda, A. M. (2012). Testing with culturally and linguistically diverse populations: Moving beyond the verbal-performance dichotomy into evidence-based practice. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp 526–552). New York: The Guilford Press. Otheguy, R., García, O., & Reid, W. (2015). Clarifying translanguaging and deconstructing named languages: A perspective from linguistics. Applied Linguistics Review, 6(3), 281–307. Otto, A., & Estrada, J. L. (2019). Towards an understanding of CLIL in a European context: Main assessment tools and the role of language in content subjects. CLIL Journal of Innovation and Research in Plurilingual and Pluricultural Education, 2(1), 31–42. https://doi.org/10.5565/rev/clil.11. Poza, L. (2017). Translanguaging: Definitions, implications, and further needs in burgeoning inquiry. Berkeley Review of Education, 6(2). https://dx.doi.org/10.5070/ B86110060. Rhodes, R. L., Ochoa, S. H., & Ortiz, S. O. (2005). Assessing culturally and linguistically diverse students. A practical guide. New York: The Guildford Press. Rix, J. (2009). A model of simplification: The ways in which teachers simplify learning materials. Educational Studies, 35(2), 95–106. Shohamy, E. (2007). Language tests as language policy tools. Assessment in Education, 14(1), 117–130. https://dx.doi.org/10.1080/09695940701272948. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95, 418–429. https://dx.doi. org/10.1111/j.1540-4781.2011.01210.x. Stavans, A. (2015). Enabling bi-literacy patterns in Ethiopian immigrant families in Israel: A socio-educational challenge. International Journal of Multilingualism, 12(2), 178–195. Stavans, A., & Hoffmann, C. (2015). Multilingualism. Cambridge: Cambridge University Press.

Assessing multilinguals 47 Stavans, A., Tahar-Eden, M., & Azar, L. (2021). 11 multilingual literacy: The use of Emojis in written communication. In E. Breuer, E. Lindgren, A. Stavans, & E. Van Steendam (Eds.), Multilingual literacy (pp 233–259). Bristol: Multilingual Matters. Szabo, C. Z., Stickler, U., & Adinolfi, L. (2021). Predicting the academic achievement of multilingual students of English through vocabulary testing. International Journal of Bilingual Education and Bilingualism, 24(10), 1531–1542. https://dx.doi. org/10.1080/13670050.2020.1814196. Takahashi-Breines, H. (2002). The role of teacher-talk in a dual language immersion third grade classroom. Bilingual Research Journal, 26(2), 461–483. Zarate, G., Lévy, D., & Kramsch, C. (Eds.) (2008). Précis du plurilinguisme et du pluriculturalisme. Paris: Editions des Archives Contemporaines.

2

Latinx bilingual students’ translanguaging and assessment A unitary approach Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López

Introduction All over the world bilingual students are assessed through monolingual tests that do not capture all that they know and can do. The difference between how language is constructed in school assessments and the ways in which students do language is substantial, and this is exacerbated when racialized bilingual children are assessed in one or even two separate languages. Bilingual children’s languaging does not fit neatly within the construction of language or bilingualism that schools have adopted, producing their minoritization and the resulting so-called ‘achievement gap.’ In this chapter we look at how translanguaging theory disrupts and questions the validity of monolingual and even bilingual assessments; and we consider what would be needed to make them just and appropriate for U.S. Latinx bilingual children. First, we use the term Latinx as a gender-neutral term instead of Latino or Latina. When we refer to Latinx bilinguals, we refer to the broad and diverse populations of Latinx individuals in the United States. The Latinx population in the United States is defined by its complexity. Latinx people from the United States can be indigenous to the United States, immigrants from many countries, children of immigrants, and combinations of these, and thus be multiethnic and multiracial. Their relationship to bilingualism is also complex, as Latinx people in the United States may speak a number and a combination of named languages such as English, Spanish, and other named languages indigenous to the Americas. Specifically, when we refer to racialized Latinx or emergent bilinguals, we refer to students who have been racialized by society because of their skin color or race. We argue that translanguaging theory is especially important for the assessment of racialized bilingual students like U.S. Latinxs. Because of misunderstandings about bilingualism, school authorities often rely on assessments that are said to be bilingual, but these assessments are merely the addition of two monolingual assessments. True, assessments in two languages, in English and Spanish, give us a fuller picture of a bilingual child than assessment in one language only. However, these bilingual assessments fail to meet the bilingual child in the borderlands and entre mundos in which they live (Anzaldúa, 1987) DOI: 10.4324/9781003177197-4

Latinx bilingual students’ translanguaging and assessment 49 because they rely on an additive definition of bilingualism. They also do not reflect a way of languaging and living that is, as Anzaldúa (1987) says: ‘Neither eagle nor serpent, but both. And like the ocean, neither animal respects borders’ (p. 84). How then can assessment be developed to work with the unitary repertoire of bilinguals, with the bilinguals’ translanguaging, which does not respect the artificial boundaries of named languages constructed by nation-states and policed in schools? Informed by the decolonial lens that has shaped translanguaging theory, we question traditional positions on language and bilingualism in society and schools and discuss what this means for assessments. We then describe ways of doing assessment ‘otherwise’ informed by translanguaging, and the import of these assessments for the education and lives of all bilingual students, and in particular racialized emergent bilinguals. Based on the work of Ascenzi-Moreno with reading assessments, we consider how translanguaging theory transforms how teachers can assess bilingual children’s reading more accurately and justly. We then describe the efforts of López and his colleagues at Educational Testing Service to develop standardized content assessments taking bilingual students’ translanguaging into account. We end by considering the reasons why these types of assessments, rooted on the bilingual children’s ways of knowing and languaging, can transform the ideologies generally held about them, as well as the students’ own subjectivities as bilingual Latinx children. Language and raciolinguistic ideologies We follow Latin American decolonial theorists (see, e.g., Menezes de Souza, 2007; Mignolo, 2000) in claiming that the way in which we conceptualize language today is a product of colonialism. That is, at the point of the encounter in the Americas, domination of Indigenous Americans was achieved not only by biologizing race, but also naturalizing ‘Castilian.’ Indigenous Americans were described as ‘non-human’ or ‘less-than-human’ (Veronelli, 2015), and thus languageless. This was the same process used later by Anglo-Saxons in the creation and expansion of the United States, as Africans were enslaved and the idea of ‘Manifest Destiny’ justified the forced removal of Native Americans and the taking over of Mexican territory. Race and language were again used to dismiss the humanity of the Mexicans in the territory, said to be ‘Indian’ or of ‘mixed tribe’ (Weber, 1973) and speaking ‘a sort of jargon of their own’ (cited in Nieto-Phillips, 2004, p. 88). Named as Castilian and English, the socalled languages were reserved for white Europeans. The process of domination and oppression based on language and racial hierarchies has continued today through what the Peruvian sociologist Aníbal Quijano (2000) has called coloniality, exclusively legitimizing knowledge and language systems as those of white European males. This is a very different concept of language from that of Chilean biologists Humberto Maturana and

50  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López Francisco Varela (1984) who coined the term ‘languaging’ to refer to the act by which all human beings are constituted, as they communicate, interact, and respond to the reception of others. U.S. Latinx have been racialized and deemed languageless, a product of raciolinguistic ideologies (Flores & Rosa, 2015; Rosa, 2019; Rosa & Flores, 2017). Assessments rendered in what has been normalized as ‘standard language’ have been important tools to validate these ideologies and to naturalize them in ways that we are hardly aware of them. Bilingualism and translanguaging Rooted in the same raciolinguistic ideologies, bilingualism was conceptualized as simply the addition of two autonomous language entities that correspond to the ways in which racially and socially dominant monolinguals use their language. Thus, the ways in which racialized bilinguals do language was looked down upon, as linguists went about finding ‘contact,’ ‘interferences,’ and describing divergence from monolingual patterns of speech in the form of ‘loans,’ ‘calques,’ and ‘code-switching.’ The thinking was that the bilingual speech of these racialized bilinguals was ‘incomplete’ (Montrul, 2008). Assessments of Latinx students in English and/or Spanish always confirmed their linguistic inadequacy, even when efforts were made to assess their linguistic performances in English and Spanish. Increasingly, however, scholars pointed to the more dynamic, and not simply additive, language practices of bilinguals (García, 2009), developing translanguaging theory. Translanguaging centers the ways in which racialized bilinguals do language and do their bilingualism with a unitary repertoire that does not reflect dual separate linguistic systems or that has a dual psycholinguistic correspondence (García & Li, 2014; Otheguy et al., 2015, 2019). Latinx bilinguals assemble and orchestrate different features and resources to do language (Li, 2017). In so doing, they construct a network of expanded signs and meanings that goes beyond, and is different from, that of monolingual English or Spanish speakers. Translanguaging not only goes beyond the concept of named languages, but also beyond the construct of oral or written language, encompassing the trans-semiotizing aspects of sense-making (Lin, 2019). All language is a semiotic system encompassing not only the verbal, but also other multimodal resources such as visuals, gestures, bodily movement, and the deployment of other objects (Moore et al., 2020). Translanguaging and assessment Translanguaging theory reframes assessments for all, insisting that there is a distinction between the child’s ability to language, and the specific features of the child’s verbal language. The ability to do language consists of whether the child can language effectively for different purposes—to communicate, to

Latinx bilingual students’ translanguaging and assessment 51 argue, to narrate, to tell a joke, to write different genres, to express content knowledge, etc. These language performances can be assessed regardless of the specific language features that the child uses. Distinguishing between general language performances and the use of specific language features that have been constructed as standard is important (García et al., 2017). Assessments that are just for Latinx bilingual children must focus on assessing the use of language, the languaging, in the sense given to us by Maturana and Varela. But many assessment tools today evaluate the children’s use of the specific features of language that have been identified as consisting of the ‘standard,’ only because they match those used by the powerful in society. In so doing, racialized speakers are penalized for drawing on features that are prevalent in their repertoire, despite not being shared by the dominant communities that produce the tests. Schools, and the curricula that they follow, remain deeply tied to the idea that language is a monoglossic entity that could be taught. And assessments have been the main tool to enforce and police linguistic behavior. By upholding only one language standard, the heteroglossic nature of language is dismissed (Bakhtin, 1981). But more importantly, these assessments then produce the failure of all those whose languaging is not contained by what has been constructed as the ‘correct, standard, academic’ language. Assessments grounded in a monoglossic perspective do not fully capture the knowledge and abilities that bilingual students possess and, even if these are bilingual assessments, may maintain the power hierarchy between different named languages (Abedi, 2011; Ascenzi-Moreno, 2018; Schissel, 2020; Shohamy, 2011; Solano-Flores, 2011). Translanguaging pedagogical practices have been often considered a scaffolding mechanism for bilingual students, and they can play a pivotal role to help them access and engage in curriculum. However, when conceived exclusively in this way, the transformative potential of translanguaging pedagogical practices is muted because it does not challenge teachers’ thinking about the ways in which their racialized bilingual students do language with a unitary repertoire. Some transformational uses of translanguaging have taken root in education (see, e.g., CUNY-NYSIEB, 2021; Fu et al., 2019; García & Kleyn, 2016; García et al., 2017). Translanguaging pedagogical practices in literacy instruction (España & Herrera, 2020; Espinosa & Ascenzi-Moreno, 2021), as well as in content instruction (Pierson et al., 2021; Poza, 2018), have been developed and explored. However, the relationship between translanguaging and assessment has been limited. In this chapter, we first consider literacy and assessment before we turn to assessment of content taking into consideration the unitary repertoire of bilinguals. We describe current efforts to develop assessments in these areas that are rooted in understandings of translanguaging to meet Latinx bilingual students where they exist with their own languaging—in historical/cultural borderlands which only make sense to them by drawing on their unitary semiotic repertoire, on their translanguaging.

52  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López Literacy, assessment, and bilingual students Current conceptualizations of reading emphasize that it is a process that not only involves language, social participation, cultural membership, and identity negotiation, but is also centered in the person and transcends language boundaries (Ascenzi-Moreno & Seltzer, 2021, García, 2020; García & Kleifgen, 2019; Kabuto, 2018). This definition of reading focuses on the dynamism of lived experience and thus emphasizes that when emergent bilinguals interact with text they do not do so in a piecemeal fashion, but rather in an integrated and unified way, merging their linguistic/semiotic resources to make sense of text. García (2020) argues that the focus of reading instruction must shift from the monolingual text to the bilingual student. She writes, ‘the monolingual text with supposedly static linguistic features is transformed and mobilized by bilingual readers who bring their entire selves—their language, with its multilingualism and multimodalities; their emotions; their bodies; and their lives—into the text’ (p. 562). This shift toward centering literacy practices in the doing and being of the bilingual person must be mirrored in the assessment policies and practices which assess literacy. Formative literacy assessment is touted as a powerful way to learn about students’ reading and thus support instruction, yet the ways in which teachers carry out formative assessments are often counter to this very intention. Rather than being instruments which support students’ literacy development, regularly the main function of literacy assessment is to report levels and sort and group students (Ascenzi-Moreno, 2016). But even when students are assessed in English and another language, monoglossic assessment practices do not allow teachers to understand students’ reading as they draw from their unitary repertoire. This mismatch between the intended purpose of assessment and actual implementation is particularly harmful for Latinx bilingual students (Ascenzi-Moreno & Seltzer, 2021). Running records is an example of formative literacy assessments that are widely used by classroom teachers. These assessments are used to glean an understanding of how students employ the complex and intertwined ‘pieces’ of the reading process—decoding, use of reading strategies, fluency, among others—within an authentic reading performance. Because this type of assessment is grounded in the experience of reading, it holds the potential for teachers to learn holistically about students as readers and knowledge-producers. Reading assessments are often posed as neutral instruments; yet they are always rooted in theories of reading and ideologies about readers which are far from neutral. Current assessments are not only tethered to a monolingual framework, but also prioritize print over multimodal ways to engage in literacy (Serafini et al., 2020). This monolingual and monomodal foundation for assessment is problematic for all students because reading is increasingly considered to be contextualized, embodied, and emplaced, and therefore not a stable set of skills that are acquired through standardized procedures (Ascenzi-Moreno

Latinx bilingual students’ translanguaging and assessment 53 & Seltzer, 2021; Compton-Lily et al., 2020). Literacy assessments grounded on translanguaging theory would consider the multilingual, multimodal, and multidimensional contextual nature of reading as bilingual readers draw from their unitary repertoire. A translanguaging perspective on literacy assessment for bilingual students In speaking about how to conduct literacy research with bilingual students, García and Kleifgen (2019) say that, ‘instead of confining the literacy act to mechanical aspects of engaging with a printed static text, scholars [need to] be comfortable with texts and bodies in movement, in the moment-by-moment interaction with multiple signs and objects in unexpected ways, and in ways that go beyond the restricted ways in which institutions have defined them’ (p. 561). Likewise, assessment of literacy needs to shift its focus from simply looking at the mechanics of how students engage with a printed text to how they make meaning in ways that are dynamic, shifting, interactive, and bodily and that are not bounded by a formal static text. Traditional reading assessments asks students to demonstrate their understandings either writing or speaking. But a translanguaging perspective would go beyond these modes, allowing students to retell by dramatizing, drawing, gesturing, and using any aspect of their linguistic and semiotic repertoires. Again, a translanguaging framework in literacy assessment would acknowledge and leverage the bilingual students’ unitary repertoire, reflecting the ways in which bilingual readers make sense of texts. Within reading assessments in classrooms, teachers can give students choices about how they would like to respond—choosing one or a combination of modalities rather than restricting students to one modality or one language. A few studies have explored how literacy assessment can consider bilingual students’ complex and unitary linguistic repertoire. For example, Briceño and Klein (2018) argue for the inclusion of a ‘second lens’ that allows teachers to determine how students’ reading miscues may be the result of language-related approximations, or ‘reading errors that are attributable to readers’ language’ (p. 3). While their research is not explicitly rooted in a translanguaging lens, they emphasize that bilingual students’ reading is intrinsically tied to their linguistic repertoire. Bauer et al. (2020) also demonstrate that when students’ translanguaging is part of the reading assessment process, the complexity of students’ linguistic practices become more apparent. This sheds light on how bilinguals make meaning of text. In working with running record assessments, Ascenzi-Moreno (2018) introduced the concept of responsive adaptations, which are flexible ways of adapting this assessment to consider students’ translanguaging. Through responsive adaptations, bilinguals’ dynamic reading practices are considered the norm. As students interact with text, they do so in a way that cuts across named language boundaries (Kabuto, 2017). For example, when a Latinx bilingual

54  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López student is thinking about a text or responding to it, they may draw on their entire semiotic repertoire, as well as experiences that they have. Miscues are then perceived not simply from a monolingual base, but from one that considers the bilinguals’ unitary languaging and reading process. This stands in contrast to the common assumption that guides literacy assessment, that, for example, if you read a book in English, you should respond to it in English, or of traditional biliteracy assessment that requires performances in one language at a time (see Ascenzi-Moreno, 2018 for more detail on responsive adaptations). Some teachers are beginning to engage with translanguaging as they conduct formative assessments of Latinx bilingual children’s biliteracy. Take, for instance, Abby, a monolingual English-speaking teacher of English as a Second Language, who works with Emilia, a 5th grader from Honduras. Abby wants to know what Emilia can do as a reader. For this reason, Abby invites Emilia to respond to a text she has read in English using whatever languaging she wishes. In school terms, this means that Abby invites Emilia to respond in English, Spanish, in a combination of both, orally, in writing, by drawing, using images, acting and gesturing. In changing the way that the assessment is typically administered, Abby provides Emilia with the opportunity to use her full linguistic/semiotic repertoire in responding to the text. Through a translanguaging view of reading, movement across named languages and modalities is fluid. This is one type of responsive adaptation which shifts the assessment instrument to consider translanguaging. It is true that this type of formative assessment of reading takes time. But it gives Abby a window into what Emilia understands when she reads a text in English. The alternative of requiring Emilia to respond to the text in English only would only mean that Abby would have limited information about Emilia as a person, a thinker, a reader, a language user, a knowledge producer. It is also true that this type of assessment requires teachers to go beyond what is required. Abby will have to use Google Translate and consult with bilingual teachers in the school to gain complete understandings of Emilia’s performances. But in so doing, Abby will be performing the only role of a teacher of Latinx bilingual students that is ethical—understanding, as well as possible, what the children know and are able to do so that instruction can meet their gifts, as well as their needs. Whereas assessments produced and performed by teachers are an integral part of instruction, teachers are also handed down standardized assessments by school systems. The next section considers standardized content assessments and how they may better engage with the translanguaging of their bilingual Latinx students. Content assessments and translanguaging In the United States students are typically required to demonstrate their content knowledge in what is deemed to be ‘Standard English’ (Escamilla, 2006; García, 2009; López et al., 2015; Shohamy, 2006). The interpretation of

Latinx bilingual students’ translanguaging and assessment 55 those scores, especially for those Latinx emergent bilinguals who fall at the beginning point of the bilingual continuum, is simply not valid (Abedi, 2006; López et al., 2017). Educators are finding ways to give bilingual students greater opportunities to demonstrate what they know and can do (Roohr & Sireci, 2017). For emergent bilingual students at the beginning end of the bilingual continuum, accommodations are often provided. Teachers can translate the test or questions or words; they can also allow students to use bilingual dictionaries or glossaries; and they can add other supports when needed (e.g., pictures, graphic organizers). Some teachers can read the questions that have been posed in English in Spanish, or they can read the questions in English and Spanish. Furthermore, teachers can allow students to demonstrate their content knowledge in English or Spanish. Although these bilingual accommodations are important, there are challenges to implementing them. Using, scoring, and interpreting bilingual content assessments might be challenging for some teachers, especially when they do not share the same linguistic resources and practices as their students. This could be mitigated by using multiple-choice questions, using multilingual translation digital applications to translate the students’ answers, or using performance-based assessments that require students to produce something (e.g., projects, experiments). Although good first steps, these efforts of bilingual accommodations fall short because they rest on understandings of standard language as an autonomous entity and bilingualism as additive. Recently scientists and psychometricians involved in construction of content assessments have started to develop assessments so that bilingual students can use all their available linguistic and semiotic resources to demonstrate their knowledge and abilities in different content areas (e.g., mathematics and science). This is the work that López has been leading for Educational Testing Service (López, 2020; López et al., 2019). The idea is to enable linguistically adaptive bilingual practices within a single assessment context (Shohamy, 2011) in order to allow students to use different semiotic resources, enabling them to perform in writing, orally or graphically (Li, 2011). With translanguaging as its theoretical underpinning, these assessments allow bilingual students to use their full unitary repertoire, drawing from all available language and semiotic resources if they want to. These content assessments incorporate the use of embedded bilingual accommodations or bilingual supports that are always available to the students and can be used whenever needed (López et al., 2017). The main goal of the embedded bilingual supports is to ensure that the English language demands of the items in the content assessment may not interfere with the students’ ability to demonstrate what they know in science or mathematics, regardless of their language proficiency. Drawing on understandings of bilingualism that do not simply view the two languages as representing two very different knowledge systems, students are allowed to use their available linguistic (e.g., English, Spanish or both) and other semiotic resources (oral and written language) to help them

56  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López

Figure 2.1  Sample science item illustrating how translanguaging is acknowledged.

understand the language of the assessments and to help demonstrate what they know and can do in a content area. Figure 2.1 shows an example of a content assessment with bilingual supports delivered on an online platform meant to assess the science understandings of students in 6th through 8th grade. The figure displays the English language version, although a Spanish language version is also available. The bilingual supports are always accessible so students can use them at any time, but they are not required to use any of them. In this English language version, initially, the students will always view the items in English, but then they can click on the language tab (Español) to view the item in Spanish; they can toggle back and forth between the two languages at any time. For constructed response questions, students are allowed to write or orally record their responses using their full repertoire. They can toggle to an oral response by clicking on the microphone icon; each response is recorded separately. In Figure 2.1, we see that a few words are glossed in the English or Spanish tab (underlined words). If students hover over the glossed words, they can see synonyms or a picture for these words. This support does not apply to content-related terminology because understanding the meaning of these terms is usually considered to be part of what the content assessments are intended to measure. Finally, students can click on the picture of the avatar to listen to someone read aloud the directions and the questions in English and/or Spanish depending on the language tab they select. Only the text in the items is read aloud; symbolic and visual representations (e.g., science and symbols, models, figures, and tables) are not read aloud because understanding and using them is part of what these content assessments are intended to measure. The same supports are present in both language versions.

Latinx bilingual students’ translanguaging and assessment 57 Since students have the freedom to perform language how they want, leveraging their full linguistic and multimodal repertoire. A conceptual scoring model is used to score the responses. Conceptual scoring allows scoring a response without regard to the language or mode in which the response is given (Barrueco et al., 2012). When using conceptual scoring, the same scoring rubric is implemented regardless of the language or mode that is used. Since online bilingual content assessments use embedded bilingual supports, it is possible that students require additional time to complete them. Thus, students should be provided with ample time to use all their linguistic resources and language modes. It is important to understand how these standardized content assessments are drawing from translanguaging theory. Because they are standardized assessments delivered on an online platform and machine scored, the instruments themselves differentiate between English and Spanish, and written and oral language. Despite what some may still perceive as limitations, these assessments allow Latinx bilingual students to show what they know by acknowledging their unitary repertoire and the ways in which they do language. Besides providing ways of assessing bilingual students that are more just and inclusive than what we presently have, these content assessments could have a formative function, allowing teachers to determine not only what students know, but how they express those understandings, and the types of supports they need. At the same time, these content assessments could be used as end of instruction summative assessments to gather evidence of how well students are meeting their learning objectives in different content areas. Conclusion Solano-Flores (2011, p. 3) argues that assessments are ‘cultural artifacts,’ and thus the site of political and social forces which deeply affect the students who take them. In general, teachers are not made aware of how bilingual students draw from a unitary language repertoire to make meaning, and of the consequences for their education if this different knowledge system is not acknowledged. Consequently, teachers do not develop a critical take on assessments, even classroom-based ones. Teachers’ learning experiences about assessment are focused on how they are administered and reported, and not on what they truly tell us about bilingual students. Teachers can be in the forefront of moving the field toward assessments which more accurately capture the knowledge and language system of bilinguals, thus making them more equitable. Making room for bilingual students’ translanguaging provides teachers with a more accurate picture of what students know and can do. Whereas shifts such as responsive adaptations are promising and a step in the right direction, when these assessments are adapted, they remain rooted in monolingual frameworks. Another less explored opportunity is related to the efforts described in this chapter, transforming assessments so

58  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López that they emerge from a translanguaging framework that taps and includes bilingual students’ unitary repertoire. The importance of this shift in the epistemes by which assessments are developed is obvious. Willingly or unwillingly, assessments have had the effect of producing and justifying the gaps in educational opportunities between the children of the dominant U.S. population, and those who have been racialized and rendered languageless. In the case of bilingual Latinx children, monolingual or bilingual assessments based on monoglossic concepts of language and additive bilingualism have been responsible for their being left behind. With little or inaccurate and invalid information, educational decisions are made about Latinx bilingual children’s capacities that render them not only nilingües (neither speakers of English nor speakers of Spanish) as teachers often say, but also cognitively deficient and culturally impoverished. Only by developing assessments that enable Latinx bilingual children to draw from their full linguistic/semiotic repertoire will Latinx bilingual children be judged fairly, assessed not in comparison to monolingual middle-class standards of language use and their knowledge/cultural system, but as true producers of their own knowledge, expressed through their own languaging. References Abedi, J. (2006). Psychometric issues in the ELL assessment and special education eligibility. Teachers College Record, 108(11), 2282–2303. Abedi, J. (2011). Assessing English language learners: Critical issues. In M. del Rosario Bastera, E. Trumbull, & G. Solano-Flores (Eds.), Cultural validity in assessment: Addressing linguistic and cultural diversity (pp 49–71). New York: Routledge. Anzaldúa, G. (1987, 2nd ed. 1999) Borderlands/La Frontera: The new mestiza. San Francisco, CA: Aunt Lute Book. Ascenzi-Moreno, L. (2016). An exploration of elementary teachers’ views of informal reading inventories in dual language bilingual programs. Literacy Research and Instruction, 55(4), 285–308. https://doi.org/10.1080/19388071.2016.1165318. Ascenzi-Moreno, L. (2018). Translanguaging and responsive assessment adaptations: Emergent bilingual readers through the lens of possibility. Language Arts, 95(6), 355–369. Ascenzi-Moreno, L., & Seltzer, K. (2021). Always at the bottom: Ideologies in assessment of emergent bilinguals. Journal of Literacy Research, 53(4), 468–490. Bakhtin, M. (1981). The dialogical imagination. Austin, TX: University of Texas Press. Barrueco, S., López, M., Ong, C., & Lozano, P. (2012). Assessing Spanish-English bilingual preschoolers: A guide to best approaches and measures. Baltimore, MD: Paul H. Brookes. Bauer, E. B., Colomer, S. E., & Wiemelt, J. (2020). Biliteracy of African American and Latinx kindergarten students in a dual-language program: Understanding students’ translanguaging practices across informal assessments. Urban Education, 55(3), 331–361. https://doi.org/10.1177/0042085918789743. Briceño, A., & Klein, A. (2018). A second lens of formative reading assessment with multilingual students. The Reading Teacher, 72(5), 611–621. https://doi.org/10.1002/ trtr.1774.

Latinx bilingual students’ translanguaging and assessment 59 City University of New York – New York State Initiative on Emergent Bilinguals (CUNY-NYSIEB) (2021). Translanguaging and transformative teaching for emergent bilingual students: Lessons from the CUNY-NYSIEB project. Oxfordshire: Routledge. Compton-Lilly, C., Mitra, A., Guay, M., & Spence, L. (2020). A confluence of complexity: Intersections among reading theory, neuroscience, and observations of young readers. Reading Research Quarterly. https://doi.org/10.1002/rrq.348. Escamilla, K. (2006). Monolingual assessment and emerging bilinguals: A case study in the US. In O. García, T. Skutnabb-Kangas, & M. E. Torres-Guzman (Eds.), Imagining multilingual schools (pp 184–199). Bristol: Multilingual Matters. España, C., & Herrera, L. (2020). En Comunidad. Lessons for centering the voices and experiences of bilingual Latinx students. Portsmouth, NH: Heinemann. Espinosa, C., & Ascenzi-Moreno, L. (2021). Rooted in strength. Using translanguaging to grow multilingual readers and writers. New York: Scholastic. Flores, N., & Rosa, J. (2015). Undoing appropriateness: Raciolinguistic ideologies and language diversity in education. Harvard Education Review, 85(2), 149–171. https://doi.org/10.17763/0017-8055.85.2.149. Fu, D., Hadjioannou, X., & Zhou, X. (2019). Translanguaging for emergent bilinguals: Inclusive teaching in the linguistically diverse classroom. New York: Teachers College Press. García, O. (2009). Bilingual education in the 21st century: A global perspective. Hoboken, NJ: Wiley-Blackwell. García, O. (2020). Translanguaging and Latinx bilingual readers. The Reading Teacher, 73(5), 557–562. https://doi.org/10.1002/trtr.1883. García, O., Johnson, S. I., & Seltzer, K. (2017). The translanguaging classroom: Leveraging student bilingualism for learning. Philadelphia, PA: Caslon. García, O., & Kleifgen, J. A. (2019). Translanguaging and literacies. Reading Research Quarterly, 55(4), 1–19. https://doi.org/10.1002/rrq.286. García, O., & Kleyn, T. (Eds.) (2016). Translanguaging with multilingual students: Learning from classroom moments. Oxfordshire: Routledge. García, O., & Li, W. (2014). Translanguaging: Language, bilingualism and education. New York: Palgrave Macmillan. Kabuto, B. (2017). A socio-psycholinguistic perspective on biliteracy: The use of miscue analysis as a culturally relevant assessment tool. Reading Horizons, 56(1), 25–43. Kabuto, B. (2018). Becoming a bilingual reader as linguistic and identity enactments. Talking Points, 29(2), 11–18. Li, W. (2011). Moment analysis and translanguaging space: Discursive construction of identities by multilingual Chinese youth in Britain. Journal of Pragmatics, 43, 1222–1235. Li, W. (2017). Translanguaging as a practical theory of language. Applied Linguistics, 39(1), 9–30. https://doi.org/10.1093/applin/amx039. Lin, A. M. Y. (2019). Theories of trans/languaging and trans-semiotizing: Implications for content-based education classrooms. International Journal of Bilingual Education and Bilingualism, 22(1), 5–16. https://doi.org/10.1080/13670050.2 018.1515175. López, A. A. (2020). Examining how Spanish-speaking English language learners use their linguistic resources and language modes in a dual language mathematics assessment task. Journal of Latinos and Education. https://doi.org/10.1080/1534843 1.2020.1731693.

60  Laura Ascenzi-Moreno, Ofelia García, and Alexis A. López López, A. A., Guzman-Orth, D. A., & Turkan, S. (2015). How might a translanguaging approach in assessment make tests more valid and fair for emergent bilinguals? In G. Valdés, K. Menken, & M. Castro (Eds.), The common core and English language learners/emergent bilinguals: A guide for all educators (pp 266–267). Philadelphia, PA: Caslon. López, A. A., Guzman-Orth, D., & Turkan, S. (2019). Exploring the use of translanguaging to measure the mathematics knowledge of emergent bilingual students. Translation and Translanguaging in Multilingual Contexts, 5(2), 143–164. https://doi.org/10.1075/ttmc.00029.lop. López, A. A., Turkan, S., & Guzman-Orth, D. (2017). Conceptualizing the use of translanguaging in initial content assessments for newly arrived emergent bilingual students (Research Report No. RR-17-07). Educational Testing Service. https:// doi.org/10.1002/ets2.12140. Maturana, H., & Varela, F. (1984). El Árbol del conocimiento. Las bases biológicas del entendimiento humano. Buenos Aires: Lumen, Editorial Universitaria. Menezes de Souza, L. M. T. (2007). Entering a culture quietly: Writing and cultural survival in indigenous education in Brazil. In S. Makoni, & A. Pennycook (Eds.), Disinventing and reconstituting languages (pp 135–169). Bristol: Multilingual Matters. Mignolo, W. (2000). Local histories/global designs: Essays on the coloniality of power, subaltern knowledges and border thinking. Princeton, NJ: Princeton University Press. Montrul, S. (2008). Incomplete acquisition in bilingualism. Amsterdam: John Benjamins. Moore, E., Bradley, J., & Simpson, J. (Eds.) (2020). Translanguaging as transformation: The collaborative construction of new linguistic realities. Bristol: Multilingual Matters. Nieto-Phillips, J. (2004). The language of blood: The making of Spanish-American identity in New Mexico. 1880s-1930. Albuquerque, NM: University of New Mexico Press. Otheguy, R., García, O., & Reid, W. (2015). Clarifying translanguaging and deconstructing named languages: A perspective from linguistics. Applied Linguistics Review, 6(3), 281–307. https://doi.org/10.1515/applirev-2015-0014 Otheguy, R., García, O., & Reid, W. (2019). A translanguaging view of the linguistic system of bilinguals. Applied Linguistics Review, 10(4), 625–651. https://doi. org/10.1515/applirev-2018-0020. Pierson, A., Clark, D., & Brady, C. (2021). Scientific Modeling and Translanguaging: A multilingual and multimodal approach to support science learning and engagement. Science Education, 1–38. https://doi.org/10.1002/sce.21622. Poza, L. E. (2018). The language of ciencia: Translanguaging and learning in a bilingual science classroom. International Journal of Bilingual Education and Bilingualism, 21(1), 1–19. Quijano, A. (2000). Coloniality of power. Ethnocentrism, and Latin America. NEPANTLA, 1(3), 533–580. Roohr, K. C., & Sireci, S. G. (2017). Evaluating computer-based test accommodations for English learners. Educational Assessment, 22(1), 35–53. Rosa, J. (2019). Looking like a language, sounding like a race: Raciolinguistic ideologies and the learning of Latinidad. Oxford: Oxford University Press. Rosa, J., & Flores, N. (2017). Unsettling race and language: Toward a raciolinguistic perspective. Language in Society, 46(5), 621–647. https://doi.org/10.1017/ S0047404517000562.

Latinx bilingual students’ translanguaging and assessment 61 Schissel, J. L. (2020). Moving beyond deficit positioning of linguistically diverse test takers: Bi/multilingualism and the essence of validity. In S. Mirhosseini & P. I. De Costa (Eds.), Sociopolitics of English language testing (pp 91–108). London: Bloomsbury Publishing. Serafini, F., Moses, L., Kachorsky, D., & Rylak, D. (2020). Incorporating multimodal literacies into classroom-based reading assessment. The Reading Teacher, 74(3), 285–296. https://doi.org/10.1002/trtr.1948. Shohamy, E. (2006). Imagined multilingual schools: How come we don’t deliver. In O. García, T. Skutnabb-Kangas, & M. E. Torres-Guzman (Eds.), Imagining multilingual schools (pp 171–183). Bristol: Multilingual Matters. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418–429. Solano-Flores, G. (2011). Assessing the cultural validity of assessment practices: An introduction. In M. del Rosario Bastera, E. Trumbull, & G. Solano-Flores (Eds.), Cultural validity in assessment: Addressing linguistic and cultural diversity (pp 3–21). New York: Routledge. Veronelli, G. A. (2015). The coloniality of language: Race, expressivity, power, and the darker side of modernity. Wagadu, 13, 108–134. Weber, D. J. (Ed.). (1973). Foreigners in their native land: Historical roots of the Mexican Americans. Albuquerque, NM: University of New Mexico Press.

3

Examining plurilingual repertoires A focus on policy, practice, and assessment in the Australian context Jacqueline D’warte and Yvette Slaughter

Introduction Changing patterns of forced and voluntary migration have significantly increased linguistic, cultural, socio-economic, and religious diversity around the world. Dynamic linguistic and cultural diversity characterises much of the educational landscape in the global north and south and is realised in many Australian schools and classrooms (D’warte & Slaughter, 2021). Australia’s 120 surviving Indigenous languages (AIATSIS, 2018) have been joined by more than 200 languages, spoken by over 20 per cent of Australians as the primary home language (ABS, 2017). Although unevenly distributed, many Australian classrooms are increasingly super-diverse (Vertovec, 2007), comprising young people who are bidialectal/bilingual and plurilingual, and include speakers of languages other than English, monolingual speakers of English, and young people who are first language speakers of Aboriginal Englishes (AEs). Over the last 10 years, for example, in New South Wales (NSW), the most populous state in Australia, the proportion of school-aged students who speak languages other than English has grown steadily from 29.4 per cent in 2010 to more than a third in 2020 (NSW DET, 2021). In 2020, 36.9 per cent of students came from homes where languages other than English were spoken by either the students themselves and/or at least one parent or carer, with the diversity of language backgrounds of students increasing by 12 per cent from 217 languages in 2010 to 243 languages in 2020 (NSW DET, 2021). In the second most populous state, Victoria, 32 per cent of school-aged students are identified as being from a language background other than English (Victoria DET, 2020), while in the Northern Territory, over 100 Indigenous languages and dialects are spoken, with around 50 per cent of the school-aged population coming from families that use languages other than English at home (NT DET, 2021a). Across the diverse educational jurisdictions in Australia, the positioning of linguistic and cultural resources in relation to educational policies, curriculum, assessment, and pedagogy is therefore of critical importance. Governmental responses to languages as a resource, however, have been highly politicised (Lo Bianco, 2010) and vacillated over many decades. Responses have also DOI: 10.4324/9781003177197-5

Examining plurilingual repertoires 63 been complicated by the federated nature of Australian politics whereby responsibility for education is predominantly devolved to each state and territory and each jurisdiction (Government, Catholic and Independent school sectors). At times, however, national directives have generated cohesion across jurisdictional contexts. In the 1980s, in predominantly English-speaking countries in the global north, Australia was considered a progressive innovator in language-in-education policy and Australia’s 1987 National Policy on Languages (NPL) (Lo Bianco, 1987) was one of the first multilingual language policies in an English-speaking country (Lo Bianco & Slaughter, 2017). Nevertheless, since this time, collaborative language policy processes across sectors, states, and territories have diminished, and Australian educational policy has persistently shifted towards monocultural and monolingual conceptualisations of language and literacy in curriculum, pedagogy, and assessment (Coleman, 2012; Eisenchlas et al., 2015). These conceptualisations fail to recognise and value students’ diverse linguistic resources, and limit opportunities for student resources to be leveraged for learning within educational settings, yet some small progressions in innovative policy and practices, as discussed in this chapter, are beginning to emerge. In this chapter, we broadly discuss current policy and practice pertaining to language and literacy in the Australian context, with a particular focus on assessment. We consider the significant linguistic, cognitive, and social benefits that can be derived from recognising and harnessing students’ plurilingual repertoire (Cummins, 2009; D’warte & Slaughter, 2021; French & Armitage, 2020; Oliver et al., 2021; Orellana & Garcia-Sanchez, 2019) alongside the competing tensions of predominantly monolingual, monoglossic educational policies and curricula, and the resulting implications for assessment practices in Australia. We present the current affordances and challenges offered for mainstream Australian classrooms and consider the role of policy and research in furthering plurilingual pedagogies and assessment in the Australian context. Language and literacy policies and assessment in Australia With a long Indigenous history, and a recent migration history, Australia has a complex story of multilingualism marked by the subjugation of Indigenous languages, and language and literacy policies that authorise monolingual, English-only practices (Lo Bianco & Slaughter, 2017; Schalley et al., 2015). Much has been written about the Australian language and literacy policy context over time. The scrutiny of policy has been substantial over recent decades, with dozens of reports into language policies being produced. These reports have illustrated the diverse linguistic needs of Australians, and the competing ideologies that can pull policies in different directions (e.g., Lo Bianco & Slaughter, 2017; Schalley et al., 2015). Policy goals have focused on three main imperatives. First has been a focus on ensuring that all Australians have English language proficiency, including English monolingual students and

64  Jacqueline D’warte and Yvette Slaughter those who are learning English as an Additional Language/Dialect (EAL/D), with particular emphasis on English literacy. Second has been a focus on the maintenance and intergenerational transmission of community languages, supported by governmental policy and funding initiatives as well as through community-driven initiatives, while the third focus has been on the acquisition of second or additional languages, with a significant focus on the study of Asian languages. At various points, these differing goals have coalesced in policy initiatives, but more often than not, the divergent tendencies they represent have created inherent tensions and policy polarities (Lo Bianco & Slaughter, 2017) and have failed to acknowledge the increasing linguistic and cultural diversity in Australia’s pre-school and school-aged population. While there have arguably been five key language and literacy policies in the last 35 years in Australia (see, e.g., Lo Bianco & Slaughter, 2017, pp. 454–456; cf. Schalley et al., 2015, p. 164), the most celebrated policy has been Australia’s 1987 NPL (Lo Bianco,1987). The NPL was a bipartisan national policy on languages, which reflected the influence of both social and economic interests within Australia, and provided broad support, including funding, for the social and educational use and development of languages and language-related services. However, the universal rationale of the NPL, informed by an extensive, nationwide consultation process, was quickly overrun by the ascendency of economic rationalism and a series of policies focused on English literacy (Scarino, 2014). The Australian Language and Literacy Policy (ALLP) was introduced in 1991, with the then Education Minister, John Dawkins, proclaiming that ‘literacy in English for all Australians must be a necessary, if not sufficient, overarching first goal’ (Brock, 2001, p. 55). In a short period of time, policies and language plans (see Scarino, 2014) moved from being ‘comfortably pluralistic and supportive of cultural and linguistic diversity’ towards the positioning of linguistic diversity as problematic and a key contributor to lower English literacy levels (Schalley et al., 2015, p. 169). In their analysis of literacy-related policies over recent decades, Schalley et al. (2015) argue that the more linguistically and culturally diverse Australia has become, ‘the more assimilationist the policies, and the more monolingual the orientation of the society that governments have sought to establish’ (p. 162). This turn towards English literacy (only) and economic rationalism in policy discourse and directions has had a significant impact on the nature of language assessment in Australia. Subsequent to the ALLP, an ongoing and relentless focus in public discourse on (English) literacy in schools and perceived underachievement among Australian students (see, e.g., Freebody, 2007), and divisive public discourse around Australia’s performance in international testing schemes such as the Programme for International Student Assessment (PISA), among other factors (see, e.g., Scarino, 2014; Schalley et al., 2015), culminated in the introduction of the National Assessment Program – Literacy and Numeracy (NAPLAN) test in 2008. The NAPLAN test is undertaken by students in Years 3, 5, 7 and 9 each year, as a ‘measure through which governments, education authorities, schools, teachers and parents, can determine

Examining plurilingual repertoires 65 whether or not young Australians have the literacy and numeracy skills that provide the critical foundation for other learning and for their productive and rewarding participation in the community’ (ACARA, 2016, n.p.). The introduction of the test, however, remains contentious and as a test construct, the NAPLAN arguably presumes a singular definition of English language proficiency, lacking recognition of and differentiation for different cohorts of students (e.g., migrants, refugees, speakers of AEs) and the varying multilingual and dialectical competencies of students (Creagh, 2014; Macqueen et al., 2019). As a result, the test both underrepresents and overrepresents the language capacities of Indigenous, refugee, and immigrant children. The NAPLAN test, for example, benchmarks against Standard Australian English (SAE) and uses cultural and linguistic norms that are unfamiliar and unsuitable for many Indigenous children (Macqueen et al., 2019; Simpson et al., 2009; Wigglesworth et al., 2011). The ramification of NAPLAN testing in Indigenous communities where English is not the first language of children, or where AE is the first language, has been profound. The mistaken attribution of bilingual teaching as a contributing force to below standard performances by Indigenous students, for example, as measured against SAE in NAPLAN testing, led to the dismantling of bilingual education in the Northern Territory, perpetuating the ongoing loss of intergenerational transmission of Indigenous languages, culture, and knowledges (Devlin et al., 2017; Simpson et al., 2009). For students from refugee and immigrant, non-English-speaking backgrounds, the NAPLAN test also misrepresents or obscures the differing challenges of students from a Language Background Other Than English (LBOTE). In an in-depth analysis of NAPLAN and school-based assessment data, Creagh (2014) found that on the surface, there appears to be no difference between the score of LBOTE and non-LBOTE students, although test results for Indigenous students (who may or may not come from a SAE background) are significantly below the average across the measures of readings, spelling, English, and mathematics. Using a multiple regression analysis, Creagh (2014) found that visa category was the most influential variable, with students arriving in Australia on refugee visas scoring significantly lower than those arriving on skill migration visas, a variation that is obscured by the broad LBOTE categorisation within the testing data. Despite repeated calls for the testing mechanism to be adapted not only to better reflect the varying proficiency levels and entry points of students, but also to incorporate measures which could recognise and understand the rich and diverse language practices that students do possess (e.g., Macqueen et al., 2019), no changes have been made in this regard. As a result, with no current national language policy in Australia, there is the chance that test mechanisms can take on the de facto role of top-down language policy. Despite a plethora of research which argues for the benefits of recognising and valuing students’ complex linguistic resources, as will be explored next, broad assessment mechanisms such as NAPLAN continue to perpetuate monoglossic

66  Jacqueline D’warte and Yvette Slaughter approaches to language and education that ‘privilege majority languages and legitimise monolingual, monocultural, and monomodal language practices’ (Kirsch, 2020, p. 15). Plurilingual repertoires and multiple meaning making opportunities We argue that the enduring monolingual lens for languages policy and associated mechanisms such as the NAPLAN test in Australia has failed to recognise the role of language and culture, including languages other than English or indeed diverse English language resources, in the cognitive and socio-emotional development of students in Australia (Cross et al., 2022). This stance has stood in opposition, for many decades, to research into language, education, and cognition (e.g., Blackledge & Creese, 2010; Lantolf & Thorne, 2006), including more recent work on heteroglossic conceptualisations of language development and use, which has moved us away from viewing languages use as being in a discrete first language and additional language relationship. Instead, research has conceptualised meaning-making as involving a single integrated system of complex linguistic and semiotic resources that work together to promote thinking and cognition, and direct our social emotional life (García, 2009; Lau & Van Viegen, 2020). We refer to this construct as a plurilingual repertoire, a dynamic and developing linguistic repertoire of an individual user or learner, a plural repertoire of linguistic and cultural resources used to communicate and interact with others. Underpinning this research is an acknowledgement of and respect for linguistic and cultural pluralism, a positioning that has increasing resonance across diverse, multilingual educational contexts globally. In English-speaking countries, including Australia, as will be explored in this section, growing consideration is given to the benefits of engaging holistically with the diverse communicative repertoires of young people. Foundational to this active engagement with students’ full linguistic repertoire in mainstream contexts is moving away from exclusively identifying what young people ‘lack’, to instead identifying and productively mobilising the full range of linguistic resources – the plurilingual repertoires young people bring to their learning (García, 2014). This positioning has led to significant conceptual work turning a heteroglossic lens onto curriculum, pedagogy and assessment policy and practice. This has included research into strategies that support teachers in developing heteroglossic practices within their diverse classrooms (e.g., Kirsch, 2020; Leung & Valdes, 2019; Slaughter & Cross, 2021), as well as engaging young people in reflective plurilingual, inquiry centred on themselves (e.g., Chik & Melo-Pfeifer, 2020; Little & Kirwan, 2019). Reinterpretation of curricula that embraces and encourages the use of multilingual and intercultural knowledge and the experiences and biographies of students and communities, and new and revised pedagogies such as, for example, linguistically responsive pedagogies (Morrison et al., 2019); culturally sustaining pedagogies (Paris & Alim, 2017); translanguaging pedagogy

Examining plurilingual repertoires 67 (García et al., 2017) and functional multilingual learning (Sierens & Van Avermaet, 2014), among others, are now well recognised in educational contexts. These developing curricula and pedagogies aim to not only recognise and challenge the biases of practitioners, but to recognise the linguistic competencies of students and to seek practices which can acknowledge, provide space for, and extend the knowledge of bilingual, dialectical, and plurilingual young people (e.g., Barac et al., 2014; Busch, 2012; Duarte, 2019; D’warte, 2021; García, 2014; Heugh et al., 2019; Orellana & Garcia-Sanchez, 2019; Sierens & Van Avermaet, 2014). Plurilingual approaches in the Australian context The research identified above is among a growing body of research crucial to the development of policy, curriculum, pedagogy, and assessment related to working in culturally and linguistically diverse classrooms, yet much of the research is generated in contexts very different from those experienced in many Australian classrooms. Context-specific research is critical. While a national curriculum – the Australian curriculum (ACARA, n.d. b) – was introduced in 2014, it is mandated curriculum for Catholic and independent education systems, whereas State governments are able to interpret the Australian curriculum. Some States draw directly on the national curriculum (e.g., South Australia) while other States incorporate the Australian curriculum with their established curriculum (e.g., the Victorian curriculum). As a result, highly varied policy and educational responses exist across Australian educational landscape. Contextualised research from Australia is continuing to emerge, offering new knowledge and understandings of how groups of teachers and students are negotiating and acquiring knowledge in multilingual classrooms (e.g., Choi & Ollerhead, 2018; Dutton & Rushton, 2021; D’warte, 2021; French, 2019; Slaughter & Cross, 2021 ). For example, in a study of bilingual primary schools in NSW, Fielding (2015) found that plurilingual children drew on their home language(s) as a resource in school contexts where other languages were used. This process increased students’ enjoyment of learning and assisted students in developing learning strategies, which built on their plurilingual experiences. In a review of research in two secondary schools in NSW and South Australia, French and Armitage (2020) also found that when students were given an opportunity to bring their own languages and knowledges to classrooms, they were active and engaged in classroom learning. Students also developed a deeper understanding of concepts, offered additional linguistic and cultural knowledge to the classroom, and involved their family and community members in the process and products of learning. Dutton and Rushton (2021, p. 108) argue that Australian students are ‘required to engage with and draw upon their own lived experiences – culturally and linguistically’ as per the national and state-based curricula (ACARA, n.d. a; NESA, 2019). In a research undertaken in secondary school English

68  Jacqueline D’warte and Yvette Slaughter Language Arts classes, with highly multilingual and multicultural student cohorts, Dutton and Rushton (2021) employed what they call translanguaging poetry pedagogy to ‘explore new possibilities that challenge the routinized everyday monolingual practices’ (p. 108). Their research found that within the translanguaging poetry space, students were able to engage in complex, multilingual expressions of language and ‘develop powerful personal representations’ (p. 108) through their work. In focusing on Australian Aboriginal children in mainstream classrooms in the state of Western Australia and the Northern Territory, Oliver et al. (2021) employed translanguaging pedagogies to draw on children’s complex linguistic practices. These researchers found that such practices could enhance home language knowledge and the development of Standard English, as well as providing greater agency for children to express their multifaceted linguistic and cultural identities. Given the critical role of teachers and pedagogy in enacting plurilingual pedagogies in their classrooms, research in this area is also key. Recent research in a variety of Australian contexts demonstrates that with effective tools, teachers can acquire new knowledge about their students and their cultural and linguistic lives, leading to teachers increasing the complexity of assigned tasks, and enriching learning activities for students (Choi & Slaughter, 2021; D’warte, 2018; French, 2019; Turner, 2019). Work is also underway into the positioning of language in teacher education. While scholars argue that it is important for teachers to position their students’ full plurilingual repertoires as a resource (Busch, 2012; Cummins, 2014; García, 2009), understanding what this looks like continues to be a challenge. Among pre-service and practicing teachers, even among those who are themselves multilingual, research highlights a lack of confidence about how to acknowledge and build on their own linguistic strengths in their teaching (Coleman, 2012; D’warte et al., 2021; Slaughter & Cross, 2021). A key contradiction lies in mainstream teacher education, where pre-service teachers are often being introduced to heteroglossic views of language learning and encouraged to recognise and build on their students’ plurilingual repertoires. Yet in their university teacher education contexts, they are overwhelmingly exposed to monolingual instructional practices, including during their professional practice experiences, and few opportunities are provided for them to use their full linguistic repertoires (D’warte et al., 2021) or to understand plurilingual pedagogies in practice. Mitigation of the monolingual mindset in assessment While the broad body of work focusing on the reframing of language, culture, and identity within education seeks to disrupt the entrenched mismatch between schools as institutions with a monolingual habitus serving linguistically diverse societies, it is important to continually reflect on the implications of this work given that education is highly contextualised and political in nature. We must be mindful that multilingualism and associated notions of hybridity

Examining plurilingual repertoires 69 can perpetuate a privileged experience of multilingualism. Kubota (2016), for example, argues that ‘[w]ithout addressing power and ideology, advocacy of multi/plural approaches and hybridity in language use can become complicit with domination and will fail to solve real problems’ (p. 9), particularly for those who need to conform to standard school-based, university-based, or work-based conventions and standard use of language (Kubota, 2016). This is where careful work in advocacy, as well as research, is necessary when seeking to shift the relationship between the positioning of linguistic resources and assessment, to ensure that changes are made in concert with each other, particularly in mainstream contexts. With teachers as the final point of departure for education policy and practice, the pedagogical choices they make in the classroom ‘ultimately constitute an enacted language and literacy policy’ (Lo Bianco, 2010, p. 165). Teachers can therefore work in small and powerful ways for their students as demonstrated in the research we have discussed (e.g., Dutton & Rushton, 2021; D’warte, 2021; Slaughter & Cross, 2021). However, assessment mechanisms are most often rigid and top-down, either as explicit assessment polices such as the NAPLAN test or in the shifting of funding and priorities that act as de facto policies. This later point is realised for example, in the reduction of support for Indigenous Languages as a First Language programme, and the increased funding for EAL/D programmes in the Northern Territory in Australia (Disbray, 2019). These challenges are also reflected in Disbray’s (2016) study in a remote urban township in Central Australia with Indigenous children who were speakers of Wumpurrarni English. While the children in the study demonstrated their narrative skills in sophisticated ways in their home language variety, these skills and understandings were not visible in a culture of high-stakes literacy testing. Her research demonstrates the critical point that unless changes are made in assessment mechanisms, the washback effect of the monolingual mindset in testing will continue to enforce a hegemonic view of (English) language in the education system. Small gains are being made in the State of Victoria, for example, the newly developed English as an Additional Language curriculum has explicitly integrated cultural and plurilingual awareness as an assessed element. The curriculum has three assessed language modes, aligned with the English curriculum – speaking and listening, reading and viewing, and writing. Each mode contains three strands: communication, linguistic structures and features, and cultural and plurilingual awareness. This last strand takes a more holistic view of students’ linguistic resources, defining this element as: understanding and using the cultural conventions of spoken and written communication in Standard Australian English – including the relationships between text and context, and audience and purpose – and drawing on the knowledge and resources of students’ other languages and cultures to negotiate communication and enhance learning. (VCAA, 2019)

70  Jacqueline D’warte and Yvette Slaughter In the Northern Territory, the recently released Framework for Aboriginal Languages and Torres Strait Islander Languages (ACARA, 2019) builds on the 1993, Australian Indigenous Languages Framework for traditional Australian languages. This framework promotes language teaching, learning, and maintenance of Indigenous Language as a Second language (ILSL) programme. Support for teachers working with English as an Additional Language or Dialect (EAL/D) includes the direction that learning environments should position home languages and dialects, and students’ knowledge of the world, as valued resources, as well as encouraging teachers to use other home language speakers to translate and make understandings clear (NT DET, 2021b). In the State of NSW, the Department of Education offers a suite of Professional Learning materials for schools and teachers, which incorporate the use of multilingual resources and the use of home language to support differentiation for EAL/D students in teaching and assessment (e.g., NSW DET, 2020). The inclusion of such policy discourse and associated resources, increasingly visible across different educational jurisdictions in Australia, is arguably creating what Flores and Schissel (2014, p. 454) term ideological and ‘implementational spaces’ which provide an opportunity for teachers to explore what plurilingual approaches may look like in practice. We must acknowledge that while positive discourse and policies have existed in the past, as noted above in the policy section, they have diminished over time, yet there is some hope for further momentum moving forward. Concluding comments In discussing language and literacy education in Australia, it is important to acknowledge that the education landscape is diverse, complex, and variable, and we have only touched briefly on some of the many issues at play. We have not, for example, considered the study of languages other than English (Languages curriculum) in the Australian education system. However, we have tried to illustrate that, from the early years and onto primary and secondary schooling, a growing body of conceptual work illuminates the possibilities in harnessing and adapting plurilingual approaches and strategies in a range of Australian classrooms. We accept that plurilingualism and the use of plurilingual pedagogies in one setting are unlikely to be the same as in another and that strategies and practices applied in one context may need continual review and adjustment in another context. However, it is clear that enabling equitable access to educational development for all children requires challenging the positioning of literacy as inextricably linked to the English language (only). Research and public discourse need to impact not only on political and social landscapes, but also on educational policy and the mechanisms used to demonstrate student achievement. Despite complex theorisations and ongoing empirical studies, and some small, visible changes in educational policies across Australia, curriculum, assessment, and more often than not, most teacher pedagogy, continue to predominantly perpetuate a monolingual bias, with the assumption

Examining plurilingual repertoires 71 that students draw on only one language system (i.e., SAE) to access, create, and recreate knowledge. More empirical work that engages Australian young peoples’ plurilingualism as a tool for learning across contexts is needed. A return to the progressive language policies that centred language as an intellectual, cultural, economic, social, citizenship and rights resource is also critical (Lo Bianco, 2010). These pressing and crucial calls, for renewed policy and applied knowledge, are motivated by the need to not only disrupt the monolingual, monocultural orientation of mainstream classrooms, but to also recognise the experiences and understandings of Australia’s increasingly diverse population. While policy change and change to educational structures are slow, they have the capacity to move us towards a more equitable education system that recognises both the strengths and needs of its students and Australia’s wider community. References ABS (Australian Bureau of Statistics). (2017). 2016 Census: Multicultural – Census Reveals a Fast Changing, Culturally Diverse Nation. http://www.abs.gov.au/ausstats/ [email protected]/lookup/media%20release3. ACARA. (2016). NAPLAN – General. https://www.nap.edu.au/naplan. ACARA. (2019). Framework for Aboriginal Languages and Torres Strait Islander Languages. https://www.australiancurriculum.edu.au/f-10-curriculum/languages/ framework-for-aboriginal-languages-and-torres-strait-islander-languages/. ACARA. (n.d. a). Intercultural understanding. https://www.australiancurriculum. edu.au/f-10-curriculum/general-capabilities/intercultural-understanding/. ACARA. (n.d. b). The Australian curriculum. https://www.australiancurriculum.edu. au/. AIATSIS (Australian Institute of Aboriginal and Torres Strait Islander Studies). (2018). Indigenous Australian Languages: Celebrating 2019 International Year of Indigenous Languages. https://aiatsis.gov.au/whats-new/events/2019-internationalyear-indigenous-languages Barac, R., Bialystok, E., Castro, D. C., & Sanchez, M. (2014). The cognitive development of young dual language learners. Early Childhood Research Quarterly, 29(4), 699–714. https://doi.org/10.1016/j.ecresq.2014.02.003. Blackledge, A., & Creese, A. (2010) Multilingualism: A critical perspective. London; New York: Continuum. Brock, P. (2001). Australia’s language. In J. Lo Bianco & R. Wickert (Eds.), Australian policy activism in language and literacy (pp 47–74). Melbourne: Language Australia. Busch, B. (2012). The linguistic repertoire revisited. Applied Linguistics, 33(5), 503–523. https://doi.org/10.1093/applin/ams056. Chik, A., & Melo-Pfeifer, S. (2020). What does language awareness look like? Visual methodologies in language learning and teaching research (2000–2018). Language Awareness 293(3–4), 336–352. https://doi.org/10.1080/09658416.2020.1785481. Choi, J., & Ollerhead, S. (Eds.). (2018). Plurilingualism in teaching and learning: Complexities across contexts. Routledge. https://doi.org/10.4324/9781315392462. Choi, J., & Slaughter, Y. (2021). Challenging discourses of deficit: Understanding the vibrancy and complexity of multilingualism through language trajectory grids. Language Teaching Research, 25(1), 81–104. https://doi.org/10.1177/1362168820938825

72  Jacqueline D’warte and Yvette Slaughter Coleman, J. (2012). Moving beyond an ‘instrumental’ role for the first language of English language learners. TESOL in Context, 22(1), 18–37. https://ojs.deakin. edu.au/index.php/tesol/index. Creagh, S. (2014). A critical analysis of problems with the LBOTE category on the NAPLaN test. The Australian Educational Researcher, 41, 1–23. https://doi. org/10.1007/s13384-013-0095-y. Cross, R., D’warte, J., & Slaughter, Y. (2022). Plurilingualism and language and literacy education. Australian Language and Literacy Journal, 45, 341–357. http:// dx.doi.org/10.1007/s44020-022-00023-1 Cummins, J. (2009). Transformative multiliteracies pedagogy: School-based strategies for closing the achievement gap. Multiple Voices for Ethnically Diverse Exceptional Learners, 11, 38–56. https://doi.org/10.5555/muvo.11.2.2420352213232u47. Cummins, J. (2014). Beyond language: Academic communication and student success. Linguistics and Education, 26, 145–154. https://doi.org/10.1016/j.linged. 2014.01.006. Disbray, S. (2016). Spaces for learning: Policy and practice for Indigenous languages in a remote context. Language and Education, 30(4), 317–336. https://doi.org/1 0.1080/09500782.2015.1114629. Disbray, S. (2019). Realising the Australian curriculum framework for aboriginal languages and Torres Strait Islander language. Babel, 54, 21–25. https://afmlta.asn. au/babel/. Devlin, B., Disbray, S., Regine, N., & Devlin, F. (2017). History of bilingual education in the Northern territory: People, programs and policies. Singapore: Springer. Duarte, J. (2019) Translanguaging in mainstream education: A sociocultural approach. International Journal of Bilingual Education and Bilingualism, 22(2), 150–64. https://doi.org/10.1080/13670050.2016.1231774. Dutton, J., & Rushton, K. (2021) Using the translanguaging space to facilitate poetic representation of language and identity. Language Teaching Research, 25(1), 105–133. https://doi.org/10.1177/1362168820951215. D’warte, J. (2018). Research Report: Enhancing English Learning: Building on Linguistic and Cultural Repertoires in 3 School Settings. NSW Department of Education and Training and Western Sydney University. http://doi.org/10.26183/5ba9a85c6759b. D’warte, J. (2021). Facilitating agency and engagement: Visual methodologies and pedagogical interventions for working with culturally and linguistically diverse young people. Language Teaching Research, 25(1), 12–38. https://doi. org/10.1177/1362168820938826. D’warte, J., Rushton, K., & Abu Bakar, A. (2021). Investigating pre-service teachers’ linguistic funds of knowledge. Western Sydney University. https://researchdirect. westernsydney.edu.au/islandora/object/uws:60711/datastream/PDF/view. D’warte, J., & Slaughter, Y. (2021). Introduction: Reframing language in teaching and learning: Leveraging students’ meaning-making repertoires. Language Teaching Research, 25(1), 5–11. https://doi.org/10.1177/1362168820971753. Eisenchlas, S., Schalley, A., & Guillemin, D. (2015). Multilingualism and literacy: Attitudes and policies. International Journal of Multilingualism, 12(2), 151–161. https://doi.org/10.1080/14790718.2015.100937. Fielding, R. (2015). Students’ use of their plurilingual resources in Australian schools. Language and Education, 30(4), 361–377. https://doi.org/10.1080/09500782. 2015.1114631.

Examining plurilingual repertoires 73 Flores, N., & Schissel, J. L. (2014). Dynamic bilingualism as the norm: Envisioning a heteroglossic approach to standards-based reform. TESOL Quarterly, 48(3), 454–479. https://doi.org/10.1002/tesq.182. Freebody, P. (2007). Literacy education in school research perspectives from the past, for the future. ACER. https://research.acer.edu.au/cgi/viewcontent.cgi?article=1001& context=aer. https://doi.org/10.1080/09500782.2015.1114628. French, M. (2019). Multilingual pedagogies in practice. TESOL in Context, 28, 21–44. https://doi.org/10.1080/09500782.2015.1114628. French, M., & Armitage, J. (2020). Eroding the monolingual monolith. Australian Journal of Applied Linguistics, 3(1), 91–114. https://doi.org/10.29140/ajal. v3n1.302. García, O. (2009). Bilingual education in the 21st century. Oxford: Wiley-Blackwell. García, O. (2014). Multilingualism and language education. In C. Leung & B. V. Street (Eds.), The Routledge companion to English studies (pp 84–99). London: Routledge. García, O., Johnson, S. I., & Seltzer, K. (2017). The translanguaging classroom: Leveraging student bilingualism for learning. Philadelphia, PA: Caslon. Heugh, K., French, M., Armitage, J., Taylor-Leech, K., Billinghurst, N., & Ollerhead, S. (2019). Using multilingual approaches: Moving from theory to practice. British Council. https://www.teachingenglish.org.uk/sites/teacheng/files/Using_multilingual_ approaches.pdf. Kirsch, C. (2020). Translanguaging pedagogies in early childhood education in Luxembourg: Theory into practice. In S. Lau & S. Van Viegen (Eds.), Plurilingual pedagogies: Critical and creative endeavors for equitable language in education (pp 15–33). Cham: Springer. Kubota, R. (2016). The Multi/Plural turn, postcolonial theory, and neoliberal multiculturalism: Complicities and implications for applied linguistics. Applied Linguistics, 37(4), 1–22. https://doi.org/10.1093/applin/amu045. Lantolf, J. P., & Thorne, S. L. (2006). Sociocultural theory and the genesis of second language development. Oxford: Oxford University Press. Lau, S., & Van Viegen, S.(Eds.) (2020). Plurilingual pedagogies: Critical and creative endeavours for equitable language in education. Cham: Springer. Leung, C., & Valdes, G. (2019). Translanguaging and the transdisciplinary framework for language teaching and learning in a multilingual world. The Modern Language Journal, 103(2), 348–370. https://doi.org/10.1111/modl.12568. Little, D., & Kirwan, D. (2019). Engaging with linguistic diversity: A study of educational inclusion in an Irish primary school. London: Bloomsbury. Lo Bianco, J. (1987). National policy on languages. Canberra: Australian Government Publishing Services. Lo Bianco, J. (2010). Language policy and planning. In N. H. Hornberger & S. L. McKay (Eds.), New perspectives on language and education: Sociolinguistics and language education (pp 143–174). Clevedon: Multilingual Matters. Lo Bianco, J., & Slaughter, Y. (2017). Language policy and education in Australia. In T. McCarty (Ed.), Encyclopedia of language and education, language policy and political issues in education (Volume 1, pp 449–461). Berlin: Springer. Macqueen, S., Knoch, U., Wigglesworth, G., Nordlinger, R., Singer, R., McNamara, T., & Brickle, R. (2019). The impact of national standardized literacy and numeracy testing on children and teaching staff in remote Australian Indigenous communities. Language Testing, 36(2), 265–287. https://doi.org/10.1177/0265532218775758.

74  Jacqueline D’warte and Yvette Slaughter Morrison, A., Rigney, L.-I., Hattam, R., & Diplock, A. (2019). Towards an Australian Culturally Responsive Pedagogy. A Review of the Narrative Literature. https://apo. org.au/node/262951 New South Wales, Department of Education (NSW DET). (2020). EAL/D Advice for Schools. https://education.nsw.gov.au/content/dam/main-education/teaching-andlearning/curriculum/multicultural-education/eald/eald_advice.pdf. New South Wales Educational Standards Authority (NESA). (2019). English K-10 syllabus. https://educationstandards.nsw.edu.au/wps/portal/nesa/k-10/learningareas/english-year-10/english-k-10. NSW DET, (2021). Schools: Language diversity in NSW, 2020. Centre for Educational Statistics and Evaluation (CESE) Bulletin. https://education.nsw.gov.au/about-us/ education-data-and-research/cese/publications/statistics/language-diversitybulletin/language-diversity-bulletin-2021. Northern Territory, Department of Education (NT DET). (2021a). English as an Additional Language/Dialect (EAL/D). https://education.nt.gov.au/__data/assets/ pdf_file/0006/1117077/english-as-additional-language-or-dialect-guidelines.pdf Northern Territory, Department of Education (NT DET). (2021b). Student Diversity. https://education.nt.gov.au/support-for-teachers/student-diversity. Oliver, R., Wigglesworth, G., Angelo, D., & Steele, C. (2021). Translating translanguaging into our classrooms: Possibilities and challenges. Language Teaching Research, 25(1), 134–150. https://doi.org/10.1177/1362168820938822. Orellana, M., & Garcia-Sanchez, I. (2019). Language and cultural practices in communities and schools. New York: Routledge. Paris, D., & Alim, S. (2017). Culturally sustaining pedagogies. Teaching and learning for justice in a changing world. New York: Teachers College Press. Rigney, L. I. (2021). Aboriginal child as knowledge producer: Bringing into dialogue indigenist epistemologies and culturally responsive pedagogies for schooling. In B. Hokowhitu, A. Moreton-Robinson, L. Tuhiwai-Smith, C. Andersen, & S. Larkin (Eds.), Routledge handbook of critical indigenous studies (pp 578–590). London: Routledge. Scarino, A. (2014). Situating the challenges in current languages education policy in Australia: Unlearning monolingualism. International Journal of Multilingualism, 11(3), 289–306. https://doi.org/10.1080/14790718.2014.921176. Schalley, A. C., Guillemin, D., & Eisenchlas, S. A. (2015). Multilingualism and assimilationism in Australia’s literacy-related educational policies. International Journal of Multilingualism, 12(2), 162–177. https://doi.org/10.1080/14790718.2015.100 9372. Sierens, S., & Van Avermaet, P. (2014). Language diversity in education: Evolving from multilingual education to functional multilingual learning. In D. Little., C. Leung., & P. Van Avermaet (Eds.), Managing diversity in education: Languages, policies, pedagogies (pp 204–222). Bristol: Multilingual Matters. Simpson, J., Caffery, J., & McConvell, P. (2009). Gaps in Australia’s indigenous language policy: Dismantling bilingual education in the Northern territory. AIATSIS. https:// aiatsis.gov.au/sites/default/files/research_pub/simpson-caffery-mcconvell-dp24indigenous-language-policy_0_3.pdf. Slaughter, Y., & Cross, R. (2021). Challenging the monolingual mindset: Understanding plurilingual pedagogies in English as an additional language (EAL) classrooms. Language Teaching Research, 25(1), 39–60. https://doi.org/10.1177/1362168820938819

Examining plurilingual repertoires 75 Turner, M. (2019). Multilingualism as a Resource and a goal. Using and learning languages in mainstream schools. New York: Palgrave Macmillan. VCAA. (2019). English as an Additional Language (EAL). Retrieved from https:// victoriancurriculum.vcaa.vic.edu.au/english/english-as-an-additional-language-eal/ introduction/structure. VCAA. (n.d.). The Victorian curriculum. https://victoriancurriculum.vcaa.vic.edu. au/ Vertovec, S. (2007). Super-diversity and its implications. Ethnic and Racial Studies, 30(6), 1024–1054. https://doi.org/10.1080/01419870701599465. Victoria, Department of Education (Victoria DET). (2020). English as an Additional Language in Victorian Government Schools, 2019. Victoria, DET. https://www. education.vic.gov.au/Documents/school/teachers/teachingresources/diversity/ eal/2019-eal-report.pdf. Wigglesworth, G., Simpson, J., & Loakes, D. (2011). NAPLAN language assessments for Indigenous children in remote communities. Australian Review of Applied Linguistics, 43(3), 320–343. https://doi.org/10.1075/aral.34.3.04wig.

4

Evaluating the effectiveness of digital tools for the assessment of young plurilingual students Emmanuelle Le Pichon, Mai Naji, Dania Wattar, and Vivian Lee

Introduction John is a sixth-grade teacher of 11–12-year-olds at a school in the Greater Toronto Area. All of his students are plurilingual and ten percent of them arrived in Canada within the last five years. The students have mastered the language of the school (English) to varying degrees, from basic, intermediate, to expert levels. One day, the researcher and first author of this chapter was in the classroom to observe the implementation of an online resource called Binogi, which provides Math and Science lessons in the form of short videos, followed by quizzes to assess students’ comprehension of content knowledge. The particularity of these videos is that students can change the language of the video, from the language of the school (English) to Arabic, Dari, Somali, or Tigrinya, among others. They can also view subtitles, in the language of the school or in one of the languages mentioned above, slow down or speed up the tempo, rewatch the video, and read the transcript placed below the video. On the day she visited the grade six class, two recently arrived students who usually did mathematics with the third-grade class were present. The teacher handed out eLearning tablets, helped the students to log in to their Binogi account, and directed them to the video and quiz in question. Both new students participated in the class like everyone else. However, the teacher noticed to his great surprise that these two students who chose to work in Arabic did answer the questions of the first quiz correctly. With this information, he realized that these students had been inappropriately placed in a lower level 3 grades below and took the necessary administrative measures to reintegrate them into his mathematics class. In this example, a simple translation of the test into the students’ primary language allowed the school to repair a potentially serious orientation error. Welcoming students into the classroom who have not mastered the school language can pose challenges for educators, most critically the inaccurate assessment of their true academic potential, which can lead to diminished academic opportunities for them at school, significant psychological and developmental disruptions, and ultimately negative educational outcomes. In fact, newcomers are often underrepresented in Science, Technology, Engineering DOI: 10.4324/9781003177197-6

Digital tools for the assessment of young plurilingual students 77 and Mathematics (STEM) pathways in education (LaCosse et al., 2020). Generally, teachers feel underprepared to cope with high levels of multilingualism in classrooms with a high percentage of newcomer students, and the declining interest in Sciences in the middle grades along with the reduced STEM practice opportunities outside of school hours (see for instance Gottleib, 2021). These circumstances have created an urgent need to improve our ability to provide appropriate support for the increased academic success of language learners, particularly in STEM. Against this backdrop, computer-based tools have recently gained traction as promising strategies to assist teachers in fostering collaboration, problem-solving and procedural thinking among plurilingual students (Johnson et al., 2011), improving higher-order thinking of learners (Akcaoglu, 2016) and increasing student engagement overall (Stacy et al., 2017). Digital tools have the unique advantage of being able to remove the language barrier by introducing translation possibilities into many different languages. It is then easier to understand that they can also provide an overview of the students’ skills in different languages. However, while the theoretical benefits of this approach seem obvious, the actual effectiveness of such tools has yet to be researched. The questions we will attempt to answer in this chapter are: 1 Why is the assessment of plurilingual students particularly complex? 2 What are the possibilities of an online resource like Binogi to facilitate these assessments? 3 What might be the limits of such a resource and, given these limits, what can be done about them? Assessing multilingual learners Failing to master the school language continues to be considered a key factor in poor academic achievement among language learners. Language learners typically need at least five years to catch up with their native-speaking peers in school (e.g., Collier, 1987; Cummins, 1981; Levin & Shohamy, 2008). During this time, they invest a substantial proportion of their efforts in learning the school language. As a result, some fall behind academically (Collier & Thomas, 1989; Cummins, 1989, 2000; OECD, 2016). Let’s take the example of the diagnostic evaluation of a newcomer student (from here on: plurilingual student). In the diagnostic assessment of plurilingual students, three scenarios can be distinguished. In the first, there is no correspondence between the learners’ academic skills in their own language(s) and in the new language they are learning. In other words, the academic content assessed does not correspond to content taught in the home countries. Thus, students do not have a knowledge of the content to be assessed in either their language or the target language. This may be the case, for example, for students who have been out of school for a significant

78  Emmanuelle Le Pichon et al. period of time or in refugee camps that do not offer them the opportunity to progress from level to level (Le Pichon-Vorstman et al., 2020). In the second, there is a correspondence between the learners’ academic skills but a possible mismatch of the content of the curricula. That is, what students have learned during their previous schooling does not fully correspond to what they are supposed to have learned in the host country for an equivalent level. The students therefore have partial access to content that they can understand in their language but not in the target language. In the third case scenario, there is correspondence as the curriculum in countries may be relatively equivalent or even richer in the country of provenance. The students therefore can treat the content in their own language and only need to acquire the target language while continuing to develop the content. The diagnosis regarding the two students presented in the introduction may probably be situated between the second and third case scenario. It reveals that if linguistic and cultural components are ignored, there is a significant risk of underestimating the academic level of students. Studies have noted the relatively high percentages of plurilingual students in lower levels of education, as opposed to so-called academic classes with more abstract teaching content (Brevik & Rindal, 2020). In the three scenarios, the students will need language instruction in the target language to meet the language requirements of the intended school system. However, the level to which they will have to overcome a lack of academic preparation will differ. Of course, the reality is much more complex than these three scenarios suggested above as there are as many variations into the academic and language levels of these students as there are different cases. In this chapter, we emphasize that the condition for a fair assessment for these students is a medium to long-term assessment that considers the linguistic and cultural backgrounds of the students in a perspective of reciprocal knowledge. We will show that in order to distinguish between the academic and language profiles of students, it is necessary, but not sufficient, to assess them in a language in which they feel comfortable. Indeed, to adequately address their needs, an advanced knowledge of the curricula of their country of provenance and of the way these curricula have been taught is equally crucial. Possibilities of an online resource like Binogi to facilitate multilingual assessments According to the Ontario Ministry of Education (2008), all teachers should be able to support language learners. However, teachers are typically trained for monolingual students and therefore not provided with the necessary knowledge to cope with high levels of linguistic diversity in the classroom and multilingual academic content resources are often incomplete or nonexistent. Consequently, assessments are often carried out in the host country’s dominant language and as a result, the assessment of academic skills in language

Digital tools for the assessment of young plurilingual students 79 learners is very likely confounded by their linguistic level in the assessment language (Attar et al., 2020; Cummins, 2017; Le Pichon-Vorstman & Kambel, 2016; Lidz, 1991). This is unfortunate as a powerful strategy for facilitating plurilingual students’ integration into their new educational and social environment is to engage their home languages to maximize the transfer of academic content. This strategy is supported by a host of studies showing its advantages (e.g., Cenoz & Gorter, 2013; Creese & Blackledge, 2010; Cummins, 2017; Garcia & Li, 2014; Reljić et al., 2015). As far as the subject of this chapter is concerned, one of the crucial advantages of digital technologies is that they can incorporate different languages, which provides a unique opportunity for teachers to assess their students in STEM content in languages they know (Stacy et al., 2017). Such technology can provide a more accurate impression of their knowledge and skills in these domains (Wylie & Lyon, 2012). Teachers need to learn, however, how to use and implement novel multilingual digital tools. If applied in the right way, such multilingual digital programs can not only serve as a learning tool for students, but also provide the teacher with insight into the students’ true academic skills. In this study, we explored this issue by examining how teachers used a multilingual digital learning platform called Binogi to assess the learning gaps of middle-grade students in STEM. We implemented and evaluated the impact of this online STEM-focused program that combines a highly engaging interface with a choice of several languages across Ontario in schools where there is a greater than average number of plurilingual students. We sought to elucidate the experiences of teachers with regard to using this technology and various strategies to support them in learning to integrate multilingual assessments across the STEM curriculum. Binogi, the multimodal platform we deployed in this study, is, to our knowledge, the only currently available program that combines a highly engaging game-like interface with a choice of several languages to access academic content (Le Pichon et al., 2021). The tool, recently aligned with the local Canadian curricula, may be used as a co-teaching tool without disrupting the course of school-related activities. Binogi’s content modules target specific areas of the STEM curriculum for students aged 10–14 years and are presented through engaging short digital video animations. Students can choose, switch, and combine the languages (through audio and captions) in which the content is presented; they can read the text or use the read aloud function, or even add captions in the language of their choice. Regardless of the language in which the student solves the problem, the teacher receives a report on the student’s use and performance in the language of the school. Thus, the teacher has access to students’ performances even if students have worked in languages that the teachers do not understand. Based on these reports, the teacher, depending on the pedagogical assessment objective, can decide to move on to the next topics or to repeat certain aspects of the lesson that have not been understood.

80  Emmanuelle Le Pichon et al. Note: Example of a quiz in Binogi: the student (left image) read the question in Tigrinya. On the right image, the student solved the same question in English.

Digital tools for the assessment of young plurilingual students 81 Additionally, this multimodality of Binogi allows to prolong the educational space from schools to families and communities. Teachers can assign homework to students, and students can work on it in discussion with their parents and communities in the language of their choice. In summary, Binogi allows teachers to interpret the work of the students, provide students and families with feedback, and report on the students’ achievements. In 2019, we conducted an exploratory study, which included the implementation of Binogi in three grade 6 classes in Toronto. Results indicated that this digital tool may support the students’ agency while stimulating the students’ overall engagement with STEM (Le Pichon et al., 2021). Building on these insights, we decided to conduct a study in which we measured the teachers’ experience of using the technology and discovered that some teachers were more than others ready to embrace the implementation of the resource. We therefore investigated the conditions that facilitated or obstructed the opening of pedagogical spaces based on the potential of the Binogi platform by teachers. We asked teachers to engage self-reflexively to report both possible challenges and benefits by means of two surveys and a final interview. A first analysis of the responses showed that in addition to the linguistic and educational profile of the students, the implementation of the platform depended on factors related to the school community despite the evident potential that the platform offered. This preliminary analysis allowed us to identify a certain number of sociolinguistic factors of resistance, starting with the teaching context. Limitations and future requirements The teaching context

To better understand how teachers use the platform, we describe the contextual singularity of three of the schools with which we worked: a bilingual Arabic-English school, a French International School, and a public school all three situated in the province of Ontario, Canada. The identified factors belong to the following categories: the linguistic profiles of teachers, schools, and families (Figure 4.1). Then, we considered the adequacy or inadequacy of the expectations of these three groups in relation to each other in terms of the pedagogical programs and academic contents of the schools in which these students were enrolled. The case of a bilingual Arabic-English school

We will present the case of a bilingual Arabic-English school to illustrate the first type of schools. In the case of this Arabic-English bilingual school, the aim of the teaching is to increase the knowledge of the students in one or the other language by building on each of the two school languages, based on an asset-based/strength-based approach. The mandate of this private school is to

82  Emmanuelle Le Pichon et al.

Figure 4.1 The relationship between the language profiles of teachers, schools, and families and its potential influence on the school community’s attitude toward multilingual assessment tools.1

guarantee a high level of education in English building on a solid knowledge of the Arabic language and culture; the school has full authority to sign up for the platform without being restricted by provincial language policies. The expectations of parents are to develop a high degree of bilingualism in their children and many teachers are bilingual in English and in Arabic. Consequently, planning, teaching, and assessing the students will happen flexibly in one of the shared languages. Binogi offering both English and Arabic, teachers showed little to no resistance to a translanguaging pedagogy. They were able to obtain a reasonable insight of their students’ language skills in each language and to understand the linguistic and cultural contexts from which they come. Examples of this implementation are presented below: I post the videos on Google Classroom in English and tell my students that they know how to change the language if they want to. Some students told me that they like to watch the videos with subtitles in their own language especially those ESL students [English as a Second Language]. The platform has been helpful as I have students with different levels of English, and I often ask students to work on it while I work with other groups. We used to watch videos as a class to introduce topic to students. As we watch the video, I would add transcripts in Arabic. After putting the transcripts and allowing students to watch the videos in the language of their choice, I have three language learners who said that they understood the topic much better after they were able to switch between languages.

Digital tools for the assessment of young plurilingual students 83 From their testimonies, it was clear that they considered that a student could be very advanced in one language and weak in another without being perceived as deficient. The teachers were able to use the languages to ensure optimal understanding of the academic content, regardless of the language used and to encourage students to use their plurilingual skills to reach their cognitive goals and get more insights into their students’ competences. In the case of this school, the platform was used to its full potential and all the proposed materials were received with enthusiasm. The teachers participated in all the activities proposed by the research team. A French international school

The second type of school is best illustrated by a so-called French international school whose mandate was very similar to that described above, except that the teachers did not necessarily share the languages of their students. For example, a Mandarin-speaking family might enroll their children, hoping to achieve a higher level of certification by guaranteeing bilingualism in the country’s two official languages, with the addition of a third family language that is not used in school education. Planning, teaching, and assessing students can therefore be done but does not necessarily imply a shared knowledge of languages and cultures. In this situation, it is more difficult for a teacher to get an accurate picture of the students’ linguistic and academic level, and parents can be particularly instrumental in this process. As they do not necessarily master both languages of the school, one of the appealing features of the Binogi platform is that it gives parents and students the opportunity to access the course content in at least one language they understand (English or French, or their own language), the use of the different languages being dictated by their own individual needs. Tools will therefore be needed to highlight student agency in assessment, and parents’ expectations are partly aligned with the use of a multilingual tool such as Binogi since both official languages are included. The independence of the school also gives them full authority to sign up for the platform without any language policy restrictions and based on easily obtained parental consents. A public monolingual school

The third type of school is a public monolingual school, under the Ministry of Education with monolingual instruction on a provincial curriculum and subject to local language policies. The multilingualism of the families does not coincide with the monolingual curriculum and the rigidity of the rules imposed by the language policies. It becomes extremely difficult for teachers to get an accurate picture of the level of their students, both in terms of language and academic content, and equally difficult to create a partnership with parents in this area. Against this context, the use of a platform such as Binogi that allows students to use the languages according to their

84  Emmanuelle Le Pichon et al. own needs in order to access the content and gain a deeper understanding might thus be perceived as controversial. Teachers or teams who decide to adopt it therefore risk opposing the local political and social ideology. It is in this situation all the more necessary to work with teachers and schools to demonstrate the value of such work, to address their ideologies, and to help them find their own scope within the context of rigid language and cultural frameworks, “exploring instructional policies and strategies that enable students to use their emerging academic language and multilingual repertoires” (Cummins et al., 2015, p. 555). It is clear then that the limitations of the multilingual resource are not necessarily located primarily in the resource itself but in the broader organization of schooling in which the resource is implemented. The combination of the three components (the linguistic, cultural, and political profiles of families, teachers, and schools) determines the teaching context in which a platform such as Binogi will be implemented or not and the extent to which it will be implemented. Thus, the dynamics of planning, teaching, and assessing plurilingual students with a tool like Binogi heavily depend on these three components. Addressing the ideologies of school stakeholders

One of the authors of this chapter conducted workshops with teacher candidates, and in-service teachers on the teaching of math to language learners and assessing students’ mathematical skills. In this workshop, participants were presented with a set of simple mathematical questions. The questions were developed with the intent to help teachers understand some of the issues that face language learners. For instance, participants had to solve simple additions and subtractions questions but these questions were presented from left to right as in Arabic rather than right to left as in most European languages. This simple change in the direction of the mathematical sentence gave teachers a better understanding of the difficulties they expose students to every day without realizing it. Another question involved solving questions using different numerical systems. By experimenting their incompetence in their own teaching area, teachers gained a better understanding of their students’ difficulties beyond language. They showed empathy and began to see the impact of linguistic and cultural differences involved in solving mathematical questions. Addressing the challenges of the curricula

Despite its high potential and beyond the societal reception of the tool, a challenge of the Binogi platform is that it is only aligned to the local curriculum. The assessment is therefore based on content that has not necessarily been taught in the student’s country of provenance. Assessing a student on content that has not been taught can be unhelpful or discouraging. Similarly,

Digital tools for the assessment of young plurilingual students 85 content that has been taught extensively can be assessed more thoroughly. In order to help teachers better interpret their students’ possible scores on the Binogi platform, the research team studied the curricula of the students’ home countries. We will illustrate this point with the example of the Syrian curriculum. Two of our team’s researchers, themselves Syrians, systematically compared the Syrian and Ontario Math curricula (see also Wattar & Le Pichon, 2022). This work allowed the team to determine a list of essential objectives for each curriculum by grade, identified by a number of mathematical and scientific concepts and to understand differences in the pedagogical approaches between the two curricula. For example, in which grades these concepts are introduced, how they are explored or approached, and how they are assessed in each curriculum. This comparison also showed that the multilingual tool (Binogi), despite an alignment with the Ontario curriculum, fairly reflects many curriculum topics taught in Syria. This finding supported the idea that Binogi may help to strengthen students’ acquisition of the school language (i.e., math and science) while reinforcing their previous understanding of mathematical and scientific concepts, thereby enhancing the potential use of the tool as a formative or diagnostic assessment. The lists of selected concepts were then compared and presented in summary tables that allowed teachers and families to quickly identify differences between the curricula and any content gaps that needed to be addressed. While the Syrian curriculum focuses more on the body of knowledge, as evidenced by the number and variation of topics covered at each level (e.g., Table 4.1), the Ontario curriculum focuses on the acquisition of four categories of science knowledge and skills, with very clear assessment criteria for each: knowledge and understanding, thinking and investigation, communication, and application, which are not explicitly identified in the Syrian curriculum. This finding shows that assessments often include performance criteria that assess more than just content knowledge. Other skills assumed to be present more or less universally might be absent from the curriculum in the country where students previously attended school. Assessing these skills in the language of the school or in the language of the students becomes difficult and may lead to an underestimation of the students’ academic achievements and skills too. This analysis shows that it is essential to take cultural differences into account with regard to the educational setting. Before being assessed, students must have the opportunity to learn to understand the expectations of the system in question with regard to assessment and to be trained in these new assessment forms. Connecting to families and communities

The results were presented by one of the researchers to the Syrian community in Canada. The parents and families, some of whom were new to the Ontario curriculum, were very enthusiastic about the comparative curriculum

86  Emmanuelle Le Pichon et al. Table 4.1  Comparison of Ontario and Syrian science curricula 2 Ontario curriculum: grade 6 (strands and overall expectations)

Syrian curriculum: grade 6 (topics and main concepts)

S1. Understanding Life Systems: biodiversity

• Relationships between living things: the importance of plants to animals and the importance of animals to plants; predation, parasitism, mutualism, and Saprotrophs • Food chains and food web • Ecosystems: biotic and abiotic constituents • Human impact on the ecosystem • Excretion: urinary system structure and function • Urinary system: health and disease • Vertebrate excretory systems • Skin (Protective Barrier): structure and function • Skin: health and diseases • Sexual reproduction in plants • Asexual reproduction in plants • Agriculture • Reproduction in humans • Pregnancy and birth • Health and family planning • Reproduction in vertebrates • Carbon Cycle • Nitrogen Cycle • Forces in nature • Frictional force: factors affecting frictional force; advantages, applications, and disadvantages of frictional force • Work: definition and calculation • Inclined plane and simple machines • Lever: definition, characteristics, and types • Pulleys and its types • Wheel and axle • Machines: simple and complex

1 Assess human impacts on biodiversity and identify ways of preserving biodiversity 2 Investigate the characteristics of living things and classify diverse organisms according to specific characteristics 3 Demonstrate an understanding of biodiversity, its contributions to the stability of natural systems, and its benefits to humans

S2. Understanding structures and mechanisms: flight 1 Assess the societal and environmental impacts of flying devices that make use of properties of air 2 Investigate ways in which flying devices make use of properties of air 3 Explain ways in which properties of air can be applied to the principles of flight and flying devices S3. Understanding matter and energy: electricity and electrical devices 1 Evaluate the impact of the use of electricity on both the way we live and the environment 2 Investigate the characteristics of static and current electricity and construct simple circuits 3 Demonstrate an understanding of the principles of electrical energy and its transformation into and from other forms of energy

• Molecules: simple and complex molecules • Law of conservation of matter • Astonishing phenomena: electric sparks and lightening • In motion and stationary: moving and reference objects, average velocity, and instantaneous velocity

(Continued)

Digital tools for the assessment of young plurilingual students 87 Table 4.1  (Continued) Ontario curriculum: grade 6 (strands and overall expectations)

Syrian curriculum: grade 6 (topics and main concepts)

S4. Understanding earth and space systems: space

• Earth: earthquakes and their effects on living things and the environment • Space: meteorites and meteors, and their effects on living things • Sun, earth, and moon: solar eclipse, lunar eclipse, and their effects on living things • Earth through time: the effect of slow changes in the earth’s crust on organisms

1 Assess the impact of space exploration on society and the environment 2 Investigate characteristics of the systems of which the earth is a part and the relationship between the earth, the sun, and the moon 3 Demonstrate an understanding of components of the systems of which the earth is a part, and explain the phenomena that result from the movement of different bodies in space

approach. It allowed them to understand the similarities and differences between the Ontario and Syrian curricula and made them feel more connected to their child’s learning and more willing to help. Their reactions were essential to us as they helped us understand the importance of introducing the concept of reciprocity in the evaluation process. Here we use the term reciprocity in the sense that Tingyang and Le Pichon (2019) use it, that is, including the perspective and feedback of the individual from elsewhere on the local school system and its curriculum. It thus brought a new perspective on the classically unidirectional dimension of evaluation by including the principle of mutual knowledge. It is through mutual knowledge that the main actors in education (teachers, parents, and students) may reach an inclusive assessment system that not only capitalizes on students’ previous knowledge but also builds on this knowledge to enrich the classroom. Conclusion Assessing plurilingual students is a complex task. In this chapter, we have tried to show that while the impact of known languages is essential and should be taken into account in assessment, it is only “a tree that hides the forest”. Our study reveals the essential role that multilingual resources like Binogi can play in terms of co-teaching if they are used in a deeply informed way. It also allows us to identify a number of preconditions for their effective use. First, it shows the essential role played by the context, that is, the context of implementation of the resource requires an adequacy between the expectations of the different actors of education (schools, teachers, and

88  Emmanuelle Le Pichon et al. families), expectations deeply impacted by the linguistic profile of these instances. Second, if these expectations are not aligned, the use of these resources depends at least in part on efforts to address teachers’ ideologies and to inform families and communities. Finally, they show that these efforts will be all the more effective if they are based on an in-depth knowledge of the curricula of the countries of origin of the students enrolled in the schools in question. This requires researchers and educational institutions to work hand-in-hand with communities and families to better understand and integrate their needs by taking a holistic approach to the student. In doing so, we are aligning ourselves with what Cummins wrote in 2007: “The more we as educators learn about our students, the more they are likely to learn from us” (Cummins, 2007, p. 4). By providing opportunities for cultural input from students and families, teachers and researchers may gain more insight into students’ backgrounds which in turn can help them overcome social, cultural, and racial factors that may also influence their expectations and evaluations. This will support them in mobilizing student linguistic and cultural resources to enhance learning and engagement inside the classroom (Cummins, 2007). Thus, the usefulness of the Binogi platform for a multilingual approach to evaluation was not contradicted by the study, on the contrary. However, the use of Binogi varied greatly according to the linguistic, cultural, and political profile of the school communities. In a previous study, Attar et al. (2020) had confirmed the relevancy of consulting parents of plurilingual students to obtain a fair assessment of their children’s level in mathematics. The principle of reciprocity that we introduce as a necessary tool to access a fairer assessment is a principle of intersubjectivity. The difficulty is that it is applied to a supposedly objective body: school curricula and their teaching, especially in the case of mathematics and science subjects. However, the results of our research seem to show that the main obstacle to fair multilingual assessment is its supposedly objective dimension. With this study we showed that plurilingual assessments should involve the discovery and enhancement of the students’ prior knowledge, not only by opening the school system to their languages, but by deepening and objectively considering their school systems and what they imply. The objective is not to assimilate the students to the local system but to allow them to enrich their knowledge and that of the class through an open sharing of curriculum content. It is a question of moving away from the supremacy of the globalized monolingual and monocultural system, which is challenged by the increasing presence of these plurilingual students in Canadian and European classrooms. In this research, we entered this process by identifying, progressively, and by successive adjustments mathematical and scientifical concepts of both curricula, through the alignment of school programs. These reference points enabled us to better understand the question of the evaluation of students from Canadian and non-Canadian school cultures through an increased reciprocal knowledge.

Digital tools for the assessment of young plurilingual students 89 Notes 1 This figure illustrates the relationship of the language profiles of these three groups to each other, the degree of adequacy or inadequacy of which might determine at least in part the attitude of the school community toward the adoption of multilingual assessment tools. 2 Table 4.1 shows the comparative mapping of the Ontario and Syrian science curricula for grade 6. The Ontario science curriculum is presented in the form of objectives and guidelines and is organized in strands (Ontario Ministry of Education, 2007). The Syrian curriculum is textbook-based and presented in the form of specific topics (Syrian Ministry of Education, 2019–2020). The strands and overall expectations of the Ontario curriculum were compared to the main topics and concepts of the Syrian curriculum.

References Abdi, A. A., Misiaszek, G. W., & Popoff, J. M. (2022).  The Palgrave Handbook on Critical Theories of Education. Springer International Publishing AG. https://doi. org/10.1007/978-3-030-86343-2 Akcaoglu, M. (2016). design and implementation of the game-design and learning program. TechTrends, 60(2), 114–123. https://doi.org/10.1007/s11528-0160022-y. Attar, Z., Blom, E., & Le Pichon, E. (2020). Towards more multilingual practices in the mathematics assessment of young refugee students: Effects of testing language and validity of parental assessment. International Journal of Bilingual Education and Bilingualism. https://doi.org/10.1080/13670050.2020.1779648. Brevik, L. M., & Rindal, U. (2020). Language use in the classroom: Balancing target language exposure with the need for other languages. Tesol Quarterly, 54(4), 925–953. https://doi.org/10.1002/tesq.564. Cenoz, J., & Gorter, D. (2013). Towards a plurilingual approach in English language teaching: Softening the boundaries between languages. Tesol Quarterly, 47(3), 591–599. https://doi.org/10.1002/tesq.121. Collier, V. P. (1987). Age and rate of acquisition of second language for academic purposes. Tesol Quarterly, 21(2), 617–641. https://doi.org/10.2307/3586986. Collier, V. P., & Thomas, W. P. (1989). How quickly can immigrants become proficient in school English? Journal of Educational Issues of Language Minority Students, 5, 26–38. https://static1.squarespace.com/static/5d854ac170e64a71d1de71d3/ t/5d9de668268cee06bbd95fb9/1570629236269/how-quickly-can-immigrants-. pdf. Creese, A., & Blackledge, A. (2010). Translanguaging in the bilingual classroom: A pedagogy for learning and teaching? The Modern Language Journal, 94(1), 103–115. https://doi.org/10.1111/j.1540-4781.2009.00986.x. Cummins, J. (1981). Empirical and theoretical underpinnings of bilingual education. Journal of Education, 163(1), 16–29. https://doi.org/10.1177/0022057481163 00104. Cummins, J. (1989). A theoretical framework for bilingual special education. Exceptional Children, 56(2), 111–119. https://doi.org/10.1177/001440298905600203. Cummins, J. (2000). Language, power and pedagogy: Bilingual children in the crossfire. Multilingual Matters Ltd. https://doi.org/10.1080/15235882.2001.10162800.

90  Emmanuelle Le Pichon et al. Cummins, J. (2007). What works? Research into practice. Literacy and Numeracy Secretariat, Ontario Ministry of Education. http://www.edu.gov.on.ca/eng/literacynu meracy/inspire/research/cummins.pdf. Cummins, J. (2017). Teaching minoritized students: Are additive approaches legitimate? Harvard Educational Review, 87(3), 404–425. http://dx.doi.org/10. 17763/1943-5045-87.3.404. Cummins, J., Hu, S., Markus, P., & Kristiina Montero, M. (2015). Identity texts and academic achievement: Connecting the dots in multilingual school contexts. TESOL Quarterly, 49(3), 555–581. https://doi.org/10.1002/tesq.241. Garcia, O., & Li, W. (2014). Translanguaging: Language, bilingualism, and education. Bilingual Research Journal, 3(37), 366–369. https://doi.org/10.1080/15235882 .2014.965361. Gottleib, M. (2021). Classroom assessment in multiple languages: A handbook for teachers. Cornwin. https://us.corwin.com/en-us/nam/classroom-assessment-inmultiple-languages/book269863. Johnson, L., Adams, S., & Haywood, K. (2011). NMC horizon report: 2011 K-12 edition. Austin, TX: New Media Consortium. LaCosse, J., Canning, E. A., Bowman, N. A., Murphy, M. C., & Logel, C. (2020). A social-belonging intervention improves STEM outcomes for students who speak English as a second language. Science Advances, 6 (40). https://doi.org/10.1126/ sciadv.abb6543. Le Pichon, E., Cummins, J., & Vorstman, J. (2021). Using a web-based multilingual platform to support elementary refugee students in mathematics, Journal of Multilingual and Multicultural Development. https://doi.org/10.1080/01434632.202 1.1916022. Le Pichon-Vorstman, E., & Kambel, E. (2016). Challenges of mathematics education in a multilingual post-colonial context: The case of Suriname. In Z. Babaci-Wilhite (Ed.), Human rights in language and STEM education (pp 221–240). https://doi. org/10.1007/978-94-6300-405-3_13. Le Pichon-Vorstman, E., Siarova, H., & Szonyi, E. (2020). The future of language education in Europe: Case studies of innovative practices. NESET Analytical Report. https://doi.org/10.2766/478776. Levin, T., & Shohamy, E. (2008). Achievement of immigrant students in mathematics and academic Hebrew in Israeli school: A large-scale evaluation study. Studies in Educational Evaluation, 34(1), 1–14. https://doi.org/10.1016/j. stueduc.2008.01.001. Lidz, C. S. (1991). Practitioner’s guide to dynamic assessment. New York: Guilford Press. OECD (2016). Education at a glance 2016: OECD indicators. Paris: OECD Publishing. https://doi.org/10.1787/eag-2016-en Ontario Ministry of Education. (2007). The Ontario curriculum grades 1–8: Science and technology. Queen’s Printer for Ontario. http://www.edu.gov.on.ca/eng/ curriculum/elementary/scientec18currb.pdf Ontario Ministry of Education. (2008). Supporting English language learners with limited prior schooling: A practical guide for Ontario educators. Queen’s Printer for Ontario. http://www.edu.gov.on.ca/eng/document/manyroots/ell_lps.pdf. Reljić, G., Ferring, D., & Martin, R. (2015). A meta-analysis on the effectiveness of bilingual programs in Europe. Review of Educational Research, 85(1), 92–128. https://doi.org/10.3102%2F0034654314548514.

Digital tools for the assessment of young plurilingual students 91 Stacy, S. T., Cartwright, M., Arwood, Z., Canfield, J. P., & Kloos, H. (2017). Addressing the math- practice gap in elementary school: Are tablets a feasible tool for informal math practice? Frontiers in Psychology, 8, 179. Syrian Ministry of Education. (2019–2020). Science textbook: Grade 6. Syria: General Printing Corporation. Tingyang, Z., & Le Pichon, A. (2019). Un dieu ou tous les dieux. Cent mille milliards, Transcultura. Paris: Gallimard. Wattar, D., & Le Pichon, E. (2022). Mobility of Syrian students and continuity of math education: A comparative curriculum mapping approach. In A. A. Abdi & G. W. Misiaszek (Eds.), International handbook on critical theories of education. Palgrave. Wylie, C., & Lyon, C. (2012). Formative Assessment - Supporting Students’ Learning. R&D Connections, 19, 1–12. https://www.ets.org/Media/Research/pdf/ RD_Connections_19.pdf

5

Functional multilingualism in educational assessment Fauve De Backer, Wendelien Vantieghem, and Piet Van Avermaet

Introduction One of the major issues with standardised tests of language proficiency is the narrow approach to what is seen as the “correct” and “appropriate” language for purposes of testing, which ignores the sociolinguistic realities of (multilingual) test takers who are still in the process of learning a language (Menken & Shohamy, 2015). As such, these tests leave no room for an alternative view of language abilities. Similarly, many tests on content knowledge take an idealised version of monolingual speakers as the norm, including the assumption of a standard language user who is equipped with academic language proficiency. This results in multilingual learners being unable to show what they know and can do. In many cases, multilingual learners have the content knowledge (e.g. mathematics or science) but the channel for expressing it (the language) is a barrier to doing so. Language proficiency and content knowledge are most often regarded as separated constructs. From the perspective of content assessment, language is seen as a source of construct-irrelevant variance – “variance in scores that is not related to the construct being assessed” (Llosa, 2017, p. 4) and vice versa (Abedi, 2004; Abedi et al., 2004; Llosa, 2017). Parallels can be drawn between the fields of language testing and that of language in the testing of content knowledge in the sense that teachers and test developers are struggling with assessing multilingual learners in a fair and valid way in both areas. Language testing associations for example are engaging in discussion on how to recognise and assess the plurilingual competences of language learners (Seed, 2020). So what both tests of language and tests of content have in common is that, when it comes to multilingual learners, they occupy an uncomfortable position in terms of their capacity to provide valid information about the competences of the test takers. The focus of this chapter is on language in testing, rather than purely language testing. In what follows, we will discuss some of the literature on assessment of language learners and investigate the extent to which accommodations can be a means of overcoming some of the difficulties language learners face when demonstrating their competences in the language of schooling (LoS). DOI: 10.4324/9781003177197-7

Functional multilingualism in educational assessment 93 Literature review Limited linguistic abilities in the LoS threaten valid conclusions of language learners’ test results (Abedi, 2017; Wolf et al., 2012). Valid conclusions are of an importance that should not be underestimated, as they inform teachers’ decisions on curriculum tracks, referral to remedial support, graduation and so on. Multilingual learners are more often assigned to lowly esteemed curriculum tracks (Vantieghem et al., 2018), run a higher risk of drop-out (Garcia et al., 2008) and are disproportionally referred to special education (Counts et al., 2018). Accommodations are proposed for large-scale content area assessments to help address issues with validity and fairness. A testing accommodation is defined as “any change to standardised testing conditions intended to make the test more fair and accessible for an individual or subgroup that does not change the construct being measured” (Educational Testing Service, 2009). Accommodations were originally introduced in the field of special education and at first, they were directly applied to emergent bilinguals in the same way as they were to pupils with disabilities (Schissel, 2014). However, giving a test in larger print for example, will not address the needs of pupils who are not yet proficient in the language of assessment, so this kind of accommodation would be labelled ineffective for these pupils. As Mihai (2017, p. 66) states: “in order to be effective, an accommodation should level the playing field so that content, not language proficiency is assessed”. Research on the effectiveness of accommodations addresses the following question: “Do accommodation strategies help to reduce the performance gap between English Language Learners (ELL) and non-ELL pupils by removing language barriers?” (Abedi et al., 2003, p. vii). In other words, “an accommodation is effective if it helps remove the construct-irrelevant sources and make assessments more accessible for the recipients” (Abedi, 2017, p. 303) and helps them overcome linguistic barriers (Cohen et al., 2017). Effectiveness of accommodations for multilingual pupils is investigated by examining the extent to which pupils receiving the accommodation demonstrate a significant increase in test scores (Cohen et al., 2017; Kieffer et al., 2009) in comparison to pupils who are also learning the language of assessment but that are tested without the accommodation (Abedi et al., 2003). The effectiveness of an accommodation is studied in combination with a test on the validity hypothesis, because a significant difference in performance between accommodated and non-accommodated pupils may also suggest the accommodation had an impact on the construct and hence changing the focus of the assessment. There is a large variety of accommodations available and different systems exist to classify them (Mihai, 2017). Butler and Stevens (1997) distinguish between modifications of the test itself and modifications of the testing procedure, while Rivera and Stansfield (2004) classified accommodations into four main types: (a) presentation, (b) response, (c) setting and (d) timing/ scheduling. More recently, Mihai and Pappamihiel (2014) propose six categories to summarise possible accommodations (Mihai, 2017), distinguishing

94  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet between test directions, presentation, support and response format as well as time and location. In the taxonomy by Kieffer et al. (2012) the emphasis is on the importance of considering language in testing, by distinguishing two categories of accommodations: indirect linguistic support which involves adjustments to the conditions of a test and direct linguistic support, which involves adjustments to the language of the test. Examples of indirect linguistic support accommodations include giving pupils more time or allowing them to be tested individually or in a small group. Examples of direct linguistic support are translations, glossaries and simplified text (Educational Testing Service, 2009). For multilingual learners, addressing the linguistic barrier is assumed to be most effectively done by means of linguistic accommodations, but research on the effectiveness of accommodations remains inconclusive (Abedi et al., 2003; Abedi & Ewers, 2013; Educational Testing Service, 2009). In this chapter we focus on a bilingual test and read-aloud accommodations, both examples of direct linguistic support accommodations. These accommodations are suggested to reduce the language barrier that multilingual learners face when they are given tests that were actually designed for monolingual learners (Abedi et al., 2003). The reasoning behind read-aloud accommodations is that some learners may prefer oral stimuli over visual stimuli and that hearing a test question with proper intonation may help pupils to understand it (Castellon-Wellington, 2000). A study by Buzick and Stone (2014) suggests that this kind of accommodation may be helpful for some subjects but not for others, as they found an increase in reading scores but not in mathematics scores. Although few in number, the majority of the existing studies on read-aloud accommodations were only conducted in the LoS. It is suggested that read-alouds in the pupils’ L1 are more responsive to their needs (Kieffer et al., 2012), even more so than written translations, because when pupils have not received any formal instruction in their L1s, their reading and writing skills are often not as developed as their listening skills (Stansfield, 2011). In a bilingual test, pupils receive the original test in the LoS and a translation in their L1. The few research studies that have been done with bilingual tests provide a positive picture on the effectiveness of this strategy. According to research by Shohamy (2011) and Heugh et al. (2017), bilingual tests have the potential to level the playing field, but as said, the effectiveness of written tests in the L1 depends on pupils’ literacy skills in that language. Hence, in this chapter, we will compare and discuss the added benefit for multilingual pupils of both read-aloud accommodations in the LoS and the L1, and of bilingual tests. Research questions The central aim of our research is to explore how multilingual children who are not yet proficient in the LoS can be assessed both fairly and in a valid way in content-related subjects. Therefore, we explored whether assessment accommodations are effective for multilingual learners taking a science test.

Functional multilingualism in educational assessment 95 a Do read-aloud accommodations (recorded audio support) in the LoS contribute to better results in science achievement for multilingual learners? b Does a written bilingual test contribute to better results in science achievement for multilingual learners? c Does a read-aloud accommodation (recorded audio support) in the pupils’ L1 contribute to better results in science achievement for multilingual learners? Research context In order to investigate the valid assessment of multilingual learners, the MuLAE-project (Multilingual Assessment in Education) was conducted from 2015 to 2020 in Flanders, the northern part of Belgium. Two characteristics of the Flemish educational system are central to understanding the research context. Firstly, Flemish education is guided by the principle of “freedom of education”. Up until recently (2021) there was no tradition of centralised testing and teachers were free to decide on the kinds of assessments they want their pupils to take (Ysenbaert et al., 2017). Secondly, monolingual beliefs continue to dominate educational practices and policies (Pulinx et al., 2015). Flanders is hardly unique in this respect; in line with many other European contexts, many of the educational staff, policymakers and parents are convinced that forbidding non-majority languages will benefit pupils’ achievement, meaning that a “Dutch-only” approach is widely supported (Blommaert & Van Avermaet, 2008; Pulinx et al., 2015). Flanders/Belgium has in fact experienced several migration waves throughout the 20th and 21st century, resulting in classrooms that are ethnically and linguistically diverse. That is, about 20% of the pupils enrolled in compulsory education in Belgium have a migration background, with the largest groups coming from Morocco, Eastern-Europe and Turkey (Verhaeghe et al., 2012). On average in 2019–2020, 18% of pupils speak another language at home than the LoS. The proportion of second language pupils is usually larger in urban regions, where the existence of ten or more different home languages within a single classroom is no exception. Hence, the Flemish context provides us not only with a research setting in which we can observe a diversity of languages present in the classroom, but also one in which the mastery of both the first language (L1) and second language (L2) varies depending on the migration history of the family. Participants and procedures From 2016 to 2017 data were collected from 1,022 fifth-grade pupils (aged 10–11 years; 49.3% boys and 50.7% girls; and 65.6% with no experience of schooling in a country other than Belgium) in 35 primary schools in Flanders. The schools were selected in order to achieve a sample with a high proportion of pupils reporting a home language other than Dutch (the LoS). All primary schools in Flanders where more than 60% of pupils did not use Dutch as their

96  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet Table 5.1   O verview of testing-conditions, participant categories and number of participating pupils

L1 Dutch pupils Multilingual pupils Total

DU/A−

DU/A+

BIL/A−

BIL/A+

Total

45 385 490

44 450 494

64 64

69 69

89 1028 1117

L1 (n = 103) were contacted. Not surprisingly, all these schools were located in urban regions, where ethnic and linguistic diversity tends to be concentrated. In total, 34% of the schools contacted agreed to participate. The most commonly mentioned reasons for non-participation were lack of time and too many invitations to take part in research. Thanks to this sampling approach, the majority of pupils in the sample (91%) were multilingual in that they did not (exclusively) speak the LoS at home. Pupils took a background survey and an online science test which consisted of 43 multiple-choice items from the fourth grade of the Trends in International Mathematics and Science Study (TIMSS). In relation to RQ 3, we obtained translated versions in Polish and Turkish from the National Project Centers in Poland and Turkey, which were chosen because they represent some of the most commonly represented immigrant languages in Flanders. The participants were randomly assigned to one of four research conditions (Table 5.1). These were: DU/A− (Dutch-only test), DU/A+ (Dutch-only test with audio-support in LoS), BIL/A− (Bilingual test without audio-support), BIL/A+ (Bilingual test with audio-support in both LoS and L1). In the cases with audio-support (A+), the pupils could listen to a read-aloud version of each question and answer options of the science test. In the cases with translations (BIL), the Dutch and the translated version (in either Polish or Turkish) of the question were simultaneously displayed on the laptop screen. Because bilingual tests and L1 read-alouds were available in only two languages, only Polish- and Turkish-speaking pupils were assigned to all four research conditions. The other pupils were randomly assigned to either the DU/A− condition or the DU/A+ condition. Results (A) Do read-aloud accommodations in LoS contribute to better results in science achievement for multilingual learners? 

The effectiveness of a read-aloud accommodation in the LoS (Dutch) on the science achievement of multilingual learners was investigated. The hypotheses whether pupils given read-aloud accommodation in the LoS perform better than their non-accommodated peers and whether we observe a differential impact of the read-aloud accommodation on multilingual pupils versus L1 speakers of the LoS were tested (Table 5.2).

Functional multilingualism in educational assessment 97 Table 5.2   O verview of testing-conditions, participant categories and number of participating pupils in the read-aloud study

L1 Dutch pupils Multilingual pupils Total

DU/A−

DU/A+

BIL/A−

BIL/A+

Total

45 445 490

44 450 494

64 64

69 69

89 1028 1117

The relationship between the type of accommodations pupils received and their science achievement was investigated using multilevel modelling conducted in MlwiN 2.29. Results indicated that L1 speakers of the LoS significantly outperform their multilingual peers (b = 1.87, SE = .62, p < .01). The read-aloud accommodations provided did not significantly contribute to a fairer assessment. The results show that the accommodation group pupils were assigned to was not significantly related to the differences in science achievement of the pupils (Multilingual accommodated: b = 0.380, χ2 (1) = 1.25, p > .05; L1 Dutch accommodated: B = 2.248, χ2 (1) = 7.70, p < .01; L1 Dutch non-accommodated: b = 3.03, χ2 (1) = 13.86, p < .001). In other words, accommodated pupils in this study did not score better or worse than non-accommodated pupils. (B) Does a written bilingual test contribute to better results in science achievement for multilingual learners? (C) Does a read-aloud accommodation in the L1 contribute to better results in science achievement for multilingual learners?

To answer the second and third research question, multilingual Turkish- and Polish-speaking pupils were randomly assigned to either a condition in which they received a written bilingual test (BIL/A−), a condition in which they received a written bilingual test and a read-aloud accommodation in both the LoS and their L1 (BIL/A+) or a non-accommodated test (DU/A−). A univariate analysis of variance shows no significant difference between conditions, F (2, 194) = .17, p = .84, η2 = .006, suggesting that neither of the two accommodations (the BIL/A− and BIL/A+) were effective in improving the test score of multilingual pupils (Table 5.3). (A + B + C) Explaining the limited effectiveness of a read-aloud accommodated test and a bilingual test for multilingual learners by investigating the within-group variance

As discussed above, we investigated the effectiveness of a (monolingual) readaloud accommodation (DU/A+) and a bilingual testing condition, a written test with (BIL/A+) and without (BIL/A−) read-aloud accommodation in the pupils’ L1. Neither experiment showed improved results for multilingual pupils on a science test. The non-significant results may be explained by the fact

98  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet Table 5.3   O verview of testing-conditions, participant categories and number of participating pupils in the bilingual study

L1 Dutch pupils Multilingual pupils Total

DU/A−

DU/A+

BIL/A−

BIL/A+

Total

45 445 105

44 450 44

64 64

69 69

89 193 282

that multilingual learners are a heterogeneous group who differ, for example, in terms of their language proficiency, both in their L1 and in the LoS. What works for one pupil does not necessarily work for other pupils. Hence, these issues might explain why we could not detect differences in the test scores between the groups. Post hoc analyses using the Scheffé post hoc criterion for significance (see Table 5.4) indicated significant differences between groups in terms of their self-reported proficiency and literacy skills. Overall, L1 Dutchspeaking pupils rate their proficiency in the LoS (M = 4.4, SD = .51) significantly higher than Turkish-speaking pupils (M = 4.06, SD = .55, F(3, 1057) = 12.64, p < .001). Also, while Turkish- and Polish-speaking (M = 4.08, SD = .64) pupils did not differ in their LoS proficiency, both groups reported significantly lower scores for proficiency in the LoS than multilingual pupils from other linguistic backgrounds (M = 4.29, SD = .55). In terms of literacy in the LoS, L1 speakers (M = 4.54, SD = .50) and other multilingual pupils (M = 4.39, SD = .59) reported significantly higher skills than both Turkish-speaking (M = 4.11, SD = .63) and Polish-speaking pupils (M = 4.01, SD = .72, F(3, 1050) = 18.65, p < .001). Proficiency skills in L1 did not differ significantly between groups (excluding the Dutch L1 speakers). Remarkably, Turkishspeaking pupils were more confident about their literacy skills in the L1 (M = 3.77, SD = 1.04) compared to pupils from other linguistic backgrounds (M = 3.26, SD = 1.30). However, the small sample size for the Polish-speaking group (n = 30) must be taken into account and may well explain why the difference in reported L1 proficiency and literacy is not significant for the Polishspeaking group (M = 4.71, SD = .65) compared to the Turkish-speaking pupils (M = 4.28, SD = .75) and pupils from other linguistic backgrounds (M = 4.36, SD = .80), F(3, 1070) = 10.10, p < .001. Table 5.4  Mean scores on a scale from 0 (= not proficient at all) to 5 (= very proficient) (SD)

L1 = Dutch L1 = Turkish L1 = Polish Other linguistic backgrounds

Proficiency Dutch

Literacy Dutch

Proficiency L1

Literacy L1

4.4 (.51) 4.06 (.55) 4.08 (.64) 4.29 (.55)

4.54 (.50) 4.11 (.63) 4.01 (.72) 4.39 (.59)

4.28 (.75) 4.71 (.65) 4.36 (.80)

3.77 (1.04) 3.8 (1.19) 3.26 (1.3)

Functional multilingualism in educational assessment 99 Table 5.5   O verview of testing-conditions, participant categories and number of participating pupils in the bilingual study with read-alouds in the L1

L1 Dutch pupils Multilingual pupils Total

DU/A−

DU/A+

BIL/A−

BIL/A+

Total

45 445 490

44 450 494

64 64

69 69

89 1028 1117

Since previous research (Acosta et al., 2008) has illustrated how the effectiveness of accommodations may vary depending on the different learners’ characteristics, our study on bilingual tests and read-aloud accommodations explicitly explored within-group variance among multilingual pupils. For a full report on the analyses and the results, we refer to De Backer et al. (forthcoming). A multiple linear regression within the condition with the bilingual test with read-alouds (BIL/A+: Table 5.5) determined how the within-group variance differentially impacts the use and functioning of accommodations, considering both main effects and interaction effects. These analyses reveal that proficiency in the L1 (F (2, 62) = 4.538, p < 0.05, f  2 = .015) and frequency of use of the read-alouds in the L1 (F (2,62) = 9.811, p > .001, f  2 = .31) significantly predict the science performance of pupils. For pupils who used the read-alouds often, the significant interaction effect with L1 proficiency indicates that the beneficial effect becomes more pronounced when pupils are more proficient in their L1. Discussion: Towards a framework for talking about multilingualism in assessment Pupils with an immigrant background score lower on all PISA tests than nonimmigrants and the gap is especially large in countries like Belgium. When considering the language spoken at home, it is clear that not speaking the language of assessment at home is a barrier to attaining high scores on standardised tests (OECD, 2019). The stakes for these pupils are high, as they are often assigned to curriculum tracks not oriented towards higher education, are disproportionately referred to special education (Counts et al., 2018) and run a higher risk of dropping out (Garcia et al., 2008; Groenez et al., 2009). One of the aspects to address in resolving these issues is removing constructirrelevant barriers in testing. This is essential to ensure fairness in scoring, because when linguistic complexity unnecessarily interferes with pupils’ ability to illustrate their competences, this poses a serious validity concern (Wolf et al., 2008). Direct linguistic accommodations such as read-aloud accommodations or bilingual tests are suggested to help overcome some of these issues. The results of the experimental research indicate that neither the readaloud assessment accommodations nor the bilingual test significantly increased multilingual pupils’ science achievement. That the bilingual tests did not produce significant benefits to the multilingual learners was rather surprising

100  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet and contradictory to the results of previous research such as that of Shohamy (2011) and Heugh et al. (2017), who reported beneficial effects of bilingual tests. An important explanation for the findings in the present study is that multilingual pupils in Flanders were very hesitant to make extensive use of the provided accommodations. The average frequency with which the read-aloud accommodation in the LoS was used was extremely limited. Out of all the available audio options in the LoS, on average only 5% were used (SD = .09). In Flemish education, pupils are unfamiliar with an assessment procedure in which listening to the questions on a computer is possible. Before letting the students take the test, the researchers provided them with video-instructions on how to make use of the accommodations, but this was most likely not sufficient in making them aware of the potential benefits of these accommodations and to take away their hesitance in using the accommodations thoroughly. Nevertheless, no negative effects were found either. Since there is “no harm” in applying the read-alouds in the LoS and the fact that it may be beneficial for some pupils (e.g. pupils with learning difficulties such as dyslexia, which were not the focus of the research reported on here), there is no reason to discourage teachers from using this approach. Reading the questions aloud is a popular accommodation that teachers spontaneously provide in Flemish classrooms. In interviews, Flemish primary school teachers explain how by either reading the questions out loud themselves or by asking the pupils to repeat the question in their own words, teachers aim to make sure the pupils understood the question (De Backer et al., 2015), an accommodation provided to the whole classroom. When used frequently and when pupils are proficient in their L1, readaloud accommodations in the L1 may well have a beneficial effect on the ability of pupils to show their science competences on a traditional test. Above all, our findings demonstrate the importance of exploring withingroup differences (what works for one group of multilingual pupils may not work for another), but also of investigating and reporting on the intensity of use of read-aloud accommodations and bilingual tests. This is admittedly probably harder to do in the case of the latter than with the former. Eyetrack research could be an interesting method to explore whether and how learners make use of a translation, such as the study by Butler (2021) which investigated university students’ results using English and Sesotho academic literacy tests. It must be noted that the concept of accommodations has also been criticised. That is, the policy seems to be granting multilingual pupils these aids on a temporary basis until a pupil achieves the norm, which is becoming as proficient as an L1 speaker. This is problematic, not only because multilingual learners rely on their L1s for a long time (Shohamy, 2011), but also because monolingualism remains the norm which is not questioned nor challenged (De Backer et al., 2019b). Also, while large-scale assessments have more resources to develop and test effective accommodations for multilingual learners, the same cannot be said of schools and teachers. Consequently, we must take

Functional multilingualism in educational assessment 101 into account how teachers can apply insights from accommodations-research into their day-to-day classroom practice and whether this approach is not only most desirable and effective, but also most feasible. Feasibility is often and understandably so an important criterion for educators. If we want to consider changes in the way we assess multilingual learners, we must also take into account the concerns and beliefs of the education staff, because beliefs and behaviour impact each other. Interviews with Flemish teachers demonstrate how they recognise the language barrier that multilingual pupils face, but most of the interviewed teachers feel strongly that bilingual tests are a bridge too far. They reason that bilingual tests would hinder the learning of the LoS (De Backer et al., 2019a). Accommodations can teach valuable lessons, but are only one possible way forward in multilingualism in assessment, as described in the framework of Functional Multilingual Assessment (De Backer et al., 2017, 2020). In this framework, two shifts are suggested: from a testing culture towards an assessment culture and from monolingual towards multilingual assessments. The first shift represents a transition from an assessment in one language towards an assessment where learners can use their entire linguistic repertoire. This includes linguistic accommodations such as the read-aloud accommodations and the bilingual tests we discussed in this chapter. However, a test with linguistic accommodations is still a traditional standardised test and as Jones and Saville (2016) argue, these kinds of tests have been systematised to a level where they do not benefit the learning process anymore as the results of these tests are often not used to inform teachers and pupils. A second shift moves away from a culture focused purely on testing towards an assessment culture. The learning-oriented approach developed by the Cambridge English Language Assessment presents a model that is systemic and ecological in the sense that all kinds of assessment – both formal and informal – contribute to better learning and meaningful interpretation of learning outcomes (Jones & Saville, 2016). It emphasises that all levels of assessment can contribute to the effectiveness of learning and reliable evaluation of outcomes (Cambridge English, 2019). The combination of both shifts (from a testing culture towards an assessment culture and from a monolingual test/assessment towards a multilingual assessment) is what is called “Functional Multilingual Assessment”. It is based on the concept of Functional Multilingual Learning (FML) by Sierens and Van Avermaet (2014). FML recognises the language abilities that language learners bring in when learning an additional language and promotes the functional use of their entire repertoire for both language and content learning. This concept is useful for the context of assessment as well. We consider learning and assessment to be inseparable and when we consider the entire linguistic repertoire of language learners to be a resource for learning both language and content, it would not make sense to disconnect learners from this resource at the moment of assessment. This approach also addresses the concerns teachers have, because of the “functional” aspect in using the linguistic repertoire as a resource.

102  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet Conclusion The central aim of this research project was to explore how multilingual children who are not yet proficient in the LoS can be assessed in a fair and valid way in content-related areas. We empirically investigated the effectiveness of read-aloud assessment accommodations and bilingual tests for multilingual pupils. Both types of accommodations did not render significant benefits in terms of multilingual learners’ science achievement, a finding that most likely could be explained by the fact that multilingual pupils were unfamiliar with these accommodations and thus made too little use of them. It would be worthwhile exploring whether intensive users of the accommodations do benefit more from it. Accommodating multilingual learners who take a test is one way to compensate for linguistic barriers experienced by these learners. These compensations and tests, however, still depart from the monolingual norm. We need to radically rethink the way we are assessing multilingual learners, taking different ways and forms of assessments into account that take the multilingual learner as a starting point, a suggestion illustrated by the framework of Functional Multilingual Assessment. References Abedi, J. (2004). The no child left behind act and English language learners: Assessment and accountability issues. Educational Researcher, 33, 4–14. Abedi, J. (2017). Utilizing accommodations in assessment. In E. Shohamy (Ed.), Language testing and assessment, encyclopedia of language and education (pp 2462–2478). Boston, MA: Springer. Abedi, J., Courtney, M., & Leon, S. (2003). Effectiveness and Validity of Accommodation for English Language Learners in Large-Scale Assessments (CSE Report 608). Los Angeles, CA: University of California, Los Angeles, National Center for Research on Evaluation, Standards and Students Testing (CRESST). Abedi, J., & Ewers, N. (2013). Accommodations for English language learners and students with disabilities: A research-based decision algorithm. Smarter Balanced Assessment Consortium. Davis, CA: University of California, Davis. Abedi, J., Hofstetter, C. H., & Lord, C. (2004). Assessment accommodations for English language learners: Implications for policy-based empirical research. Review of Educational Research, 74(1), 1–28. https://doi.org/10.3102/003465430 74001001. Acosta, B. D., Rivera, C., & Shafer Willner, L. (2008). Best Practices in state assessment policies for accommodating English language learners: A Delphi Study. Arlington, VA: The George Washington University Center for Equity and Excellence in Education. Blommaert, J., & Van Avermaet, P. (2008). Taal, onderwijs en de samenleving. De kloof tussen beleid en realiteit. Berchem: EPO. Butler, F. A., & Stevens, R. (1997). Accommodation strategies for English language learners on large-scale assessments: Student characteristics and other considerations (Vol. 448): Center for Research on Evaluation, National Center for Research on

Functional multilingualism in educational assessment 103 Evaluation, Standards, and Student Testing, Graduate School of Education & Information Studies, University of California, Los Angeles. Butler, G. (2021). University students’ utilisation of information in simultaneous exposure to the same text in different languages. Paper presented at the Language Testing Research Colloquium (LTRC). Buzick, H., & Stone, E. (2014). A meta-analysis of research on the read aloud accommodation. Educational Measurement – Issues and Practice, 33(3), 17–30. https:// doi.org/10.1111/emip.12040. Cambridge English. (2019). Learning Oriented Assessment. Retrieved from https:// www.cambridgeenglish.org/research-and-validation/fitness-for-purpose/loa/. Castellon-Wellington, M. (2000). The Impact of Preference for Accommodations: The Performance off English Language Learners on Large-Scale Academic Achievement Tests. (CSE Technical Report 524). Los Angeles, CA: CRESST/University of California, Los Angeles. Cohen, D., Tracy, R., & Cohen, J. (2017). On the effectiveness of pop-up English language glossary accommodations for EL students in large-scale assessments. Applied Measurement in Education, 30(4), 259–272. https://doi.org/10.1080/08957347 .2017.1353986. Counts, J., Katsiyannis, A., & Whitford, D. K. (2018). Culturally and linguistically diverse learners in special education: English learners. NASSP Bulletin, 102(1), 5–21. https://doi.org/10.1177/0192636518755945. De Backer, F., Van Biesen, L., Agirdag, O., Slembrouck, S., Struys, E., Van Avermaet, P., Van Houtte, M. (2015). MARS. Meertaligheid Als Realiteit op School. Flemish Department of Education: Project report. https://data-onderwijs.vlaanderen.be/ documenten/bestand.ashx?nr=6366 De Backer, F., De Cooman, E., Slembrouck, S., & Van Avermaet, P. (2019a). Multilingual assessment – beliefs and practices of geography teachers. In C. Kirsch & J. Duarte (Eds.), Multilingual approaches for teaching and learning: From acknowledging to capitalising on multilingualism in European mainstream education. London: Routledge. De Backer, F., Slembrouck, S., & Van Avermaet, P. (2019b). Assessment accommodations for multilingual learners: Pupils’ perceptions of fairness. Journal of Multilingual and Multicultural Development, 40(9), 833–846. https://doi.org/10.1080/ 01434632.2019.1571596. De Backer, F., Slembrouck, S., & Van Avermaet, P. (2020). Functional use of multilingualism in assessment: Opportunities and challenges. In Cambridge Assessment English (Ed.), What does plurilingualism mean for language assessment? (Vol. 78, pp 35–43). Cambridge: Cambridge University Press. De Backer, F., Van Avermaet, P., & Slembrouck, S. (2017). Schools as laboratories for exploring multilingual assessment policies and practices. Language and Education, 31(3), 217–230. https://www.tandfonline.com/doi/full/10.1080/09500782.20 16.1261896 De Backer, F., Van Tieghem, W., Slembrouck, S., & Van Avermaet, P. (forthcoming). Effectiveness of Multilingual Assessment of Science Competences. Educational Testing Service. (2009). Guidelines for the Assessment of English Language Learners. Stanford University. Retrieved from https://www.ets.org/content/dam/ ets-org/pdfs/about/ell-guidelines.pdf Garcia, O., Kleifgen, J. A., & Falchi, L. (2008). From English language learners to emergent bilinguals. Equity Matters. Research Review No. 1. https://files.eric. ed.gov/fulltext/ED524002.pdf

104  Fauve De Backer, Wendelien Vantieghem and Piet Van Avermaet Groenez, S., Nicaise, I., & De Rick, K. (2009). De ongelijke weg door het onderwijs. In L. Vanderleyden, M. Callens, & N. Jo. (Eds.), De sociale staat van Vlaanderen. Sint-Niklaas: Drukkerij Room. Heugh, K., Prinsloo, C., Makgamatha, M., Diedericks, G., & Winnaar, L. (2017). Multilingualism(s) and system-wide assessment: A southern perspective. Language and Education, 31(3), 197–216. https://doi.org/10.1080/09500782.2016.1261894. Jones, N., & Saville, N. (2016). Learning oriented assessment. A systemic approach. Cambridge, UK: Cambridge University Press. Kieffer, M. J., Lesaux, N. K., Rivera, M., & Francis, D. J. (2009). Accommodations for English language learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168–1201. https:// doi.org/10.3102/0034654309332490. Kieffer, M. J., Rivera, M., and Francis, D. J. (2012). Practical guidelines for the education of English language learners: Research-based recommendations for the use of accommodations in large-scale assessments. 2012 update. Portsmouth, NH: RMC Research Corporation, Center on Instruction. Llosa, L. (2017). Assessing students’ content knowledge and language proficiency. In E. Shohamy, I. Or, & S. May (Eds.), Language testing and assessment, Encyclopedia of Language and Education (pp 3–14). Cham: Springer. https://doi. org/10.1007/978-3-319-02261-1_33 Menken, K., & Shohamy, E. (2015). Invited colloquium on negotiating complexities of multilingual assessment. Paper presented at the AAAL. Mihai, F. M. (2017). Assessing English learners in the content areas. A research-intopractice guide for educators. Ann Arbor, MI: University of Michigan Press. Mihai, F. M., and Pappamihiel, N. E. (2014). Accommodations and English Learners: Inconsistencies in Policies and Practice, TAPESTRY, 6(2). Available at: https:// stars.library.ucf.edu/tapestry/vol6/iss2/2 OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed (1996– 3777). Paris: PISA, OECD Publishing. https://doi.org/10.1787/b5fd1b8f-en. Pulinx, R., Van Avermaet, P., & Agirdag, O. (2015). Silencing linguistic diversity: The extent, the determinants and consequences of the monolingual beliefs of Flemish teachers. International Journal of Bilingual Education and Bilingualism, 1–15. https://doi.org/10.1080/13670050.2015.1102860. Rivera, C., & Stansfield, C. W. (2004). The effect of linguistic simplification of science test items on score comparability. Educational Assessment, 9, 79–105. Schissel, J. L. (2014). Classroom use of test accommodations: Issues of access, equity and conflation. Current Issues in Language Planning, 15(3), 282–295. Seed, G. (2020). What does plurilingualism mean for language assessment? Cambridge English Assessment, 78. https://www.cambridgeenglish.org/blog/whatdoes-plurilingualism-mean-for-language-assessment/ Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. Modern Language Journal, 95(3), 418–429. https://doi.org/ 10.1111/j.1540-4781.2011.01210. Sierens, S., & Van Avermaet, P. (2014). Language diversity in education: Evolving from multilingual education to functional multilingual learning. Managing Diversity in Education: Languages, Policies, 33, 204–230. Stansfield, C. W. (2011). Oral translation as a test accommodation for ELLs. Language Testing, 28(3), 401–416. https://doi.org/10.1177/0265532211404191.

Functional multilingualism in educational assessment 105 Vantieghem, W., Van Avermaet, P., Groenez, S., & Lambert, M. (2018). Diversiteitsbarometer Onderwijs Vlaamse Gemeenschap. In Diversiteitsbarometer onderwijs (pp. 47–193). Brussel: Els Keytsman. Verhaeghe, P., Van der Bracht, K., & Van de Putte, B. (2012). Migrant zoekt toekomst: Gent op een keerpunt tussen oude en nieuwe migratie. Antwerpen: Garant. Wolf, M. K., Herman, J. L., Kim, J., Abedi, J., Leon, S., Griffin, N., …, Shin, H. W. (2008). Providing Validity Evidence to Improve the Assessment of English Language Learners. CRESST Report 738. Retrieved from Los Angeles. Wolf, M. K., Kim, J., & Kao, J. (2012). The effects of glossary and read-aloud accommodations on English language Learners’ performance on a mathematics assessment. Applied Measure in Education, 25(4), 347–374. https://doi.org/10.1080/ 08957347.2012.714693. Ysenbaert, J., Van Avermaet, P., & Van Houtte, M. (2017). Literatuurstudie Evaluatie en diversiteit. Ghent: Steunpunt Onderwijsonderzoek.

6

Assessing language competences of multilingual speakers – a brief review of two test approaches C-tests and TBLA (task-based language assessment) Barbara Hinger

Introduction The multilingual turn in applied linguistics has resulted in defining different constructs of multilingualism such as translanguaging, translingual practices or polylingual and dynamic languaging – to name just a few – that are of interest when conceptualizing multilingualism for language testing, as Schissel et al. (2019, p. 374) point out. Up to now, however, it seems to be much easier to consider the multilingual background of test takers in assessing content achievements than in language testing. According to Baker and Hope (2019, p. 412), when assessing content areas offering the test takers to draw upon their multilingual resources in their test responses allows them “to demonstrate their knowledge of a subject more fully.” This is especially the case for recently arrived immigrants in a country – also considered as emergent bi/ multilinguals – as their literacy skills in the language that the assessment or test tasks are provided in still are at its starting point (Shohamy & Menken, 2015, p. 254). From empirical research we know that the use of multilingual assessments significantly contributes to higher scores on academic tasks and more accurately reflects the knowledge of bi/multilingual test takers … because … “dynamic assessment” offers affordances for emergent bilinguals to use their entire linguistic repertoire flexibly and creatively to process and produce language for academic purposes through various procedures such as mediation and displaying test questions simultaneously in two languages. (ibid. 265) Therefore, it is argued that, in order to contribute to fairer assessment procedures, multilingual assessment approaches for content tests might even challenge international standardized tests like PISA (Programme for International Student Assessment) in which “emergent bilinguals underperform when compared to monolinguals” (ibid. 260). Nonetheless, if research in multilingual content assessment is regarded to be still in its infancy and needs to DOI: 10.4324/9781003177197-8

Assessing language competences of multilingual speakers 107 be explored in considerably more depth (ibid. 265), research in multilingual language testing is even less elaborated. This is the reason why the present paper aims at putting forward insights into recent studies which focus on language testing from a multilingual perspective by scrutinizing existent language test constructs and expanding them toward multilingual purposes. More precisely, in what follows, two – on a general level increasingly used – approaches for testing language competences, recently transferred to assess the language competences of multilingual speakers, will be presented, namely the C-test procedure – as put forward by Drackert and Timukova (2020a, 2020b) and in Gogolin et al. (2021) – and the task-based language assessment (TBLA) approach investigated by Schissel et al. (2018). For both, evidence-based perspectives will be offered, and their underlying theoretical constructs will be discussed. The C-test format for multilingual language speakers As to the C-test format, two studies recently investigated its relevance and use for linguistically heterogeneous learners. While one of the studies explicitly focused on the differences between heritage language learners (HLL) of Russian and foreign language learners (FLL) of Russian (Drackert & Timukova, 2020a, 2020b), the other study analyzed the development of English and French in secondary school contexts by contrasting monolingually and bilingually raised learners (Gogolin et al., 2021). Overall, both studies show that the dichotomous scoring usually applied in C-tests does not cater for measuring the language proficiency of multilingual language learners. Consequently, the authors suggest a polytomous and more complex scoring system to illustrate the learners’ many-facetted linguistic competences. Nevertheless, the C-test procedure as such is not questioned as a valid test format. In fact, the authors rather argue for more diversity in scoring to account for otherwise neglected language skills of multilingual language learners but do not challenge the C-test format as such. Before going into more detail, the C-test format will be briefly defined. In the 1970s, C-tests were developed as alternatives to Cloze tests (Klein-Braley, 1997). Contrary to Cloze tests, which follow the principle of deleting every nth word in a text, C-tests delete a certain number of letters in each word of a given text. More precisely, in a C-test the second half of every second word is deleted. However, to provide a meaningful context for test takers, the first and the last sentence of a text are not mutilated but remain intact. Hence, C-tests apply the so-called 2-2-2 rule which states that “from the second sentence [onwards], delete the half of every second word” (Roever, 2018, p. 298). Yet, proper nouns and numerals are not altered as they cannot be deduced from the context. Like Cloze tests, C-tests are based on the redundancy of languages (Klein-Braley, 1997). By following the above mentioned deletion procedures, the two test formats reduce the redundancy of a text and ask test takers to reconstruct and complete the missing parts.

108  Barbara Hinger Usually, a C-test comprises four to six texts each including between 20–and 25 gaps (see, e.g., Norris, 2018, p. 17). Test takers must reconstruct the texts by filling in the gaps in each word in order “to create a coherent text” (Norris, 2018, p. 12). Or, put differently, test takers “have to restore the deleted parts of the words […] using their lexical and grammatical knowledge of the language as well as their text-processing skills” (Drackert & Timukova, 2020a, p. 109). The reduced redundancy in C-tests does not seem to cause any problems for native speakers as they achieve high scores – usually above 90% – in this kind of tests. This supports the view that the C-test construct elicits knowledge of language structures (morphology, syntax), of the semantic meaning of words as well as collocational relationships between words and cohesive knowledge of a text. As Roever (2018, p. 296) points out: “Arguably, a high level of this kind of knowledge and the ability to access and implement it quickly is conducive to any kind of language use, supporting interpretations of scores as general indicators of proficiency.” Furthermore, numerous studies indicate C-tests as reliable instruments to differentiate among test takers’ language proficiency by highlighting that the reconstruction of deleted items in a text requires – more or less – effort of a language learner depending on his or her general language competence. Following Drackert and Timukova (2020a, p. 109), “[t]he higher the level of proficiency of a testee in the given language, the more gaps they are able to complete correctly.” As a matter of fact, C-tests often correlate “moderately to strongly with recognized criterion measures of proficiency (e.g., Daller & Phelan, 2006), and typically outperform […] other ‘short-cut’ measures in this regard (e.g., Harsch & Hartig, 2016)” (Norris, 2018, p. 15). Hence, C-tests are seen as quick and valid measures of global language proficiency in the written modality (Norris, 2018, p. 17). Traditionally, C-tests follow a dichotomous scoring procedure by awarding one point for the correct reconstruction of all deleted parts in a word. Thus, only the complete knowledge of a word is interpreted as correct answer and scored by one point. In this sense, no differentiations between semantic knowledge, spelling skills or morphological and syntactic knowledge are made. While the dichotomous scoring system is straightforward and enhances a quick assessment as well as online applications of the C-test (including its scoring), it disregards the learners’ partial and/or receptive knowledge of a word. This led Baur and Spettmann (2007) to argue for introducing a further score for valuing word recognition (which they call WE-Wert). The word recognition value refers to the written receptive abilities of test takers by considering their semantic knowledge and overlooking the test takers’ productive skills in writing a word correctly (see, e.g., Drackert & Timukova, 2020a, p. 109). However, this differentiated kind of scoring procedure is not yet widely acknowledged or used (see Drackert & Timukova, 2020a, p. 109). It would, in any case, challenge the construct validity of C-tests which claims to measure global language proficiency. However, as Grotjahn (1996, p. 96, cited in Norris, 2018, p. 14), points out: “There is no once and for all, fixed construct validity for the C-test. Rather […] the construct validity of each

Assessing language competences of multilingual speakers 109 individual test must be separately demonstrated for each specific intended use with each population.” Moreover, Norris (2018, p. 14) argues that “[…] some learners may develop language proficiency in a particular modality (e.g., oral/aural proficiency) but not in the form of literate communication ability, and it is important to keep these populations of learners in mind when using the C-text.” It is exactly at this point where the exploratory study by Drackert and Timukova (2020a, 2020b) comes into play. The aim of their study is to explore whether the C-test construct measures the general language proficiency of bi- or multilingually raised language users and, if not, whether the answer patterns in specific C-test items show differences between HLL and FLL, the target language investigated being Russian. The authors argue that so far C-tests only have focused on FLL but not on other groups of language learners. This desideratum is taken up in two research questions defined as follows by Drackert and Timukova (2020a, p. 113): “RQ1: Are heritage and foreign language learners equally successful in reconstructing the same C-test gaps? RQ 2: In what ways do the answer patterns on specific gaps differ for heritage and foreign language learners?” In order to answer these questions, 113 FLL of Russian and 89 HLL of Russian enrolled in two universities, one situated in Germany, the other in the USA, were exposed to a Russian C-test, consisting of five texts arranged in a progressive order, starting by A1/A2 and ending by C1 according to the CEFR (ibid. 114). In a first step, the responses of all learners were scored using the classical dichotomous scoring system for C-tests (one point for a correct answer, no point for an incorrect one). Next, the authors calculated whether C-test items behaved significantly different for FLLs and for HLLS by employing differential item functioning (DIF). They explain the DIF approach as follows: If an item measures the same ability in the same way across groups, the groups should display the same success rate regardless of their nature. Items that differ in success rates for two or more groups at the same ability level are said to display DIF. (Drackert & Timukova, 2020a, p. 155; Holland & Wainer, 1993) Results indicate that DIF occurs in several C-test items. More precisely, “12 items were easier for FLLs and 16 items were easier for HLLs” (Drackert & Timukova, 2020a, p. 119). To gain more in-depth insights, the authors proceeded with an error analysis which resulted in a polytomous coding system containing the following five categories: 1 2 3 4 5

(no answer given) (the word to be completed is not recognized) (the word is recognized but the reconstruction is incorrect) (the word is recognized and used in the correct form but misspelled) (correct answer) (ibid., p. 116)

110  Barbara Hinger Thereby, different answer patterns for FLLs and HLLs could be observed. Roughly said, results indicate that HLLs show a better ability in the recognition of a wider range of lexical items in the C-test texts and at the same time are less able to reconstruct the required form of the blanks due to their deficits in spelling. By contrast, for FLLs, it is easier to produce correct forms (Drackert & Timukova, 2020a, p. 123). In their explanations of the findings, the authors draw especially on Hulstijn’s (2011) differentiation between Basic Language Cognition (BLC) and Higher Language Cognition (HLC). They argue that HLLs possess largely unconscious implicit knowledge in language reception and language production but need to develop literacy skills of reading and writing which would, among other factors, belong to HLC (Drackert & Timukova, 2020a, p. 112). Hence, dichotomously scored C-test approaches “reveal predominantly HLC components of language proficiency” (ibid. 123) whereas partially scored answers also cater for BLC components which would increase scores for HLLs because of reflecting their language proficiency more appropriately. Following the authors, this is especially important when C-test scores are used in placement tests: while dichotomously scored C-tests highly tend to underestimate HLLs overall language competence, partially scored C-tests reveal the realistic nature of their language proficiency (ibid., 124). However, the authors are aware of the fact that, from a testing perspective, partial scoring is less practical than dichotomous scoring. Additionally, the authors advocate that the partial scoring approach would need further validation. Accordingly, they suggest that future “research could also explore the various ways to score C-tests and the implications of different scoring methods for construct interpretation, test reliability, and test use” (ibid., 124). In this sense, they refer to Grotjahn (1996) who argues against a fixed construct validity for C-tests and calls for defining construct validity in accordance with a specific test purpose and a specific test population (as mentioned above). Norris (2018), on the other hand, clearly bears in mind differences in the development of oral and written language modalities which may distinguish learner populations such as HLL and FLL. The study of Drackert and Timukova (2020a, 2020b) shows exactly that and can be considered as the first of its kind to offer empirical evidence as well as possible solutions for C-tests to better embrace the complex challenges an increasingly globalized and multilingual world calls for. In fact, following Drackert and Timukova, the C-test construct needs to be extended to allow scoring procedures to be adjusted to specific test contexts and test uses. Their more fine-grained rating scale, by the way, would also better suit monolingually test takers especially in the initial phases of their L2 language learning (see below). The second study focusing on the use of C-tests for heterogenous learners is carried out in the context of a broader research project by Gogolin and her team in secondary schools in Germany, called the Multilingual Development: a Longitudinal Perspective (MEZ, see Brandt et al., 2021). In a sub-study, the authors explore the quality of C-tests for comparing language competences in two foreign languages taught at school, namely English and French, as well as

Assessing language competences of multilingual speakers 111 “to clarify the merits and limits of a differentiated approach to the assessment of students’ foreign language development in multilingual settings” (Gogolin et al., 2021, p. 774). Following Drackert and Timukova (2020a, 2020b), Gogolin et al. (2021) took a closer look at C-test results for a heterogeneous learner population by dividing the participants into two groups: learners raised monolingually with German as their L1 (n = 1.010) and learners raised bilingually (either with German/Russian as their L1s or with German-Turkish as their L1s; n = 977) (ibid., 777). From the broader research project, two learner cohorts (seventh and ninth grade students) were included in the substudy and the first three waves of data collection were considered (Gogolin et al., 2021, p. 777). At each data collection point, the participants had to take C-tests in English and in French,1 comprising four different texts per language and per wave. The C-tests were scored according to two different procedures: the classical dichotomous scoring approach and the word recognition scoring approach (WE-Wert as defined by Baur & Spettmann, 2007, see above). The results for the monolingually raised participants were compared to the bilingually raised participants (Gogolin et al., 2021, 779f.). The findings show that the monolingual learners perform better than the bilingual learners in the target languages involved (English and in French) as well as in both scoring approaches. However, these differences appear to be higher in English than in French and take distinct pathways. Results indicate that in English the monolingually raised students scored significantly better than the bilingually raised students in each of the two scoring procedures (ibid. 782f.). For English, the score differences between the two groups increased throughout the three waves. By contrast, in French the differences between the two learner groups are rather small and decrease over time till becoming insignificant at the last measurement point (ibid. 782f. and 785f.). Nevertheless, the monolingually raised students perform better in French than their bilingually raised peers (ibid.). Furthermore, for both target languages as well as for both learner groups the scores for word recognition are significantly higher than the dichotomous scores. This holds true throughout all data collection points. The authors therefore conclude that word recognition – which they identify as receptive language skills or, to be more precise, as semantic and reading skills – precede productive language skills – defined as writing and spelling competences by the authors who follow Eckes and Grotjahn (2006) and Klinger et al. (2019) in this regard. According to the authors, the different scoring procedures used for the C-tests account for differences in the development of language reception and language production when learning a foreign language. As in Drackert and Timukova (2020a, 2020b), the results indicate that the definition of the construct validity for a C-test should be conceptualized by considering a specific test population and taking into account the specific test purpose as put forward by Grotjahn (1996). Likewise, the claim that learner populations may develop differences in the oral and the written language modality as stated by Norris (2018) is confirmed by Gogolin et al.’s results.

112  Barbara Hinger In what follows, the TBLA approach as put forward for multilingual learners by Schissel et al. (2018) will be presented. Task-based language assessment (TBLA) for multilingual speakers TBLA evolved as a consequence of task-based language teaching (TBLT) (see, f.i., Van Gorp & Deygers, 2013). TBLT aims at developing communicative competences in a target language by integrating TBLT’s primordial focus on meaning with a focus on linguistic forms. To enhance the communication skills in the target language students are asked to perform language tasks designed as real-world tasks and/or pedagogic tasks. TBLA evaluates the performance of these tasks by pre-established criteria for written or oral productions of the students. Hence, in the TBLA framework assessment criteria focus on “what learners are able to do with language as opposed to what they know about language” (Van Gorp & Deygers, 2013, p. 1, cited in Schissel et al., 2018, p. 171). In the context of linguistically diverse communities in Mexico, more precisely in Oaxaca, Schissel et al. (2018) carried out a project using TBLA as an alternative assessment approach with the objective to consider the multilingual background of pre-service students of English by offering them the opportunity to use their language resources in English and other languages to fulfill a given task. Accordingly, Schissel et al. (2018, p. 168) underline that their approach connects translingual practices and TBLA. To proof whether their multilingual assessment perspective leads to positive outcomes, the research team created two writing tasks for the pre-service students. Task 1 asked the participants to give advice on a specific topic, task 2 asked them to produce recommendations for another topic. To be more precise, the advice referred to prevent the onset of type II diabetes, the recommendations referred to highlight aspects for a litter clean-up program for campus (Schissel et al., 2018, p. 173). Both writing tasks were based on several reading texts providing information for each topic. While task 1 offered two reading texts in English and one in Spanish, task 2 provided the students with texts in English only. Hence, for task 1 the participants were asked to use their language skills in English and in Spanish, whereas for task 2 the monolingual use of English was focused upon. The written productions of 39 pre-service students were assessed by eight experienced raters and subsequently analyzed by the research team. The assessment procedure was carried out according to international standards, such as double rating the performances and calculating statistics for inter-rater reliability to cater for rater consistency (ibid. 174f.). The results indicate that “integrating multilingual skills within a TBLA framework” (Schissel et al., 2018, p. 175) leads to better performances in the English writing task as outcomes of task 1 which included reading texts in Spanish and English were rated higher than outcomes of task 2, which relied on reading texts only in English. However, when taking a closer look at the research design,

Assessing language competences of multilingual speakers 113 it becomes apparent that the authors consider providing the students with reading texts in two languages, namely English and Spanish, as being consistent with a multilingual approach to assess the target language English. In this context, two issues arise. Firstly, it needs to be observed that multilingualism and multilingual practice is reduced to the use of two languages, a perspective which is questioned quite from the beginning of the so-called multilingual turn in applied linguistics postulating that multilingual approaches refer to more than two languages (see, f.i., Hufeisen, 2010). Hence, the research team’s approach could also be defined as a bilingual one and would, in this sense, need a definition of what is considered as multilingual. Secondly, the regional context of the study needs to be addressed: the study was carried out in Oaxaca, a region in Mexico which comprises over 100 indigenous languages (see, f.i., López-Gopar, 2021, p. 651). This could indicate that even if the target language studied by the participants is English and their language of schooling is Spanish, the individual language backgrounds of the participants might be more varied. Had the language background of the participants been asked for in a survey it could have been possible to see whether the participants are to be considered mono-, bi- or multilingual speakers. According to possible outcomes of a language background survey, a more specific way of integrating translingual practices in the TBLA approach could have permitted the students to use more than the two languages referred to in the study. Having said that, it also needs to be considered that it would not have been easy to choose the languages or the number of languages to be integrated in the assessment procedure. Altogether, the approach chosen by the research team can be a seen as a valid first step to overcome the use of English-only in the context of a language test and to value the competences in another language the students do possess. Apart from these considerations, results altogether show that higher order thinking seems to be supported when test takers are allowed to use content-based reading texts in more than one language. In the future, these outcomes could even encourage international standardized tests like PISA, as mentioned above, to allow the use of more than one language for emergent bilinguals or other multilingual test takers in order for them to score higher and perform better in these tests. Conclusion This contribution has tried to offer insights into two approaches in the field of language testing which consider the multilingual background of test takers. While two studies (Drackert & Timukova, 2020a, 2020b; Gogolin et al., 2021) explored the use of C-tests in this context one study (Schissel et al., 2018) focused on TBLA. Results from both approaches identified the underlying test construct definitions as rather in line with monolingual concepts of language learning. Thus, they consider them as too narrow for linguistically heterogenous, multilingual populations in global societies and argue for expanding them. As to the C-test studies, considering the multilingual

114  Barbara Hinger perspective would mean to open the traditional, dichotomous scoring system toward more complex, polytomous scores to cater for the differences in the development of literacy skills, especially in bi-/multilingually raised learners. Accordingly, receptive and productive skills in a given target language would be mirrored more appropriately by the scores obtained in C-tests. As to TBLA, offering test takers content-based reading texts in two languages – instead of one language, as traditionally used – seems to suit their multi-facetted linguistic competences better. However, we need to bear in mind that the TBLA approach presented by Schissel et al. (2018) solely focuses on two languages of the test takers while disregarding other languages they might possess. Hence, an expansion of the TBLA approach by using more than two languages would be asked for. Altogether, the studies presented here are promising as they explore new pathways to consider multilingual perspectives in language testing. Moreover, they exhibit interesting results and complex insights in the development of target language skills for multilingual learners. Finally, and maybe most importantly, they contribute valuable discussion points for construct definitions in this emerging field of language testing. Note 1 English (n = 1.987) and French (n = 662) (Gogolin et al., 2021, p. 777).

References Baker, B., & Hope, A. (2019). Incorporating translanguaging in language assessment: The case of a test for university professors. Language Assessment Quarterly, 16(4–5), 408–425. Baur, R., & Spettmann, M. (2007). Screening – Diagnose – Förderung: Der C-tests im Bereich DaZ. In B. Ahrenholz (Ed.), Deutsch als Zweitsprache: Voraussetzungen und Konzepte für die Förderung von Kindern und Jugendlichen mit Migrationshintergrund (pp 95–110). Freiburg/Breisgau: Fillibach. [cit. in Drackert/Timukova 2020a] Brandt, H., Krause, M., Usanova, I., & Rahbari, S. (Eds.) (2021). Language development in diverse settings: Cross disciplinary findings from the Project “Multilingual Development: A longitudinal perspective (MEZ)”. Westport: Springer. [cit. in Gogolin et al. 2021] Daller, H., & Phelan, D. (2006). The C-test and TOELC* as measures of students’ progress in intensive short courses in EFL. In R. Grotjahn (Ed.), Der C-test: Theorie, Empirie, Anwendungen/The C-test: Theory, empirical research, applications (pp 101–119). Frankfurt/Main: Peter Lang. [cit. in Norris 2018] Drackert, A., & Timukova, A. (2020a). What does the analysis of C-test gaps tell us about the construct of a C-test? A comparison of foreign and heritage language learners’ performance. Language Testing, 37(1), 107–132. Drackert, A., & Timukova, A. (2020b). Welche Sprachkompetenz misst der Cc-Test bei HerkunftsprecherInnen: allgemeine oder doch bildungssprachliche? In A. Drackert, M. Mainzer-Murrenhoff, A. Soltyska, & A. Timukova (Eds.), Testen bildungssprachlicher Kompetenzen und akademischer Sprachkompetenzen. Zugänge für Schule und Hochschule (pp 159–186). Berlin u.a: Peter Lang.

Assessing language competences of multilingual speakers 115 Eckes, Th., & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23, 290–325. [cit. in Gogolin et al. 2021] Gogolin, I., Schnoor, B., & Usanova, I. (2021). Crossing the bridge to literacy in foreign languages: C-test as a measure of language development. Multilingua, 40(6), 771–790. Grotjahn, R. (1996). Einleitende Bemerkungen zu Band 3. In R. Grotjahn (Ed.), Der C-Test: Theorie, Empirie, Anwendungen/The C-Test: Theory, empirical research, applications (pp 1–22). Frankfurt/Main: Peter Lang. [cit. in Norris 2018] Harsch, C., & Hartig, J. (2016). Comparing C-tests and Yes/No vocabulary size tests as predictors of receptive language skills. Language Testing, 33(4), 555–575. [cit. in Norris 2018] Holland, P., & Wainer, H. (1993). Differential item functioning. Hillsdale, MI: Lawrence Erlbaum. [cit. in Drackert/Timukova 2020a] Hufeisen, B. (2010). Theoretische Fundierung multiplen Sprachenlernens – Faktorenmodell 2.0. Jahrbuch Deutsch als Fremdsprache, 36, 200–207. Hulstijn, J. H. (2011). Language proficiency in native and nonnative speakers: An agenda for research and suggestions for second-language assessment. Language Assessment Quarterly, 8, 229–249. [cit. in Drackert/Timukova 2020a] Klein-Braley, C. (1997). Towards a theory of c-test processing. In R. Grotjahn (Ed.). Der C-Tests: theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol 3, pp 23–94). Bochum: Brockmeyer. Klinger, T., Usanova, I., & Gogolin, I. (2019). Entwicklung rezeptiver und produktiver schriftsprachlicher Fähigkeiten im Deutschen. Zeitschrift für Erziehungswissenschaft, 22(1), 75–103. [cited in Gogolin et al. 2021] López-Gopar, M. E. (2021). Citizenship in language testing: A call for respectful collaborations. Language Testing, 38(4), 649–653. Norris, J. M. (2018). Developing and investigating C-tests in eight languages: Measuring proficiency for research purposes. In J. M. Norris (Ed.), Developing C-tests for estimating proficiency in foreign language research (pp 7–34). Berlin, Bern, Bruxelles, New York, Oxford, Warszawa, Wien: Peter Lang. Roever, C. (2018). Developing C-tests across eight languages: Discussion. In J. M. Norris (Ed.), Developing C-tests for estimating proficiency in foreign language research (pp 295–305). Berlin, Bern, Bruxelles, New York, Oxford, Warszawa, Wien: Peter Lang. Schissel, J. L., Leung, C., López-Gopar, M., & Davis, J. R. (2018). Multilingual learners in language assessment: Assessment design for linguistically diverse communities. Language and Education, 32(2), 167–182. Schissel, J. L., Leung, C., & Chalhoub-Deville, M. (2019). The construct of multilingualism in language testing. Language Assessment Quarterly, 16(4–5), 373–378. Shohamy, E., & Menken, K. (2015). Language assessment. Past to present misuses and future possibilities. In E. W. Wayne, B. Sovicheth, & O. García (Eds.), The handbook of bilingual and multilingual education, 1st edition (pp 253–269). Malden, MA; Oxford: John Wiley & Sons, Inc. Van Gorp, K., & Deygers, B. (2013). Task-based language assessment. In A. J. Kunnan (Ed.), The companion to language assessment, Vol. 2. Approaches and development (pp 578–593). Malden, MA; Oxford: John Wiley & Sons, Inc.

7

Assessing the first language skills of plurilingual secondary school students1 An analysis of students’ and evaluators’ perspectives Isabelle Audras

Introduction This chapter aligns with research studies on plurilingual approaches in education and training, their role in education systems, and their effects on their audiences (students, teachers, other educational stakeholders, and families). In particular, we present the first analysis of an experiment on a model for assessing home languages, created as part of a local partnership between a secondary school and an association. These analyses aim to examine the challenges (on educational, social, and didactic levels) of this type of inclusive assessment practice (benefits for the student: his/her learning and educational path). Through the difficulties observed by the test designers and evaluators, these analyses also reveal methodological questions raised by this type of assessment and contribute in a number of ways to a more general reflection on the role of translators/interpreters in this type of assessment practice2 and on the effects of these practices for the specific stakeholders. We need to ask what this experiment can teach us on the inclusive and formative potential of assessment practices considering the diversity of students’ language repertoires and backgrounds. We look at benefits offered by the implementation of this assessment model in terms of the use of plurilingual approaches in education and training (of teachers and evaluators), the role of (home) languages in school/education systems, and the evolution of stakeholders’ representations about the skills of the students and their own agency as actors promoting educational equity through language assessment. Theoretical framework Current contexts of classroom language contacts mirror the complex and varied situations characterising today’s mobility and travel concepts (Harrison et al., 2018): transformations of societies, socio-economic crises, climate change, migratory flows, and inter/transnational mobilities. Faced with these complex challenges, the role of education is to convey values such as equity DOI: 10.4324/9781003177197-9

Assessing the first language skills 117 and (respect for) diversity and to ensure access to and participation into the social and civic life (Coste, 2013). Asking questions on these issues is part of the research community. This should examine the effects of taking the languages, cultures and backgrounds of students into account in education, not only for the students themselves (cognitive learning gains and, more broadly, social-emotional and social-identity development), but for all the agents in the co-constructed learning community (teachers, families, and any other educational actors) (Auger et al., 2018). This holistic and inclusive perspective on education calls for the notion of ‘specific needs’ in education (Rigoni, 2017), considering student’s specificities, especially regarding pre-existing language skills, that is their linguistic repertoire (Grosjean, 1985). According to this inclusive perspective, where students are said to occupy a central place in the learning and assessment system, it is up to the institution to guarantee conditions favourable to learning and to the success of all students and to consider the diversity of resources (cognitive, linguistic, cultural, and emotional) of each individual. Such a move requires the institution to rely on or implement a certain ‘culture of change’ (Rousseau et al., 2017), which might lead to structural improvements, namely in terms of teaching, learning, and assessment. These improvements of the profession can potentially contribute to break down barriers between the different learning spaces (home, school, culture, sports associations, etc.) and change teachers representations of the roles played by students’ repertoires in learning (and assessment), freeing them from the ideologies of a ‘languages market’ (Auger, 2018). The coconstruction of learning communities, which are not limited to students and teachers, thus implies being open to other educational social actors with the capacity to act as cultural and linguistic mediators, to bring a dynamic of change into practices and postures, and to use cognitive and educational tools (allowing the success of all students). This dynamic of inclusion questions the place and role of educational stakeholders in the community at large, particularly those ‘who are already there’ (institutions, associations, civil society) (Constant, 2018), with regard to their agency within their communities and the meaning (co)carried by their actions, reminding us of the role of language as a form of power in the development of the capacity to act (Le Bossé & Lavallée, 1993). This approach helps us reconsider educational practices of both teaching and assessment. Additionally, it seems productive to cross-reference research on learning-oriented assessment with research privileging a systemic and ecological approach (Saif, 2002; Seed, 2020). Huver (2010; also this volume), for example, invites us to rethink assessment approaches by questioning the place and the status of linguistic diversity in order to adapt educational structures to the increasing linguistic diversity in the school system. Likewise, for Chardenet (2005, p. 91), who advocates an integrated assessment of plurilingual and interlingual skills, there is ‘a gap to be filled to promote the development of mediations that allow us to understand language learning as a training object in which languages are no longer managed independently from each other’.3 This perspective is aligned with assessment practices focused on the actual learners’

118  Isabelle Audras linguistic use: ‘The idea is that tests should match actual language practices, and that multilinguals use resources from their whole linguistic repertoire. If teaching takes the direction of a plurilingual focus, assessment should also follow the same path’ (Gorter & Cenoz, 2017, p. 243). Such a perspective specifically addresses the purposes and modalities of assessment in language teaching and learning, particularly in terms of practices and their effects on the students (Huver & Ljalikova, 2013). Critical research on the role of numbers/statistics in the evaluation system, put in place by educational policies at national or international levels, potentially leading to rankings of educational systems and ‘pressure’ on pupils and/or teachers (Shohamy, 2001), is urgently needed. In the same vein, questioning language assessment approaches also remains important, in order to associate assessment models with means of their public implementation at national and regional levels. Other elements to consider are the variety of agents involved in the assessment, the purpose of what is being assessed, and how such assessment contributes to learning. Furthermore, among critical perspectives on assessment, some studies show the effects of competitive assessment objectives on learning strategies and home practices, as it tends to point out students’ shortcomings and to negatively impact students’ capacity to learn. From a ‘training/professionalising’ or ‘inclusive/universalist’ perspective, implementing assessments that stimulate students’ capacity to learn (Trocmé-Fabre, 1999) allows teachers to exercise their ‘language power’ or agentivity (Shohamy ibid.). By actively calling students’ to use their existing linguistic resources, teachers allow pupils/students to mobilise the resources and strategies needed for their learning, while at the same time further developing those resources and strategies (Coste, 2006). Furthermore, research on the assessment of skills of bi/plurilingual speakers emphasises that language assessment is standardised according to a ‘monolingual’ conception of speakers, far from the current results of research on the development of bi/plurilingualism. For this very reason, such monolingual assessment can only capture part of the skills of bi/plurilingual students who are, for example, often able to use a word in one language but not in another (Mueller Gathercole et al., 2013). In some bilingual education models, it is difficult to implement assessment tools for languages that are sparsely or recently included as school subjects (Cenoz et al., 2013, p. 193): ‘An important step when considering a holistic view of plurilingual proficiency can be to take the students’ linguistic repertoire into account including the differences in their L1s’. The implementation of assessment tools in some bilingual education systems poses a challenge in terms of designing assessment materials. Research context and purpose This study took place in the context of the Recolang project (Resources for assessing the home language competences of migrant pupils4), supported by the European Centre for Modern Languages, in its 2020–2023 programme

Assessing the first language skills 119 ‘Inspiring innovation in language education: changing contexts, evolving competences’. Recolang collected and analysed examples of how migrant pupils’ home languages are evaluated in different European countries, to define criteria and produce materials for formative evaluation, considering the evolving nature of students’ plurilingual repertoires. Particular attention was paid to synergies between the learning of languages and the development of the language(s) of schooling. The overall aims were to support social and institutional recognition of home languages and to value pupils’ plurilingual repertoires.5 The empirical analysis of such international examples allows a broader reflection on the implementation of assessment methods, considering students’ plurilingual backgrounds and repertoires, and the international crosscomparison of existing practices and models. The project presented in this chapter was initially designed to compare different international contexts, from the students’ perspective, but the Covid pandemic did not allow to collect enough data among these participants. The analyses had to take another angle and concentrate on a single assessment model from the point of view of the evaluators (and designers of the test). The analysis presented in this contribution is based on the assessment model called Atout Langues [Language Assets], supported by the (French) association AFaLaC (Association Famille Langues Cultures). The analysis pinpoints the relevant methodological and didactic questions raised by this type of assessment practice for both the practical implementation and research, for example regarding aims and tools of language assessment, such as the construction of support material for language level tests in which writing occupies a limited place and the training of (nonprofessional) test administrators. Methodology AFaLaC and Atout Langues in the context of French education

The French education system is governed by a state public service, the Ministry of National Education. Recently, the framework texts and programmes have become more open to an inclusive approach and to taking students’ languages and cultures into account in their learning (‘School Reform Act’ (2013) revised in 2019 by the ‘School of Confidence’ law, 2020 Curricula, etc.). The 2020 (middle school) curriculum states (p. 45): The student can use the skills developed in the first language studied and in other languages of his/her repertoire, including French, to learn more quickly so as to develop a certain degree of autonomy. In this way, they can: compare certain aspects of how the learnt or known languages including French (highlighting the similarities and differences), mobilise the linguistic skills and knowledge acquired in other languages (curricular, familial or regional) to make progress in new languages based on

120  Isabelle Audras the strategies used, and develop strategies for switching and transferring from one language to another (…).6 It may be difficult to make all students’ language competences visible, whether or not these students have a history of migration (languages used in the family environment, in particular) (Adam-Maillet & Azziz, 2017). The School Reform Act (2013) conceives the principle of a ‘regional/local educational project’, in which families, schools, local authorities, and associations can collaborate through co-education as a means to promote acceptance of diversity and to act in/for an inclusive society. It is in this context and administrative framework that the Atout Langues scheme was set up by AFaLaC. Founded in 2013 and approved by the Ministry of Education in the Nantes academic area, AFaLaC7 is a training and resource centre for plurilingual education. AFaLaC supports families and professionals (educational, social and care) in contexts of linguistic and cultural diversity. AFaLaC offers academic activities complementary to and in accordance with the school curriculum, both during and outside school time. These activities promote inclusive education and society based on respect for multiple plurilingual identities. Initiated by a language teacher and driven by AFaLaC, Atout Langues has been implemented in several secondary schools in the academic regions Pays de Loire and Centre over the past four years. Through the assessment of students’ oral and/or written skills in their first language(s), Atout Langues aims at promoting students’ skills in home languages, thus increasing their value. The language level certificate obtained is placed in the school file of the students, next to the school results and allows year 10 students [14–15 years old] to validate a basic competence listed in the Common Core of Knowledge. This assessment device is different from the placement tests intended for newly arrived students.8 In 2020–2021, 67 students took these tests, ranging from A1 to B2. The materials are available in about 30 languages in the four CEFR (Common European Framework of Reference for Languages) language activities. The test designers and evaluators went through training organised by AFaLaC before each assessment session. This training covers the criteria and methods for assessing language level (written/oral skills) and is based on CEFR tools. Methodological design

This contribution is based on a qualitative and comprehensive approach. The collected corpus includes semi-structured recorded interviews with students, teachers, and evaluators in 2019 and 2021, which were later transcribed. It also includes an online student questionnaire as the health situation prevented us from conducting face-to-face interviews with students in 2021. The interviews with students were conducted in groups, with their teacher present. They focused on biographical aspects (language repertoires/uses, taste for/

Assessing the first language skills 121 openness to languages, and links with learning) and on students’ motivation to take part in the Atout Langues programme, to have their language skills assessed (experience of test administration, difficulties encountered, effects of this experience on their language practices, and, in particular, representations of the place of languages in learning). The online questionnaire takes up the same questioning themes. Fourteen responses were received. The interviews with teachers were conducted individually and had a double focus: first, on their motivations for suggesting their students to participate in the Atout Langues programme (contact with the association, experiences of plurilingual practices, forms of these practices); second, on how they perceive the effects of the assessment on their students’ and on their own personal and professional paths. The interviews with evaluators were conducted in groups (2019) and individually (2021). They focused on biographical aspects (language repertoires, teaching, training and learning experiences with students’ first languages), their motivation to join AFaLaC to assess and design material in those languages, and on the effects of evaluation practices on students and on their own personal and professional path (in terms of training, social link development, and meaning of his/her action, in particular). The participants in the study

This study focuses on the cross analysis of interviews conducted with a group of secondary school students, with their teacher present, and interviews conducted with the evaluators/material designers (mainly two of them, Amina and Hamza). At the time of the interview (2019), Sandro and Mariella, two of the students assessed, were 14 years old and in year 10. They had both just passed the DELF B1 (Diplôme d’études en langue française, official diploma in French as a foreign language) and, via Atout Langues, the A2 level in their respective home languages. Sandro had been to school in Albania until the age of 9 and had been in France for four years. He had started learning French at school in Albania (one year), before switching to English. He wants to become a nurse or an architect. Mariella had been in France for three years, having been to school in Romania until she was 11, and had learned Italian and English in primary school. Their English teacher has ten years’ experience of teaching in secondary schools. Trained in language teacher education, he progressively became interested in approaches taking into account the students’ linguistic repertoires. He introduced the Atout Langues programme in his school. The interviews with the test designers were done either as a group (Ayline, Elena and Issam in 2019) or individually (Amina and Hamza in 2021). Elena, Issam, and Ayline are speakers of Romanian, Literary Arabic, and Turkish, respectively. Issam has experience in teaching mathematics in Syria, and Elena is a Bachelor of Arts student. Amina speaks Algerian Arabic, Literary Arabic

122  Isabelle Audras and Kabyle, and has experience in teaching French in Algeria (primary school) and France (teenagers and adults). Hamza, among other languages, speaks English, Urdu, and Pashtu and has experience in teaching English and French in Pakistan and France. All of these individuals are involved in different plurilingual education activities with AFaLaC: translating materials (all), co-leading activities in class (Amina, Ayline, and Elena), participating in workshops for parents (Amina and Ayline), and training sessions related to test taking or other topics, including storytelling material design, reading sessions, and mediation in class. Cross analyses: benefits and challenges of assessing students in their first languages Effects on the students

The interviews and questionnaires show that taking the test in the first language appears to be a positive experience: it is a symbol of skill recognition thanks to the ‘diploma’, it constitutes a bridge between different skills, reinforces a taste for diversity and language learning, and creates a link between the students and their plural identity and family culture(s). Among the students who responded to the online questionnaire (2021), a large majority was born in France. The most frequently expressed motivations for taking the test were ‘to have my skills recognised on my school record’ (6) and ‘to obtain a diploma’ (7). All students wished to continue learning their first language to maintain the link with their family9: I would like to continue learning this language because all my family lives in Portugal. I could communicate with them when I go on holiday in Portugal. Also, it is a language that I find very beautiful. In the analysis of the interview with the two secondary school students presented above, we find converging elements. For example, the tests in Albanian (Sandro) and Romanian (Mariella) were experienced as a bridge between the two places of the students’ educational paths (the home and the host countries) and as a link to the countries they had left: Mariella: in France, before, there was no such thing as having a diploma in one’s own language, it seemed interesting […] I showed the diploma to my parents, they were happy […]. Sandro: [a strong point] that we practise our language, that we take a test in our language in France, in another country than ours [it’s important]10 yes if we ever go back [to Albania] […] my parents told me you could have done more, my parents always want more, I’m the eldest, I have a little sister.

Assessing the first language skills 123 Both students seem to be comfortable with their language education, relying on the languages in their repertoire to extend it: Mariella: I learned [French] quickly but it was difficult […] I like writing in French […] I like Spanish, English is also okay […]. Sandro: It sometimes helps me [to know several languages]. There are words that are similar, it’s easier to remember them or to write them, Spanish is a language that is very similar to Albanian so it’s easier, [I would like to learn] Italian or German, I have family in these countries, I would like to go and see them. Last year I went to Italy, I saw them, I learned a bit […] understanding is okay, and the languages are similar, there are words that are very similar. This exchange reveals students who are quite at ease with the diversity of their repertoires and who were made more confident in their learning because of Atout Langues. For the teacher, the effects of the tests go beyond assessing language skills: It’s even more important for students who are struggling with French, this puts them in a difficult position in almost all subjects, and to have this anchoring and recognition of existing skills is important for them, several colleagues have told me that it triggers onward reactions. […] recognising their skills is essential to enable them to access learning and the codes of the French education system. The effects observed on the students concern their (meta)-language skills in French and in other school subjects, particularly their ability to transfer attitudes and behaviours specific to the analysis of the functioning of languages to their learning; the results of the Atout Langues tests support the positive perception of the teaching team. The importance of assessments for students is also stressed by evaluators. One of the participants stresses the following: Amina: I think that it brings something to the students, because this takes place at school and because it’s important enough for them to be able to leave a class and come to take this test, that there is a specific person who comes for them. Effects on the educational stakeholders

In each interview conducted, everyone (teacher, evaluators, etc.) was invited to express the reasons for their involvement in these particular assessment practices, which seemed to reward them both at a professional and a personal level.

124  Isabelle Audras The two educational stakeholders interviewed showed interest in recognising skills otherwise made invisible by the society, at large, and by the education system, more specifically. By taking part in the dynamics of this learning (and assessment) community, they also observed and got more aware of their own agency. Hence, for the teacher: We are dealing with an important need, a society that is evolving, which is both pluricultural and plurilingual; it is essential that schools recognise this. We have more and more students who speak several languages, we need to be able to set this up quickly, it is doable and will allow us to value quite a few students. By valuing their own language(s) and culture(s), out of solidarity with their linguistic community, but also to follow the students in their learning process, teachers and evaluators aim to be useful: to promote Romanian as it is not a very well-known Latin language, to help Romanian children be better integrated, and to enable them to have this resource added to their curriculum. Ayline: to promote Turkish, which is not well-known, and perhaps for personal satisfaction, to be able to say that I’ve managed to do something, it’s satisfying to think that we’ve administered tests for children. Issam: and that we can also contribute thanks to our culture, our language. […] Hamza: It was my interest that made me become involved because I wanted to do something, because there are a lot of Pakistanis in Europe, but we don’t speak many of their languages, so I wanted to contribute, to give some resources, some content, if I could help these people. […] Elena:

Rethinking the assessment of students in (home) languages: questions of language level and the role of the evaluator/evaluatee relationship (Re)thinking the relationship between evaluators-mediators and evaluated students

Through the analysis of the exchanges on the difficulties encountered as the test was being designed (particularly translation from existing French source material) or administered (How should a level be assessed?), it is possible to see how the reflection progressed from awareness of the stakes and responsibility as an evaluator, through the wish to assume a benevolent role beyond doing a strictly certifying assessment, to making recommendations for evaluator training. The assessment material was translated from the same source material, inspired by existing test formats (of the DELF type, for instance). This raises

Assessing the first language skills 125 methodological linguistic and ethical questions in the transition from one language–culture to another, not only with regard to the themes of the chosen text, but also in the actual translation work (What should be done when a word does not exist in the target language?). These issues raised during the design phase of the test reflect the double comprehension process that is required from the translator (understanding the intention of the message and the context of the target audience), which need to mediate from one language–culture to another and reconciling the choice of words: Amina, with the example of Algerian Arabic: “Sometimes there are limits, some varieties are not suitable for schooling, there will be holes everywhere, for instance the names of animals. These languages are languages of everyday life, referring to what we see, to what is useful immediately. To designate animals, we have to switch to Literary Arabic. Because it is Arabic, there are a lot of things in common, a common sentence construction, common words […] Another thing is, the themes tackled, the themes of the environment, pollution, are not subjects that can be tackled as such in the family circle. It can be addressed at school.” Content equivalence, particularly at intermediate language levels (CEFR B level), becomes a question when translating material, but also when assessing the students. This challenge shows the limits of using only one reference grid (CEFR) in this type of assessment (designing materials and assessing) and the importance of choosing appropriate types of text, considering the diversity of educational cultures and language uses. Starting from the speakers’ uses during the test design is of paramount importance, as clearly shown by Amina in the following excerpt: Amina: the texts that we use for B level students are a little too complicated for children […] Even if all the child’s schooling has been in Literary Arabic, I believe that it will be difficult for a child of that age […] The text will seem simple in French, but as soon as it is translated into Literary Arabic it will seem like something only an adult can understand. This is both at the level of the content and the structure of the sentence. […] I would simplify, however, with B level students, there are a certain number of requirements, I would choose other texts. This comment shows how necessary it is for materials to be adapted to students’ linguistic practices and experiences with their first language(s). It is also important that the relationship between the evaluator and and student evaluated is built up through plural modalities so that the evaluator can appreciate the student’s level in a language, while at the same time taking into account varieties of this language and student’s uses of it in their repertoire.

126  Isabelle Audras Despite their ‘monolingual’ and nonstandardised format, the assessment materials allow for the target language and another language and/or the language of instruction to be concomitantly used during the test. Taking on the role of evaluator means identifying, during the activity and in the exchange, how to identify the level of a text or a discourse in the target language, what can be very challenging for evaluators without training on assessment scales: Amina: For me, it wasn’t difficult, I had already dealt with students, I have a certain experience, but I think that for some people, who don’t know the CEFR levels, the training is essential. Evaluators as a crucial link in the relationship with the evaluatee

The information collected during these tests contributes to a better understanding of the students, their linguistic biographies, and their language skills. As the students self-declare their level based on a CEFR grid, the levels do not always match, or the declared home language is sometimes different. The test administration is an opportunity for test administrators to use their knowledge of the contexts in which languages are used, to listen and to be empathetic with the students. Some situations, however, seem to lead to an interior conflict: Hamza: Everything was in Urdu, including the instructions. The B1 student understood everything, whereas I had to repeat for the others, I tried to simplify, but they didn’t understand and by talking with them, I understood that they did not speak Urdu, but Punjabi. In Pakistan, parents try to speak Urdu with their children, so when you ask the children what language they speak, they will say Urdu, as it is more valued. When I asked the students, one of them explained that his mother was from India, she didn’t speak Urdu, she spoke Sanskrit and his father spoke Punjabi. This perturbed me, what could we do? The two languages are similar, but they are still different. It was a dilemma. He passed the A1 level, our objective is to valorise. As described above, the test form is not strictly a certifying measure based on an assessment grid: the evaluator also relies on the practices observed and recorded at the time of the exchange with the students to assess their competences and encourage their progress. Hamza refers to these issues in the following terms: Hamza: some children don’t understand when we speak Literary Arabic, they think that they would be evaluated in the Arabic that they speak at home. I bring all the assessment levels with me and switch if it doesn’t match the declared level. […] They are nervous to begin with, they don’t really know what’s in store. So, I reassure them,

Assessing the first language skills 127 I try to make the atmosphere as easy as possible, they then really go for it. I ask how things are going in the family, what language they speak at home. It helps me to see how they live and to get an idea of their level. In the end, I always tell them ‘This is a start, you can come back next year, try to speak more, try to do this or that and you will come back next year to pass the next level’. I encourage them to continue to use their language. The evaluator–student relationship co-constructs the assessment process considered here as collecting information on learning, in order to better understand students’ level of proficiency and to contribute to its improvement. In fact, valuators act as mediators. This type of assessment provides students with a level of proficiency, while at the same time supporting their continuous learning process. Finally, it makes students’ linguistic repertoire visible and valued, which traditional practices in the educational system frequently hinder. Summary and perspectives Despite its fragmentary nature, this initial analysis highlights students’ interest in knowing their level in their first language(s) and seeing them recognised, as much as those of the school curriculum. Additionally, students also show interest in improving their knowledge of their first language(s). Coste (2011) stipulates as a criterion for the development of students’ plurilingualism that the learning of the language of schooling should be related to the students’ repertoires and previous language experiences in order to operationalise the inherent dynamism of their plurilingual competence, which covers the repertoire (CEFR). The empirical analysis presented in this contribution shows that home language assessment places the student’s needs (valuing and recognising their language skills) at the centre of the process and contributes to enhancing the development of their repertoires, including the language of schooling. In this sense, the results presented here allow us to (re)think about language assessment within a learning-oriented framework and from an inclusive perspective, where responsibility for learning and for assessing is shared by all the agents involved (teachers, students, and evaluators). According to this perspective, the role of the ‘evaluators– mediators’ is crucial, and training is essential: results demonstrate their capacity to adapt to the diversity of students and to their linguistic repertoires, even if this means using one or more language(s) other than the target language to make the most of the students’ resources and to assess their actual language proficiency. The account of the experiences of the designers (translators) of materials and evaluators/correctors highlights methodological and didactic questions that need to be examined during this type of assessment, concerning both the test administration and the training of test creators and test administrators. This analysis also raises the question of the resources, including human and

128  Isabelle Audras financial, that are needed to operate this type of assessment and to train all agents involved. The convergence of a learning community, via a model supported by an association, promotes the evolution of representations of the role that languages play in learning and education systems. Finally, this project helps us to reflect on the roles and statuses of languages in students’ repertoires, regarding the processes of appropriation and identity (re)composition. Notes 1 With the contribution of Carole-Anne Deschoux, Joana Duarte, Françoise Leclaire, Ildiko Lorincz, Filomena Martins and Ana-Sofia Unkart. 2 Especially regarding the languages for which there are little or no assessment materials. 3 Author’s translation from the French: “un vide à combler pour favoriser le développement de médiations qui permettent d’appréhender l’apprentissage linguistique comme un objet de formation où les langues ne seraient plus gérées indépendamment les unes des autres” (Chardenet, 2005, p. 91). 4 www.ecml.at/homelanguagecompetences. 5 For this reason, the members of the Recolang project are listed as contributors to this chapter. 6 Author’ s translation from the French: “Comparer certains aspects des fonctionnements des langues apprises ou connues dont le français (souligner les proximités et les différences), mobiliser les compétences et connaissances linguistiques acquises dans d’autres langues (curriculaires, familiales, régionales) pour progresser dans de nouvelles langues en s’appuyant sur les stratégies mises en œuvre, développer des stratégies de passage et de transfert d’une langue à d’autres correspondent à des objectifs de formation à mettre en œuvre pour mobiliser, mettre en relation et utiliser les acquis en langues” (Bulletin Officiel, 2020, p. 45), Programme pour le cycle 4, www.education.gouv.fr. 7 https://www.famillelanguescultures.com/. 8 Upon arrival, each student goes through different stages of a diagnostic assessment of their skills (cognitive, literary, reading comprehension, mathematics, etc.) The aim is to take into account their prior academic journey and place them in the appropriate level (hours in preparatory class, subjects in mainstream class). Mathematics and reading placement tests are available in 25 languages. Further information on these placement tests can be found at https://www.reseau-canope.fr/ eana-outils-devaluation-en-langue-dorigine/introduction.html#anchor-content. 9 Interviews and questionnaires were held and collected in French and translated in English for this chapter. 10 We have added complementary information from the interview in brackets in order to facilitate the comprehension of the transcript.

References Adam-Maillet, M., & Azziz, J. (2017). Pour un établissement scolaire équitable. BergerLevrault, Les indispensables. Auger, N. (2018). Bilinguisme chez le jeune enfant en famille et à l’école: quels enjeux pour la réussite à l’école? Médecine & Hygiène, « Devenir », 30, 7–66. Auger, N., Adam-Maillet, M., & Thamin, N. (2018). Inclusive education for nonFrench-speaking schoolchildren (in metropolitan France): Obstacles, institutional

Assessing the first language skills 129 opportunities and present-day teaching practices. In K. M. Harrison, M. Sadiku, & F. V. Tochon (Eds.), Displacement planet earth: Plurilingual language education policy for 21st century schools. Blue Mounds: Deep University Press, 145–170. Cenoz, J. Arozena, E., & Gorter, D. (2013). Assessing multilingual student’s writing skills in Basque, Spanish and English. In V. C. Mueller Gathercole (Ed.), Issue in the assessment of bilinguals. Bristol: Multilingual Matters, 185–204. Chardenet, P. (2005). Évaluer des compétences plurilingues et interlingues. Synergie Italie, 2, 90–102. Constant, I. (2018). Les migrants en bas de chez nous. Paris: Seuil. Coste, D. (2006). L’Europe et les langues. Multilinguisme et politiques linguistiques. In D. Coste, A. Sobrero, M. Cavalli, I. Bosonin (Eds.), Multilinguisme, plurilinguisme, éducation. Les politiques linguistiques éducatives. Aoste: IRRE-VDA, 11–36. Coste, D. (2011). Diversité des plurilinguismes et formes de l’éducation plurilingue et interculturelle. Recherches en didactique des langues et des cultures, 7(1), http:// journals.openedition.org/rdlc/2031. Coste, D., & ADEB (Eds.). (2013). Les langues au cœur de l’éducation. Principes, pratiques, propositions. Bruxelles: Éditions Modulaires Européenne. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31(3), 231–248. Grosjean, F. (1985). The bilingual as a competent but specific speaker-hearer. Journal of Multilingual and Multicultural Development, 6, 467–477. Harrison, K. M., Sadiku, M., & Tochon, F.-X. (Eds.) (2018). Displacement planet earth. Plurilingual language education policy for 21st century schools. Blue Mounds: Deep Education Press. Huver, E. (2010). “J’ai pas pu corriger parce que c’est le mélange des langues” L’évaluation de la compétence plurilingue entre résistances et tensions. Cahiers de l’ACEDLE, Recherches en didactique des langues et des cultures, (7-2), 113–137. https://doi.org/10.4000/rdlc.2144 Huver, E., & Ljalikova, A. (Dirs.) (2013). Évaluer en didactique des langues/cultures: continuités, tensions, ruptures – Dimensions épistémologiques, politiques et didactiques. Le Français dans le monde. Recherches et applications, CLE International/ Français dans le monde, 53, 7–18. Le Bossé, Y. D., & Lavallée, M. (1993). Empowerment et psychologie communautaire. Aperçu historique et perspective d’avenir. Les cahiers internationaux de psychologie sociale, 18, 7–20. Ministère de l’Education nationale. (2020). Programme du cycle 4. Bulletin Officiel no. 31, 30 July 2020, https://eduscol.education.fr/document/621/download Mueller Gathercole, V. C., Thomas, E. M., Roberts, E. J., Hughes, C. O., & Hughes, E. K. (2013). Why assessment needs to take exposure into account: Vocabulary and grammatical abilities in bilingual children. In V. C. Mueller Gathercole (Ed.), Issue in the assessment of bilinguals. Bristol: Multilingual Matters, 20–55. Rigoni, I. (2017). Accueillir les élèves migrants. Dispositifs et interactions à l’école publique en France. Alterstice – Revue Internationale de la Recherche Interculturelle, 7(1), 39–50. Rousseau, N., Point, M., Desmarais, K., & Vienneau, R. (2017). Conditions favorables et défavorables au développement de pratiques inclusives en enseignement secondaire: les conclusions d’une métasynthèse. Canadian Journal of Education/Revue canadienne de l’éducation, 40(2), 1–29.

130  Isabelle Audras Saif, S. (2002). A needs-based approach to the evaluation of ITA’s spoken language ability. Canadian Journal of Applied Linguistics (CJAL), 5, 145–167. Seed, G. (2020). What is plurilingualism and what does it mean for language assessment? Research notes. Cambridge Assessment English, 78, 5–15. https://www. cambridgeenglish.org/Images/597022-research-notes-78.pdf Shohamy, E. (2001). The power of tests. A critical perspective on the uses of the language tests. London: Routledge. Trocmé-Fabre, H. (1999). Réinventer le métier d’apprendre. Paris: Éditions d’Organisation.

8

Developing a multilingual perspective in language assessment Multilinguality and interculturality Anthony J. Liddicoat and Angela Scarino

Introduction In this chapter, we consider the nature of assessment for learning and additional language; that is in contexts where the educational purpose is to expand learners’ multilingual repertoires and develop multilinguality and interculturality. We take the position that multilinguality is inherently intercultural as it involves constant movement between languages and cultures in which both language and culture are constituent parts of meaning making and interpretation (Liddicoat & Scarino, 2013). Multilinguality is thus not simply the ability to use multiple languages but communicating and interpreting meanings across and through languages and cultures, coupled with an understanding of the ways that languages and cultures are consequential for meaning. In this chapter, we will examine how multilinguality in assessment can be conceptualised, and what constitutes multilinguality in assessment. We will consider examples of assessment from within such an orientation and examine some issues and considerations for assessment processes that emerge from such an elaborated conception. Conceptualising the ‘multilingual’ in assessment There are different conceptualisations of multilingual assessment and most importantly its purpose. The use and evaluation of multiple languages in assessment differs according to the language learning context, for example, second language learning, foreign language learning, assessment of content, assessment of language, the relative power and status of the languages involved, etc. Shohamy (2011) has argued that ‘multilingual assessment’ is used in in language learning contexts in two distinct senses. The first involves enabling learners to use all the languages in their repertoire to demonstrate their learning. The second is assessment that seeks to address ‘multilingual functioning’. We use this distinction below to discuss diverse conceptualisations of the construct. The first sense relates to a focus of assessment on content learning and is a reaction to traditional monolingual framings of assessment with rigid separation DOI: 10.4324/9781003177197-10

132  Anthony J. Liddicoat and Angela Scarino of languages that create problems for multilingual test-takers. Shohamy (2011) argues that monolingual testing regimes introduce an extra burden on multilingual test-takers as they need to deal both with the content of the assessment and the language of assessment. Multilingual assessment in this sense usually means introducing multiple languages into assessment processes to offset problems for students in expressing learning. This often takes the form of test ‘accommodations’ in which additional language support is given to students (e.g. access to bilingual resources such as dictionaries, translations of elements of the test, etc.). Shohamy (2011) notes that such accommodations are typically seen as remedial interventions. Such ways of assessing reflect what Gorter and Cenoz (2017) call a multilingualism approach towards comprehension. That is, it reflects a desire to overcome the problems caused by assessing learners in a language other than their prime language. We would argue that this framing constructs multilinguality as a background problem for effective and fair assessment, rather than placing multilinguality at the heart of assessment thinking and practice. It is necessary as a means for achieving social justice, and ethical goals but it does not specifically nurture multilinguality as an educational goal. Additionally, Gorter and Cenoz (2017) argue for a multilingual approach towards scoring, where students are assessed across their languages and then are scored by compiling the results of each performance. However, recognising multiple languages in students’ repertoires through composite scoring seems to conceal the way students operate across different languages and their use of diverse configurations of languages for different purposes in different contexts. In other words, they recognise the multiple languages that are in play in the assessment but do not appear to recognise multilinguality as entailing a distinctive set of knowings that need to be recognised in assessment. They also consider translanguaging in assessment and much thinking on multilingual assessment argues for a reframing of the construct ‘language’ in terms of repertoire rather than as single languages (García & Li Wei, 2014). Shohamy (2011) argues that assessment needs to open possibilities for learners to incorporate all their languages into assessment processes and for learning expressed through any linguistic practices to be taken into consideration and given value. However, she also notes that such translanguaging practices are typically seen in education as a liability and are taken to represent a lack of competence, when languages are conceptualised as independent, isolated entities. It thus views relationships between languages as inherently conflictual (Chalhoub-Deville, 2019). This work in multilingual assessment allows learners to express their knowledge using their full linguistic repertoires in assessing content where language is the medium for learning and assessment but does not consider the relationship between language and content, or the assessment of language itself as content. Emphasis has usually been placed on learners from non-dominant language groups being assessed in contexts where the language of the dominant groups prevails, and the critique of monolingual assessment has usually been associated with the ways in which they reproduce disadvantage (Hofer &

Developing a multilingual perspective in language assessment 133 Jessner, 2019). This view of multilingualism in assessment tends to construct multilingualism as a means for addressing problems inherent in the regular, monolingual mode of assessment that prevails in most educational contexts. Multilingual assessment thus becomes a response that allows for greater equity in assessment practices. However, while equity is crucial in assessment, it is not only in such contexts that multilingualism is relevant and, as Shohamy (2011) notes, multilingual assessment needs to be relevant for all learners regardless of how their multilingual repertoires are constructed. Shohamy’s (2011) second sense of multilingual assessment is an assessment that focuses on learners’ multilingual functioning, which suggests an understanding of the students’ capability to act successfully in the interpretation, creation, and exchange of meanings across languages and cultures. This aspect of multilingualism in assessment is much less developed in the literature. One form of assessing multilingual functioning proposed by Stathopoulou (2013, 2015) involves students reading texts in their first language (Greek) and then producing new texts in their second language (English) based on their reading. In a similar vein, Schissel et al. (2019) report on assessment tasks designed for Mexican English language teacher education students that use input materials in both Spanish and English for developing a response in English. Hofer and Jessner (2019) have proposed an assessment approach for young learners that examines other aspects of multilingual functioning. They view the construct in multilingual assessment as involving evidencing learners as ‘linguistically skilled, cognitively-flexible and culturally sensitive multilingual agents’ (Hofer & Jessner, 2019, p. 5). This involves several complex and generalised capabilities including reflecting on language structures and use, operating across languages, interpreting new linguistic material in the light of existing knowledge, and moving between languages in various ways. They propose tasks for young trilingual learners such as comparing sentences in each of their learners’ languages (Italian, German, and English) to identify which sentences share the same meaning, explaining reasons for their choices and using their knowledge of their languages to identify meanings in unknown, but related languages, again explaining their thinking. In this work on multilingual assessment, there has been a significant focus on approaches to assessing that reflect multilingual practice, understood as the presence of multiple languages in the assessment process, but less emphasis on what constitutes a multilingual capability for assessment, although there is some work in this area (Chalhoub-Deville, 2019). When multilinguality is the goal, what is needed is a recognition not only of the presence of two or more languages in learning, but also their relationship and the ways of ‘moving between’ languages and cultures and the capabilities this move entails. Recognising the role of culture

The discussion of multilingual assessment to date has focused on the multiple languages available in the societal/educational context and in the repertoires

134  Anthony J. Liddicoat and Angela Scarino of students. Absent from these discussions is a consideration of the essential role of culture in language learning and assessment. Scarino (2017), arguing for a recognition of culture in language assessment, highlights the need to consider understandings of culture that go beyond ‘factual knowledge’ associated with the focal language or as ‘content’ of language learning and beyond ‘ways of life’ or practices of social groups who are users of the focal language. Taking a hermeneutic perspective that sees language education as interpretive, she argues for understanding culture being integral to the interpretation, creation, and exchange of meanings when communicating across languages and cultures. Secondly, she highlights the mediating role of both language and culture in learning. Following Halliday’s (1993) notion of learning as a process of meaning making, she explains that students (and teachers) in their diversity, interact to exchange ideas, knowledge, understandings, elaborations, explanations as they make sense of and exchange meanings in learning. Importantly, this learning is mediated through the languages and cultures of students’ primary socialisation and involves both language and content. In other words, the meanings that learners make and represent, in both their language learning and assessment, emerge from their linguistic and cultural, historical/ experiential context. This understanding of all learning being linguistic and cultural activity leads Scarino (2017) to foreground two senses of multilingual and intercultural assessment: 1 multilingual and intercultural in the sense of multiple languages and multiple cultures being available to students, and though the assessment may involve individual or multiple languages and cultures in combination from the learners’ repertoires, there is a requirement to account for the choice of language and the use of particular languages in particular contexts, and 2 students’ performances revealing certain practices and capabilities that characterise the multilingual and intercultural practices of multilingual or ‘learning-to-be-multilingual’ users. Elaborating the conceptualisation Drawing on our work on intercultural language learning and its assessment in additional language learning (see Liddicoat & Scarino, 2013), which we conceptualise equally as multilingual, we operationalise the construct as the need to capture each of the following in both language learning and assessment:

• observation, description, analysis, and interpretation of phenomena shared when communicating and interacting;

• active engagement with the interpretation of self (intraculturality) and ‘other’ (interculturality) in diverse contexts of social exchange;

• understanding the ways in which language and culture come into play in

interpreting, creating and exchanging meaning, and the recognition and integration into communicating of an understanding of the self (and others)

Developing a multilingual perspective in language assessment 135 as already situated in one’s own language and culture when communicating with others; • understanding that interpretation can occur only through the evolving frame of reference developed by each individual: learning a new language becomes a part of the process (Liddicoat & Scarino, 2013, pp. 130–131) Students are positioned as participants, invited to act and interact in the exchange of meanings; importantly, within a multilingual and intercultural orientation, students are also analysers who are asked to consider: What is it that is really going on here? and they are positioned as reflective/reflexive persons, invited to consider the act of meaning exchange, diverse perspectives, reactions and responses and what this means to them as communicators seeking to exchange meanings in linguistic and cultural diversity. The focus is on experiences of communication where multiple languages and cultures are in play, and that encompass differing knowledge and understandings, positions, views and perspectives that are open to processes such as being interpreted, compared, connected, explained, applied, abstracted, related, questioned, and juxtaposed. These experiences include three facets which we consider necessary in multilingual and intercultural language learning and assessment: performance/interaction, analysis, and reflection, with the latter including both a consideration of the nature of the exchange and of self. Students participating in these experiences necessarily draw on their knowledge and understandings, experiences and assumptions, positionings, reactions, and values. Thus, the goals of learning an additional language and their assessment go well beyond communication alone (see Leung & Scarino, 2016) to include developing meta-awareness of what is entailed in exchanging meanings and negotiating diverse interpretations when multiple languages and cultures are in play and coming to understand the practice of meaningmaking (see also Kramsch, 2009, 2011). Much more conceptual work is needed in order to lead to a better understanding of the construct of multilingual and intercultural capabilities which recognises (1) the use of different languages, with increasing consciousness, on the part of students, about their multilingual and intercultural practices and the reasons and consequences of their choices, (2) the capability to move between languages and cultures in interacting and exchanging meanings with others, and (3) the capability to reflect critically upon the interpretation, creation, and exchange of meanings and in doing so, recognising that there is no perfect correspondence between language and meaning (Ortega, 2019). In addition to this conceptual work, an expansion is needed in assessment practices towards a stronger focus on assessment for learning (see Scarino, 2020). It is a view of assessment that moves beyond ‘finished products’ to consider the kinds of processes involved and considers dispositions such as stance, identity formation, openness to difference, and multiplicity.

136  Anthony J. Liddicoat and Angela Scarino The examples that follow are taken from our ongoing work focused on learning additional languages within a multilingual and intercultural orientation, focused on the interrelationship of language, culture, and knowledge/ content. Worth noting is the fact that it is often in the reflections/reflexivity in relation to ‘moving between’ languages and cultures that evidence of multilingual and intercultural capabilities emerges. Example 1 The following extract is taken from a pair interaction in which students are discussing a dialogue. While the activity was not set up as an assessment activity, it nonetheless provides evidence of learning and, in particular, of Jack’s emerging ability to move ideas across languages to explain an unfamiliar concept found in the text they are reading. Sarah: Jack: Sarah: Jack: Sarah: Jack: Sarah: Jack:

Sarah:

I don’t get this. She’s talking about otoosan ((father)), but why is she talking about her father? I think it’s what she’s calling her husband. Is that the word for husband?. No it’s just that’s what she calls him. Like, when I was in Japan the mother did that a lot she would say otoosan to him all the time. Not his name. She’d go “otoosan” and she meant him. And what did he call her? Okaasan. ((mother)) That’s so weird. It’s just what they say. It’s like it’s their position in the family and that’s how they talk about each other. He’s the father and she’s the mother. We use ‘dad’ and ‘mum’ just for our parents but they use them differently. Mariko used to call her brother oniisan ((older brother)) too. It’s like using titles instead of names. And when they talked about them too they’d do the same. Like mum would say to Mariko, go get oniisan or something. So you say what they are in the family. Oh I get it, she calls him father because he’s a father for the family. (Liddicoat, 2014, p. 270)

In this interaction Sarah and Jack work through issues in understanding a communicative use in Japanese. In doing so, Jack moves between languages as he attempts to articulate his knowledge of a Japanese communicative reality, of which he has direct experience, for Sarah, who does not know. Here, we can see an enactment of a particular type of multilinguality and interculturality as he attempts to respond to Sarah’s question about the interpretation of the word otoosan, literally meaning ‘father’, but used here as an address term by someone who is not the child of the person referred to. Jack begins with a mediational act similar to translation (she’s calling her husband) and this is how

Developing a multilingual perspective in language assessment 137 Sarah responds to this. Having realised that translation has failed to mediate the interpretation successfully, he adopts another approach by explaining and exemplifying the concepts he needs to mediate. What is involved here is not simply an act of translation but an act of multilingual and intercultural mediation in which Jack draws upon knowledge of both English and Japanese to explain enculturated language use. While the interaction is primarily in English, this example remains inherently multilingual as it involves the use of Jack’s and Sally’s repertories in both languages and draws on each to construct an intercultural interpretation of the act of addressing. This example shows that (1) evidence of learning comes not just from ‘performance’/interaction but also from reflection on interactions, and (2) students need to be able to explain the multilingual/intercultural exchange to demonstrate their learning; that is the need to articulate their multilingual/intercultural experience and the learning that results from it. Example 2 The second example is taken from a unit of work developed for Year 10/11 students of French in South Australia. The class profile (22 students) included students of different year levels, including adults. The class also included several students of African origin (Kenya, Burundi, Congo) and one from Laos, with a French background student and two German exchange students. Students participated in a number of activities, including discussions and writing in French and English prompted by textual work. The teacher, in his own words, sought to consider ‘students’ ability to respond to challenging multicultural situations, with an awareness of their own linguistic and cultural position at the same time’. He asked students to maintain a journal where they recorded their ‘evaluations, responses and other thoughts’. As a final assessment task, the teacher asked students to consider photographs of students wearing a hajib, one of which depicted three girls with a tricolour band blending into their hajibs. These photos were protests against a ruling in France banning wearing religious symbols and clothing in public schools. An Australian girl from this class wrote: La deuxième photo semble être une protestation paisible des elles musulmanes. Elles ont mis le tricolore sur leurs foulards. En fait ça, peutêtre elles veulent montrer qu’elles peuvent être musulmanes et françaises au même temps, que, être l’une ça ne doit pas les exclure d’être l’autre. Pour les gens qui ne supportent pas les foulards ou les autres signes religieux, ils verraient les tchadors avec le bleu, blanc et rouge comme incompatibles avec le principe de la laïcité (ou neutralité) de France. à mon avis, c’est un témoignage intelligent que les filles ont fait parce que c’est profond et pas violent. En utilisant leurs intelligences au lieu de violence (en serrant constructive au lieu de destructrice), je pensé que les filles montrent qu’elles ont pensé longue et forte au sujet de quoi elles croient.

138  Anthony J. Liddicoat and Angela Scarino Je crois que c’est admirable d’agir dans cette façon parce que c’est plus difficile se conduire calmement logiquement à propos deu qu’on semble passionnément qu’il est? (extract taken from an extended response) The second photo seems to be a peaceful protest of the Muslim women. They have put the tricolour on their scarves. Maybe they want to show that they can be Muslim and French at the same time, that being one does not exclude them from being the other. For the people who don’t support headscarves or other religious signs, they would see the chadors with the blue, white, and red as incompatible with the principle of secularism (or neutrality) of France. In my opinion, it’s an intelligent testimony that the girls have made because it’s deep and not violent. By using their intelligence instead of violence (by being constructive instead of destructive), I think the girls show that they have thought long and hard about what they believe. I think it’s admirable to act in this way because it’s more difficult to behave calmly logically about what one seems passionately to be? In this extract, the student uses French to express her reactions to the images. She attempts to construct her understanding of the perspective of the women in the images and shows awareness of how they position themselves in relation to controversial discourses and understanding of the cultural symbols that are deployed by the women to state their positions. The example shows the student transposing her thinking in English into French, and the resulting text shows many examples of unidiomatic language use, but at the same time the language is complex for the level of student and reveals her ability to express her interpretations in the language she is learning. Issues and considerations Elaborating the construct

The discussion above points to the intricacy of describing the construct of multilingual assessment. The interrelationship between language and content/ knowledge and language and culture needs to be taken into account, with a recognition that in diverse educational contexts of learning, multilingual assessment may focus differently on the assessment of ‘content’/‘knowing’ (concepts, ideas, phenomena) and of the languages that form the multilingual repertoire. The nature of multilingual assessment is such that it needs to incorporate the assessment of content/knowledge and language use. In addition, given the personal nature of students’ multilingual repertoires, there is a need to acknowledge the embodied nature of multilinguality; in ‘moving between’ languages and cultures, students come to understand others’ situatedness and their knowledge, perspectives, positionings, as well as their own. This

Developing a multilingual perspective in language assessment 139 understanding is best revealed through processes of reflection and reflexivity/ introspection, as processes that invite participants to externalise their thinking. Through these processes, students develop sensitivity to noticing multilingual and intercultural features of language use, through a meta-analysis that is language-and-culture focused. We have proposed that a consideration of the construct necessitates the integration of:

• multiple languages and cultures and related knowledge/knowing within learners’ evolving repertoires;

• multifaceted dimensions including interaction to exchange meanings,

analysis of concepts, ideas, perspectives, etc., and reflection on own/other positioning perspectives, choices, reactions, responses, and identities (see Liddicoat & Scarino, 2020).

It also necessitates characterising the nature (i.e. characteristics, features, and qualities) of multilingual and intercultural capabilities. How the various dimensions of multilingual assessment are integrated and how value is appraised in the assessment process at different levels and scales of accomplishment remain a challenge, both in theory and in practice, one that will require further conceptual consideration and investigation. Eliciting evidence

Multilingual approaches to assessment in the learning of an additional language require assessors to develop elaborated approaches to eliciting evidence of multilingual and intercultural learning and use. There is no single task that can elicit the whole range of the complex capabilities that a language learner is developing, as different tasks can only elicit aspects of the overall capabilities involved (Liddicoat & Scarino, 2010). This means that the approach to eliciting evidence of learning cannot be singular or episodic but rather requires an integrated, developmental approach to designing assessment activities. Eliciting therefore needs to begin with profiling the learner as a multilingual and intercultural user and learner of language. The focus is on learners’ agency as they participate in processes of communication, as users of languages in cultural contexts, as learners and knowers, and as persons. This involves more than representing ‘content’ or knowledge through languages, as has been the main focus of much work on multilingual assessment, and also requires a focus on how meanings are made and interpreted in and between languages, how languages are drawn upon in acts of making and interpreting meaning, and how learners are able to display awareness of the affordances and constraints of working within and between languages. Elicitation therefore needs to move beyond performance, in the sense of doing things in and through languages, and capture the learners’ analysis, understanding and interpretation of what happens in communication (Liddicoat & Scarino, 2013, 2020). It needs to be sensitive to how the activities learners engage in positioning them as users of

140  Anthony J. Liddicoat and Angela Scarino language, as enculturated beings, and as having complex relationships with the languages and cultures that are part of their repertoires. It also needs to ensure that learners have sufficient opportunities to engage with meaning making in, through, and across languages and cultures to evidence complexity and sophistication. This also requires a meta-analytic dimension in which learners have opportunities to evidence sensitivity to the ways that languages and cultures shape their meaning making and interpretation and to demonstrate how they make sense of their experiences of language and culture in their learning and use of multiple languages. Judging/evidencing multilinguality

The area of judging multilinguality is the least developed dimension of multilingual assessment. This is no doubt related to the issues surrounding the nature of the construct as well as the fundamental purposes and contexts of multilingual assessment, notably, to make considered judgements about the learners’ understanding and use of ‘content’ (where content dominates in the relationship between language/culture and content) or about their understanding and use of languages (where language-and-culture-dominate in the relationship between language/culture and content). The difficulty is also related to the fact that assessing multilinguality entails making judgements about the situated and embodied experience of students, their recognition of and response to diverse perspectives, their understanding of themselves as multilingual (or coming-to-be multilingual) users, and their capability to reflect on the mediating role of language and culture in the exchange of meanings. In other words, the assessment is enmeshed with the person, which raises questions about assessing subjectivities (McNamara, 2019) which, in turn, brings ethical considerations strongly to the fore. A further difficulty relates to the ‘architecture’ of assessment and, in particular, the expectation of a reference point for making judgements. For skills such as second language writing, language teachers and the language assessment community more broadly have both an internal (tacit) reference system in relation to which they make judgements of students’ writing, and numerous scales that seek, for example, to characterise writing performance at different levels of education and or levels of sophistication. This architecture is not available for multilingual and intercultural assessment. It must be highlighted that these ‘difficulties’ relate largely to the expectations that teachers and members of the language assessment community hold in relation to assessment. Within such expectations, in relation specifically to judging, there are normally systems of criteria, established a priori, with scales depicting standards, also established a priori. If multilingual assessment is understood as (1) a multidimensional construct, (2) as situated, and (3) as contextualised, involving different configurations of languages and learners/users operating with differing kinds and extents of capabilities in the languages of

Developing a multilingual perspective in language assessment 141 their repertoires these a priori framings become far less, if at all feasible. Furthermore, the process of assessment itself is interpretive, and this reality needs to be recognised. Because of these difficulties, we propose a notion of ‘evidencing’ rather than judging, which is intended to connote the need to identify and establish evidence that reflects diverse aspects of multilinguality and interculturality. This evidencing needs to consider the multiple languages and their relationship, and the nature and extent of functioning in each. In work on assessing interculturality, we have proposed (Liddicoat & Scarino, 2013) a framework for generating criteria, rather than criteria per se. This framework included capturing (1) the nature, scope, and sophistication of the interaction, that required ‘moving between’ languages and cultures, while noticing and managing their own and others’ perspectives and expectations and responding to diverse understandings held by participants and (2) understanding the process of interpretation/exchange and themselves and others as interpreters through reflection on the knowledge/knowing exchanged, the languages of the exchange, and the role of language and culture in the exchange. This approach towards establishing criteria recognises the distinctive context of each instance of multilingual and intercultural interaction and allows for space for emergent criteria. Sadler’s work (Sadler, 2009, 2013) on formative assessment and on connoisseurship (appreciation) provides an expansive way of thinking about criteria and standards that can cater for the distinctive and contextualised nature of each instance or experience of multilingual and intercultural interaction, allowing for evidence to emerge from the experience itself. These qualities of multilingual and intercultural assessment call for approaches that go beyond traditional ones, to recognise the interpretive nature of assessment (see Moss, 2008). Assessment in this sense is closely linked to learning and becomes more akin to processes of inquiry that seek evidence of particular practices and processes in learning in time and over time. This will also entail alternative ways of considering evidence and validating the assessments made (see Moss, 2016). Translanguaging

Translanguaging has been a common focus in multilingual assessment where much of the work has conceptualised it primarily in terms of the production of hybrid language expression (e.g. Gorter & Cenoz, 2017; Shohamy, 2011). This means that assessors try to gauge students’ knowledge through their language repertoire by drawing on whatever elements provide evidence of learning. While this understanding of translanguaging is significant for the assessment of content knowledge, assessing learning of an additional language requires a different way of thinking about the possibilities for translanguaging. In such programmes, language itself is the content and the assessment focus is the assessment of language, and of communication within a specific

142  Anthony J. Liddicoat and Angela Scarino language. This means that understanding students’ multilinguality requires evidence of different ways of engaging students’ multilingual repertoires. While much translanguaging work has focused on the use of complex repertoires as a pedagogical process in which teachers and learners draw on their repertoires to enact and develop learning (García & Li Wei, 2014; Li Wei, 2017), translanguaging does not simply refer to hybrid forms of communication but rather refers to the idea of choice, that is, selecting from the repertoire those elements which most effectively meet communicative, educational, or other goals. This means that translanguaging is sometimes manifested in, and needs to be manifested in, a ‘monolingual mode’ (Li Wei, 2017, p. 18) to meet the communicative requirements of the situation. That is, language learners need to be able to make choices knowingly and purposefully from their repertoires that address their communicative needs, which involves a continuum from monolingual modes in one or another named variety to other more multilingual modes (García, 2009). This ability to select either monolingual and multilingual modes from their repertoire is central to the construct of multilinguality and needs to be considered within a multilingual orientation to assessment. Nonetheless, it remains important to recall that even in monolingual mode, translanguaging remains present and speakers continue to draw on their whole repertoires, and more importantly, need to move ideas between language systems, for example by translation or other mediational processes (Jones & Lewis, 2014). Conclusions In thinking through multilingual assessment in the context of additional language learning, it becomes clear that a multilingual orientation to assessment is a pluralistic endeavour that attends to the contextualisation of students, their languages, and their language learning. What assessment looks like will vary according to which languages are in play, for which purposes they are being assessed, and for which purposes they are being learned. It involves an ecological understanding of the languages that students bring to their learning and to the assessment and how these languages are understood and valued within local linguistic ecologies of society and education. This ecology establishes the setting in which the assessment is undertaken and in which the students’ work is assessed. Multilingual assessment therefore depends on the type of programme and the purposes of assessment; it is not a unitary phenomenon with a single assessment architecture. It also becomes clear that the conceptualisation of the construct needs to be revisited to tease out the implications of the diversity of assessment and how multilingual capabilities are best understood in the context in which the assessment occurs. This conceptualisation needs to go beyond questions of multilingual ‘performance’ to include how the processes of communicating in and through languages are recognised, understood, and mediated by learners in their learning and in their communication. This requires a more expansive

Developing a multilingual perspective in language assessment 143 view of assessment that dislodges long-established assessment practices to open spaces for new ways of understanding both the what and the how of multilingual and intercultural assessment. References Chalhoub-Deville, M. B. (2019). Multilingual testing constructs: Theoretical foundations. Language Assessment Quarterly, 16(4–5), 472–480. https://doi.org/10.108 0/15434303.2019.1671391. García, O. (2009). Education, multilingualism and translanguaging in the 21st century. In S.-K. Tove, P. Robert, K. M. Ajit, & P. Minati (Eds.), Social justice through multilingual education (pp 140–158). Bristol: Multilingual Matters. García, O., & Li Wei. (2014). Translanguaging: Language, bilingualism and education. New York: Palgrave Macmillan. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31(3), 231–248. https://doi.org/10.1080/0950 0782.2016.1261892 Halliday, M. A. K. (1993). Towards a language-based theory of learning. Linguistics and Education, 5, 93–116. Hofer, B., & Jessner, U. (2019). Assessing components of multi-(lingual) competence in young learners. Lingua, 232, 102747. https://doi.org/10.1016/j.lingua.2019.102747. Jones, B., & Lewis, W. G. (2014). Language arrangements within bilingual education. In T. Enlli Môn & M. Ineke (Eds.), Advances in the study of bilingualism (pp 141–170). Bristol: Multilingual Matters. Kramsch, C. (2009). The multilingual subject: What foreign language learners say about their experience and why it matters. Oxford: Oxford University Press. Kramsch, C. (2011). The symbolic dimensions of the intercultural. Language Teaching, 44(3), 354–367. Leung, C., & Scarino, A. (2016). Reconceptualizing the nature of goals and outcomes in language(s) education. The Modern Language Journal (Special Centenary Issue). https://doi.org/10.1111/modl.12300. Liddicoat, A. J. (2014). Pragmatics and intercultural mediation in intercultural language learning. Intercultural Pragmatics, 11(2), 259–277. https://doi.org/10.1515/ ip-2014-0011. Liddicoat, A. J., & Scarino, A. (2010). Eliciting the intercultural in foreign language education. In A. Paran & L. Sercu (Eds.), Testing the untestable in foreign language education (pp 52–73). Clevedon: Multilingual Matters. Liddicoat, A. J., & Scarino, A. (2013). Intercultural language teaching and learning. Chichester: Wiley-Blackwell. Liddicoat, A. J., & Scarino, A. (2020). Assessing intercultural language learning. In J. Jackson (Ed.), Routledge international handbook of language and intercultural communication (pp 397–408). London and New York: Routledge. Li Wei. (2017). Translanguaging as a practical theory of language. Applied Linguistics, 39(1), 9–30. https://doi.org/10.1093/applin/amx039. McNamara, T. (2019). Language and subjectivity. Cambridge: Cambridge University Press. Moss, P. A. (2008). Sociocultural implications for assessment. Classroom assessment. In P. A. Moss, D. C. Pullin, J. P. Gee, E. H. Haertel et al. (Eds.), Assessment, equity and opportunity to learn (pp 222–225). Cambridge: Cambridge University Press.

144  Anthony J. Liddicoat and Angela Scarino Moss, P. A. (2016). Shifting the focus of validity for test use. Assessment in Education: Principles, Policy & Practice, 23(2), 236–251. https://doi.org/10.1080/096 9594x.2015.1072085. Ortega, L. (2019). SLA and the study of equitable multilingualism. The Modern Language Journal 103 (Supplement). https://doi.org/10.1111/modl.1252500267902/19/23-38. Sadler, D. R. (2009). Transforming holistic assessment and grading into a vehicle for complex learning. In G. Joughin (Ed.), Assessment, learning and judgment in higher education (pp 44–64). Dordrecht: Springer. Sadler, D. R. (2013). Opening up feedback: Teaching learners to see. In S. Merry, M. Price, D. Carless, & M. Taras (Eds.), Reconceptualising feedback in higher education. Developing dialogue with students (pp 54–63). London and New York: Routledge. Scarino, A. (2017). Culture and language assessment. In E. Shohamy, I.G. Or, & S. May (Eds.), Language testing and assessment. Encyclopaedia of language and education (3rd ed., pp 15–31). Dordrecht: Springer. Scarino, A. (2020). Mediation in The assessment of language learning within an interlingual and intercultural orientation: The role of reciprocal interpretation. In M. Poehner & O. Inbar-Lourie (Eds.), Towards a reconceptualisation of second language classroom assessment: Praxis and researcher-teacher partnership (pp 43–60). Dordrecht: Springer. Schissel, J. L., López-Gopar, M., Leung, C., Morales, J., & Davis, J. R. (2019). Classroom-based assessments in linguistically diverse communities: A case for collaborative research methodologies. Language Assessment Quarterly, 16(4–5), 393–407. https://doi.org/10.1080/15434303.2019.1678041. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418–429. https://doi. org/10.1111/j.1540-4781.2011.01210.x. Stathopoulou, M. (2013). Investigating mediation as translanguaging practice in a testing context: Towards the development of levelled mediation descriptors. In J. Colpaert, M. Simons, A. Aerts, & M. Oberhofer (Eds.), Proceedings of the international conference language testing in Europe: Time for a new framework? (pp 209–217). Antwerp: University of Antwerp. Stathopoulou, M. (2015). Cross-language mediation in foreign language teaching and testing. Bristol: Multilingual Matters.

Part II

Assessing plurilingual competence Principles and empirical approaches

9

Assessing the multiple acquisitions from bi-/plurilingual learning – the questions to ask1 Marisa Cavalli

Introduction In the early 21st century education moved from attention to processes to concern for results as products. “Standard mania” superseded “pedagogies” (Guilherme, 2012, p. 366): the latter provides a philosophical underpinning – both ontological and epistemological – and a political understanding of diversity in education, whereas the former offers a number of general guidelines aimed at homogenizing education […]. [T]hey are also radically divergent in their goals, as a critical pedagogy aims to create an active citizen prepared to intervene in society, whereas a ‘standards technocracy’ produces a ‘competent’ individual ready to adapt to and serve society as it has been structured by dominant and established powers.2 This quote provides the framework within which I wish to share my thoughts. Between the two opposing poles the author cites I side firmly with that of “pedagogies”. While I understand the importance of assessment in improving teaching and learning and providing a fair education for all, I believe that not all types of assessment are equal for this purpose. The aim of this text is threefold: 1 To argue that the meritocratic vision that sometimes still characterises compulsory education and some types of assessment is in contradiction with the principles of inclusive and emancipatory education. 2 To prove that changing thinking about assessment is as complex and arduous a task as it is a necessary one if we want to guarantee every pupil the right to an education, which, in turn, is based on linguistic rights. 3 To stress that pupils’ linguistic rights – including learning of/in not only one language but also, a fortiori, several languages – cannot be envisaged solely from the technocratic standpoint, in terms of skills and therefore “products” of the teaching-learning process: this is a current trend one might label a “neoliberal” option. The learning of one or more languages DOI: 10.4324/9781003177197-12

148  Marisa Cavalli actually draws on a whole different set of complex individual development processes and calls upon something more than “mere” communication skills, which requires different, more sophisticated and subtle forms of assessment, but without aspiring to assess everything. The purpose of these lines, through a series of questions, is to elucidate certain blind spots in the assessment of plurilingual skills, above all by exploring the educational added value or, so to speak, that hidden dimension, in terms of multiple acquisitions, that taking into account learners’ repertoires and the learning of more than one language bring to any speaker: I shall focus, therefore, on the educational function of these teachings and not (only) on their instructive function, even if the distinction may appear somewhat artificial as the two often overlap. The idea is to give body to what most plurilingual people and teachers working with plurilingual pupils know intuitively from direct experience: the cultural and cognitive depth and consistency which a bi/ plurilingual upbringing or education gives people, whether acquired formally, informally or non-formally. In other words, I mean to explore the contribution of language teaching to the purpose of creating “an active citizen prepared to intervene in society”. Changing assessment to change education Is assessment necessary? If so, what assessment for inclusion?

The proponents of standard mania will swear only by the absolute need to assess everything in a standardised and quantitative manner. Not only that: assessment nowadays has become a means of regulating school systems and reforms according to productivity standards borrowed from the liberal economy. Management by numbers, leading by test results, can be very useful in my opinion, in so far as they help to steer education systems in a positive way towards more justice and democracy (system assessment). However, relying solely on standardised quantitative evaluation fails to take into account the complex, non-comparable realities of schooling and the many elements that impact the measures, including the cultural dimensions of the different educational systems. Unfortunately, assessment is sometimes carried out in its summative form and used by some teachers and schools as a form of power to keep discipline and punish or select pupils (Shohamy, 2014; Spolsky, 1995). It is also generally expected and monitored by the society, public opinion, the media, the children’s parents and, lastly, by the children themselves, who have been accustomed to non-formative assessment practices. Often this sanctioning use of assessment encourages competition between pupils and develops their competitiveness (often encouraged by their families) rather than their spirit of solidarity and mutual assistance.

Assessing multiple acquisitions from bi-/plurilingual learning 149 What is more, in the face of “standard mania” it must be stressed that in several fields of social action various resistance movements are emerging – at least in the French-speaking world – of people fed up with assessment and measurement and calling for more humanistic approaches that take the complexity of human phenomena into account, approaches that are less standardised and homogenising than certain modern assessment measures (Cassin, 2014; Huver & Springer, 2011). They criticise the rigidity and limits of the evaluation grids in various fields (health, school, assistance needs, …) that imprison social actors working in situations where what counts is the human dimension. Summative assessment would be reserved for post-compulsory schooling, and pedagogical and didactic action would focus firmly and resolutely on constant, well-planned assessment as and for learning (Britton op. cit). This would make it possible to devote much more time to more productive, transformative and emancipatory forms of assessment. On the other hand, teachers would no longer be able to base their teaching on the threat of bad grades and would have to find other professional resources to interest their pupils in what they teach. It would be a didactic revolution, with teachers and pupils (as well as parents, media, society, decision-makers) having to make a number of changes in representations. This means that just as the teacher would have to find other incentives for pupils to learn, based on the joys of learning, on each pupil’s capability, the pupils would be rid of a form of assessment which is often stressful, classifying, selecting and stigmatising them, and which may be their only motivation to learn. They would learn to learn for the sheer pleasure of learning, not just in the hope of getting a good grade. It is also important that only a tiny part of the overall time spent on schooling should be devoted to assessment: assessment cannot and must not encroach on or limit the precious time devoted to what matters most: effective teaching/learning. Indeed, the aims and purposes of all assessment need to be specified: The major purpose of assessment in schools, however, should be to provide interpretative information to teachers and school leaders about their impact so that they have the best information possible about where to go next in the teaching process. Instead we drop tests on schools like ‘precision bombs’. We see the purpose of testing as informing the student, not the teacher, how to change and adapt, and we create tests, not reports, first. Until we see tests as aids to enhance teaching and learning and not primarily as thermometers of how much a student knows now, on this day, on this test, then developing more tests will add little and will remain an expensive distraction. (Hattie, 2015, p. 16) What is being recommended here is a positive form of assessment that would highlight each pupil’s progress and seek to fill in any gaps. Assessment,

150  Marisa Cavalli in short, would contribute to better teaching and better learning, the aim of which would be to educate a future citizen to engage actively and critically in society. That said, it is still essential to evaluate teaching and learning through formative assessment with a view to improving them (assessment as and for learning: Black & Wiliam, 1998, 2009; Britton, 2021; Gipps, 2011), by selfevaluation, which makes learners more autonomous, peer evaluation and the renewal of qualitative tools. In short, emancipatory and transformative approaches should be preferred in compulsory schooling especially, where one of the aims of the assessment – where languages are concerned − is to guarantee, inter alia, that the linguistic rights of all the children are respected. What are the linguistic rights of pupils?

All individuals have general linguistic rights, which may be summarised as follows3:

• The right of all individuals to be able to speak and learn their first language

to the full, regardless of the status of that language (dialect, regional or minority language or one linked to migration). • The right to use their own first language and its full repertoire in building their learning and knowledge of the different disciplines at school. • The right of foreigners to learn (one of) the language(s) of the host country (this is also a duty, as provided for in the Charter for regional and minority languages). • The lifelong right of all individuals to learn foreign languages according to their wishes and/or needs. • The right to learn languages that enable the person to play an active, conscious and responsible part in democratic citizenship. • The right to the acceptance of their own languages and language specificities (accents, language level, registers, etc.) and certain of their cultural specificities (Blanchet, 2016). This list of rights shows that multilingualism at school can take a multitude of different individual forms, in the classroom and throughout later life: this is a far cry from the plurilingualism often induced by most school systems: national language + English. Taking a closer look at the rights of learners at school in relation to the language of schooling, we find: general language rights concerning use of the language of schooling; rights concerning language as subject; rights concerning language as a medium of instruction for other subjects; rights concerning relations between school subjects and between subjects and pupils (Coste et al., 2009a, q.v. for further information). It is no use just listing the language rights schools must guarantee their pupils if the school does not provide its pupils with all the essential experiences

Assessing multiple acquisitions from bi-/plurilingual learning 151 they need in order for these rights to become a reality. A list of these experiences for different school levels provides teachers with concrete means for ensuring that all pupils enjoy the rights to which they are entitled (Beacco et al., 2016, pp. 78–89, para. 3.1). The aims of these experiences are not limited to language and communication skills alone, but include openness to other cultures, autonomy, personal initiative, reflection on the plurality of languages, significant learning opportunities … Achievements linked to plurality of languages and cultures in schools Plurilingualism4 is certainly a complex field that has given rise in the past to theoretical diatribes and polarisations, particularly with regard to its effects. The period of negative effects (from the early 20th century to the 1950s5) was followed by that of neutral effects (1950s–1960s) then by that of positive effects (from the 1960s) with the Canadian immersion experience (Baker & Prys Jones, 1998, pp. 62–65). Bilingual teaching and its evaluation have been extensively explored since the middle of the last century in Canada’s immersion experiments, which have been the subject of substantial research (Cummins & Swain, 1998; Lambert & Tucker, 1972; Mackey, 1976, inter alia). It was an experiment in immersion of children from the Anglophone minority in Quebec in French at school. Nowadays this kind of education and the assessment that went with it is criticised for aiming at binary monolingualism or dual monolingualism and drawing on the monolingual standards expected of native speakers (cf. “the two-solitude assumption”, Cummins, 2007, p. 221). Other models of bilingual education exist, less globally known and renowned, such as Wales (Baker, 1993; Williams, 2017) or the Italian region of Valle d’Aosta (Cavalli, 2005), which are inspired by a plurilingual conception of plurilingualism. From the start of the 21st century the phenomenon of superdiversity, that is to say the rapid, exponential increase in linguistic and cultural diversity scientifically observed in our societies (Vertovec, 2007), is largely confirmed by current demographic data (cf. the UNHCR’s 2021 statistical Report): the population migrations and displacements that have characterised human history since pre-historic times have taken on unprecedented dimensions. Humanitarian crises, wars, and climate change, but also work, studies, leisure activities, tourism and so on in a globalised world, all contribute to the acceleration of migration processes − temporary or permanent – that have always existed and are now fuelling the diversification of languages and cultures in contact with others. In spite of the enthusiasm in the media and public opinion, especially families, in favour of “international” schooling in the more prestigious and economically profitable languages,6 the scientific community remains divided as to the effects of plurilingualism, particularly the bilingual advantage. Which

152  Marisa Cavalli caused Antoniou − who wrote an extensive review article on the subject in 2019 − to say in Grosjean (2019)7: The debate surrounding the bilingual advantage is very heated and fierce. It is also repetitive. Certain research groups consistently find support for a bilingual advantage, while other groups consistently find none. Those familiar with the literature are able to surmise whether the findings will be for or against the existence of a bilingual advantage simply by peering at the list of authors. And that is precisely what we find in the literature on the assessment of bi-/ plurilingual people. If you are familiar with the authors’ ideas it is not hard to predict what their findings will be, on one side or the other. Which means that we should humbly accept that researchers are far from neutral in their approach and will eventually find … exactly what they hoped to find. What definition for plurilingualism and the plurilingual person?

Canada’s immersion and the numerous research projects that accompanied it led to a radical change in views on plurilingualism and highlighted its advantages. From the point of view of psycholinguistics the change began in the 1980s in respect of bilingualism and the bilingual individual (Grosjean, 1982): by exposing the persistent myths (Grosjean, 2010) as to what a “true” bilingual person should ideally be according to the old theories (Bloomfield, 1933, inter alia), that is, a person with native-like control of two languages or the sum of two monolingual individuals, Grosjean’s research helped to develop a realistic image of the characteristics of a bilingual person – through a functional definition of bilingualism as the regular use of two languages – and to show the specificities of his or her cognitive and discursive functioning. Since then globalisation has revealed the existence of different types of plurilingualism, the general definition of which adopted here remains that retained by the Council of Europe (Coste et al., 2009b, p. 11): Plurilingual and pluricultural competence refers to the ability to use languages for the purposes of communication and to take part in intercultural interaction, where a person, viewed as a social actor, has proficiency to varying degrees in several languages, and experience of several cultures. This is not seen as the superposition or juxtaposition of distinct competences, but rather as the existence of a complex or even composite competence on which the social actor may draw. This dynamic competence is seen as unbalanced as regards general proficiency in each language, the profiles between the linguistic abilities in the different languages and the degree of cultural knowledge for each language. The

Assessing multiple acquisitions from bi-/plurilingual learning 153 plurilingual competence, partial and plural, evolving and malleable, is at the service of the linguistic and cultural identity. Which languages are concerned by plurilingualism? The different languages present in the classroom …

Let us take the example of a multilingual class of the type you might find in Europe nowadays and simply try to list the languages certain pupils may speak, knowing that classes are generally made up of a majority of speakers of the national language of which the school teaches the school forms: 1 the standard variety of the language of teaching, 2 a variety not used at school (or a regional variety) of the national language/ the language of schooling, 3 one or more local dialects, 4 one or more dialects from other regions, 5 one or more (varieties of) languages of migration, 6 a regional or minority language, and 7 a foreign language taught at school. … but also the different status of the language of schooling, depending on the pupils …

For the different learners the national language(s) taught at school as the language(s) of schooling could be:

• a first language, in general, for most pupils, • a second language, i.e. not a first language for the migrant pupils but a language present in the context, and

• a foreign language for pupils new to the country. Plurilingual and pluricultural repertoires are therefore rich and can also be very heterogeneous, presenting highly complex configurations of the languages concerned and different levels of competence in each. The language which is to be the common language in the school – that is the language of schooling – presents almost every pupil – especially in certain highly multilingual situations – with very different learning challenges to which school and teachers should be equipped to rise. … hence the need for the school to take learners’ bi-/plurilingualism into account

The possible repertoires in a multilingual class must not be seen as an obstacle to teaching, but turning them into an advantage calls for a veritable revolution

154  Marisa Cavalli for all the actors in the school, particularly the teachers, who must feel responsible for the development of acquisitions both in the language of schooling as subject (broadening of these repertoires into genres, registers, metalinguistic conceptualisation, literature, etc.) and in the use of that language in the other disciplines taught (taking into account their language dimensions: discursive and textual genres, disciplinary rhetorics, specific terminology, etc.). In addition, to provide the necessary scaffolding to each of their pupils according to their different, specific needs, using the resources already present in their repertoires, becomes a priority (Auger, 2010). Standardised tests are not capable of taking all these differences between pupils into account. All this requires awareness of the challenges involved in the linguistic dimensions of every kind of learning (in or out of school). The challenges linked to the assessment of plurilingual competences What type of assessment for compulsory education and for a plurilingual audience?

An ethic of assessment itself, and of the assessor, inter alia, requires humility from the latter, who must acknowledge that all assessment is situated interpretation, in which the assessor is involved. Any assessment is therefore subjective, contingent/relative and not self-sufficient (Huver, 2014). When considering what language assessment should involve we also find a strong ethical component linked to the language rights (mentioned supra to be guaranteed to learners in relation to their personal development, that of their identity and their social, affective, cognitive, academic and other development. The linguistic dimensions are indeed pervasive, and developing them is essential in every human sphere. Seen in this light the forms and processes of evaluation must draw upon, and serve, the principles of justice and social equity and help to implement them. It should be possible for all the functions attributed to assessment to be geared to this vision of the rights to be guaranteed to pupils and the values of justice and equity that must inspire each action taken by the school: this is a sort of wider framework of ethical principles. Now, with reference to the pupils’ plurilingual repertoires, a number of researchers have pointed out a methodological bias in appraisal: […] language skills in bilinguals have almost always been appraised in terms of monolingual norms. The evaluation tools used with bilinguals are often quite simply those employed with the monolinguals of the two corresponding language groups. These assessments rarely take into account the bilinguals’ differential needs for their two or more languages or the different social functions of these languages, i.e. what a language

Assessing multiple acquisitions from bi-/plurilingual learning 155 is used for, with whom it is used and where (what I have since called the Complementarity Principle). The results from these tests invariably show that bilinguals are less proficient than the corresponding monolinguals. People then talk of “a bilingual disadvantage” instead of “a bilingual difference”. I maintained then, and still do, that monolingual tests are, for the most part, inappropriate to evaluate the language skills of bilinguals. (Grosjean, 2021, pp. 112–113) This raises the problem of how to take into account not both (or several) of the pupil’s languages separately (a monolingual conception of bi-/ plurilingualism), but the pupil’s overall repertoire, its inner richness, plurality, heterogeneity and dynamics. This calls for a new vision of assessment and its functions, and for different tools capable of fathoming its complexity. When we take a closer look at the realities of so many classes, it is easy to see how far we still are from this right of every plurilingual pupil. Different types of assessment exist that could help teachers get a better idea of the complexity of the repertoires present in the class. But only if assessment is considered as a subjective, non-definitive, and certainly not “objective” hypothesis that allows them to “form a (provisional and situated) idea” on the basis of which to shape a path for their pupils (Huver, 2014). Diagnostic assessment enables us to establish the starting point of the pupil/ each pupil, in terms of “déjà là”, the resources already in the repertoire and, if possible, the level of competence already achieved. This assessment is all the more important when it comes to describing the pupil’s bi-/plurilingual profile in order to seek out specific learning issues, without confusing them with the child’s bilingual development in this particular case. This function of evaluation enables the teacher to design the learning paths each pupil will follow to acquire the competences they have the right to achieve. Formative assessment – coupled with self-evaluation and peer evaluation or co-evaluation to lead the learners towards autonomy – helps to steer the linked processes of teaching and learning and regulate progress according to the needs of each learner. It focuses more on processes than on products. Lastly, certification assessment can help to recognise and show the value of the competences acquired or possessed in the different languages. For reasons already mentioned I will not take summative assessment into consideration, which pays more attention to products and is often contrary to the principles of inclusion that should characterise compulsory education. The other types of assessment listed above do more for learners’ rights and are useful to teachers in guaranteeing those rights in their work: diagnostic assessment (for detecting linguistic development problems, especially for bi-/ plurilingual children); formative assessment, or assessment as and for learning (for redirecting didactic progress and adapting pedagogical options to the difficulties pupils encounter); self-evaluation and peer evaluation, to develop pupils’ autonomy and awareness of the learning they have acquired and those areas in which more work is needed.

156  Marisa Cavalli Needless to say, the reorientation of assessment I am proposing here requires certain essential conditions, including first and foremost special training for teachers designed to develop an understanding first of all that their work should contribute to an inclusive and transformative education and, within that general framework, to a vision of assessment as something that should facilitate and be at the service of such an education.   What is assessed at present when learners are bi-/plurilingual?

Experts on assessment tell us that it is not competences as such that we can evaluate but what Cummins (1979) calls the surface features of language, those which, in his famous iceberg, appear above the waterline. Performances which have tangible (acoustic or graphic), analysable and describable visibility, what the CEFR (among other possible frameworks) defines as language activities (reception, production, interaction, mediation, each of which can be done orally or in writing). But beneath all this, which is visible or audible, we have general competences (knowledge, skills, existential competence and ability to learn) and the communicative language competence comprising linguistic, sociolinguistic and pragmatic competences. Even if we focus on language activities alone, large swaths of these competences elude assessment. The inanity of the efforts of standardisation to reflect the complexity of human practices using objectively measurable standards was clearly stated by Spolsky (1995, p. 351): The promise of absolute certainty promoted by modern science and eagerly espoused by bureaucracies, encouraged the mistaken belief in the possibility of absolute reliable measurement of a complex human skill. In short, assessment is always a subjective, partial, momentary selection of clues to understanding the efficacy of the teaching process: it is always “situated”. So it is not possible to assess “everything”, and that would not even be desirable. Knowing the world we live in (which demands ever more descriptors, criteria, transparency, etc.), it would open up the possibility that these qualitative elements would one day be turned into criteria, descriptors, grids, etc. (Huver, 2014). Instead, it is precisely on the more formative aspects that it would be more interesting and productive to work with the pupils. What else could we assess?

Yet learning (in) another language is one of the most formative of human experiences, one which does not only concern the language activities listed by the CEFR. First of all, it is the experience of the physical incorporation of another code by the sounds we hear, by the sounds we are led to produce, by the reading,

Assessing multiple acquisitions from bi-/plurilingual learning 157 decoding and writing of new signs by hand. A bodily dive into an otherness that we have to appropriate, incorporate, physically make our own. This appropriation of otherness, without the teacher’s careful linguistic scaffolding, becomes particularly cumbersome when a pupil’s language of schooling is a foreign or second language. A language is always an open doorway to other cultures. Even teaching a foreign language purely in terms of language introduces pupils to other ways of categorising the world, of using linguistic signs to express their thoughts. The cultural and intercultural aspects of language learning considerably enhance and diversify the perception of the learner. Not to mention the epistemological enrichment derived from studying different subjects and acquiring knowledge in other languages (Cavalli, 2005). Incorporation and cultures are thus other aspects that must be taken into account in in-depth assessment. Acquiring any language, or indeed, any knowledge, has an impact on the forging of the learner’s identity (Byram, 2006), be it a happy or an unhappy process. The cognitive advantages of bilinguals have been extensively explored in psycholinguistic research and recently more precisely defined: An active area of research since 2000 has followed up this early evidence for a cognitive advantage in bilinguals by focusing on the set of cognitive abilities known as the executive function. These are the processes responsible for attention, selection, inhibition, shifting, and flexibility that are at the center of all higher thought. Exciting new research is now providing strong evidence that early bilingualism has the power to set in place precocious development of these crucial skills and maintain them at a higher level than found for monolinguals through adulthood and into older age. (Grosjean & Li, 2012, p. 193) The indubitable advantages of bi-/plurilinguals occur only in situations of bi-/plurilingualism where all the languages of the repertoire have the potential to develop. When a first language is devalued, socially stigmatised or even ostracised in the eyes of the speaker, there is a risk of it withering and rusting (language attrition widely attested in the scientific literature). This does not imply any superiority of (early or late) bi-/plurilinguals over monolinguals, but it is important to know that the former are not the same speakers as the latter, as Grosjean (2021, pp. 113–114) explains based on his personal experience: “bilinguals are neither better nor worse than monolinguals but are different communicators”. This too must be a part of metalinguistic and metacognitive reflection in class with the pupils, including through portfolio approaches, where writing is a powerful tool at the service of introspection and putting thoughts into words, in order to bring out the particularities of each learner’s repertoire, as they are all “bilinguals in the making” (bilingues en devenir) (Py, 2007)

158  Marisa Cavalli

Figure 9.1  The 20 competences included in the competence model.

through the action of the school, but each in his or her own way, something which fair and differential evaluation should help to value. Some researchers have looked into the assessment of plurilingual competences and made concrete proposals (Coste et al., op. cit. 2009b; Huver & Springer, 2011; Lenz & Berthele, 2010; Serra, 2020), while others have wondered how to assess what cannot be assessed in education (Paran & Sercu, 2010). Lastly, if we look at the recent work of the Council of Europe on the culture of democracy (Council of Europe, 2019), Figure 9.1 proposes the model of competences for a democratic culture. They are competences acquired through languages and their plurality. While some of these competences may fit into habitual forms of standardised assessment, there are others which can only be measured by different types of assessment, and above all by formative assessment, self-assessment and peer assessment and qualitative methods of assessment as Portfolio (cf. Council of Europe., 2021). Conclusion I have tried to show that in plurilingual and multicultural education it is in our interest, through alternative qualitative approaches, to assess something other than the mere linguistic and communicative competences of a plurilingual and

Assessing multiple acquisitions from bi-/plurilingual learning 159 intercultural education, which are of course important and almost self-evident. I refer once again to Grosjean (2021, p. 115): […] bilingualism is the regular use of two (or more) languages, and […] bilinguals are those people who need and use two (or more) languages in their everyday lives. (I added dialects to languages in my definition later on). Although proficiency is not mentioned in the definition, it is implicit as one cannot use a language without having some level of proficiency in it.8 I would like to finish by with a brief reference to one example which, to me, well illustrates the danger of research focused solely on language acquisition and its outcome, no matter how rigorous it may be scientifically. I refer to a research project conducted to ascertain whether the hypothesis of interdependence (Cummins, 1979) between the Heritage Languages (HL) and the learning of the language of schooling, could be verified, which was not the case. It is interesting to follow the authors’ reasoning: A more radical way to go would be to claim that multilingual language learning and usage is complex and dynamic […], to enumerate all potentially important factors and to state that they may or may not play a role, depending on the particular time, context and case, as is done by certain researchers in multilingualism […]. Such theories are seductive as they appear to do justice to the complexity of the phenomena that the applied linguist is interested in. On the other hand, they are not very useful for scholarly research and educational planning: they cannot be proven wrong because they do not make any clear predictions, or their predictions are on the level of the individual which means they do not meet the needs of educational policymaking and curriculum planning. Thus, seen from the point of view of complexity theory or from a dynamic systems framework, the empirical investigation of interdependence effects may well be possible, but it remains unclear what the predictions are that could be tested. For such theories, other methods and epistemological stances than the one taken in this book might be considered. For language policymakers, however, the answer that multilingual language learning is complex and dynamic is not promising when evidence-based recommendations are expected.9 (Berthele & Lambelet, 2017, p. 4140 empl. Kindle) These researchers readily admit that other theoretical points of view exist and that in the face of the complex and dynamic phenomenon of plurilingualism factors other than interdependence may come into play. However, they do not admit the need to find a means of using these theories in policy planning and policymaking. In stating that these theories are not “promising” they deny the (ethical/political) responsibility to seek better means – which is not

160  Marisa Cavalli a scientifically responsible position. They remain attached to a (quantitative) paradigm even while acknowledging its reductionist nature. In the action-research field and observation on the ground, however, using the family language is meant to induce more self-confidence, more self-esteem, more serene identity building, better integration into school processes, a basis for learning the language of schooling, better relationships with families, in the interests of cohesion and social peace, etc. Yet it seems that these things are of no importance for “scholarly research”, or for “educational planning”, where what matters most would appear to be “the needs of research and educational policymaking and curriculum planning”. I would prefer to be wrong, but that attitude eerily echoes part of the initial quotation from Guilherme, where she speaks of “a ‘standards technocracy’ [that] produces a ‘competent’ individual ready to adapt to and serve society as it has been structured by dominant and established powers”. We cannot democratically accept that, because of the lack of suitable methodologies for analysing (some of) the other factors in play, we settle for those which are easy to find and which suit certain researchers and policymakers. Spolsky (1995, p. 355) points out that the effect of the current pressure on standards is “to encourage people to test what is easy or cheap to test, rather than what is harder and more expensive”. The path of future research thus seems to be clear: to find methodologies and tools that make it possible, in a qualitative, rich, holistic and above all equitable manner, to evaluate some of the different educational acquisitions – not only the technical or linguistic ones – that can be achieved through plurilingual and intercultural education. Some researchers have already tried: it is of fundamental importance that all the scattered efforts of the researchers concerned be assembled and made available to school professionals. Notes 1 I should like to thank Emmanuelle Huver and Michael Byram for their critical revision and their advice. Any remaining errors are my responsibility alone. Warm thanks also to my translator, who managed not to … betray my thoughts: traduttore but not traditore. 2 In this same quotation, with considerable caution Guilherme emits the hypothesis of a possible combination of the two educational orientations which, to my mind, is neither possible nor desirable. 3 The fisrt five points of the bullets list are a reformulation of a ancient text of the Council of Europe that disappeared from the website. 4 I use plurilingualism as a hyperonymous term for every form of language plurality, including bilingualism. 5 The most critical period was analysed extensively over a century (1840–1940) by Tabouret (2011). 6 These educational modes are also based on a strict separation, even opposition, between international schools and schools/classes for newcomer and allophone children, between mobility and migration and between plurilingualism and allophony. 7 See also Grosjean (2016) for change of characters dimensions. 8 My emphasis. 9 My emphasis.

Assessing multiple acquisitions from bi-/plurilingual learning 161 References Antoniou, M. (2019). The advantages of bilingualism debate. Annual Review of Linguistics, 5(1), 395–415. Auger, N. (2010). Élèves nouvellement arrivés en France. Réalités et Perspectives pratiques en classe. Préface de Jean-Louis Chiss. Paris: Editions des archives contemporaines. Baker, C. (1993). Bilingual education in Wales. In H. Baetens Beardsmore (Ed.), European models of bilingual education (pp 7–29). Clevedon: Multilingual Matters. Baker, C., & Prys Jones, S. (1998). Encyclopedia of bilingualism and bilingual education. Clevedon: Multilingual Matters. Beacco, J.-C., Byram, M., Cavalli, M., Coste, D., Egli Cuenat, M., Goullier, F., & Panthier, J. (2016). Guide for the development and implementation of curricula for plurilingual and intercultural education. Division of language policy. Strasbourg: Council of Europe. Berthele, R., & Lambelet, A. (Eds.). (2017). Heritage and school language literacy development in migrant children: Interdependence or independence? (1st ed.). Clevedon: Multilingual Matters. Black, P. J., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles Policy and Practice, 5(1), 7–73. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Blanchet, P. (2016). Discriminations: Combattre la glottophobie. Limoges: TEXTUEL. Bloomfield, L. (1933). Language. New York: Henry Holt. Britton, M. (2021). Assessment for learning in primary language learning and teaching. Bristol: Multilingual Matters. Byram, M. (2006). Languages and identities. Strasbourg: Language Policy Division. Council of Europe. Cassin, B. (2014). Derrière les grilles: Sortons du tout-évaluation. Paris: Fayard/Mille et une nuits. Cavalli, M. (2005). Education bilingue et plurilinguisme – Le cas du Val d’Aoste. Paris: Collection LAL, CREDIF, Didier. Coste, D., Cavalli, M., Crisan, A., & van de Ven, P.-H.(Eds.) (2009a). Plurilingual and intercultural education as a right. Strasbourg: Language Policy Division, Council of Europe. Coste, D., Moore, D. & Zarate, G. (2009b). Compétence plurilingue et pluriculturelle, Version révisée et enrichie d’un avant-propos et d’une bibliographie complémentaire. Strasbourg: Conseil de l’Europe (1ère éd. 1997). Council of Europe (2001). Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Language Policy Division. Council of Europe. Council of Europe (2021). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Strasbourg. Council of Europe. Council of Europe (2019). Reference framework of competences for democratic culture (Volume 1 Context, concepts and model - Volume 2 Descriptors - Volume 3 Guidance for implementation). Strasbourg: Conseil de l’Europe. https://www.coe.int/ fr/web/education/competences-for-democratic-culture. Council of Europe (2021). A portfolio of competences for democratic culture – standard version. Strasbourg: Council of Europe. Cummins, J. (1979). Linguistic interdependence and the educational development of bilingual children. Review of Educational Research, 49, 222–51.

162  Marisa Cavalli Cummins, J. (2007). Rethinking monolingual instructional strategies in multilingual classrooms. The Canadian Journal of Applied Linguistics, 10, 221–240. Cummins, J., & Swain, M.. (1998). Bilingualism in education – Aspects of theory, research and practice (1st ed., 1986). London and New York: Longman. Gipps, C. (2011). Beyond testing - towards a theory of educational assessment (1st ed.). New York: Routledge. Grosjean, F. (2010). Bilingual – Life and reality. Cambridge: Harvard University Press. Grosjean, F. (1982). Life with two languages : An introduction to bilingualism. Cambridge: Harvard University Press. Grosjean, F. (2016). The Bilingual Advantage: Three Years Later – Where Do We Go From Here? – What future for an area of research that has become so controversial. Psychology Today. www.psychologytoday.com. Grosjean, F. (2019). The Bilingual Advantage: Three Years Later – Where do things stand in this area of research? Psychology Today. www.psychologytoday.com. Grosjean, F. (2021). A journey in languages and cultures. Oxford: Oxford University Press. Grosjean, F., & Li, P. (2012). The psycholinguistics of bilingualism. Hoboken: Wiley-Blackwell. Guilherme, M. (2012). Critical language and intercultural communication pedagogy. In J. Jackson (Ed.), The Routledge handbook of language and intercultural communication (pp 357–371). London: Routledge. Hattie, J. (2015). What doesn’t work in education: The politics of distraction. London: Pearson. Huver, E. (2014). Les inaccessibles de l’évaluation en langue(s) – impensé? Impasse? Ferments? Glottopol, 23, 77–100. Huver, E., & Springer, C.. (2011). L’évaluation en langues—Nouveaux enjeux et perspectives. Paris: Didier. Lambert, W. E., & Tucker, G. R. (1972). Bilingual education of children: The St. Lambert experiment. Rowley: Newbury House. Lenz, P., & Berthele, R. (2010). Prise en compte des compétences plurilingue et interculturelle dans l’évaluation. Strasbourg: Division des Politiques linguistiques, Conseil de l’Europe. Mackey, W. F. (1976). Bilinguisme et contact des langues. Paris: Klincksiek. Paran, A., & Sercu, L. (2010). Testing the untestable in language education. Clevedon: Multilingual Matters. Py, B. (2007). Apprendre Une langue et devenir bilingue: Un éclairage acquisitionniste sur les contacts de langues. Journal of Language Contact – THEMA 1, 1(1), 93–100. Serra, C. (2020). L’évaluation des compétences en milieu plurilingue. M. Cavalli and L. Gajo (dir). Le professeur de langue 2 dans l’enseignement bilingue – Rôles, fonctions et pratiques pédagogiques: Transversalités et spécificités. ADEB (pp 75–79). Shohamy, E. (2014). The power of tests: A critical perspective on the uses of language tests. Harlow: Routledge. Spolsky, B. (1995). Measured words: The development of objective language testing. Oxford: Oxford University Press. Tabouret, A.-K. (2011). Le bilinguisme en procès (1840 – 1940). Limoges: Editions Lambert-Lucas. Vertovec, S. (2007). Superdiversity and its implications. Ethnic and Racial Studies, 30 (6), Special issue: New directions in the anthropology of migration and multiculturalism; Williams, C. (2017). Wake me up in 2050! Formulating language policy in Wales. Languages, Society and Policy, 1, 1024–1054.

10 Crosslinguistic teacher education How self-assessment and reflection tools can support plurilingual student teachers Lukas Bleichenbacher, Anna Schröder-Sura, Christof Chesini, Mara De Zanet, Cornelia Gantenbein, and Robert Hilbe Introduction: Self-assessment in a plurilingual perspective With its four national languages, one trilingual and several bilingual cantons, as well as a multitude of immigrant languages, Switzerland offers complex sociolinguistic, educational and language policy conditions that place considerable demands on teacher education. The educational institutions respond with different innovative educational models and are confronted with complex challenges, including assessing plurilingual, pluricultural and plurilingual methodological competences. Plurilingual assessment includes assessment formats that address specific needs in specific contexts (e.g., Antia et al., 2021; Gorter & Cenoz, 2017; Seed, 2020; Shohamy, 2011). The question arises which of these formats are best geared to assessing students’ plurilingual competences in teacher education. In our contribution, we focus on self-assessment of plurilingual competences, carried out by students in higher education. We will lean on a model for secondary schools presented by Estaire and Zanon (1994, p. 35) that describes different aspects of the students’ learning process and apply these from a plurilingual perspective. These aspects include the students’ coursework activities, the materials they use and produce and their abilities and attitudes. The particular added value of self-assessment lies in the promotion of autonomous learning (Chudak, 2000; Goullier, 2017). Self-assessment is also considered particularly valuable for promoting plurilingual competence, which is closely related to self-regulated learning (Martinez & Schröder-Sura, 2012), intercultural learning and, especially in teacher education, a prerequisite for plurilingual social agency (cf. Goullier, 2017). Plurilingual and pluricultural competence is a complex and dynamic construct. For its operationalisation in self-assessment tools, it is useful to decompose the competences into the domains of savoirs, savoir-être and savoir-faire. A set of descriptors in these areas is provided by the Framework of Reference for Pluralistic Approaches to Languages and Cultures (FREPA) (Candelier et al., 2012). Not only do these descriptors make the different DOI: 10.4324/9781003177197-13

164  Lukas Bleichenbacher et al. facets of the plurilingual and pluricultural competence tangible, but they also reveal the individual character of plurilingual resources in the process of their development. This conscious perception of one’s own plurilingual development is one component of plurilingual teacher education. A further argument for the use of self-assessment in language teacher education is that it can take place at different levels, which are particularly important also for the development of plurilingualism. The levels are as follows (Weskamp, 1996, p. 407):

• assessment of one’s own approach to solving a task (process assessment), • assessment of the learning outcome (product assessment), • assessment of the given or self-assigned task (task assessment), and • assessment of the learning process as a whole (overall assessment). After an introductory clarification of the use of the underlying terms functional plurilingualism and crosslinguistic teaching and learning in our context, we will present the structure and content of the crosslinguistic teacher education curriculum for language teachers. We will then propose concrete assessment and reflection tools concerned with the multilingual components in the different curricular elements and present an analysis of the students’ responses and results. Crosslinguistic teaching and learning, functional plurilingualism: Terminology used within the curriculum The terminology in the field of multilingualism, its pedagogy and methodology is extremely multifaceted and complex (cf. Candelier & Schröder-Sura, 2016). In the present paper, we will merely refer to those terms relevant to the study programme we are presenting. We first distinguish between multilingualism and plurilingualism according to the Common European Framework of Reference for Languages (CEFR) (Council of Europe, 2001) and its Companion Volume (Council of Europe, 2020). While multilingualism refers to the coexistence of several languages within the same social group or territory, plurilingualism refers to each learner’s individual linguistic repertoire, as well as his/her experience of languages and language learning. Plurilingual competence is characterised by features such as the interconnectedness of languages and linguistic varieties, transversality and distinct levels of partial competences in several languages. In the Swiss context, the term functional plurilingualism is frequently used both in the academic and official discourse (e.g., Hutterli, 2012), because of its strong conceptual coherence with the policy aim to equip as many learners as possible with the necessary basic (partial) competences in at least two languages apart from the local language of schooling (D-EDK, 2016). Based on the characteristics just mentioned, the focus is on linguistic and cultural agency and the active recourse to language use and language learning strategies.

Crosslinguistic teacher education 165 The terms crosslinguistic teaching and learning are used to account for a central organisational aspect of the St. Gallen curriculum, namely the provision of courses where students of different target languages are mixed. Within this setting, both students and lecturers need to make use of their functional plurilingualism, in that they use not only the languages they specialise in but also other languages. In doing so, they are expected to simultaneously learn about and apply the key processes and strategies of plurilingual teaching for their own learning, as well as their (future) teaching. The context: The plurilingual curriculum for the initial education of lower secondary language teachers at St. Gallen University of Teacher Education Teacher education for lower secondary level at St. Gallen University of Teacher Education consists of a 4.5-year Masters programme. Teachers typically specialise in four subjects, with German (the language of schooling) or mathematics as a major. The choice of foreign language subjects includes English, French and Italian. In schools across Eastern Switzerland, English is the first foreign language, French the second and Italian an optional third; accordingly, most students are functional in English and French. An important exception is the trilingual canton of Grisons where Italian and English are taught, while French is optional. In recent years, a complex pattern of options for student teachers resulted in a situation where a majority of students chose to study one foreign language rather than two, and much like elsewhere in Europe, the student numbers for English are higher than for other languages (cf. Lundberg, 2018). Against this background, a group of lecturers and researchers at St. Gallen University of Teacher Education seized the opportunity of an imminent curricular reform to redesign the foreign language curricula in order to gear initial teacher education more towards the affordances of language education orientated towards plurilingualism. The reform had two main aims: to facilitate the choice of national languages (French and/or Italian) and to enhance plurilingual methodology in crosslinguistic courses that are obligatory for all future language teachers. The most substantial change of this curricular reform was the replacement of approximately 50% (ca. 15 ECTS) of university courses in one foreign language subject with crosslinguistic courses where students of all three target languages – English, French and Italian – are mixed. Table 10.1 shows a simplified overview of the curriculum. All students complete the crosslinguistic courses at the top, plus the language-specific courses for one or more language(s) from the three columns at the bottom. As can be seen from Table 10.1, the curriculum contains language-specific courses (taught in the target language) as well as crosslinguistic courses in all main areas (subject discipline, subject methodology and language competence). Students are introduced to key aspects of crosslinguistic and plurilingual methodology in various subdisciplines. In linguistics and culture studies, they are familiarised with multilingual and intercultural phenomena.

166  Lukas Bleichenbacher et al. Table 10.1  An overview of the plurilingual curriculum Crosslinguistic Comparative/plurilingual literature/culture and linguistics Four courses, eight ECTS Plurilingual methodology Four courses (with an optional project week), seven ECTS Plurilingual language competence: Transversal principle in all courses Language-specific English

French

Italian

English Studies (culture, literature, linguistics) Four courses, nine ECTS Methodology: English as a foreign language Two courses, four ECTS English language competence One course, two ECTS Language stay Language Diploma Teaching practice

French Studies (culture, literature, linguistics) Four courses, nine ECTS Methodology: French as a foreign language Two courses, four ECTS French language competence One course, two ECTS Language stay Language Diploma Teaching practice

Italian Studies (culture, literature, linguistics) Four courses, nine ECTS Methodology: Italian as a foreign language Two courses, four ECTS Italian language competence One course, two ECTS Language stay Language Diploma Teaching practice

In methodology, they reflect and enact the specific didactic choices associated, for instance, with intercomprehension or intercultural communication. The curriculum is informed by a theoretical concept (Bleichenbacher et al., 2019), based on the pluralistic approaches and objectives of plurilingual teaching outlined in FREPA (Candelier et al., 2012; see also Sauer & Saudan, 2008 for the Swiss context). The main dimensions of plurilingual teaching include language across the curriculum, language awareness, plurilingual subject teaching, exchange and encounter methodology, and intercultural education. All of these dimensions inform, albeit to different degrees, the various subdisciplines in the curriculum, which include subject-specific knowledge, subject methodology, language competence development, teaching practice, and the language stay. For the implementation of plurilingual teaching in the crosslinguistic modules, the concept planned a number of prototypical scenarios (Bleichenbacher et al., 2019, 12ff.), distinguished by the typical patterns of lecturers’ and students’ language choice. The scenarios included different forms of team teaching, where lecturers from English, French and Italian co-prepare and co-teach using the respective target languages. In some cases, lecturers also teach on their own in more than one language. While German (the language of schooling) is used in some courses, the dominant pattern of language choice is a preference for a combination of the three foreign languages.

Crosslinguistic teacher education 167 A research project initiated to survey the implementation of the curriculum, Plurilinguale Ausbildung von Lehrpersonen, has documented several positive outcomes, but also some problem areas. Students still most frequently choose English only (ca. 50–60% of all students), but there is also a relatively high and stable percentage of 20–30% who choose two foreign languages. Moreover, Italian was successfully reintroduced in the curriculum. The overall commitment of lecturers to engage in cooperative university teaching across subject and language boundaries is high, and so is the students’ willingness to participate. At the same time, both lecturers and students have pointed to some organisational and conceptual difficulties. Since the curriculum was first introduced, the activities have not been conducted in an adequately plurilingual way for all students, especially members of language minorities, to understand the content. In reaction, the project team decided to create and pilot the self-assessment instruments described in this chapter as, among other aims, a means for students and lecturers to diagnose and react to difficulties during plurilingual learning at an early stage. Functional plurilingualism in practice: The plurilingual project week The plurilingual project week is an optional course for all foreign language students. It involves two or more lecturers from different language disciplines and up to 25 students, mixed mainly from the second and third year of study. The course programme includes background reading, brief theoretical inputs and task-based activities with a focus on pedagogy and methodology (e.g., how plurilingual teaching can cater for different learning styles), sociolinguistics (e.g., editing a video on the linguistic landscape of Eastern Switzerland) and culture studies (e.g., preparing food from different cultural contexts). Many activities involve the students’ engaging with languages that are little known to most, such as Swedish, Portuguese, Romanian, Romansh, Polish and Arabic. The self-assessment instrument consisted of nine activities inserted into a PowerPoint presentation. The students were asked at regular intervals, typically once or twice a day of the project week, to access and work on a specific activity related to the assessment of the process, the task or the outcome. At the end of the week, and after requesting formal consent from those students for anonymised use of the data, the 22 students’ answers were collected, crosstabulated and analysed with content and discourse analysis, as well as descriptive quantitative statistics. In this paper, we will present the results of four analysed activities that took place on different days. Data collection with the instruments used for the plurilingual selfassessment of the students was carried out during the plurilingual project week in April 2021, which took place in an online format due to the COVID-19 pandemic.

168  Lukas Bleichenbacher et al. Table 10.2  Self-assessment at beginning and end of project week (N = 22) Descriptor

1 I am able to make targeted use of the diversity of languages and cultures for language learning 2 I am able to apply and develop my listening and reading strategies in multilingual learning situations 3 I am able to reflect on my languages, on my language learning and on my plurilingualism

Beginning of the project week (t1)

End of the project week (t2)

Mean

Standard deviation

Mean

Standard deviation

3.05

0.58

4.29

0.62

3.10

0.86

4.00

0.88

3.90

0.88

4.43

0.49

Using can-do checklists: Towards individual plurilingualism

The can-do checklist consisted of three competence descriptors concerned with openness towards plurilingualism and cultural diversity, based on a logbook designed to familiarise teachers with plurilingual teaching and learning (Egli Cuenat et al., 2013, p. 7). For each descriptor, the students indicated an approximate value by shortening or lengthening an arrow from left (minimal competence) to right (maximal competence) and noted a brief example in German. The length of the arrows was converted into a figure from 0 to 5 for this analysis. The students completed this exercise twice, at the beginning of the project week (t1) and before its end (t2). Table 10.2 shows the average values at both moments. The results of this descriptive analysis show slightly positive values and a clear increase in the students’ self-assessment for the first two competence descriptors, which are concerned with aspects of their plurilingual learning. The average for the third descriptor was already relatively high at the outset, but an increase at the end of the week can still be discerned. The short prose examples offered by the students refer to a wide variety of learning strategies. With respect to the first competence, exploiting the diversity of languages, several students describe how the genetic similarity, especially of Romance languages, facilitates learning. Lexical similarities are mentioned and illustrated with examples, also beyond Romance languages, e.g., Italian lettura vs German Lektüre. While the concepts and terminology in a few statements are vague (one student uses the term code-switching to refer to the recognition of cognates), others enrich their examples with background information, e.g., a historical explanation of why Romance and other languages are similar in the first place. The frequent references to Romance languages can easily be explained by the importance of French, Italian and Latin in Swiss schools and the lecturers’ decision to work intensively with lesser-known languages, including Portuguese, Romanian or Romansh. At t2,

Crosslinguistic teacher education 169 the range of topics in the examples is broadened to include pragmatic phenomena (e.g., linguistic politeness), grammatical phenomena (inflection, word order) and sociolinguistic topics such as language dominance or linguistic minorities. Although the descriptor refers to the students’ own language learning rather than to facilitating their pupils’ learning as (future) teachers, several students mention the extent to which they feel better equipped to use plurilingual methodologies in their classrooms. With respect to question 2, students mention specifically plurilingual comprehension strategies but also a fair number of other ones. Examples include improving one’s comprehension by simultaneously listening to and reading a text, creating islands of understanding by highlighting comprehensible words and phrases to infer the context (and vice versa). One student lists several terms in English (“bridging”, “scaffolding”, “bottom-up” and “top-down”), while others state that they regularly work on their comprehension skills by watching films. Another student argues that although they manage to infer some words even in an unknown language, comprehension of an entire text often remains difficult. The same student highlights that they feel more confident in listening than in reading. At t2, examples of intercomprehension strategies recur, though with more specific detail; several students evoke the “seven sieves” (McCann et al., 2002) method and how it helped them to understand texts in an unknown Romance language. The examples for the third descriptor cover a range of primarily positive observations. These include reports of their general interest in or even enthusiasm about languages. The students describe how they go about learning languages, describing routines and strategies such as focusing on relevant and immediate goals, attempting to notice relevant structures also in everyday life or critically assessing their progress. They also explicitly mention instruments such as the language silhouette, an instrument referred to in the project week that the students had already worked with in an introductory course. Three students refer to the courses in the plurilingual curriculum at the university. At t2, students add the observation that the week has contributed to their awareness of linguistic and cultural diversity and encouraged them to devote further attention to it, as in this example: I noticed during the week that I have probably not questioned and analysed my language, language learning and plurilingualism enough. I had not really realised before that multilingualism is so omnipresent in everyday life. Reflection questions: Following one’s own intercomprehension process

A set of five fields with questions (cf. Estaire & Zanon, 1994, p. 39) was given to the students in English to reflect on their experience of solving an exercise in intercomprehension, understanding a text about a guided tour through Bucharest in Romanian. The first question was: “What did I do to understand the

170  Lukas Bleichenbacher et al. text?” The second question addressed what the students liked best and found easy about the exercise and why. The third asked about possible changes to make the task easier. The fourth enquired about the learning gain through the exercise, and the fifth asked what the students should keep in mind for their future teaching. The answers to the first question refer to the plurilingual strategies of intercomprehension. All students refer to understanding through lexical similarities with languages in their repertoire. There are also many references to structural similarities (especially syntax, occasionally also morphology). Several students refer to the “seven sieves” method. Students also report how they inferred the general plot or context from what they already understood, and one student describes the inverse strategy, i.e., to anticipate the content via the expectations based on the context and the text type. In the answers to the second question, there is an overwhelming consensus that the students’ application of intercomprehension strategies is reasonably easy and straightforward, either by definition or after some initial effort. The exercise is considered manageable and indeed motivating if not, in some cases, fascinating. In their answers to the third question, the students also offer a number of meaningful scaffolding strategies that would make the task easier for themselves and for target-level learners. These include explicit teaching of the intercomprehension strategies, visualisation, parallel versions in another language such as French or Italian, a glossary or grammatical explanations (e.g., to infer grammatical categories from word endings). A few students argue, though, that further support may be unnecessary or could even reduce the learning gain. The answers to the fourth question, where students describe their learning gain, are quite diverse. The topics include a general account of the activity itself (e.g., comparing languages to understand foreign languages, using strategies, applying the seven sieves). While some answers are reasonably short, the following example contains a detailed reflection on the entire process involved in solving and intercomprehension exercise: I have learnt that texts of foreign languages in the beginning often look very complex and seem to be incomprehensible. It’s, however, very important to not trying to understand everything from the start on. You have to go step by step through such a task and read the whole text several times. After every repetition of the words and sentences the context will get clearer. This statement can be seen to reflect a heightened awareness of the affordances of intercomprehension strategies at the target level, in that it highlights that intercomprehension is an iterative and sometimes slow process that can also be challenging for learners at the target level and, therefore, presupposes a fair amount of perseverance and exercise. In the answers to the fifth question on what should be kept in mind at the target level, some students lay an emphasis on ensuring that the learners are provided with enough explicit

Crosslinguistic teacher education 171

Figure 10.1  Example of student’s work.

instruction, time on task and exercise to prevent them from giving up too early, or failing to experience the success of understanding content in unknown languages for lack of adequate strategies. A snapshot of the students’ mood: Capturing reactions to tasks and outcomes

On one project day, the students participated in a series of activities concerned with Romansh, Switzerland’s fourth national language, which is rarely learnt as a foreign language. As a follow-up to the activities carried out, they were asked to answer how they felt immediately after solving the tasks. Thus, this reflection activity is about capturing the students’ reactions to their own outcome and to the task as can be seen in this example (Figure 10.1). For this activity, all features of the task can be relevant, e.g., the type of task, the approach to solving it and the languages involved. To capture their immediate mood and feelings after solving the task at the end of this learning sequence on a polarities profile, they were given several five-point short scales with two adjectival antonyms (e.g., happy – unhappy; nervous – calm; enthusiastic – bored, worried – unworried) corresponding to the PANAVA model, meaning Positive Activation (PA), Negative Activation (NA) and Valence (VA) (Schallberger, 2000). Basically, students who participate in this plurilingual project week are not fundamentally opposed to tackling plurilingual tasks. The results visualised in Figure 10.2 show the students’ preference to position themselves more closely next to adjectives with positive emotional connotations, namely in pairs enthusiastic – bored (0.82), highly motivated – listless (0.91) and full of energy – without energy (0.59). They correspond to high values of positive activation (PA). However, it becomes apparent that the exposure to languages they do not study and to new task formats triggers a rather high degree of activation and motivation. Other statements indicate that, on average, the task did not disturb the students. Although it deals with a language that they do not learn or use

172  Lukas Bleichenbacher et al.

Figure 10.2  Reflecting the mood after having solved a plurilingual task (N = 22).

actively, students felt rather calm (0,61), peaceful (1.18) and relaxed (0.32). The low degrees of negative activation (NA) show that the students feel neither stressed nor inhibited by these tasks. On average, with regard to the valence value (VA), students feel satisfied (0.91) or happy (0.86) after this learning sequence. However, the standard deviations indicate that students’ reactions to the task were rather broadly spread and need an individual follow-up with peers or with the lecturer. Therefore, such reflection activities are appropriate when new plurilingual task formats are used. Students develop an awareness of the components that particularly challenge them, trigger stress and even block them when dealing with multi- and plurilingualism. The students’ resolutions: The sustainability of plurilingual learning

As a pre-final task in the self-assessment portfolio, the students were asked to define resolutions for their future professional development in three areas: (a) their own plurilingualism, (b) their use of plurilingual methodology and (c) their multilingual classroom. While the three foci overlap to some degree, the distinction between plurilingual methodology and the multilingual classroom was made to encourage the students to reflect both on their teaching of one or more target (foreign) languages on the one hand and on how they intend to negotiate further languages spoken by their (future) pupils. The students’ resolutions regarding individual plurilingualism include the learning of new languages or reactivating existing competences in languages such as French and Italian. However, one student also states that they are interested in “unconventional” languages, such as Albanian or Swedish. The use of learning strategies also appears, e.g., to prefer intercomprehension to automatic translation or to work on one’s attitudes towards learning: “I want

Crosslinguistic teacher education 173 to have more courage to speak languages, even if I do not know them perfectly and still make mistakes. There is no reason to be ashamed”. With regard to the intended pedagogical actions, students describe their intention to integrate plurilingualism into their teaching, for instance by highlighting similarities between languages, capitalising on existing resources in the classroom and integrating intercultural topics in one’s teaching. In a rare sceptical statement, a student describes an uncertainty about how to balance learning time devoted to plurilingualism and an explicit focus on the target language French, but adds: “However, I am confident about this and hope that with practice I can find a way to promote both”. In the answers to the third question, the classroom is often interpreted as the actual learning space and how it can be shaped and decorated by the teacher (e.g., with multilingual posters, or language maps), but sometimes also metonymically as the students’ future class of learners, or even their teaching. The dominant topic in the resolutions is how learners’ linguistic and cultural resources can be highlighted in teaching. In two noteworthy statements by future teachers of English only, one future teacher vows to also include French and Italian in the linguistic landscape of their classroom, and another one intends to promote the students’ relying on other languages to reduce their “insécurité langagière” (French in the original). The extent to which the intentions are quite concrete mirrors a hands-on activity earlier in the week concerned with linguistic landscape. Self-assessment of students’ work: The task spider diagram

Within the project week, the students were asked to work in pairs or small groups to elaborate a teaching unit consisting of two double lessons for a foreign language (usually English, French) with, among others, clearly visible plurilingual elements and a focus on pluralistic approaches. In these teaching units, they should optionally consider other foreign, family or national languages in addition to the school foreign languages and the language of schooling. They were also invited to take into account clearly defined contents (e.g., awareness of languages and cultures, multiple intelligences) that had been dealt with in the project week on a theoretical and practice-oriented basis. Further requirements were the consideration of competence-oriented teaching principles as well as the use of several foreign languages when writing different parts of the teaching unit. These teaching units were assessed by the lecturers on a criterion-oriented basis. The students were given the task of assessing their products with the help of a spider diagram, inspired by Blömeke et al. (2006). It represented numerical values on a scale from zero to ten according to the criteria (Table 10.3), some of them referring directly to plurilingualism. The potential of the tool is to make students aware of the strengths and weaknesses of the teaching unit they have developed. A first obvious insight is that the values are very high. This positive result can be explained by several factors. The aspect of voluntary participation and the cooperative development

174  Lukas Bleichenbacher et al. Table 10.3  Assessing plurilingual teaching units Criteria

Plurilingual potential (PP) Theoretical basis (TB) Openness towards linguistic and cultural diversity (OD) Coherence of content and aims (CA) Potential for differentiation (DI) Activation of learners’ resources and strategies (RS) Interpersonal exchange (IE) Affordability (AF) Language comparison (LC) Language awareness (LA)

Assessing plurilingual teaching units (N = 22) Mean

Standard derivation

8.60 6.91 8.92

1.06 1.63 1.57

8.03 6.25 7.88

1.24 2.03 0.75

9.13 8.41 8.57 8.15

1.29 1.49 1.20 1.51

of the teaching unit certainly played a role. In addition, textbook lessons that already contained plurilingual elements served as a starting point for the creation of the lesson planning. Students also confirmed that they feel quite confident in developing teaching units. The highest scores were achieved in the areas of “openness towards linguistic and cultural diversity (OD)” (8.92) and interpersonal exchange (IE) (9.13). Both aspects were among the main foci of the project week. Areas that were only marginally addressed in the project week, such as differentiation (DI) (6.25), are also indicated as less pronounced in the diagram. If we look at the spider diagrams of individual students who have worked together on a teaching unit, there is a tendency to see strong agreement in the assessment of strengths and weaknesses of the task. One student confirms that within the group, they were “looking for meaningful links between languages”. However, the values do vary (cf. standard derivation), as critical engagement with the task is very individual. It is clear from some comments that some students felt insecure about theoretically underpinning the tasks and were aware that, e.g., in the area of language comparisons, their “suggestions” were reduced to the lexical field. The external assessment by the lecturers sometimes showed that a deeper integration of plurilingual components would have been quite possible. However, this first analysis indicates that students can carry out the reflection and, after a deeper engagement with the topic, are willing and able to transfer plurilingual elements into teaching practice. Conclusions and perspectives In what follows, we will summarise and draw conclusions on the students’ results and the usefulness of the applied tools for plurilingual learning and teaching of the project week, as well as indicate possible consequences from

Crosslinguistic teacher education 175 these findings for crosslinguistic teacher education at the St. Gallen University for Teacher Education and, finally, offer some perspectives for plurilingual assessment in general. In general, the students completed all the reflection tasks given to them carefully and in detail and reacted positively to the tool. We consider that the multifaceted self-assessment activities have proven to be stimulating. The tasks were formulated in German and in other languages, and the answers were generally given in different languages, too. The analyses have shown that the acceptance of plurilingual tasks in teacher education and the understanding of plurilingual teaching are quite pronounced. A rejecting attitude was not discernible, nor did we come across any negative feedback that would indicate, e.g., doubts about the meaningfulness or a lack of understanding of the tool. Nevertheless, it is important to ensure that each task is understood in terms of language and content, especially if it is an unknown tool. The differences between the students’ responses were quite noticeable for some aspects. In part, the answers consist exclusively of content that was taken up during the project week and do not include previous experiences. At the end of this project week, the assessment of the beneficial consequences on language competence postulated in the literature (cf. Bachmann & Palmer, 2010) was rated in the good and medium range in the students’ self-assessment. However, a longer period for experimenting, less condensed series of self-assessment tasks which were a further additional charge and a more explicit introduction are possibly needed to integrate the dimension of language competence more specifically in plurilingual self-assessment. These factors were very limited due to restrictions of the project week. Moreover, the plurilingual tasks for the students as well as the construction of the teaching unit were rather easy to manage, so that assessment of the learning processes, the outcomes as well as the tasks themselves were probably not always considered as necessary. Overcoming crisis moments (Melo-Pfeifer, 2020) during the learning process will possibly reinforce the need for the tools. In the local context of the St. Gallen curriculum, the intention is to extend the use of these instruments to compulsory courses. It is expected that such a use can help the team to address some of the challenges hinted at above. For instance, they could empower the students with a solid base for reflecting on and reacting to forms of plurilingual teaching and plurilingual learning that are beyond their abilities, and equip lecturers with further means to plan and coordinate key aspects of their teaching, including the content and activities, the target competences and the patterns of language choice. These and other tools can help lecturers to get ideas and develop a variety of criteria based on ongoing formative assessment tools in areas such as plurilingual language competence, attitudes towards multi- and plurilingualism and development as autonomous learners. The impact of self-assessment in the context of plurilingual education has been discussed mainly in relation to the European Language Portfolio (ELP, e.g., Ballweg & Kühn, 2019; Brinkmann, 2021). The tools created so far for

176  Lukas Bleichenbacher et al. the project presented in this paper take up and deepen some aspects of the EPL but are intended to be specifically tailored to the needs of plurilingual teacher education. They contribute to raising awareness and reflection as well as to promoting plurilingual and pluricultural (teaching) competences. We believe that some of the tools have substantial potential for the alignment of formative and summative assessment. In conclusion, concrete and diverse selfassessment tools, as well as their possible adaptation to summative assessment formats, can have positive effects on crosslinguistic teacher education, as aptly expressed in the following quotation: […] L’importance de l’auto-évaluation et de l’auto-analyse dans cette perspective n’exclut pas toute forme d’évaluation formelle. Elle est possible pour de nombreux aspects de cette compétence. Elle est sans doute même souhaitable pour renforcer l’impact de l’auto-évaluation: l’importance que l’enseignant ou l’institution de formation accordent à la spécificité de cette compétence par le biais d’une évaluation peut confronter l’apprenant dans son désir de progresser, peut influer positivement sur sa motivation et son estime de lui-même. (Goullier, 2017, p. 261) References Antia, B. E., Weldemichael, T., & Dyers, C. (2021). Multilingual assessment: Levelling the cognition-emotion playing field at the University of the Western Cape. Language Matters, 52(1), 50–70. https://doi.org/10.1080/10228195.2020.1839539. Bachmann, L., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press. Ballweg, S., & Kühn, B. (Eds.). (2019). Portfolioarbeit im Kontext von Sprachenunterricht. Universitätsverlag Göttingen. https://doi.org/10.17875/gup2019-1204. Bleichenbacher, L., Kuster, W., Heinzmann, S., Hilbe, R., & Annen, M. (2019). Entwicklung sprachenübergreifender curricularer Elemente für die Ausbildung von Sprachenlehrpersonen Sek I (2nd rev. ed.). Pädagogische Hochschule St. Gallen. https://www.phsg.ch/sites/default/files/cms/Forschung/Institute/InstitutFachdidaktik-Sprachen/CV/ESCEAS_Bleichenbacher%20et%20al%202019.pdf. Blömeke, S., Risse, J., Müller, C., Eichler, D., & Schulz, W. (2006). Analyse der Qualität von Aufgaben aus didaktischer und fachlicher Sicht. Ein allgemeines Modell und seine exemplarische Umsetzung im Unterrichtsfach Mathematik. Unterrichtswissenschaft, 34(4), 330–357. Brinkmann, L. M. (2021). Das Europäische Sprachenportfolio als Spiegel von Mehrsprachigkeit: Förderung der language awareness. In C. Helmchen, S. Melo-Pfeifer, & J. Von Rosen (Eds.), Mehrsprachigkeit in der Schule. Ausgangspunkte, unterrichtliche Herausforderungen und methodisch-didaktische Zielsetzungen (pp 73–102). Tübingen: Narr Francke Attempto. Candelier, M., Camilleri Grima, A., Castellotti, V., de Pietro, J.-F., Lörincz, I., Meissner, F.-J., Schröder-Sura, A., & Noguerol, A. (2012). FREPA – competences and resources. A framework of reference for pluralistic approaches to languages and cultures. Council of Europe. https://www.ecml.at/Resources/ECMLresources/ tabid/277/ID/20/language/en-GB/Default.aspx.

Crosslinguistic teacher education 177 Candelier, M., & Schröder-Sura, A. (2016). Mehrsprachigkeitsdidaktik et Didactique du plurilinguisme: Structure du champ et terminologie – Quelques repères. Synergie – Pays Germanophones, 9, 33–46. Chudak, S. (2000). Die Selbstevaluation im prozess- und lernerorientierten Fremdsprachenunterricht (Bedeutung, Ziele, Umsetzungsmöglichkeiten). Glottodidactica, 28, 49–64. Council of Europe. (2001). Common European framework of reference for languages learning, teaching, assessment. Cambridge: Cambridge University Press. Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume. Strasbourg: Council of Europe. D-EDK (Deutschschweizer Erziehungsdirektoren-Konferenz). (2016). Lehrplan 21: Fachbereich Sprachen. https://v-fe.lehrplan.ch/index.php?code=b|1|0&la=yes. Egli Cuenat, M., Klee, P., & Wirrer, M. (2013). Brücken zwischen envol und open world (voices, new inspiration). Unterwegs zur Mehrsprachigkeit. Lehr-/Lernjournal. Lehrmittelverlag St. Gallen. Estaire, S., & Zanon, J. (1994). Planning classwork: A task based approach. Oxford: Heinemann. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. International Journal of Multilingualism, 13(1), 1–18. Goullier, F. (2017). Evaluer la compétence plurilingue et interculturelle? In J.-C. Beacco & D. Coste (Eds.), L’éducation plurilingue et interculturelle - La perspective du Conseil de l’Europe (pp 257–273). Paris: Didier. Hutterli, S. (Ed.). (2012). Koordination des Sprachenunterrichts in der Schweiz – Aktueller Stand, Entwicklungen, Ausblick. Bern: EDK. Lundberg, A. (2018). Multilingual educational language policies in Switzerland and Sweden: A meta-analysis. Language Problems and Language Planning, 42(1), 45–69. https://doi.org/10.1075/lplp.00005.lun. Martinez, H., & Schröder-Sura, A. (2011). Der Referenzrahmen für Plurale Ansätze zu Sprachen und Kulturen: Ein Instrument zur Förderung mehrsprachiger Aneignungskompetenz. Die Neueren Sprachen. Jahrbuch des Gesamtverbandes Moderne Fremdsprachen, 2, 67–83. McCann, W. J., Klein, H. G., & Stegmann, T. D. (2002). EuroComRom – the seven sieves: How to read all the romance languages right away (2nd rev. ed.). Aachen: Shaker Verlag. Melo-Pfeifer, S. (2020). “Plurale Ansätze werden mich in der zukünftigen Unterrichtsvorbereitung beeinflussen” – Dilemmas und Einstellungen künftiger Spanischlehrer_ innen zu Pluralen Ansätzen. In S. Morkötter, K. Schmidt, & A. Schröder-Sura (Eds.), Sprachenübergreifendes Lernen: Lebensweltliche und schulische Mehrsprachigkeit (pp 97–117). Tübingen: Narr Verlag. Sauer, E., & Saudan, V.. (2008). Aspekte einer Didaktik der Mehrsprachigkeit. Vorschläge zur Begrifflichkeit. Passepartout. https://www.sprachenunterricht.ch/sites/default/ files/Didaktik-der-Mehrsprachigkeit.pdf. Schallberger, U. (2000). Qualität des Erlebens in Arbeit und Freizeit. Eine Zwischenbilanz. Psychologisches Institut der Universität Zürich. https://www.psychologie. uzh.ch/dam/jcr:00000000-4a5f-c2e5-ffff-ffffea52afb9/BaAAPZwBil31.pdf. Seed, G. (2020). What is plurilingualism and what does it mean for language assessment? Cambridge Assessment English – Research Notes, 78, 5–15. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessing policies. The Modern Language Journal, 95(3), 418–429. Weskamp, R. (1996). Selbstevaluation: Ein zentraler Aspekt schülerorientierten Fremdsprachenunterrichts. Fremdsprachenunterricht, 6, 406–411.

11 Plurilingualism and assessment Some issues and options Enrica Piccardo and Brian North

Introduction Much of the literature on multilingual assessment concerns linguistic accommodation for emerging bilinguals in formal tests, often taking the form of two parallel versions of the test. Shohamy (2021) suggests that scores can more or less double when bilinguals are able to access the test in two languages. De Backer et al. (2019) in Belgium and Heugh et al. (2017) in South Africa each report that over 80% of students found this kind of accommodation useful, Shohamy (2011, 2021), De Backer et al. (2019), Guzman-Orth et al. (2019) and Schissel et al. (2019) all found that it had unpredictable positive effects. However, significant though the approach is, it only concerns “bilingual tests” of subject content. In addition, accommodation is often based on assumptions of homogeneity regarding language varieties, cultural background and the appropriateness of the original test specifications (Huempfner, 2004). De Backer et al. (2017) argue that what is needed is a far broader view of assessment in order to do justice to the full content knowledge of bi-/ plurilinguals and to assess their plurilingual competence, and a shift “towards an assessment where a learner can use all of his/her multilingual repertoires to demonstrate his/her knowledge and competences” (pp. 226–227). Such an approach should take account of “actual language practices” (Gorter & Cenoz, 2017), emphasising “features such as plurilinguistic communication and translanguaging” (Chalhoub-Deville, 2019, p. 476) and, fundamentally, adopt “a languaging and linguistic repertoire theory of language” (Schissel et al., 2019, p. 376) in which the learner is encouraged to “to use all their languages, including mixing them if needed” (Lopez et al., 2017a, p. 5). Saville considers that: “a transformative shift is needed in policy-making and action planning” to adopt a plurilingual perspective (Saville, 2019, p. 470). De Backer et al.’s “broader view of assessment” and Saville’s “transformative shift” imply a reconsideration of the nature and scope of assessment itself. It is with the nature of such a shift and possible modalities for the assessment of plurilingual competence that this article is concerned.

DOI: 10.4324/9781003177197-14

Plurilingualism and assessment 179 We first consider the “broader view” of assessment and then move on to discuss some options for assessing plurilingual competence. Renewing assessment During the 1990s, whilst language testing focused on models of communicative language proficiency and test validity, in mainstream education there was a growing awareness of the need for systematic ongoing teacher assessment related to learning objectives. Publications with titles like Beyond Testing (Gipps, 1994) and Assessment for Learning: Beyond the Black Box (Assessment Reform Group, 1998) moved away from the straightjacket of measuring through “objective” tests towards an action-oriented paradigm: feedback and feedforward fully integrated into the teaching process. This implies a synergistic integration of curriculum, teaching/learning and assessment (Graves, 2008) with a change in the role of both teachers and students in a perspective of inclusive education, through a process of “constructive alignment” (Biggs, 2003). The alignment is between planning, teaching and assessment, and the constructive “refers to the idea that students construct meaning through relevant learning activities” (Biggs, 2003, p. 2). The Common European Framework of Reference for Languages, Learning, teaching assessment (CEFR: Council of Europe, 2001, 2020) takes this approach: Both the CEFR descriptive scheme and the action-oriented approach put the co-construction of meaning (through interaction) at the centre of the learning and teaching process […] In the reality of today’s increasingly diverse societies, the construction of meaning may take place across languages and draw upon user/learners’ plurilingual and pluricultural repertoires. (Council of Europe, 2020, p. 30, our emphasis) With its concept of plurilingual and pluricultural competence, the CEFR “describes the complex language practices of multilinguals in a more accurate way” (Lopez et al., 2017b, p. 9). Changes in assessment practices that the CEFR inspires are summarised by Piccardo (2013) on the basis of data from teachers’ interviews. As regards assessment, among other things, the CEFR promotes:

• alignment of syllabus aims, teaching and assessment; • increased awareness of the complexity of assessment and of all the different aspects that need to be taken into account – including plurilingual profile;

• the value and effectiveness of awareness-raising with learners, particularly

the communication of criteria, self-assessment in relation to them and consequent sharing of the responsibility for progress and assessment;

180  Enrica Piccardo and Brian North

• a new vision of error, which is not a “sin” (une faute) but should be seen, as expressed by a teacher in France, as erreur constructible;

• an appreciation of assessment as a support to learning, as opposed to summative, formal grades.

In language assessment a number of such assessment for learning approaches have emerged. In discussing a shift to multilingual assessment policies and practices, De Backer et al. (2017) define assessment for learning as follows: In assessment for learning, evaluation and instruction are not seen as two separated tasks a teacher has to do, but are integrated. Assessment is learning. Providing feedback and scaffolds during increasingly complex [tasks] are helpful tools in supporting multilingual pupils to perform in their ZPDs. (De Backer et al., 2017, p. 225) This is very similar to the concept of dynamic assessment (Poehner, 2008) and to the related learning-oriented assessment (Turner & Purpura, 2015). Accepting complexity From a plurilingual perspective, assessment becomes more complex because it involves working at different levels at the same time. Students will plurilanguage (fully exploit all linguistics, semiotic and environmental resources: Piccardo, 2017) only if they realise that they are allowed to exert their agency, feeling empowered to do so vis-a-vis the norm of using the language of schooling (LoS) or designated target language. In the classroom, they need to be helped to see the norm from two perspectives: (a) as something that has been codified through the development of the language, a code that in life they will be confronted with, and which they cannot ignore, and (b) as a pure convention that does not correspond to the fluid use of speakers’ linguistic repertoires, an abstract construction that can be stretched and challenged when the external constraints permit this. In addition, when we talk about assessment, we are talking about very different things in terms of what we assess, when we assess and who assesses whom. These issues are considered below. What to assess

There is a fundamental distinction between (a) ways of allowing plurilinguals to access all their languages in demonstrating knowledge of content (multilingual assessment: discussed in the Introduction) and (b) the assessment of plurilinguals’ language competences to give value to their plurilingual profiles, their metalinguistic capabilities and their capacity to use languages flexibly and appropriately – the focus of this chapter.

Plurilingualism and assessment 181 In addition, there is the question whether plurilingualism should be involved in assessment indirectly or directly. Indirect involvement could mean allowing collaborative work in any language(s) during the preparation phase of an assessment task, with only the final artefact being assessed. The problem here is that if something is not assessed then this can suggest it has no value, an unintended effect particularly dangerous in relation to the use of heritage languages as a resource, which is already undervalued by teachers (De Angelis, 2011; Gorter & Arocena, 2020; Young, 2014). Therefore, we consider that it is good educational policy not only to allow plurilinguals to access all their resources while preparing to accomplish a task but also to include opportunities for plurilingual behaviour in assessment, and we discuss later in the article some options for direct assessment of plurilingual competence. When to assess

Since plurilingualism implies the value of unbalance and dynamic development in individuals’ repertoires, assessment should be undertaken as a process over time, since: “variability and non-linearity seem to be a matter of course (run-of-the-mill), rather than exceptional” (Van Steendam et al., 2012, p. xx). Specific assessment moments – snapshots – should be seen as part of an ongoing process, and we need to assess the overall process as well as the snapshots as evidence of development. Dynamic assessment takes this perspective: it is through the degree of scaffolding provided by the teacher that the process of displaying the learner’s proficiency in the language is supported, and not just controlled. Who assesses whom?

The key to harnessing assessment to promote learning is an integrated approach to classroom teaching and assessment: space needs to be made for teacher, peer and self-assessment. Studies suggest that, given transparent descriptors and some training, teachers are generally well capable of monitoring and assessing aspects of their learners’ language competences (e.g., Berger, 2020; Fleckenstein et al., 2018; Lankina & Pect, 2020), as are their students in peer and self-assessment (Frost & O’Donnell, 2015; Idris & Raof, 2017; Thompson, 2015). How to assess?

Some forms of holistic assessment, like structured observation of collaborative work, are simply part of good teaching. Self- and peer assessment helped by selected descriptors can have an awareness-raising effect as well as complementing teacher assessment. Using validated descriptors to design tasks and to monitor and assess students’ actions and language in the process of completing

182  Enrica Piccardo and Brian North those tasks is a valuable experience for both teachers and learners. In piloting the new CEFR descriptors teachers made comments like the following:

• [Students reacted] very enthusiastically. They found self- and peer assessment

based on the scales very much effective, especially if followed by teachers’ comments and feedback on their performance (Tertiary, B1/B2, mediation); • I found [the descriptors] to be clear and coherent and, thus, facilitating the way to promote the plurilingual competence. The activity served to help students see that although they struggle at times with English, they have the resources and tools available to them to make meaning of new language (Primary, A1, plurilingualism). Pavlovskaya and Lankina (2019) give an example of the way CEFR descriptors can be used to combine informal and formal assessment, assessing both process and product. They monitored collaborative mediational talk using descriptors for Facilitating collaborative interaction with peers, Collaborating to construct meaning, and Encouraging conceptual talk (Council of Europe, 2020, pp. 109–113) after familiarising students with the descriptors and the concepts in them. Their data suggests not only an awareness-raising effect but also enhanced quality of language used (compared to a control group) in the final product. Supporting plurilingual/pluricultural assessment One core issue in encouraging a shift towards a complex, plurilingual perspective is supporting teachers with tools to do so. Here we certainly disagree with Schissel et al., who consider that: “[d]efining parameters, scales, or other measures common to language testing seems antithetical to the dynamism of languaging and a holistic linguistic repertoire” ( 2019, p. 375). On the contrary, descriptor scales can be very helpful, not least to illustrate a profile across that repertoire (see Council of Europe, 2020, p. 40). More fundamentally, descriptors can provide concrete, cognised goals (Bandura, 1989), to signpost to learners the purpose of activities, to suggest real world-oriented classroom tasks, to monitor small group work and for teacher, peer and self-assessment and feedback. In addition, the inclusion of descriptors for plurilingual language use in an international language policy document like the CEFR can have an empowering effect for teachers who wish to promote a plurilingual approach in contexts in which colleagues may be reluctant to do so. Descriptors are an important tool for the constructive alignment discussed earlier. They can help raise learners’ awareness, particularly in valuing their plurilingual repertoire. As Little et al. (2011) reported in their stock-taking of ten years of experimentation with the European Language Portfolio (ELP), with its CEFR-based descriptors, “learners appreciated the opportunity to reflect on their plurilingual and pluricultural experience” (2011, p. 12; see also Gorter & Cenoz, 2017).

Plurilingualism and assessment 183 Flores and Schissel (2014) call for the provision of concrete tools to assist the enactment of heteroglossic implementational spaces, and we argue that descriptors can provide relevant tools to facilitate such space. In discussing the way in which the New York State Bilingual Common Core Initiative (2013) provide a blueprint to teachers for the stages of scaffolding emergent bilinguals need, Flores and Schissel illustrate the way in which a descriptor, Describe characters in a story (e.g., their traits, motivations or feelings) and explain how their actions contribute to the sequence of events, can be turned into performance indicators for five stages of development by being modulated with a 5-point scale: (entering, emerging, transitioning, expanding and commanding). When descriptors are used for teacher, peer and self-assessment, it is common practice to modulate the descriptor in this way. Apart from helping teachers to accommodate a wide range of levels in a class, this can encourage learners to compare their abilities in different languages with the same descriptors. In addition, it is typical of creative practices to provide constraints that spark the production of ideas. In general people tend not to engage fully in a process if just left alone, acting in a vacuum (Csikszentmihalyi, 1996); they need to be provided with signposts to get hints of the new dimension they are venturing into, and which they will continue to develop in their trajectory. Here too descriptors can play a significant role. For instance, in the following two descriptors from the CEFR Companion Volume scale Building on plurilingual repertoire, learners are prompted to use all their linguistic resources, at a lower level (B1) in order to cope with a difficult situation. At a higher level (C1) learners are given hints on how to proceed when they are faced with a situation in which people need to understand and discuss the content of a text they cannot understand, Can exploit creatively their limited repertoire in different languages in their plurilingual repertoire for everyday contexts, in order to cope with an unexpected situation. (Building on plurilingual repertoire, B1) Can support comprehension and discussion of a text spoken, signed or written in one language by explaining, summarising, clarifying and expanding it in another language in their plurilingual repertoire. (Building on plurilingual repertoire, C1) Even descriptors that do not specifically mention the plurilingual/ pluricultural dimension, such as descriptors for mediating concepts, can easily serve the purpose of making learners aware of the complexity and diversity of perspectives on an issue, and how to deal constructively with it. Can consider two different sides of an issue, giving arguments for and against, and propose a solution or compromise. (Collaborating to construct meaning, B2)

184  Enrica Piccardo and Brian North Can build on people’s ideas and link them into coherent lines of thinking. (Encouraging conceptual talk, B2) These descriptors shown as examples can be used both in a teaching situation and in an assessment context. In fact, these examples show how descriptors have a strong potential for goal-setting and thus can contribute to broadening the scope of language education and instilling a plurilingual/ pluricultural attitude. For their part, Flores and Schissel lament the “relative silence on issues of bilingualism” (2014, p. 468) in US educational standards. They cite the tendency of even examples that “build on the dynamic bilingualism of emergent bilinguals and provide a roadmap for teachers to follow” (2014, p. 272) to fail to “acknowledge that translanguaging is not simply a discursive scaffold for emergent bilinguals that disappears as bilingualism develops” (2014, p. 474). In other words, they regret the implicit perspective in some bilingual education that bilingualism is tolerated only as long as it avoids educational disadvantage while learners are acquiring the LoS. By contrast, the CEFR emphasises that developing plurilingualism is a lifelong educational goal. As Gorter and Cenoz acknowledge, “[t]he CEFR endorses a multilingual focus” (2017, p. 239) and as Lopez et al., confirm “plurilingual and pluricultural competence as defined by the CEFR describes the complex language practices of multilinguals in a more accurate way” (2017b, p. 9). Whilst it is true that some countries have exploited the CEFR in an instrumental manner solely in relation to English, ignoring its plurilingual philosophy (see Savski, 2019), the fundamental aim of the CEFR is to provide the basis for a plurilingual approach to language education across the curriculum and thus counteract: “various forms of purism which regard each language and culture as a separate entity, to be preserved and protected against the threat offered by alien forces” (Trim, 2007, p. 51). While: [m]ost users of the CEFR have applied it only to a single language […] its descriptive apparatus for communicative action and competences, together with the ‘can-do’ descriptors of levels of competence, are a good basis for a plurilinguistic approach to language across the curriculum, which awaits development. (ibid.) Building upon theoretical developments and bottom-up experimentation with regard to plurilingualism since 2001, the CEFR has undergone the development that Trim was hoping for. The CEFR Companion Volume (CEFRCV: Council of Europe, 2020) updates and expands the CEFR 2001 version, emphasising dynamic plurilingual repertoires that develop throughout the life trajectory. In addition to descriptors for mediation within and across languages, varieties and modalities, it provides others for Building

Plurilingualism and assessment 185 on pluricultural competence; Plurilingual comprehension; and Building on plurilingual repertoire. In addition, the REFIC project (Référentiel de compétences de communication plurilingue en intercomprehension: De Carlo & Anquetil, 2019) provides descriptors for aspects of intercomprehension at the three broad CEFR levels and the FREPA project (Framework of Reference for Pluralistic Approaches: Candelier et al., 2012) offers a taxonomy of descriptors for different aspects of plural approaches. These three documents offer teachers the necessary support to embrace a type of assessment that both acknowledges and fosters learners plurilingual/pluricultural repertoires. A construct for assessment of plurilingualism Saville and Seed (2022) discuss the need to define the plurilingual domain for assessment. They consider that the CEFRCV: “provides a useful definition of what plurilingualism is, and has provided two sets of ‘can do’ descriptors explicitly on plurilingualism – Plurilingual comprehension and Building on plurilingual repertoire” (Saville & Seed, 2022, p. 365). As they state, the former is defined as: “the capacity to use the knowledge of and proficiency (even partial) in one or more languages as leverage for approaching texts in other languages and so achieve the communication goal” (Council of Europe, 2020, p. 127) and the latter is defined as “engaged in exploiting all available linguistic resources in order to communicate effectively in a multilingual context and/or in a classic mediation situation” (Council of Europe, 2020, p. 127). (2022, p. 366) It follows, therefore, they conclude, that “plurilingual assessment would be a test of what a test-taker can do to demonstrate their plurilingual comprehension and/or what the test-taker is doing to build on their plurilingual repertoire”. Here are some examples of descriptors on these two scales (users are invited to replace the part underlined with specific languages concerned): Plurilingual comprehension

• B1: Can use what they have understood in one language to understand the topic and main message of a text in another language (e.g. when reading short newspaper articles in different languages on the same theme). • B1: Can recognise similarities and contrasts between the way concepts are expressed in different languages, in order to distinguish between identical uses of the same word/sign and “false friends”.

186  Enrica Piccardo and Brian North Building on plurilingual repertoire

• B1: Can exploit creatively their limited repertoire in different languages in their plurilingual repertoire for everyday contexts, in order to cope with an unexpected situation. • B2: Can make use of different languages in their plurilingual repertoire to encourage other people to use the language in which they feel more comfortable situation. The CEFRCV operationalisation of the concept of plurilingualism is not the only one possible, as CARAP/FREPA and REFIC demonstrate. However, the CEFRCV descriptors for plurilingualism have a clear theoretical framework provided by the definition given in the CEFR Section 1.3 (What is plurilingualism?), clarified in CEFRCV Section 2.3. This definition of plurilingualism is related to other terms for translingual practices by Piccardo and North (2019): In different situations, a person can call flexibly upon different parts of this [plurilingual] competence to achieve effective communication with a particular interlocutor. For instance, partners may … (Council of Europe, 2001, p. 4): (a) switch from one language or dialect (or variety) to another; (this relates to codeswitching, code alternation, flexible bilingualism and translanguaging). (b) express oneself in one language (or dialect, or variety) and understand a person speaking another; (this relates to lingua receptiva and intercomprehension). (c) call upon the knowledge of a number of languages (or dialects, or varieties) to make sense of a text; (this relates to translanguaging as pedagogic scaffolding in a language class, and intercomprehension). (d) recognise words from a common international store in a new guise; (this relates to intercomprehension). (e) mediate between individuals with no common language (or dialect, or variety), even with only a slight knowledge oneself; (this is acting as an intermediary in cross-linguistic mediation). (f) bring the whole of one’s linguistic equipment into play, experimenting with alternative forms of expression in different languages or dialects, exploiting paralinguistics (mime, gesture, facial expression, etc.) and radically simplifying their use of language. (This relates to translanguaging, code crossing, code mixing, code-meshing, polylingualism and metrolingualism). (Piccardo & North, 2019, p. 224) As Cummins (2017) points out, there are situations in which plurilinguals naturally plurilanguage ((f) above), situations in which they alternate between

Plurilingualism and assessment 187 languages while acting as an intermediary for others ((e) above) and situations in which it is expected of them to operate in one language only. They may stretch the norm and mix languages in order to express their voice, but as in poetry or rhetoric, in order to do this effectively, one has to be aware of the norm in question and “transgress” it playfully but consciously. Knowing when to plurilanguage or not, which language to use, being able to perceive “the social etiquette of language choice” (Gumperz, 1964, 138), is a fundamental aspect of the [plurilingual] repertoire. And, as Busch (2017) points out, the emotionally lived experiences accumulated as the repertoire develops helps the social agent to choose how to position themselves in the interaction concerned. (Piccardo et al., 2022a, p. 7) Being able to distinguish between situations in which to plurilanguage and situations in which sticking to a norm is required is crucial from a social justice perspective. Plurilingual pedagogy involves a constant two-step process of making space for linguistic and cultural diversity by encouraging and supporting learners to build on their repertoires and resources, while also building awareness of the social expectations vis-à-vis the use of linguistic and cultural conventions. Some options for assessment of plurilingual competence Let us now turn to some concrete examples. As Shohamy points out, if we “adopt a broad and expanded view of assessment, such as when assessment is used for diagnostic, feedback, and learning purposes […] then any multilingual pedagogical strategy can be used as an assessment procedure” (2011, 427). There are a wide range of possible options; the following are just examples: a Self-assessment of plurilingualism: Galante (2018) used CEFRCV descriptors for plurilingual and pluricultural competence to design ten 30-minute plurilingual tasks and a questionnaire given to the students before and after her 3-month EAP class. b Assessing comprehension of words/sentences in a language similar to one spoken or studied: Jentges et al. (2023) gave such a test before and after a project and field trip concerning similarities between Dutch and German. c Assessing comprehension of a text in a language one does not speak – lingua receptiva (Rehbein et al. (2012) or intercomprehension (see De Carlo & Andrade, this volume). d Assessing writing in two languages “side-by-side”, with one rubric helping teachers judge bilingual strategies and characteristics across languages that reflected the nature of the holistic plurilingual repertoire (Escamilla et al., 2013). e Teacher and self-assessment of plurilanguaging in action-oriented collaborative tasks and scenarios (González-Davies, 2020; Piccardo et al., 2022b).

188  Enrica Piccardo and Brian North The first three examples concern assessment of plurilingualism itself, in the sense of the profile, of exploiting known languages in relation to “unknown” languages. The last two can be used for direct assessment of plurilingual ability, but also for taking heritage languages into account in the assessment of content. Assessment of plurilingual abilities has also been included in both oral and written examinations. In relation to the former, Steinhuber (2022) discusses the 15-minute oral for the Certificate of Plurilingualism offered in Austrian vocational colleges, which assesses fluency at alternating between languages (L1, L2 and L3), following the flow from one language/variety to another. Firstly, there is a phase involving mediating text from L1 into L2 and L3 alternately, and then a second phase of mediating concepts and mediating communication in a collaborative task undertaken with interlocutors speaking the L2 and L3. Assessment criteria consider “task achievement”; “language switch and interaction” (flexibility, spontaneity, initiative, coherence and cohesion, accommodation to interlocutor, ability to mediate), and “range and accuracy”, assessed for each language separately (Piribauer et al., 2019). Thus, the ability to alternate effectively between languages as required is one of three criteria. The assessment of cross-linguistic mediation of a text – summarising or relaying specific information in a different language in speech or writing, one of the core aspects of plurilingualism identified in the CEFR, (Council of Europe, 2020, Section 3.4.1.1) – has been included in national exams in Germany and Greece since the early 2000s. Dendrinos (2013) and Stathopoulou (2015) show a range of task types for CEFR levels B1–C1 and Stathopoulou also analyses the mediation strategies employed by students in them. Crosslinguistic tasks like these mirror activities that “occupy an important place in the normal linguistic functioning of our societies” (Council of Europe, 2001, p. 14). Thus, including plurilingual tasks in examinations is not impossible. Conclusion This article has argued that in order to implement assessment of plurilingualism, a radical change towards assessment for learning is required, so that “teaching and assessment are totally integrated and inseparable. One is embedded in the other and there are no clear and distinct divides” (Shohamy, 2011, p. 427). Well-validated descriptors, such of those of the CEFRCV, offer tools to do this. They suggest aims and tasks, help problems to emerge through diagnosis and provide a starting point to develop assessment criteria. With continuous teacher, peer and self-assessment, tasks like those described above (1–5) can be integrated regularly in the programme for teaching and assessment. Action-oriented collaborative tasks and scenarios ((5) above), in which learners plurilanguage in certain phases, are perhaps the most challenging for teachers to implement. However, experience in the Linguistic and Cultural Diversity Reinvented (LINCDIRE) project (Piccardo et al., 2022b) shows that when teachers are motivated and well supported with clear models and

Plurilingualism and assessment 189 tools, they can embrace a dynamic plurilingual vision in which assessment for learning guided by targeted rubrics based on descriptors, including descriptors for plurilingual/pluricultural competence, becomes key. This can spark a shift in the language class in which assessment is completely embedded in the learning process. Preliminary data from an ongoing international research project with over 80 Italian teachers using the LINCDIRE scenarios and e-portfolio LITE in classes of different languages from primary to upper secondary confirm this shift and highlight the importance of students’ awareness of their plurilingual/pluricultural repertoire and trajectory and their sense of empowerment that derives from leveraging their linguistic and cultural resources as springboards for learning, communicating and feeling part of a community. Formal assessments can also create a positive washback promoting plurilingualism in day-to-day teaching. As experience with teacher education in relation to the Austrian certificate mentioned above shows, when teachers are provided with a website of “plurilingual lessons” (usually action-oriented modules of several lessons), they feel empowered to introduce this approach in preparing students for the exam, especially if plurilingual assessment tasks for prior practice are also provided, as in this case.1 These examples show that a shift towards the assessment of plurilingualism in the language classroom is certainly possible if it goes hand-in-hand with the adoption of a plurilingual pedagogy: a pedagogy which uses a unified metalanguage for languages across the curriculum and recognises that all languages spoken by students are always present in the classroom; one that makes space for heritage languages and encourages learners to consciously exploit all their language repertoire, including when acting as linguistic and cultural mediators with and for peers; and, finally, a pedagogy that brings to the fore and values learners’ plurilingual profiles in learning, teaching and assessment. Note 1 Plurilingual lessons: https://www.cebs.at/home/plurilingualism/plurilingual_ lessons/.   Plurilingual exams, including sample tasks: https://www.cebs.at/home/ plurilingualism/plurilingual-exams/   Documentation: https://www.cebs.at/wp-content/uploads/2019/05/Austrian_ framework_plurilingual_oral_exams-Druckausgabequalit-1.pdf.

References Assessment Reform Group (1999). Assessment for Learning: Beyond the Black Box. Cambridge: University of Cambridge School of Educaion. https://www. researchgate.net/publication/271848977_Assessment_for_Learning_beyond_ the_black_box Bandura, A. (1999). Human agency in social cognitive theory. American Psychologist, 44, 1175–1184. Berger, A. (2020). Specifying progression in academic speaking: A keyword analysis of CEFR-based proficiency descriptors. Language Assessment Quarterly, 17(1), 85–99.

190  Enrica Piccardo and Brian North Biggs, J. (2003) Aligning teaching and assessment to curriculum objectives: Imaginative curriculum project. LTSN imaginative curriculum guide IC022. York: Higher Education Academy. https://administracion.uexternado.edu.co/real/ Recursos/Aligning_teaching_and_assessment_to_curriculum_Biggs_J_2003.pdf Candelier, M., de Pietro, J.-F., Facciol, R., Lorincz, I., Pascual, X., & Schröder-Sura, A.. (2012). CARAP–FREPA: A framework of reference for pluralistic approaches to languages and cultures. Strasbourg: Council of Europe. http://carap.ecml.at/ Chalhoub-Deville, M. B. (2019). Multilingual testing constructs: Theoretical foundations. Language Assessment Quarterly, 16(4–5), 472–480. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press. http:// www.coe.int/en/web/common-european-framework-reference-languages Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume. Strasbourg, France: Council of Europe. https://rm.coe.int/common-european-framework-of-reference-forlanguages-learning-teaching/16809ea0d4. Csikszentmihalyi, M. (1996). Creativity. New York: Harper Perennial/Harper Collins Publishers. Cummins, J. (2017). Teaching minoritized students: Are additive approaches legitimate? Harvard Educational Review, 87(3), 404–425. De Angelis, G. (2011). Teachers’ beliefs about the role of prior language knowledge in learning and how these influence teaching practices. International Journal of Multilingualism, 8(3), 216–234. De Backer, F., Baele, J., Van Avermaet, P., & Slembrouck, S. (2019). Pupils’ perceptions on accommodations in multilingual assessment of science. Language Assessment Quarterly, 16(4–5), 426–446. De Backer, F., Van Avermaet, P., & Slembrouck, S. (2017). Schools as laboratories for exploring multilingual assessment policies and practices. Language and Education, 31(3), 217–230. De Carlo, M., & Anquetil, M. (2019). Un Référentiel de compétences de communication plurilingue en intercompréhension. REFIC [A repository of plurilingual communication skills in intercomprehension. REFIC]. EL.LE, 8(1), 157–228. http:// doi.org/10.30687/ELLE/2280-6792/2019/01/008 Dendrinos, B. (2013). Testing and teaching mediation. Directions in English language teaching, testing and assessment. Athens: RCeL publications. https://rcel2.enl.uoa. gr/directions/issue1_1f.htm Escamilla, K., Hopewell, S., Butvilofsky, S., Sparrow, W., Soltero-Gonzalez, L., RuizFigueroa, O., & Escamilla, M. (2013). Biliteracy from the start: Literacy squared in action. Philadelphia, PA: Caslon Publishing. Fleckenstein, J., Leucht, M., & Köller, O. (2018). Teachers’ judgement accuracy concerning CEFR levels of prospective university students. Language Assessment Quarterly, 15(1), 90–101. https://doi.org/10.1080/15434303.2017.1421956. Flores, N., & Schissel, J. L. (2014). Dynamic bilingualism as the norm: Envisioning a heteroglossic approach to standards-based reform. Tesol Quarterly, 48(3), 454–479. Frost, D., & O’Donnell, J. (2015). Réussite: Être ou ne pas être B2: Telle est la question. (le projet ELLO Étude longitudinale sur la langue orale) [Success: To be B2 or not to be B2 – That is the question. (The Étude longitudinale sur la langue orale (ELLO) project)]. Cahiers de l’Apliut, XXXIV(2). Retrieved from https://journals. openedition.org/apliut/5195.

Plurilingualism and assessment 191 Galante, A. (2018). Plurilingual or monolingual? A mixed methods study investigating plurilingual instruction in an EAP program at a Canadian university (Doctoral dissertation, University of Toronto, Canada). Retrieved from https://hdl.handle. net/1807/91806 Gipps, C. V. (1994). Beyond testing: Towards a theory of educational assessment. London: Falmer Press. González-Davies, M. (2020). Developing mediation competence through translation. In S. Laviosa & M. González Davies (Eds.), The Routledge handbook of translation and education (pp 434–450). London & New York: Routledge. Gorter, D., & Arocena, E. (2020). Teachers’ beliefs about multilingualism in a course on translanguaging. System, 92, 102272. Gorter, D., & Cenoz, J. (2017). Language education policy and multilingual assessment. Language and Education, 31(3), 231–248. Graves, K. (2008). The language curriculum: A social contextual perspective. Language Teaching, 41(2), 147–181. Guzman-Orth, D. A., Lopez, A. A., & Tolentino, F. (2019) Exploring the use of a dual language assessment task to assess young English learners, Language Assessment Quarterly, 16(4–5), 447–463. https://doi.org/10.1080/15434303.2019.1674314. Heugh, K., Prinsloo, C., Makgamatha, M., Diedericks, G., & Winnaar, L. (2017). Multilingualism (s) and system-wide assessment: A southern perspective. Language and Education, 31(3), 197–216. Huempfner, L. (2004). Can one size fit all? The imperfect assumptions of parallel achievement tests for bilingual students, Bilingual Research Journal, 28(3), 379–399. https://doi.org/10.1080/15235882.2004.10162622. Idris, M., & Raof, A. H. (2017). The CEFR rating scale functioning: An empirical study on self- and peer assessments. Sains Humanika, 9(4–2), 11–17. Jentges, S., Knopp, E., & Sars, P. (2023). Dutch for young speakers of German – A workshop on receptive multilingualism through cultural and linguistic landscaping. In B. North, E. Piccardo, T. Goodier, D. Fasoglio, R. Margonis, & B. Rüschoff (Eds.), Enriching 21st century language education: The CEFR companion volume, examples from practice (pp 95–108). Strasbourg: Council of Europe Publishing. Lankina, O. Y., & Pect, Y. V. (2020). Classroom-based assessment of group discussion: Challenges and opportunities. CEFR Journal Research and Practice, 3, 116–125. Little, D., Goullier, F., & Hughes, G. (2011) The European language portfolio: The story so far (1991–2011). Strasbourg: Council of Europe. https://rm.coe.int/ CoERMPublicCommonSearchServices/DisplayDCTMContent?documentId=090 00016804595a7. Lopez, A. A., Turkan, S., & Guzman-Orth, D. (2017a). Conceptualizing the use of translanguaging in initial content assessments for newly arrived emergent bilingual students. ETS Research Report Series, 2017(1), 1–12. https://doi.org/10.1002/ ets2.12140. Lopez, A. A., Turkan, S., & Guzman-Orth, D. (2017b). Assessing multilingual competence. In E. Shohamy, I. G. Or, & S. May (Eds.), Language testing and assessment. Encyclopedia of language and education (pp 91–102). Cham: Springer. New York State Bilingual Common Core Initiative. (2013). NYS bilingual common core initiative: Theoretical foundations. Albany: New York State Department of Education. http://www.engageny.org/sites/default/files/resource/attachments/ nysbcci-theoretical-foundations.pdf.

192  Enrica Piccardo and Brian North Pavlovskaya, I. Y., & Lankina, O. Y. (2019). How new CEFR mediation descriptors can help to assess the discussion skills of management students: Global and analytical scales. CEFR Journal: Research and Practice, 1, 33–40. Piccardo, E. (2013) Assessment recollected in tranquillity: The ECEP project and The key concepts of The CEFR. In E. D. Galaczi & C. J. Weir (Eds.) Exploring language frameworks: Proceedings of the ALTE Krakow Conference, July 2011 (pp 187–204) Studies in Language Testing Series 36. Cambridge: Cambridge University Press. Piccardo, E. (2017) Plurilingualism as a catalyst for creativity in superdiverse societies: A systemic analysis. Frontiers in Psychology, 8, 2169. https://doi.org/10.3389/ fpsyg.2017.02169. Piccardo, E., Lawrence, G., & Germain-Rutherford, A. (2022a). Routledge handbook of plurilingual language education. London and New York: Routledge. Piccardo, E., Lawrence, G., Germain Rutherford, A., & Galante, A. (2022b). Activating linguistic and cultural diversity in the language classroom. New York, NY: Springer International Publishing. Piccardo, E., & North, B. (2019). The action-oriented approach: A dynamic vision of language education. Bristol: Multilingual Matters. Piribauer, G., Atzlesberger, U., Greinix, I., Ladstätter, Mittendorfer, F., Renner, H., & Steinhuber, B. (2019). Plurilingualism: Designing and implementing plurilingual oral exams: Framework for the Austrian upper secondary level oral leaving examination at colleges for higher vocational education. Vienna: CEBS. https://www.cebs. at/wp-content/uploads/2019/05/Austrian_framework_plurilingual_oral_examsDruckausgabequalit-1.pdf. Poehner, M. E. (2008). Dynamic assessment: A Vygotskian approach to understanding and promoting L2 development. Cham: Springer. Rehbein, J., Ten Thije, J., & Verschik, A. (2012). Lingua receptiva (LaRa) – Remarks on the quintessence of receptive multilingualism. International Journal of Bilingualism, 16(3), 248–264. Saville, N. (2019). How can multilingualism be supported through language education in Europe? Language Assessment Quarterly, 16(4–5), 464–471. Saville, N., & Seed, G. (2022). Language assessment in the context of plurilingualism. In E. Piccardo, G. Lawrence, & A. Germain-Rutherford (Eds.), Routledge handbook of plurilingual language education (pp 360–376). London and New York: Routledge. Savski, K. (2019). Putting the plurilingual/pluricultural back into CEFR: Reflecting on policy reform in Thailand and Malaysia. The Journal of Asia TEFL, 16(2), 644–652. Schissel, J. L., Leung, C., & Chalhoub-Deville, M. (2019). The construct of multilingualism in language testing. Language Assessment Quarterly, 16(4–5), 373–378. Schissel, J. L., López-Gopar, M., Leung, C., Morales, J., & Davis, J. R.. (2019) Classroom-based assessments in linguistically diverse communities: A case for collaborative research methodologies. Language Assessment Quarterly, 16(4–5), 393–407. https://doi.org/10.1080/15434303.2019.1678041. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 418–429. Shohamy, E. (2021). Multilingual assessment integrated with language policy: Lessons from research and new initiatives for equality and justice. Keynote presentation given at the ALTE 1st International Digital Symposium, 29 April 2021. Stathopoulou, M. (2015). Cross-language mediation in foreign language teaching and testing. Bristol: Multilingual Matters.

Plurilingualism and assessment 193 Steinhuber, B. (2022). Implementing plurilingual oral exams and plurilingual lessons in Austrian upper secondary vocational colleges. In B. North, E. Piccardo, T. Goodier, D. Fasoglio, R. Margonis, & B. Rüschoff (Eds.), Enriching 21st century language education: The CEFR companion volume, examples from practice (pp 109–116). Strasbourg: Council of Europe Publishing. Thompson, A. S. (2015). Are your participants multilingual? The role of selfassessment in SLA research. Language in Focus, 1(1), 51–65. Trim, J. L. M. (2007) The CEFR in relation to the policy aim of the Council of Europe. In Council of Europe (Ed.), The common European framework of reference for languages (CEFR) and the development of language policies: Challenges and responsibilities. Intergovernmental Language Policy Forum, Strasbourg, 6–8 February 2007, Report (pp 48–49). Strasbourg: Council of Europe. https://rm.coe.int/ CoERMPublicCommonSearchServices/DisplayDCTMContent?documentId=090 000168069b821. Turner, C. E., & Purpura, J. E. (2015). Learning-oriented assessment in second and foreign language classrooms. In D. Tsagari & J. Banerjee (Eds.), Handbook of second language assessment (pp 255–274). Volume 12 of Handbooks of Applied Linguistics. Berlin: De Gruyter. Van Steendam, E., Tillema, M., & Rijlaarsdam, G. (2012). Introduction. In E. Van Steendam, M. Tillema, G. Rijlaarsdam, & H. van den Bergh (Eds.), Measuring writing: Recent insights into theory, methodology and practice (pp i–xxii). Leiden/Boston: Brill. Young, A. S. (2014). Unpacking teachers’ language ideologies: Attitudes, beliefs, and practiced language policies in schools in Alsace, France. Language Awareness, 23 (1–2), 157–171.

12 EVAL-IC An integrated approach to plurilingual competences Elisabetta Bonvino, Filomena Capucho and Margareta Strasser1 Introduction Intercomprehension (IC) has been a dynamic notion since it appeared on the specific domain of language learning and teaching in the 1990s, aiming at developing new perspectives for multilingualism. The need for the recognition of IC studies and their epistemological anchoring within the field of applied linguistics led to significant progress throughout the last 25 years. At the same time, the evolution of supporting linguistic theories, as well as the social and technological changes that have occurred during these decades, has also deeply influenced the way language learning is conceived and practised, paving the way for new perspectives in IC approach to language learning too. Therefore, the several shifts that may be observed in IC conceptions and methodologies could be easily recognised and explained. If the earliest works and applications in this domain were limited to the comprehension of written texts in several languages. IC studies soon also included plurilingual listening skills. Later, with the development of IT supports, synchronous and asynchronous interactions between speakers of several languages became possible – and usual – and practices of IC in interaction became the focus of new studies. Nowadays, with the growing mobility of citizens, the new migration flow, the multinational composition of international or even national firms, and the increasing opportunities in tourism, face-to-face plurilingual interactions are no longer exclusive to small groups in social or professional interactions (Berthoud & Burger, 2014; Berthoud et al., 2013; Davignon, 2008; Di Pardo, 2009; Grin, 2013). IC is thus one of the possibilities to follow if the aim is to maintain cultural and linguistic diversity in a globalised world, and IC studies have also found a particularly rich field to explore there. The progressive changes of IC studies have also caused a conceptual move in the notion itself. If until very recently IC was seen as a partial competence mainly anchored in receptive skills, it is currently “conceived as a multidimensional process that occurs in multilingual and multicultural contexts and supports discursive encounters between plurilingual speakers” (Ollivier et al., 2019, p. 27). It is consequently based on general communicative competence “using (in particular) additional interlinguistic knowledge and highly DOI: 10.4324/9781003177197-15

EVAL-IC 195 activating psychological dimensions” (Ollivier et al., 2019, p. 27) which concern plurilingual interaction as a specific competence. As any distinct competence, IC competence needs specific tools to be assessed. Empirical studies on IC uses and IC testing revealed that for the Common European framework of reference for languages (CEFR; Council of Europe, 2001), descriptors were not adapted to adequately describe plurilingual integrated competences (Capucho, 2014; Carrasco Perea & De Carlo, 2016). In order to solve this problem and ensure that IC may dispose of rigorous tools to sustain the planning of learning activities and the assessment and certification of competences, a specific framework of reference for IC was developed within the European project EVAL-IC (Evaluation des compétences en intercompréhension, 2016-1-FR01-KA203-024155). The aim of this chapter is to present this detailed framework, as well as the holistic approach developed within EVAL-IC to evaluate plurilingual competences, based on a task-based scenario of social interaction. A framework for intercomprehension competence The competence model developed in the project is based on a holistic definition of competence, comprising knowledge (“savoir”), skills (“savoir-faire”), attitudes and values (“savoir-être”). IC competence is thus seen as the ability to make use of these attitudes and values, knowledge and skills in order to engage in multilingual and cross-cultural communication based on (linguistic) proximity (CEFR; OECD, 2018; Rychen & Salganik, 2003; Weinert, 2014). The IC framework of reference was developed following an analysis of existing approaches to describing plurilingual competences: the CEFR, the Framework of reference for pluralistic approaches to languages and cultures (FREPA; Candelier et al., 2012), and the frameworks developed within the MAGICC (Modularising multilingual and multicultural academic communication competence; Räsänen et al., 2013) and Miriadi (Mutualisation et innovation pour un réseau de l’intercompréhension à distance; Projet Miriadi, 2015) projects. This work was complemented by an analysis of the literature in the field of IC competence and a survey among experts conducted as part of a previous project, REDINTER (143339-2008-PT-KA2NW).2 As a result of this analysis, the following dimensions of IC competence were identified (Project EVAL-IC, 2019):

• Linguistic dimension • Paraverbal and nonverbal dimension • Socio-pragmatic dimension: communicative conventions, non-linguistic factors and knowledge, textual and pragmatic factors

• (Inter-)Personal dimension: personality-related factors, values and attitudes (e.g. sensitivity, curiosity, acceptance, openness, motivation, confidence, openness to the diversity of languages, people or cultures of the world; to diversity as such)

196  Elisabetta Bonvino, Filomena Capucho and Margareta Strasser

• (Inter-)Cultural dimension: knowledge of the world, of cultures and of cultural discourse conventions

• Cognitive dimension: ability to compare and relate different languages and cultures; ability to organise one’s own learning process

• Metacognitive and metalinguistic dimension declarative or procedural knowledge about one’s own competencies and strategies

• Strategic dimension

Individual dimensions can be further divided into more specific subdimensions. The linguistic dimension of receptive IC competence comprises, for instance, of an orthographic/graphemic (written receptive IC) or graphematic/ phonological (oral receptive IC), a lexical and a morphosyntactic subdimension (written and oral receptive IC). The strategic dimension is to be seen as a transverse/key competence (Weinert, 2001) in the concept of IC, as every other dimension includes a strategic component. IC is seen from a holistic perspective, considering IC as a form of plurilingual and pluricultural communication in which the communication partners rely on (potentially partial) competencies in different languages as well as on other specific competencies (e.g. intercultural/metacognitive competencies). Receptive IC and interproduction are partial aspects of this form of communication, also referred to as “interactive IC” (Ollivier & Strasser, 2013) or “intercommunication” (Balboni, 2007; Ollivier, 2019). Receptive IC thus denotes reception of oral and written discourse in languages in which one may not be able to express oneself and in which one may have only partial comprehension skills. Interproduction (Balboni, 2007; Capucho, 2018; Hédiard, 2009) refers to oral and written production in a language or languages in which one has sufficient competencies to express oneself, using specific additional skills to adapt their discourse to recipients who have not necessarily learned the respective language(s). This means, for example, that a written or spoken Italian text can be understood by recipients with knowledge of other Romance languages (Ollivier et al., 2020). Depending on language activity, the various dimensions of IC competence are activated to different extents. In receptive IC, i.e. reading and listening comprehension in usually genetically related and/or geographically close languages, the linguistic dimension plays a central role, whereas strategies of linguistic adaptation as well as paraverbal and nonverbal strategies are of particular importance in interproduction. In accordance with the competence descriptions of the CEFR, the framework for IC competencies developed within the project EVAL-IC describes six competence levels for each IC activity (written/oral reception, written/oral interproduction, written/oral interaction). These six levels can be regrouped into three broader levels: basic (levels 1 and 2), advanced (levels 3 and 4) and expert (levels 5 and 6). At basic level, language users focus on selected strategies and dimensions: they understand in a fragmentary and approximative way, strategies rely

EVAL-IC 197 mainly on the lexical dimension. Interproduction strategies concentrate on transparent words and nonverbal elements. In interactive IC, co-construction of meaning is based on selected, mainly nonverbal and paraverbal strategies. At expert level, language users implement a wide variety of strategies across all dimensions of IC competence, which they use flexibly and according to the requirements of the situation. Interaction partners at level 6, for example, can efficiently adapt their discourse to the plurilingual and pluricultural repertoires and to the socio-affective and cognitive characteristics of their interlocutors. They easily grasp what others are saying, actively and flexibly contribute to establishing a fluent and varied conversation and effectively avoid and resolve conflicts in the co-construction of meaning (Garbarino & Melo-Pfeifer, 2020; Project EVAL-IC, 2019). These six competence levels of IC are defined through “can-do” descriptors for each language activity. The following example illustrates the description of the nonverbal/paraverbal dimension for written receptive IC competence. Using nonverbal/paraverbal factors: 1 Identifies nonverbal cues (illustrations, pictures, photos, symbols, etc.) but does not use them for comprehension 2 Identifies nonverbal cues (illustrations, pictures, photos, symbols, etc.) to understand; 3 Constructs some units of meaning from nonverbal indicators (illustrations, pictures, photos, symbols, etc.) 4 Uses nonverbal cues (illustrations, pictures, photos, symbols, etc.) to construct meaning 5 Systematically uses nonverbal cues (illustrations, pictures, photos, symbols, etc.) to construct meaning and to bridge gaps in understanding 6 Systematically mobilises nonverbal cues (illustrations, pictures, photos, symbols, etc.) for full comprehension according to their reading intention (Project EVAL-IC, 2019) Not all dimensions of IC competencies are developed in terms of descriptors. Aspects of attitudes and values, for example, were explicitly integrated in other dimensions of the model, since factors like sensitivity, curiosity, acceptance, motivation and openness to languages and cultures play a crucial role in developing IC competencies. The descriptors for receptive IC define this dimension at two levels (“discovers the differences and similarities between different languages and cultures” vs. “is fully aware (both) of the differences and similarities between different languages or cultures”). However, other dimensions are excluded from evaluation. For interactive oral IC, for example, the model only includes descriptors for the linguistic, paraverbal and nonverbal, the intercultural and the pragmatic dimension. In this case, it is primarily a matter of linguistic adaptations to the interactional partner at the lexical and textual level, and the mutual negotiation of meaning (Project EVAL-IC, 2019).

198  Elisabetta Bonvino, Filomena Capucho and Margareta Strasser As already mentioned, in order to measure the competences described so far, the EVAL-IC team decided to use scenario-based assessment. Scenario-based assessment IC scholars have repeatedly highlighted the need for a system of assessment and certification of plurilingual competences (see, among others, Bonvino, 2015; Bonvino & Jamet, 2016; Jamet, 2010), which is crucial for institutional and social recognition, as well as for the valorisation of educational paths based on plural approaches in general and particularly on IC. The purpose of assessment is usually described in terms of the “construct” and, in the case of language learning, it is related to one’s conception of language, language learning and language teaching or other characteristics. The construct is an abstract theoretical concept, which can be defined from a number of perspectives, and must be explicitly described and specified in the assessment design, in order to define a basis to develop language assessments, as well as to interpret and use their results (Bachman & Palmer, 2010). The construct of plurilingual assessment adopted within the EVAL-IC project consists of a series of tasks linked by a plurilingual scenario-based macrosituation. These tasks require a series of “can do” that constitute the object of the assessment (cf. Lenz & Berthele, 2010). Thus, it is closely related to the concept of plurilingual and pluricultural competence described in the framework mentioned above. As can be seen, the project highlighted the need to distinguish between receptive and interactive IC and interproduction competencies. In the EVAL-IC test, the distinction of these three activities is purely instrumental and operational; it is aimed at identifying observable skills and should not suggest a compartmentalised conception of IC competence. Language and IC in the EVAL-IC assessment are considered as an integrated, holistic, plurilingual and communicative construct (see Ollivier et al., 2019). The various languages considered are not assessed in isolation; rather, an attempt is made to provide a comprehensive assessment of plurilingual repertoire management. This concept is reflected into the type of test chosen, which is scenariobased (Purpura, 2016). The proposed scenario corresponds to a complex task, articulated in sub-tasks, close to the users’ experience, and requires the activation of a plurilingual and pluricultural repertoire. The activities are therefore independent from each other, although linked by a macro-situation, and allow participants to experience authentic communicative situations in which they are required to use several Romance languages. Task-based language assessment is an assessment practice in which the construct is entirely based on the performance of specific communicative tasks. In this approach, the construct is not based on language as an object, intended as declarative language knowledge, but on the concept of task as a unit of analysis, and of performance as a means of testing learners’ ability to complete the

EVAL-IC 199 task (Brown et al., 2002; Norris et al., 2002). Although numerous definitions of task in the literature (cf. Ellis, 2003; Long, 2015; Nunan, 2004) emphasise different aspects, they all agree in describing the task as an activity that requires learners to achieve an extra-linguistic goal through the use of their language resources, focusing on meaning and not on form (Cortés Velásquez et al., 2018). Tasks replicate real-life situations, contents, interlocutors and other aspects and aim at gathering reliable indications of the learners’ ability to handle authentic situations. Thus, they have many advantages in both formative and summative assessment. One of the questions still unanswered in the literature on task-based language assessment is the way in which the performance elicited by the task is to be evaluated. To this regard, a “strong” and a “weak” position can be distinguished. In the first one, language assessment focuses on the ability to complete the task, and the performance aims at verifying the achievement of this objective (Long, 2015). On the other hand, in the “weak” position, the task is a tool to elicit a performance assessed according to certain criteria (TimpeLaughlin, 2018). The EVAL-IC test, being a test based on plurilingual competence in IC, does not include an assessment of the language performance in terms of communicative effectiveness or morphosyntactic and lexical accuracy. Subsequently, it adheres to the concept of “strong” task assessment, where the achievement of the task is the main aspect to be assessed. The scenario The Erasmus+ EVAL-IC project proposes a system for the assessment of plurilingual competences in IC and aims to provide university students with a certificate in IC. Starting from the assessment protocol elaborated within the project, an assessment tool was developed and tested with the aim of formally recognising the communicative competences and language skills acquired in any context (i.e. not necessarily as a result of a course), in relation to IC among the five Romance languages of the project (Portuguese, Spanish, Italian, French and Romanian). In the proposed scenario, students have to imagine applying for an international conference on sustainable development held in five Romance languages, involving some European universities. Applicants are first required to gather information about the subject by consulting a series of written and oral documents sent by the organisers and then to present valid measures to achieving the sustainable development goals in their own living and study contexts. The application involves the following stages, which correspond to the same number of tests: 1 Applying for the conference by filling in a plurilingual registration form 2 Gathering information on sustainable development by reading texts in the five languages of the conference

200  Elisabetta Bonvino, Filomena Capucho and Margareta Strasser 3 Gathering information on sustainable development from video documents in the five languages of the conference 4 Preparing a Power Point presentation entitled “Ideas and proposals for sustainable development in European universities”, in the candidate’s preferred Romance language (very often their L1) 5 Presenting the Power Point presentation and discussing it (again in the candidate’s chosen language) with a panel of five people each speaking a different Romance language The topic illustrated above is that of the first scenario proposed and tested within the project. The chosen topic is only one possibility, and it is envisaged that the scenario will be developed in future tests based on other current hot topics. Conclusion Throughout the last decades, research in IC and its subsequent applications in pedagogical situations led to a change of paradigm: from the conception of IC as a partial competence, researchers moved to facing it as a specific plurilingual communicative competence that mobilises the entire linguistic and cultural repertoire of the speakers (see, among others, De Carlo & Anquetil, 2019). This integrated view of a plurilingual competence demanded a different approach for the description of all the dimensions that compose it. The Project EVAL-IC aimed at creating a comprehensive set of descriptors that may help understand the communicative processes that support IC and describe progression in this competence. The work produced may help the structuring of specific IC courses and support institutional certification of competences. For this aim, EVAL-IC also proposed an evaluation protocol and a set of task-based assessment tools that take the form of a progressive, coherent communicative scenario. The pilot scenario has been tested and the analysis of results allowed its authors to identify some problematic areas concerning the clarity of formulation of the tasks, the specific assessment criteria and a balanced management of the time devoted to each component. Moreover, the challenges presented by the COVID-19 pandemic – and the inevitable changes it will bring in the future of educational practices – demand adaptations to a full or partial online testing environment. The new version of the scenario that is currently being produced will focus the specific reformulations that are considered necessary to increase its applicability and validity. Data resulting from the use of this reformulated tool will provide a new ground for detailed analysis and subsequent improvement. This permanent effort on quality control is essential to implement large institutional certification of IC competence, which is considered as an important requisite for the dissemination of plurilingual learning possibilities, and the academic and educational recognition of the IC approach.

EVAL-IC 201 Notes 1 All authors have contributed equally to the work. Contributions: E. Bonvino: Scenario-based assessment, The scenario; F. Capucho: Introduction and conclusion; M. Strasser: A framework for intercomprehension competence. 2 For a description of the methodology used, see Strasser and Hülsmann (2020) and Ollivier et al. (2020).

References Bachman, L. F., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press. Balboni, P. (2007). Dall’intercomprensione all’intercomunicazione romanza. In F. Capucho, A. Martins, P. Alves, C. Degache, & M. Tost (Eds.), Diálogos em intercompreensão: Actas do colóquio, Lisboa, Setembro de 2007 (pp 447–459). Universidade católica editora. https://core.ac.uk/download/pdf/223150526.pdf Berthoud, A.-C., & Burger, M. (Eds.). (2014). Repenser le rôle des pratiques langagières dans la constitution des espaces sociaux contemporains. Bruxelles: De Boeck. Berthoud, A.-C., Grin, F., & Lüdi, G. (2013). Exploring the dynamics of multilingualism: The Dylan project. Amsterdam: John Benjamins. Bonvino, E. (2015). Intercomprehension studies in Europe: History, current methodology and future developments. In R. Dolci & A. J. Tamburri (Eds.), Intercomprehension and multilingualism: An asset for Italian language in the USA (pp 29–59). New York: John D. Calandra Institute. Bonvino, E., & Jamet, M.-C. (Eds.). (2016). Intercomprensione: Lingue, processi e percorsi. Venezia: Edizioni Ca’ Foscari. Brown, J. D., Hudson, T. D., Norris, J. M., & Bonk, W. (2002). An investigation of second language task-based performance assessments. Honolulu: University of Hawaii Press. Candelier, M., Camilleri-Grima, A., Castellotti, V., De Pietro, J.-F., Lörincz, I., Meißner, F.-J., Noguerol, A., & Schröder-Sura, A. (2012). FREPA. A framework of reference for pluralistic approaches to languages and cultures: Competences and resources. Council of Europe, European Centre for Modern Languages. https://www. ecml.at/Portals/1/documents/ECMl-resources/CARAP-EN.pdf. Capucho, F. (2014). Intermar ou les nouveaux défis de l’intercompréhension. In R. Daval, E. Hilgert, T. Nicklas, & D. Thomières (Eds.), Sens, forms, langage. Contributions en l’honneur de Pierre Frath (pp 345–358). Reims: Epure. Capucho, F. (2018). Plurilingual interactions: The role of interproduction strategies. In M. Hepp & M. Nied Curcio (Eds.), Educazione plurilingue: Ricerca, didattica e politiche linguistiche = Bildung zur Mehrsprachigkeit: Forschung, Didaktik und Sprachpolitik = Plurilingual education: Research, teaching and language policies. Roma: Istituto italiano di studi germanici. Carrasco Perea, E., & De Carlo, M. (2016). Evaluer en intercompréhension ou oser le paradigme plurilingue. In E. Bonvino & M. C. Jamet (Eds.), Intercomprensione: lingue, processi e percorsi (pp 183–204). Venezia: Edizioni Ca’ Foscari. Cortés Velásquez, D., Nuzzo, E., & Nuzzo, E. (2018). La didattica con i task: Principi, applicazioni, prospettive. In D. Cortés Velásquez (Ed.), Il task nell’insegnamento delle lingue: Percorsi tra ricerca e didattica al CLA di Roma Tre (pp 13–39). Roma: Roma TrE-Press.

202  Elisabetta Bonvino, Filomena Capucho and Margareta Strasser Council of Europe (Ed.). (2001). Common European framework of reference for languages: Learning, teaching, assessment. Council of Europe Publishing. https:// rm.coe.int/16802fc1bf. Davignon, E. (Ed.). (2008). Languages mean business – Companies work better with languages: Recommendations from the business forum for multilingualism established by the European Commission. DG Education and Culture. https://op.europa.eu/ en/publication-detail/-/publication/d5ee6ef0-986c-49e6-b8ac-71da8401efc6. De Carlo, M., & Anquetil, M. (2019). Un référentiel de compétences de communication plurilingue en intercompréhension. REFIC. EL.LE Educazione linguisticalinguistica Educativa, 8(1), 157–228. Di Pardo Léon-Henri, D. (2009). Plurilingual business communication strategies for the European economy. In Actes des 2èmes assises européennes du plurilinguisme, organisées les 18-19 juin 2009 à Berlin/Genshagen. Bookelis. http://www.observa toireplurilinguisme.eu/index.php?option=com_content&view=article&id=2207:pr ogramme-et-contributions&catid=88888966&Itemid=178379078&lang=fr. Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press. Garbarino, S., & Melo-Pfeifer, S. (2020). Décrire et évaluer les compétences en intercompréhension: Du référentiel de compétences REFIC (MIRIADI) aux descripteurs de compétences en intercompréhension (EVAL-IC). In C. Hülsmann, C. Ollivier, & M. Strasser (Eds.), Lehr- und Lernkompetenzen für die Interkomprehension: Perspektiven für die mehrsprachige Bildung (pp 103–124). Münster/New York: Waxmann. Grin, F. (2013). Plurilinguisme et multilinguisme au travail: Le regard de l’économie des langues. Repères DoRiF, 4 – Quel plurilinguisme pour quel environnement professionnel multilingue? = Quale plurilinguismo per quale ambito lavorativo multilingue? https://www.dorif.it/reperes/francois-grin-plurilinguisme-et-multilinguisme-autravail-le-regard-de-leconomie-des-langues/. Hédiard, M. (2009). De l’intercompréhension à l’interproduction: Impact des usages langagiers en langue maternelle. In M. H. Araújo e Sá, R. Hidalgo Downing, S. Melo-Pfeifer, A. Séré, & C. Vela Delfa (Eds.), A intercompreensão em línguas românicas: Conceitos, práticas, formação (pp 213–223). Oficina Digital. www.galanet.eu/ publication/fichiers/Araujo_e_Sa_et_al_Ed_2009.pdf. Jamet, M. (2010). Intercomprensione, quadro comune europeo di riferimento per le lingue, quadro di riferimento per gli approcci plurilingui e valutazione. Synergies Europe, 5, 75–98. Lenz, P., & Berthele, R. (2010). Prise en compte des compétences plurilingue et interculturelle dans l’évaluation. Conseil de l’Europe. https://rm.coe.int/CoERMPublicCommonSearchServices/DisplayDCTMContent?documentId=09000016805 a1e56. Long, M. H. (2015). Second language acquisition and task-based language teaching. Malden, MA: John Wiley & Sons. Norris, J. M., Brown, J. D., Hudson, T. D., & Bonk, W. (2002). Examinee abilities and task difficulty in task-based L2 performance assessment. Language Testing, 19(4), 395–418. Nunan, D. (2004). Task-based language teaching. Cambridge: Cambridge University Press. OECD. (2018). Global competency for an inclusive world. https://www.oecd.org/ education/Global-competency-for-an-inclusive-world.pdf

EVAL-IC 203 Ollivier, C. (2019). Interkommunikation. In C. Fäcke & F.-J. Meißner (Eds.), Handbuch Mehrsprachigkeits- und Mehrkulturalitätsdidaktik (pp 292–294). Tübingen: Narr Francke Attempto. Ollivier, C., & Strasser, M. (2013). Interkomprehension in Theorie und Praxis. Wien: Praesens. Ollivier, C., Capucho, F., & Araújo e Sá, M. H. (2019). Defining IC competencies as prerequisites for their assessment. Rivista di Psicolinguistica Applicata/Journal of Applied Psycholinguistics, 19(2), 15–30. Ollivier, C., Capucho, F., & Araújo e Sá, M. H. (2020). Les compétences en interaction plurilingue: Trois dimensions saillantes. In C. Hülsmann, C. Ollivier, & M. Strasser (Eds.), Lehr- und Lernkompetenzen für die Interkomprehension: Perspektiven für die mehrsprachige Bildung (pp 69–85). Münster/New York: Waxmann. Project EVAL-IC. (2019). EVAL-IC: produits. https://evalic.eu/productions/ produits/. Projet Miriadi. (2015). Refic. Miriadi – Mutualisation et innovation pour un réseau de l’intercompréhension à distance. https://www.miriadi.net/refic. Purpura, J. E. (2016). Second and foreign language assessment. The Modern Language Journal, 100(S1), 190–208. Räsänen, A., Natri, T., & Forster Vosicki, B. (2013). MAGICC conceptual framework: Modularising multilingual and multicultural academic communication competence for BA and MA level. https://www.unil.ch/magicc/en/home.html Rychen, D. S., & Salganik, L. H. (2003). A holistic model of competence. In D. S. Rychen & L. H. Salganik (Eds.), Key competencies: For a successful life and a wellfunctioning society (pp 41–62). Göttingen: Hogrefe & Huber. Strasser, M., & Hülsmann, C. (2020). Kompetenzen für die rezeptive Interkomprehension. In C. Hülsmann, C. Ollivier, & M. Strasser (Eds.), Lehr- und Lernkompetenzen für die Interkomprehension. Perspektiven für die mehrsprachige Bildung. Münster/New York: Waxmann. Timpe-Laughlin, V. (2018). Pragmatics in task-based language assessment. In N. Taguchi & Y. Kim (Eds.), Task-based approaches to teaching and assessing pragmatics (pp 288–304). Amsterdam: John Benjamins. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. S. Rychen, & L. H. Salganik (Eds.), Defining and selecting key competencies (pp 45–65). Göttingen: Hogrefe & Huber. Weinert, F. E. (2014). Vergleichende Leistungsmessung in Schulen: Eine umstrittene Selbstverständlichkeit. In Leistungsmessungen in Schulen (3rd ed., pp 17–31). Weinheim/Basel: Beltz.

13 Towards an assessment of intercomprehension competences in coherence with plurilingual approaches Maddalena De Carlo and Ana Isabel Andrade Introduction The notion of plurilingual and pluricultural competence introduced into language teaching by the Common European Framework of Reference (CEFR) (2001) constituted a major change of perspective in language teaching, opening up to a holistic concept of plurilingual competence (Herdina & Jessner, 2002). Since then, in the last two decades, language education studies have shifted attention from the development of a (native speaker-like) linguistic competence to the recognition and building of a plurilingual and pluricultural competence (Beacco & Byram, 2007; Candelier, 2008; CEFR, Companion volume, 2018; Candelier et al., 2012), “This is made possible by diversifying the languages taught and coordinating teaching of the various languages, which are often regarded as separate subjects” (Beacco & Byram, 2007, p. 67). The key principles of this shift involve

• going beyond a monolingual perspective and adopting a plurilingual perspective in language education;

• in any action aiming at linguistic development, recognizing the role of the first language and of all languages in the learners’ repertoires;

• taking into account learners’ linguistic, cultural, cognitive, emotional resources.

These principles converge on a perspective that considers any language as “a fundamental component of the intellectual and emotional development of the individual” (De Mauro, 2009), regardless of its social prestige. Despite the fact that 20 years have passed since the first edition of the CEFR, and despite the growing attention towards pluralistic teaching approaches (Cognigni, 2020), studies on the evaluation of plurilingual competence remain quite rare: “the contrast between the expanding use of multilingual practices in pedagogy, and the absence of multilingual approaches in assessment and evaluation measures is striking” (Schissel et al., 2021: 343). It is therefore necessary to conceive criteria and evaluation tools consistent with the recent didactic orientations that are gradually, albeit slowly, being DOI: 10.4324/9781003177197-16

Towards an assessment of intercomprehension competences 205 introduced into school curricula (see Candelier & De Carlo, 2018). This change would have the advantages of recognizing the value of communication practices widespread in society and of the competences already possessed by learners, of making the learning-teaching-evaluation process homogeneous and coherent, and it would significantly contribute to the recognition of pluralistic approaches (FREPA/CARAP, 2007) in institutional education systems. We can consider the institutionalization of a certain body of knowledge, through its teaching, as a real action of educational and linguistic policy (Grin, 2005). Thus, teaching a language contributes to its diffusion and its cultural, political, and social legitimacy. In a retroactive process, this dissemination in turn stimulates its learning and reinforces its prestige. Therefore, teaching this or that language – in institutional contexts or otherwise – is never a neutral choice or an action without consequences. This phenomenon is further reinforced by assessment methods that focus on specific languages or on specific knowledge. As a matter of fact, when knowledge and skills of any kind are subjected to an evaluation process – or even more, to a certification – they automatically acquire a social legitimacy. As noted by Shohamy (2001, 2007, 2011), assessment and certification tests can modify the behaviour of teachers and students by pushing them to overestimate some content over others. The power of tests goes far beyond the discourse expressed in official documents regarding the educational objectives pursued: for example, in the face of general orientations in favour of plurilingualism and the enhancement of all the learners’ linguistic repertoires, assessment tests can make some languages prevail over others, thus constituting the real hidden agenda of educational policies. Making sure that evaluation processes and criteria are meaningful for the life of learners (e.g. by focusing on competences necessary for their fulfilment as individuals, for the development of their cognitive abilities or for their integration into society and the professional world) is therefore an ethical question that the stakeholders involved in educational contexts (teachers, policy makers, administrative managers, publishers, etc.) cannot ignore. In order to obtain a stronger institutional recognition of pluralistic approaches, researchers on the intercomprehension approach have attempted to define evaluation criteria and tools to assess effective proficiency gains and intercultural learning through intercomprehension. These have been the main objectives of the two frameworks conceived within the MIRIADI project,1 the REFIC (Référentiel de compétences de communication plurilingue en intercompréhension) (De Carlo & Anquetil, 2019) and the REFDIC (Référentiel de compétences en didactique de l’intercompréhension)2 (Andrade et al., 2019) and of the project EVAL-IC (Évaluation des compétences en intercompréhension: réception et interactions plurilingues).3 These two projects aimed to provide specific descriptors for competences in intercomprehension, enabling the assessment of learning results. Our contribution will focus on the intercomprehension approach, considered here as one of the modalities adopted in multilingual communication,

206  Maddalena De Carlo and Ana Isabel Andrade which has also proved to be a powerful catalyst for plurilingual education seeking to develop the linguistic, communicative, and cultural repertoire of the individual. In particular, we shall describe some possible modalities for assessing learners’ plurilingual repertoires, while also taking into account the discourse produced by the learners themselves in the context of tests for the evaluation of intercomprehension. Assessing competences in intercomprehension One of the first exploratory studies on the evaluation of learning achievements in intercomprehension (Carrasco Perea & De Carlo, 2016) revealed a variety of practices4 which can be categorized and described on the basis of criteria that emerged from the analysis: type of evaluation (summative, formative), object of the assessment (linguistic-communicative activity, different types of text, different linguistic skills concerning one or more of the languages treated), and modalities (self-assessment, hetero-assessment, an articulation between the two modalities, logbooks, portfolios). The differences in the choices made in assessment modalities can be interpreted according to the specific teaching contexts. Nonetheless, the need remained to develop evaluation criteria and methods consistent with the constitutive principles of the intercomprehension approach. These principles concern the development of: (i) cognitive abilities allowing transfer of knowledge and inter-linguistic comparison, (ii) intercultural competences, and (iii) metacognitive competences that imply a change of attitude by the learners towards the languages studied and their own learning process. The evaluation construct therefore has to concern not only competences acquired in one or more target languages, but transversal knowledge and know-how (Lenz & Berthele, 2010, p. 6) which can mobilize several languages simultaneously in order to interact or play a mediating role in multilingual contexts, to understand texts in different languages, and to exploit one’s plurilingual profile to carry out communication activities in a named language. Reflections on the criteria and modalities of assessment in a plurilingual perspective also modify the criteria for learning progression. In this regard, it is interesting to recall that research into language acquisition (Giacalone Ramat, 1986; Pienemann, 1998) has highlighted the fact that the ease-difficulty criteria relating to the content of language learning are very random. This type of research has especially shown that the early stages of interlinguistic development are based on universal principles, functioning mainly in pragmatic and semantic modes and relatively independent of the learners’ language of origin. Other variables also come into play, diversifying from individual to individual and depending on the interaction of several factors: linguistic (quantity and quality of input), psychological (motivation), social (integration into a community of speakers), educational (access to guided learning), affective, etc. While it is true that these studies focus more on the analysis of learners’ interlanguage rather than on comprehension processes, they show the difficulty of constructing curricula and assessment criteria that meet social needs in terms of knowledge and skills while respecting learners’ cognitive processes.5

Towards an assessment of intercomprehension competences 207 Research in intercomprehension has highlighted a completely new way of working in relation to the progression expected in the comprehension and mastery of a target language. Capucho (2014, p. 347) proposes in a way to reverse the progression proposed by the Common European Framework of Reference for Languages (2001) because, according to her, reception descriptors (written or oral) are not adapted to intercomprehension tasks if the progression in reception goes from simple to complex, from the word or sentence level to the textual level. In intercomprehension tasks, it is perfectly possible to understand the global meaning of a complex text (especially a text in areas dominated by learners) without understanding details; it is also possible not to understand a simple message if it is not framed by iconic or situational material. Other authors (Campodonico & Janin, 2018), on the other hand, consider relevant the complexity of the task in which the learner is involved. Thus, reading comprehension would be at the lowest level of difficulty, while oral reception and interaction may present great difficulties. Although empirical research6 seems to confirm this view in relation to reading, establishing whether listening comprehension is easier or less easy in an interactive activity strictly depends on the different factors involved: absence/presence of feedback, possibility/impossibility of negotiating meaning, greater or lesser influence of the communicative and psychological aspects involved. Besides, the number of languages mobilized may also have some relevance, since if on the one hand more languages may cause more difficulties, on the other hand, given the assumptions of intercomprehension approach, the wealth of linguistic material may constitute a resource to be exploited through the suppletive phenomenon, allowing greater and easier interlinguistic circulation. In view of this diversity of assessment practices, although derived from a rather modest and by no means exhaustive sample and faced with the observation that there is a central need in the field of pluralistic approaches for assessment and certification tools – a team of researchers in intercomprehension launched the European project EVAL-IC in 2016 (https://evalic.eu/ leprojet/). Its main goal was to address this need by developing, based on already established descriptors and empirical research, tools for the assessment of plurilingual receptive and interaction skills, based largely on the proximity between the languages involved. In the following paragraphs, we shall analyse how the assessment scenario designed in the EVAL-IC project impacted the subjects who participated in the experimental sessions. Analysing the potential of an assessment scenario on plurilingual repertoires Assessment scenario

In general, we can say that the assessment scenario that was designed, developed, and implemented had a positive impact on the subjects who volunteered to be assessed in tests of intercomprehension in Romance languages in the

208  Maddalena De Carlo and Ana Isabel Andrade framework of the EVAL-IC project. This positive effect was already pointed out by Fiorenza and Diego-Hernández (2020) in a study on the perceptions of 58 students from an Italian university about their participation in those same tests. In the case study presented here, we analysed the impact of participation in the EVAL-IC assessment scenario on the performance of 12 students from Education courses (Bachelor, Master and PhD) at the University of Aveiro7 who volunteered to take the tests. More specifically, we intend to present in this text the data obtained from the discourse analysis of the individuals who participated in the EVALIC-IC evaluation scenario, in order to identify a possible development of multilingual competences thanks to the intercomprehension approach when used in communicative situations with languages of the same family. Before moving to the results of the data analysis (the responses of the assessed subjects to the tests carried out), it is important to describe in more concrete terms the evaluation scenario constructed for the application of the intercomprehension tests that sought to assess in five Romance languages (Spanish, French, Italian, Portuguese, and Romanian): the comprehension of written and audio-visual documents; and the interaction with speakers of Romance languages different from the language mastered by the assessed subject, analysing the ability to understand the interlocutor(s) and to make oneself understood by them, adapting one’s oral and written expression to the interlocutors by producing and presenting a slideshow in the language chosen for expression. The aim was to construct an assessment scenario as a learning scenario containing real communicative tasks in order to recognize the value of plurilingual abilities and plurilingual repertoire (Seed, 2020). The assessment scenario that was created sought to take into account the characteristics that Gorter and Cenoz (2017) present for an assessment that favours a multi- and plurilingual approach to language teaching and learning, allowing the mobilization of different languages of the subject’s global repertoire, namely the switching of Romance language and the mixing of languages for discursive production, that is seeking not to assess the isolated mastery of the different languages. In line with this, the tests of the EVAL-IC project sought to create a realistic context capable of providing rich experiences of contact with meaningful discourse – participation in an international university conference – by mobilizing a set of multimodal resources that asked the assessed subjects to read, think, write, produce, and present academic discourse in different languages on a topic of common interest (in this case sustainable development) (cf. Fiorenza & Diego-Hernández, 2020.). Within this framework, the assessed subjects were asked to read written texts and listen to interviews presented in video format on the topic, being able to choose the language or languages for gathering information and taking a position on the concepts and the topic in focus in the tasks to prepare their participation in the conference. The assessed subjects had then to produce texts, such as answering forms, writing notes on important ideas about the topic, and producing a PowerPoint to present at the conference. The latter had to be carried out in their dominant or preferred language and presented by calling upon

Towards an assessment of intercomprehension competences 209 different languages at their disposal in order to understand and answer questions posed by their interlocutors in different Romance languages. For a better understanding of what has just been described, we show the instructions that contextualize the environment in which the assessment takes place:

The test consists of 5 parts, organised around the following scenario - You are applying to participate in an international conference to be held in five Romance languages. Instructions are available in 5 languages (ES-FR-IT-PT-RO). Choose your preferred language, the language in which you will have to express yourself and give your answers. The assessment in intercomprehension will cover all the above-mentioned languages.

Results analysis

For the analysis of results on the discourse of individuals participating in EVALIC-IC assessment scenario, we chose to analyse the results of the tests carried out by the subjects who took them in a Portuguese context, specifically, students of Education courses, attending one of the Portuguese Higher Education institutions that participated in the project. For the selection of participants in the EVALIC-IC tests, an email was sent to all students of Education courses at the University of Aveiro, Portugal, asking for volunteers to participate in the tests assessing competences in intercomprehension in Romance languages. Twelve students answered positively and the EVALI-IC team of the Portuguese institution sent an email to the volunteers to arrange a place and a date (one morning in July) to perform the tests, having previously constituted a group of evaluators, two by each Romance language. The assessment lasted about three hours, with the participants undergoing five tests organized around the following scenario: You are applying to participate in an international conference to be held in five Romance languages, and the instructions are available in five languages (ES-FR-IT-PT-RO). Choose your preferred language, the language in which you will express yourself and will give your answers. The assessment in intercomprehension will cover all the Romance languages mentioned above. Table 13.1 shows the sequence of tests the subject has taken, their duration, the type of test taken, and the competence assessed. The data analysed here are the responses of the 12 volunteer students to the multilingual questionnaire (using the different Romance languages) of general characterization on their linguistic-communicative identity, their academic pathways and relationship with languages and simulating their request

210  Maddalena De Carlo and Ana Isabel Andrade Table 13.1  Sequence of tests taken by the subject, duration, type of test taken and competence assessed Tests

Duration Task

Competence

Test 1

15 min

Written intercomprehension

Test 2

30 min

Test 3

60 min

Test 4

45 min

Test 5

20 min

Completing a multilingual questionnaire Reading five texts in the different conference languages Watching ten videos on the conference theme Preparing a slideshow on the conference theme (Powerpoint, Libre Office Impress ...) Oral presentation of the slideshow and discussion with a jury of five people, each one speaking a different Romance language

Written intercomprehension Oral intercomprehension Written interproduction Oral interproduction

for accommodation and food for the participation in a European conference on sustainability. As the table shows the answers to the questionnaire were followed by comprehension and production tests of the 12 subjects who participated in this experimental protocol with the different Romance languages. The participants in the experiment were informed that their competences in intercomprehension would be evaluated according to one of three levels – basic, advanced, or expert. The criteria used and explained to the participants were: availability (willingness to get involved in communication situations with people from other languages/cultures, discovering similarities between cultures and languages and ability to deal with communication difficulties); adequacy (ability to use adequate strategies and resources to understand and/or make oneself understood orally and/or in writing in different Romance languages); extension (ability to expand one’s participation in the communication situation, integrating elements of other languages in one’s own discourse). In this sense, the participants were encouraged by the evaluators to use the languages they could mobilize to perform the proposed tasks by EVAL-IC protocol. So, as mentioned above, all the subjects who participated in the EVALIC tests in Portuguese context were volunteers and were students or scholarship holders in the area of Education (seven were attending the doctoral programme in education) and their mother tongue and/or predominant language was Portuguese, four of whom with the Brazilian variant. All of them chose Portuguese as their communication language with the evaluators, members of the research team of the institution. The analysis of the questionnaire shows that, regarding their perception of the practice of intercomprehension, almost all of them (10) stated that they practised it by changing language and establishing connections between the languages in contact, in a process of mobilizing their whole repertoire, in family, work, or tourism situations, except for one subject who did not answer and another who clearly stated that he/she did not practise it. All subjects

Towards an assessment of intercomprehension competences 211 self-assessed themselves as speakers of more than one language, and at the level of expression the majority (8) chose English as the preferred language of communication, followed by Spanish and French (four subjects) and then Italian (three subjects identified this as a possible language for communication). Regarding their ability to understand Spanish, almost all of them (11) said that they could understand it, followed by French (eight subjects said that they could understand this language with some difficulty) and then by Italian (only three subjects said they could understand this language). As for Romanian, only one subject, the one who reported most experience of travelling and intercultural contacts, rated himself A1 in relation to this language. Overall, all languages were mobilized by the subjects in the different assessment tasks, and all languages were mobilized by the subjects even if to varying degrees. All the subjects understood what was asked of them in different languages (e.g. the questions on the form were formulated either in one language or in another), acquired knowledge about sustainable development, and executed and presented a power point document on that topic in front of a panel of assessors of different languages. Thus, they read in the five Romance languages, reflected about sustainable development, and produced discourse on the subject that they also claimed to be of interest to them. Most of the subjects responded with a genuine interest in the topic, being willing to enter realistically into the assessment scenario, expressing a willingness to engage in communication situations with people from other cultures and languages, discovering points of contact between languages and learning about the topic:

I want to understand my capacity for intercomprehension (S2). As a PhD student in Education, I intend to investigate how to develop a common, interdisciplinary curriculum around the topic of sustainable development. How the contents and competences acquired from regular subjects in schools can contribute so that students develop a critical and active look towards a society that develops with sustainable development (S7). As a citizen and responsible for the upbringing of three children, in daily life and in my professional career, I have always been a teacher, therefore attitudes, examples and teachings have always been a reference so as to contribute to the creation of a more conscious society (S9).

The assessed subjects read written texts, watched, and listened to audiovisual documents, for example video interviews on the topic, in the five Romance languages. Regarding the reading of texts, we can say that their content was identified and understood, and in general all the languages were mobilized in

212  Maddalena De Carlo and Ana Isabel Andrade the execution of the tasks (questions in different languages about sustainable development to prepare the intervention in the conference). Thus, proposing the test of language change and creating the possibility of using texts in different languages to acquire knowledge on the subject, all subjects mobilized at least three languages in the understanding and definition of the concept of sustainable development. All the 12 subjects indicated that, in order to solve the assessment tasks, they used texts in Portuguese, Spanish, and French, 11 used Romanian and 9 used Italian. These data show the competence of the subjects in intercomprehension in Romance languages, demonstrating the potential that working with these languages may have for the development of subjects’ plurilingual competence. It should be noted that the assessment situation potentiates new learning, expanding the linguistic repertoire of the subjects, So, we identified 26 references to the use of Romanian in the answers to one of the comprehension questions, 27 references to texts in French, and 19 to Italian texts, languages considered to be closer and mastered by the subjects at the time of their plurilingual self-assessment. The use of Romanian is therefore clearly an extension of the participants’ language profile thanks to the assessment tests. Lastly, the data confirm the proximity between Portuguese and Spanish and the subjects’ familiarity with this language, with 45 references to Spanish texts. Overall, we can say that a need was created for the subjects to operate in more than one language, giving them the confidence to do so, to expand their linguistic-communicative repertoires by introducing new languages, such as Romanian, and new words and expressions (lexicon on sustainable development in different languages). Thus, we can say that this case study converges with the results obtained by Fiorenza and Diego-Hernández when they observe that “the EVAL-IC test stimulated students’ self-reflection and self-assessment skills on various aspects: their experience of the plurilingual test, their IC skills in terms of repertoire composition (partial language competences, etc.), their comprehension and learning processes and strategies (and their evolution during the test), and the characteristics of the languages (not only the target languages) and their perceived distance” (Fiorenza & Diego-Hernández, 2020, p. 49). We can say that in carrying out the assessment tasks (comprehension, interproduction, …), the subjects showed that their performance intersected with the reflection on their linguistic-communicative repertoire and their representations of the proximity of other Romance languages to their mother tongue. Conclusions For several years, experts in language didactics have been advocating a holistic assessment that does not “artificially and arbitrarily” divide communicative competence into different language activities assessed in isolation (Bourguignon, 2001; Carrasco Perea & De Carlo, 2019; Gorter & Cenoz, 2017; Lenz & Berthele, 2010; Seed, 2020). This perspective is inevitable if we place ourselves in the context of a plurilingual and pluricultural approach, such as that of intercomprehension between close languages. To this end, the researchers

Towards an assessment of intercomprehension competences 213 working on the EVAL-IC project have created an assessment scenario that fulfils the fundamental principles of validity, reliability, and feasibility, in accordance with the objectives and contents of language teaching. In this text, we present a case study which aims at analysing the impact of the assessment scenario on students mastering one or several Romance languages and learning languages. To carry out this analysis, we asked ourselves the following questions: What does it mean to adopt a plurilingual perspective in language assessment? How to consider the role of the mother tongue and the other languages of the subjects? How to create opportunities for the subjects to mobilize their linguistic-communicative resources? The results obtained confirmed the potential of this type of tests for introducing other languages in the linguistic-educational offer of the respective contexts. Furthermore, despite its experimental context, this assessment scenario enabled the assessed subjects to become aware of new possibilities for expanding their plurilingual repertoire in processes of comparison between languages, changing language(s) and mobilizing knowledge of the world and of languages and their functioning, thereby gaining an understanding of the potential that the development of a plurilingual competence based on processes of intercomprehension can bring. In conclusion, the evaluation protocol conceived in the EVAL-IC project shows how it is possible to recognize communication learners’ practices and the competences already possessed by them. In addition, having designed reliable assessment tools making it possible to attest or even certify specific skills in receptive intercomprehension or plurilingual interaction has contributed to the recognition of the potential of this pluralistic approach within institutional education systems. Notes 1 MIRIADI – Mutualisation and Innovation within an Interconnected Network Aiming to Develop e-Intercomprehension, 2013–15. http://miriadi.net. 2 The updated edition of the two frameworks is available at: http://doi.org/ 10.30687/ELLE/2280-6792/2019/01 (see bibliography). 3 EVAL-IC – Evaluation des compétences en intercompréhension, 2016–2019. https://evalic.eu. 4 This is a survey on the assessment practices adopted in five European universities in three different countries (Spain, France, and Italy) and concerning simultaneous learning in four or five Romance languages with an intercomprehension methodology. 5 Reconciling these two criteria is not easy: for example, adult learners, in order to interact in their social or professional environment, often need morpho-syntactic structures and a quantity of lexicon which are well beyond their acquisitional stage, hence the conflict between targeted objectives and attainable results. 6 See, for instance, the experimental phases of projects such as Galatea or Eurom4. 7 The following researchers participated in the experimentation of the evaluation scenario of the project EVAL-IC – 2016-1-FR01-KA203-024155 – ERASMUS+, at the University of Aveiro (Portugal): M. H. Araújo e Sá, A. I. Andrade, A. R. Simões, C. Sá, F. Martins, M. Lourenço, R. Faneca, S. Pinto, and Â. Espinha.

214  Maddalena De Carlo and Ana Isabel Andrade References Andrade, A. I., Martins, F., & Pinho, A. S.. (2019). Un Référentiel de compétences en didactique de l’intercompréhension. REFDIC. EL.LE, 8(1), 253–264. https://doi. org/10.30687/ELLE/2280-6792/2019/01/010. https://edizionicafoscari.unive.it/ media/pdf/article/elle/2019/1/art-10.14277-ELLE-2280-6792-2019-01-010.pdf Beacco, J.-C., & Byram, M. (2007). De la diversité linguistique à l’éducation plurilingue – Guide pour l’élaboration des politiques linguistiques éducatives en Europe. Strasbourg: Conseil de l’Europe. Bourguignon, C. (2001). L’évaluation de la communication langagière: de la connaissance de l’objet à la compétence du sujet. ASp. la revue du GERAS, 34, 53–70. Campodonico, N., & Janin, P. (2018). Construire une progression pour l’Intercompréhension. In S. Garbarino & C. Degache (Eds.), Intercompréhension en réseau: scénarios, médiations, évaluations (pp 323–331). Travaux du CRTT. Université Lumière Lyon 2, IC2014-Projet Miriadi. https://www.miriadi.net/sites/ default/files/ic2014definitivo2018dic2018.pdf. Candelier, M. (Ed.). (2007). CARAP – Cadre de Référence pour les Approches Plurielles des Langues et des Cultures. CELV – Conseil de l’Europe. https://carap.ecml.at/ . Portals/11/documents/C4pub2007F_20080228_FINAL.pdf Candelier, M. (2008). Approches plurielles, didactiques du plurilinguisme: Le même et l’autre. Les Cahiers De l’Acedle, Recherches En Didactique Des Langues Et Des Cultures, 5(1), 65–90. Candelier, M., Camilleri-Grima, A., Castellotti,V., de Pietro, J.-F., Lörincz, I., Meißner, F.-J., Schröder-Sura, A., Noguerol, A., & Molinié, M. (2012). Le CARAP - Un Cadre de Référence pour les Approches plurielles des langues et des cultures – Compétences et ressources. Strasbourg: Conseil de l’Europe. http://carap.ecml.at/. Candelier, M., & De Carlo, M. (2018). Les approches plurielles: des outils d’enseignement et de formation pour aider les enseignants à gérer l’hétérogénéité de la classe de langue. In L. Ouvrard & E. Akborisova (Eds.), L’hétérogénéité dans la classe de langue. Comment et pourquoi différencier?(pp 7–29). Paris, Editions des Archives contemporaines. Capucho, F. (2014). Intermar ou les nouveaux défis de l’Intercompréhension. In H. Daval & T. Nicklas (Eds.), Sens, formes, langage. Contributions en l’honneur de Pierre Frath (pp 345–358). Reims: Ed. Epure. Carrasco Perea, E., & De Carlo, M. (2016). Evaluer en Intercompréhension ou oser le paradigme plurilingue. In E. Bonvino & M. C. Jamet (Eds.), Intercomprensione: lingue, processi e percorsi (pp 183–204). Venezia: Edizioni Ca’ Foscari. Carrasco Perea, E., & De Carlo, M. (2019). ¿Cómo implementar una educación plurilingüe y evaluarla? El ejemplo de la Intercomprensión. Lenguaje y Textos, [S.l.], 50, 75–85, dic. https://polipapers.upv.es/index.php/lyt/article/view/12004. Cognigni, E. (2020). Il plurilinguismo come risorsa. Prospettive teoriche, politiche educative e pratiche didattiche. Pisa: Edizioni ETS. Conseil D’Europe. (2001). Cadre Européen Commun de Référence pour les Langues: Apprendre, Enseigner, Evaluer. Paris: Didier. Conseil D’Europe. (2018). Cadre Européen Commun de Référence pour les Langues: Apprendre, Enseigner, Évaluer (CECR). Volume complémentaire. Strasbourg: Conseil de l’Europe. De Carlo, M., & Anquetil, M. (2019). Un Référentiel de compétences de communication plurilingue en intercompréhension. REFIC. EL.LE, 8 (1), 157–228. https://doi.

Towards an assessment of intercomprehension competences 215 org/10.30687/ELLE/2280-6792/2019/01/008. https://edizionicafoscari.unive. it/media/pdf/article/elle/2019/1/art-10.14277-ELLE-2280-6792-2019-01-008. pdf. De Mauro, T. (2009). Dinamiche linguistiche contemporanee. In Enciclopedia italiana. Disponible sur. http://www.treccani.it/. https://doi.org/10.4995/lyt.2019.12004. Fiorenza, E., & Diego-Hernández, E. (2020). The challenge of assessing plurilingual repertoires: The EVAL-IC project. In G. Seed (Ed.). What does plurilingualism mean for language assessment? Research Notes, 78, 43–52. Giacalone Ramat, A.(Ed.). (1986). L’apprendimento spontaneo di una seconda lingua. Bologna: Il Mulino. Gorter, D., & Cenoz, J. (2017) Language education policy and multilingual assessment, Language and Education, 31(3), 231–248. https://doi.org/10.1080/0950 0782.2016.1261892. Grin, F. (2005). L’enseignement des langues étrangères comme politique publique. Paris: Ministère de l’éducation nationale, enseignement supérieur, recherche (Haut Conseil de l’évaluation de l’école; 19. https://archive-ouverte.unige.ch/unige:14835. Herdina, P., & Jessner, U. (2002). A dynamic model of multilingualism – Perspectives of change in psycholinguistics. Bristol: Multilingual matters. Lenz, P., & Berthele, R. (2010). Assessment in plurilingual and intercultural education. Strasbourg: Council of Europe. Pienemann, M. (1998). Language processing and second language development. Processability theory. Amsterdam/Philadelphia: Benjamins. Schissel, J. L., De Korne, H., & López-Gopar, M. (2021). Grappling with translanguaging for teaching and assessment in culturally and linguistically diverse contexts: Teacher perspectives from Oaxaca. Mexico. International Journal of Bilingual Education and Bilingualism, 24(3), 340–356. Seed, G. (2020). What is plurilingualism and what does it mean for language assessment? What Does Plurilingualism Mean for Language Assessment? Research Notes, 78, 5–15. Shohamy, E. (2001). The power of tests, a critical view of the uses of language tests. London: Routledge. Shohamy, E. (2007). Language tests as language policy tools. Assessment in Education: Principles, Policy & Practice, 14(1), 117–130. Shohamy, E. (2011). Assessing multilingual competencies: Adopting construct valid assessment policies. The Modern Language Journal, 95(3), 417–429.

14 Using the European Language Portfolio for the assessment of plurilingual students in the foreign language classroom Lisa Marie Brinkmann and Sílvia Melo-Pfeifer Introduction Plurilingualism1 is a complex, continuously evolving, multifaceted and individual phenomenon (Piccardo & Capron Puozzo, 2015), as is the plurilingual competence. Its form of assessment therefore should also be multifaceted, alongside being student centred and process-oriented, i.e. providing feedback on an individual’s progress (Barootchi & Keshavarz, 2002; Baturay & Daloğlu, 2010; Burner, 2014; Hansen-Pauly, 2014; Oskarsson, 1980). One possible form of evaluating plurilingual competence is the language portfolio, an instrument that has been said to encourage learner autonomy and individual learning (Kristmanson et al., 2013; Little, 2011, 2012; Little et al., 2017): “In general, a language portfolio can be defined as a systematic and purposeful collection of learner language-related work that shows progress over time and respects instructional objectives” (Gonçalves & Andrade, 2007, p. 198). The potential of the European Language Portfolio (ELP), to encourage individual learning, counts especially for plurilingual students2 as they have heterogeneous language learning experiences (Cassany, 2002). More specifically, teaching plurilingual students could be more efficient if focusing on learning processes and on the development of metacognition through enhanced reflection. In order to respond to these requirements, Little (2011) advocates the use of the ELP which aims at promoting plurilingualism. The ELP is a document that provides a track of transversal and integrated language learning, recording the detailed and changing linguistic profile of the learner (Sheils, 2010). It is a document made for students’ use and should document the diverse linguistic experiences they go through. Its design encourages learner autonomy, awareness and reflection on language learning and intercultural competency (Council of Europe, 2020). Such a stance acknowledges the learner as a social actor, i.e. as an individual with social needs that participates in social dynamics and interpersonal communication, by acting in specific situations (Kern & Liddicoat, 2011). To be effectively implemented, the ELP requires continuous (formative) feedback about the learning performance and

DOI: 10.4324/9781003177197-17

Using the European Language Portfolio 217 process, which is narrated and documented by the learner/social actor. The metacognitive competence and learner autonomy could be enhanced through self-assessment, using evaluation grids derived from “can do” descriptors by the Common European Framework of Reference (CEFR) (López de la Torre, 2013). Self-assessment under the light of the European Language Policy was empirically researched by Oskarsson (1980) and Little et al. (2017). Research on evaluation using both the ELP and the CEFR-grids was done by López de la Torre (2013) and Little (2009). Other possible forms of evaluation with the ELP, however, remain unexplored. This article presents the state of the art on the use of the ELP as a tool to assess plurilingual students’ meta-linguistic and metacognitive strategies and autonomy in foreign language classrooms. The following two research questions will guide the investigation:

• What is the added value of using the ELP for assessing plurilingual students

in the foreign language classroom, according to available literature in the field? • Which requirements should be attained to integrate ELP assessment in the classroom? The chapter is organised as follows: first, the ELP will be presented and discussed under the framework of plurilingualism and plurilingual competence. Second, types of formative assessment and self-assessment using the ELP will be reviewed. To draw theoretical conclusions on the implications and requirements for integrating the ELP in the plurilingual classroom, the importance of reflection on plurilingual students’ cross-linguistic competency will be explored. Definition and principles of the European Language Portfolio There are more than 250 languages in Europe and 24 of which are official. These linguistic circumstances have led to efforts to construct unified linguistic and language learning policies throughout Europe. In 1991, in Rüschlikon (Switzerland), the CEFR was established as a guiding document to support linguistic diversity and the pursuit of plurilingualism for all language learners (Toulouse & Geoffrion-Vinci, 2017). It promotes plurilingualism and the development of a plurilingual competence in the school context, while at the same time recognising the value of all linguistic experiences, inside and outside the school. According to that document, plurilingual education means systematically integrating language education in schools’ curricula, being aware of the reasons for language learning, supporting transfer across languages, and capitalising on already acquired language learning strategies. It involves respecting all languages, cultures, cultural identities and the plurilingualism of others, and being able to mediate between these different languages or

218  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer cultures (Sheils, 2010). One tool for implementing CEFR principles and goals is the ELP, which is defined by the Council of Europe as follows: The ELP is a document in which those who are learning or have learned one or more languages can record and reflect on their language learning and intercultural experiences. It has three components: a language passport, a language biography, and a dossier. The ELP supports the development of learner autonomy, plurilingualism and intercultural awareness. Its main aims are: • to help learners give shape and coherence to their experience of learning and using languages other than their first language • to motivate learners by acknowledging their efforts to extend and diversify their language skills at all levels • to provide a record of the linguistic and cultural skills they have acquired (to be consulted, for example, when they are moving to a higher learning level or seeking employment at home or abroad (Council of Europe 2021). The ELP thus encourages learners to archive, reflect on and assess their language learning, creating a record and contextualising students’ linguistic and intercultural experiences. As part of the ELP, the language biography describes the language levels attained by the students. Specifically, as part of their self-assessment, learners can evaluate their language competency through “can do” descriptors following the CEFR and reflect on the learning strategies applied. The dossier summarises and presents evidence of concrete language performance. Learners can also include language certificates and individual (plurilingual) texts inside the dossier (Coste & Simon, 2009). According to Tagliante and Deleuse (2002), the use of ELP aims to offer a transparent tool to

• understand and diversify language learning at all ages and levels; • protect the different European cultures and the diverse European community;

• enable language learning at all stages of life; • make language programmes transparent; • provide a descriptive evaluation of performance. The ELP was created to value all languages and cultures, whether learnt in formal or informal contexts. Its use serves both a pedagogical function, i.e. valuing the learning process, and a documentary function, i.e. documenting the learning product (Tagliante & Deleuse, 2002). Documenting the learning process can be done by using students’ (oral or written) reflections and can allow students to perceive their progress (Kristmanson et al., 2011). Merging the pedagogical and the documentary function, the ELP can capacitate

Using the European Language Portfolio 219 foreign language students to manage their learning and reflective processes (Little et al., 2017). The European Language Portfolio and the promotion of plurilingualism: possibilities and constraints In 2005, the Summit Declaration of the Third Summit of Heads of State in Government in Warsaw declared that language education had to integrate all the languages present at school. To this, the Council of Europe’s Languages in/for Education project added the promotion of not only modern foreign languages or classical languages but also regional, minority and migrant languages (Little, 2011).3 To fulfil these aims, the ELP is suggested as an adequate concept and pedagogical instrument, since it offers the possibility to explore and document contact with different languages, diverse language learning experiences and strategies, and intercultural experiences (Council of Europe 2000; Little, 2011). Using the ELP, learners’ linguistic backgrounds are valued and they can link their plurilingual learning to intercultural experiences in the language passport. In this way, individuals, languages and cultures are linked (Gonçalves & Andrade, 2007). More specifically, linking intercultural experiences and language learning requires different tasks adapted to the specific individuals (Little, 2011). The portfolio production and its product should be plurilingual so that every student is invited to use all their languages (Little, 2011). By doing so, students can realise the importance of developing foreign language competences and the heterogeneous paths leading to it. The different languages and ways how to learn language(s) can be recognised, valued or assessed in different parts of the ELP. In the language biography, the self-assessment through the “can do” grids allows learners to evaluate one language separated from another and also to differentiate between the skills (reading, writing, speaking, mediation or listening) following the CEFR. Furthermore, since portfolios are bound to a context and represent the entirety of their linguistic context, culture and worldview (Hamp-Lyons & Condon, 2000), learners elaborate and work on their whole, dynamic (language) identity (Coste & Simon, 2009): “Consequently, the image and the model of the ‘native speaker’ is […] replaced by that of the real, plurilingual speaker” (italics in original; Coste & Simon, 2009, p. 180). Being a European tool, the grids are available in all the official European languages, which can be helpful if a plurilingual student knows one of these languages better than the one provided in the foreign language classroom. When discussing the “can do” statements for (self-) assessment, students and teacher can also develop or refine the “can do” statements according to students’ specific needs, e.g. developing cross-linguistic competences. The metacognitive competence is one of the competences the ELP is most suitable to assess. Indeed, learning a language, learning about a language, learning languages, as well as learning about linguistic require learning how to

220  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer learn, i.e. metacognitive competence. The ELP helps learners to reflect and develop this particular competence (Lamb et al., 2017). For plurilingual students, this means learning to monitor their learning process, perceive their (plurilingual) strengths and weaknesses and reflect on their learning strategies, including cross-linguistic strategies which they can develop steadily while counting on the teacher’s support and scaffolding (e.g. providing prompts or guiding questions for written reflections). The teacher should also explain the purpose of the student’s reflection. This can be done by showing examples of other portfolios where students had a moment of “eureka” and gained insight into their learning and understood how to optimise it. The teacher can also explain some common learning strategies to the whole class and guide the students in discovering strategies that fit them best through prepared material or individual questionnaires. Collective discussions can be a good method to approach diverse learning strategies since, by discussing cognition, students are situated at a metacognitive level. One aspect could be added to plurilingual students’ reflections, namely reflecting on their plurilingual identity and becoming aware of any changes in it (Coste & Simon, 2009). Working and reflecting on identity is not only important for plurilingual students but for any learner. To be able to work on students’ language biography, capitalising on their previous linguistic experiences, it is important that teachers encourage students’ awareness of their plurilingualism, identifying it as a rich cognitive and affective resource for further language learning. Thereby, plurilingual students’ specific languages and their identities are valued. Teachers should work with students on their linguistic repertoire and enhance the potential of developing meta-linguistic awareness and using cross-linguistic strategies, as shown in Dmitrenko (2016). Such cross-linguistic competence can be applied in opportunities for interactions with and through the languages present in the classroom. Consequently, in language learning, evaluating plurilingual students can focus on an additional competence, namely cross-linguistic competence. We therefore argue that the ELP, representing individual learning processes, enables the evaluation of more than linguistic competence. Concretely, teachers can evaluate the development of cross-linguistic competence shown in students’ written reflections. Evaluation could focus on the student’s reflection on the learning process and not only on the actual linguistic performances. To assess plurilingual students, teachers can evaluate if and how (well) students apply transfer and inference strategies throughout their languages and to what extent they develop cross-linguistic competence (see the section Using the European Language Portfolio in assessment of plurilingual students: envisioning formative and self-assessment practices). There are some general challenges connected to the implementation of ELP for enhancing plurilingualism: In the accompanying eBook The European Language Portfolio: A Guide for Teachers and Teacher Trainers, only one chapter addresses plurilingualism and only one task is presented (Little & Perclová, 2001). Furthermore, the ELP mentions only integrating European languages

Using the European Language Portfolio 221 and cultures and the CEFR’s descriptors are only available in the official European languages. Thus, the linguistic diversity is not fully expanded to other continents (Llorente Puerta, 2013; Morencia, 2002). In addition, the integration of minority languages is not as developed as the integration of migrant languages, as shown in the examples provided in Lazenby Simpson (2012), Cassany (2002) and the state of the art on using portfolio with plurilingual learners presented in the section Using the portfolio with plurilingual learners: research settings and competences assessed. Finally, an epistemological challenge could also be formulated: despite the promotion in the CEFR of an integrated view of individuals’ linguistic and cultural repertoires, called “plurilingual and intercultural competence”, thus viewing languages as articulated and dynamically interacting with all other languages and semiotic resources of an individual, the self-assessment “can-do” grids still refer to the languages separately. This means that the students evaluate their whole repertoire as the sum of discrete languages, not necessarily interconnected to each other, and not as part of a multilayered and interconnected repertoire. The use of the European Language Portfolio in language education: envisioning the assessment of plurilingual students We will first present case studies using ELP with plurilingual students, relying on their conclusions to answer our research questions. In a second step, we will discuss the assessment process (both formative assessment and selfevaluation), which can be promoted with the ELP in teaching education settings. Using the portfolio with plurilingual learners: research settings and competences assessed

In the following we will present four empirical results from case studies with plurilingual learners who worked with the ELP. Llorente Puerta (2013) developed a case study using the ELP in at least three classrooms with immigrants. She reports on the implementation of ELP in teaching immigrants, discussing students’ samples and the ELP’s practicability in this scenario. The students’ ELP includes information about the country and the culture they emigrated from which means that, especially with adult learners, teachers may get to know the national requirements of students’ previous language education. Indeed, the ELP is not exclusively oriented towards the curriculum but rather considers and integrates diverse educational and curricular experiences of migrant students. Together with the teacher, they are encouraged to discuss “can do” statements, enabling them to apply these “can do” descriptors in a more conscious way. Llorente Puerta (2013) also shows that the ELP values the learner’s plurilingual identity, i.e. every linguistic competence, whether acquired in formal or informal education contexts. According to that author, teaching and learning experiences with the ELP show that

222  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer the CEFR’s evaluation descriptors do not apply seamlessly to the plurilingual situation of migrant students, proposing several reasons for this:

• migrant students’ have specific needs, such as the need for basic information about the society they are living or will live in;

• their previous learning experiences are not identified and the metacognitive competences listed in the CEFR are not explained thoroughly or illustrated with examples to help them understanding what is required in European language education system; • the terms used in the CEFR, its section titles and “can do” descriptors are not provided in non-official and migrant languages, increasing problems in understanding them and reducing transparency in the classroom; • images and visual scaffolding are needed to help migrants make sense of CEFR content (Llorente Puerta, 2013).

In sum, the ELP has the potential to address the individual experiences of migrant students, but lacks formulations for standards in migrant languages. In a case study in Luxemburg, Hansen-Pauly (2014) examines the multilingual situation of the country and how teachers address it. Because the school system is multilingual, students and teachers are all plurilingual in Luxemburg. Her results show that reflecting on one’s development needs to be linked to (target) language use. In order to document and nourish their progress, students should work with “can do” grids and set themselves learning goals, thus becoming more agentive and aware of their learning path. Helping students with an inner dialogue, i.e. reflections on their own self and doing and learning, teachers and students both get a chance to analyse linguistic representations. In this inner dialogue, mediation in the broad sense takes place because the learners have to formulate their thinking. This process may involve different languages, through translanguaging practices, and include communicating ideas to the teacher in a common language. The author therefore argues for a twofold evaluation, including self-assessment and peer assessment, to identify processes and progress in language learning from different angles (HansenPauly, 2014). Thus, evaluating plurilingual students through the use of the ELP involves insights into their learning before assessing them in diverse ways. In their study, Csire and Laakso (2014) focus on heritage language learners. They show how the learners can use the ELP to reflect on meta-linguistic awareness and to consciously analyse their heritage language, according to their level of proficiency. The authors identify students’ specific needs and develop some possible explanations for these, including the development of general learning strategies, such as cross-linguistic ones,4 that are “entrenched during the formal instruction of L2/3 at school” (Csire & Laakso, 2014, p. 225). In sum, the study highlights the role that reflection plays for plurilinguals in language learning. Another study focuses on the implementation of the ELP in a class of adult immigrants: the language programme Integrated Ireland Language

Using the European Language Portfolio 223 and Training encouraged students to plan, monitor, reflect on and assess their learning and learning experiences. The portfolio developed for this programme (called Milestone ELP) presents an alternative to language tests for immigrants. The ELP aims to foster learner autonomy to improve learning and teaching strategies, highlight students’ competences and develop their metacognitive, intercultural and communicative skills. Little et al.’s (2017) case descriptions of the implementation of the Milestone ELP in different classrooms indicate that migrant students reflect on their learning with the help of the language biography. The language biography includes reflection on language learning, intercultural and personal experiences, actual proficiency in the target language, awareness and attitudes, as well as the development of individual learning strategies, setting and assessment of learning goals through the “can do” descriptors which may not have been achieved yet. The language biography displayed specific descriptors from the Education Council for immigrants, in the form of checklists, to fulfil the latter’s requirements. Little et al. (2017) present a promising way of supporting migrant students in their language learning using the ELP. Using the European Language Portfolio in assessment of plurilingual students: envisioning formative and self-assessment practices

In the ELP, assessment is done by using the CEFR’s “can do” descriptors. Because the assessment is not done separately but relates to the learning process and the contents and competences taught, it links assessment to learning and teaching (Little, 2011). This integrated form of assessment enables internal classroom differentiation. In general, the ELP can thus contribute to self and formative assessment (Kristmanson et al., 2013). Self-assessment aims to perceive individual gaps in learning. In this way, learners can set individual goals and gain insight into what is required (Oskarsson, 1980). It is crucial that they perceive a gap between the desired goal and the actual state of development so that they can then develop a plan to close it. In the ELP, self-assessment is carried out with reference to the proficiency levels of the CEFR. Checklists of descriptors derived from its illustrative scales are used to identify learning targets, monitor progress and assess learning outcomes; and in the language passport overall L2 proficiency is periodically summarized against the self-assessment grid. (Little, 2009, p. 3) The CEFR provides rubrics for self-evaluation, which are integrated into the language biography. Such a self-assessment is elaborated in autonomous learning settings and depends on learners’ cognitive and metacognitive processes (Little, 2009; Little et al., 2017). Assessment is directly linked to the learning process and to written or implicit reflection on it; within this perspective,

224  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer self-assessment becomes diagnostic (Lamb et al., 2017). Self-assessment does not exclude help from peers; on the contrary, their assistance, i.e. formative peer assessment, enriches the evaluation (Oskarsson, 1980). Thus, learners are invited to evaluate themselves by using the descriptors with the help of a teacher, so as to neither overestimate nor underestimate their competences (Tagliante & Deleuse, 2002). Students’ self-assessment in the ELP is done through evaluating their progress continuously, which is a form of formative assessment that may lead to increased autonomy. However, as Kristmanson et al. (2013) have stated, getting to know and evaluating their own competences as well as planning their learning can be difficult. Once plurilingual students learn to work individually, they can also evaluate themselves individually by considering their holistic learning context. They are encouraged to identify what they are able to do in the foreign language classroom, referring not only to the target language but their plurilingual repertoire. Therefore, in their reflections, students should document their learning strategies and include possible difficulties of learning. Through learners sharing their observations on learning, the teacher gets to know the individual strengths and weaknesses and should aim to address both in suitable ways. For this purpose, precise teacher training on supporting plurilingual students in their learning can help to evaluate the appropriateness of students’ learning strategies, based on their reflections. The support can be varied: if the learner needs more information, the teacher should prepare further material; if the learner’s texts for the dossier lack creativity, the teacher should provide some inspiration; if the learner does not understand a task, the teacher must take the time to explain it individually; if the learner cannot reflect on their learning in the classroom language, the teacher should find some sort of mediation to get to know students’ reflections on learning without losing information due to language difficulties; if the learner does not take responsibility for their own learning, the teacher should check students’ learning process, e.g. by setting deadlines. In sum, the teacher has to be aware of the difficulties that the plurilingual learner can have in self-assessment and autonomous learning. Such difficulties can be the same as those of other students, but they can include difficulties with the target language and the use of languages to provide scaffolding, difficulties with developing and using cross-linguistic strategies, and difficulties in or with reflecting on one’s language learning processes, accomplishments, paths and linguistic projects. Formative assessment means providing feedback, i.e. individualised continuous assessment (Black & Wiliam, 1998). Formative assessment focuses on skills and performances compared to previous performances and can be applied to both peer and self-assessment (Little & Erickson, 2015). In general, peer and self-assessment provide autonomous settings where “learning is driven forward by reflective evaluation” (Little et al., 2017, p. 241). For formative assessment, the dossier and the language biography of the ELP play an important role.

Using the European Language Portfolio 225 The dossier contains texts the learner produced through writing, peer assessment and improvement processes. Learners’ texts need continuous revisions by the teacher and/or peers to improve the learning process and the learning results, whereby learning is understood as a collaborative process. For both self-assessment and formative assessment, it is crucial that students and teachers understand the “can do” descriptors given by the CEFR and/or adapted “can do” descriptors (Little & Erickson, 2015). Referring to the descriptors, the teacher should support students by providing continuous feedback on their performances and competences. Language tasks and texts in the dossier are evidence of the performance. The teacher can refer to these performances by analysing both the process and the final product (text/task) and discussing them with the learner. The teacher can highlight the strengths but also show gaps in learning where the student should invest more time and learning. Together, they can develop the next steps and exploit the potential of the formative language learning situation if the teacher’s feedback takes into account the student’s reflection. Students, especially plurilingual students, can use the opportunities of formative feedback from their teacher to discuss individual learning issues and ensure that they understand what is required from them. Concretely, teachers and students should discuss achievements referring to individual strengths, encouraging improvement and advancement in the target language. One potential of formative assessment using individual ELPs is the comparison between students’ former and current stages in language development. The “can do” descriptors enable the integration and evaluation of crosslinguistic strategies. These descriptors can include achievement in transfer and inference competences by examining specific examples the students prepared, e.g. intercomprehension tasks. As explained above, the students need to understand what is expected from them and the teacher can introduce them to cross-linguistic strategies by explaining what they are, how they can be developed and how the students’ repertoires can foster them, if activated and capitalised upon. Learners can be encouraged to use transfer and inference skills in any situation, which they can document in the dossier and/or on which they can reflect in the language biography. The teacher and the students can develop specific learning products (a written text, a talk, a written/oral comprehension task or a mediation task) and reflections on the cross-linguistic competence, for which explicit descriptors might be developed, applying to language-based and non-language-based situations. Finally, such an evaluation grid containing descriptors for cross-linguistic competences can be added to the larger grid on specific linguistic competences in the target language. Through individual and collaborative reflection and discussion in the classroom, evaluation grids in the ELP can be expanded, refined and validated. For this purpose, the development of cross-linguistic strategies should be understood as a meta-level of (language) learning and evaluated as an important part of mastering the target language.

226  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer Conclusion Since the ELP is a tool that encourages plurilingualism (despite some inconsistencies that we commented on above) and since it has the potential to provide specific descriptors for plurilingual learners, this pedagogical instrument can be used to evaluate plurilingual students in a way that fosters their language learning process and boosts their metacognitive and meta-linguistic competences. Because of the philosophy attached to its implementation, the use of the ELP can be beneficial to plurilingual students with and without a migrant background, adults or younger students, learning the language of schooling or foreign languages. As we also saw, the advantages of using the ELP for assessment purposes are twofold. First, it can help students evaluate their language performance and plurilingual competence. Through its use, plurilingual students can assess their language skills and learning process in the target language and also their cross-linguistic competences. They can use the grids based on the CEFR’s descriptors to evaluate their language skills and use specific grids developed to assess their cross-linguistic competency. The peer formative assessment should be linked to self-assessment where reflections on individual learning situations and their potential for cross-linguistic competency are stressed. In this way, the ELP is a multilayered instrument of assessment, which can address the challenges and the characteristics of multifaceted and diverse language learning scenarios. One of these challenges is, of course, the presence of heterogeneous repertoires in the classroom, leading to the need for implementing differentiation, responding to the inclusive turn (as addressed by Mary et al., 2021) in assessment practices. Second, it can guide teachers through the design of tasks that address students’ concerns and foster their autonomy as language learners. The structure of the ELP and especially of the language biography serves as orientation for the development of tasks fostering cross-linguistic competence that can then be assessed using specific “can do” descriptors developed by the teacher. The literature review made clear that the evaluation of plurilingualism in an autonomous learning setting guided by the implementation of the ELP can and should be done through reflection, both individually and collaboratively, with the teacher and with peers. The teacher encourages the learners to reflect on their plurilingual repertoires and their linguistic and cross-linguistic competences. Through self-assessment, students should be called to take responsibility for their learning process, achievements and goals. In a path to a (more) autonomous learning process, the use of ELP allows students to set their own learning goals, to trace a plan and implement strategies to achieve them, and finally to evaluate the whole process and its outcomes. As a learning facilitator, the teacher can help students to determine some general aspects of their learning steps, but the reflection on and implementation of learning strategies must be embraced by the students themselves.

Using the European Language Portfolio 227 Despite the potential of the ELP, its broad implementation failed.5 A sign of the rather shy implementation of the ELP can be seen in the lack of empirical studies that provide evidence of the intended competence development related to its use as an assessment tool. This may be due to incompatibility with some national educational systems, problems of integration into the curriculum, and the unsatisfied expectation that the portfolio could resolve all problems in language learning. Additionally, internal problems of the ELP can be a reason for its failure, such as internal obstacles to plurilingualism by referring to “can do” descriptors in different, isolated languages, as if they were “entirely separate entities” (Little, 2016, p. 167), or not addressing equally the Council of Europe’s ethos for politics, culture and education in any learning setting (Little, 2016). To bring the ELP to the forefront again and address plurilingual students’ needs, the integration of languages should be more systematically recognised and valued, perhaps based on the descriptors of the Companion Volume (Council of Europe, 2020). Likewise, the design of the ELP may be reoriented according to plurilingual pedagogies such as those in the Framework of Reference for Pluralistic Approaches to Languages and Cultures (Candelier et al., 2007), using their descriptors around attitudes, knowledge and skills to review the “can do” standards. The marked difference observed between the generalised implementation of the CEFR across multiple contexts and learning settings for certification purposes, on the one hand, and the rather reduced enthusiasm around the use of the ELP for promotion of plurilingual competence, on the other, is a sign of the possible dilemmas undermining a more regular use of the ELP. Notes 1 We distinguish between plurilingualism and multilingualism, the first referring to the individual repertoires (also known in the literature under the designation “individual multilingualism”) and the second to the coexistence of languages at the societal level. 2 We should note that “plurilingual students” can refer both to students who have learnt, or in the process of learning foreign languages in the school context and to students with a migrant background which are learning the language of schooling or/and other languages. The path to become plurilingual can thus be related to (multiple) formal language learning experiences, (multiple) language acquisition from early on to adult age and a combination of both. 3 For further information, see Council of Europe (2000). 4 Cross-linguistic competences include strategies of transfer and inference and occur on the level of “dynamic construction of interlanguages” (Dmitrenko 2016, p. 264). 5 With the exception of the language programme Integrated Ireland Language and Training in Ireland (cf. Little et al. 2017).

References Barootchi, N., & Keshavarz, M. H. (2002). Assessment of achievement through portfolios and teacher-made tests. Educational Research, 44(3), 279–288.

228  Lisa Marie Brinkmann and Sílvia Melo-Pfeifer Baturay, M. H., & Daloğlu, A. (2010). E-portfolio assessment in an online English language course. Computer Assisted Language Learning, 23(5), 413–428. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. Burner, T. (2014). The potential formative benefits of portfolio assessment in second and foreign language writing contexts: A review of the literature. Studies in Educational Evaluation, 43, 139–149. Candelier, M., Camillieri-Grima, A., Castellotti, V., de Pietro, J.-F., Lörincz, I., Meißner, F.-J., Noguerol, A., Schröder-Sura, A., & Molinié, M. (2007). FREPA. Framework of reference for pluralistic approaches to languages and cultures. Strasbourg: Council of Europe and Graz: ECML. Cassany, D. (2002). Usando el portfolio europeo de las lenguas (PEL) en el aula. MOSAICO: Revista para la Promoción y Apoyo a la Enseñanza del Español, 9, 18–24. Coste, D., & Simon, D.-L. (2009). The plurilingual social actor. Language, citizenship and education. International Journal of Multilingualism, 6(2), 168–185. Csire, M., & Laakso, J. (2014). L3, L1 or L0? Heritage language students as thirdlanguage-learners. In A. Otwinowska & G. de Angelis (Eds.), Teaching and learning in multilingual contexts: Sociolinguistic and educational perspectives (pp 215–231). Bristol: Multilingual Matters. Council of Europe. (2000). Common European Framework of Reference for Languages: Learning, teaching, assessment. Strasbourg: Council of Europe. Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Strasbourg: Council of Europe. Council of Europe. (2021). What is the ELP?, https://www.coe.int/en/web/portfolio/ introduction [10/01/2021]. Dmitrenko, V. (2016). Language learning strategies for the development of plurilingual competence in European languages in adult learners (Unpublished doctoral dissertation), Universidad de Navarra, Pamplona. Gonçalves, M., & Andrade, A. I. (2007). Connecting languages: The use of the portfolio as a means of exploring and overcoming frontiers within the curriculum. European Journal of Teacher Education, 30(2), 195–213. Hamp-Lyons, L., & Condon, W. (2000). Assessing the portfolio. Cresskill: Hampton Press. Hansen-Pauly, M.-A. (2014). Teacher education: language issues in multilingual educational contexts: Sensitising subject student teachers for language issues and cultural perspectives. Strasbourg: Council of Europe. Kern, R., & Liddicoat, T. (2011). From the learner to the Speaker/Social Actor. In G. Zarate, D. Lévy, & C. Kramsch (Eds.), Handbook of multilingualism and multiculturalism (pp 17–23). Paris: Editions des Archives Contemporaines. Kristmanson, P., Lafargue, C., & Culligan, K. (2011). From action to insight: A professional learning community’s experiences with the European :Language Portfolio. The Canadian Journal of Applied Linguistics, Special Issue, 14(2), 53–67. Kristmanson, P., Lafargue, C., & Culligan, K. (2013). Experiences with autonomy: Learners’ voices on language learning. Canadian Modern Language Review, 69(4), 462–486. Lamb, T., Vieira, F., & Jiménez Raya, M. (2017). Mapping autonomy in language education: A framework for learner and teacher development. Frankfurt: Peter Lang.

Using the European Language Portfolio 229 Lazenby Simpson, B. (2012). Using the ELP with adult refugees learning the language of the host community. In D. Little (Ed.), The European Language Portfolio in use: nine examples. (pp. 41–49). Strasbourg: Council of Europe. Little, D. (2009). The European Language Portfolio: Where pedagogy and assessment meet. Strasbourg: Council of Europe. Little, D. (2011). The European Language Portfolio: A guide to the planning, implementation and evaluation of whole-school projects. Strasbourg: Council of Europe. Little, D. (2012). The European Language Portfolio in whole-school use. Innovation in Language Learning and Teaching, 6(3), 275–285. Little, D., & Erickson, G. (2015). learner identity, learner agency, and the assessment of language proficiency: Some reflections prompted by the common European framework of reference for languages. Annual Review of Applied Linguistics, 35, 120–139. Little, D. (2016). The European Language Portfolio: Time for a fresh start? International Online Journal of Education and Teaching, 3(3), 162–172. Little, D., & Perclová, R. (2001). The European Language Portfolio: A guide for teachers and teacher trainers. Strasbourg: Council of Europe. Little, D., Dam, L., & Legenhausen, L. (2017). Language learner autonomy: Theory, practice and research. Bristol: Multilingual Matters. Llorente Puerta, M. J. (2013). El portfolio europeo de las lenguas (PEL) para alumnado inmigrante, refugiado y solicitante de asilo. In B. Blecua Falgueras (Ed.), Plurilingüismo y enseñanza de ELE en contextos multiculturales: Girona, 2012 (pp 508–517). Málaga: Asociación para la Enseñanza del Español como Lengua Extranjera. López de la Torre, J. A. (2013). El portfolio Europeo de las Lenguas (PEL): herramienta de evaluación de la composición escrita en Español como Lengua Extranjera. Valencia: Universitat de València. Mary, L., Krüger, A.-B., & Young, A. (2021). Migration, multilingualism and education: Critical perspectives on inclusion. Bristol: Multilingual Matters. Morencia, C. G. (2002). Desarrollo del portfolio europeo en españa. MOSAICO: Revista para la Promoción y Apoyo a la Enseñanza del Español, 9, 8–13. Oskarsson, M. (1980). Approaches to self-assessment in foreign language learning. Oxford: Pergamon. Piccardo, E., & Capron Puozzo, I. (2015). From second language pedagogy of ‘plurilingualism’: A possible paradigm shift? Introduction. The Canadian Modern Language Review, 71(4), 317–323. Sheils, J. (2010). Council of Europe Policy for plurilingual education. In B. O’Rourke & L. Carson (Eds.), Language learner autonomy: Policy, curriculum, classroom; A festschrift in honour of David Little (pp 315–326). Frankfurt: Peter Lang. Tagliante, C., & Deleuse, J. (2002). Les portfolios européens des langues. Revue internationale d’éducation de Sèvres, 31, 138–142. Toulouse, M., & Geoffrion-Vinci, M. (2017). Electronic portfolios in foreign language learning. In N. van Deusen-Scholl & S. May (Eds.), Second and foreign language education (pp 151–166). Cham: Springer.

15 Evaluation and plurilingual competence Between epistemological consistency and ethical vigilance Emmanuelle Huver Questions about the evaluation of plurilingual competence have always accompanied reflections on plurilingual competence itself, and this has been the case from the start of work on this topic. Therefore, in the research literature on these issues, it appears from the very first reflections on the subject, included in the French-language literature, on which I will preferentially rely here.1 These different works regularly highlight the difficulty, nay the impossibility, of assessing plurilingual competence: “assessment” and “plurilingual competence” are then presented as belonging to two paradigms that are so different that their combination would practically be an oxymoron. This tension between the need to assess plurilingual competence and the (technical and/or ethical) impossibility of doing so is still present today, including in the argument of this book. The ways of trying to resolve this contradiction have been numerous. They have even multiplied as the notion of plurilingual competence was growing in importance, particularly at the institutional level: the publication of the Companion Volume of the Common European Framework of Reference (Conseil de l’Europe, 2018) can be considered as one of the most recent avatars of that process. In this chapter,2 I will argue that the research on plurilingual competence assessment has essentially consisted of a reflection on assessment tools and approaches and, logically, on designing such tools and approaches. By doing this, the research has neglected a more theoretical/epistemological reflection on the tensions mentioned above between assessment and plurilingualism, that brings up, more broadly, the problem of the status and conceptions of the notion of diversity at the basis of these tools. I don’t have enough space here to present extensively these different conceptions, but I can sum them up roughly by opposing two conceptions of that notion. One the one hand, diversity can be understood as a quantitative notion, in which case language diversity, for example, refers to the fact that languages can be objectively distinguished, enumerated, cumulated. One the other hand, diversity can be understood as a qualitative notion that questions of the very notion of language as a homogeneous system with absolute and predefined contours. In that conception of diversity, what matters is DOI: 10.4324/9781003177197-18

Evaluation and plurilingual competence 231 what’s happening in the relation between languages (that is interlingustic and translinguistic processes, but also biographical aspects when a person puts into relation different “languages” that crossed one’s life) and the experience of otherness that results from that linking.3 I will argue that this until now quite neglected reflection is yet essential to understand what can be diversely meant by “assessing plurilingual competence” and thus to define conceptions of plurilingual competence and evaluation that both are based on (qualitative) diversity as a core value whether in the engineering, political or ethical field. To this end, I will first provide an overview of recent research on assessment in language didactics, paying close attention to work on plurilingual competence. Then, I will show how monolingual assessments and the research criticizing them are based on a conception of language and assessment that is, in the end, quite convergent. This will allow me, in the last part, to make some proposals—epistemological in nature—as to what could be a languages assessment that takes diversity seriously. Recent work on evaluation in language teaching: between criticism and engineering To sum up roughly, most of the work on evaluation in language didactics can be divided into two main trends: one that we could “critical” on the one hand and one that we could call “engineering” on the other hand.4 The critical trend examines the political and ideological stakes on evaluation. If we stick to the most recent works, it consists essentially in a criticism of certifications or at least of their prevalence. Researchers denounce the fact that the predominance of certifications leads to an overvaluation of descriptors and standards, with the corollary of a homogenization of the proposed evaluations, an excessive influence of certification bodies on language teaching/ learning and a reduction of educational stakes to their market logic. Within this “critical” theoretical framework, one frequently finds the idea that evaluation, or more precisely, standardized evaluation, would be at the service not of education but of a capitalist and managerial ideology (cf., for example, Maurer & Puren, 2019). Moreover, these analyses converge, although they are not actually being treated as linked, with the work of “critical” sociolinguistics, which considers languages (and in particular linguistic diversity) as a capital inscribed in power relationships, or even participating in types of domination by capitalism (cf. the work of M. Heller, A. Duchêne or, from a slightly different angle, Ph. Blanchet). These criticisms of evaluation, and more particularly of certification, are part of a larger criticism of the ideology of evaluation, from an economistic and ideological perspective inspired by Bourdieu’s critical sociology. They simultaneously draw on reflections from other fields of the humanities (philosophy, sociology, political science, etc.) and, for language didactics, on work from English research (for example, Mc Namara & Roever, 2006; Shohamy, 2001; Spolsky, 1995, etc.). In this framework, assessment (in fact, more precisely,

232  Emmanuelle Huver standardized assessment) is seen as a “social technology” (Madaus, 1990) used to support a more or less explicit ideology of selection, which (among others) deeply question the goals of compulsory schooling. The engineering trend is essentially interested in the building of tools and approaches (and, more secondarily, on practices) to enhance relevance to a specific chosen teaching methodology (communicative, action-oriented, task based, etc.). This work can take the form of methodological guides drafting evaluation tools, guides that are either general (Bachman & Palmer, 2010) or targeted at specific didactic areas or audiences (Kunnan, 2014, which is intended to be a reference work in the field). Most of them aim to promote alternative assessment approaches to certification and, more widely, to summative assessment and to the use of numerical marks. The aim then is to foster assessment practices:

• for language learning and/or learning to learn languages (Cuq & Davin, 2016; Médioni, 2016; Soubre, 2021);

• consistent with the latest methodological approaches promoted, foremost

among which is the action-oriented perspective widely disseminated by the CEFR.

We can observe a real enthusiasm for this trend in the language testing field, as evidenced by the increase in titles published on the subject over the past five years (for example and without being exhaustive: Cheng & Fox, 2017; Davis et al., 2018; Hidri, 2018; Shohamy et al., 2017; Taguchi & Kim, 2018). Most of the tools developed may have in common to promote “alternative” (that is more positive and formative) assessment, but they do not really address explicitly (or rarely) the issue of assessing plurilingual competence, even though there is a growing interest in this topic. Moreover, they mostly stick to the observable dimensions of the competence only, without questioning the didactical and political consequences of this choice that finally avoids the “inaccessible” dimensions of assessment process and the subjectivity that it would introduce in evaluation (Huver, 2014b). What about research on plurilingual competence assessment? Similar structuring

The two trends highlighted in the field of languages assessment are also reflected in works on the assessment of plurilingual competence. The critical branch consists essentially of a criticism of monolingual evaluations on the grounds that:

• they do not allow for the entire language repertoire of individuals to be taken into account, even in situations where this is, didactically and/or politically, particularly relevant. For example, the people’s orientation at the

Evaluation and plurilingual competence 233 start of their training would benefit from being relying not only on their “level” in the target language, but on a range of information concerning their language biography (cf. in France, the assessments carried for the orientation of migrant pupils at school, which include an assessment in their so-called original language); • they are often based on a very normative conception of language: people are then evaluated in relation to a fantasized “native speaker”, who would speak a “pure” language or, at least, a very widely homogenized one (that is redacted from its variations). According to the proponents of this type of criticism, the consequences of this normativity and homogenizing conception of language are both didactic and political. On the didactic level, monolingual assessments are denounced as essentially subtractive (people are assessed on what is not conform to the dominant norm). On the political level, they are denounced for ignoring the identity dimensions of language, resulting in stigmatizing or discriminatory effects. These works criticizing monolingual assessments are quite diverse. Some of them deny descriptors and scales per se (often by referring at Bourdieu’s critical sociology) or criticize the CEFR as based on irenic plurilingualism. Some say it evacuates subjectivity and subject (Anderson, 1999; Anquetil et al., 2017; Prieur & Volle, 2016); for others it does not mention the issues of commodification of education (Maurer & Puren, 2019); for others still it lacks of a fundamental epistemological reflection on the notion of diversity, which would connect the above-mentioned other criticisms (Castellotti, 2017; Huver, 2014a). But some others recognize the CEFR, including descriptors and scales, as an attempt to overcome the homogenization of language, or at least the systematic reference to the so-called “native speaker” in assessments (cf. arguments in favour of the CEFR Companion Volume in Piccardo et al., 2019). However, in the field of plurilingual competence assessment, the engineering branch is also the most developed (like in language assessment in general). A great deal of research has been dedicated to the preparation and testing of tools and approaches to assess plurilingual competence, including:

• tools for the evaluation of the various approaches that fall within the so-

called “plural approaches” (which include: intercultural approach, integrated didactics, intercomprehension, language awareness). Thus, for example, articles or book chapters are devoted to the evaluation of intercomprehension (for example, Ollivier, 2019); • tools directly concerned with plurilingual competence. These tools can either be reference systems incorporating descriptors, such as the Framework of reference for pluralistic approaches to languages and cultures, the Harmos project,5 or the CEFR Companion Volume, or tools for assessing plurilingual competence itself. In this regard, we can mention in particular, in addition to the European Language Portfolio, the proposals of Lenz and Berthele (2010) and Goullier (2017, p. 266ff.), the experimental plurilingual

234  Emmanuelle Huver assessment scenarios of the Val d’Aosta (linked on specific curricula), and the mediated language tasks of the Inca project (Huver & Springer, 2011). While many of these tools explicitly follow the guidelines of the CEFR, others distance themselves from it, but still claim in the same time: an ‘integrated assessment’ which takes into account all the issues at stake in the language teaching-learning process; an ‘integrated plurilingual methodology’ which relies on the already existing in terms of learners’ language repertoire and on the already constructed in terms of knowledge about languages and language learning skills. (Maurer & Puren, 2019) Research on the assessment of plurilingual competence is therefore a complex and heterogeneous field, with its areas of dissensus and internal tensions. Nevertheless, however diverse they may be, these studies agree on some common principles. In particular, recommendations that transversely aim denounce the (formative and social) selection linked to assessment, the negative effects on pupil’s well-being, learning capacity, and empowerment. Reciprocally, they promote assessments that would be less “threatening” (Butera et al., 2011), less “detrimental” (Shohamy, 2001), less “glottophobic” (Blanchet, 2016), and therefore more qualitative (not based on a score, a fortiori graded), more formative (strengthening learning), and more inclusive (of people’s linguistic and cultural diversity especially)—on this point, cf. Goullier (2017, p. 260) or Cavalli in this book. This has played a major role in raising awareness of the political and even identity-related issues associated with evaluation. It has also contributed to the diversification of evaluation tools and to new ways of dealing with the linguistic and cultural diversity (cf. the emblematic example of the European Language Portfolio). The other common point is that even the critical works consider the interventionist aim of language didactics as essential, so that they propose, as a counterpoint to their criticism, other tools. But doing so, they also reinforce in the end the focus of language didactics research on the engineering dimension. What are the implications of these findings for research on the assessment of plurilingual competence? Plurilingual assessment, truly?

This focus on the engineering aspects of language didactics ultimately leads to the idea that exercises, activities and evaluation tools are in themselves destructive (or constructive), threatening (or benevolent), glottophobic (or glottophilic), depending on whether they are closed (or open), quantitative/ graded (or qualitative/ungraded), monolingual (or plurilingual), and exclusive of diversity and variation (or inclusive of these). This contributes to evacuating

Evaluation and plurilingual competence 235 from researches representations on the one hand and epistemological questions on the other. However, without this epistemological questioning, the conception and the status of diversity are notoriously excluded from reflection:

• first of all, because the notion of plurilingualism (and, more broadly, of linguistic plurality/diversity) is not defined, the work of the years 1990–2000 on the subject being generally considered sufficient6; • secondly, because linguistic diversity is maintained as an object to be assessed, based on the observable and objective dimensions of it. Thus, from this double point of view, there is paradoxically a form of (epistemological) convergence between assessments that are denounced as being monolingual and those that claim to be plurilingual. In monolingual assessments indeed, (linguistic) diversity is certainly absent; but in so-called “plurilingual” assessments, it is generally reduced to quantitative forms of diversity:

• basically, a tool is offered in X languages or includes X languages; • they rarely include trans- and inter-linguistic dimensions. And even if the

trend is growing (Chardenet, 2005; Conseil de l’Europe, 2018; Lenz & Berthele, 2010, etc.), the cited researches reduce plurilingualism to observable dimensions of the plurilingual competence (vs question the stakes of including the more unobservable dimensions of the plurilingual experience); • and even more rarely, they try to seize the ensemble of issues that question evaluation from the perspective of diversity. In other words, they do not try to imagine what could be an evaluation in which diversity would not only be the object but mainly the core value (cf. recently, nevertheless, Gajo et al., 2019). The next part of this article will try to develop more precisely what I mean by “considering diversity as core value” in the assessment field (that is what I have also called “a methodical use of diversity” or a “diversitarian” evaluation7). The same rationale can be extended to conceptions of evaluation. Indeed, even when they include elements of diversity (and/or when they claim to assess plurilingual competence), the different assessments and/or reflections on assessment eventually remain epistemologically homogeneous by the fact that they:

• generally maintain the idea that some assessment tools or practices would be inherently better than others;

• conceptualize the intrinsically interpretive (hermeneutic) nature of evalu-

ation only to a limited extent, except in an attempt to “control” or “master” the assessment process, that is then considered as (too) subjective (see below); • thus perpetuate a “technical thought” in languages assessment, beyond the formally qualitative aspects that are yet considered inseparable from

236  Emmanuelle Huver the assessment of plurilingual competence (on the question of “technical thought” in language didactics, see Debono, 2020). In the end, since diversity remains the object of assessment but not its principle, language assessment can be considered to remain ultimately monolingual deep down, including the assessment of plurilingual competence. In other words, the “reversal” promised by the introduction of plurilingual competence in language didactics (Coste, 2002) does not seem to have affected assessment, if not very superficially: Un système de gradation qui a été d’abord pensé et produit en relation à une langue reste inscrit dans une logique monolingue de progrès continu apprécié tout au long selon les mêmes critères. Un tel système ne peut tout simplement pas accommoder la pluralité linguistique et culturelle dans ce qu’elle a—justement—de plurilingue et dans les discontinuités, les recompositions, les modifications d’équilibre entre les différentes composantes du répertoire que présente l’évolution « ordinaire » des capacités plurilingues et pluriculturelles de tout acteur social. (Coste, 2021)8 Taking diversity seriously in languages assessment Then, what can it mean to let language didactics (and languages assessment) be affected (in the strongest sense of the word) by plurilingual competence, or more precisely by the principle of diversity? In other words, what can it mean to “take diversity seriously” in languages assessment and thus put diversity at the heart of “language” and “assessment”? Language(s): thinking on the basis of linguistic diversity

As explained above, assessing in a plurilingual manner is not simply a matter of assessing plurilingual competence, let alone as a juxtaposed assessment of different languages, based on level descriptors designed a priori. In my work and in the work of my research team (Dynadiv, https:// dynadiv.univ-tours.fr/), the notion of language is not initially conceptualized as a system or a means (of communication), nor is it considered primarily as a heterogeneous social phenomenon, but as ways of experiencing the world. Let us also specify that, in this orientation, an experience is not only individual and, consequently, does not evacuate the political dimension: the notion of experience as conceptualized here takes on a historical thickness and density and, in doing so, a collective dimension requiring a political treatment (cf. in this regard Romano, 2010). Furthermore, it is worth asking what “it means to look at a language as a foreign language” (Gajo et al., 2019). According to them—and I largely agree with them:

Evaluation and plurilingual competence 237 The answer immediately involves the notions of contact and context: a language is foreign because it comes into contact with at least one other language, either in a particular social context or in the trajectory of a learner. At the didactic level, the intersection of these two reflections (language as an experience of the world/foreign language learning as an experience of contact and otherness) implies considering that learning/teaching a language is first and foremost a way of experiencing otherness: Learning/teaching a language is to (let) experience and confront a certain way of being in the world; and consequently, learning/teaching a foreign language is to (let) experience and confront another way of being in the world, another way of being human. (Huver & Macaire, 2021) However, in existing assessments, the underlying conception of language is always either structural or communicative/task-based and therefore pragmatic. This also applies to assessments of plurilingual competence (cf. the tools mentioned above, which are mostly based on the methodological principle of authentic communicative tasks). Moreover, the question of language contacts is never really raised, except at worst under the angle of interference (to be banned) or, at best, under the angle of error (to be explained). However, from my approach, language evaluation is necessarily language evaluation (for example, is always plurilingual). In other words, it cannot avoid the question of contacts, considering them not only as normal, but also as a certain “way of being” (to use Beacco’s terms about plurilingualism) and as part of what Castellotti (2017) calls “appropriation”.9 Assessing plurilingual competence from a pragmatist perspective is, in the end, quite easy—even if it involves a sophisticated didactic technology— since, reduced to its semiotic dimensions, language is more easily predictable, and therefore controllable, and therefore assessable. The situation is different when language learning is considered from the perspective of appropriation. Indeed, in this framework, although the evaluation of plurilingual competence may10 still be possible, it can no longer be done through standards and descriptors designed a priori and according to the verticality of levels. In other words, working on the notion of language through the prism of the principle of diversity also implies working on the notion of evaluation through this same prism. Evaluating: understanding and interpreting

As we have already said, the question of assessing plurilingual competence is often asked in the light of the difficulties inherent in this act, or even in terms

238  Emmanuelle Huver of its impossibility (Huver, 2010). The relevance of this assessment is less often mentioned, although one might ask, with De Pietro et al. (2019): If the strictly speaking plurilingual aspects of plurilingual competence that are covered by multiple approaches are (…) an end in itself, of which proficiency could be measured? What would be, from this point of view, a “plurilingual” competence at A2 or C1 level? The question that J.-F. de Pietro is asking is, in my opinion, twofold, since it concerns both the relevance:

• to make plurilingual competence the object of assessment; • to reduce this assessment to a matching of levels using standardized descriptors.

The first part of this questioning was raised above. The other part actually refers to the evaluation design underlying the work on the evaluation of plurilingual competence. However, while the conception of language is generally not dealt with to any great extent, the conception of evaluation is not dealt with at all, even though this point is fundamental, especially in the perspective of epistemological coherence defended here. What does it mean, then, to “put diversity at the heart of evaluation”? The thesis I have been trying to defend for several years is that diversity is epistemologically evicted as soon as the assessment practices and tools are based on an imaginary of transparency and control, even if they are in several languages or if they integrate trans- and interlinguistic dimensions. However, this design of evaluation currently prevails in language didactics, including work on plurilingual competence. Many of titles of books devoted to the assessment of plurilingual competence emphasize transparency, the “quality approach” or the reliability of descriptors (cf., for example, Piccardo et al., 2019; Taylor & Weir, 2005). This discourse is so dominant in language didactics that it has become self-evident, making us lose sight of the fact that other conceptualizations of evaluation (could) exist, which are (would be) based on diametrically opposed principles (cf., for example, Ardoino & Berger, 1986 or Vial, 2001; cf. also the historical work of Desrosières (1993) on literary origin of statistic). They may be summarized roughly as follows:

• Evaluation depends most of all on the teachers’ and/or learners’ (and eval-

uators’) relationship to assessment (and more generally to diversity). To take a practical example and as I have already mentioned elsewhere (Huver & Springer, 2011; Huver, to be published), an assessment tool supposed to be very qualitative (such as the portfolio, for example) can be used in an extremely controlling way—just as, conversely, a monolingual control-type tool can be understood and invested in a different way. In other words,

Evaluation and plurilingual competence 239 practices and tools are based on representations and thus fall first under a “universe”, an imaginary, a way of conceiving the world. Therefore, for researchers and teachers’ trainers, the (reflexive) understanding of the universe in which the evaluator is involved is far more important than developing, advising, or prescribing tools. • The evaluative process is fundamentally hermeneutic rather than analytic. In analytical, objective evaluation, the quality of evaluation derives from the rigorous use of tools, from the objectivity of the evaluator and from the reproducibility of his or her judgement (see, for example, the way that certifying bodies define the quality of their tools). On the contrary, in a hermeneutic paradigm, the quality of assessment is based on the flexible use of rules, on the committed and sensitive presence of the evaluator and on the conflict of interpretations. More precisely, Romano (2015, p. 473), basing on Gadamer’s theory, defends the idea that the interpretation (in its hermeneutical sense) does not arise from the strict, exact, right use of a method, which is not in itself able to preserve from error. To the idea of “exact” rules that could be applied mechanically, Romano opposes “the (flexible) rules of experience”, which “rather require a use that itself has experience” (ibid., p. 475). The central status of experience (and reflexivity) implies to admit that there is a form of “pre-methodical” understanding that passes below, beyond and through rationalized understanding and that this “pre-methodical understanding” plays a fundamental role in the assessment process. And this applies a fortiori for plurilingual competence, which, is—when defined as below—also inherently qualitative, thus interpretative/hermeneutic. Considering evaluation from this angle is therefore the opposite of an evaluation based on a priori designed standards and ready-made psychometrically validated scales on the one hand and that delegitimizes the evaluators’ experience on the other. Conclusion: from epistemological consistency to ethical vigilance Under what conditions can we speak of effective assessment of plurilingual competence, or, more precisely, of effective plurilingual assessment? It is clear that developing tools that continue to evaluate languages in a juxtaposed relationship is not enough (and there is a consensus among researchers interested in this matter, including in this book). It is indeed on the basis of this observation that tools have been developed that try to take into account the inter- and trans-linguistic dimensions or the mediation issues inherent in any language contact. But if we wish to be consistent with the changes brought about by the notion of plurilingual competence in language didactics, developing “not monolingual assessment tools” is just a superficial way of dealing with this issue. It is also necessary (and from my point of view a priority) to question the notions

240  Emmanuelle Huver of language and evaluation by considering diversity as an epistemological and (therefore) political principle:

• on which underlies the interpretative (and therefore evaluative) judgement, considering that rigour is not to be found in the tools or in their (proper) use—in short, in the method and technique—but in the involvement of the evaluator and in the plurality of interpretations that a reflective posture must allow to make explicit and to confront; • on conceptions of what is called “language”, considering it not only as a heterogeneous social phenomenon, but also as experiences of the world, which cannot, by definition, neither be controlled nor technologized.

In my opinion, this shifts questions about what the social intervention of researchers in language didactics can mean and therefore presents ethical (and not only philosophical) issues from this point of view: In the field of teacher training: It is not about deciding instead of teachers what tools they should carry out—and therefore about training them to the proper use of these tools. It is not either about developing in them a “culture of evaluation” (Cuq & Davin, 2016), which would suggest that only one (homogeneous) culture would be valid and that some of the teachers would be, a contrario, “evaluation uneducated”. Languages assessment training in which diversity is a priority means working on the basis of the evaluators’ experiences and representations, in order to work primarily on becoming aware of one’s collective and individual stakes and imaginaries on language and assessment—in order to question them or, on the contrary, to better understand why they are important to them. Therefore, for the researchers, thinking about a form of responsible social intervention is less about creating tools than fostering reflexivity by teachers, based on the diversity of their positions and fundamentally respectful of this diversity. In the field of research: It is common to hear or read that the provision of level reference systems or tools for assessing plurilingual competence would promote its teaching (Goullier, 2017, p. 256). The responsibility of researchers who wish to promote this plurilingual turning point would then lie in developing and disseminating such devices. From the perspective I have tried to defend here, the responsibility of researchers is based on a need for consistency, not only between teaching and assessment practices but also between the concepts that underlie these practices. Since plurilingual competence is rooted in the notion of diversity, as defined here, it is indeed this notion that must serve as an epistemological thread to think evaluation and language notions and, on this basis, evaluative practices. Without this epistemological coherence, the evaluation of plurilingual competence is condemned to assess only a superficial plurilingualism. And it is by firmly and insistently placing diversity at the centre of the evaluation process

Evaluation and plurilingual competence 241 that we can guard against some manipulations of the tools created, since diversity introduces a form of relativity that excludes any expert positioning. Thus, given that language is defined as an experience of the world and its learning as an experience of contact and otherness, language assessment (but also language teaching/learning) cannot (epistemologically) be understood solely from a technical point of view and should not (ethically) be so. Placing diversity at the heart of the assessment of plurilingual competence is therefore above all to think of epistemological coherence as a form of ethical vigilance. Notes 1 This language choice lies on the fact that, on the one hand it is my main field of reflection and that, on the other hand, it seems important to me that these elements, which are often little known or little cited by the English-language research literature, can be disseminated. 2 I hereby thank Katia Schuchman, who took in charge the translation of this text. Of course, I am solely responsible for the content and the possibly remaining mistakes. 3 For more details on this matter, see, among others, Coste (2002), the work of V. Castellotti and D. Moore (for example, Castellotti and Moore 2011) or Huver (2014). 4 These two trends are of course not completely mutually exclusive, even if the critical tendency is undoubtedly more recent. This one is based indeed on multidisciplinary works on the ideology of evaluation published over the last 15 years as a part of a more general movement denunciating the excesses of neo-liberalism. 5 Harmos is a Swiss intercantonal agreement on the harmonization of compulsory schooling between the different Swiss cantons, approved by referendum in 2006. Among other things, it includes national educational standards in various school subjects, including foreign languages and schooling. For more details, see Lenz (2006). 6 The newly introduced notions of “translanguaging” or “superdiversity” do not qualitatively change this reflection. 7 See also a number of my works over the past ten years). 8 “A system of gradation that was first conceived and produced in relation to one language remains inscribed in a monolingual logic of continuous progress appreciated throughout according to the same criteria. Such a system simply cannot accommodate linguistic and cultural plurality in its plurilingual aspect and in the discontinuities, the recompositions, the modifications of balance between the different components of the repertoire that the ‘ordinary’ evolution of the plurilingual and pluricultural capacities of any social actor presents”. 9 “Appropriation” refers here not to possession and mastery, but to an existential, experiential and relational process integrating deviations, contingencies, mixtures (the “ghosts” of other languages, to use Py’s (1992) beautiful metaphor) and posing as central the place [that language] occupies in the history and the project of the person who engages in this appropriation” (Castellotti, 2017, pp. 43–45). 10 But can we still speak of “competence” in this case? And isn't all language competence in fact plurilingual?

References Anderson, P. (1999). La didactique des langues étrangères à l’épreuve du sujet. Besançon: Presses universitaires franc-comtoises.

242  Emmanuelle Huver Anquetil, M., Derivry, M., & Gohard Radenkovic, A. (2017). « En finir avec le Je contraint et réifié dans l’objet PEL: pour une didactique de la biographie langagière comme processus relationnel ». TDFLE, 70, La pensée CECR, http://revue-tdfle. fr/les-numeros/numero-70/23-en-finir-avec-le-je-contraint-et-reifie-dans-l-objetpel-pour-une-didactique-de-la-biographie-langagiere-comme-processus-relationnel. Ardoino, J., & Berger, G. (1986). L’évaluation comme interprétation ». POUR, 107, 120–127. Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press. Blanchet, P. (2016). Discriminations: combattre la glottophobie. Paris: Textuel. Butera, F., Buchs, C., & Darnon, C. (2011). L’évaluation une menace? Paris: PUF. Castellotti, V. (2017). Pour une didactique de l’appropriation. Diversité, compréhension, relation. Paris: Didier. Castellotti, V., & Moore, D. (2011). La compétence plurilingue et pluriculturelle: genèse et évolution d’une notion-concept. In P. Blanchet & P. Chardenet (Eds.), Guide pour la recherche en didactique des langues et des cultures. Approches contextualisées (pp 241–252). Paris: Editions des Archives Contemporaines. Chardenet, P. (2005). Evaluer des compétences plurilingues et interlingues. https:// www.academia.edu/5354831/Evaluer_des_comp%C3%A9tences_plurilingues_et_ interlingues_2005. Cheng, L., & Fox, J. (Eds). (2017). Assessment in the language classroom: Teachers supporting student learning. London: Palgrave. Conseil de l’Europe. (2018). Cadre européen commun de référence pour les langues: apprendre, enseigner, évaluer – Volume complémentaire avec de nouveaux descripteurs. Strasbourg: Éditions du Conseil de l’Europe, https://rm.coe.int/cecr-volumecomplementaire-avec-de-nouveaux-descripteurs/16807875d5. Coste, D. (2002). « Compétence à communiquer et compétence plurilingue ». In V. Castellotti & B. Py (coord.), dans: Notions en questions, no.6 (pp. 115–123), « La notion de compétence en langue ». Lyon: ENS Editions. Coste, D. (2021). De Rüschlikon au Volume complémentaire ou Du risque qu’il y a à passer sous les échelles ». In K. Vogt & J. Quetz (Eds.), Der neue Begleitband zum Gemeinsamen europäischen Referenzrahmen für Sprachen (pp 35–45). Berlin: Peter Lang. Cuq, J.-P., & Davin, F. (2016). « L’évaluation, un paramètre prépondérant en didactique du français langue étrangère et seconde ». In L’évaluation à la lumière des contextes et des disciplines (91–110). Bruxelles: De Boeck. Davis, J. M., Norris, J. M., Malone, M. E., & McKay, T. H. (Eds.). (2018). Useful assessment and evaluation in language education. Washington, DC: Georgetown University Press. Debono, M. (2020). La question de la technique en didactique des langues et en sociolinguistique (francophones). Habilitation à diriger des recherches, Grenoble: Université Grenoble Alpes. De Pietro, J.-F. (2019). « Faut-il évaluer la « compétence plurilingue »? Quelques réflexions à partir du Cadre de référence pour les approches plurielles (CARAP). In L. Gajo, J.-M. Luscher, I. Racine, & F. Zay (Eds). (2019). Variation, plurilinguisme et évaluation en français langue étrangère. Bern: Peter Lang. Desrosières, A. (1993). La Politique des grands nombres – Histoire de la raison statistique. Paris: La Découverte.

Evaluation and plurilingual competence 243 Gajo, L., Luscher, J.-M., Racine, I., & Zay, F. (Eds). (2019). Variation, plurilinguisme et évaluation en français langue étrangère. Bern: Peter Lang. Goullier, F. (2017). « Evaluer la compétence plurilingue et interculturelle? ». In J.-C. Beacco & D. Coste (Eds.), L’éducation plurilingue et interculturelle: la perspective du Conseil de l’Europe. Paris: Didier. Hidri, S. (Ed.). (2018). Revisiting the assessment of second language abilities: From theory to practice. Cham: Springer. Huver, E. (to be published). “Favoriser l’autonomisation par l’évaluation: s’autonomiser de l’évaluation?”. In P. Candas, P. Acker & D. Toffoli (Eds.), Autonomisation et évaluation. Lausanne : Peter Lang. Huver, E. (2010). « “J’ai pas pu corriger parce que c’est le mélange des langues” L’évaluation de la compétence plurilingue entre résistances et tensions. Recherches en didactique des langues et des cultures, 7(2), http://journals.openedition.org/ rdlc/2144. Huver, E. (2014a). De la subjectivité en évaluation à une didactique des langues diversitaire. Pluralité, altérité, relation, réflexivité, Note de synthèse d’habilitation à diriger des recherches, Université F. Rabelais (Tours), https://hal.archives-ouvertes. fr/tel-01211154v1. Huver, E. (2014b). « Les inaccessibles en évaluation. Impensé? Impasse? Ferments? ». In C. Goï, E. Huver, & E. Razafimandimbimanana (coord.), « Inaccessibles, altérités, pluralités: trois notions pour questionner les langues et les cultures en éducation ». Glottopol, 23, http://glottopol.univ-rouen.fr/telecharger/numero_23/ gpl23_03huver.pdf Huver, E., & Macaire, D. (2021). « Didactique de langue, didactique des langues, didactique du plurilinguisme: évolutions, enjeux, questions ». Recherches en didactique des langues et des cultures, 18(2), https://journals.openedition.org/rdlc/9673. Huver, E., & Springer, C. (2011). L’évaluation en langues. Paris: Didier. Kunnan, A.-J.(Ed.). (2014). The companion to language assessment. Hoboken : John Wiley & Sons. Lenz, P. (2006). Überlegungen zur Sprachkompetenzbeschreibung und Testvalidierung im Projekt HarmoS/Fremdsprachen. https://www.researchgate.net/publication/ 42384750_Uberlegungen_zur_Sprachkompetenzbeschreibung_und_Testvali dierung_im_Projekt_HarmoSFremdsprachen. Lenz, P., & Berthele, R. (2010). Assessment in Plurilingual and Intercultural Education, Conseil de l’Europe, Strasbourg. http://www.coe.int/t/dg4/linguistic/List Docs_Geneva2010.asp#TopOfPage. Madaus, G. F. (1990). Testing as a social technology. Boston: Boston College. Maurer, B., & Puren, C. (2019). CECR: par ici la sortie!. Paris: Éditions des archives contemporaines. https://eac.ac/books/9782813003522. Mc Namara, T., & Roever, C. (2006). Language testing: The social dimension. Oxford: Blackwell Publishing. Médioni, M.-L. (2016). L’évaluation formative au cœur du processus d’apprentissage. Lyon: Chronique sociale. Ollivier, C. (2019). « Évaluer les compétences en communication plurilingue – le cas de l’intercompréhension ». Les Langues Modernes, 4, « Évaluation et certification en langues: tensions, évolutions, perspectives ». Piccardo, E., North, B., & Maldina, E. (2019). Innovation and Reform in Course Planning, Teaching, and Assessment: The CEFR in Canada and Switzerland, A Comparative Study. Canadian Journal of Applied Linguistics, 22 (1), 103–128.

244  Emmanuelle Huver Prieur, J.-M., & Volle, R.-M. (2016). Le CECR: une technologie politique de l’enseignement des langues ». Educations et sociétés plurilingues, 41, 75–87. Py, B. (1992). « Acquisition d’une langue étrangère et altérité ». Cahiers de l’ILSL, 2, 113–126. http://www.unil.ch/webdav/site/clsl/shared/CILSL02.pdf. Romano, C. (2010). Au cœur de la raison: la phénoménologie. Paris: Gallimard, Folio. Romano, C. (2015). « La règle souple de l’herméneute ». Critique, 817–818, 464–479. http://www.cairn.info/revue-critique-2015-6-page-464.htm. Shohamy, E. (2001). The power of tests. A critical perspective on the uses of language tests. Essex: Pearson Education. Shohamy, E., Or, I. G., & May, S.(Eds.). (2017). Language testing and assessment. Cham: Springer. Soubre, V. (2021). L’agir évaluatif partagé: une ressource dynamique pour l’enseignement et l’apprentissage des langues étrangères dans un contexte d’enseignement supérieur, thèse de doctorat, université de Genève. https://archive-ouverte.unige.ch/unige:150813. Spolsky, B. (1995). Measured word. The development of objective language testing. Oxford: Oxford University Press. Taguchi, N., & Kim, Y. (Eds.) (2018). Task-based approaches to teaching and assessing pragmatics. Amsterdam: John Benjamins Publishing Company. Taylor, L., & Weir, C.-J. (Eds.) (2005). Multilingualism and Assessment: Achieving Transparency, Assuring Quality, Sustaining Diversity—Proceedings of the ALTE Berlin Conference, May 2005. Vial, M. (2001). Se former pour évaluer, Se donner une problématique et élaborer des concepts. Bruxelles: De Boeck.

Conclusion Somewhere over the rainbow … a place for multilingual assessment? Sílvia Melo-Pfeifer and Christian Ollivier

This volume brings together contributions of diverse nature to the cause of changing assessment practices. Some have raised ethical issues, others theoretical, political and/or pedagogical. Contributions have focused on two aspects: assessment of plurilingual learners (usually new arrivals in the education systems of host countries) and assessment of plurilingual competence (whether or not developed in formal learning contexts). The chapters of this volume thus address various dimensions of these two aspects, giving a rich overview of the state of the art in these two domains and showing the diversity of related research. Going through the chapters of this book, the reader will perceive a broad consensus among its authors. Most chapters state that assessment – and in some contexts even language policy – remains largely monoglossic and therefore stress the critical need for multilingual assessment. On the one hand, multilingual assessment would align with a widespread but not always implemented pedagogical goal in language education, that is the promotion of plurilingual competence. On the other hand, multilingual assessment of so-called nonlinguistic contents would provide more social justice for plurilingual individuals by giving them the opportunity to use and value their full repertoire and to show to a fuller extent their academic knowledge and competences. Chapters are based on different concepts, such as superdiversity and translanguaging. The concept of superdiversity was coined by Blommaert and Vertovec to describe the contexts where multilingual assessment seems to be most needed (used by D’warte & Slaughter as well as Cavalli). Translanguaging, a term that highlights the interwoven nature of languages with other semiotic resources needed in and for assessment, is referred to by several of the authors (Ascenzi-Moreno, García & López; Hinger; Le Pichon, Naji, Wattar & Lee; Liddicoat & Scarino; De Angelis & Stavans; and Piccardo & North). Some chapters provide theoretical reflections (Cavalli; Liddicoat & Scarino; Hinger; Piccardo & North) or a state of the art in particular domains (Brinkmann & Melo-Pfeifer on the use of portfolio for multilingual assessment). Others discuss practical, theoretical and epistemological challenges regarding assessment in general and plurilingual assessment in particular (such as Huver; Cavalli; and De Angelis & Stavans). Some authors present

246  Sílvia Melo-Pfeifer and Christian Ollivier innovative – sometimes disruptive – avenues for the practice, through the use of digital technologies (Le Pichon, Naji, Wattar & Lee) or other means (De Angelis & Stavans). Two critical contributions ask fundamental questions regarding (multilingual) assessment (Cavalli; Huver). Two chapters, by Bonvino, Capucho and Strasser and by De Carlo and Andrade, provide detailed, concrete practices based on a theoretical framework called “intercomprehension”. The diversity of the volume is also a diversity of perspectives and contexts: Australia (D’warte & Slaughter), Europe – including Switzerland (Bleichenbacher, Schröder-Sura, Chesini, De Zanet, Gantenbein & Hilbe), Belgium (De Backer, Vantieghem & Van Avermaet) and France (Audras) – USA (Ascenzi-Moreno, García & López) and Canada (Le Pichon, Naji, Wattar & Lee). The chapters address various educational levels, from primary school (De Backer, Vantieghem & Van Avermaet) to teacher education (Bleichenbacher, Schröder-Sura, Chesini, De Zanet, Gantenbein & Hilbe). Most of the chapters also deal with specific target audiences and contexts: migrants (Audras; Brinkmann & Melo-Pfeifer), LatinX in the USA (Ascenzi-Moreno, García & López), future teachers (Bleichenbacher, Schröder-Sura, Chesini, De Zanet, Gantenbein & Hilbe), among others. Through this thematic, contextual, theoretical, practical, epistemological “superdiversity”, the volume gives fascinating insights into the prolific and ambitious research on multilingual assessment. Nevertheless, despite the advances brought by this volume to the field, especially in terms of (designing) pedagogical practices, some dimensions remain to be elaborated and may become the catalyst of future developments and research. In the introduction to this volume five scenarios of multilingual assessment within educational settings were presented: assessment in and of a specific target language resorting to plurilingual competence principally acquired in an educational context, assessment of content in non-linguistic subjects resorting to multilingual educational settings such as CLIL scenarios, assessment in the language classroom using plurilingualism acquired through mobility, assessment of school content in non-linguistic subjects resorting to plurilingual competence acquired through mobility and holistic assessment of plurilingual competence. As we explain in the introduction, this volume focuses especially on the latter two scenarios. We therefore suggest that further developments in the remaining three fields are necessary, not only to disseminate forms of multilingual assessment but also to normalise multilingualism as a starting point to learning and teaching, as a resource and as an outcome of those processes, inside and outside educational systems. Among the open questions, we think that an analysis of how curricula align with political demands formulated at national and supra-national levels and with research results and evidence-based recommendations would require more attention from scholars. For instance, we saw in the introductory chapter that the development of individual plurilingual competence and its assessment are part of European language policies. However, if we consider the practical operationalisation of those language policies in local, regional and national

Conclusion 247 terms, there is still a monolingual and additive understanding of this competence, with the result that assessment (both of the competences of plurilingual learners and of the plurilingual competence itself) remains hostage to this monolingualised and monoglossic understanding. To this monolingualised understanding we add evaluation practices that are mostly still text-based only (in reception and/or production formats), disregarding multiliteracy practices in the contemporary world. One issue to be addressed is therefore the transformation of the curricula and programmes of different subjects (language subjects in particular) in order to make room for multilingual and multisemiotic assessment formats and instruments that create space for translanguaging. Another unresolved issue concerns the study of (language) teachers’ beliefs and practices in relation to how they consider and value learners’ plurisemiotic and plurilingual repertoires and how they assess the plurilingual competence. Knowing that teachers’ beliefs are dynamic, though resistant to change, forged during their own educational biographical trajectories, it becomes unavoidable to work on and with teachers, both in the diagnosis of beliefs and in the observation and discussion of multilingual assessment practices. The implementation of practices which value students’ plurilingual repertoires cannot be achieved without teachers, who need diagnostic and assessment tools, among other things. In this sense, it is necessary to diagnose teachers’ beliefs and pedagogical needs, addressing those needs both in initial and continuing teacher education programmes. Teachers are key agents of change and the backbone of educational transformation (Menken & García, 2010). Some of the issues this volume did not address in more depth refer to the need for assessment itself, the functions of assessment and the impacts of assessment on the lives of the assessed, whether in monolingual or multilingual formats. These reflections are well documented in the literature (Shohamy, 2001, 2022, for example). One unanswered question is whether multilingual assessment can (or should) follow the culture of monolingual assessment – based on monolingual assessment instruments, with their stereotypical assessment formats, mostly text-based, relying on ready-made assessment criteria. It is also important to consider whether the times and spaces for multilingual assessment, particularly at school, may follow the same patterns as the usual school structure and cultures: assessment in predetermined periods and prior to the teaching and learning processes themselves, final assessment based on the individual performance of each student (without considering, for example, elements of shared cognition), assessment based on an instrument constructed in a top-down manner, imposed on the assessed. The previous questions make us wonder about the transformative potential of adopting multilingual assessment formats over a rather established and stifled assessment and school culture. Can we conceptualise assessment as diagnostic, formative and summative, as is generally done in school assessment culture? Can we change ideologies associated with assessment in educational institutions and rethink the supposed inevitability, necessity and fairness of assessment practices? Is multilingualism in assessment scalable to all students,

248  Sílvia Melo-Pfeifer and Christian Ollivier schools and educational systems and achievable for example at the level of international and internationally recognised certifications? So, another open question is the massification of multilingual assessment itself: can assessment of plurilingual learners and plurilingual competence be massified on the same scale as, let’s call it, traditional assessment – bringing “multilingual assessment for all”? In view of the diversification of school populations in different contexts and of language learning (at least in the European context), the answer could tend towards a “yes”. For example, in the field of intercomprehension, the EVAL-IC project (see chapters by Bonvino et al. and De Carlo & Andrade) has shown that this could be possible by pioneering the development of an assessment protocol as the basis for an interuniversity certificate. We might then wonder what contextual characteristics (geopolitical or others) and what features of education systems are more permeable and favourable to the conceptualisation and mainstreaming of multilingual assessment practices. By disseminating assessment theories and approaches anchored in a multi/ plurilingual and heteroglossic paradigm, this volume may open avenues for the development and implementation of more diverse (and disruptive) assessment practices, and new grounds for research on multilingual assessment that engages both with evaluators and with the evaluated. We think that the implementation of multilingual assessment practices can become a fertile research area, renewing how assessment is thought and researched, eventually by revisiting the very scientific language it uses. As editors of this volume, there were more than a few times when we got stuck on the terms to use, reflecting upon the terms we wanted to use and the ones we wanted to break ties with. The reader may have noticed some fluctuation, throughout the volume, between the use of “multilingual” and “plurilingual”, for example. In this conclusion, we have chosen to use plurilingual to refer to individuals (with a plurilingual competence, as defined in the Introduction) and multilingual to refer to assessment practices. At some points we noticed an overlapping of these two concepts in the argumentation. Thus, the very epistemological impacts of a renewal of assessment practices, in the sense of pluralising and multilingualising them, are also still an open question which deserves to be highlighted in future work. Among the questions and doubts we had during the coordination process of this volume was the issue of the structure of the book. At the beginning of the project we thought to divide it into two parts: one addressing theoretical aspects and the other presenting and discussing concrete examples of implementation in various settings. Looking at the abstracts sent by the authors and later at the full chapters, it appears that the planned structure was not a feasible option because many contributions brought forward both theoretical and practical dimensions: this is perhaps due to the fact that different academic traditions and different teaching and learning cultures, developed in specific historical and geopolitical contexts, frame multilingual assessment differently, according to situated ideological, political, epistemological and pedagogical worldviews. In a highly context-sensitive research and action field, with strong

Conclusion 249 political and ideological dimensions, this is not a surprise. Thus, we came to realise, the main distinction that remains is between assessment of plurilingual competence and assessment of plurilingual individuals (both of which can be done through either monolingual or multilingual assessment formats). Even if we decided, on the basis of this observation, to divide the volume into two parts, each one dealing with one of these aspects, by editing this book, we wanted to bring together scholars working in different but strongly related fields, contributing to a greater porousness between the two fields of research. Reading the contributions in this volume we can notice that researchers working on assessment of plurilingual competence often know and refer to concepts developed by researchers working on assessment of plurilingual individuals, but the opposite is rarely true. We hope that this volume will contribute to opening and strengthening dialogue between researchers who, beyond possible epistemological differences, share common objectives and values such as the promotion of plurilingualism, cognitive equity and social justice. References Menken, K., & García, O. (Eds.). (2010). Negotiating language policies in schools. Educators as policymakers. London: Routledge. Shohamy, E. (2001). The power of the tests. A critical perspective on the uses of language tests. Essex: Pearson Education Limited. Shohamy, E. (2022). Critical language testing, multilingualism and social justice. The Modern Language Journal, 56(4), 1445–1457. https://doi.org/10.1002/ tesq.3185.

Index

Note: Page references in italics denote figures and in bold tables and with “n” endnotes. accommodation: bilingual tests 101; direct linguistic 99; effectiveness of 93, 94; language testing 92–101; linguistic 94; read-aloud 94, 96–97, 99–100, 101; types of 93–94 action-oriented paradigm 179 AFaLaC (Association Famille Langues Cultures) 119–122 Anzaldúa, G. 49 appropriation 237, 241n9 Armitage, J. 67 Ascenzi-Moreno, L. 3, 17, 49 Assessment Reform Group: Assessment for Learning: Beyond the Black Box 179 Atout Langues [Language Assets] 119–121, 123 Australia: 1987 National Policy on Languages (NPL) 63, 64; Australian Language and Literacy Policy (ALLP) 64; Framework for Aboriginal Languages and Torres Strait Islander Languages 70; Indigenous Language as a Second language (ILSL) programme 70; National Assessment Program – Literacy and Numeracy (NAPLAN) test 64–66, 69; plurilingual repertoires, examination of 62–71 Baker, B. 106 Basic Language Cognition (BLC) 110 Bauer, E. B. 53 Baur, R. 108 Berthele, R. 233 bilingualism 31; Latinx bilingual students 50; multiple acquisitions from 147–160

Binogi 78–81, 84, 85 BLC see Basic Language Cognition (BLC) Blömeke, S. 173 Blommaert, J. 245 Bourdieu, P.: critical sociology 231 Briceño, A. 53 Busch, B. 187 Butler, F. A. 100 Buzick, H. 94 Cadre de référence pour les approches plurielles des langues et des cultures (CARAP)/Framework of reference for pluralistic approaches to languages and cultures: competences and resources (FREPA) 5, 12, 163, 164, 166, 185, 195, 227 Capucho, F. 207 CARAP/FREPA see Cadre de référence pour les approches plurielles des langues et des cultures (CARAP)/Framework of reference for pluralistic approaches to languages and cultures: competences and resources (FREPA) Cassany, D. 221 Castellotti, V. 237, 241n3 CEFR see Common European Framework of Reference for Languages (CEFR) Cenoz, J. 7, 14, 132, 184, 208 Certificate of Plurilingualism 188 certification assessment 155 Chardenet, P. 117 CLIL see Content and Language Integrated Learning (CLIL) Cloze tests 107 co-evaluation 155 colonialism/coloniality 49

Index  251 Committee of Ministers: Recommendation CM/Rec(2014)5 7 Common European Framework of Reference for Languages (CEFR) 1, 2, 6, 13, 18, 109, 120, 125–127, 156, 179, 182, 195, 204, 207, 217, 218, 221, 222, 226, 234; Companion Volume 5, 12, 164, 183–188, 227, 230, 233; on language proficiency 38; on plurilingual competence 32–33; scales 13 Complementarity Principle 155 compulsory education 154–156 connoisseurship 141 Content and Language Integrated Learning (CLIL) 37, 246 Coste, D. 127, 241n3 Council of Europe 6, 152, 158, 227n3; Languages in/for Education project 219; on multilingualism 32 Creagh, S. 65 critical language testing 4 critical sociology 231 cross-linguistic mediation 2, 4, 186, 188 crosslinguistic teacher education 163–176 Csire, M. 222 C-test format, for multilingual language speakers 107–112 culture of change 117 Cummins, J. 1, 88, 156, 186–187 Dawkins, J. 64 De Angelis, G. 7–8, 35, 40 De Backer, F. 178, 180 Deleuse, J. 218 Dendrinos, B. 188 de Pietro, J.-F. 238 Desrosières, A. 238–239 diagnostic assessment 155 Diego-Hernández, E. 212 DIF see differential item functioning (DIF) differential item functioning (DIF) 109 Disbray, S. 69 discrimination: linguistic 8–10 Dmitrenko, V. 220 Drackert, A. 107–111 Dutton, J. 67–68 dynamic assessment 180, 181 EAL/D see English as an Additional Language/Dialect (EAL/D) Eckes, Th. 111

El Euch, S. 32 ELL see English Language Learners (ELL) ELP see European Language Portfolio (ELP) English as an Additional Language/ Dialect (EAL/D) 63–64, 69, 70 English Language Learners (ELL) 93 epistemological consistency 239–241 Estaire, S. 163 Europe: political will in 5–7 European Centre for Modern Languages: ‘Inspiring innovation in language education: changing contexts, evolving competences’ 118–119 European Language Portfolio (ELP) 13, 175–176, 182, 233, 234; for assessment of plurilingual students in foreign language classroom 216–227; competences assessed 221–223; definition of 217–219; formative and self-assessment practices 223–225; Integrated Ireland Language and Training 222–223; principles of 217–219; and promotion of plurilingualism 219–221; research settings 221–223 European Language Portfolio: A Guide for Teachers and Teacher Trainers, The 220 European Union 33 EVAL-IC (Evaluation des compétences en intercompréhension) project 15, 16, 20, 194–201, 205, 207–210, 212, 213, 248 Expressive Vocabulary Test 38 Fielding, R. 67 Fiorenza, E. 212 Flores, N. 70, 183, 184 formative assessment 2, 52, 155, 189, 206, 224–225 Framework of reference for pluralistic approaches to languages and cultures: competences and resources (FREPA) 5, 12, 163, 164, 166, 185, 195, 227 France: School Reform Act (2013) 119, 120 French, M. 67 functional multilingualism, in educational assessment 92–102; framework 99–101; literature review 93–94; participants 95–96,

252 Index 96; procedures 95–96; read-aloud accommodated and bilingual tests, limited effectiveness of 97–99, 98, 99; read-aloud accommodation in the L1 97, 98; read-aloud accommodations in LoS 96–97, 97; research context 95; research questions 94–95; written bilingual test 97, 98 Functional Multilingual Learning (FML) 101 functional plurilingualism 164–165; in practice 167–174 Galante, A. 187 García, O. 1, 3, 52, 53 Gathercole, M. 14 Gipps, C. V.: Beyond Testing 179 Gogolin, I. 107, 110–111 Gorter, D. 7, 14, 132, 184, 208 Gottlieb, M. 10–11 Goullier, F. 233 Grosjean, F. 152, 157, 159, 160n6 Grotjahn, R. 108, 110, 111 Guide for the development and implementation of curricula for plurilingual and intercultural education 6, 7 Guilherme, M. 160nn2 Guzman-Orth, D. A. 178 Halliday, M. A. K. 134 Hansen-Pauly, M.-A. 222 Harmos 241n5 Heritage Languages (HL) 159 Heugh, K. 94, 100, 178 Higher Language Cognition (HLC) 110 Hofer, B. 133 holistic assessment 2, 4, 13–16, 181, 212, 246; plurilingual communication competence, integrated assessment of 15–16; through translanguaging 14–15 Hope, A. 106 Hülsmann, C. 202n2 Hulstijn, J. H. 110 Huver, E. 117, 241n3 hybrid assessment: calculated 14; profile 13–14 IC see intercomprehension (IC) ILSL see Indigenous Language as a Second language (ILSL) programme Indigenous Language as a Second language (ILSL) programme 70

Integrated Ireland Language and Training 222–223 integrated language teaching 2 intercomprehension (IC) 2, 13, 15, 19, 20, 166, 169–172, 185–187, 194–200, 204–213, 225, 233, 246, 248; competence, framework for 195–198; competences in coherence with plurilingual approaches, assessment of 204–213; dimensions of 195–196; interactive 196; nonverbal/paraverbal factors of 197; plurilingual project week 169–171, 171; receptive 196; scenario-based assessment 198–200 intercultural competence 1 interculturality 131–143 Ireland: Integrated Ireland Language and Training 227n5 Jamet, M.-C. 13 Jentges, S. 187 Jessner, U. 133 Jones, N. 101 Kieffer, M. J. 94 Kleifgen, J. A. 53 Klein, A. 53 Klinger, T. 111 Kramsch, C. 33 Kristmanson, P. 224 Kubota, R. 69 LA see language awareness (LA) Laakso, J. 222 language awareness (LA) 2, 166, 233 Language Background Other Than English (LBOTE) 65 language of schooling (LoS) 92, 94, 95, 98, 100–102, 159, 180, 184; read-aloud accommodations in 96–97, 97 language proficiency 38 language separation 3 languages market 117 languaging 50; translanguaging see translanguaging Lankina, O. Y. 182 Latinx bilingual students’ translanguaging and assessment 48–58; bilingualism 50; content assessment 54–57, 56; Language and raciolinguistic ideologies 49–50; literacy assessment 52–54 Lazenby Simpson, B. 221

Index  253 LBOTE see Language Background Other Than English (LBOTE) learning as a process of meaning making 134 learning-oriented assessment 180 Lenz, P 233, 241n5 Le Pichon, A. 87 Lin, A. M. Y. 1, 3 LINCDIRE see Linguistic and Cultural Diversity Reinvented (LINCDIRE) project Linguistic and Cultural Diversity Reinvented (LINCDIRE) project 188–189 linguistic diversity 236–239 linguistic profiling 9, 10 linguistic repertoire 1, 2, 4, 8, 14, 32, 53, 66, 68, 101, 106, 117, 118, 121, 127, 132, 164, 178, 180, 182, 205, 212, 220 linguistic rights 150–151 Little, D. 182, 216, 217, 223 Llorente Puerta, M. J. 221–222 López de la Torre, J. A. 217 LoS see language of schooling (LoS) MAGICC (Modularising multilingual and multicultural academic communication competence) 195 Maturana, H. 49–51 McNamara, T. 8 MEZ see Multilingual Development (MEZ) Mihai, F. M. 93 Miriadi (Mutualisation et innovation pour un réseau de l’intercompréhension à distance; Projet Miriadi) 195, 205 monolingual mindset in assessment, pervasiveness of 1–5 Moore, D. 241n3 Mu-LAE-project (Multilingual Assessment in Education) 95 multilingual-by-design test 40 multilingual-by-translation test 40 Multilingual Development (MEZ) 110–111 multilingualism 31, 63, 69; definition of 32; distinguished from plurilingualism 32–34; functional 92–102; individual 32–34, 227n1; societal 32 multilingual perspective in language assessment, development of 131–143; culture, role of 133–134; elaboration 134–136; example 136–138; issues and considerations 138–142

multilingual speakers, language competences of 106–114; C-test format 107–112; task-based language assessment 112–113 multilingual testing and assessment 31–44; contexts 41–42; evaluators 35–36; knowledge assesment 37–39; location 42–44; method 39–41; reasons for 36–37; terminological issues 31–34; test writers 32–33 multiple acquisitions from bi-/ plurilingual learning, assessment of 147–160 NAPLAN see National Assessment Program – Literacy and Numeracy (NAPLAN) test National Assessment Program – Literacy and Numeracy (NAPLAN) test 64–66, 69 New York State Bilingual Common Core Initiative 183 Norris, J. M. 109–111 Ollivier, C. 16, 201n2 Ontario Ministry of Education 78 Oskarsson, M. 217 PANAVA model 171 Pappamihiel, N. E. 93 Pavlovskaya, I. Y. 182 Peabody Picture Vocabulary Test (PPVT) 38, 39 peer evaluation 155 Piller, I. 9 PISA see Program for International Student Assessment (PISA) pluralistic approaches 5, 166, 173, 205, 207, 213 pluricultural assessment, supporting 182–185 plurilingual assessment 4–8, 13, 16, 20, 88, 163, 175, 185, 189, 198, 234–236, 239, 245 plurilingual communication competence, integrated assessment of 15–16 plurilingual competence 1–3, 10, 152; assessment of 6, 12–16; challenges associated with 154–158; compulsory education and 154–156; critical trend of 231–232; definition of 1, 164; development of 5; evaluation of 230–241; integrated approach to 194–201; linguistic diversity 236–239;

254 Index model of 158; options for assessment 187–188; similar structuring 232–236 plurilingual curriculum 165–167, 166 plurilingual comprehension 185 Plurilinguale Ausbildung von Lehrpersonen 167 plurilingual individuals, assessment of 7–12; politics and practices 8–10 plurilingualism 3–6, 11, 15, 70; acceptance of complexity 180–182; assessment of 178–189; construct for assessment 185; definition of 152–153, 164; distinguished from multilingualism 32–34; functional 164–165; hierarchies 10; languages and 153–154; self-assessment in 163–164; support for assessment 182–185; sustainability of 172–173 plurilingual pedagogies 4, 20, 63, 68, 70, 187, 189, 227 plurilingual person, definition of 152–153 plurilingual project week 167–174; can-do checklists 168–169; reactions to tasks and outcomes, capturing 171–172, 172; reflection questions 169–171, 171; self-assessment of 168; students’ resolutions 172–173; task spider diagram 173–174, 174 plurilingual repertoire 1, 3–6, 11, 12, 16, 17, 33, 119, 154, 182–187, 198, 206, 224, 226, 247; assessment scenario 207–209; examination of 62–71; language and literacy policies 63–66; monolingual mindset, mitigation of 68–70; multiple meaning making opportunities 66–67; results analysis 209–212, 210 plurilingual secondary school students, language skills of 116–128; AFaLaC Langues 119–122; Atout Langues 119–121; effect on the students 122–123; effects on the educational stakeholders 123–124; evaluators as crucial link in relationship with evaluatee 126–127; evaluatorsmediators and evaluated students, relationship between 124–126; methodological design 120–121; participants 121–122; research context and purpose 118–119; theoretical framework 116–118

pluri-monolingualism 3 political will, in Europe 5–7 PPVT see Peabody Picture Vocabulary Test (PPVT) Program for International Student Assessment (PISA) 40, 64, 99, 106, 113 Py, B. 241n9 Quijano, A. 49 raciolinguistic ideologies 49–50 Recolang project 118 REFDIC (Référentiel de compétences en didactique de l’intercompréhension) 205 REFIC (Référentiel de compétences de communication plurilingue en intercompréhension) 205 responsive adaptations 53–54 Rivera, C. 93 Roever, C. 108 Romano, C. 239 Rushton, K. 67–68 Sadler, D. R. 141 Saville, N. 101, 178, 185 Scarino, A. 134 scenario-based assessment 198–200 Schalley, A. C. 64 Schissel, J. L. 3, 70, 107, 112, 114, 133, 178, 182–184 Seed, G. 185 self-evaluation 155 Shohamy, E. 8, 11, 15, 94, 100, 131–133, 178 Sierens, S. 101 social justice 10, 16, 132, 187, 245, 249 Solano-Flores, G. 57 Spettmann, M. 108 Spolsky, B. 156, 160 Stansfield, C. W. 93 Stathopoulou, M. 133, 188 Steinhuber, B. 188 Stone, E. 94 Strasser, M. 201n2 summative assessment 2, 155, 206 Summit Declaration of the Third Summit of Heads of State in Government in Warsaw (2005) 219 superdiversity 151, 245 Tabouret, A.-K. 160n4 Tagliante, C. 218

Index  255 task-based language assessment (TBLA): for multilingual language speakers 107, 112–113 task-based language teaching (TBLT) 112 TBLA see task-based language assessment (TBLA) TBLT see task-based language teaching (TBLT) TIMSS see Trends in International Mathematics and Science Study (TIMSS) Timukova, A. 107–111 Tingyang, Z. 87 translanguaging 2, 3, 6, 132, 245; holistic assessment through 14–15; Latinx bilingual students’ 48–58; multilingual perspective in 141–142; pedagogies 68; theory 48–50, 53 Trends in International Mathematics and Science Study (TIMSS) 96

UNESCO 7 Van Avermaet, P. 101 Varela, F. 50, 51 Vertovec, S. 245 young plurilingual students assessment, digital tools for 76–89; bilingual Arabic-English school 81–83; curricula challenges, addressing 84–85, 86–87; French international school 83; future requirements 81–87; limitations 81–87; multilingual learners 77–78; online resource, possibilities of 78–81; connecting to families and communities 85, 87; public monolingual school 83–84; school stakeholders ideologies 84; teaching context 81, 82 Zanon, J. 163