Innovation in Learning-Oriented Language Assessment 3031189493, 9783031189494

Citation preview

NEW LANGUAGE LEARNING AND TEACHING ENVIRONMENTS

Innovation in Learning-oriented Language Assessment Edited by Sin Wang Chong Hayo Reinders

New Language Learning and Teaching Environments

Series Editor Hayo Reinders Department of Education; Department of Languages Anaheim University; King Mongkut’s University of Technology Thonburi Anaheim; Bangkok, USA

New Language Learning and Teaching Environments is an exciting new book series edited by Hayo Reinders, dedicated to recent developments in learner-centred approaches and the impact of technology on learning and teaching inside and outside the language classroom. The series aims to: • Publish cutting-edge research into current developments and innovation in language learning and teaching practice. • Publish applied accounts of the ways in which these developments impact on current and future language education. • Encourage dissemination and cross-fertilisation of policies and practice relating to learner-centred pedagogies for language learning and teaching in new learning environments. • Disseminate research and best practice in out-of-class and informal language learning. The series is a multidisciplinary forum for the very latest developments in language education, taking a pedagogic approach with a clear focus on the learner, and with clear implications for both researchers and language practitioners. It is the first such series to provide an outlet for researchers to publish their work, and the first stop for teachers interested in this area.

Sin Wang Chong  •  Hayo Reinders Editors

Innovation in Learning-Oriented Language Assessment

Editors Sin Wang Chong Moray House School of Education and Sport University of Edinburgh Edinburgh, UK

Hayo Reinders King Mongkut’s University of Technology Thonburi Bangkok, Thailand

ISSN 2946-2932     ISSN 2946-2940 (electronic) New Language Learning and Teaching Environments ISBN 978-3-031-18949-4    ISBN 978-3-031-18950-0 (eBook) https://doi.org/10.1007/978-3-031-18950-0 © The Editor(s) (if applicable) and The Author(s), under exclusive licence to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover illustration © peepo / Getty Images This Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Praise for Innovation in Learning-Oriented Language Assessment “Using the seven dimensions of learning-oriented language assessment (Turner and Purpura, 2016), editors Chong and Reinders derived 10 learning-oriented language assessment principles. This edited volume of 17 practical chapters document innovative practices of learning-oriented language assessment from 10 countries. It is arguably the first exploration into learning-oriented language assessment for teachers and teacher trainers and offers excellent examples worthy of replication by teachers.” —Antony Kunnan, Duolingo “This interesting collection of papers on learning-oriented language assessment emanates from an online symposium held in October 2021. I was delighted to keynote at that event and it gives me great pleasure to applaud the efforts of the co-editors and authors in producing this timely collection. These evidence-based and practical chapters are underpinned by ten learning-oriented language assessment principles proposed by the editors in their introductory chapter. Plenty of food for thought for language teachers and researchers.” —David Carless, University of Hong Kong

Contents

1 I ntroduction: Learning-Oriented Language Assessment—Insights for Evidence-Based Practices  1 Sin Wang Chong and Hayo Reinders 2 A  Review of Formative Language Assessment Research and Implications for Practitioners 13 Tony Burner 3 Is  It Possible to Implement Learning-­Oriented Assessment Principles in Test Preparation? Evidence from a High-­­Stakes Standardised EFL Test in China 27 Ruijin Yang 4 Assessment  for Learning in the Brazilian High-School EFL Context: The Case of Digital Genres 45 Vander Viana and Juliana Jandre 5 An  Investigation into EFL Learners’ Perspectives Towards Dynamic Assessment 63 Tuba Özturan, Hacer Hande Uysal, and Prithvi N. Shrestha

vii

viii Contents

6 Exploring  the Consistency Between Self- and Teacher Assessment: Using Co-Constructed Assessment Descriptors in EAP Writing in China 81 Huahui Zhao and Beibei Zhao 7 Developing  a Learner-Centered Assessment of Academic Writing for Graduate EFL Students105 Gavin O’Neill and Katerina Petchko 8 Language  Teacher Autonomy and Written Feedback: The Case of a Hong Kong Elementary English Teacher121 Sin Wang Chong 9 Assessing  Depth of Word Knowledge of Beginner Learners of French, German and Spanish Aged 11–14 in England137 Natalie Finlayson and Emma Marsden 10 Developing  Learning-Oriented Language Assessment Literacy in Pre-­service Language Teachers163 Magnus Coney 11 Padlet  as a Formative Assessment Tool in the Online Language Classroom: Action Research181 Ghadah Albarqi 12 Application  of Virtual Reality Speaking Assessment Tasks in an EFL Context201 Mojtaba Heydari and Fahimeh Marefat 13 Technology-Mediated  Tasks for Formative Assessment in an e-Portfolio Environment219 Rosana Villares and Oana Maria Carciu

 Contents 

ix

14 Mobile-Assisted  Language Assessment for Adult EFL Learners: Recommendations from a Systematic Review237 Michelle Y. Chen and You-Min Lin 15 Pro-Gamer  Inspired Speaking Assessment257 James York 16 Developing  Language Teacher Cognition about Technology-Enhanced Assessment: A Case of Student Teachers in a TESOL Programme277 Li Li 17 Product  to Process: The Efficacy of Hybrid Feedback in Academic Writing Classrooms for Fostering Process-Oriented Writing295 Johanathan Woodworth 18 Development  and Content Validation of an E-portfolio to Assess and Promote Learner Autonomy311 Lilan Lin and Hayo Reinders I ndex331

Notes on Contributors

Ghadah Albarqi is a Fellow of Higher Education Academy (FHEA). Ghadah’s main areas of interest are second language (L2) acquisition, processing and production, and the relationship between working memory capacity and L2 processing. She is also interested in second language assessment. Tony Burner, PhD  is a Professor of English at the Department of Languages and Literature Studies at the University of South-Eastern Norway. His main research interests are English education, classroom assessment, R&D work, teacher mentoring, multilingualism and professional development. Oana Maria Carciu  is a Lecturer at the University of Zaragoza (Spain). Her research interests include corpus linguistics, genre analysis, digital literacies, and learning, teaching and assessment integrating new technologies for the development of communicative competence in English. Her current research focuses on the study of digital genres for science communication. Michelle Y. Chen  holds a PhD in measurement, evaluation and research methodology from the University of British Columbia. She is an experienced psychometrician at Prometric. Her work supports test ­development, evaluates score validity and promotes a better understanding of measurements and assessments. xi

xii 

Notes on Contributors

Sin Wang Chong  is a Senior Fellow of the Higher Education Academy (SFHEA), and is an Associate Professor in Language Education at the Moray House School of Education and Sport, University of Edinburgh. Sin Wang’s research interests include language and educational assessment, computer-assisted language learning and research synthesis. Sin Wang is Associate Editor of two SSCI-indexed journals, Innovation in Language Learning and Teaching, and Higher Education Research & Development. Magnus Coney  worked as a teacher and teacher trainer in the UK and Italy until 2018. He has an MPhil in Research in Second Language Education at the University of Cambridge, and now works in the Teaching and Curriculum team at Cambridge University Press and Assessment. Natalie Finlayson  carries out research that focuses on the construction and analysis of corpora for foreign language vocabulary teaching and testing purposes. She currently works as a Senior Resource Developer at the National Centre for Excellence for Language Pedagogy (NCELP) and has been teaching French, German and English at secondary and tertiary level since 2013. Mojtaba Heydari is a research fellow at University of Ottawa. He received his PhD in Applied Linguistics from Allameh Tabataba’i University. His field of interest is computer-aided assessment/learning and computational linguistics. Mojtaba’s publications have appeared in peer-reviewed journals including Assessing Writing and his edited volumes are published by Routledge, Palgrave Macmillan and Cambridge University Press. Juliana Jandre  is an English teacher at the school run by the Federal University of Rio de Janeiro, Brazil. She has a PhD in Education from the Catholic University of Rio de Janeiro, Brazil. Her research interests include English language education and translanguaging. Li Li  is a Professor in language education at the University of Exeter. Her research interests include language teacher cognition, classroom discourse, higher-order thinking skills and the use of new technologies in

  Notes on Contributors 

xiii

language learning. She is the author of Social Interaction and Teacher Cognition, Language Teacher Cognition and New Technologies and Language Learning. Lilan Lin  is a Professor at the School of Foreign Languages at Anhui Jianzhu University in China. Her research interest includes learner autonomy, assessment, blended learning and language teaching. You-Min Lin  is a Test Research and Development Specialist at Prometric. You-Min’s interests lie in the intersection of corpus linguistics and language assessment. Her recent work focused on integrating learner corpora research and language models to facilitate the development and validation of second language testing. Fahimeh Marefat is a Professor of Applied Linguistics at Allameh Tabataba’i University in Tehran, Iran. Her research interest includes assessing writing, genre studies and systematic reviews. She has published peer-reviewed edited volumes with Routledge, Palgrave Macmillan and CUP. Her recent study appears in Studies in Second Language Learning and Teaching. Emma Marsden  is a Professor of applied linguistics at the University of York and a former secondary school teacher. She investigates the learning and teaching of second languages. She has (co-)authored approximately 70 publications, edited the journal Language Learning and led initiatives for open materials and data (IRIS), findings (OASIS) and researchinformed pedagogy (NCELP). Gavin O’Neill  is an Assistant Professor at Hitotsubashi University in Tokyo, Japan. His research interests include technical writing instruction in EMI contexts, how students learn to use source literature, and anti-­ plagiarism education. Tuba Özturan  is an EFL instructor at School of Foreign Languages at Erzincan University. She received her master’s degree and PhD in English Language Teaching from Hacettepe University. Her research interests lie in the areas of learning-oriented assessment, dynamic assessment, sociocultural theory, L2 writing and computer-based testing.

xiv 

Notes on Contributors

Katerina Petchko  teaches research methods and thesis writing at the National Graduate Institute for Policy Studies, Tokyo, Japan, and develops academic writing curricula for international graduate students. Hayo Reinders is a TESOL Professor and Director of Research at Anaheim University, USA, and Professor of Applied Linguistics at KMUTT in Thailand. He is founder of the global Institute for Teacher Leadership and editor of Innovation in Language Learning & Teaching. His interests are in out-of-class learning, technology and language teacher leadership. His homepage is www.innovationinteaching.org. Prithvi N. Shrestha  is a Senior Lecturer in English Language in the School of Languages and Applied Linguistics at The Open University. He holds a BEd, MA in English literature (both Tribhuvan University, Nepal), MA TESOL (University of Lancaster) and EdD (The Open University), and he is a Senior Fellow of the Higher Education Academy (SFHEA). Hacer Hande Uysal  is a Professor at Hacettepe University. She received her master’s degree in English Education and her PhD in Foreign Language and ESL Education from The University of Iowa, United States. Her research interests are second language writing, academic discourse, early language teaching and language policies. Vander Viana  is an Associate Professor in Education at the University of East Anglia. He directs the Master’s in Teaching English to Speakers of Other Languages (TESOL) and is the founder/leader of the Language in Education Research Group. He has published widely on his areas of research expertise, which include English language (teacher) education, corpus linguistics and academic/pedagogic discourse analysis. Rosana Villares  is Assistant Professor at the University of Zaragoza (Spain) where she teaches English for Specific Purposes courses to undergraduate students. After completing her PhD on internationalization and language policy, she has specialized in academic writing, genre analysis and digital genres for science communication. Johanathan Woodworth, PhD is an instructor at the School of Continuing Studies at York University, Canada. Within the broader

  Notes on Contributors 

xv

scope of second language (L2) acquisition, his research interests include technology as a mediating tool, adaptive learning and corrective feedback for L2 development. Ruijin Yang  is a Lecturer and currently a post-doc fellow in the School of International Studies at Zhengzhou University. She obtained her PhD from the Queensland University of Technology (Australia). Her research interests include TESOL, Language Testing and Assessment and Mixed Methods Research. James York  is a Senior Assistant Professor at Meiji University where he teaches English. He conducts research on games and play in language teaching. He is also the editor of Ludic Language Pedagogy, an open-access journal publishing research on the integration of games and play in language teaching contexts. Beibei Zhao  is a Lecturer in Language Education at Zhejiang Shuren University. Her primary research interest lies in language testing/assessment and English for Academic Purposes. Huahui Zhao  is a Lecturer in Language Education at the University of Leeds. Her main research lies in the interplay between language assessment and technology-enhanced learning. She has published widely in prestigious journals in both areas.

List of Figures

Fig. 5.1 Fig. 9.1 Fig. 9.2 Fig. 9.3

Students’ perspectives towards DA Example of an NCELP test item on a web interface Example of an NCELP test item on a mobile phone Receptive test of form and meaning recognition (French Year 7, Term 2) Fig. 9.4 Receptive test of form and meaning recognition (Spanish Year 7, Term 2) Fig. 9.5 Productive test of spoken form recall (French Year 7, Term 2) Fig. 9.6 Receptive test of word part recognition (French Year 9, Term 3) Fig. 9.7 Receptive test of word part recognition (German Year 8, Term 3) Fig. 9.8 Receptive test of form and meaning recall (Spanish Year 7, Term 2) Fig. 9.9 Receptive test of form and meaning recognition (Spanish Year 7, Term 3) Fig. 9.10 Receptive test of form and meaning recognition (Spanish Year 8, Term 3) Fig. 9.11 Receptive test of form and meaning recognition (German Year 8, Term 2) (based on Read, 1995) Fig. 9.12 Productive test of form and meaning recall (Spanish Year 7, Term 2) (Adapted from Nurweni & Read, 1999)

71 149 149 150 151 151 152 152 153 153 154 154 155 xvii

xviii 

List of Figures

Fig. 9.13 Productive test of form and meaning recall (French Year 9, Term 1) 155 Fig. 9.14 Receptive test of concept and referent recognition (Spanish Year 8, Term 2) 156 Fig. 9.15 Productive test of concept and referent recall (Spanish Year 9, Term 3) 156 Fig. 9.16 Receptive test of association recognition (French Year 7, Term 2) 157 Fig. 9.17 Receptive test of association (German Year 7, Term 3) (Adapted from Read, 2000) 157 Fig. 9.18 Receptive test of constraints recall (French Year 8, Term 1) 158 Fig. 11.1 Using Padlet in a writing class 186 Fig. 11.2 Benefits of using Padlet in the online language class 187 Fig. 12.1 Person-item FACETS ruler 209 Fig. 13.1 Example of assessment task 1 228 Fig. 13.2 Example of assessment task 2 229 Fig. 13.3 Examples of teacher feedback 230 Fig. 14.1 The assessment triangle. (Note: Graph adapted from National Research Council (2001) and Shavelson (2010)) 238 Fig. 14.2 The Preferred Reporting Items for Systematic Reviews and Meta-­Analyses (PRISMA) flow diagram of literature search screening process (adapted from Page et al., 2021) 241 Fig. 15.1 The structure of a typical montage video 261 Fig. 15.2 Diagrammatic representation of the pedagogical procedure 264 Fig. 15.3 An example of student transcription featuring comments regarding errors 266 Fig. 15.4 Graphical representation of the structure of a montage video 267 Fig. 15.5 An example of a student’s worksheet submission (names redacted)267 Fig. 15.6 A screenshot of a student’s montage video featuring colour-coded subtitles. (Videos can be viewed here https://youtu.be/2NG8lgz1LFk)268 Fig. 15.7 An example of colour-coded subtitles 269

List of Tables

Table 1.1 Table 2.1 Table 3.1 Table 4.1 Table 6.1 Table 6.2 Table 6.3 Table 6.4 Table 6.5 Table 6.6 Table 6.7 Table 6.8 Table 6.9

The seven dimensions of learning-oriented language assessment (based on Turner & Purpura, 2016) The 44 reviewed research studies on formative language assessment in alphabetical order Information of participating teachers Summary of the unit on digital genres Student participants’ backgrounds Kappa inter-rater agreement between teacher and selfratings for constructing summaries WST: differences between teacher and self-ratings on constructing summaries WST: congruence between teacher and self-ratings for constructing summaries Kappa inter-rater agreement between teacher and selfratings for the language use of summaries WST: differences between teacher- and self-ratings on the language use of summaries WST: differences between teacher- and self-ratings on the language use of summaries Kappa inter-rater agreement between teacher and selfratings for constructing argumentative essays WST: Differences between teacher and self-ratings for constructing argumentative essays

4 16 31 52 85 87 88 89 90 91 92 94 94 xix

xx 

List of Tables

Table 6.10 WST: Differences between teacher and self-ratings for constructing argumentative essays Table 6.11 Kappa inter-rater agreement between teacher and selfratings for the language use of argumentative essays Table 6.12 WST: differences between teacher- and self-ratings for the language use of argumentative essays Table 6.13 WST: congruence between teacher and self-ratings for the language use of argumentative essays Table 10.1 LOA and non-LOA topics Table 10.2 Survey items for RQs 1–3 Table 10.3 Activity types for developing assessment literacy Table 10.4 Self-reported tutor AL Table 10.5 Most common assessment topics in CELTA input sessions Table 10.6 Importance of assessment topics for CELTA trainees Table 10.7 Most common activity types for developing LOA Table 10.8 Most and least common activity types by topic Table 12.1 Item fit statistics for the five tasks Table 13.1 Task 1 “Let me show you around” Table 13.2 Task 2 “Language learning strategies for speaking: Spain/ Romania exchange project” Table 14.1 Articles about mobile-assisted language assessment Table 14.2 Studies with learner characteristics, assessment features, and technologies Table 15.1 The evaluation rubric Table 15.2 Qualitative coding results of participant responses Table 17.1 Criterion Trait Feedback Analysis for Criterion for Version 19.3.0 Table 18.1 Content validity of the student model in Rounds 1–3 Table 18.2 Aiken’s V and H estimations of the rubric Table 18.3 Weighted values and rankings considered by the experts

95 96 97 98 167 167 168 171 171 172 173 174 210 226 227 242 244 265 269 307 317 325 327

1 Introduction: Learning-Oriented Language Assessment—Insights for Evidence-Based Practices Sin Wang Chong and Hayo Reinders

What Is Language Assessment? Language assessment is any activity, both inside and outside the classroom, that aims to collect information about language learners’ performance, proficiency, knowledge, and skills. There is an expectation that such information will be analysed and acted on by the teacher and/or the learners. Central to the processes of collection, analysis, and action is the environment (e.g., classroom, sociocultural contexts) where the assessment is enacted, and the relationship between the environment and learners. Language assessment can include a wide range of activities including quizzes, tests, exams, portfolios, feedback, teacher-learner

S. W. Chong (*) Moray House School of Education and Sport, University of Edinburgh, Edinburgh, UK e-mail: [email protected] H. Reinders King Mongkut’s University of Technology Thonburi, Bangkok, Thailand © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_1

1

2 

S. W. Chong and H. Reinders

discussions on learners’ performance, as well as automatically collected and analysed data, such as attendance and participation rates. All of these can take place in class or beyond. According to its purposes, language assessment is usually categorised into assessment of learning, for learning, and as learning (Chong, 2018; Lee, 2017). Assessment of learning, also known as summative assessment, aims to certify and measure achievements of language learners. On the formative end of the spectrum are assessment for learning and as learning. While the intention of both assessment for learning and as learning is to facilitate learning, the former emphasises the leading role played by teachers and the latter focuses more on the learners (Earl, 2013). In practice, language assessment usually serves more than one purpose. For instance, an examination can be an assessment of learning and assessment for learning activity when teachers not only provide scores and corrections on the exam paper but also provide oral feedback to learners regarding their strengths and weaknesses. An examination can even be an example of assessment as learning when examination questions are set by learners to demonstrate their understanding of common question types (Carless & Lam, 2014) (see also Chap. 3. Therefore, it is unrealistic and unhelpful to categorise language assessment activities based on stereotypical views (e.g., that a test must be summative assessment). In fact, Boud (2000) argues that, most if not all the time, assessment does “a double duty” (p. 159), serving both summative and formative functions simultaneously. From the notions of assessment of, for, and as learning, language assessment is conceptualised into a “product” and “process”. Language assessment is viewed as a “product” when assessment of learning is implemented, focusing on the achievements of language learners as captured in the assessment task. In the cases of assessment for and as learning, language assessment is understood as a teacher-led and learner-led learning process whereby learners progress from their current state of learning to the next (Hattie & Timperley, 2007). In this book, we aim to introduce an additional dimension of language assessment, that is, assessment as an “ecology” (Chong & Isaacs, 2023). Here, “ecology” is used metaphorically to refer to the spaces where language assessment takes place (Benson, 2021). An ecological perspective to language assessment, or “language

1  Introduction: Learning-Oriented Language… 

3

assessment ecology” (ibid), refers to “a dynamic environment where learners’ needs are addressed through deliberate and informed orchestration of language learning resources so that learners make effective use of assessment for their learning” (p. xx). In other words, language assessment is not simply about what teachers and learners do independently in the process. More importantly, it is about how teachers design an environment or culture of assessment that is suitable for the developmental and psychological needs of learners. To exemplify, it is not sufficient for teachers to implement peer assessment in a writing classroom without ensuring that learners are prepared to provide constructive peer feedback, and understand the purpose of peer assessment. It is only through alignment between learners’ needs and the assessment activities that learners can realise the “affordances” (i.e., the usefulness of the activities from the learners’ perspective) of such activities, thus engaging meaningfully with them (Chong et al., 2022; van Lier, 1997). In short, conceptualising language assessment as ecology underscores that language assessment does not operate in isolation, but in alignment with the needs of learners and the environments where the assessment is enacted. To underscore the importance of language assessment as an ecological construct, each chapter in this volume provides an in-depth description of the context where the assessment innovation is practised. In their chapters, the contributors justify how the assessment innovations meet the needs of a specific group of learners and the learning and sociocultural environments they are in (see Chap. 7 for an example of how an academic writing test was developed to suit the needs of a group of Japanese university English learners). Such an ecological view towards language assessment is also called “learning-oriented language assessment” (Turner & Purpura, 2016).

Learning-Oriented Language Assessment The term “learning-oriented assessment” was coined in higher education by Carless (2007) to refer to summative and formative assessment that focuses primarily on “engineering appropriate student learning” (p. 59). According to Carless (2007), there are three features of learning-oriented

4 

S. W. Chong and H. Reinders

assessment: assessment tasks as learning tasks, learner involvement, and the central role of feedback (p. 60). In a recent model by Carless (2015), more specific information is added to the framework. For example, Carless (2015) explicates that one benefit of involving learners actively in the assessment process is to develop their evaluative judgement. As for feedback, Carless (2015) highlights the paradigm shift from feedback as information to feedback as process, honing in on learners’ engagement with feedback. In the context of language education, the concept of learning-oriented assessment has been interpreted quite differently. As a budding area of research, the earliest mention of learning-oriented language assessment is in Turner and Purpura’s (2016) working model of learning-oriented language assessment. They refer to seven dimensions of language assessment that teachers need to consider: contextual, affective, interactional, instructional, learning, proficiency, and elicitation (p. 261) (Table 1.1). Different from Carless (2007, 2015), Turner and Purpura’s model of learning-­ oriented language assessment does not suggest a specific approach to assessing learners. Instead, they offer an ecological view towards language Table 1.1  The seven dimensions of learning-oriented language assessment (based on Turner & Purpura, 2016) Dimension Affective

Description

Concerns L2 learners’ emotions, beliefs about L2 learning, motivation, and personality, and how they affect L2 learning. Contextual Includes micro (e.g., curriculum) and macro (e.g., sociocultural) contexts. Elicitation Includes planned and unplanned activities which aim to provide opportunities for learners to demonstrate their language ability. Learning Relates to how L2 learners process and learn new language items, considering the roles of cognition, feedback, and self-regulation. Instructional Refers to L2 teachers’ knowledge of the language, knowledge of the curriculum, and knowledge of how to teach the language and the curriculum. Interactional Relates to the elicitation dimension but focuses on exchanges between the teacher and learners. Proficiency Refers to the learning objectives and targets of L2 teaching and assessment.

1  Introduction: Learning-Oriented Language… 

5

assessment, proposing interrelated learner and contextual dimensions that need to be considered when researching and designing language assessment. In a similar vein, the learning-oriented assessment cycle proposed by Jones and Saville (2016) takes into consideration the contextual dimension of language assessment, referring to micro-level context (“the classroom”, p. 26) and macro-level context (“setting and monitoring targets”, p. 26). From the above, it is clear that learning-oriented language assessment is not a specific type or approach to language assessment; learning-­ oriented language assessment is a perspective that views language assessment with an ecological stance, considering the interaction between learners’ needs and the environments where language learning takes place.

 en Learning-Oriented Language T Assessment Principles This edited volume includes 18 chapters that document innovative practices of learning-oriented language assessment. Each chapter considers dimension(s) of language assessment of Turner and Purpura’s (2016) model. In what follows, we synthesise the findings from the chapters and propose ten principles for implementing learning-oriented language assessment.

Principle 1 Language teachers need a renewed understanding of what “language assessment” entails, especially its close relationship with teaching and learning. For example, in Chap. 16, Li Li proposes ways to develop language teachers’ cognition about technology-mediated language assessment. These include creating concrete experience for language teachers to design and implement technology-mediated language assessment, observing technology-­ mediated language assessment designed by peers, and reflecting on their own experience and peer observation. Two key principles highlighted in the chapter are “abstract conceptualisation” and “active experimentation”.

6 

S. W. Chong and H. Reinders

The former refers to the generic principles for designing and implementing technology-mediated language assessment that teachers develop throughout the teacher training programme, while the latter focuses on how language teachers apply these principles to develop effective technology-­mediated language assessment. In a similar vein, Magnus Coney in Chap. 10 calls for developing pre-service language teachers’ assessment literacy using innovative approaches in the context of CELTA.

Principle 2 Language assessment tasks need to be designed to represent language use in authentic communicative situations. Chapter 12 reports on the application of virtual reality to the design of speaking assessment tasks. Based on their findings, Mojtaba Heydari and Fahimeh Marefat emphasise the importance of authenticity of the test-takers’ role and the relevance of scenarios in speaking assessment. The scenarios of speaking assessment need to specify communicative functions that test-takers are expected to demonstrate. At the same time, test-takers’ backgrounds, needs, and skills need to be considered when designing authentic speaking assessment.

Principle 3 Opportunities should be provided to learners to be more proactive in the feedback process. Johanathan Woodworth (Chap. 17) discusses a hybrid approach to feedback comprising automated written corrective feedback and teacher feedback. In this approach, learners first submit their essays to an automated writing evaluation system to receive computer-generated written corrective feedback. Learners then revise their essays based on the feedback they receive. When they are satisfied with the revisions, learners submit their essays to the teacher for additional feedback. Learners in the study’s experimental group (those who experience the hybrid approach to feedback) perceived writing as a process rather than a product and became more autonomous learners.

1  Introduction: Learning-Oriented Language… 

7

Principle 4 The potential of gamification in language assessment can be explored to maximise learner engagement and sustain learner motivation. In Chap. 15, James York describes a pro-gamer inspired speaking portfolio assessment that asks learners to create montage videos to showcase their best gameplay achievements. The portfolio is assessed by the instructor using a pre-­ determined rubric. Specifically, learners choose a videogame that they would like to play and record themselves playing the game. Before playing the game, they are given the chance to familiarise themselves with how to play the game and the language features that they need when playing the game. Next, learners enter a cycle of play and analysis sessions where they record and transcribe their own gameplay. Finally, learners select clips from the gameplay recordings to be included in the montage video. Together with the video, learners submit a written report detailing their reflection on their speaking performance.

Principle 5 Through assessment tasks, learners need to be orientated towards focused, purposeful, and deep self-reflection. Rosana Villares (Chap. 13) reports the development of an e-portfolio with first-year university students. One of the speaking tasks requires learners to create a video or recorded presentation to introduce a one-day itinerary for a tourist who would like to visit the learners’ hometown. To facilitate learners’ reflection, instead of asking learners generic questions such as “what you did well” or “what you could do better”, the teacher designed task-specific reflective questions so that learners can provide more insightful and focused self-assessment of their performance. For example, for the itinerary task, since it is a group task, learners are asked to reflect on communication skills that they practise during their presentation and something they learn from their peers.

8 

S. W. Chong and H. Reinders

Principle 6 A community of practice approach can be adopted by making learners’ work visible and accessible. In Chap. 11 Ghadah Albarq describes the use of Padlet as a formative assessment tool and demonstrates the benefits for sharing learners’ work in progress in a friendly and supportive environment. As part of a continuous writing assessment, a Padlet wall is created with the assessment rubric posted. Learners are encouraged to create their writing outlines and post them on Padlet. Learners are then asked to give comments on their peers’ outlines. Having obtained peer feedback, learners begin writing their essays, which they also post on Padlet. In addition to giving feedback, the teacher asks learners to give comments on their own work and the work by their peers. Because of the wall-like layout of Padlet, learners are able to view the whole writing process, from outline to essay, with the peer and teacher feedback they received.

Principle 7 Language teachers need to develop and exercise their autonomy in developing appropriate assessment or feedback strategies. In his chapter (Chap. 8), Sin Wang Chong reports on the written corrective feedback strategies of a primary English teacher in Hong Kong. He first puts forward a conceptualisation of language teacher autonomy that comprises action, development, and freedom. He then examines how an experienced primary English teacher navigates the school’s expectations, her experience in giving feedback, and the readiness of her learners when implementing bespoke written corrective feedback strategies. These feedback strategies include unfocused direct written corrective feedback, focused direct written corrective feedback (with error codes), focused complimentary indirect written corrective feedback, written comments at the end of learners’ compositions, and a generic feedback form.

1  Introduction: Learning-Oriented Language… 

9

Principle 8 The learning aspect of summative assessment can be maximised when it is designed, implemented, and followed up appropriately. Focusing on a learner-centred, high-stakes graduate-level writing test, Gavin O’Neill and Katerina Petchko (Chap. 7) documents the process of developing and validating a placement test of academic writing for a Japanese university. Summarising their experience, they present four important steps for developing a learner-centred language test. First, needs analysis is to be conducted to collect data from different sources about learners’ backgrounds, university’s writing requirements, academics’ expectations, and learners’ perceived difficulties. Second, the construct of academic writing needs to be developed to include specific skills and demonstrable outcomes. Third, academic writing instructors need to be involved in test design and rater training. Wider faculty involvement is encouraged to minimise political issues related to curricula change Finally, test results need to be used to inform curriculum change, teacher training, and learner support.

Principle 9 Language teachers can explore the opportunity to personalise learners’ assessment experience. Focusing on dynamic assessment, Tuba Özturan, Hacer Hande Uysal, and Prithvi N. Shrestha (Chap. 5) introduce how one-on-­ one interactive feedback sessions are implemented longitudinally in a writing class. In each interactive session, teachers and learners engage in dialogic feedback to discuss various areas of learners’ written texts including rhetorical and linguistic quality. In the sessions, the teacher identifies and addresses specific learning needs. Feedback dialogue between the teacher and learner is maintained through the teacher asking tailored guiding questions for individual learners to reflect on their own writing performance.

10 

S. W. Chong and H. Reinders

Principle 10 Language assessment can be used to promote language learner autonomy. Implemented in a Chinese university, an electronic portfolio for first-year non-English major undergraduates to self-assess the development of their autonomous language learning behaviours during their undergraduate study is developed by Lily Lin and Hayo Reinders (Chap. 18). The electronic portfolio enables learners to submit numerical and narrative performance data. Using learning analytics, learners’ reflection on their autonomy is supported by an array of quantitative and qualitative data. To further assist learners’ reflection, learners are provided with a rubric, peer feedback, and teacher feedback.

Conclusion With contributions from language teachers and assessment researchers based in Brazil, Canada, China, Hong Kong, Japan, Norway, Saudi Arabia, Spain, Turkey, and the UK, we hope that this edited volume sheds practical light on “learning-oriented language assessment” as an emergent approach to language assessment. In this book, you will find innovative (e.g., the use of technologies in Chap. 4) and learner-centred language assessment practices (e.g., self-assessment in Chap. 6) that focus on diverse language learner populations (e.g., assessment of young learners’ vocabulary knowledge in Chap. 9). At the same time, research syntheses that summarise existing research and practices on learning-oriented language assessment are also featured (see Chaps. 2 and 14). These syntheses provide a comprehensive review of the state-of-the-art of specific types of language assessment and offer evidence-based and practical suggestions for language teachers. To make the book more relevant to practitioners, each chapter ends with a “practice brief ” from the contributors, sharing tips with other language teachers should they consider implementing the assessment example in their own classrooms or contexts. We hope you find innovative ideas in this book to design and research your own language assessment practices.

1  Introduction: Learning-Oriented Language… 

11

References Benson, P. (2021). Language learning environments: Spatial perspectives on SLA. Multilingual Matters. Boud, D. (2000). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151–167. https://doi. org/10.1080/713695728 Carless, D. (2007). Learning-oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57–66. https://doi.org/10.1080/14703290601081332 Carless, D. (2015). Exploring learning-oriented assessment processes. Higher Education, 69, 963–976. https://doi.org/10.1007/s10734-­014-­9816-­z Carless, D., & Lam, R. (2014). Developing assessment for productive learning in Confucian-influenced settings. In C.  Wyatt-Smith, V.  Klenowski, & P. Colbert (Eds.), Designing assessment for quality learning. The enabling power of assessment, vol 1 (pp.  167–179). Springer. https://doi. org/10.1007/978-­94-­007-­5902-­2_11 Chong, S. W. (2018). Three paradigms of classroom assessment: Implications for written feedback research. Language Assessment Quarterly, 15(4), 330–347. https://doi.org/10.1080/15434303.2017.1405423 Chong, S., Isaacs, T., & McKinley, J. (2022). Ecological systems theory and second language research. Language Teaching, 1–16. https://doi.org/10.1017/ S0261444822000283 Chong, S. W., & Isaacs, T. (2023). An ecological perspective on classroom-based assessment. TESOL Quarterly. https://doi.org/10.1002/tesq.3201 Earl, L. (2013). Assessment as learning: Using classroom assessment to maximize student learning (2nd ed.). Corwin Press. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487 Jones, N., & Saville, N. (2016). Learning oriented assessment: A systemic approach. UCLES/Cambridge University Press. Lee, I. (2017). Classroom writing assessment and feedback in L2 school contexts. Springer. Turner, C. E., & Purpura, J. E. (2016). Learning-oriented assessment in second and foreign language classrooms. In D.  Tsagari & J.  Banerjee (Eds.), Handbook of second language assessment (pp. 255–274). De Gruyter Mouton. van Lier, L. (1997). Observation from an ecological perspective. TESOL Quarterly, 31(4), 783. https://doi.org/10.2307/3587762

2 A Review of Formative Language Assessment Research and Implications for Practitioners Tony Burner

Formative Language Assessment This chapter is based on approaches to formative assessment, where the philosophy is that assessment situations should have a learning potential for students. Formative assessment was coined by Scriven (1967), referring to formative evaluations of study programs. The concept of formative assessment was widely studies in the context of classroom research in the 1990s. A panel of assessment experts formed the Assessment Reform Group (1989–2010), with its epicenter in the UK, studying and developing formative assessment in schools. They coined the term Assessment for Learning, which is believed to be a more practice-oriented term stressing the learning dimension of assessment. Some use the terms formative assessment and assessment for learning interchangeably (Gardner, 2012), whereas others argue that they should be distinguished (Swaffield, 2011).

T. Burner (*) University of South-Eastern Norway, Drammen, Norway e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_2

13

14 

T. Burner

Obviously, there is not a clear consensus about their meanings, and there exists various definitions (Wiliam, 2011). The Assessment Reform Group effectively put formative assessment on the educational agenda (Black & Wiliam, 1998, 2009; Black et al., 2003). Their definition of formative assessment is “The process of seeking and interpreting evidence for use by learners and their teachers, to identify where the learners are in their learning, where they need to go and how best to get there” (Assessment Reform Group, 2002). Related to language learning classrooms, one widely used definition is provided by Bachman and Palmer (1996). They define formative language assessment (hereby abbreviated FLA) as situations in which “…students guide their own subsequent learning, and teachers modify their teaching methods and materials so as to make them more appropriate for students’ needs, interests, and capabilities” (p. 98). A seminal oft-cited review article, published by two of the most prominent scholars in the Assessment Reform Group, indicated positive impacts of formative assessment on learning (Black & Wiliam, 1998). It was followed up by empirical longitudinal studies showing promising results on student learner-centeredness (Wiliam et al., 2004), and other studies implementing and relating formative assessment to language learning classrooms (McGarrell & Verbeem, 2007; Rea-Dickins, 2001; Ross, 2005). Typically, formative assessment in language learning classrooms has been realized through classroom-based assessments, such as self- and peer assessment, portfolio assessment, and through formative feedback in classroom discussions. However, despite lots of research, there is no up-to-date summary of “what do we know” in this field, i.e., FLA. The question that is posed in this chapter is what do we know about the most recent empirical research on formative language assessment and what do the results mean for practitioners?

Method This chapter reviews the most recent empirical research on FLA.  It is limited to classroom language studies using the term “formative assessment”, bearing in mind that there may be studies about FLA but which

2  A Review of Formative Language Assessment Research… 

15

avoid “formative assessment” in title, abstract, and keywords. Alternative terms some scholars use are, for example, assessment for learning, learning-­oriented assessment, or dynamic assessment. However, most studies based on approaches to formative assessment seem to include “formative” in either title, abstract, and/or keywords. Database searches were conducted in the two largest databases for educational research (ERIC and SCOPUS) with a combination of the keywords “formative assessment” and “language”. The searches were limited to the last five years (2016–2021) from the time the searches were conducted (2021) and to peer-reviewed full text empirical articles only. The reasons for choosing the period 2016–2021 is practical; the vast amount of research in the field would make it unmanageable to include more articles. The first round of selection resulted in hundreds of hits. Selection was performed by reading the titles and abstracts. Any articles that did not revolve around FLA were excluded. Twenty-six relevant articles were found in ERIC and 20 in Scopus. The second round of selection occurred by reading the selected articles from the first round. One article (Gan & Leung, 2019) was excluded because it was theoretical and not empirical, and one article was excluded because it was a methodological study (Khodabakhshzadeh et al., 2018). The final number of 44 relevant articles (see Table 2.1) from the second round were read and prepared for analysis. The analysis categorized the articles’ contents according to the nature of formative assessment (e.g., peer assessment, self-assessment, portfolio assessment), the mode of formative assessment (e.g., written, oral, multimodal), the context of formative assessment (e.g., country, class level, language subject) and the methods used (e.g., observations, interviews, surveys). The articles are reviewed with a focus on “practice brief ” for practitioners by addressing success criteria and challenges in FLAs. In the following, three sections will provide a review of the 44 articles: first, the nature and mode of FLA in the reviewed studies; second, the contexts and methods in the reviewed studies; finally, the results in the reviewed studies with a focus on implications for practitioners.

16 

T. Burner

Table 2.1  The 44 reviewed research studies on formative language assessment in alphabetical order Afitska, O. (2017). Exploring teacher and learner views on the use of formative assessment in primary EAL classrooms: A case study. i-manager’s Journal on English Language Teaching, 7(3), 41–54. Alahmadi, N., Alrahaili, M., & Alshraideh, D. (2019). The impact of the formative assessment in speaking test on Saudi students’ performance. Arab World English Journal, 10(1), 259–270. Alam, J., & Aktar, T. (2019). Assessment challenges & impact of formative portfolio assessment (FPA) on EFL learners’ writing performance: A case study on the preparatory English language course. English Language Teaching, 12(7), 161–172. Alharbi, A. S., & Meccawy, Z. (2020). Introducing Socrative as a tool for formative assessment in Saudi EFL classrooms. Arab World English Journal, 11(3), 372–384. Antonova, K., & Tyrkheeva, N. (2021). Formative assessment of critical reading skills in higher education in Russia in the context of emergency remote teaching. The Journal of Teaching English for Specific and Academic Purposes, 9(2), 137–148. Bachelor, J. W. B., & Bachelor, R. B. (2016). Classroom currency as a means of formative feedback, reflection and assessment in the World Language classroom. NECTFL Review, 78, 31-42. Bader, M., Burner, T., Iversen, S. H., & Varga, Z. (2019). Student perspectives on formative feedback as part of writing portfolios. Assessment & Evaluation in Higher Education, 44(7), 1017–1028. Bayat, A., Jamshidipour, A., & Hashemi, M. (2017). The beneficial impacts of applying formative assessment on Iranian university students’ anxiety reduction and listening efficacy. International Journal of Languages’ Education and Teaching, 5(1), 1–11. Burner, T. (2016). Formative assessment of writing in English as a foreign language. Scandinavian Journal of Educational Research, 60(6), 626–648. Chen, D., & Zhang, Li. (2017). Formative assessment of academic English writing for Chinese EFL learners. TESOL International Journal, 12(2), 47–64. Fong, C. S., & De Witt, D. (2019). Developing Intercultural communicative competence: Formative assessment tools for Mandarin as a foreign language. Malaysian Journal of Learning and Instruction, 16(2), 97–123. Ghazizadeh, F., & Motallebzadeh, K. (2017). The impact of diagnostic formative assessment on listening comprehension ability and self-regulation. International Journal of Language Testing, 7(2), 178–194. Guo, Q., & Xu, Y. (2021). Formative assessment use in university EFL writing instruction: A survey report from China. Asia Pacific Journal of Education, 41(2), 221-237. (continued)

2  A Review of Formative Language Assessment Research… 

17

Table 2.1 (continued) Jing, M. J. (2017). Using formative assessment to facilitate learner self-­ regulation: A case study of assessment practices and student perceptions in Hong Kong. Taiwan Journal of TESOL, 14(1), 87–118. Judge, M. (2021). Covid 19, school closures and the uptake of a digital assessment for learning pilot project during Ireland’s national lockdown. Irish Educational Studies, 40(2), 419–429. Kılıçkaya, F. (2017). Improving formative assessment in language classrooms using GradeCam Go! Teaching English with Technology, 17(4), 78–92. Mahapatra, S. K. (2021). Online formative assessment and feedback practices of ESL teachers in India, Bangladesh and Nepal: A multiple case study. Asia-­ Pacific Edu Res, 30(6), 519–530. Maier, U. (2021). Self-referenced vs. reward-based feedback messages in online courses with formative mastery assessments: A randomized controlled trial in secondary schools. Computers & Education, 174, 1–16. Mäkipää, T., & Hildén, R. (2021). What kind of feedback is perceived as encouraging by Finnish general upper secondary school students? Education Sciences, 11(12), 1–15. Meza, A., Rodríguez, I., & Caviedes, L. (2021). Fostering EFL preservice teachers’ academic writing skills through reflective learning. Issues Teach. Prof. Dev., 23(1), 89–106. Micán, A. D., & Medina, L. C. (2017). Boosting vocabulary learning through self-assessment in an English language teaching context. Assessment & Evaluation in Higher Education, 42(3), 398–414. Mohamadi, Z. (2018). Comparative effect of online summative and formative assessment on EFL student writing ability. Studies in Educational Evaluation, 59, 29–40. Moser, J. (2017). Creating in-class self-directed learning through can do objectives, portfolio use, and formative assessment. The Journal of Asia TEFL, 14(4), 674–686. Naghdipour, B. (2017). Incorporating formative assessment in Iranian EFL writing: A case study. The Curriculum Journal, 28(2), 283–299. Nasr, M., Bagheri, M. S., Sadighi, F., & Rassaei, E. (2018). Iranian EFL teachers’ perceptions of assessment for learning regarding monitoring and scaffolding practices as a function of their demographics. Cogent Education, 5(1), 1–29. Önalan, O., & Karagül, A.E. (2018). A study on Turkish EFL teachers’ beliefs about assessment and its different uses in teaching English. Journal of Language and Linguistic Studies, 14(3), 190–201. Pan, Y-C. (2020). Taiwan university students’ perceptions of summative and formative classroom assessment in English courses. TESOL International Journal, 15(2), 46–64. Parsons, D. (2017). Formative assessment in discussion tasks. ELT Journal, 71(1), 24–36. (continued)

18 

T. Burner

Table 2.1 (continued) Qin, X., & Yi, C. (2021). Preservice teachers’ implementation of formative assessment in English writing class: Mentoring matters. Studies in Educational Evaluation, 70, 1–11. Saito, H., & Inoi, S. (2017). Junior and senior high school EFL teachers’ use of formative assessment: A mixed-methods study. Language Assessment Quarterly, 14(3), 213–233. Saliu-Abdulahi, D. (2017). Scaffolding writing development: How formative is the feedback? Moderna språk, 111(1), 127–155. Sardareh, S. A. (2016). Formative feedback in a Malaysian primary school ESL context. The Malaysian Online Journal of Educational Science, 4(1), 1–8. Tavakoli, E., Amirian, M. R., Burner, T., Davoudi, M., & Ghaniabi, S. (2019). Formative Assessment of Writing (FAoW): A confirmatory factor structure study. International Journal of Assessment Tools in Education, 6(3), 344–361. Tolley, L. M. (2019). Valuing teachers’ evaluative thinking: The role of teacher knowledge and practice in formative assessment. Research Issues in Contemporary Education, 4(1), 21–34. Treve, M. (2021). English for Academic Purposes (EAP) lecturers’ perceptions of formative assessment integration in the Thai EAP context. Journal of Language and Linguistic Studies, 17(2), 1096–1113. Vogt, K., Tsagari, D., Csépes, I., Green, A., & Sifakis, N. (2020). Linking learners’ perspectives on language assessment practices to teachers’ assessment literacy enhancement (TALE): Insights from four European countries. Language Assessment Quarterly, 17(4), 410–433. Wang, B., Yu, S., & Teo, T. (2018). An exploratory case study into Chinese EFL teachers’ commentary practices in oral presentations. Taiwan Journal of TESOL, 15(2), 65–94. Wicking, P. (2020). Formative assessment of students from a Confucian heritage culture: Insights from Japan. Assessment & Evaluation in Higher Education, 45(2), 180–192. Widiastuti, I. A. M. S., Mukminatien, N., Prayogo, J. A., & Irawati, E. (2020). Dissonances between teachers’ beliefs and practices of formative assessment in EFL classes. International Journal of Instruction, 13(1), 71–84. Wilson, J., Roscoe, R., & Ahmed, Y. (2017). Automated formative writing assessment using a levels of language framework. Assessing Writing, 34, 16–36. Xiao, Y., & Yang, M. (2019). Formative assessment and self-regulated learning: How formative assessment supports students’ self-regulation in English language learning. System, 81, 39–49. Yarahmadzehi, N., & Goodarzi, M. (2019). Investigating the role of formative mobile based assessment in vocabulary learning of pre-intermediate EFL learners in comparison with paper based assessment. Turkish Online Journal of Distance Education-TOJDE, 21(1), 181–196. (continued)

2  A Review of Formative Language Assessment Research… 

19

Table 2.1 (continued) Zenouzagh, Z. M. (2019). The effect of online summative and formative teacher assessment on teacher competences. Asia Pacific Education Review, 20, 343–359. Zou, M., Kong, D., & Lee, I. (2021). Teacher engagement with online formative assessment in EFL writing during Covid-19 pandemic: The case of China. Asia-Pacific Edu Res, 30(6), 487–498.

 hat Is the Nature and the Mode of FLA W in the Research Studies? Formative feedback specifically (Saliu-Abdulahi, 2017; Sardareh, 2016; Wang et  al., 2018) or formative feedback as part of other natures of assessment (Afitska, 2017; Bader et  al., 2019; Burner, 2016; Chen & Zhang, 2017; Mahapatra, 2021; Mäkipää & Hildén, 2021; Meza et al., 2021) are common themes in the reviewed studies. The same goes for FLA using digital tools (Alharbi & Meccawy, 2020; Judge, 2021; Maier, 2021; Wilson et al., 2017; Yarahmadzehi, 2019). Some other themes or natures of FLA that were found in the reviewed studies were portfolios (Alam & Aktar, 2019; Mohamadi, 2018); FLA and self-regulation (Jingjing, 2017; Xiao & Yang, 2019), self-assessment (Bachelor & Bachelor, 2016; Micán & Medina, 2017), quizzes (Bayat et al., 2017), multiple drafting and peer assessment (Naghdipour, 2017), FLA and self-­ monitoring and scaffolding (Nasr et al., 2018), oral presentations (Pan, 2020), informal FLA (Tolley, 2019), overall or general FLA frameworks—typically using Black and Wiliam’s frameworks rather than a specific nature or element of formative assessment (Guo & Xu, 2021; Saito & Inoi, 2017; Tavakoli et  al., 2019), and FLA during Covid-19 (Zou et  al., 2021). Most research focuses on written modes (e.g., Alam & Aktar, 2019 on formative writing portfolio), a few on oral (e.g., Alahmadi et al., 2019 on formative speaking tests), and a few on a combination of both modes (e.g., Afitska, 2017).

20 

T. Burner

In Which Contexts Do we Find FLA Studies and Which Methods Are Used? The reviewed studies on FLA were conducted in a range of national settings, including Norway, Finland, Turkey, Iran, Malaysia, the USA, England, Germany, Saudi Arabia, China, Hong Kong, Colombia, Russia, Ireland, India, Bangladesh, Nepal, Japan, Taiwan, Thailand, Cyprus, Greece, and Hungary. It is mostly English that is studied, but there are a couple of studies that focus on Spanish, Mandarin, Swedish, and French (Bachelor & Bachelor, 2016; Fong & Witt, 2019; Mäkipää & Hildén, 2021). Most research is on English in a context where it is a foreign language (EFL), whereas a few studies look at English where it is a first language (Tolley, 2019) or second language (ESL) (Mahapatra, 2021; Qin & Yi, 2021; Sardareh, 2016), English as an additional language (EAL) (Afitska, 2017), English for academic purposes (EAP) (Jingjing, 2017; Treve, 2021), and English for specific purposes (ESP) (Alharbi & Meccawy, 2020; Antonova & Tyrkheeva, 2021). Higher education dominates when it comes to the level of education that is studied. There are some studies from secondary schools (Burner, 2016; Judge, 2021; Kılıçkaya, 2017; Mäkipää & Hildén, 2021; Saliu-Abdulahi, 2017; Saito & Inoi, 2017; Tolley, 2019) and only a couple where the level is primary school (Afitska, 2017; Wilson et al., 2017; Sardareh, 2016). Methods of data collection are both qualitative (e.g., Nasr et al., 2018) and quantitative (e.g., Moser, 2017) in nature, including interviews, observations, texts/documents, surveys, and mixed methods (e.g., Saito & Inoi, 2017). Several studies use control and experimental groups or pre- and post-tests to measure the effect of FLA (e.g., Alahmadi et al., 2019; Yarahmadzehi, 2019). Most studies have few participants, but some have hundreds of participants, ranging from 255 students in Tavakoli et al.’s study (2019) to 727 teachers in Saito and Inoi’s study (2017).

2  A Review of Formative Language Assessment Research… 

21

 esults in the Reviewed Studies with a Focus R on Implications for Practitioners Firstly, teacher assessments tend to be traditional and thus need to become more formative. Teachers need more time to do the transformations needed towards more formative assessments (Burner, 2016; Saito & Inoi, 2017; Tavakoli et al., 2019; Treve, 2021). Secondly, both teachers and students prefer formative assessment rather than summative assessment, but they need practice in order to get it right (Pan, 2020; Parsons, 2017). Teachers perceive FLA as more effective and more beneficial and prefer process to product (Mohamadi, 2018; Nasr et  al., 2018; Önalan & Karagül, 2018). Students report that FLA makes learning more enjoyable and authentic (Bayat et al., 2017). FLA seems to be more effective than summative assessment, not only when it comes to written skills (which teachers mostly focus on (Vogt, 2020)), but also listening skills, speaking skills, critical skills, awareness-raising, attitudes, and self-regulation skills (Alahmadi et  al., 2019; Antonova & Tyrkheeva, 2021; Fong & Witt, 2019; Ghazizadeh & Motallebzadeh, 2017; Pan, 2020; Xiao & Yang, 2019). Regarding written skills, multiple drafting and formative feedback help students improve their works. Studies with experimental and control groups, and pre- and post-studies, indicate that FLA produces better learning results (Chen & Zhang, 2017; Naghdipour, 2017). Regarding formative feedback, students prefer mixed feedback rather than only from the teacher or only from peers. There is a general recommendation that teachers and students need to develop a shared understanding of FLA in order to make it work. Otherwise, contradictions might appear between how teachers and students perceive of feedback, text revision, self- and peer assessment, learner-centeredness, student involvement etc. (Bader et al., 2019; Burner, 2016). However, no matter how formative the feedback is in FLA, it is of utmost importance that teachers evaluate their assessment activities and feedback regularly in order to timely use the information they obtain from formative assessment (Mahapatra, 2021; Tolley, 2019) and students need to actively participate in the feedback process (Guo & Xu, 2021; Sardareh, 2016). Students regard the content of feedback as the most important aspect in

22 

T. Burner

formative feedback (Mäkipää & Hildén, 2021) and prefer one-to-one oral feedback (Chen & Zhang, 2017; Saliu-Abdulahi, 2017). Praise is much more effective than punishment, and reward-based feedback (making students repeat the test) outperforms self-referenced feedback (Maier, 2021). Negative feedback can, particularly in certain cultural contexts, run the risk of “losing face” among the students (Wang et  al., 2018). Other research from Confucian heritage contexts reveal that students are not as competitive as one might think, but they are more like their Anglophone peers (Wicking, 2020). Portfolio assessment seems to be a useful learner-centered tool for FLA. Students are positive towards portfolio assessment due to the possibility of multiple drafting and being able to improve one’s work. Students seem to be critical towards peer assessment, but rather prefer peer discussions as part of portfolio assessment (Bader et  al., 2019). Portfolio assessment leads to autonomous learning and students take more control of their learning (Alam & Aktar, 2019). The studies on self-assessment indicate that students are unable to benefit from self-assessment when they do not know how to address linguistic gaps, so it is important for teachers to provide the necessary language guidance (Moser, 2017). Moreover, students tend to prefer teacher aid and teacher-directed practices to that of peers (Jingjing, 2017). Combining feedback, reflection, and assessment seems to be meaningful for students, who are good at acknowledging their learning strengths and weaknesses and set learning commitments for themselves (Afitska, 2017; Bachelor & Bachelor, 2016; Meza et al., 2021; Micán & Medina, 2017). E-learning tools (e.g., Socrative quiz), software and applications (e.g., GradeCam Go! with quizzes, immediate feedback and monitoring progression) have a positive effect on students’ language assessment, make FLA easier, more accurate, and more efficient (Alharbi & Meccawy, 2020; Kılıçkaya, 2017; Yarahmadzehi, 2019; Wilson et al., 2017; Zenouzagh, 2019), but should be approached through inquiry rather than measurement (Judge, 2021) and should draw on teachers’ optimistic views of ICT and their expertise in ICT and writing assessment (Zou et al., 2021). FLA should be introduced early to preservice teachers, preferably at levels where there is no pressure of exams (Qin & Yi, 2021). Teachers

2  A Review of Formative Language Assessment Research… 

23

who participate in professional development courses show stronger beliefs in FLA, and stronger beliefs tend to lead to more formative language assessment practices (Widiastuti, 2020).

Conclusions Returning to the question posted in the beginning of this chapter about what we know regarding the most recent empirical research on FLA and what the results mean for practitioners, it is clear that FLA focuses mostly on formative feedback, self-regulation, self-monitoring, scaffolding and the use of digital tools, self- and peer assessment (more self than peer), quizzes, multiple drafting, and portfolios as mediating artefacts (Vygotsky, 1986). Moreover, most research in FLA focuses on students’ writing skills. There is a good geographical spread in studies on FLA, but studies from higher education dominate the field. More research from lower levels of education would be a good contribution to the field. Furthermore, most of the research on FLA is in EFL, thus more research is needed in English as a second language (ESL) and modern foreign languages (MFL) contexts to shed light on the similarities and differences in language status and language learning contexts in FLA. The results and the implications of the results in the reviewed studies on FLA are not surprising. They concur with other research in the field from the last decades (Andrade & Cizek, 2010; Bachman & Palmer, 2010). Language teachers’ assessments tend to be traditional and need to become more formative. Interventions where the purpose is to make language assessments more formative indicate that teachers and students appreciate the changes, students learn more and in better ways, and the mediating artefacts that are used (Vygotsky, 1986) mostly succeed in fulfilling the formative purposes. Nevertheless, teachers need assessment literacy and FLA needs to be articulated with the students in order to achieve mutual and shared understandings. Allal (2016) proposed to define assessment culture as constructed in classrooms and schools in terms of three components: teachers’ and students’ beliefs regarding assessment; the assessment practices that teachers implement and in which students participate; and the assessment tools that support, or sometimes inhibit, the assessment practices. The

24 

T. Burner

interaction between these three components is essential. Future research on FLA could take care to include these components in studying any unit of analysis. As succinctly put by Allal (2016, p. 261): “In any given classroom, recurring patterns or configurations can be found, but ongoing interactions always introduce variations and evolutions”. It is these interactions and evolutions that need to be studied, preferably in longitudinal studies from various researchers’ points of views (not only linguistics, but also psychology and educational sciences). In her article, Allal also points to a growing field of research, namely co-regulation. Whereas self-­ regulation entails active monitoring and regulating of one’s own learning, co-regulation is a transitional process in students’ appropriation of self-­ regulation strategies through interaction with a more capable other (Hadwin & Oshige, 2011; Vygotsky, 1978). FLA research could advance a step further by looking more into co-regulation in settings where learning is mediated through assessment tools such as portfolios, self- and peer assessment, or software and digital applications. Finally, as concluded by Allal (2016), any implementation of formative assessment or assessment for learning requires concerted coordination of policy, professional development, and practice in classrooms and schools. The present review is restricted to two of the largest databases for educational research and only to the last five years from the date of search. It could have been expanded to include several databases, manual searches in journals, citation searches, PhD theses (monographs), and book chapters. However, this small-scale study provides an overview of some of the most recent empirical studies on FLA. It is of utmost importance that practitioners, researchers and policy makers pay attention to such studies since they have a more generalizable relevance than single studies do.

Practice Brief Language teachers should focus more on formative than summative assessment. Formative assessment promotes students’ learning; it is learner-centered, more effective, authentic, and appreciated by students. Language teachers should talk more about assessment with their students and obtain a shared understanding of the purposes and methods of

2  A Review of Formative Language Assessment Research… 

25

assessment. They can use various tools to implement and enhance formative assessment in language classes, for example portfolios, multiple drafting, software and digital applications, self- and peer assessment, quizzes, discussions etc., and use formative feedback to engage students in the assessment process and track students’ learning over time.

References Allal, L. (2016). The co-regulation of student learning in an assessment for learning culture. In D.  Laveault & L.  Allal (Eds.), Assessment for learning: Meeting the challenge of implementation (pp. 259–273). Springer International Publishing. Andrade, H. L., & Cizek, G. J. (Eds.). (2010). Handbook of formative assessment. Routledge. Assessment Reform Group. (2002). Researched-based principles of assessment for learning to guide classroom practice. Retrieved from http://www.nuffieldfoundation.org/assessment-­reform-­group Bachman, L., & Palmer, A. (1996). Language testing in practice. Oxford University Press. Bachman, L., & Palmer, A. (2010). Language Assessment in Practice. Oxford: Oxford University Press. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5–31. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning. Open University Press. Gan, Z., & Leung, C. (2019). Illustrating formative assessment in task-based language teaching. ELT Journal, 74(1), 10–19. Gardner, J. (Ed.). (2012). Assessment and learning (2nd ed.). SAGE Publications Ltd.. Hadwin, A., & Oshige, M. (2011). Self-regulation, coregulation, and socially shared regulation: Exploring perspectives of social in self-regulated learning theory. Teachers College Record, 113, 240–264. Khodabakhshzadeh, H., Kafi, Z., & Hosseinnia, M. (2018). Investigating EFL teachers’ conceptions and literacy of formative assessment: Constructing and validating an inventory. International Journal of Instruction, 11(1), 139–152.

26 

T. Burner

McGarrell, H., & Verbeem, J. (2007). Motivating revision of drafts through formative feedback. English Language Teaching Journal, 61(3), 228–236. Rea-Dickins, P. (2001). Mirror, mirror on the wall: Identifying processes of classroom assessment. Language Testing, 18(4), 429–462. Ross, S. J. (2005). The impact of assessment method on foreign language proficiency growth. Applied Linguistics, 26(3), 317–342. Scriven, M.  S. (1967). The methodology of evaluation. In R.  W. Tyler, R.  M. Gagne, & M.  Scriven (Eds.), Perspectives of curriculum evaluation (pp. 39–83). Rand McNally. Swaffield, S. (2011). Getting to the heart of authentic Assessment for learning. Assessment in Education: Principles, Policy & Practice, 18(4), 433–449. Vygotsky, L. S. (1978). Mind in society. The development of higher psychological processes. Harvard University Press. Vygotsky, L. S. (1986). Thought and language. The MIT Press. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. Wiliam, D., Lee, C., Harrison, C., & Black, P. (2004). Teachers developing assessment for learning: Impact on student achievement. Assessment in Education, 11(1), 49–65.

3 Is It Possible to Implement Learning-­Oriented Assessment Principles in Test Preparation? Evidence from a High-­Stakes Standardised EFL Test in China Ruijin Yang

Introduction and Literature Review It is often admitted that English-as-a-Foreign-Language (EFL) tests may exert a significant impact on corresponding teaching as well as learning practices, such impact is generally referred to as washback and it could be either positive or negative washback. This study, taking the grammar and vocabulary tasks in the Senior High School Entrance Examination— English (SHSEE-E) as an example, investigates preparation of this compulsory EFL test in Chinese junior high schools. As designed, SHSEE-E evaluates students’ English proficiency required by the fifth level of the English Curriculum Standards for Compulsory Education (Ministry of

R. Yang (*) School of International Studies, Zhengzhou University, Zhengzhou, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_3

27

28 

R. Yang

Education, 2011), which advocates the development of students’ overall language use ability and communicative language use. The test scores are accordingly used as proof for senior high school admission which is one of the dual functions of this gatekeeping test. Thus, SHSEE-E exerts tremendous influence on teaching and learning. However, whether the test could achieve the student-centred goal in both teaching and summative assessment design mandated in the Curriculum (Ministry of Education, 2011) or not remains largely unknown. Learning-Oriented Assessment (LOA) (Carless, 2007; Jones & Saville, 2016), which views students or learners as the centre of learning, is considered suitable to explore the learning-oriented potential during test preparation in the current study. As LOA has potential for presenting the positive impact of tests on teaching and learning, this study thus adopts the conceptual framework of LOA to explore the synergy between teaching, learning, and assessment. The study focuses on the most controversial part of this standardised EFL test—the grammar and vocabulary test in the SHSEE-E (hereafter the GVT). In reality, the grammar-translation method has dominated EFL teaching in China for decades (Pan & Qian, 2017) and the teaching of grammar and vocabulary has been heavily taught in traditional ways such as rote-memorisation. As a result, researchers tried hard to find suitable and meaningful ways to teach and test grammar and vocabulary knowledge during test preparation, which failed to a large extent as teachers were found to use test-drilling strategies to prepare students for effective test preparation (Yang, 2015). Worse still, this negative influence goes against the curriculum writers’ positive intentions of student-­centred teaching and testing in terms of English study and the grammar and vocabulary section (Ministry of Education, 2011). In light of this contradictory situation, it is crucial to investigate the potential positive washback of the GVT and its influential factors. Even though washback studies are populous in the field (Damankesh & Babaii, 2015; Zhang & Bournot-Trites, 2021), studies that investigate specifically the positive washback of summative assessment are scarce. In view of this and to work towards positive washback (Bailey, 1996), the researcher regards it necessary to select an appropriate assessment notion to explore positive washback by using the GVT as the research context.

3  Is It Possible to Implement Learning-Oriented Assessment… 

29

Conceptual Frameworks and Principles Seeking to reconcile the conflicts between formative and summative assessments and focusing on students’ learning (Carless, 2007; Jones & Saville, 2016), LOA intends to promote positive washback of tests (Jones & Saville, 2016) and is the assessment that primarily focuses on “the potential to develop productive student learning processes” (Carless, 2015, p. 964). It involves using the evidence of test performance to make decisions on further language development and offer implications for test construction (Purpura, 2004). It prioritises L2 processing and learning outcomes and acknowledges their centrality in both planned and unplanned assessments (Purpura & Turner, 2013). In other words, in LOA, learning takes priority in any context and standards of learning and assessment are raised over time (Saville & Salamoura, 2014). It thus acknowledges the synergy between teaching, assessment, and learning (Turner & Purpura, 2016). Theoretically, LOA is usually practised in formative classroom assessment contexts (Jones & Saville, 2016; Turner & Purpura, 2016). However, since both formative assessment and summative assessment aim to measure learning outcomes, when the assessment is designed to engineer appropriate student learning, then either assessment could be claimed to achieve an LOA purpose of promoting learning in assessment (Carless, 2007). For summative assessment, it can be learning-oriented in certain conditions such as encouraging deep rather than surface approaches to learning and promoting a high level of cognitive engagement throughout a test (Carless, 2015). The current study employs LOA (Carless, 2007; Jones & Saville, 2016) as the analytical lens to explore positive washback of the GVT. Carless (2007) used three principles to construct his LOA model. The first principle of assessment tasks as learning tasks. This is relevant to test preparation practices when teachers use test-related tasks to prepare students for the test, which also applies to the current GVT preparation context. The second principle of involving students in assessment indicates that students could act as peer- or self-evaluators. The third principle of feedback as feedforward implies the important role of feedback in LOA. Likewise,

30 

R. Yang

the LOA cycle by Jones and Saville (2016) emphasises the key role of LOA tasks, interaction, and feedback in the classroom context. Gaining insights from these two models, this study thus regards the existence of student involvement in assessment, feedback, and classroom interactions as potential indications of LOA opportunities in GVT preparation. Empirically, inspired by LOA, Hamp-Lyons and Green (2014) demonstrated the application of LOA to summative assessments, using the term Learning Oriented Language Assessment (LOLA) to investigate the Cambridge English: First (FCE, now B2 First) Speaking test. Tsagari (2014) presented findings from unplanned LOA in EFL classrooms. Both studies suggest that LOA/LOLA training, its principles, and practices are necessary to integrate exams into learning, thus providing an empirical basis for LOA application in specific summative examinations. Nevertheless, the understanding of how LOA applies to summative assessments is still limited. Few studies have focused on applying LOA frameworks to high-stakes standardised English tests. Therefore, this study aims to contribute in this regard.

Research Questions From an LOA perspective, this study aims to investigate the positive washback of the GVT and two research questions (RQ) are addressed: RQ 1:  What are the opportunities for implementing LOA principles in GVT preparation? RQ 2  : What are the difficulties in implementing LOA principles in GVT preparation?

Methodology This qualitative research, focusing on three teachers and their classes (see Table 3.1), used convenience sampling and collected data in two phases. The first phase included 15 classroom observations in three junior high

3  Is It Possible to Implement Learning-Oriented Assessment… 

31

Table 3.1  Information of participating teachers Teacher

SA-TA

SB-TB

SC-TC

Pseudonyms Age Class size Teaching experience

Lan 25–30 48 students Three years of teaching experiences; first-year grade 9 teacher, no administrative position; subject teacher Master of translation and interpreting (MTI) TEM8

Hu 25–30 52 students Three years of teaching experiences; first-year grade 9 teacher; head teacher

Zhang 35–40 36 students 18 years of teaching experiences, director of teaching and discipline

Part-time master candidate

Bachelor degree (correspondence, in 2000) Not given

Education

English proficiency

TEM8

Note: SA/B/C-TA/B/C is the abbreviation for School A/B/C-Teacher A/B/C

schools in a southwestern city in China. The second phase contained three semi-structured teacher interviews. Thematic analysis (Clarke & Braun, 2017) was applied throughout the analysis stage and NVivo 12 Pro was used. Moreover, one-third (i.e., 1111) of the coded segments were co-coded by both the researcher and a Grade 9 English teacher, who had experiences in decoding the test preparation practices, and the inter-­ coder reliability was 87.57%.

Results and Discussion Generally speaking, the observation data presented key practices of LOA in the GVT preparation context, which included classroom interactions, feedback, and learning-oriented teaching methods. Semi-structured interviews further identified factors that were considered to facilitate or hinder the implementation of LOA principles in GVT preparation. Teachers’ beliefs about opportunities for the implementation of LOA principles contained alignment with students’ learning stage, developing communication abilities in real life, developing students’ learning skills

32 

R. Yang

in general, and learning-oriented test design intentions. However, although opportunities were clear, there were various challenges to the implementation of LOA principles which included efficient use of class time, the consideration of high test stakes, administrative duties, student language proficiency, class size, the concern over teaching performance, and limited teaching experiences as well as language expertise. This section thus reports and discusses the key findings.

LOA Opportunities In the interview stage, the researcher explained the LOA term to interviewees and they thus considered that their teaching practices were guided by LOA principles. Therefore, it was regarded generally possible for the GVT to exert positive washback in these three teachers’ classes. According to them, various patterns of test preparation activities aiming to promote students’ learning were identified. The two categories of learning-oriented test preparation practices identified were interactive classroom activities and learning-oriented teaching methods. Among those practices, the interactive classroom activities and learning-oriented teaching methods could offer most implications to in-­ service teachers and are thus particularly discussed as follows.

Interactive Classroom Activities Importantly, various interactive activities were practiced by the three observed teachers. In fact, a learning-oriented language classroom context normally consists of two kinds of activities: learner-centred and content-­centred activities which could complement each other in an LOA cycle (Jones & Saville, 2016). The former kind enables higher-order thinking skills and the co-construction of knowledge to occur during classroom interactions, while the latter category features the transmission of knowledge from teachers to students. Although, ideally, both learner-centred and content-centred interactions should appear in an LOA classroom, the current GVT preparation

3  Is It Possible to Implement Learning-Oriented Assessment… 

33

classes presented limited activities which focused on learners. Typical activities included group/pair discussion and positioning students as holders of linguistic knowledge.1 Noticeably, all three teachers utilised group/ pair discussion in classroom teaching which provided opportunities for learners to engage with assessment tasks with peers. This type of activities enabled teachers to foster higher-order communicative skills for problemsolving among students. As for positioning students as holders of linguistic knowledge, students were asked to share their answers in test preparation classes. This activity, mentioned by both Hu and Zhang, appeared only in Zhang’s observed classes when she frequently asked students to come to the front of the class to share knowledge/responses and answer questions. In contrast to Hu who used mainly the teacher-talk technique, Zhang used this learner-centred interactive activity to improve student learning and cultivate critical thinking among students. She explained as follows: Zhang: It can certainly [improve students’ grammar and vocabulary learning]. As I said just now, the student did the exercise, but he or she was not necessarily able to tell [how he or she did it]. This is such a process of persuading oneself (i.e., speaking out the student’s own thinking) and making clear of one’s mind, which I think is very important. In fact, the time is short, if the time is sufficient, especially ask that medium, no, the low language proficiency level students to come to the front and lecture. If those students can explain it very clear, then the other students should possibly be fine. What else? The teacher lecturing, students might not like this. But if it’s a student who lectures, the feeling of novelty, when a student is lecturing, the audience’s perception and feelings are different. (Interview) From the above interview account, Zhang encouraged students to articulate their thinking processes and apply problem-solving skills. She seemed to transfer the power and responsibility of teaching to students through  By positioning students as holders of language knowledge, it means that the teacher appointed a certain student to act as a teacher, come to the front of the class, and then explain how he finished the task to the whole class. 1

34 

R. Yang

this interaction and hoped to further improve students’ language learning. Therefore, both of these two learner-centred classroom interactions were regarded to contribute to the learning-oriented potential of the GVT. Except for learner-centred interactions, content-centred interactions were more prevalent in GVT preparation classes. As the name indicates, classroom activities focusing on knowledge of linguistic forms and test-­ related tasks were content-centred (Jones & Saville, 2016). A variety of content-centred interactions were identified in Lan’s and Zhang’s classes, which contained open-ended questioning, close-ended questioning, reading in chorus, giving bonus points, sentence completion, and student-­initiated questioning. Taking the open-ended questioning as an example, the observation data revealed that Lan and Zhang used open-ended questioning frequently to interact with students. The activity was characterised by asking “why” questions to engage individual or all students. Similar to positioning students as holders of linguistic knowledge, teachers expected students to clarify their thoughts and explain problem-solving processes through open-ended questioning. For example, Lan: No? Okay, forty-five, okay, helpful. [pause] What’s your reason? Why, why you chose “helpful”? What’s your reason? (Classroom Observation 1) It was clear that Lan expected students to explain the process behind the response. As Lan verified in the interview, she was not clear about the student’s decoding process before the student explained the reasoning. This implied that teachers hoped to motivate students to take part in classroom activities, express ideas and opinions via questioning (Liu & Zhao, 2010). In addition, compared with close-ended questions that trigger simple recall or mechanical response such as “yes” or “no”, open-­ended questions aimed at higher cognitive levels contribute more to learning (Hasselgreen et al., 2012) and thus could reduce the teacher-talk phenomenon. As such, it could be argued that open-ended questioning enabled teachers to achieve the “interaction” purpose of LOA in GVT preparation. As the complementarity between learner-centred and content-centred classroom interactions is viewed as essential for a learning-oriented

3  Is It Possible to Implement Learning-Oriented Assessment… 

35

classroom context (Jones & Saville, 2016), the above findings indicated the potential for GVT preparation classes to be learning-oriented. Although the number of learner-centred classroom interactions was limited and differences remained across the three observed classes, it was evident that both Lan’s and Zhang’s classes were more interactive than Hu’s classes.

Learning-Oriented Teaching Methods The other opportunity for GVT preparation classes to be learning-­ oriented was that teachers adopted teaching methods linked with LOA (Carless, 2007; Jones & Saville, 2016). Aiming to intentionally improve students’ learning, three methods were used by the teacher participants: using mentor-apprentice pairs, encouraging learner autonomy, and involving students in assessment. For example, “using mentor-apprentice pairs” meant asking students to help each other. It was encouraged mainly by Hu and Zhang and was particularly termed by Zhang. In Zhang’s class, there was a unique phenomenon in which a pair of students stood up at the same time when she questioned one of them in her class. According to Zhang, one mentor-­ apprentice pair normally contained both a high language proficiency student and a lower one. As she explained, her teaching and language learning beliefs motivated her to form such study pairs. In detail, Zhang confirmed that training students with high language proficiency to help those with low language proficiency is beneficial to both teachers and students. For teachers, this could help them focus on more difficult teaching content. For students, it could help high-proficiency students think clear the problem-solving process and low-proficiency students identify learning gaps. Zhang believed that the progress in learning was made jointly by both the mentor and the apprentice. In her opinion, once the mentor could clearly explain the problem-solving process to the apprentice, it meant the mentor understood the knowledge clearly. Thus, she verified that seeking help from peers could promote students’ learning. In fact, this activity assigned priority to both guidance and participation in learning

36 

R. Yang

and cognitive development activities (Rogoff, 1990). It allowed students to co-construct a zone of proximal development (ZPD) to cognitively deal with their learning tasks at a higher level (Vygotsky, 1986). It thus coincided with LOA principles since this ZPD put effective learning in a socio-cultural context rather than simply developing knowledge (Jones & Saville, 2016; Sjøberg, 2007). Additionally, learner autonomy (Salamoura & Unsworth, 2015) was highly promoted by teachers. Both Lan and Zhang regarded learner autonomy as a condition for students to improve learning. For instance, Lan suggested students learn by themselves outside the classroom. According to her, it was necessary for students to be autonomous learners and develop effective methods of self-study because English was a conduit for students to learn more about the world. To achieve this goal, Lan suggested students develop self-learning goals of vocabulary and encouraged students to take the learning initiative in test preparation. Further, in her class, Zhang reminded students to learn from a student, Mia, who was an autonomous learner during self-study time. Therefore, both teachers considered learner autonomy as a key in GVT preparation to enable learning-oriented potentials. To conclude, different practices which enabled LOA opportunities in GVT preparation were of referential value. However, although researchers such as Salamoura and Unsworth (2015) emphasised the importance of learner autonomy, it has not been foregrounded in LOA theoretical frameworks (Carless, 2007; Jones & Saville, 2016). Furthermore, Bailey (1996) proposed that positive or beneficial washback can emerge from learner autonomy and self-assessment. Therefore, current findings suggested the consideration of “encouraging learner autonomy” in exploring LOA opportunities and the positive washback in GVT preparation.

LOA Difficulties On the other hand, those teachers expressed their concerns over the implementation of learning-oriented principles in test preparation classes, especially when they were approaching the test date. The concerns expressed by teachers proved the difficulties of LOA implementation in the GVT

3  Is It Possible to Implement Learning-Oriented Assessment… 

37

context. In general, those concerns included efficient use of class time in test preparation, teachers’ limited teaching experiences and language expertise, high stakes of the test, administrative duties, class size, student language proficiency, and the concern over teaching performance. To make sure that teachers express clear opinions on the impeding factors of LOA implementation, the interviewer explained the term LOA to teachers. All three teachers considered that they understood this term and agreed that their teaching was generally guided by the belief that EFL assessment should place students’ needs and improvement as the main goal. However, they articulated different views towards in-service training when explaining their understandings of LOA. According to Lan, she confirmed attending professional development workshops organised by the school to improve their assessment literacy. On the contrary, Hu and Zhang had not received professional development in language assessment. Hu even expressed her eagerness to have such training. These showed differences in opportunities for building teachers’ assessment literacy and LOA knowledge across participating schools. However, since building teachers’ subject knowledge could influence student achievement in the long run (Hill et al., 2005), providing teachers with professional development opportunities for learning about LOA and improving their assessment literacy is essential (Carless, 2007; Zeng et al., 2018).

Effective Use of Class Time According to interview data, all three teachers first articulated their concern over the effective use of class time during the test preparation period. This priority impacted both their use of the target language and classroom activities in teaching. For instance, Lan explained as follows. Lan: ….Sometimes, for example, when I think that students could not understand, I use Chinese instead. Because, after all, sometimes you need to be efficient to prepare for the test. That is to say, you can use maybe two or three sentences in Chinese to explain something clearly, but if you use English, maybe some students need to

38 

R. Yang

comprehend for quite a while at even one sentence. Therefore, [the choice of using English language] should be carefully considered according to different teaching contexts. (Interview) Similarly, Zhang commented that considering saving time and making test preparation easy was important, she deliberately avoided using the target language in test preparation classes. However, she recognised that this time-saving compromise was not beneficial to students’ language learning and the implementation of LOA principles. The reason could be that the better acquisition of L2 was obtained when learners were exposed to the target language massively and L1 use was limited (Ellis, 2006; Lightbown & Spada, 2019).

The Concern over Teaching Performance Unsurprisingly, teachers expressed their concerns over their teaching performance if they implemented LOA principles in test preparation classes. This factor was particularly discussed by Hu. According to her, to make sure students would listen to her in class, she intentionally avoided using English, the target language, in test preparation classes. Hu: Those kids are very wayward, especially according to their current teenage psychology. If you force them to learn English, but they could not understand. They will then think “I spend time in English subject but still couldn’t improve my test score, so why should I learn?” … They will ask you the answers for that. Perhaps for me, I’d rather give up many of my previous teaching behaviours (i.e., teaching activities), in order to at least make sure that he will listen to my class. (Interview) This effort of getting students to listen to her class was a reflection of Hu’s concern for her teaching performance. As Hu commented in her interview, her students would judge whether they listened to the class based on the content of her teaching and she was afraid that if she did not use the “instruct-evaluate-do exercises” model, her students would think

3  Is It Possible to Implement Learning-Oriented Assessment… 

39

the class was useless for test preparation. In other words, Hu sacrificed the practices of using the target language and various classroom activities to meet the students’ needs to get a higher test score in test preparation classes. According to her, as the head teacher of the class, this consideration was crucial as students’ performance on tests would be viewed as the evidence for evaluating her teaching performance at the end of Grade 9. This finding coincided with other studies (e.g., Tsagari, 2011) which reported that students’ test performance was generally viewed as the criteria for evaluating teachers’ teaching effectiveness and thus aroused teachers’ anxiety. As such, it is argued that teachers’ concern over teaching performance or career security might impede their intentions to implement LOA principles in GVT preparation.

Limited Teaching Experiences and Language Expertise Additionally, limited teaching experiences and language expertise appeared to challenge the implementation of LOA principles in GVT preparation. As for Lan and Hu, their limited teaching experiences were the main challenge. As the first-year Grade 9 teachers, they felt it difficult to implement LOA principles in GVT preparation. Lan: However, Grade 9, because I am a new teacher, I haven’t taught Grade 9 before, so, many other teachers who are experienced may teach Grade 9 better than me. As for me, well, it’s still mainly me who guided more [in the class], because, recently I am quite confused. Because, in other words, you have to complete the [teaching] schedule, but if you always guide students, and for one question you have to guide for a long time, and the student still couldn’t give you the answer, that will be too much waste of time. Well, I am still falling short in teaching. (Interview) From the above comment, it was evident that Lan was feeling difficult due to her inexperience as a new teacher. In fact, Hu had a similar

40 

R. Yang

challenge since she was also a new teacher. Hence, it was not surprising that Hu believed that the model of “instruct-evaluate-do exercises” was the best teaching method she could use in test preparation.

Conclusion This study explored the potential of implementing LOA principles in a high-stakes standardised EFL test preparation context and identified both opportunities and difficulties of this implementation. This study preliminarily examined the actual test preparation classes which were common practices in many other summative test preparation situations. As the findings differed across the observed teachers, the researcher thus suggests the implementation of identified LOA practices in future studies to investigate the effectiveness. In this vein, action research could be utilised (Jones & Saville, 2016) and the following tips could be adopted by schools and teachers. Theoretically, this study uses LOA to investigate the positive washback of a high-stakes standardised EFL test and suggest the inclusion of learner autonomy in the existing LOA models. The study is thus timely and significant in that it answers how to make test preparation learning-oriented and highlights potential restrictions on the implementation of LOA principles. As the LOA literature is an under-researched area, this study contributes to a further understanding of examining positive washback in language testing from an LOA perspective.

Practice Brief The findings suggest that to bring positive washback in high-stakes standardised EFL test preparation, schools and teachers could contribute in the following ways. For schools, it is necessary to (1) provide more professional development seminars for teachers to build their LOA literacy and language expertise and (2) revise assessment criteria for monitoring teachers’ academic performance (i.e., do not simply use test scores to evaluate teaching performance). For in-service teachers, they could adopt

3  Is It Possible to Implement Learning-Oriented Assessment… 

41

learning-­oriented teaching methods (e.g., using mentor-apprentice pairs, encouraging learner autonomy, offering constructive feedback to feedforward students’ learning) to complement test-taking techniques in test preparation classes to promote students’ learning of English in examination-­oriented educational milieus. Acknowledgement  This study was funded by 2021 Higher Education Research Project of the Higher Education Association of Henan Province (number 2021SXHLX192).

References Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13(3), 257–279. https://doi. org/10.1177/026553229601300303 Carless, D. (2007). Learning-oriented assessment: Conceptual bases and practical implications. Innovations in Education and Teaching International, 44(1), 57–66. https://doi.org/10.1080/14703290601081332 Carless, D. (2015). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963–976. https://doi.org/10.1007/s10734-­014-­9816-­z Clarke, V., & Braun, V. (2017). Thematic analysis. The Journal of Positive Psychology, 12(3), 297–298. https://doi.org/10.1080/17439760.2016.1262613 Damankesh, M., & Babaii, E. (2015). The washback effect of Iranian high school final examinations on students’ test-taking and test-preparation strategies. Studies in Educational Evaluation, 45, 62–69. https://doi.org/10.1016/j. stueduc.2015.03.009 Ellis, R. (2006). Current issues in the teaching of grammar: An SLA perspective. TESOL Quarterly, 40(1), 83–107. https://doi.org/10.2307/40264512 Hamp-Lyons, L., & Green, T. (2014). Applying a concept model of learning-­ oriented language assessment to a large-scale speaking test. Teachers College Columbia University Roundtable in Second Language Studies, Teachers College Columbia University. http://www.tc.columbia.edu/tccrisls/ Hasselgreen, A., Drew, I., & Sørheim, B. (2012). Understanding the language classroom. Fagbokforlaget. Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement., 42(2), 371–406. https://doi. org/10.3102/00028312042002371

42 

R. Yang

Jones, N., & Saville, N. (2016). Learning oriented assessment: A systemic approach (Vol. 45). Cambridge University Press. Lightbown, P. M., & Spada, N. (2019). Teaching and learning L2 in the classroom: It’s about time. Language Teaching, 1-11. https://doi.org/10.1017/ S0261444819000454 Liu, Y., & Zhao, Y. (2010). A study of teacher talk in interactions in English classes. Chinese Journal of Applied Linguistics, 33(2), 76–86. Ministry of Education. (2011). English curriculum standards for compulsory education (2011) (yiwujiaoyu yingyu kecheng biaozhun, 2011). Beijing Normal University Press. http://mat1.gtimg.com/edu/pdf/edu/xkb2011/20120130 155515499.pdf Pan, M., & Qian, D. D. (2017). Embedding corpora into the content validation of the grammar test of the National Matriculation English Test (NMET) in China. Language Assessment Quarterly, 14(2), 120–139. https://doi.org/1 0.1080/15434303.2017.1303703 Purpura, J. E. (2004). Assessing grammar. Cambridge University Press. Purpura, J. E., & Turner, C. E. (2013). Learning-oriented assessment in classrooms: A place where SLA, interaction, and language assessment interface. ILTA/ AAAL Joint Symposium on “LOA in Classrooms”. Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context. Oxford University Press. Salamoura, A., & Unsworth, S. (2015). Learning oriented assessment: Putting learning, teaching and assessment together. Modern English Teacher, 24(3), 4–7. www.modernenglishteacher.com Saville, N., & Salamoura, A. (2014, October). Learning oriented assessment—A systemic view from an examination provider. Teachers College Columbia University Roundtable in Second Language Studies, Teachers College Columbia University. Sjøberg, S. (2007). Constructivism and learning. In E. Baker, B. McGaw, & P. Peterson (Eds.), International encyclopaedia of education (3rd ed.). Elsevier. http://folk.uio.no/sveinsj/Constructivism_and_learning_Sjoberg.pdf Tsagari, D. (2011). Washback of a high-stakes English exam on teachers’ perceptions and practices. Selected Papers on Theoretical and Applied Linguistics, 19, 431–445. https://doi.org/10.26262/istal.v19i0.5521 Tsagari, D. (2014). Unplanned LOA in EFL classrooms: Findings from an empirical study. Teachers College Columbia University roundtable in second language studies, Teachers College Columbia University. http://www.tc. columbia.edu/tccrisls/

3  Is It Possible to Implement Learning-Oriented Assessment… 

43

Turner, C. E., & Purpura, J. E. (2016). Learning-oriented assessment in second and foreign language classrooms. In D. Tsagari & J. Banerjee (Eds.), Handbook of second language assessment (pp. 255–273). DeGruyter Mounton. Vygotsky, L. S. (1986). Thought and language. MIT Press. Yang, W. (2015). A study on the washback effect of the grammar part in the junior secondary English achievement graduation test (Jiangxi) [Master’s thesis, Guangdong University of Foreign Studies]. Guangdong. http://www.cnki.net/ Zeng, W., Huang, F., Yu, L., & Chen, S. (2018). Towards a learning-oriented assessment to improve students’ learning—A critical review of literature. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 30(3), 211–250. https://doi.org/10.1007/ s11092-­018-­9281-­9 Zhang, H., & Bournot-Trites, M. (2021). The long-term washback effects of the National Matriculation English Test on college English learning in China: Tertiary student perspectives. Studies in Educational Evaluation, 68, 100977. https://doi.org/10.1016/j.stueduc.2021.100977

4 Assessment for Learning in the Brazilian High-School EFL Context: The Case of Digital Genres Vander Viana and Juliana Jandre

Assessment for Learning Assessment can be used for several purposes in educational environments. Traditionally, it has been equated with a systematic process aimed at evaluating one’s knowledge about a specific topic. This type of assessment is termed assessment of learning (cf. Black et al., 2004), and it has had a pervasive influence in education. In EFL contexts, this type of assessment can be used to decide, for instance, which class students can join (e.g. an upper-intermediate vs. advanced EFL class), whether students can be accepted onto an educational institution (e.g. a university), whether students are able to evidence attainment of learning outcomes and are thus ready to progress to the next stage (e.g. pass a school year).

V. Viana (*) University of Edinburgh, Edinburgh, UK e-mail: [email protected] J. Jandre Federal University of Rio de Janeiro, Rio de Janeiro, Brazil © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_4

45

46 

V. Viana and J. Jandre

Assessment, however, is not restricted to its traditional purpose of compartmentalizing students into categories, classes, levels and the like based on how much evidence of their knowledge they are able to provide. It can (and we would argue that it should) fulfill a supportive role in the classroom. When assessment is used to foster student learning, it is described as assessment for learning (e.g. Black et  al., 2004; Wiliam, 2011). The change of preposition (i.e. of vs. for) is key in indicating a completely different view of assessment. This means that learning accountability is not the main factor; instead, assessment is employed as an instrument to facilitate and develop student learning. A key practical difference between assessment of learning and assessment for learning relates to their frequency. The former is generally conducted at specific times, and it is somewhat temporally detached from regular classes—for example, in the high-school context described in this chapter, there is an “exam week” which is held at the end of every trimester. Assessment for learning, on the other hand, is fluid: it should be embedded in class and conducted in a continuous way as is the case in the present case study. The literature on assessment for learning (Assessment Reform Group, 2002; Bartlett, 2015; Berry, 2008; Cowie et al., 2013) identifies several of its features: • • • •

it focuses on the process—rather than the outcome—of learning; it promotes student motivation; it fosters students’ active role in their learning; it encourages students to support one another’s development (in addition to the role played by the teacher); • it develops students’ reflective and self-management skills; • it aims to develop student autonomy. It is not unusual to see assessment of learning being used as a synonym for summative assessment and assessment for learning being equated with formative assessment (e.g. Assessment Reform Group, 2006; Green, 2014). Black et  al. (2004), however, explain that there is a difference between these categorizations. Assessment of/for learning deals with the reason for which the assessment is conducted, and summative/formative

4  Assessment for Learning in the Brazilian High-School EFL… 

47

assessment relates to how the assessment is used. The reason and the use of assessment may be sometimes differentiated as in a task that is used to assist student learning (i.e. its reason) and to provide useful evidence to orient future pedagogical practice (i.e. its use). Regardless of how interesting this theoretical distinction may be, it might not always be the case in practice. For this reason, we adopt the term “assessment for learning” in the present chapter. The existing literature on assessment for learning indicates that its application to EFL environments has been investigated from teachers’ and/or students’ perspectives (Lee & Coniam, 2013; Wu et  al., 2021) and in different geopolitical contexts—e.g. China (Wu et  al., 2021), Hong Kong (Lee & Coniam, 2013) and Iran (Nasr et al., 2019). In this chapter, we describe a case study of assessment for learning in an underexplored EFL educational context: a public high school in Brazil. The next two sections provide relevant information about the Brazilian educational system and the specific high school where the case study took place.

EFL in the Brazilian Educational System As a result of the latest educational reform in Brazil, the compulsory teaching of a foreign language at middle and high schools was replaced by compulsory teaching of EFL (Ministry of Education, 2018).1 The justification for the most recent legislation change lies in the status of English as a lingua franca, which would be needed for international communication. However, many researchers argue this new educational law is based on a neoliberal view, favoring an education model which serves market needs (Considera, 2019; Szundy, 2019; Tílio, 2019; Santana & Kupske, 2020). Current Brazilian EFL education aims at enabling students to understand and share information in the target language as well as to position themselves critically in  local/global societies (Ministry of Education,  High schools, however, are allowed to offer a second/foreign language as an optional choice to students; and Spanish is indicated as the preferred option in the law (Ministry of Education, 2018). 1

48 

V. Viana and J. Jandre

2018). Brazil’s National Common Curricular Base foregrounds the deterritorialization of English (Ministry of Education, 2018). This means that English is not owned by a specific people, but it is used as the language of communication between various peoples. Its teaching should favor practices that target intelligibility, variety and creativity. This can be considered a disruptive view of EFL teaching in Brazil, especially as schools in the country follow a tradition of EFL grammar teaching (Batista, 2020). In relation to assessment, there is no national exam during or after basic education.2 The federal law states that evaluation must be continuous and cumulative to assess students’ development, and that teachers must focus more on qualitative aspects than quantitative results (Education Guidelines and Bases Law, 1996). Both formative and summative assessment types are recommended in the Brazilian National Curriculum Parameters (Ministry of Education and Sports, 1998, 2000). These parameters, which provide recommendations on school teaching, emphasize the use of assessment as a pedagogical tool to evaluate the success of teaching/learning. The Parameters suggest that teachers should favor continuous and process-focused assessment practices. Tests or exams should not be used as the only assessment instrument; instead, teachers are encouraged to develop various assessment instruments which must be contextually and pedagogically relevant. More recently, the government implemented Brazil’s National Common Curricular Base (Ministry of Education, 2018), which is aimed at establishing a common curriculum in Brazilian education, but the document does not offer specific and/or detailed guidance on assessment. The only recommendation in this regard is that formative and summative assessment should take into consideration context- and learning-related matters so that assessment is used to support student, teacher and school development. In summary, while the official policy documents in Brazil refer to assessment and provide educational stakeholders with some guiding principles, these documents do not stipulate what must be done in the classroom. Teachers and/or educational institutions enjoy considerable freedom in their assessment practices.  The only exception relates to the national exam that students may decide to take at the end of high school, which allows them to be considered for a place at an undergraduate course. 2

4  Assessment for Learning in the Brazilian High-School EFL… 

49

 FL Education at a Federal High School in Rio E de Janeiro, Brazil The case study presented here was conducted at a federal high school, which is considered a center of excellence in the Brazilian public education sector. In this context, middle-school students (aged 11–14 years old) take classes on EFL and French as a foreign language, and high-­ school students (aged 15–17 years old) can choose one foreign language—English, French or Spanish.3 This choice is made in the first high-school year, and students should stick to it until the third and last year.4 In this high school, EFL teaching is delivered in two weekly 50-minute classes. These classes are attended by a maximum of 12 students, thus enabling teachers to foster students’ development of the four language skills. This class size is not common at Brazilian public schools: the average student number is 35, which makes it difficult to practice speaking and listening (Batista, 2020). In recent years, EFL teachers at the focal high school have drawn on the notions of English as a lingua franca (Jenkins, 2007; Jordão, 2004) and translanguaging (Canagarajah, 2011; Lopes & Silva, 2018) to inform their pedagogical practice. Students are introduced to texts produced in numerous English-speaking countries, and they are encouraged to reflect on the use of English and other languages for meaning-making purposes in our contemporary world. In relation to assessment, the school is driven by assessment of learning, which is even institutionalized in an “exam week” that is held every three months. However, assessment for learning is recommended by the school policy. EFL teachers at this school adopt different assessment instruments such as seminars, projects, exams and self-evaluation. This  Middle school in Brazil lasts for four years, and it corresponds to Level 2 in the International Standard Classification of Education (ISCED). The three-year high-school system in Brazil corresponds to ISCED Level 3. 4  There is a mismatch between the existing federal law and the practice at this school. For example, high-school students are still given a chance to choose whether they want to study English even though EFL teaching is now compulsory in Brazilian basic education. Discussing the reasons for this practice is beyond the scope of this chapter. 3

50 

V. Viana and J. Jandre

diversity of assessment instruments is believed to be positive for the students, who may evidence their learning in different ways, and the teachers, who can gather more information about their pedagogical practice and adjust it if necessary. Online education was adopted at this school for the first time in 2020 due to the COVID-19 pandemic, and some adaptations were deemed necessary. The two weekly 50-minute sessions for high-school students were split into one synchronous and one asynchronous session. The former was jointly taught by two teachers through Google Meet, and the latter consisted of online preview/review tasks posted in Google Classroom, this high school’s virtual learning environment (VLE). Students were expected to work autonomously in the completion of these tasks, and they were then provided with feedback. The case study presented in this chapter took place with three groups of about 10 students each—all of whom were in their second high-school year. As many students at this high school live in deprived areas, they experience several technical barriers to online education in relation to both hardware (i.e. not having the appropriate equipment) and Internet connection (e.g. poor signal or lack of coverage). For these reasons, they hardly ever turn on their cameras or contribute to online lessons. As a consequence, EFL teachers opted to target the development of writing and reading skills in online classes during the pandemic. Assessment practices had to be adapted since the usual on-site exam week could not be held due to the pandemic restrictions. The school also changed the way students were graded: quantitative evaluation on a 10-point scale was replaced by a five-point conceptual evaluation (i.e. very good, good, average, unsatisfactory and insufficient).

Assessment for Learning in Practice The assessment-for-learning tasks which are the focus of the present chapter were part of a unit on digital genres delivered over eight online sessions—evenly split between synchronous and asynchronous ones. The inclusion of digital genres in the EFL syllabus at this high school coheres with the National Common Curricular Base, which suggests that digital

4  Assessment for Learning in the Brazilian High-School EFL… 

51

genres should be discussed in the language classrooms since they “impact their [students’] everyday lives in various social interactions”5 (Ministry of Education, 2018, p.  478). The focus on language use in the digital environment is seen as a way of adding to students’ meaning-making repertoire as well as empowering them to express their viewpoints either individually or collectively (Ministry of Education, 2018). Ahead of the start of the unit, students were made aware of the topic, and they were asked which digital genres were most recurrent in their experience. This discussion was used as an information-gathering exercise for the upcoming unit on digital genres. It stands as an example of bottom-­up syllabus design (i.e. the students were in charge of deciding the digital genres to be discussed in the subsequent classes), and it was a way of promoting student motivation, one of the features of assessment for learning. The most cited digital genres were online news stories and memes. Online news stories were deemed important to discuss in class because the students disclosed that, most of the times, they do not read the full stories. There was, therefore, the need to conduct some work on this front, which could potentially encourage students to read this digital genre more frequently and extensively. In addition, this genre had already been discussed in their Portuguese language classes, so it would be possible to establish an interdisciplinary link and to build on students’ prior knowledge. Memes, on the other hand, represented a digital genre of which students had both receptive and productive knowledge, that is, they reported having read memes in social media as well as having created memes themselves. The unit on digital genres started with an exploration of online news stories and then proceeded to consider memes. Table 4.1 provides a summary of the work conducted in the eight sessions in this unit. In Session 1, students’ prior knowledge was activated: they were asked to share their experiences as consumers of this digital genre. With student participation, the main features of online news stories (headline,

 Our translation of the following excerpt: “impactam seu dia a dia nos vários campos de atuação social”. 5

52 

V. Viana and J. Jandre

Table 4.1  Summary of the unit on digital genres Delivery Session mode 1

Synchronous

2

Asynchronous

3

Synchronous

4

Asynchronous

5

Synchronous

6 7

Asynchronous Synchronous

8

Asynchronous

Focal genre Task description Online news story

Meme

• Knowledge sharing regarding online news stories • Definition and identification of the main features of online news stories • Selection of one online news story and its analysis • Oral presentation based on the analytical work conducted for Session 2 • Analysis of an online news story selected by other students • Review of the genre-related features of online news stories • Introduction of memes and discussion of their main characteristics • Creation of memes • Interpretation/Understanding of another student’s meme • Reflection on meme design success and suggestions for improvement • Creation of a meme based in the online news story selected for Session 2

subhead, byline, lead, body) were defined before these features were identified in the examples that had been selected ahead of the session. For the subsequent asynchronous session, students were asked to complete an assessment-for-learning task individually. They had to find one online news story and post it in the high school’s VLE so that their choice could be checked. This assessment-for-learning provided useful information about students’ ability to identify examples of online news stories. Students were additionally requested to identify in their selected stories the main features that had been discussed in Session 1. Session 3 was dedicated to brief oral presentations: students had to present the analyses of their chosen online news stories. This assessment-­ for-­learning task was aimed at consolidating students’ knowledge of key concepts, developing their analytical skills and fostering their presentation skills in English. In line with the principles of assessment for

4  Assessment for Learning in the Brazilian High-School EFL… 

53

learning, students were asked to provide feedback to their peers, thus supporting their development. The following asynchronous session required students to take an active role in task design. In pairs, they had to select one text to be analyzed by a different pair. In other words, the students had to identify an example of an online news story; to evaluate its appropriateness in terms of, for instance, English language use and topic selection; and to analyze the story that their peers had assigned to them. This assessment-for-learning task was aimed at consolidating students’ knowledge of the online news story as a genre. It provided evidence of students’ ability to analyze an online news story outside their comfort zone since the story had been selected by someone else. Students’ answers were posted in the high school’s VLE, and teacher feedback was provided on the analysis. Session 5 worked as a transition from online news stories to memes, following the student-driven choice of digital genres. This was a challenging topic transition given the numerous differences between the two focal online genres. However, the transition was facilitated by the use of an online news story on the anniversary of a famous meme in the United States—the gorilla called Harambe6 (see https://www.huffpost.com/ entry/internet-­mourns-­harambes-­death_n_592c17c7e4b0065b20b7 77f4). Drawing on this specific text, the session progressed from a review of the online news story genre to a discussion of memes. Most of the students had not known about this meme since it went viral in 2016, a few years before this session was held. The class discussed the reasons for their lack of knowledge of this meme: memes are usually short-term context-­specific jokes which are recognizable by a specific community (e.g. age-specific and geographically specific communities). Some Harambe meme examples were shared with the students (see, for example, https://www.hitc.com/en-­gb/2021/05/29/harambe-­meme/), and they were asked to identify the features of memes (e.g. memes are captioned images/videos created for humorous purposes and aimed to be widely disseminated—mainly through social media).  After an incident of a kid falling in a gorilla’s enclosure at an American zoo in 2016 and the animal’s being killed because of it, many memes were created and circulated on the web using the Gorilla’s image. 6

54 

V. Viana and J. Jandre

For the asynchronous homework task, students were asked to work individually on the creation of a meme, which was to be posted on the high school’s VLE. Students were provided with some suggestions of free online meme generator tools (e.g. www.canva.com and www.inpixio. com/meme-­generator), and they were free to choose a topic or a situation of their interest. This production-oriented assessment-for-learning task evidenced students’ learning of this digital genre, their ability in the use of relevant software and their skills in using EFL creatively. In Session 7, students were randomly assigned a meme created by a peer of theirs and asked to explain what they understood by it. The meme creator was invited to respond by reflecting on the extent to which s/he had been successful in conveying his/her original idea. After this initial exploration of individual memes, the class was wrapped up with a final discussion on the features of memes. Students were presented with the same list of features from Session 5, and they were asked to identify how these features were present in their own memes. They were additionally encouraged to consider what changes could be made so that their previously designed memes could more successfully meet the usual features of this digital genre. The final session in the unit was aimed at bringing together online news stories and memes. Working in pairs, students were asked to review their selection of online news stories for Session 2 and choose one story that could be used as the basis of a meme. The resulting memes were posted in the VLE, and students received feedback on their memes. The assessment-for-learning tasks proposed in the unit on digital genres were characterized by a progression in their difficulty level. Students engaged in the identification of the focal digital genres before they analyzed the features of these genres. All the examples were made available in the high school’s VLE, thus increasing the input for these high-school students. The analyses were similarly shared with the entire class—either synchronously or asynchronously—as a way of providing students with a chance to learn from their peers. The final assessment-for-­ learning task that students were asked to undertake related to genre production. This was considered as a more difficult task in that they had to evidence (1) their knowledge of genre-related features, (2) their ability to

4  Assessment for Learning in the Brazilian High-School EFL… 

55

apply this knowledge productively, and (3) their ability to repackage the information discussed in one genre into another. The assessment-for-learning tasks prioritized a process-oriented approach to EFL education. Students’ learning journeys were more important than a quantitative evaluation of the outcome of the tasks, which used to be the focus in this high school before the COVID-19 pandemic. Students were provided with a chance to learn more about digital genres, and their evolving understanding could be monitored by the assessment-for-learning tasks.

Reflection on Assessment Practice Deviating from what Freire (1996) has described as the banking model of education, students were encouraged to take an active role in the EFL classroom and, more specifically, in assessment-for-learning tasks (e.g. Berry, 2008). They were invited to share their prior knowledge and to develop their understanding of the forms and functions of the two focal digital genres. Students’ mastery of online news stories and memes was scaffolded with different but related assessment-for-learning tasks. The scaffolding afforded by assessment-for-learning tasks enabled students to identify their learning difficulties (if any) and solve their doubts. The assessment-for-learning also provided evidence of most students’ attainment of the learning outcome. The assessment-for-learning tasks presented in this chapter had numerous advantages as summarized below. • Application-wise: The tasks can be conducted in numerous contexts. They were originally conducted online, but they can equally be implemented in on-site classes with minor changes. • Language-wise: The tasks followed a three-stage approach in which students were asked to identify, analyze and produce digital genres, thus providing them with a chance to expand their receptive (e.g. reading an online news story) and productive (e.g. creating their own meme) EFL knowledge.

56 

V. Viana and J. Jandre

• Motivation-wise: With regard to the link between assessment and motivation (e.g. Harlen, 2012), the teacher could observe that most of the students engaged in the activities proposed even though they knew they were not being graded. In order words, they seemed to be intrinsically—rather than extrinsically—motivated to participate and contribute to the sessions. The relationship between assessment for learning and motivation was also observed in Lee’s (2011) study with secondary-school students in Hong Kong, which evidenced an increase in their agreement with statements like “I am ready to take responsibility to improve my own writing” after the implementation of assessment for learning. • Learning-wise: Students seem to have perceived the assessment-for-­ learning tasks as regular class activities. For this reason, they experimented with language in a much freer environment than it would have been the case if the tasks had been perceived as part of high-stakes assessment of learning. Students had chances to learn with and from their peers rather than just the teacher. The teacher could notice that, while there were punctual infelicities in English language use, students were not facing any issues in their understanding of the two focal digital genres. This coheres with Lee and Coniam’s (2013) results, which also highlighted the role played by assessment for learning in supporting the EFL learning of secondary-school students in Hong Kong. • Teaching-wise: The teacher was able to monitor student learning in a continuous way. This is an important aspect in any educational context, but it was even more important in online teaching during the pandemic. The lack of visual contact with the students (i.e. most had their cameras off) made it very difficult to tell whether students were following the sessions. The teacher had to develop some strategies in order to stimulate students’ active participation. For instance, having students comment on one another’s presentations in Session 3 and memes in Session 7 was a strategy to engage students in the tasks more actively. This ensured that students paid attention to the presentations and read the memes as well as encouraged them to analyze the focal digital genres  critically. The assessment-for-learning tasks thus provided evidence of student learning and helped the teacher to make sound pedagogical decisions for the following classes.

4  Assessment for Learning in the Brazilian High-School EFL… 

57

• Assessment-wise: The assessment-for-learning tasks prepared students for the assessment of learning, which was similar in nature to the activities proposed during the synchronous and asynchronous sessions. All students met the passing criteria in the assessment of learning, and most of them excelled in it (i.e. their work was evaluated as either good or very good). This led the teacher to conclude the sequence of the assessment-for-learning tasks supported the students in evidencing their knowledge successfully. It was believed that digital genres would stimulate student participation in and engagement with online classes given the pervasiveness of these genres in students’ lives. Although several students actively contributed to the sessions, some of them did not complete the asynchronous assessment-for-learning tasks or participated actively in the synchronous sessions. Online education offers a lot of challenges, and one of them is the potential lack of contact between students and teachers. The teacher made an effort to contact the students who were not engaging in the activities with the help of the school’s pedagogical supervisor, but sometimes it was not possible to reach out to them. Should this have taken place in an on-site class, we believe it would have been easier to gauge student engagement and to address any issues. Another challenge faced in the implementation of the assessment-for-­ learning tasks was that most of them were conducted asynchronously. This was necessary given the limited amount of class time during the pandemic. Student completion of homework is an issue in the Brazilian context, and it was not different in this experience. We believe that more students would have completed the assessment-for-learning tasks if these tasks had been conducted during class time—ideally on-site, which was not possible during the pandemic. One challenge that teachers in other contexts might face in reproducing the sequence described in this chapter relates to class size. In this specific high-school context, EFL class size is set to a maximum of 12 students, which is not the case in many other educational contexts. The small group allowed the teacher to implement individual student presentations, which would be challenging with a large class. This means that

58 

V. Viana and J. Jandre

the activities would need to be tailored to these other contexts through, for instance, group work. As with any pedagogical proposal, the implementation of assessment for learning in this context was not a straightforward task (see also Black & Wiliam, 1998). However, it was a worthwhile initiative where the educational advantages to both the teacher and the students outweighed the challenges.

Practice Brief Some of the points that language teachers should consider when adapting the assessment-for-learning proposal discussed in the present chapter are summarized below: • Involve students in syllabus design (e.g. ask them about their preferences) as a way of raising their motivation to complete the assessment-­ for-­learning tasks and, more generally, to engage them in their learning. • Choose genres with which students are familiar so that the assessment-­ for-­learning tasks will help them deepen their knowledge of such genres and enhance their (analytical/critical) reading skills. • Aim to work with current and authentic texts which are socially relevant to students’ local context. • Carefully consider the sequencing of assessment-for-learning tasks in order to support student learning. • Analyze the evidence of learning attainment provided by the assessment-­for-learning tasks and use this information to orient pedagogical decisions.

References Assessment Reform Group. (2002). Assessment for learning: 10 principles— research-based principles to guide classroom practice. https://www.aaia.org.uk/ storage/medialibrary/o_1d8j89n3u1n0u17u91fdd1m4418fh8.pdf

4  Assessment for Learning in the Brazilian High-School EFL… 

59

Assessment Reform Group. (2006). The role of teachers in the assessment of learning. https://www.nuffieldfoundation.org/sites/default/files/files/The-­role-­of-­ teachers-­in-­the-­assessment-­of-­learning.pdf Bartlett, J. (2015). Outstanding assessment for learning in the classroom. Routledge. Batista, F. (2020). English language teaching in Brazil: A gap in policy, problems in practice. English Language Teaching, 13(8), 135–140. https://doi. org/10.5539/elt.v13n8p135 Berry, R. (2008). Assessment for learning. Hong Kong University Press. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8–21. Black, P. J., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. Canagarajah, S. (2011). Translanguaging in the classroom: Emerging issues for research and pedagogy. Applied Linguistics Review, 2, 1–28. Considera, A. (2019). Um museu de grandes novidades: A reforma do Ensino Médio e a BNCC [A museum of great news: The high-school reform and the national common curricular base]. In A. Gerhardt & M. Amorim (Eds.), A BNCC e o ensino de línguas e literaturas [The national common curricular base and the teaching of languages and literatures] (pp. 41–86). Pontes Editores. Cowie, B., Moreland, J., & Otrel-Cass, K. (2013). Expanding notions of assessment for learning: Inside science and technology primary classrooms. Sense Publishers. Education Guidelines and Bases Law, 9394. (1996). http://www.planalto.gov. br/ccivil_03/leis/l9394.htm Freire, P. (1996 [1970]). Pedagogy of the oppressed (M. B. Ramos, Trans.). Penguin. Green, A. (2014). Exploring language assessment and testing: Language in action. Routledge. Harlen, W. (2012). The role of assessment in developing motivation for learning. In J. Gardner (Ed.), Assessment and learning (2nd ed., pp. 143–152). Sage. Jenkins, J. (2007). English as a lingua franca: Attitude and identity. Oxford University Press. Jordão, C. M. (2004). A língua inglesa como ‘commodity’: Direito ou dever de todos? [The English language as a ‘commodity’: Right or duty of all?]. In J. P. Romanowsky, P. L. O. Martins, & S. R. A. Junqueira (Eds.), Conhecimento local e conhecimento universal [Local knowledge and universal knowledge] (pp. 287–296). Champagnat.

60 

V. Viana and J. Jandre

Lee, I. (2011). Bringing innovation to EFL writing through a focus on assessment for learning. Innovation in Language Learning and Teaching, 5(1), 19–33. https://doi.org/10.1080/17501229.2010.502232 Lee, I., & Coniam, D. (2013). Introducing assessment for learning for EFL writing in an assessment of learning examination-driven system in Hong Kong. Journal of Second Language Writing, 22(1), 34–50. https://doi. org/10.1016/j.jslw.2012.11.003 Lopes, A. C., & Silva, D. N. (2018). Todos nós semos de frontera: Ideologias linguísticas e a construção de uma pedagogia translíngue [We are all from the borderlands: Linguistic ideologies and the construction of a translingual pedagogy]. Linguagem em (Dis)curso, 18(3), 695–713. Ministry of Education. (2018). Base nacional comum curricular [National common curricular base]. http://basenacionalcomum.mec.gov.br/images/BNCC_ EI_EF_110518_versaofinal_site.pdf Ministry of Education and Sports. (1998). Parâmetros curriculares nacionais: Terceiro e quarto ciclos do ensino fundamental—Língua estrangeira [Brazilian national curriculum parameters: Primary education—Foreign language]. http:// portal.mec.gov.br/seb/arquivos/pdf/pcn_estrangeira.pdf Ministry of Education and Sports. (2000). Parâmetros Curriculares Nacionais: Ensino Médio—Linguagens, códigos e suas tecnologias [Brazilian National Curriculum Parameters: High school –Languages, codes, and related technologies]. http://portal.mec.gov.br/seb/arquivos/pdf/blegais.pdf Nasr, M., Bagheri, M. S., Sadighi, F., & Rassaei, E. (2019). Iranian EFL teachers’ assessment for learning practices and barriers: Do textbooks taught and teaching context matter? Cogent Arts & Humanities, 6(1), 1646691. https:// doi.org/10.1080/23311983.2019.1646691 Santana, J., & Kupske, F. (2020). De língua estrangeira à língua franca e os paradoxos in-between: (Tensionando) o ensino de língua inglesa à luz da BNCC. [From foreign language to lingua franca and the in-between paradoxes: (Tensioning) the teaching of English language in the light of the national common curricular b]. Revista X, 15(5), 146–171. Szundy, P. A. (2019). Base nacional comum curricular e a lógica neoliberal: Que línguas(gens) são (des)legitimadas? [The national common curricular base and the neoliberal logic: Which language(s) are (de)legitimized?]. In A. Gerhardt & M. Amorim (Eds.), A BNCC e o ensino de línguas e literaturas [The national common curricular base and the teaching of languages and literatures] (pp. 121–152). Pontes Editores.

4  Assessment for Learning in the Brazilian High-School EFL… 

61

Tílio, R. (2019). A base nacional comum curricular e o contexto brasileiro [The national common curricular base and the Brazilian context]. In A. Gerhardt & M. Amorim (Eds.), A BNCC e o ensino de línguas e literaturas [The national common curricular base and the teaching of languages and literatures] (pp. 7–16). Pontes Editores. Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. Wu, X., Zhang, L.  J., & Dixon, H.  R. (2021). Implementing assessment for learning (AfL) in Chinese university EFL classes: Teachers’ values and practices. System, 101, 102589. https://doi.org/10.1016/j.system.2021.102589

5 An Investigation into EFL Learners’ Perspectives Towards Dynamic Assessment Tuba Özturan, Hacer Hande Uysal, and Prithvi N. Shrestha

Introduction This chapter aims to present what Turkish tertiary level EFL students’ perspectives towards DA are and how they are engaged with it in Turkey. Driven by the growing interest in embedding learning into assessment, learning-oriented approaches, such as DA, have gained prominence in T. Özturan (*) School of Foreign Languages, Erzincan University, Erzincan, Turkey e-mail: [email protected] H. H. Uysal Department of Foreign Language Education, College of Education, Hacettepe University, Ankara, Turkey P. N. Shrestha School of Languages and Applied Linguistics, The Open University, Milton Keynes, UK e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_5

63

64 

T. Özturan et al.

language assessment (Poehner & Inbar-Lourie, 2020). DA is informed by Vygotskian Sociocultural Theory (SCT) and blends assessment with instruction. Accordingly, teachers actively intervene in the assessment procedure and simultaneously assess and instruct/mediate learners through dialogic interactions and graduated prompts in an implicit-­ explicit continuum (Poehner, 2008). DA deals with aligning feedback/mediation to L2 learners. Feedback has long been acknowledged as the hallmark for qualified instruction (Ferris, 2006) and to promote learners’ motivation (Hyland & Hyland, 2006). Therefore, a wealth of research has relied on feedback’s efficacy in different settings and the comparison of feedback types. Yet, no consensus could be achieved to date since these studies paid little attention to L2 learners’ differential needs and had unidirectional teacher-student relations. Heeding the inconsistent results of the related studies in unveiling the most effective feedback type/s, scholars, such as Nassaji (2017) and Storch (2018), have called for a need to adopt the SCT in feedback studies, which asserts that cognitive abilities are not fixed and cannot be pre-­ defined; rather their development is triggered by social interaction. Accordingly, as individuals interact with the people in their environment, they begin to think and behave in new ways, and mediation is embedded in this social interaction, which is unique to each person and affects each person’s cognitive development differently (Vygotsky, 1978). On the other hand, DA rests on an iterative, dialogic interaction-­ oriented, and systematic assessment process during which both the teacher and the students actively work in tandem, and there are three pivotal terms, namely microgenesis, the Zone of Proximal Development (ZPD), and mediation, in this cooperative work (Poehner, 2008). Vygotsky (1978) defines microgenesis as a quick response to reconstruct the learners’ ZPD (i.e., longitudinal development in a short span of time in a specific sociocultural setting) that reveals what they can achieve in the future without any assistance and mediation, and Feuerstein et  al. (2010) describes mediation as a goal-oriented interaction with the

5  An Investigation into EFL Learners’ Perspectives… 

65

learners to help them understand and solve problems in their outputs and trace their transfer abilities in new contexts. Within this nature, each learner’s matured abilities and needs should be identified first, and then, s/he should be supported through negotiated feedback based on the implicit-explicit continuum by the teacher during dialogic interaction in DA sessions. Because DA moots that each learner can succeed at a different rate and in different ways, the learners’ microgenetic development and likelihood to shift from other-regulation to self-regulation are salient to unpack their (ZPD) (Poehner, 2008; Shrestha, 2020). Also, the teacher assesses the learner’s responsiveness towards mediational moves to diagnose how s/he governs L2 in his/her mind and tracks microgenetic development over time. In a typical DA session, the most implicit mediational move is delivered initially, and the learner’s responsiveness is analyzed. No more subsequent mediational moves are provided when the learner can self-correct. However, if the learner fails to self-correct, the subsequent mediational moves become more explicit based on the learner’s needs. This dialogic teacher-learner interaction continues in many turns (Aljaafreh & Lantolf, 1994; Poehner, 2008). Recently, some influential studies have explored the use of DA in different L2 settings, and there has been a growing tendency to employ learning-oriented assessment approaches and analyzing teachers’ praxis (Poehner & Inbar-Lourie, 2020); however, far less work has taken account of the students’ insights into engaging with DA. Two recent studies (Davin & Herazo, 2020; Leung, 2020) reported the teachers’ positive perspectives towards learning-oriented assessment approaches, and Shrestha (2020) demonstrated learners’ positive perspectives on DA. However, more studies examining learner engagement with DA in different writing contexts are warranted. Against this backdrop, adopting a qualitative research methodology, this study aims to address the following research questions: 1 . How are EFL learners engaged with DA in a L2 writing setting? 2. What are the EFL learners’ perceptions of adopting DA in L2 writing classes?

66 

T. Özturan et al.

Methodology Participants Twenty female and five male students studying in the department of English Language Teaching at a state university in Turkey were recruited for this research. Regarding the importance of voluntary participation and ethical consent issues, the researcher informed the participants about the flow and aim of the study. A proficiency exam was administered at the beginning of the semester to diagnose the participants’ English language proficiency level. The group’s mean score was 21.68 out of 100, indicating that their English proficiency level was rather low. Besides, all participants were learning English as a foreign language for eight years, and they were aged between 17 and 19 years (Mean: 18).

Data Collection Procedure and Instruments This study adopted a grounded theory-oriented approach to collect data for thirteen weeks. It included the introduction of the study to the participants and the ethical consent procedure, the administration of the proficiency exam, individual interactive DA sessions, and the semi-­ structured focused group interviews. This study was a part of a Writing Skills course taught five hours weekly and comprised different writing genres, such as description, opinion, comparison/contrast, problem/solution, and narrative. After each genre was introduced to the students during class-hour, they were asked to write an approximately two-hundred-word text independently. Each student was provided with one-on-one interactive DA two times a week for ten weeks. During DA sessions, the teacher checked each text’s rhetorical (organization/coherence/cohesion) and linguistic (grammar/vocabulary choice/spelling/ punctuation) quality, assessed each learner’s needs, and provided interactive dialogic feedback through fine-tuned prompts based on Aljaafreh and Lantolf ’s regulatory scale (1994) grounded in their needs. These DA sessions (16 hours and 56 minutes) were audio-recorded to illustrate students’ engagement with DA. Lastly, semi-structured focused group interviews (two hours and six minutes) were conducted in five groups (five

5  An Investigation into EFL Learners’ Perspectives… 

67

students in each group) in Turkish to explore their perspectives on DA. The interview questions were as follows; however, the participants were free to share other comments and/or suggestions.

I nterview Questions 1 . What did you like about adopting individual DA in L2 writing? 2. What did not you like about adopting individual DA in L2 writing? 3. How and to what extent did individual DA contribute to the improvement of writing skills? 4. What were the disadvantages of adopting individual DA in L2 writing?

D  ata Analysis To illustrate how the students were engaged with DA, initially the audio recordings during dialogic interaction between the teacher/mediator and the students were transcribed through Transana, following some transcription conventions (see Appendix). The transcribed data shed light on microgenetic (within a session) and macrogenetic (between sessions) development of the participants (Nassaji & Swain, 2000). Then, interview sessions were transcribed, translated into English, and coded. Two different researchers worked on the coding procedure to assure inter-­ coder reliability. The consistency coefficient between the coders was found to be 0.83, which was a strong indicator for inter-coder reliability (O’Connor & Joffe, 2020).

Findings Students’ Engagement with DA To illustrate how the students were engaged with DA, some excerpts from two learners, Mia and Fiona’s (pseudonyms) sessions, would be discussed to unearth their growths and/or resistance to change within-­ session and between-sessions. Also, their differential needs in terms of

68 

T. Özturan et al.

mediational moves and changing reciprocity acts would be unpacked through these excerpts. Both students were chosen on purpose as they had difficulty in singularity-­plurality and subject-verb agreement, so the excerpts based on those linguistic items were also chosen on purpose to enlighten the individualized space for learning and assessment. The following excerpt, for example, belonged to Mia’s DA session held in the second week. Excerpt 1: (T: Teacher, M: Mia) T: Can you please read it again? (Mediational move 1—existence of an error) M: It has four windows that has a curtain ((she is reading the sentence)) T: four windows that ↑has (Mediational move 2—location of an error) M: yes (Reciprocity act 1—no response) T: that have (Mediational move 3—providing the correct form) M: yeah, that have. Windows are plural (Reciprocity act 2—partial autonomy)

That part of the dialogic interaction between Mia and the teacher included three graduated mediational moves and two reciprocity acts. Initially, the teacher implied that there was an error by underlining a sentence and wanted Mia to reread it. As she showed no response, the teacher provided a more explicit mediational move through intonation and by narrowing the location of the error, but Mia still could not self-­ correct it. Therefore, the teacher delivered the correct form, and then, Mia could find the underlying reason for the error. Mia also had a similar problem during the same session, and she could self-correct it after the teacher merely underlined the whole sentence that had an error (an implicit mediational move). The following excerpt reported her private speech in a session as she could find the error and self-correct it upon an implicit mediational move. Excerpt 2: (M: Mia) M: there is a yellow bags—no-no—there is a yellow bag ((self-correction)) (Reciprocity act 1—full autonomy)

5  An Investigation into EFL Learners’ Perspectives… 

69

Two weeks later, Mia had a similar error on singular-plural nouns. Initially, the teacher underlined the whole sentence with an erroneous part and wanted Mia to revise it (an implicit mediational move). Mia could narrow the location by responding ‘one out of every three person’ but could not find the error exactly and correct it. Then, the teacher guided her to revise the error again, and Mia answered that she was not sure and asked for metalinguistic cues. Upon this request, the teacher underlined the erroneous part and prompted Mia to focus on grammar. Mia could correct the error, yet with hesitation, so the teacher delivered the most explicit mediational move to clarify the topic. Excerpt 3: (T: Teacher, M: Mia) T: can you please read these parts again? ((by showing the underlined places)) (Mediational move 1—existence of an error) M: one out of every three person … (Reciprocity act 1—partial autonomy) T: so do you think it is correct? (Mediational move 2—looking for revision) M: I am not sure ↓ hm ↓ What kind of mistake?↓ (Reciprocity act 2—hesitation and looking for metalinguistic cues) T: Please focus on grammar … the blue part ((which was underlined by the mediator)) (Mediational move 3—nature and location of error) M: every three people? (Reciprocity act 3—partial autonomy) T: yes, we need a plural noun here

Fiona, also, had a singularity-plurality problem in her writing in the second week, and the teacher started to mediate her with the most implicit mediational move. She could immediately find the erroneous word; however, she was unable to correct it. Then, an implicit mediational move was again provided. She was still far from correcting the error and/or understanding the underlying reason for the error. Therefore, the teacher moved on with explicit mediational moves by giving metalinguistic cues, such as the location and nature of the error, and then Fiona could immediately self-correct and verbalize the reason. She had no similar problems in the same week.

70 

T. Özturan et al.

Excerpt 4: (T: Teacher, F: Fiona) T: do you think that this is a correct usage here? (Mediational move 1—the existence of an error) F: I think hmm true but hmm maybe destination (Reciprocity act 1—partial autonomy) T: what is wrong with it? (Mediational move 2—asking for clarification) F: ((no response)) (Reciprocity act 2—no autonomy) T: can you please focus on the grammar … “one↑ of the most attractive destinati↑on” ((the teacher uses intonation to emphasize the places which have mistakes)) (Mediational move 3—nature and location of error) F: destinations plural I need (Reciprocity act 1—full autonomy)

Fiona’s subsequent tasks were also analyzed to determine whether she had a similar problem or not. Although she had some problems in her writings, no erroneous outputs on singularity-plurality and/or subject-­ verb agreement were detected. Overall, the sample excerpts’ micro-analysis yielded two salient dimensions: First, the learners might have different needs in terms of feedback. Mia seemed to need more explicit feedback/mediational moves than Fiona. Second, their reciprocity acts differed. Fiona could display partial autonomy right after the most implicit mediational move, and she was autonomous to provide the correct answer and verbalize its reason after she got metalinguistic cues on the erroneous part. Mia was also delivered metalinguistic cues, but she replied with hesitation, which was an obstacle placed in the way. Thus, their performances during the process should be assessed accordingly to offer them mediation targeting their differing ZPDs.

Students’ Perspectives Towards DA The interview data analysis revealed three broad themes and a variety of sub-themes germane to the student participants’ perspectives towards employing DA in L2 writing classes. Students were asked the benefits of DA and drawbacks of DA during the interview, and their further comments and suggestions constituted the third theme, the need for integration of DA into L2 writing courses. Figure 5.1 shows the emerging themes and sub-themes:

5  An Investigation into EFL Learners’ Perspectives… 

71

Fig. 5.1  Students’ perspectives towards DA

B  enefits of DA The interviewees expressed four sub-themes on what they liked about adopting DA in L2 writing courses and what advantages it presented. A common view amongst the interviewees was individualized learning opportunities through DA (N: 21). Most of the participants highlighted that DA sessions helped them to recognize errors and understand their underlying reasons through dialogic interactions, which paved the way

72 

T. Özturan et al.

for narrowing the gap between their interlanguage and accurate L2 form/ model. Gina (pseudonym) illustrated this point of view by saying: It was the time that I could improve myself. Working individually with you was very productive to see my mistakes. It was especially good for me since I was bad in grammar. And the sessions helped me improve not only in correct language use but also in effective writing.

Furthermore, some students were likely to ignore the salience of rhetorical features, such as organization, cohesion, cohesive devices, and were unable to operate them in writing because they assumed that writing grammatically correct sentences would be enough for a well-written text. Although the students were instructed and assisted on these features during writing courses, some students reported that DA sessions created individualized learning spaces, which would lead to higher control in L2 writing. The comments of Sue and Sam (both pseudonyms) shed light on the issue: I was not good at writing, even in Turkish. I was thinking that writing sentences would be enough for a good text, I did not care how to combine the sentences because I did not know how to organize my ideas. I learned it during individual DA sessions. I could see my mistakes and weaknesses during these sessions. These sessions helped me improve myself. (Sue) I think writing is a very hard skill to achieve. It requires patience and a long process. We must control many things while writing. I could learn how to achieve them, see my mistakes and the reasons for my mistakes. (Sam)

Another emergent sub-theme was the teacher’s attentive and supportive role throughout DA sessions (N: 12). Theoretically, DA posits that instruction needs to be fused with assessment; in so doing, the teachers are supposed to actively intervene in the assessment process and provide mediational moves in an ad hoc manner. The emergent themes from the interview data revealed that almost half of the participants valued the merit of DA in L2 writing settings in situ. Bill (pseudonym) expressed his opinion by saying:

5  An Investigation into EFL Learners’ Perspectives… 

73

It might be difficult for our teacher to take care of all of us attentively during the course. Thanks to DA, more detailed analysis and evaluation of my writing made me focus on much. The tolerant attitude of our teacher was also the greatest support to me in this process. During DA sessions, it garnered my interest in the course even more positively by showing us everything in the most detailed and attentive way. (Bill)

Nine participants thought that one overarching feature of DA was related to the motivational dimension (sub-theme: increased motivation towards writing). They mentioned directly or implied their increased motivation towards writing was due to individual DA sessions. Ben (pseudonym) commented on the motivation-driven aspect of DA by saying “… When it is provided personally according to my needs, it motivates me and makes me feel valued…”. A minority of the participants (N: 5) emphasized the relaxing atmosphere of DA sessions, which helped them reduce their anxiety (sub-­ theme: lower anxiety) because, unlike a traditional classroom setting, DA sessions created a secure atmosphere and individualized learning space to talk about the learners’ errors, making the learners felt less anxious and more valued.

D  rawbacks of DA In addition to the benefits, the interviewees were also asked to share their opinions about the disadvantages of DA.  Though the majority of the students agreed that there was nothing to report as a drawback, some students touched upon limited time for each student during DA sessions (N: 4) and increased anxiety (N: 2) as the drawbacks of DA.  Even though most students were satisfied with the duration of the DA sessions (approximately 10 minutes each), a minority of the students criticized that the short-lived sessions posed a challenge for them. For example, Mia thought if she had had more allotted time, she could have had more opportunities to learn. She clarified her opinion by saying “I think the sessions should have lasted longer. Maybe the longer time we spent, the better we would be at writing.” Albeit her comment and desire to have longer time for each

74 

T. Özturan et al.

session, it is worth mentioning that the teacher dealt with each student’s needs in each session, and if needed, the teacher spent extra time with the student. On the other hand, a few students faced challenges and reported their increased anxiety during sessions. They mentioned that when the teacher showed an error or guided him/her for self-correction, they were disappointed and could not focus on the task. Therefore, they failed to take the advantage of learning opportunities during the dialogic interaction with the teacher. Sue, for example, clarified the sub-theme with these words: When I could not find my mistakes during sessions, I felt anxious. For example, we were moving step by step with your guidance, and sometimes you underlined a sentence and wanted me to revise it. At that time, I was very disappointed. One day you underlined a sentence again, but I could not find the mistake despite all your efforts. The problem was the lack of preposition; I had forgotten to write ‘to’ after ‘listen’, but because of my anxiety, I could not see it.

Integration of DA into L2 Writing Courses Some participants also shared their further suggestions or comments on the use of DA in L2 writing courses. The preoccupying idea of these students was the need to integrate DA into L2 writing courses (N: 23) since DA could present promising steps both for teachers and learners, such as individualized feedback and assessment, a more effective way of learning, and progress over time (sub-themes). Notably, most students (N: 14) valued DA because of its individualized feedback and assessment nature, and they criticized static assessment approaches as they lacked the opportunity for feedback and timely diagnosis of each learner’s development profile. DA might be a better alternative approach to fill the gap and assist the students to narrow the gap between their output and accurate L2 form.

5  An Investigation into EFL Learners’ Perspectives… 

75

Discussion and Conclusion Theoretically, DA is acknowledged as a merit for qualified instruction integrated with assessment, and some influential studies illustrated dialogic interaction during DA in different L2 settings. However, the learners’ perspectives towards adopting DA in L2 writing classes in situ is an underexplored strand. Against this backdrop, this chapter aimed to enlighten the issue by examining the mediation data (engagement with DA) and learner perspectives on DA, and the findings have a bidirectional relationship with the hallmarks of DA. First, DA values individuals’ differential needs for feedback and mediation (Poehner, 2008). The excerpts of Mia and Fiona are in accord with this assertion. To diagnose each learner’s needs and align feedback/mediation accordingly, teachers in this study acted as active agents during DA, which is in accord with Shrestha’s (2020) study. Second, DA is an iterative process, and the learners’ growth and needs between sessions and within-session are of importance. Moreover, promoting learning opportunities through tailoring mediational moves are at the forefront (Lantolf & Poehner, 2004). The sub-themes, such as individualized learning opportunities and a more effective way of learning are germane to these theoretical premises and consistent with that of Shrestha (2020). Third, subsequent mediational moves during DA are decided by observing and analyzing the learner’s reciprocity acts. Tracking each learner’s development over time is salient to reveal what that learner can achieve without assistance in the future (Poehner & Lantolf, 2013). The sub-themes individualized feedback and assessment and progress over time are in line with the underlying premises of DA. Fourth, Vygotsky (1978) highlights the intra-psychological plane or private speech in his writings since it might be a good indicator of shifting from other-regulation to self-regulation, having intentional control over his/her cognition, and acting independently. In this study, Mia’s excerpt displays her private speech and indicates her microgenetic growth within a session. Lastly, lower anxiety during DA sessions in this study corroborates the earlier findings of Davin and Herazo (2020), who stated decreased anxiety as a change experienced by the students (p. 209).

76 

T. Özturan et al.

Overall, dynamic assessment presents some pedagogical implications for L2 writing classes. Negotiation on the learners’ errors perpetuates language learning since it paves the way to narrow the gap between the learners’ interlanguage and accurate language use. Assisting the learners to self-correct through implicit mediational moves, explaining the underlying reasons for the errors through explicit mediational moves, and guiding the learners to verbalize the reasoning for their answers might create an individualized space within the cognitive-interactionist nature of DA (Ellis et  al., 2019; Infante & Poehner, 2019). Moreover, these factors might create opportunities for the learners to notice the error, understand its underlying problem, and learn it (Schmidt, 1990). Furthermore, DA is a process-oriented approach; therefore, tracking the nature and number of mediational moves provided to the learners and analyzing their development and/or resistance to change in time are essential to diagnose what a learner’s potential to achieve alone in the future is. Without a doubt, DA has some challenges for both teachers and students. The limited allotted time during individual DA sessions was mentioned by some interviewees since they feel that the more time they spend in a session, the more they learn.

Practice Brief To be able to use L2 accurately and communicatively is inherent in effective L2 writers; however, L2 writing teachers might be prone to handle merely linguistic errors of the students. However, L2 learners might need getting feedback for organizing a text effectively. Moreover, each student in a classroom shows differences in terms of his/her needs, so providing feedback in an individualized space would be helpful. Yet, employing one-on-one DA inside the classroom might not be applicable, so teachers may be prone to administering DA sessions outside the classroom. It might be hard to control time effectively, especially for the teacher who works in crowded classes. Alternatively, technology, such as Google Docs, can be integrated into DA practices. Writing teachers may also check all learners’ writing and note common errors down; then, they can

5  An Investigation into EFL Learners’ Perspectives… 

77

negotiate these errors inside the classroom. Further individual DA sessions might be offered to the students who are left behind the classroom. Even though initial DA sessions might create an extra burden for the teachers, subsequent DA sessions might require less time since the students’ development is faster.

Appendix Transcription Conventions ((comments)) Transcriber’s comments, includes non-verbal behavior ? Rising intonation, a question (hm, hh) these are onomatopoetic representations of the audible exhalation of air ↑↓ up or down arrows are used to indicate that there is sharply rising or falling intonation. The arrow is placed just before the syllable in which the change in intonation occurs Infante and Poehner (2019, pp. 87–88); Sert (2017, p. 24)

References Aljaafreh, A., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal, 78(4), 465–483. https://doi.org/10.2307/328585 Davin, K.  J., & Herazo, J.  D. (2020). Reconceptualizing classroom dynamic assessment: Lessons from teacher practice. In M.  E. Poehner & O.  Inbar-­ Lourie (Eds.), Toward a reconceptualization of second language classroom assessment (1st ed., pp. 197–217). Springer. Ellis, R., Skehan, P., Li, S., Shintani, N., & Lambert, C. (2019). Cognitive-­ Interactionist Perspectives. In Task-Based Language Teaching: Theory and Practice. Cambridge Applied Linguistics, (pp. 29–63). Cambridge University Press. https://doi.org/10.1017/9781108643689.006 Ferris, D. R. (2006). Does error feedback help student writers? New evidence on the short- and long-term effects of written error correction. In K. Hyland & F.  Hyland (Eds.), Feedback in second language writing: Contexts and issues (pp.  81–104). Cambridge University Press. https://doi.org/10.1017/ CBO9781139524742.007

78 

T. Özturan et al.

Feuerstein, R., Feuerstein, R. S., & Falik, L. H. (2010). Beyond smarter: Mediated learning and the brain’s capacity for change. Teachers College Press. Hyland, K., & Hyland, F. (2006). Feedback on second language students’ writing. Language Teaching, 39, 83–101. https://doi.org/10.1017/ S0261444806003399 Infante, P., & Poehner, M. E. (2019). Realizing the ZPD in second language education: The complementary contributions of dynamic assessment and mediated development. Language and Sociocultural Theory, 6(1), 63–91. https://doi.org/10.1558/lst.38916 Lantolf, J. P., & Poehner, M. E. (2004). Dynamic assessment of L2 development: Bringing the past into the future. Journal of Applied Linguistics, 1, 49–72. https://doi.org/10.1558/japl.1.1.49.55872 Leung, C. (2020). Learning oriented assessment: More than the chalkface. In M.  E. Poehner & O.  Inbar-Lourie (Eds.), Toward a reconceptualization of second language classroom assessment (1st ed., pp. 85–106). Springer. Nassaji, H. (2017). Negotiated oral feedback in response to written errors. In H.  Nassaji & E.  Kartchava (Eds.), Corrective feedback in second language teaching and learning: Research, theory, applications, implications (pp. 114–128). Routledge. Nassaji, H., & Swain, M. (2000). A Vygotskian perspective on corrective feedback in L2: The effect of random versus negotiated help on the learning of English articles. Language Awareness, 9(1), 34–51. https://doi. org/10.1080/09658410008667135 O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 1–13. https://doi.org/10.1177/1609406919899220 Poehner, M.  E. (2008). Dynamic assessment: A Vygotskian approach to understanding and promoting L2 development. Springer. Poehner, M.  E., & Inbar-Lourie, O. (2020). An epistemology of action for understanding and change in L2 classroom assessment: The case for praxis. In M.  E. Poehner & O.  Inbar-Lourie (Eds.), Toward a reconceptualization of second language classroom assessment (1st ed., pp. 1–20). Springer. Poehner, M. E., & Lantolf, J. P. (2013). Bringing the ZPD into the equation: Capturing L2 development during computerized dynamic assessment (C-DA). Language Teaching Research, 17(3), 323–342. https://doi. org/10.1177/1362168813482935 Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129–158. https://doi.org/10.1093/applin/11.2.129

5  An Investigation into EFL Learners’ Perspectives… 

79

Sert, O. (2017). Creating opportunities for L2 learning in a prediction activity. System, 70, 14–25. https://doi.org/10.1016/j.system.2017.08.008 Shrestha, P.  N. (2020). Dynamic assessment of students’ academic writing: Vygotskian and systemic functional linguistic perspectives. Springer. Storch, N. (2018). Written corrective feedback from sociocultural theoretical perspectives: A research agenda. Language Teaching, 51(2), 262–277. https:// doi.org/10.1017/S0261444818000034 Vygotsky, L. S. (1978). In M. Cole, V. John-Steiner, S. Scribner, & E. Souberman (Eds.), Mind in society. The development of higher psychological processes. Harvard University Press.

6 Exploring the Consistency Between Self- and Teacher Assessment: Using Co-Constructed Assessment Descriptors in EAP Writing in China Huahui Zhao and Beibei Zhao

Tell me and I forget, teach me and I may remember, involve me and I learn. —Benjamin Franklin

Introduction The role of self-assessment in language learning has been proliferated in recent years to develop learner agency. However, the development of learner agency depends on its embedded social contexts that can afford (Larsen–Freeman, 2019). In a teacher-driven learning context, learners have been heavily relying on teachers to provide feedback and doubted

H. Zhao (*) University of Leeds, Leeds, UK e-mail: [email protected] B. Zhao Zhejiang Shuren University, Hangzhou, China © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 S. W. Chong, H. Reinders (eds.), Innovation in Learning-Oriented Language Assessment, New Language Learning and Teaching Environments, https://doi.org/10.1007/978-3-031-18950-0_6

81

82 

H. Zhao and B. Zhao

their own capacity of assessing their learning (Chang, 2016; Zhao, 2010). Likewise, teachers have been accustomed to teacher assessment and doubt self-assessment (Zhao, 2018). In a teacher-driven learning context, a comparison study between self-and teacher assessment is essential to provide learners and teachers with empirical evidence of how reliable self-­ assessment is, where differences might lie and how to enhance self-assessment for learning.

Consistency of Self- and Teacher Assessment Different findings were reported across instructional settings when selfand teacher assessment results were compared. Sung et al. (2005) reported that teacher assessment provided a higher average mark than self-­ assessment while Chang et al. (2012) observed in their study that students provided much higher ratings than teachers. Overall, the meta-analysis of studies in higher education by Falchikov and Boud (1989) shows an average coefficient of 0.39 between self- and teacher marks and an average agreement percentage of 64% between self- and teacher assessors. The inconsistent findings across studies were explained by Falchikov and Boud (1989) and Falchikov and Goldfinch (2000). They suggested that the congruence of teacher and student ratings could be affected by assessment focuses and assessment criteria: analytic judgement is associated with a lower agreement than holistic judgement and well-­understood criteria; however, a higher level of familiarity with and ownership of criteria are associated with better agreements. Inconsistency between selfand teacher assessment could vary from the focuses of assessment. Chang et al. (2012) found that significant differences in self- and teacher grades existed in three out of seven assessment aspects. Bouzidi and Jaillet (2009) observed that the explicitness of assessment aspects and assessment instructions could decide the agreements. Training has been commonly believed to increase the congruence of teacher and student-led assessment (Rahimi, 2013; Zhao, 2014). The classroom settings (i.e., the instructional context relating to the curriculum and design of teaching and learning activities) could affect learning including assessment (Dörnyei, 2009).

6  Exploring the Consistency Between Self- and Teacher… 

83

Research Contexts This study was conducted during the English testing reform in China launched in 2014, aiming to develop China’s Standard of English Language Ability (hereafter as CSE) that bridges teaching, learning, assessment and learner autonomy (Ministry of Education of the People’s Republic of China, 2018). To fulfil this objective, this study integrated self-assessment into the existing teacher assessment with an adapted version of the European Language Portfolio (ELP), derived from CEFR and bearing resemblance with the action-oriented CSE (i.e. can-do statements) (Jiang, 2016). The project aimed to enrich the limited guidance on using action-oriented assessment descriptors in assessment and examined how the exam-driven learning culture in the Chinese would influence the implementation of self-assessment to promote coherence among teaching, learning and assessment (Zhao, 2018).

 esearch Context: Teacher-Driven R Writing Instruction Before the introduction of self-assessment, writing tutors played a dominant role in assessment. EAP writing in this research context was heavily based on a product-oriented writing approach. A typical lesson started with a teacher-led analysis of an exemplar article in terms of its structure and language use. The assessment was conducted solely by the writing tutors. Due to the large class size (over 50), little formative feedback was provided to justify the marks and explain the strengths and weaknesses of student writing. Introducing self-assessment into writing instruction was expected to develop learners’ autonomy of assessing their writing and the checklist of ‘I can’ statements at three scales [i.e. achieved (), almost there () and not there yet ()] was designed to use the minimum class time to maximise the value of self-assessment for improving writing quality and proficiency.

84 

H. Zhao and B. Zhao

Research Design This study was carried out in three phases. A pre-assessment phase addressed learners’ concerns over self-assessment and introduced the CEFR and ELP descriptors, followed by training in self-assessment with teachers’ demonstration of how to use the assessment criteria. In the assessment phase, students and teachers conducted self-assessment (inside classrooms) and teacher assessment (outside of classrooms) of the same piece of writing, using the co-constructed assessment criteria by teachers and students (Zhao & Zhao, 2020). In the post-assessment phase, the participants’ experiences of self-assessment were investigated. This paper focused on the data from the assessment phase and answered the following two research questions: 1. What was inter-rater agreement between self- and teacher ratings, using co-constructed assessment criteria? 2. What were the differences between self- and teacher ratings, using co-­ constructed assessment criteria? The quantitative results will be discussed with potential factors in the discussion section.

Participants Two tutors and 146 students from four classes and three subjects participated in the project for one semester voluntarily. All participants were Chinese who spoke English as a foreign language (EFL). Both tutors had been working at the institution for more than ten years and teaching the EAP module since 2016. The student participants were second-year university students, majoring in Network Media (Class 1–2), Public Management (Class 3) and Chinese Linguistics and Literature (Class 4). Most of the students had been learning English for more than 10 years since their primary school, with an approximate English proficiency level around B1-B2, based on their entrance English exam scores, writing scores and tutors’ judgement.

6  Exploring the Consistency Between Self- and Teacher… 

85

Table 6.1  Student participants’ backgrounds Class ID

Number of students

Gender

Major

Final writing marks (average)

1

35 (taught by Tutor 1) 35 (taught by Tutor 1) 29 (taught by Tutor 2) 47 (taught by Tutor 2)

Male: 16 Female: 19 Male: 13 Female: 22 Male: 8 Female: 21 Male: 3 Female: 44

Network Media

69.69 (SD = 7.66)

Network Media

67.79 (SD = 7.17)

Public Management

71.31 (SD = 7.79)

2 3 4

Chinese Language and 75.98 (SD = 6.22) Literature

They had limited self-assessment experience. Table  6.1 summarises the participants’ backgrounds. Descriptive analysis and an ANOVA test of English writing proficiency across classes showed no significant difference in writing proficiency among Class 1–3 but students in Class 4 scored 5 points higher in their average writing score than the other three classes. It is worthy of noticing the unbalanced number of male and female students which could affect the results as existing studies have suggested the gender difference in self-­ efficacy and use of CEFR in self-assessment (Denies & Janssen, 2016). Due to the limited space, the impact of participants’ backgrounds on the results would not be discussed in this paper.

Data Collection and Analysis Data were collected from two genres: summaries of reading about society today and food security and argumentative essays on sustainable energy and sustainable fashion. Thirty minutes were allocated for students for self-­ assessment. They were asked to tick one of the three options for each descriptor in the co-constructed assessment grids. Teachers used the same criteria to assess the same essays without reading the student self-ratings to avoid their possible influence on teacher ratings. Comparative analysis was conducted between self- and teacher ratings for each task. Kappa agreement for nominal data (K) was used to assess the agreement between teacher and self-assessment on a scale of slight

86 

H. Zhao and B. Zhao

(0.0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), or almost perfect (0.81–1.00) (Frey, 2018). Difference analysis was carried out to reveal whether students over- or underrated themselves, compared to their teachers via the Wilcoxon signed-rank test (WST) which suggested the specific number of the same self- and teacher ratings and different ratings with higher self-ratings and lower self-­ratings, respectively.

Findings The results were reported in the order of assessing constructing summaries, language use in summaries, constructing argumentative essays and language use in argumentative essays.

Self- and Teacher Ratings on Constructing Summaries Table 6.2 showed that teachers and students achieved significant agreements in six out of the nine descriptors (p