Equity and Formative Assessment in Higher Education: Advancing Culturally Responsive Assessment 3030716430, 9783030716431

Citation preview

Dorit Alt Nirit Raichel

Equity and Formative Assessment in Higher Education Advancing Culturally Responsive Assessment

Equity and Formative Assessment in Higher Education

Dorit Alt · Nirit Raichel

Equity and Formative Assessment in Higher Education Advancing Culturally Responsive Assessment

Dorit Alt Kinneret College on the Sea of Galilee Jordan Valley, Israel

Nirit Raichel Kinneret College on the Sea of Galilee Jordan Valley, Israel

ISBN 978-3-030-71643-1 ISBN 978-3-030-71644-8 (eBook) https://doi.org/10.1007/978-3-030-71644-8 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

During the past decades, cultural diversity has emerged as one of the most prominent challenges facing higher education institutions in multicultural societies. The reality of a diverse global student population seeking higher education has intensified the necessity to ensure that all students succeed regardless of their ethnic, religious and/or cultural differences. Indeed, instruction, learning, and assessment in a manner that takes into consideration the various needs of different students may lead to positive outcomes; however, incorporating such techniques into multicultural learning environments is often fraught with difficulties frequently related to multiple academic challenges encountered by minority students upon entering the higher education system (Hofhuis et al. 2016; Jiosi and Zalmanson-Levi 2018). These diversity-sensitive trends have amplified the calls for a pedagogy that embodies justice and equity that is suited to a multi-cultural society that aspires to meet culturally diverse student populations’ unique learning needs. This approach is characterized by ‘culturally responsive teaching’, which is adapted to the learners and the reality of their lives (Aronson and Laughter 2016). However, merely modifying teaching practices cannot solve the challenges faced by ‘minoritized’ students (Gay 2018). To be particularly successful in teaching all students equitably, the importance of ensuring that assessment is multifaceted and incorporates manifold manifestations of excellence should be acknowledged. The assessment instruments and procedures should be customized for students of different cultures and languages. To achieve equity, educators must question the standards and criteria being used to evaluate students by considering multiple paradigms and methods of assessment and evaluation that recognize existing cultural variances and address the needs of diverse student populations. Such a culturally responsive assessment, which might perhaps be deemed more appropriate in diverse, multicultural classrooms, could lead to more targeted improvements in both teaching and learning. This could be achieved by allowing students to demonstrate their learning in a variety of ways, reflect on their learning experiences, and actively participate in the learning and assessment process (Montenegro and Jankowski 2017). Despite increasing interest in and writing about culturally responsive teaching, very few studies in Europe have examined strategies that teachers employ to incorporate cultural responsiveness in their student assessments, nor have they considered v

vi

Preface

the challenges inherent in assessing minority students (Nayir et al. 2019). To fill this lacuna, this book explores the relationship between equity and assessment. It hinges on the notion that for an assessment to fulfill the goal of improving student learning in diverse classrooms, a culturally responsive approach to assessment is essential, and the entire institution must search for an optimal combination of solutions and support mechanisms that enable students to be successful. This can only be achieved by establishing a satisfactory, responsive culture of assessment. This book puts forth a rationale as to why change is necessary and proposes ways to make assessment culturally responsive. Throughout its chapters, aspects of instruction, learning, and assessment in higher education are discussed while shedding light on several formative assessment tools and methods used in higher education learning environments. Each chapter provides a rich theoretical review, followed by a case study accompanied by episodes, thoughts, and feelings experienced by students and instructors throughout the assessment processes. The challenges in using the proposed assessment tools and methods in culturally diverse classrooms are discussed, and suggestions for removing potential barriers, especially for minority students, are put forward. The case study research method defined by Yin (2014, p. 16) as “an empirical inquiry that investigates a contemporary phenomenon (the ‘case’) in depth and within its real-world context” was found fitting for the purposes of the current book. Case study methodology allows for a holistic, in-depth investigation of complex issues as expressed in the core theme of this book, which enable the researcher to go beyond the empirical results and evaluate the phenomenon directly through the subjective eyes of the participants. The case studies narrated throughout the book are informed by experiences gained mainly within the context of a joint international three-year European Union (Erasmus+) co-funded project titled Assessment Tools for Higher Education Learning Environments (ASSET, 2017–2020). In the ASSET project, assessment tools were either revised or newly devised to meet the needs of students from different cultures as they strive for academic success. The cases pertain to multicultural higher education institutions located in the northern periphery of Israel. In Israel, the Arab population represents 20.95% of the country’s population. Minority students comprise 14.3% of the overall student population; however, in the north of the country resides a majority of Arabs (53%). The students’ cultural diversity is characterized by language, religion, family structure, and ideological differences. Most Israeli Arab students are Muslim, and their first language is Arabic. In the public elementary and secondary schools in Israel, Jewish and Arab populations attend completely separate education systems with curricula governed by the Ministry of Education. In the Arab sector, the language of instruction is Arabic, with Hebrew taught as the compulsory second language, whereas in the Jewish system, Hebrew is the language of instruction and Arabic is an elective subject. Therefore, for most students, intercultural integration occurs primarily in higher education institutions (Pieterse et al. 2018). This radical transition from the familiar Arabic-speaking setting to the Hebrew-speaking setting of a college or university leads to manifold difficulties experienced by minority students but also provides fertile ground for the development of culturally responsive teaching

Preface

vii

and assessment approaches, like those discussed throughout this book. It should be noted that the book does not address specific cultural characteristics and needs of each group of students; it rather adheres to challenges experienced by minority students in higher education worldwide, as indicated in a preponderance of literature discussed in the following chapter. Hence, the conclusions and suggestions deriving from the cases narrated in this book can be implemented in other countries’ higher education institutions with appropriate adjustments that should be made to address the needs of their unique cultural groups of students. The book contains nine chapters: Chapter 1: Advancing Culturally Responsive Assessment Practices. This chapter reviews contemporary theory pertaining to culturally responsive assessment and discusses the multiple academic challenges encountered by minority students upon entering higher education. The call to address culturally diverse student populations’ unique learning needs by advancing culturally responsive assessment tools and methods is expounded. The chapter further presents an overview of the culturally responsive assessment method which will be discussed in later chapters. The following chapters are structured around three main facets: (1) intrapersonal; (2) interpersonal; and (3) institutional. The intrapersonal facet refers to the ability of assessment to diagnose and advance the individual’s skills that are requisite for the learning process, as elaborated and discussed in the following chapters: Chapter 2: Self-assessment of Digital Literacy Skills in a Multicultural Classroom. Student self-assessment mainly involves learners making judgments about their achievements and the outcomes of their learning. This chapter underscores the ways in which self-assessment can be used as a formative assessment method. In this chapter, students were encouraged to monitor and manage their digital literacy skills by developing the ability to effectively evaluate their own competence throughout their learning process. Chapter 3: Reflective Journal Writing and Multicultural Diversity. The ability to reflect on one’s learning is a fundamental and vital skill for lifelong learning. Therefore, developing students’ capacity to engage in reflective practices has been recognized as an essential goal in higher education and especially for preparing students effectively for their personal and professional development. Given the challenges experienced by minority students in multicultural higher-education environments, the question of how such higher order thinking skills can be practiced in a diverse classroom lies at the core of this chapter. Minority students’ challenges and the benefits associated with reflective writing processes are discussed. Chapter 4: Using Digital Badges in a Multicultural Classroom. With the growing attention being devoted to the phenomenon of academic underperformance among ethnic minority students, which has been widely explained by lower motivation for learning compared to their majority counterparts, this chapter seeks to explore how digital badges can be used effectively within multicultural learning environments. Additionally, it portrays the opportunities and challenges this assessment method holds for enhancing students’ motivation for learning.

viii

Preface

Chapter 5: Using Digital Concept Maps as an Assessment Tool in a Culturally Diverse Learning Environment. This chapter demonstrates how concept mapping can be used as an efficient instructional scaffold to support diverse students’ learning processes. It explores how these students design concept maps, use well-defined criteria for self-assessing the maps, and express their views on the use of concept mapping as an assessment method that measures and supports meaningful learning. The interpersonal facet has two main dimensions. The first is the connection between minority students and their peers from both the minority and majority groups. The second is the connection between these students and the teaching staff. The chapters related to this facet are the following: Chapter 6: Peer Assessment in Multicultural Classrooms: Tensions and Opportunities. Culturally responsive assessment strategies and methods provide opportunities for students to demonstrate their learning. One of these methods is peer assessment, which helps ethnic groups take ownership of their educational progress and assessment and, in a wider sense, enhances their involvement and inclusion in the classroom. In this chapter, multicultural students’ perceptions of and attitudes towards peer assessment are presented, with a focus on both the opportunities and the challenges posed by this assessment approach for students of diverse ethnic and cultural backgrounds. Chapter 7: Assessing Collaborative Learning in a Culturally Diverse Classroom. This chapter presents a collaborative group-learning activity designed for a multicultural learning environment and analyzes diverse students’ experiences and attitudes toward collaborating with students from a cultural background different from their own. The chapter attempts to equip higher-education teachers with useful information for designing, implementing, and assessing effective collaborative learning experiences among diverse populations. Chapter 8: Culturally Responsive Feedback. Feedback is a core component of the assessment process in higher education and an integral part of the learning experience, with a recent upsurge in scholarly attention being devoted to this domain. This chapter presents several contemporary aspects of feedback (seeking and giving) in higher education as perceived by students and teachers in multicultural learning environments. The institutional facet involves establishing an assessment culture within academic institutions as discussed comprehensively in Chap. 9, entitled “Culturally Responsive Culture of Assessment”. The main thrust of this chapter is to illustrate contemporary efforts to establish a culturally responsive culture of assessment in several multicultural academic institutions in Israel. This work might benefit assessment practitioners and aid in further theorization about how cultures of assessment can be developed in multicultural academic institutions worldwide. Lastly, the epilogue reviews the benefits of culturally responsive assessment, restates the goals of the book, and summarizes the best practices of formative assessments for higher education students of diverse ethnic and religious backgrounds. This volume provides theoretical, methodological, and practical information on how to mobilize instructional and assessment methods for social justice and equity

Preface

ix

in education for pedagogical and research purposes. Recognizing the power that educational research holds for advancing social justice and equity can be seen as a further step forward in generating practical societal, institutional, and educational change. This book provides readers with invaluable information about theory, classroom practices, and formative assessment methods applicable to students in a multicultural setting. A wide audience can benefit from this book, including students, teachers, policymakers, curriculum designers, and teacher educators who are interested in fostering initiatives in higher education aimed at adapting teaching-learningassessment processes to the unique learning needs of culturally diverse student populations. Israel

Dorit Alt Nirit Raichel

References Aronson, B., & Laughter, J. (2016). The theory and practice of culturally relevant education: A synthesis of research across areas. Review of Educational Research, 86(1), 163-206. Assessment tools for higher education learning environments [ASSET]. (2020). Erasmus+ Capacity Building in the Field of Higher Education. Erasmus+ programme of the European Union. https:// www.asset-erasmus.com/ Gay, G. (2018). Culturally responsive teaching: Theory, research and practice. (3rd ed.). Multicultural Education Series. New York: Teachers College Press, Columbia University Hofhuis, J., van der Rijt, P. G. A., & Vlug, M. (2016). Diversity climate enhances work outcomes through trust and openness in workgroup communication. SpringerPlus, 5, 714. doi: s40064016-2499-4 Jiosi, V., & Zalmanson-Levi, G. (2018). Multiculturalism and co-living between Jews and Arabs in teacher training academic institutions. Recommendations document. Tel-Aviv: The Association for Civil Rights in Israel. (Hebrew) Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment (Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Nayir, F., Brown, M., Burns, D., O’hara, J., Mcnamara, G., Nortvedt, G., Skedsmo, G., Silje Kristin Gloppen, S. K., Eline F. Wiese, E. F. (2019). Assessment with and for migration background students-cases from Europe.Eurasian Journal of Educational Research, 79, 39-68. Pieterse, E., Greenberg, R., & Santo, Z. (2018). A multicultural approach to digital information literacy skills evaluation in an Israeli college. Communications in Information Literacy, 12(2), 107-127. Yin, R. K. (2014). Case study research design and methods (5th ed.). Thousand Oaks, CA: Sage.

Contents

1 Advancing Culturally Responsive Assessment Practices . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Minority Students in Higher Education: Barriers Related to Academic Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culturally Relevant Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culturally Responsive Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culturally Responsive Assessment as a Formative Assessment Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culturally Responsive Assessment Realigns Power Relations Between Teachers and Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culturally Responsive Assessment Is an Outcome-Based Approach . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part I

1 1 1 3 5 6 8 10 12

Intrapersonal Facet

2 Self-assessment of Digital Literacy Skills in a Multicultural Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Self-assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Self-assessment in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Literacy Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Literacy Skills and Minority Students . . . . . . . . . . . . . . . . . . . . . . . . Digital Literacy Skills and the Learning Environment . . . . . . . . . . . . . . . . . Case Study: Using Self-assessment to Support Digital Literacy Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Intervention Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Self-assessment Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Between-Tests Differences in the Cultural Digital Literacy Skill . . . . . . . . Students Voicing Their Challenges and Gains . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19 19 19 19 21 23 24 26 27 28 29 30 31 35

xi

xii

Contents

Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36 37

3 Reflective Journal Writing and Multicultural Diversity . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflection: A Lifelong Learning Skill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflective Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflective Writing and Future Professional Development . . . . . . . . . . . . . . Reflective Journal Writing and Diversity . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reflective Journal Dimensionality and Structure . . . . . . . . . . . . . . . . . . . . . Minority Students’ Challenges and Benefits Related to Reflective Writing Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thinking Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emotional Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collectivist Versus Individualist Culture . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prior Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41 41 41 42 43 44 45

4 Using Digital Badges in a Multicultural Classroom . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is a Digital Badge? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Badges and Lifelong Learning Skills . . . . . . . . . . . . . . . . . . . . . . . . Can Digital Badges Increase Motivation for Learning? . . . . . . . . . . . . . . . . Minority Students and Motivation for Learning . . . . . . . . . . . . . . . . . . . . . . Digital Badges—Limitations and Challenges . . . . . . . . . . . . . . . . . . . . . . . . The Intervention Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Students’ Perspectives on Digital Badges in a Diverse Learning Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Motivation for Learning: Who Benefits the Most from Digital Badges? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Self-regulated Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Digital Badges and Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Openness to Change and Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Using Digital Concept Maps as an Assessment Tool in a Culturally Diverse Learning Environment . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What Is a Concept Map in Teaching? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Maps for Constructivist Learning . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Maps in Social Studies Education . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Mapping and Individual Differences . . . . . . . . . . . . . . . . . . . . . . . .

46 48 50 50 51 52 52 54 57 57 57 58 59 60 61 62 64 64 65 66 67 68 69 70 73 73 73 74 75 76

Contents

xiii

Assessment of Concept Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Concept Mapping in a Diverse Classroom—A Case Study . . . . . . . . . . . . Visual Clarity and Simplicity of Presenting the Material Using a Concept Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Organizing the Learning Material—Concept Mapping as a Means of Self-assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creativity and Connection to Previous Knowledge . . . . . . . . . . . . . . . . . . . The Technological Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Importance of Criteria and Feedback in Assessment Processes . . . . . Implications for Future Professional Practice . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part II

77 78 80 82 83 84 84 85 85 86 88

Interpersonal Facet

6 Peer Assessment in Multicultural Classrooms: Tensions and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Peer Assessment: Challenges and Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Peer Assessment in a Diverse Classroom: Opportunities and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Peer Assessment Implementation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Gaining Experience over Time and Practice . . . . . . . . . . . . . . . . . . . . . . . . . 97 Feedback for the Purpose of Advancement and Growth . . . . . . . . . . . . . . . 98 Teacher-Centered Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Cultural Group Dynamics and Peer Assessment . . . . . . . . . . . . . . . . . . . . . 99 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Implication and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 7 Assessing Collaborative Learning in a Culturally Diverse Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Collaborative Learning for Meaningful Interactions Across Cultures . . . . Minority Students and Collaborative Learning . . . . . . . . . . . . . . . . . . . . . . . Designing a Collaborative Group-Learning Activity in a Diverse Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessing Perceptions Towards Collaborative Learning . . . . . . . . . . . . . . . Lack of Language Proficiency and Uneven Workload Distribution . . . . . . Coping with Different Cultural Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Who Benefits from the Fruits of Collaboration? . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implication and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

105 105 105 107 108 109 109 110 111 111 112 114

xiv

Contents

8 Culturally Responsive Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback in Higher Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Effective Feedback for Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Barriers to Effective Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Barriers to Effective Feedback in Culturally Diverse Learning Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback in a Multicultural Learning Environment: The Students’ Voice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Feedback and Ways of Obtaining It . . . . . . . . . . . . . . . . . . . . . . . . Familiarity with Feedback Prior to Pursuing Academic Studies . . . . . . . . Barriers of “Respect” and “Embarrassment” Against Seeking Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Who to Address: Alternative Sources for Feedback . . . . . . . . . . . . . . . . . . . Sharing Feedback with Others . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Feedback for Personal, Group, or Collective Work . . . . . . . . . . . . . . . . . . . Building Bridges of Trust: Lecturers’ Points of View . . . . . . . . . . . . . . . . . Providing a Continuous Flow of Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . Personal Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

117 117 117 118 120 120 123 124 124 125 126 127 128 128 129 129 130 131 132

Part III Institutional Facet 9 Culturally Responsive Culture of Assessment . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culture of Assessment: Definitions and Features . . . . . . . . . . . . . . . . . . . . . Commitment to a Culture of Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . Challenges in Building a Culture of Assessment . . . . . . . . . . . . . . . . . . . . . Culture of Assessment in Israeli Academic Institutions . . . . . . . . . . . . . . . Lack of Suitable Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lack of Commitment to a Common Vision . . . . . . . . . . . . . . . . . . . . . . . Lack of Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disparities in Learning Skills Among Students from Multicultural Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implications and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137 137 137 139 140 142 149 150 150 151 151 153 154

Epilogue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Chapter 1

Advancing Culturally Responsive Assessment Practices

Introduction During the past several decades, cultural diversity has emerged as one of the most prominent challenges facing higher education institutions in multicultural societies. In this context, a preponderance of literature indicates that instruction, learning, and assessment adapted to students’ various needs may contribute to effective learning; however, incorporating such techniques and methodologies into multicultural settings might be met by numerous academic challenges experienced by minority students in higher education (Hofhuis et al. 2016; Jiosi and ZalmansonLevi 2018). This chapter reviews current theories regarding contemporary assessment strategies and discusses ways in which these strategies can be responsive to the multiple academic challenges which minority students encounter in higher education. The call to address the unique learning needs of culturally diverse student populations by advancing culturally responsive assessment tools and methods is emphasized herein. The chapter opens with an overview of the unique needs of higher-education students in a multicultural setting, followed by a description of the properties of culturally responsive assessment, which will be elaborated on and exemplified in later chapters.

Minority Students in Higher Education: Barriers Related to Academic Skills Research shows that when transitioning into academic institutions, minority students have to bridge two divergent cultures: that of their own ethnic community and that of the campus (Hager and Jabareen 2015). Among the numerous obstacles that may hinder minority students on their path to graduation, two are particularly prominent: language proficiency and scholastic skills. Across Europe, robust monolingual © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_1

1

2

1 Advancing Culturally Responsive Assessment Practices

education policies and practices are notable, “The emphasis lies on language proficiency in the dominant societal language as the key to educational success… The use of languages or linguistic repertoires, other than the dominant societal one, are not valued in educational policies, nor in educational practices” (De Backer et al. 2017, p. 217). These policies and practices, in turn, affect the assessment of students’ (language) proficiencies and practices. Language proficiency is thus recognized as a strong factor, impacting on minority students’ opportunities to participate and succeed in classroom and assessment practices. Similar policies exist in the Israeli educational system, which consists of sectors divided according to ethnicity and religious affiliation. The Arab population, for example, is taught in an Arabic-speaking K-12 education system; however, in academia, these minority students are expected to understand and summarize lectures delivered in Hebrew (the dominant national language), to read, comprehend and write academic papers in Hebrew and in English, and to present their academic work orally in either their second or third language (Manor 2016). Even those students whose Hebrew (L2) language skills are relatively high find themselves, for the first time in their lives, facing the challenge of coping with a fully Hebrew-speaking system (Dirasat 2011). Hence, these first- and second-generation college students in their family, whose first language is not the language of instruction, often have significant linguistic handicaps and needs (Burns et al. 2019). According to Nordvedt (2018), adaptive practices such as preparing glossaries or translating key vocabulary might be more of a hindrance than anticipated due to the lack of transferability and translatability of many of the concepts. Lack of students’ proficiency in the country’s dominant language may lead to their instructors adopting the “deficit” model approach whereby students are considered a “problem” requiring compensatory education similar to remedial education for students with special needs (Burns et al. 2019). This negative response can devalue students’ ethnic culture and native language and “can have a negative impact on their overall achievement and motivation… Evidence suggests that treating linguistic and cultural diversity as a resource, rather than a deficit, and adopting a multilingual habitus in educational policy and practice can also be a valuable approach to promote communicative competence and foster academic performance” (Herzog-Punzenberger et al. 2017, p. 24). Indeed, students must learn the dominant language in order to succeed academically; however, a meaningful component is the manner in which teachers approach this issue within a social justice framework; that is, without working from a deficit perspective or marginalizing students’ first languages and home cultures (Skinner and O’Toole 2018). The second major barrier related to academic skills can be partly explained by the disparity that exists between minority and majority students in secondary education. As argued by Richardson (2011, p. 125) with regard to UK students: One reason for this attainment gap is that ethnic minority students enter UK higher education with lower entry qualifications than those of White students…This explains about half of the disparity in degree attainment between White and ethnic minority students… Nevertheless, even when the effects of variations in entry qualifications and other demographic and institutional variables have been statistically controlled, White students are still more likely to

Minority Students in Higher Education: Barriers Related …

3

obtain good degrees than are students from other ethnic groups... An alternative possibility is that ethnic minority students are more likely to encounter discriminatory teaching and assessment practices or more subtle exclusionary attitudes and behaviour on the part of their teachers or other students…

In Israel, this qualifications gap is also explained by the differences that exist between the formal Arab education K-12 system in Israel, which mainly employs traditional teaching methods, and academia teaching, which encourages creative and critical thinking (Ben-Rabi and Hanadin 2013). In Arab schools, the traditional cultural environment, which includes patterns of unquestioned acceptance of authority, lags behind in the adoption of digital technology. As a result, “the prospective academic students are not exposed at all to learning experiences that encourage questioning and critical reasoning, are not required to independently process information … to independently formulate an argument or conclusion, and so on. These are the basic skills required for academic studies” (Shaviv et al. 2013, p. 42). Drawing on previous research (Cummins 1979, 2001; Little 2010), Skinner and O’Toole (2018) explained both barriers by highlighting the distinction between social/conversational versus academic language proficiency. Conversational language is typically context-embedded and supported by paralinguistic cues and is generally acquired within one to two years, whereas academic language is contextreduced, more abstract, and involves complex vocabulary and features such as hypothesizing, persuading, classifying, arguing, and speculating. The acquisition of academic proficiency requires five to seven years (Hancock 2018). Hence, minority students pursuing higher education face a two-fold challenge; first, they lack adequate academic and linguistic skills such as reading, writing, and analyzing of texts, independent work, analytical discussion, and independent use of different resources at the level required for university studies. Second, many of them prefer a teacher-centered learning environment wherein lecturers are expected to convey the required information rather than having students seek out answers for themselves (Hager and Jabareen 2015). Notably, with the growing attention devoted to international migration, issues of the integration and adaptation of foreigners have emerged as being crucial for the immigrants and the host society alike (Dixon and Wu 2014). Academic skills are considered key elements among many success factors in integration. Without adequate skill sets, it is very hard to acquire further education (Choi and Ziegler 2015).

Culturally Relevant Teaching These diversity trends have generated an increase in the calls to adopt a pedagogy of justice and equity, suitable for a multicultural society, which addresses the unique learning needs of culturally diverse student populations. For example, Banks (2017) proposed an interim approach whereby the curriculum should include a common core of knowledge and values for all citizens, while at the same time reflecting the interests of deprived groups. In addition, programs should be developed to provide learners

4

1 Advancing Culturally Responsive Assessment Practices

from society’s weaker populations with knowledge and critical thinking skills. Banks presented five elements of education that are suitable for a multicultural society: (1) integrating learning materials and content reflecting a diversity of cultures into the curriculum; (2) reducing prejudices; (3) developing teaching and assessment methods that suit the learning strategies of all the groups; (4) raising awareness regarding the powers that shape knowledge; and (5) forming and developing an empowering school culture. Ladson-Billings (1994) coined the term ‘culturally relevant teaching’ to describe a pedagogy that “empowers students intellectually, socially, emotionally, and politically by using cultural referents to impart knowledge, skills, and attitudes” (pp. 17– 18). This pedagogy focuses on social justice and depicts the classroom as a fertile place for social change, together with a practical and moral acknowledgement of the rights of diverse groups. There are three central tenets underpinning this pedagogy: (1) upholding high expectations for all students, (2) assisting students in the development of cultural competence, and (3) guiding students in the development of a critical cultural consciousness. This approach is characterized by ‘culturally responsive teaching’, which uses knowledge to conduct relevant learning that is adapted to the learners and the reality of their lives. Culturally responsive teaching validates the culture of each student and bridges the gaps between multicultural learning programs and varied teaching strategies (Aronson and Laughter 2016). Developing cultural awareness is crucial to proper functioning in a culturally diverse society. Consequently, educators are expected to possess the knowledge and professional flexibility to understand their students’ ethnic, social and cultural differences, to enrich their knowledge about multiculturalism, and to identify explicit and implicit dilemmas in the cultural context (Abington-Pitre 2015). Culturally responsive pedagogy is thus a student-centered approach to teaching in which the uniqueness of each student is not just acknowledged but nurtured (Ford and Kea 2009). This pedagogy contains cultural references and recognizes the centrality of students’ cultural backgrounds and experiences in all aspects of learning. It integrates a student’s background knowledge and lived experiences into the teaching and learning experiences that take place in the classroom. Gay (2013, 2018) pointed to a number of signs of culturally relevant education. The prominent ones include the following: 1. Erecting bridges between the learners’ culture and academic ideas, while underlining the essential aspect of culture and its effect on teaching that includes historical facts and events and the eradication of stereotypical and racist perceptions; 2. Developing critical reflection through the use of programs and activities that include analysis of all the cultures represented in the classroom; 3. Fostering the learners’ cultural pride. Others suggest conducting a dialogue centered around identity and diversity that takes place within an educational framework so as to develop awareness and empowerment among the students, as well as changing the educational environment so that it promotes respect and social justice for all learners. Additionally, it is suggested to develop self-awareness that includes learning one’s own ethnic and cultural heritage, thereby, in turn, enhancing self-esteem and motivation (Aronson and Laughter 2016).

Culturally Relevant Teaching

5

However, it has been argued that solely modifying teaching practices cannot solve the challenges faced by ‘minoritized’ students (Gay 2013, 2018). To be successful in teaching all students equitably, Ladson-Billings (1994) also underscored the importance of ensuring that assessment is multifaceted and incorporates multiple forms of excellence. The assessment instruments and procedures should be valid for students of different cultures and languages. Tests, for example, that are not sensitive to students’ cultural and linguistic backgrounds “will often merely indicate what the students don’t know (about the mainstream culture and language) and very little about what they do know. Thus, the opportunity to build on what students know is lost” (Krasnoff 2016, p. 19). Notably, much attention has been paid to culturally responsive teaching in schools, and several studies dealing with its integration in higher education have been produced. For example, Ginsberg and Wlodkowski (2009) produced a comprehensive guide pertaining to higher education. In their book, they help educators create a supportive community of learners, facilitate equitable discussions in linguistically diverse classrooms, design engaging lessons, and assess students fairly. The book features teaching practices that can be easily adapted to a range of postsecondary settings. Yet, measurement tools included in the appendices of their book reflect the expectations of educators who are members of the dominant culture (Carjuzaa and Ruff 2010). In further navigating the literature, it seems that the field of assessment has been largely silent when it comes to issues of equity. Instead, assessment has remained largely unchanged in the context of inclusivity, and little urgency has been assigned to ensure that students are provided with just and equitable means to demonstrate their learning, as noted by Montenegro and Jankowski (2017, p. 5): Intentionally choosing appropriate assessment tools or approaches that offer the greatest chance for various types of students to demonstrate their learning so that assessment results may benefit students from all backgrounds advances our collective interest in student success. Without examining issues of equity, the students who may stand the most to gain from assessment efforts may have the least benefit since their learning is not accurately assessed, and feedback may not be relevant to impact learning.

In summary, the culturally responsive classrooms tenets that need to be considered for equitable participation of and for minority students are: ‘Curriculum’ (Banks 2017); ‘Philosophy’ (Ford and Kea 2009); ‘Instruction’ (Gay 2018); ‘the Learning Environment’ (Alt and Raichel 2018); and ‘Assessment’ (Montenegro and Jankowski 2017). This book centers on one core tenet of cultural responsiveness, often referred to as culturally responsive assessment.

Culturally Responsive Assessment One of the challenges faced by education in the twenty-first century is to instill humanistic values and social skills, which are necessary to shape the learner into a complete person who can survive and thrive in the world. There is, consequently, an

6

1 Advancing Culturally Responsive Assessment Practices

urgent need to take steps to cultivate the social values and skills based on ethical foundations (UNESCO 2002). Tolerance, appreciation, pluralism, and respectful engagement are worthy of greater emphasis in education policy (Veugelers et al. 2017). This type of learning develops mutual respect, caring, social responsibility, acceptance, and solidarity towards a diversity of (ethnic, social, cultural, sectoral, national, etc.) individuals and groups. The goal is to solve problems by means of cooperative practices so as to create a just, free, peaceful and democratic society (Council of Europe 2016). To achieve equity, educators must delve into the standards being used to evaluate the student by considering multiple paradigms and methods of assessment and evaluation that are cognizant of cultural variations that exist and meet the needs of diverse student populations in addition to and beyond the dominant worldview (Gay 2018). However, Zeichner (2003) points to a cultural divide between teachers and their students which becomes “further complicated by the lack of sustained attention to preparing teachers to teach across lines of ethnicity/race, language, and social class in most teacher education programs” (p. 493). This cultural divide is just as apparent in higher education as it is in K-12 classrooms, and efforts to bridge this gap by modifying instructional strategies have been widely documented by numerous scholars. However, published literature discussing a multicultural perspective of assessment in higher education is scant (e.g., Carjuzaa and Ruff 2010). Similarly, in the context of schools, very few studies in Europe (e.g., Erasmus + funded project titled Aiding Culturally Responsive Assessment in Schools [ACRAS] 2018) have examined assessment strategies utilized by teachers to integrate cultural responsiveness into student assessments, but such efforts have overlooked the challenges these strategies pose to assessing minority students with a migrant background. As will be discussed below, culturally responsive assessment can be described as a formative assessment that goes beyond simply modifying instructional strategies to meet the needs of diverse learners and providing a representative view of multiple cultures. It specifically utilizes strategies to acknowledge and respect learners’ diverse cultural backgrounds and approaches to learning as they strive for academic success (Nayir et al. 2019).

Culturally Responsive Assessment as a Formative Assessment Approach Several methods of assessment delineated in the following chapters have the potential to be culturally responsive. These are associated with the formative assessment approach rather than a summative assessment. It has been well documented that in educational institutions, the most visible assessments are summative; specifically, this is an assessment that attempts to evaluate learning outcomes in stark contrast to formative assessment, which aims to improve teaching and learning (Dixson and Worrell 2016). Hue and Kennedy (2015) argue that although teachers are supposed

Culturally Responsive Assessment as a Formative Assessment …

7

to be responsible for implementing assessment, research has confirmed that the summative assessment has an impact on teacher’s assessment practices. Indeed, the benefits of standardized testing cannot be overlooked. Tests are used to ensure high-quality education, monitor national progress, and provide right of entry into tertiary education (Nayir et al. 2019). However, one of the major concerns when testing students in a language that is not their mother tongue is the issue of validity. De Backer et al. (2017) point out that the use of other “foreign” languages or linguistic repertoires is not valued in educational policies, nor in educational practices. Students who do not speak the local dominant language are nonetheless required to take content-related tests in a language with which they are often insufficiently familiar. Drawing on Vygotsky’s (1978) theory of cognitive development and the zone of proximal development (ZPD), the authors claim that teachers actually require these students to perform and demonstrate achievements outside of their ZPD. Kim and Zabelina (2015) provide some examples to substantiate the premise that cultural and social norms affect how students understand and interpret the wording of test questions: How they make sense of the test items can be influenced by their values, beliefs, experiences, communication patterns, teaching and learning styles, and epistemologies of their cultures and societies… For instance, the meaning of educacion in Spanish is different from that of education in English... The social skills of respectful and correct behavior are important to the Spanish when they state educacion, whereas only cognitive processing is important to education in many Western societies (p. 131).

Alternatively, formative assessment is described as assessment for learning, in which evaluation and instruction are regarded as two separated but integrated tasks which must be performed by the teacher. Assessment is learning. Feedback and scaffolds are provided during increasingly complex tasks to encourage multilingual students to perform within their ZPDs (De Backer et al. 2017). Formative assessment refers to all the activities performed by teachers, and by students in self-assessment, which provide information that can serve as feedback to change the teaching and learning activities that it relates to. Formative assessment is defined as: ... frequent, interactive assessments of student progress and understanding to identify learning needs and adjust teaching appropriately. Teachers using formative assessment approaches and techniques are better prepared to meet diverse students’ needs – through differentiation and adaptation of teaching to raise levels of student achievement and to achieve a greater equity of student outcomes (OECD/CERI 2008, p. 1).

Formative assessment takes advantage of a wide variety of assessment tools. The tool must fit the type of task performed by the students. The students are directly involved in assessments, for example, by means of self-assessment or peerassessment techniques, which is how they participate not just in structuring the learning but in assessing it as well. These forms of student participation in assessment refer to dialogic and dynamic relationships between teachers and students, and to what López-Pastor (2009) referred to as ‘shared assessment’. The teacher’s role is to identify the learning outcomes, clearly define (with the students) criteria for success, and design learning activities and assessment tasks. The students’ involvement in

8

1 Advancing Culturally Responsive Assessment Practices

their own learning assessment helps them to obtain descriptive feedback from the teacher that allows them to improve their own learning process. Formative assessment also provides the teacher with information about the quality of his/her own teaching (Alt and Raichel 2018). This assessment method requires identifying the desired learning outcomes and how to achieve them; giving clear and detailed feedback in an effective and efficient way; encouraging active involvement of the students in their learning process; enhancing good student–teacher communication; and maximizing the availability of the teacher for the students. It involves teachers and students in a joint activity that is characterized by shared involvement in the process, collection of and reactions to evidence of learning by providing scaffolds that include feedback, and self-regulation by the learners. Additionally, formative assessment occurs within a community in which both the teacher and the students take part, set goals, experience practices and norms of mutual support, trust, respect and cooperation during interactions aimed at supporting diverse student populations, and thus can be considered a culturally responsive assessment approach.

Culturally Responsive Assessment Realigns Power Relations Between Teachers and Students A natural tension exists between standards-based assessment and culturally responsive assessment. The latter challenges the traditional power relations that exist in the university and its academic and professional fields. Developing a culturally responsive assessment does not mean merely adopting new assessment techniques; it signifies fundamentally realigning power relationships and redefining what it means to be an ‘authority’ or an ‘expert’ in a particular field (Lorente and Kirk 2013). Alt and Itzkovich’s (2018) review of previous studies reinforces this notion by demonstrating how teacher-student power relations are reflected differently in the traditional teacher-centered versus the student-centered approach to teaching. The former’s view of teaching, learning, and assessment holds that learners must submit themselves to the authority of their teachers, which is never to be challenged. The underlying assumption is that without the communication of the teacher’s power over the student, the student is unable to learn, and, to some extent, power becomes the essence of the learning process rather than a means to its ends (McCroskey and Richmond 1984). This teacher-centered approach is based upon the process of overcoming natural inclination and replacing it with habits acquired under external pressure applied by teachers (Dewey 1938). In traditional classrooms, knowledge flows only in one direction from teacher to student. As a result, students tend to have an attitude of docility, receptivity, and obedience (Hammerness et al. 2005; Sfard 1998). This traditional mode is, in essence, one of imposition from those in authority upon their students. Among other inherent flaws of traditional education are imposition of adult standards; teaching knowledge as a set of static principles; suppression of creativity, expression, and the cultivation of individuality; lack of learning through

Culturally Responsive Assessment Realigns Power Relations Between …

9

experience; acquisition of isolated skills without context; construction of the world as static rather than ever-changing; and authority-based professional discourses (Beck 2009; Dewey 1938; Oral 2013; Walkin 1990). In higher education settings, the lecture course is associated with traditional instruction, or back-to-basics, which means implementing conventional teachercentered methods used in schools which society has traditionally deemed appropriate. Frontal lectures are designed to expedite the transmission of knowledge to large numbers of students, although there is no guarantee that effective learning will result (Peters 2000). Students are told when and how to learn the course materials and then are assessed in ways that overlook the diverse types of intelligence that exist in every student population (Erwin 2004). Similarly, Biggs and Tang (2011) maintain that this teacher-centered approach overlooks the diversity that exists among students attending higher education today, who undoubtedly have unique needs which should be addressed, and which necessitate “teaching that engages students’ learning activities appropriately” (p. 5). The traditional approach has been found to be associated with students’ conceptions of learning; that is, students who conceive learning as a quantitative increase in knowledge or as memorizing are likely to adopt a surface approach to learning. When learning is perceived by the learner as something external that just ‘happens’ or is ‘done to you’ by teachers, they are pragmatically motivated and seek to meet the demands of the task with minimum effort, and are passively involved in learning without reflecting on purpose or strategy (Biggs and Tang 2011; Prosser and Trigwell 1999). The main criticism of this type of teaching lies in the inherent difficulty in pinpointing exactly what the student would be able to do as a result of a successful learning process. According to the traditional epistemology, the teacher is being endowed with incontestable authority. Fromm (1941) describes this type of authority as irrational, one that promotes a kind of authoritarian relationship, whether explicit or implicit, that impedes the emergence of conflicts, leaving everything tacit but not expressed, therefore unresolved. In contrast, rational authority aims to establish a more democratic relationship, by which students are respected by their educators and which favors the emergence of conflicts that can be resolved through dialogue. It represents the aims of individual growth and development. Fromm asserted that in the traditional learning environment, the educational methods used tend to discourage original thinking by putting “emphasis on knowledge of facts, or I should rather say on information. The pathetic superstition prevails that by knowing more and more facts one arrives at the knowledge of reality. Hundreds of scattered and unrelated facts are dumped into the heads of students; their time and energy are taken up by learning more and more facts so that there is little left for thinking” (pp. 213–214). In contrast, culturally responsive assessment is based on the student-centered approach to teaching, learning, and assessment and is informed by the work of critical pedagogy (Freire and Macedo 1995; Giroux 2011), a term used to describe “what emerges when critical theory encounters education” (Kincheloe and Steinberg 1997, p. 24). Realignment of power relations in the classroom can be achieved by enabling students to be at the core of the educational process through practices based

10

1 Advancing Culturally Responsive Assessment Practices

on dialogic learning, equal relationships, and democratic values. Thus, the purposes of offering alternatives to traditional forms of assessment are to elicit dialogue, democratic practice, and critical thinking. To achieve the clear goal of social change, it is essential to create opportunities in the classroom to develop a cadre of critical thinkers who are able to be agents of change in society through their experience. Critical pedagogy seeks to promote a democratic educational process through thinking about equality and social justice and questioning power structures in the classroom, which reproduce hegemonic relations in society more broadly. As pointed out by Giroux (2011), critical pedagogy is based on “a recognition that education was important not only for gainful employment but also for creating the formative culture of beliefs, practices, and social relations that enable individuals to wield power, learn how to govern, and nurture a democratic society that takes equality, justice, shared values, and freedom seriously…” (p. 4). Critical pedagogy means enabling students to think critically and to use the knowledge they gain to critique the world in which they live. Hence, critical pedagogy “is about more than a struggle over assigned meanings, official knowledge, and established modes of authority: it is also about encouraging students to take risks, act on their sense of social responsibility, and engage the world as an object of both critical analysis and hopeful transformation…” (Giroux 2011, p. 14). While examples of critical pedagogy are accumulating, it has been argued that they rarely reflect corresponding changes in assessment practices (Nayir et al. 2019). As noted by Reynolds and Trehan (2000), “Where assessment does depart from mainstream practice, alternatives are typically based on humanistic, studentcentered aspirations for social equality, rather than on an analysis of the assessment process in terms of institutional power” (p. 268). The realignment of power relations during assessment processes might be retained by allowing students to be involved throughout the entire assessment process, including the development of learning outcome statements. As further discussed in this book, such assessment practices are ever more necessary in multicultural learning environments, as in addition to the power relation that exists between teachers and students, power relations between the majority and minority are present.

Culturally Responsive Assessment Is an Outcome-Based Approach Despite the numerous definitions, often similar to each other, of learning outcomes, the common denominator is that learning outcomes should focus on what the learner has achieved rather than on the teacher’s intentions and should reflect what the learner is able to do at the end of the educational activity. Learning outcomes should be phrased as statements of what the learner is expected to be able to demonstrate after completion of a process of learning. Based on Kennedy et al.’s (2007) guidelines, learning outcome statements should be capable of being validly assessed, achievable

Culturally Responsive Assessment Is an Outcome-Based Approach

11

in a given period of time, and take into account the students’ level and challenge it. Therefore, an assessment tool or technique is needed that can determine the degree to which the learning outcomes were achieved. The challenge for teachers is the ability to ensure alignment between teaching/learning activities, assessment techniques, assessment criteria, and learning outcomes, and endorse the transparency of the overall learning process. Although it is student-oriented, the outcome-based approach still places the responsibility for formulating the learning outcomes primarily on the teacher. Indeed, clear expectations of the student are one of the key elements of effective learning (National Institute for Learning Outcomes Assessment 2016). But, as asserted by Montenegro and Jankowski (2017), teachers should involve students in the development process of writing learning-outcome statements, to increase the chances of students understanding what is expected of them and be properly guided into thinking very carefully about the semantics and syntax of their learning-outcome statements, “… being mindful of how the language of learning-outcome statements might be appropriate for/inclusive of certain student groups but not others can lead to more holistic assessments” (p. 12). Learning-outcome statements should be written with attention to cultural relevancy so as to genuinely infer the learning gains of diverse groups of students. To do so, Adelman (2015, p. 3) posits that the approach to writing learning-outcome statements should be confined to operational verbs: While virtually all excursions onto the field of writing student learning outcomes regard “active verbs” as the required fulcrums of such statements, I hold to a difference between “active” and “operational” that is rarely noted. In this context, an operational verb references student actions that are directly observed in external contexts and subject to judgment. For example, the stand-alone “recall” references an internal cognitive dynamic. It is not operational. One does not know that a student has “recalled” anything until the act of memory is externalized in another action. The same can be said for “recognize,” “develop,” “value,” and “relate,” for example. While some might well disagree with my illustrative selections, the approach to writing learning outcome statements set forth here is confined to operational verbs. Yes, some merely “active” verbs somehow sneak in, but “operational” rules the roost.

Learning outcomes embrace ethical, cultural, and social skills which standard assessments are not always able to capture. Those refer to the ability to function in a certain discipline with and in other contexts and regionally around the world. This should have an impact on the academic requisites for graduates as they can be expected to be interculturally competent. However, “this expectation should not be reduced to being able to identify cultural differences and sensitivities. It is more about the ability to think and act interculturally. An intercultural learning outcome should refer to an ability to value cultures without judging, enabling effective and appropriate communication and cooperation with people of all cultures” (Aerden 2015, p. 12). The OECD’s Learning Framework 2030 (OECD 2018) echoed this argument by noting the increasing social and cultural diversity that is reshaping countries and communities today and positing that the use of the twenty-first century’s broader set of skills should be mediated by attitudes and human values (e.g., respect for life,

12

1 Advancing Culturally Responsive Assessment Practices

human dignity, and respect for the environment) that cannot be evaluated by means of summative assessment tools. Hence, the need to deepen the role of education to ensure that learners acquire solid social, civic and intercultural competences is widely recognized (European Commission 2018). Drawing on these culturally sensitive learning outcomes, educators should undergird practices that enhance openness to cultural diversity and intercultural understanding. By embracing such culturally sensitive teaching practices, teachers may become more aware of the individual student’s knowledge, skills and attitudes that influence his/her respect for and engagement with the cultures of others, including their value systems. As intercultural understanding and cultural diversity are closely linked to competencies of language; personal, social and learning; civic; and cultural awareness and expression (Council of Europe 2018), addressing them might palliate some major challenges faced by students of diverse ethnic, religious, and cultural backgrounds in higher education settings, as outlined at the beginning of this chapter. Educators should also acknowledge that diverse and innovative assessment tools and methods are needed to address these competencies, as a single assessment instrument cannot meet all of these objectives, as noted by Lamb et al. (2015, p.1): While test scores have been centre stage in international comparisons, there has been growing recognition of the effects that education has on the development of interpersonal and intrapersonal skills and capabilities that affect the success of students in school and success in the labour market… Assessing such skills may also help make visible the achievements of students who do not perform well in academic tests, and the contribution schools make to their learning.

Drawing on the above review, it can be concluded that culturally responsive assessment might be considered more appropriate in diverse classrooms and could lead to more targeted improvements in teaching and learning “that allow students to demonstrate their learning in various ways while also being transparent about the learning that is taking place, help students reflect on their learning experiences, and allow students to actively participate in the learning and assessment process” (Montenegro and Jankowski 2017, p. 12). Nonetheless, current studies indicate that in Europe, culturally responsive assessment practices are less prevalent, conspicuously scarce in higher education, and consequently far less discussed (Nayir et al. 2019). To fill this lacuna, the following chapters address aspects of instruction, learning, and assessment in higher education while placing an emphasis on several formative assessment tools and techniques used in multicultural learning environments.

References Abington-Pitre, A. (2015). Where did we go wrong? Eight characteristics of multicultural schools. Universal Journal of Educational Research, 3, 99–103. Adelman, C. (2015). To imagine a verb: The language and syntax of learning outcomes statements. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA).

References

13

Aerden, A. (2015). An introduction to international and intercultural learning outcomes. European Consortium for Accreditation in Higher Education. Retrieved from http://ecahe.eu. Aiding Culturally Responsive Assessment in Schools (ACRAS). (2018). Exploring culturally responsive assessment practices in European schools - Interim report. Oslo: University of Oslo. Alt, D., & Itzkovich, Y. (2018). The connection between perceived constructivist learning environments and faculty uncivil authoritarian behaviors. Higher Education, 77(3), 437–454. Alt, D., & Raichel, N. (2018). Lifelong citizenship: Lifelong learning as a lever for moral and democratic values. Leiden and Boston: Brill and Sense Publishers. Aronson, B., & Laughter, J. (2016). The theory and practice of culturally relevant education: A synthesis of research across areas. Review of Educational Research, 86(1), 163–206. Banks, J. A. (2017). Citizenship, global migration, and education. Washington DC: American Educational Research Association. Beck, R. H. (2009). The three r’s plus: What today’s schools are trying to do and why. MN: University of Minnesota Press. Ben-Rabi, D., & Hanadin, A. (2013). Preparation for successful integration of Arab students in higher education: Evaluation study on an improvement plan of pre-academic preparation. Report submitted to the Council for Higher Education. Jerusalem: The Myers-JDC-Brookdale Institute. (Hebrew) Biggs, J. B., & Tang, C. (2011). Teaching for quality learning at university: What the student does (4th ed.). Berkshire: Open University Press. Burns, D., Brown, M., McNamara, G., O’Hara, J., Altrichter, H., Nayir, F., Nortvedt, G., Fellner, M., Helm, C., Punzenberger, B., Skedsmo, G. and Wiese, E.F. (2019) Aiding Culturally Responsive Assessment. A User-Friendly Handbook for Culturally Responsive Assessment in Educational Settings. Carjuzaa, J., & Ruff, W. G. (2010). When western epistemology and an indigenous worldview meet: Culturally responsive assessment in practice. Journal of the Scholarship of Teaching and Learning, 10, 68–79. Choi, J., & Ziegler, G. (2015). Literacy education for low-educated second language learning adults in multilingual contexts: The case of Luxembourg. Multilingual Education, 5(1), 1–21. Council of Europe. (2016). Competences for democratic culture. Living together as equals in culturally diverse democratic societies. Retrieved from https://rm.coe.int. Council of Europe. (2018). Reference framework of competences for democratic culture. Retrieved from https://www.coe.int. Cummins, J. (1979). Linguistic interdependence and the educational development of bilingual children. Reprinted in C. Baker, & N. Hornberger, (Eds.) (2001). An introductory reader to the writings of Jim Cummins (pp. 63–95). Clevedon: Multilingual Matters. Cummins, J. (2001). Negotiating identities: education for empowerment in a diverse society, 2nd ed. Ontario, CA: California association for Bilingual Education. De Backer, F., Van Avermaet, P., & Slembrouck, S. (2017). Schools as laboratories for exploring multilingual assessment policies and practices. Language and Education, 31, 217–230. Dewey, J. (1938). Experience and education. Kappa Delta Pi. Dirasat. Arab Center for Law and Policy. (2011). The obstacle course: Challenges and directions of action for a substantial integration of Arab citizens into the Israeli higher education system. Report submitted to the Council for Higher Education. Nazareth: Dirasat. (Hebrew) Dixson, D. D., & Worrell, F. C. (2016). Formative and summative assessment in the classroom. Theory into practice, 55(2), 153–159. Dixon, L. Q., & Wu, S. (2014). Home language and literacy practices among immigrant secondlanguage learners. Language Teaching, 47(4), 414–449. Erwin, J. C. (2004). Classroom of choice: Giving students what they need and getting what you want. Alexandria, VA: Association for Supervision and Curriculum Development. European Commission. (2018). Commission staff working document. Accompanying the document Proposal for a Council Recommendation on Key Competences for LifeLong Learning. Retrieved from https://eur-lex.europa.eu.

14

1 Advancing Culturally Responsive Assessment Practices

Ford, D. Y., & Kea, C. D. (2009). Creating culturally responsive instruction: For students’ sake and teachers’ sake. Focus on Exceptional Children, 41, 1–18. Freire, P., & Macedo, D. (1995). A dialogue: Culture, language and race. Harvard Educational Review, 65, 377–402. Fromm, E. (1941). Escape from freedom. New York: Holt, Rinehart and Winston. Gay, G. (2013). Teaching to and through cultural diversity. Curriculum Inquiry, 43, 48–70. Gay, G. (2018). Culturally responsive teaching: Theory, research and practice. (3rd ed.). Multicultural Education Series. New York: Teachers College Press, Columbia University. Ginsberg, M. B., & Wlodkowski, R. J. (2009). Diversity and motivation: Culturally responsive teaching in college. Hoboken, NJ: John Wiley & Sons. Giroux, H.A. (2011). On critical pedagogy. New York: Continuum International. Hager, T., & Jabareen, Y. (2015). From marginalisation to integration: Arab-Palestinians in Israeli academia. International Journal of Inclusive Education, 20(5), 455–473. Hammerness, K., Darling-Hammond, L., Bransford, J., Berliner, D., Cochran-Smith, M., McDonald, M., & Zeichner, K. (2005). How teachers learn and develop. In L. DarlingHammond & J. Bransford (Eds.), Preparing teachers for a changing world: What teachers should learn and be able to do (pp. 358–389). Jossey-Bass: San Francisco. Hancock, A. (2018). Inclusive practices for pupils with English as an additional language. In B. Skinner & B. O’Toole (Eds.). Minority Language Pupils and the Curriculum: Closing the Achievement Gap (pp. 24–27). Retrieved from https://www.mie.ie. Herzog-Punzenberger, B., Le Pichon-Vorstman, E. & Siarova, H. (2017). Multilingual education in the light of diversity: Lessons learned. NESET II report, Luxembourg: Publications Office of the European Union. Hofhuis, J., van der Rijt, P. G. A., & Vlug, M. (2016). Diversity climate enhances work outcomes through trust and openness in workgroup communication. SpringerPlus, 5, 714. https://doi.org/ 10.1186/s40064-016-2499-4. Hue, M., & Kennedy, K. J. (2015). Promoting cultural responsiveness: Teachers’ constructs of an assessment classroom environment for ethnic minority students in Hong Kong secondary schools. Teachers and Teaching, 21, 289–304. Jiosi, V., & Zalmanson-Levi, G. (2018). Multiculturalism and co-living between Jews and Arabs in teacher training academic institutions. Recommendations document. Tel-Aviv: The Association for Civil Rights in Israel. (Hebrew) Kennedy, D., Hyland, A., & Ryan, N. (2007). Writing and using learning outcomes: A practical guide. Retrieved from http://reforma.fen.uchile.cl. Kim, K. H., Zabelina, D. (2015). Cultural bias in assessment: Can creativity assessment help? International Journal of Critical Pedagogy, 6, 129–148. Kincheloe, J., & Steinberg, S. (1997). Changing multiculturalism. London: Open University Press. Krasnoff, B. (2016). Culturally responsive teaching. A guide to evidence-based practices for teaching all students equitably. Portland, Oregon: Region X Equity Assistance Center at Education Northwest. Ladson-Billings, G. (1994). The Dreamkeepers: Successful teaching for African-American students. San Francisco: Jossey-Bass. Lamb, S., Jackson, J., & Rumberger, R. (2015). Measuring 21st Century Skills in ISCY. ISCY Technical Paper. Centre for International Research on Education Systems, Victoria University. Little, D. (2010). The linguistic and educational integration of children and adolescents from migrant backgrounds. Council of Europe Concept paper. Geneva, Switzerland: Council of Europe. López-Pastor, V. (2009). Evaluación formativay compartida en educación superior [Formative and Shared Assessment in Higher Education]. Madrid: Narcea. Lorente, E., & Kirk, D. (2013). Alternative democratic assessment in PETE: An action-research study exploring risks, challenges and solutions. Sport, Education and Society 18(1), 77–96. Manor, R. (2016). Arab students in Israel write a second language - in Hebrew. In Proceedings the International Online Teaching Conference Hebrew as another language for diverse populations in Israel and around the world, 15 June 2016 (pp. 80–96). Tel-Aviv, Israel. (Hebrew)

References

15

McCroskey, J. C., & Richmond, V. P. (1984). Power in the classroom II: Teacher and student perceptions. Communication Education, 33, 125–136. Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment (Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). National Institute for Learning Outcomes Assessment. (2016, May). Higher education quality: Why documenting learning matters. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. Nayir, F., Brown, M., Burns, D., O’hara, J., Mcnamara, G., Nortvedt, G., Skedsmo, G., Silje Kristin Gloppen, S. K., Eline F. Wiese, E. F. (2019). Assessment with and for migration background students-cases from Europe. Eurasian Journal of Educational Research, 79, 39–68. Nordvedt, G. (2018). What previous research tells us about assessment and migrant students. Aiding Culturally Responsive Assessment in Schools (ACRAS) Exploring Culturally Responsive Assessment Practices in European Schools - Interim Report. Oslo: University of Oslo. OECD/CERI International Conference. (2008). Learning in the 21st century: Research, innovation and policy. Assessment for learning formative assessment. http://www.oecd.org/site/educeri21st/ 40600533.pdf. OECD. (2018). The future of education and skills Education 2030. RETRIEVED FROM http:// www.oecd.org. Oral, Y. (2013). “The right things are what I expect them to do”: Negotiation of power relations in an English classroom. Journal of Language, Identity, and Education, 12(2), 96–115. Peters, M. (2000). Does constructivist epistemology have a place in nurse education? Journal of Nursing Education, 39, 166–172. Prosser, M., & Trigwell, K. (1999). Understanding learning and teaching: The Experience in higher education. Buckingham: SRHE and Open University Press. Reynolds, M., & Trehan, K. (2000). Assessment: a critical perspective. Studies in Higher Education, 25(3), 267–278. Richardson, J. T. E. (2011). The academic engagement of White and ethnic minority students in distance education, Educational Psychology, 31, 123–139. Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(3), 4–13. Shaviv, M., Binstein, N., Stone, A., & Fudem, O. (2013). Pluralism and equal opportunity in higher education. Expanding access for Arabs, Druze and Circassians in Israel. Report of the Planning and Budgeting Committee of the Council for Higher Education. Jerusalem: Council for Higher Education. Skinner, B., & O’Toole, B. (Eds.). (2018). Minority language pupils and the curriculum: Closing the achievement gap. Papers from a seminar on teaching English as an additional language. Retrieved from https://www.mie.ie. UNESCO. (2002). Learning to be: A holistic and integrated approach to values education for human development: Core values and the valuing process for developing innovative practices for values education toward international understanding and a culture of peace. (UNESCO-APNIEVE Sourcebook, No. 2). Veugelers, W., de Groot, I., & Stolk, V. (2017). Research for CULT Committee – Teaching Common Values in Europe. Brussels: European Parliament, Policy Department for Structural and Cohesion Policies. Vygotsky, L. S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press. Walkin, L. (1990). Teaching and learning in further and adult education. Cheltenham, Stanley Thorne. Zeichner, K. M. (2003). The adequacies and inadequacies of three current strategies to recruit, prepare, and retain the best teachers for all students. Teachers College Record, 105, 490–519.

Part I

Intrapersonal Facet

Chapter 2

Self-assessment of Digital Literacy Skills in a Multicultural Classroom

Introduction Student self-assessment essentially involves learners making judgments about their own achievements and learning outcomes (Boud and Falchikov 1989). This chapter emphasizes the ways in which students should be supported by their instructors to monitor and manage their skills by developing the ability to effectively evaluate, through their learning process, their own competencies. Information-access inequality in minority populations—informed by race, age, educational level, nationality, and economic factors—is well-documented. This chapter focuses on digital literacy skills and, particularly, on the way in which teachers might deliberately further opportunities which support minority students and enable them to assess and express effective and informed judgments about the quality of obtained digital information. The case study narrated in this chapter illustrates how self-assessment can be used to evaluate the process of creating gamification projects based on authentic problem-solving to help multicultural students acquire and assess digital literacy skills for learning.

Literature Review Self-assessment Based on previous studies (Boud 1995; Paris and Paris 2001; Andrade et al. 2008; Summers et al. 2019; Yan and Brown 2017) self-assessment is defined in this chapter as a process during which students are given the opportunity to collect information about their own performance, evaluate, and reflect on the quality of their work, learning, knowledge, and performance in a way that helps them identify their strengths and weaknesses, according to selected criteria, which ultimately leads to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_2

19

20

2 Self-assessment of Digital Literacy Skills in a Multicultural …

improvement. Self-assessment requires the induction of learners into reflective practice, revealing the purpose and skills required to identify issues that do not often meet expected quality standards (Andrade and Du 2007; Andrade and Brown 2016). This process has been perceived as authentic assessment or “assessment as learning” (Dann 2014) and a useful construct that increases students’ self-regulation and ability to autonomously control their own learning. Some advantages of self-assessment have been outlined by Summers et al. (2019) as follows: (a) it encourages the development of self-awareness while enhancing reflective learning; (b) it gives learners the opportunity to analyze their abilities based on their learning experiences; (c) it takes advantage of adults’ abilities to think about, understand, and act on educational objectives; (d) it is cost-effective; (e) it can be motivating for students; (f) it eliminates concerns regarding cheating and test security; and finally, (g) it benefits teachers by reducing the testing burden and elevating awareness of their students’ confidence levels. On this same continuum, researchers (Sambell et al. 2013) contend that higher education should prepare students to become effective self-assessors, to make complex judgments about their own work, and thus to be prepared to make decisions in the complex professional contexts they are likely to encounter in the future. As put by Boud and Falchikov (2007), “self-evaluation underpins an individual’s capacity for independent, reflective thinking and the aptitude to take responsibility for one’s own actions. These are features which employers expect of a graduate workforce and the capacity to monitor one’s own work is typically seen as a graduate attribute” (p. 121). These researchers also emphasize the scaffoldings which students require to successfully manage and assess their own learning. Many students need explicit help to develop the capacity to do so; therefore, teachers cannot assume that their students already possess the requisite disposition, skills and qualities to self-assess their work. “The development of learners’ self-evaluation skills is too vital to leave to serendipity and a causal process of ‘picking it up as one goes along’” (p. 125). Although guided by the teacher, self-assessment relies primarily on the studentcentered approach and not on the teacher as the sole source of evaluative judgments. It also engages students in reflecting on the quality of their own work. It is perceived as a subset of assessment for learning that supports students’ metacognitive thinking and is largely used for formative purposes to improve students’ performance, such as identifying the strengths and weaknesses in one’s work for revision (Wong 2017). Bourke (2016) associates the term reflective intelligence with self-assessment because it encompasses the ability to engage in the metacognitive monitoring of one’s own learning and enables students to self-regulate their learning, develop a greater sense of control over their learning, accept responsibility for their learning, develop reflective thinking skills, and apply metacognitive strategies. Self-assessment can motivate students to learn, enable them to think for themselves and be more confident in their ability to learn, be useful for enhancing academic performance, and perhaps help learners to build lifelong learning skills (Brown and Harris 2014). To achieve these goals, researchers (e.g., Wong 2017) emphasize that using self-assessments

Literature Review

21

for summative purposes of grading and reporting tends to reduce the accuracy of self-assessment and could turn students’ attention away from the quality of their work. Other researchers (Bourke and Mentis 2013; Bourke 2016) perceive selfassessment as a process for inclusion and an act of liberation which engages students in their learning process by enabling them to share their own reality, which is done by setting and assessing their goals. Self-assessment, identity, and inclusion are interlinked, as they occur through lived experiences within multiple and diverse sociocultural contexts. Therefore, assessment should be related to inclusion, and educators should determine how assessment practices can be modified for the benefit of all students, only then do inclusion and participating in learning belong to the student, as asserted by Bourke and Mentis (2013, p. 855). For inclusion to be meaningful, each student’s own values, goals and aspirations need to be acknowledged. Students need to be encouraged to see the relevance of and connect meaning to a task or activity for learning and self-assessment to become a priority for them. Understanding learning then, through a different lens, portrays students in various ways, but self-assessment, particularly, makes their sense of identity and belonging visible.

Indeed, the advantages of self-assessment over practices that seek standard or measured outcomes were widely acknowledged in prior research; however, over the years, self-assessment has not been spared from criticism. Some researchers (e.g., Raaijmakers et al. 2019; Summers et al. 2019) have drawn attention to several disadvantages; for instance, learners are often imprecise in their self-assessments, lack the confidence and training necessary to accurately self-assess, view teachers as the ultimate authority in assessment, or are afraid of self-assessing improperly. Despite these concerns, the researchers bring forth several indications showing that learners are more aware of their weaknesses than they are of their strengths, and with more experience they become more accurate in assessing both their strengths and weaknesses. “The advantage of situating self-assessment as a competence is that competencies usually have levels of development (e.g., ranging from novice to expert)… and, consequently, can be used as the basis for a teaching curriculum (Brown and Harris 2014, p. 25).

Self-assessment in Higher Education Clearly the study of self-assessment has borne much fruit in the field of higher education, in which self-assessment has been proposed as beneficial to student learning in various disciplines. For example, Taras (2010) evaluated four models of self-assessment in higher education: ‘standard’, ‘self-marking’, ‘learning contract design’, and ‘sound standard’, which assist teachers in comparing the different processes and the degree of engagement of learners and tutors in self-assessment. In her study, Taras discusses how each model had the potential to develop expertise in different contexts. For instance, the self-marking model has the advantage

22

2 Self-assessment of Digital Literacy Skills in a Multicultural …

of providing students with immediate feedback on their work; the standard model permits learners to consider their work against criteria and standards and has the advantage of encouraging learner independence. The sound standard model focuses on students’ learning and appreciation of standards and on making grading transparent. In the learning contract design model, learners assume responsibilities, and major decisions are placed in the learner’s hands. These models permit different levels of tutor ‘handover’ of assessment in accordance with the philosophy of student-centered education, which higher education purports to endorse. Wolffensperger and Patkin (2013) explored self-assessment within the framework of a mathematics teaching seminar course at a teachers’ training college in Israel. Their research findings indicated that students’ self-assessment experience is “far from being faultless”. Self-assessment, they argue, necessitates considerable teaching efforts devoted to preparing students for this task due to their inadequate self-assessment skills. They concluded that self-assessment is a complex competence and, hence, we cannot expect students to be able to perform it immediately. To this end, extensive teaching-learning and appropriate preparation, including actual opportunities to experience it are required. Negotiations and continuous discussions between teachers and students regarding potential misunderstandings might instigate learners’ deeper comprehension of the learning assignment. The authors recommend introducing self-assessment assignments in the first or second year of their studies, when they are more inclined to accept changes. Hill (2016) explored students’ (of accounting) perceptions of self-assessment in learning and demonstrated the importance of investigating the benefits from a student perspective. She gauged the perceptions of accounting students of self-assessment after exposure to the process on three occasions in a facilitated environment created by their lecturers. According to her findings, most students did not self-assess unless encouraged to do so. After the facilitated process, they believed that self-assessment could improve their overall academic performance and indicated that they would practice self-assessment in future. A more recent study (Yan and Brown 2017) identified the actions involved in a self-assessment process in a qualitative study with 17 undergraduate students from a teacher education institute. The authors identified a cyclical self-assessment process that involves a sequence of first determining relevant performance criteria, seeking feedback, self-reflection, and a gradual calibration of judgments. Their study supports the idea that self-assessment skills can be learned by students, but further work needs to be undertaken to develop realistic and accurate self-assessment. Self-assessment is discussed in this chapter in conjunction with digital literacy skills; therefore, the following section provides a comprehensive definition of these skills and shows their heightened importance for personal, professional, and social benefits as recorded in the literature. Then, the term “digital divide” is discussed to illustrate the gap the exists among higher education students from different economic, ethnic, and educational levels in relation to digital literacy competencies. Finally, it outlines learning environments that might support these skills in higher education settings.

Digital Literacy Skills

23

Digital Literacy Skills Digital literacy, which is becoming a central empowering agent in educational institutions, requires competencies in locating, processing, producing, and communicating information. It is additionally defined as the awareness, attitude, and ability of individuals to use digital tools appropriately and “…to identify, access, manage, integrate, evaluate, analyze and synthesize digital resources, construct new knowledge, create media expressions, and communicate with others, in the context of specific life situations, in order to enable constructive social action; and to reflect upon this process” (Martin 2005, pp. 135–136). These skills are essential elements of digital citizenship that enable people to achieve the benefits of thriving in a digital age (Frau-Meigs et al. 2017). The digital literacy skill set includes reading and writing, speaking and listening, familiarization with new technologies, critical viewing of visual communication content, and the ability to create messages using a wide variety of technologies (Alt and Raichel 2020a). The messages or ‘texts’ consist of any mode of expression and communication that uses a system of symbols: language, images, graphic design, icons, sound, music, and so forth (Hobbs 2011). These digital literacy skills may stimulate curiosity and creativity, help to increase understanding, expand critical thinking, and develop assessment and judgment skills (Horton 2007). Drawing on this framework and others (Hunter 2018), the following types of digital skills can be distinguished by: (1) formal operational skills required to navigate digital media, access information by locating and sharing materials; (2) analytical and evaluative skills; i.e., analyzing messages in a variety of forms, obtaining information critically, identifying the source of information, and evaluating the quality and credibility of the content; (3) creating and communicating digital information—e.g., creating content in a variety of forms, making use of language, images, sound and new digital tools and technologies, and collaborating to construct new knowledge or digital artifacts using technology and media. The first skill set includes reading information in various forms, finding information from varied sources and selecting information relevant to the task at hand (Hobbs 2011). The learner makes responsible choices and accesses information by locating and sharing materials and comprehending information and ideas. Hunter (2018) maintains that in the information age, information overload, or managing pushed information, is a bigger issue than a lack of relevant information, stating that while “research used to be about which of a finite number of sources to check, now it is often about knowing when to stop when faced with an almost infinite number of possible sources” (p. 57). To enhance the abilities of finding and selecting information, teachers should provide students with instructions on how to construct their web search so that students are more likely to find the relevant information they seek (Zhao et al. 2018). According to the second set of skills, a digitally literate learner should know how to employ effective mechanisms of information assessment, to verify the credibility of the information, distinguish between opinions and facts, and select only credible information. This set of skills is important to learners as it allows them to filter the

24

2 Self-assessment of Digital Literacy Skills in a Multicultural …

information they use to consolidate knowledge. These digital literacy skills help to identify erroneous, irrelevant, or biased information and prevent its penetration into the learner’s scheme of considerations. Scholars (e.g., Koltay 2011) agree on the necessity of introducing those skills that are rooted in the nature of the present era, characterized by an exponential surge of available information. In addition, the information available through multiple media is of indeterminate quality. Many sources of the unlimited digital information to which we are exposed are unreliable because of the unrestricted opportunities to publish information and manipulate it. This ‘information explosion’ intensifies the importance of introducing skills of information consumption. Digitally literate consumers are always prepared to critically doubt the quality of information and are not tempted to accept it unquestionably, even if the source seems authoritative and reliable, as expected of a citizen in a democratic society (Eshet 2012). The third set of digital literacy skills refers to the ability to create and express ideas through an array of technological tools. This activity encourages students to formulate messages that express creativity and teaches them to exploit opportunities to express themselves (Hobbs 2011). This component of literacy contributes to cultivating a learning environment in which students practice skills of leadership, freedom of expression, and decision-making. Lacking these skills, learners will not be effectively engaged with others in overcoming the challenge of problem-solving through teamwork and collaboration, requisite skills for participation in a democratic society (Alt and Raichel 2018). Teachers can facilitate effective group work by supporting and encouraging students to develop strategies of collaboration through the use of digital technologies.

Digital Literacy Skills and Minority Students Numerous studies show how higher education institutions are being challenged to meet the needs of students with technological skill deficits, especially of firstgeneration college students, and minority students. Among these skills is the ability to access information technology tools, which has been identified as critical to the education and professional development of individuals in today’s information-laden society (Torralba 2015); or the ability to create knowledge. Technologies provide multiple opportunities to construct knowledge. Blogs, digital stories, and wikis are some of the many digital tools that can be used to communicate in and out of the classroom. They can help to reduce the social isolation that often characterizes/typifies minority group members and create ways in which they can construct and maintain their identities as well as build communities (Thorne 2009). The term “digital divide” is frequently used to highlight the large disparities in access to computer technology and Internet-based activities among people from different economic, ethnic, and educational levels (Torralba 2015). This digital gap is a serious concern for higher education, especially for those that reach out to underserved populations whose technology access and ownership are less prevalent

Digital Literacy Skills and Minority Students

25

than those found among majority students. The underserved were found to have had less preparation in the use of the internet, core computer applications, and digital resources for academic pursuits (Buzzetto-Hollywood 2017; Dixon 2017). Bridging the digital gap has been discussed in several studies. BuzzettoHollywood et al. (2018) for example, reported the results of their study which examined the technological readiness of students attending a Mid-Atlantic minorityserving university. Their findings illustrated the efficacy of computer software application courses and the value of a simulated learning and assessment system in building the computer and information literacy skills of students. The researchers dispute the prevalent notion that students enter college with the digital literacy skills necessary for success and, therefore, do not require additional support. In fact, they maintain that “students expressed that they wished they had come to college more prepared to use computer software applications and that their high school had not adequately prepared them for use of technology in higher education” (p. 89). In a similar vein, Ktoridou and Eteokleous-Grigoriou (2011) evaluated the effectiveness of a computer course offered to unemployed women learners with insufficient computer literacy skills, characterized as “digital immigrants”. The study sought to map participants’ perceptions and experiences regarding technology, any barriers and challenges they had faced throughout the course, and its effectiveness in assisting them in becoming computer literate and employable. Results demonstrated the necessity of computer courses for this target group. The participants succeeded in developing computer literacy skills and further acknowledged the importance of these skills, specifically in relation to their employability, professional path, and career development. These results underscore the necessity of exploring further instructional design to support digital immigrants’ education and training with regard to computer technology. Similarly, the effectiveness of free digital learning in bridging the digital gap was assessed by the European Commission (Castaño-Muñoz et al. 2018). The extent to which MOOCs (Massive open online courses) and other free digital learning options (such as free mobile learning) are efficient in cultivating migrants’ and refugees’ skills needed for inclusion, civic integration, education, and employment was assessed. According to the findings, awareness of free digital learning options was very low among the target population. Most participants do not necessarily use social media or mobile phones for learning purposes. Usage of language Apps was found to be the most prevalent. The majority of participants also believed that free digital learning should complement face-to-face learning and underscored the importance of physical networking for their optimal integration. For free digital learning to be effective for migrant and refugee inclusion, it should take into consideration the fragility and diversity of migrant/refugee target groups, including their digital literacy, educational background, and access to technology and connectivity. Torralba (2015) examined how Latino immigrant families in California use information technology tools for the personal, educational, and work-related activities that exist inside their homes. His work re-examined the concept of access, highlighting the importance of the local, social, and economic contexts of users and the implications for the design, implementation, and use of these technologies among

26

2 Self-assessment of Digital Literacy Skills in a Multicultural …

immigrant communities. Results indicated that integrating appropriate information technology tools to support immigrant families’ personal, educational, and workrelated needs and aspirations might enhance their digital and language literacies and form new ways of interacting with formal institutions in the USA. Those literacy events, where individuals with very limited English language skills learn to design and produce forms with specific intended functions to address their needs or goals within existing societal demands, may in fact be used as a mechanism of integration of minority individuals and families seeking political, economic, and educational mobility. Hence, scholars who embrace sociocultural perspectives in learning (e.g., Lam 2012) argue that understanding the digital practices of immigrants might aid in designing curriculum topics that would allow students to utilize their skills in different languages to learn about these issues from multiple points of view. In so doing, we may foster ‘digital equity’, defined as the lack of differences in digital literacy among subpopulations based on race/ethnicity, gender, and national origin at given levels of educational attainment (Reder 2015). Similar to the above-surveyed studies, many Israeli first-year undergraduate students find it challenging to use information sources efficiently and to develop academic digital literacy (Yoo and Huang 2011). For example, Pieterse et al. (2018) reviewed multiple studies showing the impact of cultural diversity on usage of information These studies showed that cultural, linguistic, and technology-adoption are some of the factors connected to Israeli Arab minority students’ difficulties in gaining the skills needed for narrowing the digital literacy gap (Merdler and Peled 2016). Pieterse et al. (2018) reinforced these findings by presenting further insights on the differences between the diverse multicultural and multilingual student groups. In their study, they found that native Hebrew-speaking students preferred digital sources whereas Arabic-speaking students preferred printed sources, highlighting the need for greater mediation in the dimensions of information literacy for minority students.

Digital Literacy Skills and the Learning Environment Several researchers (e.g., Zhao et al. 2018) have pointed to the digital learning environment as a key factor in nurturing students’ digital literacy skills, arguing that digital technologies present multiple opportunities for teaching digital literacy skills. For ˇ example, in their review of forms of creativity with digital technology, Cernochová and Selcuk (2019) suggest several platforms, such as digital story-telling, which can be utilized to engage students in creating a digital story, by promoting a variety of literacy skills, such as researching, writing, organizing, presenting, interviewing, problem-solving, and assessing. Others suggest gamified-based platforms as a means of teaching digital literacy through creative games. Gamification works by making technology more fascinating and utilizes people’s psychological tendency to play. Thus, the use of game elements in non-game contexts is a way to utilize digital technologies and video game features

Digital Literacy Skills and the Learning Environment

27

to increase people’s involvement and motivation to perform activities. One of those activities, described in this chapter, is learning by enhancing digital literacy skills (Dubois and Tamburrelli 2013; Sadler et al. 2013). In contrast, several researchers (De-Marcos et al. 2014) claim that meaningful integration of competencies such as digital literacy may not be attained by digital technology platforms unless these environments have a careful instructional design driven by clear objectives and learning outcomes based on sound underlying pedagogy. Traditional settings which tend to “crush students’ natural inclinations toward creative and divergent thinking” (Henriksen et al. 2016, p. 28) should be replaced with pedagogical models that are grounded in the social constructivist approach. Constructivist activities with digital technology might be perceived as a promising approach to digital literacy-skill development because they are based on the learners’ strong motivation to do something completely new and interesting by incorporating problem-solving elements (Blum-Ross et al. 2019; Byrne et al. 2018). Researchers argue that basic digital literacy skills alone are not sufficient, and students need to be prepared, as put by Frank and Castek (2017), to “skillfully use digital tools and develop a discovery and risk-taking mindset toward navigating online” (p. 66). To do so, they suggest moving beyond the basic digital literacy skills towards digital problem-solving skills, which include “navigating a range of digital resources to locate, evaluate, create, and communicate information” (p. 66) to solve different kinds of real-world problems (Quann 2015; Vanek 2017). Students are therefore required to be able not only to use digital tools but to solve problems in technology-rich environments. This preponderance of literature indicates that students enter higher education with deficits in their digital skills, and this is a particularly relevant issue among minority students. Nonetheless, while this assumption lies on a solid empirical foundation, a somewhat opaque landscape emerges in terms of developing and evaluating effective interventions to enhance digital skills among multicultural students. The following case study provides a useful design to illustrate, perhaps, the effect of using selfassessment tools to evaluate and support diverse students’ digital literacy skills.

Case Study: Using Self-assessment to Support Digital Literacy Skills The participants comprised a total of 74 second-year undergraduate students of Education at a major college in Israel, of whom 73% were Muslim students and 24% Jewish students. The students participated in a three-month intervention program aimed at developing digital literacy skills. Given the heterogeneity of the classroom, self-report measures were filled out by the students twice: before (pretest) and after (posttest) the intervention, in order to trace strengths and weakness related to digital literacy skills and, consequently, to support the students accordingly by providing

28

2 Self-assessment of Digital Literacy Skills in a Multicultural …

adequate guidance. Another self-assessment tool employed was a reflective diary in which the participants were asked to describe predominant skills they had developed as a result of the intervention.

The Intervention Procedure In this learning experience, an open internet platform named To-Be Education was used (hereinafter: the platform). This platform facilitates the creation of learning communities that share opinions to solve authentic problems related to the studied discipline. In the current case, the dilemmas selected by the students related to the promises and pitfalls of educational approaches such as homeschooling and online learning. This platform was chosen as it allows students to convert learning materials into an interactive role-play and to plan and create a lesson that inspires independent learning, digital literacy skills, critical thinking, and dialogue skills by means of forums that allow voicing alternative viewpoints. In the present study, the participants were asked to convert learning materials into a game by means of the platform and to present the designed game in the classroom. The students worked in multicultural groups; however, given the composition of the participants, the Muslim students significantly outnumbered the Jewish students. The following steps outline the procedure of the game construction and the desired learning outcomes: 1. 2. 3.

4.

5.

6.

Construct a story that includes an ill-structured problem. Devise five different solutions to the problem to enable players to examine and experience the problem from multiple perspectives. For each solution, the game creators design an avatar through which the player argues either in favor of or against a certain option. Avatars are images that represent different perspectives in the story and solutions to the problem. An example of an avatar can be a fictitious character who espouses a certain opinion. This character’s profile (including physical and personal features) is creatively designed by the game creators. For each avatar, the game creators provide corresponding learning materials (resources), which are carefully selected, analyzed, and evaluated before being uploaded. During the game, players share their informed opinions about the dilemma at hand. The game creators prepare in advance the rewards and feedback (badges) that will be awarded to the players during the game and the criteria for obtaining these rewards. After sharing opinions, the players freely (regardless of their avatars) vote for one solution, and a face-to-face discussion takes place, in which the players can convey their own opinions. The game creators prepare several points for discussion in advance.

The Intervention Procedure

7.

29

Finally, the game creators present the game to their peers by means of the platform.

Self-assessment Measurement The learning tasks for the activity (game construction) were explained to the students prior to the outset of the intervention program. An initial discussion was held in class surrounding the importance of digital literacy in the twenty-first century. The discussion revealed that most of the students were not sufficiently familiar with these skills from their previous studies and, consequently, there arose a need to explain these skills to them. The students were informed that at the beginning of the assignment (pretest) they would be asked to evaluate their own knowledge and abilities regarding digital skills and to re-evaluate their progress at the end of the process (posttest). The criteria for assessing the skills are presented in Table 2.1. This tool enabled the students to better understand the operational indicators that attested to each digital Table 2.1 Scale items and factor descriptions Factor

Item

Analyze

Generally, I can easily sort out the information I read online to address my academic work-related requirements When I read information online, I easily identify the sections relevant to my assignment In general, while reading online, I can distinguish between facts and the author’s interpretation When I read online, I can easily distinguish between what matters and what is less important to my work When I watch an online video, I can easily identify the pieces of information that are relevant to my work

Access

Usually, I know how to get specific online information that might help me with my assignment Usually, I know which websites or other Internet resources might be relevant to my assignment Usually, I know how to access online databases (e.g., databases of journals or those of authorities, such as government resources) I usually know how to find online sources of information (e.g., magazines, journals, newspaper) that might be relevant to my assignment Usually, I know how to narrow the search options and obtain exactly what I need

Evaluate

Usually, I check whether it’s an academic source (e.g., books, academic journal articles, or published expert reports) or a non-academic source Usually, I attend to source information (e.g., author, date of publication) while reading online

30

2 Self-assessment of Digital Literacy Skills in a Multicultural …

literacy skill. The tool included three sets of criteria: Access (five items), Analyze (five items), and Evaluate (two items). 1. 2. 3.

Access—accessing information by locating and sharing materials. Analyze—analyzing messages in a variety of forms, obtaining information critically. Evaluate—evaluating the quality and credibility of the content.

All items were scored on a Likert-type score ranging from 1 = strongly disagree to 6 = strongly agree. Table 2.1 shows the three sub-scales of the digital literacy tool, along with the items. Satisfactory validity and reliability results are displayed in Alt and Raichel’s (2020a) study. Another single item alongside an open-ended question was added to evaluate the students’ involvement in creating digital materials (Create) pre- and postintervention. In this item the students were asked the following question: ‘In your studies to date, have you created new knowledge or digital artifacts using technology?’ 1 = No; 2 = Yes. If you marked Yes, briefly describe the activity you participated in. After filling out the questionnaire below, the students were asked to identify which skill they felt was their weakest, which earned the lowest average score, and which skill they deemed the “strongest”, and then to record their conclusions in their reflective journal. The students kept one copy of the questionnaire, and a second copy was submitted to the lecturer anonymously. They were also asked to write how they felt they could improve these skills during the course of the assignment. For example, what type of help would they need? Who would they approach for help? What objectives should they establish? At the end of the intervention program, they were asked to describe what methods they had used to overcome the challenges, what weak points still remained, and which skills they had attained, and to record them in their reflective journal.

Between-Tests Differences in the Cultural Digital Literacy Skill A set of t-tests demonstrated that prior to the intervention, Access scored the highest mean result, followed by Analyze and Evaluate, respectively, in the Jewish group. In contrast, the Muslim students were found to be less skilled, according to their self-evaluation, in both Access and Evaluate compared with the Jewish group, as can be seen in Fig. 2.1 and Table 2.2. Considering these findings, an Arabic-speaking teaching assistant was added to the team of instructors so as to support the Muslim students, with emphasis placed on skills that primarily deal with searching for, locating, and assessing relevant information. Improvement was noted in both groups following the intervention. However, a significant result was shown for the Muslim group only in Evaluate; and for the Jewish group only in Create (Figs. 2.2 and 2.3).

Students Voicing Their Challenges and Gains

31

Fig. 2.1 Differences between pre/post tests within cultural groups for Access, Analyze, and Evaluate

Table 2.2 Mean scores, and SD, values, of the cultural groups for pre/post tests Jewish group Pretest

Muslim minority group Posttest

Pretest

M

SD

M

SD

Access

4.59

1.19

4.91

Analyze

4.33

0.99

4.84

Evaluate

3.95

1.33

Create

1.08

0.28

Posttest

M

SD

M

SD

1.04

4.39

0.92

4.66

0.94

0.90

4.45

0.80

4.65

0.77

4.35

1.19

3.60

1.33

4.19

1.22

1.70

0.47

1.10

0.30

1.11

0.31

Students Voicing Their Challenges and Gains An analysis of students’ reflective journals entries was conducted to further understand their challenges and gains during the process, and more specifically with respect to their cultural differences. Three main categories that emerged during the content analysis were: 1.

Arab (Muslim) students’ first exposure to digital skills during their academic education.

32

2 Self-assessment of Digital Literacy Skills in a Multicultural …

Fig. 2.2 Differences between pre/post tests within the Jewish group for Create

Fig. 2.3 Differences between pre/post tests within the Muslim minority group for Create

2.

3.

Improving digital literacy skills—referring to the participants’ ability to acquire relevant and reliable sources of information that would yield what was needed to create the game. The contribution of the lecturers’ guidance to developing digital literacy skills.

Students Voicing Their Challenges and Gains

1.

33

Arab students’ first exposure to digital skills during their academic education

The Muslim students reported that this was the first time they had been exposed to a learning method that required problem solving by building educational games for the purpose of acquiring digital skills. The education systems from which they came used solely traditional teaching methods. This was mentioned by 29 students in their reflective journals; for example, Learning through games was new for me. I was exposed to this method for the first time during this course. I was accustomed to learning in schools using traditional methods and we were not exposed to new methods. The activity was therefore challenging. (Rana, Muslim female) This was the first time I had encountered this type of learning method. At the school in my village, we didn’t use this method because the teachers don’t want to change their methods and prefer to encourage traditional methods, which they use throughout the entire period of studies. (Rola, Muslim female)

Sa’id (Muslim male) supported his peers by writing the following in his journal: We learned how to present a dilemma and find different solutions using this learning method. We learned that we could find more than one solution to a dilemma. Everyone has a different perspective that they can suggest as a solution. The objective of the game is to learn a new method that we never learned before that will help us as teachers in the future.

2.

Improving digital literacy skills

This category consists of students’ reports that relate to developing and attaining digital literacy skills while working on gamification projects. The students understood the objective of the activity and the importance of acquiring the skills that were the basis for the activity. As Roan (Muslim female) wrote: I became extremely attached to this topic because it’s new to me, and I was interested in searching and researching it. I was asked to search for information through articles and books, and to read about the topic. I had to thoroughly understand what they were talking about in order to begin the assignment. I think that the goal of the activity was to challenge ourselves so that we would be active, search for material ourselves, and know how to use it.

However, as the qualitative data analysis showed, many of the Arab students encountered numerous difficulties, mainly with the skills involved in searching for and locating appropriate information to describe the character (avatar) they had selected in the game. Ahmed (Muslim male), for example, described the difficulties he encountered in finding articles “My character (avatar) was a person with an opinion that was not mainstream, and it was extremely difficult for me to find suitable articles”. Fatma (Muslim female), also explained that she was hesitant about “having to gather suitable information for the game”. Her main concern was “the time I would need to invest in finding suitable materials”. Yet, in fact, she admitted that she had learned how “to search for resources quickly according to topics and by combining close topics. I discovered that I really enjoy looking for materials because it’s like looking for the missing word in an answer.”

34

2 Self-assessment of Digital Literacy Skills in a Multicultural …

The Jewish students, on the other hand, reported that it had not been difficult for them to locate information (stated by 10 students). For example, Liat (Jewish female) described how she was able to rapidly assimilate the use of technology in locating resources that were needed for the game. Her description shows that having the ability to locate various sources relatively easily made it easier, in turn, for her to present different viewpoints on the problem that her game sought to present, “My classmates and I decided to create a game about homeschooling that would present an authentic problem that is connected to reality. The ability to find various articles enabled me to present different angles of looking at the problem […] to extract the most relevant points, and to combine all of them into a game that would enable students to learn and broaden their orbit of knowledge.” However, these [Jewish] students reported that while they encountered difficulties in analyzing the academic resources, they were able to overcome these obstacles by themselves, by helping each other, and/or by using various technological tools, without the help of the lecturer. Rachel (Jewish female) described the methods of digital technology used by her group to better organize a collaborative project, One of the skills that I learned was to create cooperation between the students, to let the students express their opinions, and to get them to come up with ideas. When we began to work together, we immediately created a WhatsApp group and a document folder that enabled us to work together all the time. Everyone received an update immediately if anything was added or changed. We saw for ourselves how a few lines grew into an entire computer game.

3.

The contribution of the lecturers’ guidance to developing digital literacy skills

The Arab students repeatedly emphasized the importance of the role of the course instructors in strengthening their skills and motivation. They (15 students) repeatedly described the connection the instructors made with them and the quality of their help. Unlike the Jewish students, who depended upon their friends and on technological skills to help them overcome difficulties, the Arab students depended upon personal guidance from the course instructors. For example, Aseel (Muslim female) explained: It took us a long time to make progress with this assignment. The group met more than once, but we succeeded in making progress only after we met with the course instructors who gave us directions mainly on how to find resources. We managed to overcome the difficulties. Afterwards I sat at home and read my articles and the material that we had on Moodle. I watched the videos and I began to understand what we needed to do.

Yousef (Muslim male) also described the type of help that he received from the course instructors and emphasized that in addition to attaining skills, the meeting increased his motivation and “gave him a push” to complete the assignment. The meeting helped us. She [the instructor] gave me a push forward because I felt that it was hard for me and that I wasn’t making progress. I began to write down important things, and when things were difficult for me, I would send her emails and ask her for advice. I received a lot of help and support.

Students Voicing Their Challenges and Gains

35

Gadeer’s (Muslim female) remarks help summarize the skills that were acquired during the intervention, as perceived by the Arab students. Her remarks testify to the fact that she was able to improve her skills of traditional literacy such as writing in a language (Hebrew) that was not her mother tongue as well as digital skills to which she had not been exposed previously: I acquired skills in using technology to find up-to-date sources to complete the assignment… My writing ability improved, especially my ability to summarize and combine materials. I learned to construct creative ideas […] and not to be afraid of trying new things that I was not used to. That is the only way to learn. I learned that if things are difficult, it’s all right to ask for help even more than once from my friends and instructors. I think that this is the best way to teach future generations.

Discussion This chapter devotes particular attention to the ways in which self-assessment processes in higher education might expressly be designed and managed in order to build capacity for self-regulation among students of different backgrounds. While most studies conducted on the topic of self-assessment focus on instilling knowledge, the study described in this chapter illustrates how today’s learning skills (digital or otherwise) can be augmented by means of self-assessment tools and reflective journals. These were found to be an especially insightful means of examining the differences between students of varying abilities and offered a concise description of the operational criteria on which the students needed to focus when evaluating their digital abilities. This self-assessment report also pointed to the different types of guidance that should be provided for the different groups. Arab students, for instance, reported having difficulty with retrieving digital information. They further reported that learning methods that involve problem-solving through construction of an educational game based on digital skills was not something they had previously encountered in the education systems they had attended. They described the difficulties they encountered during the course, particularly in skills associated with searching for and locating information that would be suitable for depicting the character (avatar) they had chosen for the game. These difficulties were exacerbated by a sense of frustration at the lack of ability to access and evaluate sources, some of which included research studies found in scientific publications. This is supported by a previous study (Greenberg and Bar-Ilan 2014) which presented insights into the differences between multicultural and multilingual student groups in higher education regarding digital literacy skills. The study noted that Arab students needed more guidance than their Jewish counterparts in searching for and retrieving information. The Arab students then went on to describe the important skills they had gained during the intervention. These included obtaining relevant and reliable sources of information and sorting out what was needed to meet the project’s aims. Alongside the difficulties they experienced in locating, selecting, and evaluating information, they underscored the vital role played by the instructors and their [Arabic-speaking]

36

2 Self-assessment of Digital Literacy Skills in a Multicultural …

assistant in mitigating these challenges. The instructors provided appropriate scaffolding and acted as mentors who motivated and encouraged the students to exercise the skills they were learning at their own pace. The Jewish students, on the other hand, reported relatively little difficulty in locating information. When confronted by difficulties in sorting sources, they managed to overcome them independently or with help from their peers, or by using various technological tools, without help from the instructor(s). In other words, these students were more savvy about the technological tools at their disposal, and able to rely on their friends’ digital abilities, for sorting information to complete the assignment. They described how they had coped with the challenges by using various technologies, such as WhatsApp. These technological tools helped them to communicate efficiently with one another to achieve their common, agreed-upon goals. In summary, whereas the Arab students underscored the role of the course instructor in strengthening their skills, a role based mainly upon personal, one-onone mentoring, their Jewish classmates preferred to seek help from their peer group. The interaction with the instructor was especially important for Arab students as it not only strengthened their digital skills but offered them emotional benefits as well.

Implications and Recommendations The self-assessment tools employed in this study encouraged students to identify and reflect on their learning strengths and weaknesses, to seek guidance when necessary, and exert an effort towards achieving their goals. Self-assessment also enabled the course instructors to offer students more precise direction and to differentially adapt their teaching practices and support for individual students. The initial assessment enabled instructors to understand the type of guidance that would be best suited to the Arab students. Therefore, an Arabic-speaking teaching assistant was hired for the purpose of providing culturally responsive and sensitive guidance to these students. The initial assessment also increased the students’ awareness of their weak points and their capabilities vis-a-vis their digital skills, thus helping them pinpoint the various challenges they faced and, consequently, seeking help to improve those skills which they identified as being weaker. Hence, self-assessment can help students diagnose their baseline in relation to the academic skills necessary to support the learning process. In line with Boud (2010), this self-assessment process comprises two main elements: first, determining the expected standards of performance. In the current intervention, it was decided to employ a validated tool due to the students’ poor prior knowledge and insufficient digital literacy skills; and second, making judgments about the quality of the individual’s performance in relation to these standards. In the case study investigated and presented here, this involved evaluations by both students and instructors. Hence, the self-assessment in our case study consists of a process of formative assessment during which students can reflect on and evaluate the quality of their skill set, identify the strengths and weaknesses in their work, and set goals accordingly.

Implications and Recommendations

37

This study also points to increased self-regulation of learning as an important consequence of self-assessment. In the learning process outlined in the present study, the students monitored and evaluated their own performance and generated feedback as to what should be done next. Self-assessment has been found to improve this metacognitive functioning (Alt and Raichel 2020b); hence, the importance of self-assessment interventions in promoting students’ use of deep-learning strategies should be acknowledged by instructors in higher education settings.

References Alt, D., & Raichel, N. (2018). Lifelong citizenship: Lifelong learning as a lever for moral and democratic values. Leiden and Boston: Brill and Sense Publishers. Alt, D., & Raichel, N. (2020a). Enhancing perceived digital literacy skills and creative self-concept through gamified learning environments: Insights from a longitudinal study. International Journal of Educational Research. https://doi.org/10.1016/j.ijer.2020.101561. Alt, D., & Raichel, N. (2020b). Reflective journaling and metacognitive awareness: Insights from a longitudinal study in higher education. Reflective Practice, 21, 145–158. Andrade, H. L., & Brown, G. T. L. (2016). Student self-assessment in the classroom. In G. T. L. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 319– 334). New York: Routledge. Andrade, H. & Du, Y. (2007). Student responses to criteria-referenced self-Assessment. Assessment and Evaluation in Higher Education, 32(2), 159–181. Andrade, H. L., Du, Y., & Wang, X. (2008). Putting rubrics to the test: The effect of a model, criteria generation, and rubric referenced self-assessment on elementary school students’ writing. Educational Measurement: Issues and Practice, 27, 3–13. Blum-Ross, A., Kumpulainen, K., & Marsh, J. (Eds.). (2019). Enhancing digital literacy and creativity. Makerspaces in the early years. London: Routledge. Boud, D. (1995). Enhancing learning through self-assessment. London: Kogan Page. Boud, D. (2010). Sustainable assessment: Rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151–167. Boud, D., & Falchikov, N. (1989). Quantitative studies of student self-assessment in higher education: A critical analysis of findings. Higher Education, 18, 529–549. Boud, D., & Falchikov, N. (2007). Introduction: assessment for the longer term. In D. Boud, & N. Falchikov, (Eds.) Rethinking Assessment for Higher Education: Learning for the Longer Term (pp. 14–25). London: Routledge. Bourke, R. (2016). Liberating the learner through self-assessment. Cambridge Journal of Education, 46, 97–111. Bourke, R., & Mentis, M. (2013). Self-assessment as a process for inclusion. International Journal of Inclusive Education, 17, 854–867. Brown, G. T. L., & Harris, L. R. (2014). The future of self-assessment in classroom practice: Reframing self-assessment as a core competency. Frontline Learning Research, 2, 22–30. Buzzetto-Hollywood, N. (2017). Establishing an institutional commitment to the digital and information literacy of underserved college students. In Proceedings of the 2017 Drexel Assessment Conference, 13–15 September 2017 (pp. 14–27). Philadelphia, Pennsylvania. Buzzetto-Hollywood, N., Wang, H. C., Elobeid, M., & Elobaid, M. E. (2018). Addressing information literacy and the digital divide in higher education. Interdisciplinary Journal of e-Skills and Lifelong Learning, 14, 77–93.

38

2 Self-assessment of Digital Literacy Skills in a Multicultural …

Byrne, JR., Sullivan, K., & O’Sullivan, K. (2018). Active learning of computer science using a Hackathon-like pedagogical model. In Proceedings of the Constructionism 2018: Constructionism, Computational Thinking and Educational Innovation conference, 21–25 August 2018 (pp. 138–149). Vilnius, Lithuania. Castaño-Muñoz, J., Colucci, E., & Smidt, H. (2018). Free digital learning for inclusion of migrants and refugees in Europe: A qualitative analysis of three types of learning purposes. International Review of Research in Open and Distributed Learning, 19(2) Retrieved from https://ezproxy.kin neret.ac.il:2059/docview/2056772236?accountid=39607. ˇ Cernochová, M., Selcuk, H. (2019). Digital literacy, creativity, and autonomous learning. In A. Tatnall (Ed.), Encyclopedia of education and information technologies (pp. 1–8). Switzerland: Springer Nature. Dann, R. (2014). Assessment as learning: Blurring the boundaries of assessment and learning for theory, policy and practice. Assessment in Education: Principles, Policy & Practice, 21, 149–166. De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75, 82–91. Dixon, J. (2017). First impressions: LJ’s first year experience survey. Library Journal. Retrieved from http://lj.libraryjournal.com/2017/04/academic-libraries/first-impressions-ljs-first-year-exp erience-survey/. Dubois, D. J., & Tamburrelli, G. (2013). Understanding gamification mechanisms for software development. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, 19–23 August 2013 (pp. 659–662). ACM. Eshet, Y. (2012). Thinking in the digital era: A revised model for digital literacy. Issues in Informing Science and Information Technology, 9, 267–276. Frank, T. H. J., & Castek, J. (2017). From digital literacies to digital problem solving: Expanding technology-rich learning opportunities for adults: The resource for adult education. Journal of Research and Practice for Adult Literacy, Secondary, and Basic Education, 6, 66–70. Frau-Meigs, D., O’Neill, B., Soriani, A., & Tomé, V. (2017). Digital citizenship education: Overview and new perspectives. Strasbourg: Council of Europe Publishing. Greenberg, R., & Bar-Ilan, J. (2014). Information needs of students in Israel — A case study of a multicultural society. Journal of Academic Librarianship, 40, 185–191. Henriksen, D., Mishra, P., & Fisser, P. (2016). Infusing creativity and technology in 21st century education: a systemic view for change. Educational Technology & Society, 19, 27–37. Hill, T. (2016). Do accounting students believe in self-assessment? Accounting Education, 25, 291–305. Hobbs, R. (2011). Digital and media literacy: Connecting culture and classroom. Thousand Oaks, CA: Corwin Press. Horton, F. W. (2007). Understanding information literacy: A primer. Paris: UNESCO. Hunter, I. (2018). Digital literacy in the workplace: A view from the legal sector. Business Information Review, 35 56–59. Koltay, T. (2011). The media and the literacies: media literacy, information literacy, digital literacy. Media, Culture & Society, 33, 211–221. Ktoridou, D., & Eteokleous-Grigoriou, N. (2011). Developing digital immigrants’ computer literacy: The case of unemployed women. Campus-Wide Information Systems, 28, 154–163. Lam, W. S. E. (2012). What immigrant students can teach us about new media literacy. Phi Delta Kappan, 94, 62–65. Martin, A. (2005). DigEuLit–A European framework for digital literacy: A progress report. Journal of eLiteracy, 2, 130–136. Merdler, M., & Peled, Y. (2016). Tendencies and preferences of choosing information sources in academic learning: A case of native Hebrew and native Arabic speakers in Israel. Journal of Educational and Social Research, 6, 39–47. Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated Learning. Educational Psychologist, 36, 89–101.

References

39

Pieterse, E., Greenberg, R., & Santo, Z. (2018). A multicultural approach to digital information literacy skills evaluation in an Israeli college. Communications in Information Literacy, 12, 107– 127. Quann, S. (2015). Integrating digital literacy and problem solving into instruction. LINCS Regional Professional Development Center for Adult Education. Retrieved from http://www.worlded.org. Raaijmakers, S. F., Baars, M., Paas, F., van Merriënboer, J. J. G., & van Gog, T. (2019). Effects of self-assessment feedback on self-assessment and task-selection accuracy. Metacognition and Learning, 14, 21–42. Reder, S. (2015). Digital Inclusion and Digital Literacy in the United States: A Portrait from PIAAC’s Survey of Adult Skills. Retrieved from https://static1.squarespace.com/static/51bb74b8e 4b0139570ddf020/t/551c3e82e4b0d2fede6481f9/1427914370277/Reder_PIAAC.pdf. Sadler, T. D., Romine, W. L., Stuart, P. E., & Merle-Johnson, D. (2013). Game-based curricula in biology classes: Differential effects among varying academic levels. Journal of Research in Science Teaching, 50, 479–499. Sambell, K., McDowell, L., & Montgomery, C. (2013). Assessment for learning in higher education. Abingdon, U.K.: Routledge. Summers, M. M., Cox, T. L., McMurry, B. L., & Dewey, D. P. (2019). Investigating the use of the ACTFL can-do statements in a self-assessment for student placement in an Intensive English Program. System, 80, 269–287. Taras, M. (2010). Student self-assessment: processes and consequences. Teaching in Higher Education, 15, 199–209. Thorne, S. L. (2009). Mediating technologies and second language learning. In J. Coiro, M. Knobel, C. Lankshear, & D. J. Leu (Eds.), Handbook of research on new literacies (pp. 415–447). New York, NY: Routledge. Torralba, J. A. (2015). A survey of emergent digital literacy inside the homes of Latino immigrants in California: Digital literacy inside the homes of Latinos in California. Journal of International Migration and Integration, 16, 491–515. Vanek, J. (2017). Using the PIAAC framework for problem solving in technology-rich environments to guide instruction: An introduction for adult educators. Retrieved from https://static1.square space.com. Wolffensperger, Y., & Patkin, D. (2013). Self-assessment of self-assessment in a process of coteaching. Assessment & Evaluation in Higher Education, 38, 16–33. Wong, H. M. (2017). Implementing self-assessment in Singapore primary schools: effects on students’ perceptions of self-assessment. Pedagogies: An International Journal, 12, 391–409. Yan, Z., & Brown, G. T. L. (2017). A cyclical self-assessment process: Towards a model of how students engage in self-assessment. Assessment & Evaluation in Higher Education, 42, 1247– 1262. Yoo, S. J., & Huang, W. H. D. (2011). Comparison of Web 2.0 technology acceptance level based on cultural differences. Educational Technology & Society, 14, 241–252. Zhao, P., Kynäshlahti, H., & Sintonen, S. (2018). A qualitative analysis of the digital literacy of arts education teachers in Chinese junior high and high schools. Journal of Librarianship and Information Science, 50 77–87.

Chapter 3

Reflective Journal Writing and Multicultural Diversity

Introduction The use of reflection as a learning tool (Boud et al. 1985; Kolb 1984; Schön 1983) has been practiced over the last 30 years in educational programs and for professionaldevelopment training. In the reflection process, the learner purposely contemplates past or imminent experiences, with a view toward effecting future improvement; s/he also identifies and seeks to resolve a problem or question whose answer is likely to be of practical use to her/him. This process should lead to new understandings and appreciations (Cowan 2014). Notably, a large corpus of research has demonstrated the importance of reflection in learning, particularly in higher education (Rogers et al. 2019). Yet, in contrast to the plethora of literature on the teaching and learning of reflection, the almost complete lack of extensive research related to promoting reflective practices in multicultural learning environments is inescapable. Given the challenges which multicultural students encounter in higher education, the question of how such higher-order thinking skills can be practiced by culturally diverse students lies at the core of this chapter.

Reflection: A Lifelong Learning Skill The ability to reflect on one’s learning is a fundamental skill necessary for lifelong learning (Ryan 2015). As reflection is well established as an essential component of lifelong learning and professional development, it is widely acknowledged that reflective practices offer higher education potential for engendering lasting and effective changes in students’ personal and/or professional lives (Griggs et al. 2018; Waggoner-Denton 2018). Lifelong learning is an approach whereby the individual engages in a continuous process of adjusting his capabilities in relation to the changing expectation of the work and learning environments (Archer 2007). Arguably, in the current fluid and © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_3

41

42

3 Reflective Journal Writing and Multicultural Diversity

changing environment, individuals must actively manage their capabilities in a meaningful manner. Individuals who are able to reorient themselves in new ways are more likely to have a sense of agency and make effective choices which require a reflective approach to learning. Therefore, developing students’ capacity to engage in reflective practices has been recognized as an essential goal in higher education and for preparing students effectively for their future personal and professional development (Adie and Tangen 2015). Lifelong learning generally includes the notions of self-regulation and selfassessment (Ryan 2015). The latter comprises our use of the information and feedback we receive from multiple sources. Schön (1983) maintained that through these practices, the practitioner can make new sense of situations of uncertainty. Andrade and Du (2007) viewed self-assessment as a process of formative assessment “during which students reflect on and evaluate the quality of their work and their learning, judge the degree to which they reflect explicitly stated goals or criteria, identify strengths and weaknesses in their work, and revise accordingly” (p. 160).

Reflective Journals Assessment as learning encourages students to increase their ‘meta’ skills so that they become conscious of how they learn (O’Farrell 2017). An active learning method to promote the development of those higher-order skills is keeping a reflective journal, also referred to as a learning diary/journal, or a learning/response log. Such a journal is perceived as a vehicle for reflection (Moon 2003). Reflective journals are defined as “… written documents that students create as they think about various concepts, events, or interactions over a period of time for the purposes of gaining insights into self-awareness and learning” (Thorpe 2004, p. 328). Journal writing is related to metacognition as it slows the pace of learning, increases the sense of ownership of learning, acknowledges the role of emotion in learning and… is claimed to enable writers to stand outside their experience, seeing it more objectively, and being detached from emotional outcomes… It has been described as a bridge across which learners can move from the specific to the general, while developing a habit of reflection (Cowan 2014, p. 54).

Students who are aware of their tasks and have knowledge of themselves as learners will more effectively apply metacognitive strategies. Hence, reflection and metacognition are pillars for a richer learning experience (Desautel 2009). The aims of this tool are to express the observation of the learning process and evidence of reflection (Wallin and Adawi 2018). An essential benefit of this assessment tool is the creation of an environment where students feel encouraged and safe to express their concerns and explore their thoughts, reflect on their beliefs, values, experiences, and the assumptions that influence their learning, as well as their development and progress over time (Minott 2008). Reflective journals can focus on learning content or behavior (Fabriz et al. 2014), directed by self-monitoring prompts. While activity

Reflective Journals

43

prompts help students to reflect on and improve their work, self-monitoring prompts help students to think about their learning approaches and processes, and thus bring them to the fore (English and Kitsantas 2013; Wallin and Adawi 2018). Reflective journals, which have been employed in a wide array of higher education disciplines, have been proven effective in supporting students’ in-depth learning, and assisting students in exploring their values, beliefs and assumptions. In this way, they facilitate students’ learning from experience, support career management, develop critical thinking, and increase the development of reflective judgment and problem-solving skills. Furthermore, reflective diaries contribute to making connections between old and new knowledge, while promoting behavioral change, supporting planning and progress in research or a project, enabling students to identify complexity and conflicts in the workplace, and building professional practice confidence (Moon 2006; Pack 2014).

Reflective Writing and Future Professional Development A central goal of education is to provide learning experiences that are useful beyond the specific conditions of initial learning (Lobato 2012; Marton 2006). Those refer to activities undertaken by students which generate knowledge beyond the information given (Biggs et al. 2001). Reflection represents a deeper form of learning and moves from questioning assumptions and viewpoints to changing conceptual meanings, altering internal perspectives, and modifying future behaviors (Peltier et al. 2005). Drawing on Kolb’s (1984) theory of experiential learning, which suggests that learners must first reflect before they can move on to active experimentation, Brown et al. (2011) suggest using reflective journal writing as a technique designed to encourage trainees to learn the content and assess their experience in real time as they apply it. Hence, arguably, when students are guided to reflect deeply on their learning, they are further encouraged to contextualize their learning in relation to their current academic and future professional lives (Adie and Tangen 2015; Dyment and O’Connell 2011). Based on this notion, two sets of dimensions for questions to prompt students to engage in reflection processes were set by the Learning to Learn Project (2002) team. The first set entitled “Explore a learning experience” deals with the specific and the immediate learning processes. This helps students’ reflect on their current performance. The second concerns lifelong learning skills or long-term issues and widening the perspective of students, helping them appreciate what they are learning, and engaging them to contemplate their learning experience in relation to academic, professional, and/or personal development.

44

3 Reflective Journal Writing and Multicultural Diversity

Reflective Journal Writing and Diversity Several attempts have been made to investigate whether different cultural educational approaches impact student’s ability to engage in reflective thinking. Rushton and Duggan (2013), for example, used reflective practice to enable multicultural students to engage more effectively in their personal and professional development. Students were encouraged to learn, through reflection, from opportunities that arise throughout their experience of studying, living and working as international exchange students in Sheffield, UK. The students were required to engage in reflective activities through both formative and summative coursework. Content analysis of the completed questionnaire returns, accompanied by focus-group discussion, showed that students from all cultures experienced varying degrees of difficulty in reflective writing. Written reflection was not familiar in their respective educational cultures, yet they also noted that in a safe environment they felt they were able to engage in this practice. Language and communication issues proved to have the greatest impact on the nature of the difficulties that students encountered. The researchers observed that students appreciated and valued the reflective approach, yet also came to realize the need to teach thoroughly the skill of self-reflection and reflective writing and not assume that it came naturally… in many languages there is no word for ‘reflection’ as used in western academia. Students need substantial practice and practical support in the classroom to enable their effective engagement with this process. In addition to theoretical teaching and underpinning, students need examples, practice and immediate supportive feedback- feed forward on relation to reflective practice (Rushton and Duggan 2013, p. 959).

Another pivotal aspect is the importance of incorporating reflection into the education of teachers, who may use this valuable practice in their future diverse learning environments, as asserted by Bailes et al. (2010), “… the complexity of school change, and the need for teachers to engage in lifelong learning have encouraged teacher education programs to seek sources of knowledge that lead to self-directed growth. That goal underlies the rationale for incorporating reflection into teacher education” (p. 235). In a similar vein, Miller-Dyce and Owusu-Ansah (2016) maintained that “critical self-reflection provides a third space, where preservice teachers can begin the process of critiquing their own positionality as it relates to their teaching and learning practices when working with diverse students and families” (p. 332). In their study, multicultural education was found effective in increasing preservice teachers’ knowledge of diversity. The researchers conclude that, Thinking and reflecting critically on diversity, culture, and schooling may be enhanced when preservice teachers are exposed to critical pedagogies in education such as culturally relevant teaching, social justice education, and multicultural education. An understanding of these critical frameworks provides preservice teachers with pedagogical tools to create culturally sensitive classroom teaching and learning experiences that will benefit all children (p. 351).

In this same line of thought, Moloney and Oguro (2015) maintained that teachers’ development of critical cultural skills is essential to their awareness of how they communicate with students in the multilingual, multicultural classrooms

Reflective Journal Writing and Diversity

45

of today. The researchers analyzed 33 preservice teachers’ written narrative reflections designed to elicit description and analysis of their experiences of linguistic and cultural difference. The reflective narrative was found effective in supporting critical intercultural understanding for the preservice teachers and sharpened the awareness of their role in affording opportunities for critical cultural thinking in their future classrooms. The researchers stressed the importance of using a structured reflective narrative task to enhance an active and continuous process of personal and professional growth for all teachers, which is “fundamental to their ability in reducing prejudice and racism in the culturally and linguistically diverse classrooms which are increasingly common today. This is relevant… to all areas of the curriculum, as it has the potential to contribute to understandings of citizenship, human rights and anti-racism” (p. 105). Similarly, in Kyles and Olafson’s (2008) study, teacher candidates, who were placed in an urban, culturally diverse practice-teaching site in an elementary school, were required to investigate personal beliefs and past experiences with cultural diversity in the form of writing reflective response letters and cultural autobiographies. The researchers found this practice beneficial: The use of reflective narratives in combination with field experiences may provide a more effective vehicle from which preservice teachers can begin to uncover and articulate their beliefs about diversity and teaching diverse learners… every opportunity for contact with cultural diversity, both personally and professionally, provides teacher candidates with added experiences for reflection that connect with and build on previous experiences, reaffirm cultural diversity, or challenge existing notions and prejudices (p. 505).

Despite the recognized benefits to preservice teachers, a review of the literature (Standal and Moe 2013) on reflective practices in teacher education since 1995 points to preservice teachers’ inability to reflect critically. Some encounter problems such as lack of relevant training and the need for sensitivity to individual needs and differences (Cowan 2014). It should also be noted that most of the studies on reflection in teacher education (e.g., Degago 2007; Knapp 2012) did not use a comprehensive framework to evaluate or induce reflective writing. Notably, alongside the numerous studies on reflection is a paucity of empirical work on the dimensionality of reflective writing, and, more specifically, in the context of multicultural teacher education learning settings. Consequently, this chapter sets out to establish a reflection scheme based on the analysis of multicultural students’ reflective journal entries. Second, it offers prompts that might encourage competent and profitable reflection. Lastly, it outlines minority students’ challenges in reflective writing processes.

Reflective Journal Dimensionality and Structure The use and research of reflective journals to encourage higher order learning outcomes is a growing field in higher education research and practice. Therefore, engaging students’ in reflective practices has been recognized as a central goal for deep learning and transformation and for preparing students effectively for their

46

3 Reflective Journal Writing and Multicultural Diversity

professional lives. Several researchers (Alt and Raichel 2021; Assessment tools for higher education learning environments [ASSET] Erasmus+ project 2020; Learning to Learn Project 2002) have created a reflection scheme with a bank of prompting questions (Table 3.1). This scheme includes two dimensions. The first refers to students’ current experiences, or “short-term related reflections.” This dimension deals with students’ in-process experiences during the course. The reflection includes the following levels: 1. 2. 3.

Cognitive—relates to the course content, learning skills, and learning purposes; Behavioral—refers to the student’s behavior during the learning process; Emotional (affective)—pertains to emotions arising during the learning experience.

The second dimension pertains to long-term related reflections and includes students’ learning experiences in relation to their future in terms of their: 1. 2. 3. 4.

academic development; professional development; personal development; multicultural development.

In addition, three essential metacognitive abilities were foregrounded which include: awareness, evaluation, and regulation. Table 3.1 illustrates the scheme, with a bank of suggested prompting questions.

Minority Students’ Challenges and Benefits Related to Reflective Writing Processes Alt and Raichel (2021) used reflective journaling with 75 first-year undergraduate preservice teachers (30% Jewish students, 70% Muslim and Christian students). The students were enrolled in a mandatory two-semester course entitled “Philosophy of Education”. The course included lectures and two group assignments. During the first month of the first semester, an unstructured reflective journal technique was used, whereas in the second month, a semi-structured reflective journal was provided to the students, including several prompting questions from which students could choose to answer in each entry. These questions were aimed at leading students to identify any difficulties they encountered during the learning process and to devise plans and remedies for solving these problems. Students were directed to contemplate things that had the greatest personal significance to them and convey them freely. During the first semester of the course, students submitted a journal entry following each lesson. Throughout the second semester, students were required to submit four journal entries. In this phase, an additional set of prompting questions was used, with the aim of engaging students to think of learning experiences in relation to their future professional career or personal lives. Following this activity, 30 semi-structured interviews were conducted with 15 Arab (both Muslim and Christian) students and 15

Minority Students’ Challenges and Benefits Related …

47

Table 3.1 Reflective writing scheme and prompting questions Dimension 1

Short-term related reflections Awareness

Evaluation

Cognitive

– What have I learned? – What were the topics of the activity? – What was I required to do? – Why should I learn this? (What was the purpose of the activity/lesson?)

– Do I understand – What can I do to what I have learned? gain a better – What else do I need understanding? to learn? – Where can I find – What do I think more information? about these – What experiences purposes? can I gain from this – What other purposes activity to improve can be suggested? my learning skills? – Were my learning milestones and goals mostly met, and how much did I deviate from them, if at all?

Behavior

– How did I learn/do it – How effective is this – Is the way I did it is (a specific topic)? strategy? the best way? – What strategy did I – How can I make this apply in learning this strategy more topic? effective? (I wrote everything down, learned with others in a group?) – How did I work in the group?

Emotions (affective level)

– How did I feel during the activity? – What did I like or dislike about it? – How do I think my peers in the group felt during the activity?

Dimension 2

Long-term related reflections Awareness

Academic development

– Why did I feel this way? – What contributed to these feelings?

Evaluation

Regulation

– What can improve my feelings towards the activities? – Who can I consult with to manage my feelings?

Regulation

– What obstacles did I – What is the source – How can I remove encounter? of the obstacles? those obstacles? – What did I find easy – How did I link this – What techniques can to do? activity to my prior I use to link my – What can be knowledge? learning to prior connected to my – What can be done to knowledge and prior knowledge? better connect the skills? activity to the students’ prior knowledge? (continued)

48

3 Reflective Journal Writing and Multicultural Diversity

Table 3.1 (continued) Dimension 2

Long-term related reflections Awareness

Evaluation

Regulation

Professional development

– How can I transfer – How can the – What experiences the knowledge and knowledge and skills can I gain from this skills I learned I learned be adapted activity to improve during the activity to to my professional my professional my professional work? development? (What should be work? – What knowledge added/omitted?) and skills will be useful in my professional work?

Personal development

– What have I learned from the activity about myself on the personal level? – What have I learned are my greatest strengths/areas for improvement – At what moments was I most proud of my efforts? – What personal traits have I discovered?

– What does this – How can I use these learning experience insights for my tell me about my future personal potential and myself development? as a person? – What’s the one thing about myself, above all others, I would like to work on improving?

Multicultural development

– What have I learned from the activity about my or other cultures? – At what moments was I most proud of my culture? – What cultural traits did I discover?

– How can this activity be linked to my/others’ culture/s?

– What can be done to better connect the activity to the students’ culture? – What techniques can I use to connect my learning to my culture?

Jewish students, which were then analyzed to glean information on the reflective writing process from a multicultural point of view, and to identify the challenges and opportunities facing diverse students in this context. The analysis foregrounded four categories: 1. thinking skills; 2. emotional challenges; 3. collectivist vs. individualist culture; and 4. prior knowledge and experience.

Thinking Skills Ten of the Muslim students reported that this was a new experience for them. Their descriptions were filled with words that expressed hardship. As time went on, they reached a greater understanding of the essence of the process; moreover, the students’

Thinking Skills

49

ability to understand the advantages of reflection in promoting learning skills grew. This is expressed in Asma’s (Muslim female) remarks: It was difficult for me to perform reflection in philosophy. At first, I didn’t manage to express myself: How much can you write about the topic you are studying? I had a hard time overcoming it. It was boring. I felt as if every lesson I was writing the same thing. As time went by, I understood the need to think about the processes during the lesson. I developed critical thinking while writing my reflections. This helped me considerably to understand the material. I discovered not only what I understood and what I didn’t, but also what helps me understand and what I need to do. I highlighted all the important things with a marker during the lesson. I noted what I had to ask. I learned to write down comments during the lesson. Before performing reflection, I quickly read what I wrote. That helped me to comprehend the aspects that I understood less.

Similarly, Rola (Muslim female) noted that the journal writing process helped her regulate her learning, and that over the course of time, the process helped her overcome learning challenges: I learned that it’s good for me to think about what I learned, and that I’m capable as well. It compelled me to think about what I understood and what I didn’t, and why. I saw that it’s really good for me to devote a few minutes to what occurred in class and to answer the questions, because I feel that I am teaching myself. The number of learning difficulties I have declined dramatically from the first to the second semester.

Alaa (Muslim female) noted that this kind of reflective practice is not taught in the Arab school system. She describes how writing helped her think about ways to improve her learning—things that are difficult to think about during the lesson: Writing a journal helps organize the material and teaches me what I didn’t understand or what is difficult for me, and I think about how I can learn better. This is the first time I have ever learned to perform reflection. I don’t think anyone is talking about reflection in the Arab schools. God willing, I will be a teacher and will teach this to my students.

Samar (Muslim female) described how the process of reflective journaling helped her improve her writing in a language that is not her mother tongue and remember the course material better: Writing reflections helped me to improve my Hebrew. While I write my reflections, I recall what was in the lecture. It interests me and also helps me learn. I discovered that I remember and retain things in my memory. When I began to study for the test, it was enough for me to skim over some of the material because it was all in my mind from the reflections.

This topic came up during interviews with five Jewish students, but in a less comprehensive manner. They described the initial difficulty in writing reflections and its contribution to developing the ability to think and express oneself in writing. Maayan (Jewish female) explains: I had difficulty writing reflections. Sometimes I had a lot of difficulty expressing myself, thinking about how I was learning, and what bothers me, and to analyze the process in every lesson. I think that [eventually] reflection enabled me to develop my writing abilities and my thinking about how I learn. I learned how to improve my learning: when to write, when to listen, when to chat, and when to concentrate.

50

3 Reflective Journal Writing and Multicultural Diversity

Emotional Challenge Another aspect that arose, which was unique to the Muslim students, pertained to emotion. Reflective writing helped these students, as reported by 11 of them, overcome obstacles such as fear and embarrassment, and was perceived as a very personal process that enables voicing one’s opinions freely and compensates for the fear of speaking in the classroom, as explained by Rana and Suha (Muslim females), respectively: Reflection lets us feel and express our own ideas without fear or embarrassment. It helps me develop the feelings and opinions that I write in my journal but wouldn’t dare to express in class. Reflection is a part of learning. Thinking about the topic and how we learn it is even more important than learning it. I want to be a teacher. It’s important for me to know what helps students learn. I learned that I shouldn’t feel shy or inferior. I have to ask when I don’t understand something. It needn’t be unpleasant when classmates answer and explain things. Once I feel better about asking questions, I’ll be able to contribute more to members of the group.

Collectivist Versus Individualist Culture One of the main topics the students raised related to the role that writing in a journal plays in encouraging their thinking about cultural aspects. Nine Jewish students related to the difficulty in working with their colleagues from minority groups (me versus them), I got to know the Arab students through working together, which included both the Arab and Jewish sectors. There were language gaps and differences in the level of understanding of the material, writing, and verbalizing (Adam, Jewish male).

Eight Muslim and Christian students, on the other hand, related to the connection to their own culture. Writing in a journal caused them to think and to better understand the connection between the topics of the course and culture. Rana, Ela (Christian and Muslim females, respectively), and Ahmed (Muslim male) explained that including material relevant to their culture in the curriculum increased their interest in the course: As a Christian, I saw what I had written about the Church when we talked about education during the Middle Ages, when the Church was the only vehicle to move from one social class to another. I read the information I had received when we learned about the Renaissance, when education and culture always took place through the Church, and anyone who wanted to be educated would go to the church and learn from the monks. Even girls could receive an education from the nuns. I felt that this spoke to me a lot when I reviewed topics that had to deal with the church. I was pleased that I learned about the church even in general learning. I asked myself what I felt and why, and my answer was that my learning enriches me and contributes more to me when I grow, from the perspective of my culture as well. I read the reflection I wrote about Al-Ghazali. Among other reflections, I read the introduction to Arabic literature with the lecturer’s permission. It was easy for me because I finally read something in my own language, in which the text was written. It was special and different.

Collectivist Versus Individualist Culture

51

Even now, I almost wrote the philosopher’s name in Arabic. I wanted to do that because I felt ‘a sense of belonging’ with the material. I learned from reflection how important Arabic literature is to me. It is important to me for people to respect our culture. They should also teach it and present its authors and philosophers. The lecturer should give examples of good behavior from Islam as well. When that happened, the lesson was better. Other lecturers should also try to make connections in class to others’ cultures.

On the individualistic level, mainly the Jewish students (seven) elaborated on the contribution that writing in a journal made towards their personal development. This was described by Shir and Iris (Jewish females) respectively: I learned about myself through reflection. I learned that I am sensitive when working in groups. I expect everyone to do an equal share. I assume responsibility for tasks and then become angry due to the pressure and have difficulty withstanding it. I wrote an “angry” reflection following the self-learning. After some time went by, I reread it and [as a result] during the second group project I worked differently. I still have things to learn but I’ve made progress. I’ve learned something about myself. After an entire year I feel comfortable with reflection. Writing reflections transformed part of the process of learning the subject. I was apprehensive about the lessons in educational philosophy, but writing made them relevant and personal. It didn’t help me in exams, but it helped me learn about my own learning and strengthened my growth as a curious student.

Prior Knowledge One of the main topics that appeared in statements made by seven Jewish students but was absent among their Arab counterparts was the connection to prior knowledge, mainly knowledge related to topics they had learned in previous courses or acquired during training in the field of education. This was expressed by Sigal and Sarit (Jewish females) respectively: I like these moments of writing reflections in philosophy when I see connections to knowledge from other courses, previous knowledge, and experience in the field. They assume a broader dimension in my observation. It’s like a puzzle with pieces of my life in which the entire picture is slowly revealed. Knowledge and experience, combined with new insights, complete a picture of myself as I want to be with a defined “raison d’etre” and the path to pursue that is best for me. I was taught to find the connection between the topic being studied – the philosophy of education – and my life. I thought through reflection about how the lessons were connected to me. I was in Europe about a month ago. When I came back, we talked about school building designs and how it is connected to a philosophical approach and to education. When I wrote my reflection and answered the questions, everything suddenly became clear. It was an important moment in which my life became connected to what I had studied, and my knowledge of what I had seen and heard became more profound.

52

3 Reflective Journal Writing and Multicultural Diversity

Discussion Analysis of the interviews showed that the process of reflection enabled the Arab students to improve the Hebrew language in which they study, which is not their mother tongue. It also enabled them to think about the learning process and ways to improve it, to express feelings, and to examine effective ways of overcoming obstacles such as fear of speaking in front of an audience and expressing their opinions in class. The analysis also showed that writing in a journal was initially challenging for the Jewish students as well. As was the case among their Arab peers, as time went on, they too recognized the advantages of keeping a journal. This attests to the need to encourage students to persevere in journal writing. In addition, the findings show that, despite their initial feelings of deterrence and wariness, the Arab students came to view the journal as a tool that enabled them to think about the important place of their culture in the contents of the course. This thinking included descriptions that attest to their heightened motivation to take an interest in those contents and participate in class discussions and activities. Aside from the importance of including contents in the course curriculum that are relevant to the variety of cultures represented in the class, reflective journal writing may serve to increase students’ awareness of the diversity of cultures exemplified by the course material. An effective way of doing so is by posing prompting questions. It is also evident that the Arab students attributed a significant emotional role to writing in a journal. Their journal was perceived as a personal space in which they could make their voice heard far more than in the classroom. These students seldom express their opinions openly due to cultural constraints and personal barriers. They view journal writing as a more private, intimate channel of communication with the teacher that allows them to express themselves freely. Lastly, it is evident that the journal enabled the Jewish students to draw a connection between the course contents and their academic and non-academic experiences such as travel abroad or teaching. This may also be attributed to the Jewish students being somewhat older than their Arab peers, having done 2–3 years of compulsory military service and subsequently traveling abroad, which is customary in the Israeli Jewish culture, before commencing their academic studies. On the other hand, the Arab students typically pursue higher education directly after their high-school graduation and are therefore younger and less mature than their Jewish counterparts.

Implications and Recommendations It appears that for some Arab students, an invitation on the part of the lecturer to express their thoughts and feelings by writing them down in a journal is perceived as a better means of communication than maintaining direct contact with the lecturer. After several interactions involving the student’s writing and the lecturer’s feedback, the level of trust between the lecturer and the student eventually increased. This in

Implications and Recommendations

53

turn enabled students to open up emotionally and express their feelings, difficulties, and successes. In this context, it seems that in multicultural learning environments, students benefit more from a reflection process that features cognitive, interpersonal, and emotional facets, since the learner’s emotional state, according to Cowan (2014) is ‘the point where body, culture and mind come together” (p. 58). It is worthwhile encouraging students from diverse cultures to think and express themselves about cultural issues that arise during the learning process. In this way, for example, the voice of the majority group in the class can be heard in relation to the challenges of their interactions with the minority group, and vice-versa. As illustrated in this book, this is an important aspect to consider in a multicultural setting. These reflections can help teachers better understand the dynamics in a heterogeneous classroom and learn from the students’ viewpoint as to which practices work and which do not. Students’ testimonies as expressed in their journals can reveal the strengths found in the interactions between them as well as whatever identified weaknesses may require improvement. It may be necessary to encourage students in multicultural classrooms to draw a connection between their personal experiences and the courses they took towards their degree with an emphasis on promoting thinking associated with prior knowledge. Consideration of students’ prior life experiences and learning may aid in recognizing and surmounting the limitations imposed by traditional subject-based categories of knowledge and contribute to a holistic perspective toward learning rather than a narrow ‘here and now’ view. Importantly, these reflections may also help teachers to tailor the instructional methods and course contents to students’ prior knowledge. This chapter also highlights the importance of using questions related to different dimensions along the provided scheme (long- and short-term related prompts). The use of questions may enhance students’ awareness of the tasks and to identify what they do and do not know. Moreover, it encourages them to consider the effectiveness and limitations of their thinking and of a particular chosen strategy and to think about setting goals and taking actions needed to facilitate their learning process. Teachers’ use of prompting questions may also signal to the students that difficulties along the path of learning can be expected and that discussing ways to deal with these challenges is a legitimate practice that generates a routine examination of past experiences and planning for the future. At the beginning of the process, the structured journal might serve as a scaffold that leads the students through their reflective process. However, over time, the prompting questions might be internalized by the students and become an integral part of their high-quality reflective thinking and self-directed learning processes (Alt and Raichel 2020). Despite the widely recognized assumption that preservice teachers must acquire knowledge about cultural and social diversity and consider effective strategies to cope with minorities and their families, research of teacher training in the past two decades reveals the absence of a clear, consistent policy and of a distinct educational policy that fosters multiculturalism. Therefore, establishing a supportive environment for multiculturalism is of great importance in teacher-training institutions (Alt and Raichel 2018). The current chapter underscores the benefits of reflective journaling

54

3 Reflective Journal Writing and Multicultural Diversity

for establishing and maintaining these supportive environments. Incorporating reflective writing in the learning process provides students with opportunities to ponder on cultural aspects and think of ways to connect the learning activities and contents to different cultures and points of view.

References Adie, L., & Tangen, D. (2015). The use of multimodal technologies to enhance reflective writing in teacher education. In M. E. Ryan (Ed.), Teaching reflective learning in higher education (pp. 127–138). New York and London: Springer. Alt, D., & Raichel, N. (2018). Lifelong citizenship: Lifelong learning as a lever for moral and democratic values. Leiden and Boston: Brill and Sense Publishers. Alt, D., & Raichel, N. (2020a). Reflective journaling and metacognitive awareness: Insights from a longitudinal study in higher education. Reflective Practice, 21, 145–158. Alt, D., & Raichel, N. (2021). Reflective journal writing: Scheme and scale preliminary validation by using Facet Theory with Smallest Space Analysis. Paper accepted by the 17th International Facet Theory Conference (July, 2021). https://www.action-m.com/ft2020/. Andrade, H. & Du, Y. (2007). Student responses to criteria-referenced self-Assessment. Assessment and Evaluation in Higher Education, 32(2), 159–181. Archer, M. (2007). Making our way through the world: Human reflexivity and social mobility. Cambridge: Cambridge University Press. Assessment tools for higher education learning environments [ASSET]. (2020). Erasmus+ Capacity Building in the Field of Higher Education. Erasmus+ programme of the European Union. https:// www.asset-erasmus.com/. Bailes, C. N., Hulsebosch, P., & Martin, D. S. (2010). Reflective journal writing: Deaf pre-service teachers with hearing children. Teacher Education and Special Education, 33, 234–247. Biggs, J. B., Kember, D., & Leung, D. Y. P. (2001). The revised two factor study process questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71, 133–149. Boud, D., Keogh, R., & Walker, D. (1985). Reflection: Turning experience into learning. London: Kogan Page. Brown, T., McCracken, M., & O’Kane, P. (2011) ‘Don’t forget to write’: how reflective learning journals can help to facilitate, assess and evaluate training transfer. Human Resource Development International, 14, 465–481. Cowan, J. (2014). Noteworthy matters for attention in reflective journal writing. Active Learning in Higher Education, 15, 53–64. Degago, A. T. (2007). Using reflective journals to enhance impoverished practicum placements: A case in teacher education in Ethiopia. Teaching Education, 18, 343–356. Desautel, D. (2009). Becoming a thinking thinker: Metacognition, self-reflection, and classroom practice. The Teachers College Record, 111, 1997–2020. Dyment, J. E., & O’Connell, T. S. (2011). Assessing the quality of reflection in student journals: A review of the research. Teaching in Higher Education, 16, 81–97. English, M. C., & Kitsantas, A. (2013). Supporting student self-regulated learning in problemand project-based learning. Interdisciplinary Journal of E-Learning and Learning Objects, 7(2), 128–150. Fabriz, S., Ewijk, C. D. Van, Poarch, G., & Büttner, G. (2014). Fostering self-monitoring of university students by means of a standardized learning journal – A longitudinal study with process analyses. European Journal of Psychology of Education, 29(2), 239–255. Griggs, V., Holden, R., Lawless, A., & Rae, J. (2018.) From reflective learning to reflective practice: assessing transfer. Studies in Higher Education, 43, 1172–1183.

References

55

Knapp, N. F. (2012). Reflective journals: making constructive use of the “apprenticeship of observation” in preservice teacher education. Teaching Education, 23, 323–340. Kolb, D. A. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice Hall. Kyles, C. R. & Olafson, L. (2008). Uncovering preservice teachers’ beliefs about diversity through reflective writing. Urban Education, 43, 500–518. Learning to Learn Project. (2002). Reflective Learning Journal (Teacher Guide). The Hong Kong Polytechnic University. Retrieved from https://www.polyu.edu.hk/learn-to-learn/. Lobato, J. (2012). The actor-oriented transfer perspective and its contributions to educational research and practice. Educational Psychologist, 47, 232–247. Marton, F. (2006). Sameness and differences in transfer. The Journal of the Learning Sciences, 15, 501–538. Miller-Dyce, C., & Owusu-Ansah, A. (2016). Yes, we are still talking about diversity: Diversity education as a catalyst for transformative, culturally relevant, and reflective preservice teacher practices. Journal of Transformative Education, 14, 327–354. Minott, M. A. (2008). Valli’s typology of reflection and the analysis of pre-service teachers’ reflective journals. Australian Journal of Teacher Education, 33(5), 55–65. Moloney, R., & Oguro, S. (2015) The effect of intercultural narrative reflection in shaping pre-service teachers’ future practice. Reflective Practice, 16, 96–108. Moon, J. (2003). Learning journals and logs, reflective diaries. Exeter UK: University of Exeter Center for Teaching and Learning. Moon, J. (2006). Learning journals: A handbook for reflective practice and professional development (Second ed.). London and New York: Routledge. O’Farrell, C. (2017). Assessment for lifelong learning. Dublin: Academic Practice, University of Dublin, Trinity College. Pack, M. (2014). Practice journeys: using online reflective journals in social work fieldwork education. Reflective Practice, 15, 404–412. Peltier, J. W., Hay, A., Drago, W. (2005). The Reflective Learning Continuum: Reflecting on Reflection. Journal of Marketing Education, 27, 250–263. Rogers, J., Peecksen, S., Douglas, M., & Simmons, M. (2019). Validation of a reflection rubric for higher education. Reflective Practice. https://doi.org/10.1080/14623943.2019.1676712. Rushton, D., & Duggan, C. (2013). Impact of culture on reflective writing in masters level students. Procedia - Social and Behavioral Sciences, 93, 956–963. Ryan, M. E. (Ed.). (2015). Teaching reflective learning in higher education. New York and London: Springer. Schön, D. A. (1983). The reflective practitioner – how professionals think in action. New York: Basic Books. Standal, Ø. F., & Moe, V. F. (2013). Reflective practice in physical education and physical education teacher education: A review of the literature since 1995. Quest, 65, 220–240. Thorpe, K. (2004). Reflective learning journals: From concept to practice. Reflective Practice, 5, 328–343. Waggoner-Denton, A. (2018). The use of a reflective learning journal in an introductory statistics course. Psychology Learning & Teaching, 17(1), 84–93. Wallin, P., & Adawi, T. (2018). The reflective diary as a method for the formative assessment of self-regulated learning. European Journal of Engineering Education, 43, 507–521.

Chapter 4

Using Digital Badges in a Multicultural Classroom

Introduction A digital badge is an online display that acknowledges and validates the achievement of skills and competencies through explicit evidence (Blackburn et al. 2016). A digital badge has been defined as “an assessment and credentialing mechanism that is housed and managed online. Digital badges are designed to make visible and validate learning in both formal and informal settings and hold the potential to help transform where and how learning is valued” (The MacArthur Foundation 2017, para. 1). As an innovative form of educational certification, badges could be offered by teachers as assessment tools in higher education which might potentially increase learner motivation while providing high-quality feedback (Grant 2016). With the growing attention paid to academic underperformance among ethnic minority students, largely explained by a lower motivation for learning compared to their counterparts, this chapter illustrates the opportunities and challenges which this assessment tool possesses for enhancing students’ motivation for learning in multicultural classrooms and explores how digital badges can be used effectively and creatively in multicultural learning environments.

What Is a Digital Badge? Digital badges are visual representations of skills, experience, knowledge, accomplishments, interests or affiliations, which are available on virtual platforms. Digital badges store information in the form of metadata about the institution or company issuing them, the person receiving the badge, and the context in which the badge was awarded. One kind of process, for example, might be “the peer and expert review of artifacts of work showing what someone knows and can do, and the accompanying validation and credentialing of that person’s knowledge or capability” (Gibson et al. 2015, p. 404).

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_4

57

58

4 Using Digital Badges in a Multicultural Classroom

Fanfarelli and McDaniel (2019) distinguish between the term ‘digital badges’ and other relative terms, for example, ‘micro credentials.’ The latter implies an additional level of validation by a central authority such as the academic institution, whereas digital badges “can be used even to recognize work in progress that has not yet been centrally validated” (p. 11). Digital badges might be useful for motivating certain behaviors without requiring an “outside central authority’s blessing to do so” (p. 12). Others (Ippoliti 2018) suggest deeming digital badges as a specific form of micro credentialing which also requires some form of validation from an organization or entity verifying that the badge is authentic and meets certain criteria for quality. Another frequently used term is ‘open badges’ which use open-source technologies such as the Open Badge Infrastructure (Mozilla Foundation 2011). These technologies allow open badges to be shared between different systems and to manage the exchange and distribution of digital badges.

Digital Badges and Lifelong Learning Skills Digital badges can address some of the preponderant issues associated with highereducation learning outcomes such as meeting the demands of enhancing lifelong learning skills (Jacobson 2018). Lifelong learning skills refer to a range of diverse elements establishing close links to the identity of the learner, derived from multiple views about oneself, such as interactions and collaborations (Hickey and Willis 2015). For example, when individuals work together to complete assignments, they need to master certain social skills and competencies as they form personal as well as professional relationships (Johnson and Johnson 2014). Grant (2014) argues that if we seek to instill in the learners twenty-first century skills such as leadership, critical thinking, and problem solving, multiple-choice tests designed to assess these qualities might not be sufficient since at “no point during the test are learners asked to problem-solve with others or perhaps critique the questions” (p. 31). In contrast, badges can be designed to demonstrate various types of skills and may be used to support both formative and summative assessment. Assessment is usually comprised of learning outcomes and measures of the outcomes, and badges can be seen as digital representations of these learning outcomes. Mozilla provides a short list of lifelong learning skills, which includes hard skills (e.g., mastering a programming language) and soft skills (e.g., critical thinking and collaboration). This source provides a full list of skills and how assessment works for each one (Parker 2015). As an instrument of summative assessment, digital badges are used to mark the achievement of a certain level of proficiency upon the completion of a learning event. As an instrument of formative assessment digital badges are used to provide feedback and outline possible paths of progress in a further course of learning (Grant 2014).

Can Digital Badges Increase Motivation for Learning?

59

Can Digital Badges Increase Motivation for Learning? As badges might engage and motivate people in games, there is great interest in how this “game mechanic” element can enhance engagement and motivation in learning settings. Abramovich and Wardrip (2016) suggest a number of learning motivation theories that can inform a potential interaction between badges and learners’ motivation, such as the Achievement Goal Theory (Pintrich 2000). They suggest that motivation to earn badges can be linked to different learning goals, “learners could have a performance approach goal orientation and be motivated to earn more badges than their peers. Or have a performance avoidance goal orientation and want to earn enough badges to be similar to their peers” (p. 56). Therefore, educational badges have begun to gain traction among reformers and instructional designers. Similar to videogames, the issuers of educational badges can bestow a symbolic award for any type of skill, knowledge, or achievement in the form of a badge, that can subsequently be shown by the learner to inform others of their mastery or knowledge. Digital badges might also be useful as an assessment tool that stimulates learners to self-regulate their learning by providing challenging tasks, clear standards, affirmation of performance, novelty, and authenticity (Abramovich et al. 2013). Selfregulated learning (Butler and Winne 1995; Zimmerman and Schunk 2001) is defined as: an active, constructive process whereby learners set goals for their learning and then attempt to monitor, regulate and control their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment (Pintrich 2000, p. 453).

This process involves learners’ regulation not only of their cognition but also their behavior and motivation, according to their self-set goals. It refers to the degree to which learners can regulate aspects of their thinking, behavior and motivation (Pintrich and Zusho 2002). Self-regulated learners are individuals who are already assessing their own performance to varying degrees, while generating their own selfassessment actively and consciously controlling their own learning from cognitive, affective, motivational and behavioral points of view (Boekaerts et al. 2005). In this process, digital badges might be useful as an assessment tool that stimulates learners to take control of their learning. However, due to the preliminary stage of development and testing, empirical evidence of how digital badges might implement assessment strategies supporting the development of self-regulated learning is still quite limited. Studying the associations between digital badges and self-regulation could increase our possibilities for implementing strategies for assessment-promoting skills for lifelong learning (Cucchiara et al. 2014). Therefore, this chapter seeks to understand how digital badges can be used to motivate multicultural students’ self-regulation of learning.

60

4 Using Digital Badges in a Multicultural Classroom

Minority Students and Motivation for Learning Globally, the phenomenon of academic underperformance of ethnic minority students has drawn increased attention across disciplines. One of the factors recognized by researchers as being crucial in explaining this achievement gap is motivation for learning (Ryan and Deci 2000). A recent study (Isik et al. 2018) sought to establish a comprehensive overview of precursors that may affect motivation of ethnic minority students from their own point of view. A systematic review of 45 studies regarding the motivation of ethnic minority students was conducted. The authors found a wide range of (individual, family-related, school-related, and social) factors that influence the academic motivation of ethnic minority students. In a similar vein, Bembenutty (2007) showed, among other motivational factors, how college students’ self-regulatory strategies and academic performance varied as a function of their ethnicity (Caucasian vs. minority college students). The author advises educators to support their minority students by giving them opportunities to manage and regulate their time and their study environments. Young et al. (2011), in their study of undergraduates from three distinct ethnic groups, showed that socioeconomic status, family generations with college experience, and perceived social support were predictive of both intrinsic and extrinsic motivation for African Americans. Isik et al. (2018) assert that school-related and social factors can be influenced by interventions aimed at augmenting the motivation of ethnic minority students. They wrote, “Examples of measures that can be used to influence students’ motivation in a positive manner are supporting students’ self-efficacy and helping them to accept that they have to take control of their own learning, establish positive achievement goals and communicate well with educators and peers in the classroom” (p. 18). Other researchers (Gavala and Flett 2005; Zhao 2015) who investigated motivation among ethnic minority students suggest that teachers should change their traditional teaching methods and create suitable teaching contexts to satisfy student’s needs and that a comfortable academic environment should be provided to minority students. Such a culturally compatible environment might increase perceived psychological well-being and academic enjoyment and motivation. Notably, previous studies have pointed to a disparity between minority and majority students regarding their motivation for learning. However, their suggestions merely provide general instructional approaches to enhance motivation for learning, such as, “educators could help all of their students by teaching them how to use cognitive strategies. The teacher’s role in the classroom is not just teaching the content area, but also promoting active learning” (Bembenutty 2007, p. 611). In the context of this chapter, and with the growing attention being devoted to digital badges, it seems worthwhile to explore how digital badges can be used to increase minority students’ engagement and motivation to perform learning activities.

Digital Badges—Limitations and Challenges

61

Digital Badges—Limitations and Challenges Despite mounting interest in and writing about the merits of digital badge usages, this assessment method has not been spared of criticism. Among its detractors are some researchers of rewards and gamification who have raised concerns, namely that such incentives may undermine individuals’ intrinsic motivation and interest in the activity by focusing on the acquisition of incentives and losing interest in the material itself. As explained by Fanfarelli and McDaniel (2019, p. 59), When badges are used as rewards similar to praise, stickers, grades, or points, they are serving as positive reinforcers that increase extrinsic motivation… However, specific badge designs that help users feel more competent, autonomous, or socially successful can also impact users’ intrinsic motivation. An example is a badge that helps users improve their abilities through progressive goal setting.

In a similar line of thought, Finkelstein et al. (2013) suggest that digital badges in and of themselves should not be seen as the source of motivation for many learners and should not be promoted as such, There is a very real risk of alienating otherwise well-intentioned, self-motivated learners by holding a badge in front of them as the reason to pursue a learning opportunity. Of course, every case is different, but it is fair to say that most experiences that will result in badges should not be positioned first and foremost as “badging programs” but rather as programs that happen to offer badges (pp. 9–10).

Although, badges are used to positively reinforce and stimulate students’ behavior and self-efficacy by measuring their learning progress and learning feedback, researchers suggest that the effects of educational badges vary among different ability learners (Abramovich et al. 2013). Badge earning could be driven by learner motivations and therefore have a positive effect on critical learner motivations. Hanus and Fox (2015), for example, found that students in the gamified course showed less motivation, over time, than those in the non-gamified class. They also suggest that caution should be taken when applying certain gamification mechanics to educational settings, Offering tangible, expected rewards to individuals who are already interested in a topic may cause them to shift motivations from intrinsic (i.e., because they wanted to) to extrinsic (i.e., because they want to earn a reward). When the reward is present, one may be interested in completing the task, but once the reward is removed one will no longer have a reason to perform a behavior (p. 154).

Other studies (e.g., Reid et al. 2015) linked the effectiveness of digital badges to the motivation and the expectancy-value of the learner. More specifically, digital badges proved to be more effective for learners with higher levels of expectancy-value motivational goals compared to those who placed little value on the learning tasks. The authors conclude that digital badges cannot be perceived as a ‘magic bullet’ for learning in all environments, “Rather, badges should be utilized as a way to measure learning progress in conjunction with many other instructional tools and strategies. Digital badges should function as recognizers of the learning that already has taken place rather than affirmations or rewards” (p. 393).

62

4 Using Digital Badges in a Multicultural Classroom

To tackle these limitations and motivate adult learners, by offering them digital badge-yielding learning opportunities, it is necessary to underscore the benefits of earning digital credentials. For example, digital badges can illuminate the steps on the pathway to achieving learning and life goals by providing “natural representations for achievable milestones that can help learners see that the big picture is possible if they can achieve the needed steps along the way” (Finkelstein et al. 2013, p. 20). Digital badges can offer credentials for skills not currently certified; for example, they can play a role in the recognition of prior learning and can acknowledge progressive levels of achievement across multiple venues and institutions. With the scant research done thus far, studying how digital badges may be perceived by students of diverse ethnic backgrounds, in terms of motivation for learning, could increase our possibilities for implementing various assessment strategies and promoting motivational learning goals. This objective is manifested in this book, in that it involves a deep understanding of whether or not this form of evaluation is efficient in the context of multicultural groups. We wish to learn the advantages and disadvantages of digital badges for these groups and how they can be better suited to students of different cultures. The intervention program described herein was implemented for this purpose and its impacts on students in a heterogeneous classroom were examined.

The Intervention Procedure The intervention was implemented as part of an EU project (Assessment tools for higher education learning environments [ASSET] 2020; Alt 2021) aimed at promoting formative assessment methods in higher education. It was conducted as part of a one-semester course of teaching methods and was collaboratively led by the course lecturer and a teaching assistant. To create a badge-based intervention plan, course learning outcomes were mapped. Next, a group assignment was developed that included four micro-assignments, each relating to a specific skill, such as information literacy and creativity. Badges that align with each learning outcome were developed, as described in Table 4.1. For each of the four group micro-assignments it was specified how students were expected to achieve the outcome, how they would be assessed (criteria were designed, shared with the students, and revised where needed), and would ultimately earn the badges. In-class guidance was provided to support students’ success in each assignment. Despite being a group assignment, the badges were awarded to each student individually, as each one was required to describe the part s/he had taken in the assignment; for example, which papers were analyzed by the student and what part of the educational activity the student had developed. In the plenary session, each student’s contribution to the presentation was observed and assessed by the teacher. This pilot study utilized a Credly account to create and distribute badges. Students’ progress and achievements were tracked. After carefully reviewing the students’ work

The Intervention Procedure

63

Table 4.1 Digital badge description (Alt 2021) Information literacy: earners of this badge have successfully gathered, evaluated and used information to describe two twenty-first century skills

Creating information: earners of this badge have created an educational activity plan based on predefined criteria aimed at enhancing a twenty-first century skill among their pupils

Creativity: earners of this badge have presented their planned activity in the plenary in a creative and active manner

Reflective thinking: earners of this badge have submitted reflective journal entries during the course and showed in-depth reflective thinking regarding their experiences during the assignment, while addressing the following core dimensions: awareness, evaluation, and regulation

by both the lecturer and the teaching assistant, detailed feedback was provided, and finally the badges were awarded. As illustrated in Table 4.1, each student could earn one badge per microassignment. However, the level of achievement was reflected in the number of stars indicated on the badge, whereby a student could earn up to three stars (indicating that all the criteria for the assignment had been met). In total, 208 badges were awarded during the intervention.

64

4 Using Digital Badges in a Multicultural Classroom

Students’ Perspectives on Digital Badges in a Diverse Learning Environment Eighteen undergraduate college students were interviewed to examine and identify their perceptions regarding the use of digital badges in the assessment process as well as its advantages and disadvantages. Twelve of the students were Arabs (Muslims, Christians, Druze, and Bedouin) and six were Jews. All were students in the course described above. The analysis process involved examining the gathered data and generating themes from the data (Creswell 2014). Participants were quoted extensively to support the themes. The analysis revealed four main categories, which are presented below: (1) motivation for learning (who benefits the most from digital badges?); (2) self-regulated learning; (3) digital badges and competition; and (4) openness to change and challenge.

Motivation for Learning: Who Benefits the Most from Digital Badges? Analysis of the interviews reveals that differences exist between high-achieving students in the course and low achievers (mainly Arab students) vis-à-vis their motivation to learn as a consequence of using digital badges. The badges increased the low achievers’ motivation to learn. Nassrin and Shirin (Christian females) respectively described their progress and the efforts it required as a highly emotional process characterized by extreme frustration, alternating between despair and hope, and between a painful sense of “failure” and motivation for “success”: After I received my first badge, I felt very frustrated… we [our group] received only one star, which meant that the assignment wasn’t very good. All of us felt disappointed and wondered how to continue because all we had up until now were very few stars. We continued on to the next assignment, hoping that we would receive three stars. When we received two, our confidence in our ability increased. The first and second assignment helped us become stronger in the third assignment. After we received one star in the first assignment, I thought about the end of the course and about my future. How would I continue? What would I do about my future if I received so few stars for the assignments? It is a failure for me. I began to think about what I needed to do and what I needed to improve in order to complete the rest of the assignments well, and how I could achieve the goals that were demanded of me in the work.

Students who attained high achievements in the course (both Jewish and Arab students) also reported that badges increased their motivation to learn. However, it appears that these students derived the most benefit from the feedback they received by implementing a suitable work plan which they described during their interviews. Hence, they channeled their “negative feelings” into more meaningful learning in order to succeed in the next assignments. This was described by Rana (Muslim female):

Motivation for Learning: Who Benefits the Most from Digital Badges?

65

It’s true that we were disappointed at first because we received a badge with less than three stars for the first assignment. But afterwards, we felt motivated to succeed more in the next assignments. First of all, we knew what our mistakes were. We sat together and examined what we had done right and what we had done wrong and how we could improve and work better to receive more stars in the future. We could then avoid the negative feeling we had at the beginning. Each of the members of the group had an ambition to succeed, and there was a great desire to succeed in the next assignments. Sure enough - we received a badge with three stars on assignment number 2 because of our motivation and desire to succeed.

However, most of these students claimed that they had high motivation to learn anyway, and the badges did not serve as an incentive to increase their motivation. Nir (Jewish male) explained: I didn’t feel that I had higher or lower motivation to succeed just because of the badge system. I felt the same amount of drive to get the best grade as possible even with the regular grading system. I always strive to achieve the maximum, and it doesn’t matter whether I receive three stars or a hundred. From my standpoint, you have to complete every assignment in the best way possible and it doesn’t matter whether the grade is a badge or a number.

Self-regulated Learning It appears that for Arab minority students, the use of digital badges aided in a variety of aspects that relate to both individual and group learning. At the individual level, the use of digital badges led to increased awareness of the importance of reading the guidelines before submitting the assignment and strengthening the communication with the lecturer during the assignment. This was explained by Yara (Christian female): I learned about my learning process from the feedback I received - about things that I will want to apply in my learning in the future to strengthen weak points in my learning. In order to do this, I’ll need to ask for help from the lecturer to learn the material before I begin the assignment. I’ll use the lecturer’s guidelines. I’ll read them and understand them well. If I don’t understand something in particular, I’ll ask the lecturer or my friends. I’ll go to the library and look for sources about the assignment. Then I’ll finish the assignment according to the lecturer’s guidelines. After I finish, I’ll send the lecturer an email and ask her to meet to talk with me about the assignment. After she reads my work and tells me that everything is correct, I’ll know that I’m headed in the right direction.

Seven of the Arab interviewees noted how the assessment criteria and feedback helped them better understand how to optimally organize studying for and preparing the assignments during group work. Narmin and Ahmed (Muslim female and male students) respectively described how evaluation using badges influenced their work as a team, as well as the understanding that they had to assume more responsibility in the group despite the language barrier: The badges helped us organize our work in the group better than our work on other assignments. Earning the badge entailed criteria. For example, teamwork – describing each team member’s contribution in detail, searching for academic articles, films, searching for information, cooperation, and setting schedules. We were asked to describe how we conducted our

66

4 Using Digital Badges in a Multicultural Classroom meetings together. All in all, the badge evaluates us as a group and evaluates each person’s work, and so all of us had to work. Every person influenced everyone else, and we learned to work together from one assignment to the next. I have to assume a lot of responsibility towards the group because of the language. Like me, Arabic is the mother tongue for most members of the group. This assignment required a higher level of academic language than in other courses.

Unlike the Arab minority students who attended more to the group work dynamics, their Jewish counterparts reported that the badges made them think about their learning and how to make it more efficient at the personal, individual level. For example, Yael (Jewish female): I learned about my learning process. I need to make an effort from the beginning, from the first assignment and not wait until I receive a badge to succeed on other assignments.

With regard to the Jewish students’ group level and the possible contribution of badges towards increasing the efficiency of group learning, it appears from their feedback that the use of badges did not contribute to developing skills in group work as noted by four students. For example, Rachel (Jewish female) stated, “Our group cooperated in meetings and in the WhatsApp group for the purpose of sharing and dealing with questions about the topic. From that standpoint, the badge did not improve the organization of studying together in the group in any way.”

Digital Badges and Competition A predominant intercultural difference arose with respect to increased competition between the groups in the class associated with the badge system. Ten of the Arab interviewees noted that showing the badges increased motivation because it created an element of competition, as explained by Muna (Muslim female): “The badges increased our motivation, challenge, and competition between the groups. This gave us the incentive to get more stars. There was motivation in my group, especially after we received very few stars in the first assignment. We wanted to prove that we are capable of doing better.” Similarly, Sabrin and Yasmin (Bedouin and Christian females) explained, respectively: When I received the first badge, I was very disappointed because it had only one star. That meant that our work wasn’t very good. But it gave us an incentive to improve our ability and to complete the next assignment better and at a higher level so as to get more stars. I think there was competition in the class between the groups and each group strived to get more stars. We viewed this as a competition between the other groups to see who could collect the most stars at the end of the assignment. The badge system is more competitive than giving grades.

Four of the Arab students even attributed negative aspects to the competitiveness, particularly when the badge did not award the highest evaluation level of three stars. As Nadin (Muslim female) explained, “The exposure was liable to lead to feelings

Digital Badges and Competition

67

of embarrassment. What I didn’t like was that the entire class could see my badge because it’s embarrassing to get a badge that isn’t very good.”

Openness to Change and Challenge Interviews with students of all cultures revealed that this method of assessment was new to them. None of the students had been exposed to this method before. “It’s amazing for me because this is the first time I received a badge instead of a grade” (Manar, Muslim female). This method was unfamiliar, “In our educational system in the Arab sector our evaluation of assignments and tests is according to grades and not badges, because that’s the way it is in school. There are no badges on our report cards or on matriculation exams. That’s the way things are” (Ahmed, Muslim male). However, discrepancies between the Jewish and the Arab students was shown with regard to their perception of the objectives of using digital badges and the need for their use in the academic world. The Jewish students addressed the lifelong learning skills that could be obtained and promoted by implementing this assessment approach. Sigal and Yael (Jewish females), respectively, endorsed the advantages of the use of digital badges for the enhancement of “abilities that we need to acquire in order to become integrated into the work market and life and to succeed”, The use of digital badges symbolizes that the lecturer examines different parameters that were not examined in other courses. I can therefore understand the need for awarding badges. They offered me the possibility of understanding which skills I need to improve. Additional skills were examined in this course that had not been expressed previously in learning or in student evaluation…When I received a badge, I knew that I was being examined for a particular skill that can’t be expressed in papers or exams that assessed only my ability to think without relating to my other abilities. It’s important that students also acquire these skills during their studies in the academy. It’s not enough to know how to study just to get good grades. I had a better feeling when other parameters were examined during our evaluations that are important for us to acquire during our studies. The idea is that students are not only evaluated according to grades they receive on tests, for which they learn and memorize material, but also according to several other skills that aren’t expressed in “regular” learning. I was pleased to know that other indexes are being examined, in addition to how well we memorize material, such as teamwork, cooperation, creativity, and the ability to get up in front of an audience. The assignment was a challenge that I wasn’t accustomed to previously.

Six Arab students also voiced positive views about the use of the badges, but their rationale was based upon psychological aspects, such as reduced tension and fear that are connected to the system rather than aspects connected to improving learning skills for which badges are awarded. For instance, Yara (Christian female) said, “I like the badge system because it’s not frightening like grades are”, and similarly Rana (Muslim female) said, We students are used to receiving grades in our courses, because none of the [other] lecturers use badges. They all use grades. After experiencing the badge method, I think that lecturers

68

4 Using Digital Badges in a Multicultural Classroom should make a change and use badges or another method instead of grades, because the badge system made us more relaxed.

Paradoxically, despite the advantages mentioned by the Jewish students, five of the interviewees in this group also expressed a preference for the “old familiar” method of evaluation. They recognized that the badge system can examine a variety of skills but reported that they preferred the “safe” prevailing environment of grades. The main reason for this is that they have been familiar with the “old” method for many years. As Ido (Jewish male) explained, “I would prefer to remain with what is familiar. It’s more comfortable for me. I could receive a separate grade for each assignment that could reflect just as well as the system of stars can…. Despite the fact that our minds are flexible, it still takes time for students like me, who have been accustomed to a different method of evaluation throughout our lives, to adjust and try a new system. The new method doesn’t make things better”. In contrast, although this method was also innovative and unfamiliar for the Arab students, they expressed an openness towards it, as Yasser (Muslim male) explained, “I would prefer to receive a grade because we are more used to grades than badges. They are clearer. But after I got to know the badge system, I prefer it over grades because they can encourage each student to work more seriously to attain the best outcome.”

Discussion The theory presented in this chapter shows how badges may attest to students’ learning ability more broadly than a traditional degree or certification while also meeting learners’ needs. The array of badging systems suggests new ways of selecting what we “want to count, what we want to value and acknowledge and credit and reward” (Davidson 2011). A badge system can be a visible proof of the quality of participation and contribution that were previously unrecognized. In contrast to the traditional diploma, grades and test scores that are the culmination of a formal educational system, digital badges might raise new possibilities, such as the validation of skills, learning experiences, and knowledge by a wide array of badge issuers in informal and formal education alike (Gibson et al. 2016). By virtue of its broad technical possibilities, the digital badge has given rise to a global discussion on educational practices and opportunities centered on evidencebased learning and assessment. Analysis of the students’ interviews points to a number of issues that might inform such discussions on the implementation of digital badges in the context of ethnic and cultural diversity. First, it is evident that for weak students, badges serve as an incentive for learning while only partially serving as a motivating factor for academically strong students. Indeed, high achievers stated that they would persevere in their learning and devote efforts to attaining maximum achievements, regardless of the system for evaluation and its objectives. It appears,

Discussion

69

therefore, that the digital badge is most effective for students who are in need of continuous feedback that reflects their progress and helps them attain achievements step by step. It is also evident that the use of badges benefits the regulation of learning for a wide range of students at both the individual and group levels. While the tasks in the intervention program described herein took place in groups, each member was asked to describe the group work process and explicitly specify each student’s contribution to the overall group effort. This aspect constituted one of the main evaluation criteria, and the teacher’s feedback contained points for improvement of the group’s work. The students made use of the preliminary feedback offered by the teacher, together with the digital badge, to improve the learning process in subsequent assignments. Another point that arose was the competition that was spurred by publicly exposing the badges earned by others. This aspect appeared mainly among the Arab students. Some students reported that the competition helped motivate them to show their peers what they are capable of. For others, on the other hand, the public visibility of the badges could potentially result in detrimental emotional effects as shame or embarrassment when the student’s badge does not exemplify a high degree of success. Finally, it appears that the digital badge system was new to all the students without exception. Jewish students noted the advantages of the system for evaluating skills that are not currently assessed in the academic world, whereas Arab students related to its psychological and emotional merits such as reducing the fear and tension that they associate with traditional evaluation methods. The latter were more open to the new method than their Jewish peers, who noted its advantages but still preferred the traditional methods of evaluation.

Implications and Recommendations In heterogeneous classes attended by students with varying abilities in both independent and group learning, it is advisable to employ a method such as digital badges to provide feedback on the personal and/or group learning process and how to improve it. In other words, apart from establishing criteria that center on a specific skill signified by a particular badge, such as information literacy, for instance, it is also advisable to attend to the students’ overall learning process, including their difficulties, their contribution to group work, and their strong and weak points, together with citing ways to make learning more efficient. It is also important to display sensitivity in matters connected to the public aspect of digital badges. In our opinion, it is advisable to hold a discussion with the students before introducing the digital badge system. These discussions can clarify the merits and demerits of displaying the badges publicly. We must bear in mind that while displaying badges in public can foster healthy competition, it might also potentially elicit negative feelings that are liable to impede the learning process. It should be noted that alongside the pedagogical theory in which this topic is grounded, and against the background of the scant empirical research that has been

70

4 Using Digital Badges in a Multicultural Classroom

conducted to date, it is advisable to conduct additional research on how students of different cultural traditions and needs perceive this method of evaluation. As it relates to the topic of this book, it is important to ascertain how to adapt this method to a wide range of students from different countries and cultures. It should be noted that this chapter examined only one intervention program which was implemented within the framework of a one-semester undergraduate college course. This is not sufficient for us to draw far-reaching conclusions. Future research will be needed to shed more light on the insights that have been presented in this chapter.

References Abramovich, S., & Wardrip, P. (2016). Impact of badges on motivation to learn. In L. Y. Muilenburg, & Z. L. Berge (Eds.), Digital badges in education (pp. 53–61). NY: Routledge. Abramovich, S., Schunn, C., & Higashi, R. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61, 217–232. Alt, D. (2021). Who benefits from digital badges? Motivational precursors of digital badge usages in higher education. Current Psychology. https://doi.org/10.1007/s12144-021-02002-0. Assessment tools for higher education learning environments [ASSET]. (2020). Erasmus+ Capacity Building in the Field of Higher Education. Erasmus+ programme of the European Union. https:// www.asset-erasmus.com/. Bembenutty, H. (2007). Self-regulation of learning and academic delay of gratification: Gender and ethnic differences among college students. Journal of Advanced Academics, 18, 586–616. Blackburn, R., Porto, S., & Thompson, J. (2016). Competency-based education and the relationship to digital badge. In L.Y. Muilenburg, & Z. L. Berge (Eds.), Digital badges in education: Trends, issues and cases (pp. 30–38). NY: Routledge. Boekaerts, M., Pintrich, P., & Zeidner, M. (2005). Handbook of self-regulation. Elsevier, San Diego. Butler, D. L., & Winne, P. H. (1995) Feedback and self-regulated learning: a theoretical synthesis. Review of Educational Research, 65, 245–281. Creswell, J. W. (2014). Research design. Qualitative, quantitative, & mixed method approaches (4th ed.). Los Angeles: Sage Publications. Cucchiara, S., Giglio, A., Persico, D., & Raffaghelli, J. E. (2014). Supporting self-regulated learning through digital badges: A case study. In Y. Cao, T. Väljataga, J. K. T. Tang, H. Leung, & M. Laanpere (Eds.), Proceedings from the international conference on web-based learning (133– 142). Tallinn: Springer. Davidson C. N. (2011). Could badges for lifelong learning be our tipping point? Retrieved November 20, 2019, from https://www.hastac.org/blogs/cathy-davidson/2011/11/14/could-badges-lifelonglearning-be-our-tipping-point. Fanfarelli, J. R., & McDaniel, R. (2019). Designing effective digital badges. Application for learning. NY: Routledge. Finkelstein, J., Knight, E., & Manning, S. (2013). The potential and value of using digital badges for adult learners. Final report. Washington, DC: American Institutes for Research. Gavala, J. R., & Flett, R. (2005). Influential factors moderating academic enjoyment/motivation and psychological well-being for Maori university students at Massey University. New Zealand Journal of Psychology, 34, 52–57. Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2015). Digital badges in education. Education and Information Technologies, 20, 403–410.

References

71

Gibson, D., Coleman, K., & Irving L. (2016). Learning journeys in higher education: Designing digital pathways badges for learning, motivation and assessment. In D. Ifenthaler, N. BellinMularski, & M. Dana-Kristin (Eds.), Foundation of digital badges and micro-credentials. Demonstrating and recognizing knowledge and competencies (pp. 115–138). Switzerland: Springer. Grant, S. L. (2014). What counts as learning: Open digital badges for new opportunities. Irvine, CA: Digital Media and Learning Research Hub. Grant, S. L. (2016). History and context of open digital badges. In L.Y. Muilenburg, & Z. L. Berge (Eds), Digital badges in education: Trends, issues, and cases (pp. 3–11). New York, Routledge. Hanus, M., & Fox, J. (2015). Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Computers & Education, 80, 152–161. Hickey, D., & Willis, J. E. (2015). Research designs for studying individual and collaborative learning with digital badges. In Proceedings of the Open Badges in Education (OBIE 2015) Workshop, Poughkeepsie, New York, USA. Ippoliti, C. (2018). History of mico-credentialing. In K. L. O’brien, & T. E. Jacobson (Eds.), Teaching with digital badges (pp. 3–14). London: Rowman & Littlefield. Isik, U., El Tahir, O., Meeter, M., Heymans, M. W., Jansma, E. P., Croiset, G., & Kusurkar, E. A. (2018). Factors influencing academic motivation of ethnic minority students: A review. SAGE Open, 1–23. Jacobson, T. E. (2018). Forces of change for higher education: Opening gates for digital badging. In K. L. O’brien, & T. E. Jacobson (Eds.), Teaching with digital badges (pp. 15–30). London: Rowman & Littlefield. Johnson, D. W., & Johnson, R. T. (2014). Cooperative learning in 21st century. Anales de Psicología, 30, 841–851. MacArthur Foundation. (2017). Digital badges. Retrieved from https://www.macfound.org/pro grams/digital-badges/. Mozilla Foundation. (2011). Mozilla launches open badges project. Mozilla Blog. Retrieved from https://blog.mozilla.org/blog/2011/09/15/openbadges/. Parker, H. E. (2015). Digital badges as effective assessment tools. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Pintrich, P. R. (2000) The role of goal orientation in self-regulated learning. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds), Handbook of self-regulation (pp. 451–502). San Diego: Academic Press. Pintrich, P. R., & Zusho, A. (2002). The development of academic self-regulation: the role of cognitive and motivational factors. in A. Wigfield, & J. S. Eccles (Eds.), Development of Achievement Motivation (pp. 249–284). San Diego, CA: Academic Press. Reid, A. J., Paster, D., & Abramovich, S. (2015). Digital badges in undergraduate composition courses: Effects on intrinsic motivation. Journal of Computers in Education, 2(4), 377–398. Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68–78. Young, A., Johnson, G., Hawthorne, M., & Pugh, J. (2011). Cultural predictors of academic motivation and achievement: A self-deterministic approach. College Student Journal, 45, 151–163. Zhao, L. (2015). Teaching strategies of enhancing motivation among ethnic minority students. Theory and Practice in Language Studies, 5, 2176–2182. Zimmerman, B. J., & Schunk, D. H. (2001). Self-regulated learning and academic achievement: Theoretical perspectives. NJ: Laurence Erlbaum.

Chapter 5

Using Digital Concept Maps as an Assessment Tool in a Culturally Diverse Learning Environment

Introduction The idea of a concept map was first developed by Novak and Gowin (1984) who sought to illustrate the main concepts involved in a study by using a diagram. The idea is based on the constructivist approach to learning, which highlights the active role played by learners in constructing and developing knowledge and extracting meanings from their prior knowledge, experiences, and beliefs. Concept maps have been applied in education systems for more than 30 years (Kinchin 2014); however, applying them to formative assessment to support student learning has been less commonly practiced in the classroom. In view of the growing attention being devoted to diversity of student characteristics and learning needs, this chapter elucidates how concept mapping can be used as an effective instructional scaffold to support culturally diverse students’ learning processes. It also explores how these students design concept maps, use well-defined criteria for self-assessing the maps; it additionally ascertains their views on the use of concept maps as an assessment tool that measures and supports meaningful learning.

What Is a Concept Map in Teaching? A concept map is a visual representation of knowledge. It can be used to organize and structure information within a particular domain. This may be done in a wholly graphical manner to highlight differing concepts and their linkages or by identifying key concepts by names or titles and enclosing them in visual boxes, and then providing connecting navigation to lesser concepts (Jennings 2012). Concept maps can be incorporated in introductory units, midterm reviews and assessments, peer assessment, self-assessment, and/or end-of-course reviews and assessments (Croasdell et al. 2003). In contrast to lecture-based teaching methods, the concept map is a student-centered method which has been demonstrated repeatedly to have a positive © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_5

73

74

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

impact on the quality of student learning. It does so by improving students’ thinking and decision-making skills, abstract reasoning, critical thinking skills, meaningful learning levels and creativity much more than the traditional lecturing method (Chan 2017). Erickson et al. (2017) argue that teachers who use concept mapping need to engage students in inquiry and identify conceptual lenses as well as craft quality generalizations. Campbell (2016) offers a step-by-step plan for teaching Novak’s (1998) concept mapping in elementary, middle, and high school. Campbell cautions that concept mapping should begin only after an instruction process where the students will have been taught how to build a concept map by, for example, showing them several samples of concept maps and discussing the elements that make a great concept map. Campbell adds that at the final stage of the introduction, she creates anticipation by asking the students if they would like to take a picture of inside their brain. I then describe concept maps as an ‘Instagram’ of a student’s brain about a particular subject. Just like an Instagram portrays a visual and a photo filtered be added to create a tone, so too are their thoughts, their interrelationships, and their experiences evidenced on their concept map. Life experiences often add the ‘tone’ that makes the map unique from other maps (p. 76).

Concept Maps for Constructivist Learning Concept maps afford a way to gain twenty-first century skills identified as critical thinking and problem solving (Partnership for 21st Century Skills 2014). Students need to know how to frame, analyze, and synthesize information so as to solve problems and answer questions. In higher education settings, recent studies, mainly in medical and nursing education, have concluded that concept mapping, when used for problem solving, can provide learners with better and deeper learning instead of the mere surface learning exemplified by traditional teaching methods. For example, Chan (2017) explored the manner in which students develop concept maps and their views on the use of concept maps as a learning activity in a problem-based learning class. Specifically, she investigated the issue of whether and how innovative concept maps, supplemented with art elements, improves the creative thinking abilities of students. The study was conducted in a school of nursing in a university in Hong Kong. The students were asked to draw two concept maps based on a given scenario and then to participate in a follow-up interview. The author demonstrated how concept maps could be implemented in a problem-based learning class to enhance the students’ creativity, active participation, and motivation to learn, and thus should be considered as an alternative way to facilitate student learning. Hung and Lin (2015) examined the patterns of concepts and differences in the knowledge structures of students taught with and without a problem-based learning approach among occupational therapy undergraduates. A descriptive analysis of the morphology of the concept maps yielded three categories: isolated, departmental, and integrated. The students in the control group constructed more isolated maps, while the students in the problem-based learning group tended toward integrated

Concept Maps for Constructivist Learning

75

mapping. The authors concluded that problem-based learning had a strong effect on the acquisition and integration of knowledge. The important properties of problembased learning, including small-group interactions, can help students acquire more concepts, achieve an integrated knowledge structure, and enhance clinical reasoning. These studies (Chan 2017; Hung and Lin 2015) show the superiority of constructivist learning environment, over traditional methods of teaching, utilized with concept mapping in medical education. This approach can help students engage in integrated concept mapping and achieve a more integrated knowledge structure during a problem-solving process. “Students connect descriptive knowledge with procedural knowledge and create more details and cross linkages in their knowledge structures, which will benefit clinical reasoning in the future” (Hung and Lin 2015, p. 8). The authors advise educators to enhance their students’ knowledge structures by incorporating problem-based learning and concept mapping in the curriculum. Drawing on social constructivism (Vygotsky 1978), according to which knowledge is constructed through social interactions and is developed collaboratively, increasing research has been devoted to investigating the effectiveness of collaborative concept mapping (Cicognani 2000; Kwon and Cifuentes 2009). Using collaborative concept mapping, students co-construct their common knowledge by collaboratively negotiating and developing the relationships they write on their maps. Lin et al. (2016) investigated the effectiveness of using Google Docs in collaborative concept mapping, in contrast to a traditional paper-and-pencil approach, in enhancing middleschool students’ learning achievements, physics concept representation, attitudes toward science, and attitudes toward collaborative concept mapping. The results suggested that while the use of Google Docs did not significantly affect physics achievement, it did foster physics concept representation and more positive attitudes toward science, and collaborative learning.

Concept Maps in Social Studies Education While there is a considerable corpus of literature on concept maps in healthcare education, scant research has been conducted in the field of social studies education. For example, drawing on prominent theories of attitude structure, Keating and Bergan (2017) showed how university students, who were asked to construct concept maps about how political concepts interrelate (political attitude structure), were more likely to exhibit ideologically coherent attitudes. In the field of teacher education, Leumann (2017) demonstrated how concept maps can be used as a methodological and targeted instrument for the exploration of the deep structure of teachers’ conceptions with respect to general or domain-specific aspects of teaching-related phenomena and processes. Campbell (2016) highlights the benefits derived from adding this learning tool to a teacher’s instructional toolbox and invites them to adopt and integrate digital concept mapping into the social studies classroom “as a means to build content knowledge, to understand what students are thinking about content, and to provide

76

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

opportunities for students to make relevant and important connections within the body of knowledge defined as social studies” (p. 74).

Concept Mapping and Individual Differences The diversity of student characteristics and learning needs can present seemingly infinite challenges to teachers as they deliver complex content into a diverse classroom. Teaching strategies are often thrust into crisis by the presence of diverse students with socialization and/or learning problems. Teachers are often required to adapt their teaching methods and adopt new models that promote meaningful learning for all students. A diffused use of concept maps has shown an improvement in students’ cognitive and metacognitive skills and allowed taking suitable actions to mitigate the diversity represented by immigrant students and students with specific learning disorders or disabilities. For example, a study at the University of Oregon (McCoy and Ketterlin-Geller 2004) showed how reading difficulties can be overcome when teachers assume responsibility for identifying and elucidating concepts. Overt identification of concepts and the use of concept maps reduce the reading comprehension demands placed on students with low abilities. The researchers argue that due to language barriers, or low skill levels, struggling readers often focus on decoding the text at the expense of understanding the material and thus face serious difficulties in understanding the material. Consequently, student learning is potentially jeopardized because of inappropriate instructional techniques adapted to serve the needs of all students. Their study showed that students prepared with the concept-based approach outperformed students in a more traditional classroom on a problem-solving task. Recent studies showed how higher prior knowledge is associated with effective concept mapping utilization that enables a deeper understanding of the topic and effective information problem solving. For example, Whitelock-Wainwright et al. (2020) devised a concept map and search tool to help undergraduate students write two short essays. The findings showed that even when presented with opportunities (i.e., a concept map) to support effective information problem solving, not all learners took advantage or gleaned the benefits of such tools. Prior knowledge has been found to impact information problem solving strategies, with higher levels being attributed to better navigation and planning. McGowen and Davis (2019) detected differences between students who made substantive gains and those who made only limited gains over the course of a semester in the way they used concept maps. Students with high gains tend to construct, organize, and restructure knowledge in ways that are qualitatively different from the processes utilized by students with low gains. Concept maps revealed radically different processes of knowledge construction and organization by the students— those with low gains seemed unable to integrate new knowledge productively into an existing structure. These students might encounter difficulties building on prior knowledge and integrating new knowledge into a core cognitive structure.

Concept Mapping and Individual Differences

77

Chan’s (2017) study showed how specific aspects of concept mapping might aid students with language barriers. Concept mapping helps such students to express knowledge by using non-verbal expressions. Using art-based concept maps in nursing education, the study revisited the definition of concept maps and explored alternative ways of expressing the interconnections among key terms on a visual platform, which could help to enhance the students’ creativity in future clinical practice. The participants used drawings to depict patients’ nutritional care and required surgical treatment by using creative images. The author concluded that such techniques can help students develop better critical thinking skills and engage them in deep learning and creative thinking.

Assessment of Concept Maps Although concept maps have been used as pedagogical tools for several decades, their application to formative assessment to support student learning has been less commonly utilized in the classroom. This may be due to teachers’ unfamiliarity with using concept maps in this context or due to time constraints (Trumpower et al. 2014). The detailed concept map evaluation criteria which follow illustrate why “creating good assessment tasks that measure meaningful learning can take a considerable amount of time, and providing detailed feedback to each student after they have completed an assessment task may take even more time” (Trumpower et al. 2014, p. 234). One method of assessing concept maps was proposed by Novak and Gowin (1984). This approach, based on the components and structure of the map, awards points for criteria such as the amount of branching, crosslinks, and specific examples. A less formal approach to grading concept maps is to use a set of assessment criteria within a rubric. The following six criteria are graded on a four-score scale from exemplary to unacceptable: breadth, interconnectivity, use of descriptive links, efficiency of links, layout, and development of the map over time. Based on these scoring criteria for concept mapping, a more intricate concept map can reflect a more expounded cognitive structure on the given knowledge field. Therefore, concept mapping can also be used as a tool to assess students’ knowledge construction and cognition (Lin et al. 2016). Other researchers (Trumpower et al. 2014) suggest using an expert’s map to assess the quality of students’ maps and encourage educators to use concept maps for formative assessment goals to measure higher-order knowledge and skills. This process enables a comparison of students’ map to a referent expert map; identification of strengths and weaknesses of students who are provided with useful feedback, suggested by the discrepancies between students’ and experts’ maps (students can compare their map with an expert’s or other students’ maps in the course at predetermined time points or at multiple intervals over time). Technology-enabled concept maps are widely suggested to support the formative assessment process. These are considered user-friendly for students and teachers

78

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

as they can be evaluated automatically and generate feedback instantly, thus potentially enabling teachers to identify students’ typical misconceptions throughout the learning process. In this process, students have the chance to witness their development and are provided with feedback that can be used to effectively improve learning. Bittencourt et al.’s (2013) research shows how technology accompanied by well-defined assessment criteria can be used to evaluate concept maps produced by students. The methodology used in their research comprised an interactive reading of concept maps, usage of Cmap Tools software, and construction of concept maps. Concept evaluation criteria featured coherence, propositions, clarity of ideas, and logical relations between concepts. This evaluation of maps revealed consistency among concepts, significant relationships, clarity of ideas, and logical relations between the stages of a research project. Throughout the concept mapping process, students gradually developed the capacity for critical analysis and synthesis of ideas. The process inspired students to search for additional information to explain concepts, and apply logical reasoning to determine coherent relationships between these concepts. Curiosity, creativity, and flexibility of ideas to discuss the possibility of using concepts maps during the development of learning and research activities were also detected. The above-cited researchers conclude that concept maps can be deemed a valid strategy to evaluate the learning-teaching process; however, they also emphasize that an efficient evaluation of concept maps goes hand in hand with providing useful formative feedback to students—a crucial scaffold for students’ learning outcomes. Lachner et al. (2017a, b) reinforced this premise by implementing three different computer-based feedback methods and investigating effects of the format (outline versus concept map) and the specificity (specific versus general) of the feedback on students’ perceived difficulty and on their production of cohesive texts. The above study showed that specific concept map feedback can be an efficient instructional scaffold to support students’ writing a cohesive text. It further demonstrated that this scaffold has to be implemented properly so as to fully utilize the potential of formative computer-based feedback. Based on the above-surveyed research, the aim of the following case study is to explore how students develop concept maps and how they view the use of concept maps as a learning and assessment tool in problem-solving activities in a diverse higher education setting.

Concept Mapping in a Diverse Classroom—A Case Study The following activity included 85 students who were studying for their undergraduate degree in Education at a college located in the northern periphery in Israel. Fifty were first-year students and the remainder were in their second year of the three-year academic study track. The intervention program took place within the framework of two courses: Philosophy of Education and Teaching Methods.

Concept Mapping in a Diverse Classroom—A Case Study

79

The ethnically heterogeneous group of students included Jews, Muslims, Druze, and Circassians. The group also comprised religious and secular Jews, immigrants and native-born Israelis who came from cities, villages, and rural communities. There were differences among the students in both age and life experience. Some of the women were already 30–40 years old, were mothers, and were active in the informal educational system. Others were young people aged 18–22 who were continuing their education immediately upon graduating from high school. Some were participating in programs for aiding elementary school pupils. The assignment that involved using a concept map varied from one course to another. In the Philosophy of Education course, students were asked to analyze the various approaches that had been taught during the semester and to describe the similarities and differences between the various philosophies. In the Teaching Methods course, a problem-based learning technique was implemented that involved perusing a dilemma related to their profession. Perusing a dilemma entails teamwork and discussing the pros and cons of problem-solving alternatives, while considering ethical aspects and finding information that is necessary to establish arguments. This practice is rooted in democratic education, which is based on an exchange of opinions and understanding through listening to the other. At its center, we find conflicts that engage the students in discussion and present divergent opinions. This approach exposes students to problems relevant to their profession, clarifies the moral values related to these problems, and allows them to form an independent opinion, while instilling good practices of dialogue. Peer-dialogue revolving around experiences concerned with ethical dilemmas might be an auspicious beginning on the path to finding a solution. The dilemma represented several different educational approaches, such as education for multi-culturalism and education focused upon national culture. The students were asked to select the educational alternative that reflected their philosophy. After reading about it, they were asked to create two maps: The first one had to present the alternative they had chosen from the approaches presented in the dilemma and offer three to five different arguments to explain why they had chosen it. For the second map they were asked to read academic and non-academic material (such as press articles or films) that would serve as a basis for their initial arguments. They were then asked to compare the arguments by adding arrows accompanied by explanations on the concept map (see Fig. 5.1). They were told to refer to ethical values that arose in the dilemma and to draw connections between the values and the arguments which they had presented. The students received rubrics (see Table 5.1) that included criteria for assessing each of the maps (Panadero et al. 2013). This enabled them to assess the improvement that had taken place between the first and second map, to clarify the difficulties and ask for help accordingly, to acknowledge the skills they had attained during the process, and to clarify which skills were lacking. The two maps were examined by the instructors, who offered feedback aimed at facilitating improvement. A detailed Power Point presentation was used to provide instructions for the optimal use of the Mindomo program for designing the maps. Finally, the students were asked to describe through a reflective journal their progress from creation of

80

5 Using Digital Concept Maps as an Assessment Tool in a Culturally … Main position. Values associated with the arguments. Three arguments. Supporting information to substantiate the arguments, including hyperlinks to elaborated materials. Explanations regarding similarities and differences between the arguments.

Fig. 5.1 An example of a concept map detailing the arguments, evidence, and values

the first map to the second one. The students described difficulties in using a concept map as well as its advantages. They also described methods that they used to make their work more efficient. The following analysis of the students’ descriptions in their reflective journals emphasizes how they perceived the use of a concept map and the role that characteristics such as age, culture, and previous knowledge played in their perceptions. Figure 5.1 is an example of a map created by the students. The analysis of the reflective journals rendered six main categories: 1. 2. 3. 4. 5. 6.

Visual clarity and simplicity of presenting the material using a concept map. Organizing learning material. Creativity and connection to previous knowledge. The technological challenge. The importance of criteria and feedback in the assessment process. Implications for future professional practice.

Visual Clarity and Simplicity of Presenting the Material Using a Concept Map Most of the students noted that the map helped them to emphasize and summarize important items, enabling them to clarify the important points for themselves and for those reading the map. “We were able to emphasize the important things and eliminate what was less important. In that way the reader will understand it better” (Yaara, Muslim female). For Arab students, who have difficulty writing in Hebrew, the language of instruction, the advantages of using a concept map were even greater due to the demand to write in a concise manner. This was described by Assil (Muslim female) and 12 other Arab students: “The work with the map was orderly and easy to understand. The map helped me to explain my argument in a simple manner without so much detail. It focused me on the things that were important for me to understand,

Visual Clarity and Simplicity of Presenting the Material …

81

Table 5.1 Evaluation of the maps—checking improvement between step 1 and 2 based on wellestablished criteria Criteria/score

4

3

2

1

Components of The position, 3–5 argumentation substantiating reasons (map 1) and supporting facts (map 2) are included

The map contains the position with fewer than 3 substantiating reasons (map 1) or facts (map 2)

The position is Most key stated but components are secondary reasons lacking (map 11) or facts are not

Hierarchy

The organization is complete and correct, as reflected by the map

The organization is correct but incomplete; some levels or elements are lacking

The organization The organization is complete but is incomplete and incorrect; there incorrect are components in the wrong places

Relationships among components in different hierarchical levels

Relationships: They are correct, making connections among the correct components LINKS: They are explicit and help to better understand the relationships among components

Relationships: They are correct but incomplete; some connections are lacking LINKS: They are incomplete. Only some are explicit but they are correct

Relationships: Some are incorrect, making connections among components without a relationship LINKS: Only some are explicit, and others are incorrect

Relationships: The majority are incorrect or there are only a few that are correct LINKS: They are incomplete and incorrect

Relationships among concepts from different columns

There are several connections, making relevant relationships

There is only one

None

None

Simplicity and ease of understanding

Its design is simple and easy to understand

There are examples. Some relationships are difficult to understand

There are only a few examples. There is an excessive number of connections

There are no examples. Neither the relationships nor the hierarchy are understandable

without deviating from the subject.” Ruan (Muslim female) also described these added advantages: The advantages are that you don’t have to write a lot. You have to write a small amount that is clear. One can’t give many details on the map. I have to write a lot in regular assignments in Word format, and half of what you write is repetitive and redundant…On the map you only write what you have to. It helped me a lot because the material was explicit and clear. It was easier for me to understand than other assignments.

Another aspect that was regarded as an advantage by 11 Arab students was connected to the visual representation of the maps. This aspect contributed to the

82

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

students’ ability to illustrate hierarchy, to differentiate between the argument and supporting evidence, and to use visualizations such as pictures in lieu of words. Raon (Muslim female) explained, “I liked the fact that the map was attractive and clear, with colors. This method helped me to emphasize the most important points and to arrange the points from the title downward. I chose different colors to show the various arguments and explanations that supported them”. Assad (Muslim male) added, “The greatest advantage is that we can use the map to minimize a broad subject and use pictures in addition to words. Material sticks in your mind better that way. It’s a simple tool to work with. All the concepts are on one screen and it’s easier to understand when it’s presented in a visual manner.” It should be noted that these comments also appeared among Jewish students, but less frequently. Four students addressed this issue while mainly referring to the potential use of a concept map in their future teaching. They understood the potential of using the map for teaching children who have difficulty reading and writing. In Nurit’s (Jewish female) words, “I know a lot of children for whom learning the material this way would be very helpful. There is a lot of visual order and you don’t have to look for material in endless text.”

Organizing the Learning Material—Concept Mapping as a Means of Self-assessment Fifteen students noted that the map helped them to organize the learning material, to create a logical coherent sequence among the various concepts, to find a connection between concepts and theories, and to make the learning material clearer. These comments had no connection to the students’ cultural background or other characteristics. Stav (Jewish female) described how the concept mapping process helped her group to review the material in a more critical manner while understanding the similarities and differences between the various philosophies: The map helped us get better organized for learning. In this assignment, we reviewed all the material that we decided to work on. The assignment enabled us to concentrate on every period, its characteristics and its philosophers separately. That also helped us understand the similarities and differences between one philosopher and another. Our group chose the Renaissance. We went over every point in the material and then studied each of the main characteristics and the three concurrent revolutions: cultural, religious, and technological. We wrote what was important in each of these, and we learned how they influenced the different philosophers.

Fatma (Muslim female) also described how the map helped her group to organize the material and summarize it before the final summative exam at the end of the course. In her words she addressed the concept mapping activity as a means of self-assessment that enabled her to be better prepared for the final exam: The map helped me understand what is important and what is less important, and what is relevant to remember and what is less relevant. It helped me focus and summarize the

Organizing the Learning Material—Concept Mapping as a Means …

83

material in a clear, easy way so that I could remember it. Each period in history has its characteristics and philosophers that belong to that period and their educational perceptions, and the similarities and differences between the periods. The fact that I reviewed the material more than once to understand what was important and what was not helped me focus on the [most pertinent] material. It really helped me. I was also able to arrange the material effectively before the exam because it’s easier to remember the material as a map.

Bella (Jewish female) also noted the advantages of using a concept map that is organized coherently and in a way that contributes to the effectiveness of the learning process. Her remarks create a connection between a learning process using a concept map and the constructivist approach to learning that enables students to play a part in constructing learning material. It appears that the concept mapping helped the students to reflect on the knowledge they had gained during the course and evaluate their level of understanding of the material taught, I learn better and understand and remember the material better when it’s arranged in a flow chart, tree, or map. I really like this style. I think it’s more orderly and focused, and the fact that I was the one who constructed the map, made the connection between the different philosophers, and found the similarities and differences helped me understand the material better. It also helped me to read the map and better understand it. To construct the map, we have to go over the material again, know how to summarize it, and write down comments and main points. Focusing on maps helps us understand the material better.

Creativity and Connection to Previous Knowledge Six Jewish students with previous teaching experience underscored aspects that were connected to the hidden potential of the concept map method for nurturing creativity and curiosity, not only for students but for various age groups. It appears that their prior teaching experience contributed to their ability to profoundly understand the advantages of working with a concept map that extend beyond academic skills. Gila (Jewish female) explained, It appears that this map/program is “the land of endless discoveries.” It permits a great deal of creativity and leads to curiosity. The map is suitable for all ages. It seems to me that it is also suitable for very young children if the teacher creates the map and the children have to search for, associate, and connect the concepts. Aside from that, creating the map and the final visible product is very rewarding. It looks attractive. Going around and observing the quality of the work is extremely rewarding.

Sivan (Jewish female) recalled how she had been asked to prepare a mind map in elementary school and how the assignment had aroused a spark of creativity within her. When I was in elementary school, I was asked to prepare a mind map. I remember cutting out newspaper clippings, planning, drawing, cutting things out, gluing colors and being happy while doing it. It’s a pleasant memory of exploration and the joy of creativity, and it’s even more pleasant to do it again while studying at the college. During this assignment, I used my imagination more. I think I was more creative, and I thought up all kinds of ideas.

84

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

The Technological Challenge As mentioned previously, the students received instructions in the use of the Mindomo Internet platform for building and sharing their concept maps. However, it appears that working with the platform was challenging for many of the older students (most of whom were Jewish) as well as for some of the Arab students. Despite their positive attitude towards using a concept map, they required more extensive instructions and technological support, as Anabel (Jewish female) explained, “I definitely see the map as a method of coping with large amounts of material, but I have to practice with the map several times to get to know it better. I also needed technical support.” Yael (Jewish female) offered examples of the difficulty they experienced during the group work process: The material can accidently be deleted and is not saved…One mistake can accidently delete everything that was prepared. This is particularly true in group work when several people are working on the same map. It’s rather ironic to attribute human qualities to the map, but I get a feeling that it is very sensitive. During the first part of the assignment, we were all occupied with the technical aspect of creating the map. I don’t think we should assume that computer learning programs are necessarily clear. If I want to learn how to use a specific computer program, I first need to learn the program thoroughly and then begin the actual work.

The Importance of Criteria and Feedback in Assessment Processes At the beginning of the process, the students received criteria for self-assessing the maps they were about to create, as well as feedback from the instructors. The students’ reflective journals revealed that these criteria and feedback helped them in the process of constructing the map. The Arab students were aided by the criteria and sought feedback more intensely than their Jewish colleagues. This was in light of the fact that they had never worked with a concept map before and had difficulty constructing it according to the exact requirements. Doaa and Nadin (Muslim females) explained, respectively: The criteria that we received helped me to understand what needed to be on the map. It helped us understand the structure that we were supposed to plan and to know what we needed to focus on in our map. It mainly helped me because I had a bit of a problem with this learning method. It was new for me, and there were difficult parts that took me a long time to do. After completing every stage of the work, we looked at the criteria we had been given. The criteria helped us assess the map and understand how to construct it properly. I went over the criteria for assessing the map and explained them to the others in my group to make sure that all of us understood the instructions and knew exactly what we needed to do.

The Jewish students also testified to the effectiveness of criteria for self-assessment but referred to it in connection to their future teaching and the effectiveness of using self-assessment when working with children. Mira (Jewish female) explained,

The Importance of Criteria and Feedback in Assessment Processes

85

I think that this is an excellent thing that can help a lot in the educational system. If children can perform a type of personal assessment when they finish an assignment, they will know exactly on what they are receiving or not receiving a grade. They will be able to personally critique their work. We often complete assignments without knowing whether or not we achieved exactly what the person who gave the assignment wanted. We will be able to critique the work with this type of criteria and make corrections accordingly. I was really enthusiastic about this, and I talked about it to several of my friends who are teachers and suggested this idea.

Implications for Future Professional Practice Forty-one of the students who took part in this study talked about the possibility of using concept maps in the future. Using a concept map was not a one-time activity for them. The recognition of the method’s effectiveness aroused motivation to transfer the approach to future activities. This was explained by Misa (Muslim female), I think the most important advantage is that this knowledge – the use of the map – will help us in our studies and even in our work. For example, I can adapt the method of constructing a map to other subjects. That way it will be easier for me to learn and remember the material. It can also help with work in the future such as building a work plan or intellectual games for children using the map.

Ruan (Muslim female) emphasized the differences between the traditional approaches used in the Arab sector and the use of the map, while highlighting the advantages of using the map for children in the Arab sector, In my opinion, using the map as future educators is very important. On the one hand, it gives us a possibility of sharpening and focusing the material that we need to teach, and on the other hand, it presents the material to adolescents or children in a focused manner. In most cases it will increase the children’s motivation to learn and concentrate on the material because it appears in the form of a map rather than the way it usually does as “spilling learning material on paper.” It’s important to use a concept map and to let children experience it because it will deepen their thinking and enable them to organize the material more clearly, focus the material, and understand what is more relevant and what is less relevant.

Gaben (Muslim female) also noted the advantage of using the map for children with whom she would be working in the future. She also suggested an idea for an activity with the map within the framework of the mentoring project in the Arab village where she lives: I volunteer for a mentoring project with children who are 7-14 years old. Using the map can help me adapt the activity to different children. For example, one of the activities deals with violence in the Arab sector. It’s a subject that’s important to me to talk about, and I can use the map to construct an informative activity for children and adolescents.

Discussion This chapter showed how constructing a concept map can be used as part of the assessment process among students with varying needs and interests. The process

86

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

consisted of constructing a concept map, determining criteria for assessing it, and reflecting on the process. In the Teaching Methods course, the map enabled students (and instructors) to assess the quality of the arguments raised in connection to a professional dilemma. It further enabled them to determine whether the backing for their argument was well-based, to examine the depth of the understanding of the supportive arguments for one of the alternatives proposed for dealing with the dilemma, by assessing the set of linkages the student had created between the argument and its support. The entire process allowed for examining the cogency of the argument, relationships between ideas, consistencies, primary ideas versus secondary ideas, and tracing the development of the argument from the first map to the second one. The assessment findings indicated that by using a digital platform to create their concept maps, students succeeded in improving their skills. In the Philosophy of Education course, the concept map helped the students organize the learning material and find connections between concepts, arguments, and theories. This in turn enabled them to grasp the continuity of issues, think about them in a critical manner, and thus gain a deeper understanding of the learning material. Identifying similarities and presenting them as lines on the map helped the students to trace common characteristics between historical periods and educational perspectives according to the course objectives. Some were even able to assess and critique each of the educational approaches suggested by the philosophers and to indicate this on the map. Other students noted that the process had helped them to remember the material better for the summative examination at the end of the course. The concept map was further found to have helped the students nurture and hone various thinking skills, ranging from lower-level skills of simple processing and remembering information to higher-level thinking skills such as the ability to analyze and organize information in a critical manner and to assess it according to given criteria. As for the instructors, the concept map enabled them to spot incorrect perceptions, examine the level of students’ understanding, and adapt their teaching to the various levels that were observed in a heterogeneous classroom. As a pedagogical tool, the concept map was well received by the students aiming towards a future in teaching. The majority of students expressed a willingness to use it to increase the effectiveness of their teaching process, and especially noted the advantages of the concept map being used by children who have difficulty expressing themselves in words. Students, who had already accumulated teaching experience, related to the concept map not only from the academic-cognitive aspects but from the psychological aspect as well. They mainly referred to the concept mapping’s ability to serve as a basis for enhancing curiosity and motivation for learning citing the numerous possibilities it offers for implementing creative solutions.

Implications and Recommendations In summing up the experience of constructing a concept map in a heterogeneous classroom, the overall conclusion was that it contributed to the learning processes

Implications and Recommendations

87

in several aspects. Many students noted that building the map fostered important skills, including the ability to distinguish between what is important and trivial and presenting things in a concise manner. Students whose mother tongue was not the language of instruction (Hebrew) viewed the map construction positively as it required them to express themselves in a concise manner rather than writing long texts. It seems that constructing and using a concept map is beneficial for students with reading and writing difficulties since it requires considerably less extensive reading material and minimizes writing. Alongside the advantages described by the students, analysis of their comments revealed that for some, the activity was somewhat complicated as it required using a new, unfamiliar Internet platform. The older students (aged 30–40) sometimes had difficulty using the technology, which hampered their ability to complete the task. In this case, although students received instruction in the use of the platform prior to the activity, it appears that continuous guidance needs to be provided for those students who are less familiar with technological environments. This issue was raised by relatively older Jewish students, but many Arab students also required individual guidance in the technological aspect. Being provided this type of guidance would probably increase the students’ motivation to use the platform as well as considerably reduce the gaps between students, which stem from discrepancies in their familiarity with digital formats. At the beginning of the activity, criteria for self-assessment of the map were established and students received them as part of the preparation for the assignment. It is important to verify, from the outset, that the students understand the requirements of the assignment and how it will be assessed. The criteria helped students understand what was expected of them and clarified the requirements of the assignment. This is particularly important in the case of students for whom constructing a map is a new learning experience. Some of the students expressed a willingness to use self-assessment in their future classrooms. This point is best exemplified by Arab students who stated that, in their view, use of a concept map would contribute to alterations in the conventional methods of instruction that are used in schools in the Arab sector. In other words, their positive experience with using the map and their acknowledgement of its contribution to the learning process may spur a process of change towards applying constructivist approaches to teaching and learning in schools that currently use only traditional methods. This important process is referred to in the literature as “transfer of learning” (Lobato 2003). A central goal of education is to provide learning experiences that are useful beyond the specific conditions of basic learning. Based on our findings, it seems that the concept map-building activity fostered a deeper understanding of the purpose of the activity, which might in turn lead to greater transfer. It may be inferred that using a concept map in curriculum settings that reflect the constructivist underpinnings of the tool may lead students to recognize its benefits, as they master the skill at a level which they feel confident to apply in new situations. Special attention should be paid to the teacher’s role in the concept mapping process. S/He should have clear instructional objectives and assessment criteria for the use of concept mapping which must then be conveyed to students. Hence, the

88

5 Using Digital Concept Maps as an Assessment Tool in a Culturally …

teacher should foster a learner-centered classroom which focuses on the learning processes rather than the products of learning. The teacher becomes a learning orchestrator rather than an information transmitter—and this implies moving from a traditional to an actor-oriented mode by applying social constructivist frameworks that structure the students’ generalized experiences. As today’s society is characterized by the rapid development of information and communications technology, graduates are expected to be able to apply knowledge efficiently to solve complex problems in real-life situations. In keeping with the constructivist approach to learning, and with special consideration devoted to students’ different needs, technology-enabled concept mapping can be viewed as a student-centered teaching method which nurtures deep learning associated with high-order thinking skills. Our main conclusion is that concept maps can serve as an effective means for assessing the learning process. However, it is important to note that for assessment to be effective and contribute to students’ advancement, it must be accompanied by criteria, feedback, and appropriate assistance from peers and instructors, alike, which is optimally suited to the students’ diverse characteristics and needs.

References Bittencourt, G. K. G. D., Nobrega, M. M. L., Medeiros, A. C. T., & Furtado, L. G., (2013). Concept maps of the graduate programme in nursing: experience report. Revista Gaúcha de Enfermagem, 34, 172–176. Campbell, L. O. (2016). Concept mapping: An “Instagram” of students’ thinking, The Social Studies, 107, 74–80. Chan, Z. C. Y. (2017). A qualitative study on using concept maps in problem-based learning. Nurse Education in Practice, 24, 70–76. Cicognani, A. (2000). Concept mapping as a collaborative tool for enhanced online learning. Educational Technology & Society, 3, 150–158. Croasdell, D., Freeman, L., & Urbaczewski, A. (2003). Concept maps for teaching and assessment. Communications of the Association for Information Systems, 12. https://doi.org/10.17705/1CAIS. 01224. Erickson, H. L., Lanning, L. A., & French, R. (2017). Concept-based curriculum and instruction for the thinking classroom. Thousand Oaks, United States: SAGE Publications Inc. Hung, C. H., & Lin, C. Y. (2015). Using concept mapping to evaluate knowledge structure in problem-based learning. BMC Medical Education, 15. https://doi.org/10.1186/s12909-0150496-x. Jennings, D. (2012). The use of concept maps for assessment. Ireland: UCD Teaching and Learning. Retrieved from https://www.ucd.ie/teaching/t4media/concept_maps_assessment.pdf. Keating, D. M. & Bergan, D. E. (2017). Mapping political attitudes: The impact of concept mapping on ideological constraint. Communication Studies, 68, 439–454. Kinchin, I. M. (2014). Concept mapping as a learning tool in higher education: a critical analysis of recent reviews. The Journal of Continuing Higher Education, 62, 39–49. Kwon, S. Y., & Cifuentes, L. (2009). The comparative effect of individually-constructed vs. collaboratively- constructed computer-based concept maps. Computers & Education, 52, 365– 375.

References

89

Lachner, A., Burkhart, C., & Nückles, M. (2017a). Mind the gap! Automated concept map feedback supports students in writing cohesive explanations. Journal of Experimental Psychology: Applied, 23(1), 29–46. Lachner, A., Burkhart, C., & Nückles, M. (2017b). Formative computer-based feedback in the university classroom: Specific concept maps scaffold students’ writing. Computers in Human Behavior 72, 459–469. Leumann, S. (2017). Representing Swiss vocational education and training teachers’ domainspecific conceptions of financial literacy using concept maps. Citizenship, Social & Economics Education, 16, 19–38. Lin, Y. T., Chang, C. H., Hou, H. T. & Wu, K. C. (2016). Exploring the effects of employing Google Docs in collaborative concept mapping on achievement, concept representation, and attitudes. Interactive Learning Environments, 24, 1552–1573. Lobato, J. (2003). How design experiments can inform a rethinking of transfer and vice versa. Educational Researcher, 32, 17–20. McCoy, J. D., & Ketterlin-Geller, L. R. (2004). Rethinking instructional delivery for diverse student population: Serving all learners with concept-based instruction. Intervention in School and Clinic, 40, 88–95. McGowen, M. A., & Davis, G. E. (2019). Spectral analysis of concept maps of high and low gain undergraduate mathematics students. Journal of Mathematical Behavior, 55. https://doi.org/10. 1016/j.jmathb.2019.01.002. Novak, J. D. (1998). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. Mahwah, NJ: Erlbaum. Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. New York: Cambridge University Press. Panadero, E., Romero, M., & Strijbos, J. W. (2013). The impact of a rubric and friendship on construct validity of peer assessment, perceived fairness and comfort, and performance. Studies in Educational Evaluation, 39(4), 195–203. Partnership for 21st Century Skills. (2014). ICT literacy maps: Social studies map. Retrieved from: http://www.p21.org/images/stories/matrices/ICTmap_ss.pdf. Trumpower, D. L., Filiz, M., & Sarwar, G. S. (2014). Assessment for learning using digital knowledge maps. In D. Ifenthaler & R. Hanewald (Eds.), Digital knowledge maps in education (pp. 221–237). Springer New York. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Cambridge. MA: Harvard University Press. Whitelock-Wainwright, A., Laan, N., Wen, D. Gasevic, D. (2020). Exploring student information problem solving behaviour using fine-grained concept map and search tool data. Computers & Education 145. https://doi.org/10.1016/j.compedu.2019.103731.

Part II

Interpersonal Facet

Chapter 6

Peer Assessment in Multicultural Classrooms: Tensions and Opportunities

Introduction We have seen that culturally sensitive and responsive methods of assessment strategies provide multiple opportunities for students to demonstrate their learning. One of these methods is peer assessment, which helps ethnic groups take ownership of their educational progress and, in a wider sense, of their involvement and inclusion in society (Nayir et al. 2019). Peer assessment is the application of criteria and standards to evaluate and deliver feedback on the work of peers or colleagues. It is considered a form of collaborative learning whereby groups of individuals rate their peers via a judgment, or assessment, of the amount, level, value, worth, quality and/or success of the learning products or outcomes (Topping 2017). However, while it is widely acknowledged that peer assessment practices offer higher educational potential for engendering enduring and effective changes in students’ learning skills, research aimed at understanding the implications of culture on peer assessment is only in its nascent stages. To fill this void, this chapter focuses on culturally diverse students’ perceptions of, and attitudes toward, peer assessment, and attempts to shed light on both the opportunities and the challenges of this assessment approach for culturally diverse students.

Peer Assessment: Challenges and Benefits Topping (2019, p. 1) defines peer assessment as “an arrangement for learning to consider and specify the level, value, or quality of a product or performance of other equal-status learners, then learn further by giving elaborated feedback to them and discussing their appraisals with those who were assessed to achieve a negotiated agreed outcome.” Topping suggests 43 variations of peer assessment; for example, summative or formative, single product versus several products, in class or out of

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_6

93

94

6 Peer Assessment in Multicultural Classrooms …

class, process monitored or not. Although there are numerous varieties of peer assessment, Topping emphasizes that peer assessment aimed at inspiring reflection and developing personal and professional skills, should be formative; i.e., the assessment type in which students help each other identify their strengths and weaknesses and target areas for remedial actions. Peer assessment is a form of collaborative learning (Webb 2012), also referred to as participative assessment, “a process in which students and tutors share, to some degree, responsibility for making evaluations and judgments about students’ written work, gaining insight into how such judgments are made and finding appropriate ways to communicate them” (Reynolds and Trehan 2000, p. 270). The assessment process should comprise transparency and explicit grading criteria, which may either be given or influenced by the students themselves. Grading criteria are worded as statements that convey what the student is expected to perform in order excel (Panadero 2016). These statements help identify various levels of performance among the students or of each individual student. Determining these criteria can motivate students to strive for, and ultimately achieve, the highest levels of performance. Therefore, assessment should contain a guide or indicator by which those areas in need of improvement can be identified and addressed (Alt and Raichel 2018). The criteria provided should be clear to the students, as well; when they are involved in the development of the assessment criteria, it makes them feel intrinsically engaged in the process. The students need to understand the criteria by which their work will be evaluated and to work jointly with their teacher in order to become autonomous and reflective people (Topping 2019). Drawing on the social-constructivist approach, peer assessment emphasizes shared activity during the learning process. It involves teachers and students in joint activity characterized by: (1) a shared engagement in the process, (2) a collection of and reactions to evidence of learning by providing scaffolds that include feedback, and (3) self-regulation by the learners. Hence, peer assessment occurs within a community in which both the teacher and the students take part, set goals, experience practices and norms of mutual support, trust, respect and cooperation during interactions aimed at supporting learning (Wu et al. 2012; Zhou et al. 2019). Alongside the benefits, Topping (2017) points to several drawbacks of peer assessment. On the upside, there are immediate benefits for learning and achievement and also longer-term benefits with respect to transferable skills in communication, collaboration, and self-regulation of the student’s own learning. These benefits pertain to both the assessors and the assessees. As Topping states, “Peer assessment of learning and social behaviors sharpens and broadens the assessor’s capabilities. Combining assessment of product and process can enhance student understanding of the consistency or mismatch between these, and different ways of learning beyond their own” (p. 3). Peer assessment was found to have positive effects on domain-specific assessment skills because it provides immediate, individualized, and richer formative feedback, and thus has a clear potential of fostering the subsequent learning process (De Grez et al. 2012; Topping 2019). On the downside, peer assessment has been criticized for being less “correct” than teacher feedback, as suggested by Topping: “Assessees just need to be somewhat

Peer Assessment: Challenges and Benefits

95

cautious about peer feedback, and assessors [need to be] cautioned that if they are uncertain, they should not claim to be right, since they are not expected to always be right” (Topping 2017, p. 3). Reliability, however, can be increased by providing assessors with scoring checklists or rubrics, defined as a document that articulates the expectations for performance of an assignment by listing the criteria and describing the levels of quality. Although frequently used by teachers to gauge student learning and assess whether or not the learning outcome was demonstrated, rubrics can serve as part of a formative assessment, and thus can teach as well as evaluate. Rubrics have the potential to help students understand the learning outcomes and standards of quality for a particular assignment, as well as make judgments about their own work, which can inform revision and improvement (Reddy and Andrade 2010). When shared with students, rubrics become a way for diverse students to accurately assess learning while allowing variation in how the learning is demonstrated, and thus be deemed culturally responsive (Montenegro and Jankowski 2017). The reports of peer assessment help to identify various performance levels among the students or of each individual student and can motivate students to strive for, and ultimately achieve, the highest performance levels. In contrast, summative assessment, which entails giving a single grade to a student, is not deemed appropriate feedback for the student’s performance but merely indicates an overall performance level. This approach is unable to identify strengths and weaknesses in relation to learning outcomes. Therefore, assessment should include a guide or indicator by which areas in need of improvement can be identified and addressed. The assessor, according to this approach, can, for example, be either the teacher or a student who assesses fellow students. Transparency is intrinsic for effective peer assessment and should be implemented by notifying students on whether the assessment will be entered in any kind of highstakes assessment, and in what proportion. Moreover, as asserted by Van Zundert et al. (2009), as peer assessment can be adapted to students’ individual needs, an enormous variety exists both in peer assessment practices and in research on their effects. Hence, the conditions under which peer assessment occurs differ, a diversity of methods can be applied, and many different outcomes can potentially emerge, For example, one might imagine that students who already have some experience in assessing their peers (condition) might gain fewer learning benefits (outcome) from extensive assessment training (method) than students who have never assessed their peers before. This multiplicity in itself is positive; that is, peer assessment can be customised to individual needs. However, it does complicate the drawing of inferences about causes and effects. This is because the literature usually describes peer assessment in a holistic fashion; that is, without specifying all the variables present in terms of conditions, methods and outcomes (p. 270).

To constitute an effective peer assessment Van Zundert et al. (2009) suggest improving it by training potential peer assessors. They found that the development of peer assessment skills and positive attitudes towards it are associated with students’ thinking style, academic achievement, training, and experience.

96

6 Peer Assessment in Multicultural Classrooms …

Peer Assessment in a Diverse Classroom: Opportunities and Challenges The opportunity for students to evaluate each other’s work could facilitate, in Giroux’s words (1988), “diffusing authority along horizontal lines…under such conditions, social relations of education marked by dominance, subordination, and an uncritical respect for authority can be effectively minimized” (p. 39). In contrast, Reynolds and Trehan (2000) assert that sharing in the procedures of assessment does not necessarily result in more critical or democratic processes and advise looking into the cultural bases of power which influence relationships among the students themselves, power bases that are likely to affect the assessment process. Indeed, many studies have demonstrated the contribution of peer assessment to students’ learning, motivation, and performance (Hsia et al. 2015; Panadero et al. 2016). However, despite these cumulative reports, minimal attention has been devoted to evaluating peer assessment implications in the culturally diverse highereducation classroom, and the cultural bases of power are absent from most accounts of participative approaches. In a school-level context, Bryant and Carless (2010) reported on the perspectives that a class of Hong Kong primary-school students and their teachers had on their engagement with peer assessment. Given the challenges of implementing assessment change, they cautioned against assumptions that the promotion of peer assessment solves problems inherent to summative assessment and have suggested to focus on broader cultural contexts. For example, they observed that students from collectivist cultures are likely to perform better in peer cooperative processes than those from more individualistic settings. Chinese students also had more confidence in teacher feedback which could provide specific explanations and concrete suggestions for revision. Given the academic challenges encountered by minority students in higher education, and given the scant empirical research done thus far aimed at examining these challenges in conjunction with peer assessment, the study described below sought to delve deeper into multicultural students’ perceptions of, and attitudes towards, peer assessment, and to pinpoint the main opportunities and challenges of this assessment approach in a culturally diverse classroom. This study may expand our understanding of how peer assessment might be used beneficially in multicultural higher education settings.

Peer Assessment Implementation Design The following is an extended report based on Alt and Raichel’s (2020) work. The process included 120 first-year undergraduate Education students (90% females) from one major college located in the northern periphery of Israel. The college was expressly established in this region with the intention of widening access to

Peer Assessment Implementation Design

97

higher education for Arab minorities. The students were enrolled in a course entitled ‘Informal and Formal Education’ at the first year of their studies. The ethnic distribution of participants was: 30% Jewish students, and 55% Muslim, 8% Christian, and 7% Druze Arab minority students, with a mean age of 22.85 (SD = 5.06) years. In addition to Alt and Raichel’s (2020) quantitative analysis report, semi-structured interviews were conducted with 14% of the participants (N = 17, 14 females, 3 males), of whom nine were minority students (five Muslim students, three Christian students and a Druze student). The interviews were used to ascertain the participants’ subjective experiences and attitudes during peer assessment processes in a multicultural learning environment by propounding questions such as “Describe the challenges you had during the learning and assessment process in each assignment”. During the peer assessment process, the participants were given two similar group-assignments in line with the project-based approach. The first intervention included converting a selected instructional unit into a dilemma-based game (a twomonth project); the second intervention was also a unit of dilemma-based learning which required searching for different solutions to a professional dilemma (an ethical problem), based on a selected instructional unit (a two-month project). The students worked in groups of four to five students and were required to prepare and deliver a presentation of their learning (PoL). Next, the students were asked to present their individual and group assessments of their peers’ PoLs, based on indicators that were previously designed in collaboration with the teachers. As noted by the teacher, The process of determining criteria is important but complex. I created the first draft and presented it to the class. The students read it and offered comments. Some of the criteria were consequently changed. For example, during the second assignment the students commented that the claims supporting the group’s position need to be distinguished from each other. Those criteria were then added to the list. I had planned that this activity would take half an hour, but in actuality it took an hour and a half. During the next lesson, we reviewed the wording of the criteria following additional feedback on the part of the students who had reread it at home.

The students were interviewed following the second assessment process. The following analysis involved examining the collected qualitative data and generating themes from the data (Creswell 2015). The analysis centered around four themes: Gaining experience over time and practice; feedback for the purpose of advancement and growth; teacher-centered assessment; and cultural group dynamics and peer assessment.

Gaining Experience over Time and Practice At first, some minority students were concerned about their inexperience in assessing their peers and felt uncomfortable critiquing others’ work. For all of them, this was the first time they had ever experienced individual instruction that encourages reflective thinking and critique, learning in groups, and peer assessment. However, after two

98

6 Peer Assessment in Multicultural Classrooms …

iterations eight interviewees felt better able to engage in peer assessment. Jasmin (Druze female) explained: I had a lot of difficulty when we met with the advisor, and I didn’t know what was happening with our work. The lecturer asked us questions, but we didn’t have an answer. She talked to the group about dividing up the work and showed us how to cooperate. We had never worked in groups or assessed our peers before. This wasn’t acceptable in Arab schools and wasn’t done in the school I attended.

Asma’a explained the improvement that took place in her skills and feelings of competence she had gained after two peer assessment repetitions, This is the first time that I worked with a group. The assessment method is also new to me. I wasn’t accustomed to working in a group, and it was difficult for me. Everyone had an opinion, and everyone criticized in a way that was comfortable for them. Some people claimed we had not cooperated enough, and that was hard for me. Some people said there was no cooperation at all. Everyone did part of the work and in the end, we tried to put all the parts together. That’s the reason why it was hard for me to present the project in front of the class. I was a bit confused and wasn’t sure of myself. I stood in front of the class and simply lectured. I felt flustered when I had to assess other groups. But during the second assignment, I understood how to work differently. After I received feedback from the teacher and from other students, I understood that I can [do it], and that’s a good feeling. I wanted to get up again and do it better. (Asma, Muslim female). Similarly, Suha and Nadin shared the novelty of peer assessment for them, and emphasized its importance for their learning, I never learned the ability to listen to critique, respond to critique, or to speak in a positive manner. We need to learn to express our opinion with respect without getting angry, and to say how one should improve. We need to be taught these things from preschool. The ability to accept critique is also very important. If we begin to learn these things from a young age, then the process will be much easier for students (Suha, Muslim female). I think that dealing with the issue of assessment is important. You receive a lot of feedback throughout your life at work and in school. Giving students the opportunity to become actively familiar with this topic is important. It’s an excellent tool for learning. It’s important to learn this as a tool, with all the hidden and obvious messages of the objectives, challenges, and possibilities. It’s undoubtedly something that I need to take with me, but we need to know how to do it correctly (Nadin, Christian female).

Feedback for the Purpose of Advancement and Growth The students who were asked about the assessment method and their feelings towards it, attributed improvement in the assessment methods to scaffolds provided by the teacher during the process. Over time, their sense of efficacy grew in terms of their ability to assess others. Seven interviewees emphasized the teaching method which facilitated personal contact with the advisor and receipt of her feedback, for example: It was very difficult for me to accept critique. It was hard for me to hear my faults, or as the lecturer defined it – ‘critique for the purpose of advancement and growth’. Then I had a conversation with my grandfather, who had been a principal and a teacher. I told him what had happened and about the lecturer’s assessment, and he told me that in order to get to the

Feedback for the Purpose of Advancement and Growth

99

top you need a lot of strength, and you don’t get there easily, and that you learn from your failures. This taught me that nothing is perfect and that you have to work hard to succeed. The second time, the feedback was very positive, and I felt good that the teacher saw that there was improvement. I felt that I was doing better and that I was then capable of making corrections and comments to other groups (Shirin, Christian female). During the second assignment it was easier and more comfortable for us as a group because we already understood each other and had learned how to work and when to meet. We had already come to an agreement about most things and there was more cooperation between us. We were aware that the instructor required that we play a different role than in the past in this type of learning. The instructor was available to offer direction, advice, and support in developing the new skills. As a result, the product was better and more organized. The presentation was also more comfortable for me. Maybe that’s because the first time is always difficult. During the assessment I thought about the groups that made presentations and about the process they underwent. I felt more mature than I had the previous time (Samar, Muslim female).

Teacher-Centered Assessment During the interviews, eight students explained that despite the importance of peer assessment, the lecturer still plays a central role in assessment due to professional reasons because “as a professional, the lecturer is likely to offer more accurate feedback”. This premise was shared by students of all cultural backgrounds. For example, Dotan, (Jewish male) claimed that the lecturer was aware of the group’s entire learning process and therefore, “as an assessor she has the possibility of knowing who really did the work. The lecturer has to assess the process. We can only assess the final presentation”. Ravit (Jewish female) similarly claimed: As a professional, the lecturer is likely to offer more accurate feedback because he knows the material, and I don’t know the material like the lecturer does. From the standpoint of contents, for example, I listen to students talking about educational reform, and I can listen, but how do I know whether what was said is correct? I don’t think that I can offer feedback from the standpoint of knowledge.

Cultural Group Dynamics and Peer Assessment After the first peer assessment, minority students gave higher scores to their Arab peers for their presentation than the Jewish students did, while after the second iteration the gap between the groups narrowed. These findings can be explained by the high abilities that minority students attributed to the others in the class. After the second experience, however, the students became better acquainted with their classmates, with whom they initially had preferred working, which led to a more balanced critique of their peers and posing of more questions. The process also led some to reach out for assistance from their Jewish classmates. The following description of Basma (Muslim female), reveals her feelings during the first and second assignments,

100

6 Peer Assessment in Multicultural Classrooms …

During the first assignment, another student and I led the work, but I felt that the two of us were doing all the work. All the rest [of the members] submitted the material at the last minute, just one day before the presentation. They didn’t give us any respect. There were disputes among the students in my group, and as a result we stopped being in contact and didn’t speak to each other, even after the work was finished. What’s frustrating is that these same students who didn’t do the work still offered feedback to other groups. That’s not fair. During the second group assignment, I decided that all participants would take an equal part. I would not take pity on anyone. I wouldn’t relent and I wouldn’t work hard all by myself. I approached Paz (a Jewish student) and told her that I would take responsibility for the presentation in addition to my portion of the work. Furthermore, I connected with Michal (a Jewish student) who promised that we would sit together to make sure there were no linguistic errors. I was afraid that we would fight and that it wouldn’t succeed, but I was wrong. Aside from one student, we all worked and contributed to the assignment.

At the beginning of the process, five Arab interviewees felt they were one cultural group that was competing against another—the majority. As such, they felt that they had to promote their members by giving them higher assessments, as clarified by Iyat (Muslim female), “I had to offer [honest] feedback to my friends, and I felt uncomfortable doing that because I was studying [in the course] with a friend whom I knew from school. It was hard for me.” These testimonies exemplify the issue of bias during the assessment process stemming from social considerations. Vered and Rachel, (Jewish females), added: During the first peer assessment, when the group met to discuss the assessments, the Arab students had difficulty giving low grades to their friends from the village. They gave them assessments that were higher than the actual quality of their work. We [Jewish students] had disagreements. We worked according to the criteria.

It may be inferred that during the first peer assessment, the obligation to one’s ethnic group received more weight among most of the Arab interviewees. However, after gaining more familiarity with other students’ work methods in heterogeneous groups, Arab students’ critique of their classmates increased.

Discussion This chapter exemplifies Reynolds and Trehan’s (2000) main assertion that simply to exchange one set of power relations (teacher-student) for another (student–student) does not of itself guarantee equality. As seen in the students’ reports, peer assessment brings to fore a new set of complex power relations which need to be understood, i.e., those that exist among students of diverse backgrounds and cultures. These relations may stem from academic skills that are not shared equally by Arab and Jewish students as they enter college. These essential twenty-first century skills are typically nurtured by constructivist learning environments, which are in reality rarely practiced in Israeli Arab schools. These schools have been criticized for not keeping in step with the challenges of modern education, while opting instead to perpetuate the traditional ‘banking’ view of one-directional teacher-student instruction (BenRabi and Hanadin 2013). As a result, the Arab students reported that they lack such

Discussion

101

essential learning skills as collaborating, persuading, classifying, and arguing. The key to narrowing the wide gap between the two student sectors lies in revamping the instructional benchmarks in minority schools and replacing them with up-to-date twenty-first century approaches. One of the ways to narrow the gap is to enhance the professional level of teachers working in schools located in Arab population areas. In addition to the critical discrepancies in socio–economic conditions, infrastructures, and budgets, there are also claims that these segregated populations have different professional standards for hiring teachers, and this may be responsible for insufficient learning outcomes. Miaari (2015), an Israeli Arab researcher, who specializes in the inequality and conspicuous social disparities in non-Jewish-sector education, claims that high professional standards, first and foremost, in the appointment and placement of principals and teachers, are lacking. The solution to this would call for transforming the mechanisms for appointing and placing principals and teachers into ones that are professional, transparent, and capable of withstanding criticism. A central finding of this study is related to groups’ dynamics in a cultural context. Initially, the obligation toward their cultural group held more weight among the minority group. However, after several attempts—some of which were unsuccessful—working together in homogeneous groups and, consequently, gaining more familiarity with other students and work methods in heterogeneous groups, minority students’ critique of their classmates increased. It is reasonable to assume that increasing familiarity between the two groups (Arabs and Jews) resulted in a greater willingness on the part of the Arab students to critique their peers and assess their capabilities according to more professional and less subjective criteria. Hence, it can be inferred that activities in heterogeneous groups combined with proper guidance can promote critical thinking among students towards their peers and reinforce their preference of unbiased considerations over subjective ones. These results can further be attributed to the “peer effect”, namely “the positive outcome derived from the fact that students with learning difficulties benefit from sharing the educational space with their more advantaged peers” (Council of Europe 2017, p. 13). Research in this field emphasizes that heterogeneity in learning levels is positive, especially for non-native students, while the observable “loss” derived from heterogeneity for native students is marginal. As stated by Tonello (2016), these non-native students may be academically disadvantaged either because they are immigrants entering a new, unfamiliar education system or because they need to learn a new language; therefore, they require special or extra attention. In this context, inter-group social interactions may play a crucial role in narrowing the gap that exists between non-native and native students.

Implication and Recommendations An important implication is related to the teacher’s role in the peer assessment process. The students felt that for professional reasons the lecturer should assume a

102

6 Peer Assessment in Multicultural Classrooms …

central role in the assessment process. Based on their reports, the full endorsement of peer assessment should be rejected. It seems that the teacher’s place in the process is deemed invaluable especially by minority students. However, as students’ faith in themselves and their peers as being capable of offering fair and professional feedback increases, their perception of the teacher as playing the prominent role in assessment may diminish. Thus, as argued in the introductory chapter of this book, this finding underscores the need for designing a curriculum that supports instructors in becoming more culturally aware and sensitive. Educators should understand that there is not a one-size-fits-all answer to improving learning outcomes; they should strive to deepen their knowledge of their students’ needs and adapt their work methods to meet those needs. The present study emphasizes that when applying alternative assessment methods, it is necessary to recognize the students’ learning challenges. It appears that some of the students needed a significantly more emotional support in order to boost their perception of themselves as being capable of successfully assess others. Most importantly, peer assessment requires placing an emphasis upon strengthening assessment skills over a period of time. This will eventually contribute to the students’ perception of their role in assessing their peers and will increase their perception of themselves as being capable of offering a fair and professional assessment. Lastly, it appears that students who come from collectivist societies are confronted with a greater challenge when attempting to offer a fair assessment to their peers since social considerations often outweigh professional judgment. Therefore, activities in heterogeneous groups combined with proper guidance should be offered by teachers, to promote critical thinking among students towards their peers and encourage the preference of professional considerations over subjective ones. In summary, the findings of this study can aid in understanding the factors that contribute to accepting this change; these include reinforcing students’ positive perceptions of their ability and that of their peers to offer fair and unbiased feedback. As students’ faith in themselves and their peers increases, their perception of the teacher as playing a prominent role in assessment diminishes (Alt and Raichel 2020). Teachers should also acknowledge, however, that effectively implementing peer assessment in heterogenous classrooms necessitates iteration, appropriate scaffolding, constant monitoring, and adaptation to the students’ different needs. This chapter focused on the tensions and opportunities arising from the implementation of peer assessment. One of the most critical sources of tension was related to collaboration among students of different cultures. Therefore, the following chapter is dedicated to illuminating collaborative learning in a diverse, multicultural setting. In our view, its insights elaborate and compliment the implications of the current study.

References

103

References Alt, D., & Raichel, N. (2018). Lifelong citizenship: Lifelong learning as a lever for moral and democratic values. Leiden and Boston: Brill and Sense Publishers. Alt, D., & Raichel, N. (2020). Higher education students’ perceptions of and attitudes towards peer assessment in multicultural classrooms. Asia-Pacific Education Researcher. Ben-Rabi, D., & Hanadin, E. (2013). Preparation for successful integration into higher education of Arab students: An evaluation study of a program to improve pre-academic preparation. Jerusalem: The Board of Higher Education. (Hebrew) Bryant, D. A. & Carless, D. R. (2010). Peer assessment in a test-dominated setting: empowering, boring or facilitating examination preparation? Educational Research for Policy and Practice, 9, 3–15. Council of Europe. (2017). Fighting school segregation in Europe through inclusive education. Council of Europe: Commissioner for Human Rights. Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (5th ed.). Thousand Oaks, CA: Sage. De Grez, L., Valcke, M., & Roozen, I. (2012). How effective are self- and peer-assessment of oral presentation skills compared with teachers’ assessment? Active Learning in Higher Education, 13(2), 129–142. Giroux, H. A. (1988). Teachers as intellectuals: Toward a critical pedagogy of learning. Massachusetts: Bergin & Garvey Publishers, Inc. Hsia, L. H., Huang, I., & Hwang, G. J. (2015). A web-based peer-assessment approach to improving junior high school students’ performance, self-efficacy and motivation in performing arts courses. British Journal of Educational Technology, 47(4), 618–632. Miaari, S. (2015). What is the problem of schools in the Arabic sector? Themarker. Retrieved from https://www.themarker.com/opinion/1.2719625. (Hebrew) Montenegro, E., & Jankowski, N. A. (2017, January). Equity and assessment: Moving towards culturally responsive assessment (Occasional Paper No. 29). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Nayir, F., Brown, M., Burns, D., O’hara, J., Mcnamara, G., Nortvedt, G., Skedsmo, G., Silje Kristin Gloppen, S. K., Eline F. & Wiese, E. F. (2019). Assessment with and for migration background students-cases from Europe. Eurasian Journal of Educational Research, 79, 39–68. Panadero, E. (2016). Is it safe? Social, interpersonal, and human effects of peer assessment: A review and future directions. In G. T. L. Brown, & L. R. Harris (Eds.), Handbook of social and human conditions in assessment (pp. 247–266). New York: Routledge. Panadero, E., Jonsson, A., & Strijbos, J. W. (2016). Scaffolding self-regulated learning through selfassessment and peer assessment: Guidelines for classroom implementation. In D. Laveault, & L. Allal (Eds.), Assessment for Learning: Meeting the challenge of implementation (pp. 311–326). Boston, MA: Springer. Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35, 435–448. Reynolds, M., & Trehan, K. (2000). Assessment: a critical perspective. Studies in Higher Education, 25(3), 267–278. Tonello M., 2016, Peer effects of non-native students on natives’ educational outcomes: mechanisms and evidence. Empirical Economics, 51, 383–414. Topping, K. J. (2017). Peer assessment: Learning by judging and discussing the work of other learners. Interdisciplinary Education and Psychology, 1(1), 1–17. Topping, K. J. (2019). Using peer assessment to inspire reflection and learning. UK: Tylor and Francis. Van Zundert M., Sluijsmans D., & Van Merriënboer, J. (2009). Effective peer assessment processes: Research findings and future directions. Learning and Instruction, 20(4), 270–279.

104

6 Peer Assessment in Multicultural Classrooms …

Webb, M. (2012). Beginning teacher education and collaborative formative e-assessment. In S. Hatzipanagos & R. Rochon (Eds.), Approaches to assessment that enhance learning in higher education (pp. 107–128). London and N.Y.: Routledge. Wu, K., Davison, L., & Sheehan, A. H. (2012). Pharmacy students’ perceptions of and attitudes towards peer assessment within a drug literature evaluation course. American Journal of Pharmaceutical Education 76(4), 62. Zhou, J., Zheng, Y., & Tai, J. H-M. (2019): Grudges and gratitude: the social-affective impacts of peer assessment. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/026 02938.2019.1643449.

Chapter 7

Assessing Collaborative Learning in a Culturally Diverse Classroom

Introduction Given the substantial cultural segregation within K-12 schools in Israel, and increasingly in Europe as well (Council of Europe 2017), attending college, for most minority students, plays a unique role in promoting opportunities for meaningful interactions across cultures (Halabi 2017) and provides an important context for shaping young adults’ emerging long-term attitudes, values, and behaviors. One of the methods offered to support this role is ‘collaborative learning’ defined as “a reacculturative process that helps students become members of the knowledge communities whose common property is different from the common property of knowledge communities they already belong to” (Bruffee 1993, p. 3). In this chapter, we describe a collaborative group-learning activity designed for a multicultural learning environment (de Hei et al. 2018) and analyze multicultural students’ experiences when collaborating with students from cultural backgrounds different from their own. This chapter attempts to provide higher education teachers with useful information for designing, implementing, and assessing appropriate collaborative learning experiences with multicultural populations.

Collaborative Learning for Meaningful Interactions Across Cultures Collaborative learning is based on social constructivism, which holds that learning is acculturation into knowledge communities. Indeed, collaborative learning can enhance openness toward and appreciation of diversity; however, attaining these learning outcomes depends, in large part, on the design of the group activity and its proper implementation. Such activities should be focused on the collaborative learning process rather than on the product (de Hei et al. 2019). © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_7

105

106

7 Assessing Collaborative Learning in a Culturally …

As noted by Smith and MacGregor (1992), collaborative learning is an active process whereby the student assimilates the information, actively engages with his/her peers, processes and synthesizes information, and benefits from interacting with people of varied backgrounds. In collaborative learning environments, learners are provided with opportunities to participate in multicultural learning communities, ask questions, share knowledge, and create interest-based networks that promote the development of their shared ideas and projects. This element includes an emotionalsocial aspect (de Hei et al. 2019) deriving from placing an emphasis on developing sensitivity to various communities and appreciating the diversity of a multicultural society. This type of learning develops mutual respect, caring, social responsibility, acceptance, and solidarity towards a variety of (ethnic, social, cultural, sectoral, national etc.) individuals and groups. The goal is to solve particularly complicated problems that gain from having multiple groups’ views and perspectives (Alt and Raichel 2018; Kezar and Holcombe 2020). Drawing on the constructivist approach to learning, collaborative learning highlights learning as a social activity affected by internal personal characteristics as well as by external social elements. Knowledge is built through interaction between one’s existing knowledge and social situations (Vygotsky 1978). This principle underscores the collaborative nature of the learning process, the goal of which is to promote dialogical thinking, and has been described in terms of transferring the learners to a space of discourse. Discussions could lead to active processing of information and reprocessing of ideas, thereby helping students learn better, retain information longer than working alone, and enhance their academic achievements. The process includes furthering the growth of research communities through the use of spaces that facilitate expressing an alternative voice and recruiting students to engage in real dialogue in a varied and multicultural society (Schwarz and de Groot 2011; Snowman and Biehler 2006). Collaborative learning can take place in a classroom, where practices of discussion and teamwork transpire, or outside the classroom, i.e., in online or other forms of learning environments (Alt 2014). It might be deemed a long-term process in which the learners gradually become part of an active community. At first, the learners play a marginal role, but with experience and practice, they take on additional commitments. Such experiences may shape students’ long-term attitudes, values, and behaviors provided that the activities indeed require and generate actual collaboration that could not have been accomplished by dividing up the work between the individual participants; and in addition, provided that the problem at the core of the activity is complex and productive (Hesse et al. 2015). Researchers (de Hei et al. 2019) assert that collaborative learning can lead to (shared) knowledge acquisition, higher student motivation, improved meta-cognitive skills, and social/collaborative skills. However, they also emphasize that attaining these learning outcomes depends on the design of the group activities by the teacher, who has a unique role in this learning process, especially in multicultural environments, where more salient cultural differences exist among students (GregersenHermans 2016). Teachers should model interculturally competent behavior, help

Collaborative Learning for Meaningful Interactions …

107

students manage their group work by assuring meaningful interaction and engagement in the group work, by creating a safe and inclusive learning environment, and by stimulating social interaction. The latter role requires a learning environment in which the students practice tasks that spur positive interdependence, regular group evaluations and personal reflection on their behavior and communication, and social activities related to the assignment (de Hei et al. 2018).

Minority Students and Collaborative Learning Because of the recent status of Europe as a growing destination for immigrants, migrants, and refugees, issues of integration have become a reality also in countries that were ethnically homogenous just a few decades ago (Jutengren and Medin 2019). According to Tonello (2016), non-native students often start the school path in the host country with little or no knowledge of the language of instruction and may be academically disadvantaged and need special or extra attention from their teachers. The educational disadvantage experienced by those students is also affected by the new environment and the institutions they encounter; hence “social interactions may play a crucial role in narrowing the existing gap—integrating nonnative with native students—or exacerbating it—if they engage in self-clustering or rejection behaviors” (p. 384). This assertion has been discussed in previous studies that addressed the nature of the interaction between majority and minority students. For example, Jiosi and Zalmanson-Levi (2018) observed that college students usually interact only with their own clique unless otherwise required. Without guidance, most minority students’ social and academic interactions occur in and among themselves. Because of a sense of closeness, both in language and culture, these students seek out each other’s company to elevate their sense of belonging, as the national group is the only place they feel at home (Halabi 2017; Severiens and Wolff 2008). Tutri (2009) noted that this manner of conduct has implications for these students’ participation in class. For example, participation in classroom discussions is perceived as asymmetrical; that is, whereas most ‘majority students’ participate in classroom discussions unhindered, few minority students do so. This is partly rooted in their difficulty to articulate their thoughts in the language of instruction, and partly in a minority’s concern about openly expressing opinions in a setting in which they do not feel fully equal. Current studies in Israel (Jiosi and Zalmanson-Levi 2018) have also shown that majority students are less willing than Arab minority students to cooperate in multicultural educational activities. These gaps stem, among other things, from the low degree of familiarity between the groups in numerous areas such as culture, customs, etc. Insufficient familiarity with the ‘other’ may potentially lead to forming negative attitudes and stereotypes toward the ‘other’ group, its culture, language, and those that speak it. Other studies (Boymel et al. 2009) found that the only activities which minority students shared with majority students were instrumental, i.e., related to their studies, and that they usually avoided contact that extended beyond that.

108

7 Assessing Collaborative Learning in a Culturally …

Designing a Collaborative Group-Learning Activity in a Diverse Classroom With the growing attention paid to academic skills such as collaboration and communication as key elements among the many success factors involved integration, attaining adequate levels of those skills has become crucial for attaining further education (Choi and Ziegler 2015). A central question in this context is how to design a learning environment that is optimally suited to collaborative work in a multicultural classroom and how to evaluate it (Alt and Raichel under review). The following case study outlines a group-learning activity in a diverse, multicultural class attended by 86 first-year Education students (90% females) from one major college located in Israel. The students were enrolled in an introductory course entitled: ‘The Education System in Israel’. The distribution in terms of ethnicity was 24% Jewish students; 76% Arab (Muslim) students, with a mean age of 22.71 (SD = 6.28) years. The Muslim group students were significantly younger and with a reported lower socioeconomic level than that of the Jewish students. Throughout the first semester of their first academic year, the students were required to take part in a group-learning activity in a course which focused on the educational system policy and its implications for learning. This course was selected because it best fits process-oriented course learning outcomes such as collaboration. Following the GLAID (Group-Learning Activities Instructional Design) framework (de Hei et al. 2019) the course was re-designed to include three key components: (1) interaction; (2) learning objectives and outcomes; and (3) assessment criteria. Students’ interaction featured investigating, planning, and ultimately presenting an educational reform, such as autonomy in school management, ICT, integration in education, cost-free education, state education, the involvement of non-profit organizations in education, etc. During the process, they were required to reach a consensus on an approach with which to work on the assignment, find and present information from both theoretical research and the field, integrate theory with material gathered from the field, and demonstrate their knowledge of the reform by explaining their findings initially to the group members and, later, to the course participants. Learning objectives and outcomes were determined before implementation of the course. Collaboration skills were made explicit in the course description for the students. Means and criteria of assessment were constructed together with the students at the beginning of the course. The rubrics designed were intended to ascertain whether students had clearly communicated the theme, background, research methods, educational reform implementation, and resources and to determine whether all group members participated meaningfully. Next, components of didactics and logistics were developed. Regarding didactics, task characteristics were determined, including the activity description, frequency of group meetings and reporting procedure tools. Furthermore, the interaction among group members was documented in a group reflective journal. Each group submitted

Designing a Collaborative Group-Learning Activity …

109

four entries during the course. This process included receiving and giving peer feedback among the multicultural group members as to the quality of their contributions. Guidance was given by a teacher and a teaching assistant, who also reviewed the reflective journals and gave written and oral feedback on the quality of the students’ evaluation of the group meetings, and when needed, provided guidance for effective collaboration. Logistics included two components, (1) group constellation—the teams (of five students) were assembled by the teacher to be as culturally diverse as possible including Arab minority and Jewish students, and (2) facilities—the group worked in several locations including primarily their classroom and the school library. In addition, the students were required to attend weekly group-work sessions which were not teacher-guided. The total amount of group-work expected of students was approximately three hours per week. After the implementation, groups’ products and performance were evaluated using the previously designed criteria.

Assessing Perceptions Towards Collaborative Learning In this chapter we sought to illuminate multicultural students’ perceptions of the group dynamics throughout the group-learning activity. Qualitative semi-structured interviews were conducted with a total of 14 students, of whom seven were Arab Muslim students, so as to learn how the participants perceived this activity: what the significance of multicultural shared work was for them, how they felt about the work processes and products, and how they thought the collaboration could be improved. The students of both cultures described in their own words the group dynamics throughout the group-learning activity. The qualitative data analysis process involved examining the gathered data and generating themes from the data (Creswell 2014). Content analysis of the students’ interviews revealed three main themes: (a) Lack of language proficiency and uneven workload distribution; (b) coping with divergent cultural values, and (c) determining ‘who benefits from the fruits of collaboration.’

Lack of Language Proficiency and Uneven Workload Distribution During the group-learning activity, the students were required to read articles in English. Six Arab students described their difficulty in reading the articles, the time it took them to complete this task, and the quality of the reading products. Four Jewish interviewees reported an expansion in their workloads that engendered negative feelings within the group overall. Abir (Muslim female) described her experience and expressed her understanding of her Jewish teammates’ negative feelings.

110

7 Assessing Collaborative Learning in a Culturally …

I told Ronit [a Jewish student] that it was hard for me to understand the article. I asked her if we could sit together in the library and read, and if I didn’t understand a word, I could ask her. We worked together, but I felt that I was wasting her time and making slow progress. When I told her I felt uncomfortable, she took on my articles too. I could see she was trying not to get angry.

Ronit (Jewish female), on her part, described her feelings about Abir: I felt sorry for Abir. We sat in the library for two hours, and I realized from her questions that she didn’t understand anything. What kind of summary would she bring to the group? What could we learn from it? Although I had a lot to do, I took on her article too. How did they– the lecturer, the college, whoever - expect the minority students to cope with reading articles in languages that were hard for them?

A lack of command of the Hebrew language was also evident in the issue of the presentation, which was one of the products of the group work, and in Nur’s (Muslim female) words, There were mixed sensations of fear, nervousness, and pressure. We wanted to make our presentations without having to consult a page of notes like everyone else and we prepared to do that, but at the last minute we broke down. We wrote all our notes down on a page so that we wouldn’t get confused. Perhaps we should have put more trust in ourselves…

The Jewish students described the ‘entreaties’ of their Arab friends that they make the presentation to the class and exempt them from having to stand in front of an audience. Amit [Jewish male] stated that: Aya [a Muslim student] asked me with tears in her eyes not to take part in the presentation to the class with excuses such as ‘I’m shy’, ‘I have stage fright’, etc. Her excuses didn’t convince me, because the atmosphere in class was very good and accepting. She then used not knowing the language as a reason: ‘I speak with mistakes; I won’t be understood. Why don’t you present instead of me…?’ I felt sorry for her. I tried to convince her. I said that she could learn by experiencing it, and that the first time was the hardest and then it got easier. My urging didn’t work. I finally agreed to present her part, too…

Whereas for the minority students, standing in front of an audience was distressing, those Jewish students who had an emotional barrier to speaking to an audience saw this initial experience as a challenge that must be overcome. Nira (Jewish female) told us:” I have stage fright, and I get very agitated standing before the class. Sometimes I stutter, and that doesn’t boost my confidence. I knew that this was a skill I had to master. So, I didn’t dare think that someone would present my part for me”.

Coping with Different Cultural Values Jewish and Arab students alike mentioned aspects that hindered their work together. The most obvious one was responsibility. Five Jewish students described several instances in which they had scheduled a meeting at the college or a virtual meeting to work on the project, and their Arab colleagues failed to show up. The recurrent

Coping with Different Cultural Values

111

explanation for this behavior was related to family commitment that outweighed academic commitment, including their commitment to the work group, as described by Nira (Jewish female), We scheduled a meeting in the library to decide on the research question that would guide the work and to distribute the materials among us. We all agreed on the day and time, and Jamila didn’t show up… When I saw her in class, I was angry with her. She was offended. She said that her mother was ill, and she was more important than our work. Okay, I’m sorry her mother is ill. But why can’t you pick up the phone and explain? Why not try to find out later about your part? She didn’t come to the presentation either, without informing us. I understood from her that she was the eldest sister, and she had responsibilities in the home. What about her responsibility to her group?

Jamila (Muslim female) described her contribution to the group work; however, when asked about her commitment to this collaborative work, she explained that she couldn’t be at the other meetings because “my sister was ill. My family is very important to me, more than my studies. Some days I couldn’t be there. I missed classes, too. I explained this in the group; I think they understood me”.

Who Benefits from the Fruits of Collaboration? In principle, all the Jewish and Arab students mentioned their openness and willingness to know ‘the other’, their curiosity about cultures and lifestyles different from their own, and their belief that they should study and work together. However, whereas the Arab students felt that they had learned and profited from the shared work—to quote Fatma (Muslim female), “I enjoyed working in a group. The Jewish students helped me when it was hard, pushed me to cope, and believed in me. I hope we continue to work in mixed groups”—the Jewish students were disappointed, as described by Noa (Jewish female): I started out happily. I invited everyone to my home; it was a pleasant meeting. We got to know each other and divided up the work. I thought it would be interesting and empowering. That was the only meeting everyone came to. From then on, all of the work fell on Amit and me, virtually everything. The phone calls, the willingness to explain, the attempt to find time for meetings that were convenient for all of us – nothing helped. Some materials arrived [from the minority students] here and there. In class, after the lesson, we talked about how the work was not the way I had wanted it to be. And I admit, the grade, which was eventually given to the entire group, mattered to me too.

Discussion The key concern emerging from this experience is related to the majority students’ views regarding collaborative learning. Tensions were detected between the cultural groups, one of them stemming from the minority students’ insufficient command of the Hebrew language that ultimately resulted in an unequal distribution of the

112

7 Assessing Collaborative Learning in a Culturally …

workload. The Jewish students felt they had carried more of the burden than their minority counterparts. Some of the Jewish interviewees sympathized with their minority friends and attested to helping them more than necessary. Others blamed the institution for setting overly-high expectations of the minority students, who were supposed to be as proficient in Hebrew as those whose mother tongue is Hebrew, despite these expectations being unrealistic. In addition, the gap in life experience between the cultural groups affected the group dynamics. Whereas most minority students start their academic studies directly after high school, Jewish students start much later following military service and the socially-accepted practice of working and saving money for traveling abroad before undertaking academic studies. Hence, for minority students, entering college necessitates a rapid maturation process on several fronts, beyond their academic studies, such as acquiring adult life skills that involve responsibility to one’s peers, time management, withstanding pressures etc. The students who were interviewed gave pertinent examples of these challenges; speaking before an audience, taking responsibility for group work, attending meetings on time, and adhering to prearranged schedules were all found to be relatively new and thus more challenging for the minority group. An additional aspect that emerged from the interviews was the divergent cultural values of the groups, whereby, in the minority group, family commitment outweighed academic commitment, including one’s commitment to the study group. As shown in previous research (Abu-Baker 2005), in contrast to the individualist orientation espoused by the Israeli Jewish sector, the Israeli Arab family emphasizes, in conformity with the Islamic tradition, a close-knit collective orientation. This tight web of kinship shapes their understanding of themselves and of the surrounding world, providing solidarity, help, and unwavering support to their family members.

Implication and Recommendations Indeed, college students need to be prepared for a global environment, and designing collaborative activities is critical to this preparation. However, as shown in this study, given the formidable challenges in fostering intercultural competencies in higher education, mere participation in group-learning activity will not automatically foster positive attitudes toward collaboration nor lead to profound interpersonal engagement. To achieve this outcome, special attention should be devoted to the design and implementation of the group-learning activity, with a focus on the process and intended learning outcomes. This process pertains to the nature of how students work together and is influenced, as shown in this study, by how they perceive collaborative work. Therefore, in view of the current study’s findings, it is recommended that the design of any group-learning activity should include three additional elements. The first element refers to special scaffoldings that should be provided to minority students throughout the group work.

Implication and Recommendations

113

It is recommended to establish a support system which might help them to develop academic skills such as critical reading of academic articles in their second (Hebrew) or third (English) language, writing or reading papers, and giving oral presentations of academic work in the language of instruction. In addition, the support system should deal with life skills such as fulfilling one’s responsibility, time management, and withstanding pressures. The second element pertains to the group level, and specifically to the structure of the group, which needs to be carefully designed. Though mainly used in studies investigating schoolchildren with disruptive behavior, our findings may be interpreted in reference to the uneven models of interaction between native and nonnative students, as presented by Tonello (2016). Keeping in mind the high proportion (70%) of minority students in the current study, a central inference might be related to this disproportionate share of minority students in the groups, which, in turn, could account for the reported negative effects. Such an imbalance might harm both groups of students and negatively impact the overall learning process in the class, “Whenever non-native students become prevalent enough to form a ‘critical mass,’ the native type could reject them because the effort of integration becomes too high… Natives may be willing to make sufficient effort to include a few minority members but unwilling to make the effort to include numerous non-native schoolmates (Tonello, p. 392). The third element pertains to the classroom diversity climate (Hofhuis et al. 2016). In group-learning, activities should be designed with a special focus placed on openness toward and appreciation of cultural differences. Teachers should communicate and discuss the concept of ‘diversity’ with the students in preparation for the activity. They should encourage students of both cultural groups to freely and openly discuss actual or foreseen academic, social, or emotional challenges they may encounter during the collaborative learning and propose ideas and suggestions to remove these barriers. In summary, it appears that encouraging collaborative work among students in multicultural higher-education settings may be challenging and perhaps imposes greater liability on educators to design inclusive, collaborative activities that advance openness and tolerance towards differences. To properly address this task, several elements should be considered when designing such activities: on the individual level—it is the goal of the teacher to ensure that each and every student learn effectively, with special attention devoted to minority students’ needs; on the group level—at the outset, the number of minority students in each group should be more balanced, to ease the effort undertaken by majority-group students in supporting their minority-group counterparts and improve overall group performance; and, finally, on the class level—whole-group discussions should incorporate methods to enhance diversity-positive attitudes, trust, fairness, and open communication.

114

7 Assessing Collaborative Learning in a Culturally …

References Abu-Baker, K. (2005). The impact of social values on the psychology of gender among Arab couples: A view from psychotherapy. Psychiatry Related Science, 42, 106–115. Alt, D. (2014). The construction and validation of a new scale for measuring features of constructivist learning environments in higher education. Frontline Learning Research, 2(3). https://doi.org/10. 14786/flr.v2i2.68. Alt, D., & Raichel, N. (2018). Lifelong citizenship: Lifelong learning as a lever for moral and democratic values. Leiden and Boston: Brill and Sense Publishers. Boymel, Y., Zeevi, A., & Tutri, M. (2009). What do Jewish and Arab students learn from each other? Report of the Research and Evaluation Authority. Kiryat Tivon: Oranim College of Education. (Hebrew) Bruffee, K. A. (1993). Collaborative learning: Higher education, interdependence, and the authority of knowledge. Baltimore: Johns Hopkins University Press. Choi, J., & Ziegler, G. (2015). Literacy education for low-educated second language learning adults in multilingual contexts: The case of Luxembourg. Multilingual Education, 5(1), 1–21. Council of Europe. (2017). Fighting school segregation in Europe through inclusive education. Council of Europe: Commissioner for Human Rights. Creswell, J. W. (2014). Research design. Qualitative, quantitative, & mixed method approaches (4th ed.). Los Angeles: Sage Publications. de Hei, M. S. A., Admiraal, W. F., Sjoer, E., & Strijbos, J. W. (2018). Group learning activities and perceived learning outcomes. Studies in Higher Education, 43, 2354–2370. de Hei, M. S. A., Tabacaru, C., Sjoer, E., Rippe, R., & Walenkamp, J. (2019). Developing intercultural competence through collaborative learning in international higher education. Journal of Studies in International Education. https://doi.org/10.1177/1028315319826226. Gregersen-Hermans, J. (2016). The impact of an international university environment on students’ intercultural competence development (Doctoral dissertation). Retrieved from https://www.researchgate.net/publication/313388295_Intercultural_Competence_Develo pment_in_Higher_Education. Halabi, R. (2017). Graduate Arab students in a teacher education college: Guests or welcome guests? Megamot: Interdisciplinary Journal of Social Sciences, 52, 141–162. (Hebrew) Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A Framework for teachable collaborative problem solving skills. In P. Griffin, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 35–56). Netherlands: Springer. Hofhuis, J., van der Rijt, P. G. A., & Vlug, M. (2016). Diversity climate enhances work outcomes through trust and openness in workgroup communication. SpringerPlus, 5, 714. https://doi.org/ 10.1186/s40064-016-2499-4. Jiosi, V., & Zalmanson-Levi, G. (2018). Multiculturalism and co-living between Jews and Arabs in teacher training academic institutions. Recommendations document. Tel-Aviv: The Association for Civil Rights in Israel. (Hebrew) Jutengren, G., & Medin, E. (2019) Cross-ethnic friendship and prosocial behavior’s potential significance to elementary children’s academic competence, The Journal of Educational Research, 112, 38–45. Kezar, A., & Holcombe, E. (2020). The role of collaboration in integrated programs aimed at supporting underrepresented student success in STEM. American Behavioral Scientist, 64, 325– 348. Schwarz, B., & De Groot, R. (2011). Breakdowns between teachers, educators and designers in elaborating new technologies as precursors of change in education to dialogic thinking. In S. Ludvigsen, A. Lund, I. Rasmussen & R. Säljö (Eds.), Learning across sites: New tools, infrastructures and practices (pp. 261–277). London: Routledge. Severiens, S. & Wolff, R. (2008) A comparison of ethnic minority and majority students: social and academic integration, and quality of learning, Studies in Higher Education, 33, 253–266.

References

115

Smith, B. L., & MacGregor, J. T. (1992). What is collaborative learning? In Goodsell, A. S., Maher, M. R., & Tinto, V. (Eds.), Collaborative learning: A sourcebook for higher education (pp. 9–22). National Center on Postsecondary Teaching, Learning, & Assessment, Syracuse University. Snowman, J., & Biehler, R. (2006). Psychology applied to teaching (11th ed.). Boston: Houghton Mifflin. Tonello, M. (2016). Peer effects of non-native students on natives’ educational outcomes: mechanisms and evidence. Empirical Economics, 51, 383–414. Tutri, M. (2009). Arab student needs in a college of education and the extent of their absorption into it. Report of the Research and Evaluation Authority. Kiryat Tivon: Oranim College of Education. (Hebrew) Vygotsky, L. S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.

Chapter 8

Culturally Responsive Feedback

Introduction Research into assessment feedback has led to a debate as to the precise definition of the term. The term assessment feedback may include “all feedback exchanges generated within assessment design, occurring within and beyond the immediate learning context, being overt or covert (actively and/or passively sought and/or received), and importantly, drawing from a range of sources” (Evans 2013, p. 71). Feedback information can be received in the form of handwritten or typed comments on assessment tasks, be they written/verbal, specific/general, group/individual, or can be given faceto-face, in rubrics, using automated systems, or through creative uses of technology, as well as via peer- and self-evaluation modes of feedback (Ball 2010; Deeley 2018; Huisman et al. 2019; Rowe and Wood 2008; Ryan and Henderson 2018). Feedback can also be relayed by offering a conclusive evaluation of the quality of reflections made in the journals (Bruno and Dell’Aversana 2017). However, despite a growing body of research engaging in feedback for students, there appears to be a paucity of literature which comprehensively conceptualizes or details its practical application for multicultural students. To fill this lacuna, this chapter depicts several contemporary aspects of feedback (both seeking and giving) in higher education, as perceived by students and teachers in diverse, multicultural higher-education learning environments, and discusses ways in which feedback-giving can be adapted to students of different cultures.

Feedback in Higher Education It is widely acknowledged that feedback is a core component of the assessment process in higher education and an integral aspect of the learning experience, witnessed by an increase in scholarly attention paid to this domain. Feedback is known to exert considerable influence on achievement and to enable students to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_8

117

118

8 Culturally Responsive Feedback

become independent learners “who are able to monitor, evaluate, and regulate their own learning… beyond graduation” (2013, p. 75). Yet, higher education institutions are being criticized for inadequacies in the feedback they provide to students. For example, Bell and Brooks (2018) argue that feedback tends to be poorly understood and executed in higher education. Their study in the UK showed that dissatisfaction with feedback is one of the most problematic aspects of the student experience, arguing that “students appear to place most emphasis on teaching, course organisation and personal development with very little on assessment and feedback” (p. 1138). Furthermore, in terms of formative assessment, it has been argued that although constructivist approaches to learning have led to shifts in conceptions of teaching and learning within higher education, this shift has been slower to emerge (Evans 2013). Ball (2010) maintains that by definition the word “feedback” can be mistakenly associated with the teacher-centered approach, i.e., the one-way transmission of information from teacher to student, as it invokes connotations of passivity and dependence, “Feedback is a process by which the students can have their work examined by a tutor who is experienced in the demands of the academic environment… Given this, the student borrows the perceptive skills of the lecturer” (p.139). However, Ball maintains that the degree to which the students are ‘mouths to feed’ depends on the student–teacher rapport. Ideally there should be a progressive shift towards student-centered approaches that enhance self-regulated learning whereby students learn by doing and applying their new knowledge. Boud and Molloy (2013b, pp. 6–7) strengthen this notion by associating feedback with the student-centered approach. Based on their definition, feedback is a “process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work.” In the feedback process, the student should be placed at the center of the activity; the crucial role of external standards shared with students should be recognized; it should be regarded as a continuous process and not as a single act; variation between the standards and the work itself should be appreciated; and it should be deemed necessary for the action. This definition poses a challenge to teachers who need to present feedback in a way that encourages and motivates the learner. Therefore, key factors are essential in learning and assessment, with a direct bearing on the student’s journey towards intellectual independence and motivation.

Effective Feedback for Learning Feedback can be seen as an end product, regarding aspects of one’s performance or understanding (Hattie and Timperley 2007), or as an integral part of learning (Cramp 2011). Drawing on the work of Ramaprasad (1983) and Sadler (1989), a functional aim can be attributed to feedback for students, namely, to bridge the gap between the actual level of performance and the desired learning goals or outcomes. Yet, despite the existence of guiding principles, bridging this gap effectively remains

Effective Feedback for Learning

119

a multifaceted and complex challenge, and there is no consensus with respect to the specific features of effective feedback. To tackle this challenge, Evans (2013) distinguishes between a cognitivist and a socio-constructivist view of feedback. The cognitivist perspective is closely associated with the traditional teacher-centered approach, whereby an expert supplies information to the passive recipient/s. Whereas, within the socio-constructivist paradigm, …feedback is seen as facilitative in that it involves provision of comments and suggestions to enable students to make their own revisions and, through dialogue, helps students to gain new understandings without dictating what those understandings will be… In such situations, interactions between participants in learning communities lead to shared understandings as part of the development of communities of practice… with the student taking increased responsibility for seeking out and acting on feedback (p. 71).

Nelson and Schunn (2009) maintain that effective feedback should be considered a motivator that increases a general behavior of the learner. According to Ryan and Henderson (2018), “students need to be amenable to the messages contained within the information in order to feel motivated enough to act upon them… when students experience adverse emotional reactions as a result of the feedback process, their receptiveness may be limited” (p. 881). Hence, negative emotional reactions might stem from critical feedback and may reduce students’ self-esteem and perceived selfefficacy. Effective feedback should consist of information used by a learner to change performance in a particular direction. It can be defined as information provided to students about their performance that includes written comments on assignments (Rowe and Wood 2008). In this context, research suggests that feedback is most effective when provided soon after task performance; that is, “effective feedback is based on assumptions that it is clear, meaningful and timely, and also that it is understood, engaged with and acted upon… However, feedback becomes redundant if it is returned too late for students to improve their performance in the next assessment” (Deeley 2018, p. 440). Based on this notion, several researchers (Carless et al. 2011) maintain that to render feedback effective in terms of sustainability, a broader definition of feedback should be embraced rather than the functional one alone; namely, “all dialogue to support learning in both formal and informal situations” (Askew and Lodge 2000, p. 1). This formative feedback should be viewed as part of interactive and dialogic pedagogy, which affords students opportunities to self-monitor their own work and increase their self-regulative capacities; that is, “the crux of the matter is how students interpret and use feedback. Possible ways of enhancing feedback processes involve viewing feedback more as dialogue than information transmission” (p. 2). Taking this perspective, Nicol (2010) argues that dissatisfaction with written feedback can be interpreted as “symptoms of impoverished and fractured dialogue” (p. 503).

120

8 Culturally Responsive Feedback

Barriers to Effective Feedback Ample explanations for the lack of those key factors essential for effective formative assessment feedback in higher education have been brought forth by several researchers (Boud and Molloy 2013a). These may be categorized based upon three main tenets: organizational aspects, teachers’ perceptions, and students’ experiences. The first tenet pertains to contemporary changes in higher education settings. For example, the rising number of students pursuing higher education has led to a corresponding increase in graded tasks (with no verbal feedback), which has greatly reduced the amount of information provided to students about their work (Boud and Molloy 2013a; Henderson et al. 2019a, b). Due to the growth in student numbers, “written comments have become detached from this supportive context. The result is a great deal of dissatisfaction with feedback by students and teachers, as evidenced in a number of surveys, research studies and reports” (Nicol 2010, p. 501). Moreover, within the constraints of modularized learning in higher education it is increasingly difficult to handle feedback effectively. The modularized structures have reduced the number of tasks given each week, which means that students gain less feedback on their practice and less systematic knowledge and awareness of their performance. The second tenet deals with the students’ feedback experiences. To better understand the barriers to effective feedback, a recent study (Henderson et al. 2019a, b) explored feedback challenges identified by university students and staff. The authors’ findings showed that for students, the need for feedback specificity was often framed around a desire for comments that are less generic and more individualized and feedback that specifically focuses on both the negative and positive aspects of the assessed work. In addition, students in that study noted an absence of constructive comments containing information that could be utilized to improve their future work. The last tenet pertains to the barriers perceived by staff members. Henderson et al. (2019a, b) point out several concerns, such as time as a limited resource as well as the constraints associated with the need to provide great quantities of detailed and personalized feedback in large classes. Additional themes related to difficulties in pointing out students’ weaknesses sensitively and to ascertaining whether the provided feedback comments are useful, and whether students are actually able to understand and benefit from them.

Barriers to Effective Feedback in Culturally Diverse Learning Environments While considerable literature exists that discusses the subject of feedback for students, only a small portion of previous research addresses the use of feedback in culturally diverse learning environments. Nonetheless, the few studies that have dealt with this aspect in higher education focused on the perspective of feedback seeking. In an early study of Sully de Luque and Sommer (2000), the authors assert

Barriers to Effective Feedback in Culturally Diverse …

121

that although research scholars have addressed the issue of an individual’s behavior in feedback-seeking activity, studies have overlooked the role of culture in this context. In their paper they presented a theory-based cross-cultural model of feedback-seeking behavior. In their model, the authors identified four cultural variables (syndromes) of feedback-seeking behavior; namely, specific-holistic orientation, tolerance for ambiguity, individualism-collectivism, and status identity. The focus of the specific-holistic syndrome involves the way in which a culture interprets the richness of relationships. Individuals in holistic cultures care about relationships and avoid losing face; therefore, less inquiry and more monitoring are displayed in holistic culture. Sully de Luque and Sommer (2000) postulate that feedback-seeking behavior by individuals shaped by a holistic-oriented culture is more likely to be affected by face costs (i.e., the amount of social value or respect that others grant the individual), whereas individuals shaped by a specific-oriented culture are more likely to be influenced by effort and inference costs, “that is, the resources necessary versus the potential for interpretation error” (p. 835). Drawing mainly on Heine’s work (2007), MacDonald (2008) reviewed several studies in crosscultural psychology which illustrate the specific-holistic syndrome. In general, her findings suggest that whereas Westerners are motivated to maintain or enhance a positive self-view, East Asians are predominantly motivated to be viewed favorably by significant others. Moreover, East Asians tend to avoid disclosing any information that could lead to loss of face. Concerns about saving face “may motivate feedback seekers to ask for negative performance information insofar as it gives them information that they can use to improve and maintain face with others” (p. 12). East Asians value and therefore seek negative feedback information also because negative (rather than positive) feedback is more consistent with the Eastern social norm of behaving modestly. In contrast, seeking positive feedback from one’s instructor or employer is liable to be regarded as immodest, even disrespectful, behavior that may result in an unfavorable evaluation. The tolerance for ambiguity cultural syndrome refers to the extent to which ambiguous situations are perceived as threatening and to which society members are open to change and innovation. In cultures with a low tolerance for ambiguity, there is an extensive reliance on rules and procedures. Therefore, individuals shaped by these cultures will tend to engage in greater feedback seeking, will be less influenced by feedback-seeking costs, and will perceive effort (and face) as less costly, given the benefits of averting risk and uncertainty. In contrast, cultures with high tolerance for ambiguity are more accepting of and less threatened by risk and uncertainty; therefore, in these cultures, individuals are not threatened by opinions and behaviors different from their own and feel less compelled to need or seek feedback (Sully de Luque and Sommer 2000). The most widely studied cultural dimension is the conceptualization of individualism-collectivism. In an individualistic culture, individual goals become the primary focus of behavior, whereas collectivism exemplifies a societal situation in which people share a mutual concern for each other (Triandis 1995). Sully de Luque and Sommer (2000) assert that in an individualistic environment, individuals will probably seek feedback as “a means of personal impression management…

122

8 Culturally Responsive Feedback

or ego protection” (p. 840). Feedback-seeking behavior in collectivist cultures will be oriented toward “the betterment of the group through improving accuracy and understanding the task” (p. 840). A culture with a high collectivist orientation will not encourage direct-inquiry behavior since it might draw too much attention to the individual. Instead, indirect inquiry and monitoring may be a preferred mode of seeking feedback. Hence, the authors proposed that, Cultures with more of a collective identity favor behavior in harmony with the group… Individuals shaped by an individualistic culture will more frequently use direct-inquiry feedbackseeking strategies. Individuals shaped by a collectivist culture will more frequently use monitoring and indirect inquiry feedback-seeking strategies (pp. 840–841).

In addition, several researchers (MacDonald et al. 2013) suggest that an emphasis on maintaining interpersonal relationships results in a greater social concern in public situations among individualistic-culture individuals in contrast to collectivistculture individuals. Therefore, the act of seeking individual feedback in public situations is expected to be less unsettling for individuals from Western cultures. “Overt expressions of individuality that cause an individual to ‘stand out’ from others are more consistent with the Western values of individuality and uniqueness… crosscultural research also demonstrates a stronger avoidance orientation and a great concern for others’ evaluations in public situations among East Asians as compared to Westerners” (MacDonald 2008, pp. 16–17). Ashford et al. (2003) maintain that this cultural dimension of feedback-seeking in a cross-cultural context is of relevance to organizational life around the globe, …what happens with feedback-seeking dynamics when an individualistic person works in a collectivist culture (or vice versa)? For example, consider a request by an individualistic person for feedback (to validate progress and prompt positive recognition) from collectivist colleagues. Both parties could find this transaction frustrating. For the collectivist colleagues, the feedback seeking may be viewed as selfish and rude; further, providing positive feedback may be viewed as a sign of weakness. The failure to receive positive feedback is likely to frustrate the individualistic person as the attempt to obtain performance recognition is thwarted and goal progress remains ambiguous. Conversely, a group of predominately individualists may develop doubts about the competence and engagement level of a collectivist colleague who seldom seeks nor gives feedback (p. 793).

The cultural syndrome of status identity, also seen in research on power distance, embodies the notion that “cultural members are stratified into categories or a hierarchy based on culturally salient criteria” (Sully de Luque and Sommer 2000, p. 841). High status-identity cultures imply a greater distance between supervisors and subordinates. In these cultures, feedback-seeking is lessened, and peers become more important sources of feedback than superiors and subordinates. Whereas in egalitarian cultures individuals are less receptive to power distance, individuals belonging to a lower status identity culture will seek feedback from superiors and subordinates more than from peers. Similarly, MacDonald (2008) argues that decisions to seek feedback from different sources may be explained by cultural status identity concerns. Whereas Western societies are characterized by low power distance, in Eastern societies power is distributed

Barriers to Effective Feedback in Culturally Diverse …

123

unequally; hence, subordinates feel reluctant to confront their superiors with questions because “it could be interpreted as an indirect criticism of the superior’s or the organization’s effectiveness” (p. 19). While Western individuals value supervisor’s feedback more than that of their peers because it may be seen as having more instrumental value than a peer’s evaluation, Eastern individuals prefer seeking feedback from peers because it may be seen as less disrespectful than to confront or question a person in authority. In the context of academia, Ryan and Henderson (2018) maintain that receiving feedback from an authoritative individual, such as a lecturer or tutor, is a complicated form of social interaction in which factors such as power, discourse, identity, and emotion may arise. In a hierarchical organization such as a university, students may feel personally slighted when a more knowledgeable individual points out insufficiencies in their work. As suggested by Carless (2006, in Ryan and Henderson 2018, p. 881) “the ideology of universities demarcates novice students from authoritative academics, and this is often reinforced by the role of correcting and judging scripts, rather than engaging in a debate on them.” With only limited studies engaging in feedback sought and provided in culturally diverse learning environments, the following interview analysis of undergraduate students and faculty sought to exemplify several contemporary aspects of feedback in higher education as outlined by students and teachers in a multicultural college located in northern Israel.

Feedback in a Multicultural Learning Environment: The Students’ Voice To delve deeper into the concept of feedback, its nature, its objectives, and how to obtain it, 18 first-year undergraduate students pursuing their degree in two multicultural teacher training colleges were interviewed for this chapter. Ten were Arab students (Muslim and Christians) and eight were Jewish. Six Jewish lecturers who taught both Jewish and Arab students at the two teachers’ training colleges were also interviewed about the feedback process from their standpoint. The lecturers discussed ways of coping with some of the problems that arose among the minority students. A semi-structured interview was performed in which participants were asked to describe the process of feedback-seeking, its challenges, and its benefits. The analysis involved examining the gathered data and generating themes from the data (Creswell 2012). Participants were extensively quoted to support the validity of the inductively drawn themes and the reliability of the findings. The analysis revealed six main categories as follows: (1) defining feedback and ways of obtaining it; (2) familiarity with feedback prior to pursuing academic studies; (3) barriers of “respect” and “embarrassment” against seeking feedback; (4) who to address: alternative sources of feedback; (5) sharing feedback with others; and (6) feedback for personal, group, or collective work.

124

8 Culturally Responsive Feedback

Defining Feedback and Ways of Obtaining It The interviews revealed that both Jewish and Arab students underscored that the main source of feedback was the lecturer. They described feedback as a response from the lecturer or teaching assistant that is designed to augment their knowledge, as described by the following students: “The lecturer’s response to my work usually emphasizes what is wrong. Some lecturers guide us on how to correct mistakes” (Sachar, Muslim female); “Lecturers relate to the entire assignment and offer comments about what is good as well as emphasizing what is wrong” (Miriam, Jewish female); “The lecturer offers feedback before giving a grade and gives us an opportunity to improve our grade, which is good” (Gaban, Muslim female). Unlike the Arab students who associated feedback only with the academic authorities (i.e., the lecturer and teaching assistant), the Jewish students reported having received feedback from family members as well. Oded (Jewish male) explained: “I sometimes share what I do with my father, who was an academic student, and I’m pleased to receive his feedback.” Similarly, Tamar (Jewish female) stated: “When I need direction or feedback about something I have done or am going to do I approach whomever is available; my friends, my advisor, the lecturer, or - if there’s no other option - my mother, who is a high-school teacher.” Dvorah (Jewish female) reported: “I often received feedback on large projects in high school from my mother and older sister. In most cases I asked for feedback so that my work would be better and more meticulous. My mother directed me, and my sister also helped me.”

Familiarity with Feedback Prior to Pursuing Academic Studies The interviews revealed discrepancies among students regarding their previous experience with feedback before they began their academic studies. Nine of the Arab students reported that they had received feedback on their assignments from their teachers in educational institutions, but that the objective was merely to offer verbal comments in conjunction with the final grade rather than to improve or promote learning. Nardin (Christian female) explained: “We received grades on our work, but if there were comments we didn’t have to submit the work again with improvements. We had already received our grade, so what would be the point?” Asma (Muslim female) also clarified this point: “We didn’t wait to receive comments before our work was graded, even when we had worked in a group.” When asked if it was acceptable to ask for comments about the learning process before submitting assignments, three Arab students explained that this was not an accepted practice. “I never asked a teacher for comments or feedback about my work. What, is he my friend or something? That wasn’t acceptable.” (Yara, Muslim female). Fatma (Muslim female) noted that if she asked her teacher for feedback, her objective was to “adapt herself” to the teacher’s opinion, but not to receive comments

Familiarity with Feedback Prior to Pursuing Academic Studies

125

about the learning process, “The teachers’ objective was to direct me more precisely to what he or she wanted. If I wanted to succeed on the assignment, I would have to make an effort to understand what the teacher wanted me to write, and I would write exactly what the teacher wanted. It wasn’t acceptable to dissent in our school.” Similarly, Lubna (Muslim female) explained: “It wasn’t acceptable to ask teachers for feedback. They offered it when they wanted to and when they thought it was appropriate.” She remembered an instance in which, after deliberating, she got up the courage to approach a teacher to ask for feedback. I only asked my English teacher for feedback once during my last year of high school. I liked English a lot, and I had a good teacher. I thought about it a lot at home and I was nervous. I approached her after class and asked her if she thought I was making progress because I was making a lot of effort. I asked her if she could suggest how I could make even better progress. She was very surprised at my personal question and perhaps at my courage, but she told me that she saw that I was investing effort and that I had made progress. She suggested that I read easy books out loud, and I did what she said.

Jewish students, on the other hand, were familiar with feedback from their experience in high school, “It was acceptable, a regular thing, part of being at school. We worked in groups until the end of 10th grade. We did a lot of assignments and received feedback and we could correct mistakes” (Hannah, Jewish female). According to Yariv (Jewish male), “I didn’t ask for feedback, but I received it for both individual and group assignments.” Anat (Jewish female) noted that during her high-school studies, students would do research projects and their teachers would write detailed feedback, particularly for large projects that were read and examined by external teachers. She said, “The feedback gave us direction, made us work meticulously, and correct mistakes. It seems natural to me – a part of the learning process”.

Barriers of “Respect” and “Embarrassment” Against Seeking Feedback Seven of the Arab students interviewed stated that they do not approach the lecturers at the college to ask for feedback or direction. Most of them explained that despite their desire to improve their learning, they were “embarrassed” to do so. Lecturers also did not offer sufficient feedback at their own initiative. According to Mohammed (Muslim male), “Despite the fact that it’s important to me, only some of our lecturers offer us guidance about how to correct mistakes. I don’t always know what’s wrong or how to correct it, but I don’t ask for explanations. I’m embarrassed.” Similarly, Arjuan (Christian female) emphasized her reluctance to request feedback from her lecturer. “I’m embarrassed and think that maybe it’s not her job to talk with me. It’s not proper to approach her, and I’m awfully afraid that she’ll see that I don’t know a lot and am not educated enough.” Asma (Muslim female) also feared that asking for feedback would expose her weak points in understanding the material. “She’s not my friend. I’m afraid that she won’t be nice to me…. I’m afraid she’ll discover that I don’t understand what she is teaching.”

126

8 Culturally Responsive Feedback

Aside from considerations of apprehension about how lecturers would perceive them, seven Arab interviewees also raised misgivings regarding the place and circumstances in which feedback is offered. Some students have negative feelings about receiving feedback in a large forum such as the classroom—even if the feedback is positive. In Nardin’s (Christian female) words, When the lecturer said positive things about me and said that she knew I was capable, I felt as if she were taking pity on me because I sit so quietly and make an effort to understand and I don’t always succeed. I felt that the nice things she said to me weren’t true and that I didn’t deserve them. I thought the others felt the same way, and I was ashamed.

Sachar (Muslim female) also explained, “I won’t ask for help when others are around. I prefer to wait until we are alone.” Lubna (Muslim female) added, “I don’t like receiving comments when everyone is around. I’m embarrassed if the lecturer talks about problems or mistakes in my work, and I blush and feel uncomfortable if she says something complimentary in front of my friends.” Mohammed (Muslim male) noted that “despite curiosity about how my group performed in comparison with other groups, exposing feedback in front of the entire class is liable to cause disrespect to members of the group if it’s not presented well.” Another barrier that contributes to “embarrassment” is the difficulty of the Arab students to express themselves in the language of instruction (Hebrew). Difficulty with the language constitutes a barrier for the Arab students when they want to request feedback. According to Mohammed (Muslim male), “My Hebrew isn’t very good. I’m embarrassed to talk with the lecturer and ask her and I’m afraid I won’t understand what she says and that she will think I’m stupid.”

Who to Address: Alternative Sources for Feedback Nine of the Arab students reported finding “indirect” ways to receive feedback in lieu of approaching the lecturer. They prefer to ask teaching assistants or friends for feedback about their learning. As Gaban (Muslim female) explained, “If I need someone to tell me whether my assignment is good and if I understood and answered correctly, I’ll ask my friends or the teaching assistant. I’ll never ask the lecturers. I’m afraid they’ll see that I don’t understand.” Asma (Muslim female) described how her difficulty to fully understand Hebrew causes her to avoid approaching the lecturer, and she prefers to seek help from the wide range of tutors that are available at the college, “The lecturers speak on a high level of Hebrew and speak very fast. I’m afraid they’ll explain something to me, and I won’t understand what they are saying. I prefer to ask my tutor, who speaks more slowly and is willing to repeat things for me.” Mohammed (Muslim male) prefers to receive feedback from his Jewish friends. He said, “I have several friends in the class who I became close with because there are not many men in the class. Sometimes I ask my Jewish friends for their opinion about my work and whether I correctly understood what the lecturer wanted. Sachar

Who to Address: Alternative Sources for Feedback

127

(Muslim female) also explained, “I don’t like to ask for help. I never ask the lecturer to read my work and comment on it. Why should she see my problems? If I ask anyone, I’ll ask my friends from my village who study with me, and I’ll help them as well.” On the other hand, seven of the Jewish interviewees explained that they frequently ask the lecturers for feedback. Some even resorted to self-criticism for approaching their lecturers too often. Sigal (Jewish female) explained, “Our lecturer is an extremely nice person, and I have no problem approaching him if I’m in need of comments or direction. He’s usually willing to advise me, and if he thinks I consult with him too often, he tells me to make more of an effort before I consult with him.” The students described various channels of communication that they use to receive feedback from the lecturer, as described by Yariv (Jewish male): “If I need direction or feedback, I approach the lecturer after the lesson or send him an email asking to arrange a time for a short meeting.”

Sharing Feedback with Others Nine of the Arab students admitted sharing with others the “big picture” of what is going on in their studies, but do not share the feedback they receive in a specific course. However, these students reported that if they received positive feedback for a particular assignment in a course, they would consider sharing it with their family, friends or with the teaching assistants with whom they are in contact. Arjuan (Christian female), for example, explained, “My family is interested in how I am doing. I tell them what is going on generally. They have been very helpful in paying for my studies. I don’t involve them in what goes on in class. Sometimes I show the comments to my friends from class and we compare them.” Similarly, Mohammed (Muslim male) explained, “I share comments with my friends. My family is only interested in what is going on generally. I have a good relationship with one of the teaching assistants, and sometimes I show her the lecturer’s comments even if they are not connected to her class because she understands Arabic and explains them to me.” Sachar (Muslim female) reported that she shows comments to her friends; however, “when I don’t understand the feedback or part of the comments or what they mean, I show them to my tutor who knows Arabic. I’m not ashamed to confide in her because I know she understands me.” Seven Jewish students reported sharing the feedback they receive mainly with their classmates. If they received help from a teaching assistant or family member, they show it to them. Hannah (Jewish female) explained this, similarly to most of the other Jewish participants, “I show the feedback I receive to my friends who study with me. If the feedback is special, particularly if it contains praise, I also show it to my family. If someone helped me prepare the assignment or project, I always show them.”

128

8 Culturally Responsive Feedback

Feedback for Personal, Group, or Collective Work Six of the Jewish students stated that they would like to receive individual feedback alongside collective feedback for group work. They are interested in learning how their group dealt with a joint project but would also like to receive an evaluation of their part in the work. Yariv (Jewish male) explained, “Feedback for group work should direct each member of the group how to work together better; [To hear] what was not good enough. For that reason, I’m interested in knowing how I did. What do I need to do in order to work better with my colleagues?” In addition to highlighting the importance of receiving personal feedback within the group feedback, Rachel (Jewish female) is not afraid of exposing feedback in front of the other students in the class. She said, “I’m curious to know how our group performed, but I also want to know how we did in comparison with the other groups…but it’s also important to me to learn how I did.” Seven Arab students, on the other hand, reported feeling that in group work the entire group is responsible for the final product and, therefore, feedback must be directed towards the entire group. As Asma (Muslim female) explained, “When we work together the feedback is for all of us. We are all responsible for the assignment. We all have to make an effort and receive direction together.” Mohammed (Muslim male) also feels that group work needs to be given feedback that is designated for the entire group, saying, “This is a group assignment, and there’s no need to cause division and disagreement within the group. The feedback needs to be given to the group.” Analysis of the interviews of the six Jewish lecturers who taught Jewish and Arab students at the two teachers’ training colleges revealed some useful coping strategies with the problems that emerged from the students’ reports. They suggested eliminating the Arab students’ perceived sense of the threat posed by the feedback process by building trust between the feedback provider and the recipient, providing a continuous flow of face-to-face feedback, and making the feedback more personalized by employing Arab tutors.

Building Bridges of Trust: Lecturers’ Points of View The lecturers raised several insights that can help eliminate the sense of threat when approaching a source of authority. The first insight is to boost trust between the lecturer and minority students by creating clear and honest communication. This is explained by Shiran, Creating trust between a lecturer and a student is essential for designing feedback that promotes learning. This applies to all students, but my experience has taught me that more effort is required to accomplish this with minority students. The student must believe that the lecturer truly wants to help him/her progress and that s/he is not merely a ‘burden’ or an ‘unpleasant obligation’. The process of creating trust is a long one – particularly with minority students.

Building Bridges of Trust: Lecturers’ Points of View

129

Shiran asked her students to write feedback at the end of each lesson that included several points describing any difficulties they had during the lesson. She was then able to provide a broader and clearer solution for all of her students. At first, she felt that the minority students did not understand the objective of the feedback. They were afraid to describe their difficulties. They were afraid that if they described difficulties, I would think they were not capable and that they were ‘stupid.’ The first feedbacks they wrote were meaningless: ‘Everything is OK’; ‘The lecturer explains things clearly.’ or ‘I have no difficulty understanding the assignments’, etc. I had to explain that they could teach me the best way to teach them and that we were actually helping each other.

Shiran described a positive development in the students’ ability to express their difficulties over time after continuing the reflective process at the end of each lesson. She emphasized that the students’ trust increased the more she responded to their feedback by clarifying the material in class.

Providing a Continuous Flow of Feedback Perseverance in providing feedback was raised as another aspect intended to eliminate the minority students’ sense of threat. Four lecturers described their surprise when they discovered that the minority students were not accustomed to receiving feedback. They emphasized that precisely because of this, it is important to persevere and devote time to offering face-to-face feedback to those who have never experienced the process significantly, as described by Sivan: I had the feeling that the minority students didn’t believe that I would ‘waste time’ on discussions to provide feedback, or that feedback was worth the time I devoted to it. Our meeting and the fact that I related seriously to what they had written caused the change…

Personal Feedback Personal feedback is offered as a third method designed to eliminate the sense of threat. Four lecturers who have tutors working alongside them in their courses describe the personal rapport that develop between tutors (third-year students) and their first-year Arab students in contrast to the openness that lecturers have difficulty in attaining in their relationships with students. Rachel and Naomi describe this respectively: Asma talked extremely openly with her Arab tutor about problems at home and about the difficulty she was having understanding graphs that I presented in class. She never shared this with me. She only alluded to difficulties with her family at home that were preventing her from making progress during the meeting we had at the end of the semester. I think that from the minority students’ standpoint, openness implies a blurring of the boundaries between the status of a lecturer and the status of a friend. They open up and talk with a friend, but with a lecturer they pretend they understand, everything is OK, things are moving

130

8 Culturally Responsive Feedback

along, and progress is being made. The tutors, who are usually closer in age to the students and speak their language, are perceived as friends, and I feel that students feel comfortable opening up to them.

Three lecturers whom we interviewed offered using face-to-face dialogue for providing feedback to minority students. They feel that using Email, Moodle, or other electronic media is not suitable at the outset for providing feedback during courses, particularly in the case of minority students. Sharona explained, Using Moodle was totally ineffective at the beginning of the first semester, despite the fact that it was used after numerous explanations in class. I felt a significant change [only] when I met with the students personally. After two such meetings there was significant improvement in the way they related to the feedback I gave them. I felt they had assimilated it and would perhaps apply it in the future.

Nava described a similar process with her immigrant Jewish students from Ethiopia, Face-to-face dialogue with the students enabled me to understand how each student related to the topic. I understood that offering feedback was something new for them. We were able to eliminate gaps in a few sentences, questions, and answers face-to-face that I would have had difficulty eliminating if I had communicated by computer.

Discussion This chapter dealt comprehensively with the role played by culture in individuals’ behavior when seeking feedback, according to the specific-holistic orientation dimension of the cross-cultural model of feedback-seeking behavior (Sully de Luque and Sommer 2000). A conspicuous finding is that the Arab minority students feel “embarrassed” when seeking and receiving feedback for various reasons. The first reason has to do with interaction with the lecturer, whom they perceive as a distinguished and authoritative figure who should not be approached directly. The students fear that approaching a lecturer to request feedback will expose their weak points and embarrass them. Secondly, they must cope with a limited ability to express themselves in Hebrew, which is the language of instruction but not their mother tongue. Finally, receiving feedback publicly in front of their classmates constitutes a source of embarrassment, irrespective of whether the feedback is positive or negative. Consequently, there is a need to be sensitive and cognizant of cultural differences and consider them when offering feedback. These differences are characterized, among other things, by minority students’ culture of respect for lecturers, embarrassment, and fear of misunderstanding due to language difficulties, manifest in their preference for feedback that is delivered personally rather than in a public forum. Analysis of the Arab students’ responses with regard to receiving feedback in high school showed that it was culturally unacceptable to seek feedback. If feedback took place at all, its sole objective was to clarify precisely what the student needed to do in order to succeed based on what the teacher wanted the student to do. In line with

Discussion

131

the teacher-centered approach, the students were liable to ‘pay a high price’ if they attributed a different meaning to what the teacher said, as found in low tolerance for ambiguity cultures (Sully de Luque and Sommer 2000). The Jewish students, on the other hand, reported that while they did not actively seek feedback, they were given it, nonetheless. The feedback had the sole objective of promoting accuracy in their work and was perceived as a natural part of the learning process. Apart from the teacher’s opinion, the opinions of family members and friends were also an acceptable form of feedback. Another finding was that the Arab students preferred feedback that was given to the group and that related collectively to the assessment criteria, in line with the individualism-collectivism cultural dimension (Sully de Luque and Sommer 2000). However, they felt that group feedback should not be given in front of the entire class due to considerations of respect and avoiding hurt feelings among their classmates and members of other groups. The Jewish students, on the other hand, attributed greater importance to personal feedback even when offered as part of the overall group feedback. From their standpoint, recognition of each person’s contribution or weak points was necessary and even warranted, so as to make his/her work more efficient in the future.

Implications and Recommendations For minority student, receiving feedback in front of the entire class—immediately following presentations of study products, for example—is liable to result in hurt feelings among the minority group, regardless of whether the feedback is positive or contains constructive criticism. Therefore, it is suggested that instructors take these individual and cultural differences into consideration when assessing group work and consider providing feedback individually, and discreetly, to minority students. Unlike their Jewish peers who felt more comfortable utilizing a wide variety of communication channels to elicit the lecturer’s feedback, minority students perceived approaching figures of authority as threatening. To circumvent this challenge, it is suggested using private face-to-face dialogues for providing feedback to minority students. The interviewed lecturers conceded the ineffectiveness of e-communication when offering feedback to minority students who are either uncomfortable or unfamiliar with the process. Building trust between the lecturer and the students is suggested as a means to eliminate those barriers. This can be achieved by explaining the aims of feedback and providing it continuously during the course. Finally, it appears that one of the main ways to cope with the sense of threat felt by minority students when seeking feedback from their lecturers is to make use of tutors. In the participants’ colleges, these tutors are either Jewish or minority graduates of these same institutions who are employed by them to aid minority students. Each lecturer is assigned one or more tutors who provide help and support for minority students who need or request it. The tutors work in full cooperation and transparency with the lecturers. It is evident, on the backdrop of the cultural and emotional barriers

132

8 Culturally Responsive Feedback

that characterize the feedback-seeking process, that minority students perceive this system as an efficient and beneficial tool for receiving feedback on their learning process. They feel more comfortable seeking feedback from tutors who come from their culture and speak their language. This chapter has presented several key insights concerning feedback-seeking as it relates to culture. In addition, it has proposed ways to promote feedback-providing strategies suitable for teachers/instructors of multicultural learners and suggested that such strategies will greatly improve feedback effectiveness and fulfill its objectives for all students. The first step in accomplishing this is to increase awareness among instructors with respect to cultural differences that characterize and affect the act of feedback-seeking. Educators must recognize that this process is new for many students who are unaccustomed to feedback. Secondly, we must understand that for minority students, this process may involve mixed feelings, grounded in their culture, that will prevent them from freely asking their lecturers for feedback. It is therefore advisable to encourage them to do so in a discreet, one-on-one conversation. Finally, it is recommended to train college students to serve as tutors of minority students and to work closely with lecturers to mediate the feedback process.

References Ashford, S. J., Blatt, R., & VandeWalle, D. (2003). Reflections on the looking glass: A review of research on feedback-seeking behavior in organizations. Journal of Management, 29, 773–799. Askew, S., & Lodge, C. (2000). Gifts, ping-pong and loops – Linking feedback and learning. In S. Askew (Ed.), Feedback for learning (pp. 1–17). London: Routledge. Ball, E. C. (2010). Annotation an effective device for student feedback: A critical review of the literature. Nurse Education in Practice, 10, 138–143. Bell, A. R., & Brooks, C. (2018). What makes students satisfied? A discussion and analysis of the UK’s national student survey. Journal of Further and Higher Education, 42, 1118–1142. Boud, D., & Molloy, E. (2013a). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38, 698–712. Boud, D., & Molloy. E. (2013b). What is the problem with feedback? In D. Boud & E. Molloy (Eds.), Feedback in Higher and Professional Education (pp. 1–10). New York: Routledge. Bruno, A., & Dell’Aversana, G. (2017). Reflective practice for psychology students: The use of reflective journal feedback in higher education. Psychology Learning & Teaching, 16, 248–260. Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36, 395–407. Cramp, A. (2011). Developing first-year engagement with written feedback. Active Learning in Higher Education, 12, 113–124. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Upper Saddle River, NJ: Merrill. Deeley, S. J. (2018). Using technology to facilitate effective assessment for learning and feedback in higher education. Assessment & Evaluation in Higher Education, 43, 439–448. Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83, 70–120. Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112.

References

133

Heine, S. J. (2007). Culture and motivation: What motivates people to act in the ways that they do? In S. Kitayama & D. Cohen (Eds.), Handbook of cultural psychology (pp. 714–733). New York: Guilford Press. Henderson, M., Ajjawi, R., Boud, D., & Molloy, E. (Eds.). (2019a). The impact of feedback in higher education: Improving assessment outcomes for learners. Switzerland: Springer Nature. Henderson, M., Ryan, T., & Phillips. M. (2019b). The challenges of feedback in higher education. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2019.1599815. Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: a Meta-Analysis. Assessment & Evaluation in Higher Education, 44, 863–880. MacDonald, H. A. (2008). Motivational differences in feedback-seeking Intentions: A cultural analysis. (Doctoral dissertation). Retrieved from https://uwspace.uwaterloo.ca/handle/10012/ 3696. MacDonald, H. A., Sulsky, L. M., Spence, J. R., & Brown, D. J. (2013). Cultural differences in the motivation to seek performance feedback: A comparative policy- capturing study. Human Performance, 26, 211–235. Nelson, M. M., & Schunn, C. D. (2009). The nature of feedback: How different types of peer feedback affect writing performance. Instructional Science, 37, 375–401. Nicol, D. (2010). From monologue to dialogue: improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35, 501–517. Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28, 4–13. Rowe, A. D., & Wood, L. N. (2008). Student perceptions and preferences for feedback. Asian Social Science, 4, 78–88. Ryan, T., & Henderson, M. (2018). Feeling feedback: students’ emotional responses to educator feedback. Assessment & Evaluation in Higher Education, 43, 880–892. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. Sully de Luque, M. F., & Sommer, S. M. (2000). The impact of culture on feedback-seeking behavior: An integrated model and propositions. The Academy of Management Review, 25, 829–849. Triandis, H. C. (1995). Individualism and collectivism. Boulder, CO: Westview.

Part III

Institutional Facet

Chapter 9

Culturally Responsive Culture of Assessment

Introduction This chapter centers on several definitions of the culture of assessment employed primarily by scholars in the United States (Fuller and Skidmore 2014; Maki 2010). A culture of assessment refers to the “deeply embedded values and beliefs collectively held by members of an institution who influence assessment practices on their campuses… This culture undergirds assessment practices on a campus. Assessment culture is the system of thought and action that reinforces what ‘good’ conduct of assessment looks like” (Fuller et al. 2015, p. 332). Maki’s (2010) definition of culture of assessment is focused on the promotion of student learning and of learner-centered assessment. It is about making meaning from complex learning processes that take place in higher education. While it is widely acknowledged that cultures of assessment contribute to institutional conditions for student learning, a somewhat opaque landscape emerges in terms of empirically exploring the existence, or the dearth, of these cultures in higher-education institutions, in part due to the relative difficulty in collecting data from assessment practitioners and faculty (Skidmore et al. 2018). To bridge the gap, and in the context of this book’s theme, this chapter seeks to depict contemporary efforts to establish a culturally responsive culture of assessment in several academic institutions in Israel. This work may benefit assessment practitioners and aid in further theorization on how cultures of assessment can be developed in multicultural academic institutions.

Culture of Assessment: Definitions and Features Drawing on Maki’s (2010) definition of culture of assessment, the primary institutional approach is designed to utilize evidential data to improve students’ lifelong learning skills, in that “beyond its role of ascertaining what students learn in individual courses, assessment, as a collective institutional process of inquiry, examines © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8_9

137

138

9 Culturally Responsive Culture of Assessment

students’ learning over time. It explores multiple sources of evidence that enable us to draw inferences about how students make meaning based on our educational practices” (p. 2). Freedman (2013) underscores two core objectives of the culture of assessment: value and quality. Value is equated to the benefit or moral justification of a service or action, as related to the needs that the service or activity is designed to provide. It further refers to the extent to which the assessment contributes to the recipients of the service or the beneficiaries of the contents or the activities; for example, the degree to which the assessment contributes to the honing of skills that are necessary for optimal learning in the twenty-first century. Quality is defined as the extent to which the objectives that have been established are attained. The quality of the activity being assessed is examined by the answer to the question of whether or not the people being assessed are doing what they are supposed to do satisfactorily. The concepts of value and quality of a program are independent of each other from the standpoint of assessment. A program may have a high level of quality by fully attaining its objectives, but those objectives may be of little value to the participants. Hence, the assessment program must focus upon two components: mapping needs and attaining objectives. Assessment goals and objectives are shaped by the needs that were identified as well as to the guidelines, standards, and overall organizational culture of the academic institution. These objectives are complex and, as put forward by Maki (2010), we expect our students to develop higher-order thinking abilities, to respond to environmental, social, technological, scientific and international developments, to solve ill-structured problems by using multiple perspectives, and to develop lifelong learning habits of mind. Designing activities to attain assessment objectives requires a collective institutional process of examining students’ learning outcomes against the standard of judgment. Weiner (2009) describes fifteen institutional components of cultures of assessment focused on improving student learning: (1) setting general education goals by which the institution should choose the core manageable competencies that all students, regardless of major, are expected to demonstrate; (2) common use of a glossary of assessment terms; (3) faculty’s participation in planning and developing an assessment program; (4) offering ongoing professional development programs to ensure extensive understanding; (5) administrative support and understanding, e.g., “If faculty members think that their administration views assessment as a fad that will go away, or even that it sees it only as the job of faculty, they will hesitate to engage in the assessment process.”; (6) a practical, sustainable assessment plan that needs to be cost-effective, realistic, and frequently revisited; (7) an assessment plan based on a methodical and systematic process of evaluation; (8) the institution must understand what students will be able to do (learning outcomes) by the end of each course; (9) a comprehensive program review, which could be accompanied by formats designed to guide the academic programs’ review; (10) assessment of co-curricular activities that may provide learning opportunities; (11) institutional effectiveness, in addition to student learning outcomes, such as assessing campus climate by measuring students’ perceptions and attitudes toward their campuses and college communities;

Culture of Assessment: Definitions and Features

139

(12) information sharing provides opportunities for departments to engage in peer review, steer away from failed experiences, and replicate successes; (13) planning and budgeting, whereby “assessment results can demonstrate areas of need within a department, and faculty can evaluate how much money might be necessary to rectify a problem”; (14) celebration of success provides acknowledgment and appreciation that reinforce the notion that assessment is important and that assessment reports are valued. and (15) new initiatives—evaluating a new initiative or proposal stimulates lively discussions at meetings around the college, such that “when such a response occurs, it will confirm that an institution possesses a culture of assessment”.

Commitment to a Culture of Assessment Some researchers have pinpointed several principles of commitment that may contribute to a strong culture of assessment. For example, Lee et al. (2019) presented important features of these principles such as organizational commitment to assessment across all levels, multiple stakeholder involvement, clear lines of accountability, and a sense of a common purpose. With the establishment of its own culture of assessment, the institution must ensure that fundamental support systems such as data management/reporting systems, reward and recognition strategies, and training are also in place. The process should be conducted in an iterative fashion, with continuous feedback and consequent revision of the process. The purpose of assessment must be clearly articulated by institutional leaders, and integrity and trust must thrive within the environment. During the evolution of an assessment culture, collective values, missions, resources, and structures should be considered. Furthermore, a commitment to stakeholder accountability as well as continuous quality improvement and innovation are necessary to establish a true culture of assessment. All stakeholders need to be engaged in the processes with relation to “the rationale supporting assessment, the processes involved, the artifacts used to report meaningful feedback and evaluation, and the expectations for how results are communicated and used to assure quality and improvement” (p. 130). Finally, the administration’s support for the commitment of required resources should be ensured. According to Maki (2010, p. 9), an inclusive commitment to a culture of assessment should center on student learning. This commitment is established when it is (1) meaningfully anchored in the educational values of an institution—articulated in a principles-of-commitment statement; (2) intentionally designed to foster interrelated positions of inquiry about the efficacy of education practices among educators, students, and the institution itself as a learning organization; and (3) woven into roles and responsibilities across an institution from the chief executive officer through senior administrators, faculty leaders, faculty, staff, and students. Without meaningful institutional anchors, the commitment to assessment may become… an action that has to be performed but not necessarily for any meaningful purpose.

140

9 Culturally Responsive Culture of Assessment

Challenges in Building a Culture of Assessment As noted by several researchers (Baas et al. 2016; Farkas 2013), the “Assessment Movement” has been a guiding force in the world of higher education for 30 years in which colleges and universities are asked to show that they are actually delivering on the promised student learning outcomes. Faculty are increasingly expected to be able to exhibit what students have learned and how that corresponds with the programlevel and campus-wide learning outcomes. However, despite this greater focus on accountability, few institutions of higher education have developed a sustainable culture of assessment. Baas et al. (2016) attribute this failure to the tension generated by the “Assessment Movement” in higher education. On one hand, colleges and universities are expected to deliver student learning outcomes while, on the other hand, efforts to foster a “culture of assessment” among institutions of higher learning have frequently encountered resistance, as witnessed by, …one of the biggest deterrents – if not the principal obstacle—encountered in the quest to cultivate campus-wide “cultures of assessment” stems from inadequately understood viewpoints held by key parties to the enterprise. Particularly important in this regard are the perspectives of faculty and administrators as institutions embark on the creation of such cultures in a manner able to satisfy external accrediting agencies while upholding or advancing the quality of the college’s core educational mission (p. 2).

Similarly, Walvoord (2010) maintains that faculty often fear external control over what is taught and tested. When results are used irresponsibly with a preponderance of standardized test-driven instruction, narrowing the goals of higher education to what is measurable, then “educators will be forced to create costly and time-consuming bureaucratic systems that comply with accreditors’ demands for assessment but that do not really result in improved student learning” (p. 3). This tension is not confined to higher education and can be traced also to other educational frameworks, for example in the case of curricular reforms which have been gradually implemented in science and technology education. These reforms emphasize the interconnection between teaching, learning, and assessment to create meaningful learning. The main objectives of those reforms are to enable students to interpret information, create links between facts, think about and regulate their learning, apply their understanding of new concepts to new situations, employ creative thinking, solve problems, experience a change in attitudes and opinions, develop scientific skills, construct knowledge, and search for possible solutions to ill-structured questions by means of inquiry-based discussions. Formative assessment used to inform subsequent learning is considered an integral part of teaching and learning, according to the new reforms in science education. Policymakers formulate and declare objectives that seek to advance 21st-century abilities and evaluate them using formative assessment tools and methods. However, at the same time, they perpetuate the use of comparative standardized tests used throughout the world. This test structure has caused teachers to assume responsibility for their students’ results as well as for the repercussions of their students’ failures. This contradiction

Challenges in Building a Culture of Assessment

141

between objectives declared by policymakers and what actually occurs in practice has created a situation in which many teachers reject formative assessment methods and, rather, focus their attention on the use of standard comparative tests (Alt 2018). This duality is liable to deter the formulation of a clear evaluation program on an organizational level as well as to inhibit the desire and ability on the part of the educational staff to implement it. Aydeniz and Southerland (2012) showed that a majority of American high-school and middle-school science teachers held negative views of the role of standardized testing in bringing about improvements in student learning in science. Among other variables, their views were influenced by their beliefs about what they thought was important for students to learn in science classrooms: “Those that valued students’ acquisition of scientific inquiry and critical thinking skills opposed the administration of the standardized tests for accountability purposes” (p. 251). Science teachers’ instructional and assessment practices depend on how the expectations are being communicated by the stakeholders and how the teachers are being supported as they attempt to make the goals of science education reform prevail in science classrooms. To circumvent this conflict, faculty and staff must feel empowered to develop their own assessment program and measures, and administrators need to lead by example and support their efforts (Farkas 2013). It should be noted that many scholars promote the notion that, through it all, the focus must be on student learning, rather than accreditation or other external pressures that grant preference to standardized tests, yet, “despite its purported benefits to student learning, current discourses of accountability and accreditation dominate the practice and scholarship of assessment. Such a focus on teaching and learning as well as the recognition of assessment as a personal policy area begs for an examination of assessment efforts using personcentred approaches” (Skidmore et al. 2018, p. 1244). Contrary to those who value assessment for learning in a well-established culture of assessment institutions, many faculty members do not recognize the value of assessment in discovering more about student learning. Instead, they view this assessment, which is mandated from above, as something that they have to do because it is required by administrators or accreditors. For some, this goes against their belief in academic freedom to use an assessment method of their choice. Walvoord (2010) contradicts these argumentations asserting the violation of academic freedom, echoing a counter argument put forth by the Association of American Colleges and Universities’ Board of Directors Statement on Academic Freedom and Educational Responsibility (2006). This statement emphasizes that academic freedom has to do with faculty members’ responsibilities for establishing goals for students learning, designing and implementing programs, cultivating the intended learning, and assessing students’ achievements. These matters demand collaboration among colleagues and relevant administrators in their respective departments, schools, and institutions, in the belief that “academic freedom is necessary not just so faculty members can conduct their individual research and teach their own courses but so they can enable students – through whole college programs of study – to acquire the learning they need to contribute to society” (p. 8).

142

9 Culturally Responsive Culture of Assessment

Farkas (2013) argues that in light of these challenges, initiating and spearheading changes can be difficult and that constructing a sustainable culture of assessment necessitates the collaboration of the entire organization; sporadic efforts of faculty are not sufficient. In her study, she discusses the idea of using Kotter’s (1995) eightstep model for establishing leadership to create a culture of assessment: (1) create a sense of urgency; (2) form a guiding coalition; (3) create a vision; (4) communicate the vision; (5) empower others to act on the vision; (6) plan for and create short-term wins; (7) consolidate improvements to generate more change; and (8) institutionalize new approaches. Drawing on this model, Farkas concludes that “a change process that ends with changes in faculty and staff behavior will not create lasting effects. People will eventually resume their old habits once the urgency has subsided. For a change process to be successful over time, the organization must also change structures and policies to both accommodate the change and embed it in the culture” (p. 28). Based on the above-surveyed studies, it seems that although cultures of assessment in higher education have long been advocated as being beneficial to institutional conditions for student learning, empirical research exploring the establishment or lack of these cultures in higher education institutions is notably scant. Drawing on the above-surveyed studies, the present chapter conceptualizes culture of assessment as institutional initiatives that support, or hinder, the integration and use of components that lead to changes in teaching and learning, with a particular emphasis placed on the needs of multicultural students. This premise will be illustrated in the following section through the prism of Israeli faculty members’ perceptions of their institutional culture of assessment.

Culture of Assessment in Israeli Academic Institutions In order to assess the culture of assessment in academic institutions in Israel and how it is manifest in the perceptions of senior faculty members, nine senior lecturers were interviewed from nine institutions of higher learning. These included three universities, three teachers’ training colleges, and three regional academic colleges. In the 1990s, regional colleges signified an avant-garde in the higher education revolution in Israel. These colleges were established in the outlying areas to enable students of various population groups—many of which multicultural—to study for academic degrees. All the interviewees have been active faculty members in their institutions for more than ten years and have held senior positions. All the institutions are multicultural, and their student bodies include Hebrew speakers (mostly Jews, who comprise the majority in Israel) and Arabic speakers who belong to three main minority groups (Muslims, Christians, and Druze). All the participants were asked to relate to the policy in their respective academic institutions regarding culture of assessment, as defined for the purposes of this study, and to the suitability of the assessment to individuals belonging to minority groups. Participants are referred to by the following pseudonyms to preserve their anonymity: Dror, Irit, and Aviv (university), Rafael,

Culture of Assessment in Israeli Academic Institutions

143

Yonatan, and Maya (regional colleges), and Yair, Nava, and Anat (teacher-training colleges). The interviewees were asked about the guiding principles of pedagogical policy and attitudes towards assessment in their institution, policies that are implemented there for assessing minority students, whether or not these policies are sensitive to their cultures, motives for developing alternative tools for teaching, learning, and assessment, training the faculty in this domain, international projects for advancing teaching and their implications for developing assessment, and their outlook regarding their preferred evaluation policy in general and their preferred policy with respect to minority students in particular. Content analysis revealed five main themes pertaining to pedagogical policy and culture of assessment as perceived by the research participants. These themes reflect the policy relevant to the advancement of teaching-learning and the culture of assessment in multicultural academic institutions: A. B. C. D. E.

A.

The conflict between academic freedom and advancing teaching-learningassessment. International connections as an impetus for promoting teaching-learningassessment. Perception of the role of the lecturer in the institution. Placing responsibility for academic advancement of minority students upon other entities within the institution. Barriers to the advancement of teaching-learning-assessment and its adaptation to the individual: lack of suitable training, lack of commitment to a common vision, lack of resources, and disparities in learning skills among students from multicultural groups. The conflict between academic freedom and advancing teaching-learningassessment

The six participants from the universities and regional colleges emphasized that preservation of academic freedom and the autonomy of the faculty in the fields of content, teaching, and evaluation served as a criterion for determining the pedagogical policy of the academic institution they represent. At the same time, the administration strives to inform the faculty about the changes implemented in teaching-learningassessment in recent years, to encourage them to become familiar with them and to adopt the changes that are appropriate for their particular content knowledge and needs. This may be attained by offering assistance and learning tools to those who are interested in trying modern methods of learning and evaluation. The process may be led by key stakeholders such as the president of the institution. All the participants noted that alternative assessment is considered the weak link in the chain of implementing advancement of teaching. When speaking about the institution’s policy on assessment, none of the participants related either to the matter of assessment in general or assessment of a cultural group in particular. For example, Aviv, a lecturer from a university in central Israel, emphasized the involvement of the

144

9 Culturally Responsive Culture of Assessment

rector in advancing teaching and the efforts made by the Center for Advancement of Teaching, which invites lecturers to participate in its activities. However, he explained that lecturers are given freedom to decide what method of teaching and evaluation they prefer. Aviv stated, The rector is active, and his presence is evident in the issue of advancement, modernization, and improving teaching. There is an awareness of the need for developing cognitive skills in new ways. The Center for Advancing Teaching encourages lecturers to attend workshops about various methods of teaching and assessment. The center promotes innovation and encourages lecturers to attend and be updated. There is no doubt that these topics are raised during meetings in the various departments. In the end, however, the lecturer is the sole person who decides which learning and assessment methods will be used in the course.

The participants from the colleges revealed that there is slightly greater emphasis placed upon advancement of teaching in the colleges than in the universities. This feature serves as a tool for attracting students from the periphery in the competition between the regional colleges and the universities. However, the colleges also emphasize academic freedom. The component of assessment is seldom referred to, and the matter of alternative assessment is not dealt with at all. Rafael, from a regional college in southern Israel, described the pedagogical policies as part of the overall policy declared by their colleges and the efforts made by their respective institutions to change teaching-learning methods. As he noted: The regional colleges and the universities are competing for students, and the right way to attract students to the college is to improve teaching. Another rationale for opening the regional colleges was to make academic education more readily available. The college administration encourages lecturers to keep up-to-date about teaching methods and initiates workshops and conferences but does not determine teaching methods and doesn’t force lecturers to change students’ learning methods. The lecturers’ autonomy is respected. There is no special reference made to assessment. It’s part of the alternative methods of teaching and learning...

In contrast to the faculty members of regional colleges and universities, Yair, Nava, and Anat from the teacher-training colleges describe a clear-cut, unequivocal policy of demands for modern pedagogical methods. They present examples of frequent direct involvement of the college administration in issues of advancing teachinglearning-assessment without taking lecturers’ rank or status into consideration. Yair, for example, described the pressure placed on lecturers to diversify their methods of teaching-learning, but when dealing with assessment this takes the form of a request rather than a directive. In his words, There is a demand to update teaching-learning, and department heads are asked to examine how this is expressed in the syllabus and in class. We have established a department that works to advance “active teaching” and its members relate to the lecturers directly. They ask to observe lessons, provide help and guidance, and of course, report on their activities. There is definitely pressure to train future teachers using methods that are suitable to the present era. On the other hand, you know that once the lecturers close the classroom door, they teach as they please. There is also a request – not a directive – to include alternative assessment alongside the final examinations that many lecturers prefer. Despite all the talk, the administration is still reluctant to discontinue the use of closed examinations as an assessment tool.

Culture of Assessment in Israeli Academic Institutions

B.

145

International connections as an impetus for promoting teaching-learningassessment

The participants described how international projects serve as an unexpected source of advancement of teaching-learning-assessment as an integral part of their academic institutions’ policies. All of them explained that their institutions participate in various international projects and that they themselves have also taken part. Most institutions have a department or office that promotes international projects and aids interested members in participating. The desire to attract students from abroad within the framework of international programs has “compelled” the faculty to adapt their teaching and assessment methods to a format that will be acceptable to the institutions of potential foreign students. Aviv, a university lecturer, explained, Attention is being devoted to assessment as a result of the international programs, too. Assessment methods for foreign students are often adapted to suit the criteria of their universities abroad. Lecturers who teach and assess these students become enriched with “alternative” assessment tools and learn while implementing these methods. Adapting the curriculum and assessment methods is relevant to universities that send students for a semester or a year, but I have no doubt that these methods permeate to other courses that these lecturers teach.

Similar to their university colleagues, regional college faculty reported the development of international research projects, advancement of short-term student exchange programs, and their hopes that more students from abroad would attend classes in their institutions. They reported exchanges of lecturers for short periods and research cooperation that have resulted largely from personal initiatives of lecturers or sometimes from their participation in projects such as Erasmus + (co-funded by the European Union). These programs introduce the importance of learning-teaching-assessment into the agenda. As Maya explains, The academic faculty and the administration are motivated to create international connections and devote relatively large resources to the issue. Every program, such as Erasmus+, that we lead or participate in enriches and enables the college to invest in international relationships. I think that some of the programs that were accepted which deal with teaching-learningassessment and multi-culturalism will permeate to the topics we are dealing with in this interview.

The teacher-training colleges are additionally involved in advancing international connections. The faculty members’ statements reveal that this issue is gaining importance in their college policies. They consider it extremely valuable and view it as a tool for advancing the academic status of the teachers’ colleges, which often suffer from an unflattering image on the part of the public and their academic colleagues. Yair, Nava, and Anat, who are involved in some of the projects, note their contribution to advancing teaching and assessment. Anat related to the advancement of learning and assessment that results from projects that involve teaching and learning in Erasmus+ and other frameworks, Some of the projects require the implementation of unique methods of teaching and assessment within the college itself. This enriches the students and often relates to assessment of students on both an individual and group basis. Other lecturers are also becoming aware of

146

9 Culturally Responsive Culture of Assessment

these methods and they are presented at college meetings. Some are incorporated into the methods of teaching, learning, and assessment in the college and enable better training of future teachers.

C.

Perception of the role of the lecturer in the institution

The interviewees underscored the efforts on the part of the institutions to improve the lecturers’ teaching skills. However, the interviews revealed that despite their desire to advance methods of teaching and assessment, the participants do not perceive themselves as trained teachers who are assigned to expand their knowledge in methods of teaching and assessment. Obligations that are placed on the faculty and on young lecturers who are interested in becoming future faculty members to conduct research demand resources that these lecturers would prefer to channel into research rather than teaching. Dror, for example (a university lecturer), claims that teaching is not within his realm of expertise. He attempts to diversify his teaching methods and to “interest” and engage his students, but he does not see himself as an “educator” or as someone who is qualified to teach. The principle academic activity of a lecturer who wishes to attain tenure at the university is conducting research in his or her field of knowledge. All lecturers want to advance their students. They want to enjoy classes and to encounter bright-eyed, curious students following a unique lesson and alternative assessment. However, that demands extensive efforts outside their area of knowledge, re-planning lessons, thinking, and cooperating with experts in assessment to find alternative ways to assess students. Who has time for that? Education is not my field. I try and interest students. I try to show consideration to students who don’t know Hebrew well, especially during their first year of study. But I can’t offer them a different scale of assessment than the one I use for other students or teach fewer topics each semester.

Yonatan, Maya, and Rafael, who are senior faculty members at regional colleges, reported that the conditions for professional promotion are starting to resemble those at universities and require them to conduct research, but at the same time the existing conditions for research are unsatisfactory. They spoke about the difficulties posed by numerous hours of classroom instruction together with insufficient budgets for performing research. All three lecturers identify with the need to improve learningteaching-assessment methods but voice reservations about further defining the role of the lecturer. They claim that extensive investment in improving alternative assessment methods would come at the expense of research. This distinction is the foundation for the belief of those interviewees who differentiate between the role of an academic lecturer/researcher and the role of “school teachers.” They categorically do not identify with objectives such as developing students’ social skills, which they associate with “school teachers.” They strongly feel that such objectives are not “academic” and therefore are not part of the role of an academic lecturer and researcher. Maya expressed this belief in her remarks, Traditional teaching methods need to be changed, and alternative teaching methods need to be implemented. At the same time, we need to ensure that we uphold an appropriate academic level and the need for personal and critical reading within the new frameworks. I sometimes get the feeling that social aspects of building groups and dealing with the components of alternative teaching come at the expense of devoting myself to the actual learning material

Culture of Assessment in Israeli Academic Institutions

147

and subjects. I sometimes sense that instead of communicating my academic knowledge to my students and presenting them with conflicts and dilemmas, I’m busy functioning as an elementary-school teacher who is occupied with solving social problems. Excuse me…but there is also the matter of the stagnation in my own academic development. I have to devote many hours to reading group projects, checking their authenticity, and to procedures that precede the projects. Assessing group work mainly demands an investment on my part.

Yair, Nava, and Anat, the lecturers from the teachers-training colleges, explained that the colleges attribute great importance to the advancement and modernization of learning-teaching-assessment. The issue is always on the agenda. The college devotes conferences and meetings to the issue, and lecturers who develop new methods of learning-teaching-assessment are given an opportunity to be heard and to share what they have done. Indeed, the advancement of teaching and assessment is declared openly as an important component of the lecturers’ role in the institution. However, all three lecturers’ statements reveal a distinct gap between the declaration, augmented by verbal and written support offered to those who advance teaching and assessment, and the demands and dictates which—according to the college administration— stem from the Council of Higher Learning for lecturers to conduct research as a prerequisite for receiving academic rank. This seeming contradiction redefines the role of the lecturer by prioritizing research at the expense of fulfilling the objectives of implementing varied methods of teaching and assessment. Nava, a lecturer in a teacher-training college, describes the key role played by the president of the college in changing the perception of the lecturer’s role: The president of the college is personally involved in the issue of advancing teaching-learning but slightly less in the matter of assessment. I feel that there is no culture or policy regarding assessment. The president expresses both oral and written praise for taking action in these issues. However, I can personally testify that when the president and I discussed my academic promotion, he suggested that I “leave everything” [else] and publish another three articles so that I can be promoted. It’s clear that to leave everything implies that I should stop investing my time in alternative assessment and teaching.

D.

Placing responsibility for academic advancement of minority students upon other entities in the institution

The interviews revealed that the participants are aware of the need for helping students from different cultures and offering them an opportunity to become successfully integrated into society. All participants from all three types of academic institutions feel that minority students require constant help to narrow the achievement gap between them and the majority group. However, they said that the institution—not the lecturers—must be responsible for dealing with the issue. Other entities within the institution must support struggling students—mainly among the increasing number of Arab students—and help them pave their way in the college. While mainly emphasizing learning skills, they made no reference to the vital connection between teaching and assessment and these skills. Dror, a university lecturer, distinguishes between the sets of skills that need to be advanced: lecturers are responsible for cultivating cognitive skills, while the Deaconate of Students is responsible for developing “soft” skills:

148

9 Culturally Responsive Culture of Assessment

The main skills which faculty at the college tend to develop in students are cognitive skills, critical thinking, creativity, authentic and independent thinking, critical scrutinizing of information, developing ethics, etc. Less emphasis is placed on developing personal and interpersonal skills. The discussion about these “soft” skills began recently, and there is room for learning them in depth. The social and multicultural areas are currently in the hands of the Deaconate of Students and various entities that help students to become active within the university framework.

Like the university lecturers, the regional colleges lecturers, too, designate responsibility for dealing with Arabic-speaking students to the Deaconate of Students. Maya’s statements represent the view of her colleagues: The Deaconate of Students is the entity that should guide, advise, and provide assistance for Arab students, and that is what our Deaconate of Students does. More and more social workers, assistant teachers, and Arabic-speaking teachers from various departments initiate meetings with students, offer them options for receiving tutorial help, and accompany them throughout the duration of their academic studies. When the deaconate of students approaches me about a particular student, I’m always happy to help. I provide material from the lessons, suggest articles, or fill out forms - whatever is needed. The activities make it much easier for these students to become integrated into classes, workshops, and seminars.

Dror, a university lecturer, feels that when there is a large sub-group of students from a different culture, their needs cannot be ignored. However, he also stressed that the responsibility for dealing with these difficulties, which constitute an obstacle to success in academic studies, must be primarily handled by the Deaconate of Students. Their difficulties, needs, and rights to equal opportunities must not be ignored, and any university that is interested in successfully assimilating Arab students must do it with the help of the Deaconate of Students, experts in education, and experts in Arab culture. If possible, organizations that specialize in multi-culturalism should also be consulted. There must also be a willingness, knowledge, and experience as well as financial resources devoted to this objective.

Dror describes a local program which features a more balanced blend between lecturers who teach in the department and other entities to which he attributed responsibility for advancing skills to help minority students become better integrated in the department. In actuality, fruitful cooperation and joint responsibility exist in the “mentoring program”, The mentoring program (lecturer-student meetings): This program assigns students to members of the faculty who are interested in serving as mentors. Participants in the program include first-year students and advanced students who are about to be interviewed before they begin their specialization. The meetings take place twice each semester and are conducted in a comfortable and informal atmosphere. There is also a series of structured meetings and a social - academic meeting that focuses on providing information about career options. During these meetings attorneys share their personal stories and explain about the essence of their work and its daily characteristics.

Culture of Assessment in Israeli Academic Institutions

E.

149

Barriers to the advancement of teaching-learning-assessment and adapting them to the individual

The content analysis identified a variety of barriers to implementing methods of teaching and assessment that are designed to support various student population groups, and barriers to adapting them to individuals and their abilities: Lack of suitable training for lecturers who do not know how to make teaching and assessment methods more efficient; lack of commitment to a common vision; lack of resources such as time; and disparities in learning skills among students from multicultural groups.

Lack of Suitable Training The interviewees referred to their commitment to advancing minority students from an academic standpoint. This is an explicit objective of the regional colleges, which were established to provide academic education for residents of the periphery— particularly Arab students who have been encouraged to study in the regional colleges that are in proximity to their places of residence. However, the interviewees had difficulty translating this commitment into teaching and assessment, explaining that they feel that the difficulty stems from a lack of suitable training. For example, Yonatan, a lecturer in a regional college, understands the commitment but admits that he “doesn’t understand how it can be expressed in assessment.” The colleges open their doors to Arab students. That is part of their integration policy, but it’s also a commitment. It’s a commitment to take these students’ culture into consideration, to become familiar with it, to consult with those who are familiar with it, to respect it, and to integrate it within the college. It means taking the fast during the month of Ramadan into consideration as well as festivals, holidays, family obligations, and other cultural characteristics that are different from the culture of the [Jewish] majority. This has to be expressed in teaching, but I don’t know how this can be expressed in assessment. I know that it is expressed through help in becoming integrated into the college, It’s not perfect. There is more to learn and do, but we’ve begun…

Irit, a lecturer in a university, highlighted the importance of the Center for Advancement of Teaching but testified that the center at the university does not always rise to the expectations. The center provides lecturers with limited tools for coping with the challenge of adapting teaching and assessment to the individual. She said, Our university is definitely moving in that direction. Out of an awareness of our academic role and the duty of the research colleges, we established an active center for those interested in advancing teaching, I hope the center will promote the issue of alternative teaching together with the appropriate assessment tools. Meanwhile, a lot of lip service was paid to the topic in the workshop in which I participated, but I didn’t gain any wisdom about implementing it…

150

9 Culturally Responsive Culture of Assessment

Lack of Commitment to a Common Vision Anat, a lecturer in a teacher-training college, offered another explanation as to why ‘other’ assessment methods she used have become obsolete. Her statements reveal a sense that a discrepancy exists between the “vision” and guiding principles that were stated and what is actually happening in reality. There is supposedly a “common vision” about assessment, yet many lecturers—including those who hold high administrative positions—do not act to uphold the vision and continue to perform assessment by traditional methods. This makes it difficult to create a united front that expresses the vision: Anat recounted, During my classes using the problem-based learning approach, students in each group had to assess each other. In order to do this, we compiled an index for assessment. That [activity] took almost the entire lesson. Some of the students disliked the idea that they would be assessed by their peers and that they would have to assess them. They asked if it was not a waste of time, wondered if we would manage to cover all the material in the syllabus, and if not, would they have to learn it by themselves. What I found most irritating was that they asked why I am the only one who teaches in this cumbersome manner. They claimed that even department heads, who teach them more than one class, teach the material clearly, summarize the articles, and give them one closed exam. ‘Everything is so structured and simple.’ The things several students said felt like a painful slap in the face.

Lack of Resources Anat, Nava, and Yair referred to the multi-culturalism that exists in their teachertraining colleges and spoke about the need for “alternative” teaching-learningassessment. According to Anat, alternative methods encourage work groups and develop social and interpersonal skills, “Alternative teaching relates to all students, their cultures, and their skills that are taken into consideration.” However, it is “extremely complex and requires extensive investment and thought.” Aviv, a university lecturer, feels that it is impossible for lecturers in the various fields to “stop and devote a lot of time” to innovative teaching methods and, mainly, to vary their methods of assessment. The reasons he gives for this are lack of resources such as time, training, and the need to deal with social conflicts. He himself is unwilling to “pay the price” that alternative methods require, saying: I don’t have enough time for games or group research projects either from the standpoint of what I plan to complete during the academic year, or from the aspect of time that is required to implement different procedures of group work, or from the aspect of the chaos and social conflicts that you need to cope with when you create groups during the first years of academic studies. I’m not even going to talk about the time it takes to read the reams of pages students write and to offer them the feedback they expect to receive, which is justified. If my teaching assistant would read them, then the comments would merely be technical, and that’s not what’s intended… I’m not good at constructing indexes or wracking my brains about assessment. I know the price of this from my wife, who is a lecturer at a teacher-training college.

Culture of Assessment in Israeli Academic Institutions

151

Nava also explained that in her teacher-training college there was clear-cut guidance to use various assessment methods, but as time went by, lecturers reverted to summative methods of assessment. The main reason for this were numerous instances of students’ dishonest behaviors and the introduction of more sophisticated methods of checking exams that were attractive to lecturers because they saved time: Until two years ago there were explicit instructions at the college that, in addition to a final exam, students must be allowed to submit a paper, portfolio, or project that would enable them to express themselves and their uniqueness. However, the frequency of instances of students copying each other’s papers and projects, together with mixing up tests and the relative ease of grading them by computer, led lecturers to use a final exam as the main tool for assessment.

Disparities in Learning Skills Among Students from Multicultural Groups Yair, a lecturer in a teacher-training college, claims that alternative assessment is liable to pose a challenge with respect to heterogeneous groups that are composed of minority students who come from conservative local education systems and have difficulty getting accustomed to the methods used at his college. He suggests solving the problem by adding an additional semester for students who are experiencing difficulty in order to bridge the gaps. He explained: During the first year, I teach classes of women in which half the students are Arabs and half are Jews. There is a huge gap in the conditions for success that are open to Arabic-speaking students – mainly because they are Arabic speakers and all the lectures and activities are in Hebrew! In addition, many come from a teaching-learning culture that is very different, conservative, and suppresses criticism of the contents and of the lecturer. Most of them explained to me that you don’t argue with the teacher and that when you write the exam, you simply have to write what the teacher said in class. I raised the issue of the gaps with the president of the college and discussed ways of taking them into consideration and dealing with them. The issue also came up in discussions with my colleagues who are similarly dealing with the gaps I described. Most of the responses leaned towards adding a semester for Arab students before they begin studying in the preparatory year. However, it seems to me that we also have to think about possibilities for effectiveness and inclusion that are perhaps better and fairer.

Discussion Analysis of the interviews generally shows that no declared structured culture of assessment exists in the academic institutions that were examined. No institutional efforts, such as constructing a common vision or facilitating cooperation among administrators and policymakers (Weiner 2009), were made in this context. The interviewees described initiatives for advancing teaching and learning in their institutions; however, their statements reveal that the issue of “alternative assessment” or

152

9 Culturally Responsive Culture of Assessment

“other assessment” was referred to only sporadically, and assessment was referred to as the weak link in the teaching-learning-assessment triad. While there were instances of local initiatives that had the objective of advancing teaching and learning, they rarely involved culturally responsive teaching methods. The participants’ general descriptions of institutional pedagogical policies seldom cited any guiding principles that focused on culture of assessment, and no mention was made of assessment that was adapted to groups of minority students. The issue arose only when the participants were asked explicitly about assessment in general or assessment of minority students in particular. It appeared that the institutions lacked any clear-cut policy in this area. One of the barriers to designing a culture of assessment is the institution’s emphasis on commitment to academic freedom for its faculty members. The principal claim is that the lecturers themselves determine the level of academic quality, and it is their view that adapting teaching or assessment methods to suit minority students is liable to lower the level of academic quality. This claim is not new and is supported by several studies such as that of Walvoord (2010). This perception is one of many barriers, the main one of which is the perceived role of the lecturer in the college. The interviewees make a point of distinguishing themselves from school teachers. They feel, for example, that while teachers need to increase their knowledge of teaching and assessment, college and university lecturers should be occupied with research in order to further their academic career. The interviewees also stressed that they lack knowledge and training in progressive methods of teaching and assessment. Those who have some knowledge in this realm testify that investing effort in implementing these methods in large classes consumes precious time at the expense of conveying knowledge and assert that such methods are better suited to high schools than to colleges and universities. This dichotomy between teaching and research is further expressed in contradictory messages—both implicit and explicit—that are transmitted by the administration and policymakers in the institutions, primarily the colleges. The institution’s professed obligation to teaching and assessment methods that are suitable to the entire student body, with emphasis on weak students, is visibly pronounced and frequently even serves as a marketing and advertising strategy. However, lecturers are constantly exposed to implicit messages to conveying the institution’s expectation that they devote a large portion of their time to research. These contradictory messages impede the ability to create uniformity based on the overtly endorsed message. Despite the interviewees’ recognition of the complexity of teaching in heterogeneous classes and the importance of suiting teaching and assessment methods to culturally diverse groups in the class, they continued to focus on the barriers. These included a lack of resources necessary for incorporating advanced methods of teaching and assessment in higher education, such as a professional support staff capable of dealing with the variety of challenges that developing and implementing these methods entails. Due to these challenges, there is a greater temptation for instructors to rely on traditional methods of assessment, especially summative exams consisting of easy-to-grade multiple-choice questions. This is aided by technology that eliminates the time-consuming burden of grading exams manually. Furthermore,

Discussion

153

to their credit, traditional assessment methods lower the odds of academic dishonesty; therefore, lecturers view them as being more objective.

Implications and Recommendations The principal initiative for training lecturers to develop advanced methods of teaching and assessment emanates from the Center for Advancement of Teaching, a pedagogical body which has representation in all academic institutions in Israel. All the participating lecturers noted that this body offers training to novice instructors as well as those who request assistance chiefly in refining their teaching methods. However, they noted that with the minimal emphasis placed on the assessment aspect, the effectiveness of these centers is debatable. Our suggestion is for the centers to expand their title and the definition of their objectives to include the advancement of assessment in addition to the advancement of teaching. This would fill the void of the lacking assessment component in the teaching-learning-assessment triad. It can be said that the participants exhibited a clear awareness of the needs of groups of students from different cultures. The difficulties experienced by Arabicspeaking students are well-documented, and the study participants expressed a sincere desire for their minority-group students to be offered an opportunity to succeed equal to that of their majority-group peers. However, they contend that the road to equity and success is not part of the lecturers’ job description but is, in fact, the responsibility of the multi-professional entities that operate outside the classroom to assist and mentor students. These entities consist of experts in all matters of social support as well as young Arabic-speaking mentors who represent academics from minority groups who have experienced the same phenomenon. These entities can serve to narrow the gap between majority-minority students and help the latter to meet the institutions’ academic demands without having to modify them. Despite the lack of a clear-cut established culture of assessment in the academic institutions, it appears that the initiative for establishing a culture of assessment that is sensitive to minorities, their culture, and their needs must arise from the bottom up rather than being directed from top to bottom. The interviews reveal that international academic intervention encourages openness toward teaching-learningassessment and adoption of alternative and innovative learning environments, alternative assessment tools, and innovative teaching strategies. Institutions are interested in attracting foreign students, and in order to do so they are “compelled” to introduce modifications in their teaching and assessment methods that will be acceptable to those students’ institutions abroad, and to enable them to receive accreditation for the courses they take in the host college or university. This path of training will have implications generally for the teaching and assessment methods of the lecturers in the institution. It may be inferred that the seeds of a culture of assessment aimed at advancing learning are likely to sprout from the need to establish international connections in the academic world.

154

9 Culturally Responsive Culture of Assessment

Other examples of bottom-up activities can be observed in local initiatives on the part of academic departments that develop and maintain extensive support systems tailored to the individual. These types of programs, whose objective is to advance students’ skills within an informal, personal atmosphere, have been deemed successful. Sometimes entire departments embrace assessment and are regarded as models at the university or college, while other departments may not necessarily follow suit. Yet, these innovative programs may potentially lead to the creation of a model that will supplement the traditional methods and establish the beginnings of a program that could yield tangible results in other departments as well. However, for a culturally responsive culture of assessment to be all-encompassing, communication must be established between the various entities within the institutions with the ultimate aim of engaging them and adapting the model to their unique needs, thus creating an optimal culture of assessment. It appears that against the background of the numerous barriers described in this chapter, together with local initiatives, implementing student–centered methods of learning and assessment may be difficult for lecturers. The majority of policymakers have difficulty providing effective or meaningful assistance via the entities that are designated to do so within the institution. Consequently, the interviewees raised the possibility of designating the care of weak students to other entities—primarily to the students’ deaconate. These agencies must assume responsibility for imparting both soft skills and cognitive skills by means of a mentoring system, group practice, and other methods. In the end, however, the work of creating a successful culture of assessment that supports the learning of the most vulnerable students should not be the sole responsibility of any single institutional entity, no matter how efficient it may be.

References Alt, D. (2018). Science teachers’ conception of teaching, attitude towards testing and their use of contemporary educational activities and assessment tasks. Journal of Science Teacher Education, 29, 600–619. The Association of American Colleges and Universities’ Board of Directors Statement on Academic Freedom and Educational Responsibility. (2006). Academic freedom and educational responsibility. Retrieved from https://www.aacu.org/about/statements/academic-freedom. Aydeniz, M., & Southerland, S. A. (2012) A national survey of middle and high school science teachers’ responses to standardized testing: Is science being devalued in schools?, Journal of Science Teacher Education, 23(3), 233–257. Baas, L., Rhoads, J. C., & Thomas, D. B. (2016). Are quests for a “culture of assessment” mired in a “culture war” over assessment? A Q-methodological Inquiry. Sage Open. https://doi.org/10. 1177/2158244015623591. Farkas, M. G. (2013). Building and sustaining a culture of assessment: Best practices for change leadership. Library Faculty Publications and Presentations. 78. Retrieved from https://pdxsch olar.library.pdx.edu/ulib_fac/78. Freedman, I. (2013). Formative assessment as an organizational culture – the class in the center. Maof & Maase, 15, 1–14. (Hebrew)

References

155

Fuller, M., & Skidmore, S. T. (2014). An exploration of factors influencing institutional cultures of assessment. International Journal of Educational Research 65, 9–21. Fuller, M., Henderson, S., & Bustamante, R. (2015) Assessment leaders’ perspectives of institutional cultures of assessment: a Delphi study. Assessment & Evaluation in Higher Education, 40, 331– 351. Kotter, J. P. (1995). Leading change: Why transformation efforts fail., Harvard Business Review, 73, 59–67. Lee, K. C., Rudolph, M. J., Assemi, M., Bray, B. S., Daugherty, K. K., Karpen, S. C., … Weck, M. A. (2019). Factors associated with cultures of assessment at US schools and colleges of pharmacy. Currents in Pharmacy Teaching and Learning, 11, 129–138. Maki, P. (2010). Assessing for learning: Building a sustainable commitment across the institution (2nd ed.) Sterling, VA: Stylus. Skidmore, S. T., Hsu, H-Y., & Fuller, M. (2018) A person-centred approach to understanding cultures of assessment. Assessment & Evaluation in Higher Education, 43, 1241–1257. Walvoord, B. (2010). Assessment clear and simple: A practical guide for institutions, departments, and general education (2nd ed.). Hoboken, NJ: Jossey-Bass. Weiner, W. F. (2009). Establishing a culture of assessment: Fifteen elements of assessment success – how many does your campus have? AAUP Academe Online. Retrieved from http://www.aaup. org/AAUP/pubsres/academe/2009/JA/Feat/wein.htm.

Epilogue

Significant changes in migration patterns throughout the world have compelled educational systems to cater to the changing needs of minority students or those with a migration background. This has resulted in the elevated importance of complementary formative assessment strategies that offer support throughout the K-12 learning process. These strategies are gaining prominence in higher education as well. This book suggests that these learning and assessment strategies also have the potential to address and resolve issues of inequity in higher education and to be adapted to suit students of different cultures. It mainly shows that in the process of developing a culturally responsive pedagogy, key stakeholders should realize that some instructional practices are liable to inhibit academic achievement among students whose mother tongue is not the language of instruction. Therefore, assessment methods need to be implemented with careful consideration for personal and cultural aspects; and instructors need to understand the factors responsible for advancing or hindering learning among minority students. It appears that students’ learning activities during their academic studies are inseparable from their prior personal, domestic, social, educational, and occupational experiences. These experiences are frequently influenced by the individual’s culture and by the differences between it and the culture of their lecturers and colleagues. These differences may detrimentally impact students’ learning performance. Therefore, there is a need to plan and implement formative methods of teaching and assessment that take cultural differences into consideration and to understand how assessment can mitigate and ultimately eliminate social disparities. It is also important to understand the structure of the heterogeneous composition of the class in order to adapt teaching and assessment methods to the entire array of students and to engage them in an inclusive learning process. This book asserts that the use of assessment tools that are effective for the majority group but are not always sensitive to minority students and their needs puts the latter group at a disadvantage. This is liable to result in a decline in their quality of education, create alienation between the student and the institution, and result in disparities

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. Alt and N. Raichel, Equity and Formative Assessment in Higher Education, https://doi.org/10.1007/978-3-030-71644-8

157

158

Epilogue

in achievements. It is important to acknowledge that students have different ways of demonstrating their knowledge and skills, and we instructors need to use assessment methods that produce proof of these differences and take personal and cultural differences into consideration. Our assessment approaches must afford students an opportunity to express their abilities and enable them to overcome challenges and promote solutions that help them pursue higher education. Continuing to assess students of diverse backgrounds and cultures as if there is no difference between them will merely perpetuate inequality and widen the disparity in their achievements. If we wish to enable students from different cultures to succeed, we must offer them equitable opportunities to learn the course material and acquire lifelong learning skills. We can do this by using a wide range of assessment formats which take students’ culture into account. This book offers several insights regarding the assessment of students in a multicultural society that will help decision-makers in academic institutions formulate a comprehensive vision and a responsive and sensitive policy on this issue. Since it is impossible to detach the assessment component from teaching and learning, our conclusions and recommendations, which are discussed and elucidated below, are directly connected to these processes and refer to the entire teaching-learningassessment triangle. The recommendations are presented along a continuum ranging from the intrapersonal to the interpersonal facets and extending from them both to the institutional facet.

Recommendations Related to the Intrapersonal Facet This facet refers to the ability to assess oneself—known as self-assessment—using explicit criteria to evaluate and foster the individual’s skills essential for optimal learning. Using self-assessment tools has been shown to help students understand the precise criteria for assessing the skills and other requirements of the academic system in this context from the beginning of their studies. The students gradually become more aware of their skills, strengths, and weaknesses with the help of reflection throughout their learning process. Identifying these competencies enables them to understand what type of help and guidance they need to seek and from whom. At the same time, for their part, instructors are better able to adapt their practices for teaching and helping their students in a personalized manner and to provide effective, targeted feedback based on their students’ self-assessment. For example, establishing criteria for self-assessment of an activity, such as creating concept maps, might help students develop their abilities of critical analysis and synthesis of ideas, while using a set of predefined criteria for analysis and assessment. This process might induce students to search for information to explicate the ideas they have raised, to write their claims in a concise manner, and to establish the connections between them in a coherent way, while elucidating the logic of the connections between these components. Self-assessment empowers students to reflect upon, and collect information about their own performance, to assess the

Epilogue

159

quality of their learning, and to identify their strong and weak points according to predefined and clear-cut criteria. This authentic assessment process encourages the individual to assume responsibility and control of his/her own learning. These selfassessment skills bear considerable importance in both higher education and the occupational world. The reflective journal is an additional tool with which students can assess their own learning styles and point to ways of making them more effective. From the intrapersonal standpoint, in addition to improving their [second] language skills, this practice can be effective for minority students who have difficulty expressing themselves in a multicultural classroom setting. Writing in a journal enables them to express themselves freely on matters such as the challenges entailed in interacting with other [majority] students and/or whatever emotional or cultural barriers they may be facing. Reflective journaling also enables them to think about ways to improve their learning while coming to recognize and appreciate their strengths in the process. However, it was also observed that minority students are unaccustomed in their culture to this method of integrating reflection in their learning. They fear that disclosing their weaknesses will harm their image in the instructor’s perception and evaluation of them. For this reason, it is necessary to define the objectives of writing in a journal and to discuss the importance of self-awareness and self-criticism, to encourage students to identify the challenges they face, and to contemplate together with them about how to deal with these challenges. It is also highly advisable to point out students’ strengths and thereby motivate them to take full advantage of their potential, with the ultimate aim of empowering them. Providing helpful and consistent feedback will gradually increase students’ trust in the effectiveness of this tool and their ease in using it to their advantage. Finally, the intrapersonal dimension refers to the tools that promote self-regulation for learning, such as the use of digital badges. These badges may serve as a motivational tool for minority students by creating a detailed, step-by-step progress plan that breaks the main assignment down into smaller parts. In this way, the students could receive feedback for each separate part of the assignment. They could then correct their performance, set new objectives as they progressed, plan the rest of the assignment, review and hone the learning strategies they had used, and implement further changes if necessary. They could then better prepare themselves to complete the rest of the learning assignment successfully. With all of these benefits, teachers should be encouraged to use such digital platforms to scaffold students’ learning differently and gradually lead them to achieve their goals.

Recommendations Related to the Interpersonal Facet This facet has two main dimensions. The first relates to the relations between minority students and their classmates from both the minority and the majority groups. The second dimension pertains to the rapport between the student and the teaching staff. In both aspects, it is important to note that the transition from a teacher-centered to

160

Epilogue

a student-centered approach may not in itself result in equitable relations between students. The latter approach might tend to blur the balance of power between instructors and students. However, if not implemented properly, it is liable to result in the creation of a new focal point of hierarchy, i.e., between minority and majority students, thereby widening the disparities between the two groups even further. The balance of power between minority students and their majority counterparts comes into play when applying strategies such as peer assessment. Creating a heterogeneous group dynamic is likely to establish peer assessment that is based solely upon professional criteria. This is no simple matter, since students might be inclined to remain within their comfort zone and to interact exclusively with members of their own cultural group. This situation is liable to hinder students’ ability to obtain the mutual trust that serves as the basis for peer assessment. Effective inter-cultural encounters, which many students experience for the first time in academic institutions, are prerequisites for developing skills for peer assessment, giving and receiving constructive criticism for support and growth in the learning process, as well as for enhancing future personal and professional development and growth. This experience is not devoid of challenges. Instructors must recognize that the intercultural encounter is liable to present barriers to interpersonal and inter-group cooperation. These barriers stem from gaps between majority and minority students in terms of language, learning skills, and prior life experiences. Assigned work groups, in and out of the classroom, should be heterogeneous and comprise a balanced number of students from the different cultural groups so as to mitigate the difficulties that are involved in integration. The teacher’s role is to assess the abilities of the participants in the learning groups and to offer proper support for minority students who are experiencing difficulty, so that they will not be regarded as a burden on the majority group. Such mitigating support can come in the form of a mentor who speaks the minority students’ mother tongue and is familiar with their culture. These mentors, such as third-year students, who have already successfully undergone the process of entering the academic world, might serve to enrich minority students’ academic, interpersonal, and intrapersonal abilities. Finally, the issue of cultural differences must be a topic for legitimate, transparent, and open dialogue within the heterogeneous classroom. Teachers and students should discuss the challenges that typify the learning and assessment processes in this context and even present suggestions for ways to alleviate tensions that may arise in the class. The ability to communicate with members of a different culture is deemed one of the core skills of the twenty-first century. Hence, it is important that this skill be declared officially as a central learning outcome by teachers and students alike. Evaluating the acquired skills can be done by collecting and analyzing qualitative observations and quantitative data before and after activities, as well as reports on how individuals are progressing, their contribution to the group, feedback for the aspect of group dynamics, and students’ reflection on the intercultural interactions taking place. This assessment may enhance awareness among the participants as to the importance of intercultural discourse and ways to improve it, toward the ultimate goal of attaining better interpersonal skills.

Epilogue

161

The second component in the interpersonal dimension is the connection between minority students and the faculty. This connection often represents a significant turning point for students who come from conservative educational systems that elevate the teacher’s status and place him/her at the center. We therefore feel that the faculty must understand this point of origin and gradually nurture the minority students’ trust in a teaching method that invites new and unfamiliar learning experiences. For example, if we are contemplating a transition to peer assessment, we must consider how to mitigate the transition from a singularly teacher-centered approach to a student-centered approach for those who are accustomed to attributing primary importance to the teacher in student assessment. It is advisable to use a tool such as a reflective journal to moderate the transition. Faculty can use this tool to adapt their teaching methods to individuals, based on the challenges the students record in their journals. This dynamic creates a dialogue between the two sides that enables teachers to better ascertain and understand their students’ needs. By doing so, faculty members can implement teaching and assessment practices that will have a greater impact on their students’ learning. Equally important is that this exchange of ideas and provision of customized guidance can potentially raise the level of trust between the students and the teacher—something that is particularly significant if the teacher belongs to the majority culture and the student is a member of a minority group. Elevating the level of trust between minority students and their teachers can be accomplished by using alternative assessment tools while still taking cultural aspects into consideration. For example, teachers should be sensitive to cultural differences when delivering feedback. Offering feedback publicly might be perceived by some students as a negative experience. Some students may prefer to receive personal feedback in a private setting. It is therefore important to become closely familiar with these preferences, which might be linked to cultural influences, and to attune the channel of feedbacking to students’ preferences. Adapting feedbacking encounters in this way can contribute to strengthening the personal rapport between teachers and students, thereby increasing the latter’s trust in this type of communication and encouraging them to more widely and willingly seek feedback. Another recommendation related to the interpersonal aspect is to train student mentors who can help teachers successfully implement formative assessment methods. These mentors may be advanced students or graduates of the study program who offer their assistance to minority students and speak their mother tongue. The ability to communicate with a person who comes from their own cultural background and speaks their language can potentially increase minority students’ confidence in their ability to complete assignments successfully. The use of students’ own languages can promote successful learning experiences, increase students’ engagement, and enable them to gain a more sophisticated understanding of the concepts being taught. Using the student’s mother tongue can support students to better understand the teacher’s questions or instructions for the given activity and to find the right words to express what they want to communicate. This might provide minority students with equitable learning opportunities in that they may feel more comfortable approaching their mentor, rather than the lecturer, to receive feedback on their work. This stems from cultural barriers that may involve feelings of embarrassment that

162

Epilogue

often accompany encounters with faculty. In addition to training mentors, various ways to strengthen lecturer–mentor–student inter-connections should be weighed. This would enable lecturers to better understand the mentoring process, obtain indepth feedback from the mentor(s), and thus further the students’ cultural integration and academic progress.

Recommendations Related to the Institutional Facet This facet involves establishing a culturally responsive assessment culture within academia. Academic institutions attended by students from different cultures are expected to establish teaching-learning-assessment policies that are sensitive to cultural differences, featuring a clear policy for the assessment of culturally diverse students at an institutional level. The strategies for formative assessment need to be carefully reviewed and revised to cater to students’ varying needs. Higher education institutions evidently do not uphold a clear, uniform policy regarding the assessment of students from different cultural backgrounds. It is also evident that, while faculty members overall declare their appreciation for cultural diversity, this support for cultural diversity is not necessarily translated into the institutional culture. To tackle these challenges, professional development opportunities should be made available for staff to hone their skills in integrating culturally relevant materials into teaching practices. The training provided to faculty should be adequate and well-suited to enabling them to support, communicate, and assess students of different cultures. It is evident that not enough emphasis is being placed on culturally responsive assessment despite the rhetoric that endorses interculturalism. As such, there is a need to evaluate the scope of culturally responsive professional development opportunities available to faculty. It is no wonder then that, considering the inadequate culturally responsive training provided to faculty, the perceived supreme importance of academic freedom, the perceived predominant role of the lecturer as a researcher rather than an instructor, and an overall lack of resources, no comprehensive assessment culture exists that is sensitive to the diverse cultural needs in the institutions that were examined. To increase the level of professionalism in assessment, which appears to be lacking, such major inhibitors to introducing more innovative and effective assessment practices must be removed. Formal professional training for the use of innovative assessment approaches is likely to influence staff’s willingness to integrate new pedagogical ideas, educational policies, appropriate practices in assessment design and implementation, and, in the context of this book’s core issue, culturally sensitive and responsive assessment strategies. Responsive assessment practices can be initiated by instructors—beyond their traditional role—in small steps. Examples of such initiatives are a system of extracurricular activities that provide learning opportunities featuring experts on certain subjects and that support the training of Arabic-speaking mentors who are themselves minority students or graduates who have completed the same study program, and who in turn guide novice students. This system can serve to eliminate the academic and

Epilogue

163

social gaps between majority and minority students and enable the latter to fulfill the academic requirements. It is therefore recommended to higher education institutions to consider ways to establish and support such initiatives. International projects can also be an avenue by which to advance openness toward adopting different, innovative learning environments, formative assessment tools, and advanced teaching strategies. Such initiatives may impact higher education institutions which seek to develop support systems tailored to the individual. Examples of such international initiatives are European capacity-building projects such as Erasmus+ in the field of higher education. These transnational joint projects, based on multilateral partnerships, may match the objective of elevating awareness toward culturally responsive assessment strategies as they address the challenges facing higher education systems and institutions worldwide. However, it should be noted that no matter how effective these initiatives might be, they cannot be regarded as a general culture of assessment that is totally sensitive to students’ cultural needs. An overall assessment culture must be all-encompassing, include a common vision, and obtain the cooperation of all entities within the institution. Effective cooperation among these partners requires communication, a willingness to take risks, and a sincere intention to influence the culture of assessment with the ultimate aim of improving learning among students in a multicultural setting. The experiences that have been presented and analyzed in this book can help the reader to better understand the components that need to be included when building teaching-learning-assessment programs that are sensitive to the various cultures that comprise the human mosaic of the contemporary campus. In doing so, we must consider the intrapersonal and interpersonal facets that are influenced by cultural characteristics and the intercultural encounters that impact this process. The component of assessment, which is the underlying foundation of this book, must be sensitive to cultural differences and altered to suit them optimally. Against the background of difficulties that minority students encounter in their academic studies, we have shown throughout this book how the use of a variety of formative assessment tools has the promising potential to support the learning of culturally diverse students. We feel that not having appropriate formative assessment is liable to leave many minority students behind, lacking the proper support systems and methods to provide a fitting response to their needs. To enhance the benefits of formative assessment, a vision with compatible objectives and strategies should be implemented within the institution’s assessment culture. This will ultimately narrow the gaps between students by developing a set of learning skills in harmony with the institution’s objectives and the desired image of its graduates. A well-defined vision and well-planned strategy will reflect what graduates should be capable of doing when they complete their studies, including skills for intercultural collaboration. There is also a need to rethink the formula that would strike a balance between teaching and research, to strengthen the teaching component in the definition of the lecturer’s role so as to incentivize lecturers to become engaged in advancing their teaching and assessment practices. Indeed, resources and incentives must be allocated to facilitate the implementation of individually-tailored assessment methods, and effective training must be provided as well. Furthermore,

164

Epilogue

there is a need to communicate local initiatives within the institution and to make the faculty, the administration, and key stakeholders aware of them. It is important to examine which assessment methods have proven helpful in developing skills and which need improvement. This information must be brought to the attention of all parties involved in developing a common vision, and it can serve as an impetus for generating a philosophy that nurtures and galvanizes the newly formed assessment culture. This book has examined the application of several innovative, formative assessment tools used with the aim of supporting learning for students of different cultures and ethnic backgrounds. Documentation of additional modalities and activities performed in other multicultural institutions, that have similarly determined a key objective of eliminating intercultural disparities, will certainly contribute to the enrichment of this field of knowledge, which, regrettably, has been sorely neglected both in literature and in practice in the realm of higher education. For this purpose, future studies are needed to delve more deeply into formative assessment tools and methods used in multicultural settings worldwide. A much more systematic study would promote a more profound understanding of the various methods introduced in this book for contending with the intercultural challenges discussed within it. Furthermore, it may provide a valuable source of knowledge for understanding and improving the overall quality of higher education.