Service-Learning Capacity Enhancement in Hong Kong Higher Education (Quality of Life in Asia, 14) 9811924368, 9789811924361

Citation preview

Quality of Life in Asia 14

Grace Ngai Daniel T. L. Shek   Editors

Service-Learning Capacity Enhancement in Hong Kong Higher Education

Quality of Life in Asia Volume 14

Series Editors Alex C. Michalos, University of Northern British Columbia, British Columbia, MB, Canada Daniel T. L. Shek, Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hunghom, Hong Kong Doh Chull Shin, University of California, California, MO, USA Ming-Chang Tsai, Department of Sociology, Academia Sinica, Taipei, Taiwan

This series, the first of its kind, examines both the objective and subjective dimensions of life quality in Asia, especially East Asia. It unravels and compares the contours, dynamics and patterns of building nations by offering innovative works that discuss basic and applied research and emphasizing inter- and multi-disciplinary approaches to the various domains of life quality. The series appeals to a variety of fields in humanities, social sciences and other professional disciplines. Asia is the largest, most populous continent on Earth, and it is home to the world’s most dynamic region, East Asia. In the past three decades, East Asia has been the most successful region in the world in expanding its economies and integrating them into the global economy, offering lessons on how poor countries, even with limited natural resources, can achieve rapid economic development. Yet while scholars and policymakers have focused on why East Asia has prospered, little has been written on how its economic expansion has affected the quality of life of its citizens. This series publish several volumes a year, either single or multiple-authored monographs or collections of essays.

Grace Ngai · Daniel T. L. Shek Editors

Service-Learning Capacity Enhancement in Hong Kong Higher Education

Editors Grace Ngai Service-Learning and Leadership Office The Hong Kong Polytechnic University Hong Kong, China

Daniel T. L. Shek Department of Applied Social Sciences The Hong Kong Polytechnic University Hong Kong, China

Department of Computing The Hong Kong Polytechnic University Hong Kong, China

ISSN 2211-0550 ISSN 2211-0569 (electronic) Quality of Life in Asia ISBN 978-981-19-2436-1 ISBN 978-981-19-2437-8 (eBook) https://doi.org/10.1007/978-981-19-2437-8 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Foreword

It is with great pleasure that I present this comprehensive and wide-ranging book about service-learning, published during the year in which we celebrate the 85th Anniversary of The Hong Kong Polytechnic University (PolyU). I joined PolyU in 1994, the same year the institution gained full university status. I have since been able to witness the University’s remarkable progress in education and research. The development of service-learning at PolyU into a rigorous undergraduate academic requirement engaging different disciplines is undoubtedly among the most positive, impactful changes I have seen at the University over the past decade. At PolyU, fostering a strong sense of social responsibility is part of our DNA, exemplified by our motto, “To learn and to apply, for the benefit of mankind.” As a result, we strive to promote civic engagement among our undergraduate students, enhancing their commitment to serving society. In 2020, PolyU received the International Research Award from the International Association for Research on ServiceLearning and Community Engagement for its service-learning programme, which was described as “one of the world’s most impressive and impactful service-learning initiatives in higher education.” In 2022, we celebrated another achievement as the ranking agency Times Higher Education (THE) recognized our service-learning programme as the “Teaching and Learning Strategy of the Year” at the 2022 THE Awards Asia. Our service-learning teachers have also been recognized by the University Grants Committee with two Teaching Awards, and our service-learning subjects have obtained two Gold awards and one Bronze award at the Quacquarelli Symonds Reimagine Education Awards. Our remarkable achievements in service-learning would not have been possible without the hard work and tireless dedication of our colleagues, including the editors of the present volume: Dr Grace Ngai, Associate Professor of PolyU’s Department of Computing, and current head of the Service-Learning & Leadership Office (SLLO); and Professor Daniel T.L. Shek, Chair Professor of PolyU’s Department of Applied Social Sciences, Li and Fung Professor in Service Leadership Education, and Associate Vice President (Undergraduate Programme). They are both distinguished practitioners and scholars of service-learning, whose contributions have local, regional and global recognition. v

vi

Foreword

This volume contains works by colleagues from The Education University of Hong Kong (EdUHK), Hang Seng University of Hong Kong (HSUHK), Hong Kong Baptist University (HKBU), Lingnan University (LU), and PolyU. The authors of the different chapters present empirical findings, theoretical frameworks, and practical experiences which illustrate their shared commitment to promoting service and community engaged learning in Hong Kong’s higher education sector. Readers interested in service-learning will find in this volume not only ideas for servicelearning practice, but also means to make service-learning pedagogy more grounded on validated measures and evidence-based approaches. On behalf of the PolyU community, I sincerely thank the editors and authors of this volume for this much-needed contribution to service-learning scholarship. Jin-Guang Teng President The Hong Kong Polytechnic University Hong Kong, China

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Grace Ngai and Daniel T. L. Shek

2

What Matter in a Quality Service-Learning Course: A Study on Faculty Members’ Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huixuan Xu

17

Faculty Perceived Impacts of the Support from a Centralized Service-Learning Office . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huixuan Xu and Maureen Yin Lee Chan

35

3

4

5

6

Assessing Students in Academic Service-Learning: Faculty Experiences and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rina Marie Camus, Grace Ngai, Kam-Por Kwan, and Daniel T. L. Shek How Hong Kong University Teachers Assess Students in Service-Learning: An Exploratory Study . . . . . . . . . . . . . . . . . . . . . . Grace Ngai, Ka Hing Lau, Kam Por Kwan, Stephen C. F. Chan, and Daniel T. L. Shek Promotion of Teachers’ Readiness for Service-Learning Through a Professional Development Course . . . . . . . . . . . . . . . . . . . . Grace Ngai and Stephen C. F. Chan

1

47

71

99

7

Conceptual Framework for Key Process Ingredients Salient for Effective Service-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Robin Stanley Snell and Ka Hing Lau

8

Validating an Instrument for Measuring the Developmental Outcomes for Students Arising from Service-Learning . . . . . . . . . . . . 137 Ka Hing Lau and Robin Stanley Snell

vii

viii

9

Contents

Investigating Service-Learning Impacts on University Students’ Developmental Outcomes Through a Control Group Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Chad W. F. Chan, Robin S. Snell, and Constance W. Y. Chan

10 Service-Learning in the Context of Business-Community-University Partnership: The Project WeCan in Hong Kong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Daniel T. L. Shek, Xiang Li, Yammy L. Y. Chak, and Li Lin 11 The Impact of E-Service-Learning on Tui-na Teaching in a Chinese Medicine Course—From the Perspectives of Service-Recipients and Service-Partner . . . . . . . . . . . . . . . . . . . . . . . 203 Muhammad Hafiz, Feng Tu, Chun Hoi Cheung, and Kin Man Kevin Yue 12 Action Research as a Tool for Faculty Development in Service-Learning: The Impact of Action Research on Faculty’s Knowledge, Skills and Attitudes Towards Service-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Muhammad Hafiz, Yuki Yuet Yee Choi, Jeffrey Ching To Keung, Joy Lai Ki Lam, and Kevin Kin Man Yue 13 The Developmental Impacts of Interdisciplinary and Inter-Institutional Service-Learning Collaboration . . . . . . . . . . . 239 Chad Wing Fung Chan, Muhammad Hafiz, Yan Ho Siu, Feng Tu, Shane Sheung Yuen Siu, Joy Lai Ki Lam, Robin Stanley Snell, and Kevin Kin Man Yue 14 Co-Designing with People with Special Needs: An Analysis from Contact Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Huixuan Xu and Maureen Yin Lee Chan 15 From Industrial Fermentation to Life Planning: The Meaning of Meaningful Service in a Biology Service-Learning Course . . . . . . 273 Huixuan Xu and Maureen Yin Lee Chan 16 Assessing Community Impact After Service-Learning: A Conceptual Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Robin Stanley Snell and Ka Hing Lau 17 Promotion of Service-Learning in Hong Kong: Experiences Surrounding Collaboration Among Higher Education Institutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Daniel T. L. Shek and Grace Ngai

Chapter 1

Introduction Grace Ngai and Daniel T. L. Shek

Abstract Service-learning is gaining recognition in higher education institutions as an effective pedagogy for holistic student learning, and many universities have integrated service-learning into their academic curriculum in some form. While servicelearning has gained much attention in many Western countries, its development in different Chinese societies is still at its infancy. In Hong Kong, most of the higher education institutions implement some form of academic service-learning. However, service-learning as a pedagogy is not something that is familiar to most higher education teachers and teachers generally hold the myth that service-learning is simply another form of community service, and they lack the skills and knowledge as to how to properly implement service-learning. As such, there is an urgent need to build capacity of university teachers for service-learning. With this background, we have implemented a large-scale teaching development research project with an objective to promote the service-learning capacity of university teachers in Hong Kong. In this book, we outline what has been achieved in this project as well as some issues related to SL. In this chapter, we outline the background of this project and the proposed initiatives to promote capacity in service-learning teachers in the university context in Hong Kong. Keywords Service-learning · Teacher training · Student development · Formal and informal learning · Peer learning · Building community

G. Ngai (B) Service-Learning and Leadership Office, The Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China D. T. L. Shek Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_1

1

2

G. Ngai and D. T. L. Shek

1.1 University Social Responsibility and Student Development A university’s mission is to prepare its students to become productive and socially responsible members of society. In a report titled Aspirations for the Higher Education System in Hong Kong (Hong Kong University Grants Committee, 2010) to the Hong Kong Legislative Council in 2010, it was clearly pointed out that. “Universities also undertake an important task beyond the transmission of academic, disciplinary or professional knowledge. Students should acquire a greater sense of the wider world and the moral or ethical tools with which they can contribute to that world. The experience of university should firmly root an individual’s sense of personal and social responsibility…” (p. 16)

Simultaneously, society invests in its own future by educating its citizens through the university. Hence, university students should actively engage in society and learn not only from the classroom, but also from the community. This is evidenced by the increasing emphasis on the role of universities in educating students to be socially responsible citizens with a heart to serve the community. This responsibility has been reinforced by critical voices attacking the education system in Hong Kong for being too examination-oriented and generally lacking emphasis on holistic development in young people (Shek, 2010; Shek & Siu, 2019; Shek & Wong, 2011). Among the different pedagogies that focus on students’ civic learning, servicelearning has been identified as a high-impact educational practice that can effectively prepare students for citizenship, work and life (Kuh, 2008). There are several different definitions of service-learning. One of the most cited is that of (Bringle & Hatcher, 1995): A "course-based, credit-bearing educational experience that allows students to (a) participate in an organized service activity that meets identified community needs and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of civic responsibility" (p. 112)

Service-learning is a form of experiential learning, in which students learn by doing or observing. Experiential learning, including study tours, site visits and internships, has been part of university education for a long time. Service-learning, however, differs from other forms of experiential learning in that it integrates community service with academic learning and places equal emphasis on the benefits to the students and to the community: (Sigmon, 1994, as cited by Furco, 1996): Service and learning goals are of equal weight, and each enhances the other for all participants. (p. 3)

Service-learning engages students in activities that address human and community needs, together with structured opportunities of reflection to achieve the intended learning outcomes (Jacoby, 1996). Research based on rigorous research methodology has shown that well-designed and implemented service-learning programs can have strong positive impact on students’ intellectual, social, personal and, in particular,

1 Introduction

3

moral and civic development (Celio et al., 2011; Conway et al., 2009; Lundy, 2007; Warren, 2012; Yorio & Ye, 2012). In Hong Kong, there is also encouraging evidence that service-learning does have potential to be a transformative learning experience for students, beyond what conventional classroom-based learning can offer (Ngai et al., 2016; Shek et al., 2017; Yang et al., 2016) and the positive impacts do not differ from traditional face-toface mode and online mode (Leung et al., 2021b; Lin & Shek, 2021a). Specifically, students can improve their critical thinking, problem solving, teamwork and leadership skills, learn to communicate effectively with people from different social or cultural backgrounds, understand more of the problems faced by the underprivileged members of the society, develop empathy for people in need, and become a more responsible member of the community. There are also research findings showing that service-learning benefits the service recipients as well (Shek, Yang, et al., 2021).

1.2 Challenges Arising from Implementing Service-Learning Even though service-learning has a long history, there is much variation in the way that it is actuated by different universities or different faculty and staff. Many of these are due to contextual and cultural variations between different locales or institutions. For example, at liberal arts colleges where small class sizes and seminar groups are de rigueur, one might expect service-learning subjects to focus on the social issues and the associated theoretical concepts, perhaps even through a critical servicelearning lens. At large comprehensive universities, especially those which have a strong tradition in the applied sciences and engineering, one might expect service projects that apply technology to address humanitarian challenges. These diversities, and the somewhat fluid nature of what it means to “meet identified community needs,” can lead to big variances in the interpretation of what constitutes as servicelearning, and the design and teaching of service-learning subjects. The quality-of-life outcome indicators may also differ in different projects with varying contextual and institutional attributes. This unfamiliarity with the pedagogy poses a great challenge for teaching servicelearning subjects which, because of their emphasis on experiential and reflective learning, involve more complexities in their curriculum design, delivery and assessment than classroom-based subjects. The attitudinal and behavioral elements, together with the community service component, make it especially important that service-learning subjects be managed and taught well, lest harm be caused to the community, or disillusionment and cynicism in students (Ngai & Chan, 2015; Wood et al., 2011). As (Billig, 2007) pointed out, students’ learning outcomes from servicelearning are not automatic but rather, highly dependent on how the program is planned and conducted. The challenge of teaching service-learning is especially great in Asian societies where students have been conditioned to be passive learning agents who

4

G. Ngai and D. T. L. Shek

are generally expected to follow teachers’ instructions and teachers are expected and pressured to be authority figures in classroom and society. The morbid emphasis on academic achievement also means that service-learning is easily marginalized as it is perceived to be not “academic enough” and thus a “waste of time” that could be spent on further examination coaching for students. The importance of proper teacher preparation and quality control is even more important when service-learning is a compulsory element in the curriculum. There are many arguments for and against such a policy: detractors argue that “civic responsibility” and “empathy,” both among the main learning outcomes of service-learning, cannot be mandated, and in fact, forcing students to serve the community when they are not such inclined can have a backlash and create students who are even more antagonistic toward the community than they were to begin with; and worse yet, it may “poison the pool” for other students or cause harm to the community. Proponents of such requirements argue that a civic learning requirement (which essentially is what service-learning is about) is no different from the language or mathematics requirements that are common in most undergraduate colleges, and in fact, requiring service-learning has the benefit of reaching students who may not be inclined to enroll in such a program of their own accord. As such, to successfully implement service-learning, we must help teachers overcome their perceptions and see servicelearning in a positive light, including appreciation of its academic rigor and positive impact on the service providers (i.e., students), service recipients and community. In our own studies (Chan et al., 2019), we have shown that whether a student takes an service-learning subject out of interest or to fulfill a requirement is not significantly correlated with their learning outcomes; on the other hand, it is the learning experience within the subject that is the most significant correlator of whether or how well students will learn. A later study (Chan et al., 2021) backed this up by showing that students’ perceptions can and do change; a student who is initially negative toward the idea of doing service-learning may become an advocate if their experience in the subject is a positive one; on the other hand, students who are initially supportive of the requirement may also turn negative given a disappointing learning experience. Unfortunately, it is observed that the expertise in designing and teaching servicelearning subjects varies much among university faculty and staff. For example, a study of international service-learning programs found that the impact on the host community is rarely considered, or assumed to be positive, by the designers or implementers of the programs (Wood et al., 2011). This is a surprising finding since one would assume that, especially given the principle of reciprocity, such a basic issue would be among the first issues considered. This finding is especially alarming given that “benefits of the service to the community” are highly correlated with student learning, and basic common sense suggests that it would not be ethical to simply treat the community as a classroom without giving something back in return. Despite its popularity in the US, service-learning, especially academic servicelearning in the form of credit-bearing subjects, is relatively new in Hong Kong. Given this, most colleagues are not adequately equipped to design, teach or assess servicelearning subjects or projects. There is also much misunderstanding and misconceptions about service-learning on the part of the academic staff—for example, despite

1 Introduction

5

several years of academic service-learning in the curriculum, some colleagues still refer to it as “informal learning,” or conflate it with volunteerism or community service. Some faculty even feel that civic learning is simply another form of student development, which is often delegated to non-academic units such as Student Affairs. Most colleagues also do not appreciate the fact that service-learning is a pedagogy which can be used to consolidate the academic learning experience of the students, and indeed, can be integrated into a subject. All these factors can adversely affect students and bring harm to the community. This suggests that enhancing the development and support to teaching service-learning as an effective pedagogy is needed among Hong Kong higher education institutions. This recommendation is particularly important because constructive and healthy collaboration of researchers across institutions is not common in Hong Kong.

1.3 Designing a Cross-Institutional Project for Capacity Building To build up capacity in implementing service-learning in university teachers in Hong Kong, there are several important building blocks. First, teacher training and education is critical. This training can be in terms of formal professional development or in informal peer learning. Second, construction of a database of good practice would be useful. From the database, colleagues can learn from the best practice which facilitates the implementation of service-learning. Third, to promote collaboration across institutions, it would be exciting to develop collaborative service-learning projects involving colleagues in different institutions. Fourth, to sustain collaboration and mutual support among service-learning teachers and to facilitate informal and peer learning, building up a community of practice dedicated for service-learning. Fifth, with advances in information technology, the utility of which has been thoroughly demonstrated in the crisis of COVID-19, it would be helpful to develop online programs such as MOOC and SPOC. These online programs can also help to scale up service-learning in Hong Kong. Sixth, to objectively assess the impact of servicelearning on the service providers (i.e., university students) and service recipients, it would be academically and practically important to develop validated measures to evaluate the learning experience and impacts. Finally, as there are many factors influencing service-learning implementation, it would be helpful to conduct action research to test out new ideas and pedagogies. Figure 1.1 presents the conceptual model behind the design of our project. Our model is based upon learning theories behind adult learning (Knowles, 1984). Knowles lists four characteristics of adult learners: 1.

Self-concept: Unlike child learners who are dependent upon their teachers, adult learners are self-directed. They take responsibility and make decisions about what they will learn and how they will learn it.

6

G. Ngai and D. T. L. Shek

Fig. 1.1 Conceptual model behind the project design

2.

3.

4.

Experience: Adult learners, unlike child learners, have a lifetime (or years) of experience that they can draw upon. This experience is a resource that supports their learning, and learning gained from experience is more meaningful than learning that is passively acquired. Readiness to learn. Adult learners learn when they experience a need to learn it. Usually, these needs are aligned with career or personal goals, or when triggered by real-life tasks or problems. Orientation to learning. Adult learners learn in order to acquire skills and competencies that are required in their immediate context. Hence, the learning should be immediately applicable and problem-based.

In addition to these general characteristics, the learners that we are targeting in our project are all teachers in higher education. Most of our target group have attained their qualifications through intense study and research of theory and concepts in a particular field. Therefore, they need to be convinced that the topic that they are teaching (service-learning in this case) is bona fide academic knowledge and is properly grounded within a particular discipline. This is especially important in the Hong Kong context, where teaching in higher education in Hong Kong focuses primarily on intellectual learning instead of affective learning. In institutions where many of the teachers are from the STEM (science, technology, engineering and mathematics) fields, this situation is exacerbated, since STEM teaching functions in a very different paradigm from service-learning and education. In our experience,

1 Introduction

7

many STEM teachers are uncomfortable with the affective learning that is central to civic learning. Given these characteristics and observations, our project approaches teacher capacity building from the following angles: 1.

2.

3.

Teacher training: The project components under this dimension seek to educate teachers on the basic theories, concepts and practices of service-learning. Even though adult learning theory advises against conceptual teaching, covering theoretical knowledge helps to build a case for service-learning for our teachers. In particular, we also see that higher education teachers have an added disadvantage. K-12 teachers usually have to be certified for their job through the attainment of a teachers’ certificate or diploma that ensures that they are familiar with concepts of education and pedagogy. In Hong Kong, K-12 teachers have to fulfill continuous professional development requirements in which they have to attend a certain number of teacher training courses every year. This ensures that they are kept aware and abreast of the current state-of-the-art in teaching development. University teachers, on the other hand, are not qualified for their position or hired because of their ability to be good teachers. Much of university learning also focuses on cognitive learning, or the attainment of new knowledge and skills, but service-learning also seeks to impact students’ attitudes through affective learning. This is likely to be new territory to many higher education teachers, and as such, it is beneficial to give them an overview of the underlying concepts and to ensure that they have a proper understanding of the field. Expertise sharing and peer support: Even though service-learning is relatively new in Hong Kong, some early pioneers have already done substantial work and accumulated significant expertise. Since these experiences and practices are situated in the Hong Kong context, they are more immediately applicable or adaptable. It would be beneficial to facilitate teachers to learn from each other and to share their practices. This has the potential added benefit of building up a community of like-minded individuals who, through regular interactions, can provide each other with peer support and learning. This is important especially in the context of higher education. In other (regular) academic subjects, teachers have the peer community of other faculty members in their own department; however, most departments generally only offer one or two service-learning subjects. Since the siloed nature of higher education oftentimes means that faculty members do not interact much outside of the boundaries of their discipline, service-learning teachers can often feel very lonely and isolated within their own department. Helping them to build community not only facilitates mutual support, it also facilitates peer learning, where teachers can learn from each other’s best practices, perhaps even in an interdisciplinary manner. In the process, we emphasize the strengths of teachers (even novice teachers) as well as humility, respect and the value of collaboration. Resources support: In addition to training and education, teachers often also require support in the form of resources that can be used in their teaching, or for evaluating their programs. For example, teachers often ask us for help with

8

4.

G. Ngai and D. T. L. Shek

teaching the “basic concepts of service-learning” to students, or for teaching students proper “dos and don’ts” while on service. These concepts are standardized and do not vary much between projects, and hence, it makes sense to have a set of resources that can be shared. In addition, many university teachers are researchers by training, and universities encourage teaching that is underpinned by research. Resources such as validated instruments for measuring student learning outcomes or the learning process, or for soliciting community partner feedback, would be helpful in this direction. Improvements of current practices: Service-learning is a form of pedagogy, and teaching and learning at the university level are expected to be revised and updated constantly. In the disciplines, teachers are expected to also be conducting research in their own field and integrating state-of-the-art into their teaching. These disciplines are quite different from service-learning. Therefore, we also feel that it would be beneficial to support teachers to expand on their subjects and to improve their teaching in an evidence-based manner. Since most teachers are not familiar with this kind of research, support needs to be provided in the form of research expertise and resources.

1.4 Project Deliverables Given the above principles, we designed our project to have the following deliverables:

1.4.1 A Formal Professional Development Program The core of good teaching is teacher training. For most teachers in Hong Kong, service-learning is an unfamiliar pedagogy. However, there is a sizable contingent of teachers who have previously been involved in bringing students on service projects, mostly in the form of community service. These two groups of teachers are quite different: Some of them will require formal, systematic training, and others may benefit more from informal sharing of practices from colleagues. For formal systematic training, we propose the development of a professional development program on service-learning. This course should cover both the conceptual and the practical—in other words, it should have concepts and theories to provide a concrete underpinning of the pedagogy; as well as case studies to give teachers practical, replicable examples. In particular, since the experiential learning nature of service-learning is not always very easy to visualize, especially for teachers who are used to lecturing as their primary way of teaching, where possible, teachers should be brought to the field so that they can see how a service project works for themselves. This, we have also found, is the best way to persuade skeptics—facilitating them to can see for themselves what students can achieve and learn on the field, and

1 Introduction

9

the impact that students can have on the community. However, the integration of an observational component of course complicates matters as it is more difficult to arrange and coordinate. Hence, to accommodate for the constraints of adult learners (teachers), the course should be modular and adaptable, and flexibly use e-Learning technology when possible. Completion of this program would potentially require significant time and commitment on the participant. Therefore, teachers should be recognized for their attainment through certification.

1.4.2 Resources to Support Service-Learning Subjects and Projects Another important component of teacher support, especially with complex subjects such as service-learning which target multiple diverse student learning outcomes, are resources that can be used during the design, operation and evaluation of servicelearning subjects and projects. These include tools and learning components that can be flexibly integrated into service-learning subjects and projects, or which can be used for monitoring or feedback purposes.

1.4.2.1

A Student E-Learning Module

Teachers are not the only ones who misunderstand service-learning. Quite frequently, students also have misperceptions of service-learning, and question how they are supposed to learn from “doing charity.” Given that the service-learning experience is quite different from most disciplinary subjects, students are also often not prepared to learn from it—and indeed may question the need for it. They may also have some concerns about how they would be evaluated, or whether they might be at a disadvantage (this applies particularly to students who are more introverted, or who are from disciplines that are traditionally very objective and deterministic, such as the hard sciences and engineering). Hence, it is also necessary in service-learning classes to educate students on what service-learning really is, why they should do service-learning, and how they can and should learn from service-learning. These topics are important, as they prepare students to learn the most from their SL experience and to properly orient them with the necessary skills, knowledge and attitudes for service-learning. However, they are very generic and can be covered in a variety of subjects without much modification. This essentially means that these concepts lend themselves easily to a blended-learning approach. We therefore propose an e-Learning module that is designed to complement service-learning subjects—with the objective being to free up classroom time for more interactive activities. Potential topics include: i.

Concepts and myths about service-learning

10

ii. iii. iv. v.

G. Ngai and D. T. L. Shek

Benefits to service-learning to students, clients and the related communities Social responsibility, ethical/moral responsibility Cases of good practice in previous student projects Reflecting and learning from observations and experiences.

To make it interesting for students, such an e-Learning module should include learning activities to stimulate students to interact with the materials, such as short videos, concept tests, case studies, experience sharing by students having completed service-learning subjects. The objective is to provide opportunities for them to apply or think more deeply about the content. Most importantly, we have to evaluate the impact of the online module on student learning and benefits.

1.4.2.2

A Bank of Tools For Student and Community Assessment

To help teachers to better monitor their subjects and improve on their teaching, we also propose to develop tools that can be used for feedback and hopefully lead to evidencebased improvements in their subjects. Examples are psychometric instruments that measure aspects of student learning outcomes that are commonly seen across different service-learning programs—such as social responsibility, empathy and respect for diversity. Although some related instruments exist now, most of them were developed in the US and European contexts, and in previous work, we have found that some of them are not appropriate for use in the Hong Kong context (Lo et al., 2016). Hence, there is a need to develop and validate a set of instruments in the local context, such that they can be appropriately used for our students. The development of objective measures can also demonstrate the claim that SL is an academic matter which can be rigorously assessed. Student learning outcomes can be measured in various ways. A rigorous research study would treat the learning experience as an intervention, and measure students’ attainment of the learning outcomes before and after the learning experience—i.e., at the beginning and the end of the subject—and analyze the difference, and perhaps compare it to a control group. Although alternative explanations may be involved, this approach has commonly been used to examine changes in students after taking a subject (Li & Shek, 2020; Lin & Shek, 2021b). However, most teachers are not conducting student surveys to do research; instead, they are interested in receiving reliable feedback which they can use to evaluate their own subject. For feedback purposes, it is often enough to measure the student-perceived learning outcomes— which ask students whether they have gained in a particular area as a result of the learning experience. These instruments have the advantage that they are usually more focused, and more sensitive, than the research-based pre- and post-intervention instruments. In fact, the client satisfaction approach has been commonly used in the higher education context to examine the perceived outcomes of the learning experience students (Yu et al., 2021) as well as the perceived benefits of the learning experience (Ngai et al., 2018a; Shek, Yang, et al., 2021).

1 Introduction

11

In addition to student learning outcomes, the student learning experience should also be of interest to teachers as it allows them to view the design of their course from the students’ point of view. Previous work (Ngai et al., 2018b) has identified certain practices to be strongly correlated with learning gains in students—for example, students appear to learn better when they are well-motivated when they find themselves challenged, and when they are given ample opportunities to interact with the served community. However, there is significant variation between different courses and projects, and what might be impactful in a particular project may not be practicable in another. An instrument that helps teachers to ascertain what is impactful in their course would therefore be of value. Service-learning purports to benefit the community as well as the student. Since service-learning requires teachers to work with the community and NGO partners to arrange a project, it follows that the service will usually be of some value; after all, NGOs and communities are busy with their own affairs and are not likely to entertain requests which they do not find helpful. That said, what was understood during the planning may differ with what actually happens, and teachers should be encouraged to ask their community partners for feedback with which to guide future activities. Therefore, an instrument to collect community feedback and ascertain community impact should also be of value.

1.4.2.3

A Database of Exemplars of Good Practices in Service-Learning

The formal learning afforded by the professional development course is designed to give teachers a underpinning of the concepts and theories, and supplementing with some resources and cases. This is useful, but when teachers actually have to plan and teach a service-learning subject themselves, they often benefit from real examples, which they can then adapt for their own context. To this end, we also propose to create a collection of real cases of good practice in designing, delivering and assessing service-learning. These cases will include examples of notable subjects, exemplary projects, and innovative service-learning pedagogy, including creative and effective approaches to designing, teaching and assessing service-learning. The content will focus on teachers’ reflections on their own classes and projects, with an aim to documenting quality elements in servicelearning—e.g., best practices, what worked, what didn’t work, and reasons and rationales. To help teachers to make sense of the cases, and for better organization, we propose a framework that focuses on seven elements of a quality service-learning course/project/program: meaningful service activity, link to curriculum, reflection, diversity, duration and intensity, partnership and youth voice. Within each element of the framework, two types of good practice will be documented: good practice that reveals the key quality indicators in each element and the course/project/program that reveals the interplay of several elements that lead to good student learning outcomes. It is anticipated that this database will support colleagues to design and improve their service-learning courses/projects by incorporating high-quality elements. It also

12

G. Ngai and D. T. L. Shek

constitutes as a sense of empowerment for teachers and colleagues who advocate for the importance of service-learning.

1.4.3 Promoting and Facilitating Evidence-Based Improvement in Teaching Critical reflection and continued improvement are keys to good teaching, and especially when teaching a new course, as mistakes and inefficiencies are bound to happen in the first few years. This is especially true of service-learning, which as it is composed of many different components, is more complex by nature. This is a neglected domain in higher education because critical reflection is not commonly emphasized in teaching and learning. In fact, many teachers may simply say that they do not know how to reflect and positively deal with the reflective experiences. We therefore propose to set up a scheme to support action research in servicelearning. Through this scheme, we hope to encourage teachers to take an evidencebased, critically reflective approach to identifying room for improvement in their own subjects and projects, implementing revisions based on the findings, and verifying the benefit of the revisions. To prepare colleagues for these projects, we propose a set of workshops and seminars on methodologies in action research, such as framework design, data collection, analysis. Interested teachers can then propose their intended projects, and they will be expected to document, share and/or publish resulting improvements to their subjects/projects.

1.4.4 Facilitating Peer Learning and Support: A Community of Practice Peer learning is a significant component of adult learning, and recent years have seen the development of communities of practices (Wenger, 2013) that link together like-minded professionals to learn and grow together. These communities can be in any profession, but they are often seen in education. These communities facilitate learning in an adaptable manner, as members can flexibly and nimbly decide their topics of common interest. They are also a good source of peer support. We feel that a cross-institutional community of practice in service-learning would be an ideal way to link together teachers who already have some knowledge of service-learning, or who are interested in service-learning. This community of practice will host regular workshops, seminars and sharing sessions on related topics, such as the impact of service-learning on students/service recipients/the community, problems and challenges encountered, engaging and assessing students in critical reflection, doing action research. Some seminars can be given by teachers, and external guests, including community partners and international experts, can

1 Introduction

13

be invited for others. The cross-institutional nature of the community of practice would also help colleagues to build up trust and meaningful collaboration, which is particularly important in our context because competition among the Government funded institutions in Hong Kong is very intense, and there is a need to promote healthy peer appreciation and learning.

1.4.5 Research Studies on Service-Learning A final piece of our project is to study service-learning practices that have proven challenging for teachers. These studies, while they do not immediately build capacity of the community, are nonetheless important as they help to identify further areas that need improvement. Assessment is part and parcel of academic studies. When service-learning becomes academic, assessment of students also necessarily becomes part of the picture. Unlike many disciplinary studies, however, service-learning learning outcomes involve affective (attitudinal) as well as cognitive learning (knowledge). For many teachers, especially for those from the sciences, assessing student learning in these areas is challenging as it involves subjective judgment, which is not something that they are trained to do. There are two common problems involved here. First, it is a challenging task to develop rubrics for service-learning assessment because it covers multiple diverse assessment components, including understanding and application of academic knowledge, conception of the service-learning proposal, implementation and performance on project, evaluation of project and reflection. Second, for many of these components, besides intellectual assessment, qualitative evaluation is also needed. We hence propose a research study on practices in student assessment as part of our project. Through this study, we hope to understand which learning outcomes teachers deem important, and what learning components are actually assessed for student learning in practice. We hope that this study will help us to better understand this vital part of academic learning and help to guide further capacity building initiatives in the future. In another project, we present and summarize the evaluation findings of the service-learning projects arising from the Project WeCan. The Project WeCan is a project initiated by the Wharf (Holdings) Limited to support schools admitting students with social deprivation. Through collaboration with The Hong Kong Polytechnic University, PolyU students provided service in the Project WeCan schools. As this is an innovative project involving the business sector (Wharf), university and the community (high school and the community), there is a need to examine the impact of service-learning projects on the university students (i.e., service providers) as well as the high school students (i.e., service recipients). To give an integrative picture on the impact of the service-learning projects in the context of the Project

14

G. Ngai and D. T. L. Shek

WeCan, we have put together the relevant evaluation findings and discuss the benefits of the Project WeCan projects on the different stakeholders (Leung et al., 2021a; Shek et al., Shek, Chak, et al., 2021; Shek et al., 2020).

1.5 Conclusion Academic service-learning is growing in popularity in Hong Kong higher education institutions; however, teachers face various challenges in actuating service-learning programs in their own practice. This calls for efforts to build the capacity of the teachers who are involved. We propose a multi-faceted approach to capacity building for service-learning, ranging from professional development to experience sharing and peer support, and an investigative study into some of the more challenging aspects of academic service-learning for future initiatives. The chapters in this book will describe and evaluate our efforts in each of these dimensions.

References Billig, S. H. (2007). Unpacking what works in service-learning: Promising research-based practices to improve student outcomes (Growing to Greatness 2007, Issue). Bringle, R. G., & Hatcher, J. A. (1995). A service-learning curriculum for faculty. Michigan Journal of Community Service Learning, 2, 10. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164–181. https://doi.org/10.1177/105382 591103400205 Chan, S. C. F., Ngai, G., & Kwan, K. P. (2019). Mandatory service learning at university: Do lessinclined students learn from it? Active Learning in Higher Education, 20(3), 189–202. https:// doi.org/10.1177/1469787417742019 Chan, S. C. F., Ngai, G., Lam, C. H. Y., & Kwan, K. P. (2021). How participation affects university students’ perspectives toward mandatory service-learning. Journal of Experiential Education, 44(2), 137–151. https://doi.org/10.1177/1053825920948889 Conway, J. M., Amel, E. L., & Gerwien, D. P. (2009). Teaching and learning in the social context: a meta-analysis of service learning’s effects on academic, personal, social, and citizenship outcomes. Teaching of Psychology, 36(4), 233–245. https://doi.org/10.1080/009862809 03172969 Furco, A. (1996). Service-learning: a balanced approach to experiential education. In Corporation for National Service (Ed.), Expanding boundaries: Serving and learning (pp. 2–6). Cooperative Education Association. Hong Kong University Grants Committee. (2010). Aspirations for the Higher Education System in Hong Kong. https://www.legco.gov.hk/yr10-11/english/panels/ed/papers/edcb2-602-1-e.pdf Jacoby, B. (1996). Service-learning in higher education: concepts and practices. Jossey-Bass Publishers. Knowles, M. (1984). The Adult Learner: A Neglected species. Gulf Publishing Company, P.O. Box 2608, Houston, TX 77001 ($7.95). http://files.eric.ed.gov/fulltext/ED084368.pdf Kuh, G. D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. AACU.

1 Introduction

15

Leung, H., Shek, D. T., & Dou, D. (2021a). Evaluation of service-learning in project WeCan under COVID-19 in a Chinese context. International Journal of Environmental Research and Public Health, 18(7), 3596. Leung, H., Shek, D. T. L., & Dou, D. (2021b). Evaluation of service-learning in Project WeCan under COVID-19 in a Chinese Context. Int J Environ Res Public Health, 18(7). https://doi.org/ 10.3390/ijerph18073596 Li, X., & Shek, D. T. L. (2020). Objective outcome evaluation of a leadership course utilising the positive youth development approach in Hong Kong. Assessment & Evaluation in Higher Education, 45(5), 741–757. https://doi.org/10.1080/02602938.2019.1696944 Lin, L., & Shek, D. T. (2021a). Serving children and adolescents in need during the covid-19 pandemic: Evaluation of service-learning subjects with and without face-to-face interaction. International Journal of Environmental Research and Public Health, 18(4), 2114. Lin, L., & Shek, D. T. L. (2021b). Serving children and adolescents in need during the COVID19 pandemic: Evaluation of service-learning subjects with and without face-to-face interaction. International Journal of Environmental Research and Public Health, 18(4). https://doi.org/10. 3390/ijerph18042114 Lo, K. W. K., Kwan, K. P., Chan, S. C. F., & Ngai, G. (2016). Cross-Cultural validation of the global citizenship scale for measuring impacts of international Service-Learning in Hong Kong setting. In International Association for Research on Service-learning and Community Engagement. New Orleans, LA. Lundy, B. L. (2007). Service learning in life-span developmental psychology: Higher exam scores and increased empathy. Teaching of Psychology, 34(1), 23–27. https://doi.org/10.1080/009862 80709336644 Ngai, G., Chan, S. C., & Kwan, K. P. (2018a). Challenge, meaning and preparation: Critical success factors influencing student learning outcomes from service-learning. Journal of Higher Education Outreach and Engagement, 22(4), 55–80. Ngai, G., Chan, S. C. F., & Kwan, K. P. (2018b). Challenge, meaning, interest, and preparation. Critical success factors influencing student learning outcomes from service-learning, 22(4), 55– 80. http://www.scopus.com/inward/record.url?scp=85059743039&partnerID=8YFLogxK Ngai, G., & Chan, S. C. F. (2015, 2015). How much impact can be made in a week?: designing effective international service learning projects for computing. Sigcse ’15, New York, NY, USA. Ngai, G., Chan, S. C. F., Wong, W., & Kwan, K. P. (2016). A Multivariate Study of Factors Influencing Students’ Learning Outcomes from Service-Learning International Association for Research on Service-Learning and Community Engagement, New Orleans, LA. Shek, D. T. L. (2010). Nurturing holistic development of university students in Hong Kong: Where are we and where should we go? The Scientific World Journal, 10, 563–575. Shek, D. T. L., & Siu, A. M. H. (2019). “UNHAPPY” environment for adolescent development in Hong Kong. Journal of Adolescent Health, 64(6), S1–S4. https://doi.org/10.1016/j.jadohealth. 2019.01.007 Shek, D. T. L., & Wong, K.K.-Y. (2011). Do adolescent developmental issues disappear overnight? reflections about holistic development in university students. The Scientific World Journal, 11, 353–361. Shek, D. T. L., Yu, L., & Chai, W. Y. (2017). Evaluation of the general university requirements: What did students say? International Journal of Adolescent Medicine and Health, 29(1), 75–82. https://doi.org/10.1515/ijamh-2017-3010 Shek, D. T. L., Ma, C. M. S., & Yang, Z. (2020). Transformation and development of university students through service-learning: A corporate-community-university partnership initiative in Hong Kong (Project WeCan). Applied Research in Quality of Life, 15(5), 1375–1393. https://doi. org/10.1007/s11482-019-09738-9 Shek, D. T. L., Chak, Y., Lin, L., Li, X., Leung, E., Li, P., Mok, B., & Shek, V. (2021a). Servicelearning under Covid-19: Experience gained from project WeCan in Hong Kong. International Journal of Child and Adolescent Health, 14.

16

G. Ngai and D. T. L. Shek

Shek, D. T. L., Yang, Z., Ma, C. M. S., & Chai, C. W. Y. (2021). Subjective outcome evaluation of service-learning by the service recipients: scale development, normative profiles and predictors. Child Indicators Research, 14(1), 411–434. https://doi.org/10.1007/s12187-020-09765-1 Warren, J. L. (2012). Does service-learning increase student learning?: A meta-analysis. Michigan Journal of Community Service Learning, 18, 56–61. Wenger, E. (2013). Communities of practice: learning, meaning, and identity. Cambridge University Press. https://doi.org/10.1017/CBO9780511803932 Wood, C. A., Banks, S., Galiardi, S., Koehn, J., & Schroeder, K. (2011). Community impacts of international service-learning and study abroad: An analysis of focus groups with program leaders. Partnerships: A Journal of Service Learning & Civic Engagement, 2. http://libjournal. uncg.edu/prt/article/view/430 Yang, M., Luk, L. Y. Y., Webster, B. J., Chau, A.W.-L., & Ma, C. H. K. (2016). The role of international service-learning in facilitating undergraduate students’ self-exploration. Journal of Studies in International Education, 20(5), 416–436. https://doi.org/10.1177/1028315316662976 Yorio, P. L., & Ye, F. (2012). A meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9–27. https://doi.org/10.5465/amle.2010.0072 Yu, L., Lin, L., Shek, T. L. D., & Chai, W. (2021). Students’ perceived attributes and benefits of a leadership course: Subjective outcome evaluation. Research on Social Work Practice, 10497315211042824. https://doi.org/10.1177/10497315211042823

Chapter 2

What Matter in a Quality Service-Learning Course: A Study on Faculty Members’ Perception Huixuan Xu

Abstract The present paper attempts to explore what consists of key elements in the design and delivery of service-learning in higher education in Hong Kong, East Asia. Ten individual interviews with academics from four higher education institutions in Hong Kong were conducted in a pilot study. Each interview lasted around 1.5 h and interview questions focused on what types of service-learning the faculty delivered, what were perceived as good practice and experience in teaching the course, and what were the barriers and challenges they encountered. The findings echo the research into the importance of five key elements: link to curriculum, partnership, service placement quality, reflection, facilitator, or tutor’s support. The study also provided empirical evidence on the three elements that had rarely been reported in the previous quantitative studies: uncertainty avoidance, student training, and assessment. In addition, two novel elements were identified: community of practice, role model. Keywords Service-learning · Program elements · Higher education · Review

2.1 Introduction Service-learning, as a form of experiential learning experience, intends to “equally benefit the provider and the recipient of the service as well as to ensure equal focus on both the service being provided and the learning that is occurring” (Furco, 2002). Service-learning has obtained a good record of development in higher education since 1990s. In higher education, service-learning is more often structured as a creditbearing learning experience, which is referred to “a course or competency-based, credit-bearing educational experience in which students (a) participate in mutually identified service activities that benefit the community, and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader H. Xu (B) Department of Curriculum and Instruction, the Education University of Hong Kong, Hong Kong, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_2

17

18

H. Xu

appreciation of the discipline, and an enhanced sense of personal values and civic responsibility” (Bringle & Clayton, 2012). This definition referring to academic service-learning (ASL) stresses the importance of connecting academic learning and service-based activities, in which service-learning serves the role of pedagogy. Although service-learning is intuitively attractive to be a pedagogy to enhance student learning and personal growth, the mechanism that triggering student learning has yet well unfolded. Extensive research has examined students’ learning outcomes in the context of service-learning (Salam et al., 2019), however, less research has been conducted to address what consist of high quality service-learning. The present chapter reported a study on the exploration of what consists of key elements in the design and delivery of service-learning in higher education in Hong Kong, East Asia. The findings were obtained from interviews with 16 faculty members from four universities in Hong Kong. All the interviewees have accumulated experience in teaching service-learning and have also encountered challenges posed by servicelearning. We attempted to identify which elements have a critical role in the design and delivery of good service-learning.

2.1.1 Essential Elements in Service-Learning Both conceptual discussion and empirical evidence have been accumulated about what comprises good practice in teaching service-learning. Regarding the conceptual analysis of principles of good service-learning, a few institutions have suggested some key elements based upon literature analysis. For instance, the K-12 standards were created on the basis of an extensive review of the literature (Billig, 2009), comprising eight categories: duration and intensity, meaningful service, link to curriculum, reflection, diversity, youth voice, partnerships, and progress monitoring. According to certain researchers (Eyler et al., 2010), these K-12 standards may serve as a definition for how quality service-learning practice might look, which may also be applied to the higher education settings. Similar conceptual work can be found in a few of scholars, such as the work of Keith (1997), National Service-Learning Cooperative (1998), Billig (2002). Empirical studies have examined what elements in service-learning programs impact upon students’ learning outcomes. Some elements have attracted much research attention over the past two decades, including reflection (for instance, Chang, Anagno-seopoulos, & Omae, 2011), youth voice (for instance, Covitt, 2006), relationship and partnership (for instance, Clayton, Bringle, Senor, & Morrison, 2010), duration and intensity (for instance, Conway et al., 2009), quality of service placement (for instance, Chang, Anagno-seopoulos, & Omae, 2011), link to curriculum (for instance, McCrea, 2010). Celio et al. (2011) also examined how the interplay of multiple elements work in service-learning programs. In the following, we review the recent research on individual elements and then introduce the present study.

2 What Matter in a Quality Service-Learning Course …

2.1.1.1

19

Reflection

Reflection is the key element in the pedagogy of service-learning. Reflection refers to an ongoing and forming part of teaching and learning process, during which students’ thoughts, feelings or actions are taken into consideration in order to gain deeper understanding of their current or subsequent experience (Brockbank & McGill, 1998). In the constructivist perspective of experiential learning, reflection is constructed as a key component functioned as transforming experience to knowledge. Some research has focused on the amount of reflection. Covitt (2006) examined the effects of service-learning requirements on students’ attitudes toward servicelearning and their environmental education outcomes. His study showed that students who reported having greater opportunities for reflection reported more positive ASL. Dahan (2016) examined the effects of service hours and frequency of contact with community members on students’ changes in civic engagement and academic gains. This study confirmed that weekly in-class and out-of-class reflection were effective in producing positive outcomes for social values and civic attitudes. Chang et al. (2011) examined 212 pre-service teachers’ service experience and multicultural learning in a multicultural education course, by using pre-posttest design. Apart from MSL site characteristics, individual characteristics and the relationship between the participants and children being served, pedagogical engagement in terms of reflective discussion (reflection) were examined in the study. It was reported that more intensive discussions in their university course seminars had a positive significant effect on all three MSL learning outcomes. In their longitudinal study, Astin et al. (2000) examined the relationship between reflection and student outcomes in academic, values, self-efficacy, leadership, career, further service. It was reported that both class discussion and written reflection fostered positive service-learning experience. The frequency with which professors connected the service experience to the course subject matter was an especially important determinant of academic outcomes. In a meta-analysis of service-learning’s effects, Conway et al. (2009) examined the extent to which structured reflection as a moderator contributing to greater change in students’ academic, personal, social, and citizenship learning. The result showed that service with structured reflection produced larger effects on student participants. Yorio and Ye (2012) conducted a meta-analysis on the effects of service-learning with 40 studies and examined the effects of reflection as a moderator. The analysis found that the inclusion of a discussion reflection component in the service-learning experience impacted on students’ understanding of social issues significantly more than those service-learning experiences that included a written reflection component only. With the effort to deepen the understanding of reflection, Sturgill and Motley (2014) compared the affordances of three different models of written reflection in engendering students’ higher-order thought processes. Student reflections were compared across axes of guided versus free response, dialogic versus expressive reflection, and public versus private reflection. Results indicate that guided reflection yields more response than free reflection does. Dialogic and guided reflections

20

H. Xu

both yield more integration of knowledge from service-learning activities within a larger context. Apart from the research on the features of reflection, some researchers focus on developing practical guidance for teacher practitioners. Ash, Clayton, and their colleagues developed a DEAL (Describe, Examine, and Articulated Learning) model that provided a practical guidance of how to effectively facilitate students’ reflection in service-learning. The model was originally based on Ash and Clayton’s work on Articulated Learning (Ash & Clayton, 2004). The protocols and tools that intentionally linked the formative assessment of student learning with reflection were developed in the model, which were tested in a follow-up study (Ash et al., 2005). The study reported that the tools improved students’ high order thinking and critical thinking skills relevant to academic enhancement, civic engagement, and personal growth, after experiencing a process of draft, revise, and final version of written reflections (Ash et al., 2005). Empirical examination reported that the model and associated rubrics were useful in documenting student learning. The study conducted by Molee et al. (2010) reported that students could identify, describe, and apply their learning by using a tailor-made tool; they had difficulty, however, evaluating their learning and thinking critically.

2.1.1.2

Meaningful Service

Service is a term that covers a variety of service activities to serve multiple groups of underprivileged people with various goals in promoting student learning over different periods of time. In the K-12 Service-Learning Standard (Billig, 2009; National Youth Leadership Council, hereafter called NYLC, 2008), meaningful service means that servicelearning actively engages participants in meaningful and personally relevant service activities. Five indicators can be used to evaluate whether a service experience is meaningful: 1. 2. 3. 4. 5.

Service-learning experiences are appropriate to participant ages and developmental abilities. Service-learning addresses issues that are personally relevant to the participants. Service-learning provides participants with interesting and engaging service activities. Service-learning encourages participants to understand their service experiences in the context of the underlying societal issues being addressed. Service-learning leads to attainable and visible outcomes that are valued by those being served.

Research that addressing the features of service activities has focused on the quality of service placements. Simons and Cleary (2006) examined students’ learning outcomes according to the placement site and placement activity. There were no significant differences in personal and social outcomes according to the type of

2 What Matter in a Quality Service-Learning Course …

21

placement but there were differences in social outcomes between service activities. Mentors had higher social justice scores compared to tutors, also had higher community interests scores than tutors. Chang et al. (2011) examined 212 pre-service teachers’ service experience and multicultural learning in a multicultural education course by using pre-posttest design. Multiple site characteristics (i. e. the type of organization, student populations, and support from site supervisors) were examined in the study. The study reported that only two aspects of service settings, supportive site supervisors, and exposure to ELL students, appear to significantly impact pre-service teachers’ learning. Some research focused on the feature of social interaction during service. According to the seven features of Contact Theory (CT), Conner and Erickson (2017) differentiated 13 service-learning into high CT and low CT service-learning courses and observed the impacts of different CT SL course and individual characteristics on students’ changes in attitudes to racial minority groups. The data showed that courses with higher alignment with CT conditions did predict lower levels of overall colorblindness on the posttest. Cooperation between students and those with whom they served and institutional support for the service each generates significant associations with the degree of change in students’ racial attitudes.

2.1.1.3

Student Autonomy and Youth Voice

Researchers have noted the importance to allow students to make choices and make decisions in different stages of a service-learning experience, which may promote student motivation and engagement. This is consistent with one standard in K-12 service-learning, youth voices. Indicators in K-12 service-learning include 1. 2. 3. 4. 5.

service-learning engages youth in generating ideas during the planning, implementation, and evaluation processes. Service-learning involves youth in the decision-making process throughout the service-learning experiences. Service-learning involves youth and adults in creating an environment that supports trust and open expression of ideas. Service-learning promotes acquisition of knowledge and skills to enhance youth leadership and decision-making. Service-learning involves youth in evaluating the quality and effectiveness of the service-learning experience.

In the studies in higher education, Levesque-Bristol and Stanek (2009) revealed the relationship between students’ perceived autonomy-supportive learning environment and their changes of motivation and skills developed during a service-learning course focusing on research methods (conducting research, analyzing data, statistics, writing a research report, interacting with community partners, class presentations). Covitt (2006) examined the effectsof service-learning requirements on students’ attitudes toward service-learning and their environmental education outcomes. Students

22

H. Xu

who reported having greater opportunities to make decisions about service-learning had more positive ASL.

2.1.1.4

Partnership

As defined by Clayton et al. (2010), relationship refers to interactions between persons while partnership refers to a particular subset of relationships between the community and the campus. Clayton et al. (2010) developed an instrument to identify different types of partnership in service-learning, and transactional and transformational relationship was differentiated by using the Transformational Relationship Evaluation Scale (TRES). Transactional relationships are defined as instrumental and often designed to complete short-term tasks while transformational relationships are wherein both persons grow and change because of deeper and more sustainable commitments (Enos & Morton, 2003). Some researchers have investigated the characteristics of good partnership in service-learning. Mayer et al. (2017) reported a case study on the communityacademic partnership (CAP) between a nonprofit organization (NPO) and a school of nursing (SON). Multiple sources of qualitative data were collected from stakeholders and finally four main themes emerged that capture the relationship characteristics between partners and how the relationship could be improved: Time (frequency and duration of time community and academic participants spent in the community, in community activities, and with each other.), Communication (defining and understanding roles and expectations within and between organizations), Goals (overall goals for the partnership and community as well as for programs and specific projects conducted by respondents.), and Sharing (giving of resources, advice, information and connections between participants throughout the CAP). Meierhofer et al. (2013) examined a case that reported the partnership between a student pharmacist organization and a clinic serving underprivileged people. The responsibilities associated with the design of the training process, student activities at the clinic, and program evaluation were left primarily to the students, guided by their faculty mentors and clinic partners. As a result of this need, BFM faculty pharmacists worked with the student group to create a medication reconciliation program for student volunteers, to be carried out on Wednesday evenings at the clinic. In addition, the state Board of Pharmacy was contacted to verify eligibility of the clinic nurse practitioner to serve as a preceptor for student pharmacists performing medication reconciliation. Tryon et al. (2008) interviewed 67 community representatives and unraveled their voices and perceptions on service-learning. Regarding partnership between the community and the campus, it was found that open, regular communication and trust in the relationship are identified as important element in developing good partnership. Success indicators include the level of commitment made by the academic and community partners, the effectiveness of communication, and the compatibility of the service-learning program and the student to the community organization’s goals.

2 What Matter in a Quality Service-Learning Course …

2.1.1.5

23

Duration and Intensity

To address community needs and meet specified outcomes, service-learning should have sufficient duration and intensity. However, the cut-off duration and intensity that ensure the desired learning outcomes to occur has no answer. In a meta-analysis of service-learning’s effects, Conway et al. (2009) examined the extent to which the number of service hours and length of service contributed to greater change in students’ academic, personal, social, and citizenship learning. The result reported that slightly larger effects were reported from smaller numbers of hours and weeks, larger number of service hours and weeks did not contribute to larger effects. Dahan (2016) examined the effects of service hours and frequency of contact with community members on students’ changes in civic engagement and academic gains. This study confirmed that more than 15 h of civic engagement produced positive outcomes for students’ civic attitudes. The only consistent predictor for the civic engagement outcomes in this study was greater than 30 h. Other differences were detectable at 15 or fewer hours, but these results were mixed. Frequent contact was strongly related to positive social values and civic attitudes, weakly related to community service self-efficacy and personal and social development gains.

2.1.1.6

Interplay of Multiple Elements

Rather than studying on individual elements, Celio et al. (2011) examined the impacts of service-learning programs that included different number of recommended practices (link to curriculum, youth voice, community involvement, and reflection) upon students’ learning in attitudes to self, attitudes to school and learning, civic engagement, social skills, academic achievement. All studies, regardless of how many of the four recommended practices they contained, produced significant positive mean effects on the fiveoutcomes. However, including more of the four recommended practices did not lead to successively better outcomes.

2.2 Methods Sixteen individual interviews with service-learning course instructors from four higher education institutions in Hong Kong were conducted in a qualitative study. These course instructors were recommended by core members from a crossinstitutional service-learning project team and the members were experts in teaching service-learning and familiar with their individual service-learning colleagues. Each interview lasted around 1.5 h. The elements reviewed above served as sensitizing concepts (Bowen, 2006) in the present study were employed in the interview questions as probes, as well as in data analysis as reference to develop the coding system. Interview questions focused on what types of service-learning the

24

H. Xu

faculty participants delivered, what were perceived as good practice and experience in teaching the course, and what were the barriers and challenges they encountered. Subsequent questions probed the specific elements in service-learning, such as reflection, youth voice, partnership, link to curriculum, etc. All interviews were conducted in Cantonese, audio-recorded and transcribed verbatim. One of the senior research assistants randomly selected three case courses and coded the transcripts, then the author of the present chapter checked the coding. When there were any discrepancy occurred, the two researchers discussed until the consensus reached. After the coding system was developed, the senior research assistant used the coding system to code the remaining transcripts. The author of this chapter checked all the coding to confirm the results. The elements emerged from the literature served as a framework to code the transcripts. Any novel elements that perceived as good practice in service-learning in the Hong Kong context were also identified. The variations across the academics were noted during the analysis, and sub-codes were then created to specify the categories in each element. Relevant documents such as course outlines and teaching materials were also collected and analyzed to triangulate the data from different sources.

2.3 Findings and discussion The findings from the present study echoed the literature on five elements: linkage and integration, partnership, meaningful service, reflection, assessment for learning.

2.3.1 Linkage and Integration 2.3.1.1

Linkage to Curriculum

As Howard (1998) points out, “The service and the learning are reciprocally related; the service experiences inform and transform the academic learning, and the academic learning informs and transforms the service experience” (p. 21). Blending service-learning with the academic curriculum provides students with more experiential learning, allowing them to bring the theoretical concepts taught in the traditional classrooms to apply in the practical society. In this way, students can yield substantial benefits from the integrated program to learn more about the subject knowledge, the community, as well as themselves in the experiential and contextualized environment. Though it is intuitive to understand the importance of linkage, the challenge in the design and delivery of a coherent service-learning course cannot be missed. This means that various sections of the course, including objectives, content, teaching activities, assessment, and readings, are connected with students’ concrete experience in a systematic way. This is consistent with the principle of constructive alignment in curriculum development. The challenges during the process of design teaching

2 What Matter in a Quality Service-Learning Course …

25

included how the abstract academic theory was actualized in concrete experience, which parts of course content was possible to be linked to service, what facilitation was needed to achieve the synergy, and how to use alternative assessment methods to identify student learning outcomes. Faculty members that we interviewed taught one or two service-learning to facilitate students to learn academic knowledge and promote students’ personal growth. The constructive alignment was mentioned by all participants as a key factor in the design and delivery of service-learning. Though the importance of linkage was recognized by all participants, the difficulty to coherently connect all parts was varied across the courses. For some courses in the applied disciplined such as engineering, psychology, education, Chinese medicine, the faculty participants felt it easier to find a link between the specialization knowledge and concrete service experience. For some courses in the pure disciplines such as biology, however, faculty participants encountered challenges to find proper service activities. Course lecturers should think of alternative ways to help students see the link. For instance, one faculty member in the department of Biology taught a course about Fermentation and Enzyme Technology. The course lecturer attempted to use metaphor to help students see the link between fermentation and service activities. The course lecturer collaborated with one colleague in the department of social work and used “fermentation” as a metaphor for life planning. University students were engaged in leading activities to engage school students to “ferment” their life and set goals for the learning and life at the next stage. The linkage between content and service is not only important for the course lecturer to develop a coherent design from the supply side, but also important to make it explicit to students. When students were aware of the relationship between what they learned in the classroom and that in the service placement, it was more likely that they construct the meaning of their service experience from the perspective taken by the course lecturer, the more likely students achieved the desired learning outcomes.

2.3.1.2

Integration Between Teaching and Research in Service-Learning

Among 16 interviewees, nine were academic staff working on a tenure track, and seven were teaching staff whose major work was teaching. For academic staff, they reported the tensions in teaching service-learning: service-learning required much more time and effort in the organization and communication with different parties. This time-consuming work likely discouraged academic staff, since the major criteria for evaluating an academic staff’s work performance was research. In our study, two types of academic staff reported how they released the sense of resistance of teaching service-learning: one faculty received strong support from a central service-learning office; two participants attempted to integrate their research focus and service-learning to make a win–win result. One faculty participant narrated how she, with the substantial help from a central office of service-learning, developed

26

H. Xu

her experience of teaching service-learning from zero, such as collaborating with the office to extend the partnership with community centers, enhancing the training and preparation for students, and providing prompt feedback to facilitate student learning during the period of service. For academic staff, the support from the central unit and policy of the home university was important, in terms of encouraging and sustaining academic staff in the service-learning teaching and guaranteeing the quality of a service-learning course.

2.3.2 Partnership Developing good partnerships between university and community has been widely believed as one of the key factors in the successful delivery of academic servicelearning (ASL) (Maurrasse, 2002; Mihalynuk & Seifer, 2002). Good partnerships bring individuals together from different areas (Dougherty, 1992) and build up a long-term commitment in collaborative engagement (e.g., Maurrasse, 2002). All participants acknowledged the key role of partnership in the success of a service-learning course. A few factors were mentioned by the interviewees as critical to sustain a good partnership. Common goals in the service-learning partnership was one of the important factors, which meant the outcomes of service might be maximized when all the participating parties shared common goals, properly distributed the duties and resources, and provided mutual support. Such goals addressed the learning goals of university students, as well as met the genuine needs of service recipients. Sufficient communication between the university and the partner organization before the beginning of service activities was also important to reach the understanding of common goals.

2.3.3 Meaningful Service Activities In service-learning, service activities were concrete experience that students experienced and reflected upon, from which students were engaged in the social interactions, encountering and interpreting concrete experience, transferring academic content to real life settings, getting a picture of social issues in society. Provision of meaningful service activities may maximize the possibility to get students motivated and engaged in a learning environment. Concerning what comprising meaningful service activities, the faculty members from our study indicated the effects of direct service, features of service recipients, students’ voices to be heard, and provision of chances for students to see the benefits of their service to real people and community. Students performing in direct and indirect service had been perceived as having different benefits on student learning. For example, exposing participants to have person-to-person contact with the recipients could lead students to be more attached

2 What Matter in a Quality Service-Learning Course …

27

to the community, to feel more autonomous in social interactions whereas indirect service could result in higher level of academic engagement, and provide students with immediate outcomes and physical evidence for their accomplishments. These findings were consistent with previous research (Billig et al., 2005; Kackar-Cam & Schmidt, 2014). Exposing students with the individuals whose backgrounds were dissimilar with students’ might, to some extent, made students feel shocked at the first sight, thereby leading to empathy to the recipients. Findings also showed that when students helped the homeless or deprived individuals, it was more likely for them to facilitate their civic awareness and future intention of civic involvement. For instance, one collaborative service-learning project involved students from two universities in the provision of homework tutoring for primary school students from deprived family backgrounds. During preparation and training, university students had chance to know the background of individual tutees, which made them more responsible and discussed the tutoring work with a focus on how to provide additional education support for deprived children to achieve a more justice system in Hong Kong. Service experience though which students have chance to see the benefit to service recipients were perceived by participants as potent to change students. Some servicelearning courses we examined were a requirement from the university. Students showed diversified motivation in these courses, some having high motivation which others would like to meet the minimum requirement. For the latter, one of the chances to change their attitude was seeing the benefits of their own action. In addition, in our study, course lecturers provided varying degrees of chances for students to make decision on their own study. Not all students would like to complete a service experience by their own, but they enjoyed some chances to make decisions. The main effect was to enhance their autonomy in the process of learning.

2.3.4 Reflection Reflection is the key element in the pedagogy of service-learning that refers to an ongoing and forming part of teaching and learning process which students’ thoughts, feelings, or actions are taken into consideration in order to gain deeper understanding of their current or subsequent experience (Brockbank and McGill, 1998, p. 56 cited in Ikeda, 2000, p. 2). The perceptions shared in our study were consistent with the four principles of reflection, developed by Eyler et al. (1996): continuous, connected, challenging, and contextualized, by using multiple types of reflection activities. Reflection was conducted continuously before, during and after the services. For example, “reflective preparation” started before an experience by informing students the issues, the community, the agency, and the service recipients they would deal with. Most reflection took place during the experience, referring to reflection-inaction (Schön, 1992), when they tried to figure out what was going on and how to find solutions for the problems they encountered and considered the improvement of situations. The reflection after the experience, referring to reflection-on-action,

28

H. Xu

reviewed the service experience, integrated between new and old understandings / knowledge, and planed for future action (Connors & Seifer, September, 2005). In our study, most faculty participants provided multiple chances for students to approach to course lecturers and get feedback on their progress of learning. Reflection connects service to theories and concepts of the course by integrating activities with the lesson contents. Students reflected on whether they met the desired learning outcomes, as what the course aims for, through understanding and practicing what they learned from the lectures and classwork along with the development of personality, communication, teamwork, interpersonal skills, and attitude toward community services. Concrete aspects of teaching included the provision of appropriate readings and guiding questions in reflection activities to guide the direction of students’ reflection and learning. Challenging reflection was considered to be “most difficult to implement” since students were challenged to explore issues or participate in the activities with more critical ideas. Thus, the faculty participants mentioned that they should take the role of facilitator to suggest unexamined, unfamiliar and uncomfortable ideas as well as asking questions to challenge students’ assumptions or opinions. As a result, students developed new perspectives or alternative explanations for what they had experienced or observed that challenge their inherent perceptions of the issues. Balancing both challenge and support was essential in this process. Without adequate support and encouragement, students would not leave their “comfort zones” and try on new cogitations in order to learn and grow (Eyler et al., 1996; Sanford, 1967 cited in Jacoby, 2015, p. 28; Connors & Seifer, September, 2005). Service-learners reflect within the contexts. Students’ backgrounds should be considered such as their thoughts, developmental levels, life situations (Jacoby, 2015, p. 28), knowledge, experiences, and learning styles (Connors & Seifer, September, 2005). Scaffolding was important to identify the needs of students.

2.3.5 Assessment for Learning Assessment is defined as “conscious and systematic activities used by teachers and students for gathering information, and analyzing and interpreting it, drawing inferences, making wise decisions, and taking appropriate actions in the service of improving teaching and learning” (Berry, 2008, p. 6). There are traditionally two types of assessment in regular curricula, formative and summative. Formative assessment is often known as assessment FOR learning (AfL) and conducted during the teaching and learning process that provides information about students’ learning progress and revises or readjusts curriculum, whereas summative assessment, as commonly known as assessment OF learning (AoL), collects information about students’ levels of achievement in relation to the final goal of educational activities and selects students for prizes or award of certificates.

2 What Matter in a Quality Service-Learning Course …

29

Studies suggest that the ongoing, formative, embedded assessment throughout the service-learning course has the best evaluating results. Thus, a shift from traditional summative assessment (AoL) to formative assessment (AfL), where evaluation is embedded during learning and teaching, is urged for service-learning curricula. Bringle et al. (2016) also stress that the implementation of formative assessment associates with course design and monitoring mediating variables that help students achieving learning objectives since good assessment of learning relies on connecting learning goals and objectives with course design, implementing the course based on course design, and adopting suitable and meaningful measurement on learning in the service-learning courses that often fall into unpredictable situations. Comparing with the traditional summative examinations, tests, and quizzes that only evaluate academic learning, these self-report measures appraise “students’ attitudes, behavioral intentions, past behaviors, feelings, or beliefs” in order to assess civic learning and personal growth. Nevertheless, the negative sides of these instruments were inaccuracies, biased memories from experiences, and biased “selfassessments of skill, character, and learning.” Furthermore, the formative performance measures could be reflective journals, individual and group work for service activities, and evaluation by community partners that provided clear learning objectives and criteria for assessment as well as feedback for both summative and formative purposes. Giving meaningful feedback to students about their actual performance levels after assessment helped them understand their learning situations and improved their performance in right directions. Quality feedback should be in adequate detail and appropriate timing. The results indicated that the “ongoing feedback” and the grading framework of the assessment helped lecturer adapt and adjust the pedagogical techniques as well as distribute grades to students. It eventually fulfilled the expectations of all parties including the lecturer, students, and community partners.

2.4 Conclusion Service is a term that covers a variety of service activities to serve multiple groups of underprivileged people with various goals in promoting student learning over different periods of time, however, a systematic understanding of the extent to which individual service-learning program elements foster student learning and how the interplay of program elements influence upon the positive changes of student learning is still lacking. The present study shows what works best, what works best for whom, and which components of service-learning course design and implementation result in particular student learning outcomes in the Hong Kong context. What comprise high quality service-learning and how to integrate the key elements into individual service-learning teaching to enhance students’ learning are key issues when faculty design and develop a service-learning course/subject/project. The present study identified ten key elements in service-learning programs that positively foster students’ learning in the Hong Kong context. Such knowledge

30

H. Xu

will provide practical implication for faculty to develop and improve their servicelearning teaching. Suggestion on how an individual faculty member may translate a particular element in individual service-learning course design will be given at the end of the paper. Acknowledgements The author acknowledges the University Grants Council of Hong Kong for the support through the Funding Scheme for Teaching and Learning Related Proposals (PolyU4/T&L/16–19) and the Education University of Hong Kong for supporting the preparation of this manuscript. The authors also would like to thank the faculty members those involved in the project for their kind support.

References Ash, S. L., & Clayton, P. C. (2009). Generating, deepening, and documenting learning: The power of critical reflection in applied learning. Journal of Applied Learning in Higher Education, 1, 25–48. Ash, S. L., & Clayton, P. H. (2004). The articulated learning: An approach to guided reflection and assessment. Innovative Higher Education, 29(2), 137–154. Ash, S. L., Clayton, P. H., & Atkinson, M. P. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service Learning, 11(2), 49–60. Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service learning affects students. Higher Education Research Institute, UCLA. Berry, R. (2008). Assessment for learning (Vol. 1). Hong Kong University Press. Billig, S. H. (2000). Research on K–12 school-based service-learning: The evidence builds. Phi Delta Kappan, 81(9), 658–664. Billig, S. H. (2002). Support for K–12 service-learning practice: A brief review of the research. Educational Horizons, 80(4), 184–189. Billig, S. H. (2009). Does quality really matter? Testing the new K–12 service-learning standards for quality practice. Creating our identities in service-learning and community engagementIn B. E. Moely, S. H. Billig, & B. A. Holland (Eds.), Advances in service-learning research (Vol. 9, pp. 131–157). Information Age. Billig, S. H., Root, S., & Jesse, D. (2005). The relationship between the quality indicators of service-learning and student outcomes. Improving service-learning practice: Research on models to enhance impacts, 97–115. Bloom, B. S. (Ed.). Engelhart, M. D., Furst, E. J., Hill, W. H., Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Longmans. Bowen, G. A. (2006). Grounded theory and sensitizing concepts. International Journal of Qualitative Methods, 5(3), 12–23. Bradley, J. (1995a). A model for evaluating student learning and in academically based service. In M. Troppe (Ed.), Connecting Cognition and action: Evaluation of student performance in service-learning courses. Campus Compact. Bradley, J. (1995b). A Model for Evaluating Student Learning and in Academically Based Service. In M. Troppe, Connecting Cognition and Action: Evaluation of Student Performance in ServiceLearning Courses. Providence, RI: Campus Compact. Bringle, R. G. and Hatcher, J. A. (1999). Reflection in Service Learning: Making Meaning of Experience. In Introduction to Service Learning Toolkit: Readings and Resources for Faculty. Providence: Campus Compact.

2 What Matter in a Quality Service-Learning Course …

31

Brockbank, A., & McGill, I. (1998). Facilitating reflective learning in higher education. St. Edmundsbury Press Ltd. Bringle, R.G., Reeb, R. N., Brown, M. A., & Ruiz, A. I. (2016). Service learning in psychology: Enhancing undergraduate education for the public good. American Psychological Association. Burch, S. (2013). Strategies for service-learning assessment in dental hygiene education. The Journal of Dental Hygiene, 87(5), 265–270. Carrington, S., & Selva, G. (2010). Critical social theory and transformative learning: Evidence in pre-service teachers’ service-learning reflection logs. Higher Education Research & Development, 29(1), 45–57. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164–181. Chang, S., Anagnostopoulos, D., & Omea, H. (2011). The multidimensionality of multicultural service-learning: The variable effects of social identity, context and pedagogy on preservice teachers’ learning. Teaching and Teacher Education, 27(7), 1078–1089. https://doi.org/10.1016/ j.tate.2011.05.004 Clayton, P. H., Bringle, R. G., Senor, B., Huq, J., & Morrison, M. (2010). Differentiating and assessing relationships in service-learning and civic engagement: Exploitative, transactional, or transformational. Michigan Journal of Community Service Learning, 16(2), 5–21. Conway, J. M., Amel, E. L., & Gerwien, D. P. (2009). Teaching and learning in the social context: A meta-analysis of service learning’s effects on academic, personal, social, and citizenship outcomes. Teaching of Psychology, 36(4), 233–245. Covitt, B. A. (2006). Self-determination and student perceptions in environmental service-learning. Applied Environmental Education and Communication, 5(3), 171–181. Conner, J., & Erickson, J. (2017). When does service-learning work? Contact theory and servicelearning courses in higher education. Michigan Journal of Community Service Learning, 23(2), 53–65. Connors, K., & Seifer, S. D. (2005, September). Reflection in higher education servicelearning. Community-Campus Partnerships for Health. Retrieved November 15, 2015, from http://servicelearning.org/resources/fact_sheets/he_facts/he_reflection/index.php?search_ term=Reflection%20in%20higher%20ed Dahan, T. A. (2016). Revisiting pedagogical variations in service-learning and student outcomes. International Journal of Research on Service-Learning and Community Engagement, 4(1), 3–15. Dougherty, D. (1992). Interpretive barriers to successful product innovation in large firms. Organization Science, 3(2), 179–202. Dubinsky, J. (2006). The role of reflection in service learning. Business Communication Quarterly, 69(3), 306–311. Enos, S., & Morton, K. (2003). Developing a theory and practice of campus-community partnerships. Building Partnerships for Service-Learning, 20–41. Eyler, J. (2001). Mapping service-learning reflection activities. In M. Canada (Ed.), Service learning: Practical advice and models. Jossey Bass. Eyler, J., Giles, D. E., & Schmiede, A. E. (1996). A practitioner’s guide to reflection in servicelearning: Student voices and reflections. Vanderbilt University Press. Eyler, J, L., Bradley, R., Goldzweig, I., Schlundt, D., & Juarez, P. (2010). The relationship between the quality of service-learning interventions and teen seatbelt use. In J. Keshen, B. A. Holland & B. E. Moely (Eds.), Research for what? Making engaged scholarship matter (1st ed., pp. 91–120). Information Age Publishing, Inc. Furco, A. (2002). Is service-learning really better than community service? A study of high school service program outcomes. In A. Furco, & S. H. Billig (Eds.), Service-learning: The essence of the pedagogy (pp. 23–50). Information Age Publishing. Hatcher, J. A., & Bringle, R. G. (1997). Reflections: Bridging the gap between service and learning. College Teaching, 45, 153–159. Hildenbrand, S. M., & Schultz, S. M. (2015). Implementing service learning in pre-service teacher coursework. Journal of Experiential Education, 38(3), 262–279.

32

H. Xu

Howard, J. P. (1998). Academic service learning: A counternormative pedagogy. New Directions for Teaching and Learning, 73, 21–29. Hullender, R., Hinck, S., Wood-Nartker, J., Burton, T., & Bowlby, S. (2015). Evidences of transformative learning in service-learning reflections. Journal of the Scholarship of Teaching and Learning, 15(4), 58–82. Jacoby, B. (2015). Service-learning essentials: Questions, answers, and lessons learned. JosseyBass. Kackar-Cam, H., & Schmidt, J. A. (2014). Community-based service-learning as a context for youth autonomy, competence, and relatedness. The High School Journal, 98(1), 83–108. Kalles, S., & Ryan, T. G. (2015). Service-learning: Promise and possibility in post-secondary education. International Journal of Progressive Education, 11(1), 132–148. Keith, N. Z. (1997). Doing service projects in urban settings. In A. S. Waterman (Ed.), Servicelearning: Applications from the research (pp. 127–149). Lawrence Erlbaum Associates. Kolb, D. (1984). Experiential learning: Experience as the source of learning and development. Prentice Hall. Levesque-Bristol, C., & Stanek, L. R. (2009). Examining self-determination in a service learning course. Teaching of Psychology, 36(4), 262–266. Maurrasse, D. J. (2002). Higher education-community partnerships: Assessing progress in the field. Nonprofit and Voluntary Sector Quarterly, 31(1), 131–139. Mayer, K., Braband, B., & Killen, T. (2017). Exploring collaboration in a community-academic partnership. Public Health Nursing, 34(6), 541–546. McCrea, E. A. (2010). Integrating service-learning into an introduction to entrepreneurship course. Journal of Management Education, 34(1), 39–61. McMillan, J. H. (2007). Classroom assessment: Principles and practice for effective standardsbased instruction (4th ed.). Pearson Education, Inc. Meierhofer, J., Baumgartner, L., Howard, K., Lounsbery, J., Reidt, S., & Moon, J. (2013). Facilitating student pharmacist learning through student-led development of a service-learning opportunity. Currents in Pharmacy Teaching and Learning, 5(6), 611–615. Mihalynuk, T. V., & Seifer, S. D. (2002). Partnerships for higher education service-learning. National Service-Learning Clearinghouse. Misyak, S., Culhane, J., McConnell, M., & Serrano, E. (2016). Assessment for learning: Integration of assessment in a nutrition course with a service-learning component. NACTA Journal, 60(4), 358–363. Molee, L. M., Henry, M. E., Sessa, V. I., & McKinney-Prupis, E. R. (2010). Assessing learning in service-learning courses through critical reflection. Journal of Experiential Education, 33(3), 239–257. Morgan, W., & Streb, M. J. (2003). First, do no harm: Student ownership and service-learning. Metropolitan Universities: an International Forum, 14(3), 36–52. National Service-Learning Cooperative. (1998). Essential elements of service learning. National Youth Leadership Council. National Youth Leadership Council. (2008). K-12 Service-learning standards. Retrieved from https://www.nylc.org/page/standards Satterley, D. (1989). Assessment in schools (2nd ed.). Basil Blackwell. Salam, M., Iskandar, D. N. A., Ibrahim, D. H. A., & Farooq, M. S. (2019). Service learning in higher education: A systematic literature review. Asia Pacific Education Review, 20(4), 573–593. Sanford, N. (1967). Self and society: Social change and individual development. Atherton Press. Schlesselman, L., Borrego, M., Bloom, T. J., Mehta, B., Drobitch, R. K., & Smith, T. (2015). An assessment of service-learning in 34 US schools of pharmacy follow up on the 2001 professional affairs committee report. American Journal of Pharmaceutical Education, 79(8), 1–11. Schön, D. A. (1987). Educating the reflective practitioner. Jossey Bass. Schön, D. A. (1991). The reflective practitioner. Temple Smith. Schön, D. A. (1992). The theory of inquiry: Dewey’s legacy to education. Curriculum Inquiry, 22(2), 119–139.

2 What Matter in a Quality Service-Learning Course …

33

Sheafer, V. (2014). Using service learning to teach classic learning theories. Psychology Journal, 11(2), 77–82. Simons, L., & Cleary, B. (2006). The influence of service learning on students’ personal and social development. College Teaching, 54(4), 307–319. Stiggins, R. J. (2005). Student-involved assessment for learning (4th ed.). Pearson Merrill Prentice Hall. Sturgill, A., & Motley, P. (2014). Methods of reflection about service learning: Guided vs. free, dialogic vs. expressive, and public vs. private. Teaching and Learning Inquiry, 2(1), 81–93. Tryon, E., Stoecker, R., Martin, A., Seblonka, K., Hilgendorf, A., & Nellis, M. (2008). The challenge of short-term service-learning. Michigan Journal of Community Service Learning, 14(2), 16–26. Welch, M., & James, R. C. (2007). An investigation on the impact of a guided reflection technique in service-learning courses to prepare special educators. Teacher Education and Special Education, 30(4), 276–285. Yancey, K. B. (1998). Reflection in the writing classroom. Utah State University Press. Yorio, P. L., & Ye, F. (2012). A meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9–27. Yun, J. A. H. (2009). Critical reflection among elementary school teachers: an examination of content, cognitive style, and integrative complexity (Doctoral Dissertation). University of Southern California. ProQuest/UMI. (Publication No. AAT).

Chapter 3

Faculty Perceived Impacts of the Support from a Centralized Service-Learning Office Huixuan Xu

and Maureen Yin Lee Chan

Abstract Service learning (SL), as a form of experiential learning, is a challenging pedagogy for faculty members because they have to restructure the traditional creditbearing courses, liaise with community agencies and NGOs, arrange service activities that help students connect service with academic content, etc., all of which are out of the “comfort zone” of traditional university teaching. This chapter will introduce the support that was provided by a central service-learning office in LU, and then outline a faculty member’s experience of how she collaborated with the office to extend the partnership with community agencies, to improve the training and preparation for students, and to enhance the prompt feedback to facilitate student learning during the period of service. This chapter will explain the importance of the support from a centralized service-learning office from faculty member’s perspective and discuss the issues in relation to the institutionalization of service learning. Keywords Institutionalization of service learning · Faculty engagement · Institutional support

3.1 Introduction Over the past two decades, many higher education institutions have invested significant financial and human resources toward the support of service-learning pedagogy (Rosing et al., 2010). The growing popularity of service-learning results from the positive student learning outcomes reported (e.g., Brown & Bright, 2017; Ma et al., 2019; Shek et al., 2019). In Hong Kong, service learning has also gained a footing in the tertiary education over the past decade (Shek et al., 2020): some institutions have institutionalized service learning with setting a central service-learning office to coordinate the development and implementation of service-learning courses; some H. Xu (B) · M. Y. L. Chan Department of Curriculum and Instruction, The Education University of Hong Kong, Hong Kong, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_3

35

36

H. Xu and M. Y. L. Chan

have required service learning as a required credit-bearing experience; others simply have arranged service learning as extra-curriculum activities. With the growing importance and increasing popularity of service learning as a pedagogy in Hong Kong educational institutions, the expectations on faculty members for their contributions to service-learning courses have become higher. When an academic staff considers teaching service learning, they need to spend time and effort to develop a new course with service learning as a part of the course, or transform their existing courses into ones with service-learning elements amidst their already bestowed heavy workload. Although teaching service learning results in faculty members’ expanded roles from their traditional classroom teaching and research to include community engagement, barriers are existed that discouraging faculty members’ commitment to teaching service learning. Hou (2009) explored challenges of faculty members who were adopting service learning and indicated that the time commitment and additional planning are required. Students’ concerns about service learning, such as their placement in the community, time, and scheduling also put additional pressure on faculty members (Cooper, 2014). Abes et al. (2002) found that the two strongest potential deterrents were the time-intensive nature of service learning and challenges with coordinating projects in the community. For service learning to be effective, an additional stakeholder, a community partner is of critical importance for structuring a service-learning course. Murphy and Rasch (2008) highlighted that the students and the community should contribute in defining and achieving goals of servicelearning endeavors. Vernon and Foster (2002) indicated that educational institutions should outreach more in explaining and increasing the level of awareness of community members in their responsibilities when planning and implementing servicelearning courses. However, faculty members who teach service-learning courses are the persons to take actions on the abovementioned tasks. In particular, the most significant deterrent to faculty involvement in service learning is its lack of recognition in the faculty reward structure (Abes et al., 2002; Morton & Troppe, 1996; Stanton, 1994). To understand the challenges faced by the faculty members who are engaged in teaching service learning and the support they need to successfully implement service learning, Furco and Moely (2012) consolidate different views of scholars and summarize four conditions that are important for faculty buy-in and support for an instructional innovation. The four conditions include: 1. 2.

3. 4.

The goals of the innovation must be clearly communicated to faculty members and must be consistent with their values and concerns; Faculty members must have opportunities to gain expertise with the innovation and to explore their questions, without making inordinate demands on their time (Baxley et al., 1999; Dotolo, 1999; Faculty members must perceive an institutional commitment to provide ongoing support for the innovation (Young et al., 2007); Members should see rewards for their participation, in terms of their own professional development (Williams et al., 2007).

3 Faculty Perceived Impacts of the Support …

37

In addition, Demb and Wade (2012) and other scholars (Jaeger & Thorton, 2006) acknowledge that the involvement of faculty in service appears to see mostly intrinsic rewards and participate in engagement because they feel a responsibility to society. Ma et al. (2016) claim that faculty involvement is one of the important factors for students’ learning, especially for academic service learning. They advise that it is valuable to investigate the factors that affect the adoption of service learning in teaching and service-learning’s impact on faculty from faculty’s perspectives. Some researchers have mentioned that a centralized approach, or office of outreach and engagement, is critical to institutionalizing engagement efforts (Antonio et al., 2000). Although the institutionalization of an educational innovation is a complex process that requires sustained institutional commitment and support from key stakeholders (Furco & Moely, 2012), it has been acknowledged the importance of institutional support in smoothly running service-learning courses. Thus, this chapter will explain the importance of institutional support from a centralized service-learning office from faculty member’s perspective and discuss the issues in relation to the institutionalization of service learning. The faculty member is Clare (Pseudonym), who was a young tenure-track academic staff when interviewed. She was from the Department of Management of LU and taught two three-credit service-learning courses, titled “Strategic Management” and “Leadership and Teamwork.” In this chapter, we focused on how she collaborated with the central office of service learning in LU to extend the partnership with community centers, improve the training and preparation for students, and enhance the prompt feedback to facilitate student learning during the period of service. The outcomes of this effective faculty engagement and corresponding support system included the transformation of students from dislike to participative attitude in joining the service-learning courses as well as the smooth integration of service-learning components in her academic courses. The establishment of sustainable relationship with community partners was another fruitful outcome.

3.2 Method A purposeful sampling strategy (Creswell & Poth, 2018) was applied for selection of a case for this study. The faculty member of this case was selected based on her successful attempts of running two service-learning courses with her being a young faculty member without prior service-learning experiences and knowledge. When other faculty members complained about the heavy workload of a service-learning course, she had an entirely different view and attitude toward her two service-learning courses. Her positive attitudes worth the efforts of exploration into the contributing factors. A case study design (Yin, 2014) was employed as it involves the study of the practical experiences of a particular faculty member in leading service-learning course. The data sources covered documents of course information, examples of students’ assignments, a published journal paper regarding service learning and an interview.

38

H. Xu and M. Y. L. Chan

The course documents provided information regarding the course structure, learning objectives and the assessment plan. The interview was conducted to know more about the practical experiences of the faculty member in the course of service-learning development of the two courses. Data analysis focused on the factors that fostering or constraining the faculty member to develop and improve her service-learning courses.

3.2.1 Brief Information of the Two Service-Learning Courses The two service-learning courses led by the faulty member were Strategic Management and Leadership and Teamwork and they were opened to all undergraduates. For the service-learning part in the Strategic Management course, about 30 students of the course were divided into 5 or 6 groups and they worked with a social enterprise. Students did not need to serve the service recipients directly but provided indirect service by giving consulting services on strategic management to the social enterprise. Students used relevant knowledge of strategic management to help the organization handle some tasks based on the actual situation in which it operated. Another course Leadership and Teamwork required students to provide direct service. For example, when cooperating with the Elder Academy, the vision of which is to inspire elders to pursue learning and to foster a sense of worthiness among them. The students were required to serve the elderly in person, such as teaching them how to use smartphones. Unlike Strategic Management, this course required each group to serve a different institution, with each institution accommodating fewer students. Therefore, it required the establishment of the partnership with a few community partners for the whole class of about 30 students.

3.3 Findings 3.3.1 The Challenges Faced by the Faculty Member 3.3.1.1

Challenges Posed by Students

Clare said that when she first came to LU, she found many students were not favorable for service learning. The main reason might be that the workload of service learning was relatively heavy. After some years, Clare observed that students had improved their acceptance level of service- learning. Some students even chose more than one service-learning courses. This was explained as learning motivation from Clare’s view. Clare noted that students had different motives to join a required servicelearning course and she identified three main motives: being instrumental to acquire knowledge and skills as well as gain more experiences which were helpful for their

3 Faculty Perceived Impacts of the Support …

39

future career pursuit; to meet their personal interest and enjoy the learning through carrying out the corresponding service-learning activities; and to simply meet the graduation requirement. After identifying students’ different motives to join a service-learning course, Clare began to strategically arrange service activities to meet different students’ expectation. For instance, her courses were credit-bearing ones and many students would like to have a good academic result. Clare added some learning support such as consultation with a focus on how to integrate service with course academic learning and how to get a better performance in her courses. Thus, students felt that their participation in the service-learning course could help them get higher scores in their academic work. In addition, she also made association of students’ personal learning goals with the service-learning projects and choices were provided, in such regard, students’ participation in the projects could benefit and help them achieve their personal goals. Apart from different motives in service learning, Clare also noted the challenges to respond to students’ negative emotions generated during the service process, in particular, when the service experience was not necessarily pleasant. For example, she had cooperated with an institution on fund-raising projects. However, the students failed to achieve the fund target and felt extremely frustrated. She suggested to the students that if they could reflect on the reasons and provide clear explanations, they would be given satisfactory scores in the end. This experience posed challenges to her on how to arrange service activities that could avoid failure experience of students.

3.3.2 Challenges Posed by the Teaching-Research Divide Clare believed that a very important factor during the development of service learning in university was staff participation. However, a young tenure-track academic staff was expected to get external competitive funding awarded and to publish articles in high ranking journals. Though teaching is also a major task, it is not as significantly important as research and publication. The study of Roob et al (2020), which focused on faculty engagement in physics outreach, reported that service learning provided opportunities for university research faculty who desired to seek ways to make meaningful and broader impact contributions to include as an essential part of research grant proposals and to enhance personal commitment. Service Learning contributed to the research outputs of faculty and therefore it was an attraction to faculty members so far if their engagement created value for them more than their inputs to service-learning activities. However, an integration of teaching and research was not easy to achieve for most faculty members when they were assigned to teach service learning. When Clare was invited by the Head of Department to teach service-learning courses, she mentioned that the timely support received from the university to maintain her interest and commitment to teaching service learning was very important.

40

H. Xu and M. Y. L. Chan

3.3.3 The support from Office of Service Learning (OSL) LU set up the OSL to fully support academic staff in running service-learning courses. When Clare first came to LU as a young faculty member, she was not familiar with service learning. At that time, the OSL provided her with advice for implementing service learning, offered her human resource assistance in arrangement of orientation meetings, field trips and consultation services, financial assistance to operations of service-learning projects and established a community of practice for continuous improvement. Without such support, the workload of the faculty would be heavier and the threshold for faculty to participate in service learning would be higher. The following paragraphs show her positive experience in taking up the servicelearning courses. Through her story of developing service-learning courses, we will draw the effective contextual factors for service-learning development in higher education institutions.

3.3.3.1

Human Resources

With the establishment of OSL, the OSL staffs have focused their effort on building up network with community partners, helping faculty members to connect to community partners, and providing mentoring to new faculty members on running servicelearning courses. Clare had benefited from the support of OSL when she newly took up service-learning courses. OSL helped students fulfill the graduate requirement by launching a registration system, provided training to students to prepare them prior to the service, and guided them through the orientation process to know the community partners and service recipients. The OSL monitored the service-learning process and continued to support and coordinate the alignment of expectations of different stakeholders or to cope with new arising needs throughout the whole period. Initially, the OSL staff were responsible for all the support work. They later began to hire outstanding students with service-learning experience as Teaching Assistant (TA). In offering supporting work, TA was mainly responsible for communicating with institutions and students.

3.3.3.2

Network Resource Support

The OSL had the network resources of relevant cooperative institutions. It recommended some suitable institutions with reference to the needs of course instructors, and let the instructors choose by themselves. Before the commencement of the semester, if the initial impression of the recommended institution was suitable, an instructor would discuss with the institution some details, which included the service time and mutual expectations. If the instructor was not satisfied with the selected institution, he would invited OSL to continue to search and explore. Clare

3 Faculty Perceived Impacts of the Support …

41

said that through this specific arrangement, instructors often could find suitable institutions. After the identification of a partner institution, a course instructor needed to undertake more communication work, such as discussing with the institution how to develop plans for specific service arrangements. If they were happy with the cooperation, the course instructor would be responsible for the follow-up contact with the same institution the next year. Based on the remarks made by Clare, a trusting relationship had been developed among her, the academic staff and colleagues from OSL. With the support of OSL, she could find further suitable community partners, and therefore build up her network at the same time. The support by OSL in preparing her students to do the servicelearning project also released her much time and effort and she remarked that there was no additional workload for her to lead her service-learning courses.

3.3.3.3

Teaching Support

The tailor-made service training course for students At LU, the pre-service training was mainly provided by OSL. The training it provided comprised of two parts: one was about general knowledge and skills, and another part was course specific skills. OSL also tailored training to the students with regard to the nature of the service-learning project arrangements between academic staff and community partners. For example, Clare had organized courses to serve ethnic minorities, the OSL staff then provided training with students that was similar to a cultural tour, took students to visit mosques and taste local food, and helped students understand their culture. Clare commented, “OSL is very flexible. Many of our teachers’ requirements of training can basically be met.” “The staff in OSL are very professional in the provision of training for students.”

As part of the training, at the beginning of the semester, OSL also led the students to meet the person in charge of the partner institution to get a preliminary understanding of the institution and the service recipients. Co-facilitating students’ reflection The role of reflection in the process of service learning has been much discussed and recognized. For instance, Jacoby (1996) emphasizes that reflection and reciprocity are key components of service learning. Jay Conger (1992) proposes that personal growth is involved in learning through opportunities to reflect on one’s behaviors. Similarly, Allen and Hartman (2008) indicate the sources of learning and claim that service learning provides a source of learning for personal growth while other sources of learning for personal growth includes group reflection, individual reflection, team building, and developmental relationships (Lester, 2015). When reflection activities are a regular part of the course and serve to clarify personal values, it enhances the quality of the course (Hatcher et al., 2004). In Clare’s case, the OSL staff also took part in some reflection related sessions. For example, during the training, the OSL staff included a part of training on conducting reflection exercises, which included the knowledge of reflection process and how to

42

H. Xu and M. Y. L. Chan

self-facilitate in the related reflection exercises. In the middle of the semester, sometimes, in the course of Strategic Management, Clare invited an institution representative as well as one OSL staff to the class. Students could ask him questions based on the problems encountered in their service experience and assignments. During that interactive process, the OSL staff helped facilitate the discussion.

3.3.3.4

Financial Support

OSL does not only support with human resources but also financial resources. Students sometimes need funding for purchasing materials for service delivery. The financial assistance was offered mainly for purchase of training courses and for daily expenses of a student group, which included transportation and activity materials.

3.4 Discussions The support from a central service-learning unit contributes to fulfilling the four conditions as presented by Furco and Moely (2012) for faculty involvement in conducting service-learning courses. In this case, there were details of the four conditions to explain for their effects on facilitating the engagement of faculty members in running service-learning course. OSL is a symbol of institutional support while the professionalism of OSL builds the trust of the faculty members in running the service-learning courses. The open attitude of faulty members, such as Clare that reported in the present chapter, in accepting the new pedagogy of service learning while acquiring the expertise from OSL helped them run service-learning course successfully. Identification of suitable case is a challenge in case study design (Creswell & Poth, 2018, p. 102). As only a single case was selected in the presents study, the generalizability of the findings was a concern. Accordingly, the case selected may be narrow in scope as there may be other factors besides the institutional ones that worth exploration for positive impacts. To address the limitations of generalizability, in future studies data could be collected from several key stakeholders such as the service partners and other faculty members. In addition, a multiple-case design may be applied to show the extent which faculty members provide different responses to the role of a central unit in supporting teaching service learning.

3.5 Conclusion Service learning, grounded in the experiential learning model, is categorized in the literature as both a philosophy and a pedagogy utilizing the service experience as the fundamental program (Deeley, 2010; Kenworthy-U’Ren & Peterson, 2005). The

3 Faculty Perceived Impacts of the Support …

43

experience of the faculty member presented in the present chapter informs the value of a facilitating role of a centralized service learning supporting unit for faculty members and students in this experiential learning model. In practice, every stakeholder in the service-learning process undergoes a separate experiential learning process and which may cause misalignment of expectations and objectives if without active interactions between them. With the inclusion of active interaction among different stakeholders, the experiential learning processes may yield effective learning outcomes for every stakeholder involved. The service-learning development is a continuing process as new challenges will arise with increasing demand by the educational institutions and the community partners. The successful outcome of a service-learning course requires a pooled effort of stakeholders and these include institutional support, faculty engagement, students’ commitment and community involvement. Between stakeholders, there should be effective interactive process to ensure alignment of objectives and to ensure quality of service-learning output. OSL acts like a central unit representing the institutions, perform this bonding role among institutions, faculty members, students, and community partners. The support offered by OSL relieves the pressure brought by the expanded roles of faculty members as well as the students. Their professionalism in service-learning projects help engage the young faculty members and build the service-learning resources. With careful selection of community partners, LU helps community development while meeting students’ course learning objectives and individual learning goals as well. To summarize, the increase in service-learning demand requires more human resources and funding to drive forward different service-learning projects. The support of a central unit is nonetheless getting more important as it is not only supporting but helping the development of service-learning curriculum in higher education institutions. It represents the institutional support to faculty members and it symbolizes the emphasis that institutions place on civic engagement and contributions of our future generations. Acknowledgment The authors acknowledge the University Grants Council of Hong Kong for the support through the Funding Scheme for Teaching and Learning Related Proposals (PolyU4/T&L/16–19) and the Education University of Hong Kong for supporting the completion of this manuscript. The authors also would like to thank the faculty members involved in the project for their kind support.

References Abes, E., Jackson, G., & Jones, S. (2002). Factors that motivate and deter faculty use of servicelearning. Michigan Journal of Community Service Learning, 9(1), 5–17. http://hdl.handle.net/ 2027/spo.3239521.0009.101 Allen, S. J., & Hartman, N. S. (2008). Leadership development. SAM Advanced Management Journal, 73(1), 10–19, 62–63. https://www.proquest.com/scholarly-journals/leadership-develo pment-exploration-sources/docview/231151159/se-2?accountid=11441

44

H. Xu and M. Y. L. Chan

Baxley, E. G., Probst, J. C., Schell, B. J., Bogdewic, S. P., & Cleghorn, G. D. (1999). Programcentered education: A new model for faculty development. Teaching and Learning in Medicine, 11(2), 94–99. Antonio, A. L., Astin, H. S., & Cress, C. M. (2000). Community service in higher education: A look at the nation’s faculty. Review of Higher Education, 23(4), 373–398. Medicine, 11(2), 94–99. https://doi.org/10.1207/S15328015TL110207 Brown, K., & Bright, L. (2017). Teaching caring and competence: Student transformation during an older adult focused service-learning course. Nurse Education in Practice, 27, 29–35. https:// doi.org/10.1016/j.nepr.2017.08.013 Conger, J. (1992). Learning to lead: The art of transforming managers into leaders. Jossey-Bass. Cooper, J. R. (2014). Ten years in the trenches. The Journal of Experiential Education, 37(4), 415–428. https://doi.org/10.1177/1053825913513721 Creswell, J. W., & Poth, C. N. (2018). Qualitative inquiry & research design: Choosing among five approaches (4th ed.). Sage Publications. Demb, A., & Wade, A. (2012). Reality check: Faculty involvement in outreach & engagement. The Journal of Higher Education, 83(3), 337–366. Deeley, S. J. (2010). Service-learning: Thinking outside the box. Active Learning in Higher Education, 11(1), 43–53. https://doi.org/10.1177/1469787409355870 Dotolo, L. G. (1999). Faculty development: Working together to improve teaching and learning. New Directions for Higher Education, 1999(106), 51–57. https://doi.org/10.1002/he.10606 Furco, A., & Moely, B. E. (2012). Using learning communities to build faculty support for pedagogical innovation: A multi-campus study. The Journal of Higher Education (columbus), 83(1), 128–153. https://doi.org/10.1353/jhe.2012.0006 Hatcher, J. A., Bringle, R. G., & Muthiah, R. (2004). Designing effective reflection: What matters to service-learning? Michigan Journal of Community Service Learning, 11(1), 38–46. http://hdl. handle.net/2027/spo.3239521.0011.104 Hou, S. (2009). Service learning + new master of public health student = challenges for professor. International Journal of Teaching & Learning in Higher Education, 20(2), 292–297. https://files. eric.ed.gov/fulltext/EJ864347.pdf Jacoby, B. (1996). Service-learning in higher education: Concepts and practices (1st ed.). JosseyBass. Jaeger, A. J., & Thornton, C. H. (2006). Neither honor nor compensation: Faculty and public service. Educational Policy, 20(2), 345–366. Kenworthy-U’Ren, A., & Peterson, T. (2005). Service-learning and management education: Introducing the “WE CARE” approach. Academy of Management Learning & Education, 4(3), 272–277. https://doi.org/10.5465/amle.2005.18122417 Lester, S. W. (2015). Melding service learning and leadership skills development. The Journal of Experiential Education, 38(3), 280–295. https://doi.org/10.1177/1053825915576196 Ma, C. H.K., Law, S., Lo, D. F. Y., Chan, C., (2016). Service-learning impact on faculty in Hong Kong through faculty engagement model. Proceedings of the 2nd International Conference on Service-Learning, 69–75. Lingnan University. Ma, C. M. S., Shek, D. T. L., & Li, P. P. K. (2019). Service leadership through serving socially deprived students: Experience gained from Corporate–Community–University Partnership (Project WeCan). In Quality of Life in Asia (Vol. 12, pp. 83–112). https://doi.org/10.1007/ 978-981-13-0448-4_6 Morton, K., & Troppe, M. (1996). From the margin to the mainstream: Campus compact’s project on integrating service with academic study. Journal of Business Ethics, 15(1), 21–32. https://doi. org/10.1007/BF00380259 Murphy, J. W., & Rasch, D. (2008). Service-learning, contact theory, and building black communities. Negro Educational Review, 59(1/2), 63–78. Roob, E., Bouscher, R. F., Kleinbaum, E. I., Sederberg, D., & Csathy, G. (2020). Too cool two— Service learning and faculty engagement in physics outreach. The Physics Teacher, 58(1), 28–31. https://doi.org/10.1119/1.5141967

3 Faculty Perceived Impacts of the Support …

45

Rosing, H., Reed, S., Ferrari, J. R., & Bothne, N. J. (2010). Understanding student complaints in the service learning pedagogy. American Journal of Community Psychology, 46(3), 472–481. https:// doi.org/10.1007/s10464-010-9338-5 Stanton, T. K. (1994). The experience of faculty participants in an instructional development seminar on service-learning. Michigan Journal of Community Service Learning, 1(1), 7–20. http//hdl.handle.net/2027/spo.3239521.0001.101 Shek, D. T., Dou, D., Zhu, X., & Chai, W. (2019). Positive youth development: Current perspectives. Adolescent Health, Medicine and Therapeutics, 10, 131–141. Shek, D. T., Ma, C. M., & Yang, Z. (2020). Transformation and development of university students through service-learning: A Corporate-Community-University Partnership Initiative in Hong Kong (Project WeCan). Applied Research in Quality of Life, 15(5), 1375–1393. https://doi.org/ 10.1007/s11482-019-09738-9 Vernon, A., & Foster, L. (2002). Community agency perspectives in higher education servicelearning and volunteerism. In S. Billig & A. Waterman (Eds.), Studying service learning: Innovation in education research methodology (pp. 1–12). Mahwah, NJ: Earlbaum. Williams, B. C., Weber, V., Babbott, S. F., Kirk, L. M., Heflin, M. T., O’Toole, E., Schapira, M. M., Eckstrom, E., Tulsky, A., Wolf, A. M., & Landefeld, S. (2007). Faculty development for the 21st century: Lessons from the society of general internal medicine-hartford collaborative centers for the care of older adults. Journal of the American Geriatrics Society (JAGS), 55(6), 941–947. https://doi.org/10.1111/j.1532-5415.2007.01197.x Yin, R. K. (2014). Case study research: Design and methods (5th ed.). SAGE. Young, C. A., Shinnar, R. S., Ackerman, R. L., Carruthers, C. P., & Young, D. A. (2007). Implementing and sustaining service-learning at the institutional level. The Journal of Experintial Education, 29(3), 344–365. https://doi.org/10.1177/105382590702900306

Chapter 4

Assessing Students in Academic Service-Learning: Faculty Experiences and Challenges Rina Marie Camus, Grace Ngai, Kam-Por Kwan, and Daniel T. L. Shek

Abstract Service-learning contributes to student development in multiple ways, through academic, social, personal, and civic outcomes. Because it intends varied outcomes and is itself a complex pedagogy, formally assessing students in servicelearning courses can prove particularly challenging for faculty. Challenges include managing on-site observation of service performance, working with multiple assessors, and a heavy workload. This chapter is a qualitative study of faculty challenges in formally assessing students in academic service-learning courses. The study uses phenomenographic research methods and is based on interviews with 17 faculty members from 4 universities in Hong Kong. It is hoped that difficulties faculty face in assessing service-learning will be better understood and that practical insights can be drawn to guide assessment practices in academic service-learning. Results show that all participants combined traditional and experiential assessment methods in their courses and that reflection was commonly used. However, on-site performance was not assessed by all and was the most reported area of challenge. On-site performance is primary means for faculty to see how students are applying their knowledge and skills. It also helps ensure the quality of service rendered to the community. Hence, the study recommends reinforcing course components and assessment methods that have direct bearing on service work.

R. M. Camus (B) · G. Ngai · K.-P. Kwan Service-Learning and Leadership Office, The Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] G. Ngai e-mail: [email protected] K.-P. Kwan e-mail: [email protected] D. T. L. Shek Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_4

47

48

R. M. Camus et al.

Keywords Service-learning · Faculty · Formal assessment/grading · Challenges · Reflection · On-site performance

4.1 Introduction Service-learning is an experiential pedagogy which links academic learning with organized service to the community. Through service, close interaction with the community, and reflection on experience, students learn to apply academic or professional knowledge and skills to real-life contexts, enhance their social capacity and personal growth, and develop civic-mindedness. The extensive literature about service-learning assessment that exists to date include a good number of studies about its varied outcomes (e.g., Astin et al., 2000; Conway et al., 2009). Many of these are program evaluations based on scales or measures from surveys administered to students which rely on personal perceptions of learning. This chapter differs in exploring the issue of “assessment” in the sense of grading practices. Specifically, we refer to methods employed by faculty to grade students based on their achievement of course-intended learning outcomes. Unlike assessment in the sense of program evaluation, this topic is not well addressed in the literature despite the popularity of academic service-learning programs and the importance of assessment in education. Assessment is an indispensable component of formal education and is ultimately about improving student learning. From students’ perspective, assessment is the curriculum (Biggs, 2003; Ramsden, 1992), and also what drives learning (Cowan, 2005). More clarity about assessment experiences and practices in service-learning can contribute to faculty work and redound to better learning for students. Like other academic courses, academic service-learning requires rigorous methods of assessment for which teachers must go beyond self-reports to direct evidences of learning from students’ works and artifacts. Academic service-learning should be assessed rigorously if it is to be taken seriously. In concrete, it is necessary to assess the performance of individual students according to their attainment of intended learning outcomes, and to assign grades that reflect their performance in a fair and accurate manner. Together with written artifacts and reports, students’ on-site service performance, deliverables, and attitude are also important evidences of learning. Assessment in higher education is, as it is, a complex and multifaceted enterprise, one which calls for observation, measurement, and judgment from faculty (Simpson, 1996). Service-learning appears to exacerbate assessment challenges for several reasons. Compared to conventional, instruction-based academic courses, servicelearning intends outcomes that go beyond academic ones which faculty are more accustomed to deal with. Besides, assessment in service-learning is not confined to the classroom nor campus, and often involves multiple assessors which may include community collaborators (Quinn & Shurville, 2009). Admittedly, there is real “difficulty of rigorous and authentic assessment of service-learning outcomes” (Butin,

4 Assessing Students in Academic Service-Learning …

49

2003, p. 1674), and as Bringle and Hatcher (1995) state, it is inevitable to employ a combination of assessment methods, i.e., “traditional measures of mastery of course content (as well as) supplementary forms of evaluation through reflection activities (such as) papers, directed writings, class presentations” (p. 117). Assessing academic service-learning is thus markedly different from assessing conventional academic subjects and may generate new challenges for faculty. For academic service-learning to be successful, these challenges need to be better understood and addressed. There are many studies about the impact of service-learning programs on the learning, development, and transformation of students (e.g., Lai & Hui, 2021; Shek et al., 2020), and this includes students initially averse to mandatory service-learning courses (Chan et al., 2019, 2021). Very few studies, however, look into how teachers grade their students in academic service-learning courses. Investigating assessment practices is particularly important in Asian contexts where research on service-learning is still inchoate (cf. Shek et al., 2019). This chapter helps to fill the gap in literature by presenting a qualitative study about assessing students in service-learning focusing on challenges experienced by faculty. What challenges do faculty experience in formally assessing students in academic service-learning courses? The study addresses this question drawing from in-depth interviews with 17 faculty members from four universities in Hong Kong. It is hoped that through this study, difficulties faculty face in assessing service-learning will be better understood and that practical insights can be drawn to guide grading practices.

4.2 Literature Review What sets academic service-learning apart from other academic courses in terms of assessment? How is this reflected in the assessment practices of service-learning instructors? Service-learning is one pedagogical practice where social, personal, and civic outcomes are as important as academic ones. Academic or professional outcomes of service-learning vary greatly according to course content, and, generally speaking, redound to mastery of course knowledge and skills and the linking and application of these to real-life contexts (Steinke & Fitch, 2007). Besides this type of outcome, service-learning intends a range of graduate attributes or generic cognitive and attitudinal gains, such as problem-solving and critical thinking, personal development (e.g., self-awareness, self-efficacy, moral growth), interpersonal abilities (e.g., communication, teamwork, collaboration, empathy, cultural understanding), and, most importantly, civic-mindedness or community engagement (e.g., prejudice reduction, social responsibility, committed citizenship) (cf. Bringle & Wall, 2020; Conway et al., 2009; Eyler & Giles, 1999; Felten & Clayton, 2011; Simons & Cleary, 2010). As mentioned, many studies about service-learning assessment are impact or outcome evaluations of service-learning programs which use survey methodology and rely on self-reports (Kezar, 2002; Lambright, 2019; Rubin & Matthews, 2013).

50

R. M. Camus et al.

Steinke and Fitch (2007) recapitulate a list of tools for direct and indirect measurement specifically used to gauge cognitive outcomes. Although surveys can be highly informative about the efficacy of service-learning programs, they are limited means to determine individual learning gains and are hardly basis for formal grading. Several sources concur about the need for more comprehensive approaches to service-learning assessment. Observing narrow focus on cognitive outcomes, Kezar (2002) proposed, for instance, using Howard Gardner’s (1993) theory of “Multiple Intelligences” as framework for assessing service-learning outcomes. Kezar argued that adopting such holistic framework would capitalize on service-learning’s strength as a pedagogy that gives import to interpersonal and intrapersonal abilities, seeing these as matters to be honed through formal education along with traditionally recognized forms of intelligences (p. 17). Kezar suggested deploying a variety of assessment types to allow students to demonstrate their learning in different ways—e.g., through portfolios, projects, exhibitions, and presentations—thus going beyond “paper-and-pencil essay tests and objective multiple-choice tests (which) tend to favor linguistic ability and logical and mathematical ability” (p. 18). In a similar vein, Chan (2012b) explains how Kolb’s Learning Theory—which begins with concrete experience to higher-order, operative thinking—can capture the learning process and outcomes that students demonstrate in their service work and reflection. Chan’s conclusion from her study is worth transcribing, With experiential learning, one important consideration is the assessment of the learning experience. It is beyond doubt that assessment drives student learning, and that the types of assessment will have an impact on students’ learning experiences… A further exploration into the different types of assessments for experiential learning, such as a reflective journal, direct observation, presentation, oral assessment and interview assessment will be beneficial to formulate suitable assessments for experiential learning to assess various learning outcomes. (pp. 413–414)

An earlier article by the same author helpfully evaluates widely used assessment methods in service-learning, in concrete, reflective journals, direct observation, presentation, and oral assessment (cf. Chan, 2012a). Assessment methods used in service-learning are varied. Reflection—whether in the form of journal writing, oral presentation, or other media—is widely recognized as an essential form of assessment in service-learning and serves both pedagogical and practical functions (Bringle & Hatcher, 1999; Camus et al., 2021; Eyler, 2002). Focusing on academic and cognitive outcomes, Ash and Clayton (2004) give prime importance to written reflections as articulations of student learning. Arguing that reflective artifacts constitute a rigorous basis for assessment, they offer sample rubrics to evaluate higher-order thinking skills manifested by students in their essays (cf. Ash et al., 2005). In a later work, the same authors propose using the “DEAL” Model— acronym for describe, examine, and articulate learning—a process which guides students to think deeply and critically about service experience (Ash & Clayton, 2009). Applying the DEAL Model in assessing service-learning courses, Molee et al. (2010) found that while the model and its accompanying rubrics are useful for documenting and determining “what students are learning (and) how well they are

4 Assessing Students in Academic Service-Learning …

51

learning” (p. 250), student participants of the study overall did not manifest higher levels of evaluating their learning and critical thinking. Molee et al. recommend experimenting with other ways of measuring depth of learning and critical thinking. They note too that in service-learning, “improving reflection skills is only part of what students are expected to accomplish… (they are also) required to work in community organizations” (p. 252). Noting similarities between study abroad programs and service-learning—both of which “don’t only examine knowledge and skills but also… attitudinal and dispositional outcomes”—Rubin and Matthews (2013) recommend adopting assessment practices established in the former to the latter. Likewise, Champagne’s (2013) study about service-learning projects of students in the health field illustrates the usefulness of triangulating different sources of data from students about their learning, for instance, written reflections, perceptions surveys, and annotated portfolios. For grading portfolios, Champagne specifically suggests an evidence-based approach in which students present written and visual materials to showcase competencies gained. Champagne also recommends involving multiple reviewers and using a shared rubric to facilitate “uniform assessment criteria so that the role of subjective opinions or multiple evaluators would be minimized” (p. 141). Investigating student outcomes achieved through service-learning projects, Lambright (2019) shows how student evaluations of group projects can be used with final exams to measure learning outcomes. These and other assessment methods are recapitulated by Oates and Leavitt (2003). According to Oates and Leavitt’s summary of useful materials for assessment, journal entries, evaluations by different parties, and narratives or case studies are helpful for assessing service-learning experience; tests and quizzes, videos and presentation for assessing learning of course content; finally, reflections and portfolios can be used to assess students’ integration of course content with service experience. There is thus an assortment of methods that can be used to evaluate different learning components and outcomes of service-learning courses. Each method, however, has limitations which may, on occasion, put faculty in a quandary. Reflective artifacts, for instance, have limitations associated with being self-reports (Eyler, 2000); portfolios and presentations run the risk of being mere showcase products in which visual aspects override content (Chan, 2012a; Rennert-Ariev, 2005; Shulman, 1998); group projects may generate free riders (McCrea, 2010). As for on-site observation, assessing higher-order outcomes and developing an appropriate rubric can be difficult (Chan, 2012a). Besides, visiting service locations to see students in action and examine their work often presents no small amount of logistic hurdles for faculty (Kelseyak et al., 2007). How does assessment with its variety and complexities play out in the actual experiences of service-learning faculty? In continuation, we describe the context and details of a phenomenographic study of service-learning assessment from the perspective of faculty.

52

R. M. Camus et al.

4.3 The Research Speaking from the field of Social Work, Shek et al. (2005) sounded off a wakeup call to strengthen the quality of qualitative research, among other things by being clear about different types of qualitative research and being more sensitive to and explicit about criteria such as reflexivity and auditability. Faculty challenges in service-learning assessment is a complex issue with variables that are not easily measurable. A qualitative study was needed to explore the topic in a way that would lead to in-depth and detailed understanding of experiences around it (Creswell, 2013; Merriam & Tisdell, 2016). We undertook, in particular, a phenomenographic research which allowed us to approach the issue by openly attending to how faculty interpret and describe their experiences without projecting researcher preconceptions or categories from literature. Phenomenography’s terms and methods largely coincide with phenomenology with which it shares an interpretive view of knowledge (Cibangu & Hepworth, 2016). Compared to phenomenology, however, phenomenography is more empirical in orientation and attempts a collective, descriptive analysis of experiences of a phenomenon rather than focusing on varying individual experiences to capture the essence of a phenomenon (Sin, 2010). To address the issue of reflexivity, we disclose our background and biases as researchers behind the study. We are academic and project staff affiliated with The Hong Kong Polytechnic University’s (PolyU) Service-Learning & Leadership Office (SLLO), a unit which supports the development, implementation, and evaluation of service-learning subjects offered by the different departments of the university. There are two so-called “biases” that we bring to the present research. As educators, we hold a student-centered view of learning and believe that pedagogical methods— including assessment—should be primarily driven by considerations of what helps student learning, learning being an active process where the student is protagonist. As service-learning practitioners, we believe that service-learning programs or subjects (particularly when intentionally designed and duly implemented) can be transformative experiences for students and offer real benefits to communities.

4.3.1 Participants of the Study The target participants of the study were faculty involved in the teaching of academic service-learning courses or programs in 4 universities in Hong Kong, including the host institution of this research. The four universities are members of a joint university project entitled “Cross-Institutional Capacity Building for Service-Learning in Hong Kong Higher Education Institutions.” Funded by Hong Kong’s University Grants Committee (UGC), the project was launched in 2016 and consists in collaborations to advance service-learning in Hong Kong’s higher education through research and by forming a community of practice. Collaborators from the service-learning offices

4 Assessing Students in Academic Service-Learning …

53

Table 4.1 Distribution of faculty according to experience and discipline (N = 17a ) Distribution by academic discipline Hard Sciences

Soft Sciences

Applied Mathematics; Biology; Building Services Engineering; Computing; Mechanical Engineering; Life Sciences

Chinese Medicine; Education; Management; Optometry; Rehabilitation Sciences; Textiles & Clothing

Years of experience teaching service-learning:

≥3

>3

≥3

>3

No. of faculty:

3

3

3

8

a

Females, 7; Males, 10

of the member universities apart from the host institution were asked to nominate service-learning faculty who could be approached for interview. Phenomenographic research relies on experiences and conceptions of participants. For this, a heterogeneous group of participants is important to increase conceptual variations and optimize the set of categories inductively drawn from the data set. We employed a purposive sampling design to achieve diversity among participants, specifically in terms of years of experience in teaching service-learning (3 or more years versus less than 3 years) and of academic background (hard versus soft sciences). The distinction in academic backgrounds broadly follows Biglan’s (1973) classification of disciplines between those having a defined and structured cognitive paradigm (“hard sciences”) and those with more flexible and variable paradigms (“soft sciences”) (cf. Doberneck & Schweitzer, 2017). The distribution of faculty participating in the study is shown in Table 4.1.

4.3.2 Data Collection and Analysis Principles of auditability, validity, objectivity, reliability, and research ethics were observed throughout the research process, as we shall explain. From the outset, approval for the study was sought from and granted by the host institution’s Human Subjects Ethics Sub-Committee (Ref. no. HSEARS20190816001). Candidates for interview were invited to attend face-to-face, semi-structured interviews in their workplace, that is, the natural setting where participants experience the issue being explored. Initial invitations were e-mail letters explaining to potential participants the purpose of the research, the expected length and period of interviews, and the freedom to withdraw from the research at any stage. Non-respondents were followed up twice by e-mail or phone.

54

R. M. Camus et al.

The interviews took place between August and November 2020. To enhance objectivity and reliability, efforts were made to faithfully capture and present participants’ views. Prior to actual interviews, participants were asked to complete a short, 7-item questionnaire about their service-learning experience, such as how long they had been teaching service-learning and details about their service-learning courses (e.g., origin, course title, service type, percentage of service-learning component, enrollment size; see Appendix 1 for the pre-interview questionnaire). Data from the survey together with interview records and course artifacts such as course syllabi and rubrics shared by participants provided useful details to contextualize statements made during the interviews and enabled us to triangulate findings. All interviews were individual and each lasted from 30 min to an hour. The interviews were conducted by two members of the research team: one took charge of facilitating the discussion, the other of note-taking and ensuring that all interview questions were asked (see Appendix 2 for the interview questions). To encourage participants’ free expression, interviews were carried out in a congenial manner and open-ended questions were used. Interviewers avoided interrupting participants when they spoke or thoughtfully paused. When needed, follow up questions were raised to ascertain or probe deeper into participants’ replies. Explicit permission was sought from participants to audio-record and transcribe the interviews for analysis. Interviews were transcribed verbatim by teaching assistants and each transcript was checked for accuracy by members of the research team. The resulting 17 pairs of audio files and transcripts obtained from the interviews were separately studied and returned to again and again by the principal researchers throughout the research process, first to familiarize and immerse themselves in the data, then to identify emerging themes, later to verify or count similar conceptions in an iterative process of drawing categories from the data set. Subjective interpretation was minimized by the active involvement in the study of multiple researchers. The research team held weekly meetings over the course of three semesters. During these meetings, research progress and findings were discussed, challenges were raised, and doubts and disagreements were resolved. Initial content analysis of interview records enabled the researchers to compare and decipher emerging themes around assessment practices and challenges. Once discussions led to a consensus about salient themes and categories, two members carried out parallel coding. To facilitate overall analysis, an anonymized spread sheet was created containing key contextual information and significant statements by participants in response to main interview questions. The spreadsheet and other research documents (e.g., interview notes, coding results, research meeting reports and notes, and manuscript drafts) were filed for reference and auditability. Substantial quotes provided in support of findings are language edited, that is, the minimum necessary to guarantee readability while staying close to the original utterances. The quotes are anonymized and details that could give away source identity were removed.

4 Assessing Students in Academic Service-Learning …

55

4.4 Findings To give some context before reporting findings about faculty challenges in servicelearning assessment, we first present background details about the assessment practices of participants, in particular, learning outcomes they intended, and assessment methods or components they used in their service-learning courses.

4.4.1 Learning Outcomes What learning outcomes did participants intend or assess in their service-learning courses? The question elicited a wide variety of answers which can be classified under three categories of discipline-/profession-specific, generic/soft skills, and civic learning outcomes. Discipline- or profession-specific learning outcomes were mentioned by all participants. Examples of these varied much according to participants’ disciplines and the content and focus of courses they taught. Fourteen participants taught servicelearning courses that impart specialized knowledge and skills to either majors of a discipline or as general education for students of any discipline, while three participants’ service-learning courses had interdisciplinary academic content. Soft skills or generic learning were also mentioned by most participants (12/17) as assessed outcomes of their courses, communication and teamwork being most common. Finally, civic learning outcomes were mentioned by a lesser though still majority number of participants (10/17). The summary of learning outcomes assessed in participants’ service-learning courses is shown in Table 4.2. Table 4.2 Learning outcomes of participants’ service-learning courses Discipline-/Profession—specific Soft Skills & Civic Learning Outcomes Generic Outcomes Outcomes Specified by 17/17 participants Examples (mentioned by no. of participants)

• specialized knowledge/ skills in different disciplines (14) e.g., health sciences, tourism, statistics, engineering, fashion • interdisciplinary learning (3) e.g., leadership theories, Design Thinking, understanding poverty

12/17 participants

10/17 participants

• communication (6) • teamwork (5) • problem-solving (3) • self-reflection/ evaluation (2) • leadership ability (2) • etc

• social awareness (6) • empathy/compassion/ care (4) • social responsibility (3) • application of learning to serve community (2) • etc

56 Table 4.3 Assessment methods used by participants (N = 17)

R. M. Camus et al. Assessment method

Used by no. of participants

Reflection

16/17

On-site service performance

12/17

Oral presentation/report

12/17

Group projects

11/17

Others • Written tests, 3/17 • Peer evaluation, 3/17 • Portfolio, 2/17 • Preparation, 1/17 • Attendance, 1/17 • Pre-/Post-survey, 1/17

4.4.2 Assessment Methods What assessment methods or components did participants use in their servicelearning courses? All participants used multiple assessment methods in their servicelearning courses. These ranged from three to 11 different types of individual or group assessment methods (see Table 4.3). Reflection—usually individual, written works—was an assessment component in almost all of the participants’ (16/17) courses. Apart from reflection, on-site service performance (12/17), oral presentations or reports (12/17), and group projects were also widely used (11/17). Other assessment methods mentioned by participants were written tests, peer evaluation, and portfolios.

4.4.3 Assessment Challenges The core findings of the study hinged around the question, What challenges or difficulties do faculty experience in formally assessing students in academic service-learning courses? Among assessment methods used by participants, assessing students’ service performance was the most reported area of challenge (10/17), followed by student-related concerns (6/10), workload or manpower (5/17), institutional guidelines on formal assessment (4/17), developing or applying rubrics (3/17), and working with multiple assessors (2/17). These items are inter-related and involve intricacies. In continuation, we discuss in detail each item summarized in Table 4.4 below.

4 Assessing Students in Academic Service-Learning … Table 4.4 Summary of assessment challenges experienced by service-learning faculty

4.4.3.1

57

Type/Area of challenge

Reported by no. of participants

Assessing students’ service performance (logistics, fairness, authenticity)

10/17

Student-related concerns (grade appeals, free riders)

6/10

Workload/manpower (large class size, insufficient instructors)

5/17

Institutional assessment guidelines (i.e., against grading leniency)

4/17

Rubrics (development or application to service-learning)

4/17

Multiple assessors (consensus, uniform standards)

2/17

Assessing Students’ Service Performance

Students’ on-site service performance and deliverables were important assessment components in the service-learning courses of most participants. In these, they invariably focused on criteria such as how students “applied” or “practiced” course content, the “design and quality of service” they delivered, and soft skills and attitude they manifested in the process of working with others and interacting with the community. One participant explained the importance and challenge of on-site observation of students’ service performance in the following manner, Despite the help of a collaborator, I feel I am not able to sufficiently observe service performance, which is important for students to demonstrate their attitude, knowledge, and skills. It is also important for the university and its partnership with the community. Besides, on-site observation helps to motivate students who may otherwise perform badly. It is so important in my course that anyone failing in service performance fails the whole course… I would like to spend more time on on-site observation, to know students’ concerns and performance, not just for grading. But performance needs to be observed over time for us to be able to judge students’ attitudes and knowledge. Unfortunately, students have different timetables, and I cannot go to the community center each time they serve.—T12

Granted the value of on-site observation through which faculty can assess students in action and interacting with the community, participants found it challenging for different reasons. According to the example above, logistics was a main source of difficulty. The matter was cited by at least eight participants whose courses involved different groups serving in different locations, or whose class sizes meant more group or individual service projects than faculty could supervise. Participants facing this difficulty claimed to do their best to visit all or most groups even for a limited time, or took turns with colleagues or other course collaborators to visit students in their service locations. A common strategy was to count on feedback or evaluation

58

R. M. Camus et al.

from community partners and service clients to assess students’ service performance. Some deployed this strategy to supplement direct observation by faculty, while others more constrained by manpower and time relied on it as substitute for direct observation by faculty. A less common measure mentioned by four participants was to “glean students’ performance from their reflections.” Logistical difficulties aside, on-site observation was also a challenge in terms of “fairness.” As three participants explained, there are many variables between group or individual services, be it in the type of service rendered, the service location, the standards of community partners helping to evaluate students, or the clients students happened to be assigned to when assessment took place. Such variables led some participants to feel uncertain about grading students’ services. In their words, Grading students on the field somewhat depends on luck, for example, on the clients the students get on the day they are assessed. It makes a big difference when the client whom the student needs to interview is over 90 years old or younger, a local or a non-local. —T3 I will not – cannot – apply on-site assessment in this course – it is (logistically) impossible and unfair. The students’ projects are very different from each other. —T5 On-site supervisors really vary, so (their assessment of students’ skills) may not be fair or really reflect students’ performances. Fair grading is indeed a challenge. —T12

Also worth mentioning is the issue of “authenticity” raised by one participant, who expressed the following qualm about on-site assessment: It seems strange, unnatural to assess service performance, I mean, students know they’re being watched. —T13

4.4.3.2

Student-related Concerns

Four participants reported assessment hurdles coming from students, such as complaints about grades received especially from written reflections. The following examples show this: Some students tend to complain about their reflective journal (grades). From my experience, it’s usually students who are content with moderate service performance who want to make up by trying to get higher marks from writing. —T2 Marks for written stuff get challenged by students, and I’m the one who has to mark all these... We need some guidelines for grade appeals. I can’t deny that journal-grading is also based on how the marker feels about their work. And sometimes I can’t tell free-riders. —T3 I get grade appeals – usually from the extremes of the scale. Those who got very low grades, and those who got high grades but want higher grades. —T13

The second quote mentions another student-related concern, namely, identifying and dealing with free riders in group projects. Another participant with the same concern used peer review to address the problem: (The challenge is) free-riders, because the course involves projects where students work as teams. Because of this, I ask students to do peer review, which I use as reference to evaluate their group work, not as assessment. —T16

4 Assessing Students in Academic Service-Learning …

4.4.3.3

59

Workload/Manpower

A heavy workload, closely correlated with lack of manpower (particularly when teaching assistants are not available), is known to be a common drawback of teaching service-learning and a major deterrent for new faculty to try service-learning (Abes et al., 2002; Banerjee & Hausafus, 2007; Darby & Newman, 2014). The issue surfaced too in this study where workload and manpower were explicitly named by five participants as a challenge in student assessment. Difficulties with workload and manpower were associated with large class sizes and scheduling on-site visits. For instance, We have 70 to 80 students divided into 12 groups. That makes in-class and on-site observation time and manpower consuming. —T6 There are 100 students and only two instructors for this course. It is impossible to see them all implementing their service—T7 We need more tutors to help with grading, especially on-site assessment. —T9 More manpower for assessment is needed. The challenging part is service performance. Students face difficulties on-site and need support. Over and above assessing them, we need manpower to work with students on-site and to debrief them about their work. —T12 I’m the only one teaching this subject so I cannot visit them (on-site). Hence, I cannot guarantee the kind of volunteer work they do, and I wonder about how real or valid their experiences were. I can only trust them and rely on their reflections about their experience. —T14

One participant explained an interesting reason behind manpower problem, that is, lack of understanding within the department about the heavy workload which service-learning courses entail: We are a team of four leading 200 students. We used to be five, but after some colleagues (without experience in service-learning) questioned the number of teachers involved in the subject, one teacher was removed from the team, so the workload of the rest increased! —T10

4.4.3.4

Other Challenges

Institutional assessment guidelines. There were other less cited but no less important challenges discussed by participants. Several (4/17) wrestled with institutional guidelines or expectations about formal assessment, in particular, those which guard against grading leniency. Such guidelines appeared problematic to some participants specifically when applied to their service-learning courses where they felt most students deserved high marks for their “hard work” or “good service,” as the two examples illustrate, (The guidelines) assume a certain percentage of students who do well or do less. But empathy wise, many deserve high marks… If all the students work very hard in their projects and are all-out in doing what the NGOs ask, why should we limit A’s and B’s to only 70% of the class? It can give the impression that we teachers don’t believe all students can perform well and demonstrate empathy. Also, if they fulfill all the criteria in the rubrics, a conflict arises

60

R. M. Camus et al. between the rubrics (telling us who deserve high marks) and the policy (limiting high marks to a few). —T3 In my service-learning subject, all students are screened and must pass an interview, which means they’re all good students! It’s difficult to give lower marks when all your students are excellent. Besides, how can I give C’s to students who did good service overseas? This is the challenge I need to solve each semester: justifying the number of A’s I give to students. —T4

The second example above involves an international service-learning course for which students were pre-selected. Since the course does not draw randomly from the general distribution of students whereas assessment guidelines assume a normal distribution of students, the participant experienced a conflict between following assessment guidelines and giving students grades they seemed to deserve. Both examples illustrate Brookhart’s (2013) claim that “teachers at all levels” tend to “mix effort and behavior into their achievement grades” (p. 269). McMillan (2019) attributes this phenomenon to teachers’ awareness that grading does not only record learning but also promotes learning and affects student motivation. Rubrics. As mentioned in a previous quote, developing suitable rubrics or knowing how to apply them in service-learning courses was another challenge, cited by at least four participants: Few of us service-learning teachers are trained in social work and find it challenging to grade “empathy.” I can create rubrics for the academic part, but for empathy… —T3 (Even with the rubrics) it’s hard to distinguish between A+ and A, between B+ and B… Besides, there are some issues that are important which go beyond the rubrics, like mistakes they make, or misbehavior outside service hours – should we take these into account in the evaluation? —T4 It’s hard to give a grade to students for the service they do. They all work hard and sacrifice their free time, so it’s hard to decide who gets A or B. So, I assume they all do their best in service delivery and just assess their reflection. –T11 I find it difficult to evaluate service performance – to assign a grade or follow a rubric for it. —T14

Some faculty members thus had difficulty with using rubrics, particularly in appraising a non-academic outcome like empathy or non-academic factor like misbehavior, likewise in accurately grading service work. Multiple assessors. Service-learning courses often require a team of instructors and may involve a heterogeneous group formally assessing students. For example, instructors and teaching assistants may collaborate in marking reflective journals; NGO supervisors and service clients may help evaluate and give feedback about student performance; peers or audience invited to exhibits or presentations of students’ works may take part in judging these. With different assessors involved, difficulties such as achieving consensus or applying the same criteria may arise. Two participants explained, Sometimes teachers may have different opinions and might be a little bit subjective. One teacher pays attention to one aspect and another one to another aspect. So, we take turns observing the different groups, in such a way that all the teachers get to see how each group performed in the course of a week. —T1

4 Assessing Students in Academic Service-Learning …

61

Assessing journals or performance is not that difficult. Grading with the same criteria is what’s challenging for me as subject leader. Anyway, the rubrics helps a lot in trying to keep the same standards. —T5

In fact, dealing with multiple assessors was a matter mentioned by more participants, though not necessarily in the context of assessment challenge. The two participants above relied on shared rubrics or shifts in grading students to achieve consistency and fairness in grading. Other participants found other ways. A common approach was to discuss and compare grades, another was to calculate the mean of scores given by different graders, still another to double-mark a random set of items to calibrate marking standards.

4.5 Discussion It is clear from qualitative data and findings furnished here that assessing students in service-learning courses presents faculty with challenges that do not have simple solutions. Some of these also arise in non-service learning courses, particularly those which involve group work, performance evaluation, or multiple assessors. Besides, participants expressed underlying concerns also often found in academic courses involving formal assessment. These include concerns about, fairness, making accurate judgments about demonstrated learning, and managing subjectivity—be it faculty subjectivity in deciding about students’ grades, or students’ subjectivity in appealing for grades which they think they deserve. Here we shall focus on areas of the findings which are intrinsically related to service-learning. We also bring up practical ideas from the data set and literature. The findings are encouraging on at least two counts. First, all participants used a combination of traditional and experiential assessment methods in their servicelearning courses. Service-learning being an experiential pedagogy that intends a variety of outcomes, it cannot but employ different methods to evaluate knowledge and skills, performance, attitude, and behavior. Second, reflection was an assessment component in almost all participants’ courses. This too is laudable given the wellacknowledged critical importance of reflection in service-learning. By reflecting on experience, students become more conscious of and responsible for their learning, allowing them to assimilate it more deeply and be better disposed toward positive changes from within and without. Participants’ use of mixed methods of assessment and wide use of reflection are in line with what is recommended in literature by service-learning educators and are positive findings from the study. On the other hand, on-site service performance was not assessed by a handful of participants, and on-site observation was the most reported area of challenge in service-learning assessment. The matter merits attention in light of service-learning’s educational and social objectives. Service-learning is an experiential pedagogy: students learn to apply knowledge to real-life situations and learn more from applying.

62

R. M. Camus et al.

Observing students on-site is an important means for faculty to supervise and evaluate students’ performance. What one source wrote about task assessment in higher education can well be said of service performance assessment, that it provides “useful learning experiences as well as information about a student’s level of competency… (and is) an integral part of instruction” (Linn, 1993, p. 13). Further, service-learning is also essentially civic education and promises to benefit not only student learners but also the clients and communities they purport to serve. Faculty supervision and evaluation of on-site performance ensures the quality of service rendered by students and guarantees that communities are not exploited. It is thus necessary to place due weight on course components and assessment methods directed toward actual service work. These are as important to student learning as reflective activities and have repercussions on the community. Recall that participants focused less on civic outcomes compared to discipline- or profession-specific and generic or soft skills outcomes. This piece of data together with findings about onsite observation suggest the need for faculty to review whether their courses are aligned with service-learning’s objectives and pay equal regard to stakeholders in the community. In view of the pedagogical and social significance of service rendered by students, we think it is important to guard against a simplistic interpretation of the idea that “grades should be based on learning, not service” (Bringle & Hatcher, 1995, p. 117; cf. Howard, 1993). Service is a crucial demonstration of student learning and should also be assessed in academic service-learning. In fact, most participants intend civic outcomes and realize the importance of onsite performance. However, they are challenged by matters such as large class sizes, having to travel to different locations, and a heavy workload. As one participant pointed out, gauging student performance and understanding what they are learning as they work and interact with the community cannot be achieved by one site visit only but requires observation through time; besides, faculty conducting on-site visits are not only there to assess but also to give students valuable support and feedback. Workload and manpower were together a major assessment challenge that emerged from the study and were in most cases linked to the unusual time and effort taken up by organizing and overseeing student services. Whether in teaching, supervising, or assessing, service-learning generally ask more from faculty than conventional courses. In this regard, assessment challenges presented here point to the dire need for departmental and institutional support for service-learning faculty. To mention forms of support suggested by participants themselves, increasing manpower (e.g., more instructors, teaching assistants, or peer mentors) could help improve supervision and assessment. Faculty work could also be facilitated through discretionary arrangements concerning class size and assessment guidelines which work better for conventional courses than academic service-learning. Given the uniqueness of the pedagogy which calls for different ways of teaching and assessing, challenges reported by participants further disclose a fundamental form of support that is needed: staff development for faculty involved in servicelearning. Such training can be provided by service-learning offices, by communities of practice, or, simply by more experienced colleagues, who can share “many ways

4 Assessing Students in Academic Service-Learning …

63

to conduct the supervision and assessment of students (and) present various ways they have dealt with these issues to illustrate choices for those planning courses” (Bringle & Hatcher, 1995, p. 118). On this note, colleagues from social and health disciplines—i.e., disciplines where community-based practicum is a well-established part of the curriculum—may have useful practices to share. Two participants from different health science disciplines did express confidence about assessing students’ service on-site. One commented that grade appeals were hardly an issue since “students already knew how they performed from structured debriefings (after scheduled services): what they achieved, what did not go well, why” (T5), while another explained that experiences in supervising “clinical placements” and conducting “competency tests” made service-learning assessment unproblematic (T8). Other participants also shared strategies for coping with assessment challenges, for instance, using student peer assessment to discourage free riding. Worth mentioning are flexible assessment methods adopted by participants in response to feedback from students. For instance, typical group-by-group presentations at the end of a course (which can be repetitive and lengthy) were turned by one into a “project showcase event,” another to a “running stations” presentation, a third to a “final exhibition.” Through these novel methods, students not only had the chance to explain their work but also received feedback about their work from peers and guests invited to the events. One participant stated the lesson well, My students complained about having too many presentations, so I changed the format of the final presentation to an exhibition – it turned out very good, students loved it! It is good for teachers to give students space and adopt innovative formats. —T16.

An essential part of most strategies was good communication, for example, with NGO supervisors or service clients, to explain and provide examples of how to evaluate students’ performance; with students making grade appeals, to acknowledge first where they did well before pointing out areas of deficiency; with colleagues, to have a consensus about the rubrics and convene about disparities in grading. To end, we are hardly able to offer definite solutions to assessment challenges here. Nevertheless, a couple of insights by educators about assessment may serve as overarching suggestions. Speaking from a health discipline, Schuwirth and van der Vleuten (2006) address assessment challenges from the root by voicing the need to create or promote a “positive culture of assessment.” They write, “the major challenge is to change the culture of assessment into one where assessment is informative, helps people to improve their work, and where the goal is not to be better than others but to be better today than you were yesterday” (pp. 545–546). Meanwhile, noting how varying grading practices reflect teachers’ educational beliefs and values, McMillan (2019) invites teachers “to fully understand their reasons for using different factors in grading… by reflecting on the purposes of grading” (p. 13). Clearer ideas about the purpose of assessment can help dispel misconceptions about assessing students in service-learning.

64

R. M. Camus et al.

4.6 Conclusion The foregoing study is only exploratory. It is based on a small group sample and although multiple institutions are involved, all participants and their experiences are from the limited context of Hong Kong. Readers would have to bear this in mind when evaluating the transferability of the study and its implications. Certainly, more investigation about service-learning assessment is needed, and studies with larger group samples, different foci and contexts could place findings and ideas presented here in better perspective. The topic of this study is not an easy one. Simpson (1996) rightly remarked that research about assessment in higher education is the type where “there are more questions than solutions to challenges” (p. 70). Steinke and Fitch (2007) intimated that assessment in service-learning may shed light on assessment in higher education as a whole. In the heart of assessment challenges glimpsed from this study is the charge laid upon service-learning faculty to assist students in integrating academic learning with useful service to the community, likewise to promote graduate attributes that prepare students for the real world. Graduate attributes such as being reflective, ethical practitioners, problem-solvers, team players, effective communicators, and civic-minded members of society are best honed through experiential pedagogies like service-learning, but they are also hardest to assess and often “require a radical adjustment on the part of academics… to move from didactic to facilitated experiential teaching and learning approaches” (Quinn & Shurville, 2009, p. 336). Applying knowledge, skills, and abilities from the academia to the real world and becoming engaged, responsible members of society are crucial processes for undergraduates regardless of discipline. They are vital goals of higher education which are patently addressed by academic service-learning and to which different disciplines can contribute. In this light, the experiences and concerns related by the participants of this study may find resonance too in other academic courses and among non-service-learning faculties.

4 Assessing Students in Academic Service-Learning …

65

Appendix 1: Pre-Interview Questionnaire

Subject name/Course code that you are involved with:

.

How long have you been teaching /involved in assessing academic Service-Learning subject(s)? < 1 year

1–3 years

4–6 years

7–9 years

>10 years

Which type of academic Service-Learning subject have you taught/are you teaching? Open for all General University Requirements (GUR) DSR (Discipline-Specific Requirements) subject

Which statement best describes the Service-Learning subject you taught/are teaching? Intentionally designed Service -Learning subject Existing subject with added Service -Learning component Others: Please specify

.

Which type of service-learning is utilized in your academic Service-Learning course/subject? Direct service Indirect service Advocacy Community-based research

What is the enrollment of your Service-Learning subject in the last semester (estimate)? .

Appendix 2: Interview Questions Background 1. 2.

Please provide a brief background of the Service-Learning (SL) subject that you are involved in. What is the SL component? Learning Outcomes

3.

What are the major learning outcomes that you would like to achieve through your SL subject/course?

66

R. M. Camus et al.

4.

What are the major learning outcomes from your SL subject/course that you assess? Assessment and Grading

5. 6. 7. 8. 9.

10.

How do you assess students in your SL program or course? What assessment methods or tasks do you use to assess those outcomes? What are the criteria and standard you use to grade students’ performance in those tasks? (e.g., service product(s), feedback from community partner(s)? What are the weightings of the methods or tasks? What percentage of your assessment is group-based? What percentage of it is individual-based? How do you assign individual grades for group-based service projects? For each of the assessment methods, could you provide more details answering the following questions: • Do you assess individual student’s performance and learning in your SL program or course? Why or why not? • Is the course grade a Pass/Fail grade or a letter grade (e.g., A to F)? • Do you assess students’ performance in service delivery/execution? Why? How? • Do you assess students’ reflections on their SL experience? Why? How? Format (journals, discussion, blog …)? • How much/which part of it is graded and ungraded, respectively?

11.

How many teachers are involved in the grading of the students?

Application 12.

Recall a recent student who received an A grade or had an outstanding performance in your SL program or course. • What made you grade the student as outstanding? • Why did you think that the student deserved an A grade?

13.

Recall a recent student who received a very low grade or had a poor performance in your SL program or course.

Challenges 14. 15.

What made you grade the students’ performance as poor? What help or support would you need to improve your assessment practices relating to SL?

Others 16.

We would also like to collect some artifacts pertaining to your gradings and assessment. Thus, we would like to request for samples of your SL subject course: • Course assignment and guidelines for students

4 Assessing Students in Academic Service-Learning …

67

• Rubrics • Any other samples related to grading and assessment that you would like to share with us

References Abes, E. S., Golden J., & Jones, S. R. (2002). Factors that motivate and deter faculty use of servicelearning. Michigan Journal of Community Service Learning, 9(Fall), 5–17. http://hdl.handle.net/ 2027/spo.3239521.0009.101 SL Ash PH Clayton 2004 The articulated learning: An approach to guided reflection and assessment Innovative Higher Education 29 137 154 https://doi.org/10.1023/B:IHIE.0000048795.84634.4a Ash, S. L., & Clayton, P. H. (2009). Generating, deepening, and documenting learning: The power of critical reflection for applied learning. Journal of Applied Learning in Higher Education, 1(1), 25–48. http://hdl.handle.net/1805/4579 Ash, S. L., Clayton, P. H., & Atkinson, M. P. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service Learning, 11(2), 49–60. http://hdl.handle.net/2027/spo.3239521.0011.204 Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service learning affects students (Executive Summary). Higher Education Research Institute. University of California, Los Angeles. http://digitalcommons.unomaha.edu/slcehighered/144 Banerjee, M., & Hausafus, C. O. (2007). Faculty use of service-learning: Perceptions, motivations, and impediments for the human sciences. Michigan Journal of Community Service Learning, 14(1)(Fall), 32–45. Biggs, J. B. (2003). Aligning teaching for constructing learning. The Higher Education Academy. Retrieved on July 7, 2021, from https://www.heacademy.ac.uk/sites/default/files/resources/ id477_aligning_teaching_for_constructing_learning.pdf A Biglan 1973 The characteristics of subject matter in academic areas Journal of Applied Psychology 57 3 195 203 RG Bringle JA Hatcher 1995 A service-learning curriculum for faculty Michigan Journal of Community Service Learning 2 1 112 122 Bringle, R. G., & Hatcher, J. A. (1999). Reflection in service learning: Making meaning of experience. Educational Horizons (Summer), 179–185. RG Bringle E Wall 2020 Civic-minded graduate: Additional evidence Michigan Journal of Community Service Learning 26 1 1 17 https://doi.org/10.3998/mjcsloa.3239521.0026.101 SM Brookhart 2013 Grading JH McMillan Eds Sage Handbook of Research on Classroom Assessment Sage 257 271 DW Butin 2003 Of what use is it? Multiple conceptualizations of service learning within education Teachers College Record 105 9 1674 1692 https://doi.org/10.1057/9781403981042_6 RM Camus G Ngai KP Kwan JHY Yau S Chan 2021 Knowing where we stand: Mapping teachers’ conception of reflection in service-learning Innovative Higher Education 46 285 302 https://doi. org/10.1007/s10755-020-09534-6 N Champagne 2013 Using the NHEC areas of responsibility to assess service learning outcomes in undergraduate health education students American Journal of Health Education 37 3 137 145 Chan, C. K. (2012a). Assessment for community service types of experiential learning in the engineering discipline. European Journal of Engineering Education, 37(1), 29–28. https://doi.org/ 10.1080/03043797.2011.644763 Chan, C. K. (2012b). Exploring an experiential learning project through Kolb’s Learning Theory using a qualitative research method. European Journal of Engineering Education, 37(4), 405–415. https://doi.org/10.1080/03043797.2012.706596

68

R. M. Camus et al.

SCF Chan G Ngai KP Kwan 2019 Mandatory service learning at university: Do less-inclined students learn from it? Active Learning in Higher Education 20 3 189 202 https://doi.org/10. 1177/1469787417742019 SCF Chan G Ngai CHY Lam KP Kwan 2021 How participation affects university students’ perspectives toward mandatory service-learning Journal of Experiential Education 44 2 137 151 https:// doi.org/10.1177/1469787417742019 SK Cibangu M Hepworth 2016 The uses of phenomenology and phenomenography: A critical review Library and Information Science Research 38 2 148 160 JM Conway EL Amel DP Gerwien 2009 Teaching and learning in the social context: A metaanalysis of service learning’s effects on academic, personal, social, and citizenship outcomes Teaching of Psychology 36 4 233 245 https://doi.org/10.1080/00986280903172969 Cowan, J. (2005). Designing assessment to enhance student learning. The Higher Education Academy.http://www.heacademy.ac.uk/assets/ps/documents/practice_guides/practice_guides/ ps0069_designing_assessment_to_improve_physical_sciences_learning_march_2009 J Creswell 2013 Qualitative inquiry and research design 3 Sage A Darby G Newman 2014 Exploring faculty members’ motivation and persistence in academic service-learning pedagogy Journal of Higher Education Outreach and Engagement 18 2 91 119 DM Doberneck JH Schweitzer 2017 Disciplinary variations in publicly engaged scholarship: An analysis using the Biglan classification of academic disciplines Journal of Higher Education Outreach and Engagement 21 78 103 Eyler, J. (2000). What do we most need to know about the impact of service-learning on student learning? Michigan Journal of Community Service-Learning (Fall), 11–17. http://hdl.handle.net/ 2027/spo.3239521.spec.102 J Eyler 2002 Reflection: Linking service and learning—Linking students and communities Journal of Social Issues 58 3 517 534 https://doi.org/10.1111/1540-4560.00274 Eyler, J., & Giles, D. E. (1999). Where’s the learning in service-learning? Jossey-Bass. http://hdl. handle.net/2027/spo.3239521.0006.115 P Felten PH Clayton 2011 Service-learning New Directions for Teaching and Learning 128 75 84 https://doi.org/10.1002/tl.470 H Gardner 1993 Multiple intelligences: The theory in practice Basic Books Howard, J. (1993). Community service learning in the curriculum. In J. Howard (Ed.), Praxis I: A faculty casebook on community service-learning (pp. 3–12). Office of Community Service Learning. NT Kelseyak M Simmer-Beck MM Bray CC Gadbury-Amyot 2007 Evaluation of an academic service-learning course on special needs patients for dental hygiene students: A qualitative study Journal of Dental Education 71 3 378 392 Kezar, A. (2002). Assessing community service learning. About Campus (May-June), 14–20. SC Lai CP Hui 2021 Service-learning: Impacts of learning motivation and learning experience on extended social/civic engagement Higher Education Research & Development 40 2 400 415 https://doi.org/10.1080/07294360.2020.1756748 K Lambright 2019 Lessons outside of the classroom: Examining the effectiveness of service learning projects at achieving learning outcomes Journal of Public Affairs Education 14 2 205 217 RL Linn 1993 Educational assessment: Expanded expectations and challenges Educational Evaluation and Policy Analysis 15 1 1 16 EA McCrea 2010 Integrating service-learning into an introduction to entrepreneurship course Journal of Management Education 34 1 39 61 McMillan, J. H. (2019). Surveys of teachers’ grading practices and perceptions. In T. R. Guskey & S. M. Brookhart (Eds.), What we know about grading. Association for Supervision and Curriculum Development. Retrieved on July 7, 2021, from http://www.ascd.org/Publications/Books/Ove rview/What-We-Know-About-Grading.aspx SB Merriam EJ Tisdell 2016 Qualitative research: A guide to design and implementation 4 Jossey Bass

4 Assessing Students in Academic Service-Learning …

69

LM Molee ME Henry VI Sessa ER McKinney-Prupis 2010 Assessing learning in service-learning courses through critical reflection Journal of Experiential Education 33 3 239 257 https://doi.org/ 10.1177/105382590113300304 Oates, K. K., & Leavitt, L. H. (2003). Service-learning and learning communities: Tools for integration and assessment. Association of American Colleges and Universities. Quinn, D., & Shurville, S. (2009). From little things big things grow: Scaling-up assessment of experiential learning. Campus-Wide Information Systems, 26(5), 329–344. P Ramsden 1992 Learning to teach in higher education Routledge P Rennert-Ariev 2005 A theoretical model for the authentic assessment of teaching Practical Assessment, Research, and Evaluation 10 2 1 11 https://doi.org/10.7275/a7h7-4111 DL Rubin PH Matthews 2013 Journal of Higher Education Outreach and Engagement 17 2 67 86 Schuwirth, L. W. T., & van der Vleuten, C. P. M. (2006). Challenges for educationists. Retrieved July 7, 2021, from bmj.com, https://doi.org/10.1136/bmj.38952.701875.94 DTL Shek VMY Tang XY Han 2005 Evaluation of evaluation studies using qualitative research methods in the social work literature (1990–2003): Evidence that constitutes a wake-up call Research on Social Work Practice 15 3 180 194 https://doi.org/10.1177/1049731504271603 Shek, D.T.L., Ngai, G., & Chan, S.C.F. (Eds.). (2019). Service-learning for youth leadership. Springer Nature. DTL Shek CMS Ma Z Yang 2020 Transformation and development of university students through service-learning: A corporate-community-university partnership initiative in Hong Kong (Project WeCan) Applied Research in Quality of Life 15 5 1375 1393 https://doi.org/10.1007/s11482-01909738-9 LS Shulman 1998 Theory, practice, and the education of professionals The Elementary School Journal (special Issue) 98 5 511 526 Simons, L., & Cleary, B. (2010). The influence of service learning on students’ personal and social development. College Teaching, 54(4), 307–319. https://doi.org/10.3200/CTCH.54.4.307-319 R Simpson 1996 The challenging task of assessment in higher education Innovative Higher Education 21 2 69 72 S Sin 2010 Considerations of quality in phenomenographic research International Journal of Qualitative Methods 9 4 305 319 https://doi.org/10.1177/160940691000900401 P Steinke P Fitch 2007 Assessing service-learning Research & Practice in Assessment 2 Summer 24 29

Chapter 5

How Hong Kong University Teachers Assess Students in Service-Learning: An Exploratory Study Grace Ngai, Ka Hing Lau, Kam Por Kwan, Stephen C. F. Chan, and Daniel T. L. Shek Abstract Service-learning is known to be a high-impact educational practice, and academic service-learning in particular has benefits of facilitating student learning through linking the academic discipline with social issues, and allowing students to practise and apply academic knowledge and skills in serving the community, with the ultimate purpose of promoting the well-being of the service recipients and the community. For student learning, and for institutional measures, this learning needs to be assessed just like any other academic subject. Despite academic service-learning being widely deployed in universities in Hong Kong and worldwide, there have not been a lot of studies on how teachers assess service-learning, or on the challenges and needs that become apparent during the process. This study attempts to fill in the research gap through a multi-institutional study on this topic. Keywords Academic Service-Learning · Student Assessment · Survey Study · Teacher Practices · Criteria · Outcomes

G. Ngai (B) · K. H. Lau · K. P. Kwan · S. C. F. Chan Service-Learning and Leadership Office, Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] K. H. Lau e-mail: [email protected] K. P. Kwan e-mail: [email protected] S. C. F. Chan e-mail: [email protected] D. T. L. Shek Department of Applied Social Sciences, Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_5

71

72

G. Ngai et al.

5.1 Introduction Academic service-learning, defined as a “course-based, credit-bearing educational experience in which students (a) participate in an organized service activity that meets identified community needs and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of civic responsibility” (Bringle & Hatcher, 1995, p. 112), is popular in higher education worldwide (Shumer et al., 2017). In the Asia–Pacific region, there has been rapid development in service-learning during the past two decades (Ma, 2018; Snell & Lau, 2020; Xing & Ma, 2010). In Hong Kong, most of the higher educational institutions have incorporated service-learning as part of their academic curriculum in some way (Lau & Snell, 2020; Shek et al., 2015). Since academic service-learning awards students with credits for achieving the intended learning outcomes, this implies a necessity to assess students. This is, however, not a straightforward task. Fundamentally, service-learning focuses on very different aspects of student development, as compared to the traditional disciplinebased courses. Most academic courses have learning outcomes that focus on students’ cognitive, intellectual, or skills-based development. In contrast, service-learning has intended learning outcomes which aim to develop students’ self-awareness and understanding (Eyler & Giles, 1999), empathy, and social responsibility (Bringle & Hatcher, 2009; Einfeld & Collins, 2008; Lundy, 2007; Prentice, 2007; Richard et al., 2016; Simons & Cleary, 2006; Snell et al., 2015; Warren, 2012). While conventional assessment methods such as multiple-choice questions, problem sets, quizzes, exams, or short essays have been demonstrated to be able to capture student performance on cognitive and intellectual development, they are not necessarily appropriate for assessing students’ achievement of the more affective outcomes from servicelearning, including promotion of the well-being of the service providers (i.e. university students) and service recipients. Using conventional methods to assess servicelearning may cause mismatch between goals, pedagogy, assessment, and outcomes (Kezar, 2002). Service-learning courses or programmes thus may need different assessment methods that accurately assess students’ achievement of the intended outcomes from the experience. Assessment is important for student learning, and it is known that misalignment between assessment and learning outcomes will result in negative impact to student learning (Kennedy et al., 2006). From the students’ perspective, assessment is the curriculum, as students will learn what they think will be assessed (Ramsden, 2003). For faculty members, assessment provides “evidence … to support their belief that service-learning has a rightful place in the academy” (Strouse, 2003, p. 78). From the institutional perspective, for academic service-learning courses or programmes that bear academic credits, establishing reliable and accurate assessment methods to grade students, either on pass/fail or a letter/numeric grade basis, is of a primary concern to administrators to ensure parity (such as avoiding grade inflation and leniency) and academic rigour, and to justify resource allocation. The further development and integration of service-learning into the academic curriculum, therefore, relies in part

5 How Hong Kong University Teachers Assess Students …

73

on whether the service-learning community can assess student learning outcomes in a fair, consistent, and valid manner. Assessing learning outcomes is regarded as one of the most difficult parts of experiential learning (Quinn & Shurville, 2009). Service-learning, being a type of experiential learning, is similarly difficult to assess (Chan, 2012). Therefore, it is important to research and study how to grade students properly in service-learning. This chapter reports a study that takes a first step in this direction by looking into Hong Kong university teachers’ actual practice of assessing students in service-learning.

5.2 Literature Review There have been a lot of studies on students’ learning outcomes arising from servicelearning (e.g. Astin et al., 2006; Celio et al., 2011; Driscoll et al., 1996; Eyler et al., 2001; Yorio & Ye, 2012). However, most of them focus on evaluating the impact or effectiveness of service-learning courses or programmes on students’ learning outcomes. The evaluation methodology often rely on either self-reporting questionnaires by students (e.g. Ma et al., 2019; Ngai et al., 2018; Snell & Lau, 2020; Toncar et al., 2006; Yu et al., 2018) or focus groups and interviews with students, community partners, and instructors after the completion of service-learning projects (Shek & Chan, 2013; Snell et al., 2015, 2019; Wong et al., 2021). This is clearly a different matter from assessing or grading individual students in their attainment of the intended learning outcomes, as evidenced through their performance or course deliverables. There exist some studies that examine the appropriate assessment methods for service-learning. Ash and Clayton (2004) proposed the “Articulated Learning Model” to guide and assess reflection in service-learning to facilitate student learning, and to produce a final grade that reflects the extent to which the student has attained the intended learning outcomes of the course or programme. Ash et al. (2005) proposed the DEAL model for critical reflection which integrates reflection and assessment to capture student learning in terms of critical thinking and reasoning about course concepts. Some other examples include Molee et al. (2011) who found assessment rubrics useful for training and evaluating student reflection based on the DEAL model. Schmidt and Brown (2016) also introduced reflection as an assessment method and identified its best practices, as well as advocating the use of rubrics in grading reflection. There has also been work in applying rubrics into other aspects of assessment in service-learning. Menéndez-Varela and Gregori-Giralt (2016) integrated rubrics in different components of a service-learning project in a conservation-restoration and design course, including the project proposal, documentation, and service delivery, with different criteria and performance levels set out for each component. They involved multiple assessors in assessing different stages of the service-learning project, and used multivariate statistical analyses to analyse the score data. Their results show that rubrics-based performance assessment is conducive to student

74

G. Ngai et al.

learning. They further argue that the role of rubrics should be extended to that of an instructional resource that conveys expectations about intended learning outcomes and assignments to students, instead of merely that of an assessment tool. Misyak et al. (2016) utilised a structured, multi-faceted assessment system in a community nutrition service-learning course, with carefully designed student assignments and multiple feedback points to promote reflection and learning. Their assessment methods include in-class discussion, reflective assignment, needs assessment plan, and poster presentation. Rubrics were developed to indicate different levels of performance criteria and community partners involved in assessment. They also illustrate how various assessment methods could be utilised to meet different instructors’ needs in different subjects. Apart from the use of reflection and rubrics, other researchers also offer alternative assessment methods in grading service-learning. Peterson (2004) described a method for assessing problem-based service-learning projects via service reports, teamwork, and other contributions, mostly in soft skills and problem-solving. The described assessment approach covers several critical aspects in assessing problembased service-learning projects; but the examples are mostly derived from his own experience in one type of service-learning project. McDonald (2012) reviewed the current circumstances of assessing servicelearning and raised several operational questions in assessment, including: 1. 2. 3. 4.

5. 6.

How can various intended learning outcomes, particularly the more affective outcomes of social and personal development, be assessed? When and where should the assessment take place in the service-learning process? How can the assessment identify individual contribution in teamwork settings? How should the assessment be designed to cater for the complexities of servicelearning that students would encounter in the field, such as utilising multiple skills to solve real-life problems? How should student performance be assessed in the context of a real-life setting, where there are often no absolute answers or solutions? How can fairness in assessment be ensured when students encounter different service contexts and experience different challenges?

These questions point to the need to clarify the key elements in service-learning assessment, including the intended learning outcomes to be assessed, the assessment methods to be employed, as well as the criteria and standards to be adopted in assessment. Chan (2012) summarised several common approaches used by service-learning teachers in assessing student performance in their service-learning courses, including: reflective journal, direct observation, presentation, and oral assessment, and explores students’ and teachers’ perceptions towards different assessment approaches through a review of an assessment framework of one service-learning project. The findings are insightful, but it is difficult to generalise given the limited coverage.

5 How Hong Kong University Teachers Assess Students …

75

5.3 Study Objectives The objective of the current study is to identify the assessment methods currently in use among Hong Kong university teachers for assessing students in academic servicelearning, as well as their perceptions and criteria when using different methods. We anticipate that this inventory taking exercise may be able to serve as a foundation for future research. Our research questions are: 1. 2. 3. 4.

5.

What are the important student learning outcomes that university teachers in Hong Kong aim to achieve in their service-learning courses or programmes? What learning outcomes are most commonly assessed by these teachers, and what are the most common assessment methods they use to assess each of those? What criteria and standards do they use to determine the grade when assessing students in their service-learning courses or programmes? How self-efficacious are the teachers in assessing students fairly and accurately in their service-learning courses or programmes? What kinds of support do they need in assessment? To what extent are teachers’ assessment beliefs and practices related to their demographic characteristics: i.e. years of teaching experience (in general and in service-learning), disciplinary background; and the features of their service-learning courses or programmes: i.e. course type (general education vs. major-specific), and course nature (courses or programmes designed specifically for service-learning vs. existing courses or programmes converted to service-learning)?

5.4 Method 5.4.1 Study Design Our study employed a survey research design (Rea & Parker, 2014), which is commonly used to assess needs and establish baselines for future comparisons with quantitative descriptions (Isaac & Michael, 1995; Kraemer, 1991). It has several merits, including its ability to collect information from large numbers of respondents, lower cost in development and administration, and ease in describing the sample demographic composition and generalising from the results (Glasow, 2005).

5.4.2 Target Participants We targeted teachers who have been engaged in teaching and assessing servicelearning in three public universities in Hong Kong. The three universities were selected because their service-learning curriculum has been well institutionalised

76

G. Ngai et al.

Table 5.1 Key Features of the Three Participating Universities in the Study University A

B

C

Type of university

Public comprehensive, Public liberal arts Public liberal arts application-oriented

No of undergraduate students

Around 15,000

Around 7,000

Around 2,600

No of teachers engaged in service-learning

106

40

34

Compulsory or optional

Graduation requirement

Optional

Graduation requirement

Course-based

Yes

Yes

Yes

Credit bearing

Yes

S-L design

Yes

Yes

Minimum service hours 40 for endorsed SL courses

10

30

Centralised service-learning office

Yes

Yes

Yes

and integrated into their undergraduate programmes. Moreover, all of them provide systematic support, training, and coordination service to service-learning faculty members and students through a centralised office. Table 5.1 describes the current situation of service-learning and the number of target participants invited from the three universities. We invited 106, 40, and 34 teachers from University A, B, and C respectively, yielding a total of 180 individuals. At University A and C, servicelearning is a graduation requirement for their undergraduate programmes, whereas University B has an institutionalised service-learning programme that has been rapidly expanding in the past few years. University A and B are relatively large-scale in number of students; whereas University C has a longer history of implementing service-learning.

5.4.3 Instruments The survey instrument was designed by the research team, guided by pilot interviews with nineteen service-learning teachers from six universities in Hong Kong and insights from the literature review described in the previous section. The resultant questionnaire contains, among other things, the following five sets of questions. Participants were asked to respond to all items, and base their answers on their experience of the service-learning course or programme that they were most involved in.

5 How Hong Kong University Teachers Assess Students …

77

1. Demographics and Course or Programme Information The first set consists of items for collecting respondents’ demographic information, including gender, age, teaching institution, disciplinary background, years of teaching experience, and years of experience of teaching and assessing servicelearning. It also collects relevant information of the service-learning course or programme on which their responses were based, including: course type (general education course vs major-related course), course nature (specifically designed course for service-learning vs converted from existing course), service types (direct service, indirect service, advocacy, or community-based research), hours of direct service or interaction with the clients, class size, and whether the course is credit bearing or non-credit bearing. 2. Important Learning Outcomes for Service-Learning The second set asks respondents to rate, on a five-point scale (1 as not important; 3 as somewhat important; 5 as very important), the level of importance of the following eight learning outcomes to their service-learning courses or programmes: (a) deeper understanding of academic concepts and theories, (b) applying knowledge to deal with real-life complex issues, (c) problem-solving and critical thinking skills, (d) teamwork and effective communication skills, (e) understanding community problems and needs, (f) becoming a more responsible member of the community, (g) better understanding of personal strengths and weaknesses, and (h) understanding other people’s perspectives. 3. Assessment Methods for Different Service-Learning Outcomes The third set asks respondents to indicate, first on a “yes/no” basis, if each of the above eight intended learning outcomes were assessed in their respective servicelearning courses or programmes and, if yes, the types of assessment methods used. Respondents were allowed to choose multiple responses from the following list: (a) tests and exams, (b) academic term paper, (c) service project proposal, (d) on-site performance rated by instructor, (e) community partner evaluation/feedback, (f) peer assessment, (g) student self-assessment, (h) presentation, (i) service report, (j) service deliverable, and (k) reflective essay/journal/blog. 4. Assessment Criteria and Standards The fourth set asks respondents to rate, on a five-point scale (1 as not important; 3 as somewhat important; 5 as very important), the importance of ten factors in determining the grade they give in their service-learning courses or programmes, including: (a) demonstrating understanding of the subject matter, (b) amount of effort put into the service, (c) punctuality and attendance, (d) ability to deal with uncertainty during the service, (e) enthusiasm and motivation to serve, (f) demonstrating awareness of own strengths and weaknesses and how to improve, (g) responsibility in the service project, (h) caring attitude for others, (i) ability to apply class learning into real-life situations, and (j) ability to examine issues from multiple perspectives.

78

G. Ngai et al.

5. Teachers’ Efficacy, Challenges, and Needs for Support The fifth set asks respondents to rate, on a five-point scale (1 as strongly disagree; 3 as neutral; 5 as strongly agree), their level of efficacy, challenges, and needs for support in assessing service-learning: Self-efficacy. This includes four items, as follows: (a) I can assess my servicelearning students fairly and accurately, (b) I find it difficult to assess soft skills such as communication and teamwork, (c) I can justify my grading if my students were to question the grades they receive, and (d) I find grading service-learning courses/programmes harder than grading other courses or programmes. Challenges. This includes two items: (a) Grading service-learning courses/programmes take more time and effort than other subjects, and (b) There is too much variation in the criteria and standards across different service-learning courses or programmes. Needs for Support. This includes two items: (a) More training should be provided to staff on how to grade service-learning courses or programmes, and (b) I need more manpower for grading and assessment in my service-learning course/programme.

5.5 Procedures The survey was administered online to the targeted participants between March and June 2020. They were invited to participate via three rounds of emails, which included an information sheet explaining the purpose of the study. Respondents were assured that their information and responses would be kept confidential and used solely for research purposes. Non-respondents were followed up twice by reminder emails to boost the response rate. Access to the survey was controlled via a unique token generated for each participant to prevent unauthorised access or multiple submissions. To incentivise participation, each participant received a HKD100 coffee ecoupon by email upon completion of the survey. The study was given approval by the University’s Institutional Review Board.

5.5.1 Data Analysis SPSS version 26.0 was employed to perform data analysis. Descriptive statistics were computed to show the means, standard deviations, and frequency distribution of the responses. Independent t-tests or One-way ANOVA were conducted to examine the impact of teacher characteristics (years of teaching experience, years of experience in teaching and assessing service-learning, and disciplinary background) and course features (general education vs. major-related courses, and new courses specifically designed for service-learning vs. existing courses converted to service-learning) on

5 How Hong Kong University Teachers Assess Students …

79

respondents’ perception of the importance of different intended learning outcomes to service-learning, the importance of different criteria and standards, as well as their self-efficacy, challenges, and needs for support in assessing students in servicelearning. As advocated by past researchers (e.g. Sullivan & Feinn, 2012), effect sizes (Cohen’s d) were estimated to reveal the magnitude of the differences found between groups (Cohen, 1990). (Kotrlik & Williams, 2003) suggested that d = 0.2, 0.5, and 0.8 correspond to small, medium, and large effects. The likelihood of the intended learning outcomes being assessed by teachers from different backgrounds or across different course features were compared by chisquare test to reveal any statistically significant differences across groups. Cramér’s V was computed to show the effect size as weak: V ∈ [0.1,0.2); moderate: V ∈∈ [0.2,0.4); relatively strong: V ∈∈ [0.4,0.6), or strong: V ≥ 0.6 (Kotrlik & Williams, 2003).

5.6 Results and Discussion 5.6.1 Respondents and Response Rates Of the 180 university teachers invited to take part in the survey, 124 eventually submitted their responses, achieving an overall response rate of 68.9%. The response rates were 78.3% (n = 83), 55.0% (n = 22), and 55.9% (n = 19) for University A, B, and C respectively, with a male/female split of 54.8%/44.8% (one respondent (0.8%) did not disclose gender). The respondents are mainly aged between 31 and 60 years old, and the majority (66.6%) came from University A. Most (66.9%) of the respondents came from a background in the soft sciences and humanities. Over 40% of them had 10 years or more teaching experience, one-third had between one to four years of experience teaching service-learning, and another one-third had between four to seven years. Table 5.2 presents the detailed demographic data. We next considered the characteristics of the service-learning courses or programmes on which the responses were based. The majority (66.1%) were general education courses or programmes open to all students. Around half (51.6%) were specifically designed to be service-learning courses or programmes and almost all (96.8%) were credit bearing. In terms of class size, over 40% of them had an enrolment between 21 and 40, and almost 15% had more than 100 students. Direct service projects were involved in almost all (91.9%) of the courses or programmes, with about half (51.6%) exclusively involving direct service projects. The duration of the service mainly fell between 11–30 h and 31–40 h, with each interval taking up roughly a third of the distribution. Table 5.3 presents the details of the service-learning courses or programmes.

80

G. Ngai et al.

Table 5.2 Respondents’ Demographics Frequency

Percentage

Gender Male

68

54.8

Female

55

44.4

Prefer not to disclose

1

0.8

Age 20–30

6

4.8

31–40

32

25.8

41–50

33

26.6

51–60

40

32.3

Above 60

8

6.5

Prefer not to disclose

5

4.0

A

83

66.9

Institution B

22

17.7

C

19

15.3

Disciplinary Background Soft sciences Arts and humanities

28

22.6

Architecture and design

3

2.4

Business

14

11.3

Education

5

4.0

Social sciences

33

26.6

Hard sciences Engineering

15

12.1

Medical and healthcare

18

14.5

Science

7

5.6

Others

1

0.8

Year of Teaching Experience Less than 1 year

7

5.6

1 to less than 4 years

18

14.5

4 to less than 7 years

28

22.6

7 to less than 10 years

16

12.9

10 years or more

55

44.4

Year of Experience in Teaching and Assessing Service-Learning Less than 1 year

20

16.1

1 to less than 4 years

41

33.1 (continued)

5 How Hong Kong University Teachers Assess Students …

81

Table 5.2 (continued) 4 to less than 7 years

Frequency

Percentage

45

36.3

7 to less than 10 years

8

6.5

10 years or more

10

8.1

5.6.2 What Student Outcomes Did Teachers See as Important to Service-Learning? Table 5.4 depicts the respondents’ ratings on the importance of various intended learning outcomes to, and whether the outcomes were assessed in, their servicelearning courses or programmes. Overall, all of the listed intended learning outcomes were rated as important (mean value over 3) to the respondents. The three highest rated outcomes include “teamwork and effective communication skills” (Mean = 4.53 on a five-point scale; 62.9% choosing “5 (very important)”); “applying knowledge to deal with real-life complex issues” (4.47, 59.7%), “understanding community problems and needs” (4.43, 58.9%), and “problem-solving and critical thinking skills” (4.43, 55.6%). The three lowest rated items were “understanding other people’s perspectives” (4.39, 51.6%), “better understanding of personal strengths and weaknesses” (4.06, 36.3%), and “deeper understanding of academic concepts and theories” (3.86, 32.3%). Breakdown analyses according to respondents’ backgrounds revealed no statistically significant differences in the mean ratings of the outcomes across years of teaching experience and years of experience of teaching and assessing servicelearning. On the other hand, respondents from a soft sciences or humanities background tended to rate the importance of “deeper understanding of academic concepts and theories” higher than those from a hard sciences background (p = 0.04; effect size [Cohen’s d] = −0.40). Comparisons of respondents’ ratings across different course features show that teachers of general education-type courses open to all students generally place a lower importance on “deeper understanding of academic concepts and theories” (p = 0.06; d = 0.37) but more importance on “becoming a more responsible member of the community” (p = 0.01; d = −0.51) and “understanding other people’s perspectives” (p = 0.06; d = −0.37) than teachers involved in discipline-related courses or programmes. Likewise, teachers of courses or programmes specifically designed for service-learning tend to give higher importance to “becoming a more responsible member of the community” (p = 0.04; d = −0.37) and “better understanding of personal strengths and weaknesses” (p = 0.05; d = −0.36), and lower importance to “deeper understanding of academic concepts and theories” (p = 0.02; d = 0.40) than teachers of service-learning courses or programmes that were converted from existing courses. The above findings should be viewed in the context of the goals and objectives of service-learning. As a pedagogy that leverages service to the community to enhance

82

G. Ngai et al.

Table 5.3 Characteristics of the Service-Learning Courses or Programmes on Which the Responses Were Based Frequency

Percentage

A new course/programme specifically designed for the purpose of SL

64

51.6

An existing course/programme that was converted into an SL course/programme by adding in an SL component

58

46.8

Others

2

1.6

82

66.1

Students from within my faculty /broad disciplinary area

17

13.7

Students who are majors in my programme

23

18.5

Others

2

1.6

Course Type

Course Nature General education type course/programme, targeting: Any student in the university Major-related course/programme, targeting:

Service Type# Direct

114

91.9

Indirect

36

29.0

Advocacy

17

13.7

Community-based research

35

28.2

Total number of hours of direct service or interaction with clients 10 h or below

15

12.1

11–20 h

16

12.9

21–30 h

24

19.4

31–40 h

44

35.5

41–50 h

16

12.9

More than 50 h

9

7.3

20 students or below

14

11.3

Class Size 21–40 students

52

41.9

41–60 students

18

14.5

61–100 students

22

17.7

More than 100 students

18

14.5

120

96.8

Credit Bearing or Not Yes

(continued)

5 How Hong Kong University Teachers Assess Students …

83

Table 5.3 (continued) Frequency No

Percentage

4

3.2

# Multiple

Remark: responses are allowed; some service-learning courses or programmes involve more than one type of service project.

Table 5.4 Respondents’ ratings on the importance of the intended learning outcomes to the servicelearning courses or programmes and whether they were assessed Overall

Intended learning outcomes n Mean Deeper understanding of academic concepts and 124 3.86 theories Applying knowledge to deal with real-life complex 124 4.47 issues 124 4.43 Problem-solving and critical thinking skills 124 4.53 Teamwork and effective communication skills 124 4.43 Understanding community problems and needs Becoming a more responsible member of the 124 4.42 community Better understanding of personal strengths and 124 4.06 weaknesses 124 4.39 Understanding other people’s perspectives Remarks: 1. The table cells shaded in light grey indicate the top three ratings 2. Underlined and italic numbers indicate the bottom three ratings

Frequency distribution (%)

sd 0.97

Not important 1 0.0

Fairly important 3 2 8.9 28.2

30.6 32.3

% Assessing this learning outcome 83.1

0.76

0.8

0.8 8.9

29.8 59.7

91.1

0.75 0.73 0.83 0.72

0.8 0.8 0.8 0.8

0.0 1.6 3.2 0.0

33.1 30.6 29.8 37.1

55.6 62.9 58.9 53.2

95.2 99.2 91.1 83.1

0.87

0.0

4.8 20.2

38.7 36.3

87.1

0.76

0.8

1.6 7.3

38.7 51.6

90.3

10.5 4.0 7.3 8.9

4

Very important 5

students’ sense of civic responsibility and understanding of course content (Bringle & Hatcher, 1995), it would be expected that the course objectives should go beyond merely facilitating students’ understanding of disciplinary skills and knowledge, and extend to non-disciplinary skills and knowledge, such as teamwork, communication skills, knowledge application, understanding community, problem-solving, and critical thinking. (Felten & Clayton, 2011) summarise the three main learning goals of service-learning as academic learning, personal growth, and civic learning. In particular, fostering students’ sense of social responsibility and civic literacy distinguishes service-learning from other types of experiential education (Lisman, 1995, as cited in Hatcher & Bringle, 1997). The learning outcomes that Hong Kong university teachers find important in service-learning align with the intended learning outcomes of service-learning, and dovetail with the three main learning goals. Besides civic learning, service-learning has also been viewed as an effective design in enhancing students’ soft skills and personal growth (Eyler et al., 2001; Furco & Root, 2010). These are also the skillsets demanded by today’s society. Our findings suggest that Hong Kong university teachers also expect service-learning to achieve these intended learning outcomes. For example, “teamwork and effective communication skills” belongs to the social and emotional skills; and “applying knowledge to deal with real-life complex issues” and “problem-solving and critical thinking skills” can be regarded as cognitive and meta-cognitive skills, under the twenty-first-century skills advocated by OECD (OECD, 2019). We also note that respondents from a “soft sciences” or humanities background tended to rate the importance of “deeper understanding of academic concepts and

84

G. Ngai et al.

theories” higher than those from a hard sciences background. This may be because skills and knowledge taught in soft sciences (e.g. social sciences) and humanities background may be more readily applicable in service contexts, which means that service-learning can be more easily used as a vehicle to achieve intended disciplinary learning outcomes than those from a hard science background. This stronger connection between service-learning experience and course content may also give teachers from soft sciences or humanities greater confidence in assessing these outcomes. In terms of course features, respondents of general education-type servicelearning courses or programmes, which are generally open to all students, generally place less importance on disciplinary learning outcomes, but more importance to civic, intrapersonal, and interpersonal outcomes than teachers involved in disciplinerelated courses or programmes. This is perhaps because of the nature of general education, which is intended to promote the holistic development of students, rather than their disciplinary knowledge. This phenomenon was also seen in teachers of courses or programmes that were designed for service-learning, vs their colleagues teaching service-learning courses or programmes converted from existing courses. In this case, this may be because an existing course, already having intended disciplinerelated learning outcomes of its own, may not afford teachers much space in integrating new outcomes or learning components. This suggests a need for further studies that compare student learning outcomes between these different types of courses or programmes in service-learning.

5.6.3 What Student Learning Outcomes Were Actually Assessed? The student learning outcomes that are assessed by the teachers are broadly consistent with those learning outcomes that are perceived as important. Almost all respondents indicated that they would assess “teamwork and effective communication skills” (99.2%); “problem-solving and critical thinking skills” (95.2%); “applying knowledge to deal with real-life complex issues” (91.1%), “understanding community problems and needs” (91.1%); and “understanding other people’s perspectives” (90.3%). However, relatively fewer respondents reported assessing “better understanding of personal strengths and weaknesses” (87.1%); “becoming a more responsible member of the community” (83.1%), and “deeper understanding of academic concepts and theories” (83.1%). (See Table 5.4, last column). These outcomes can also be regarded as indicators of well-being of students. Breakdown analyses according to respondents’ backgrounds revealed no statistically significant differences in the mean ratings of respondents across years of teaching experience and years of experience of teaching and assessing servicelearning. However, respondents from a “soft sciences” or humanities background are more likely to assess “deeper understanding of academic concepts and theories” than those from hard sciences (p = 0.01; effect size [Cramér’s V ] = 0.24).

5 How Hong Kong University Teachers Assess Students …

85

No statistical differences in respondents’ ratings were found between general education versus discipline-related courses or programmes. On the other hand, teachers of courses or programmes that were specifically designed for servicelearning are more likely to assess “becoming a more responsible member of the community” (p < 0.01; V = 0.33), “problem-solving and critical thinking skills” (p = 0.01; V = 0.24), “understanding other people’s perspectives” (p = 0.01; V = 0.24), “understanding community problems and needs” (p = 0.02; V = 0.22), and “applying knowledge to deal with real-life complex issues” (p = 0.02; V = 0.22). These findings suggest that teachers recognise that the primary focus of servicelearning is not about disciplinary skills and knowledge. Intrapersonal awareness is less likely to be assessed probably because it is not easily quantified and demonstrated in service-learning projects, thereby making valid and objective assessment difficult. Our results also suggest that teachers may not be comfortable assessing civic learning outcomes. These outcomes are usually assessed via reflection (e.g. Ash et al., 2005; Richard et al., 2016), and previous studies have explored various challenges faced by teachers in this aspect, such as logistical challenges in large classes, inadequate knowledge and skills for facilitation, and difficulty in setting standards for grading (Chan & Lee, 2021). Our own studies (Camus et al., 2021b) also suggest that teachers may not be confident in assessing civic learning outcomes via reflection, even if reflection is an integral part of service-learning. Interestingly, ease of assessment alone does not seem to be a major reason why certain learning outcomes carry more weight with teachers. Take the disciplinary learning outcomes, which can be easily handled using traditional assessment methods that are already familiar to teachers. We do not find that teachers are more likely to assess these outcomes as a result—in fact, disciplinary learning outcomes are, as previously mentioned, seen as being less important. This suggests that teachers’ reluctance to use reflections for assessment may be more deep-rooted than simply logistics, (un)familiarity or difficulty in setting standards. Our results also revealed that respondents from a “soft sciences” or humanities background were more likely to assess “deeper understanding of academic concepts and theories” than those from the hard sciences. As previously mentioned, “soft sciences” and humanities are more readily applicable to service-learning contexts, making it easier for teachers and students to link the service project with the disciplinary skills and knowledge, which in turn encourages the assessment of these outcomes. Teachers of courses or programmes that were specifically designed for servicelearning are more likely to assess the civic and higher-order thinking outcomes. One reason may be that integrating a service project into an existing course involves displacing existing learning components. It is inevitable that there will be a pressure to adapt the service project to cater for the learning outcomes that were previously assessed by the displaced component, which are likely to be disciplinary outcomes, hence the civic learning outcomes run the risk of becoming an afterthought. In contrast, designing a new course from scratch is an intensive exercise that requires teachers to reflect on the reasons and the need for this course. Teachers are therefore more likely to consider all the desired learning outcomes when starting from a

86

G. Ngai et al.

blank slate, which also allows a tighter integration of the different components of the learning experience. Finally, many courses that “become” service-learning through the integration of a service project are courses that allow students a choice as to whether they wish to participate in the service project or to complete a conventional term paper or other project. In these cases, since the students are taking the same course, the learning outcomes need to be similar for both groups, again limiting the space for integration of the civic learning outcomes. This finding has implications on the “best way” to integrate service-learning into the curriculum. Ultimately, this links to the purpose that the institution wishes to achieve via service-learning. If the purpose of service-learning is to afford students opportunities to apply their classroom knowledge and skills in a real-world setting, and to serve the community at the same time, then conversion of existing courses may be a better option, as it more tightly links the service project with the disciplinary knowledge and skills. However, if civic learning is the desired learning outcome, it appears that a course that is specifically designed for the purpose may be more appropriate, as trying to “retrofit” existing resources or tools for a new purpose is often more challenging than starting from a blank slate.

5.6.4 What Assessment Methods Did Teachers Actually Use? Table 5.5 presents the choice of assessment methods used by the respondents to assess the different intended learning outcomes of their service-learning course or programme. The learning outcomes “deeper understanding of academic concepts and theories”, “applying knowledge to deal with real-life complex issues”, and “understanding community problems and needs” were most commonly assessed via “reflective essay/journal/blog” (69.9 to 71.8%), “presentation” (58.4 to 63.1%), and “service project proposal” (54.7 to 54.9%), and least commonly via “test and exams” (4.4 to 15.5%), “academic term paper” (10.5 to 17.5%), “peer assessment” (10.7 to 15.0%), and “student self-assessment” (14.6%). For assessing “problem-solving and critical thinking skills” and “teamwork and effective communication skills”, teachers most commonly use “reflective essay/journal/blog” (56.6 to 68.6%), “presentation” (56.8 to 58.2%), and “on-site performance rated by instructor” (56.1 to 58.5%), and are least likely to use “test and exams” (0.8 to 5.9%), “academic term paper” (4.1 to 8.5%), “peer assessment” (20.3%), “student self-assessment” (20.3%), and “service report” (19.7%). The assessment of “better understanding of personal strengths and weaknesses” and “understanding other people’s perspectives” are most commonly carried out using “reflective essay/journal/blog” (81.3 to 90.7%), “on-site performance rated by instructor” (32.4 to 39.3%), “presentation” (29.6 to 39.3%), and “service deliverable” (29.6 to 39.3%), and the least commonly via “test and exams” (0.9 to 1.8%), “academic term paper” (4.6 to 5.4%), and “service report” (13.9 to 19.6%). Finally, to assess “becoming a more responsible member of the community”, the three commonest methods are “reflective essay/journal/blog” (79.6%), “on-site performance rated by instructor” (41.7%), and “service deliverable” (37.9%), and

5 How Hong Kong University Teachers Assess Students …

87

Table 5.5 Assessment methods used to assess different intended learning outcomes Intended Learning Outcomes

Deeper understanding of academic concepts and theories Applying knowledge to deal with real-life complex issues Problem-solving and critical thinking skills Teamwork and effective communication skills Understanding community problems and needs Becoming a more responsible member of the community Better understanding of personal strengths and weaknesses Understanding other people’s perspectives

% of Respondents reported using the method On-site Community Test Service performance partner Student evaluation/ Peer Service Service Reflective and Academic project rated by selffeedback assessment assessment Presentation report deliverable essay/journal/blog exams term paper proposal instructor

15.5 17.5

54.7

43.7

23.3

10.7

14.6

63.1

24.3

32.0

71.8

4.4

10.5

54.9

54.0

35.4

15.0

22.1

58.4

30.1

44.2

69.9

5.9

8.5

45.8

58.5

27.1

20.3

20.3

56.8

32.2

44.1

68.6

0.8

4.1

34.4

56.1

27.0

35.2

22.1

58.2

19.7

41.8

56.6

2.7

8.0

51.3

35.4

31.9

6.2

13.3

54.0

27.4

45.1

72.6

1.0

2.9

17.5

41.7

32.0

13.6

20.4

36.9

21.4

37.9

79.6

0.9

4.6

20.4

32.4

19.4

16.7

23.1

29.6

13.9

29.6

90.7

1.8

5.4

29.5

39.3

30.4

20.5

21.4

39.3

19.6

39.3

81.3

Remarks: 1. The table cells shaded in light grey indicate the percentages of the top three assessment methods used to assess a specific intended learning outcome 2. Underlined and italic numbers indicate the percentages of the bottom three assessment methods used to assess a specific intended learning outcome

the least common methods include “test and exams” (1.0%), “academic term paper” (2.9%), and “peer assessment” (13.6%). The finding that reflective essay/journal/blog was the most commonly employed methods for assessing students in service-learning is not surprising, as it is broadly consistent with the common use of student reflection to connect the service experience and learning outcomes in service-learning (Bringle & Hatcher, 1999; Eyler, 2002), and the use of reflection as a vehicle for assessment (Ash et al., 2005). However, it should be noted that assessing service-learning via reflection is not easy for teachers, as has been explored in previous work (e.g. Camus et al., 2021a, 2021b; Chan & Lee, 2021). Students’ on-site performance was also commonly used as an assessment component by our teachers. This makes sense since many learning outcomes, such as teamwork, communication skills, sense of social responsibility, and problem-solving, are demonstrated and readily observable during service delivery. This is particularly salient in direct service projects. However, differences between student perception of their own performance and teacher judgement may lead to assessment results being contested and challenged, especially since many teachers do not keep clear documentation of the service delivery process and many students are not very well able to judge their own performance! This points to a need for clear grading standards for all assessors to ensure efficiency and fairness (Chan, 2012), and perhaps for more emphasis on the intrapersonal learning outcomes of understanding one’s own strengths and weaknesses. Grading the service delivery may also increase teacher workload, particularly

88

G. Ngai et al.

when large classes and distant project sites are involved (Camus et al., 2021a). That being said, observing student on-site performance is an effective complement to other self-reporting methods such as reflection in assessment. The common use of presentation as an assessment method can also be linked to reflection. Some teachers prefer to use a verbal form of reflection as opposed to a written form; this is especially since in Hong Kong, the medium of instruction (English) is not the mother tongue for most of our students. Presentations also allow teachers to question, clarify, and offer feedback in real time. In a more pragmatic dimension, presentations may also be more efficient, as they are frequently on a group basis. It is also an assessment method commonly used for grading cognitive outcomes in conventional discipline-specific subjects and hence familiar to most teachers. The least commonly used assessment methods were tests, exams, and academic term papers. These conventional assessment methods are designed for traditional pedagogies, which aim at differentiating students’ mastery of disciplinary knowledge and skills. The fact that they are less commonly deployed in service-learning among our respondents is consistent with their general perception of service-learning as a vehicle that educates students beyond the conventional discipline-based outcomes. In fact, that service report, which is often similar to a term paper, was also not frequently employed by our respondents. One interesting observation is that, despite the fact that teachers commonly use on-site performance in their assessments, they rarely use community partner evaluations or feedback, peer and student self-assessments, which would presumably lower their workload in this aspect. There may be two reasons for this. First, it may be that teachers do not have confidence that students can accurately and honestly report their own (or their peers’) performance. This appears to be consistent with the aforementioned teachers’ lack of confidence in assessing the civic learning outcomes, which can only be assessed via self-reported learning artefacts. Second, community partner evaluation and feedback, even if they are reliable, may not be very logistically feasible. Community partners need to understand the course content and learning outcomes. They need to be familiar with all participating students and their contribution to the project, which implies a need to closely monitor and observe the performance of each individual student in delivering the service, often on multiple occasions, which is expensive. They also need to be familiar with skills for assessment. All the above may be potential concerns that discourage teachers from introducing community partners into student assessment.

5.6.5 What Criteria and Standards Did Teachers Use to Assign Grades? Table 5.6 presents the importance of various criteria and standards in determining the grades they gave in their service-learning courses or programmes, as rated by

5 How Hong Kong University Teachers Assess Students …

89

Table 5.6 Importance of Different Criteria and Standards in Determining Grades Overall Assessment criteria and standards Demonstrating understanding of the subject matter Amount of effort put into the service Punctuality and attendance Ability to deal with uncertainty during the service Enthusiasm and motivation to serve Demonstrating awareness of own strengths and weaknesses and how to improve Responsibility in the service project Caring attitude for others Ability to apply class learning into real-life situations Ability to examine issues from multiple perspectives

n 124 124 124 124 124 124

Mean 4.25 4.35 4.29 4.27 4.60 4.13

124 124 124

4.72 4.51 4.40 124 4.38

sd 0.74 0.70 0.72 0.75 0.57 0.80

Frequency distribution (%) Not Fairly important important 1 3 2 0.0 0.8 15.3 0.0 0.0 12.9 0.0 0.0 15.3 0.0 1.6 12.9 0.0 0.0 4.0 0.0 1.6 21.0

4 41.9 39.5 40.3 41.9 31.5 40.3

Very important 5 41.9 47.6 44.4 43.5 64.5 37.1

0.49 0.67 0.69 0.66

0.0 0.0 0.0 0.0

25.0 29.8 37.1 42.7

73.4 60.5 51.6 47.6

0.0 0.0 0.0 0.0

1.6 9.7 11.3 9.7

Remarks: 1. The table cells shaded in light grey indicate the top three ratings 2. Underlined and italic numbers indicate the bottom three ratings

the respondents. It is noted that all the criteria and standards were perceived as important, since all or almost all respondents gave a rating of three or above (fairly to very important) to all of the items. Among them, the three highest rated criteria and standards include “responsibility in the service project” (Mean = 4.72 on a five-point scale; 73.4% choosing “5 [very important]”); “enthusiasm and motivation to serve” (4.60, 64.5%), and “caring attitude for others” (4.51, 60.5%). The three lowest rated items were “ability to deal with uncertainty during the service” (4.27, 43.5%), “demonstrating understanding of the subject matter” (4.25, 41.9%), and “demonstrating awareness of own strengths and weaknesses and how to improve” (4.13, 37.1%). Further breakdown analyses according to respondents’ backgrounds revealed no statistically significant differences in the mean ratings of respondents across years of teaching experience. Analysing against respondents’ years of experience of teaching and assessing service-learning yielded some statistically significant differences, but a post-hoc least significant difference (LSD) test revealed no systematic patterns. Respondents from a soft sciences or humanities background tend to rate the importance of “ability to apply class learning into real-life situations” (p = 0.01; effect size [Cohen’s d] = −0.50) and “demonstrating understanding of the subject matter” higher than those from the hard sciences (p = 0.04; d = −0.41). Comparisons of respondents’ ratings across different course features (general education vs discipline-related) show no statistically significant differences in the mean ratings. However, teachers of courses or programmes specifically designed for service-learning tend to place more importance on “demonstrating awareness of own strengths and weaknesses and how to improve” (p < 0.01; d = −0.55), “enthusiasm and motivation to serve” (p < 0.01; d = −0.54), “caring attitude for others” (p < 0.01; d = −0.52), and “responsibility in the service project” (p = 0.01; d = −0.50). These findings are consistent with our previous conjectures. Teachers will use assessment criteria and standards related to the outcomes that they wish to achieve in their classes. Our teachers place higher emphases on the affective outcomes of service-learning, this is consistent with the highest rated criteria and standards: responsibility in the service project, enthusiasm and motivation to serve, and caring

90

G. Ngai et al.

attitude for others. In addition, since cognitive learning outcomes are not the primary goals that teachers intend to achieve, they would obviously focus less on associated assessment criteria and standards such as “demonstrating understanding of subject matter”. Other criteria, such as punctuality and attendance, may be deemed to be too low-level to really demonstrate learning on the part of the student. Similar arguments extend to teachers from the soft sciences or humanities and those teaching courses or programmes that are designed for service-learning.

5.7 Teachers’ Efficacy, Challenges, and Needs for Support Our survey also asked the respondents to rate their levels of efficacy, perceived challenges, and needs for support in service-learning assessment, which can be regarded as quality-of-life measures for teachers. The results are presented in Table 5.7. Efficacy-wise, respondents are generally confident in their ability to assess students fairly and accurately (80.6% choosing “4 or 5 [Strongly Agree]”; Mean = 4.07 on a five-point scale), and justifying their grading results in case of being questioned (82.3%, 4.11). On the other hand, half of the respondents (50.0%, 3.31) admitted that grading service-learning courses or programmes was harder than other courses or programmes, while around a quarter of them (27.4%, 2.68) found it difficult to assess soft skills, such as teamwork and communication skills, in service-learning. With respect to challenges, three quarters of the respondents found that grading service-learning courses or programmes takes more time and effort than grading other courses or programmes (75.0%, 4.02). Nearly half of them felt that there was too much variation in the criteria and standards across different service-learning courses or programmes (49.2%, 3.55). When asked about their need for support, over half of the respondents feel that more manpower (54.1%, 3.53) and training (53.3%, 3.45) is needed in grading and assessing service-learning courses or programmes. The above results suggest that although teachers are quite confident in their ability to grade or assess servicelearning, they still feel a need for training in service-learning assessment. They also face challenges of workload (time and effort required for assessment), which translates into a need for support in manpower. Further breakdown analyses according to respondents’ backgrounds revealed that there were no statistically significant differences in the mean ratings of respondents across years of teaching experience. Analysing respondents’ years of experience of teaching and assessing service-learning yielded statistically significant differences in some of the items, but the post-hoc LSD test revealed no systematic patterns. In terms of disciplinary background, respondents from the hard sciences tend to find grading service-learning courses or programmes harder than other courses or programmes than those from the soft sciences or humanities (p < 0.01; effect size [Cohen’s d] = 0.66). Comparisons of respondents’ ratings across different course features show no statistically significant differences in the mean ratings between teachers of general

124 124

I can justify my grading if my students were to question the grades they receive

I find grading SL courses/programmes harder than grading other courses/programmes

124

There is too much variation in the criteria and standards across different SL courses/programmes 124 124

More training should be provided to staff on how to grade SL courses/programmes

I need more manpower for grading and assessment in my SL course/programme

Needs for Support

124

Grading SL course/programme takes more time and effort than other subjects

Challenges

2.68

124

I find it difficult to assess soft skills such as communication and teamwork

3.53

3.45

3.55

4.02

3.31

4.11

4.07

I can assess my SL students fairly and accurately 124

Efficacy

1.05

0.88

0.91

0.94

1.24

0.80

1.10

0.76

0.8

3.2

1.6

2.4

1.6

11.3

0.8

12.1

0.8

13.7

12.9

5.6

4.8

13.7

2.4

40.3

2

Frequency distribution (%) sd

Strongly Disagree 1

Mean

Overall n

29.0

32.3

42.7

18.6

25.0

14.5

20.2

17.7

Neutral 3

34.7

45.2

33.1

40.3

32.3

49.2

22.6

51.6

4

19.4

8.1

16.1

34.7

17.7

33.1

4.8

29.0

Strongly Agree 5

Table 5.7 Respondents’ ratings on perception of efficacy, challenges, and needs for support in assessing students in service-learning courses or programmes

5 How Hong Kong University Teachers Assess Students … 91

92

G. Ngai et al.

education or discipline-related courses or programmes, or between teachers of service-learning courses or programmes specifically designed for service-learning and those converted from existing courses. There may be several reasons for the observation that teachers from a hard science background found grading service-learning courses harder than other courses. Hard science disciplinary knowledge is usually very deterministic and objective, while service-learning involves civic and affective learning outcomes. The two are naturally very dissimilar, and teachers from the hard sciences may be less comfortable with assessing these kinds of outcomes. Teachers from the hard sciences may also be less comfortable with making a subjective judgement without cross-referencing with another source (e.g. performance across multiple service sessions, or another colleague’s assessment) hence increasing the need for manpower and support. This finding suggests that additional training may be needed for teachers from the “harder” sciences.

5.8 Implications for Research and Practice Our study raises several implications for research and practice. First of all, university teachers in Hong Kong are generally well aware of the role of service-learning as more than merely facilitating academic understanding. In other words, the affective and civic learning aspects are well recognised and not neglected. Second, despite the sense that it is “easier” and “enough” to create a service-learning course by integrating a service project into an existing course, our analysis suggests that if the intention of doing service-learning is to cultivate students’ civic outcomes such as empathy and sense of social responsibility, customised service-learning courses or programmes tend to be more readily aligned with these outcomes. Third, although half of the teachers claim that they do not find assessing service-learning difficult and challenging, most do request support in terms of manpower and training. Given that the nature of service-learning involves multiple stakeholders and variations in learning contexts, support at the institutional level seems necessary. We argue that servicelearning should be viewed as similar to practicum-based subjects, and resources and staff members for regular on-site monitoring and support should be provided accordingly. We also argue that the service-learning community needs to move on from evaluating the effectiveness of service-learning courses or programmes via student selfreporting (e.g. interviews, surveys, and psychometric instruments) to demonstrating service-learning as an effective pedagogy via rigorous and authentic assessment of individual students’ attainment of the intended learning outcomes. This also involves demonstrating that the assessment is fair, consistent, and actually aligned to what students are supposed to learn. This is important as, from the student’s perspective, the assessment “is” the curriculum, and they will learn what they think will be assessed (Ramsden, 2003), so obviously inaccurate or misaligned performance

5 How Hong Kong University Teachers Assess Students …

93

assessment will not help students to achieve the learning outcomes, and might even turn them off with regard to service-learning as legitimate academic study. Our study also illustrates the complexity of service-learning assessment. The premise of service-learning is that students learn to serve, and then deepen their learning through service to the community and meaningful and effective reflection upon their service experience. This implies that service plays a critical role in service-learning. “Learning to serve”, in particular, suggests that the service is a demonstration of as well as a vehicle for student learning. One often-cited principle of good practice in service-learning is that “academic credit is for learning, not for service” (Howard, 2001, p. 16). This is often (mis)interpreted that performance in service is not important and should not be assessed. However, even Howard states that a student’s demonstration of academic and civic learning comes from both resources that are found in “traditional” courses, as well as from the community service. As affective and civic outcomes very often integrate (or intertwine) with academic learning in service, if a student achieves good learning in service-learning, this will usually manifest through desirable service performance. Hence, one should guard against the extreme of totally ignoring the role of service performance in assessment, as it serves as evidence of student learning, especially for affective and civic outcomes. Our study also raises some implications and further research directions. First, many intended learning outcomes, as well as their assessment criteria and standards, perceived as important by the teachers are about affective and civic learning, and related to student attitude and commitment. How they can be assessed reliably becomes a primary concern. Are reflection and rubrics effective and reliable methods of capturing those outcomes? Second, students learn what they perceive will be assessed. It therefore makes sense to understand their view and perceptions towards service-learning assessment. Which intended learning outcomes do students perceive are measured in service-learning? What kinds of assessment methods do they think would be helpful in achieving those outcomes? What do they perceive as good performance, and bad performance? Those questions should be further studied.

5.9 Limitations There are several limitations to the current study. Our results were drawn on a small sample from only three universities in Hong Kong. This naturally limits the generalisability of the study. Future studies could recruit a larger and more representative sample with more institutions and broader geographical coverage. Moreover, as discussed above, the current study only surveyed opinions from the teachers’ perspectives. Further studies should also investigate how other stakeholders, including students, school management, and community partners, view service-learning assessment. Also, this survey only collected teachers’ opinions on assessment outcomes, methods, criteria and standards, and their efficacy and challenges. This data is useful yet superficial. More in-depth studies, such as the mechanism behind such perception

94

G. Ngai et al.

and what can be done to help them cope with the challenges they face in assessment, can be carried out in the future.

5.10 Conclusion This multi-institution study explores the practice of assessing service-learning in Hong Kong universities. Teachers are surveyed as to the student learning outcomes that they deem important, the outcomes that they assess, the methods they use, as well as the criteria and standards they apply in assessing their students. We find that by and large, the civic learning nature and affective outcomes of service-learning are well recognised by Hong Kong university teachers. However, teachers acknowledge that assessing service-learning subjects is more challenging and time-consuming than assessing regular academic subjects. Teachers from the hard sciences in particular also appear to harbour reservations or lower self-efficacy in assessing self-reported student learning artefacts including reflections. Teachers of service-learning subjects that are converted from existing subjects also appear to be less likely to evaluate the civic and affective learning outcomes. This calls for more training and support for teachers. More research into alignment between different assessment methods and learning outcomes, and how they might impact student outcomes from servicelearning, would also be useful, as would research into students’ perceptions and experiences or expectations of assessment in service-learning. (around 7,800 words)

References Ash, S. L., & Clayton, P. H. (2004). The articulated learning: An approach to guided reflection and assessment. Innovative Higher Education, 29(2), 137–154. Ash, S. L., Clayton, P. H., & Atkinson, M. P. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service Learning, 11(2), 49–60. Astin, A. W., Vogelgesang, L. J., Misa, K., Anderson, J., Denson, N., Jayakumar, U., Saenz, V., & Yamamura, E. (2006). Understanding the effects of service-learning: A study of students and faculty. Report to the Atlantic Philanthropies, 1155. Bringle, R. G., & Hatcher, J. A. (1995). A service-learning curriculum for faculty. Michigan Journal of Community Service Learning, 112–122. Bringle, R. G., & Hatcher, J. A. (1999). Reflection in service learning: Making meaning or experience. Educational Horizons, 179–185. Bringle, R. G., & Hatcher, J. A. (2009). Innovative practices in service-learning and curricular engagement. New Directions for Higher Education, 147, 37–46. Camus, R. M., Ngai, G., Kwan, K.-P., & Shek, D. T. L. (2021a). Assessing students in academic service-learning: Faculty experiences and challenges. In G. Ngai, & D. T. L. Shek (Eds.), ServiceLearning Capacity Enhancement in Hong Kong Higher Education, Springer. Camus, R. M., Ngai, G., Kwan, K. P., Yau, J.H.-Y., & Chan, S. (2021b). Knowing where we stand: Mapping teachers’ conception of reflection in service-learning. Innovative Higher Education, 46(3), 285–302.

5 How Hong Kong University Teachers Assess Students …

95

Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164–181. Chan, C. K. Y. (2012). Assessment for community service types of experiential learning in the engineering discipline. European Journal of Engineering Education, 37(1), 29–38. https://doi. org/10.1080/03043797.2011.644763 Chan, C. K. Y., & Lee, K. K. W. (2021). Reflection literacy: A multilevel perspective on the challenges of using reflections in higher education through a comprehensive literature review. Educational Research Review, 32, 100376. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45(12), 130–1312. Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for servicelearning: Comprehensive case studies of impact on faculty, students, community, and institution. Michigan Journal of Community Service Learning, 3(1), 66–71. Einfeld, A., & Collins, D. (2008). The relationships between service-learning, social justice, multicultural competence, and civic engagement. Journal of College Student Development, 49(2), 95–109. Eyler, J. (2002). Reflection: Linking service and learning—Linking students and communities. Journal of Social Issues, 58(3), 517–534. Eyler, J., & Giles, D. E. (1999). Where’s the Learning in Service-Learning? Jossey-Bass. Eyler, J., Giles, D. E., Stenson, C. M., & Gray, C. J. (2001). At a glance: What we know about the effects of service-learning on college students, faculty, institutions and communities, 1993–2000 (3rd ed.). Vanderbilt University. Felten, P., & Clayton, P. H. (2011). Service-learning. In W. Buskist & J. E. Groccia (Eds.), New directions for teaching and learning (Vol. 2011, pp. 75–84). Jossey-Bass. Furco, A., & Root, S. (2010). Research demonstrates the value of service learning. Phi Delta Kappan, 91(5), 16–20. Glasow, P. A. (2005). Fundamentals of survey research methodology. McLean. Hatcher, J. A., & Bringle, R. G. (1997). Reflection: Bridging the gap between service and learning. College Teaching, 45(4), 153–158. Howard, J. (2001). Service-Learning Course Design Workbook. Michigan Journal of Community Service Learning. http://hdl.handle.net/2027/spo.3239521.spec.201 Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation: A collection of principles, methods, and strategies useful in the planning, design, and evaluation of studies in education and the behavioral sciences (3rd ed.). Educational and Industrial Testing Services. Kennedy, D., Hyland, A., & Ryan, N. (2006). Writing and using learning outcomes, a practical guide. EUA Bologna Handbook. Making Bologna Work. Berlin: European University Association http:// www.eua.be/publications/bologna-handbook Kezar, A. (2002). Assessing community service learning: Are we identifying the right outcomes? About Campus, 7(2), 14–20. Kotrlik, J. W., & Williams, H. A. (2003). The incorporation of effect size in information technology, learning, and performance research. Information Technology, Learning, and Performance Journal, 21(1), 1–7. Kraemer, K. L. (1991). Introduction. In K. L. Kraemer (Ed.), The information systems research challenge (vol. III): Survey research methods (pp. xiii–xvii). Harvard University Graduate School of Business Administration. Lau, K. H., & Snell, R. S. (2020). Service-learning outcomes measurement scale (S-LOMS): The user manual. Office of Service-Learning, Lingnan University. https://doi.org/10.14793/978988 7522201 Lundy, B. L. (2007). Service learning in life-span developmental psychology: Higher exam scores and increased empathy. Teaching of Psychology, 34(1), 23–27. Ma, C. (2018). Service-learning development in higher education in Hong Kong. In T.-W. Lim & T. Y. Kong (Eds.), Studying Hong Kong: 20 years of political, economic and social developments (pp. 43–61). World Scientific.

96

G. Ngai et al.

Ma, C. H. K., Chan, C. W. F., & Tse, I. P. H. (2019). A common outcome measurement for service-learning in Hong Kong. Journal of Higher Education Outreach and Engagement, 23(3), 3–20. McDonald, B. (2012). Assessment in Service-Learning. In U. o. T. a. Tobago (Ed.). Arima: ED535897. Menéndez-Varela, J.-L., & Gregori-Giralt, E. (2016). The contribution of rubrics to the validity of performance assessment: A study of the conservation–restoration and design undergraduate degrees. Assessment & Evaluation in Higher Education, 41(2), 228–244. Misyak, S., Culhane, J., McConnell, K., & Serrano, E. (2016). Assessment for learning: Integration of assessment in a nutrition course with a service-learning component. NACTA Journal, 60(4), 358–363. Molee, L. M., Henry, M. E., Sessa, V. I., & McKinney-Prupis, E. R. (2011). Assessing learning in service-learning courses through critical reflection. Journal of Experiential Education, 33(3), 239–257. Ngai, G., Chan, S. C., & Kwan, K. P. (2018). Challenge, Meaning and Preparation: Critical Success Factors Influencing Student Learning Outcomes from Service-Learning. Journal of Higher Education Outreach and Engagement, 22(4), 55–80. OECD. (2019). OECD future of education and skills 2030: OECD learning compass 2030. OECD Publishing. Peterson, T. O. (2004). Assessing performance in Problem-Based Service-Learning projects. New Directions for Teaching and Learning, 100, 55–63. Prentice, M. (2007). Service learning and civic engagement. Academic Questions, 20(2), 135–145. Quinn, D., & Shurville, S. (2009). From little things big things grow: Scaling-up assessment of experiential learning. Campus-Wide Information Systems. Ramsden, P. (2003). Learning to teach in higher education. Routledge. Rea, L. M., & Parker, R. A. (2014). Designing and conducting survey research: A comprehensive guide (4th ed.). Jossey-Bass. Richard, D., Keen, C., Hatcher, J. A., & Pease, H. A. (2016). Pathways to adult civic engagement: Benefits of reflection and dialogue across difference in higher education service-learning programs. Michigan Journal of Community Service Learning, 23(1), 60–74. Schmidt, N. A., & Brown, J. M. (2016). Service learning in undergraduate nursing education: Strategies to facilitate meaningful reflection. Journal of Professional Nursing, 32(2), 100–106. https://doi.org/10.1016/j.profnurs.2015.06.006 Shek, D. T., & Chan, S. C. (2013). Service-learning from the views of university teachers: A qualitative study based on focus groups. International Journal of Adolescent Medicine and Health, 25(4), 385–393. Shek, D. T. L., Yu, L., Wu, F. K. Y., & Chai, W. Y. (2015). General University Requirements at Hong Kong Polytechnic University: Evaluation findings based on student focus groups. Assessment & Evaluation in Higher Education, 40(8), 1017–1031. https://doi.org/10.1080/02602938. 2014.960362 Shumer, R., Stanton, T. K., & Giles, D. E. (2017). History and pre-cursors of service-learning theory, development, and research. In R. Shumer (Ed.), Where’s the wisdom in service-learning (pp. 1–32). Information Age Publishing. Simons, L., & Cleary, B. (2006). The influence of service learning on students’ personal and social development. College Teaching, 54(4), 307–319. Snell, R. S., Chan, M.Y.-L., Wu, C. X., & Chan, C.W.-Y. (2019). Service leadership emergence through service-learning internships in Hong Kong. Michigan Journal of Community Service Learning, 25(2), 167–199. Snell, R. S., Chan, M. Y. L., Ma, C. H. K., & Chan, C. K. M. (2015). Developing civic-mindedness in undergraduate business students through service-learning projects for civic engagement and service leadership practices for civic improvement. Asian Journal of Business Ethics, 4(1), 73–99. https://doi.org/10.1007/s13520-015-0044-0

5 How Hong Kong University Teachers Assess Students …

97

Snell, R. S., & Lau, K. H. (2020). The development of a service-learning outcomes measurement scale (S-LOMS). Metropolitan Universities, 31(1), 44–77. https://doi.org/10.18060/23258 Strouse, J. H. (2003). Reflection as a service-learning assessment strategy. Journal of Higher Education Outreach and Engagement, 8(2), 75–88. Sullivan, G. M., & Feinn, R. (2012). Using effect size—or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. Toncar, M. F., Reid, J. S., Burns, D. J., Anderson, C. E., & Nguyen, H. P. (2006). Uniform assessment of the benefits of service learning: The development, evaluation, and implementation of the SELEB scale. Journal of Marketing Theory and Practice, 14(3), 223–238. https://doi.org/10. 2753/mtp1069-6679140304 Warren, J. L. (2012). Does service-learning increase student learning?: A meta-analysis. Michigan Journal of Community Service Learning, 18(2), 56–61. Wong, M. M. L., Lau, K. H., & Chan, C. W. F. (2021). The impacts and success factors of a workfrom-home service-learning internship during COVID-19. Journal of Work-Applied Management, 13(2), 284–301. https://doi.org/10.1108/jwam-01-2021-0003 Xing, J., & Ma, C. H. K. (2010). Service-learning in Asia: Curricular models and practices (Vol. 1). Hong Kong University Press. https://doi.org/10.5790/hongkong/9789888028467.001.0001 Yorio, P. L., & Ye, F. (2012). A meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9–27. Yu, L., Shek, D. T., & Zhu, X. (2018). The influence of personal well-being on learning achievement in university students over time: Mediating or moderating effects of internal and external university engagement. Frontiers in Psychology, 8, 2287. https://doi.org/10.3389/fpsyg.2017. 02287

Chapter 6

Promotion of Teachers’ Readiness for Service-Learning Through a Professional Development Course Grace Ngai and Stephen C. F. Chan

Abstract Academic service-learning, especially at the higher education level, poses a variety of challenges for teachers. In addition to the obvious challenges with coordinating and managing service opportunities for students, academic service-learning requires that students’ learning be linked with their service project, and that their learning gains be assessed like other academic subjects. The breadth and the diversity of the experiences means that teachers need to possess knowledge and expertise in a broad range of educational areas—for example, experiential education, reflective learning, civic learning, etc. Keywords Teacher readiness · Professional development · Course design Academic service-learning, especially at the higher education level, poses a variety of challenges for teachers. In addition to the obvious challenges with coordinating and managing service opportunities for students, academic service-learning requires that students’ learning be linked with their service project, and that their learning gains be assessed like other academic subjects. The breadth and the diversity of the experiences means that teachers need to possess knowledge and expertise in a broad range of educational areas—for example, experiential education, reflective learning, civic learning, etc. It is well-known in the education community that the learning experience is the most important factor that impacts student learning. This has also been shown to be true for service-learning (Chan et al., 2019). In previous work, it has been shown that even students who take a service-learning course primarily to fulfill graduation requirements do not learn significantly less than those who enroll in a course out of interest or a desire to serve. It has also been shown that students’ perceptions can and do change; a qualitative study of 49 students in 2013/2014 showed that all 8 G. Ngai (B) · S. C. F. Chan Hong Kong Polytechnic University, Hong Kong, China e-mail: [email protected] S. C. F. Chan e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_6

99

100

G. Ngai and S. C. F. Chan

students who held very negative views of a service-learning requirement changed their views and took on a supportive stance after participating in the service-learning course (Chan et al., 2021). Seeing the impact on students, or, more precisely, being a factor in creating this impact on students, also has the potential to positively impact teachers. In a qualitative study carried out in 2017–2018 (Camus, Ngai, Kwan, & Chan, In Submission), a large majority of the interviewed faculty (20 out of 24) stated that service-learning resulted in better understanding and increased appreciation of students. One teacher, who had never previously been involved in volunteer work and started teaching service-learning simply because it was an assigned duty by the department, stated that they found it “worthwhile to teach service-learning”, since it allowed them to “influence them (students) through teaching”. Another teacher even went as far as to say that service-learning “changed my view of students”. If students’ perceptions can be changed from being hostile to a requirement to being supportive of it, the reverse is also true. (Chan et al., 2021) also found that while all the initially hostile students took on a positive view of a service-learning requirement after the experience, it was not true that all of the initially supportive students stayed positive about the requirement. A few of the students who were initially positive became ambivalent or even negative about the requirement—not so much because they philosophically opposed the idea of civic education via servicelearning, but because their experiences in the service-learning project did not match with their expectations. For example, one student recalled that she had expected to be able to spend a lot of time interacting with the community, but instead was tasked to create a computer application for an NGO (non-governmental organization) for her project, during which she spent most of the time in the lab. Service-learning does not only educate the aptitude of students, it also educates the attitude. In this sense, it is rather unique in university learning, much of which focuses upon cognitive or skills-based learning of students. However, students do not automatically learn when sent into the community to serve. When improperly done, service-learning can reinforce stereotypes and notions of superiority in students. For example, a student wrote in her reflection essay after a service experience in a developing country: Even for the job they live on, they don’t have a serious attitude... We can see that it is very usual for them to be late for different kinds of activities. Generally, the impression was that in this country, people are not in favor of reading or learning. They are neither punctual nor serious towards their works. This has already become the culture of the country.

Badly executed service-learning projects can also bring harm to the community— the community may be left with an improperly functioning product or deliverable that cannot be used; or community members may be left with feelings of hurt and disempowerment. All of these reasons suggest that proper teacher preparation is critical for servicelearning. This is important in general, but perhaps more critical in higher education. Unlike K-12 teachers, the majority of higher education faculty do not attain their qualifications through an educational training. Instead, they attain their qualifications

6 Promotion of Teachers’ Readiness …

101

via research in their own disciplines. The hiring process also considers mainly the candidates’ demonstrated ability in research, namely their publication track record. It is not too surprising, then, that a professor in mathematics or chemistry or engineering does not have much experience in experiential education or other various pedagogical concepts or practices that underpin service-learning. An added complicating factor is the fact that service-learning focuses on the “softer”, more affective learning outcomes of personal or civic development of students, which is very different from the cognitive or intellectual learning outcomes that are prevalent in some of the hard sciences (e.g., the “STEM” disciplines of science, technology, engineering, and mathematics). In these disciplines, student learning focuses almost wholly on the learning and application of objective theories, with very little or no room for personal interpretation, at least at the undergraduate level. Something like reflection, or the reflective essay, which is so commonplace in service-learning that it is often said that “without reflection, there is no learning”, is almost unheard of in courses in these disciplines. Hence, it is not surprising that in many institutions, teachers who are involved in service-learning are mainly from the “softer” fields of social sciences, health sciences, and education (Antonio et al., 2000). In this chapter, we will first explore some potential misconceptions about servicelearning that are commonly held by teachers, and documented in previous work as well as from our own experiences. Based on these misconceptions, we will then lay out a set of requirements for an effective professional development course in service-learning. We will then describe the elements of our own course that was first developed in 2016 and has been continually refined ever since. Finally we will conclude with an evaluation of the course as experienced by the participants, and discuss our results.

6.1 Misconceptions and Myths about Service-Learning Literature and our own studies have shown that there are a number of “myths” and misconceptions about academic service-learning. (Howard, 2001b) identifies four myths: – The myth of terminology—conflating academic service-learning with similar programs such as student community service and co-curricular service-learning – The myth of conceptualization—conflating academic service-learning with internships – The myth of synonymy—conflating experience in the community with learning – The myth of marginality—conflating academic service-learning with a simple addition of a community-based project to a traditional course Our own investigations (Shek & Chan, 2013) bear out these findings. The Hong Kong Polytechnic University started making academic service-learning a graduation requirement in 2012. As preparation for the process, a number of interviews

102

G. Ngai and S. C. F. Chan

with teachers were conducted to ascertain their general understanding and views on service-learning. They were also asked about potential challenges that they anticipated they would have to handle if they were assigned to teach a service-learning subject. The majority of the teachers correctly perceived service-learning to benefit students’ problem-solving, organization, professional, and social/interpersonal skills. Their perception of the process of service-learning, however, seemed to be along the lines of a glorified form of volunteer work or charity. Many of them stated that the primary goal of service-learning was to facilitate students to apply their academic knowledge in a practical context that would benefit the community. It is not completely off the mark, but still quite far from the complete picture, or even the main objective. Part of the consequence of such misunderstandings is the misconception that service-learning would only be applicable for students with particular professional skills. For example, while students in the health sciences disciplines, such as nursing or physiotherapy, would be able to readily make use of their skills to serve the community, this would not be as applicable for students in Hotel and Tourism Management. A follow-up confusion was then the conflation of service-learning with the professional practicums that are required as part of the curriculum by certain disciplines; teachers queried why students in, for example, social work or nursing, would still be required to do service-learning. Faculty in Engineering had the same query, albeit coming from a different angle: a commonly heard argument was that since engineering, being an applied science, in essence prepares students to use their academic skills to build and construct for the society, service-learning would not add to the learning outcomes already required by their program. Given these perceptions, it is not surprising that many teachers’ perceived benefits of service-learning to students revolved around skill acquisition. In particular, teachers believe that service-learning could help students actualize professional knowledge and skills, and accentuate their generic skills (self-understanding, interpersonal skills, communication skills) and problem-solving. In contrast, the psychosocial outcomes of civic learning and social responsibility were infrequently mentioned, if at all. A separate study conducted in 2020 (Camus et al., 2021) on the perceptions of teachers with respect to the process and the role of reflection in service-learning similarly uncovered a wide diversity of perceptions toward reflections. Some teachers took a very instrumental approach to reflection—for them, reflection was mainly for assessing students’ performance, and, in some cases, for reviewing the project logistics and operations. In this sense, the reflection is very much a tool for the teacher’s use, as opposed to a pedagogy to facilitate the learning process of the student. These perceptions have implications on the teaching process. If a teacher believes that service-learning is primarily to allow students to serve the community through applying their academic knowledge, he/she will spend more time and energy into teaching and practicing those skills with the students. The so-called service-learning

6 Promotion of Teachers’ Readiness …

103

course might then resemble a discipline course, with the addition of a communitybased project. For example, in the computer science discipline, a faculty member teaching a subject in databases might assign students a final project in which they would be asked to contact NGOs and offer to develop inventory management systems for them, rather than to develop a toy project for a pretend client in class. The teacher would then focus primarily on ensuring that the students had a solid grasp of the database concepts and the programming knowledge, and perhaps also to ensure that there was a reasonable match of expectations between the NGO and the students. For reflections, students might be expected to evaluate the communication process between themselves and their clients, how it impacted the user requirements acquisition, or how various database concepts were applied in the developed system, or their own strengths and weaknesses as evidenced through the process of the project. These projects are undoubtedly useful to the community, and indeed, database development is one of the most frequent requests that we get from NGOs. However, these types of project offer the students very little learning outside of what they would get through internships, and may not even be as effective as internships in preparing students for their profession. One observation that we made during the interviews was that the civic learning outcome (social responsibility and citizenship) was very rarely ever mentioned by the teachers. This is disappointing and warrants attention as, while service-learning can contribute to a number of different outcomes, many of these outcomes can also be fulfilled with other pedagogies. For example, problem-solving and communication skills are also among the learning outcomes for project-based learning. However, the civic learning outcome is not commonly covered by curricular learning activities at the higher education level. This implies that the civic learning outcome should not be neglected, as it would not be “made up for” through other learning activities.

6.2 Teachers’ Concerns and Needs In addition to teachers’ misconceptions of service-learning, it is also important to understand their concerns and needs in the event that they were asked to teach a service-learning subject. This is a concern that is perhaps more accentuated in Asia, where the department management in general has more authority over the teaching assignments, and it is not unheard of for teachers to be assigned to teach a course in a subject area that they are not familiar with. (Shek & Chan, 2013) also polled teachers on their needs and concerns in the event that they were asked to teach a service-learning subject. As expected, quite a number of teachers were worried about their own expertise and experience. Quite a number of teachers, especially those in the hard sciences or the engineering sciences, admitted to not being familiar with qualitative or subjective modes of student assessment, and opined that they were not confident about guiding students to reflect on psychosocial topics such as social responsibility.

104

G. Ngai and S. C. F. Chan

The operational issues also concerned teachers. Unsurprisingly, quite a number of teachers were worried about the workload, in particular, the workload incurred by the service project component of the course. Finding opportunities for service was also an area of concern, so many teachers were worried about negotiating with the NGOs, or, as said by one teacher “to convince the NGOs to take [our] students”.

6.3 Course Requirements Based on our findings, we designed our course to fulfill the following requirements:

6.3.1 Clarification of Core Concepts Given the confusion between service-learning and other related forms of experiential learning, it is important that the course clarifies the concepts underpinning servicelearning, and distinguish service-learning from similar forms of experiential learning. In particular, care must be taken to explain the differences between service-learning, “community service” and “internships”. The former is a very common co-curricular activity in universities, and many institutions have a requirement for a certain number of hours of service to the community as a prerequisite for graduation or eligibility for scholarships or other honors. Internships are also widely used in undergraduate education, and in some disciplines—in particular the healthcare-related disciplines— students are often placed in settings such as the public hospitals that mainly serve the socioeconomically disadvantaged. It is therefore quite understandable if one jumps to the conclusion that service-learning is redundant since students are already “helping the underprivileged” through their discipline practicums.

6.3.2 Assurance that Service-Learning is Genuine Academic Learning Traditionally, higher education in Asia has played the role of professional or career training. In recent years, there is increasing awareness of the need for holistic education at the university level, and general education is getting more attention, but most people, including faculty, still perceive that the primary educational mission of universities is to prepare workers for the society. The result is that non-disciplinerelated learning activities, including service-learning, or even broadening or language requirements, are often not taken seriously. The aforementioned conflation between service-learning and other related forms of experiential learning, which are very common in higher education institutions, exacerbates the problem with many teachers

6 Promotion of Teachers’ Readiness …

105

and university management having the misconception that service-learning is simply informal learning and not academically rigorous. Our course therefore needs to convince teachers that service-learning and even experiential learning is genuine academic learning and worth academic credit.

6.3.3 Concrete Illustrations for Clear Understanding Service-learning is a complex pedagogy, and there are many parties involved and many moving pieces that need to be put together. In addition, most faculty at the higher education level are not trained as teachers, many of the practices and concepts that are common in the education community are not familiar to them, including experiential learning and reflective learning, two of the mainstays of service-learning. It is not surprising then to see many teachers being concerned that they would not be adequately prepared to teach or assess service-learning. Given this, we need to provide concrete illustrations to show teachers how to operationalize the concepts. For example, we can provide some case studies with actual projects, including implementation details such as project schedules and grouping principles, with enough details such that it is possible for the teacher to replicate the project with his/her own students. Other information useful to teachers are examples of course-intended learning outcomes, grading rubrics, reflection prompts, or even graded assessments for benchmarking.

6.3.4 Discussions for Peer Learning and Community Building Finally, human learning is by nature a social activity. (Wenger, 2013) suggests that community support and peer learning are important for sustaining and deepening impact from teaching and learning. This insight has led to the development of communities of practice, which allows individuals to gather together with peers who share a common interest, and collectively, to learn how to do it better through regular interaction and peer learning. In our experience, service-learning teachers tend to be rather isolated within their work circles. In traditional discipline subjects, faculty are grouped together into academic departments and all faculty within that department teach subjects related to that discipline, and prerequisites and corequisites create dependency between the different subjects and interaction between faculties. However, it is relatively rare for an academic department to have more than one or two service-learning subjects. Hence, service-learning teachers can often feel isolated and misunderstood in their work. The added administrative workload that is incurred by service-learning— for example, coordinating service sessions from students—can also cause friction

106

G. Ngai and S. C. F. Chan

between colleagues. (Camus et al., In Submission) report that among the 24 teachers involved in the study, 6 teachers reported that service-learning affected their work relations, in particular with administrative and support colleagues who were assigned to handle the extra work. We therefore feel that the course needs to allow opportunities for teachers to build rapport with peers. While it is perhaps unrealistic to expect a professional development course to be able to build community, one can facilitate the process by building in opportunities for participants to interact and discuss in an open-ended manner.

6.4 Course Design Principles Our course design draws from principles and practices in andragogy (adult education). It is known that adult learners exhibit different characteristics from child learners. These result in different principles that need to be taken into consideration when designing for adult learners. (Knowles, 1984) lists the Four Principles of Andragogy as follows: 1. 2. 3. 4.

Adults need to be involved in the planning and evaluation of their instruction Experience provides the basis for the learning activities Adults are most interested in learning subjects with immediate relevance and impact to their needs; Adult learning is problem-oriented rather than content-oriented.

Drawing from these principles gives us the following design guidelines for our course: 1.

2.

3.

The course needs to give learners freedom and control over different modes of learning, and also (up to a certain point) different levels of commitment. For our context, the course should allow learners to exit early, or to decide for themselves the depth that they would like to go into, while still ensuring an adequate understanding of the topic. The instruction needs to be practical and grounded in real instances. In particular, we have noted that it is oftentimes more, or just as, helpful to share negative rather than positive examples. This is substantiated by learning theory, in which it has been observed that providing both positive and negative examples help to guard against over- and under-generalization from the learners (Ali, 1981). In our experience, we find that teachers welcome negative examples as a concrete “boundary” as to what is desirable or allowed. The instruction needs to be task-oriented rather than theoretical or didactic. In our context, the topics of concern that need to be addressed should be the different components of the learning experience that make up a service-learning program—for example, the project, the reflection, and the assessment.

6 Promotion of Teachers’ Readiness …

107

6.5 Course Design—Selection of Topics At a broad level, the topics that will be covered in the course should include the basic academic theories and concepts of service-learning (“What is service-learning”), the benefits of service-learning to its various participants, such as students, teachers, community, and institution (“Why do service-learning”), and the operational details and principles behind designing, executing, and assessing a service-learning program (“How to do service-learning”). The exact topics that will be covered in the course need to be carefully thought out. Service-learning originated and has a long history in North America. It is therefore not surprising that most of the resources are very North American (in particular US) centric. However, education is very context- and culture-dependent. Servicelearning, because it involves the community and has the objective of impacting students’ attitudes, is even more dependent upon the context and culture (Thomson et al., 2011). We illustrate with an example. It is commonly accepted in the service-learning community that allowing students to have a role in the planning and design is a good practice in service-learning. For example, (Billig et al., 2005) state that “In high quality service-learning projects, students have considerable voice in determining activities, and teachers facilitate knowledge and skill acquisition”. Our own research, however, does not back this up. In a study of 2,880 students enrolled in 55 creditbearing academic service-learning subjects, “student autonomy” was not among the service-learning practices with the strongest positive correlations to student learning outcomes (Ngai et al., 2018). Similarly, some topics in service-learning may not be relevant or suitable for practitioners outside of the Western context of North America or Europe, or they may take on a very different meaning outside of North American contexts. For example, “cross-cultural service-learning” in North America is often used to refer to middleclass, predominantly white college students serving working class, predominantly black communities. This takes on a very different meaning in regions which are ethnically/culturally more homogeneous, such as many countries/regions in East and South-East Asia. We thus classify the topics that could be covered in such a course along two dimensions. Conceptually, topics can be classified according to the function that they serve: to provide a conceptual understanding, to motivate, or to teach operationalization details. Pedagogically, the mode in which the topics can best be delivered can also be classified into self-guided, interactive, and experiential.

108

G. Ngai and S. C. F. Chan Self-guided

Interactive

Experiential

• Theory and Conceptual definition of Understanding (“What service-learning is service-learning?”) • Conceptual differences between service-learning and other forms of experiential learning

• Student learning outcomes from service-learning • Impact on the community

• Different models of service-learning programs

Motivation (“why do service-learning?”)

• Benefits of service-learning to different stakeholders

• Impact/benefit to the teacher

Operational (“how to do service-learning?”)

• Good practices in service-learning program design • Theories of reflection • Concepts of assessment

• Balancing service with • Facilitating and learning grading reflections • Negotiating with • Practices of NGOs service-learning • Turning a conceptual assessment design into a practical course • Supervising/supporting students in service • Funding service-learning • Organizing international service-learning

6.6 Course Design The aforementioned principles and the classification of the concepts help to inform the design of our course. In line with the principles of andragogy, the designed course consists of modular components that are intended to build upon each other, but also complete enough such that learners may stop after any level as needed. Self-guided, critical concepts are covered in a self-paced e-learning level, which are then reinforced with face-to-face instructional-based learning, and experiential-based learning for observations of real cases.

6 Promotion of Teachers’ Readiness …

109

6.7 Conceptual Learning—Self-Paced e-Learning The foundation of the professional development program are the concepts and principles behind the pedagogy of service-learning, and its applications at the higher education level. These are critical topics which are conceptual and mainly involve reading, meaning that they can easily be adapted to be delivered in an online module, which also has the advantage that it allows students to set their own pace through the content. We developed a MOOC (Massive Open Online Course) titled “Service-Learning Pedagogy and Practice”. The MOOC contains six units that take the learners from understanding what service-learning is, to motivate them as to why to do servicelearning, and finally, to scaffold them with concrete practices on how to design and operationalize a service-learning program: 1. 2. 3. 4. 5. 6.

Theories and Fundamentals of Service-Learning Benefits and Challenges of Service-Learning Developing and Sustaining Community Partnerships Designing and Implementing Service-Learning Facilitating Reflection Assessing Students in Service-Learning

Each of the topics involves a mix of theory, practice and assessment. For example, the first unit “Theories and Fundamentals of Service-Learning” gives the definition of service-learning, covers the difference between service-learning and other types of experiential learning, and introduces the theory of experiential learning (Kolb, 1984). It then concretizes these theories with examples of service-learning from diverse disciplines ranging from engineering, to tourism, to the health sciences and the social sciences. The motivation unit seeks to alleviate concerns from teachers about the impact of service-learning upon themselves, or about working with students who might be reluctant to engage in community. “Benefits and Challenges of Service-Learning” covers the twenty-first century competencies, the benefits of service-learning to communities, and to the institutions and the teachers themselves. At the same time, the unit acknowledges and addresses teachers’ concerns. Problems and challenges of service-learning are covered, as well as potential risks that might arise from improperly designed or poorly prepared projects. To encourage learners who might be daunted by all the challenges, the unit also includes research results that show that student perceptions can, and do, change, as a result of participating in service-learning (Chan et al., 2021). Because of its multi-faceted nature, service-learning operation involves more aspects than conventional academic subjects. Hence, this portion of the course is divided into units covering community engagement, service project design, student learning, and student assessment. The third and the fourth units cover the community aspect of “Developing and Sustaining Community Partnerships” and “Designing and Implementing ServiceLearning”. Because of the diverse nature of community partners, the “Developing

110

G. Ngai and S. C. F. Chan

and Sustaining” unit covers broad design guidelines such as reciprocity and principles of partnership between institutions and community organizations and members. The “Designing and Implementing” unit likewise covers broad design guidelines, such as the principles of effective practices (Howard, 2001a). With an understanding of the potentially international nature of the learners, two different sets of guidelines are included: one drawn from the North American context (National Service-Learning Cooperative, 1998) and another from a Hong Kong-based large-scale study (Ngai et al., 2018). Our experience has shown us that teachers often struggle to link academic concepts with a real service project, especially when the civic learning perspective is concerned. Therefore, the “Designing” unit includes examples from a diverse selection of disciplines, with a side-by-side comparison between the learning outcomes of the program and a description of the project. Two examples from Agriculture and Engineering are shown below: Agriculture Learning Outcomes

Service Project Description

Upon completion of the subject, students will be able to: Understand the environmental damages and impact of different forms of agriculture and aquaculture practices Apply knowledge and skills on responsible agriculture and modern agricultural practices to deal with complex issues in the service setting Work effectively in teams to solve problems encountered in planning and delivering the service Communicate effectively with clients from different backgrounds and/or other stakeholders Demonstrate empathy for people in need and a sense of civic responsibility Reflect on their role and responsibilities both as a professional in their chosen discipline and/or as a responsible citizen

Students will be required to use their knowledge and skills acquired in class to help farmers and the community in agricultural production. The program is targeted to enhance resource efficiency, especially energy, and to raise the productivity level, reduce the default rate, and to enhance crop quality and yield. Students will use computer simulated models, fabrication of agricultural devices, and other models or demonstrations, including active involvement in routine farming activities, to help and to communicate with the rural farmers and community in need

Engineering (continued)

6 Promotion of Teachers’ Readiness …

111

(continued) Upon completion of the subject, students will be able to: Demonstrate an understanding of the importance and impact of a healthy indoor living environment, and the linkage with basic human rights issues; Articulate the specific challenges related to unsustainable and/or unhealthy living environments faced by low-income communities and individuals; Apply the knowledge and skills they have acquired to carry out basic indoor environmental/living assessments, and recommend/implement basic indoor environmental improvement solutions; and Reflect on their role and responsibilities both as a professional and as a responsible citizen

Students will be tasked to: Carry out client survey to identify the gap to basic needs for the residents in the low-income regions; Propose recommendations on improvement of living standard in low-income families, and Carry out construction/renovation/development of engineering systems which improve the living quality in low-income regions

Since there are many moving parts to a service-learning program, the unit also includes a “Design Checklist” to scaffold learners as they flesh out the content of their subject. Our focus group studies revealed that reflection—both facilitating and assessing it—and assessment appear to be the biggest concerns from teachers. The MOOC thus includes dedicated units for these two topics. The “Reflection” unit reinforces the role and importance of reflection in service-learning, supplements it with theory from the Gibbs’ Model (Gibbs, 1988), and scaffolds teachers with concrete effective practices and learning activities and reflections taken from actual courses. Finally, the “Assessing Students” unit lays out conceptual principles for assessment, steps in designing assessment tasks and developing assessment rubrics, and also gives examples from actual service-learning courses. In addition to knowledge delivery, the MOOC makes provisions for learners to interact. As we anticipate that learners will most likely be experienced teachers, we hope to facilitate peer learning and perhaps collaboration and cross-fertilization through the interaction. The MOOC therefore provides discussion fora encouraging learners to introduce themselves, share experiences on community engagement at their institution, and reflect upon the practices and policies at their own institution. In the style of a “flipped classroom”, the course also contains assignments which are designed to encourage interaction between learners and thereby facilitate peer learning. For example, the unit on “Assessment” contains anonymized samples of actual reflective essays from students, and learners are asked to assess and grade the essays using the provided rubric, and to discuss their grades with each other. The entire MOOC is designed to take around 20 h of study effort. It can be viewed as a primer covering the critical concepts underlying service-learning. As such, learners who complete the MOOC can be considered to have an understanding of the fundamentals of service-learning.

112

G. Ngai and S. C. F. Chan

6.8 Interactive Learning—Face-to-Face Seminars Students completing the MOOC should have a good grasp of the theories underlying service-learning. However, given the nature and the duration of the MOOC, some concepts are difficult to include. Examples of these include exploratory topics, such as emerging trends in service-learning, or open-ended topics that require intensive moderation. The second level of the course was therefore designed to use interactive learning. Face-to-face seminars were used, and participants were partitioned into smaller groups for the duration of this component. The smaller groups are designed to enable and facilitate group work and discussion. Topics covered in these sessions go further in-depth into those covered by the MOOC, and some new topics are also included. For example, while the MOOC covers “Designing and Implementing Service-Learning”, the face-to-face component expands upon this with exemplars of both positive (good) and negative (poorlydesigned) proposals or examples of service-learning programs. Participants are then asked to work in groups to identify the good practices evidenced in the positive examples and the problematic situations that might be created by the negative exemplars. Likewise, the “Reflection” and “Assessment” sections were supplemented with interactive activities, for example, participants were given “borderline” reflective essays to read and grade, and discuss. The face-to-face format also allows us to cover some emerging themes. Topics that were covered in the past include discussions on critical service-learning and research and evaluation in service-learning. These are topics which are not central to service-learning in general, and therefore do not really warrant taking up the extra student study effort in the MOOC. However, they are of interest to some practitioners (in particular, research and evaluation are of interest to higher education colleagues). Therefore, they are incorporated into the seminars instead of the MOOC, which is intended for a broad-base audience. This component of the course usually lasts for two full days, with approximately ten to twelve hours of student study effort.

6.9 Experiential Learning—Experiential Projects and Class Observations The final component of the professional development course is also the most intensive (both for the students as well as for the teachers). It takes the form of immersive learning in an experiential environment, and is intended to create an intensive environment that closely mirrors what teachers and students might experience in a short-term service-learning project. This component of the course usually lasts for approximately three days to a week. The learning activities during this time include visits and conversations with NGOs

6 Promotion of Teachers’ Readiness …

113

and experienced service-learning teachers, observations of actual student projects in operation, and practice and role-playing with reflections and assessment. Before COVID-19, the final component of the course would take place in Cambodia, where the university has a two-week-long summer school program in service-learning and leadership. Since COVID-19, this component has been held in place in Hong Kong, with student project observations taking place via Zoom to adhere by social distancing regulations. This component of the course essentially draws teacher, NGO partner, and student together to play different facets of the educator role. An example are the visits to NGOs. NGOs often expect that potential service-learning teachers wish to know about ways in which they can collaborate, hence they often share about the communities that they are serving, and service gaps that they wish to address. Our experience is that it is very easy for learners to get excited by the work that the NGOs are doing and get carried away with plans for collaboration; and many times these collaborations may never be actualized. In addition, what would be useful for learners is not so much the potential for collaboration with this particular NGO, but rather, issues that they need to be aware of. In particular, it might actually be more useful for NGOs to share about bad examples of service-learning collaborations, and why those collaborations did not work! In other words, the NGOs that are visited need to play the role of a co-educator, albeit on the community side. Observational learning—bringing learners to learn from observing student projects in action—brings another set of challenges. Service project sites are very different from the familiar environment of the classroom or laboratory, and can often appear rather chaotic to the casual observer. It is therefore crucial that the learners be prepared with information about the details of the project—what the project is about, what they will see the students doing, what background preparation the students have undergone, the learning objectives that are supposed to be imparted, etc. This information will scaffold the learners and prime them as they are supposed to observe.

6.10 Construct Your Own Learning The final stage of the course encourages participants to construct their own learning, practice what is learned, and embody their own construction of service-learning. Participants are asked to develop a course or project as the activity that ties all the learning activities together. This course/project can be an existing course/project that the participant is currently involved in, which can be further refined. It can also be something new that the participant is developing. It can even be something that the participant is interested in, as an exercise. The course/project acts as an anchor, providing a context for all the discussions on myths, concepts, and challenges. It provides also a concrete target for all the work on partnerships, pedagogies, reflection, and assessment. Opportunities are provided throughout the course to relate the topic of discussion to their proposals. Toward

114

G. Ngai and S. C. F. Chan

the end, each is invited to make a presentation of their proposal, to which both participants and course teachers make comments and suggestions. The course/project proposal is a concrete manifestation of the participant’s own conceptual model of service-learning, constructed throughout the course. Since participants oftentimes come from very diverse academic disciplines, are interested in diverse target communities, and come with varying degrees of preparedness, relating the generated proposals to the course material pose diverse but serious challenges. This requires the course instructors to have substantial experience, not only in doing service-learning on their own, but also in helping other teachers to design their service-learning programs. In our case, each of our two main instructors have more than 10 years of relevant experience, and have worked for a long time as a team, in service-learning. They have been developing and teaching service-learning courses and projects in a wide range of communities and countries, as well as advising many instructors in different disciplines on developing their own courses. They have also sat on vetting committees, reviewing hundreds of course proposals. These factors combine to make the discussions among the instructors and participants potentially the most effective and fruitful elements of the course, by tying together the material as needed for participants.

6.11 Evaluation Participants were asked to fill in an online feedback form at the end of the course. The form consists of quantitative questions on a Likert scale of 1–5 (1 = strongly disagree, 3 = neutral, 5 = strongly agree). The following table shows the feedback results from the 2020 and 2021 years (a different feedback form was used prior to 2020). Most of the participants agreed or strongly agreed that the course had enabled them to – explain what service-learning was and how it differed from other forms of experiential learning or service programs (97%); – identify the benefits of SL to the students, the community, and the SL teachers (97%); – describe the salient features of an effective SL course (95%); – formulate ideas for developing an SL subject/project/assessment plan relevant to their interest and discipline (93%); and – plan and teach a service-learning course in the future (95%). After taking the course, I am able to…

mean n

a. Explain what service-learning (SL) is and 4.56 how it differs from other forms of experiential learning or service programs

Agree Neutral Disagree (4 or 5) (%) (3) (%) (1 or 2) (%)

39 97

3

0

(continued)

6 Promotion of Teachers’ Readiness …

115

(continued) After taking the course, I am able to…

mean n

b. Identify the benefits of SL to the students, the community, and the SL teachers

4.49

39 97

3

0

c. Describe the salient features of an effective SL course

4.37

39 95

5

0

d. Formulate ideas for developing an SL 4.41 subject/project/assessment plan relevant to my interest and discipline

39 93

5

3

e. Plan and teach a service-learning course in the future

39 95

5

0

4.41

Agree Neutral Disagree (4 or 5) (%) (3) (%) (1 or 2) (%)

With respect to the different components of the course, the majority of the participants found all the components useful to them. However, it should be noted that a significant percentage of the recipients were neutral on the usefulness of the cognitive learning components—the e-learning module and the interactive seminars. This suggests that even though the experiential component is more labor-intensive and expensive to operate, it affords learning outcomes that may not be met via the more large-scale components of the MOOC and the seminars. How useful was:

Mean

N

Useful (4 or 5) (%)

Neutral (3) (%)

Not useful (1 or 2) (%)

The e-learning module

4.16

39

87

11

3

The interactive seminars

4.05

39

80

21

0

16*

100

0

0

35

94

6

0

The community visit and 4.75 conversation with the NGOs The sharing sessions by the service-learning teachers, community partners, and students

4.34

*The experiential component, including the community visit and conversations with the NGOs, were not organized in 2020 because of the COVID-19 pandemic

Many of the graduates of the teacher development course have since successfully taught or proposed their own service-learning courses. Many have also continued to stay active in the service-learning community with participation in events such as conferences, workshops, symposia, and the like. Some early graduates have been returning to share their experiences with the later cohorts of participants.

6.12 Reflections and Future Work We find that through our course, we can facilitate the discussion of basic concepts as well as numerous real-life case studies. Just like our students, SL teachers learn

116

G. Ngai and S. C. F. Chan

from the rigorous studies as well as the immersive field experience that we arrange. The conversations with the NGO partners are also very valuable, as are seeing the projects live in the field. Teachers report that seeing the real situation helps them to concretize the concerns from the NGOs and think about real impacts from their own projects. The variety of NGOs and types of projects that we arrange (ranging from engineering to language to health) provide a lot of stimulation for the teachers in developing their own SL teaching subjects and projects. The service-learning scene in Hong Kong has changed a lot since the teacher development course was first started. At that point, we were the only institution that was doing service-learning as a graduation requirement and our primary objective was to train up enough teachers so that there would be enough subjects for our own students. Since then, other universities have started academic service-learning programs and our cohort of learners expanded to colleagues from other universities. They brought along with them a different culture of learning and a different set of requirements specific to their institution. We were also joined by colleagues from tertiary institutions in mainland China and South-East Asia (Philippines and Vietnam). The expansion of the participants helped to clarify the “core” of academic servicelearning and to identify the critical aspects; in particular, participants from outside Hong Kong have offered unique insights of their own. Going forward, it may be beneficial to study service-learning from an Asian perspective, so as to identify, or develop, aspects of service-learning pedagogy which will be more appropriate for students from this part of the world.

References Ali, A. M. (1981). The use of positive and negative examples during instruction: Some important issues related to the design and development of instructional materials. Journal of Instructional Development, 5(1), 2–7. http://www.jstor.org/stable/30220664 Antonio, A. L., Astin, H. S., & Cress, C. M. (2000). Community service in higher education: A look at the nation’s faculty. Review of Higher Education, 23(4), 373–398. https://doi.org/10.1353/rhe. 2000.0015 Billig, S., Root, S., & Jesse, D. (2005). The relationship between the quality indicators of servicelearning and student outcomes: Testing professional wisdom. Advances in Service-learning Research: Improving Service-learning Practice: Research on Models to Enhance Impacts, 97–115. Camus, R. M., Ngai, G., Kwan, K. P., & Chan, S. C. F. (In Submission). Transforming teaching: Service-learning’s impact on faculty. Camus, R. M., Ngai, G., Kwan, K. P., Yau, J. H. Y., & Chan, S. (2021). Knowing where we Stand. Mapping Teachers’ Conception of Reflection in Service-Learning, 46(3), 285–302. https://doi. org/10.1007/s10755-020-09534-6 Chan, S. C. F., Ngai, G., & Kwan, K. P. (2019). Mandatory service learning at university: Do lessinclined students learn from it? Active Learning in Higher Education, 20(3), 189–202. https:// doi.org/10.1177/1469787417742019

6 Promotion of Teachers’ Readiness …

117

Chan, S. C. F., Ngai, G., Lam, C. H. Y., & Kwan, K. P. (2021). How participation affects university students’ perspectives toward mandatory service-learning. Journal of Experiential Education, 44(2), 137–151. https://doi.org/10.1177/1053825920948889 Gibbs, G. (1988). Learning by doing: A guide to teaching and learning methods. Howard, J. (2001a). Principles of good practice for service-learning pedagogy. Michigan Journal of Community Service Learning. Howard, J. (2001b). Service-learning course design workbook. Michigan Journal of Community Service Learning. http://hdl.handle.net/2027/spo.3239521.spec.201 Knowles, M. (1984). The adult learner: A neglected species. Gulf Publishing Company ($7.95). Kolb, D. A. (1984). Experiential learning: experience as the source of learning and development. Prentice-Hall. Essential elements of service-learning, (1998). Ngai, G., Chan, S. C. F., & Kwan, K. P. (2018). Challenge, meaning, interest, and preparation. Critical success factors influencing student learning outcomes from service-learning, 22(4), 55– 80. http://www.scopus.com/inward/record.url?scp=85059743039&partnerID=8YFLogxK Shek, D. T. L., & Chan, S. C. F. (2013). Service-learning from the views of university teachers: A qualitative study based on focus groups. International Journal of Adolescent Medicine and Health, 25(4). Thomson, A. M., Smith-Tolken, A. R., Naidoo, A. V., & Bringle, R. G. (2011). Service learning and community engagement: A comparison of three national contexts. VOLUNTAS: International Journal of Voluntary and Nonprofit Organizations, 22(2), 214–237. https://doi.org/10.1007/s11 266-010-9133-9 Wenger, E. (2013). Communities of Practice: Learning, meaning, and identity. Cambridge University Press.

Chapter 7

Conceptual Framework for Key Process Ingredients Salient for Effective Service-Learning Robin Stanley Snell and Ka Hing Lau

Abstract Although service-learning is an established pedagogy, widely adopted in tertiary education around the world, the nature of the key ingredients for successful service-learning experience in terms of yielding positive developmental outcomes for students has not been fully discerned. In this chapter, we analyse previous literature to identify six key ingredients typically found in successful service-learning experience. First, students provide meaningful service, using academic knowledge to address real needs. Second, the partner organisation representative (POR) plays a positive role in supporting the project. Third, there is thorough preparation and support for students, who receive appropriate orientation, training, and consultation. Fourth, there is effective reflection by students, supported by exercises that help them recall and make sense of surprises and key choice points. Fifth, there is effective course design with integration of service-learning. Sixth, there is stakeholder synergy in terms of collaboration, communication, and co-ownership, as well as reciprocity. Looking ahead to future research, we argue that it is essential to obtain inter-subjectively fair and trustworthy data about the extent to which these six key ingredients, plus any additional success factors, are perceived to have been achieved. Accordingly, we propose a multi-stakeholder approach for data collection. Possible cultural influences when applying these ingredients in the Chinese context are discussed. Keywords Service-learning experience · Process variables · Assessment · Outcomes · Multi-stakeholder approach

R. S. Snell (B) Department of Management, The Hang Seng University of Hong Kong, Siu Lek Yuen, Hong Kong e-mail: [email protected] K. H. Lau Office of Service-Learning, Lingnan University, Tuen Mun, Hong Kong e-mail: [email protected] Service-Learning and Leadership Office, The Hong Kong Polytechnic University, Kowloon, Hong Kong © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_7

119

120

R. S. Snell and K. H. Lau

7.1 Introduction Service-learning, a form of experiential learning which aims to enable and empower students to apply knowledge learnt in class to serve people and/or organisations in a community setting, has been adopted by a number of higher educational institutions in Hong Kong over the past decade (Ma, 2018; Snell & Lau, 2020; Xing & Ma, 2010). In adopting service-learning pedagogy, these institutions appear to be adhering to the original objective of service-learning, defined as “a form of experiential education in which students engage in activities that address human and community needs together with structured opportunities intentionally designed to promote student learning and development” (Jacoby, 1996, p. 5). A broad range of intended developmental outcomes for students have been invoked. For example, Lingnan University’s service-learning programme aims to enhance seven graduate attributes, including problem-solving skills, critical thinking skills, and social competency (Ma & Chan, 2013). Also, the Hong Kong Polytechnic University launched its service-learning programme in 2010 to facilitate four developmental outcomes: (a) knowledge and skill application; (b) empathy, civic engagement, and responsibility; (c) becoming professional and responsible citizens; and (d) connecting between the academic content and the need of society (Chan & Ngai, 2014). The developmental outcomes arising from service-learning for students have been extensively researched and documented (Astin et al., 2006; Celio et al., 2011; Conway et al., 2009; Eyler et al., 2001; Salam et al., 2019; Warren, 2012; Yorio & Ye, 2012). However, the key arrangements for and processes of implementing successful service-learning, which may lead to positive developmental outcomes for students, have not been as systematically investigated, particularly in Hong Kong. The current paper therefore builds an analytical framework based on the past literature, encompassing three major parts, namely: (a) relevant literature documenting factors for effective service-learning; (b) prior research on good practices that are conducive to student learning outcomes in quality service-learning; and (c) relevant research by Hong Kong based researchers to provide local insights. The paper goes on to propose a measurement instrument based on the key ingredients that are associated with successful service-learning experience, so as to bridge a gap in servicelearning research in Hong Kong and beyond. In this chapter, by successful servicelearning experience, we mean that the associated project has been able to achieve its intended student developmental outcomes. Although we recognise that servicelearning should also create beneficial impacts for the community partners and servicerecipients, due to the limited length of this chapter, we shall take student development as our focal criterion for effectiveness.

7 Conceptual Framework for Key Process …

121

7.2 Literature Review 7.2.1 Key Ingredients for Effective Service-Learning Proposed in “Classic” Literature First we shall summarise Mintz and Hesser (1996) review of earlier literature on principles of and good practices in service-learning. The major principles and practices that they noted comprised: meaningful action; empowerment of those serving and those being served; structured reflection opportunities; training and support; evaluation and assessment; community voice; and continuous improvement. They further proposed a service-learning “kaleidoscope” with three lenses, or meta-principles, for examining the extent to which the above service-learning principles have been put into practice for a given service-learning project. The first lens is collaboration, which involves common goals and purposes, including shared responsibility and authority, sharing of resources, and mutual trust, so as to enhance the capacity of each stakeholder to contribute. The second lens is reciprocity, under which each stakeholder functions both as a learner and as an educator, thereby avoiding exploitation. The third lens is diversity in the servicelearning context, which enables all stakeholders to appreciate and respect differences and not be confined by their own perspective. Mintz and Hesser (1996) claimed that if the three meta-principles were to be adopted, the stakeholders of service-learning, including instructors, students, and community partners would be able to apply the major principles and practices of effective service-learning. Stakeholders would thus be able to ensure that the projects would: achieve their goals; meet genuine community needs; bring benefits to both students and the community; respect differences and diversity through reciprocal collaboration; and be subject to ongoing evaluation and continuous improvement. Eyler and Giles (1999) reported findings from a research study that gathered data from interviews with students regarding the factors perceived to have made their service-learning effective. They distinguished five programme characteristics that are predictors of effective service-learning outcomes. These were: (a) placement (or service) quality; (b) quality of knowledge application; (c) inducement of effective student reflection; (d) exposure for students to diverse groups; and (e) influence of community voices. Eyler and Giles (1999) argued that although the above five characteristics were in most cases positive predictors of students’ development, there were sometimes exceptions regarding diversity and community voices. These could turn out to be negative predictors if unresolved tensions were to arise from differences between stakeholders in terms of their expectations and interests. Such problems imply that it is important to ensure that stakeholders’ expectations are aligned before and throughout the service-learning intervention, and that stakeholders perceive that their collaboration is a source of mutual benefits. They argued, further, that with these provisos, the five above-mentioned programme characteristics would result in the achievement of desired student developmental outcomes, including better tolerance, personal

122

R. S. Snell and K. H. Lau

and interpersonal development, citizenship, enhanced problem-solving and critical thinking skills, and perspective transformation. In addition, Eyler and Giles (1999) also invoked the “Five Cs” principles for effective service-learning engagement and reflection for students: Connection, Continuity, Context, Challenge, and Coaching. Godfrey et al. (2005) reviewed prior literature that identified three critical ingredients for successful service-learning and characterised them as the “Three Rs”. The first of these is Reality, in terms of how service-learning can connect real situations with academic content. The second is Reflection, which enables students to think deeply about their service-learning experience and its personal impact on them. The third is Reciprocity, which involves the students and the community contributing and combining their different bodies of knowledge and working together so that synergy occurs as both parties gain from their collaboration. Godfrey et al. (2005) added a fourth R, Responsibility, in the context of businessrelated courses. By this, they referred to the need for professionalism and virtue in applying business techniques, skills, and knowledge for the enhancement of the communities that are being served. They argued that the “Four Rs” can overcome the narrowness in business education and bring students with broader education experiences in successful service-learning projects. Godfrey et al. (2005) also prescribed four additional principles for faculty to embrace as means for successful service-learning experiences. The first of these is Centrality, with service-learning activities closely tied to course objectives. The second is Commitment, which underscores faculty members’ role and requires them to invest time, energy, and work in creating meaningful service-learning experience. The third is Community Engagement, viewing community agencies and servicerecipients as partners for mutual benefit. The fourth is Continuous Improvement, requiring self-reflection by faculty so that the efficiency and quality of their servicelearning projects in yielding developmental outcomes for students can be improved over time. Among the sources that we have summarised thus far, there appear to be five process variables conducive to student developmental outcomes that are commonly alluded to. These are: quality service that meets genuine community and service recipient needs; deep reflection on service experience; reciprocity among stakeholders; inclusion of both student and community voices; and exposure to diversity.

7.2.2 Good Service-Learning Practices Proposed in Past Literature on K-12 and Youth Leadership Education We consider that a number of insights from K-12 (i.e. high school) and youth leadership education are also salient for effective academic service-learning and below we review some key sources. Youth Service California (2006) advocated seven elements of high-quality service-learning. These are itemised below.

7 Conceptual Framework for Key Process …

123

• Integrated learning. This means that in service-learning, the academic curriculum should be connected with the service, with curriculum goals clearly articulating knowledge, skills, and value that are to be learned both inside and outside the classroom. • High-quality service. This should address real community needs and should be well-organised, supported by ongoing communication between educational institutions and community partners, to achieve significant benefits for both sides. • Collaboration. All stakeholders, i.e. students, community partners, faculty members, and service-recipients should be involved in planning, executing, and evaluating the service and should all gain benefits from the collaboration. • Reflection. This should be held before, during, and after the service-learning project, with the aims of helping students to connect their service experiences with the academic curriculum and of engaging in deeper critical thinking. • Civic responsibility. This should be a major goal of service-learning and is concerned with caring for others and contributing to the community. • Evaluation. All stakeholders should be involved in the processes of assessing progress towards the learning goals and the service goals. • Student voice. This means that students should play active roles in choosing and planning service-learning projects. The National Youth Leadership Council (2008) released a set of good practices in service-learning, considered as conducive for positive developmental outcomes for K-12 students. These practices had previously been recommended by Billig (2007), based on analysis of statistical evidence. They comprise the following. • Link to curriculum or curriculum integration. This means that service-learning projects should be designed to explicitly link to curricular goals and course content standards. • Meaningful service, which enables students both to demonstrate intended learning outcomes and to meet actual community needs through the projects. The design of the service should be tailored to student age and ability and should carry a sense of purpose, connection, relevance, and usefulness. • Cognitively challenging reflection. This should occur during all phases of the service-learning project. It should involve various activities that enable students to connect knowledge with service experiences, as well as challenging students to examine their preconceptions and assumptions. • Respect for diversity. This should involve developing understanding and mutual respect for multiple perspectives and different points of view. Service-learning projects should therefore provide opportunities for students to recognise and overcome stereotypes. • Reciprocal partnerships. All stakeholders in service-learning projects, i.e. students, educators, community partners, and community members, should collaborate throughout the project on the basis of reciprocity, for mutual benefit. • Appropriate duration and intensity. There should be a sufficiently long and sufficiently concentrated block of time occupied by the service-learning project.

124

R. S. Snell and K. H. Lau

• Youth voice. Students, under guidance, should have inputs into the processes of creating, implementing, and evaluating the service-learning project. • Progress- and process-monitoring. This is defined as gathering information to check the progress of the service-learning project. It should include demonstrating the attainment of intended learning outcomes and pinpointing areas for improvement. It can involve using simple tools, such as rubrics for assessing the quality of student work.

7.2.3 Key Ingredients Identified in Two Hong Kong Based Service-Learning Studies We shall next review two recent studies by Hong Kong based researchers. The first of these was conducted by Chen et al. (2018). They identified three interrelated concepts in addition to the above-mentioned “Three Rs” of Godfrey et al. (2005), which distinguish service-learning from non-service-learning. The three concepts are project experiences, partner organisation representative (POR) responsiveness, and project efficacy belief, and are explained below. Project experiences in service-learning that set them apart from non-servicelearning typically involve serving or liaising with community organisations rather than (for business students) corporations. They also involve support from a dedicated service-learning office or unit, and a communication network that connects these stakeholders. In today’s university settings, such support is mainly provided by a centralised unit, such as the Office of Service-learning of Lingnan University. POR responsiveness is the extent to which PORs offer open access to people, resources, and information that are essential for furthering the project. Project efficacy belief refers to the extent to which students believe that through service-learning they are bringing about or will be able to bring about positive impacts for community beneficiaries and/or the community partner organisation(s) (CPOs). Besides the “Three Rs”, three additional sources of project efficacy belief among students comprise: a sense of mastery experience; perception of enacting effective social persuasion; and positive role modelling by other students engaged in the same service-learning project. In their study, Chen et al. (2018) showed that project efficacy belief mediated the effect of service-learning project experience on student development, whereas POR responsiveness moderated the indirect effect of service-learning project experiences on student development. Through these pathways, students can achieve developmental outcomes, in terms of emerging service leadership attributes and meaning schema transformation, the latter being a milder form of perspective transformation. The findings of Chen et al. (2018) are consistent with an earlier paper by Snell et al. (2015), who derived a set of ten principles for service-learning based on a qualitative study in Hong Kong. The first of these is that service-learning projects should address authentic problems or needs. Second, PORs should commit to being available for consultation throughout the project. Third, students should receive a

7 Conceptual Framework for Key Process …

125

complete orientation to service-learning. Fourth, project themes should align with the course curriculum. Fifth, there should be a foundation of inter-institutional commitment and trust. Sixth, there should be initial site visits by students prior to the main project phase. Seventh, there should be in-class project consultations during servicelearning. Eighth, there should be shared and supportive leadership within the student project team. Ninth, instructors should provide templates to help students to reflect on the processes and outcomes of their personal development. Tenth, project reports and reflective reports should be included among the graded coursework assessment requirements. Including the above ten principles as part of a road map, Snell et al. (2015) pointed out that several desirable outcomes can be achieved through service-learning. These outcomes include: that students and PORs should perceive that good service has been delivered; that students should attain the course level learning goals; and that the students should be able to explain how they practised and developed as emerging service leaders.

7.2.4 Large-Scale Survey Research on Key Process Variables for Successful Service-Learning Large-scale survey research has yielded findings about key process variables that are broadly similar to those reported in the above sections. For example, Astin et al. (2000) in a survey of more than 20,000 college students identified five main factors that were associated with positive service-learning outcomes. These were: interesting service, connection between service experience and course subject matter, amount of training prior to service, and good reflection. Another study of over 1,000 high school students participating in service-learning in the United States, conducted by Billig et al. (2005) identified nine elements of service-learning practice that were associated with positive student developmental outcomes. These were: • • • • • • • • •

clear educational goals, cognitively challenging tasks, use of assessment and evaluation, service meeting genuine community needs, valuing of diversity, promoting interaction with the community, good preparation for students prior to service, regular reflection, and appreciation of students’ service.

Ngai et al. (2018) analysed data of over 2,000 students from service-learning courses of The Hong Kong Polytechnic University. They found that positive

126

R. S. Snell and K. H. Lau

service-learning outcomes were associated with five factors. These were: challenging and meaningful tasks; interest in the service project; perceived benefits for service-recipients, service preparation; and appreciation of service by service-recipients. Some research studies have sought to determine the sufficient duration and intensity of service-learning projects. For example, Mabry (1998) suggested that servicelearning is more effective if students engage in at least 15 to 20 hours of service with frequent contact with service-recipients and regular reflection. A national survey of over 3,000 youths in the United States (Spring et al., 2006) indicated that the service duration needed to be at least one semester long, including preparation, service, reflection, and presentation of findings.

7.3 Process Variables as Key Ingredients for Successful Service-Learning Experiences Based on the above literature review, we consider that the key ingredients that contribute to successful service-learning outcomes can be analysed into six overall sets of ingredients. The first of these is meaningful service. This comprises significant action (for students in service), inclusion of community voices in establishing service needs, and the experience for students of encountering social diversity during the service. The second is that the POR plays a constructive role by, for example, being readily available for, and responsive to demands requested by other stakeholders in the entire process. The third is that the students receive effective preparation and support, so that they are sufficiently well informed and trained to carry out the planned service in the field to achieve the expected outcomes agreed upon by all stakeholders. This includes domain-specific training and orientation for service, consultation during the service-learning project, and logistical assistance and support by the instructor and the school. The fourth is engagement in effective reflection as a means for connecting the student’s service experiences to the course curriculum. The fifth is effective course design, such that project themes are closely linked to the course curriculum, project duration is long enough for such course integration to happen, and that student’s project reports and reflections form an integral part of the assessment requirements for the course. The sixth is stakeholder synergy. This is based on effective collaboration, communication, and co-ownership, with student voice adequately included. There is reciprocity between stakeholders in terms of resource commitments and derived benefits. Table 7.1 summarises these six key sets of ingredients, and how they match with the literature reviewed in the previous sections.

7 Conceptual Framework for Key Process …

127

Table 7.1 A summary analysis of the six key ingredients Key ingredients

Reference Sources

Meaningful service

Significant action: 1. Meaningful action (Mintz & Hesser, 1996) 2. Addressing authentic problems (Snell et al., 2015) 3. Reality (Godfrey et al., 2005) 4. Project efficacy belief (Chen et al., 2018) 5. Meaningful service (Billig, 2007; National Youth Leadership Council, 2008) 6. High-quality service (Youth Service California, 2006) 7. Service quality (Eyler & Giles, 1999) 8. Service promoting civic responsibility (Youth Service California, 2006) Inclusion of community voices: 9. Community voices are included (Mintz & Hesser, 1996) 10. Influence of community voices (Eyler & Giles, 1999) Diversity being experienced in service: 11. Diversity in service (Eyler & Giles, 1999) 12. Service promoting respect for diversity (Billig, 2007; Mintz & Hesser, 1996; National Youth Leadership Council, 2008)

POR plays a constructive role

1. POR responsiveness (Chen et al., 2018) 2. POR commitment (Snell et al., 2015)

Effective preparation and support provided to students

1. Sufficient support, coordination, orientation & training (Mintz & Hesser, 1996) 2. Project experiences (i.e. support and connection with community organisations (Chen et al., 2018) 3. In-class project consultation, orientation, and initial site visits (Snell et al., 2015)

Effective reflection

1. Effective reflective activities (Mintz & Hesser, 1996) 2. Effective student reflection (Eyler & Giles, 1999) 3. Reflection (Godfrey et al., 2005) 4. Cognitively challenging reflection (Billig, 2007; Godfrey et al., 2005; National Youth Leadership Council, 2008; Youth Service California, 2006) 5. Measures to enhance student reflection (Snell et al., 2015) (continued)

128

R. S. Snell and K. H. Lau

Table 7.1 (continued) Key ingredients

Reference Sources

Effective course design

Service-learning project design: 1. Service aligning with course curriculum (Billig, 2007; National Youth Leadership Council, 2008; Snell et al., 2015) 2. Integrated learning (Youth Service California, 2006) 3. Quality of knowledge application (Eyler & Giles, 1999) 4. Responsibility, i.e. the need for professionalism and virtue in applying business techniques, skills, and knowledge for enhancing the community (Godfrey et al., 2005) Duration: 5. Appropriate duration (Billig, 2007; National Youth Leadership Council, 2008) Evaluation and monitoring: 6. Effective evaluation (Mintz & Hesser, 1996; Youth Service California, 2006) 7. Progress- and process-monitoring (Billig, 2007; National Youth Leadership Council, 2008) 8. Continuous improvement (Mintz & Hesser, 1996) 9. Grading service-learning project reports (Snell et al., 2015)

Stakeholder synergy

1. Common goals, purposes, responsibility & resources (Mintz & Hesser, 1996) 2. Reciprocity (Godfrey et al., 2005; Mintz & Hesser, 1996) 3. Reciprocal partnership (Billig, 2007; National Youth Leadership Council, 2008) 4. Collaboration (Youth Service California, 2006) 5. Inter-institutional commitment and trust (Snell et al., 2015) 6. Student/youth voice (Billig, 2007; National Youth Leadership Council, 2008; Youth Service California, 2006) 7. Shared and supportive leadership within student teams (Snell et al., 2015)

7.4 Cultural Considerations in Chinese Contexts We have identified six common sets of ingredients for successful service-learning experiences, and we believe that the requisites for planning and implementing service-learning projects in Western and Chinese contexts are similar, reflecting the

7 Conceptual Framework for Key Process …

129

inclusion of some Hong Kong based studies in our review. We consider, nonetheless, based on a comparison by Snell and Lau (2020) of the goals and approaches of service-learning in Hong Kong with those in the west (mainly the USA), that some adaptations may be needed when applying the six sets of ingredients in Chinese contexts, reflecting the following four factors. First, Chinese institutions are likely to prefer a relatively de-politicised approach to student development via service-learning, focusing on student’ moral development rather than on fostering their awareness of and participation in building democracy. Second, Chinese culture places greater emphasis on one’s self-cultivation and selfimprovement. This factor, along with the first, may have implications for student reflection, which in Chinese contexts may be more inwardly focused and less concerned with issues of power imbalance and social injustice than in the West. Third, as compared to the West, Chinese education appears to place even greater emphasis on developing professional skills as a pathway to upward social mobility, resulting in relatively vocationally oriented skill development in service-learning. This stronger emphasis may overshadow considerations of respecting diversity, advocated by many service-learning practitioners in the West. It may also lead to interpretations of the nature of “meaningful service” that focus more on whether service-learning can help students advance in their professional development, than on furthering well-being in the community. Fourth, given that Chinese societies tend to assume high power distance (Hofstede, 1983), we expect that service-learning in Chinese contexts may tend to be more driven by faculty members and the respective university, and that the extent to which voices of students and grassroots participants are taken into account may be less than in the West. Our above analysis implies that it is possible that cross-cultural adjustments may be required when ascertaining the relative importance of the six sets of ingredients in determining the extent to which service-learning projects lead to successful developmental outcomes. Further research is needed to clarify the picture.

7.5 A Multi-Stakeholder Approach for Assessing the Key Ingredients Effective service-learning requires close collaboration among the stakeholders, who comprise students, instructors, PORs, end-beneficiaries of the service, and the coordinating centre, which is typically an office of service-learning (Wade, 1997). Accordingly, in further research into the key ingredients of effective service-learning, including cross-cultural issues such as those noted above, we recommend that data are sought from multiple stakeholders. Since the salient sets of ingredients appear to have already been established, we consider that mainly quantitative data collection methods would be employed, supplemented where appropriate by qualitative methods, paying attention to any special cultural issues.

130

R. S. Snell and K. H. Lau

Table 7.2 lists a set of sample items that could be used in measuring stakeholders’ perceptions about the relative presence or absence of the key ingredients that were summarised in Table 7.1. For example, ingredients related to the extent to which the POR has played a constructive role would be answered by instructors and students, whereas questions about whether students engaged in effective reflection would be answered by instructors and PORs. A limitation of Table 7.2 is that the “answered by” columns omit one important stakeholder group, the end-beneficiaries, who Table 7.2 A framework for assessing the presence of key ingredients for effective service-learning Process aspect/variable

Sample items

Answered by* I

S

C

(a) Meaningful service 1. Significant action

(i) The service was aligned with the Y real needs of our service-recipients (ii) The service providers (students) were able to address the concerns of our service-recipients (iii) The service providers (students) were able to make a positive contribution for our service-recipients (iv) The service providers (students) were able to help the CPO to improve its service

Y

Y

2. Inclusion of community voices

(v) The planning of the service was informed by community voices (vi) The execution of the service involved community voices (such as in adjustment)

Y

Y

Y

3. Diversity being experienced in service

(vii) The service enabled students to interact with people from different backgrounds

Y

Y

Y

(b) POR plays a constructive role, i.e. whether they: 1. Responsive

(i) responded helpfully to questions and enquiries

Y

Y

2. Available

(ii) were available when needed

Y

Y

3. Taking up a positive role

(iii) provided constructive feedback on ideas and suggestions

Y

Y

(c) Effective preparation and support provided to students, including: 1. Training

(i) training for providing the service

Y

Y

Y

2. Orientation

(ii) orientation about service-learning Y (iii) orientation about the CPO(s)

Y

Y

3. Consultation in the process

(iv) consultation in the process when difficulties arose

Y

Y

4. Support by the instructor

(v) support by the instructor

Y

Y

Y

(continued)

7 Conceptual Framework for Key Process …

131

Table 7.2 (continued) Process aspect/variable

Sample items

Answered by* I

5. Support by the school

(vi) support by the school (such as Y Office of Service-Learning) (vii) support to prevent or resolve any problem of free riding in the student team

(d) Effective reflection

(i) The students conducting the Y service were able to perform effective reflection on their service-learning experience

S

C

Y

Y

Y

(e) Effective course design 1. Project themes integrating with (i) The service was well-matched Y course curricula without compromise with the course curriculum (ii) The service was closely aligned with the academic goals and topics of the course

Y

2. Project duration is long enough for (iii) The service-learning project course integration to happen duration is long enough for effective learning and service

Y

Y

(iv) The course made the student’s Y service project reports and reflections an integral part of the assessment

Y

(i) All stakeholders collaborated well during the service (ii) All stakeholders communicated well with each other during the service (iii) All stakeholders shared ownership of the desired service outcomes (iv) All stakeholders were committed to achieving the desired service outcomes (v) All stakeholders were willing to go the “extra mile” for the service (vi) All stakeholders were able to think “outside the box”, and apply questioning insight

Y

3. Making student’s service-learning reports part of assessment

Y

(f) Stakeholder synergy 1. Collaboration, communication, and co-ownership

(vii) Student voice is included in the service project, from preparation to implementation and outcome evaluation

Y

Y

Y

(continued)

132

R. S. Snell and K. H. Lau

Table 7.2 (continued) Process aspect/variable

Sample items

Answered by* I

2. Reciprocity

(viii) During the service, the Y end-beneficiary or beneficiaries was/were able to learn from other stakeholders, such as the student, the instructor, and the CPO(s) (ix) During the service, the students were able to learn from other stakeholders, such as the end-beneficiaries, the instructor, and the CPO(s) (x) During the service, the instructor was able to learn from other stakeholders, such as the students, the end-beneficiaries, and the CPO(s) (xi) During the service, the CPO(s) was/were able to learn from other stakeholders, such as the student, the end-beneficiaries, and the instructor

S

C

Y

Y

Note * I: Instructors; S: Students; C: PORs

might directly receive services from the students or who might benefit from service improvements made by the CPO as a result of the service-learning intervention. The perceptions of end-beneficiaries should not be overlooked, as their levels of satisfaction with services received can be regarded as key indicators of the effectiveness of a service-learning programme. However, for various reasons such as limited time availability, data protection, or difficulty in maintaining contact, in many cases it may not be practicable to collect data directly from end-beneficiaries. We suggest two pragmatic solutions. First, it may be possible to conduct in-depth qualitative interviews or focus groups with those end-beneficiaries, who can be accessed, about their assessments of the extent to which the services they received met their needs, and about their perceptions and attributions regarding the reasons for their satisfaction/ dissatisfaction. Second, the POR may serve as a channel for conveying the assessments, perceptions, and attributions of the end-beneficiaries. In addition to survey methods, focus groups and in-depth interviews may also be used to gather data from instructors, students and PORs. They may play a formative role prior to the planning and execution stages of a service-learning project in helping to ensure the initial alignment of stakeholder expectations. They may be especially useful for identifying process issues with newly launched service-learning programmes.

7 Conceptual Framework for Key Process …

133

7.6 Conclusions Based on a review of prior literature, this paper has identified six sets of key ingredients for successful service-learning experiences and a framework to guide the assessment of the extent to which these are present. These six sets of ingredients comprise: • • • • •

meaningful service; a constructive role for the POR; effective preparation and support for the students; effective reflection by the students; effective course design that integrates the service-learning project with the course curriculum and academic assignments; and • stakeholder synergy in terms of collaboration, communication, co-ownership, and reciprocity. We have noted that cultural differences may to some extent alter the picture in Asian contexts. More comparative studies are needed to reveal the influence of cultural and institutional expectations. Another limitation of our analysis is that we have framed “successful” service-learning experiences in terms of achieving student developmental outcomes, reflecting a general bias in the existing literature, which has tended to pay less attention to community impacts. We suggest that further research should place greater emphasis on community impacts as dependent variables. We have proposed a multi-stakeholder approach for data collection, where perceptions are collected from students, instructors, and PORs. If opportunities for access to the end-beneficiaries are restricted, we have suggested that PORs can serve as proxies or channels for feedback. To supplement survey methods, we recommend focus groups and in-depth interviews. These can be especially useful in helping to align stakeholders’ expectations prior to the launching of new service-learning programmes, and in identifying important process variables affecting the quality thereof. In summary, we have suggested a general approach for research that seeks to validate and if necessary refine our framework of six sets of key process ingredients. First, a researcher may conduct an exploratory study, seeking viewpoints from various stakeholders about the factors, which they perceive to be salient in supporting or bringing about effective service-learning experiences. Second, a researcher can construct measurement instruments for assessing the extent to which each process ingredient from our framework is present, including additional items identified in any exploratory study. Process measurement instruments are themselves subject to investigation regarding their sensitivity in detecting stakeholders’ key concerns and their effectiveness in accounting for the relative success or failure of service-learning projects to achieve their intended outcomes.

134

R. S. Snell and K. H. Lau

Author Note An earlier version of this paper was presented at the conference “6th International Conference on Higher Education Advances (HEAd’20)”, which was convened by the Universitat Politècnica de València (UPV), Valencia, Spain, in June 2020. This paper results from a cross-institutional project named “Cross-institutional Capacity Building for Service-Learning in Hong Kong Higher Education Institutions (PolyU4/T&L/16-19)”, aiming at enhancing and supporting the development of service-learning as an effective pedagogical strategy under the collaboration of Lingnan University, The Hong Kong Polytechnic University, Hong Kong Baptist University, and The Education University of Hong Kong. The project was launched in 2017 and has been funded by the University Grants Committee (UGC) of the HKSAR government. The authors wish to thank the UGC for funding the project, and the above institutions for their participation in the process.

References Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service learning affects students. Higher Education Research Institute, UCLA. Astin, A. W., Vogelgesang, L. J., Misa, K., Anderson, J., Denson, N., Jayakumar, U., Saenz, V., & Yamamura, E. (2006). Understanding the effects of service-learning: A study of students and faculty. Report to the Atlantic Philanthropies, 1155. Billig, S. (2007). Unpacking what works in service-learning. Promising research-based practices to improve student outcomes. In J. Kielsmeier, M. Neal, & N. Schultz (Eds.), Growing to Greatness 2007: The State of Service-Learning (pp. 18–28). National Youth Leadership Council. Billig, S., Root, S., & Jesse, D. (2005). Impact of participation in service-learning on high school students’ civic and academic engagement. RMC Research Corporation. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164–181. Chan, S. C. F., & Ngai, G. (2014). Service-learning as a core academic component in undergraduate programs-a brief introduction to the Hong Kong Polytechnic University model. The Journal of Development Communication, 25(1), 84–99. Chen, T., Snell, R. S., & Wu, C. X. (2018). Comparing the effects of service-learning versus nonservice-learning project experiences on service leadership emergence and meaning schema transformation. Academy of Management Learning & Education, 17(4), 474–495. Conway, J. M., Amel, E. L., & Gerwien, D. P. (2009). Teaching and learning in the social context: A meta-analysis of service learning’s effects on academic, personal, social, and citizenship outcomes. Teaching of Psychology, 36(4), 233–245. Eyler, J., & Giles, D. E. (1999). Where’s the learning in service-learning? Jossey-Bass. Eyler, J., Giles, D. E., Stenson, C. M., & Gray, C. J. (2001). At a glance: What we know about the effects of service-learning on college students, faculty, institutions and communities, 1993–2000 (3rd ed.). Vanderbilt University. Godfrey, P. C., Illes, L. M., & Berry, G. R. (2005). Creating breadth in business education through service-learning. Academy of Management Learning & Education, 4(3), 309–323. Hofstede, G. (1983). Dimensions of national cultures in fifty countries and three regions. In J. B. Deregowski, S. Dziurawiec, & R. C. Annis (Eds.), Expiscations in cross-cultural psychology (pp. 335–355). Swets and Zeitlingers.

7 Conceptual Framework for Key Process …

135

Jacoby, B. (1996). Service-learning in higher education: Concepts and practices. Jossey-Bass. Ma, C. (2018). Service-learning development in higher education in Hong Kong. In T.-W. Lim & T. Y. Kong (Eds.), Studying Hong Kong: 20 years of political, economic and social developments (pp. 43–61). World Scientific. Ma, C. H. K., & Chan, A. C. M. (2013). A Hong Kong university first: Establishing service-learning as an academic credit-bearing subject. Gateways: International Journal of Community Research and Engagement, 6, 178–198. Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter. Michigan Journal of Community Service Learning, 5(1), 32–47. Mintz, S. D., & Hesser, G. W. (1996). Principles of good practice in service-learning. In B. Jacoby (Ed.), Service-learning in higher education: Concepts and practices (pp. 26–52). Jossey-Bass. National Youth Leadership Council. (2008). K-12 standards and indicators of high quality service learning. National Youth Leadership Council. Retrieved May 24, 2021, from https://www.nylc. org/page/standards Ngai, G., Chan, S. C., & Kwan, K. P. (2018). Challenge, meaning and preparation: Critical success factors influencing student learning outcomes from service-learning. Journal of Higher Education Outreach and Engagement, 22(4), 55–80. Salam, M., Iskandar, D. N. A., Ibrahim, D. H. A., & Farooq, M. S. (2019). Service learning in higher education: A systematic literature review. Asia Pacific Education Review, 20(4), 573–593. https://doi.org/10.1007/s12564-019-09580-6 Snell, R. S., Chan, M. Y. L., Ma, C. H. K., & Chan, C. K. M. (2015). A road map for empowering undergraduates to practice service leadership through service-learning in teams. Journal of Management Education, 39(3), 372–399. Snell, R. S., & Lau, K. H. (2020). The development of a service-learning outcomes measurement scale (S-LOMS). Metropolitan Universities, 31(1), 44–77. https://doi.org/10.18060/23258 Spring, K., Dietz, N., & Grimm, R. J. (2006). Youth helping America—Educating for active citizenship: Service-learning, school-based service and youth civic engagement. Wade, R. C. (1997). Community service-learning: A guide to including service in the public school curriculum. State University of New York Press. Warren, J. L. (2012). Does service-learning increase student learning?: A meta-analysis. Michigan Journal of Community Service Learning, 18(2), 56–61. Xing, J., & Ma, C. H. K. (2010). Service-learning in Asia: Curricular models and practices (Vol. 1). Hong Kong University Press. https://doi.org/10.5790/hongkong/9789888028467.001.0001 Yorio, P. L., & Ye, F. (2012). A meta-analysis on the effects of service-learning on the social, personal, and cognitive outcomes of learning. Academy of Management Learning & Education, 11(1), 9–27. Youth Service California. (2006). Seven elements of high quality service-learning. Retrieved August 12, 2021, from https://usm.maine.edu/sites/default/files/service-learning-volunteering/ 7%20Elements%20of%20High%20Quality%20Service%20Learning.pdf

Chapter 8

Validating an Instrument for Measuring the Developmental Outcomes for Students Arising from Service-Learning Ka Hing Lau and Robin Stanley Snell Abstract Service-learning has been adopted in Hong Kong tertiary education for over a decade, yet its developmental outcomes for students have not been systematically researched. One reason for this has been the lack of a standardized and comprehensive outcomes measurement instrument. Based on a review of literature on developmental outcomes arising from service-learning, consultation with local practitioners, and the need for alignment with local educational goals, the ServiceLearning Outcomes Measurement Scale (S-LOMS) was developed specifically for the Hong Kong context. A pilot study was undertaken, followed by six studies to test the validity and reliability of S-LOMS. Results indicate that although S-LOMS does not have good divergent validity, it has good content, construct, and criterion validity, and is a reliable tool with high levels of Cronbach’s alpha and test–retest reliability values. Keywords Developmental outcomes · Scale development and validation · Validity · Reliability · Hong Kong

8.1 Background Service-learning is an experiential pedagogy, through which “students engage in activities that address human and community needs together with structured opportunities intentionally designed to promote student learning and development” (Jacoby,

K. H. Lau (B) Service-Learning and Leadership Office, The Hong Kong Polytechnic University, Kowloon, Hong Kong e-mail: [email protected] Office of Service-Learning, Lingnan University, Tuen Mun, Hong Kong R. S. Snell Department of Management, The Hang Seng University of Hong Kong, Siu Lek Yuen, Hong Kong e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022 G. Ngai and D. T. L. Shek (eds.), Service-Learning Capacity Enhancement in Hong Kong Higher Education, Quality of Life in Asia 14, https://doi.org/10.1007/978-981-19-2437-8_8

137

138

K. H. Lau and R. S. Snell

1996, p. 5). Since Lingnan University became the first Hong Kong local university which institutionalized service-learning by establishing the Office of ServiceLearning in 2006 (Ma & Chan, 2013), service-learning has undergone rapid and substantial development in Hong Kong (Ma, 2018; Xing & Ma, 2010). The adoption of service-learning in Hong Kong followed major educational reforms of secondary and tertiary education, which were intended to promote whole-person development and foster the cultivation of students into socially responsible citizens. Currently, all local and some private universities in Hong Kong are operating service-learning. They have adopted various ways for doing this. Many have integrated servicelearning projects into academic courses, and some have also made service-learning a graduation requirement. Despite the emergence of service-learning as a major auxiliary pedagogy in Hong Kong, research on the developmental outcomes for students arising from servicelearning in Hong Kong and other Asian contexts remains limited, in comparison with studies conducted in the west (Shek & Chan, 2013). In part, this gap reflects the absence of a reliable and valid measurement instrument to capture student developmental outcomes (Snell & Lau, 2020). Existing studies of the developmental outcomes from service-learning in Asia have either employed qualitative methods (e.g., Shek & Chan, 2013; Snell et al., 2019) or have “borrowed” existing scales from the west (e.g., Lo et al., 2016; Ngai, 2009). Although such studies have provided insights into the operation and outcomes of service-learning in particular contexts, it is difficult to make comparisons between them. The Service-Learning Outcome Measurement Scale (S-LOMS) was accordingly developed for the purpose of meeting the need for a reliable and standardized measurement instrument to assess salient developmental outcomes arising from service-learning, as perceived by students. This paper reports the construction process and validation journey of S-LOMS (see also Lau & Snell, 2020, 2021; Snell & Lau, 2020). We will first review past literature on the developmental outcomes for students arising from service-learning. We shall then analyze how the Asian cultural context appears to have shaped the educational aims of service-learning in Hong Kong, such that there may be important differences from the west. We shall explain how these differences were taken into account when developing the conceptual framework for S-LOMS and establishing the developmental domains for inclusion. We shall also report the process through which the items for S-LOMS were constructed by taking the voices of local practitioners into consideration. Following this, we shall share the journey of validation by reporting the results of six individual studies, through which the validity (including content, construct, and criterion) and reliability (Cronbach’s alpha, and test–retest reliability) of S-LOMS have been established. The following two sections report the process of the initial scale development with a pilot study designed to establish the scale’s content validity. This is followed by brief reports of the first three studies, which confirmed the scale construct validity by using factor analysis (exploratory and confirmatory factor analysis) and tests of internal consistency (Cronbach’s alpha and test–retest reliability). These are summarized from the published papers (Lau & Snell, 2020, 2021; Snell & Lau, 2020). The last three studies, involving comparisons between S-LOMS and other scales, and known-group analysis, provide evidence

8 Validating an Instrument for Measuring …

139

about the scale’s criterion validity, and will be explained in detail. We conclude the paper by discussing practical implications for how the S-LOMS can be deployed by educators and administrators as a tool for assessing developmental outcomes for students, along with limitations and recommendations for further research.

8.1.1 Past Literature on Developmental Outcomes Arising from Service-Learning In this section, we will review the major literature on the developmental outcomes for students from service-learning, which will serve as reference point for establishing the conceptual framework for S-LOMS applicable to the Hong Kong context. How service-learning impacts the developmental outcomes of students has been well documented in the west (e.g., Astin et al., 2000; Celio et al., 2011; Conway et al., 2009; Eyler & Giles, 1999; Novak et al., 2007; Warren, 2012; Yorio & Ye, 2012). Several typologies of developmental outcomes have been proposed. For example, Driscoll et al. (1996) identified the following typology of outcome domains: awareness of community, involvement with community, commitment to service, career choices, self-awareness, personal development, academic achievement, sensitivity to diversity, autonomy/ independence, sense of ownership, and communication. Subsequently, Eyler et al. (2001) identified another typology of domains where service-learning had been found to have positive impacts on college students. This comprised: personal development (including personal efficacy, personal identity, spiritual growth, and moral development); interpersonal development (including ability to work with others, leadership, and communication skills); reduced stereotypes and prejudice, along with enhancement of cultural understanding; sense of social responsibility and commitment to service; enhanced academic results and knowledge application; cognitive development, problem analysis, and critical thinking skills; and students’ career development in the long term. Eyler and Giles (1999) sought to categorize these various developmental outcomes into a broader conceptual framework comprising four areas: understanding and applying knowledge; personal and interpersonal development; cognitive development (including critical thinking, engagement curiosity, reflective practice, and perspective transformation); and citizenship. More recently, Felten and Clayton (2011) developed another conceptual framework, linking three components of service-learning, i.e., academic material, relevant service, critical reflection, with three corresponding outcome domains, i.e., academic enhancement, personal growth, and civic learning. We consider that a tripartite framework for developmental outcomes is compatible with past research indicating that service-learning can enhance knowledge understanding and application, personal and professional skills development, and civic orientation and engagement (e.g., Ash & Clayton, 2009; Astin et al., 2000; Celio et al., 2011; Conway et al., 2009; Driscoll et al., 1996; Prentice, 2007; Richard et al.,

140

K. H. Lau and R. S. Snell

2017; Shek & Chan, 2013; Simons & Cleary, 2006; Snell et al., 2019; Warren, 2012; Yorio & Ye, 2012). We consider also that a tripartite framework can accommodate a diversity of benefits from service-learning, including self-efficacy, self-awareness and self-confidence (Astin & Sax, 1998; Astin et al., 2000), empathy (Lundy, 2007), passion for lifelong learning (Bonnette, 2006; Rama, 1998), and creativity (Shek & Chan, 2013). Table 8.1 summarizes the developmental outcomes that have been identified in prior literature, along with the typologies that have been proposed, and subsumes all these under the three broad developmental outcome categories of academic enhancement, personal growth, and civic learning. Table 8.1 Preliminary conceptual framework for the developmental outcomes of students from service-learning Researchers

Academic Enhancement

Personal Growth

Civic Learning

Eyler and Giles (1999)

Understanding and applying knowledge

Personal and Interpersonal development Critical thinking Reflective practice Perspective transformation Engagement, curiosity

Citizenship

Driscoll et al. (1996)

Academic achievement Personal development Communication skills Career choices Self-awareness Autonomy/ independence Sense of ownership

Awareness of community Involvement with community Commitment to service Sensitivity to diversity

Eyler et al. (2001)

Enhanced academic Interpersonal results development Knowledge application Ability to work with others Leadership and communication skills Problem analysis Critical thinking skills Personal efficacy Personal identity Career development Spiritual growth

Moral development Reduced stereotyping and prejudice Enhanced cultural & racial understanding Sense of social responsibility Citizenship skills Commitment to service

Felten and Clayton (2011)

Academic knowledge/ skills/ dispositions

Civic learning

Source Snell and Lau (2020)

Personal growth Teamwork Critical thinking skills Intercultural competence

8 Validating an Instrument for Measuring …

141

8.1.2 Local Adaptation of Service-Learning to the Hong Kong Context So goes an old saying: “When in Rome, do as the Romans do.” When practices are introduced or imported to a new location, adjustments are to be expected, and service-learning is no exception. Service-learning originated in the United States in the 1960s, with the common good, civil society, and learning by doing as its three foundational “pillars” (Shumer et al., 2017), augmented by a concern for democratic values, social change, and the furtherance of social justice. Western approaches to service-learning seek to encourage students to discover the potential of participatory democracy and to build their ability to serve as change agents within communities and as advocates for social justice (Battistoni, 1997; Einfeld & Collins, 2008; Mitchell, 2008; Wade, 1997). When service-learning arrived in Hong Kong, the characteristics described above may have appeared relatively less well matched to the depoliticized cultural and institutional context of this Asian city (Leung & Yuen, 2012) at the time. Confucian values have underpinned educational policies in Hong Kong and other Asian societies (Ma, 2018), where general education tends to emphasize spirituality, self-cultivation, harmonious relationships, and preservation of the status quo (Lee, 2004). An important feature of Hong Kong culture is that the large power distance created by Confucian values inclines members to accept authoritarian supervision and prefer structure and stability in social and organizational settings, including education (Hofstede, 1980). Also notable is that students undergoing academic education in Hong Kong tend to expect to develop practical and vocational skills, and good academic performance has been regarded as the yellow brick road toward job success and upward social mobility (Kennedy, 2002; Shek & Chan, 2013). The above characteristics and expectations appear to have influenced how service-learning was introduced to Hong Kong. Snell and Lau (2020) reviewed the descriptions of service-learning on the webpages of the main Hong Kong universities. They observed a tendency to emphasize opportunities for knowledge application, skill acquisition, and personal and moral development while contributing to the community rather than opportunities to further the causes of democracy, justice, human rights, and social change. Confucian cultural preferences can also be observed in the design of service-learning programs in Hong Kong, which tend to involve undertaking service-learning projects within a pre-ordained course structure and under close instructor guidance. It appears, therefore, that among the three foundational pillars mentioned at the beginning of this section, service-learning in Hong Kong’s focuses primarily on “learning by doing.” While the common good and civil society are not ignored, they are addressed by arranging for students to serve the community through existing institutions, and there are relatively few signs that proactive civic engagement and the furtherance of democratic ideals play a role in such community participation.

142

K. H. Lau and R. S. Snell

Having discussed how the Confucian values that permeate Hong Kong society have shaped the localization of service-learning, we shall next consider the implications for our conceptualization of the intended developmental outcomes of service-learning.

8.1.3 A Framework of Developmental Outcomes Arising from Service-Learning in the Hong Kong Context Based on prior literature, and on our discussion how service-learning has been adapted to Hong Kong’s institutional and cultural context, we developed a modified conceptual framework for S-LOMS to take account of educational priorities in Hong Kong. The framework (see Table 8.2) comprises 15 domains under four categories: knowledge application; personal and professional development; civic orientation and engagement; and self-awareness. Our localized framework entails five differences from the “Western” framework in Table 8.1. First, the original category of personal growth has been relabeled as personal and professional skills, reflecting greater emphasis on pragmatism by Hong Kong’s tertiary education institutions. Second, self-awareness has been designated as another superordinate outcome category, reflecting the tradition of self-cultivation and self-reflection in Confucian educational philosophy. Third, subsumed under this new superordinate category, we have identified three outcome domains, labeled Table 8.2 The modified conceptual framework specific to Hong Kong context Conceptual Category

Knowledge Application

Personal & Professional skills

Civic Orientation & Engagement

Self-awareness

Developmnetal Outcome Domain

1. Knowledge Application

2. Relationship Skills

8. Sense of Social Responsibility

13. Self-understanding

3. Team Skills

9. Commitment to Social Betterment

14. Commitment to self-improvement

4. Problem-solving 10. Skills Understanding Community 5. Critical thinking 11. Respecting Skills Diversity 6. Self-reflection Skills 7. Creativity Source Snell and Lau (2020)

12. Empathy & Caring for Others

15. Self-esteem

8 Validating an Instrument for Measuring …

143

self-esteem, self-understanding, and commitment to self-improvement. Fourth, we replaced the label of the broad category of academic enhancement with knowledge application, reflecting that we have dropped the domain of subject knowledge, on the grounds that graded assignments within a course would be a better vehicle than S-LOMS for assessing students’ understanding of the academic content of the respective course. Fifth, we have attempted to distill and select from a plethora of outcome domains to arrive at a relatively small set of 15 domains. The conceptual framework in Table 8.2 was an important reference point in the development and validation of S-LOMS. The objectives of developing S-LOMS were to develop an instrument that is: (a) applicable to the Hong Kong context; (b) comprehensive in covering salient desired developmental outcomes arising from service-learning; (c) standardized in terms of format and administration methods; and (d) flexible in affording researchers and practitioners opportunities to adopt shorter versions, tailored to measuring selected developmental domains. The following sections will explain the validation journey through a pilot study and six separate validation studies, which together have followed standard scale development procedures (Boateng et al., 2018; DeVellis, 2003).

8.2 Pilot Study: Item Development and the Confirmation of Face and Content Validity The pilot study focused on scale meaning clarification and item development and involved both deductive and inductive approaches. The deductive approach began with the literature review reported in the previous sections of this paper, which enabled us to formulate the conceptual framework for S-LOMS, given in Table 8.2. Moreover, we examined relevant preexisting scales (e.g., Bringle et al., 2004; Eyler, Giles, & Braxton, 1997; Ma et al., 2019; Olney & Grande, 1995; Reeb et al., 1998) as reference points. Next, we adopted an inductive approach by collecting comments and suggestions from at least one subject matter expert (SME), from each of the four collaborating Hong Kong based universities (namely Lingnan University, The Hong Kong Polytechnic University, Hong Kong Baptist University, and The Education University of Hong Kong). All SMEs were instructors of service-learning courses at their respective institutions, and most were Chinese. We consulted these SMEs via interviews, focus groups, and electronic media, about the appropriateness of the proposed developmental domains encompassed by S-LOMS. Regarding constituent items for each of the 15 domains, from the literature, and in consultation with the SMEs, we first identified parameters for items representing shades of meanings salient to each of the 15 S-LOMS domains. Next, we worked with SMEs to co-create potential items within these agreed-upon parameters. Following consultation and co-creation with the SMEs, we compiled a total of 103 prospective S-LOMS items under the 15 developmental domains.

144

K. H. Lau and R. S. Snell

A draft version of S-LOMS, comprising these 103 self-reporting descriptive items written in English, based on a 10-point Likert scale (from 1, “strongly disagree” to 10, “strongly agree”), was then subjected to a pilot study to test content validity from the perspective of target respondents, i.e., students, along with checks on item readability. A total of six pilot sessions, each lasting around one hour with up to 20 students, were conducted face-to-face with volunteering students at the aforementioned four universities. Each session consisted of two parts. First, the students answered the draft version of S-LOMS, which took around 40 min, and were invited to write comments if they encountered any difficulty in understanding any of the items. Second, during the last 20 min, the students were invited to explain their written comments to the administrator as well as expressing any additional concerns about language, wording, ambiguity, etc. In all, 83 students (29 males, or 34.9%; 54 females, or 65.1%; mean age: 20.5) participated in the pilot study. A report on their comments was presented to and analyzed by the researchers, who revised the wording of 35 of the items and discarded another two items, in order to address readability and language problems, ending up with a 101-item draft version of S-LOMS. With its content and face validity supported by the SMEs’ participation and by this pilot study, the 101-item draft S-LOMS was tested for its construct validity in three further studies, reported next.

8.3 Studies 1 & 2: Streamlining S-LOMS with Confirmation of Construct Validity Through Factor Analysis This section briefly reports two studies on validating S-LOMS in terms of construct validity. The first one employed exploratory factor analysis (EFA), starting from the above 101-item draft S-LOMS, through which the measurement instrument was streamlined to a 56-item scale (for details, please read Snell & Lau, 2020). This study recruited 400 full-time students from the above four local universities. Two-thirds of them were female (65.0%), and the mean age was 20.9. The students came from a variety of academic disciplines (Arts: 23.4%, Social Science: 15.6%, Business: 22.4%, Engineering & Science: 27.5%, Nursing: 11.1%). Most of them (70.5%) had already had service-learning experience. They completed the 101-item draft S-LOMS on a voluntarily basis. The minimum average partials (MAP) test and the principal components method with oblimin rotation of EFA were used to analyze the data. An item reduction exercise was conducted, based on two criteria: (a) removing those items with the highest factor loadings under 0.4 in absolute values; and (b) removing double-loaded items. Due to the large number of items relative to the sample size, four separate EFAs were conducted for the items of the four overarching categories. As a result, S-LOMS was streamlined into a 56-item version. The analysis indicated that the remaining items formed coherent and meaningful factors, representing

8 Validating an Instrument for Measuring …

145

their respective developmental domains and broader categories, and corresponded well with the modified conceptual framework for S-LOMS. Within the overarching category of knowledge application, the factor loadings of items on its single developmental domain ranged from 0.799 to 0.881. The corresponding ranges of factor loadings on the developmental domains included within the other overarching categories were as follows: 0.411–0.886 within personal and professional skills; 0.467–0.861 within civic orientation and engagement; and 0.527–0.941 within self-awareness. Within the overarching category of knowledge application, the Cronbach’s alpha value for its single developmental domains was 0.872. The ranges of Cronbach’s alpha values for the domains included within the other overarching categories were as follows: 0.751–0.925 within personal and professional skills; 0.813–0.919 within civic orientation and engagement; and 0.829–0.859 within self-awareness. However, the statistical output indicated a necessary modification to the model structure, in the form of four higher-order developmental domains, each arising from combinations of two original lower-order domains. These emergent higher-order developmental domains were: creative problem-solving skills, combining creativity and problem-solving skills; relationship and team skills, combining relationship skills and team skills; caring and respect, combining empathy and caring for others with respecting diversity; and community commitment and understanding, combining commitment to social betterment with understanding community. Armed with this evidence, we conducted confirmatory factor analysis (CFA). The CFA involved comparisons with a series of alternative models, including the original 15-domain version of S-LOMS represented in Table 8.2. For details, please read Lau and Snell (2021). To this end, another sample of 629 university students was recruited from the above four local universities, with an average age of 20.5, and comprising 59.5% females. They came from various academic disciplines (Engineering & Science: 40.9%, Business: 19.9%, Social Sciences: 14.3%, Arts: 12.7%, and Healthcare: 12.2%). A majority of them (65.8%) had previous service-learning experience. In this study, the participants answered the 56-item S-LOMS on a voluntarily basis. Overall results of the CFA, which were obtained by means of the full information maximum likelihood (FIML) method with robust correction, conducted in EQS 6.4 for Windows showed acceptable model fit (NNFI, CFI ≥ 0.90; RMSEA < 0.05), good factor loadings (ranged from 0.573 to 0.842) and high reliability (0.790–0.957) for two models that fitted the data better than the alternative models, which were rejected. The first surviving model was the 11-domain structure obtained in the EFA but without the four overarching categories. The second surviving model was a new 10-domain structure (combining the domains of sense of social responsibility and community commitment and understanding), subsumed under four overarching categories. Both of these models also survived a further test on their stability based on multi-sample analysis over gender. Nonetheless, the CFA results also indicated the possibility of further simplification, given that several factor loadings and interfactor correlation coefficients in the surviving two models were significantly high. We will return to this point in the final discussion section.

146

K. H. Lau and R. S. Snell

8.4 Study 3: Validating the Stability of S-LOMS Over Time The previous two studies offered empirical evidence for the construct validity and internal consistency of S-LOMS by using factor analysis and Cronbach’s alpha reliability. This study sought to validate S-LOMS in terms of test–retest reliability, which indicates whether the instrument remains stable over a time period during which a salient intervention, i.e., service-learning, is absent. For details, please read Lau and Snell (2020). This study recruited 266 students as participants to complete the 56-item-11-domain version of S-LOMS twice within a period of two weeks. Among the recruited students, 122 finished both the test and retest phases (completion rate: 45.9%). Regarding the students, who completed both phases, the mean age was 20.4 and 75.4% were females. They came from various academic disciplines (35.2% from Engineering & Science, 34.4% from Arts, 13.9% from Social Sciences, 10.7% from Business, and 5.7% from Healthcare), and 34.4% of them had prior service-learning experience, while 25.4% were concurrently taking service-learning programs. Intra-class correlation coefficients (ICCs) were employed in this study as the test– retest reliability index. ICCs indicated a mixture of moderate and good test–retest reliability. The entire measurement instrument achieved an ICC value of 0.78, with ICCs for the four overarching categories ranging between 0.67 and 0.78, and ICCs for the 11 developmental domains ranging between 0.60 and 0.77. Cronbach’s alpha scores ranged between 0.74 and 0.93 at the domain level, and between 0.86 and 0.96 at the overarching category level, and were 0.98 for the overall scale, indicating satisfactory results for one-off reliability.

8.5 Study 4: Testing S-LOMS in a Pretest–Posttest Setting With the construct validity and reliability basically established, the next step involved confirming the criterion validity of S-LOMS. Study 4 used S-LOMS to measure selfperceptions among students before and after a service-learning intervention and the differences were compared. In Study 4, we collected responses from the students from the four aforementioned local universities, who had taken service-learning courses during the fall or spring semesters in the academic year of 2019–2020. As a course requirement, 923 students completed S-LOMS both before commencement of their service-learning projects (pretest) and after completion of their projects (posttest). For the demographics, please refer to Group C in Table 8.4. Paired-samples t-tests were employed to analyze differences among the pretest–posttest scores. Since not all the universities required their students to compete items on all 11 developmental domains of S-LOMS, there may appear to be discrepancies in sample size in the results. Table 8.3 shows the paired-samples t-test results, which indicated that the posttest scores for all learning domains, overarching categories, and S-LOMS as a whole were significantly higher than the pretest scores. This was consistent with our expectation

6.87 7.08

923

6.90

923 655 923

Self-understanding

Commitment to Self-improvement

S-LOMS Overall

7.15

7.21

7.13

7.11

923 655

Self-awareness

7.41

7.68

Self-efficacy

923

Sense of Social Responsibility

6.99

655 923

Community Commitment & Understanding

Caring & Respect

7.46

923

Civic Orientation & Engagement

7.00

655 655

Self-reflection Skills

Critical Thinking Skills

6.95

Relationship & Team Skills

6.94

923 923

Personal & Professional Skills

6.84

SD.

1.17

1.39

1.36

1.38

1.32

1.37

1.33

1.28

1.26

1.33

1.35

1.36

1.27

1.25

1.30

923

655

923

655

923

923

923

655

923

655

655

923

923

923

923

N

Mean

N 923

Creative Problem Solving Skills

Knowledge Application

Posttest

Pretest

Table 8.3 The paired-samples t-test comparisons results Mean

7.73

7.73

7.70

7.64

7.68

7.93

8.10

7.71

7.97

7.72

7.76

7.56

7.57

7.59

7.47

SD

1.12

1.21

1.25

1.25

1.21

1.26

1.19

1.12

1.15

1.21

1.22

1.29

1.28

1.23

1.38

Posttest-Pretest

0.58

0.52

0.56

0.74

0.57

0.52

0.42

0.72

0.50

0.72

0.68

0.70

0.62

0.65

0.63