Polyphonic Construction of Smart Learning Ecosystems: Proceedings of the 7th Conference on Smart Learning Ecosystems and Regional Development (Smart Innovation, Systems and Technologies, 908) 9811952396, 9789811952395

Citation preview

Smart Innovation, Systems and Technologies 908

Mihai Dascalu Patrizia Marti Francesca Pozzi   Editors

Polyphonic Construction of Smart Learning Ecosystems Proceedings of the 7th Conference on Smart Learning Ecosystems and Regional Development

123

Smart Innovation, Systems and Technologies Volume 908

Series Editors Robert J. Howlett, Bournemouth University and KES International, Shoreham-by-Sea, UK Lakhmi C. Jain, KES International, Shoreham-by-Sea, UK

The Smart Innovation, Systems and Technologies book series encompasses the topics of knowledge, intelligence, innovation and sustainability. The aim of the series is to make available a platform for the publication of books on all aspects of single and multi-disciplinary research on these themes in order to make the latest results available in a readily-accessible form. Volumes on interdisciplinary research combining two or more of these areas is particularly sought. The series covers systems and paradigms that employ knowledge and intelligence in a broad sense. Its scope is systems having embedded knowledge and intelligence, which may be applied to the solution of world problems in industry, the environment and the community. It also focusses on the knowledge-transfer methodologies and innovation strategies employed to make this happen effectively. The combination of intelligent systems tools and a broad range of applications introduces a need for a synergy of disciplines from science, technology, business and the humanities. The series will include conference proceedings, edited collections, monographs, handbooks, reference books, and other relevant types of book in areas of science and technology where smart systems and technologies can offer innovative solutions. High quality content is an essential feature for all book proposals accepted for the series. It is expected that editors of all accepted volumes will ensure that contributions are subjected to an appropriate level of reviewing process and adhere to KES quality principles. Indexed by SCOPUS, EI Compendex, INSPEC, WTI Frankfurt eG, zbMATH, Japanese Science and Technology Agency (JST), SCImago, DBLP. All books published in the series are submitted for consideration in Web of Science.

Mihai Dascalu · Patrizia Marti · Francesca Pozzi Editors

Polyphonic Construction of Smart Learning Ecosystems Proceedings of the 7th Conference on Smart Learning Ecosystems and Regional Development

Editors Mihai Dascalu Department of Computer Science University Politehnica of Bucharest Bucharest, Romania

Patrizia Marti Dipartimento di Scienze Sociali University of Siena, Politiche e Cognitive Siena, Italy

Francesca Pozzi Istituto Tecnologie Didattiche Consiglio Nazionale delle Ricerche Genova, Italy

ISSN 2190-3018 ISSN 2190-3026 (electronic) Smart Innovation, Systems and Technologies ISBN 978-981-19-5239-5 ISBN 978-981-19-5240-1 (eBook) https://doi.org/10.1007/978-981-19-5240-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Preface

After the shock of the pandemic, learning ecosystems—and in particular schools, universities, and territorial learning communities—all over the world—are facing a new phase characterized by the search for and the experimentation of ways to their “new normality”. This is within a world that seeks to protect and incentivize the economic recovery and to find a social and political solidity that still seems far to be achieved. These proceedings contribute to the ongoing scientific debate about such transition, by focusing on the notion of learning ecosystems and reflecting on the way they are changing, between face-to-face, online, and hybrid settings. The main aim is to understand how such changes may be related to the achievement of “a better learning for a better world” as a contribution to the United Nations 2030 Agenda for Sustainable Development Goals (SDGs) and how they will contribute to the reduction of inequalities and to the empowerment of each individual, according to her/his expectations and talents. In particular, the research results presented here cover different dimensions, targeting various contexts (from school to university), different technological solutions (e.g., digital games, automated systems, e-learning environments), and different involved stakeholders (i.e., students, teachers, school principals, parents, communities). SLERD 2022 was a hybrid event organized by the University Politehnica of Bucharest, Romania, with the partnership of ASLERD (Association for Smart Learning Ecosystems and Regional Development) an international non-profit interdisciplinary and scientific-professional association committed to support learning ecosystems to get smarter and play a central role to regional development and social innovation. The University of Aveiro, Portugal, represented by its DigiMedia Research Group, also plays a central role in the co-organization of this SLERD hybrid edition. Received submissions had authors from 14 different countries including EU member states (i.e., Austria, Belgium, Estonia, Finland, France, Italy, Portugal, and Romania), India, Iraq, Palestine, Serbia, UK, and USA, an indicator of the global interest and partnership in the promoted topics. After a rigorous doubleblind peer-review and meta-review process, 14 full papers were accepted to be included in SLERD 2022 conference proceedings, published by Springer, in the v

vi

Preface

Series Smart Innovation, Systems and Technologies. We believe these proceedings are relevant for researchers, post-graduate students, teachers, designers, and policymakers concerned with learning ecosystems, methods, and empirical evidence of alternatives for post-pandemic learning scenarios. It has been an honor to belong to this scientific community and serve as the publishing chair in this SLERD edition, with a final selection of papers that represent the excellence of all the authors’ work, coupled with rigor throughout the decision and publishing processes. Much of this work would not have been possible without the effort and support of our Conference and Program Committees, which included 40 international expert researchers. We would like to thank them all, for their time to organize the event with enthusiasm and commitment. Bucharest, Romania Siena, Italy Genova, Italy May 2022

Mihai Dascalu Patrizia Marti Francesca Pozzi

Contents

1

2

3

4

5

Context-Aware Classrooms as Places for an Automated Analysis of Instructional Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Philippe Dessus

1

Student Smartphone Experience Narratives Mediated by the Phygital School Library for Learning Ecosystems . . . . . . . . . . Maria José Fonseca and Óscar Mealha

13

Space and Time in Hybrid Teaching and Learning Environments: Two Cases and Design Principles . . . . . . . . . . . . . . . . . Teemu Leinonen and Tiina Mäkelä

29

The Italian School Ecosystems Two Years After the Lockdown: An Overview on the “Digital Shock” Triggered by the Pandemic in the Perceptions of Schools’ Principals and Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Carlo Giovannella, Licia Cianfriglia, and Antonello Giannelli Automated Paragraph Detection Using Cohesion Network Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robert-Mihai Botarleanu, Mihai Dascalu, Scott Andrew Crossley, and Danielle S. McNamara

47

77

6

Assessing Readability Formulas in the Wild . . . . . . . . . . . . . . . . . . . . . Scott Crossley, Stephen Skalicky, Cynthia Berger, and Ali Heidari

91

7

pROnounce: Automatic Pronunciation Assessment for Romanian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Dan Ungureanu, Stefan Ruseti, Irina Toma, and Mihai Dascalu

8

Community Pacts and we4SLE as Tools to Support the Implementation of Smart Learning Ecosystems . . . . . . . . . . . . . . . 115 Irene Urbanetti, Maria Rosaria Autiero, Vincenzo Baraniello, and Carlo Giovannella

vii

viii

9

Contents

Digital Games as Tools of Innovative Pedagogy in Education . . . . . . 129 Maneesh Dubey and Kunal Sinha

10 Environment Challenges of E-Learning in Higher Education—The Teachers’ Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Janika Leoste, Larissa Jõgi, Tiia Õun, Ugljesa Marjanovic, Slavko Rakic, Simone Schöndorfer, and Zoe Lefkofridi 11 Parents’ Voices: Inclusion of Students with Intellectual and Developmental Disabilities in Higher Education . . . . . . . . . . . . . . 157 Isabel Catarina Martins, Oksana Tymoshchuk, Eulália Albuquerque, Paula Santos, and Geert Van Hove 12 De-Identification of Student Writing in Technologically Mediated Educational Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Langdon Holmes, Scott Crossley, Nick Hayes, Dylan Kuehl, Anne Trumbore, and Gabriel Gutu-Robu 13 Conceptualising Micro-credentials in the Higher Education Research Landscape. A Literature Review . . . . . . . . . . . . . . . . . . . . . . . 191 Alexandru Cart, is, , Janika Leoste, Romit, a˘ Iucu, Kaido Kikkas, Kalle Tammemäe, and Katrin Männik 14 The Reaction of the University Ecosystem to the Pandemics in a Mid-East Country: The Case of IRAQ—A Compared Analysis of Students’ and Teachers’ Perceptions . . . . . . . . . . . . . . . . . 205 Alaa Alkhafaji, Haider Mshali, Marcello Passarelli, and Carlo Giovannella Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

About the Editors

Mihai Dascalu is a professor at University Politehnica of Bucharest with a strong background in Computer Science applied in Education. He has extensive experience in national and international research projects with more than 200 published papers, including 30+ articles at top-tier conferences, 100+ papers indexed ISI at renowned international conferences, and 10+ Q1 journal papers. Complementary to his competencies in NLP, technology-enhanced learning and discourse analysis, Mihai holds a multitude of professional certifications and extensive experience on strategic projects on non-refundable funds. Moreover, Mihai has obtained a Senior Fulbright scholarship in 2015, has become a fulbright ambassador since 2018, and holds the US patent. Mihai is also a corresponding member of the Academy of Romanian Scientists. Patrizia Marti is a professor of Experience Design at the University of Siena. She is the director and Rector’s delegate of Santa Chiara Fab Lab where she manages several participatory innovation projects with external partners. Patrizia has an interdisciplinary background in design, philosophy, and computing and a Ph.D. in Interaction Design. Her research activity concerns designing interactive systems in various fields of application including special education for people with disability also mediated by robots. He has been a principal investigator on many EU-funded projects. She has been an expert advisor to many EU and international bodies, including EU Commission, EU Future & Emerging Technologies Program, EU Intelligent Information Interfaces, Eurocontrol, EU Disappearing Computer, UX group at University of Warsaw (Poland), and Swedish Agency for Innovation Systems. She has been an invited keynote speaker at various international conferences. She has also been the editor for special issues of international journals. Francesca Pozzi is a lead researcher at the Istituto Tecnologie Didattiche (ITD)— CNR (Consiglio Nazionale delle Ricerche), where her main research area is the theory and practice of Technology Enhanced Learning (TEL). In particular, her main research interests include learning design; collaborative learning; computer

ix

x

About the Editors

supported collaborative learning (CSCL); teacher training and innovation in education; from formal to non-formal learning: ICT in cultural heritage education; technology enhanced learning for healthcare professionals’ training. She is the co-editor of the Italian Journal of Educational Technology (IJET) and is a member of various scientific boards and committees of international journals and conferences of the field. She has participated in and led several international research projects in TEL. She has published over 180 scientific papers, including +10 on Q1 journals, and several on books and conference proceedings.

Chapter 1

Context-Aware Classrooms as Places for an Automated Analysis of Instructional Events Philippe Dessus

Abstract Context-Aware Classrooms (CACs), or ambient classrooms, are places in which instructional events can be captured and analyzed, thanks to advanced signal processing techniques. For CACs to be used for a better understanding of the educational events (teaching or learning), theoretically grounded approaches have to be reviewed and their main variables of interest presented. In this paper, three types of approaches to study the use of CACs (behavioral, ecological, and enactivist) are discussed, first theoretically, then about what each approach brings to the research on educational research. Some implications to build more ecologically sound in-presence or hybrid instructional sessions after the COVID-19 are drawn.

The COVID-19 pandemic has challenged education systems because remote instruction was to be quickly implemented at a large scale, often without specific guidance [1]. This sudden change did not allow stakeholders to properly take care about attendees’ privacy, their disengagement [2], or negative emotional mood [3] and to get information to analyze and reflect about the situation [4]. Two years after this shift, it is time to take a step aside to try to build a novel ecosystem in which build both in-presence and hybrid instruction would meet the following requirements: accounting for more ecologically sound places that allow various hybrid instructional situations; using artificial intelligence-based tools to analyze behavioral, cognitive, and emotional features more cautiously, notably in respecting attendees’ privacy; allowing in-depth teachers’ reflection on their practice. A promising path is to consider the use of context-aware classrooms as containers to capture, observe, and transmit instructional events. Capturing and observing instructional events occurring in classrooms, for teachers and researchers, is a fruitful approach to help progress, reflection, and comprehension about these events. The first observation systems were human-based, then were assisted by a large range of tools or instruments, like audio recorders, video cameras, P. Dessus (B) University Grenoble Alpes, LaRAC, 38000 Grenoble, France e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_1

1

2

P. Dessus

or mobile eye trackers. More recently, ambient classrooms have become a sort of “meta-device” embarking these tools [5] either to make rooms more reactive to instructional events or to engage deeper studies about the instructional and learning processes. A Context-Aware Classroom (henceforth, CAC), also known as an ambient, ubiquitous, adaptive, intelligent, responsive, smart, or pervasive classroom, represents any physical environment in which instructional and learning events occur, and in which specific ways to capture and analyze these events are enabled. The data capture and analysis are supported by several digital devices using a very disparate bulk of models, techniques, and tools: signal analysis and processing techniques, robotics, artificial intelligence, sensors, controllers, and effectors [6], or, simpler, many kinds of media [7]. In that way, they can help overcome human observers’ attentional limits and some biases in the observation of instructional situations [8]. However, the very role of CACs is seldom elicited: ambient classrooms are built and tested for very different purposes, from triggering a specific device to measure the classroom’s climate. So far and broadly, the educational use of CACs is mainly behavior-centered, where the latter respond to some very shallow events (e.g., attendees are entering a room), akin to demoing materials, and solutionism [9] appears to be one of the main drivers. At second sight, CACs are rooms with ears and eyes and can either be seen as intensive surveillance tools [10] or tools to better understand instructional or learning events. The question we will answer in this paper is how CACs can be used to have a better understanding of the educational events (teaching or learning), using theoretically grounded approaches, that would help build new ecosystems for a “post-pandemic new normality.” The remainder of this paper is as follows. Section 1.2 gives an overview of the various variables to be observed in classroom events, either by humans or automated methods, embedded in CACs. Section 1.3 will introduce three main theoretical approaches of classroom observation that can be implemented in CACs, and Sect. 1.4 will develop in more depth how these three types of CACs work.

1.1 Educational Variables Subject to Observation What are the raw variables that can be observed in classrooms? Four main types of variables can structure the observational process [11]: – presage variables are about the teachers’ characteristics that can have an effect on the instructional process, like their experience, their beliefs, and knowledge. These variables can be investigated by questionnaire before or after observation sessions; – contextual variables are related to the classroom size and the material—they are often named “structural quality” [12]. These variables are easier to measure and quantify than the other ones [13];

1 Context-Aware Classrooms as Places …

3

– process variables are behaviors and events that take place in the teaching–learning context (often named “process quality”). Even though these variables are more difficult to perceive and analyze either by humans or machines, because they are mostly hidden and transient [14], they play a crucial role in students’ achievement [15]; – product variables, or outcomes, are measurable consequences of the teaching– learning processes, such as students’ achievement, attitudes, and beliefs. These variables, often evaluative, are rather easy to collect and analyze digitally [16]. It is noteworthy that process variables can roughly pertain to three different categories [17]. Socio-emotional support encompasses the ways teachers promote a positive climate supporting students’ autonomy and well-being. Classroom and resource management relates to the ways teachers manage students’ behaviors and propose high-quality learning resources. Cognitive and content-related relates to the ways teachers support students’ learning, creativity, and understanding of the taught content. We will focus on process variables, for which both variability during instruction and their impact on learning are crucial. Some questions arise: Which theoretical perspective to adopt in studying them? Which are their pros and cons?

1.2 Classrooms as Context-Aware: Three Theoretical Accounts Technology-enhanced learning research is an under-theorized field [18]: roughly a third of the investigated research papers mention a theory explicitly. The lack of theories underlying the development of techniques or devices is harmful to the validity of the research done. Using a device or a computer-based system without being aware of the underlying theory can lead to solutionism: the implementation does not address any specific problems and can be seen as purposeless. Before detailing the different types of CACs it is worth discussing the most influential educational theories which can be invoked in the functioning of a CAC. To make these theories more understandable and real-life related, let us take a reallife example. Sarah, a middle school mathematics teacher, teaches every Monday morning in a context-aware classroom. We will follow her to see how the theoretical perspectives would change the analysis of her classroom management and her students’ activities.

1.2.1 Behavioral Approach The most known approach is to consider that teachers and students are reacting to different stimuli in the classroom, these stimuli being triggered by humans or purely mechanical, as external forces. For instance, a ringing bell gives an information

4

P. Dessus

about the end of a lesson and calls for getting out of a classroom; a teacher manages her classroom through various cues (facial expressions, comments, and feedback), hereby preventing her students from engaging themselves in misbehaviors. Teachers use these methods very commonly. Four basic behavioral operations are promoted in classrooms [19]: • positive versus negative reinforcement, when a positive versus negative stimulus is added contingently to a desired behavior versus a misbehavior. For instance, badges or incentives are such positive reinforcers; • extinction, when a behavior comes to decrease because its reinforcement decreases. For instance, students become less and less misbehaved because they do not get any advantage in their misbehavior, which is not recognized by the teacher over time; • response cost punishment, when a positive stimulus is removed upon undesirable behaviors. For instance, students who misbehave cannot be praised by their teacher anymore; • punishment with aversives, when a negative stimulus is added upon undesirable behaviors. For instance, a scholarship can be withdrawn to students who do not show up for exams. The information flow of the behavioral approach, as typically implemented in CACs, is depicted in Fig. 1.1. First, sensors get information (visual and auditory features, etc.) from the instructional scene, then its main features are processed to further infer the likely behaviors. The status—a psychological construct using behaviors as proxies—can then be determined, and effectors can trigger an action or assign tags to people or objects. This loop is enacted continuously and helps teachers and students do, for example, clerical tasks. Some process variables can still be captured with this approach, even though they are fully behavioral, like resource and classroom management-oriented. In Simondon’s [20] words, behavior-oriented CACs behave mostly as tools, which help action.

Sarah’s classroom: In this approach, Sarah and her students enter into the CAC and automatically trigger the lights on, the students’ faces are automatically

Fig. 1.1 CACs information flow in behavioral approaches

1 Context-Aware Classrooms as Places …

5

recognized, and an attendance report is sent to the school administration. When Sarah praises a student, the student’s name is retrieved and she is given a credit in a specific badge (like in ClassDojo, https://www.classdojo.com). This purely behavioral approach cannot be used neither to analyze cognitive processes nor to account for the environment. The two following approaches are more devoted to these points.

1.2.2 Ecological Approach In this second approach, stemming from ecological psychology, behavior is determined by the interaction of the individuals and the environment: ecological psychology theories […] strive to explain the natural patterns of stimuli, both social and physical, which exist in the individual’s immediate environment and subsequently impact the individual’s behavior and experience. [21, p. 4]

People and other objects are considered as part of an environment, inserted into multiple perception–action loops mediated by devices. An individual perceives an element of the environment (e.g., an object, another individual) and exerts an action consequently, to continue to be well-balanced within the environment. In this approach, both internal and external forces are considered in interaction [22] and there is an interdependence between the people and the room’s devices. This case entails that the situated perspective is crucial: the actual place where we are living plays an important role in the individuals’ experience. For instance, a research showed that the likelihood that a group of students in a STEM (Science, technology, engineering, and mathematics) course are engaged in an activity is related to both the teacher’s close presence and the frequency of her interaction with the group [23]. Information is gathered from the environment to take decisions. The “thinking body” is taken as a point of departure, whereby teaching and learning are natural events occurring in a natural world [24]. Compared to the previous approach, both the classroom and the individuals’ cognitions are relevant entities. Observational units are about social events that emerge from the analysis and are not pre-established from the individual’s point of view [25]. The behaviors are not purely disconnected, but connected to practice [24]. The main information–action flow in CACs within an ecological approach is depicted in Fig. 1.2. Individuals trigger various perception–action loops in relation to the objects and other individuals with which and whom they interact, the greyed loops showing previously triggered loops. After Simondon [20], ecologically oriented CACs behave mostly as instruments, which help perception.

6

P. Dessus

Fig. 1.2 CACs information–action flow in ecological approaches

Sarah’s classroom: In this approach, Sarah’s and her students’ behaviors are implicitly linked to each other because they are all immersed in the same environment. For instance, when Sarah leads a discussion about a given topic, participants’ utterances obey some implicit rules, like the following: 1. When an individual finishes her turn, she can select the next speaker, by a specific gaze or body orientation; 2. If no speaker is selected, an individual can choose to speak (auto-selection) [26]. The environment’s features are guides that ecologically constrain individuals’ behaviors, even without explicit rules. In that case, a CAC can help observe and measure this turn taking by capturing and analyzing individuals’ face orientation and body pose to predict turn taking [27].

1.2.3 Enactivist Approach An enactivist approach considers that the changes in an individual do not lie in the individual or the changes by themselves, like in the previous approach, but by the interaction between the individual and the environment at a personal and first-person level [28]. Perception, cognition, and emotion are fully integrated with sensorimotor action [29], and knowledge emerges from this action. The immersed individuals perceive the other attendees and objects, and the observation units are not pre-established but are found through inquiry [24], concrete experience, and activity, being coupled to each other. In that way, the individuals’ practice emerges from the situation: they are led by the situation, but they do not possess it [30]. Fundamentally, answering the question of how to teach in an enactivist way is trying to answer the question: “How is it like to be a teacher or students in a classroom?”

1 Context-Aware Classrooms as Places …

7

Fig. 1.3 CACs in enactivist approaches

As the previous, this approach is fully compatible with the multimodal analysis of the classroom environment [31], but needs a step further: the account of first-person perception within its environment. To data about attendees’ location, speech, gesture, and posture, one has to add information about gaze, emotion, or other physical-based measures (e.g., electrodermal) [30]. Figure 1.3 below depicts the main situation of individuals in CACs in an enactivist approach. Attendees’ actions, equipped by devices that extend their capabilities, are coupled with the environment [32].

Sarah’s classroom: In this approach, Sarah’s and her students’ perceptions, sensorimotor actions, emotions, are all involved and updated all along their action to build knowledge of the situation. Their posture, gestures, and speech, form traces of their practice as, respectively, teacher and students, and the artefact they use extend their mind. This approach is closer to the enactment of authentic activities and their capture and analysis is fully multimodal.

1.3 What CACs Can Do? In this section, we dive deeper into how a CAC can capture and analyze instructional events, and the limitations of each approach, notably in terms of privacy. Table 1 summarizes the main characteristics of each approach: their behavioral account, the main scrutinized classroom variables, and their privacy compliance (Table 1.1).

8

P. Dessus

Table 1.1 Main characteristics of the observational approaches Approach

Behavioral account

Behavioral Isolated

Classroom variables

Privacy compliance

Classroom and resource management High

Ecological

External, 2nd person Socio-emotional

Mid

Enactivist

Internal, 1st person

Low

Socio-emotional and cognitive

1.3.1 Behavior-Centered CACs From a purely behavioral standpoint, the role of a CAC is to capture elementary simple human behaviors and to trigger some events in turn. The application range of this view is already large: for instance, a CAC can react when a teacher enters and put lights and some devices on. It also can have some action recognition processes to assist attendees’ actions [33], e.g., recognizing teacher’s gestures to adjust the camera’s focus. Most of the behavior-centered CACs gather some raw features of the situation to trigger a low-level information action or a label attribution, like students’ attendance, students’ badges, powering a device on or off, etc. Their goal is simply derived from computer-based learning analytics systems that gather and compute student logs to deliver raw feedback. These classrooms are based upon the idea that a large number of students can be scrutinized by the CAC (a teacher cannot pay attention to all the students’ problems at the same time), and that the teacher can derive an overall management strategy from this log data [8]. The limitations encountered in the implementation of such CACs are as follows. First, they focus on the behaviors of some students (e.g., with special needs) because they occur more than others’. Second, the context in which behaviors are undertaken is often opaque and not captured, as behaviors are captured as isolated. Third, privacy concerns arise, since the attendees can be object to identification, but the intrusiveness of the approach is less important than this of the following approaches because fewer personal data is processed, from fewer people.

1.3.2 Ecological CACs Ecological CACs are centered on the multiple social events occurring within them. For instance, video footages can be analyzed to provide information about some attendees’ performance or status. In that vein, a recent study [34] investigated the teacher–students’ engagement behavior in the classroom, double-coded by humans. A set of teachers’ and students’ behaviors was determined (e.g., writing, asking, and pointing to the presentation) and their congruency over time was human-coded (e.g., a student who is writing when the teacher is pointing at something on the board is likely to be disengaged), then a classifier automatically attributed a students’ engagement score depending on the previous teachers’ behavior. The results show

1 Context-Aware Classrooms as Places …

9

that the classifier more accurately predicts engagement when previous teachers’ behavior is accounted for in the model and bound to students’ own behavior. In that way, ecological CACs can be the place to implement a more thorough and direct behavior rating [8] and also be the place for testing more elaborated pedagogical practices, like active learning [35]. The problems encountered in the implementation of ecological CACs are the following. First, even if some contextual elements can be captured, the way they are actually perceived by the CAC’s attendees is not accounted for. Second, and compared to behavioral CACs, more data is gathered from the environment (e.g., video, location, and posture) and explanatory models of activity relying on this data are needed to explain or predict attendees’ behavior. Third, since more personal data is processed in this approach, attendees’ privacy is hindered.

1.3.3 Enactivist CACs Up to now, enactivist CACs per se do not exist yet, even though recent apparatus like mobile eye trackers [36, 37], electrodermal response trackers [31], or other types of sensors capturing various individual data, like pulse, body temperature [38], as well as multimodal learning analytics capturing emotion and gaze recognition systems [39], can help gather and analyze instructional situations in an enactivist way: first-person-based and accounting for context more fully. As a promising example of what an enactivist CAC could be, researchers [40] developed ACORN, a multimodal machine learning system that analyzes audio and video features of instructional events footages to infer classroom climate, as modeled in the Classroom Assessment Scoring System (CLASS) [41], a reliable and well-studied classroom observation system. The results showed medium correlations between human and machine coding on two CLASS dimensions (positive and negative climate). The intensive computer processing time needed in this approach prevents to deliver real-time information, which is a benefit since many of the information is very intrusive. This point is the most concerning about enactivist CACs, because the first-person data will allow to gather and infer privacy-related information about attendees.

1.4 Discussion In this paper, we presented three theoretically grounded approaches to automatically capture and analyze instructional events in Context-Aware Classrooms. Since a large extent of research has been devoted so far to behavioral approaches, designing ecological or enactivist classrooms is a path toward a better understanding of classroom environments and of the many activities teachers and students undertake. More complex variables like socio-emotional and cognitive can be observed and analyzed.

10

P. Dessus

However, this benefit comes with a price, which is a huge processing of personal data hindering attendees’ privacy. CACs as an instructional technology allow to shed light on some new research concerns: CACs as components of smart universities and CACs as distance learning places. With the spreading of the Internet of things, an often-encountered story is to consider smart spaces as Russian dolls: several CACs compose a smart university, which in turn can be part of a smart city, etc. We consider CACs as the first bricks of smart universities very cautiously: the intention to improve students’ experience and learning outcomes cannot be fulfilled by the massive capture of personal data, which can entail massive surveillance [42]. Our point is that the intensive processing power available could either be employed to obfuscate personal data at a group level [43]. Even though classrooms are either places for direct instruction or more distant forms thereof, CACs are not neutral media that simply deliver learning or teaching experience in distant places, and these two forms of instruction differ in many aspects (e.g., students’ characteristics and pedagogical strategies). Our point is that e-learning or hybrid situations need to be carefully designed for a sound integration in CACs and that our categorization may help, depending if the emphasis is on behaviors, ecology, or enaction. Another crucial and final point is not to put teachers and students on the sideline by devising automated decision-making tools that replace teachers’ care and empathy by sharper and colder CACs “decisions” [44]. Our point is to deliver CACs’ analysis after the lesson and to prevent from surveillance and cognitive load overwhelming. In this way, the post-pandemic teachers and researchers would benefit from a novel place to capture, observe, and analyze instructional events more comprehensively. Acknowledgements We wish to thank Romain Laurent and Dominique Vaufreydaz for their comments on a previous version of this paper. Our work was partly funded by the “IDEX formation 15-IDEX-0002” grant, University Grenoble Alpes.

References 1. Hodges, C., Moore, S., Lockee, B., Trust, T., Bond, A.: The difference between emergency remote teaching and online learning. EDUCAUSE (2020) 2. Walker, K.A., Koralesky, K.E.: Student and instructor perceptions of engagement after the rapid online transition of teaching due to COVID-19. Nat. Sci. Educat. 50(1), (2021) 3. Panadero, E., Fraile, J., Pinedo, L., Rodríguez-Hernández, C., Balerdi, E., Díez, F.: Teachers’ well-being, emotions, and motivation during emergency remote teaching due to COVID-19. Front. Psychol. 13, 826828 (2022) 4. Bartlett, L.: Specifying hybrid models of teachers’ work during COVID-19. Educ. Res. 51(2), 152–155 (2022) 5. Montebello, M.: The ambient intelligent classroom. Beyond the indispensable educator. Springer, Cham (2019) 6. Antona, M., Leonidis, A., Margetis, G., Korozi, M., Ntoa, S., Stephanidis, C.: A Student-centric intelligent classroom. Ambient Intell. 248–252 (2011)

1 Context-Aware Classrooms as Places …

11

7. Phoong, S.Y., Phoong, S.W., Moghavvemi, S., Sulaiman, A.: Effect of smart classroom on student achievement at higher education. J. Educat. Technol. Syst. (2019) 8. Woolverton, G.A.: An Exploration and Critical Examination of How “Intelligent Classroom Technologies” Can Improve Specific Uses of Direct Student Behavior Observation Methods. Educational Measurement: Issues and Practice (2021) 9. Morozov, E.: To save everything, Click Here: Technology, Solutionism, and the urge to fix problems that don’t exist. Lane, Londres (2013) 10. De Hert, P., Gutwirth, S., Moscibroda, A., Wright, D., González Fuster, G.: Legal safeguards for privacy and data protection in ambient intelligence. Personal Ubiquitous Comput. 13(6), 435–444 (2008) 11. Dunkin, M.J., Biddle, B.J.: The study of teaching. Holt, Rinehart and Winston, New York (1974) 12. Perlman, M., Falenchuk, O., Fletcher, B., McMullen, E., Beyene, J., Shah, P.S.: A systematic review and meta-analysis of a measure of staff/child interaction quality (the Classroom Assessment Scoring System) in early childhood education and care settings and child outcomes. PLoS One 11(12) (2016) 13. Macmillan, S. (ed.): Designing better buildings. Quality and value in the built environment. Spon Press, London (2004) 14. Nuthall, G.: The hidden lives of learners. NZER Press, Wellington (2007) 15. Rimm-Kaufman, S.E., Baroody, A.E., Larsen, R.A.A., Curby, T.W., Abry, T.: To what extent do teacher–student interaction quality and student gender contribute to fifth graders’ engagement in mathematics learning? J. Educ. Psychol. 107(1), 170–185 (2015) 16. Shermis, M.D., Di Vesta, F.: Classroom assessment in action. Rowman & Littlefield, Lanham (2011) 17. Praetorius, A.-K., Klieme, E., Herbert, B., Pinger, P.: Generic dimensions of teaching quality: the German framework of Three Basic Dimensions. ZDM Mathematics Education 50(3), 407– 426 (2018) 18. Hew, K.F., Lan, M., Tang, Y., Jia, C., Lo, C.K.: Where is the “theory” within the field of educational technology research? Br. J. Edu. Technol. 50(3), 956–971 (2019) 19. Landrum, T.L., Kauffman, J.M.: Behavioral approaches to classroom management. In: Evertson, C.M., Weinstein, C.S. (eds.) Handbook of classroom management, pp. 47–71. Routledge, New York (2006) 20. Simondon, G.: On the mode of existence of technical objects. University of Minnesota Press, Minneapolis (2017) 21. Swartz, J.L., Martin, W.E.: Ecological psychology theory: Historical overview and application to educational ecosystems. In: Swartz, J.L., Martin, W.E. (eds.) Applied ecological psychology for schools within communities, pp. 3–27. Erlbaum, Mahwah (1997) 22. Conoley, J.C., Haynes, G.: An ecological approach to intervention. In: D’Amato, R.C., Rothlisberg, B.A. (eds.) Psychological perspectives on intervention: A case study approach to prescriptions for change, pp. 177–189. Longman, New York (1992) 23. Lermigeaux-Sarrade, I.: Rôle de l’organisation de l’espace de travail dans les activités effectives et empêchées des enseignants [Role of work space organization in impeded or actual teacher activities]. Université Grenoble Alpes, Grenoble, PhD Thesis in education (2018) 24. Jornet, A., Dam¸sa, C.: Unit of analysis from an ecological perspective: Beyond the individual/social dichotomy. Learn. Cult. Soc. Interact. 31, 100329 (2019) 25. Poizat, G., Flandin, S., Theureau, J.: A micro-phenomenological and semiotic approach to cognition in practice: a path toward an integrative approach to studying cognition-in-the-world and from within. Adapt. Behav. (2022) 26. Sahlström, J.F.: The interactional organization of hand raising in classroom interaction. J. Classr. Interact. 37(2), 47–57 (2001) 27. Kawahara, T., Iwatate, T., Takanashi, K.: Prediction of turn-taking by combining prosodic and eye-gaze information in poster conversations. In: Proceedings of International Conference Interspeech, pp. 727–730, Portland (2012)

12

P. Dessus

28. Dieumegard, G., Nogry, S., Ollagnier-Beldame, M., Perrin, N.: Lived experience as a unit of analysis for the study of learning. Learn. Cult. Soc. Interact. 31, 100345 (2019) 29. Noë, A.: Out of our heads. Why You Are Not Your Brain, and Other Lessons from the Biology of Cousciousness. Hill and Wang, New York (2009) 30. Davis, B., Sumara, D.J.: Cognition, complexity, and teacher education. Harv. Educ. Rev. 67(1), 105–125 (1997) 31. Horvers, A., Tombeng, N., Bosse, T., Lazonder, A.W., Molenaar, I.: Detecting emotions through electrodermal activity in learning contexts: a systematic review. Sensors, 21(7869), (2021) 32. Yu, C.: Linking words to world. An embodiment perspective. In: Shapiro, L. (ed.) The Routledge Handbook of Embodied Cognition, pp. 139–149. Routledge, London (2014) 33. Franklin, D., Flachsbart, J., Hammond, K.: The intelligent classroom. Intell. Syst. Appl. 14(5), 2–5 (1999) 34. Zhao, J., Li, J.M., Jia, J.: A study on posture-based teacher-student behavioral engagement pattern. Sustain. Cities Soc. 67 (2021) 35. Talbert, R., Mor-Avi, A.: A space for learning: An analysis of research on active learning spaces. Heliyon 5(12), e02967 (2019) 36. Cortina, K.S., Miller, K.F., McKenzie, R., Epstein, A.: Where low and high inference data converge: Validation of CLASS assessment of mathematics instruction using mobile eye tracking with expert and novice teachers. Int. J. Sci. Math. Educ. 13(2), 389–403 (2015) 37. Dessus, P., Cosnefroy, O., Luengo, V.: Keep your eyes on ‘em all!: A mobile eye-tracking analysis of teachers’ sensitivity to students. In: Verbert, K., Sharples, M., Klobuˇcar, T. (eds.) Adaptive and adaptable learning. In: Proceedings of 11th European Conference on Technology Enhanced Learning (EC-TEL 2016), pp. 72–84. Springer, New York (2016) 38. Chiu, C.-K., Tseng, J.C.R.: A Bayesian Classification Network-based Learning Status Management System in an Intelligent Classroom. Educ. Technol. Soc. 24(3), 256–267 (2021) 39. Ahuja, K., Shah, D., Pareddy, S., Xhakaj, F., Ogan, A., Agarwal, Y., Harrison, C.: Classroom digital twins with instrumentation-free gaze tracking. In: 39th Annual SIGCHI Conference on Human Factors in Computing Systems (CHI’21). ACM, Yokohama (Japan) (2021) 40. Ramakrishnan, A., Zylich, B., Ottmar, E., LoCasale-Crouch, J., Whitehill, J.: Toward Automated Classroom Observation: Multimodal Machine Learning to Estimate CLASS Positive Climate and Negative Climate (2020) 41. Pianta, R.C., La Paro, K.M., Hamre, B.K.: Classroom Assessment Scoring System: Manual K-3. Brookes, Baltimore (2008) 42. Kwet, M., Prinsloo, P.: The ‘smart’ classroom: a new frontier in the age of the smart university. Teach. High. Educ. 25(4), 510–526 (2020) 43. Petrova, A., Vaufreydaz, D., Dessus, P.: Group-level emotion recognition using a unimodal privacy-safe non-individual approach. In: 8th Emotion Recognition in the Wild Challenge (EmotiW) International Conference Joint to the ACM International Conference on Multimodal Interaction (ICMI 2020). ACM, Utrecht (2020) 44. Selwyn, N.: Less work for teacher? The ironies of automated decision-making in schools. In: Pink, S., Berg, M., Lupton, D., Ruckenstein, M. (eds.) Everyday Automation: Experiencing and Anticipating Automated Decision-Making. Routledge, London (2022)

Chapter 2

Student Smartphone Experience Narratives Mediated by the Phygital School Library for Learning Ecosystems Maria José Fonseca and Óscar Mealha

Abstract This paper reports a Case Study that took place in the educational community of the D. Maria II school cluster at Famalicão, Portugal, from October 2019 to October 2021. It integrated 236 participants—students, teachers, and parents— with the goal of identifying the students´ daily infocommunicational experience narratives on their smartphones, capable of also being used for classroom activities. A Design-Based Research process was used with a conceptual prototype that simulated user narratives. The findings of this study were proposed as a heuristic model of infocommunicational services that students would like to have on their smartphones to be used in/out of the classroom with the mediation of the physical and digital (phygital) school library. Socialization and collaborative learning solutions were the most popular, such as student/teacher profile information, peer assessment, inter-student and teacher infocommunication possibilities, and student performance continuous follow-up. This study’s research process revealed potential to be used in other school ecosystems and the school library and librarian teacher proved to be absolutely fundamental agents to mediate the development, discussion, and validation of the findings and final results.

2.1 Introduction The research reported in this paper delivers contributions and nurtures some answers to issues of great concern for SLERD 2022—“What should we expect for future learning ecosystems? How “smart learning ecosystems” are changing? How such changes may be related to the achievement of “a better learning for a better world” M. J. Fonseca (B) School Cluster D. Maria II de Famalicão, 4760-067 Gavião, Vila Nova de Famalicão, Portugal e-mail: [email protected] M. J. Fonseca · Ó. Mealha Department of Communication and Art/DigiMedia, University of Aveiro, 3810-193 Aveiro, Portugal e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_2

13

14

M. J. Fonseca et al.

as a contribution to the United Nations 2030 Agenda for Sustainable Development Goals (SDG)? How will they contribute to the reduction of inequalities and, at the same time, to the empowering of each individual according to her/his expectations and talents?” Nowadays teenagers and younger generations have clearly adopted a digital technology-mediated way of life. Many of their daily tasks are managed with the help of digital services interfaced with their smartphones, tablets, or personal computers. Learning ecosystems, namely schools, have also adopted technological platforms (Google Classroom—https://classroom.google.com/u/0/h; Google Meet—https:// meet.google.com/; Google Drive—https://myaccount.google.com/; Padlet—https:// padlet.com/; Socrative—https://www.socrative.com/; YouTube; Kahoot—https:// kahoot.it/; Plickers—https://get.plickers.com/) designed for this purpose but guided by the professional and specialist’s perspective of optimal services, functions, and characteristics, rarely considering the student or learner’s perspective or user narrative during the design process. Future learning ecosystems’ technologies should tend to adapt to the student’s experience narrative, optimized to align with a student’s expectation and fruition, and promote an optimal experience [1]. Only in this way can the student-centered learning process make sense and be coherent with the trendy active learning strategies. All have to be in place and designed to deliver the same learning strategy and approach: a student-centered learning process; adequate learning strategies, materials, and infrastructures; technology-mediated services that incorporate the student experience narratives. Another pertinent issue is related to the promotion and engagement of change in a learning ecosystem and who are the best players to manage it. Could the school library and the librarian be one of these agents? In line with a new educational paradigm, it is necessary to promote and enhance a different way of learning by modeling an education for autonomy, critical thinking, and learning, in different formats, throughout life. In this sense, the educational ecosystem now has a greater responsibility in the development and promotion of new fundamental skills associated with information triage such as research, selection, and its critical and careful treatment, so that students can meet the demands of a status quo influenced by information. This “new era” mediated/immersed in technologies has also changed the trajectory of the role and mission of the librarian teacher. According to the American Association of School Librarians (AASL), the librarian teacher as an agent of change with specific skills can motivate and influence the different actors of the educational ecosystem, identify the needs, considering the uniqueness of students, to act and plan with teachers to “co-plan, co-teach, and co-assess a lesson or unit of instruction” [2]. As such, AASL proposes that the librarian teacher can contribute to “leveraging” educational success by providing training in the use of technology and information to promote the critical and creative freedom of students, while respecting the freedom to access and use information ethically. The democratic, collaborative, and leadership connection with different educational actors is an added value in sharing resources, enhancing more learning opportunities in educational ecosystems. For the same reason, Hughes et al. [3] consider that: the librarian teacher plays an

2 Student Smartphone Experience Narratives Mediated …

15

important role in creating a student-friendly library environment that is conducive to learning. With dual qualifications as teacher and information professional, teacherlibrarians have a specialist role that combines key responsibilities, as curriculum leader, information specialist, and information services manager (p. 322). The necessary accompaniment of changes presents itself as an educational mission, sometimes also diplomatic, for the librarian teacher and naturally for the library, as the interface between the user and the access to information that takes place in different formats and spaces. The key roles of the librarian teacher in teaching, management, leadership, collaboration, and community engagement reinforce the centrality of the library in the educational ecosystem as a collaborative partner in the overall learning and formation of students. This new positioning implies the appropriation of other formats of organization and management of the library, to respond to the wishes and interests of different educational actors, in a close articulation among all. Unlike the heritage of a physical, controlled space, the library can be anywhere, accompanying the users wherever they are (at school, at home, on the bus, or in the city park), providing informative content at the click of the user. This ubiquity will enable impactful engagement as the users can easily find what they are looking for. IFLA [4] emphasizes that “physical and digital access to the library should be maximized” through technology, and digital access to school library information resources can be provided throughout the school and beyond. This two-way format provides democratic access to school library resources at any time and place. School libraries end up becoming spaces of symbiosis between the physical and the digital, with the relation of different document formats and information, more than collection spaces, they become connected-user environments. According to Conde [5] school libraries “challenge the traditional way of pedagogical organization focused on the classroom’s closed space” becoming a “powerful element of change and innovation, capable of providing new ways of learning and interacting”, inside or outside the school space “extending the time and space of learning from the classrooms to outside them and inducing new practices and literacies”. These new, open, furnished, and enjoyable workspaces can be made available at different levels, triggering a renewed positioning, contemplating the physical space and the digital space of the school library, and establishing the link between “the two worlds”. Anywhere, anytime “mixed” reality extends the physical environment into the digital environment, in many cases using augmented reality (AR) to enrich the experience beyond the tangible, for instance, “a phygital interface is capable of conveying distinct layers of information related to a heritage object depending on the actual communication medium, ranging from traditional displays to portable or wearable AR technology” [6]. In an updated record, the ability to democratically respond to the needs of the educational ecosystem takes place beyond the physical space of the library, augmented by digital literacy in a plan of interdisciplinary literacy as a phygital environment [7].

16

M. J. Fonseca et al.

The work reported in this paper also addresses the process that should be adopted to develop change and effectively achieve “a better learning for a better world”, a major concern announced in SLERD 2022, as a contribution to the United Nations 2030 [8] Agenda for Sustainable Development Goals (SDGs). A participatory research process developed in the scope of a community was adopted and capable of engaging all educational community stakeholders. Students, teachers, and parents were participants of a qualitative inquiry, inductive research project contextualized as a case study at the head school of school cluster D. Maria II of Vila Nova de Famalicão, Portugal. The student’s perspective and experience narratives were the driving force to codesign an infocommunicational model mediated by a smartphone, in fact capable of aligning with a Bring Your Own Device (BYOD) [6] approach.

2.2 Related Work As Chou and Chang [9] emphasize, educational ecosystems need to explore other creative formats associating BYOD to the methodologies used. The use of personal mobile devices may solve some problems, namely the lack of resources, and replace traditional assessment tools. The BYOD phenomenon, after its appearance 10 or 15 years ago, has undergone constant changes, in particular the inclusion of smartphones (about 90%), due to increased connectivity and computing capabilities that allow a performance like certain laptops Barlette et al. [10]. The students, by having the possibility to bring their own device to school, will facilitate a reorganization of practices, with emphasis on “networked learning”, promoting student–student and student–teacher connection, in a community of resources, interests, and learning. Reference [11] also consider that the increased use of digital devices in educational environments has decentralized the knowledge of the teacher and the manual, through access to online teaching resources, on the screens of the students’ devices, assuming new educational practices. It should also be added that the response system, more focused on the student and his/her learning, constitutes a tool for motivation and improvement, arousing the students’ attention. Given the empirical intervention of [12] on the experiences of students regarding the advantages and challenges of the BYOD model, results revealed that the devices authorized for classroom activities provided a more comfortable environment for studying. Reference [13], in a study conducted in Australia, highlight the role of libraries as cultural spaces for learning in the communities where they are located. It was found that libraries have been using technology to promote a participatory learning culture to bring together and promote inclusion. The examples presented in the mentioned study, although referring to public libraries, clearly expose the need for school libraries to also seize the opportunity of technology convergence in mobile devices to bring communities closer, increasing the interaction and the respective social impact. Libraries as information search spaces had necessarily to adjust to the possibilities of mobile technology, and users are always waiting for information to reach them. According to [14] quoted by [13]: “People today are tied to the mobile

2 Student Smartphone Experience Narratives Mediated …

17

phone as the center of their information ecosystem, and more and more of these phones are providing an interface to nearly all of their informational needs” (p. 14). The need to integrate different communities in the daily life of libraries has been and continues to be a concern, and smartphones can enable a new model of participation and implication, facilitating dialog and collaboration and promoting the creation of new contents. As students enter the school library with their smartphones at hand, it is important to rethink, reposition, and speak the same language and bring services closer to the user, envisaging new infocommunicational formats available to all, in a fast, intuitive, and valid way. This does not mean that the user should be tied to the device and ignore all the physical resources of the library, but that they should know how to manage, optimize, and use different instruments and formats, always on a conscious, balanced, and critical basis when accessing, searching for, and validating information. The clear valorization of new practices, considering the digital spaces and artifacts used by young people in learning and in the possibility of doing other things, is present in the Transliteracy H2020 Research and Innovation Actions. However, it is precisely through the creation of an emotional environment and the co-design path that teachers can tune in with students. This activity in listening and empathy, even beyond mediacentric teaching practices, will have a significant impact on students and will create an effective bridge between their interests and the institutional aims of the school [15]. In the MILAGE app Aprender + (https://apps.apple.com/pt/app/milage-learn/id1140 872254) and in the SMARTEEs Project—Smartphones in Educational Ecosystems [16] there is a need for mobile devices to be used in the educational process, in particular the smartphone. At the same time, the resistance to technology is still recognized, when any artifact in the classroom can be considered a technology, since the time of the blackboard, the scratchpad, and even the book itself. However, it is not enough to equip schools with computers if they are not used or updated. The associated fears or the lack of training puts obstacles in most cases, note what Scolari [15] states: “With this, there arises the need to promote a media education to learn to move within the social, political, cultural and educational context in which we are living” (p. 129). The situation described results from the use of Modernity interfaces in a school still hostage to many eighteenth-century parameters, in a mismatch in need of “redesigning the educational and political interfaces” [15]. For this redesign, it is pertinent to listen to the different actors, mapping the intentions, giving voice to the key elements in a holistic path of modernization, and informing the purposes of change, in a process of iterative cycles, as in the case of SMARTEEs. This process needs to engage students, teachers, and families for a common design and leverage the possibility of including the smartphone, inside or outside the classroom, enhancing new phygital environments for experiential and reflective smarter learning.

18

M. J. Fonseca et al.

2.3 Method Vila Nova de Famalicão is a Portuguese city in the Ave sub-region, belonging to the Northern region and Braga district, with 20 935 inhabitants in its urban perimeter. It is the seat of the municipality of Vila Nova de Famalicão, with a total area of 201.59 km2 and 133 574 inhabitants, subdivided into 34 parishes. The principal researcher, a librarian teacher, was also a contextual observer of the relevant situations in and outside the library. A Design-Based Research (DBR) process was chosen to better design and validate an infocommunicational model mediated by smartphones to be used for learning purposes. The infocommunicational model was, since a very early stage, represented and discussed as a conceptual digital prototype and used as a research instrument during the different inquiry phases to validate and improve the model. The following Fig. 2.1 depicts a symbolic representation of the methodological approach.

2.3.1 Procedure The empirical study was contextualized by a case study and gathered empirical data in different moments of the research timeline, Fig. 2.1, integrating different research instruments organized by a DBR process: (i) a systematic literature review (SLR) that informs the main concepts, methods, and related work reported in this paper; (ii) the benchmarking process that compared similar platforms and applications that are being used in the classroom and other social/collaborative platforms, used by students capable of also being configured to be used in the classroom; (iii) questionnaires, mediated by the research team and applied to students, teachers, and parents, to collect

Fig. 2.1 Symbolic representation of the Case Study with different research phases and data collection, integrating a Design-Based Research process

2 Student Smartphone Experience Narratives Mediated …

19

a corpus of student experience narratives and opinions; (iv) direct observation; and (v) iterative (DBR) design of a conceptual prototype used to test participant’s experience and rate their opinion on a set of scenarios and user narratives co-designed with data collected from student’s experience narratives and opinions. The first questionnaire, applied as open (OQ) and closed questions (CQ), was organized into three main dimensions, as detailed in Table 2.1—media literacy, smartphones, and school library. The organization of the questionnaires, with similar questions, was processed into three questionnaires directed to the three groups of participants: students (5th, 7th, and 9th grades), teachers (2nd and 3rd study cycles), and parents. It was structured with closed questions to score opinions on a linear numerical scale between 0 (minimum) and 9 (maximum), as well as open questions, to collect the opinions and/or comments regarding the score in the closed questions. Table 2.1 Relationship of the Survey Questions to the Dimensions under Study: Media Literacy, Smartphones and School Library, CQ, and related OQ Media literacy CQ1. Do you have a mobile phone with smartphone features? CQ2. How long have you been using it? CQ3. Would you like to have a smartphone and what for? CQ4. How many hours do you use your smartphone per day? CQ5. How many days do you use your smartphone per week? OQ5. For what purpose and which applications do you usually use your smartphone? CQ6. Do you usually use your smartphone at school? RF: Yes or No CQ7. Where in school do you use it? RF: Yes RF: Multiple choice answers:(1) Classroom; (2) Library; (3) Refectory; (4) Playground OQ7. ANS: No 7. What is the reason? OQ7. ANS: Yes 7. What use do you make of your smartphone in these places? Smartphones Classroom CQ8. Can the smartphone be a working tool, for teaching and learning, in a classroom context? CQ9. What activities could be added in the classroom by using the mobile phone (smartphone) to enhance teaching and your learning? RF: Multiple choice answers: (1) Content sharing (teacher-student); (2) Clarification of doubts (student–teacher and between students); (3) Study organization; (4) Self-study; (5) Information search; (6) Group work; (7) Presentation of work; (8) Evaluation (teacher-student and between students); (9) Information (teacher-student and between students); (10) Decision-making (class voting process); (11) Curriculum challenges; (12) No activity OQ9. Given your answers to the previous question, in what ways would you use the smartphone to carry out the activities? And to solve what? OQ9. In which subjects could the smartphone be used, inside or outside the classroom? Please indicate why? (continued)

20

M. J. Fonseca et al.

Table 2.1 (continued) CQ10. Do you agree that there should be rules on the use of smartphones at school? OQ10. In view of your score, would you please comment on and state the rules of your school regarding the use of the smartphone? Outside the classroom CQ11. What about outside the classroom? Can the smartphone still be a working tool? CQ12. Outside the classroom what activities would you use on the smartphone to enhance teaching and your learning? RF: Multiple choice answers: (1) Distance learning (class constitution, presentation, and discussion of work); (2) Content sharing (teacher-student and teacher-tutor); (3) Clarification of doubts (student–teacher, between students, teacher-tutor); (4) Organization of study; (5) Monitoring school progress; (6) Self-study; (7) Researching information; (8) Group work; (9) Evaluation (teacher-student and between students); (10) Information (teacher-student and between students); (11) Decision-making (class voting process); (12) Curriculum challenges. No activity OQ12. Given your answers to the previous question, do you use your smartphone differently outside the classroom? Why do you do this? OQ12.What have you enjoyed most related to using the smartphone at school? OQ12. What are the biggest problems with using the smartphone at your school? OQ12. What about at home? Do you have problems using the smartphone that you would like to share with us? School Library CQ13. The school library is a physical and digital (virtual) space for co-producing and sharing content OQ13. Given your score, how might the school library improve its services to enhance your learning? CQ14. Would the existence of a dedicated app make it easier to access the information and services that the school library provides? CQ15. Given your answer to the previous question, please tick below all the key features that this app should have? RF: Multiple choice answers: (1) Distance learning area (e.g. Google Meet, Microsoft Teams, Zoom, etc.); (2) Dissemination/reaction area (e.g. Classroom, Edmodo, Facebook Groups, Youtube, Reddit, etc.); (3) Personal/collaborative work area (e.g. Google Drive, Jamboard, Dropbox, Microsoft Office Online, Google Meet, Microsoft Teams, Mic, Zoom, etc.); (4) Organization/study planning/tasks/events area (e.g.: Google Classroom, Google Calendar, Edmodo, Padlet, Socrative, Google Keep, Microsoft Outlook, Notion, Asana, Trello, etc.); (5) Communication area (e.g. Messenger, WhatsApp, Slack, Microsoft Teams, Zoom, Skype, etc.); (6) Assessment (teacher-student and between students) (e.g.: Google Classroom, Kahoot, Google Forms, Edmodo, Padlet, Socrative, Poll Everywhere, etc.); (7) School performance/progress visualization (e.g. Edmodo, Padlet, etc.); (8) Information/document sharing (e.g.: Google Classroom, Google Drive, Microsoft Teams, Zoom, Skype, Email, Dropbox, WeTransfer, etc.); (9) Content search (Google, Reddit, Youtube, Wikipedia, etc.); (10) Decision-making (in a voting/comment process) (e.g. Kahoot, Poll Everywhere, Socrative, etc.); (11) Posting curriculum challenges (e.g. Kahoot, Facebook, Instagram, WhatsApp, Reddit, etc.); (12) No opinion OQ15. Why don’t you agree with an app with these features? OQ16. In this conversation do you think we have missed any issues related to the use of smartphones in your school or at home?

2 Student Smartphone Experience Narratives Mediated …

21

These questionnaires were applied to a sample of 115 participants, 75 students (5th grade, n = 33; 7th grade, n = 25; 9th grade, n = 17); 20 teachers; and 20 parents. The research process took place at the head school of D. Maria II school cluster at Famalicão, Portugal, from October 2019 to October 2021. Most of the empirical work took place during COVID-19 and more than 80% of the data collection process was done online via Zoom audiovisual platform. The research process had the formal approval of the Management Board of the school cluster and of the Portuguese Ministry of Education with code no. 0,576,100,002 that included the approval of research instruments, research process, and all ethical consent forms and procedures.

2.4 Student Experience Narratives and the Prototype After identifying how students use their smartphones, (at school, classroom, and outside the classroom), Figs. 2.2 and 2.3 , results reveal that only 9.33% (7/75) do not own a smartphone. Regarding this and asked whether those who did not have a smartphone would like to have one, the five participants of the 5th grade answered yes to play, communicate, and socialize and to be able to play with my friends, to talk to them from places where they are not, and to talk to my parents. As for grade 7 participants, the two students said they did not have a smartphone, but they would like to have one to communicate better with everyone in the classroom and do activities. Associated with the previous elements, the students were asked about the purpose for which they used the smartphone—there are activities common to three grades (5th, 7th, and 9th grades), namely playing games; communicating; watching videos; sending assignments; researching; taking photographs; social networks. The data collected from this initial inquiry process was processed by the research team and designed into eight scenarios and corresponding user narratives as systematized in Table 2.2. The following relation of smartphone screen images, depicted in Fig. 2.4, represents two of the student’s experience narratives, Profile/Disciplines and Former Grades, used in a 1st version of the conceptual prototype, two left screens, and then re-designed in an iterative DBR participatory process to achieve the two screens on the right. All eight scenarios and user narratives, designed in a DBR iterative participatory process, can be explored in detail in the 2nd version of the conceptual prototype available at this link—https://bit.ly/3Od96Xy.

2.5 Heuristic Model of Infocommunicational services This section describes and discusses the main findings and contributions of this work, the infocommunicational services validated with the DBR process, and the conceptual prototype, in the case study. These services, a contribution of this study,

22

M. J. Fonseca et al.

Fig. 2.2 Student score mentioning how they use their smartphones collected in the first questionnaire

Fig. 2.3 Student experience narratives collected in the first questionnaire applied and mediated by the research team

were identified in the specific context of this research and their pertinence is not generalizable to other learning ecosystems. The research process on the other hand proved to be adequate to be applied in other similar learning ecosystems because the method and data collection techniques are not context dependent.

2 Student Smartphone Experience Narratives Mediated …

23

Table 2.2 Scenarios used to iteratively design the conceptual prototype in a DBR process Scenarios

Goals

1

Simulate the registration experience, visualize the tutorial, and explore the information available in the student’s personal profile

2

Explore the information available about the subjects (general objectives, timetable, and teacher profile)

3

Consult student’s former grades and school calendar

4

Initiate a private conversation with teachers to ask questions, from the teacher’s profile or from the conversations tab

5

Create a conversation group from the option in the main menu

6

Access an activity, create a document, and view the information provided by teachers from the library

7

Explore notifications and understand the process of submitting activities and assessments

8

Identify the functionalities of the library and consult the books available

Fig. 2.4 Student experience narratives of Profile/Disciplines and Former Grades—two left screens 1st version and two screens on the right—2nd version of the conceptual prototype

The infocommunicational services that were identified and validated by the different sample’s cohorts, collected from students’ experience narratives can be described as follows: • Access to personal information—related to the student’s participation in extracurricular activities (reading contests, spelling, school sports, mathematics Olympiad, clubs, or others). It includes a private component, where the student can consult the school calendar, the timetable, information about the parent/tutor, and his/her former grade history.

24

M. J. Fonseca et al.

• Implementation of “interest badges” —this is associated with the personalization of information and motivation that may influence learning, allowing the student to select three badges to add to the profile. Filters may be needed in the sharing of interests and should be supervised by parents. The students consider that knowing the interests of his/her classmate will mean understanding him/her better, which, in part, may prevent, e.g. the activation of verbal bullying situations and alternatively foster empathy. Badges were mentioned as graphical representations of a typology of interests to quickly identify common interests among community members such as students or teachers, e.g. fishing, birdwatcher, volley-ball player, science-fiction reader, etc. • Interpersonal communication—resulting from the proximity of the different educational actors, namely teacher-student, which will favor the learning process, such as the clarification of questions (student–teacher), as well as more confident and motivated situations in the classroom. The proximity relationship that arises from this will provide greater sociability between the different players due to the common interests identified, favoring the creation of a more empathic climate in the educational ecosystem. • Organization of information—intuitive access to useful and relevant information, adjusted to the educational ecosystem, with updated content, directed and organized for each school year and subject, with the possibility of following the activities/tasks in each one of them. According to the empirical study, if the information were interfaced on a single device and customizable, it would encourage more timely and effective usage assimilation. • Student performance information—the immediate and systematized access to relevant information presenting a generic and detailed on-demand view of the student’s performance regarding each subject, curricular and extracurricular activities, will facilitate a closer and effective monitoring of the student’s school career. • Synchronous and asynchronous conversations—possibility to start conversations with colleagues and teachers, as well as to create groups to support collaborative work or just to chat. • Collaborative work—in conceptual terms, will facilitate group work, the creation and sharing of documents in various formats (text, presentations, excel…). This collaborative work possibility, with colleagues, will allow fine-tuning details and automating procedures so that the student does not get lost. Consider the insertion of a whiteboard, for the execution of joint and synchronous activities. • Submitting work for assessment—in a useful and easy procedure, the student should be able to develop and submit some of the schoolwork forms anywhere and anytime, inside or outside the classroom. The fact that “all” schoolwork could be on the same platform highlights the interest and purpose of such a submission service. • Peer-to-peer evaluation—creation of new learning opportunities, in symbiosis with the experience and the critical, positive, and constructive reflection that will enhance, in the classroom or outside the classroom, a more attentive positioning

2 Student Smartphone Experience Narratives Mediated …

25

toward disciplinary contents. When critical exercises are carried out, the students are called to participate and to know how to evaluate each other; however, critical thinking and attention may be a challenge, easily overcome with learning, training, and the opportunity to do so in a common collective process. • Personalized feedback to the student—considers a new format of evaluation where positive criticism may favor learning by valuing the student’s work and encouraging her/him to continue. The possibility for the teacher to give timely feedback with a comment about the student’s work, explaining how to improve for next time, as well as to follow the hetero-evaluation discussions could be more efficient and the feedback faster and more assertive. Both, peer assessment and the feedback that is given by the teacher in this study, consider that young people appreciate these interactions, and it is essential to allow them to do so. • Study management—study management, associated with a notification service, sent, e.g. a week before the day of an evaluation sheet, constitutes an extra alert for the student to start studying for a certain subject. In this way, they will have more time to organize themselves in a more effective and attentive way, using the services of the school ecosystem/library. This should also avoid last-minute study and facilitate access to the content made available by the teachers and library. This set of infocommunicational services systematizes the most pertinent opinions, and proposals collected from this study and at a certain point of the research process were validated by all populations of the case study. Figure 2.5 depicts the complexity of the holistic relation of all elements that integrate the infocommunicational heuristic model. As a research process it can be inspiring, a reference, and generalizable for other smart school ecosystems. The specific contributions of this educational community, as pertinent as it can seem, are only effectively applicable to the D. Maria II school cluster at Vila Nova de Famalicão, Portugal. Although some of these services are not innovative, it is not common to see them being promoted in formal learning ecosystems, such as schools. It was interesting to see that the students suggested some characteristics for the common services that can improve considerably the sociability and empathy and contribute to a more humane learning context. The several approaches mentioned by the students and at a certain point validated or highlighted by teachers and parents also augment the opportunity for collaborative and autonomous learning processes.

2.6 Conclusions The infocommunicational model proposed in this study was nurtured by stakeholders from inside and outside the formal educational ecosystem, students, teachers, and also parents. The comparison between the social worlds, school and home, as well as inside and outside the classroom, constituted deeply structured information of the heuristic model.

26

M. J. Fonseca et al.

Fig. 2.5 Symbolic representation of the holistic Infocommunicational heuristic model

This heuristic model of infocommunication services takes on a legitimate character by being built with the experience narratives of the 236 participants. Its versatility may constitute a reference of good practice inside or/and outside the classroom, promoting interaction, collaboration, and reflection, both experiential and critical. The prevalence of new media usage by students is real and should be considered since it can potentiate the development of student autonomy in a functional and correct way. The ease of access to the school library, as a phygital space (physical + digital), effectively allows supporting the needs and interests of stakeholders, with the student as both consumer and producer of information and agent of critical self and peer assessment. The ability to respond in real time to classroom challenges positions the student in a more attentive and active role. The librarian teacher as an agent of change, close and attentive to the wishes and needs of students, teachers, and parents, will leverage new configurations of technological mediation with all the actors of the educational ecosystem, particularly with students. The innovative, highly empirical, and participatory nature of the model allows for the creation of empathic relationships between students, teachers, student–teacher, and vice versa, paving the way for a new strategic look at school. The possibility for teachers to provide constant, more assertive, and faster feedback in formative and

2 Student Smartphone Experience Narratives Mediated …

27

summative assessment strengthens pedagogical ties and improves “relationships of trust”. This study, coordinated by a phygital (physical + digital) school library, with the active participation of the students (identification of their User eXperience/narratives—UX mediated by the smartphone) within a specific context, D. Maria II school cluster in Vila Nova de Famalicão, Portugal, based on infocommunicational constructs revealed clear evidence that such a research process can enhance new learning opportunities for the students and potentiate social, experiential, and reflective skills. Acknowledgements To the educational community of D. Maria II school cluster, Vila Nove de Famalicao, Portugal, namely all the students, teachers, and parents that participated in this case study, the research team is grateful for your critical reflections, suggestions, comments, and precious time you dedicated to this cause. A very special acknowledgment to team members that also integrated this SMARTEES project: Eleonor Silva—Multimedia Communication Masters student; Adriana Machado, Ana Beatriz Bastos, Irla Vaz, and Rejane Fernandes—New Communication Technologies bachelor students at the University of Aveiro, Portugal; and Cândida Pinto, Director of the D. Maria II school cluster at Famalicão, Portugal, for facilitating and being the first supporter and fan of this research process.

References 1. Nakamura, J., Csíkszentmihályi, M.: The concept of flow. In: Snyder, C.R., Lopez, S.J. (eds.) Handbook of Positive Psychology, pp. 89–105. Oxford University Press (2001) 2. AASL Empowering Leadership for School Librarians (2018) 3. Hughes, H., Bland, D., Willis, J., Burns, R.E.: A happy compromise: Collaborative approaches to school library designing. Australian Library Journal, 64(4), 321–334 (2015). https://doi.org/ 10.1080/00049670.2015.1033380 4. IFLA. C. P., S. B. E.: Diretrizes da ifla / unesco para a biblioteca escolar. In C. P. da IFLA & Portuguese (Eds.), Library (Barbara Sc, Issue até 2000, pp. 1–28) (2016). Rede de Bibliotecas Escolares 5. Conde, E.: Bibliotecas Escolares...porque sim! Redes, bibliotecas e literacias -Cidehus (pp. 1– 18) (2016). https://books.openedition.org/cidehus/2559 6. Nofal, E., Reffat, R.M., Moere, A.V.: Phygital heritage: an approach for heritage communication. Online Proceedings from the Third Immersive Learning Research Network Conference (2017). https://doi.org/10.3217/978-3-85125-530-0-36 7. ASLERD: Timisoara Declaration: Better Learning for a Better World Through People Centred Smart Learning Ecosystems (2016) (pp. 1–9). ASLERD. 8. UN Sustainable Development Goals - SDG. United Nations (2015) 9. Chou, P., Chang, C.: BYOD or Not: A Comparison of Two Assessment Strategies for Student Learning (2017) 10. Barlette, Y., Jaouen, A., Baillette, P.: Bring Your Own Device (BYOD) as reversed IT adoption: Insights into managers’ coping strategies (2020). https://doi.org/10.1016/j.ijinfomgt.2020. 102212 11. Alirezabeigi, S., Masschelein, J., Decuypere, M.: The Agencement of Taskification: On New Forms of Reading and Writing in BYOD Schools (2020). https://doi.org/10.1080/00131857. 2020.1716335

28

M. J. Fonseca et al.

12. Kibar, P., Gündüz, A., Akkoyunlu, B.: Implementing Bring Your Own Device (BYOD) model in flipped learning: advantages and challenges. Technol. Knowled. Learn. 465–478 (2019). https://doi.org/10.1007/s10758-019-09427-4 13. Hopkins, P., Hare, J., Donaghey, J., Abbott, W.: Geo, Audio, Video, Photo: How Digital Convergence in Mobile Devices Facilitates Participatory Culture in Libraries (2015). https://doi.org/ 10.1080/00049670.2014.984379 14. Griffey, J.: Mobile Technology and Libraries (Facet) (2010) 15. Machado, A., Bastos, A., Vaz, I., Fernandes, R.: SMARTEEs - Smartphones in Educational Ecosystems (2021). http://hdl.handle.net/10773/32320 16. Silva, E.: Smartphones em Ecossistemas Educativos: co-design de um protótipo conceptual. [Universidade de Aveiro.]. hdl.handle.net/10773/31971 (2021) 17. Scolari, A.: Media ecology, transmedia literacy, and redesign of interfaces. Matrizes, 3(12), 129–139 (2018a) 18. Scolari, A.: Introduction: from media literacy to transmedia literacy. In Teens, media and collaborative cultures. Exploiting teens’ transmedia skills in the classroom (2018b)

Chapter 3

Space and Time in Hybrid Teaching and Learning Environments: Two Cases and Design Principles Teemu Leinonen and Tiina Mäkelä

Abstract The opportunities and challenges of teaching and learning in the same and different space and time have been discussed in the field of distance education for several decades. Within COVID-19, a new type of experimenting and research interest in so-called hybrid learning has emerged. In this article, we present the results from exploring “hybridity” from the perspective of classical categorizations of different forms of learning in terms of time and space. We explored the phenomenon through two cases with the same high-level strategic objective, serving hybrid interaction. In case A, we evaluated university library spaces renovated to serve hybrid teaching and learning. In case B, the focus was on the implementation of a university course redesigned during the pandemic from blended learning to include more hybrid interaction. Multifaceted data was collected, including video recordings (case A), recorded videoconference sessions, and written student feedback (case B). Qualitative data analyses relied on ethnography and contextual inquiry. Based on the analyses of the case studies, we propose five design principles for designing hybrid teaching and learning that aim to overcome the limitations of the same space and time: (1) Ensuring access to required tools, infrastructure, and support; (2) Design primarily for same time, different place learning; (3) Design primarily for same time, same place learning; (4) Less is more; and (5) “Le bon Dieu est dans le détail.” These design principles provide guidance to the design process of hybrid teaching and learning to increase the chances of reaching a successful solution.

T. Leinonen (B) School of Arts, Design and Architecture, Aalto University, Helsinki, Finland e-mail: [email protected] T. Mäkelä University of Jyväskylä, Jyväskylä, Finland e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_3

29

30

T. Leinonen and T. Mäkelä

3.1 Introduction With the COVID-19 pandemic, several concepts have gained popularity, as practitioners have tried to find ways to organize teaching and learning in the middle of the crisis. One of these is “hybrid,” of which the aim is to enable teaching and learning in situations where some of the participants are unable to attend the classroom. Although in recent years, hybrid teaching and learning has been a buzzword, there is a relatively long history, especially in the field of distance learning, of defining possible best practices to overcome space and time restrictions. During COVID-19, various ways of organizing teaching and learning in exceptional situations of lockdowns and restrictions were implemented. Concurrently, some of the practices designed in these times have made many practitioners ask if we should reconsider the entire existing paradigms of teaching and learning in ways similar to those discussed in relation to the future of work. If increasing remote working will be part of the “new normal” of work life, could remote teaching and learning also become part of mainstream practices? In this article, we aim to open up the current research related to hybrid teaching and learning. We also present two real-life cases where the concept has been used. Based on the analyses, we present five design principles for hybrid teaching and learning environments, supporting particularly synchronous teaching and learning that occurs in different places. The design principles are guidelines aimed to help the design process and, thereby, increase the chances of reaching a successful solution. In the following section, we start with a concept analysis of hybrid teaching and learning, and frame the phenomenon of interest. Then we continue with a brief introduction of the two cases with analyses. The results from the analyses are presented and discussed as five design principles for hybrid teaching and learning environments, where an essential part of teaching and learning is synchronous collaboration that takes place partly face-to-face and partly online.

3.2 Hybrid Teaching and Learning in Space and Time To categorize different forms of learning, a matrix (Fig. 3.1) has been developed to present the dimensions of same and different spaces and times, depending on where and when the learning activities take place [1]. The upper-left area represents traditional teaching and learning, where teachers and students get together in the same space and time. The other three areas represent different kinds of distance learning with their own opportunities and challenges. Broadly speaking, the term “hybrid” can entail participation in multiple locations either synchronously or asynchronously. As a concept, it became popular during the COVID-19 pandemic and the growing need for remote meetings, teaching, and learning. Simultaneously, desktop videoconference solutions and the internet infrastructure had become mature enough to experiment with “hybrid meetings” and

3 Space and Time in Hybrid Teaching and Learning Environments …

31

Fig. 3.1 Shared space and time of participants in teaching and learning situations modified from the original by Miller [1]

“hybrid teaching,” whereby some participants are present in the same place while others attend the session via videoconferencing. Before the pandemic, the term “hybrid” has been used in various educational contexts. For instance, authors such as Graham and Allen [2] use blended and hybrid learning environments as synonyms for learning environments that combine face-to-face teaching with technology-mediated remote teaching and studying. The term “blended learning” is commonly used in relation to practices where variations exist between the space and time where learning takes place [3–5]. However, hybrid teaching and learning in today’s mainstream means that some participants may share the same time and place, while others participate at the same time but in different places. It may also consist of recordings for their posterior use. In practice, this means that participants have more freedom to choose their preferred study place and time. When we consider a hybrid teaching and learning environment, we are precisely interested in the educational arrangements as a whole, including variations in time and place, the physical and virtual spaces and tools (i.e., technology-enhanced physical environments), services, practices, and participants’ activities and interactions (i.e., psychosocial environment) [6]. In the recent research literature, there are several more concepts related to hybrid teaching and learning. According to Irvine, Code, and Richards [7], in “multi-access learning,” students are given opportunities to choose between synchronous online or onsite or asynchronous online modes of participation. As described by Elder [8], the “multi-options” teaching methodology allows students to choose weekly either

32

T. Leinonen and T. Mäkelä

onsite, synchronous online, or asynchronous online attendance at their convenience. He et al. [9] use the term “flexible hybrid instruction” as a model in which all course materials, including lecture videos, are offered online one week before class, and students are also given the freedom to choose whether to attend onsite classes or not. Bower et al. [10] describe “blended synchronous learning environments” as a model whereby some students participate in face-to-face classes through synchronous technologies such as videoconferencing and virtual worlds. A similar design is also applied in Bell et al. [11] experiments, which they call “synchromodal classes”. Beatty [12] proposes a term “hybrid-flexible” (HyFlex) for learning environments in which students can flexibly choose to participate (1) in face-to-face synchronous sessions in-person (in a classroom), (2) in face-to-face synchronous sessions via videoconference, or (3) fully asynchronously via videos and other available online materials. Eyal and Gil [13], in turn, emphasize the fluidity of “hybrid,” in the sense that it crosses boundaries between virtual and physical, formal, and informal. Also, Goodyear [14] highlights the importance of the students’ role as active coconfigurators of hybrid learning and spaces with “rich mixtures of material and digital tools and resources.” A recent systematic literature review on synchronous hybrid learning [15] concluded that most of the existing research is still exploratory and mostly describes students’ experiences on a course in which both onsite and remote students participated in learning activities simultaneously. Earlier research also reports organizational and technological implementations. Thus far, research evidence on the advantages or disadvantages of hybrid teaching and learning is scant. Having online materials always available for students has been found to increase the time spent on learning and improve learning outcomes in blended learning [16]. For synchronous blended/hybrid learning, in particular, some studies highlight the greater educational access, increased student autonomy, and flexible choices that this mode offers [7, 10, 17]. Additionally, an enhanced sense of community and interchanges of experiences between campus students and, often, more professionally experienced online students have been reported [7, 10, 18]. However, a common challenge faced by teachers in this mode is dividing their attention between onsite and online participants [10, 18, 19]. Teachers also need to adapt their teaching methods and learning activities to the synchronous mode [11, 15]. We see that a need still exists for more explorative research to gain experiences from different implementations of hybrid teaching and learning. This means experimenting with existing courses by expanding them in the direction of hybrid teaching and learning, and considering what should be taken from these to establish permanent practices. We also see that hybrid teaching and learning needs a physical space. This means that schools, classrooms, and libraries need to be redesigned to facilitate hybrid interaction. In the following, we present two cases and analyses of them.

3 Space and Time in Hybrid Teaching and Learning Environments …

33

3.3 Case Studies In these cases, the attempt was to design a hybrid teaching and learning environment. The cases are very different, but they share the same high-level strategic objective: being and serving hybrid interaction. Two independent case studies were conducted in two Finnish universities. The first case focuses on evaluating the potential of a newly renovated university library for hybrid teaching and learning. The focus is on the teachers’ perspectives. The second case presents a university BA-level course that has been taught for a couple of years using a blended-learning approach, and in autumn 2021, it was quickly modified to include more online activities and, therefore, became hybrid interaction. In this case study, the focus is more on the students’ perspectives.

3.3.1 Case Study with an Action Research Orientation With the case studies, we aim to explore the phenomenon of hybrid teaching and learning, rather than answering any predefined research questions. Therefore, this research consists of two exploratory case studies to develop pertinent hypotheses [20] that are presented in the form of design principles. We have collected multifaceted data from the cases, including video recordings from test sessions (case A) and recorded videoconference sessions and written feedback from students (case B). In the qualitative data analysis, we have relied on ethnography and contextual inquiry as described in the context of design research [21, 22]. Its aim is to specify and develop a domain, which, in our case, is hybrid teaching and learning environments. Both authors participated in the data analysis. Instead of transcribing entire video recordings, the researcher in charge of the particular case reviewed collected recordings by writing down notes that responded to our research aim [see 23]. The same procedure was followed with the written student feedback. In the data analysis, we followed an interpretative approach, focusing on significant (i.e., meaningful or relevant) themes and on identifying “clusters of meaning” about problem areas. Trustworthiness and credibility of the analysis was strengthened by cross-checking the initial themes identified by the researchers in each case. The design principles were finally derived from the themes that were identified in both cases. This was also thought to augment their transferability to different contexts. The authors have been an essential part of the research, participating in the activities, facilitating the participants, observing, and taking notes during the cases, with the intention of recognizing challenges and looking for opportunities to solve them. Therefore, in this research, there is also an action research orientation, as applied in educational research. In educational action research, the aim is to conduct research that will result in improvements in educational practice [24].

34

T. Leinonen and T. Mäkelä

3.3.2 Case A: From a Library to a Hybrid Teaching and Learning Space In autumn 2021, a newly renovated library was opened at the University of Jyväskylä in Finland. The renovated library offers a spacious and multipurpose, interdisciplinary meeting point for all citizens. The aim is to be “hybrid,” connecting formal, non-formal, and informal learning: an open entry point to research knowledge, and a meeting place of knowledge, learning, and wellbeing. The building is described as a science living room that draws people to scientific knowledge. The library entails an open science center (open science, library, and museum services), digital services, and student life center, promoting student wellbeing. There is also a café and a restaurant in the building. The library includes a number of teaching facilities equipped with the latest technology as well as two highly equipped studios for multimedia productions. There are various reading halls, group working spaces, and spaces for quiet work. There are also spaces for science and art education and exhibitions. The library is promoted as a venue for various events that can be easily streamed online. Public events of the University, such as doctoral dissertations, are streamed on a wall-sized screen in the café for a wider audience. In January 2022, the adequacy of the library’s multipurpose learning spaces and technologies for hybrid teaching and learning—recorded synchronous onsite and online sessions—were evaluated. The evaluation was conducted as part of the university’s education development program (“JYULearn”), with the aim of involving the university community in the development of pedagogically and digitally relevant teaching practices, new kinds of learning opportunities, as well as structures to support multi-sited hybrid interaction. A researcher (author 2) and a coordinator invited a representative of the university’s digital services to introduce the available technologies in the learning space. Technologies were then tested with four university teachers from the faculties of information technology, education, and humanities and social sciences. Two participated in testing onsite and two online via videoconferencing. The testing took three hours, and the sessions were video recorded for later analysis. The multipurpose learning space where most of the testing took place (Fig. 3.2) entails reconfigurable furniture: tables and chairs that allow for individual and group configurations, sofa groups, and individual chairs with writing pads. For presentations, chairs and tables can be arranged facing the wall where the presentation, which can also be shared with online participants, is displayed by means of a data projector. There is also a video camera on the ceiling that can be directed towards the presenter/s using a remote control. There are two wearable microphones for presenters and two handheld microphones that can be used by the onsite participants. The aim of the multiple microphones is to enable online participants to clearly hear the presentation. Audio speakers in the room make it possible for everyone in the space to hear the comments from the remote participants. Online participants’ presentations can also be displayed on the screen. The space also includes a movable interactive whiteboard

3 Space and Time in Hybrid Teaching and Learning Environments …

35

Fig. 3.2 Videoconference view (screenshot) of the testing of different furniture and technology configurations at the university library’s multipurpose space

computer with a video camera, which can be used to display online participants and provide them with another view of the space. Based on the evaluation, the library’s multipurpose learning spaces and their technological equipment enabled both frontal teaching and working in differentsized groups synchronously with onsite and online participants. It is also possible to record the sessions for asynchronous use. Further, the live video stream makes it possible to have an audience, for instance, in the library café with the wall-sized screen. This was seen to enable informal learning and opening-up sessions for a wider public. For synchronous hybrid sessions entailing group discussions between onsite and online participants, an interactive whiteboard computer with a video camera can be connected to three ceiling microphones. This way, online participants can hear everyone speaking in the room. It was noticed, however, that there was some delay and difficulty following the conversation online, if onsite participants moved between microphones or if various people talked in the room at the same time. It was also necessary to turn off the audio from devices that were not in use during the discussion. For small group discussions mixing onsite and online participants, using individual laptops was found to be more suitable. With some distance between laptops, interference between them was avoided. Small groups could also spread outside the room to other spaces of the library. The group testing the space discussed that it could be beneficial to have onsite participants log into videoconferencing systems with cameras from their mobile devices or laptops. This would allow online participants to see onsite participants individually. This would also enable participants to chat and get access to shared links. In addition to video cameras, it was noticed that connecting mobile devices and their

36

T. Leinonen and T. Mäkelä

cameras to videoconferencing systems would enable moving with the camera around the space. In the case of being connected to a videoconference with various devices and cameras, it was seen as necessary to direct participants’ focus, either with the videoconference software or by asking the participants to “pin” one of the cameras to their view. Based on the three-hour testing, it was concluded that library spaces and multimedia equipment offer possibilities for a wide range of synchronous and asynchronous hybrid teaching from more traditional lecturing to collaborative multidisciplinary and multi-located sessions. Teachers with no previous experience with hybrid environments would, however, need detailed instructions, and plenty of support and training in order to familiarize themselves with all options and to adapt their teaching accordingly. Students also need clear instructions on how to interact efficiently in hybrid environments. It was noted that participants need to adapt to different interaction patterns when involving onsite and online participants. For instance, while it was easy for onsite participants to focus on one of the simultaneous conversations in the room, for online participants, it sounded chaotic. Knowing that sessions are recorded or followed elsewhere was also considered a possible limitation for spontaneous interactions between participants.

3.3.3 Case B: Expanding Blended-Learning to Synchronous Hybrid Teaching and Learning In autumn term 2021, the Process Management for Media and Design studio course of the Aalto University School of Arts, Design and Architecture in Finland was organized partly on campus and partly online. The 6 ECT credits course of seven weeks was taken by twenty-four 18–25-year-old students from the BA in Design program. The course was taught by one professor of new media design (author 1), one lecturer in industrial design, and an English language teacher. The language teacher was running a parallel language course, so some of the assignments of the courses were shared. Students received credits and grades from both courses. The course introduces students to the fundamentals of the innovation process for both products and services. The focus is on the role and value of design within such processes. Students learn to work in a team that is implementing a design process with a customer. Students are also introduced to the basics of social psychology and organizational dynamics, and their practical application in design. The in-class lectures and discussion cover both physical and digital development processes and methods as well as the tools that can be used in them. Through assignments and inclass activities, students learn how to participate in and facilitate creative processes. In the course, students are divided into six teams, each with four students. The teams are assigned a customer—a non-governmental organization—with whom they practice design consultation and, finally, propose a solution to their customer’s design brief or any other challenge they have recognized during the process. Within the

3 Space and Time in Hybrid Teaching and Learning Environments …

37

Fig. 3.3 Student team presented their design proposals for the Finnish Red Cross and the rest of the class on a videoconference

course, students study the customers’ operations and facilitate design workshops with them. The course ends with presentations to the customers (Fig. 3.3). The other assignments of the course include note-taking drawing from the lectures and classes, preparing a team presentation (brochure and oral presentation), preparing a facilitation guidebook, and finally, writing a personal study report in the format of a reflective learning journey. At the end of the course, students should be able to recognize the difference between process-driven and philosophy-driven development processes for products and services. They should be able to plan and participate in an innovation process and account for group dynamics. They should be able to plan and visually facilitate a creative session for a group of people and utilize facilitation tools and methods in a creative development process. The course was planned to include 60 h of classroom teaching and learning. In addition to this, students were expected to spend 60 h on preparing for the classes and doing their teamwork independently. The students were also given 40 h to complete their independent writing assignments. The classes included lectures, covering topics of organizational behavior, management, teamwork, design facilitation, agile software development, and offline and online workshop methods and tools. The course was designed to be a blended learning, so that all the learning materials, including slides from the lectures, are available in the online learning management system, and students are asked to return their assignments to the same platform. Still, it was decided that the most crucial mode of working was the classes in the same time and place with lectures and classroom discussions on homework assignments. Due to the pandemic, some of the classes were organized so that students who could not attend the class were offered the option of following the lectures and discussions via videoconference. The rule, however, was that students must have a pandemic-related reason, such as quarantine, symptoms, or travel restrictions, for not attending the face-to-face class.

38

T. Leinonen and T. Mäkelä

Students were encouraged to organize their first team meetings face-to-face before meeting via videoconferencing. Due to the pandemic, most of the meetings with the customers were conducted via videoconference, although some teams also participated in the organizations’ meetings and voluntary work. To analyze the implications of rapidly moving from blended learning to more hybrid teaching and learning, multifaceted data were collected from the course. The data consisted of recorded videoconference sessions and written feedback from the students. From most parts, the change to hybrid interaction was deemed successful. Students were already familiar with the main technical tools—videoconference and industry standard online whiteboard software—used in the course. More challenging than the use of the tools was reaching agreement on the norms and practices. For instance, there were some discussions on what are considered acceptable reasons for students to attend the onsite classroom sessions remotely and how do teams reach an agreement on their collaboration and meetings. Related to remote participation, it was discussed how sick students may attend the class remotely or if they should focus on resting and getting better. For the teamwork, students were asked to prepare a team contract, defining their team members’ roles and responsibilities, meeting practices, online tools that are used, and ways to solve possible conflicts in the team. The team contract was found very useful. The students found the online sessions with customers beneficial because they felt that their future work with customers may also be partially conducted online. However, those students who were able to attend face-to-face events and conduct interviews with the customers also performed somewhat better in the course, which was also reflected in their grades. In this case, those students who, for one reason or another, preferred to work more online most likely did not reach the same level of competency as those who were actively mixing online and face-to-face collaboration.

3.4 Results: Design Principles for Hybrid Teaching and Learning Environments Building on earlier research and two case studies, in the following, we present a set of design principles to consider when designing hybrid environments. By design principles, we mean rules of thumb that are delivered inductively from our experience and provide guidance to the design process to increase the chances of reaching a successful solution [25]. First, we define the hybrid teaching and learning environment as the physical and virtual spaces, tools, services, and practices where the participants’ activities and interactions can take place both in the same time and place and in a different time and place (Fig. 3.4). Designing hybrid environments refers to the design of both psychosocial and technology-enhanced physical and virtual environments. The design of the learning environment is such that the different activities in different

3 Space and Time in Hybrid Teaching and Learning Environments …

39

Fig. 3.4 Hybrid teaching and learning where the same time but in different place form is becoming mainstream

spaces and times are either programmed by the teacher, or the students are given a choice of where and when to do them, depending on the situation. There seems to be a trend [e.g., 13, 15, 19] that more teaching and learning is going in the direction of extending the freedom of the physical place where learning takes place. In the following, we present five design principles (DPs) to consider when designing hybrid teaching and learning that aim to overcome the limitations of the same time and space. These DPs are derived from the analysis of two cases aimed at serving hybrid interaction.

3.4.1 DP1: Ensuring Access to Required Tools, Infrastructure, and Support To enable hybrid teaching and learning, several technological infrastructure requirements and technological affordances are necessary. In practice, it means that all participants have a broadband internet connection, laptops with front cameras/microphones, and access to online services or software enabling asynchronous and synchronous communication and collaboration. In case B, all students had relatively new laptops and a broadband connection at home and on campus. It was, however, found important to confirm this before the class and be ready to adapt

40

T. Leinonen and T. Mäkelä

the course arrangements accordingly, if tools or infrastructure were found lacking among the students. In the case of educational institutions, they have several opportunities to overcome limitations. The institution can provide all students with the necessary tools or provide them as a library service. As found in case A, in synchronous hybrid interaction, it is vital that physical classrooms incorporate a high-quality infrastructure (screens, microphones, stereos/loudspeakers, cameras), enabling bi-directional interactions between onsite and online participants, and supporting teachers’ tasks of encouraging student participation and distributing their attention between onsite and online students. Instructions as well as time for testing and becoming familiar with the tools are needed. In case B, part of the classroom time was used to confirm that all students knew how to use the collaborative online tools. Particularly, hybrid synchronous environments with various cameras, sound systems, and devices require careful planning in advance as well as a detailed script, which is introduced to all participants in the beginning of the course. In case A, it was also noticed that one should be highly organized in advance; for instance, all technology should be set up and tested [see also 10]. One should also determine how to use audio, visual, and text modalities or how to group onsite and online participants. In case A, the participants pointed out that there should also be some time to test the technology before each session.

3.4.2 DP2: Design Primarily for Same Time, Different Place Learning (with Recordings) The current trend seems to be moving more in the direction of the same time but in different place modes of teaching and learning. However, the possibility to provide participants with the freedom to choose a different time for learning is also there, as the synchronous sessions can be recorded for later viewing. This was considered an advantage in case A, where the technology in the library enabled various modes of teaching and learning. There are many advantages to providing opportunities for learning at the same time but in a different place. Firstly, students are free to choose where they attend classes. In case A, the library could work as a meeting point for students who may want to socialize with others, regardless of whether the main focus was on online sessions with those students who prefer to attend the course remotely. Secondly, the latest software for videoconferencing and online collaboration is now widely used in work life. By using these during studies, students will learn important work life skills. In case B, students reported this as one important competency developed during the course. Thirdly, the schedule of the sessions will give students a rhythm to their studies. They can work as checkpoints for teachers to discuss with students their progress with their studies, to give assignments, and to discuss the assignments in a flipped classroom way [26, 27].

3 Space and Time in Hybrid Teaching and Learning Environments …

41

Fourthly, as noticed in case A, by recording the synchronous sessions, those students who could not attend the session could watch it afterwards and get some idea about what was happening in the class session. Based on our experiences in both cases, students who participate in synchronous sessions also use video recordings, for instance, to review parts that they found challenging to understand during the session. It is also possible to add subtitles and automatic translations, increase or decrease the speed, and pause the video. This provides possibilities to adapt resources based on one’s personal needs. There are, however, some disadvantages related to the recordings. Especially, if the synchronous online classes discuss sensitive topics, participants may not be willing to share their thoughts when the sessions are recorded. This happened in case B, where the teacher did not feel comfortable discussing ethical issues or customers with the group when the session was being recorded. The recording may also cause challenges related to privacy and security.

3.4.3 DP3: Design Primarily for Same Time, Same Place Learning (with Recordings) When considering hybrid teaching and learning, it might sound contradictory to organize synchronous online sessions when the participants are in the same physical space. An online session in the same place, however, makes it possible for some participants to attend the session from a distance. This can be done in spaces specifically designed for this, such as in case A. All students may attend the online class from their own laptop at the same time and place on campus, such as small meeting rooms or the campus library/learning center. While the main focus would be on onsite studying, this design would also enable students to participate remotely from their own choice of place. In case A, it was also found that having laptops or mobile phones with cameras helped the remote participants, as they could see close-up faces of all the participants and not just an overview of the classroom. Using laptops in the same space can be a solution that serves both pedagogy and various students’ needs. This model was experimented within case B. It was found especially feasible when working in teams on campus, although all students did not use their own laptops all the time; they also shared one laptop and met with remote participants via videoconference. Additionally, when students presented their progress in the classroom, some teams also had their remote participants do their part of the presentation over videoconference. In case A, it was also found that enabling all students to chat simultaneously and share links could enhance their interactions. There are several advantages to this type of hybrid working. When invited to the campus, social events can be arranged between classes, such as having a cup of coffee or lunch with the teachers and the other students. In both cases, having informal discussions on the course content during breaks was considered important for various reasons. During informal discussions, students may clarify many topics discussed during the class. Non-formal get-together events build social cohesion and

42

T. Leinonen and T. Mäkelä

professional identity and, in many cases, have a positive impact on general wellbeing. In case B, after a long period of remote studies, most of the students highly appreciated the possibility of being in the same classroom. Although it is possible to organize virtual coffee breaks, too, one should not underestimate the power of getting together face-to-face. As found in case B, another advantage of organizing synchronous online sessions in the same place is that those students who cannot come to campus are still able to more or less equally participate in the class. For instance, one student in case B was in Australia and could not travel to the University because of the travel restrictions. Without the hybrid learning arrangement, he would have missed the first half of the course and would not have been able to complete the course. From the analysis of case B, it can be concluded that if everyone is on their laptop, the situation does not differ too much between the participants. Everyone could have the same tools to participate in class activities, and the teacher is obligated to design the activities so that everyone can participate in them equally. This was also noticed in case A. A disadvantage of this arrangement is that remote participants are left out of informal discussions. This requires the teacher to brief the remote participants on all essential discussions related to the course that occurred during the breaks. In case B, the teacher found this very difficult. Knowing who has been in which sessions and who needs what kind of information and support became an overwhelming challenge of information management. Likewise, in case A, it was noticed that providing possibilities for synchronous or asynchronous remote participation limits the flexibility of movement and speech onsite when aiming to transmit everyone’s image and voice without interference. In both cases, it was also found that choices in the use of cameras and microphones also weakens both onsite and online participants’ control over the interactional space [see also 19].

3.4.4 DP4: Less is More The complexity of the hybrid environment grows when the number of ways to participate in classes and modes of working increases. When designing the hybrid teaching and learning environment, it is important to think about students’ abilities to focus on their studies, rather than overloading them with needing to make decisions on how they can do it. Simple rules, such as in case B where they discussed acceptable reasons for not attending the face-to-face classes, will help students and the teacher. Also in case B, one team assignment was to prepare a team contract, defining how they will organize their hybrid teamwork, who is responsible for what, and how potential conflicts will be resolved. This was found very useful for both the students and the teacher. In case A, it was noticed that the library and its technology provided so many possibilities to design and implement teaching and learning that choosing the right tools and methods that support the pedagogy became difficult. Therefore, it would be important to have simple default examples and settings for beginners. Teachers

3 Space and Time in Hybrid Teaching and Learning Environments …

43

should not spend their time configuring the technology for their use, but rather apply it to their pedagogy. However, access to technological tools, such as in case A, may open up teachers to reconsider their pedagogy and build new best practices. In case B, the assignment for all lectures, online and in class, was to take notes with a pen and paper, photograph them, and then share them with everyone in the class’s online environment, which both teachers and students found beneficial. It was found to help students concentrate on the class, which can be difficult when there are many other applications and services available in the same classroom. For the teachers, the students’ notes were useful for following up students’ progress and final evaluations.

3.4.5 DP5: “Le Bon Dieu est dans Le Détail” What then works in teaching and learning onsite in class and online is often difficult to grasp in all its detail and, more so, to explain profoundly. A various number of factors affect the experience, and many of them are out of the teachers’ control. Therefore, not everything can be designed. There are, however, many details to which we may pay attention. As already mentioned, it is important that all technology to be used works properly, and if there are challenges, students and teachers must get prompt support. In case B, there was a moment when the classroom video projector did not work, and the teachers had to call support for help. In this situation, the best possible solution was to go to the nearby cafeteria to discuss the theme of the day in small teams and return to the classroom when support had solved the issue. Similarly, in case A, the library offers several spaces for meaningful collaboration, if the technology in one of the classrooms does not work as expected. In these situations, personal laptops can be used to connect with the remote participants. In case B, because everyone agreed to keep their video on during the online classes, it made it easier for the participants to have feelings of belonging and community. It was, however, agreed that it was not obligatory, if someone felt uncomfortable or had concerns related to their privacy. Similarly, it was agreed that the sessions would not be recorded. This made it possible to have discussions also on sensitive topics. This was also discussed in case A. In case B, there were also some practicalities that were expected to have an impact on the students’ well-being. For instance, it was agreed that the videoconference room would always open 15 min before the actual class session started and reopen after the session for another 15 min. The 15-min periods were used to play music for all and to have informal discussions. These were also opportunities to discuss topics that one did not want to be recorded and posteriorly shared. At these times, students could ask questions in a small group or one to one with the teacher. Both the students and the teachers found these sessions important. In the case of recorded sessions, students could use the informal sessions before or after class and breaks to discuss sensitive topics that would be left out of the recordings.

44

T. Leinonen and T. Mäkelä

3.5 Discussion and Conclusion The trend in designing teaching and learning opportunities is to provide more freedom of the physical space where learning takes place. We, however, see that teaching and learning at the same time and in the same physical space will continue playing an important role in the future. However, more opportunities should be provided, at times, to organize teaching completely online and to provide possibilities for synchronous and asynchronous online participation for students with limited access to synchronous onsite sessions. For this purpose, we have presented five DPs derived from two case studies to be considered when designing hybrid teaching and learning environments. The first design principle (DP1) presented in this paper reminds us that ensuring access to and support for tools and infrastructure is a precondition for designing teaching and learning environments with more flexibility in terms of time and space. As reminded by Bower and colleagues [10], it is important to match technologies to lesson requirements. Additionally, mobile devices can be used to participate in a session, and mobiles with cameras may also facilitate the visual input [10]. Further, our results suggest that it may be recommendable to prioritize either designing synchronous online (DP2) or synchronous onsite (DP3) interactions. In the design of the actual course and class, one should consider, however, that one model does not fit all students. Attention should be given to designing ways to enhance educational access, flexibility, and student autonomy [7, 10, 17]. Therefore, it is important to consider how to design activities that take place at the same time but in different locations but also activities that take place at the same time and the same place. That is, opportunities for face-to-face interactions may still be provided when using mainly online environments. Likewise, it is possible to provide possibilities for remote participation for some individuals when working mainly onsite. With all these alternatives, we can take advantage of digital tools that can work as platforms for interactions, and we may provide recordings of it for later viewing, thereby providing access to resources when synchronous participation is not possible. Recordings may also be used when creating materials for flipped learning. Finally, as always in design, less is more (DP4) and God is in the details (DP5). These principles, often discussed in design circles, are unfortunately not well understood or applied when designing teaching and learning environments. Design is always a matter of making choices. Often, fewer choices are better than allowing everything. There is also a need to consider, for instance, ways to ease teachers’ cognitive loads in adapting new teaching methods and activities [10, 15] and dividing their attention between online and onsite participants [10, 18, 19]. We see that the possible “new normal” of more remote teaching and learning may challenge the existing forms of institutional learning. There is the possibility of a new kind of interplay between formal and informal learning with new kinds of educational models and practices. For example, videoconferencing enables having external experts in teaching and learning from all around the world [15, 18]. Working in

3 Space and Time in Hybrid Teaching and Learning Environments …

45

hybrid teaching and learning environments may provide authentic learning opportunities connected to real-world challenges and enable practicing various skills needed in our society, such as flexibility, patience, empathy, creativity, and problem solving. As described by Pichetola [28], hybridity is very much about blurring the boundaries between physical and virtual, private and public, personal and professional in a context of uncertainty, diversity, and freedom. This is something that can be seen as a characteristic of our time and needs to also be considered in education. At the same time, we understand that introducing new models and practices is challenging, and the consequences are not necessarily only positive. Therefore, there is a need to codesign the future of education where the same space and time do not play such an essential role, while also considering both the teachers’ and students’ wellbeing and the quality of education. We also see that this may open up new opportunities for regional development. This research has several limitations. The DPs presented in this study are based on only two case studies conducted in one country. They are, however, in line with the contemporary literature, indicating that they may be generalizable in other contexts. We are currently extending this preliminary and exploratory study to a participatory design of hybrid learning environments with a greater number of participants and stakeholder groups. This will also lead to further development of these preliminary DPs. The principles presented in this paper could, for example, be divided into more specific sub-principles, whereby the course designer could choose those that fit the context she is working for and in best. Also, more concrete examples could be offered for their actual application, followed by an evaluation of their usefulness. We also look forward to mixing qualitative data gathering methods with quantitative methods, for instance, when evaluating the effectiveness of hybrid learning environments for teaching and learning. There is also a need to deepen our understanding of how teachers can orchestrate hybrid sessions without the over-extensive cognitive load identified in previous studies [10, 15, 18, 19]. Acknowledgements Special thanks to the teachers, the project coordinator, and the representative of digital services for participating in case study A, organized as a part of the JYULearn development program at the University of Jyväskylä. Special thanks to the students of the Process Management for Media and Design studio course at the Aalto University School of Arts, Design and Architecture in Finland, who participated in case study B. We are also thankful to the digital learning environments specialist Jaana Brinck for her review and comments on the manuscript.

References 1. Miller III, T. K.: Delivering Engineering Education via Distance Learning. https://www.nsf. gov/pubs/1998/nsf9892/deliver.htm (1998). Last accessed 01 Mar 2022 2. Graham, C. R., Allen, S.: Blended learning environments. In: Encyclopedia of Distance Learning, pp. 172–179. IGI Global (2005) 3. Power, M.: The emergence of a blended online learning environment. J. Online Learn. Teach. 4(4), 503–514 (2008)

46

T. Leinonen and T. Mäkelä

4. Singh, H., Reed, C. A.: White paper: achieving success with blended learning. Centra Software, 1–11 (2001) 5. Garrison, D.R., Kanuka, H.: Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education 7(2), 95–105 (2004) 6. Mäkelä, T., Leinonen, T.: Design framework and principles for learning environment co-design: Synthesis from literature and three empirical studies. Buildings 11(12), 1–22 (2021) 7. Irvine, V., Code, J., Richards, L.: Realigning higher education for the 21st century learner through multi-access learning. J. Online Learn. Teach. 9(2), 172 (2013) 8. Elder, S.J.: Multi-options: an innovative course delivery methodology. Nurs. Educ. Perspect. 39(2), 110–112 (2018) 9. He, W., Gajski, D., Farkas, G., Warschauer, M.: Implementing flexible hybrid instruction in an electrical engineering course: The best of three worlds? Comput. Educ. 81, 59–68 (2015) 10. Bower, M., Dalgarno, B., Kennedy, G.E., Lee, M.J., Kenney, J.: Design and implementation factors in blended synchronous learning environments: Outcomes from a cross-case analysis. Comput. Educ. 86, 1–17 (2015) 11. Bell, J., Sawaya, S., Cain, W.: Synchromodal classes: designing for shared learning experiences between face-to-face and online students. Int. J. Designs Learn. 5(1) (2014) 12. Beatty, B. J.: Beginnings: where does hybrid-flexible come from? In: Beatty, B. J. HybridFlexible Course Design: Implementing Student-Directed Hybrid Classes. EdTech Books (2019). https://edtechbooks.org/hyflex/book_intro. Last accessed 01 Mar 2022 13. Eyal, L., Gil, E.: Hybrid learning spaces—a three-fold evolving perspective. In: Gil, E., Mor, Y., Dimitriadis, Y., Köppe, C. (eds.) Hybrid Learning Spaces. Springer (2022) 14. Goodyear, P.: Design and co-configuration for hybrid learning: theorising the practices of learning space design. Br. J. Edu. Technol. 51(4), 1045–1060 (2020) 15. Raes, A., Detienne, L., Windey, I., Depaepe, F.: A systematic literature review on synchronous hybrid learning: gaps identified. Learn. Environ. Res. 23(3), 269–290 (2020) 16. Means, B., Toyama, Y., Murphy, R., Bakia, M., Jones, K.: Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies (2009) 17. Bülow, M. W.: Designing synchronous hybrid learning spaces: challenges and opportunities. In: Gil, E., Mor, Y., Dimitriadis, Y., Köppe, C. (eds.) Hybrid Learning Spaces. Springer (2021) 18. Szeto, E., Cheng, A.Y.: Towards a framework of interactions in a blended synchronous learning environment: What effects are there on students’ social presence experience? Interact. Learn. Environ. 24(3), 487–503 (2016) 19. Leijon, M., Lundgren, B.: Connecting physical and virtual spaces in a HyFlex pedagogic model with a focus on teacher interaction. J. Learn. Spaces 8(1), 1–9 (2019) 20. Yin, R. K.: Case Study Research: Design and Methods (Vol. 5). Sage (2009) 21. Müller, F.: Design Ethnography: Epistemology and Methodology. Springer Nature (2021) 22. Leinonen, T.: Designing learning tools. Methodological insights. Aalto University (2010). http://urn.fi/URN:ISBN:978-952-60-0032-9 23. McLellan, E., MacQueen, K.M., Neidig, J.L.: Beyond the qualitative interview: Data preparation and transcription. Field Methods 15(1), 63–84 (2003) 24. Sáez Bondía, M. J., Cortés Gracia, A. L.: Action research in education: a set of case studies? Educat. Action Res. 1–16 (2021) 25. Fu, K. K., Yang, M. C., Wood, K. L.: Design principles: the foundation of design. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (Vol. 57175, p. V007T06A034). American Society of Mechanical Engineers (2015) 26. Abeysekera, L., Dawson, P.: Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research. High. Educ. Res. Dev. 34(1), 1–14 (2015) 27. Fung, C.H.: How does flipping classroom foster the STEM education: a case study of the FPD model. Technol. Knowl. Learn. 25(3), 479–507 (2020) 28. Pischetola, M.: Teaching novice teachers to enhance learning in the hybrid university. Postdigital Sci. Educat. 4(1), 70–92 (2022). https://doi.org/10.1007/s42438-021-00257-1

Chapter 4

The Italian School Ecosystems Two Years After the Lockdown: An Overview on the “Digital Shock” Triggered by the Pandemic in the Perceptions of Schools’ Principals and Teachers Carlo Giovannella, Licia Cianfriglia, and Antonello Giannelli Abstract Two years after the shock undergone by the Italian school ecosystem— due to the lock-down imposed by the COVID-19 at beginning of March 2020—the effects generated by the resulting fully digital immersion have been investigated by a survey administered to a representative sample of school teachers and principals. The analysis of the outcomes, together with their comparison with similar investigations performed in May 2020 and March 2021, shows that the activities carried out in the last two years—from emergency teaching to integrated teaching and, finally, to teaching in a “new normal” condition—have triggered an apparently fast innovation process with beneficial effects on the e-maturity of the system, on the educational processes and on the state of the individual well-being. Although some symptoms of normalization start to be glimpsed, and despite the workload increase induced by the adoption of technologies, the system seems ready to carry out an optimization of the educational processes to include a stable use of didactic activities augmented by technologies and, more in general, of forms of integrated on-line learning. As a corollary, emerges the relevance attributed, in a plebiscite manner, to continuous training (LLL) on learning technologies and digital pedagogy, as well as the need to implement permanent forms of smart working to thin the organizational processes.

4.1 Introduction As well known, in the spring of 2020 all learning ecosystems around the world— including the schools on which we focus in this paper—underwent a severe lockdown C. Giovannella (B) University of Rome Tor Vergata – Dip. SPSF, Rome, Italy e-mail: [email protected] ASLERD, Rome, Italy L. Cianfriglia · A. Giannelli National Association of School Principals (ANP), Rome, Italy © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_4

47

48

C. Giovannella et al.

as a non-pharmacological intervention to support the social distancing needed to contain the spread of the COVID-19 pandemic [1]. The abrupt transition from face-toface (f2f) classes to distance and online learning subjected the learning ecosystems to considerable stress that has been largely documented by grey literature, storytelling, reports [2], and scientific investigations [3] that showed both the lack of plans to cope with the emergency but, as well, the ability of the systems to react and respond to the emergency in a period of time varying between few days and 2–3 weeks depending on the level of IT infrastructures and of the digital skills available to the specific context. Many contributions have highlighted the great self-denial and spirit of sacrifice of the teaching staff, but also a few problems such as a widespread deficit of specific preparation (digital and pedagogical) and the existence of geographical areas where segments of the population—from a few percent to over 50% in the most disadvantaged regions of the world—were still subjected to a veritable digital divide. In any case, the abrupt transition to distance and online learning can be considered a “digital shock” and represents a unique opportunity to study the characteristics and the effects of a very peculiar innovation process. In fact, as described in [4], the schools were faced with a forced process during which the awareness phase overlapped with the acceptance one, no one had a clear idea of which factors could influence the process and, finally, the surrounding conditions changed continuously and substantially over the past two years (from distance teaching to forms of integrated teaching—presence-distance and parallel blended learning—till a “new normality” characterized by large flexibility in the procedural customization of the teaching process, functional to the contingent situations determined by the various pandemic waves). During the last two years we have followed the evolution of the process both in the school [4, 5] and in the university [6–9] at our best. In particular as far as the school, with the help of the teachers, who kindly took part in two surveys, we were able to take a snapshot of the steady state of the emergency (two months after the lockdown [5]) and one year later [4]. The intention was to analyze the progress of the ongoing innovation process, in particular: to investigate its nature, identify the factors that can determine its development, stagnation, or retreat, and, finally, to make emerge a model capable of representing it: Model for Attitude to get Engaged in Technological Innovation (MAETI) [5, 9]. The outcomes of these investigations provide a comparative baseline of fundamental importance for the investigation presented in this paper, carried out two years after the lockdown. At present, the system has reached a new steady state characterized by learning processes able to adapt in an extremely flexible way to the contingent evolution of the pandemic that may require, for example, a limited number of students to follow the lessons from home for a certain number of days. The “new normality” is, therefore, a phase characterized by a variable and personalized geometry in the delivery of the educational processes that does not imply, however, also personalization of contents and methods. Due to the achievement of such a new steady state (that we can define a dynamical one), it is of great interest to take stock of the situation, verify the evolution of the perceptions, and identify the transformations induced by the “digital shock” both at the individual

4 The Italian School Ecosystems Two Years After the Lockdown …

49

and systemic level, understand if there were any chance that they could stabilize over time and, finally, bring out useful elements for the development of future policies. The present survey, thus, presents both elements of continuity with respect to the ones carried out in the past—to monitor the evolution of the teachers’ perception with respect to the operational context—and also differences intended at: (a) collecting also the point of view of the schools’ principals (to be compared with that of the teachers); and investigating the effects that these last two years had: (b) on the perception of the elements that contribute to defining the e-maturity of the schools (the ecosystems considered here); (c) on the factors that determine the perception of the individual well-being, also in relationship with technologies. Furthermore, this study is intended also to: (d) refine the understanding of the peculiarities characterizing the forced innovation process undergone by the learning ecosystems; (e) identify the respondents’ propensities and expectations for the future. (a–e) are the themes around which the discussion of the data analysis reported in Sect. 4.3 is organized.

4.2 The Experimental Setting 4.2.1 Factors Investigated by the Survey E-maturity. One of the working hypotheses of our survey is that the “digital shock” provoked by the pandemic may have induced an increase in the e-maturity of the schools which, before the lockdown, struggled to incorporate the learning technology into everyday practices and were characterized by a significant proportion of teachers who did not routinely use technology for learning [10]. The e-maturity is a multilevel construct [11, 12] that measures the digital maturity of a complex organizational context, such as a learning ecosystem, to evaluate the ability of the context to fully exploit the opportunities offered by the digital technologies and culture, and to prepare, therefore, strategic plans for the development of the learning ecosystem. Among the several attempts done in the past to elaborate models and methods to measure such construct, we will focus on two of them that summarize, albeit in a different way, all previous attempts. The first one, developed by Sergis and Sampson in 2014 [13] is based on three main relevant dimensions: (a) the individual competencies (which in principle includes all the actors who take part in the development of the educational processes delivered by the learning ecosystems); (b) the tangible assets (infrastructures, hardware, and software used by the various sub-processes);

50

C. Giovannella et al.

(c) the overall organizational culture of the learning ecosystems (vision and planning, professional development, etc.). The second one is the framework presented in 2015 by the JRC-IPTS, known by the acronym DigCompOrg [14], that considers 7 dimensions: (i) the infrastructure (that actually includes mainly management aspects); (ii) leadership and governance practices that together with collaboration and networking refer to the organizational culture of the structure); (iii) the professional development (also through a subsection of the teaching and learning practices) that deals with the individual competencies and their development; (iv) three dimensions that describe the educational processes (including digital resources), namely content and curricula, teaching and learning practices and assessment practices. In both the frameworks, briefly summarized above, a rather large number of very specific indicators are taken into consideration (i.e. 74 in the case of the DiCompOrg [14]). The level of granularity for each of the considered dimensions is, thus, very high. A framework like DiCompOrg can be used to perform a detailed diagnostic survey at the school level by means of the tool called SELFIE [15] (to be carried out with the approval of the school’s principal) but, at present, cannot be used to carry out online surveys to investigate the e-maturity of the whole school ecosystem and/or additional issues (e.g. the characteristics of the innovation process, etc.) like we did in the past [4, 5]. To this end, we have identified a smaller number of factors that on the one hand cover the dimensions considered in the two models described above and, on the other hand, allow us to explore also other aspects, while minimizing the fatiguing effects, and the related drop out, of the respondents. Our choice of factors (see Table 4.1) allowed us to investigate the perceptions about the level of the available resources (technological setting and competencies), some organizational factors, and some aspects related to the learning processes, with the aim to measure for a part of them not only the absolute value but also the perceived variation during the last two years. It is important to underline that, of course, this survey does not intend to measure the e-maturity of a single school but that of the whole Italian school system thanks to the participation of a suitable sample of school teachers and principals. The innovation process. The variation of the e-maturity level is obviously linked to the innovation process and to the modalities of its progress, especially in the case of a positive sign. In the survey carried out in March–April 2021 [4]—one year after the lockdown—it emerged that in the case of the innovation process triggered by the pandemic, being a forced one, the technology awareness phase has developed in parallel with the technology acceptance. However, at that time, it was not possible to clearly understand if one year time was enough to foster also a transition toward the adoption phase which, of course, would make any increase in e-maturity more stable over time. For this reason, by means of the present questionnaire, we tried to make emerge to what extent (percentage) the technologies to which the teachers were exposed were also accepted and possibly adopted, and in how much time.

4 The Italian School Ecosystems Two Years After the Lockdown …

51

Table 4.1 List of the main factors considered in the present study, organized by domains Factors domain

Factors considered in the present investigation

Learning ecosystem: technological resources

School Connectivity (SC); School Technological Adequacy (STA); Student Technological Adequacy (StTA)

Learning ecosystem: Competences

Average Teachers’ Technological Preparedness (ATTP) and its variation (dATTP); Average Teachers’ Pedagogical Preparedness (ATPP) and its variation (dATPP); Average Students’ Technological Preparedness (ASTP) and its variation (dASTP); Quality of Training (QT) and its variation (dQT)

Learning ecosystem: Organizational factors and relationships

School Digital Leadership (SDL) and its variation (dSDL); School Digital Future Vision (SDFV) and its variation (dSDFV); Operational Assistance (OA) and its variation (dOA); Usage of Smart Organization (USO) and its variation (dUSO) Cohesion among colleagues (CC) and its variation (dCC); Best Practices Sharing (BPS) and its variation (dBPS)

Personal factors: competences

Individual Technological Preparedness (ITP) and its variation (dITP); Individual Pedagogical Preparedness (IPP) and its variation (dIPP)

Personal factors: well-being

variation in Self-Fulfillment (dSF); variation in Self-Esteem (dSE); variation in Esteem from Others (dEfO); variation in Autonomy Level (dAL); variation in the Involvement Level (dIL); variation in the Intrinsic Motivation (dIM); variation in the Extrinsic Motivation (dExM)

Personal factors: perceived changes (Individual and Process levels)

variation of Personal Time Management Capacity (PTMC); variation of Student Time Management Capacity (STMC); Workload Increase (WI); variation of the Interest in Digital Challenges (dIDC); variation in the Individual Innovation Propensity (dIIP); variation in the Individual Feeling with the Technologies (dIFT); variation in the Individual e-Maturity (dIeM)

Technology enhanced educational activities/processes and their variations

Technology Enhanced Collaborative Activities Percentage (TECAP); Technology Enhanced Design Activities Percentage (TEDAP); Technology Enhanced Evaluations Percentage (TEEP) and its variation (dTEE); Variation in the Production of Digital Resources (dPDR); Variation in the Generation of Digital Archives (dGDA); Variation in the Communication with Students (dCWS); Variation in the Ability to Motivate the Students (dAMS); Variation in the Digital Inclusion (dDI); Reproducibility of Classroom Dynamics (RCD) (continued)

52

C. Giovannella et al.

Table 4.1 (continued) Factors domain

Factors considered in the present investigation

Educational activities/processes and their variations

Collaborative Activities Percentage (CAP) and its variation (dCA); Design Activities Percentage (DAP) and its variation (dDA); Competence Based Learning Percentage (CBLP); Changes in Didactic Activities Percentage (CDAP); Variation in the Integration of New Didactic Activities (dINDA)

Perceived values of technologies

Usefulness of the Didactic Technologies (UDT) and its variation (dUDT); Easiness of Use of the Didactic Technologies (EUDT) and its variation (dEUDT); Efficacy/Efficiency of Integrated didactic (EEDT) and its variation (dEEDT)

Outcomes: Learning ecosystems

Degree of e-Maturity (SeM) and its variation (dSEM)

Outcomes: Learning processes

Sustainability of Technology Augmented Didactics (SoTAD) and its variation (dSoTAD); Sustainability of Integrated Didactic (SID) and its variation (dSID); Usefulness of Education on Learning Technologies (UELT) and its variation (dUELT); Extent to which School should Rely on Technology Augmented Didactic (SRTAD); School should Rely on Smart Organization (SRSO)

Outcomes: Individual intentions

Intention to attend Training in Learning Technologies (ITLT); Intention to Use Technology Augmented Didactics (IUTAD); Intention to Use Integrated Didactic (IUID)

Well-being. Also, very important for a learning ecosystem is the level of smartness achieved, that is the well-being of all the actors participating in the educational process. The augmentation of spaces, services, and activities determined by the integration of ICTs is only part of the smartness of a learning ecosystem [16, 17]. The technologies, indeed, allow to improve and/or optimize many of the dimensions of the ASLERD pyramid (see Fig. 4.1), but not all. The dimensions located higher in the pyramid, in fact, although supported by the improvements of the lower levels, open up important psychological and social implications. It is no coincidence that for some time now, the design and evaluation of the human–machine interaction processes have gone beyond the efficiency vision to fully embrace first the multidimensionality of the experience (design for the experience) [18, 19] and more recently started to consider the well-being of the actors [20]. Among the theories that have major implications for well-being is the SelfDetermination Theory (SDT) which identifies three factors capable of generating well-being: autonomy, competence, and relatedness [21]. All are in strict relationship with the highest level of the ASLERD pyramid. In particular, relatedness coincides with the need for social relations (fourth last level) which is certainly the basis for both self-appreciation and public appreciation. As discussed in [22] the last levels of

4 The Italian School Ecosystems Two Years After the Lockdown …

53

Fig. 4.1 The dimensions that contribute to defining the smartness of a large techno ecosystem (ASLERD pyramid [27])

the pyramid are closely linked to the ability of the techno ecosystem to stimulate the state of flow [23], which is a state of engagement characterized also by a high level of satisfaction, To this latter contributes also the feeling to own adequate skills to address the challenges posed by the technological context. As well known, to maintain the state of flow, with the time, the level of the challenges must increase to prevent the development in the individual/community of a state of boredom and, as well, to foster the development of a higher level of competences. This is a mechanism that reminds the learning stimulated by the encroachment in the proximal space of development [24]. Maintaining the flow state is therefore the main way towards self-fulfillment and self-realization of individuals and, as well, of the community. Certainly, the engagement associated with the state of flow, generated by the adequacy of the challenges, also implies an adequate perception of autonomy and, probably, increases the level of extrinsic motivation. The situation is different for the intrinsic motivations which have a different and potentially very varied origin. Nevertheless, the achievement of an adequate degree of self-fulfillment can also provide reinforcement of intrinsic motivations. Summarizing: the key assumption—related also to the ongoing innovation process—is that the adoption of one or a set of technologies is related to the perceived level of personal well-being, fostered in the actors that take part in the educational processes by the evolution of the technology augmented learning ecosystem. With this in mind, we used this survey also to investigate if these two years of experience have modified the perceived value of the individual factors related to personal well-being. Outcomes and future intentions. As in previous surveys, also during this one, we have investigated if and how the factors taken into consideration—related to the innovation process, the variation of e-maturity, and the perceived wellbeing— have been capable to influence the future intentions of the participants (teachers and principals in this case). For this purpose, as in the past, we have first defined and

54

C. Giovannella et al.

measured the value of a set of factors that could be considered potential outcomes of the ongoing process and, then, we have extracted the causal relationships existing among all factors considered in the survey (see Sect. 3.2).

4.2.2 The Questionnaire and the Participants Following the elaboration of Table 4.1, we have designed a six-sections questionnaire presenting a total of 81 items. Section 1 comprises seven socio-biographical background items (gender, age, role, school level, school curriculum and teaching subject [only teacher], geographical location). Section 2 presents 27 items (requiring a multiple-choice or numerical answer); this section focuses on respondents’ perceptions of the general operating conditions and the technological context. Section 3 is composed of 7 items (5 questions requiring a single multiple-choice or numerical answer; 2 questions requiring multiple numerical answers); this section deals with the learning process and the didactic activities. Section 4 presents 5 items and deals with more general operating conditions and with the self-feeling (4 questions requiring a single multiple-choice or numerical answer; 1 question requiring multiple numerical answers). Its focus is on changes in teachers’ opinions about self-perception and about the repercussions of the last two years on the didactic activities. Section 5 presents 17 items dedicated to the feeling with technologies at the individual level (9 questions requiring a multiple-choice or numerical answer and 8 open questions or requests for explanatory comments). Section 6, finally, comprises 18 items (13 questions requiring a multiple-choice or numerical answer and 5 open questions or requests for explanatory comments); this section investigates mainly changes in teachers’ expectations for the future. The complete questionnaire (in Italian) is available at [25]. Compared to the questionnaires used in the previous surveys, the open answers have been greatly reduced and, alongside the measure of the absolute value of the factors considered in the survey, we have tried also to measure (on a scale of 5/5) the perceived variation that some of such factors have undergone during the last two years. As for the previous survey [4, 5], the teachers were contacted mainly by means of announcements on social media and via emails. The principals were contacted through the newsletter distributed by the Principals National Association (ANP) [26] that promoted the investigation, together with ASLERD [27]. Since the goal was to investigate the evolution of the Italian school ecosystem in about two years since the lockdown (March 5th, 2020), the survey was open from March 17th, 2022 till April 19th, 2022. The survey was completed by 231 teachers and 153 principals, overall 79,5% females and 20,5% males from all the Italian regions. Teachers are employed in kindergarten (14), primary (81), lower secondary (52), or upper secondary (84) schools. Principals are employed in Instituto comprensivo (primary and first secondary school level—78) and in secondary schools (75). The sample vs entire

4 The Italian School Ecosystems Two Years After the Lockdown …

55

population comparison revealed an almost perfect balance in terms of teacher gender (84% females in the sample, compared to 83% in the target population), and a somewhat imbalance as far as the principal gender (79% females in the sample compared to 68,2% in the target population) [28] and the geographical distribution [29]. The sample appeared to be also representative of the mean population age of both the teachers (49,86 vs. 48.90 years) [26] and principals (55,9 vs. 55,6 years) [30]. The fatigue effect induced in the respondents by the length of the questionnaire turned out to be as usual quite low; slightly more than 4% of respondents skipped the last multiple-choice and numerical-answer questions of the survey.

4.3 Results As we did in the past [4–9], to analyze the opinions of the respondents we pursued multiple strategies. First, we carried out descriptive and univariate analyses (Appendix–Table 4.2), that allow us to perform a comparison between the teachers’ and principals’ feelings and, partially, with the results we obtained in 2020 and 2021. Then we worked out the network of relations connecting the factors taken into consideration in this study (see Table 4.1) and tried to infer the direction of causality for such associations. More specifically, we used the PC algorithm [31, 32] to infer the direction of causality in the graph (Sect. 3.2).

4.3.1 Descriptive Analysis Table 4.2 shows the average values detected for each factor taken into consideration during this survey. The first general impression obtaind from Table 4.2 is that the “digital shock” undergone by the learning ecosystem during these last two years was perceived as positive by both teachers and principals, significatively more by the latter. The intensity of the effect, as we will see in detail below, varies from factor to factor but concerns all the variables examined and, with some exceptions, tends to show also a positive trend as a function of time. A second general observation concerns the measure of the absolute values of the factors (scale 1–10) and of the variation (scale -5/5) that such factors have undergone in the last two years in the perception of the respondents. As shown in Fig. 4.1, these two measures have an important correlation (R > 0.5). This tells us that the perceived variation in the factors’ value does not allow us to estimate their initial absolute value (assumed before the lockdown) because of the correlation among perceptions. On the other hand, the positive values of all the variations confirm that the effect of the “digital shock” on the ecosystem was positive.

56

C. Giovannella et al.

The e-maturity of the ecosystems. Despite the problems detected during the past surveys and which still afflict the Italian schools, the evaluation of the school connectivity (6.20 - > 6.40 T; 7.17 P) and of the school IT equipment (6.19 - > 6.54 T; 7.88 P) has slightly improved over time. The evaluation of the students’ computer equipment returns a slightly lower value (6.03 T; 6.57 P). Since almost all students have a smartphone, it is very likely that the teachers’ and principals’ evaluation concerns the availability of laptops whose purchase is not yet within the financial resources of all the families. As regards human resources and competencies, we note a high evaluation of her/his own skills, both digital (7.91 T; 7.93 P) and pedagogical (7.35 T; 7.26 P). On the other hand, the evaluation given of the competence available on average within the context is more parsimonious. Indeed, the evaluation assigned to the competence owned by the colleagues is somewhat lower than her/his own competencies, albeit the values have significatively increased with respect to 2020: 5.93 - > 6.49 T and 7.18 P for digital skills, 5.85 - > 6.36 T and 6.98 P for pedagogical ones. A little bit higher and closer to her/his own the evaluation assigned to students’ digital skills: 6.86 T; 7.55 P. This difference between the evaluation of one’s own competencies and the competencies of the others could be ascribed both to an over-evaluation of one’s own competencies and/or to a higher level of competencies owned by those who are more active on social networks and more willing to participate in the survey. Unfortunately, the structure of the questionnaire does not allow to clarify this point. It is also important to underline that despite the globally positive trend observed between 2020 and today, we observe a small, although not fully significant, decrease in the value assigned to the competencies of the colleagues compared to 2021. A more significant negative trend is found in the level of Cohesion and collaboration among colleagues (−0.89 on the absolute value), despite still a persisting perception of improvement with respect to 2020 (+1.40 in terms of perceived variation). These observations represent a warning because indicate a possible setback, if not a retreat, with respect to the climate of strong collaboration between colleagues that seemed to have arisen during the first year of the pandemic, possibly due to the need to share problems and solutions [4]. The largest differences between the evaluations provided by teachers and principals are observed, as might be expected, for the factors relating to the organizational aspects: digital leadership (7.97 P vs. 6.48 T), digital future vision (7.88 P vs. 6.27 T), operational assistance (7.75 P vs. 6.23 T), technology-enhanced processes and organization or smart organization (7.81 P vs. 6.35 T) and best practices sharing (7.09 P vs. 6.16 T). Also for these factors, the perceived change compared to 2020 turns out to be positive for all factors and all respondents. The positive picture illustrated so far is partly confirmed by the global assessment of the e-maturity of the learning ecosystem (7.12 P vs. 6.33 T) and by the perception of a relatively high variation in the value of such factor compared to 2020 (+2.82 P vs. 2.04 T), even if the same type of setback is observed with respect to the absolute value of the factor perceived by the teachers in 2021.

4 The Italian School Ecosystems Two Years After the Lockdown …

57

The individual well-being. As discussed in Sect. 2.1, in order to stabilize a process of technological innovation and the development of a digital culture for teaching, a perceived improvement also on a personal level would be required. Even more, if we consider that all respondents think that technologically augmented, integrated and remote teaching generates an increase in the perceived working load, of the order of 60% for teachers and somewhat higher for the principals (67%). It is therefore important that the perception of positive change develops both at a personal level and in the relationship with technologies. As regards the latter, we find a rather positive perception of improvement in all the parameters considered by the survey, more for the principals than for the teachers, being these latter always more cautious: + 2.4 P versus + 1.65 T for the variation of Interest in Digital Challenges (dIDC); + 2.28 P versus + 1.84 T for the variation in the Individual Feeling with Technologies (dIFT); + 2.68 P versus + 1.96 T for the variation in the Individual Innovation Propensity (dIIP) and finally + 2.49 P versus + 2.05 T in the variation of the perceived Individual e-Maturity (dIeM). Also regarding personal wellbeing, the survey returns us a positive picture, albeit more attenuated in the values. At the top is the sense of greater autonomy (+2.37 P vs. + 1.73 T), followed by the perception of greater involvement (+2.08 P vs. + 1.34 T) and therefore the sense of higher esteem by others in what is being done (+1.82 P vs. + 1.39 T). Somewhat lower, but still positive, the variation in personal motivation, both internal (+1.63 P vs. + 0.95 T) and external (+1.67 P vs. + 0.95 T), and the level of self-esteem (+ 1.63 P vs. + 0.94 T). Further down we find the variation in self-fulfilment (+1.17 P vs. + 0.59 T). These results tell us that on the one hand there is a true increase in personal wellbeing, but on the other that the elements that compose the most “external” layer encounter some difficulty in being transferred to the most “internal” layer, i.e. to generate an equivalent level of self-fulfilment. To this landscape should be added an improvement in the organization and management of her/his own time supported by the introduction of technologies (+1.07 P vs. 1.04 T). This improvement, albeit contained, constitutes a substantial leap in quality compared to the trouble created by the impact of the lockdown; in fact, we recall that in 2020 there was a worsening in the ability of teachers to manage their time (−1.08) [4]. This is therefore a further confirmation of the increased feeling between teachers and technologies. The learning processes. This part of the survey concerned only the teachers who highlighted how the experience undertaken during the last two years led to a 49% change in current didactic practices including also the Integration of New Didactic Activities (+1.81). The practices implemented by teachers include Collaborative Activities (40% of the total) that may include or not Design Activities (41%) strongly Technology Enhancement. In line with these figures also Technology Enhanced Evaluations stands at 39%. Very interesting is the fact that teachers declare to have also largely implemented Competence Based Learning (48%). It is very likely that the average values reported above may vary among the various levels of schools, but the segmentation of the target is not the focus of this paper and will be the subject of future elaborations of the data.

58

C. Giovannella et al.

The effects of the “digital shock” were also felt on the Production of Digital Resources (+1.86) and on Generation of Digital Archives (+1.93), slightly less on the ability to communicate with the students (+1.34) and motivate them (+1.34). A positive effect (+1.38) has been detected also on the Digital Inclusion that, as we know from the previous investigation affected between 6 and 10% of the student population [5]. The panorama that has emerged so far, overall very positive, has induced an extremely positive perception of the learning technologies in use by both teachers and principals: we measured 7.43 T versus 8.35 P for the Usefulness of the Learning Technologies (ULT) and 7.19 T versus 7.88 P for their Efficacy and Efficiency. All these, most likely, are thanks also to a high perception of the Easiness of Use Learning Technologies (7.27 T vs. 7.49 P). The capability of the technologies to Reproduce the Classroom Dynamics continues to be perceived by teachers as low (6.00). However, this is deemed only partially relevant since in any case, as it has been declared in a few comments by the teachers, the dynamics of the didactic activities enhanced by the technologies should be largely modified to work. Notes on the characteristics of the innovation process. All the elements gathered up to now seem to point towards the stabilization of the technological innovation process triggered by the pandemic. The investigations carried out previously had identified the parallelism between the awareness and acceptance phases of the process and, as well, an advanced status of the acceptance phase but it was not possible to understand whether the acceptance phase had given rise to the adoption one. To shed light on this aspect of the undergoing innovation process, during this survey we have explicitly asked the teachers to provide information on the technologies tested and in use and on the time required for their acceptance and adoption. Apparently, the percentage of the technologies to which the teachers have been exposed but that they decided not to use is equal to 40%. Unfortunately, this data does not allow us to understand whether teachers have been exposed to a small number of technologies or if they have adopted a high percentage of them. However, the percentage of technologies tested and used in the didactic processes, according to the respondents, is equal to 52%, which, added to the previous 40%, give a figure close to 100%. Apparently 45% of the technologies that have been used by the teachers during the past two years have been definitively adopted. The most surprising outcome, however, concerns the time interval needed for the transition from technology acceptance to technology adoption. The average time taken to accept technology and use it in the ongoing learning process has been 17.9 days while the average time to adopt it has been 22,9 days. If we assume that the respondents have well understood the difference between acceptance and adoption it appears that during a forced technology innovation process the awareness phase was more than twice longer than the acceptance one and that in 5 days the transition from acceptance to adoption was completed. Such a result can be explained and justified by the concomitance of two factors: the need to be operative in a time as short as possible and the extreme easiness in the use of the technologies. If this

4 The Italian School Ecosystems Two Years After the Lockdown …

59

were the case, this could explain why the most adopted applications during the past two years have been those designed for domains other than the educational one and that were tested by thousands and thousands of people before the pandemic (like Zoom, Teams, Mirò, etc.). Not by chance the respondents indicated among the digital competencies/abilities most developed during the last two years the use of applications to foster communication (68%); followed by: the use of digital environments for collaboration (59%); collection, sharing and organization of digital resources (55%) as well as the ability to search for and filter on-line content (34%); the use of applications capable to reproduce virtual classes (48%); the preparation of content for offline (44%) and online (41%) presentations, together to the use of applications for the personal production (30%); the digital processing of movies (28%) and images (25%), etc. Future intentions. Finally, as for the previous surveys we have investigated the future intentions and expectations of the respondents both to evaluate the stability of the positive effects found so far, and to outline a reference framework within which to develop new research and policies, in order to improve didactic processes and increase the smartness of the learning places. Quite strong is the perception of Sustainability of the Technology Enhanced Didactics (6.94 T vs. 7.87 P) that teachers intend to adopt, on average, for 56% of the didactic activities. A result that is consistent with their belief that the schools, as a whole, should use the support of technologies, on average for 57% of the didactic activities (69% in the principals’ opinion). As regards the Sustainability of the Integrated Didactic, the teachers appear less convinced than the principals (5.87 T vs. 7.43 P) and seem to send out another regression signal compared to 2021 (6.17 - > 5.87). Still, very open teachers appear on the use of online technologies to implement smart school management, strongly desired by the principals (6.53 T vs. 7.76 P). Another relevant outcome is the strong expectation for continuous training on learning technologies and technology-enhanced didactics (7.90 T vs. 8.71 P), a training that teachers seem keen to attend (7.10).

4.3.2 Causal Discovery The search for the causal relations returned the picture in Fig. 4.3 that, as in previous investigations [4, 5, 7, 8], should be interpreted as a tentative one, due to possible hidden variables. To better determine the direction of the relationships we have considered either the answers of the teachers and those of the principals as a unique dataset. In Fig. 4.3 are not shown the respondents’ feelings about the variation of the values, due to the dependence of the former on the latter (see Fig. 4.2). Moreover, are not shown, also the variables concerning the didactic activities and the changes in the factors related to the individual well-being because they form isolated clusters

60

C. Giovannella et al.

Fig. 4.2 Plot of the perceived absolute values versus their perceived variation (Delta) during the last two years for both teachers (blue dots) and principals (fuchsia squares). Linear fits are reported at the bottom

not connected to the main causal network shown in Fig. 4.3. In this latter, we can distinguish two main macro-areas: a first one that has as chain terminal the e-maturity of the learning ecosystems (SeM) and a second one that has as main terminals the Extent to which School should Rely on Technology Augmented Learning (SRTAD) and to Smart Organization (SRSO). As far as the first macro-area one has to note that the Student Technological Adequacy—BYOD (StTA) is influenced by the Technological Preparedness of the Students (ASTP) and, at the same time by the School Connectivity (SC). This latter influences, as expected, the Technological Adequacy of the schools (STA) as also StTA does. Very logically STA is strictly related to the Operational Assistance (OA) that through the Digital Leadership (SDL) leads to the Smart Organization (USO). SDL influences, as expected, the Digital Future Vision (SDFV) that contributes also to USO. The terminal of this chain, as anticipated above, is the e-maturity of the ecosystem and it is interesting to note that the perception of SeM is sustained by a competence factor (average teachers competencies, ATTP), a strictly organizational factor (USO) and a cultural/behavioral factor (Best Practices Sharing, BPS), to underline that the digital adequacy of the school, the leadership and the quality of the training/collaboration are all needed to sustain the digital innovation of educational processes. Possibly, an additional warning arrives from the influence of BPS and SDL on the Cohesion and collaboration among colleagues (CC). In fact, one year after the lockdown, following a difficult year, CC was considered a pillar of the causal chain

4 The Italian School Ecosystems Two Years After the Lockdown …

61

Fig. 4.3 Causal structure of the main factors considered in this study

while now, during the “new normality”, it appears to be a product of the leadership so that this latter is considered relevant also in the definition of policies on the sharing of the best practices. It is also interesting to note that along another chain the propensity to share the best practices (BPS) contributed to the Quality of the Training (QT) and this, in turn, contributes not only to SDL but also to the perception of Sustainability of the Integrated Didactics (SID) before to converge into the Extent to which School should Rely on Smart Organization (SRSO), a terminal of the second macro area. Within the second macro area, another chain starts from the perception of Efficacy/Efficiency of Learning Technologies (EELT) that influences the Easiness of Use of the Learning Technologies (EULT). This latter is also influenced by the perception about her/his own Technological Preparedness (ITP), while both EULT and ITP determine the perception of Pedagogical Preparedness IPP. EELT influences also the perception of the Usefulness of the Learning Technologies (ULT), directly and through the perception of the Sustainability of Technology Enhanced Didactics (SoTAD). This latter influences SRSO either directly and through SID. It is also interesting to note that SoTAD can be considered a “preliminary perception” that influences SID. Finally, SoTAD influences SRSO also through the path that goes through ULT and the Extent to which School should Rely on Technology Augmented Learning (SRTAD). In addition, the path through ULT arrives also to the Intention to attend Training in Learning Technologies (ITLT) and to the intention to use technology augmented didactic activities (IUTAD) before converging in SRTAD.

62

C. Giovannella et al.

In summarizing we can state that the perception of sustainability of both the technology augmented didactics and the integrated didactics, together with the expectations about the future use of the technology augmented learning, determines the expectations of a smart organization of the school. It is interesting to note, however, that IUTAD represents a sort of intermediate terminal influenced by EULT, ITLT, and the percentage of the changes made to the didactic activities (CDAP—only teachers), being this latter connected to the working load (WI) and the capability of technologies to support the reproducibility of the classroom dynamics (RCD). One has also to note that ITLT determines the perception of Usefulness of Education on Learning Technologies (UELT). A further notation concerns the factors considered relevant by the Technology Acceptance Model (TAM) [33], that is ULT and EULT. In the previous investigations [4, 5, 9] their role appeared to be quite collateral, an impression confirmed by Fig. 4.3. Nevertheless, although EULT occupies a side position and ULT could be bypassed through other paths they contribute directly or indirectly to IUTAD and SRTAD. To conclude, one cannot avoid noting that for the respondents the variations induced by these two years’ experience in the organization of their own and students’ time are not so relevant for their future intention and expectations. PTCM and STMC form, in fact, an isolated cluster.

4.4 Final Considerations and Conclusions Two years have passed since the lock-down, a lot of water has flowed under the bridges and the feeling of those who ensured the continuity of the educational processes in the Italian school—first of all teachers and principals—has changed and evolved over time, as shown by the present and previous investigations [4, 5]. By means of surveys carried out during some topical moments of the pandemic evolution, we have been able to return pictures of the emergency and, as well, of the adaptation of the schools to new dimensions of the teaching (like the parallel blended learning). Evolutions that have been subjected also to a continuous modification of the social landscape and of the political indications. In addition, we have also tried to return a picture of the technological innovation process triggered by the lock-down and to make emerge its transformative potential that extended till the time of the “new normality”. At present, we are facing a school ecosystem divided between the firm will of those who have deeply suffered the lockdown and wish to restore as soon as possible the full normality, and the desire of others that wish to encourage a reflection on the potential and the opportunities that the “digital shock” and its positive effects can provide well beyond the emergency. Which message did the schools’ teachers and principals wish to convey with this survey? They certainly highlighted that the ecosystem has been subjected to a “digital shock” and—although their judgments might have been affected by a bias when referring to their own person, or to the structure for which they are responsible—that

4 The Italian School Ecosystems Two Years After the Lockdown …

63

it had and still have a positive effect on many relevant aspects: the development of digital skills, the management of the school and the future vision about a technology augmented education, the personal well-being. A positive judgment that in no way represents an unconditional sponsorship of online education but, rather, a meditated reflection on an experience lived in the first person, not by hearsay. An absolute positive result was the acquired awareness about the need and the relevance of adequate training on the support that technologies can provide to the teaching processes and to the improvement of the didactic methodologies (digital pedagogy), a training that cannot be limited to the initial one but that should accompany teachers and principals along with the whole working life. The use of digital technologies and remote practices is not effortless: after two years, in fact, everyone has understood that it involves a greater workload (a reason for which many do not want to be involved in). However, such workload could be reduced considerably by an adequate preparation, possibly, focused also on aspects that go far beyond the preparation and management of content and the trivial use of communication and socialization tools. From these last two years of experience we have also understood that, to go further away in the innovation process, the technologies made available to teachers should be easy to use and should return immediate results. The information gathered on the innovation process, in fact, tells us that when you combine a didactic need and a technology easy to use, then, the technology is adopted in a time much shorter than one might imagine. If on the one hand this is a good new, on the other it explains the failures of decades of European policies and efforts in the development of learning technologies. For sure such efforts have partly discounted the effects of a rapid technological evolution but, on the other hand, they have been rendered useless by the scarce attitude to the “market” of the European scholars. Basically, the research groups and the project managers have not understood that the target was not the few participants to case studies or pilots but the entire school population which, in turn, also represents a large slice of the entire population. The lesson that emerges from the pandemic time is clear: it doesn’t matter that an application has been developed for the educational domain, what is important is that it works also for the educational domain. Moreover, it appears that only when the technologies are perceived as simple to use that the individuals perceive also an improvement in their pedagogical competencies. One of the questions that arises, then, but which cannot be answered right now is the following: ICT and internet majors will have ever the time and the interest to direct their efforts toward the development of applications a little bit more pedagogically driven or should be always the teachers that have to find a way to adapt the available applications to the teaching needs? This question could also stimulate policymakers to reflect on how to use at the best the available resources for the digitalization of the society. This survey also made clearly emerge that the use of digital technology does not harm the social and psychological dimensions but, on the contrary, contributes to the individual well-being. It strengthens the autonomy level, as well as the esteem of others and of the individual in her/himself which, in some cases, transform also into a perception of higher self-realization. The social and psychological factors, as well

64

C. Giovannella et al.

as the political ones [5], in any case, do not influence the perceptions of teachers and principals about the factors that contribute to defining the e-maturity of a context, nor determine their expectations on how the context should exploit at best the potential of the digital, or an individual should operate in a digitally augmented context. It seems clear to the most that the e-maturity of a context is determined mainly by three macro-factors: the improvement of the infrastructures, the continuous increase of the average digital competencies of the individuals and, even more, an adequate leadership and future vision of the digital innovation. Nobody seems to have nostalgia for the learning activities realized during the emergency. Nevertheless, in a very clear and mature way, a large majority believes that we must seriously reflect on the deeper integration of the technologies in the school ecosystem. 64% of the respondents, in fact, consider viable and feasible the use of integrated didactic (and not because of the pandemic). In particular, they think that the integration of the in-presence with the online can be very useful to carry out strengthening activities (55%), foster collaborative activities (50%), carry out exercises (47%), watch again the lectures (45%), deliver documents (34%) and correct homework (26%), carry out complementary activities (26%), deliver lectures (20%), implement dual-educational schemes (called PCTO in Italy—15%) and, finally, carry out assessments (10%). Moreover, although almost 64% of the respondents are not aware of what digital micro certification of competencies are, 66% would like to test them in their own schools. In addition, we detected also a clear desire to take full advantage of the technologies also in the management of the school to optimize the individual time and improve the efficiency of the system, for example to hold meetings, for teacher training, to stimulate participation and collaboration between colleagues, to interact with families and stakeholders of the community and of the territory of reference. All the elements that emerged during this investigation seem to point toward a very different meaning of the “new normal”, which should not be perceived and experienced as a period of restoration but, rather, as a period of evolution of the learning ecosystems and of the learning processes to exploit the full support of the technologies, the integration of remote activities and advanced forms of smart organization. As we have seen, however, a few warnings are also emerging: collaboration among colleagues does not seem to be as strong as one year ago, as also the perception of the level of available competencies. Up to now, they are small signals that do not affect the very positive landscape that emerges from the survey, but they tell us that we are at a crossroads between treasuring the positive legacy of a dramatic experience or dispersing it by nullifying the many sacrifices that have been done and by making the school assume an anti-historical position.

Appendix

V = 6093 p < 0.001 Cohen’s d = 1.12 W = 15,802 p = 0.004 V = 6435 p < 0.001 Cohen’s d = 2.24 W = 18,224 p < 0.001 V = 5445 p < 0.001 Cohen’s d = 0.78 W = 14,694 p = 0.22 V = 6555 p < 0.001 Cohen’s d = 1.99 W = 16,950 p < 0.001 V = 6328 p < 0.001 Cohen’s d = 1.35 W = 15,835 p < 0.001

M = 7.17 [6.89, 7.44]

M = 7.88 [7.68, 8.07]

M = 6.57 [6.32, 6.83]

M = 7.18 [7.02, 7.33] (+3.18)

M = 6.98 [6.78, 7.19] (+2.79)

School Connectivity (SC)

School Technological Adequacy (STA)

Student Technological Adequacy—BYOD (StTA)

Average Teachers’ Technological Preparedness (ATTP)

Average Teachers’ Pedagogical Preparedness (ATPP)

Learning ecosystem

Wilcoxon tests

Mean P

Factors

M = 6.36 [6.15, 6.57] (+2.41)

M = 6.49 [6.29, 6.68] (+2.62)

M = 6.03 [5.77, 6.29]

M = 6.54 [6.27, 6.80]

M = 6.40 [6.12, 6.68]

Mean T

V = 20,806 p < 0.001 Cohen’s d = 0.54

V = 22,201 p < 0.001 Cohen’s d = 0.66

V = 17,308 p < 0.001 Cohen’s d = 0.27

V = 20,853 p < 0.001 Cohen’s d = 0.50

V = 19,874 p < 0.001 Cohen’s d = 0.42

Wilcoxon test

M = 5.85 [5.65, 6.05] W = 32,658 p = 0.0002

M = 5.93 [5.72, 6.14] W = 32,444 p = 0.001

M = 6.36 [6.10, 6.62] W = 38,022 p = 0.60

Mean T 2020

(continued)

M = 6.55 [6.37, 6.74] W = 35,462 p = 0.014

M = 6.69 [6.52, 6,85] W = 36,088 p = 0.14

M = 6.19 [5.90, 6.47] W = 31,820 p = 0.20

M = 6.20 [5.93, 6.46] W = 32,452 p = 0.36

Mean T 2021

Table 4.2 Outcomes of the principals’ and teachers’ perceptions about the factors listed in Table 4.1: Mean values. A 10 points Likert-like scale (1–10) was used unless otherwise indicated. Comparisons are performed against the column “Mean T” by means of a two samples Wilcoxon test

4 The Italian School Ecosystems Two Years After the Lockdown … 65

V = 6459 p < 0.001 Cohen’s d = 1.79 W = 16,156 p < 0.001 V = 6528 p < 0.001 Cohen’s d = 1.97 W = 18,658 p < 0.001 V = 6555 p < 0.001 Cohen’s d = 2.48 W = 12,806 p = 0.62 V = 6058 p < 0.001 Cohen’s d = 1.54 W = 11,655 p = 0.20 V = 6407 p < 0.001 Cohen’s d = 2.12 W = 18,569 p < 0.001

M = 7.55 [7.34, 7.65] (+2.98)

M = 7.89 [7.66, 8.11] (+2.89)

M = 7.93 [7.74, 8.12] (+3.07)

Individual Technological Preparedness (ITP)

Individual Pedagogical M = 7.26 Preparedness (IPP) [7.05, 7.48] (+2.41)

M = 7.97 [7.76, 8.19] (+3.45)

Quality of Training (QT)

School Digital Leadership (SDL)

Wilcoxon tests

Mean P

Factors

Average Students’ Technological Preparedness (ASTP)

(continued)

M = 6.48 [6.21, 6.76] (+2.03)

M = 7.35 [7.17, 7.53] (+2.73)

M = 7.91 [7.73, 8.08] (+3.32)

M = 6.39 [6.10, 6.68] (+1.88)

M = 6.86 [6.65, 7.06] (+2.61)

Mean T

V = 19,820 p < 0.001 Cohen’s d = 0.46

V = 25,688 p < 0.001 Cohen’s d = 1.32

V = 26,573 p < 0.001 Cohen’s d = 1.77

V = 19,547 p < 0.001 Cohen’s d =0 .40

V = 22,567 p < 0.001 Cohen’s d =0 .87

Wilcoxon test

Mean T 2020

(continued)

M = 7.43 [7.02, 7.83] W = 34,502 p = 0.54

M = 7.68 [7.52, 7.83] W = 30,368 p = 0.05

Mean T 2021

66 C. Giovannella et al.

Technology enhanced educational

V = 18,508 p < 0.001 Cohen’s d = 0.33

M = 6,16 [5.90, 6.41] (+1.75)

V = 6220 p < 0.001 Cohen’s d = 1.32 W = 16,479 p < 0.001

M = 7.09 [6.86, 7.31] (+2.52)

Best Practices Sharing (BPS)

V = 19,676 p < 0.001 Cohen’s d = 0.46

Cohesion and collaboration among colleagues CC

V = 18,228 p < 0.001 Cohen’s d = 0.33

M = 6.47 [6.19, 6.75] (+1.40)

V = 6388 p < 0.001 Cohen’s d = 1.97 W = 19,236 p < 0.001

M = 7.81 [7.59, 8.03] (+3.11)

Usage of Smart Organization (USO)

M = 6.23 [5.94, 6.52] (+2,01)

V = 18,805 p < 0.001 Cohen’s d = 0.37

Wilcoxon test

(Only teachers)

V = 6286 p < 0.001 Cohen’s d = 1.98 W = 18,234 p < 0.001

Operational Assistance M = 7.75 (OA) [7.54, 7.96] (+3.28)

M = 6.27 [6.00, 6.55] (+1.79)

Mean T

V = 20,182 p < 0.001 Cohen’s d = 0.45

V = 6200 p < 0.001 Cohen’s d = 1.97 W = 18,610 p < 0.001

M = 7.88 [7.65, 8.11] (+3.13)

M = 6.35 [6.10, 6.60] (+1.78)

Wilcoxon tests

Mean P

Factors

School Digital Future Vision (SDFV)

(continued) Mean T 2020

(continued)

M = 7,36 [7.13, 7.59] W = 41,902 p < 0.001

M = 6.72 [6.46, 6.98] W = 38,052 p = 0.005

Mean T 2021

4 The Italian School Ecosystems Two Years After the Lockdown … 67

(Only teachers)

(Only teachers)

(Only teachers)

Variation in the Integration of New Didactic Activities (dINDA) (−5, 5 scale)

Technology Enhanced Collaborative Activities Percentage (TECAP) %

Technology Enhanced Design Activities Percentage (TEDAP) %

V = 21,736 p < 0.001 Cohen’s d = 1.42 V = 20,503 p < 0.001 Cohen’s d = 1.36

M = 40 [37, 44]

M = 41 [37, 45]

V = 18,457 p < 0.001 Cohen’s d = 0.81

V = 25,200 p < 0.001 Cohen’s d = 1.99

M = 49 [45, 52]

(Only teachers)

Changes in Didactic Activities Percentage (CDAP) % M = 1.81 [1.51, 2.11]

V = 23,436 p < 0.001 Cohen’s d = 1.62

M = 48 [44, 52]

(Only teachers)

Competence Based Learning Percentage (CBLP) %

V = 21,945 p < 0.001 Cohen’s d = 1.48

M = 43 [39, 47]

(Only teachers)

V = 22,366 p < 0.001 Cohen’s d = 1.50

Design Activities Percentage (DAP) %

Wilcoxon test

Wilcoxon tests M = 41 [37, 44]

(Only teachers)

Mean T

Mean P

Factors

Collaborative Activities Percentage (CAP) %

(continued) Mean T 2020

(continued)

M = 50 [48, 53] W = 39,241 p < 0.001

Mean T 2021

68 C. Giovannella et al.

V = 13,686 p < 0.001 Cohen’s d = 0.56 V = 13,106 p < 0.001 Cohen’s d = 0.57 V = 14,352 p < 0.001 Cohen’s d = 0.57 V = 16,394 p < 0.001 Cohen’s d = 0.20

M = 1.34 [1.03, 1.65]

M = 1.34 [1.03, 1.66]

M = 1.38 [1.06, 1.71]

M = 6.00 [5.68, 6.33]

(Only teachers)

(Only teachers)

Variation in the Generation of Digital Archives (dGDA); (−5, 5 scale)

Variation in the Communication with Students (dCWS); (−5, 5 scale)

Variation in the Ability (Only teachers) to Motivate the Students (dAMS); (−5, 5 scale)

Variation in the Digital (Only teachers) Inclusion (dDI); (−5, 5 scale)

Reproducibility of Classroom Dynamics (RCD)

(Only teachers)

V = 19,100 p < 0.001 Cohen’s d = 0.82

M = 1.86 [1.56, 2.16]

(Only teachers)

Variation in the Production of Digital Resources (dPDR); (−5, 5 scale) V = 18,290 p < 0.001 Cohen’s d = 0.87

V = 19,306 p < 0.001 Cohen’s d = 1.27

M = 1.93 [1.64, 2.23]

Wilcoxon test

M = 39 [35, 43]

Wilcoxon tests

Mean T

Mean P

Technology Enhanced (Only teachers) Evaluations Percentage (TEEP) %

Factors

(continued)

M = 5.32 [5.08, 5.57] W = 30,888 p < 0.001

Mean T 2020

(continued)

M = 5.95 [5.72, 6.19] W = 31,574 p = 0.31

Mean T 2021

4 The Italian School Ecosystems Two Years After the Lockdown … 69

V = 6415 p < 0.001 Cohen’s d = 1.91 W = 15,300 p = 0.003

Efficacy/Efficiency of M = 7.88 [7.65, Learning Technologies 8.12] (EELT) (+2.64)

V = 3817 p < 0.001 Cohen’s d = 0.37 W = 13,458 p = 0.69 V = 3166 p < 0.001 Cohen’s d = 0.47 W = 15,483 p = 0.004

Personal Time M = 1.07 Management Capacity [0.55, 1.59] (PTMC); ((−5, 5 scale)

Student Time M = 0.87 Management Capacity [0.53, 1.21] (STMC); (−5, 5 scale)

Personal factors

V = 6056 p < 0.001 Cohen’s d = 1.56 W = 13,588 p = 0.32

Easiness of Use of the M = 7.49 Learning Technologies [7.25, 7.73] (EULT) (+2.59)

Wilcoxon tests V = 6199 p < 0.001 Cohen’s d = 2.27 W = 16,384 p < 0.001

Mean P

Usefulness of the M = 8.35 Learning Technologies [8.12, 8.59] (ULT) (+3.08)

Factors

(continued)

M = 0.07 [−0.24,0.39]

M = 1.04 [0.71, 1.36]

M = 7.19 [6.94, 7.45] (+2.16)

M = 7.27 [7.03, 7.50] (+2.19)

M = 7.43 [7.18, 7.68] (+2.29)

Mean T

V = 9571 p = 0.69 Cohen’s d = 0.31

V = 13,277 p < 0.001 Cohen’s d = 0.41

V = 22,890 p < 0.001 Cohen’s d = 0.87

V = 24,094 p < 0.001 Cohen’s d = 0.98

V = 24,336 p < 0.001 Cohen’s d = 1.01

Wilcoxon test

M = -.43 [-.74, -.12] W = 27,232 p < 0.001

Mean T 2020

(continued)

M = -1.08 [-1.40, -0.76] W = 18,836 p < 0.001

Mean T 2021

70 C. Giovannella et al.

V = 6105 p < 0.001 Cohen’s d = 2.54 W = 14,948 p = 0.03 V = 4403 p < 0.001 Cohen’s d = 1.52 W = 13,306 p = 0.04 V = 4554 p < 0.001 Cohen’s d = 1.23 W = 12,742 p = 0.22 V = 4858 p < 0.001 Cohen’s d = 1.55 W = 13,574 p = 0.03 V = 4812 p < 0.001 Cohen’s d = 1.45 W = 12,558 p = 0.22

M = 67 [62, 72]

M = 2.4 [2.09, 2.71]

Variation in the M = 2.28 Individual Feeling with [1.93, 2.64] the Technologies (dIFT); (−5, 5 scale)

Variation in the M = 2.68 Individual Innovation [2.35, 3.01] Propensity (dIIP); (−5, 5 scale)

M = 2.49 [2.15, 2.82]

Variation of the Interest in Digital Challenges (dIDC); (−5, 5 scale)

Variation in the Individual e-Maturity (dIeM); (−5, 5 scale)

Wilcoxon tests

Mean P

Factors

Workload Increase WI %

(continued)

M = 2.05 [1.75, 2.36]

M = 1.96 [1.64, 2.29]

M = 1.84 [1.52, 2.16]

M = 1.65 [1.32, 1.98]

M = 60 [57, 64]

Mean T

V = 16,662 p < 0.001 Cohen’s d = 0.90

V = 17,234 p < 0.001 Cohen’s d = 0.80

V = 14,402 p < 0.001 Cohen’s d = 0.77

V = 15,372 p < 0.001 Cohen’s d = 0.66

V = 25,878 p < 0.001 Cohen’s d = 2.20

Wilcoxon test M = 65 [63, 68] W = 43,220 p = 0.03

Mean T 2020

(continued)

M = 68 [66, 71] W = 39,676 p < 0.001

Mean T 2021

4 The Italian School Ecosystems Two Years After the Lockdown … 71

V = 2797 p < 0.001 Cohen’s d = 0.94 W = 13,128 p = 0.03 V = 3232 p < 0.001 Cohen’s d = 1.08 W = 12,214 p = 0.25 V = 3193 p < 0.001 Cohen’s d = 0.78 W = 12,858 p = 0.07 V = 3296 p < 0.001 Cohen’s d = 0.87 W = 12,900 p = 0.05

Variation in Esteem by M = 1.82 Others (dEfO); (−5, 5 [1.49,2.14] scale)

M = 1.63 [1.23,2.04]

M = 1.67 [1.30,2.04]

Variation in the Intrinsic Motivation (dIM); (−5, 5 scale)

Variation in the Extrinsic Motivation (dExM); (−5, 5 scale)

Variation in Self-Esteem (dSE); (−5, 5 scale)

M = 1.63 [1.29,1.97]

Wilcoxon tests V = 2458 p < 0.001 Cohen’s d = 0.60 W = 13,046 p = 0.11

Mean P

Variation in M = 1.17 Self-Fulfillment (dSF); [0.79,1.55] (−5, 5 scale)

Factors

(continued)

M = 0.95 [0.61, 1.29]

M = 0.95 [0.61, 1.30]

M = 1.39 [1.09, 1.70]

M = 0.94 [0.63, 1.25]

M = 0.59 [0.27, 0.92]

Mean T

V = 10,130 p < 0.001 Cohen’s d = 0.37

V = 10,429 p < 0.001 Cohen’s d = 0.37

V = 11,646 p < 0.001 Cohen’s d = 0.62

V = 10,558 p < 0.001 Cohen’s d = 0.41

V = 9808 p < 0.001 Cohen’s d = 0.24

Wilcoxon test

Mean T 2020

(continued)

Mean T 2021

72 C. Giovannella et al.

V = 3938 p < 0.001 Cohen’s d = 1.13 W = 12,858 p = 0.07 V = 3828 p < 0.001 Cohen’s d = 1.46 W = 13,346 p = 0.05

M = 2.08 [1.72,2.43]

Variation in Autonomy M = 2.37 Level (dAL); (−5, 5 [2.06,2.68] scale)

V = 6260 p < 0.001 Cohen’s d = 1.63 W = 15,194 p < 0.001 V = 6346 p < 0.001 Cohen’s d = 1.71 W = 15,570 p < 0.001 V = 5903 p < 0.001 Cohen’s d = 1.05 W = 15,194 p < 0.001

M = 7.12 [6.94, 7.31] (+2.82)

M = 7.87 [7.61, 8.13] (+2.62)

M = 7.43 [7.09, 7.78]

Degree of e-Maturity (SeM)

Sustainability of Technology Augmented Didactics (SoTAD)

Sustainability of Integrated Didactic (SID)

Outcomes

Wilcoxon tests

Mean P

Factors

Variation in the Involvement Level (dIL); (−5, 5 scale)

(continued)

M = 5.87 [5.52, 6.22]

M = 6.94 [6.66, 7.21] (+1.75)

M = 6.33 [6.06, 6.59] (+2.04)

M = 1.73 [1.42, 2.03]

M = 1.34 [1.01, 1.66]

Mean T

V = 14,236 p = 0.008 Cohen’s d = 0.14

V = 20,032 p < 0.001 Cohen’s d = 0.70

V = 17,984 p < 0.001 Cohen’s d = 0.42

V = 13,788 p < 0.001 Cohen’s d = 0.76

V = 12,760 p < 0.001 Cohen’s d = 0.55

Wilcoxon test

M = 5.17 [4.93, 5.42] W = 29,164 p < 0.001

M = 6.36 [6.13, 6.59] W = 36,474 p = 0.68

Mean T 2020

(continued)

M = 6.17 [5.91, 6.43] W = 31,976 p = 0.47

M = 6.51 [6.27, 6.75] W = 33,004 p = 0.24

Mean T 2021

4 The Italian School Ecosystems Two Years After the Lockdown … 73

(Only teachers)

(Only teachers)

Intention to Use Technology Augmented Didactics (IUTAD) %

V = 5904 p < 0.001 Cohen’s d = 1.26 W = 15,256 p < 0.001

School should Rely on M = 7.76 Smart Organization [7.43, 8.09] (SRSO)

Intention to attend Training in Learning Technologies (ITLT)

V = 6105 p < 0.001 Cohen’s d = 3.39 W = 14,814 p < 0.001

Extent to which School M = 69 should Rely on [65, 73] Augmented Didactic (SRTAD) %

Wilcoxon tests V = 6421 p < 0.001 Cohen’s d = 2.37 W = 16,692 p < 0.001

Mean P

Usefulness of M = 8.71 [8.46, Education on Learning 8.96] Technologies (UELT) (+3.32)

Factors

(continued)

V = 16,790 p < 0.001 Cohen’s d = 0.40

V = 19,378 p < 0.001 Cohen’s d = 0.65 V = 22,791 p < 0.001 Cohen’s d = 2.11

M = 7.10 [6.77, 7.43]

M = 56 [53, 60]

V = 22,578 p < 0.001 Cohen’s d = 2.19

V = 22,227 p < 0.001 Cohen’s d = 1.18

Wilcoxon test

M = 6.53 [6.18, 6.87]

M = 57 [54, 61]

M = 7.90 [7.63, 8.17] (+2.64)

Mean T M = 8.04 * [7.81, 8.27] W = 37,187 p = 0.25

Mean T 2020

M = 7.30 [7.00, 7.60] W = 35,928 p < 0.001

M = 7.59 [7.31, 7.86] W = 29,038 p = 0.28

Mean T 2021

74 C. Giovannella et al.

4 The Italian School Ecosystems Two Years After the Lockdown …

75

References 1. UNESCO, https://en.unesco.org/themes/education-emergencies/coronavirus-schoolclosures (2020). Accessed Apr 10 2022 2. ETF (European Training Foundation) Coping with COVID-19: Distance digital learning during COVID-19 in ETF partner countries, 6 June 2020. Last accessed February 2021 at: www.etf.eur opa.eu/sites/default/files/2020-06/mapping_covid_ddl_050620_1.pdf.. Accessed Apr 10 2022 3. As an example of the huge amount of papers that have collected, narrated and analyzed educational experiences during the pandemic. In: Sahin, I., Shelly. M. (eds.) Educational Practices During the COVID-19 Viral Outbreak: International Perspectives, ITES Organization (2020) 4. Giovannella C.: Between awareness and acceptance: a more mature school teachers’ perspective on integrated learning one year after the pandemic outbreak. Interaction Design Architect. J. IxD&A, N. 52, 2022, to be published 5. Giovannella, C., Passarelli, M., Persico, D.: The Effects of the Covid-19 Pandemic on Italian Learning Ecosystems: the School Teachers’ Perspective at the steady state, Interaction Design and Architecture(s) Journal - IxD&A, N. 45, pp. 264–286 (2020) 6. Giovannella, C., Passarelli, M.: The effects of the Covid-19 pandemic seen through the lens of the Italian university teachers and the comparison with school teachers’ perspective”, Interaction Design and Architecture(s) Journal - IxD&A, 46, 264–286 (2020) 7. Giovannella C., A year after the outbreak of COVID-19: how has evolved the students’ perception evolved concerning the on-line learning?” in Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education. Smart Innovation, Systems and Technologies, vol 249. Springer, 2022, pp. 105–121 8. Giovannella C., Dascalu M., Dodero G., Mealha O., Rehm M., The Year of Living Dangerously, Interaction Design and Architecture(s) Journal - IxD&A, N. 46, 2020, pp. 5–12 9. Giovannella C., Passarelli M., Alkhafaji A.S.A. , Pérez Negrón A. P., A Model for the Attitude to get Engaged in Technological Innovation (MAETI) derived from a comparative study on the effects of the SARS-CoV2 pandemic seen through the lens of the university teachers of three different national learning ecosystems: Iraq, Italy and Mexico.”, submitted to Interaction Design and Architecture(s) Journal - IxD&A, 47, 2021, pp. 167–190 10. Akar, S.G.M.: Does it matter being innovative: Teachers’ technology acceptance. Educ. Inf. Technol. 24(6), 3415–3432 (2019) 11. Durando, M., Blamire, R., Balanskat, A. & Joyce, A.: E-mature schools in Europe. InsightKnowledge building andexchange on ICT policy and practice (2007). http://www.eun.org/doc uments/411753/817341/emature_schools_in_europe_final.pdf/e5d4b90e-24ff-454f-9722-167 b402ce7f4 Accessed 2022/04/10 12. Underwood, J., Baguley, T., Banyard, P., Dillon, G., Farrington-Flint, L., Hayes, M., Selwood, I.: Understanding the impact of technology: Learner and school level factors (2010).http://dera. ioe.ac.uk/1434/1/becta_2010_understandingimpacttechnology_report. Accessed 2022/04/10 13. Sergis, S., Zervas, P., Sampson, D.G.: A Holistic approach for managing school ICT competence profiles towards supporting school ICT uptake. Int. J. Digital LiteracyDigital Competence 5(4), 33–46 (2014) 14. Kampylis, P., Punie, Y., Devine, J.: Promoting Effective Digital-Age Learning: A European Framework for Digitally-Competent Educational Organisations, EUR 27599 EN. Publications Office of the European Union, Luxembourg (2015) 15. https://education.ec.europa.eu/selfie 16. https://publications.jrc.ec.europa.eu/repository/search?query=selfie 17. Giovannella, C.: Smartness as complex emergent property of a process: the case of learning eco-systems. ICWOAL 2014, IEEE Publisher, pp. 1–5 (2014). 18. Giovannella, C.: The ASLERD Pyramid of Smartness: A Study on the Stability of Indices and Indicators in Schools in Project and Design Literacy as Cornerstones of Smart Education, Springer, pp. 81–91 (2021) 19. Hassenzahl, M.: Experience design: technology for all the right reasons. Synthesis lectures on human-centered informatics 3(1), 1–95 (2010)

76

C. Giovannella et al.

20. Giovannella, C.: Is complexity tameable ? Toward a design for the experience in a complex world. Interaction Design and architecture(s) IxD&A J. N. 15, pp. 18–30 (2012) 21. Calvo, R.A., Peters, D.: Positive Computing: Technology for Wellbeing and Human Potential. The MIT Press, Cambridge, Massachusetts (2014) 22. Desmet, P.M., Pohlmeyer, A.E.: Positive design: an introduction to design for subjective wellbeing. Int. J. Design 7(3): 5 (2013). https://repository.tudelft.nl/islandora/object/uuid%3A06ec 60ac-0363-43ea-9ccd-8426ef0d6b64 23. Giovannella, C.: From Simplex to Complex: Design for Wellbeing at Scales. https://doi.org/ 10.13140/RG.2.2.30457.13922 24. Czisikszentmihalyi, M.: Flow - The Psychology of Optimal Experience, Harper & Row (1990) 25. Lev Vygotsky Adolescent Pedagogy The Development of Thinking and Concept Formation in Adolescence (1931). https://www.marxists.org/archive/vygotsky/works/1931/adoles cent/ch10.htm. Accessed Apr 10 2022 26. https://docs.google.com/forms/d/e/1FAIpQLSd0K7MFQsP2GL0JhF4H668H1NbswOD2C S2ZzkaMg7sdrASXJg/viewform 27. https://www.anp.it/ 28. http://www.aslerd.org/ 29. http://www.piolatorre.it/rubrica/read-art.asp?id=2609; https://www.truenumbers.it/eta-degliinsegnanti/ 30. https://www.orizzontescuola.it/avvio-anno-scolastico-suddivisione-dei-docenti-regione-e-liv ello-istruzione/ 31. https://www.orizzontescuola.it/dirigenti-scolastici-maggioranza-donne-eta-media-55-annidal-concorso-esclusi-precari/ 32. Hayduk, L., Cummings, G., Stratkotter, R., Nimmo, M., Grygoryev, K., Dosman, D., Boadu, K.: Pearl’s D-separation: one more step into causal thinking. Struct. Equat. Model. 10(2): 289–311 (2003) 33. Kalisch, M., Mächler, M., Colombo, D., Maathuis, M.H., Bühlmann, P.: Causal inference using graphical models with the R package pcalg. J. Statist. Software 47(11): 1–26 (2012). http:// www.jstatsoft.org/v47/i11/ 34. Davis, F.D.: A Technology Acceptance Model for empirically testing new end-user information systems: theory and results. Massachusetts Institute of Technology (1985)

Chapter 5

Automated Paragraph Detection Using Cohesion Network Analysis Robert-Mihai Botarleanu, Mihai Dascalu, Scott Andrew Crossley, and Danielle S. McNamara

Abstract The ability to express yourself concisely and coherently is a crucial skill, both for academic purposes and professional careers. An important aspect to consider in writing is an adequate segmentation of ideas, which in turn requires a proper understanding of where to place paragraph breaks. However, these decisions are often performed intuitively, with little systematicity in sequencing ideas. Thus, an automated method of detecting the optimal hierarchical structure of texts using quantifiable features could be a valuable tool for learners. Here, we aim to define a framework grounded in Cohesion Network Analysis to establish the structure of a text by modeling paragraphs as clusters of sentences. The analogy to clustering enables us to identify paragraph breaks that maximize inter-paragraph separation while ensuring high intra-paragraph cohesion. Our approach consists of two steps acted on texts without paragraph breaks. First, the number of paragraphs is automatically inferred with an absolute error of 1.02 using a Recurrent Neural Network, which relies on text features and cohesion flow. Second, paragraph splits are detected using two algorithms: top k which selects the largest cohesion gaps between adjacent utterances, and divisive clustering which iteratively splits the text into paragraphs. Silhouette scores are used to assess performance and the obtained values denote adequately inferred structures. R.-M. Botarleanu · M. Dascalu (B) University Politehnica of Bucharest, 313 Splaiul Independentei, 060042 Bucharest, Romania e-mail: [email protected] R.-M. Botarleanu e-mail: [email protected] M. Dascalu Academy of Romanian Scientists, Str. Ilfov, Nr. 3, 050044, Bucharest, Romania S. Andrew Crossley Georgia State University, Department of Applied Linguistics/ESL, Atlanta, GA, USA e-mail: [email protected] D. S. McNamara Arizona State University, Department of Psychology, PO Box 871104, Tempe, AZ, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_5

77

78

R.-M. Botarleanu et al.

5.1 Introduction Learning to write is an important aspect of education and a useful skill across many circumstances. An important aspect of writing is text structure, which needs to be taken into account to convey content and facilitate understanding. Stark [1] found that paragraphs are discourse units that affect what ideas are considered to be important. As such, paragraph breaks represent a central delimitator of ideas and impact the structure and the coherence of a text. However, the task of identifying where to place paragraph breaks in a sequence of sentences without them is not trivial. This challenge is mainly because paragraph composition relates to the flow and sequencing of ideas, in tight relation to text cohesion, both at local (i.e., in-between sentences) and global (i.e., among paragraphs) levels. In addition, writers have a personal style and may place paragraphs differently. For instance, they may group more sentences to maximize the content of each paragraph, or they may prefer to use a more fractionated structure with more fine-grained, individualized ideas per paragraph. In more extreme cases, students may have single paragraphs with many sentences or a large number of very short paragraphs, with only one sentence each. Our proposed system analyzes the quality of a text’s paragraph structure and enables automated feedback integrated into a smart learning environment, to provide instruction to students on how to better organize sentences into cohesive paragraphs. The model can be considered as groundwork for analyzing texts on a topological level, to guide learners towards improving their writing, in particular text structure and organization, by providing easily understandable, explicit feedback. In this paper, we present a two-stage approach to the issue of paragraph identification. First, we detect the optimal number of paragraphs through a Recurrent Neural Network. Second, we identify the optimal structure of paragraphs in the text based on text cohesion, to maximize inter-paragraph separation, while ensuring high intraparagraph cohesion. Thus, our research question is the following: to what extent is our automated model capable to predict the optimal number of paragraphs in a text, as well as how adequate is the proposed topology of a text derived from the proposed paragraph segmentation?

5.2 Related Work Paragraph detection is a sub-task of the wider concept of paragraph segmentation. It is a separate topic from the detection of already existing paragraphs, such as those that can be found in PDF files [2]. The task of detecting the optimal paragraph structure of a text in the current study falls under Automated Writing Evaluation (AWE) [3] because our tool suggests paragraph break positions in student writings in order to maximize the readability of their work while structuring and compartmentalizing sentences into distinct, cohesive groups. In contrast to Automated Essay Scoring (AES), AWE systems provide targeted feedback to users to help them improve their

5 Automated Paragraph Detection Using Cohesion Network Analysis

79

texts. The problem of identifying the optimal paragraphs is either modeled as: (a) a text with all paragraphs removed in which boundaries need to be established or (b) a sequence of sentences for which a paragraph break exists, and its optimal location needs to be identified. The problem of detecting paragraph boundaries was proposed by Genzel [4] for use in downstream tasks, such as grammar checking or as a restoration step after OCR processing. The arbitrary nature of paragraph boundary placement is noted. In this work, paragraph detection is modeled as a classifying task that establishes whether a sentence is paragraph-starting or not. The model used is a sparse voted perceptron, with an emphasis being placed on the features the model learns to use, which may be indicative of what constitutes a paragraph-starting sentence for humans. Various statistical features were used, such as the cosine similarities between sentences or their first words, parse tree statistics, part-of-speech statistics, surface statistics (e.g., length of sentences), centering types, and others. Accuracies between 64 and 82% were reported for a model trained on “War and Peace” by Leo Tolstoy and evaluated on other texts. The main issue with this approach is that there is a significant imbalance between paragraph-starting and non-paragraph-starting sentences, which means that: (a) the model can outperform a baseline that always predicts the majority class by 17 percentage points for some texts, while (b) the model is 6 percentage points below the oracle baseline for other texts. Another example of paragraph detection can be found in Sporleder and Lapata [5], who introduced a state-of-the-art system for marking paragraphs by proposing methods of paragraph detection using hand-crafted features. They achieved a performance that is within 6% of the human baseline, with an accuracy of 82.91% on their corpus. The authors used fiction texts, news, and parliamentary transcripts in three languages (English, Greek, and German) and included the following features: • Sentence signatures composed of part-of-speech tags; • Parse tree complexity considering the distribution of parts of speech and other geometrical features, such as the branching factor and depth; • Statistical features such as the distance between the current sentence and the previous paragraph break, the length of the sentence, and the relative position in the text; • Other heuristics such as the presence of quotation and punctuation marks; • Word features, for the first three words of a sentence and the aggregated words of a sentence. However, their method of modeling the task of paragraph detection as a classification problem has several drawbacks, similar to Genzel [4]. First, the number of paragraphs in a text is significantly lower than the number of sentences. This means that, given any succession of sentences, there is a bias towards the sentences belonging to the same paragraph which produces an imbalanced classification dataset. As such, measuring accuracy may be misleading. Genzel [4] also reported the results of an oracle that only predicts the majority class. Second, the introduced features do not take into account the overall structure of the text and consider only surface indicators of sentence complexity and length to split paragraphs. Because the primary purpose of

80

R.-M. Botarleanu et al.

a paragraph is to structurally group cohesive blocks of sentences, we hypothesized that a graph-like cohesion-centered structure of a text would capture the properties of paragraphs more effectively. The purpose of our work is to detect adequate locations in a text where a writer should consider placing paragraph breaks that maximize two properties: intraparagraph cohesion and decoupling between paragraphs. As such, we propose an unsupervised approach that takes into account the structural information extracted from a text. We assume that sentences in the same paragraph should have a higher cohesion with each other, as compared to sentences in differing paragraphs. This is similar to how points within the same clusters should be closer to each other than they are to points from other clusters. Therefore, we model a text in a clustering space, where sentences represent nodes, paragraphs are clusters, and text cohesion is used to measure the relatedness of nodes. In other words, we analyze how a document is split into paragraphs by measuring whether the clusters (i.e., the paragraphs) form distinct sequences of points (i.e., sentences) that are separable. With this in mind, a text can be viewed as having a good paragraph structure if each paragraph is highly cohesive and, simultaneously, if there exists a clear separation between one paragraph and another. Our method is grounded in Cohesion Network Analysis [6] which considers text structure based on semantic links established between different constituents (i.e., sentences, paragraphs, and the entire document). Our approach generates new paragraph structures for texts by maximizing the two previously mentioned properties (i.e., intra-paragraph cohesion and inter-paragraph decoupling). We propose a twostep approach. First, we extract various information from the text to predict the optimal number of paragraphs. Second, we develop two algorithms to detect the optimal configuration of the sentences into paragraphs. We compare the structural clustering metric scores from our two models to those measured in the original, human-paragraphed texts.

5.3 Method 5.3.1 Corpus We used a combination of documents from the TASA (Touchstone Applied Science Associates, Inc.) corpus (http://lsa.colorado.edu/spaces.html) and essays gathered using the Writing Pal intelligent tutoring system (ITS) [7] in various experiments. The TASA corpus was selected because the initial texts were split into different selfdefined short documents (1–7 paragraphs) by experts, whereas the human essays represent a more relaxed and free-form structure, which varies greatly due to a large number of writers. From these, only texts that have at least 3 and at most 7 paragraphs were selected, resulting in a dataset containing 4,704 filtered texts of which 3,893

5 Automated Paragraph Detection Using Cohesion Network Analysis

81

Fig. 5.1 Histogram of paragraph counts for the paragraph detection corpus

are taken from the TASA corpus and 811 are essays written by students (see Fig. 5.1 for the histogram with the distribution of paragraph length).

5.3.2 Predicting the Number of Paragraphs First, we built a model to predict the number of paragraphs in an unstructured text. A list of features from each text is generated to train a regressor. These features correspond both to surface-level statistics, as well as features derived from the cohesion between sentences, computed using cosine similarity on sentence embeddings. The features extracted included: • Counts of words, sentences, different word lemmas, and stopwords; • Distributions of part-of-speech tags using the Penn Treebank Part-of-speech tagger and determining the frequency of each tag per document; • Counts of specific connectors (e.g., cause and effect, comparison, emphasis, etc.); in total, 88 connectors are used, representing common sentence and paragraph boundaries taken from various word lists; • A measure of cohesion flow operationalized as a vector with cohesion scores between two consecutive sentences; we use the cosine similarity between two sentences to measure their semantic distance and, through the sequence of such distances in the text, the cohesiveness of the entire text;s • Various statistical measures were applied to the cohesion flow: mean, standard deviation, kurtosis, skewness, 0.1/0.25/0.5/0.75/0.9 quantiles and 25/50/75 percentile values. Two types of models are considered while assessing the flow of cohesion, namely: (a) a pre-trained word2vec [8] model using the Google News word embeddings, and (b) a word2vec model trained on the Corpus of Contemporary American English— COCA [9] dataset.

82

R.-M. Botarleanu et al.

Fig. 5.2 Architecture for the paragraph counting model

The aggregation of individual word embeddings into phrase-level vector representations is achieved either through an unweighted mean of word vectors or through the Smooth Inverse Frequency (SIF) [10] which assigns a larger weight to less common words: vs =

a 1  vw |s| w∈s a + p(w)

(5.1)

where v describes an embedding vector, s is a sentence, w is a word in s, p is the probability of seeing w in s, and vw is the vector representation of w. The regressor used for the task of predicting the number of paragraphs in a text (see Fig. 5.2) is a neural network composed of a hidden linear layer with 64 neurons, followed by a Rectified Linear Unit activation [11] for the statistical input features, and a single LSTM [12] layer with 32 units that receives the cohesion flow vector as input. The results of these two layers are concatenated, and the final prediction is given through an output layer with a single neuron. The intuition behind the architecture introduced in Fig. 5.2 is that the fully connected layer should generate new representations of the statistical features, while the LSTM should learn to understand that cohesion gaps are correlated with the number of paragraphs. The model is trained using the Adam optimizer [13] for 10 epochs and optimizes the mean squared error. A cosine annealing learning rate scheduler [14] is used, with the learning rate going from a maximum of 0.01 to a minimum of 0.001. Additionally, we perform a z-score normalization of the target variable and report the results after denormalization.

5 Automated Paragraph Detection Using Cohesion Network Analysis

83

5.3.3 Identifying Paragraph Breaks Once the number of paragraphs has been estimated, we can find the optimal paragraph structure of the text. Two algorithms—top k and divisive clustering—were tested to detect where paragraphs should be placed in an arbitrary set of sentences. Each algorithm assumes that the number of paragraphs (P) is given as input. Both algorithms preserve the original order of the sentences in the text. The top k algorithm detects the highest P cohesion breaks and marks them as paragraph splits. A cohesion break is defined as a succession of two adjacent sentences for which the cohesion between them is low. The divisive clustering considers the following: a. Consider all sentences to belong to one cluster. b. For (P – 1) repetitions: (1) Find the sentence that is most dissimilar (on average) to its cluster. (2) Split the paragraph such that the sentence found previously and all sentences following it in the cluster are assigned to a new paragraph.

5.3.4 Evaluation Using the analogy that representations of sentences occur in a clustering space wherein the paragraphs are clusters, we adapt the Silhouette score [15] to assess the performance of our two algorithms. The Silhouette score is a frequently employed clustering metric used to evaluate the extent to which an algorithm maximizes intracluster similarity and minimizes inter-cluster similarity. We expect good paragraphs to lead to high Silhouette scores since sentences inside such a paragraph are likely to be more connected than those belonging to different paragraphs. Our adapted Silhouette score considers the following steps: 1. Compute the average distance between each point i and all other points in the clustering space (C j denotes sentences from cluster/paragraph C j ): a(i ) =

1 |C i | − 1



(1 − cohesion(i, j ))

(5.2)

j∈Ci ,i= j

2. For each point i, compute the distance to the nearest neighbor in another cluster: b(i ) = min k=i

1  (1 − cohesion(i, j )) |C k | j∈C

(5.3)

k

3. Compute the Silhouette score for each point i as: s(i ) =

b(i ) − a(i ) max(a(i ), b(i))

(5.4)

84

R.-M. Botarleanu et al.

4. Finally, the document silhouette score is the average of the individual silhouette scores for all sentences (D denotes the entire document): S=

1  s(i ) |D| i∈D

(5.5)

As such, the Silhouette score provides higher values to texts where sentences are well placed inside their paragraphs.

5.4 Results Our corpus was split into a training and a test set with a ratio of 4:1 for each sub-task. The first component consists of a regressor that predicts the number of paragraphs in a text. Results from Table 5.1 indicate that neither the word2vec corpus nor the aggregation method (i.e., unweighted mean of the word vectors versus Smooth Inverse Frequency) affected the performance of the four models. The mean absolute error measures the average difference between the actual and the predicted number of paragraphs. An MAE of 1 reflects a difference of 1 paragraph in the proposed text structure, making our model usable in practice. Table 5.2 provides the Silhouette scores measured using all variations of aggregation methods, word2vec models, and the two proposed algorithms (i.e., top k and divisive clustering). We considered P to be the predicted number of paragraphs resulting from the previous sub-task, as well as the actual number of paragraphs. The later assessment using the actual number of paragraphs from the reference tests was performed to evaluate this sub-task individually, without errors induced by the previous prediction component. Results indicate that the use of the word2vec embeddings trained on the Google News corpus led to better performance in detecting optimal paragraphs. This is consistent with the idea that word embeddings trained on larger corpora are better suited for measuring the semantic relatedness between sentences because they are more likely to capture the relations between words. In addition, divisive clustering outperforms the simple “top k” algorithm, showing that a top-down approach is better suited for this task. Results also indicate there is no noticeable difference between using an unweighted average of the word vectors of a Table 5.1 Mean Absolute Error for the paragraph counting regressor as a function of aggregation method and word2vec corpus

Aggregation method word2vec corpus Mean Absolute Error (MAE) Mean

Google news

1.03

SIF

Google news

1.02

Mean

COCA

1.03

SIF

COCA

1.02

5 Automated Paragraph Detection Using Cohesion Network Analysis

85

Table 5.2 Silhouette scores for the paragraph detection test set Aggregation Method

word2vec Corpus

Paragraph Detection Algorithm

Silhouette score using Silhouette score the predicted # of using the actual # of paragraphs paragraphs

Mean

Google news

Top k

0.22

0.19

SIF

Google news

Top k

0.22

0.19

Mean

COCA

Top k

0.19

0.15

SIF

COCA

Top k

0.19

0.16

Mean

Google news

Divisive clustering

0.23

0.20

SIF

Google news

Divisive clustering

0.23

0.20

Mean

COCA

Divisive clustering

0.20

0.17

SIF

COCA

Divisive clustering

0.21

0.17

sentence and using a weighted average where the weights are given by the frequency of a word’s apparition (i.e., using SIF). An additional experiment was performed that relied only on the Silhouette score as the selection criterion to determine the optimum number of paragraphs. This was achieved by running the paragraph detection algorithms on all paragraph counts from 3 to 7, followed by the evaluation of corresponding Silhouette scores. This experiment provided insights into whether the Silhouette score is a metric correlated to the number of paragraphs in the text and whether the paragraph count regressor is necessary for the algorithm to detect the correct number of paragraphs (see Table 5.3). The mean average errors obtained in this experiment are approximately twice as high as those measured using the regressor (see MAE values in Table 5.1). This indicates that having a separate model to estimate the correct number of paragraphs is, indeed, useful as cohesion is a key constituent, but not sufficient by itself, to accurately predict how many paragraphs a text should contain.

5.5 Discussion This study introduces an automated method for detecting the optimal hierarchical structure of texts using quantifiable features. The framework is grounded in Cohesion Network Analysis and models paragraphs as clusters of sentences, allowing us to identify paragraph breaks that maximize inter-paragraph separation while ensuring high intra-paragraph cohesion. Figure 5.3 describes how the paragraphs in the generated texts are structured as a function of the number of sentences. In the original texts,

86 Table 5.3 Mean Absolute Error using the Silhouette score as an indicator of the number of paragraphs

R.-M. Botarleanu et al. Aggregation Method

word2vec Corpus

Paragraph Detection Algorithm

Mean Absolute Error (MAE)

Mean

Google news

Top k

2.52

SIF

Google news

Top k

2.53

Mean

COCA

Top k

2.45

SIF

COCA

Top k

2.48

Mean

Google news

Divisive clustering

2.51

SIF

Google news

Divisive clustering

2.53

Mean

COCA

Divisive clustering

2.39

SIF

COCA

Divisive clustering

2.42

Fig. 5.3 Distribution plot showing the number of sentences in each a original paragraph and b generated paragraph.

the average number of sentences per paragraph was 4.21, with the 50th percentile being at 4 sentences per paragraph. The generated paragraphs have an average of 7.4 sentences, with the 50th percentile being at 6 sentences per paragraph. Thus, our system appears to generate longer paragraphs than those found in the original texts, which would suggest that higher Silhouette scores result from more compacted blocks of sentences.

5 Automated Paragraph Detection Using Cohesion Network Analysis

87

To better understand how paragraph breaks are marked, we sample from each of the three possible cases in which the splits into paragraphs are different than the initial ones (see Table 5.4). In the first example, an intermediate paragraph is generated from the second and third source text paragraphs. While the beginning of this supplementary paragraph is not ideal since it starts with “Instead of like a pack of wolves” which requires the previous sentence for continuity, the final paragraph in the altered text more clearly delimits the essay’s conclusions. In the second example where an equal number of paragraphs is used, the generated text in our approach elects to start the second and third paragraphs with connectors such as “Although” and “For example”. In the final example where only one paragraph is generated using all three paragraphs from the source text, our system detects that the three paragraphs are highly interconnected and can be more fluently expressed as a single paragraph. Overall, we find that our method generates more paragraphs than in the original text; the system generates an equal number of paragraphs to the original text, but at different places; and the system generates a greater number of paragraphs than in the original text. Given that the paragraph count regressor has an MAE of 1.24, deviations of more than one paragraph are rare. The results indicate that the introduced paragraph Table 5.4 Samples of generated paragraphs Actual paragraphs

Generated paragraphs

More generated paragraphs I believe that any kind of material such as books, movies, magazines, etc., should be censored except for mature content such as pornography, violent and horrid movies and video games. But that’s why we have ORGANIZATION1’s made. So when asked, we can show our age. Some age groups react differently to mature contents. Usually NUM1 and up is a tolerable level for maturity to set in Maturity is maintained at different paces in all age groups. For example, perhaps a NUM2 year old is more mature than an NUM3 year old. It is when you as a individual can act like a human being. Instead of like a pack of rabid wolves. Just because you do not act mature dosen’CAPS2 mean you don’t know how to be civilized The best way to prevent a child from discover something inappropriate or vulgar is to, separate the books by an age grouping system. Keep adult videos out of site and reach of children. Also put safety locks on your CAPS2.V. to prevent kids from sneaking behind your back

I believe that any kind of material such as books, movies, magazines, etc., should be censored except for mature content such as pornography, violent and horrid movies and video games. But that’s why we have ORGANIZATION1’s made. So when asked, we can show our age. Some age groups react differently to mature contents. Usually NUM1 and up is a tolerable level for maturity to set in Maturity is maintained at different paces in all age groups. For example, perhaps a NUM2 year old is more mature than an NUM3 year old. It is when you as a individual can act like a human being Instead of like a pack of rabid wolves. Just because you do not act mature dosen’CAPS2 mean you don’t know how to be civilized. The best way to prevent a child from discover something inappropriate or vulgar is to, separate the books by an age grouping system Keep adult videos out of site and reach of children. Also put safety locks on your CAPS2.V. to prevent kids from sneaking behind your back

Equal number of paragraphs (continued)

88

R.-M. Botarleanu et al.

Table 5.4 (continued) Actual paragraphs

Generated paragraphs

There are some ways to be original but most have already been done. Some will use others originals and maybe perfect it. Although we value uniqueness and originality, also we like to see others enhance someone else idea to make it better There is a ongoing debate about can people be truly original. I think in some ways this is true because people can still have ideas that no one has thought of before. Although, people use others ideas they can make them better in the future or make them look like they are themselves original. For example, people buy clothing from the store and the clothing is a designer’s creation. People take that and put it with other designer clothing and make their own original style. I think that is kind of original, because its their own way of wearing the clothing they buy So although people may use others design or something similar to it. It can still be original it all depends on how people use it in a different way

There are some ways to be original but most have already been done. Some will use others originals and maybe perfect it Although we value uniqueness and originality, also we like to see others enhance someone else idea to make it better. There is a ongoing debate about can people be truly original. I think in some ways this is true because people can still have ideas that no one has thought of before. Although, people use others ideas they can make them better in the future or make them look like they are themselves original For example, people buy clothing from the store and the clothing is a designer’s creation. People take that and put it with other designer clothing and make their own original style. I think that is kind of original, because its their own way of wearing the clothing they buy. So although people may use others design or something similar to it. It can still be original it all depends on how people use it in a different way

Fewer generated paragraphs I do think that there should be a censorship in not just in libraries, but everywhere Personally, I think that the way that the libraries have the books are appropriate and if the parents do not want their children going anywhere that is not privy to them keep a hand length away As for the parents, the parents know the areas that interest them, therefore the parents should go there

I do think that there should be a censorship in not just in libraries, but everywhere. Personally, I think that the way that the libraries have the books are appropriate and if the parents do not want their children going anywhere that is not privy to them keep a hand length away. As for the parents, the parents know the areas that interest them, therefor the parents should go there

breaks make quite good sense as the system splits paragraphs based on the similarity between ideas. For example, the new collided version of the sentences from the last sample into one paragraph seems more adequate and cohesive, especially in contrast to the highly segmented initial version which had two paragraphs with only one sentence each.

5.6 Conclusions A novel automated approach was introduced to analyze how texts are structured into paragraphs by considering cohesive clusters of sentences. Cohesion Network

5 Automated Paragraph Detection Using Cohesion Network Analysis

89

Analysis was used to capture text structure and maximize the cohesion between sentences in a paragraph while keeping distinct paragraphs separate. The marking of paragraphs is performed as a two-stage process. First, a regressor using a Recurrent Neural Network is trained to estimate the number of paragraphs in a text; this sub-task achieves a mean absolute error of 1.02, which indicates that we can predict the number of paragraphs in our combined corpus with an average error of around 1 paragraph. Second, the Silhouette clustering score, in conjunction with a paragraph splitting algorithm, is used to determine an optimal paragraph structure for the predicted number of paragraphs. Our primary contribution is that our proposed clustering approach is unsupervised and independent of the quality of the human paragraph segmentations within a corpus. Our method does not require training and it offers an explainable and intuitive approach to find the optimal paragraph structure of a text. Moreover, our method provides a quantitative measure of a text’ paragraph topology. We demonstrate that our method works best in a two-stage approach, wherein we first determine the number of paragraphs that should exist in the given text, and then select the appropriate sentence distribution for those paragraphs. Our experiments have shown that attempting to simultaneously determine (a) the number of paragraphs and (b) what sentences belong to them, leads to inferior quantitative outcomes. Our system can generate paragraphs similar to the ones chosen by a human and appears to select coherent breakpoints. An interesting observation is that our algorithm tends to merge very short consecutive paragraphs. Moreover, our method of modeling paragraphs as a clustering task can provide insights into how well a written work is structured. In an applied setting, learners might receive suggestions on an alternative paragraph structure that is topologically more cohesive and potentially separable in terms of ideas. The proposed framework also opens the possibility of further, more complex, use cases. For now, the order of sentences is maintained in the task of paragraph selection. However, the same technique could be applied to determine whether a sentence is well suited in its paragraph, or whether a portion of a text is ill-placed and should be removed or placed within a different paragraph. Thus, follow-up extensions envision the identification of out of context sentences and the generation of potential suggestions for sentence reordering. Ultimately, our objective is to guide learners toward improving their writing through automated feedback on how to better and more clearly express their ideas. The feedback introduces key positions where to introduce paragraph breaks to enhance the overall organization of the text. Beyond providing feedback to learners, our method is also a first and essential step towards improving text structure automatically, by accounting for the maximization of text cohesion, both globally at the paragraph level, and locally, between sentences within paragraphs. Overall, the research reported here represents an initial step in developing an algorithm that can be used to guide students as to how to better organize their writings. However, we have not explored the usage of such a system in an educational setting within the context of this paper, with such an experiment representing an important future step for the work detailed so far.

90

R.-M. Botarleanu et al.

Acknowledgements This research was supported by a grant from the Romanian National Authority for Scientific Research and Innovation, CNCS—UEFISCDI, project number TE 70 PN-III-P1-1.1TE-2019-2209, ATES— “Automated Text Evaluation and Simplification”, the Institute of Education Sciences (R305A180144 and R305A180261), and the Office of Naval Research (N00014-17-12300; N00014-20-1-2623). The opinions expressed are those of the authors and do not represent the views of the IES or ONR.

References 1. Stark, H.A.: What do paragraph markings do? Discourse Process. 11(3), 275–303 (1988) 2. Darvishy, A., Nevill, M., Hutter, H.-P.: Automatic paragraph detection for accessible PDF documents. In: International Conference on Computers Helping People with Special Needs, pp. 367–372. Springer, Linz, Austria (2016) 3. Roscoe, R.D., Varner, L.K., Crossley, S.A., McNamara, D.S.: Developing pedagogicallyguided algorithms for intelligent writing feedback. Int. J. Learn. Technol. 25, 8(4), 362–381 (2013) 4. Genzel, D.: A paragraph boundary detection system. In: International Conference on Intelligent Text Processing and Computational Linguistics, pp. 816–826. Springer, Berlin, Heidelberg (2005) 5. Sporleder, C., Lapata, M.: Automatic paragraph identification: a study across languages and domains. In: International Conference on Empirical Methods in Natural Language Processing, pp. 72–79. ACL, Barcelona, Spain (2004) 6. Dascalu, M., McNamara, D.S., Trausan-Matu, S., Allen, L.K.: Cohesion Network Analysis of CSCL Participation. Behav. Res. Methods 50(2), 604–619 (2018) 7. Roscoe, R.D., Varner, L.K., Weston, J.L., Crossley, S.A., McNamara, D.S.: The Writing Pal intelligent tutoring system: Usability testing and development. Comput. Compos. 34, 39–59 (2014) 8. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representation in vector space. In: Workshop at ICLR, Scottsdale, AZ (2013) 9. Davies, M.: The 385+ million word Corpus of Contemporary American English (1990–2008+): Design, architecture, and linguistic insights. International journal of corpus linguistics 14(2), 159–190 (2009) 10. Arora, S., Liang, Y., Ma, T.: A simple but tough-to-beat baseline for sentence embeddings. In: 5th International Conference on Learning Representations (ICLR 2017), Toulon, France (2017) 11. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010) 12. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997) 13. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (2014) 14. Loshchilov, I., Hutter, F.: SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv: 1608.03983 (2016) 15. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)

Chapter 6

Assessing Readability Formulas in the Wild Scott Crossley , Stephen Skalicky , Cynthia Berger , and Ali Heidari

Abstract Recent advances have facilitated major improvements in developing intelligent and purpose-oriented readability formulas to predict the overall difficulty of a text in terms of text comprehension and processing. Such readability formulas are mediating technologies that help match appropriate reading texts with students, thus enabling the development of smart learning environments that adapt learning resources to learner skills. Newer readability formulas include linguistic features that are more predictive of human judgments of text readability than traditional readability formulas, such as Flesch-Kincaid Grade Level. However, in many cases, these formulas have not been tested beyond their ability to predict reading scores. The purpose of this study is to examine the validity of newer readability models along with more traditional readability formulas using behavioral data and text comprehension scores. The results indicate that readability models employing linguistic features more theoretically related to text processing and comprehension outperform readability models that do not employ similar features. The findings support the long-term growth of readability formulas that are continuously improved to increase the wellbeing of learners.

6.1 Introduction In the United States, students perform below average on standardized reading tests regardless of grade level [1]. This performance minimizes opportunities for future student success and lowers wellbeing within communities ranging from the social to S. Crossley (B) · A. Heidari Georgia State University, Atlanta, Georgia e-mail: [email protected] S. Skalicky Victoria University of Wellington, Wellington, USA C. Berger Duolingo, Pittsburgh, PA, USA © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_6

91

92

S. Crossley et al.

the academic [2]. The primary reason for low reading success rates is the inherent difficulty of developing reading skills [3]. This difficulty can be mediated using a number of strategies to help learners develop stronger reading skills [4]. Chief among these is the careful selection of texts to ensure that readers have challenging texts that are comprehensible [5]. A common technique to match readers with appropriate texts has been the use of readability formulas meant to access text difficulty, which in return ensures a well-balanced and personalized experience for learners. Since the 1940s, over 200 readability models have been created, indicating a long-term vision to help students succeed through better text matching [6]. Traditional readability formulas developed through the 1940s to the 1980s like the Flesch-Kincaid Grade Level [7] are based on surface level textual features that can be hand counted. These features include the number of letters (or syllables) per word to approximate lexical sophistication and the number of words in a sentence to approximate syntactic complexity. Newer formulas developed after the advent of desktop computing are more intelligent and purpose-oriented, allowing for automatic assessment of text difficulty using deeper linguistic features often inspired by theories of reading. These features allow newer readability formulas to better assess elements of reading including decoding (i.e., word recognition). Traditional readability formulas, which rely on the number of letters or syllables per word to measure decoding do not tap directly into the linguistic components of readability [8]. Newer formulas, however, can compute the frequency of words,lexical properties of words including concreteness, imageability, and age of acquisition; psycholinguistic norms including word naming and lexical decision times; and phonological neighborhood effects (among many others) that are more strongly related to decoding. Similar examples exist in how newer and traditional formulas calculate syntactic complexity. Chiefly, traditional readability formulas examine sentence length while newer readability formulas measure phrasal and clausal complexity. Additionally, newer formulas calculate features related to text cohesion and semanticity, which are not measured by traditional formulas. Even though traditional readability formulas are not very smart, they have been widely adopted by publishers, researchers, primary and secondary schools, universities, the military, and testing agencies where they are used to select reading materials for a variety of learners [9]. The purpose of this study is to compare newly developed and more intelligent readability formulas to traditional readability formulas using behavioral data (i.e., text processing data) and reading comprehension scores. The goal is to better understand how these formulas perform in comparison to one another and how they perform when co-varied with individual difference variables (e.g., reading skills, reading confidence, number of books read), study design (e.g., order of texts read), and demographic information (e.g., age and gender).

6 Assessing Readability Formulas in the Wild

93

6.2 Method 6.2.1 Participants Sixty undergraduate English-speaking students from a southeastern public research university participated in a reading experiment. The participants were recruited from undergraduate linguistics courses. Demographic information for the participants was collected using a self-reported online survey. Complete data were collected from 54 participants and included in the present study (43 female, 10 male, and one participant who declined to choose). The participants’ average age was 23 (SD = 7.01, min = 18, max = 51). Participants self-identified as either monolingual English speakers or bilinguals (seven participants) who spoke additional languages including German, Spanish, Vietnamese, and Portuguese. All the participants had normal to corrected vision. Since the study was advertised as an ordinary reading study, all the participants were naïve about the purpose of the study. Participants received $20 Amazon gift cards in compensation for participating in the experiment.

6.2.2 Materials Questionnaire. An online questionnaire was used to collect reader literacy and demographic information including reading habits, reading confidence, reading enjoyment, exposure to TV programs, and demographic information including age, gender, knowledge of second languages, and vision quality. Reading Comprehension Scores. Reading comprehension ability of the participants was assessed using the Gates-MacGinitie (4th ed.) reading comprehension test (form S) level 10/12 [1]. The reading comprehension test included 48 multiple-choice questions that measured comprehension of short passages. Each passage was followed by two to six questions which measured reading comprehension competence involved in both surface and deeper level comprehension processes such as inference, text recall, and main ideas. The test involved standard instructions, two practice questions, and the comprehension items. Participants were allowed 25 min to complete the test. Corpus. Twelve texts from a previous study assessing the development of new text readability formulas [11] were used in the experiment. The texts included six texts from Simple English Wikipedia and six texts from regular Wikipedia. Simplified and regular Wikipedia texts were selected to provide variation in the difficulty of the texts. Two simplified and two regular texts were selected from each of three topic domains: history, technology, and science. The average length of the texts was 159 words (SD = 27.6). Three multiple-choice comprehension questions were developed for each text. Each set of comprehension questions contained a question related to making inferences from the content, one related to text recall, and one related to the main idea of the text.

94

S. Crossley et al.

Readability Formulas. We selected three traditional readability measures: Flesch Reading Ease [10], Flesch Kincaid Grade Level [7], and the FOG Index [12] to assess reading speed and comprehension. All traditional readability formulas assess syntactic complexity (through sentence length) and lexical sophistication (through word length). We selected three NLP inspired readability formulas for comparison. First, we selected the New Dale-Chall readability formula [13], which includes both traditional and newer measures of text difficulty. The formula measures syntactic complexity using a traditional approach (sentence length), but it measures lexical sophistication using a list of the 3,000 most frequent words in English. We also selected two newer readability formulas: Crowdsourced Algorithm of REading Comprehension (CAREC) and the Crowdsourced Algorithm of REading Speed [11]. CAREC measures 13 language variables related to lexical sophistication, n-gram features, text cohesion, and sentiment. CARES predicts judgments of reading speed using NLP features related to number and types of words, sophisticated words, syntactic complexity, and variation in paragraph size. Crossley et al. reported that both CAREC and CARES outperformed traditional readability formulas in predicting judgments of reading comprehension and speed. All calculated readability formulas are available in the Automatic Readability Tool for English [4]. ARTE provides free and easy access to a wide range of readability formulas and automatically calculates different readability formulas for batches of texts (i.e., thousands of texts can be run at a time) to produce readability scores for individual texts in an accessible spreadsheet output.

6.2.3 Experimental Design The experiment was designed using Eye-gaze Edge experiment builder software, Nyan 2.0. All texts and reading comprehension questions were typed in doublespaced, Times New Roman font (font size 14) in block text format with landscape text orientation. Nyan 2.0 software was used to randomize texts and reading comprehension questions for each text. In total, the experiment included 13 texts (one practice trial and twelve original texts) and 39 questions (three practice trial questions and thirty-six original text questions).

6.2.4 Procedure Participants were first assigned a unique participant code and then provided informed consent. After consent, they were guided to a computer to complete the online demographic information survey and the Gates-MacGinitie (GMG) reading comprehension test. Participants were then led to another testing booth where they began the reading experiment portion of the study. This portion included a brief introduction

6 Assessing Readability Formulas in the Wild

95

of the experimental procedure to participants. The procedure involved the initial presentation of one practice trial text and three multiple-choice questions to familiarize the participants with the nature of the experiment. After finishing the practice trial, participants read the twelve unique texts in random order and answered the three multiple-choice questions for each text at their own pace (neither text reading nor the multiple-choice comprehension test items were time limited). Questions were presented immediately after each text and participants could not return to the passages once they started answering questions. The participants used the keyboard number pad (keys 1–4) to record their responses to comprehension questions, and then pressed space bar to move forward between passages and corresponding questions. The collection of demographic information, reading comprehension ability, and reading data was done in a single session, which lasted approximately 60 min. Data for seven participants were unusable. In one case, a participant mentioned to the researcher that they had a reading processing disorder, and, in another case, a participant was observed using their cell phone while reading the texts. Additionally, data for three participants were lost due to software errors. Finally, analyses of accuracy revealed one participant to have below 5% total accuracy on the comprehension questions and another that spent less than 10 s reading each text, indicating a lack of engagement with the study task. Thus, the final dataset included data from 53 participants.

6.2.5 Statistical Analysis Because the readability formulas all purport to measure similar constructs, they were assessed for multicollinearity using non-parametric correlations and variance inflation factors (VIF). Multicollinearity was defined as any two variables correlated at a higher absolute value than 0.7 or with a higher VIF value than 2.5 in the context of other predictors. In order to systematically compare the effects of the different measures used in this study, all numerical predictors were standardized into z-scores. Additionally, we did not include text domain or simplification level as categorical factors in our models because there were only 12 texts in total, and this small number of texts resulted in multicollinearity between the readability formulas and levels of these variables, making it difficult to associate variance with the categorical label or differences in the readability formulas predictors. We used Linear Mixed Effects (LME) models to test which effects exerted significant influences on reading times and comprehension scores. We built our models in R using the lme4 package [1]. For the reading time model, we entered reading time (in seconds) as the dependent variable. We then entered the following predictor variables as fixed effects: presentation order of the texts (to control for reading fatigue over time and labeled as trial order), participants’ scores from the GMG reading comprehension test, age, English L1 status (yes/no), and participant survey responses for reading and television behavior, and a single readability formula (e.g., CAREC, Dale-Chall,

96

S. Crossley et al.

or Flesch-Kincaid Grade Level). Subjects were entered as a random effect. We then hand-pruned the models to keep only those predictors that were significant. After hand-pruning, we also checked for evidence of suppression effects as manifested through mismatched correlation and regression coefficients. We followed the same general procedure to test participant comprehension accuracy using overall comprehension scores for the three questions in each text. We used the MuMIn package in order to obtain the marginal and conditional R2 values for our models [14].

6.3 Results 6.3.1 Correlations We conducted correlations between dependent variables (comprehension and reading speed) and our readability formulas and individual difference measures. Initial correlation analyses showed that Flesch Reading Ease, FOG, and Flesch-Kincaid Grade Level were highly multicollinear. We thus removed Flesch Reading Ease and FOG from the subsequent analyses and kept Flesch-Kincaid Grade Level because it showed the highest correlation with comprehension scores and reading times. Correlations between readability formulas and comprehension scores indicated weak correlations for CAREC, Flesch-Kincaid, and Dale-Chall. Correlations between readability formulas and reading speed indicated a weak correlation for CARES (see Table 6.1). Correlations between individual differences and comprehension scores indicated a weak correlation with GMG. Correlations between individual differences and reading speed indicated weak to moderate correlations with education background, amount of time reading, enjoyment of reading, confidence in reading, and GMG (see Table 6.2). Table 6.1 Correlations between comprehension scores/reading times and readability formulas Variables

1

2

3

4

5

6

1. Comprehension

1

−0.04

0.043

−0.167

−0.135

−0.242

2. Reading speed

−0.04

1

0.236

−0.087

−0.045

−0.067

3. CARES

0.043

0.236

1

−0.252

−0.354

−0.5

4. CAREC

−0.167

−0.087

−0.252

1

0.297

0.659

5. Flesch Kineaid

−0.135

−0.045

−0.354

0.297

1

0.503

6. Dale Chall

−0.242

−0.067

−0.5

0.659

0.503

1

6 Assessing Readability Formulas in the Wild

97

Table 6.2 Correlations between comprehension scores/reading times and individual differences Variables

1

2

3

1. Comprehension

1

−0.04

−0.005 −0.008 −0.006 −0.005 0.049

4

5

6

7

8

1

−0.112 −0.138 0.087

0.192

2. Reading speed

−0.04

−0.29

−0.221 −0.328

3. Education

−0.005 −0.112 1

0.211

0.004

0.279

0.08

−0.053

4. Amount read

−0.008 −0.138 0.211

1

0.141

0.609

0.243

0.103

5. Amount TV

−0.006 0.087

0.004

0.141

1

0.153

−0.031 −0.022

6. Enjoy read

−0.005 −0.29

0.279

0.609

0.153

1

0.303

0.208

7. Confidence read 0.049

−0.221 0.08

0.243

−0.031 0.303

1

0.293

8. GMG

−0.328 −0.053 0.103

−0.022 0.208

0.293

1

0.192

Table 6.3 LME models to predict comprehension scores Model

Variable 1

Variable 2

t value (var l)t value (var 2)

Model r2

1

CAREC

GMG

−4.28** 4.926**

0.065

2

FKGL

GMG

−3.422** 4.888**

0.055

3

Dale-Chall

GMG

−6.296** 5.009**

0.095

6.3.2 Comprehension Models We conducted three linear mixed effects models to predict comprehension scores with each model featuring a different readability formula and all models including individual difference features. For each model, only the readability formula and the GMG scores were predictive. Model summaries including the variables kept, the t values for those variables, and the overall variance explained by each model (i.e., r2) are reported in Table 6.3. The strongest model was reported for the Dale-Chall readability formula model, which explained ~10% of the variance. The CAREC model explains ~7% of the variance while the Flesch-Kincaid Grade Level model explained ~6% of the variance.

6.3.3 Reading Speed Models We conducted three linear mixed effects models to predict the reading speed with each model featuring a different readability formula and all models including individual difference scores. Model summaries including the variables kept, the t values for those variables, and the overall variance explained by each model (i.e., r2) are reported in Table 6.4. The strongest model was reported for the CARES readability formula model, which explained ~19% of the variance and included GMG scores and trial order for the texts. The Dale-Chall model explained ~ 14% of the variance and also included the trial order of the texts and GMG scores. Flesch-Kincaid Grade Level

98

S. Crossley et al.

Table 6.4 LME models to predict comprehension scores Model Variable 1

Variable 2

Variable 3 t value (var l)t value (var 2)t value Model r2 (var 3)

1

CARES

Trial order GMG

8.903**

−5.248** −3.497**

0.185

2

Trial order

GMG

−5.857**

−3.487** NA

0.134

3

Dale−Chall Trial order GMG

• 2.239*

−5.737** −3.489**

0.137

NA

was not a significant predictor. The model using only the trial order of the texts and GMG scores explained ~ 13% of the variance.

6.4 Discussion This study examined if readability formulas were predictive of text reading times and comprehension scores stemming from a behavioral reading study. The results provide evidence that a model with the new Dale-Chall Readability formula explained the most variance in text comprehension scores while a model including the CARES formula explained the most variance in reading times when other factors related to text readability (e.g., reading proficiency), individual differences, and experimental design (i.e., trial order) were also considered. These findings suggest that newer formulas that better tap into the reading construct such as the New Dale-Chall and CARES formulas are likely the best predictors of text comprehension and processing speed, respectively, for the small corpus of text analysed in this study. These formulas, which are intelligent improvements over previous formulas, can be used as technological mediators to better match learners with texts to keep learners on track to better achieve reading goals. With reference to comprehension scores, the model including the New DaleChall readability formula was the strongest predictor of comprehension scores. The negative coefficient indicated that texts with higher Dale-Chall scores were more difficult to comprehend. Beyond readability formulas, Gates-MacGinitie (GMG) Reading Test scores were also significant predictors. As would be predicted, the reading score coefficient indicates that students with higher reading scores had higher comprehension accuracy. In total, the New Dale-Chall and GMG explained ~10% of the variance. The models including GMG scores and either CAREC or FleschKincaid Grade Level explained ~7% and ~6% respectively. In terms of reading time, the model that included CARES explained the most variance in reading time. Other significant predictors in this model included trial order and GMG Test scores. The trial order results indicate that participants began to read texts more quickly as the experiment moved forward. The reading test scores indicate that more proficient readers took less time to read the texts. None of the other fixed factors were significant predictors of reading times. The model with CARES explained ~19% of the variance. A model including GMG scores, trial order, and the

6 Assessing Readability Formulas in the Wild

99

New Dale-Chall formula explained around ~14% of the variance. Flesch-Kincaid Grade Level, when moderated by GMG scores and trial order, was not included as a significant factor in an individual model. These findings make important contributions to our understanding of the reliability and validity of various readability formulas in terms of their prediction success for text processing speed and comprehension for adult readers. We find that the New Dale-Chall formula explains the most variance for the data examined here. The New Dale-Chall formula contains an updated version of frequent words and the implementation of it in ARTE includes an expanded list of morphologically related words. Word frequency more strongly taps into lexical sophistication and should have stronger overlap with decoding than traditional lexical measures based on number of letters per word.. In contrast, the formula measures syntactic complexity solely as a function of average sentence length, striking a balance between new and old approaches toward measuring comprehension. Surprisingly, CAREC performed weaker than Dale-chall even though recent studies have shown improvements for CAREC when assessing text readability over other formulas [11, 12]. This may be a function of the manner in which CAREC was trained (using crowd-sourced judgments), the manner in which comprehension was operationalized in this study (i.e., multiple-choice comprehension questions), the population studied, or the small number of texts analysed. We find that a model built on top of CARES was the strongest predictor of reading speed. Considering that CARES is the only readability formula specifically normed for processing speed, the results seem intuitive. Examinations of correlations between the other readability formulas and reading speed indicated few associations, providing evidence that most traditional formulas do not measure features of texts which influenced processing for this data set.

6.5 Limitations There are a number of limitations to the approach used in this study and we discuss the most salient below with the goal of guiding future research. Most importantly, this study only examined 12 texts, which is not a large enough sample size to generalize about the strength of readability formulas beyond the scope of this study. Sample size is a continuous problem with reading comprehension studies because collecting readability criteria across large text samples is time consuming. However, the use of larger samples would help to extend these findings to a broader population. A larger sample size would allow for greater inclusion of a number of fixed factors as well, since autocorrelation was a problem with only 12 texts. For instance, domain and simplification levels were not included in the models because of autocorrelation. Posthoc examinations of text reading times indicated that all texts regardless of the domain were read at about the same speed, although simplified texts took a bit longer to read. In a larger corpus, these differences may become significant, potentially because of the extra length of simplified texts attributable to text elaboration. Additionally, it is

100

S. Crossley et al.

likely that there are effects based on text topics. Sampling a large number of texts would also allow researchers to include the topic as a random effect. Another concern is that comprehension for this study was operationalized as multiple-choice comprehension questions, which may not be the most effective way to measure comprehension. Other approaches to measure comprehension include cloze-tests and textual recall, summarization, and inferential activities. Lastly, we did not assess text domain familiarity in this study, which may have an influence on text processing and comprehension. Study examined if readability formulas were predictive of text reading times and comprehension.

6.6 Conclusion This study tested the prediction rates for classic and newer readability formulas for text comprehension and reading speed. We find that a model including the New Dale-Chall formula was the strongest predictor of comprehension (along with GMG scores) and that a model including newer readability formula related to text processing was the strongest predictor of reading speed (along with GMG scores and trial order). While the reading speed model explained about 19% of the variance, the reading comprehension model explained only around 10% of the variance. Overall, this study provides some validation for the use of smarter readability formulas which incorporate NLP inspired measures to predict text reading speed and comprehension. These newer readability formulas tap into the reading construct more intuitively and their increased accuracy over traditional readability formulas can act as technological mediators to increase reading success and the development of the reading process, thus creating the foundations for a smart personalized learning environment which presents learning resources adequate for each learner.

References 1. Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4 (2014). arXiv:1406.5823 2. Benjamin, R.G.: Reconstructing readability: recent developments and recommendations in the analysis of text difficulty. Educ. Psychol. Rev. 24, 63–88 (2012) 3. Chall, J.S., Dale, E.: Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books (1995) 4. Choi, J., Crossley, S.A.: Automated readability web app for English. In: Twenty-second IEEE International Conference on Advanced Learning Technologies (ICALT 2022), Bucharest, Romania (2022) 5. Crossley, S.A., Greenfield, J., McNamara, D.S.: Assessing text readability using cognitively based indices. TESOL Q. 42, 475–493 (2008) 6. Crossley, S. A., Heintz, A., Choi, J., Batchelor, J., Karimi, M., Malatinszky, A.: A large-scaled corpus for assessing text readability. Behav. Res. Methods (2022) 7. Crossley, S.A., Skalicky, S., Dascalu, M.: Moving beyond classic readability formulas: new methods and new models. J. Res. Reading 42(3–4), 541–561 (2019)

6 Assessing Readability Formulas in the Wild

101

8. Davison, A., Kantor, R.: On the failure of readability formulas to define readable texts: a case study from adaptations. Read. Res. Q. 17(2), 187–209 (1982) 9. DuBay, W.H.: The Principles of Readability. Costa Mesa, CA: Impact Information (2004) 10. Flesch, R.: A new readability yardstick. J. Appl. Psychol. 32(3), 221–233 (1948) 11. Fry, E.: Fry’s readability graph: Clarifications, validity, and extension to level 17. J. Read. 21(3), 242–252 (1977) 12. Gunning, R.: The technique of clear writing. McGraw-Hill, New York, NY (1952) 13. Kincaid, J. P., Fishburne, R.P., Rogers, R.L., Chissom, B.S.: Derivation of new readability Formulas: (Automated readability index, fog count and Flesch Reading Ease Formula) for Navy enlisted personnel. (No. RBR-8–75). Naval Technical Training Command, Millington, TN: Research Branch (1975) 14. Kuznetsova, A., Brockhoff, P.B., Christensen, R.H.B.: lmerTest package: tests in linear mixed effects models. J. Statistical Softw. 82(13) (1975) 15. MacGinitie, W.H., MacGinitie, R.K., Cooter, R.B., Curry, S.: Assessment: Gates-Macginitie Reading Tests. Read. Teach. 43(3), 256–258 (1989) 16. McNamara, D. S., Levinstein, I.B., Boonthum, C.: iSTART: Interactive strategy training for active reading and thinking. Behav. Res. Methods Instrum. Comput. 36(2), 222–233 (2004). https://doi.org/10.3758/BF03195567 17. Nakagawa, S., Schielzeth, H.: A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol. Evol. 4(2), 133–142 (2013) 18. National Assessment of Educational Progress. The Nation’s Report Card: Writing (2011) 19. Newbold, N., Gillam, L.: The linguistics of readability: the next step for word processing. In: Proceedings of the NAACL HLT 2010 Workshop on Computational Linguistics and Writing: Writing Processes and Authoring Aids, pp. 65–72. Association for Computational Linguistics (2010) 20. Pitler, E., Nenkova, A.: Revisiting readability: a unified framework for predicting text quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 186–195. Association for Computational Linguistics (2008) 21. Powell, P.R.: Retention and writing instruction: Implications for access and pedagogy. Coll. Compos. Commun. 60, 664–682 (2009) 22. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics: The nation’s report card. National Center for Education Statistics, Washington, DC (2020) 23. Wolfe, M.B., Schreiner, M.E., Rehder, B., Laham, D., Foltz, P.W., Kintsch, W., Landauer, T.K.: Learning from text: Matching readers and texts by latent semantic analysis. Discourse Process. 25(2–3), 309–336 (1998)

Chapter 7

pROnounce: Automatic Pronunciation Assessment for Romanian Dan Ungureanu, Stefan Ruseti, Irina Toma, and Mihai Dascalu

Abstract Automatic pronunciation assessment has the potential to play a key role in the process of improving proficiency in a foreign language. The process represents a modern, smart learning alternative where students can receive immediate feedback with very low friction. As an overview, such systems are built on top of regular Automatic Speech Recognition engines using data collected from various native speakers but operate at a lower level, where the system recognizes phonemes instead of individual words. Another key difference is that such systems do not attempt to perform automatic corrections using a language model; instead, the output is a measure of resemblance with the learned baseline with emphasis on the detection of mispronunciations. In this study, we introduce pROnouce, a tool designed for the Romanian language, which also considers gamification to ensure a more pleasant experience. Two approaches for pronunciation assessment were considered, both using Deep Neural Network models, coupled with a method to further expand the training dataset using recordings for other languages that share the phoneme set. Our system was evaluated by more than 150 individuals at Expo Dubai 2020 who were interested in experimenting with introductory Romanian words and provided feedback.

D. Ungureanu · S. Ruseti · I. Toma · M. Dascalu (B) University Politehnica of Bucharest, 313 Splaiul Independentei, 060042 Bucharest, Romania e-mail: [email protected] D. Ungureanu e-mail: [email protected] S. Ruseti e-mail: [email protected] I. Toma e-mail: [email protected] M. Dascalu Academy of Romanian Scientists, Str. Ilfov, Nr. 3, 050044, Bucharest, Romania © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_7

103

104

D. Ungureanu et al.

7.1 Introduction Learning a new language is a complex task that considers multiple dimensions, including grammar, vocabulary, and pronunciation. In general, pronunciation plays an important role, and improving it speeds up the overall learning process as students become comfortable speaking in public, which further exposes them to the language [1]. However, students seem to downplay in practice the importance of pronunciation [2]. Depending on the learner’s native language and the targeted new language, some sounds may be more difficult to reproduce. Practice is extremely important and having an automated tool increases flexibility in the learning process by providing accurate and immediate feedback. Automatic pronunciation assessment models can be developed starting from an Automatic Speech Recognition model trained on native speech that also provides the distribution for each phoneme. These distributions are then used to measure the extent to which the pronunciation is statistically close to that of a native speaker. Pronunciation evaluation covers multiple dimensions, for example: discrete (i.e., “correct” or “mispronounced”) versus continuous (i.e., score-based) feedback; type of error or level where the error occurs (i.e., “phonemic”, “prosodic”); the quantized unit of speech (i.e., “word” or “phoneme”); annotated training data with or without mispronunciations; the number of baselines used to compute the score (L1—speaker’s first language or L2—target language); approach—statistical versus machine learning based. In this paper, we introduce pROnounce, an automatic pronunciation assessment tool for Romanian that was developed starting from existing state-of-the-art Automatic Speech Recognition systems and is considered a specialized model based on Deep Neural Networks (DNN). We leveraged training data from more popular languages that share the Romanian phoneme set to train a robust pronunciation model. We also built a smart learning application on top of the assessment model with a built-in gamified experience, which was showcased at ExpoDubai 2020. In the following sections, we describe the state-of-the-art methods, followed by our research method, the considered datasets, data augmentation techniques, the speech evaluation model implemented with Kaldi [3], our specialized neural network model, as well as the architecture of our application. The last two sections introduce the results, followed by conclusions and further improvements.

7.2 State of the Art Automatic pronunciation assessment methods evolved together with Automatic Speech Recognition (ASR) and, at present, the specialized literature contains a wide variety of approaches. Witt [4] discussed the state-of-the-art research as of early 2012 and covered all major components involved in pronunciation assessment. The author emphasized the ambiguity of the term “pronunciation error” and the difficulties that

7 pROnounce: Automatic Pronunciation Assessment for Romanian

105

arise in quantifying errors. Pronunciation errors were divided into phonemic and prosodic error types, stating that the former is the “severe” one, while the latter is more related to the problem of native-live versus intelligible pronunciation. One of the oldest and most notable results was obtained by Kim, Franco, and Neumeyer [5] who used a Hidden Markov Model-based (HMM) ASR model, computed log-likelihood, log-posterior probability, and segment duration scores. Among their experiments, the log-posterior probability scores correlated most with human scores. Various other papers [6, 7] considered variations of log-posterior probability scores that used the techniques developed for Hidden Markov Model-based systems. In the past decade, deep learning technology advanced significantly and became a standard for the development of Automatic Speech Recognition technologies. In comparison with the classic approaches based on Hidden Markov Model, Deep Neural Networks can model longer connections between input features, while mapping more accurately input and output features using multiple layers of various sizes and types. Hidden Markov Models consist of sequences of large pipelines that processed the speech signals, most often optimized individually, which in return led to sub-optimal results. End-to-end speech systems built using Deep Neural Networks reached maturity and outperformed previous architectures. The same trend followed in automatic pronunciation evaluation. For example, Hu et al. [8] proposed a Deep Neural Network-system that outperformed the state-of-the-art Hidden Markov Model system by 22% at the word level. Hu et al. [9] also proposed a Deep Neural Network with transfer learning for the task of detecting “phonemic” pronunciation errors. They leveraged multi-layer neural networks for learning the nonlinear characteristics of speech signals, followed by a top layer to sharpen the posterior probabilities of the phonemes. Their goal was to train an L1-independent system and to build a more general architecture suitable for commercial applications. Other studies [10] train more than one acoustic model but do not correlate the information learned between the trained models, using individual systems for pronunciation evaluation. In contrast, more recent work [11] investigates the role of L1 in automatic pronunciation evaluation of L2 speech and argues that there is an improvement in the model’s capability to evaluate pronunciation when both L1 and L2 features are used in the model.

7.3 Method In our research, we focus exclusively on models based on Deep Neural Networks that are trained using data recorded from native speakers. We decided to experiment with two approaches, one starting from a general-purpose automatic speech recognizer architecture based on Kaldi that was trained with a subset of perfect speech, and another novel, but simpler, architecture trained with a custom dataset.

106

D. Ungureanu et al.

7.3.1 Reusing the Kaldi Speech Recognizer Kaldi is a well-known open-source toolkit for speech recognition that contains the necessary components for building speech recognition systems, including examples and documentation of complete pipelines. Kaldi was successfully employed to build accurate Romanian speech-to-text models [12, 13]. The model is based on a Time-Delay Neural Network (TDNN) which can classify patterns with shift-invariance and learn context at each layer of the network. This type of network has the advantage that it does not require explicit segmentation before classification, similar to sequence-to-sequence models. In this case, the TDNN has 5 layers of Rectified Linear Unit (ReLU) followed by a renormalization block (i.e., “relu-renorm” Kaldi layers). The network receives as input 40 Mel-Frequency Cepstral Coefficients (MFCCs) [14] and 100 identity vectors (i.e., iVectors) inputs. MFCCs have the advantage to create a significantly more compressible representation using significantly less memory than the bands from a spectrogram. MFCCs are usually used with smaller systems due to their decorrelated nature. The MFCCs were extracted from segments of 25 ms with a stride of 10 ms. This model used Linear Discriminant Analysis (LDA) for processing the input of the network before passing it further to the TDNN layers. In order to assess the pronunciation of a word, we first used force alignment at the phoneme level. This is a necessary first step to correct errors of misrecognized phonemes due to severe mispronunciations. Up until this point, the system is similar to the decoding stages of the regular Automatic Speech Recognition system. After the force alignment, the acoustic model provides the start and end frame indices of each phoneme. The pronunciation score of the target phoneme p is:   P(O p | p) ∗ P( p) log(P( p|O p )) = log  G O P( p) = /|O p | q |q) ∗ P(q) |O p | P(O q∈Q where Q is the set of all phonemes, O p is the feature matrix of phoneme p, and |O p | is the number of frames of phoneme p after the alignment. Our assumption is that the priors of all phonemes are equal; as such, the score can be computed as: ⎞

⎛ G O P( p) = log⎝

P(O | p) ⎠ /|O p | max P(O q |q) p

q∈Q

Since this model requires a complete automatic speech recognizer, we used a regular annotated speech dataset for training. However, we selected a dataset with clear recordings of speech without a regional accent to ensure that pronunciation scores are consistent with native speech. For this purpose, we chose to use the SWARA corpus [15], which consists of 17 speakers and a total duration of 21 h.

7 pROnounce: Automatic Pronunciation Assessment for Romanian

107

7.3.2 Specialized Deep Neural Network Model The second developed architecture is completely different from the previously described model in all aspects, starting with feature extraction, the architecture of the neural network itself, the method for computing scores from the network’s output, and even the training data. The new model was developed incrementally, starting from a basic sequenceto-sequence model using Connectionist Temporal Classification (CTC), and then expanded to a hybrid connectionist temporal classification and attention model. We ended up with a simpler architecture based on the Transformer architecture [16]. Transformers are deep learning models that use the mechanism of self-attention, differentially weighting the significance of each part of the input data. Transformers are used mainly in the fields of text processing in Natural Language Processing (NLP) but have also been successfully employed for Automatic Speech Recognition [17–19]. A common way of extracting acoustic features relies on a cepstral representation of the audio recording, such as MFCCs from Kaldi. The alternative is to use the entire Mel spectrograms which contain more information and neural networks can benefit from them. In contrast to the previous model, the features for this approach were the Mel spectrograms computed from audio recordings, with 512 Fast Fourier Transform bins used for dividing the window and a hop length of 512. The extracted features were then passed through a Transformer-based Deep Neural Network. The input spectrogram is passed through 3 layers of 2D convolutional layers, a dense layer, and 4 transformer encoder layers. The phonemes are generated with another 4 transformer decoder layers and a final SoftMax layer. A heuristic is then applied on the logits output by the last layer in the Deep Neural Network to give a mark for each recognized phoneme, “poor”, “average” or “good” pronunciation. The heuristic takes into consideration the logit for the target phoneme and its rank (i.e., how many phonemes with a higher logit were recognized for the same audio window).

7.3.3 Data Augmentation for the Specialized Deep Neural Network Model One of the biggest challenges in training speech-related systems for the Romanian language is the lack of high quality data. Since this system has a much smaller architecture that lacks the generalization abilities of a larger Automatic Speech Recognition, we decided to use recordings of single words. For this reason, we extracted all recordings from LibriSpeech [20] “dev-clean” dataset matching the criteria, summing up to 99,889 recordings of 10,235 unique words. Data augmentation is frequently employed in Automatic Speech Recognition when data is sparse. SpecAugment [21] is a simple and fast data augmentation method

108

D. Ungureanu et al.

for speech that consists of a series of basic operations (i.e., warping, masking of blocks in the frequency domain, or masking of time steps) and is applied directly to the extracted features (i.e., inputs of the neural network). Previous research [21] showed a decrease of up to 1.5 percentage points in word error rates for the task of automatic speech recognition; similarly, we noticed a significant increase in accuracy and robustness while training our second model architecture. To further expand the training dataset, we started with the assumption that other foreign languages share similarities with Romanian and implicitly have a common phoneme set. We aligned the Librispeech “dev-clean” dataset to word boundaries using a previously trained Automatic Speech Recognition system, essentially resulting in a list of words with starting and ending timestamps. We then extracted the words in new single word recordings and selected only those recordings consisting exclusively of common phonemes. This way, we obtained an additional 11,014 recordings of 2,924 unique words. For the phonetic transcriptions, we trained a grapheme-to-phoneme model in a similar manner to the one described by Ungureanu, Badeanu, Marica, Dascalu, and Tufis [13]. In total, we had 42 h of recordings of 110,903 words.

7.3.4 Automatic Grapheme-to-Phoneme Conversion A requirement for both pronunciation assessment architectures is a phonetic dictionary representing a mapping between words and their phonetic pronunciation as a list of phonemes. This can be either a predefined dictionary or a dynamic grapheme-to-phoneme converter that extracts the phonemes. For example, the NaviRo dictionary [22] contains 138,500 unique words in Romanian. In order to build a generic method, we started from the NaviRo dictionary, manually corrected misspellings and standardized phoneme names, while words with missing characters (for example, “q”, “x” and “w”) were added. Afterward, another Transformer-based grapheme-to-phoneme model was trained. The same model was later on trained using an English dictionary extracted from Wikimedia Foundation’s Wiktionary,1 while ensuring that the common phonemes have the same representation in both dictionaries.

7.3.5 pROnounce—The Learning Environment for Romanian Pronunciations Learning new skills includes several aspects such as student’s prior knowledge, presentation method for new information, challenging and stimulating the user, providing and improving upon feedback, and finally, repetition. We designed a smart 1

https://en.wiktionary.org/.

7 pROnounce: Automatic Pronunciation Assessment for Romanian

109

Fig. 7.1 User interface showing the word and indications

learning application—pROnounce2 —on top of the previous assessment model that also included a built-in gamified experience to create a pleasant experience that covers most of the previous dimensions for a broad audience (see Fig. 7.1). This web application was showcased at ExpoDubai 2020. Only the transformer-based model was used for prediction in the version displayed at the expo. The developed user interface was oriented toward newbie learners of the Romanian language, of various demographic characteristics such as age, education, or native language. We optimized our system for the best user experience, ease of usage with very few controls, and being suitable for people of all ages. The final setup considered a simple single-page web application deployed on a large touchscreen and a professional uni-directional microphone. Presentation of new information was made possible through the help of hired native actors that pronounced the subject word at two speeds (normal and slower), while also displaying a suggestive picture of the considered word. Each word was presented alongside its English translation so that users could make connections and better remember the new information. For the astute students, the screen included a complete word definition and phonetic translation. Every time a new word was recognized, it was automatically assessed by our system, and feedback at the phoneme level was provided (see Fig. 7.2). A mark is assigned to each phoneme that is colored accordingly (red—very poor pronunciation of phoneme, orange—average pronunciation, green—very good pronunciation; see Fig. 7.2). The overall score is displayed as stars between one and three and is computed as an average of the phoneme scores. After each try, the user has a choice between retrying pronunciation or going to the next word.

2

https://pronounce.readerbench.com/.

110

D. Ungureanu et al.

Fig. 7.2 Result of pronunciation assessment

The gaming experience was structured on three levels of difficulties (i.e., easy, medium, and difficult), each consisting of five words. Each new level increased the overall complexity by the incremental introduction of characters with diacritics and groups of vowels. For example, the first level contained basic words such as “salut” (en. “hello”), “carte” (en. “book”), “prieten” (en. “friend”). The second level introduced words that are in general more difficult to pronounce because of the composing phonemes such as “melc” (en. “snail”), “izvor” (en. “(water) spring”), limpede (en. “clear”). The final level introduced words with many diacritics which are mapped to rare phonemes like “pes, tis, or” (en. “small fish”) or “pârâu” (en. “(water) stream”). The application included a feedback form (see Fig. 7.3) used to evaluate the learning system and to gather ideas for future improvements. The feedback form was designed for fast completion on a touchscreen and contained six questions to find out: (a) users’ country of origin, (b) their age, (c) their rating for the overall game experience, (d) how much fun they had, (e) their opinion about the received feedback, and (f) whether they would play a more complete version of the game. The last 4 questions used a 5-point Likert star-based scoring, ranging from 1 (minimum score, “very poor”) to 5 (maximum score, “great”).

7.4 Results Our system was showcased at ExpoDubai 2020 for two weeks in the last month of the event. Out of all the participants, 150 people were interested in offering their feedback and helping us understand their impressions of the application; each individual experimented on average with 5 words. Most people (47%) who tried the system

7 pROnounce: Automatic Pronunciation Assessment for Romanian

111

Fig. 7.3 Screenshot of the feedback form

were under 18 of age, 3% age 18–24, 10% age 25–34, 20% 35–44, 3% age 45–54, 7% age 55–65 and 10% over 65 (Tables 7.1 and 7.2). The overall feedback was positive and people enjoyed our application. Each question had an average score of 4.5 stars out of 5, with people willing to play a more complete version of the game. Table 7.1 Number of recorded plays per word Level 1

Level 2

Level 3

Word

Plays

Word

Plays

Word

Plays

Salut

142

Melc

44

Petior

38

Mulumesc

100

Ploaie

39

Pr˘ajitur˘a

36

Amuzant

84

Izvor

36

Pârâu

30

Carte

64

Dorin˘a

33

Mlatin˘a

21

Prieten

49

Limpede

34

Furtun˘a

41

112 Table 7.2 Average scores for each question

D. Ungureanu et al. Question

Average

How would you rate the overall game experience?

4.54

How fun was the game?

4.52

Was the scoring fair?

4.52

Would you like to play the full version of the game?

4.62

7.5 Discussion There are a few discussion points that can be deduced from the saved feedback forms: the overall experience was pleasant, the game was fair, the prediction rates were good, and people were interested in this learning method and would play the game again. One of the central design points of pROnounce was its enjoyability. We used principles of gamification and a joyful graphic design to catch the eye of people passing by, stimulating them and keeping their interest. The game was designed with a predefined list of words split among three levels of difficulties, thus ensuring a challenge even for short-term visits as expected in the exhibition. Another important outcome was that all individuals considered that the evaluation models are performing adequately. There were no scores below 3 on the corresponding question, thus arguing that their opinion was at worst neutral towards the automated scoring. We are aware of the limited number of words available in the user interface, but this is an initial prototype tailored as an experience for the exposition, which we plan to further extend. Nevertheless, we emphasize that the assessment model generalizes to any pronounced word since it performs a mapping at the phoneme level. Moreover, we must stress out inherent limitations of the models. First, there are missing labeled training entries that influence the pronunciation scoring. Second, subjectivity arises while operators label the data and are faced with various problems, for example: establishing the extent to which regional accents are “incorrect” or how the mispronunciation of a letter/phoneme has a higher impact than another. In addition, to our knowledge, there is currently no dataset available that even mentions the subject of pronunciation for the Romanian language.

7.6 Conclusions In this paper, we described two approaches developed for pronunciation evaluation using deep neural networks, one using existing Automatic Speech Recognition systems and another introducing a novel Transformer-based Deep Neural Network architecture. A modern web application, pROnounce, was developed to introduce users to the Romanian language and to teach them a few basic to more advanced

7 pROnounce: Automatic Pronunciation Assessment for Romanian

113

words. The application was successfully deployed to a varied demographic and can be considered a success. The pROnounce application was designed as a fun game for discovering the Romanian language, therefore it only used a fixed list of 15 words. The proposed automated approach is not limited in any manner by the number of considered words and it is even language-independent if phoneme level annotations are provided for words. Future work includes expanding the capabilities of the existent system to leverage phonetic information learned from other languages. The system could be extended to a larger set of languages, which could also improve the performance of the prediction model by showing the same phonemes pronounced in other languages. Furthermore, we plan to create a full learning cycle—establish various ranking criteria for word difficulty, use a speech-to-text model to provide a first lecture of the word, followed by the automated evaluation of the pronunciation. Acknowledgements This work was supported by a grant from the Ministry of Research, Innovation and Digitization, CNCS/CCCDI - UEFISCDI, project number PN-III-P2-2.1-SOL-2021-2-0223, within PNCDI III.

References 1. Gilakjani, A.P.: A study on the situation of pronunciation instruction in ESL/EFL classrooms. J. Stud. Educat. 1(1), 1–15 (2011) 2. Simon, E., Taverniers, M.: Advanced EFL learners’ beliefs about language learning and teaching: a comparison between grammar, pronunciation, and vocabulary. Engl. Stud. 92(8), 896–922 (2011) 3. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P.: The Kaldi speech recognition toolkit. In: IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE, Waikoloa, HI, USA (2011) 4. Witt, S.M.: Automatic error detection in pronunciation training: Where we are and where we need to go. In: International Symposium on Automatic Detection on Errors in Pronunciation Training, pp. 1–8 (2012) 5. Kim, Y., Franco, H., Neumeyer, L.: Automatic pronunciation scoring of specific phone segments for language instruction. In: Fifth European Conference on Speech Communication and Technology (1997) 6. Witt, S.M., Young, S.J.: Phone-level pronunciation scoring and assessment for interactive language learning. Speech Commun. 30(2–3), 95–108 (2000) 7. Zhang, F., Huang, C., Soong, F.K., Chu, M., Wang, R.: Automatic mispronunciation detection for Mandarin. In: 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5077–5080. IEEE (2008) 8. Hu, W., Qian, Y., Soong, F.K.: A new DNN-based high quality pronunciation evaluation for computer-aided language learning (CALL). In: Interspeech, pp. 1886–1890 (2013) 9. Hu, W., Qian, Y., Soong, F.K., Wang, Y.: Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers. Speech Commun. 67, 154–166 (2015)

114

D. Ungureanu et al.

10. Tao, J., Ghaffarzadegan, S., Chen, L., Zechner, K.: Exploring deep learning architectures for automatically grading non-native spontaneous speech. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6140–6144. IEEE (2016) 11. Tu, M., Grabek, A., Liss, J., Berisha, V.: Investigating the role of L1 in automatic pronunciation evaluation of L2 speech. arXiv:1807.01738 (2018) 12. Georgescu, A.-L., Cucu, H., Burileanu, C.: Kaldi-based DNN architectures for speech recognition in Romanian. In: 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 1–6. IEEE (2019) 13. Ungureanu, D., Badeanu, M., Marica, G.-C., Dascalu, M., Tufis, D.I.: Establishing a baseline of Romanian speech-to-text models. In: 2021 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 132–138. IEEE (2021) 14. Mermelstein, P.: Distance measures for speech recognition, psychological and instrumental. Pattern Recogn. Artific. Intell. 116, 374–388 (1976) 15. Stan, A., Dinescu, F., Tiple, C., Meza, S., ¸ Orza, B., Chirila, M., Giurgiu, M.: The SWARA speech corpus: A large parallel Romanian read speech dataset. In: 2017 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), pp. 1–6. IEEE, Bucharest, Romania (2017) 16. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advanc. Neural Informat. Process. Syst. 30 (2017) 17. Wang, Y., Mohamed, A., Le, D., Liu, C., Xiao, A., Mahadeokar, J., Huang, H., Tjandra, A., Zhang, X., Zhang, F.: Transformer-based acoustic modeling for hybrid speech recognition. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6874–6878. IEEE (2020) 18. Synnaeve, G., Xu, Q., Kahn, J., Likhomanenko, T., Grave, E., Pratap, V., Sriram, A., Liptchinsky, V., Collobert, R.: End-to-End asr: From Supervised to Semi-Supervised Learning with Modern Architectures. arXiv:1911.08460 (2019) 19. Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y.: Conformer: Convolution-Augmented Transformer for Speech Recognition. arXiv: 2005.08100 (2020) 20. Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: an asr corpus based on public domain audio books. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206–5210. IEEE (2015) 21. Park, D.S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E.D., Le, Q.V.: Specaugment: A Simple Data Augmentation Method for Automatic Speech Recognition. arXiv:1904.08779 (2019) 22. Domokos, J., Buza, O., Toderean, G.: 100K+ words, machine-readable, pronunciation dictionary for the Romanian language. In: 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO), pp. 320–324. IEEE (2012)

Chapter 8

Community Pacts and we4SLE as Tools to Support the Implementation of Smart Learning Ecosystems Irene Urbanetti, Maria Rosaria Autiero, Vincenzo Baraniello, and Carlo Giovannella Abstract This article describes and discusses the conditions under which starting from the Italian school system it would be possible to develop smart learning ecosystems. In particular, we focus on the so-called community pact—an opportunity recently introduced and promoted by the Ministry of Education—and on how dedicated technologies may support it. Starting from the description of the regulatory context and the consultation of teachers/students/territorial stakeholders, we illustrate the needs and motivations that led to the design and development of a web portal, we4SLE, which aims at fostering and facilitating: (a) the development of the identity and the increase of the internal cohesion of the community of reference; (b) the cultural growth of the students, also as future active citizens. Among the objectives of the web portal: provide an overview of the available resources, as well as methods for finding those necessary to implement the various initiatives; highlight the contribution made by the members of the community and give adequate visibility to the path of growth of each individual; support the strengthening of existing relationships in the community and encouraging the establishment of new ones; foster awareness about the community’s state of development through a participatory approach to its evaluation.

I. Urbanetti · V. Baraniello · C. Giovannella (B) University of Rome Tor Vergata, Rome, Italy e-mail: [email protected] M. R. Autiero IIS E. Amaldi, Rome, Italy V. Baraniello · C. Giovannella ASLERD, Rome, Italy © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_8

115

116

I. Urbanetti et al.

8.1 Introduction Faced with the pressing and non-trivial challenges generated by both a stormy present and the expectations for the next future (see the SDGs promoted by UNESCO [1] and adopted by the European Union), the foundations on which is based the architecture and the educational project of the present school system appear to be obsolete and not suitable to the needs of those who are its main target: the students, that will become also the future citizens. In this regard, reference can be made to a very famous popular video by Robinson [2] that highlights the limits of the present school organization—often referred to as a “school factory”—which has its roots in the industrial revolution and, obviously, in the needs expressed by the associated productive model under development at that times and, as well, by a society that was undergoing a rapid transformation. The video was realized, mainly, to support the thesis that the main outcome of such a school model is that of repressing the development of individual creativity. Actually, it has a much broader value because it could be easily adapted to highlight the compression operated by the school standardization in the development of any LIFE skill [3] and/or individual talent. Paradoxically, such a rigid structure not only limits the development of LIFE skills and other competencies in the students but, due to the lack of flexibility, also makes it difficult to adapt the school system to the needs expressed by a productive system and society that at present are undergoing again, and since few decades, a veritable revolution. From all this derives the well-known phenomenon of the “skill mismatch” [4] and the growing and ever more widespread feeling of inadequacy of the educational system—schools and universities—even among those who have always been the main beneficiaries in terms of workforce and human resources: the companies and the productive world. The impression is that, at present, we are in a stalemate: in the face of the increasingly widespread perception of the inadequacy of the educational system, very little seems to move. The productive world limits itself to reporting critical issues and needs but does not seem willing to undertake any serious involvement in the search for a solution to the problem that, actually, should engage it in a direct and participatory manner. Policymakers and governments, on the other hand, show an enormous slowness in tackling the necessary reforms and, often, tend to chase the most fashionable trends of the moment with the idea that the structural problems can be solved by introducing the hour of coding or through introductory lectures on artificial intelligence or virtual reality. The transformation of the educational system requires a much deeper commitment and awareness that should lead to questioning the nature and role that this system could and should play today, as well as to considering the expectations of the various stakeholders: students, families, and productive system. This is of paramount importance and cannot be delayed if one doesn’t want to see the replacement of the traditional educational agencies, that are presiding over the territory, with a new impalpable and delocalized agency, a sort of educational non-place: the web. One possibility in this sense is to go beyond the vision of schools as service structures and bring them back to the center of their communities of reference, both

8 Community Pacts and we4SLE as Tools …

117

in territorial terms to foster the development of a sense of civic duty and belonging, and, in a broader sense, in cultural terms to foster cross-fertilization. Indeed, unlike other entities, schools have the advantage of having a regulated territorial coverage and, altogether, constitute the most widespread territorial network that a country owns and can use to get in contact with and support the growth of, future generations of citizens. To step forward, thus, it is not enough to transform the architecture and the educational project associated with schools, one needs also to change the citizens’ perception about schools: no longer a service but, rather, a driving force for the growth of the whole territory. What, then, are the actions to be taken to restore the role of the school as the driving engine of the territory? To give the opportunity to the schools to train students at 360°, well beyond the curricular expectations, and prepare them to face the complexity of the nowadays world with the aim to make them act as agents of a sustainable future? In other words, how do transform the schools into veritable smart learning ecosystems [5, 19, 20] capable to behave also as hubs [21] and contribute to the increase of the well-being of their students and all members of the community of reference?

8.2 The Operational Context of Reference 8.2.1 General Conditions The questions listed at the end of the previous section are not easy to answer also because the actions to be taken are often dictated by the characteristics of the context in which one operates, as demonstrated by the articles presented during the SLERD conferences organized by ASLERD since 2016 [5, 6], the guide to action about Learning Cities and SDGs by UNESCO [23, 24] and, finally, the initiative promoted by WISE at the end of 2020 [7]. In any case, to happen the cultural transformation considered here requires the satisfaction of given boundary conditions: – if the context is characterized by a strong regulatory framework, it is necessary that this latter allows the learning ecosystem to operate with a certain degree of flexibility and autonomy in order to adapt the actions to the characteristics of the context and of the territory of reference; – if the laws allow operating with flexibility and autonomy, then, equally flexible operational models are needed since they should be, most likely, contextualized and customized according to the characteristics of the school and of its territory of reference; – if you also have flexible operational models, then you need people with adequate characteristics to activate a virtuous circle capable of reproducing itself year after year. For example, you need a principal with adequate leadership, networking skills, and a farsighted vision of the future. Moreover, you also need a group of

118

I. Urbanetti et al.

teachers who know how to involve students and families, who are also available to learn lifelong and capable to work—when deemed necessary—behind the scenes to act as facilitators. A role that implies also the capability to act as a connecting hub to ensure that the baton is passed from one group of students and families to the next, and to attract former students so that they could contribute to the growth of the community through their wealth of experience. In principle, the role of facilitator can initially be carried out by an external player (e.g. an association operating in a specific cultural and territorial context) but what is very important is that, over time, skills are transferred to the school and its teachers so that the processes can be replicated in the future, also in case of the turnover of principal and some of the teachers. It is essential to underline that the learning ecosystem could become much easier a veritable engine for the whole territory, if also the territorial stakeholders would be animated by a true interest in the development of their own community and would contribute to the growth of the social capital by making their own resources available, primarily in terms of skills and competences but also in terms of assumption of co-responsibility, in compliance with the principle of subsidiarity enshrined, for example, by the Italian Constitution [22]. Very pragmatically, however, it must be recognized that the availability of people may not be enough in the face of the lack of material resources, in which case it will also be necessary to implement alternative forms of funds raising, such as, for example, the crowd-funding and other forms of social support. In fact, it is not possible to activate a virtuous circle of growth if one depends only on structural funding and/or calls for tenders. This latter could work as a stimulus but cannot guarantee continuity.

8.2.2 Opportunities Offered by the Italian Regulatory Framework After having outlined some general conditions—applicable to any context—in this article, we will concentrate on the opportunities that are offered by the Italian context—including the so-called “community pacts”—that will be described briefly later on. In Italy, the Ministry of Education has defined the regulatory framework for the functional autonomy of the schools already 25 years ago (Law no.59 of 15/3/1997 and its implementing decree, Presidential Decree no. 275 of 8/3/1999 [8]). Among other things, this law allows all schools to organize additional activities to integrate the curricular ones with the main purpose of responding to the specific needs of the students and of the context of reference, as well as of providing each learning ecosystem with the opportunity to stand out and excel.

8 Community Pacts and we4SLE as Tools …

119

Participatory evaluation, co-design and improvement plans. In such a context— alongside the self-assessment obligations imposed by the Ministry [9]—it makes sense, for example, to exploit autonomy to develop its own participatory assessment tools by mean of which all the actors interested in given educational processes— students, teachers, parents, territorial stakeholders—can express their opinion in order to generate a more objective and complete picture of how the learning ecosystems is perceived by the community. The participatory evaluation, experimented by ASLERD in various contexts [10–12], indeed, allowed the schools to trigger a participatory reflection accompanied by co-design processes and by the definition of improvement plans that could be considered a first step toward the development of a solid territorial community. Alternation schemes (PCTO: paths for transversal competencies and orientation). Among the tools provided by law 107 [13], there is the obligation to involve students in work-inspired learning experiences [25]. An obligation that can be managed in full autonomy by the schools and that, in principle, is based on quite challenging networking and co-design actions that are expected to involve local productive and associative entities in the definition of experiences that can help students: (a) to approach future job activities; (b) to develop skills and competencies that go beyond the curricular and/or the purely technical (hard-skills) ones—to include transversal skills (LIFE skills [3]) and soft digital ones (DigiComp) [14]. The dual education experiences could also be easily oriented towards social innovation and the Sustainable Developments Goals (SDGs) and may serve as a support for the development of a sense of identity and belonging with respect to the community of reference. In the past, ASLERD has been involved also in the design and management of alternation schemes [15, 16] that led students to reflect and hypothesize solutions for the needs and expectations expressed by the community of reference and, possibly, to answer also to the SDGs, having as payback the possibility to develop competence and to obtain a blockchain anchored e-certification. Community pacts. Participatory practices and alternation schemes, to which we made reference in the last two paragraphs, could be integrated and find a stable placing and setting within a tool proposed quite recently by the Ministry of Education to reinforce the sense of identity of the territorial community of reference of any school: the socalled community pact (described within the Document for the planning of the school education and training activities in all institutions of the National Education System for the school year 2020–2021). The community pact implements the idea—supported by the ministry since 2012—that each school could become a veritable civic center. The main goal of the community pact, in fact, is the activation of the local community to identify shared solutions to serious problems that may affect the most disadvantaged areas of a given territory: such as the dropout at school, the high delinquency rate, etc. It is no coincidence that just during the pandemic the Italian government enacted guidelines and gave impetus to the community pacts, that could be considered a strong basis for the establishment and development of veritable learning ecosystems. During the pandemic, in fact, the community pacts were used, among other things, to sustain the

120

I. Urbanetti et al.

search for spaces and resources useful to counteract the confinement and the social distancing imposed by the lockdown. However, looking forward, beyond the first intentions and the emergency, the community pacts can also be considered a tool to implement to foster the achievement of excellence, the culture of active and digital citizenship, to support SDGs, social innovation and territorial development, and, therefore, the overall increase of smartness of territories and cities. They represent, in fact, a tool by means of which is possible to stimulate the engagement of the local communities and involve them in the processes of co-evaluation, co-design, and co-responsibility, with the aim to support the growth of students’ competencies and, as well, those of all other citizens belonging to the local communities, in a coevolutionary perspective. There are many community pacts already active in Italy, a few hundred accordingly to a recent preliminary survey [26]. A first study carried out on about fifteen of them [27] seems to confirm that many of the general conditions described in the previous section are necessary for their development and sustainability. From the same study, it appears also extremely useful the inclusion in the community pact of activities like those described previously in this section: participatory evaluation, co-design, alternate schemes, etc.

8.2.3 The Case Study of the IIS E. Amaldi To take inspiration for our action, we focused on a specific pact promoted by the IIS E. Amaldi, a high school based in Rome, that has been entitled "Schools in common— We generate the change”. Such pact involves also a Comprehensive Institute (Melissa BASSI), the metropolitan city of Rome, the Municipio VI, two parishes and more than 20 cultural associations, most of them active in the territory of reference of the school—Tor Bella Monaca—a suburban area characterized by an intrinsic economic and social fragility. The main objective of this pact is to foster the educational coresponsibility of citizens in the training of students, together with the socio-cultural development of the territory by means of the implementation of what has been defined as a community school, i.e. a hub capable to connect the public institutions and citizens’ associations. The pact proposes to take advantage of the skills and competencies of each member to counteract the educational poverty, the uneasiness of the youths, the school dropout and, as well, to favor the integration of formal, informal and non-formal learning. In order to make emerge the charachteristics of the community of reference of the IIS E. Amaldi and, as well, the relationship of this community with the territory, we carried out a survey to which took part a sample of the categories that compose such community: 28 teachers, 24 students and 10 territorial stakeholders. The initial four questions (see Table 8.1) were intended to quantitatively measure the community members’ sense of belonging with respect to the school and its territory of reference, as well as the level of personal commitment toward activities organized by the school for the benefit of its students or the socio-cultural growth of the territory.

8 Community Pacts and we4SLE as Tools …

121

Table 8.1 Outcomes of the questionnaire that was administered to a sample of teachers (T) and students (S) of the IIS E. Amaldi, and of stakeholders (SH) of the territorial pact Mean S

Mean SH

Sense of belonging to and identity with M = 8.27 the school (IDSC) [7.72, 8.83]

M = 6.76 [5.96, 7.56]

M = 6.45 [5.39, 7.51]

Sense of belonging to and identity with M = 6.83 the territory (IDTE) [5.93, 7.73]

M = 5.52 [4.51, 6.52]

M = 6.09 [4.99, 7.19]

Commitment in collaboration in activities to enrich student preparation (CCASP)

M = 7.76 [7.06, 8.45]

M = 7.44 [6.51, 8.37]

M = 7.44 [6.51, 8.37]

Commitment in collaboration in activities to develop the territory (CCADT)

M = 6.90 [6.12, 7.67]

M = 6.35 [5.26, 7.43]

M = 5.63 [4.50, 6.77]

Usefulness of a competences bank (UCB)

M = 7.71 [6.95, 8.47]

M = 8.04 [7.35, 8.73]

M = 7.63 [6.67, 8.60]

Usefulness of a crowd resources collection (UCRC)

M = 7.89 [7.25, 8.53]

M = 7.83 [7.01, 8.66]

M = 7.73 [6.82, 8.63]

Usefulness of competences micro-certification (UCMC)

M = 7.76 [7.06, 8.46]

M = 7.71 [6.98, 8.44]

M = 7.82 [6.83, 8.81]

Usefulness of a students’ competences M = 7.54 showcase (USCS) [6.71, 8.36]

M = 7.62 [6.79, 8.46]

M = 7.73 [6.68, 8.77]

M = 5.39 [4.53, 6.26]

M = 6.62 [5.65, 7.60]

M = 6.82 [5.45, 8.19]

Variable

Usefulness of a participatory evaluation section (UPES)

Mean T

The survey results show a significantly lower sense of ownership and identity (IDSC) toward the school in students and stakeholders than in teachers. With surprise, we verified that for all categories the average value of IDSC drops consistently when the focus moves from school to territory (IDTE). The average level of commitment toward activities promoted for the benefit of students (CCASP) appears very similar for all categories that participated in the survey, and for all of them it decreases as the object of commitment becomes the socio-cultural development of the territory (CCADT). Particularly surprising is the very low value found for territorial stakeholders. For the significance of the differences between mean values of the measured factors in the case of teachers and students, please refer to the comparison table in Appendix. The numerical results extracted from the survey on the one hand tell us of different categories of community members ready to collaborate actively with the school, and within the school, to support the growth of students but, on the other of individuals who little recognize themselves in a territorial reality, like that of Tor Bella Monaca, affected by no minor social problems. The IIS E. Amaldi is seen as a sort of cultural bulwark by those who responded to the survey, but the same respondents do not seem to believe in the school’s ability to promote the socio-cultural growth of the territory. This present situation, therefore, seems scarcely favorable to the establishment of a

122

I. Urbanetti et al.

Fig. 8.1 Word cloud realized with the answers given to the question: What do you think is the purpose of the community pact promoted by the IIS E. Amaldi?

fruitful exchange with the territory, while it appears more conducive to the development of the school as a cultural hub and pole of attraction. The bet could be to leverage the latter to improve the former. By means of another open-ended question, we tried to elicit respondents’ perceptions about the community pact and its purpose. Students were unable to clearly express their opinion about its meaning, demonstrating a lack of understanding of the values the community pact stands for. The opinion of teachers and territorial stakeholders was otherwise expressed more knowledgeably and produced the word cloud shown in Fig. 8.1. From that figure, as well as from the analysis of the responses, it is evident that there is a focus on strengthening the relationship between the school and the territory for educational, social and cultural purposes as well as to counteract critical phenomena such as, for example, the dropout. To close this section is worthwhile to mention that ASLERD, as a supernational entity, did not sign the pact but supports it through a Memorandum of Understanding (MoU) and by contributing with its vision of smart learning ecosystems and, as well, with its experience in participatory evaluations, co-design and co-management of alternation schemes, and micro-certifications of competences.

8.2.4 The we4SLE Web Portal The one described in the previous section is a context in which it is likely that a dedicated web portal could be used for strengthening the ties already existing within the territory, or fostering new ones, around the initiatives proposed by the school and by the members of the community of reference. The main question, thus, becomes: how to design a technological environment capable to support the development of a

8 Community Pacts and we4SLE as Tools …

123

learning ecosystem without overlapping other tools that are freely available on the web? For example, it would not make sense to develop tools to foster socialization and rapid communication since anyone can access social channels such as Facebook, WhatsApp, etc. To answer such questions, a few interviews and focus groups were conducted with the school principal and other territorial stakeholders. The following needs emerged: – have access to a showcase dedicated to the community activities (past, present and future ones) and to the adherents to the pact; a showcase structured and stratified in a way that allows minimizing the dispersiveness that affects the socials; – have access to a concise and comprehensive representation of the skills available within the community; – have an application that allows collecting all kinds of resources needed for the development of the proposed activities; – enhance and make visible the skills developed by present and former students; – have centralized access to micro-certification processes; – have an area specifically dedicated to the participatory evaluation process and, as well, to explore the outcome of such process; – have an application that supports co-design activities. In summarizing, it can be stated that the needs emerged from the interviews and the focus groups concern: (a) the reduction of the dispersion and the increase in the structuring of the information deemed essential to the life of the community; (b) getting aware and optimizing the use of the competences and resources available within the community; (c) the support to participatory activities: co-design, implementation, and evaluation. Before translating such needs into a web portal, by means of the same survey referred to in Sect. 2.3, we have checked further for the perceived usefulness of the functionalities indicated by the participants in the brainstorming. The results are shown in Table 8.1. Apparently all—teachers, students and territorial stakeholders— agree (with similar numerical evaluation) on the usefulness of the web portal and almost all the proposed functionalities. In particular, they agree about the need to create a competencies bank, to have a mechanism for crowd resources collection, to support processes of competencies micro-certification and to create a students’ competencies showcase. Less relevant is deemed the opportunity to participatory evaluate the activities promoted and realized by the adherents to the community pact. The less interested in this latter functionality are the teachers. Despite these results, we decided to include anyway a section dedicated to the participatory evaluation because it stands at the basis of the virtuous cycle: participatory evaluation, co-design of improvement plans, and implementation of designed actions. The expectations of the members of the IIS E. Amaldi community were, then, translated into a web portal, called we4SLE (we for smart learning ecosystems), realized using a responsive layout to allow anyone to use it from any device. Although the main goal of we4SLE is to support Amaldi community, the portal, actually, offers itself also as an open template environment that could be used by other schools wishing to implement a smart learning ecosystem starting from a community pact.

124

I. Urbanetti et al.

Fig. 8.2 Portion of the homepage of the we4SLE portal designed to support the development of a Smart Learning Ecosystems following the community pact promoted by the IIS E. Amaldi

A picture of a portion of the homepage of the first prototype, realized by means of WordPress is shown in Fig. 8.1, while Fig. 8.2 shows the organization of the main areas of the portal. At the present we4SLE is being evaluated by the members who have adhered to the community pact, while the school is selecting an editorial staff with the aim of entrusting it with the updating of the portal contents with the needed continuity and, possibly, managing the portal. A very preliminary assessment with students would indicate a liking rate of 44% for the portal section devoted to support the micro-certification processes, 38% for the section dedicated to the proposal of new initiatives, 37% for the section dedicated to the crowd resources collection, 33% for the students’ competencies showcase and the informative section, 27% for the competencies bank, and finally 13% for the section devoted to the participatory evaluation (see the portal’s sections shown in Fig. 8.3). In the next future, we intend to carry out usability tests and more accurate liking tests with all categories of stakeholders and, in the medium term, to verify the effectiveness of the portal to support the growth of the local community and, more in general, of the smartness level of the learning ecosystem [11].

8 Community Pacts and we4SLE as Tools …

125

Fig. 8.3 Blocks representation of the portal structure and sections

Appendix In this appendix we report the outcomes of the survey described in Sect. 2.3. The means of the factors that have been investigated with the survey are tested with respect to the mid of the scale (5.5) with the Wilcox test since their distribution is not fully gaussian. Also the comparison between the teachers’ and students’ perceptions have been based on the Wilcox test (Tables 8.2, 8.3, 8.4 and 8.5).

Table 8.2 Teachers’ perceptions Variable

Mean

Wilcox test

Sense of belonging to and identity with the school (IDSC)

M = 8.27 [7.72, 8.83]

V = 430, p < 0.001 Cohen’s d = 1.90

Sense of belonging to and identity with the territory (IDTE)

M = 6.83 [5.93, 7.73]

V = 345, p = 0.006 Cohen’s d = 0.56

Commitment in collaboration in activities to enrich student preparation (CCASP)

M = 7.76 [7.06, 8.45]

V = 403, p < 0.001 Cohen’s d = 1.23

Commitment in collaboration in activities to develop M = 6.90 the territory (CCADT) [6.12, 7.67]

V = 364, p = 0.001 Cohen’s d = 0.68

Usefulness of a competences bank (UCB)

M = 7.71 [6.95, 8.47]

V = 382, p < 0.001 Cohen’s d = 1.13

Usefulness of a crowd resources collection (UCRC)

M = 7.89 [7.25, 8.53]

V = 371, p < 0.001 Cohen’s d = 1.47

Usefulness of competences micro-certification (UCMC)

M = 7.76 [7.06, 8.46]

V = 408, p < 0.001 Cohen’s d = 1.22 (continued)

126

I. Urbanetti et al.

Table 8.2 (continued) Variable

Mean

Wilcox test

Usefulness of a students’ competences showcase (USCS)

M = 7.54 [6.71, 8.36]

V = 408, p < 0.001 Cohen’s d = 0.95

Usefulness of a participatory evaluation section (UPES)

M = 5.39 [4.53, 6.26]

V = 207, p = 0.93 Cohen’s d = -.05

Variable

Mean

Wilcox test

Sense of belonging to and identity with the school (IDSC)

M = 6.76 [5.96, 7.56]

V = 262, p = 0.007 Cohen’s d = 0.65

Sense of belonging to and identity with the territory (IDTE)

M = 5.52 [4.51, 6.52]

V = 172, p = 0.80 Cohen’s d = 0.01

Commitment in collaboration in activities to enrich student preparation (CCASP)

M = 7.44 [6.51, 8.37]

V = 282, p = 0.001 Cohen’s d = 0.86

Table 8.3 Students’ perceptions

Commitment in collaboration in activities to develop M = 6.35 the territory (CCADT) [5.26, 7.43]

V = 192, p = 0.10 Cohen’s d = 0.33

Usefulness of a competences bank (UCB)

M = 8.04 [7.35, 8.73]

V = 293, p < 0.001 Cohen’s d = 1.56

Usefulness of a crowd resources collection (UCRC)

M = 7.83 [7.01, 8.66]

V = 278, p < 0.001 Cohen’s d = 1.20

Usefulness of competences micro-certification (UCMC)

M = 7.71 [6.98, 8.44]

V = 287, p < 0.001 Cohen’s d = 1.28

Usefulness of a students’ competences showcase (USCS)

M = 7.62 [6.79, 8.46]

V = 278, p < 0.001 Cohen’s d = 1.08

Usefulness of a participatory evaluation section (UPES)

M = 6.62 [5.65, 7.60]

V = 240, p = 0.009 Cohen’s d = 0.49

Table 8.4 Territorial stakeholders’ perceptions Variable

Mean

Wilcox test

Sense of belonging to and identity with the school (IDSC)

M = 6.45 [5.39, 7.51]

V = 52, p = 0.09 Cohen’s d = 0.61

Sense of belonging to and identity with the territory (IDTE)

M = 6.09 [4.99, 7.19]

V = 46, p = 0.26 Cohen’s d = 0.36

Commitment in collaboration in activities to enrich student preparation (CCASP)

M = 7.44 [6.51, 8.37]

V = 53, p = 0.08 Cohen’s d = 0.61

Commitment in collaboration in activities to develop M = 5.63 the territory (CCADT) [4.50, 6.77]

V = 35, p = 0.86 Cohen’s d = 0.08

Usefulness of a competences bank (UCB)

M = 7.63 [6.67, 8.60]

V = 66, p = 0.004 Cohen’s d = 1.49 (continued)

8 Community Pacts and we4SLE as Tools …

127

Table 8.4 (continued) Variable

Mean

Wilcox test

Usefulness of a crowd resources collection (UCRC)

M = 7.73 [6.82, 8.63]

V = 66, p = 0.004 Cohen’s d = 1.65

Usefulness of competences micro-certification (UCMC)

M = 7.82 [6.83, 8.81]

V = 66, p = 0.004 Cohen’s d = 1.58

Usefulness of a students’ competences showcase (USCS)

M = 7.73 [6.68, 8.77]

V = 66, p = 0.004 Cohen’s d = 1.43

Usefulness of a participatory evaluation section (UPES)

M = 6.82 [5.45, 8.19]

V = 52, p = 0.09 Cohen’s d = 0.65

Table 8.5 Comparison among teachers’ and students’ perceptions Variable

Mean T

Mean S

Wilcox test

Sense of belonging to and identity with the school (IDSC)

M = 8.27 [7.72, 8.83]

M = 6.76 [5.96, 7.56]

W = 544 p = 0.001

Sense of belonging to and M = 6.83 identity with the territory (IDTE) [5.93, 7.73]

M = 5.52 [4.51, 6.52]

W = 484 p = 0.03

Commitment in collaboration in activities to enrich student preparation (CCASP)

M = 7.76 [7.06, 8.45]

M = 7.44 [6.51, 8.37]

W = 390 p = 0.63

Commitment in collaboration in activities to develop the territory (CCADT)

M = 6.90 [6.12, 7.67]

M = 6.35 [5.26, 7.43]

W = 379 p = 0.39

Usefulness of a competences bank (UCB)

M = 7.71 [6.95, 8.47]

M = 8.04 [7.35, 8.73]

W = 300 p = 0.51

Usefulness of a crowd resources collection (UCRC)

M = 7.89 [7.25, 8.53]

M = 7.83 [7.01, 8.66]

W = 311 p = 0.82

Usefulness of competences micro-certification (UCMC)

M = 7.76 [7.06, 8.46]

M = 7.71 [6.98, 8.44]

W = 364 p = 0.77

Usefulness of a students’ competences showcase (USCS)

M = 7.54 [6.71, 8.36]

M = 7.62 [6.79, 8.46]

W = 333 p = 0.96

Usefulness of a participatory evaluation section (UPES)

M = 5.39 [4.53, 6.26]

M = 6.62 [5.65, 7.60]

W = 220 p = 0.03

References 1. https://en.unesco.org/education2030-sdg4/targets. Accessed Apr 2022 2. Changing Education Paradigms. https://www.youtube.com/watch?v=zDZFcDGpL4U. Accessed Apr 2022 3. https://apps.who.int/iris/handle/10665/63552, Accessed Apr 2022; http://disco-tools.eu/dis co2_portal/terms.php, Accessed Apr 2022; http://en.wikipedia.org/wiki/Life_skills. Accessed Apr 2022 4. Vandeplas A., Thum-Thysen A.: Skills Mismatch and Productivity in the EU, Publications Office of the European Union (2019). https://ec.europa.eu/info/sites/default/files/economy-fin ance/dp100_en.pdf. Accessed Apr 2022

128

I. Urbanetti et al.

5. 6. 7. 8. 9. 10.

https://en.wikipedia.org/wiki/ASLERD. Accessed Apr 2022 http://slerd.uniroma2.it/. Accessed Apr 2022 https://www.wise-qatar.org/wise-learning-ecosystems-living-lab/. Accessed Apr 2022 https://www.parlamento.it/parlam/leggi/97059l.htm (in Italian). Accessed Apr 2022 https://snv.pubblica.istruzione.it/snv-portale-web/public/scuole/rav. Accessed Apr 2022 Giovannella C.: Participatory bottom-up self-evaluation of schools’ smartness: an Italian case study. Interact. Design Architect. J. N. 31, pp. 9–18 (2016) Giovannella C.: The ASLERD Pyramid of Smartness: A Study on the Stability of Indices and Indicators, in Schools in Project and Design Literacy as Cornerstones of Smart Education, Springer, pp. 81–91 (2020) Meahla O., Giovannella C., Delgado F.: School Smartness augmented by educational community members: a pilot contribution from K9 students. In: D’Andrea Fabio, Baldi, V. (eds.) Codice e luoghi. Abitare le relazioni nel reale/digitale, Roma, Meltemi Editore, pp. 143–164 (2019) La Buona Scuola (2014). https://labuonascuola.gov.it/documenti/La%20Buona%20Scuola.pdf (in Italian). Accessed Apr 2022 Vuorikari R., Kluzer S., Punie Y.: DigComp 2.2: The Digital Competence Framework for Citizens—With New Examples of Knowledge, Skills and Attitudes. https://publications.jrc.ec. europa.eu/repository/bitstream/JRC128415/JRC128415_01.pdf, Accessed Apr 2022 Giovannella, C.: An Analysis of Alternation Schemes to Increase Student Employability and the Smartness of Secondary Schools, in Ludic, pp. 39–51. Springer, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education (2021) Giovannella, C.: Smart alternation schemes and design practices during pandemics, in Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education. Smart Innovation, Systems and Technologies, Vol. 249. Springer, pp. 3–14 (2022) https://www.miur.gov.it/documents/20182/2467413/Le+linee+guida.pdf/4e4bb411-1f909502-f01e-d8841a949429 (in Italian). Accessed Apr 2022 https://www.liceo-amaldi.edu.it/index.php/patto-educativo-di-comunita-scuole-in-comunegeneriamo-il-cambiamento Giovannella C.: Smartness as complex emergent property of a process. The case of learning eco-systems. ICWOAL 2014, IEEE Publisher, pp. 1–5 (2014) Giovannella C., Andone D., Dascalu M., Popescu E., Rehm M., Roccasalva G.: Evaluating the Resilience of the Bottom-up Method used to Detect and Benchmark the Smartness of University Campuses. ICS2 2016, IEEE publisher, pp. 341–345 (2016) Fuster M., Polchar J., Burns T.: Back to the Future of Education. Four OECD Scenarios for Schooling (2020). https://espas.secure.europarl.europa.eu/orbis/sites/default/files/genera ted/document/en/178ef527-en.pdf. Accessed Apr 2022 https://www.senato.it/istituzione/la-costituzione/parte-ii/titolo-v/articolo-118 (in Italian). Accessed Apr 2022 http://www.local2030.org/library/403/Learning-Cities-and-the-SDGs-A-Guide-to-Action. pdf. Accessed Apr 2022 https://uil.unesco.org/lifelong-learning/learning-cities. Accessed Apr 2022 Chatzichristou S., Ulicna D., Murphy I., Curth A.: Dual Education: A Bridge over Troubled Water, EU DG for Internal Polices—Culture and Education (2014). http://www.europarl. europa.eu/RegData/etudes/STUD/2014/529072/IPOL_STU(2014)529072_EN.pdf. Accessed Apr 2022 https://www.labsus.org/2022/02/piccole-scuole-patti-e-comunita/ (in Italian). Accessed Apr 2022 https://www.forumdisuguaglianzediversita.org/patti-educativi-territoriali-e-percorsi-abilit anti-unindagine-esplorativa/ (in Italian). Accessed Apr 2022

11.

12.

13. 14.

15.

16.

17. 18. 19. 20.

21.

22. 23. 24. 25.

26. 27.

Chapter 9

Digital Games as Tools of Innovative Pedagogy in Education Maneesh Dubey

and Kunal Sinha

Abstract There are several obstacles that conventional education institutions face in today’s information age to improving students’ learning. Technology may be the catalyst for educational transformation by providing unique potential for constructing successful learning environments. Innovative teaching practices can be developed and implemented using digital games as a technology enabler. Learners acquire and create knowledge through game-based learning in a pleasant and focused learning environment. Therefore, Digital games are used to teach students about a particular subject. However, only a few researchers have explored the effect of these strategies on the development of higher-order cognitive skills and emotional and motivational outcomes. As a result of the lack of any adequate evaluation methods based on gaming, more research into the educational benefits of digital games is required. Therefore, this article provides an overview of digital game-based learning, emphasising its use in higher education, such as in colleges and universities. In conclusion, integrating the digital game into conventional pedagogy in higher education improves students’ learning outcomes. The secondary literature has been used to compile the findings for this research.

9.1 Introduction In recent years, digital game has grown increasingly popular in education. In addition, it allows students to study in a multi-sensory, active, and experiential setting, among other advantages. This (Simulation games, Role-playing games, Strategy M. Dubey (B) Research Scholar, Centre for Studies in Science, Technology and Innovation Policy (CSSTIP), School of Social Science (SSS), Central University of Gujarat, Gandhinagar, India e-mail: [email protected] K. Sinha Assistant Professor, Centre for Studies in Science, Technology and Innovation Policy (CSSTIP), School of Social Science (SSS), Central University of Gujarat, Gandhinagar, India e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_9

129

130

M. Dubey and K. Sinha

games, Adventure games:) type of game-based learning can help students practice critical thinking, problem-solving, and different critical thinking abilities in a fun and engaging way [1]. It’s also possible to acquire quick feedback from students rather than relying on traditional assessment methods that take a long time to provide results (e.g. tests and examinations). As a result, certain educational gamification may help to minimise limits, including time and location, as portable gadgets allow students to study and learn at any time and in any place. These easy-to-use tools can help students learn and retain challenging concepts [2]. Furthermore, the learning process is regarded to be more exciting [3], motivating, achieving knowledge retention and boosting attention. It can even enhance peer communication and social skills through educational games [4, 5]. The advancement of digital technology has changed how individuals communicate, access information, conduct daily tasks, and gain knowledge. As a re-sult of recent technological developments, people from all over the world may now access a wide range of digital resources at any time, create and share their content, and communicate with one another over a long distance. Many socioec-onomic elements, such as globalisation and the digital economy, have changed today’s societies. Several new difficulties confronting education must be ad-dressed by implementing the necessary modifications. We require these changes to penetrate the educational services offered by traditional educational facilities and focus on transforming educational services and current pedagogical practices [6, 7]. Educators have access to a wide range of technological tools that can be utilised to implement pedagogical innovations that can either be integrated into the current curriculum or serve as the basis for generating new ones. A digital game is the focus of this article since it has the potential to aid in educational reform. This paper discusses issues related to learning efficacy, explains current research programmes, and recommends new investigations based on a critical appraisal of current scientific understanding based on secondary literature from 2000 to 2020.

9.2 Digital Games as Innovative Educational Tools Digital games include predetermined rules and constraints that guide the player toward a specific goal. In addition, these games constantly interact with the player and provide feedback through scoring or modifications to the game world. Digital games are software applications and innovative educational tools used in education that combine the features of video games and computer games (action games, adventure games, puzzle games, role-playing games, simulation games, and strategy games). They want to create engaging educational opportunities that successfully correlate to particular learning goals and outcomes. The digital game as an innovative educational tool also considers the desire and pleasure of the students to play while promoting the development of logical reasoning, learning ability and skills. Digital game-based learning, a novel area of study in innovative educational technology-enhanced learning, has drawn considerable interest from researchers

9 Digital Games as Tools of Innovative Pedagogy in Education

131

and educators alike [8, 9]. It has been defined as adding instructional information to computer games to obtain the same or better results as standard teaching methods [10]. Reference [11] “Using a computer games-based technique to deliver, assist, and improve teaching, learning and evaluation” is another definition of digital game-based learning. In 2009, Chen and Wang emphasised strengthening information construction processes. They characterised digital game-based learning as “an excellent innovative education tool to assist learners in creating understanding by playing, sustain stronger inspiration and apply the obtained knowledge to real-life issues” [11, 12]. The growing fame of digital games and the gaming business has prompted academic attention to ways and methods of using digital games as innovative educational technology in teaching–learning tools. Digital games have a variety of structural properties that separate them from other forms of play and make them both motivating and fascinating. More significantly, digital games feature rules and goals, complex narrative components and storey boards, challenge players, allow for interaction (both player-to-player and player-to-interface), and provide “just-in-time” or “on-demand” feedback. Innovative educational tools-based learning study explored the possible application of digital games’ in higher education [13, 14]. On the other hand, current research is focused on developing meaningful digital game-based learning activities that meet specified general, topical, or cross-domain learning objectives.

9.3 Digital Learning Games: Interactive Learning or Edutainment Currently, there has been an emphasis on creating educational video games for computer and mobile devices. Owing to the need to capitalise on the motivation that commercial-off-the-shelf digital games provide and their ability to aid learning via doing [15]. The first generation of educational games, which are sometimes referred to as “edutainment,” were developed based on the assumption that by incorporating educational content into a game-like scenario, learning would be more enjoyable and hence more effective than traditional instructional approaches. This initial wave of educational games did not, however, have the desired influence on user engagement and learning facilitation because of the following: • In terms of audio and graphical quality and the obstacles given in the game world, they are far less complex than commercial digital games. • Their inability to keep players’ motivation and interest high, despite seeing gaming assignments as “a bit easier to stomach form of drill-and-practice learning”. • The confined range and poor design of the shown activities could not help students learn higher-order cognitive skills.

132

M. Dubey and K. Sinha

In 2010 Whitton claimed that commercially available digital games are developed mainly for pleasure, with tasks and difficulties perfectly aligned with the game’s goals. For example, “The Sims” is a widely used commercial computer game. On the developer’s website, the game replicates life in a small town, allowing users to deal with various social concerns by creating and customising their virtual characters, making decisions for their lives, and helping them achieve their goals. In contrast, most instructional digital games do not match gaming activities with targeted learning objectives, with the latter being provided as a reward for completing game-based learning assignments [9]. For example, in "Math Blaster," the user is faced with easy math problems and must jump up to platforms to catch the proper solution that appears in the sky. The rise of the “serious games” movement can be seen as an attempt to address the shortcomings of the first generation of educational digital games. Creating games that provide users with authentic and meaningful learning environments that foster higher-order cognitive skills and domain-specific knowledge [16]. Michael and Chen said that Seriously games are games “designed to educate rather than entertain. ” The LUDUS project (http://www.ludus-project.eu) website lists several areas of interest for serious games, including education, politics, engineering, city planning, and health care. For example, the “Squire’s Quest” game (http://www.bcm.edu/cnrc/ consumer/archives/videogames.htm) was developed to help young people develop healthy eating habits, while the “Re-Mission” (http://www.re-mission.net) game was developed to help teenagers and young adults with cancer [16, 17]. Young et al. [18] select 39 publications that match the inclusion criteria relating to video games and academic attainment, focusing on using traditional games vs video games for instructional reasons. The studies are organised by subjects, such as History, Mathematics, Physical Education, Science, and Languages. Findings show that there is little evidence to support the use of educational games in regular classrooms, which contrasts with the conclusions of the previous studies. Researchers Smetana and Bell [19] examined computer simulations in science education. Using a side-by-by-side comparison, the researchers found that computers can be just as, if not more effective, than conventional games at fostering knowledge acquisition, skill development, and conceptual growth. Tsekleves highlighted in 2014 the advantages and disadvantages of incorporating serious games into classrooms across all grade levels and subject areas. Achievement and rewards, interactivity, feedback, motivation, playfulness and problem-based learning, collaborative learning, progression and repetition, and realism and immersion are listed as benefits by the writers. For those who are interested in implementing serious games in education, they offer some pointers. The same may be said regarding Bellotti’s at. all. in 2013 rules for evaluating the performance of serious games. Serious games can be evaluated for their educational value based on user feedback. This study’s findings demonstrate the importance of serious games in motivating students to meet their academic targets while also emphasising the necessity of giving them helpful feedback. It also shows that new game types are most effectively implemented when they are used under the supervision of an experienced

9 Digital Games as Tools of Innovative Pedagogy in Education

133

teacher. Furthermore, they emphasise essential factors, such as adaptability, personalisation, and satisfying needs on a one-to-one basis (e.g. learning styles, information provision rates, feedback, etc.).

9.4 Active Learning Through Digital Games Digital games as “software applications that can be performed using suitable technological infrastructure”, it takes a model that focuses more on the gaming activity aspect. It thus defines (digital) games as “rule-based formal sandboxes.” In Juul’s definition of (digital) games, six distinguishing traits can be described in the following way [20]. 1. Regulation based digital games 2. Variable and quantifiable results are a hallmark of digital games (due to the player-provided feedback). 3. Achieved outcomes can be valued positively or negatively by players. 4. The players are working hard to get the desired results. 5. As a result, players get emotionally invested in their performance. 6. Players’ actions have defined outcomes, which are not always the same as each game is played. Similarly, in 2008 Klopfer describes digital games as “purposeful, goal-oriented, rule-based activities that the players perceive as fun”. Where rules are sets of instructions embedded in the game’s design that define legitimate actions, and goals or outcomes determine the reasons for participating in gameplay [21]. When students play a digital game, student players are often faced with a variety of challenging and vague difficulties [9], which must be solved by formulating plans of action and executing those plans [22]. Since issue solving necessitates applying and developing a wide range of higher-order cognitive skills connected to problem-solving, there is the opportunity to apply and improve these skills [9]. In 2007 Gee argues that digital games might build problem-solving abilities by proposing a four-step process in which players actively interact when confronting in-game obstacles. Specifically, the "probe, theorise, reprove, rethink cycle" process incorporates the following steps by players: 1. The user is encouraged to "probe the virtual world" while exploring the game’s virtual environment. 2. During the game’s exploration of the virtual environment, the player forms a hypothesis based on what they have learned from their actions. 3. To test the hypothesis, the player re-explores the game world and engages in an active exploration of the environment. 4. The player “accepts or rethinks his or her original premise” and repeats the above-described sequence of activities based on the feedback supplied [23].

134

M. Dubey and K. Sinha

In 2002 Garris, Ahlers, and Driskell suggested the “Input-Process-Output Game Model” of utilising digital games to fulfil (officially specified) learning objecti/////ves. Learning content and game characteristics are critical to a successful game-based learning activity’s design, and the model’s input, output, and process phases revolve around these aspects. It is a cyclical process that includes the user’s engagement in making judgments about the virtual world of the game (“user judgements”), as well as the generation of output. The specific actions were taken by the user, as evidenced by their behaviours. With the help of “system feedback,” the refining of judgments and activities can be achieved. Kolb’s four-phase experiential learning model is consistent with the “probe, hypothesise, reprove, rethink cycle”. Therefore the “Input-ProcessOutput Game Model” describes how perceived learning experiences can trigger a four-phase circular process (namely, “concrete experience,” “reflective observation,” “abstract conceptualisation,” and “active experimentation”) [24, 25]. Virtual characters can adopt identities and explore the virtual world of the game. Interact with virtual objects to discover meanings embedded in them and communicate and negotiate with other virtual characters. Investigate cause and effect relations, resolve conflicts, search for relevant information, and make decisions with repercussions. In addition, virtual environments, such as those, provided opportunities for learning through trial-and-error procedures [9]. Thus, players can create meanings for the virtual world by integrating their experiences into existing knowledge schemas or constructing new ones to address the “cognitive disequilibrium” they feel [13, 26]. In addition, digital gaming is a social activity that requires players to work together to overcome hurdles to overcome in a virtual world in the game, with players engaged in joint and coordinated activities [9]. Through digital games, which have a goaloriented and rule-based character, players can participate actively in exploration and experimentation as part of collaborative activity. Furthermore, they can mediate and support one other’s actions. As a result, in addition to the actions and interactions that occur during the gaming activity. There is also a network of actions and interactions outside of the gaming activity, including reading books and magazines about games, visiting relevant websites, and posting comments on forums. Systems of game-related activities can be regarded as “affinity spaces” characterised by participants’ commitment to a joint endeavour in an interconnected network of tools and human actors. These digital games can assist learning communities, practice communities, and identity communities [27].

9.5 Encouraging Students Learning Through the Use of Digital Game Digital games in higher education learning environments have garnered much study attention due to their accessibility and educational capabilities. However, it is only by considering various academic levels’ general goals and objectives. Therefore, the consequences that these goals might have concerning the exploitation of digital games

9 Digital Games as Tools of Innovative Pedagogy in Education

135

are that we can fully understand the usage of digital games in higher educational contexts and their potential to aid the attainment of standard curriculum learning objectives [28]. Higher education institutions should provide education to assist students in becoming engaged and responsible citizens. Therefore, career-specific education helps students acquire domain-specific knowledge and skills. As a result, education focuses on developing essential skills such as reading and numeracy. The educational abilities needed for personal fulfilment and growth even while both levels of education aim to build higher-order cognitive skills and competencies. However, there are substantial distinctions in the subject-specific educational objectives that are supposed to be achieved. Moreover, several variations characterise the education provided by institutions (e.g. available infrastructure, time limits, usage of evaluation techniques) that might have a significant impact on the integration of digital games into daily educational activities [29]. Suppose digital games are used to enhance lower-level cognitive abilities connected to subject-specific educational objectives. In that case, there must be a direct alignment between the game’s content and the content meant to be provided. Another important argument for adopting digital games in the classroom is that they can increase student motivation [9]. However, due to time limits and a focus on testbased assessment, the sorts of games that can be employed are severely restricted. However, the type of learning outcomes sought in higher education. The need for direct relevance between the game and real-world applications and greater flexibility in assessment allows for the use of more “sophisticated” digital games, which have the potential to enhance the acquisition of higher-order cognitive skills [30]. We must underline that a wide range of learning outcomes may be achieved using the proper game genres. There have been efforts to create typologies of digital games-based on the learning outcomes they provide. There is a type of typology proposed by Dondi and Moretti in 2007: the first type of typology, drill-and-practice exercises in quizzes and puzzles, can aid in learning factual knowledge. The second type of typology, sports games and action games can be used to apply concepts and rules that are previously learned. The third type of typology strategy, role-playing, and simulation games are regarded as appropriate for problem-solving and decision-making abilities. Through technique, role-playing, and simulation games, engaging in social interactions and developing ethical principles can be facilitated through strategy, role-playing, and simulation games [31]. This is why we have concentrated on recent studies on game-based learning published in the previous decade and are concerned with examining possible effects on both subject-specific and general educational goals and learning outcomes. In addition, we’ll go over some of the things that experts say are crucial to the success of game-based learning. Digital games can have a significant impact on the achievement of standard curricula educational objectives and learners’ motivation [32], as well as on the motivation of students [33]. Researchers used educational games in many of these studies, such as drill-and-practice activities and specially designed educational games which primarily target the acquisition of factual knowledge. Game-supported

136

M. Dubey and K. Sinha

learning activities can at least equal non-gaming techniques’ effectiveness in accomplishing officially defined, subject-specific educational objectives. Despite this, relatively little research has been done to examine the possible impacts of digital games on the development of higher-order cognitive skills and identify specific game affordances or learn design aspects that might significantly contribute to this direction [34]. We utilise “Achievements” to assess a videogame’s soft skill training. Following the notion that when we play a video game, we conduct a set of activities according to its rules, guidelines, and objectives, it is necessary to develop abilities connected to such actions and regulations. Achievement is the record of successful activities. We can measure skill training by assessing the skills needed to accomplish a sequence of steps ‘packaged’ in an Achievement and offering quantitative values based on acquisition complexity. Janette Vazquez et al. in 2022 also looked at how well biometric data could predict a video game’s preference and level of engagement compared to survey data. We utilised our algorithms to classify the success of three video games, collecting survey and biometric data from participants during and after playtime. Our results showed that biometric data did a better job of suggesting the level of engagement of a video game than survey data, with a Random Forests prediction model performing best for both datasets with an accuracy of 74.67% for the biometric data. These results imply that biometric data is a superior indicator of engagement. Additional investigation of this approach to measuring gaming input should continue, especially if corporations want to design games that engage consumers and keep them playing for more extended periods [35]. The study shows that active involvement in long-lasting learning activities, fully supported by specially designed educational puzzle games, can facilitate the development of strategic thinking and reasoning skills, which are directly reliant on solving complex problems. Furthermore, playing games can help develop these skills by providing feedback, increasing difficulty gradually, backtracking functionalities (the software application’s ability to retrace performed actions in the game), and functions related to providing users with tips during gameplay [36]. Clark et al. [37] examine papers to study the effects of digital games on learning outcomes, concluding that games encourage productive learning and stressing the relevance of gaming design beyond its medium. However, in a similar vein, Backlund and Hendrix (2013), in their meta-analysis, indicated excellent learning effects when employing serious games in the educational process before this review. Additionally, Wouters (2013) used comparisons to determine whether serious games are more effective and inspiring than standard educational methods. Compared to traditional teaching approaches, they observed a higher level of learning and retention but a lower-level of desire. On the other hand, serious games are more effective when used to supplement different teaching approaches and when students participate in groups and several training sessions [38]. But in addition to ensuring that in-game learning experiences assist the acquisition of domain-specific information and higher-order cognitive abilities, learners must be engaged in debriefing and reflection activities as part of any game-based learning

9 Digital Games as Tools of Innovative Pedagogy in Education

137

method. The specified learning outcomes will not be met if students are not involved in debriefing and reflection activities unless provided with scaffolding and assistance. To maximise the educational value of the digital game, students must participate in the essential debriefing and reflection exercises [39]. The importance of implementing appropriately designed game-based learning activities to support the development of subject-specific and general educational objectives. Authors say it is possible to achieve higher-order cognitive skills aligned with Bloom’s taxonomy of educational objectives by involving learners in describing intended actions and anticipated results and justifying experiences based on provided feedback [33]. This section continues our discussion of the challenges raised by research into digital games in elementary and higher education, examining efforts to use such games in higher education. A digital instructional game called “Marketplace” (http:// www.marketplace-simulation.com/) was utilised in a marketing course at Manchester Metropolitan University in the United Kingdom. Students had to work together to create virtual organisations, do market research, devise marketing strategies, and design suitable goods for development. In addition, students had to “compete against one another for market share and position” in a competitive climate encouraged by the game’s design. Students’ work was evaluated by having them give presentations, fill out worksheets with information about their decisions, and complete individual assignments following the conclusion of the game-based learning activities. Using both qualitative and quantitative methodologies, students reported that they were given a chance to apply theoretical concepts to a real-world issue and received quick feedback on their performed activities [40]. However, the input provided was limited and “did not explain clearly why actions had led to particular effects,” according to their account. For students at the same university, a web-based educational digital game (the “Retail Game”) was used to familiarise them with the decisions and actions they must take to run a retail outlet business, as outlined in this paper. Students were allowed to take on different responsibilities, work with data about the business’s current state and market demands, and make management decisions for their store. And then justify those judgments. “knowledge of marketing principles and retail operational challenges,” as well as the opportunity to practise interpersonal skills, were provided by this method [9]. “PeaceMaker,” a commercial computer game, was utilised in this context to help students analyse “the relationship between ethical problems and international politics”. As a result of presentations and reflection exercises, students were shown to have “emotional responses to the digital game that might help them of the challenges in their region. There was a more substantial knowledge of the situation at hand after implementing the game-supported learning activities, as evidenced by discussions following the implementation of the game-supported learning activities [9].

138

M. Dubey and K. Sinha

9.6 Discussion Research shows that using digital games in higher education provides a unique opportunity to build higher-order cognitive skills essential for workplace success. Consequently, we must apply rigorous evaluation methodologies to evaluate the development of higher-order cognitive abilities while using digital games in the classroom. As a result of reviewing the current studies, we can derive that assessments are almost always based on subjective rather than objective metrics in most circumstances. While standardised examinations can be used to examine the attainment of specific educational objectives, the extent to which higher-order cognitive skills have been developed is mostly assessed by gauging the perceptions of those engaged (i.e., instructors and students). The success of game-based educational systems can be evaluated by providing proof of the perceived degree of learning, but this should not be the sole metric used. Another method is to use artefacts (such as presentations, written or oral reports, and portfolios) that students create as part of their participation in game-based learning activities to evaluate how well they develop higher-order cognitive skills. Evaluation of these artefacts provides helpful insights into the consequences of these actions and hence includes evidence for the learning that occurred. There is a tendency in most research projects to assign arbitrary values to the outputs of their efforts, which necessitates the establishment of objective criteria for evaluating these outputs. The use of digital games as instruments for measuring the development of higherorder cognitive skills has not been effectively addressed in the context of already done research and needs to be emphasised. It was previously said that digital games provide their users with authentic and meaningful environments in which they may acquire a wide range of abilities. As a result, digital games might be used to teach and test a wide range of capabilities that students need to succeed in the real-world. Such an approach, however, requires well-defined criteria that may provide us with credible proof for the learning efficacy of digital games, and at this point, there is a lack of research efforts in this area.

9.7 Conclusion and Future Perspective Digital games may be used as learning aids that can promote the acquisition of particular knowledge and the development of more advanced cognitive abilities. However, research on the effects of digital games on the development of higher-order cognitive skills and competencies has thus far been limited. Further evidence of the efficiency of digital games in achieving this goal is required. According to the European Commission, entrepreneurship1 is a vital competency for lifelong learning, personal and professional growth and fulfilment, and social inclusion and active citizenship. Developing an entrepreneurial culture among young people can be facilitated by using digital games, as demonstrated by research that highlights the potential of

9 Digital Games as Tools of Innovative Pedagogy in Education

139

digital games for the achievement of learning objectives related to entrepreneurship education. A growing body of evidence from studies undertaken at college and university levels shows that digital games can help students develop higher-order cognitive skills. However, the absence of rigorous evaluation methodologies necessitates further research in this area. Since digital games may be used to mediate student interactions by including specific meanings in their design, their efficacy can be assessed by drawing on performed interactions and their results as proven by displayed behaviours and articulated by generated artefacts. As a result, digital games as assessment tools in digital game-based learning research should also gain traction. Furthermore, due to their authenticity and meaningfulness, digital games may give their users environments that can be used to test their knowledge and abilities and evaluate them. As a result, digital games will no longer be regarded as “black boxes”, and the success of their educational value may be evaluated in context across a wide range of learning areas. This study contributes to research on the digital game as innovative educational technology tools such as educational and digital games in higher education. Because, in today’s world, the requirement for student-centred teaching approaches necessitates the implementation of game-based activities and simulations that will equip students to meet the difficulties of the new era as it comes into being.

References 1. Adachi, P.J., Willoughby, T.: More than just fun and games: the longitudinal relationships between strategic video games, self-reported problem-solving skills, and academic grades. J. Youth Adolescence 42, 1041–1052 (2013). https://doi.org/10.1007/s10964-013-9913-9 2. Hanus, M.D., Fox, J.: Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput. Educ. 80, 152–161 (2015). https://doi.org/10.1016/j.compedu.2014.08.019 3. Calliari, D.: Using games to make learning fun. Rehabilitation Nursing J. 16, 154–155 (1991). https://doi.org/10.1002/j.2048-7940.1991.tb01202.x 4. Sun-Lin, H.Z., Chiou, G.F.: Effects of gamified comparison on sixth graders’ algebra word problem solving and learning attitude. Educ. Technol. Soc. 22, 120–130 (2019) 5. Liao, C.C., Chen, Z.H., Cheng, H. N., Chen, F. C., Chan, T.W. My-Mini-Pet: A handheld pet-nurturing game to engage students in arithmetic practices. J. Comput. Assisted Learn. 27, 76–89. https://doi.org/10.1111/j.1365-2729.2010.00367.x (2011) 6. Becta: The Impact of Digital Technology: A Review of the Evidence of the Impact of Digital Technologies on Formal Education. Accessed from (2009). 7. Kearney, M. L.: Higher education, research and innovation: Charting the course of the changing dynamics of the knowledge society. In: Meek, V.L., Teichler, U., Kearney Higher Education, Research and Innovation: Changing Dynamics, pp. 7–23. Kassel: Unesco Forum on Higher Education, Research and Knowledge (2009) 8. Chen, Z. H., Chan, T. W.: Using game quests to incorporate learning tasks within a vir- tual world. In: Proceedings of 10th IEEE International Conference on Advanced Learning Technologies 2010, pp. 750–552. Sousse, Tunisia: 10th IEEE International Conference Sousse,Tunisia (2010)

140

M. Dubey and K. Sinha

9. Whitton, N.: learning with Digital Games: A Practical Guide to Engaging Students in Higher Education. Routledge, New York (2010) 10. Prensky, M.: Digital Game-Based Learning. Minnesota: Paragon House (2007) 11. Chen, M. P., Wang, L. C.: The effects of types of interactivity in experimental game- based learning. In: Proceedings of 4th International Conference on eLearning and Games, Edutainment 2009, pp. 273–283. Banff, Canada: Proceedings of 4th International Conference on eLearning and Games, Edutainment (2009) 12. Ahmad, M., Rahim, L.A., Arshad, N.I.: An analysis of educational games design frameworks from software engineering perspectiv. J. Inf. Commun. Technol 14, 123–151 (2015) 13. Van Eck, R.: Building artificially intelligent learning game. In: Gibson, D., Aldrich, C., Prensky, M.: Games and simulations in online learning, pp. 271–307. Hershey: Information Science Publishing (2007) 14. Aslan, S., Balci, O.: GAMED: digital educational game development methodology. Simulation 91, 307–319 (2015) 15. Hense, J., Mandl, H.: Learning in or with games? Quality criteria for digital learning games from the perspectives of learning, emotion, and motivation theory. In: Proceedings of IADIS International Conference on Cognition and Exploratory Learning in Digital Age. Madrid, Spain: Proceedings of IADIS International Conference on Cognition and Exploratory Learning in Digital Age (2012) 16. Aguilar, J., Altamiranda, J., Diaz, F.: Specification of a managing agent of emergent serious games for a smart classroom. IEEE Lat. Am. Trans 18, 51–58 (2020) 17. Michael, D. R., Chen, S. L.: Serious Games: Games that Educate, Train and Inform. Cincinnati: OH: Muska & Lipman/Premier-Trade (2006) 18. Williamson Shaffe, D.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006) 19. Smetana, K.L., Bell, L.R.: Computer simulations to support science Instruction and Learning: a critical review of the literature. Int. J. Sci. Educa. 34, 1337–1370 (2012). https://doi.org/10. 1080/09500693.2011.605182 20. Juul, J.: The game, the player, the world: Looking for a heart of gameness. In: Proceedings of Level Up: Digital Games Research Conference, pp. 30–45. Utrecht, The Netherlands: Proceedings of Level Up: Digital Games Research Conference (2003) 21. Klopfer, E.: Augmented Learning: Research and Design of Mobile Educational Games. Cambridge: MA: MIT Press (2008) 22. Belova, N., Zowada, C.: Innovating higher education via game-based learning on misconceptions. Educ. Sci 10, 221 (2020) 23. Gee, J.P.: What Video Games Have to Teach us About Learning and Literacy. Palgrave McMillan, New York (2007) 24. Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice mode. Simul. Gaming 33(4), 441–467 (2002) 25. Byun, J., Joung, E.: Digital game-based learning for K–12 mathematics education: a metaanalysis. Sch. Sci. Math 118, 113–126 (2018) 26. Denham, A.R.: Using the PCaRD digital game-based learning model of instruction in the middle school mathematics classroom: a case study. Br. J. Educ. Technol 50, 415–427 (2019) 27. Sedano, C.I., Sutinen, E., Vinni, M., Laine, T.H.: Designing hypercontextualized games: a case study with LieksaMyst. Educ. Technol. Soc. 15(2), 257–270 (2012) 28. Denham, A.R., Mayben, R., Boman, T.: Integrating game-based learning initiative: increasing the usage of game-based learning within K-12 classrooms through professional learning groups. TechTrend 60, 70–76 (2016) 29. Hawlitschek, A., Joeckel, S.: Increasing the effectiveness of digital educational games: the effects of a learning instruction on students’ learning, motivation and cognitive load. Comput. Hum. Behav 72, 79–86 (2017) 30. Jagušt, T., Botièki, I., So, H.J.: Examining competitive, collaborative and adaptive gamification in young learners’ math learning. Comput. Educ 125, 444–457 (2018)

9 Digital Games as Tools of Innovative Pedagogy in Education

141

31. Dondi, C., Moretti, M.: A methodological proposal for learning games selection and quality assessment. Br. J. Edu. Technol. 38(3), 502–512 (2007) 32. Annetta, L.A., Minogue, J., Holmes, S.Y., Cheng, M.T.: Investigating the impact of video games on high school students’ engagement and learning about genetics. Comput. Educ. 53(1), 74–85 (2009) 33. Panoutsopoulos, I., Sampson, D.: A study on exploiting commercial digital games into school context. Educational Technology and Society Journal 15(1), 15–27 (2012) 34. Karakoc, B., Eryılmaz, K., Özpolat, E.T., Yıldırım, I.: The effect of game-based learning on student achievement: a meta-analysis study. Technol. Knowl Learn 1, 1–16 (2020) 35. Vazquez, J., Abdelrahman, S., Wasden, C., Jardine, S., Judd, C., Davis, M., Facelli, J.C.: Using Biometric Data to Measure and Predict Emotional Engagement of Video Games. bioRxiv, 01–25 (2022) 36. Santos, A. F., Alloza, S., Escribano, F.: Relationship between video game genres and soft skills. Retrieved from gecon.es: https://gecon.es/game-genres-and-soft-skills, (2018, 04 11). 37. Tsekleves, E., Cosmas, J., Aggoun, A.: Benefits, barriers and guideline recommendations for the implementation of serious games in education for stakeholders and policymakers. Br. J. Edu. Technol. 47(1), 164–183 (2014). https://doi.org/10.1111/bjet.12223 38. Clark, D.B., Tanner-Smith, E.E., Killingsworth, S.S.: Digital games, design, and learning a systematic review and meta-analysis. Rev. Educat. Res. 86(1), 79–122 0034654315582065 (2015) 39. Law, E.L., Sun, X.: Evaluating user experience of adaptive digital educational games with Activity Theory. Int. J. Hum. Comput. Stud 70, 478–497 (2012) 40. Marklund, B.B., Taylor, A.S.: Educational Games in Practice: the challenges involved in conducting a game-based curriculum. Elect. J. Elearn 14, 122–135 (2016)

Chapter 10

Environment Challenges of E-Learning in Higher Education—The Teachers’ Perspective Janika Leoste , Larissa Jõgi , Tiia Õun , Ugljesa Marjanovic , Slavko Rakic , Simone Schöndorfer, and Zoe Lefkofridi Abstract The COVID-19 pandemic enforced higher education institutions worldwide to start using various digital technologies for providing synchronous digital online teaching and learning support. The new e-learning environment became a main challenge for the teachers, students, and university management. This paper is based on a study that aims to investigate the main challenges of digital teaching (a teacher-centered aspect of e-learning) from the teachers’ perspective. The data for this research is gathered through the ENLIVEN Video Podcasts, using the method of semi-structured interviews, in different locations of the Paris Lodron University of Salzburg. Qualitative analysis was used for data analysis. The results of this study provide five common themes which reflect teachers’ challenges: personal barriers to digital teaching, structural barriers to digital teaching, challenging factors to ensure the quality of teaching, recommendations for colleagues, and future vision. Furthermore, this paper presents a collective story of teachers’ experiences related to digital teaching practices and gives an overview of the major challenges when teaching via an e-learning environment.

10.1 Introduction and Theoretical Background This paper is based on the study that was conducted in the context of a larger project, called ENLIVEN. ENLIVEN, which stands for “ENhanced Learning and teaching in International V irtual ENvironments” is funded by an Erasmus + program J. Leoste (B) · L. Jõgi · T. Õun School of Educational Sciences, Tallinn University, Narva rd 25, 10120 Tallinn, Estonia e-mail: [email protected] U. Marjanovic · S. Rakic Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovica 6, 21000 Novi Sad, Serbia S. Schöndorfer · Z. Lefkofridi Department of Communication Studies, Paris Lodron University of Salzburg, Rudolfskai 42, 5020 Salzburg, Austria © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_10

143

144

J. Leoste et al.

(2021–2023) and aims at responding to the educational challenges resulting from the COVID-19 pandemic. Within this project, teachers across Europe share their own and their universities’ practices as well as findings of related research. One of the ENLIVEN partners, the Paris Lodron University of Salzburg (PLUS), produced a video about digital learning (students’ perspective) as well as a series of video podcasts about digital teaching (teachers’ perspective) in January 2022. The COVID-era [1] enforced educational systems worldwide to start using various digital technologies for providing synchronous teaching and learning in conditions when in-person lessons were forbidden [2, 3]. One of the major challenges was reaching out to and engaging all learners, regardless of their specific needs and circumstances [4]. The situation matched the idea of French historian Fernand Braudel, one of the founders of world-systems theory, according to which people start adopting new technologies only when their imagination about what is possible exceeds the limits of their every day (technological) structures [5, 6]. If novel technologies are able to push the limits of everyday structures by enabling things that were previously considered impossible then these technologies will be sustained [5]. Research indicates that appropriately adopted digital technologies can improve teaching and learning processes [4, 7], improve student learning engagement, or develop student employability skills such as problem-solving, collaboration, critical thinking, and creativity [8]. In the educational context, the following four technological areas can be distinguished [7]: (a) redefining teaching and learning; (b) schooling materials and related resources, such as computers or tablets; (c) shaping teachers’ technology-related knowledge, skills, and attitudes; and (d) improving the classroom learning environment. According to the Braudel Rule, the digital technologies for synchronous online teaching and learning, often known as e-learning, will only be sustained if they are able to continue benefiting the educational stakeholders after the COVID-era restrictions are lifted. The previous studies indicate that of the various COVID-era digital teaching and learning technologies, blended learning (integrating technology with traditional classroom activities) continues to be by far the most common delivery model across higher education institutions [9]. The main request for the sustained digital transformation of the e-learning process is that the management of the higher education institutions understands the value of e-learning and support transformation from traditional to digital education [10]. The situation before the COVID era shows that a major challenge for every lecturer was to understand whether student learning outcomes depend on the frequency of digital resources used at the e-learning platform [11, 12]. Additionally, the situation during the COVID era shows that students ready for digital learning have better performance at the e-learning platform [13]. Accordingly, one of the major challenges for teachers during the e-learning process is increasing student engagement at the e-learning platform [14]. However, the OECD Teaching and Learning International Survey showed that only 39% of educators in the EU felt well or very well prepared for using digital technologies in their daily work, with significant differences between the EU countries [15]. Moreover, previous research shows that the main critical issues in the transformation from traditional to digital learning from a teacher’s perspective are (a) difficulty in increasing the sense of belongingness through distance learning gained

10 Environment Challenges of E-Learning in Higher Education …

145

by participation, empathy, and effective communication; (b) difficulties in delivering "practical" subjects or laboratories through distance teaching and learning; (c) digital divide/IT issues; and (d) increased stress and physical fatigue [13]. Research also shows that 70% of teachers did not use any of the tools for student’s engagement [16]. In addition, the previous research suggests that the lack of resources and the lack of support from the university management results in a lack of motivation for teachers to use e-learning tools [9]. Nonetheless, results from a previous study demonstrate the growth of student satisfaction when the innovative tools of e-learning are used [14]. Moreover, digital technologies such as virtual reality or augmented reality increase student’s involvement in the e-learning environment [14]. Taking into consideration our main interest in the teaching aspect of e-learning, namely the challenges related to digital teaching, and being guided by the literature gap, the authors proposed the following research question: • What are the main challenges for university teachers in an e-learning environment? We begin the paper with an overview of the research context and theoretical background, then explain the methodological framework of the study and present findings. We conclude our paper with a discussion and reflection of the research questions. Finally, we present the limitations and proposals for future research.

10.2 Methodological Framework The present paper focuses on the qualitative analysis of the video podcast series that brings together the diverse and common perspectives on digital teaching, based on the COVID-19 era teaching experiences of the teaching staff of PLUS. All the PLUS teachers were provided with the same tools and information; hence, focusing on teachers from a single institution (PLUS) allows us to hold an important contextual source of variation (University Management of digital teaching and learning) constant across teachers; this enables an exploration of differences within this particular unit.

10.2.1 Sampling and Consent The ENLIVEN Video Podcasts are aimed at covering a range of standpoints. To ensure diversity in teachers’ perspectives within the PLUS, our selection of interview partners was guided by the following considerations: • academic discipline: we reached out to teachers from different disciplinary backgrounds; • national background: we sought to include Austrian teachers, who have been at PLUS for a longer time, as well as teachers that have an international background;

146

J. Leoste et al.

• gender: we sought to recruit both female and male teachers; • age: we reached out to teachers below and above 40 + years old; • care work/parenthood: since the lockdown has proven to be more burdensome for teachers that are responsible for minors or elderly, we recruited teachers with and without care responsibilities; • familiarity with the study goals: we sought to include perspectives of teachers that are part of the ENLIVEN Consortium as well as teachers that are completely unrelated to the project. Ideally, we would have been able to produce video podcasts with a large group of teachers. However, due to the difficulties posed by pandemic rules with regard to meeting in person (necessary for filming) and severe time constraints, we could produce video podcasts based on interviews with five PLUS’ teachers (convenience sampling) during this phase of the ENLIVEN project. The number of produced video podcasts was further determined by the available financial resources. Nonetheless, the video podcasts we produced do reflect the diversity within the PLUS (for example, teachers of different academic disciplines, genders, and nationalities). All the video podcasts are available on the following Google Drive repository link: https://bit.ly/ 3DodON4. Teachers were first provided with a form for declaring their consent for filming and disseminating (broadcasting) their interview. After the post-production, each teacher was able to review the video podcast for the final acceptance. An important aspect of the process was to give teachers enough space and time to reflect on their online teaching. Each teacher told their personal story about e-teaching, while the interview was guided by questions that touched on topics from their past experiences to future expectations, from pitfalls to successes, from the level of the subject to structures, from giving advice to reflecting on barriers and difficulties, etc. The core questions that guided the interviews were the following: • What have been or still are your personally experienced barriers to digital teaching? • What are structural barriers to digital teaching? • Which good or bad experiences did you have with digital teaching? • How do you see digital teaching in terms of the class as a collective? • Which are your suggestions for the implementation of digital teaching in the future?

10.2.2 Data Collection and Medium Typically a podcast—be it audio and/or video—consists of a live conversation between the interviewer and interviewee. Our reason for choosing the podcast format was based on the idea of publishing the interviews stepwise, stretched over a longer period of time. When creating the ENLIVEN Video Podcasts, however, we took a slightly different approach: by largely excluding the interviewing person and their

10 Environment Challenges of E-Learning in Higher Education …

147

point of view, the emphasis was put on the interviewed person. This was based on the idea of using the empirical data collection instrument of semi-structured interviews, combining thus the elements of expert and narrative interviews. The posed questions were inserted in the picture after the filming of the interview. For the purposes of this paper, the podcasts were coded (Teacher 1, Teacher 2, etc.). The teachers were informed of the interview questions, enabling them to prepare their replies. With respect to the medium, they were asked to keep their replies short and dense; a time frame of about 5–10 min was to be respected, in order to keep the content relevant for viewers. Each interview and interview question was recorded only once, to keep the conversations authentic.

10.2.3 Recording and Technical Information The ENLIVEN Video Podcasts’ recording demanded a great flexibility of both the production team’s/interviewer’s side and the interview partners’ side. In January 2022, the interview partners were selected and the video podcasts were recorded. Until the middle of February 2022, the post-production was realized. Both of the production/interview team members have a background in Communication Studies. The technical equipment basically consisted of three video cameras, one microphone, and two spotlights as well as two laptops and editing software. The video podcasts were recorded in different locations of PLUS. The rooms were selected by the teachers, the only demand was to choose a quiet and undisturbed setting, with adequate light and acoustic conditions. The phase of post-production demanded viewing, sorting, and cutting, in order to double-check that the teachers’ statements were not distorted in their meanings.

10.2.4 Data Analysis and Findings We based the analysis of the empirical data from the interviews on the qualitative approach. A qualitative approach to the field of empirical investigation means that researchers study things in their natural settings, attempting to make sense of and interpret phenomena in terms of the meaning people bring to them [17]. We combined three approaches: thematic inductive analysis [18], narrative coding [19], and retelling and construction of the collective story [20]. Thematic analysis was used as it is one of the most flexible data analysis methods, which we combined with the narrative coding [19]. Thematic inductive analysis has been applied, based on two stages (Fig. 10.1): 1. Listening to video podcasts and familiarizing with the content; inductive coding of the data.

148

J. Leoste et al.

Fig. 10.1 The data analysis process

2. Searching and generating the main codes; searching, reviewing themes, and defining the main themes. Analysis during the first and second stages involved six iterative steps [18]. One of the main focuses of thematic inductive analysis was related to personal and structural barriers, to analyze experience from digital teaching and challenges in the e-learning environment. Narrative coding with story construction was applied during the third stage (Fig. 10.1). The aim of the narrative coding was related to the analysis of the empirical findings across the data and “between the lines”. We aimed to explore from the empirical material the elements of the stories from participants related to challenges, personal and structural barriers to digital teaching, suggestions from teachers, and future visions. Finally, we used a retelling approach that involves re-reading and rewriting the story during the data analysis process. We subjected and constructed the collective story from the findings of thematic analysis and narrative coding, using elements of inductive analysis [20]. Below we present the main findings from the results of inductive thematic analysis related to the digital teaching experience with empirical examples (quotes) from the interviews. From the first and second stages of the thematic inductive analysis, we identified a variation of themes related to different aspects of digital teaching in the e-learning environment listed below as findings from the second stage of analysis in the thematic table (Table 10.1). Digital teaching experience means for the teachers barriers and challenges, possibilities, experimenting, supporting students, and novel opportunities for student learning. They experienced a lack of skills, lack of guidelines, and difficulties in engaging students in the learning process due to the reduction of social support. Participants of our study described their teaching experience and stressed the need for additional paid time, especially when preparing for digital teaching. This challenge emerged from the theme “time” and two subthemes: “time pressure” and “time for the preparation for digital teaching”. Time seems to be one of the personal barriers and challenges for the teachers: You need to take time. Digital teaching takes a lot of preparation, I try to be prepared for all of my students. (Teacher 1)

10 Environment Challenges of E-Learning in Higher Education …

149

Table 10.1 Themes related to digital teaching experience Themes and subthemes identified from the empirical material Barriers to digital teaching • • • • • • •

regional influence on technology internet quality workplace ergonomics, equipment, health, age gap using time previous knowledge, skills, experience, abilities lack of training lack of guidelines, conflict with existing guidelines

Challenging factors to ensure the quality of teaching • • • • • • •

different methods and materials for successful engagement preparation for teaching, learning routines student motivation in e-learning environment lack of concentration lack of suitable learning environment reduced social support visually challenged students

Suggestions for the future • • • • • • • • • •

communication and networking of the group exchange ideas, practices, experience experimenting support for students and teachers time for creating a sense of social belongingness avoiding full-time online using the best of both worlds hybrid form as an enabler for students opportunities for learning inclusive and flexible education

Learning in an e-environment raised a number of topics that teachers had encountered before, and many themes were related to environmental factors, digital opportunities, and tools they used in their teaching. Surprisingly, the themes “internet”, “internet quality”, and “necessary equipment”, related to structural barriers, emerged from the data analysis. The quality of the internet connection was considered by the teachers to be one of the determining factors that may have affected the learning activity as a whole, as a learner with a poor internet connection cannot hear well or show their video. This affects the learner’s contact with both the teacher and other learners. In several cases, teachers mentioned that learners in rural areas seemed to have fewer opportunities and a more unstable internet connection, which could also lead to inequalities for learners. Internet quality determined whether students could use the camera. However, without a camera, online learning changes fundamentally: “you are there, but you are invisible”. (Teacher 3)

It seems that the quality of the internet, equipment, and also student engagement are the structural and social barriers that limit the choices of communication and

150

J. Leoste et al.

interaction in the learning process and are still constantly the issues and objective challenges in teaching in an e-learning environment. Using equipment and different tools is not only barrier for teachers but is also a kind of obstacle for the students. I have equipment but my students do not… I need to stay simple. (Teacher 1)

When sharing their experiences, the teachers emphasized that in digital learning a lot of attention has to be paid to testing, pre-preparation of lectures and seminars, and trying out all the tools and different methods used. All interviewed teachers paid attention to the different stages of the learning process, made sense of their activities, and constantly compared it to contact teaching to ensure the same quality of learning. In many cases, teachers highlighted the learner’s experience. It means that the teachers have had to try and experiment with everything from the learner’s point of view in order for the learner to have a good and effective digital learning experience. The struggles are always equipment and using different tools from my sight and from the sight of my students… (Teacher 2)

However, online platforms and software are not experienced as barriers. Teachers found the opportunities to use different digital tools good, offering them a positive challenge and an opportunity to learn. Some resources also provided benefits that real classrooms do not allow. For example, one teacher mentioned that if she could not see the students on the back benches in real time in the classroom, then the digital environment ensured that the faces of all the students were very close at the same time and she could be in contact with everyone. However, it was mentioned that digital learning can be a problem for eye health. It was also pointed out that using some digital devices can be difficult for people with vision problems. As a whole, teachers considered various digital tools to be a positive challenge that was not difficult to develop. Handling the software was not a big challenge. Educational online platforms have been used for the last twenty years, but (asynchronous) discussion forums proved to be less successful in motivating interaction compared to (synchronous) online teaching and learning. That said, the camera is an important element for this success. (Teacher 3)

Also the themes “engagement” and “student motivation” emerged from the analysis as personal barriers in a digital learning environment and common challenges for the teachers. All teachers emphasized the need to involve students because, in digital learning, each learner has their own individual environment with many distractions and the learner has to pay much more attention to learning than in the classroom— requiring students to exercise higher learner self-motivation. Attention and motivation could be fostered by the involvement of all learners, through communication between them, which was identified as a challenge by all teachers. My personal barriers are my knowledge and skills, also how to work with the white board and different tools, how to make students engage with each other, how to prepare the content, what and how you would like to communicate. (Teacher 2)

10 Environment Challenges of E-Learning in Higher Education …

151

The challenge, as well as the positive factors related to digital teaching, emerged through the themes “flexibility”, “opportunities”, and “engagement “. Learning was seen as more effective when students were engaged and supported each other in digital learning. The flexibility of digital learning was seen both as the flexibility of everyday learning and more broadly. Digital learning makes university learning as a whole more flexible and accessible, regardless of where the student is located or what his or her home situation is. As an example, a teacher mentioned that parents of very young children can also learn from a distance, whereas, at earlier times university, education was not always possible for them. I am happy being more flexible and using more opportunities… If you have such mode to give the possibility to be more connected with each other it will give more possibilities for learning… (Teacher 1) It is very important that people do not feel that they are excluded. I really like a breakroom, you can be yourself in this space… (Teacher 4)

Significant challenges related to future vision and recommendations are perceived by teachers. One of the teachers pointed out that the best situation for her would be for all learning activities to take place again in the classroom: “My ideal future is of course people coming to class…” Teacher 2. At the same time, all teachers thought that hybrid learning has its place in the future, with some learners in the classroom and others at a distance. However, it was also found to be very complex, in which case teachers would also need an assistant to help them with e-learning, as the teacher cannot do two activities at the same time. The future is a hybrid form of teaching… you need a teacher-tutor, this person needs to take care of the group on-line. It is important that people do not feel that they are excluded. (Teacher 4)

There were recommendations related to better engagement of students, exchanging best teaching practices, and learning from each other. In addition, learning from students is considered an important recommendation. Teachers emphasized the importance of sharing best practices, which was done often at the beginning of the pandemic but not so much later. Teachers pointed out that learning from each other is crucial for teaching quality and helps to improve digital teaching and learning in online environments. We need to exchange the best practices of teaching, we need to learn from other universities, from each other, we need to learn from students… (Teacher 2)

Our findings indicate that the teaching experience of interviewed teachers consists of common themes. After reviewing and re-coding all themes, the final results from the whole process of the analysis suggest that the five common themes emerged from the empirical data related to digital teaching from the teacher´s perspectives: (a) personal barriers to digital teaching; (b) structural barriers to digital teaching; (c) challenging factors to ensure the quality of teaching; (d) recommendations; and (e) future vision (Fig. 10.2). Next, we will share the collective story focusing on the main themes as a part of the findings. We still have an insufficient number of shared stories related to the

152

J. Leoste et al.

Fig. 10.2 Common themes related to digital teaching

challenges and innovations in the e-learning environment and experience in digital teaching and learning. This common experience from teachers of different fields and disciplines might be unique and, as a collective story, it would make it possible to listen to teachers and to learn from them. Creating a story is primarily a process that organizes human experiences into meaningful episodes [17]. This story was told by teachers in 2022 and is based on their experience of teaching during the COVID-19 crisis when universities were closed but digital teaching made higher education and learning possible. We believe that the collective story is one way to understand the meanings and themes of the common experience of teachers working in an e-learning environment and value their story as a unique empirical and professional example from the current time. This story was constructed by researchers and follows the structure and coherence of the narrative, but belongs to the teachers from different fields and disciplines of Paris Lodron University of Salzburg.

10.2.5 Collective Story Universities switched over to digital teaching and learning very quickly and it was the largest challenge to adapt to the new situation. Digital teaching makes education possible and it kept universities open for learning. When I started to teach on-line it was a challenge to learn how the online tools work. Digital teaching takes a lot of preparation. Digital teaching means more than just classroom teaching. You will need to take time and I’ll try to be prepared for all of my students.

10 Environment Challenges of E-Learning in Higher Education …

153

In small groups teaching becomes easier because you can be yourself, it is exciting how it works. I really like break-rooms, you can be yourself in this space. I noticed that it enables students to learn more together. A challenge in digital teaching is a distance between teachers and students, students and students, also concentration on learning—it is a barrier to concentrating in the learning process. In terms of personal digital teaching experience there are a lot of barriers: internet is not working, speakers are not working, laptops are not working, and cameras are not used. It is difficult to have interaction without seeing people, without a camera. However, without a camera, online learning changes fundamentally: you are there, but you are invisible. Internet quality determined whether students could use the camera. It is very tiring for the eyes. You are working with a screen longer than people usually do. Too much screen, I would say. Handling the software was not a big challenge. Educational online platforms have been used for the last twenty years, but (asynchronous) discussion forums proved to be less successful in motivating interaction compared to (synchronous) online teaching and learning. That said, the camera is an important element for this success. My personal barriers are my knowledge and skills, also (knowledge about) how to work with the white board and different tools, how to make students engage with each other, how do you prepare the content, what and how would you like to communicate. In an on-line learning environment it is much more difficult to understand how communication is done between and among students. The struggles are related to how you communicate with your students and how you prepare the content, also the struggle is related to the equipment and using different tools, as perceived by me or my students. I have equipment but my students do not. I need to stay simple and it is important that people do not feel that they are excluded. I really like a break-room, you can be yourself in this space. Biggest challenge for me as a parent and teacher is that I need to take care of my child and teach students. You need to create more of those kinds of environments that enable students to establish relationships between each other and to exchange knowledge and ideas among themselves. Practice makes you better, but it will never make you perfect. There is no such thing as a perfect teacher. Being present as much as possible and really trying to engage students is a goal that we always need to have. The direction for the future is collaborative learning. I would prefer classroom teaching. In my ideal future students come to class. However the future of teaching in higher education is in hybrid form. Universities do not have enough necessary tools, equipment and rooms for hybrid teaching. You need a teacher-tutor, this person needs to take care of the group on-line. It is important that people behind the screen do not feel that they are excluded. We need exchanging the best practices of teaching, we need to learn from other universities, from each other, we need to learn from students. I am happy being more flexible and using more opportunities. If you have means that to give you the possibility to be more connected with each other it will give you more possibilities for learning.

154

J. Leoste et al.

10.3 Discussion and Conclusions This research investigates the e-learning environment challenges in higher education institutions from the teacher’s perspective. The data for this research was gathered during the ENLIVEN Video Podcasts with semi-structured interviews in different locations of the Paris Lodron University of Salzburg. Through analysis of the empirical material from the interviews, findings explain the main elements of the stories from participants related to challenges, digital teaching, and future visions of elearning in higher education institutions. Consistent with the previous research, this study highlights five common themes which reflect teachers’ challenges: (a) personal barriers to digital teaching; (b) structural barriers to digital teaching; (c) challenging factors to ensure the quality of teaching; (d) recommendations; and (e) future vision. This manuscript provides several important theoretical and practical implications for the e-learning environment challenges in higher education institutions. Based on research findings, the authors find the main digital teaching challenges according to personal barriers to digital teaching such as previous experience including knowledge and skills of teachers, different methods and materials for successful student’s engagement, and workplace ergonomics. In addition, according to the structural barriers to digital teaching, the main challenges are internet quality and necessary equipment. From the student’s perspective, the main barriers are lack of concentration due to distractions in the home study mood, reduced social (peer) support, and lack of a suitable learning environment. To pass these challenges teachers need to make recommendations to students to take time to create social belongingness and exchange ideas, practices, and experience during the e-learning platform. Moreover, results show that the future outlook from the teacher’s perspective sees the solution to these challenges by using a hybrid learning process that enables a more inclusive and flexible education. From the theoretical perspective, this study confirms previous research, which shows that lack of resources and lack of knowledge is one of the main barriers to the digital transformation of the learning process [9]. Also, this research shows that the main challenge for the teacher in the work with the students on e-learning platforms is how to increase student engagement and how to motivate them to be more active in a digital environment [14]. However, different from other studies this research illuminates different social groups of students as a challenging factor for digital teaching [16]. Hence, students who don’t have good internet quality, equipment, and a suitable learning environment to follow lectures have a lower motivation to follow lectures in a digital environment. In addition, this query needs to open a new discussion among higher education institutions on how to enable these students to be equal with the students from different social groups. From the practical perspective, findings show that teachers in the e-learning environment still have an insufficient number of shared stories related to the challenges and innovations in digital learning. To solve this challenge, the common experience of teachers from the different higher education institutions could be unique and as a story, it gives the possibility to learn from it. Furthermore, the collective story on the common e-learning platform for

10 Environment Challenges of E-Learning in Higher Education …

155

teachers could enable sharing of knowledge and skills between them. With this new expertise, teachers could make a new course at their university which will increase student engagement with the new innovative digital tools. With the new digital tools, students will be more motivated to attend e-learning and to achieve more knowledge and higher grades. The main critical issue that could increase teachers’ competencies in an e-learning environment is support from the university management which enables resources and training for the teachers to be ready for digital learning. Also, the support from the university management could result in better equipment and internet quality at the campus for teachers and students. In this way, the students without an adequate suitable learning environment can go to the university library to follow e-lectures. On the other hand, with the new equipment teachers could simulate practical exercises in the digital environment. With respect to the methodology, this study has limitations in the dataset and in the data collection procedure. Future research needs to include interviews with university representatives from different countries.

10.4 Limitations and Future Implications The limitations in this study are in sample size and in applied methods of data collection and analysis. A larger sample size, including persons with different regional backgrounds, would allow making more generalizable conclusions. Utilizing more thorough forms of data collection, such as semi-structured interviews with longer duration, together with quantitative instruments would enable gathering more comprehensive data. Future studies would also need to use both quantitative and qualitative data analyzing methods to make a more reliable overview of the e-learning environment. A great opportunity for further research could be to explore the possibilities of using artificial intelligence in the execution of e-learning content [21]. Additionally, the limitation of this study is in the teacher’s perspective on digital teaching, future research needs to include the management and students’ perspectives along with the teacher’s perspective about the digital learning environment. To achieve a more reliable overview of the e-learning environment, the authors recommended research on the participant’s behavior at the common e-learning platform with representatives from different universities. With this information, researchers could make quantitative and qualitative analyses to give recommendations from the perspective of teachers, students, and management on how to improve the e-learning process.

156

J. Leoste et al.

References 1. Kolata, G.: Past Pandemics Remind us Covid Will Be an Era, Not a Crisis That Fades. The New York Times (2021). https://www.nytimes.com/2021/10/12/health/when-will-covid-end. html. Last accessed 04 Apr 2022 2. Marinoni, G.: The Impact of COVID-19 on Higher Education Around the World. International Association of Universities (2020) 3. Crawford, J., et al.: COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. J. Appl. Learn. Teach. 3(1), 9–28 (2020). https://doi.org/10.37074/jalt.2020.3.1.7 4. Ulzheimer, L., Kanzinger, A., Ziegler, A., Martin, B., Zender, J., Römhild, A., Leyhe, C.: Barriers in Times of Digital Teaching and Learning – a German Case Study: Challenges and Recommendations for Action. J. Interact. Media Educ. 1(13), 1–14 (2021). https://doi.org/10. 5334/jime.638 5. Braudel, F.: Civilization and Capitalism, 15th–18th Century. In: The Structures of Everyday Life: The Limits of the Possible, vol. 1. Harper and Row, New York (1982) 6. Flint, C., Taylor, P.J.: Political Geography: World-Economy, Nation-State, and Locality (7 ed.). Routledge (2018) 7. Kalolo, J.F.: Digital revolution and its impact on education systems in developing countries. Educ. Inf. Technol. 24, 345–358 (2019). https://doi.org/10.1007/s10639-018-9778-3 8. Samuels, P., Haapasalo, L.: Real and virtual robotics in mathematics education at the school– university transition. International Journal of Mathematical Education 43(3), 285–301 (2012) 9. Gaebel, M., Zhang, T., Stoeber, H., Morrisroe, A.: Digitally Enhanced Learning and Teaching in European Higher Education Institutions. European University Association absl. (2021) 10. European Commission: Digital Education Action Plan 2021–2027. European Union (2020) 11. Rakic, S., Pavlovic, M., Softic, S., Lalic, B., Marjanovic, U.: An evaluation of student performance at e-Learning platform. In: 17th International Conference on Emerging ELearning Technologies and Applications (ICETA), pp. 681–686 (2019). https://doi.org/10.1109/ICETA4 8886.2019.9040066 12. Rakic, S., Softic, S., Vilkas, M., Lalic, B., Marjanovic, U.: Key Indicators for Student Performance at the E-Learning Platform: An SNA Approach. 16th International Conference on Emerging ELearning Technologies and Applications (ICETA), 463–468 (2018). https://doi. org/10.1109/ICETA.2018.8572236 13. Leoste, J., Rakic, S., Marcelloni, F., Zuddio, M. F., Marjanovic, U., Oun, T.: E-learning in the times of COVID-19: The main challenges in Higher Education. 19th International Conference on Emerging ELearning Technologies and Applications (ICETA), 225–230 (2021). https://doi. org/10.1109/ICETA54173.2021.9726554 14. Radianti, J., Majchrzak, T.A., Fromm, J., Wohlgenannt, I.: A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 147, 103778 (2020). https://doi.org/10.1016/j.compedu.2019.103778 15. OECD: PISA 2018 Results (Volume V): Effective Policies, Successful Schools. OECD Publishing (2020). https://doi.org/10.1787/ca768d40-en. 16. Enliven: Report on the online survey: Assessing the International Digital Learning Environments (2021). https://www.enlivenproject.eu/wp-content/uploads/2022/01/IO3-A1-EnlivenRequirement-analysis-vFinal.pdf, last accessed 2022/04/04. 17. Moen, T.: Reflections on the Narrative Research Approach. Int J Qual Methods 5(4), 56–69 (2006). https://doi.org/10.1177/160940690600500405 18. Braun, V., Clarke, V.: Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006). https://doi.org/10.1191/1478088706qp063oa 19. Saldana, J.: The Coding Manual for Qualitative Researchers. Sage Publications Ltd (2019). 20. Jõgi, L., Karu, K.: Nordic-Baltic cooperation in adult education: a collective story of Estonian adult educators. Int. Rev. Educ. 64, 421–441 (2018) 21. Spasojevi´c, I., Havzi, S., Stefanovi´c, D., Risti´c, S., Marjanovi´c, U.: Research trends and topics in IJIEM from 2010 to 2020: a statistical history. Int. J. Industr. Eng. Manag. 12(4), 228–242 (2021). https://doi.org/10.24867/IJIEM-2021-4-290

Chapter 11

Parents’ Voices: Inclusion of Students with Intellectual and Developmental Disabilities in Higher Education Isabel Catarina Martins , Oksana Tymoshchuk , Eulália Albuquerque , Paula Santos, and Geert Van Hove Abstract Making Higher Education more inclusive for students with Intellectual and Developmental Disabilities can bring significant benefits for the students themselves and for the academic community, in general. Family involvement is essential for the successful transition to post-scholar life, namely Higher Education. This paper presents and discusses the research and results of a study conducted with four Focus Group interviews involving 22 Portuguese parents of students with Intellectual and Developmental Disabilities. The main goal was to understand the perspectives and expectations of parents about the inclusion of their children in Higher Education. This study was carried out within the HiLives project scope, which includes, among its goals, to create strategies and provisions to facilitate and promote the inclusion of students with Intellectual and Developmental Disabilities in Higher Education, and the transition to an active and independent life. The results showed the prevalence of parents’ positive perceptions about the possibility of their sons/daughters attending Higher Education and allowed them to identify their preferred teaching models and methods, needs for support, barriers and incentives in this transition. This paper brings up some final recommendations for improving the processes of transition to Higher Education Institutions and their organization to be able of including students with Intellectual and Developmental Disabilities.

I. C. Martins · E. Albuquerque · P. Santos AVISPT21, R. Casimiros, 51-5, 3510-061 Viseu, Portugal e-mail: [email protected] O. Tymoshchuk (B) University of Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal e-mail: [email protected] G. Van Hove Ghent University, Sint-Pietersnieuwstraat 25, 9000 Gent, Belgium © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_11

157

158

I. C. Martins et al.

11.1 Introduction Different longitudinal studies indicate that, after secondary school, students with disabilities have lower employability, less probability of continuing their studies and less success in their professional training [1, 2]. These difficulties lead, frequently, to inactivity and social isolation, affecting these individuals’ well-being. In this regard, an OECD report [3] shows that the lack of possibilities after secondary education has a strong effect on disconnecting young people from the community, restricting their participation, and depriving them from social and economic independence and well-being. Education and post-scholar possibilities for students with disabilities are scarcer when compared to those of their peers [4]. In Portugal, transition possibilities to post-scholar life presented to adolescents and their families usually include protected employment, participation in community activities, daily living experiences, or inclusion in occupational activity centres. Furthermore, there is an increasing trend of students with disabilities, applying to enter in Higher Education (HE). For example, in 2019, there was an increase of 28%, when comparing to the previous year in Portugal [5]. Although an increasing number of students with disabilities consider, nowadays, post-secondary educational possibilities, these students very rarely have Intellectual and Developmental Disabilities (IDD). Persons with IDD have significant limitations both in intellectual functioning and in adaptive behaviour as expressed in conceptual, social and practical adaptive skills, shown before the age of 18 [6]. Several studies carried out in Portugal [7–9] show that the lack of transition programs and low social participation were the factors that most affected the quality of life of people with disabilities. There are few activities to be performed by those individuals who are autonomous to accomplish socially useful tasks but cannot access a job and do not want (or do not need) institutionalized support [10]. Preparatory activities for post-scholar life have been considered a responsibility of the teachers, families, and students themselves [4]. In spite of the significant importance of their participation in post-secondary preparation programs and developing self-advocacy skills, students are rarely involved in these activities and decisions [11]. Most families struggle with many challenges and frequently refer they feel lonely during this transition process. Compulsory education often fails to prepare them for successful post-secondary experiences, and parents frequently report low expectations of the school staff. Commonly, parents have low expectations concerning this stage of their lives, and previous research [12] show that parental attitudes, beliefs, expectations and involvement strongly influence transition processes. Due to the significant importance families have in transition processes, listening to their worries, dreams and conceptions is extremely important.

11 Parents’ Voices: Inclusion of Students with Intellectual …

159

11.2 Listening to Families’ Voices/Perspectives To understand the transition process, we must use multiple sources: teachers, students, social agents and parents. Parents are an essential source of information as they have the best knowledge about their children’s characteristics, abilities and needs. Families should be encouraged and supported to empower students with disabilities to make options and to be involved in decisions about their lives. Considering the importance of supporting families, Dunst et al. [13, 14] introduced the concepts of “enabling” and “empowerment”. “Enabling” implies the creation of opportunities for family members to develop skills that reinforce their functioning. The concept of “empowerment” [14, 15] proposes the implementation of interventions so that family members acquire or maintain a feeling of control over important aspects of their life, such as the result of their own effort to achieve goals. Studies also show that it is essential to consider young people’s desires and dreams in transition to adulthood, adopting a person-centred methodology [16, 17]. These authors show the effectiveness of applying PATH methodology [18]. PATH is a planning and problem-solving methodology for individuals and schools. It consists of a structured process which is underpinned by a focus on finding an alternative way for achieving a desired vision for the future [19]. Thus, the PATH process aims to create meaningful changes for individuals, placing their aspirations and wishes at the forefront of decision-making processes. Beirne-Smith et al. [20] suggest the Gentle Teaching (GT) approach as a methodology that can optimize the transition process of students with IDD to postsecondary life. The central premise of GT is based on the analysis of the person, providing the caregiver a deep understanding of their needs and motivations, and bringing interactional change, characterized by a more intense and frequent expression of unconditional appreciation towards the person [21, 22]. Despite being born as a therapeutic approach in the institutional context, GT has been proposed as a valid and effective approach for promoting the quality of relationships with students in the school context [23–25]. Based on PATH and GT assumptions, some Non-governmental organizations (NGO) in Portugal got together to create alternative answers to promote the inclusion of people with disabilities in the community. The project called “Support office for programs included in the community” (GAPRIC) aims to help adults with disabilities under the following objectives: (i) create social inclusion projects for people who find themselves without support; (ii) enable them to become autonomous through activities in the community; (iii) contribute to a more inclusive community, mobilizing the potential of community structures; (iv) foster the development of more inclusive forms of support that are less demanding in terms of human and financial resources [26]. GAPRIC show very positive results and are examples of inclusive social responses promoted by the community, although they still cover a reduced number of disabled people at the national level [26]. People who live active and meaningful lives have higher levels of psychological and emotional health [27], and less maladaptive behaviours [21], important indicators

160

I. C. Martins et al.

of students’ self-determination, as they promote an active role in their present and the construction of their future. It is essential to develop a clear understanding of parents’ perspectives of school practices regarding transition processes. This information can facilitate parent-school partnerships, which are crucial for implementing inclusive education [28] and for a successful transition to post-compulsory education.

11.3 Methodology This study is part of a research work developed within the HiLives Project—Including and Connecting in HE: networking opportunities for independent lives. The HiLives project, based on a strategic partnership between European Higher Education Institutions (HEI) and NGO, aimed at creating opportunities to share knowledge and practices in the field of (i) inclusion of students with IDD in HE, and (ii) transition to an active and independent life, exploring the role that digital media can play in this process. This study intends to show the organization process and main results of four Focus Group (FG) interviews that intended to identify parents’ perspectives of students with IDD about the possibility of their inclusion in HE. Sessions were conducted by three researchers, members of the HiLives project. FG is a method of interviewing in which “there are several participants (in addition to the moderator/facilitator); there is an emphasis in the questioning on a particular fairly tightly defined topic; the accent is upon interaction within the group and the joint construction of meaning” [29]. Morgan [30] argued that the typical group size is six to ten members. However, smaller groups are recommended when participants are likely to have a lot to say on a specific theme or topic.

11.3.1 Participants Twenty-two parents (4 male; 18 female) participated in the FG, which represented 20 youngsters with IDD (Appendix1). All tables can be found in the following URL: https://zenodo.org/record/6466484#.Ylx0htPMKuU. The selection criteria were to have a son/daughter aged plus than 15 years old. The selection of the target audience did not contemplate any other criteria, so participants represent different profiles in terms of education, age, gender and role performed in society. Five youngsters still attended compulsory school (corresponding to seven parents, as two were couples), and 15 participants were parents of people who had completed compulsory education. The parents were contacted by the researchers involved in the investigation. The invitations were formalized by email.

11 Parents’ Voices: Inclusion of Students with Intellectual …

161

11.3.2 Procedure Four sessions, using the FG methodology, took place in 2020, from June to December. Sessions were held in an online environment, using the Zoom (Colibri) platform video conferencing resource. For each session, seven parents were invited, although some parents were unable to attend. FGs were identical regarding how they were organised and facilitated by one researcher, supported by two colleagues. Participants gave informed consent and were explained the goal of the FG and the HiLives project. Participants had the option of leaving at any time if they considered the interview to be harmful. The researchers guaranteed the participant’s anonymity, that no information would be shared with third parties, and the confidentiality of the research. The researchers established an interview guide with some questions for the meetings. The starting questions for exploring the first interviews were as follows: – – – –

What dreams/expectations do you think your child has for his/her own life? Have you ever thought about him/her pursuing studies for HE? In your opinion, what other supports should there be in HE? What could be the role of NGO (linked to disability) in this process?

Each FG lasted about 150 min; the sessions were recorded on video and audio (after the interviewee’s permission) and later transcribed verbatim. The data transcribed were imported to WebQDA, where they were coded and analysed. The coding process followed a mixed coding approach [31], where an initial set of categories was defined based on the research questions but then evolved according to the codes and categories that emerged during the data analysis. Once there was a first version of the coding tree, researchers discussed it for improvement. Appendix 2 presents an overview of the interview guide by topics, the dimensions observed under each topic, and the studies used to frame each concept. It is important to note that the researchers adopted the O’Brien model’s categories [32] to analyse the results: Model, Support, Policies and Inclusive Pedagogy. However, there were collected a lot of inputs from parents about their dreams and those of their children and plans for the near future. In this context, we chose to create new categories based on PATH [33]: Dreams, Factors associated with successful transition, Factors that constitute barriers to transition, Formal transition planning, Pathways transition resources. A set of other inputs from parents were collected related to well-being and fundamental human values, and the desire for their sons/ daughters to feel safe, valued, loved, and respected. To facilitate these inputs’ analysis, we created a group of categories based on the Gentle Teaching methodology [34]: Feeling safe, Having safe and loving relationships, Feeling a sense of connectedness, Body integrity, Feeling self-worth, Feeling secure, Having meaningful daily activities and a meaningful and valuable day, Feeling inner contentment.

162

I. C. Martins et al.

11.4 Results During the four FG interviews, the researchers collected 2077 contributions from the participants. The results are presented according to the main research topics and dimensions introduced in the previous section.

11.4.1 Characterization of Youngsters The four parents of 20 young people with IDD involved in the FG, were living in Continental Portugal and in Madeira Island. Some youngsters participated in associations that support people with disabilities. The youngsters are aged between 15 and 40 years old (M = 21.5 years) (Appendix 3). Most young people are male (12/20); eight are female (8/20) (Appendix 4). Regarding education level, parents reported that eight youngsters had concluded 12th grade, and the others were still attending secondary school (Appendix 5). Parents also mentioned that seven youngsters benefited from the Individual Educational Plan at school, and seven youngsters had an Individual Specific Curriculum. Only two students attended a Regular Curriculum (with no significant differences from the ordinary curriculum). Fifteen youngsters were attending or have attended a professional course. Regarding youngsters’ special needs reported by the parents, 81 references emerged (Appendix 6). Participants presented intellectual difficulties (20/81); visual difficulties (9/81); reading difficulties (8/81); difficulties in socialization (7/81); writing difficulties (8/81) and motor difficulties (6/81). It’s important to mention that most parents said that their children have a very high level of autonomy; only one young person needs support in eating. Some parents referred that youngsters already had some professional experience: participants made 31 references for the 16 different places where young people worked. Among the workplaces mentioned, parents slightly highlighted coffee shops (6/31), hospitals (4/31) and kindergarten (3/31) (Appendix 7). According to parents, these socio-professional experiences, even when shortterm, positively impacted their sons/daughters’ development. In some cases, these experiences, after some time, became a job contract, as it happened to a young woman doing a socio-professional experience under the GAPRIC project, whose mother mentioned, “There is already a prospect of a one-year contract at the hospital” (E4.4). Most parents reported that their children had or have had an unpaid socioprofessional experience developed within the scope of the individual transition plan (PIT) at secondary school (13) and the GAPRIC Project (5). In three cases, the youngsters were volunteers; only one mother mentioned that her son worked as an employee.

11 Parents’ Voices: Inclusion of Students with Intellectual …

163

11.5 Listening to Families’ Voices/Perspectives 11.5.1 Planning Alternative Tomorrows with Hope (PATH) Parents mentioned factors that they consider beneficial for realizing these dreams and overcoming several barriers in this process. To perform an in-depth analysis of this information, we chose to adopt the principles of the PATH methodology [33]. The researchers collected 15 references about children’s dreams and 22 references about parents’ dreams. Among children’s main dreams, participants mentioned continuing studies (12/57), having work, being happy, and having an independent life. Among parents’ dreams, to see their sons/daughters happy (9/22) and autonomous (5/22) were the most frequent (Appendix 8). Researchers collected several factors pointed out by parents as facilitators in building a happy future for their children, that were assembled into six groups: Inclusive relationships; Therapies; Specialty medical appointments; Support at school; Support from associations and Participation in extracurricular activities (Appendix 9). There were identified 14 parents’ concerns about their son’s/daughter’s future, organized into two groups: (1) “My child is finishing school. What now?” (2) “What will happen to my child when I am no longer here?”. As a mother mentioned, “the father nor I will live forever, he doesn’t have siblings, so it’s a worrying situation, which I don’t even want to think about, so I can’t get sick. If he can finish a course, great; if not, at least I wish he develops enough to get a job and support himself” (E4.3). The researchers collected five testimonies about the worries of the parents. The main reasons pointed for participants’ frustrations were difficulties in accessing education (14/116); lack of inclusion (14/116); difficulties in relationships with school (13/116); low acceptance from teachers/staff at school (13/116) and difficulties in relationships with colleagues (13/116) (Appendix 10). The current pandemic has also been referred to as one of the significant limitations in the youngster’s transition to post-school life. Many parents reported that their children were forced to interrupt their professional experience because of Covid-19. “It was a prison. My son stayed at home doing nothing”, refers one mother (E4.3). Parents mentioned the importance of having specialized training for teachers to promote sensitivity and develop knowledge relevant to supporting pupils with IDD. I think there should be special training for teachers, for example, my daughter has certain difficulties, she does not make certain movements, certain tasks. So, one must be prepared to know how to deal with and know each disability, every limitation of each, and the things they can and cannot do. (E2.1). If the teachers who dealt with my son had such an experience at the university, maybe my son would have had a different support from teachers at school. (E2.5).

164

I. C. Martins et al.

11.5.2 Gentle Teaching (GT) The data collected show that the parents’ dreams were also linked to fundamental human values: the desire that children feel safe, valued, loved, and respected. The researchers considered that these factors are linked to the main principles of the GT theory [33]. Appendix 11 shows that parents gave more importance to factors such as having significant daily activities and a significant and valuable day (18/48); feeling a sense of connection (10/48); and having safe and loving relationships (7/48). As one mother said: “I think the most important thing will be the emotional domain, to be more autonomous, to have more self-confidence and feel more secure in themselves.” (E3.1).

11.5.3 Higher Education and Youngsters with IDD The researchers analysed parents’ perspectives regarding the possibility of youngsters with IDD attending HE courses based on O’Brien’s model [30]. The categories considered were the same that emerged within the systematic review carried out within the scope of HiLives Project. Thus, it is expected to compare the data of the FG with those collected in scientific articles. Regarding the teaching model to be implemented in HE, most parents preferred the full inclusive model. This model provides individualized support to students, with a support system (tutor/coach, mentors), in curricular units from undergraduate/master’s degree courses. The student’s interests and goals motivate the choices of the curricular units and their supports [33]. Considering the Inclusive Model, parents said: I think that Pais-em-Rede (Networked Parents) association should be considered a model for planning higher education courses where these young people will be included. What they do there is what is significant, they commit themselves. (E2.2). The aim is really that these young people have the opportunity to attend an academic environment in Higher Education, that they are included, but that it really has a design appropriated to their characteristics. (E2.3).

Four parents prefer the mixed model: I know little about HE, but there are schools in other countries, where there is a possibility for mix models, favourable to our children. (E4.5).

Parents stressed the importance of offering courses and developing a support network, which mobilizes resources that can positively influence the academic experience, providing personalized support and responding to the needs of students with IDD in HE. Concerning this support, parents highlighted the Including university tutoring (24/124) (Table 11.2). Several parents have pointed out that “this tutor figure seems very important because it can help manage the student’s expectations and career while in college”

11 Parents’ Voices: Inclusion of Students with Intellectual … Table 11.1 Models of Inclusion in HE

Table 11.2 Supports

165

Model

Total

Full inclusion

10

Mixed/hybrid programs

4

Segregated model

1

Supports

Total

Preparation and transition from secondary to post-secondary

4

Support services

6

Support in adaptation

11

Support in daily life

5

Support in guidance

13

Support from associations/community

16

Collaboration of several professionals (teachers/administrative/collaborators)

6

Collaboration with family

3

Inclusive university tutoring

24

Volunteering

2

Digital solutions supporting inclusion in HE

9

Zoom

6

Mail

1

Teams

1

Mobile applications for accessibility

5

Opportunities for training and professional development 9 (staff, mentors and volunteers) Special education training for teachers

6

(E2.2). Besides, parents mention that “there should have someone who could understand them.” (E4.1). The second most prominent dimension was Support from associations/ community (16/124), mentioning the mediating role associations can play in this process. Parents also mentioned the critical role of communities/associations in promoting changes in society. For example, one mother said: I think that if people, organizations, associations come together and fight for something concrete, it is easier to get where we want to go. So, if there is a desire to do something for these young people, I think the more people and associations come together in this sense, the easier it will be to take our message further. (E3.3).

Regarding the role of NGO, parents used expressions such as: It is good to share our children’s problems with each other because then we do not feel so lonely, we can help each other. (E4.3).

166 Table 11.3 Public policies

I. C. Martins et al. Polices

Total

Funding

8

Accreditation and regulations

–

Admission policy

2

Graduation policy

1

Disability policy

3

The path opened by professionals is completely different from being us showing up and asking for favours, and I think it helps a lot. (E2.2).

As shown in Table 11.3, the FG participants also highlighted the importance of Support in orientation (13/124), Support in adaptation (11/124), and Digital solutions to support inclusion in HE (9/124). Parents mentioned the importance of having a period of adaptation to the University with a more significant follow-up, for example, help with guidance on Campus, support with the study organization and management, help with the library, and other services. This monitoring should decrease over time, promoting youth autonomy. Concerning digital solutions, parents evoke the time of lockdown in which “the technologies helped a lot with teams, zoom, and the online activities that were wonderful, as the isolation ended.” (E4.3). Among the digital solutions that can facilitate young people’s learning process, parents highlighted Zoom (6/124) and mobile accessibility applications (5/124). Regarding public policies related to admission, accreditation and funding of courses “for all” in HE institutions, parents referred to the funding of these courses (Table 11.3). As one parent mentioned: the question of the resources and means needed is fundamental for the courses to work. We must understand that our young people are all different from each other, have skills in completely different subjects, and need very personalized monitoring; without funding, the course will hardly ‘have legs to walk’. (E2.2).

The realization of FG made it possible to understand the parents’ perspectives of the methodologies and pedagogical strategies that can facilitate the inclusion and learning of students with IDD in HE. As shown in Table 11.4, the FG participants attributed the most significant importance to (peer) mentoring (19/116). As one mother said: “the role of peers is fundamental in all aspects, whether collaborating, interacting or helping her to be more uninhibited.” (E3.3). The parents also highlighted other categories, such as Universal Design for Learning (UDL) (17/116); Academic tutoring (17/116); Curricular Accommodations (12/116) and Collaboration between teachers, tutors, mentors, academic community, families and other stakeholders (12/116). Parents mentioned that the learning programs in inclusive educational environments must be flexible, adaptable, and designed to overcome barriers that students with IDD may face in HE. As one parent mentioned, “the aim is that these young

11 Parents’ Voices: Inclusion of Students with Intellectual …

167

Table 11.4 Inclusive pedagogy Inclusive Pedagogy

Total

Universal Design for Learning (UDL)

17

Problem-based learning (PBL)

1

Service-Learning—experiences in the community

1

Transition to Profession/Employment—Socio-professional experiences for employment

6

Group dynamics

2

Seminars

–

Person Centred Planning (PCP)

15

Cooperative learning

1

Curricular Accommodations

12

Practical courses

4

Alternative Resumes

8

(Peer) mentoring

19

Academic tutoring

17

Collaboration between teachers, tutors, mentors, academic community, families and other 12 stakeholders

people also have the opportunity to attend an academic HE environment, that they are included, but with an adequate curriculum” (E4.1). Among the HE courses in Portugal, parents also mentioned the accessible and inclusive program of the University of Aveiro (UA) (10); the course for students with IDD at the Polytechnic of Santarém (3) and the General Studies Course at the University of Lisbon (1). As one mother said: “I think the most important part of this experience [UA] was the way colleagues saw her, accepted her, and included her” (E2.4). Three parents reported the great hope of opening the new course for young people with IDD at the UA and mentioned that their children really want to attend this course. Parents also discussed the situation regarding accessible education to students with IDD in universities in other countries. For example, one mother said: "In Spain, there are around 30 courses like this; in the United States, I don’t know how many universities but a lot, in Iceland, the same.” (E4.1). Regarding the possibility of their child attending HE, most (14/21) answered positively. The parents consider it could be a very interesting social experience (Table 11.5). Even if they do not graduate, they could develop more skills. As a father said: At the level of Higher Education, I think it would be a fantastic experience for him and his peers too, for sure. I think it would be a personal gain for him, as a person, as a young adult, at all levels. I don’t think the academic part is essential, but being part of such a project, I think, would be very beneficial for him as a person. (E1.3).

A group of parents (6/21) had never thought that their children could go to HE and refer that no one has ever spoken to them, nor have they put that hypothesis. However, six parents reported not wanting their children to attend HE (Table 11.5).

168

I. C. Martins et al.

Table 11.5 Perspectives about the possibility of attending the University Youngsters

Total

Positive

11

Positive

14

Total

Negative

4

Negative

6

Not defined

6

Not defined

1

In four cases, these answers were related to the fact that their sons/daughters were working or volunteering in a place they like very much, so parents prefer to continue. In two cases, this response was due to the negative experiences of the young person at school. As a mother said: I have the notion that he was not happy during 12 years in school, so he doesn’t want a university project of more than 3 or 5 years or whatever it is, he will not be happier because of it, I want him to be happy (E1.1).

Even parents who revealed low expectations were happy to hear that a team is working on the possibility of students going to HE. They mentioned expressions like “We can already dream, this was something unimaginable, thankfully someone is thinking about this” (E3.2). Regarding the parents’ perception of young people’s interest in attending HE, eleven parents stated that their son/daughter would like to participate in these courses. Three parents said that their children did not want to continue studying, and six parents were unsure. Some of these parents mentioned that they never spoke to their sons/daughters about this subject, unaware that these opportunities could exist. As shown in Table 11.6, the parents also mentioned several academic subjects they consider their children would like to attend, highlighting English (10/52), computer science (9/52) and arts (8/52). This study shows that parents believe that, with appropriate methodologies, it is possible to include people with IDD in HE. However, parents also point that.

Table 11.6 Academic curricular subjects Academic curricular subject

Total

Academic curricular subject

Total

English

10

Orientation and mobility course

2

Computing

9

Health

2

Arts

8

Biology

2

Music

3

Artistic expressions

2

Financial literacy

3

Psychology

1

Child education

3

Science

1

Dance

2

Physiotherapy

1

Painting

2

Psychology

1

Dance

2

Total

52

11 Parents’ Voices: Inclusion of Students with Intellectual …

169

We need to change mentalities, although we know that at the HE level, it may not be easy to change some mentalities, but with a little bit of will and, above all, the will of those who are responsible to decide on this, I think it might be possible. (E2.2).

11.6 Discussion From an inclusive perspective, the collaboration between the family and HEI is crucial in searching for the best solutions that allow them to respond to these students’ specific needs. According to Traina and Hoogerwerf [35, p.33], parents can provide valuable information about prior learning, namely “about competencies already acquired and about what, according to them, could be the next steps in the development of independence in all life areas”. Besides, these authors report that “through input from the persons who know the potential learner, it can be easier to define learning programs” [35, p. 33]. Although it is important to discuss learning needs and the motivation for learning with the youngsters directly involved, this study shows that parents’ involvement is also essential. However, it would be beneficial to triangulate parents’ perspectives with data published in scientific studies to understand their perspectives. It is important to know factors that stimulate the development of youngsters with IDD and identify the barriers in this process because it allows the creation of strategies and networks that encourage the inclusion in society. Parents state that high school experiences often do not prepare well for a successful transition to HE. The factors that stimulate young people’s development with IDD were clustered into six groups: Inclusive relationships; Therapies; Speciality medical appointments; Support at school; Support from associations (NGO); Participation in extracurricular activities. Several factors that facilitate or hinder the transition process of people with IDD to post-scholar life arouse. Parents consider social skills, interpersonal relationships as some non-academic facilitator factors. Extracurricular activities are considered too, but as already shown in previous studies, not as a significant factor [36]. Some needs have been identified. It is clear for parents that peers, teachers’ attitudes, and knowledge appear to be essential in the process. Parents mentioned that digital technologies could also be useful for creating new forms of learning and teaching. According to Woodward and Rieth [37], digital technologies can improve the acquisition of skills and content knowledge of these students, as computers are used to deliver well-designed and managed instructions through curricular adaptations. Other studies show that digital technologies allow students to engage in drill and practice, role-play, exploratory or communication activities that meet their individual needs and capabilities [35, 37]. Among the barriers in this process, parents highlighted difficulties in accessing education as lack of inclusion, difficulties in the relationship with the school, lack of acceptance from teachers/staff at school, and difficulties in relationships with colleagues. Other studies have observed these barriers [36, 37] in the inclusion

170

I. C. Martins et al.

of adults with IDD in HE. For example, the study by Corby et al. [41] identified barriers as Transition or preparation; Coping with the demands of education at a higher level; Support requirements, including access issues for those with physical disabilities; Accessing supports; Availability of appropriate level courses; and Difficulties are associated with the staff’s level of knowledge and awareness within the HE institutions. According to Mara [42, p. 80], among the obstacles that “hamper their access to HE may be: high costs, almost prohibitive for universities to provide suitable means for assimilation of the curriculum for these people with disabilities, physical barriers, material, lack of qualified staff for educational mentoring people with disabilities, lack of guidelines/proper supervision with mentors help for a smooth learning and motivation processes”. Parents mentioned that transition models and practices do not encourage students with IDD to consider HE as an option. Reflecting on the model, some parents consider that their children can go to HE since they can attend technical courses. Others consider it could be important as a social experience. Even if they do not graduate, they could develop more skills. Parents argue that there should be a structure that continues in HE, essentially for those students who, even with difficulties, are very fond of school. This study demonstrates that parents believe that learning pathways should be implemented, whenever possible, on an individualized basis, using a person-centred approach, considering the life-relevant goals for these youngsters. Parents suggest some methodologies in line with UDL. UDL applies neuroscience knowledge on the understanding of how brain processes information to the design of curricula that can meet individual student needs [38]. It’s considered a potentially promising approach to enabling students with significant IDD to access a more supportive learning environment [39]. According to Ainscow et al. [40], curricular adaptations and contextualized educational strategies can facilitate these students’ teaching and learning process. Because the needs of these youngsters in the training and learning pathways may be very diverse, the educational processes must be flexible and adaptable to each situation. The study of Corby et al. [41, p.72] shows the “requirement for a flexible approach from all stakeholders involved, ensuring a realistic approach to students’ needs as they begin to consider the option of higher-level education”. Other studies evidence that if “teachers/trainers/educators use a flexible methodology to plan activities for appropriate training, can guarantee the necessary support for the acquisition of knowledge and skills” [33, p. 36]. Results also show that most parents consider that attending HE can benefit their sons/daughters. Other studies show that attending HE allows these students to improve their self-esteem and confidence [43, 44]; their future job opportunities and earning prospects [44]; increases their friendship networks, allowing them to feel more accepted and included in the community [29].

11 Parents’ Voices: Inclusion of Students with Intellectual …

171

11.7 Concluding Remarks This study investigated parents’ perspectives of youngsters with IDD about the possibility of inclusion in HE. We can organize parents’ answers and perspectives into three categories: – Parents who would like their children to have the opportunity to attend an HE course: some parents said openly that their sons/daughters should go to HE and that they do not understand why this possibility is not allowed. This group includes parents of students who have already had the opportunity to participate in the experimental course at the UA or know this course at national and international universities. These parents were more involved in the discussion about the models that should be applied, suggestions about programs and courses, the role of the teachers/peers, and financing of the courses. – Parents who do not want their children to attend HE: some of them have had a “bad” experience in school education and prefer sons/daughters to develop professional competencies. This group also includes parents of youngsters who were successful at the workplace, so they prefer to continue working. This group focused a lot on the barriers in the teaching process, difficulties in accessing work, and the lack of inclusion in the community in general. They consider that if it is already difficult in secondary education, one can expect that it will be more difficult in HE. – Parents were surprised by the possibility of their children continuing their studies at HE as they had never thought about it. This group mainly includes parents of the youngsters (aged 15–18), who are still attending school. They refer to it as the first time they were approached on this issue and still do not have a defined opinion on the subject. They became more involved in the discussion about the types of support that should exist for these people to be able to continue their studies. These parents have attributed great importance to the transition to an active life and the existence of transition plans/projects that promote this process, such as GAPRIC. When youngsters are more autonomous and have more skills, it seems logical to parents that they could attend HE. However, parents reveal low expectations, and they were happy to hear there is a team working on the possibility of students going to HE. Our central aim was to hear parents’ perspectives on the possibility of youngsters with IDD attending HE. However, this research exceeded expectations, allowing us to collect information beyond the parents’ perspectives on HE and the conditions that can increment this process. The interviews offered a lot of relevant data to understand better the complexity of the development processes, the school path, and the transition of these youngsters to post-secondary life. Therefore, the research has brought valuable results on – the process of development of youngsters, comprising facilitators/barriers in this process in Portugal;

172

I. C. Martins et al.

– different paths and experiences of youngsters in school education, realizing the difficulties and the conditions enhancing this process; – different processes of transition to adulthood, types of support that exist or are missing, and the worries of some parents who unfortunately feel alone in these experiences; – youngster’s expectations about their future, gathering dreams and plans for the near future, and realizing their reasons for willing or not to attend HE. Understanding these factors is essential in creating HE possibilities for these youngsters. The present study has some implications and suggests clues for secondary education and HE inclusion. Parents made many important suggestions to facilitate the transition of young people with IDD to life after school and to promote more inclusive HE. Among these suggestions, we can highlight. – Institutions (secondary and HE) could develop practices that support the transition and encourage people with IDD and their families to consider HE as an option, if they wish. – High Schools should be encouraged to develop transition planning that should take place in due time. – Dreams and perspectives in a near future may be considered. – It is important to listen and empower parents and involve them in the learning processes of youngsters, including transition. – At HE there must be a period of adaptation, with more intensive monitoring and mentoring programs. – HE institutions should consider different models of education and different models of support. It is not enough to offer courses but also necessary to develop a support network. – Curricula may be adapted according to the interests and skills of people with IDD. The resources may be adapted to their characteristics. – We must keep in mind that digital technologies can make HE more accessible to these students. – Since some youngsters with IDD may need ongoing support and services, they should be aware that these supports and services are available at HE and have continuity. – In defining HE courses for students with IDD, it may be pertinent to include the PATH and GT (well-being and human values, such as security, affection, being able to be affectionate and bonding). – NGO may have a mediating role between the university (teachers/tutors/employees) and the family, namely in supporting the transition process of youngsters to HE, defending young people’s rights, and raising community awareness. Another highly focused aspect was the training of teachers. NGO can train teachers and tutors in knowledge and methodologies about Inclusion and Inclusive Pedagogy. Furthermore, it is essential to highlight that this research qualitative nature implies

11 Parents’ Voices: Inclusion of Students with Intellectual …

173

that its generalizability is limited. As discussed in the previous paragraph, the results concur with previous research, giving reliability and robustness to the findings. Acknowledgements This work is financially supported by national funds through FCT—Foundation for Science and Technology, I.P., under the project UIDB/05460/2020.

References 1. Patrick, L., Edwards, C.: Youth Quality of Life Instrument – Short Form: Intervention in School and Clinic. University of Washington, Seattle, Washington (2010) 2. Wagner, M., Newman, L., Cameto, R., Garza, N., Levine, P.: After High School: A First Look at the Postschool Experiences of Youth with Disabilities. Menlo Park, CA: SRI International (2005) 3. OECD.: Inclusion of Students with Disabilities in Tertiary Education and Employment: Training Policy. OECD Publishing (2011) 4. Garrison-Wade, F.: Listening to their voices: factors that inhibit or enhance postsecondary outcomes for students with disabilities. Int. J. Special Educat. 27(2), 1–15 (2012) 5. GTAEDES: Alunos com deficiência no ES. (2019). http://www.gtaedes.pt. Last accessed 22 Mar 2022 6. Schalock, R.L., Luckasson, R.A., Shogren, K.: The renaming of mental retardation: understanding the change to the term intellectual disability. Intellect. Dev. Disabil. 45(2), 116–124 (2007) 7. Fânzeres, L.: Transição para a vida adulta de alunos com necessidades educativas especiais: Percursos de formação no sistema educativo português. PhD diss., University of Minho (2017). http://hdl.handle.net/1822/50880. Last accessed 21 Mar 2022 8. Ferreira, M.: Perspetivas sobre a transição para a vida pós-escolar de alunos com dificuldade intellectual. PhD diss., University of Minho (2019). http://hdl.handle.net/1822/60249 9. Veiga, C., Domingues, I., Saragoça, J., Fernandes, L.: Inclusão profissional e qualidade de vida. Vila Nova de Famalicão: Formem (2014) 10. Martins, I.C., Albuquerque, E., Pereira, M.: Enhancing participation, creating good memories: an experience of inclusion in community. Dev. Med. Child Neurol. 62(S4), 34–52 (2020) 11. Skinner, M., Lindstrom, B.: Bridging the gap between high school and college: strategies for the successful transition of students with learning disabilities. Preventing school failure: alternative education for children and youth 47(3), 132–137 (2003) 12. Doren, B., Gau, J., Lindstrom, E.: The relationship between parent expectations and postschool outcomes of adolescents with disabilities. Except. Child. 79(1), 7–23 (2012) 13. Dunst, J., Leet, E., Trivette, M.: Family resources, personal wellbeing, and early intervention. J. Special Educat. 22(1): 108–116 (1988) 14. Dunst, J.: Early intervention in the USA: programs, models, and practices. Prevention and intervention in childhood and adolescence 11–52 (1996) 15. Santos P., Tymoshchuk O., Sousa T., Almeida M.: Compartilha: a sharing social network in Special NEEDs aiming to empower parents and experts. INTED2015 Proceedings. In: 9th International Technology, Education and Development Conference. Madrid, Spain. 2–4 March (2015). http://library.iated.org/ 16. Caldwell, K., Parker Harris, S., Renko, M.: Inclusive management for social entrepreneurs with intellectual disabilities: “how they act. J. Appl. Res. Intellect. Disabil. 33(2), 204–218 (2020) 17. Clark, H.: Involvement of the young person in transition planning. In Incarcerated Youth Transitioning Back to the Community, 35–58. Springer, Singapore (2018). 18. Pearpoint, J., Forest, M., J. O’Brien.: PATH: A Workbook for Planning Positive Possible Futures: Planning Alternative Tomorrows with Hope for Schools, Organizations, Businesses, Families. Inclusion Press (1993)

174

I. C. Martins et al.

19. Wood, H., O’Farrell, K., Bjerk-Andersen, C., Mullen, C., Kovshoff, H.: The impact of planning alternative tomorrows with Hope (PATH) for children and young people. Educ. Psychol. Pract. 35(3), 326–338 (2019) 20. Bernie-Smith, M., Patton, J., Kim, S.: Mental Retardation (7th ed.). Upper Saddle River, NJ: Pearson Education (2006) 21. McGee, J., Menolascino, J., Hobbs, C., Menousek, P.: Gentle Teaching: A Non-aversive Approach to Helping Persons with Mental Retardation. Human Sciences Press, New York (1987) 22. Van de Siepkamp, P.: An introduction to GT. Learn. Disabil. Pract. 13(6), 25–27 (2010). https:// doi.org/10.7748/ldp2010.07.13.6.25.c7888 23. Cullen, C., Mudford, O.C.: GT Controversial therapies for developmental disabilities: fad, fashion, and science in professional practice, 423–432 (2005) 24. Martins, C., Marques, R., Linharelhos, M., Martins, I.C.: Sentir-se seguro e amado na escola: Gentle Teaching em contexto educativo. Gestão e Desenvolvimento, 29–50 (2021) 25. Pereira, M.: Pertencer & Participar para Aprender. Edição ASSOL (2017) 26. Martins, I.C., Ribeiro, C., Ferreira, A.: Quality of life in cerebral palsy: the importance of social support. Aten. Primaria 48, 133–155 (2016) 27. Canha, L. Fernandes, C.: Sintra Inclui - Projeto E Manual. Camara Municipal de Sintra (2018) 28. Lui, J.H., Marcus, D.K., Barry, C.T.: Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Prof. Psychol. Res. Pract. 48(3), 199 (2017) 29. Bryman, A.: Social Research Methods, 4th edn. Oxford University Press, Oxford, New York (2012) 30. Morgan, L.: Planning Focus Groups. Sage, Thousand Oaks, CA (1998) 31. Creswell, J., Clark, V.: Designing and Conducting Mixed Methods Research, 2nd edn. SAGE Publications, Inc., Los Angeles (2010) 32. O’Brien, P., Bonati, L., Gadow, F., Slee, R.: People with intellectual disability experience university life: theoretical underpinnings, evidence and lived experience. Brill (2019) 33. Pearpoint, J., Forest, M., O’Brien. J.: Path: A Workbook for Planning Positive Possible Futures: Planning Alternative Tomorrows with Hope for schools, Organizations, Businesses, Families. Inclusion Press (1993) 34. Machado, M., Santos, P., Espe-Sherwindt. M.: Inclusão de pessoas com dificuldades intelectuais e desenvolvimentais no ensino superior: contributos da literatura no contexto europeu. Revista Portuguesa de Investigação Educacional 20, 143–165 (2020) 35. Traina, I., Hoogerwerf, E.: Possible framework for the design of learning programs in assistive technology for people with intellectual disabilities in inclusive educational environments. Psychol. Behav. Sci. 7(2), 29–37 (2018) 36. Lombard, P.: Factors that influence transition from high school to higher education: a case of the Junior Tukkie programme. Afr. J. Career Develop. 2(1), 1–14 (2020) 37. Woodward, J., Rieth, H.: A historical review of technology research in special education. Rev. Educ. Res. 67, 503–536 (1997) 38. Rose, D., Meyer, A.: Teaching Every Student in the Digital Age: Universal Design for Learning. Alexandria, VA: ASCD (2002) 39. Coyne, P., Pisha, B., Dalton, B., Zeph, L., Smith, N.: Literacy by design: a universal design for learning approach for students with significant intellectual disabilities. Remedia Special Educat. 33(3), 162–172 (2012) 40. Ainscow, M.: Tornar a educação inclusiva: como esta tarefa deve ser conceituada. Tornar a educação inclusiva 1, 11–24 (2009) 41. Corby, D., Cousins, W., Slevin, E.: Inclusion of adults with intellectual disabilities in postsecondary and higher education: a review of the literature. Lifelong Learn. Commun. Develop. 7, 69–86 (2012)

11 Parents’ Voices: Inclusion of Students with Intellectual …

175

42. Mara, D.: Higher education for people with disabilities–Romanian education experience. Procedia Soc. Behav. Sci. 142, 78–82 (2014) 43. Weinkauf, T.: College and university? You’ve got to be kidding: Inclusive postsecondary education for adults with intellectual disabilities. Crossing Boundaries 1(2), 28–36 (2002) 44. Blumberg, R., Carroll, S., Petroff, J.: Career and community studies: an inclusive liberal arts programme for youth with intellectual disabilities. Int. J. Incl. Educ. 12(5), 621–637 (2008)

Chapter 12

De-Identification of Student Writing in Technologically Mediated Educational Settings Langdon Holmes , Scott Crossley , Nick Hayes, Dylan Kuehl , Anne Trumbore , and Gabriel Gutu-Robu Abstract When conducting research with data from smart learning systems, there is a need to protect user identities because the release of personally identifiable information (PII) poses a significant risk to participants and creates a barrier to analyzing data and/or creating open datasets. Massive open online courses (MOOCs) are a good example of learning systems where PII concerns may hamper data analysis, the well-being of users, and system innovation. PII is particularly hard to locate and clean because of the variations in formatting, texts, and assignments found in unstructured data. In particular, identifying and removing students’ names has proven difficult. This study examines the potential to use large, pre-trained language models to deidentify MOOC data and compares performance on these language models to human annotations. On a validation set, a pre-trained language model fine-tuned using spaCy default hyperparameters achieved 97% recall of student names in the validation set, including partial matches, and 30% precision. On a larger, unseen test set (n = 3,077), the model achieved 93% recall and 24% precision. The majority of the false positives leading to lower recall in the test set were known names belonging to authors and/or lecturers. The results of the ensemble approach used here show considerable promise for a difficult de-identification task and indicate that automated de-identification is, likely, mature enough for use on some education datasets. Clearing PII from smart L. Holmes · S. Crossley (B) · N. Hayes · D. Kuehl Georgia State University, Atlanta, Georgia e-mail: [email protected] L. Holmes e-mail: [email protected] N. Hayes e-mail: [email protected] D. Kuehl e-mail: [email protected] A. Trumbore University of Virginia, Charlottesville, VA, USA G. Gutu-Robu University Politehnica of Bucharest, Bucharest, Romania © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_12

177

178

L. Holmes et al.

learning systems would ethically protect learners within the systems, allowing for the release of large datasets that could be analyzed for intelligent insights to forward innovation within smart learning systems.

Online learning systems have become an important component of many university systems and programs allowing for greater enrollment in courses, a more diverse student population, lower costs, and increased scalability [1]. There are, of course, drawbacks to online learning systems. One specific problem is that the large enrollments found in systems like Massive Open Online Courses (MOOCs) preclude interaction between teachers and students and, often, between students. One solution to low opportunities for interaction in MOOCs has been the inclusion of discussion forums that help increase student discussions as well as peer-reviewed assignment scoring that helps to generate student collaboration. While the interaction between students can assist in the learning process and is a valuable repository for educational data mining [2, 3], accessing, analyzing, and sharing student data from MOOC forums and assignments is complicated by issues related to personally identifiable information (PII) shared between students. This PII may include names, e-mail addresses, phone numbers, or addresses [4]. Because of the sensitive nature of PII in educational data and laws protecting students (e.g., the Family Educational Rights and Privacy Act [FERPA] in the United States, or the General Data Protection Regulation in the European Union), data containing PII cannot be publicly released and, in most cases, cannot be accessed by researchers without institutional approval. This creates significant barriers to analyzing data and/or creating open datasets that can help researchers create tools and interventions to help students maximize learning within smart learning systems. The simple solution to this problem is to remove PII from educational data so that they can be shared. For small datasets, this is time-consuming but possible. In the case of large datasets, PII needs to be removed automatically because the time and resources necessary to clean the data by hand are overwhelming. In some cases, automatically removing some types of PII from the large dataset is trivial, such as in the case of e-mail addresses or phone numbers that have common formatting. However, removing other types of PII like names and locations is nontrivial [4]. Computational approaches to removing names and locations rely mainly on named entity recognition (NER), which focuses on finding and annotating names, locations, and other instances of entities in the text [5]. While NER approaches traditionally work well on the domains they have been trained on, this training does not always transfer well to new domains [6, 7]. Another approach to removing PII is to use dictionaries of known words and remove these known words from educational data such that the remaining candidates are more likely to be named. Reference [4] successfully used this approach along with a variety of text-based features (e.g., whether the word was capitalized, where the word occurred in the sentence, and whether the word occurred in census lists) and machine learning techniques to classify PII information in MOOC forum posts.

12 De-Identification of Student Writing in Technologically …

179

Similar to [4], the goal of this study is to examine the potential to automatically annotate PII information (specifically names) in MOOC data. Unlike Bosch et al., we focus on assignments written by participants (as compared to forum posts). Additionally, we focus on the potential for newly developed transformer models to automatically annotate PII information. [8] used a similar approach to examine medical PII with structured and unstructured texts and found strong but imperfect recall. Successfully annotating large educational datasets for PII would allow enhanced research of educational conditions that consider privacy and consent concerns [9]. Research insights that come from PII-free data will help ensure a personalized experience for students in smart learning environments that are ethically based and protects the well-being of learners.

12.1 Methods 12.1.1 Critical Design MOOC Data was collected from a MOOC that focuses on teaching critical thinking skills from a design perspective. The course was the product of a collaboration between an educational content provider and a North American university. It is estimated that course completion required around six hours. Over 360,000 students had enrolled in the course at the time of data collection. Data includes posts in forum discussions, assignment submissions, assignment scores, and demographic data for the participants. From this data, we selected 6,293 users using the criteria discussed later. Selected demographic data for the dataset of 6,293 participants used in this study are presented in Figs. 12.1, 12.2, and 12.3. Not all data fields were available for all users. Of the top 10 countries (labeled according to the ISO-2 standard), India represented the most students, followed by the United States and Nigeria. In terms of educational attainment, the largest number of respondents reported at least a Master’s degree, closely followed by a bachelor’s degree. We derived the first language using available browser language codes provided by the users’ internet browsers when they use the MOOC’s web interface. Five of the top ten most frequent browser language codes represent varieties of English, and three represented varieties of Spanish. In terms of student gender (reported or inferred), there were more men (N = 2,531) than women (N = 1,527). No other genders were reported (or inferred).

12.1.2 Corpus The MOOC data was collected and stored in a relational database (PostgreSQL) that contains 111 tables. Compared to a single table, relational databases allow for more

180

L. Holmes et al.

Fig. 12.1 Most common countries for students in Critical Design MOOC. IN = India, US = United States, NG = Nigeria, MX = Mexico, BR = Brazil, AE = United Arab Emirates, DE = Germany, CO = Columbia, SA = Saudi Arabia, PH = Philippines

Fig. 12.2 Educational attainment level for students in Critical Design MOOC

information to be stored with less redundancy. Of interest to the current investigation were the assignment submissions from participants that had also made forum posts, both of which contained numerous instances of PII.

12 De-Identification of Student Writing in Technologically …

181

Fig. 12.3 Most common browser language codes for participants in Critical Design MOOC. en-US = English (USA), en-GB = English (Britain), en = English, es-ES = Spanish (Spain), en-IN = English (India), en-us = English (USA), es-41 = Spanish (Latin America), pt-BR = Portuguese (Brazil), fr-FR = French (Frances), es = Spanish

A single table stored 221,043 assignment submission events. Most of these submission events were associated with a publicly accessible URL, where the submission file was stored (for example, on an Amazon Web Service server). Each row also contained an identifying number that uniquely identified a user across the course, as well as metadata such as a submission timestamp and the contents of the “title” and “comments” fields that some students chose to fill out for each assignment. Each submission was graded based on the average of at least three peer review scores from other students participating in the MOOC. The average of these scores was also used as the course grade, serving as the sole discriminator between passing and failing the course. Despite this, some users were associated with multiple submissions that received passing scores. Due to uncertainty about the nature of these submissions, they were excluded from the analysis. Additionally, some submission events were associated with invalid URLs or files that were not provided in a PDF format. As the PDF format was an explicit requirement for the assignment, these participants were also excluded from this study. The remaining 32,525 assignments were automatically parsed using the PyMuPDF Python package. The texts of these files were then checked to ensure that they were written in English, contained at least 50 white-spacedelimited tokens, and were no longer than 5 pages in length. Automatic language detection was provided by the pycld2 Python package, which uses the same language detection engine as chromium-based web browsers. After checking for these parameters, 3,383 assignments were removed. The remaining 29,142 documents constitute the assignments portion of the corpus available for this analysis.

182

L. Holmes et al.

Of the final 29,142 assignments, 3,216 of these were submitted by users who also posted in the assignment submission forum. Since more text data was available from these users, they were prioritized for de-identification techniques. An additional 3,077 assignments not associated with users active in the discussion forum were also randomly selected and included as part of this investigation. In total, 6,293 assignment submissions were included in this study.

12.1.3 Human Annotations of Names To collect a gold standard for PII, we had human raters tag words and terms in each assignment as potential PII. The human raters were two undergraduate students at a major research university in the United States. The undergraduate students were working as interns on the project and were students in an applied linguistics department. The students were in their final year of study. Collaborative annotation was facilitated by Doccano [10], see Fig. 12.4, which is an open-source annotation software comparable with products such as TagTog and spaCy’s Prodigy. It allows for the specification of a labeling scheme and can export annotated documents in a JSON line format. When hosted on a web server, annotators can work on the project simultaneously without splitting and versioning the data. The following annotation guidelines were developed in house through discussion with the lead authors and the annotators. The guidelines were also made available through Doccano’s interface:

Fig. 12.4 Human annotation interface used to tag student names

12 De-Identification of Student Writing in Technologically …

183

Label students’ names as PERSON. Try to avoid labeling non-student names such as “Thomas Jefferson”. When in doubt, err on the side of caution and label the name as PERSON. Label physical addresses as LOC. These may include locations of schools, student residences, local businesses, or other locations that could identify the student. Label e-mail addresses E-MAIL and phone numbers PHONE.

Annotating the assignments was done independently by each rater, but raters and lead authors came together to discuss problematic instances. Discussions generally arose wherever there was doubt with regard to the most appropriate label for the PII. Another major concern regarded instructor’s names as well as published authors found in the course materials which frequently appeared in submission texts. Since it was difficult to discern whether these names might refer to students, annotators were instructed to take a conservative approach and label them as PERSON when in doubt. Over the course of annotation, names were identified as the most challenging type of PII to identify for several reasons. As opposed to emails, phone numbers, and URLs, names do not have any predictable patterns that distinguish them from dictionary words besides capitalization of the first letter, which is not always standardized. Additionally, some names are also dictionary words, such as “Rob”. Beyond these two concerns, the open nature of MOOCs leads to a diverse student population with a number of unique names. Furthermore, students have varying degrees of proficiency with English and various L1 backgrounds that can affect spelling, formatting, and other writing conventions. Because of the inherent difficulty of flagging student names and the strength of names to personally identify students, the present study focused exclusively on the identification of student names. In total, the human annotators flagged 1,347 names in the dataset of 6,293 assignments. This equated to.21 names per assignment with a standard deviation of 0.70 (and a maximum of 18 names in one assignment). These names included many conventional English names such as Brian, Andrew, and Diana as well as the unconventional spelling of English names like Sem, Walcoln, and Michell and non-English names including Fatima, Vidya, and Ricardo. The majority of names in the dataset were non-English names. Formatting of names was also non-standardized with many names being capitalized (e.g., AMIT, CHAWLA), including spaces between letters (e.g., M A R I N U S F E R N A N D O), or having different initialization structures (e.g., H. Borkar, Kumar V, Ashank T, C N T Sairam).

12.1.4 Automatic Annotation of Names Automatic annotation of student names was aided by the spaCy NLP engine [11] and Presidio (2022), an open-source Microsoft project that provides a suite of data classes and convenience functions for orchestrating automatic de-identification of sensitive data.

184

L. Holmes et al.

The identification of student names is a specific sub-problem of named entity recognition (NER), one of the core tasks in NLP and one that spaCy is particularly well-suited for. SpaCy’s best-performing model is “en_core_web_trf”, which is a transformer-based language model based on RoBERTa (Liu et al. 2020) that has been configured for general purpose NER and linguistic annotation. Since this is a general purpose model, it is not expected to outperform domain-specific systems. However, the model can be improved by training on a labeled dataset in a process called fine-tuning. The 3,216 assignment submissions from users who posted in the assignment forum were split 75/25 into a training set and a validation set. The SpaCy NER model was fine-tuned using the training set and the validation set using default hyperparameters to tag sequences labeled by human annotators as representing possible student names. The remaining 3,077 assignment submissions from users that did not post in the assignment forum were reserved as an out-of-bag set that was passed through the system only once, for the purpose of evaluation (i.e., a test set).

12.1.5 Evaluation and Analysis In order to evaluate the performance of the automatic annotation system described above, two criteria were established that differently define what constitutes a match between the human and the automated annotations. The first approach defines matches on a per-token basis. Thus, given a tokenized document and a list of spans annotated as student names, all tokens belonging to those spans can be labeled as “name”. All other tokens are not names. This approach has the benefit of being highly precise and also allowing for a well-defined measure of true negatives. However, trailing spaces and punctuation marks are often included as part of the labels, which may contribute to a counterintuitive estimate of the model’s performance. Also, as a result of inconsistent formatting in the corpus, some names may be represented as letters separated by spaces (eg. “J O H N M A L K O V I C H”). As each of these letters constitutes a spaCy token, it would weigh disproportionately in the evaluation statistics. As a result, the model’s performance was also evaluated on a “per name” basis. This analysis assumes that any overlap between the actual labels and the predicted labels constitutes exactly one true positive (predictions never overlap with each other). If there is a name that does not overlap with any predicted annotations, this constitutes exactly one false negative. While more straightforward to interpret, it is possible that the overlap between the human annotation and the automatic prediction may not be sufficient for the purpose of anonymization, so both evaluation strategies are reported. In calculating prediction rates, we rely on recall and precision. Recall is the percent of relevant instances retrieved (i.e., the number of true positives divided by the number of true positives and false negatives). Precision is the percentage of relevant instances among the recalled items (i.e., the number of true positives divided by the true positive and false positives). There is a general understanding that recall is substantially more

12 De-Identification of Student Writing in Technologically …

185

important to a de-identification system than precision [12] because false negatives do not strongly affect the purpose of the system. That is to say, the goal of a PII system is to remove named entities, and removing false positives likely will not strongly interfere with subsequent analyses. If recall is insufficient, all documents would need to be reviewed by a human annotator, rendering the automation inefficient. However, a system with poor precision could be paired with human annotators and not all documents would need to be reviewed. Because of these differences, we adopt an F β measure as our final measure of accuracy because it weights recall (β = 2) more highly than precision  2  β + 1 · pr ecision · r ecall Fβ − measur e = ;β = 2 β 2 · pr ecision + r ecall

12.2 Results 12.2.1 Validation Set The ensemble of the “en_core_web_trf” model and the domain-specific, fine-tuned model achieved a recall of 97% and a precision of 29%, F β = 65.90 (per name) on the validation set. Per token, the ensemble achieved 92% recall and 36% precision, F β = 69.47 on the validation set (see Figs. 12.5 and 12.6).

Fig. 12.5 Confusion matrix for model accuracy (per name) on validation set

186

L. Holmes et al.

Fig. 12.6 Confusion matrix for model accuracy (per token) on validation set

12.2.2 Test Set For the out-of-bag test set, the ensemble model achieved a per name recall of 93% and a precision of 24%, F β = 59.18. Per token, recall was calculated at 70% and precision was calculated at 30%, F β = 55.04 (see Figs. 12.7 and 12.8).

Fig. 12.7 Confusion matrix for model accuracy (per name) on test set

12 De-Identification of Student Writing in Technologically …

187

Fig. 12.8 Confusion matrix for model accuracy (per token) on test set

12.3 Discussion The goal of this paper was to develop and test an automated system for annotating PII information in student assignments found in a MOOC. Unlike previous systems used in educational PII annotation, our system focused on student assignments and leveraged a NER transformer-based language model. Overall, the system performed well in terms of recall reporting recall of 97% for names in the validation set and 93% for names in the test set. Precision was much lower at 29% and 24%, respectively, indicating that models are not ready for application in their current form. Overall, recall and precision were higher per name than per token and lower for the test set than the validation set. One explanation for this is that the test set represents a different subset of the MOOC data, namely users that did not post in the assignment forum. As this is a different sampling strategy, student names may appear with different patterns. The second explanation is that there may have been more names in which each letter was tokenized separately (e.g., “J O H N M A L K O V I C H”), and these names may appear several times in a single document in the test set resulting in dozens of false negatives on a per-token basis. A third explanation is that the test set was annotated prior to the training and validation sets. As a result, annotators may have been less consistent with their annotations and became more consistent over time. Post-hoc analyses of rater accuracy need to be undertaken to better understand differences in potential temporal differences. In a limited hand-analysis of the data, it was noted that at least one text described the student’s relationship and activity with a public figure in sufficient detail that the student could feasibly be identified as the public figure. Cases such as these, likely, pose significant challenges to automatic de-identification systems. A related issue with de-identification is triangulation. It is possible that several pieces of information that are not personally identifiable by themselves might be combined to identify a

188

L. Holmes et al.

student. Triangulation for automated systems would be difficult, but humans should be able to infer student names in this manner. The biggest problem in our dataset was the names in which each character was separated by a space. This formatting posed a problem for the ensemble models used here. It is relatively trivial to create a regular expression rule that would automatically tag these sequences as possible PII, but hand coding is needed to identify the problem initially. If a name with this pattern did not appear in the training and validation sets, it would likely not be correctly tagged by the system and could hypothetically result in the consistent leaking of PII. It is unknown how common this structure is in similar datasets and there is the possibility that the formatting was a result of automatically parsing the PDF files into text files.

12.4 Conclusion In this study, we have introduced an automated model to identify PII in student assignments in a MOOC. The model performed well on recall in the validation and test set, but still leaves room for error (i.e., the model missed between 3 and 6% of names in the dataset). Knowing that any leak of PII could influence the integrity of a system, work is still needed on automatic PII removal. There are also solutions to PII leaks that mix automated PII systems with other approaches to ensure student privacy is protected. For instance, pseudonymization of identified names could help obfuscate student data and help protect student identities. In this case, identified names would be randomly switched among essays to ensure that the writer’s work and ideas are not identifiable. Additionally, multi-token names (e.g., first and last names) could be broken apart and randomly resampled. Lastly, data enclaves can be used in which researchers never have physical access to the texts. In these instances, researchers can access data with PII through a secure network in which data can be stored and analyzed. Researchers cannot access the texts, but they can process and analyze the data. Overall, this study presents the promise of automated PII classification which would open up the potential for large public datasets based on educational settings. These datasets could help researchers create tools and interventions to help students maximize learning in smart systems in an ethical and open manner that protects the well-being of learners.

12 De-Identification of Student Writing in Technologically …

189

References 1. Anjum, M. M., Mohammed, N., Jiang, X.: De-identification of unstructured clinical texts from sequence to sequence perspective. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 2438–2440 (2021). https://doi.org/10.1145/346 0120.3485354 2. Bosch, N., Crues, R. W., Shaik, N.: “Hello, [REDACTED]”: Protecting student privacy in analyses of online discussion forums. In: Proceedings of the 13th International Conference on Educational Data Mining, Vol. 11 (2020) 3. Chen, B., Chang, Y.H., Ouyang, F., Zhou, W.: Fostering student engagement in online discussion through social learning analytics. Internet Higher Educat. 37, 21–30 (2018). https://doi. org/10.1016/j.iheduc.2017.12.002 4. Crossley, S., Paquette, L., Dascalu, M., McNamara, D.S., Baker, R.S: Combining click-stream data with NLP tools to better understand MOOC completion. In: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 6–14. New York, NY, USA (2016) 5. Deming, D.J., Goldin, C., Katz, L.F., Yuchtman, N.: Can online learning bend the higher education cost curve? Am. Econom. Rev. 105(5), 496–501 (2015). https://doi.org/10.1257/aer. p20151024 6. Ferrández, O., South, B.R., Shen, S., Friedlin, F.J., Samore, M.H., Meystre, S.M.: Evaluating current automatic de-identification methods with Veteran’s health administration clinical documents. BMC Med. Res. Methodol. 12(1), 109 (2012). https://doi.org/10.1186/1471-228812-109 7. Gayed, J.M., Carlon, M.K.J., Oriola, A.M., Cross, J.S.: Exploring an ai-based writing assistant’s impact on English language learners. Comput. Educat. Artific. Intell. 3, 100055 (2022). https:// doi.org/10.1016/j.caeai.2022.100055 8. Honnibal, M., Montani, I., Van Landeghem, S., Boyd, A.: spaCy: Industrial-strength Natural Language Processing in Python [Python]. Explosion AI (2020) 9. Jiang, R., Banchs, R.E., Li, H.: Evaluating and combining named entity recognition systems. In: Proceedings of the Sixth Named Entity Workshop, pp. 21–27 10. Kleinberg, B., Mozes, M., Arntz, A., Verschuere, B.: Using named entities for computer automated verbal deception detection. J. Forensic Sci. 63(3), 714–723 (2018). https://doi.org/10. 1111/1556-4029.13645 11. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: A Robustly Optimized BERT Pretraining Approach. http://arxiv.org/ abs/1907.11692 12. Murugadoss, K., Rajasekharan, A., Malin, B., Agarwal, V., Bade, S., Anderson, J.R., Ross, J.L., Faubion, W.A., Halamka, J.D., Soundararajan, V., Ardhanari, S.: Building a best-inclass automated de-identification tool for electronic health records through ensemble learning. Patterns 2(6), 100255 (2021). https://doi.org/10.1016/j.patter.2021.100255 13. Nakayama, H., Kubo, T., Kamura, J., Taniguchi, Y., Liang, X.: doccano: Text Annotation Tool for Human (2018). https://github.com/doccano/doccano 14. Nanda, G., Douglas, K.A.: Machine learning based decision support system for categorizing MOOC discussion forum posts. In: Proceedings of the 12th International Conference on Educational Data Mining (EDM 2019), pp. 619–622 (2019) 15. Meystre, S.M., Friedlin, F.J., South, B.R., Shen, S., Samore, M.H.: Automatic de-identification of textual documents in the electronic health record: a review of recent research. BMC Med. Res. Methodol. 10, 70 (2010). https://doi.org/10.1186/1471-2288-10-70 16. Presidio—Data Protection and Anonymization API. (2022). [Python]. Microsoft. https://git hub.com/microsoft/presidio. Original work published 2018 17. Young, E.M.: Educational privacy in the online classroom: FERPA, MOOCS, and the Big Data Conundrum. Harvard J. Law Technol. 28(2) (2015)

Chapter 13

Conceptualising Micro-credentials in the Higher Education Research Landscape. A Literature Review Alexandru Cart, is, , Janika Leoste , Romit, a˘ Iucu , Kaido Kikkas , Kalle Tammemäe , and Katrin Männik Abstract The educational research landscape is still in a very early stage regarding the research in micro-credentials and how they are defined in scientific and policy documents. Whereas the micro-credentials trend is on rapid rise at the international and European level, coherent approaches to the concept, understanding and use of them are yet to be clarified. We addressed the issue through a literature review on the existing terminology and understandings related to micro-credentials in higher education in current scientific papers and policy documents, clustering them based on the granulation level from small units of learning to standalone certifications. Research shows that further research is needed to create a coherent approach for defining what micro-credentials are and how they can be integrated into educational processes, especially in higher education. Even though policy documents and scientific papers converge in numerous cases on creating a common definition for microcredentials, differences are still present. The vision and practice do not always concur when describing the meaning and use of these innovative certification instruments.

A. Cart, is, (B) · R. Iucu University of Bucharest, Panduri St. 90, 050663, Bucharest, Romania e-mail: [email protected] R. Iucu e-mail: [email protected] J. Leoste · K. Kikkas · K. Tammemäe Tallinn University of Technology, Ehitajate tee 5, 12616, Tallinn, Estonia e-mail: [email protected] K. Kikkas e-mail: [email protected] K. Tammemäe e-mail: [email protected] K. Männik Tallinn University, Narva mnt 25, Tallinn, Estonia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_13

191

192

A. Cart, is, et al.

13.1 Introduction Rapidly evolving and emerging areas in technology, such as artificial intelligence, cyber security, robotics, virtual and augmented reality, quantum computing, etc., create demand for people with specific up-to-date skills. On the one hand, only 3% of employers consider a university degree as somewhat important in several key sectors [1]. Instead, they seek people with certain specific competencies relevant to their hiring needs. On the other hand, micro-credentials can be perceived as an added value to academic programmes, making it possible to acknowledge competencies not necessarily included in traditional programmes, mitigating discrepancies between labour market needs and academic offers [2]. In this regard, there can be a balance between academic practices and micro-credentials use, since the latter can “strengthen traditional degree programmes, support competency-based programmes, and link badge earners to potential employers and professional organisations” [3]. Indeed, it has been suggested that the relative institutional autonomy of universities, together with the academic freedom they enjoy, have allowed them to provide education that is not always focused on practice and can be irrelevant to job market needs [4]. To combat this gap between job market needs and somewhat inflexible curricula of educational facilities, governments together with business stakeholders have started lobbying in order to re-orient higher education to adopt a narrower focus on preparing students for work [4]. In this context, the concept of micro-credentials has been used as an opportunity to bridge the skill gap between formal education and real-world work requirements [2], helping companies to remain competitive and adaptive. Micro-credentials in this context are certified documentary proofs of skills acquired from certain short-term educational or training activities [5]. Micro-credentials are an approach where the boundaries between formal and informal learning are less defined—student competency can not only be validated and certified by universities, but also by third parties [5]. However, for universities, micro-credentials approaches can become challenging, as they need to overcome their existing knowledge transfer paradigms and adopt active learning models with authentic assessment scenarios instead [2]. Additional obstacles for universities can be the reluctance of their upper management to understand micro-credentials, their aversion to changes, lack of resources, and inability to access and interpret up-todate data about the latest market needs [5]. Yet, as micro-credentials are considered a valuable approach for universities to become able to address the rapidly changing needs of employers and job seekers [1], it is necessary to study this topic more deeply. In this paper, we are examining the current research on micro-credentials in the context of higher education. Our study is guided by the following research question (RQ): How are micro-credentials linked to higher education practices presented in current research papers and policy documents, in terms of definitions and connected concepts?

To answer this, we conducted a literature review on the existing academic papers and policy documents that tackle the use of micro-credentials in higher education.

13 Conceptualising Micro-credentials in the Higher Education …

193

13.2 Methodology We propose analysing the current research interest expressed in defining and conceptualising micro-credentials and their relationship with the higher education landscape. To answer our research question, we propose analysing academic papers and policy documents using the following criteria: 1. The documents are not focused on study cases about specific programmes or platforms designed and implemented for delivering micro-credentials or microcredentials-like certifications. 2. The main interest around the micro-credential definitions is focused on higher education and tertiary education or lifelong learning at universities. 3. The documents are either academic journal papers, conference proceedings papers, or public policy documents relevant to the higher education sector. 4. Micro-credentials as a concept are clearly defined, and the authors mention some of the opportunities and challenges linked with implementing micro-credentials in higher education practices. The current research landscape on micro-credentials is not so rich, since many of the developments around the concepts have flourished in just the past two or three years, especially because of the COVID-19 pandemic forcing governments and public authorities to turn their focus on educational needs, and universities were seeking new ways for reaching out to the educational beneficiaries [4]. Of course, in many cases, the alternative terminologies were present even before, being mainly referred to by aspects that are today grasped by micro-credentials—be it digital badges [3, 6– 8], alternative credentials [9–11], or terminologies linked to modular education [12, 13]. Nevertheless, as the current literature and policy-level developments show, the concepts are not to be confused or mixed, but clearly differentiated and defined [7]. This is also to be seen when analysing the concepts’ presence through the Google Ngram Viewer platform (Fig. 13.1), where “micro-credential” is meaningful only since 2010 [14]. To identify the scientific literature subject to our study, we conducted a search for scientific papers in Scopus and Web of Science databases, including journal

Fig. 13.1 Frequency of the term “micro-credential” in books indexed by Google [14]

194

A. Cart, is, et al.

papers, book chapters, proceedings and other publications present in the databases. The first round of results offered us only papers and documents published after 2014, reinforcing what the Google Viewer suggested. These two scholarly sources were considered since the papers published have undergone a strict evaluation process that assures the quality of the contents, as well as the scientific rigour of the research process. We used the following search string: “micro-credentials” OR “microcredentials” OR “micro credentials” OR “micro-credential” OR “microcredential” OR “micro credential”, for all fields of search (topic, title, abstract, etc.), since there are some cases in which the spelling differentiates between these three formats. For the purpose of our paper, we will use only the format “micro-credential(s)”. Based on this search string, we obtained a total number of 411 sources, which was further reduced to 353 after we removed the duplicates. Further on, we removed the sources which did not fall under the criteria we established for the search, meaning here the sources which were referring to specific case studies or descriptive analyses, and the ones that were not linked at all with the higher education sector or were not relevant to our study. During this stage, we removed 228 sources, remaining with 125 sources to be analysed in depth for complying with the criteria. The analysis of the remaining sources was conducted manually by the researchers by reading the full text and selecting the papers which included definitions and conceptualizations of micro-credentials, as well as opportunities and challenges linked with implementing micro-credentials in higher education practices. During this stage, 17 papers were eliminated because of missing full text, and 40 more were removed because they did not contain information relevant to our study. After this process, 68 documents were selected for review. The entire selection process was systematised according to the PRISMA guidelines applicable to the literature review [15, 16] (Fig. 13.2). Most of the papers were not entirely focused on analysing micro-credentials as they are and their implications for the higher education sector but tackled these understandings and concepts in aspects especially linked to the use of alternative credentials in teaching and learning practices as well as the emergence of digital credentials and open badges in education. As it can be seen, more than half of the selected papers (36 out of 68) had the concept of “micro-credentials” included in the title (in the various formats already mentioned), whereas a smaller number of 21 selected papers focused mainly on digital badges and digital credentials. To tackle some policy-level conceptualizations related to the implementation of micro-credentials in the higher education sector, we included some extra documents issued by high-level stakeholders in education, such as [13, 17–26], or the input provided by some of the European Universities [27, 28], considering their major involvement in this topic and in supporting national authorities and universities to overcome barriers and challenges in adopting micro-credentials in the academic practices.

13 Conceptualising Micro-credentials in the Higher Education …

195

Fig. 13.2 PRISMA flowchart presenting the selection of papers

13.3 Results: Defining and Understanding Micro-credentials Based on the analysis of the selected papers and documents for our study, we noted that there are many ways in which micro-credentials are defined and characterised by different authors and institutions, confirming the notions of some researchers [4, 7, 9] that there is a lack of consistency and cohesion about what micro-credentials are and how they are understood. In most cases, very broad and comprehensive definitions are used [5, 17]—researchers tend to avoid offering complete definitions but rather describe different components that build up micro-credentials and their use in education and training. In the European context, there is an important movement for the adoption of microcredentials in the higher education sector, somewhat influenced and supported on the policy level [17–23, 25, 27–29]. However, this approach can also benefit (perfectly in some cases) from the research areas that are not necessarily emerging from policylevel perspectives, as can be seen below (see Table 13.1). It also identifies the most

196

A. Cart, is, et al.

appropriate means for creating a cohesive approach for defining and conceptualising micro-credentials in the context of higher education processes and practices. In order to answer our research question and identify the main areas where the selected papers and documents define micro-credentials, we proposed a series of clusters that group the views of micro-credentials and their definitions in current literature (Table 13.1). Notably, some resources express multiple perspectives on defining micro-credentials and are present in more than one cluster. Our analysis shows that we cannot adhere to any specific definition or understanding of what micro-credentials are and how they should be understood. Yet, similarities can be identified between descriptions from the same cluster, as we can see, Table 13.1 Clusters of definitions for micro-credentials in selected documents Cluster

Description

Resources

Standalone certifications

Micro-credentials are [2, 5, 9, 17–23, 25, 28–35] solid certifications that can be used without being part of a larger certification context (the micro-credentials can become parts of larger certifications, but it is not mandatory)

Micro-credentials are [2, 4, 7–9, 12, 13, 20, 22, 29–31, 36–50] Components of large certifications viewed as building blocks for large certifications (based on stack-ability and recognition) and not as standalone certifications (small learning proofs as parts of larger educational framework) Small units of learning

Micro-credentials are viewed as educational components, as small learning units, and not necessarily as certifications at all

[4, 8, 37, 39, 51–59]

Umbrella concepts Micro-credentials are [6–8, 13, 24, 29, 35–37, 42, 52, 53, 57, 59–64] viewed as an umbrella concept that includes variations of certifications and small learning components (such as badges, nanodegrees, micro-masters, small courses, etc.)

13 Conceptualising Micro-credentials in the Higher Education …

197

e.g., in the following two definitions from the first cluster: the Commission’s definition of micro-credentials [17]: “A micro-credential is a proof of the learning outcomes that a learner has acquired following a short learning experience. These learning outcomes have been assessed against transparent standards”, and the definition by [5]: “Micro-credentials are certified documents that provide recognized proof of the achievement of learning outcomes from shorter, less duration, educational or training activities. They focus on the validation of competency-based skills, outcomes and/or knowledge using transparent standards and reliable assessments, which can enhance graduates’ employability prospects”. We notice identical concepts being used in both definitions, such as “proof” (referring to the role of the credential), “learning outcomes” (referring to the evidence proved by the credential), “short” (referring to the duration of the learning activity), and “assess/assessment” (referring to the method of attesting the solidity of the contained outcomes, competencies, or skills). Similar analyses would be of high interest for future research and possibly developing a coherent definition for micro-credentials that would assure widespread functionality and understanding in higher education teaching and training. During the analysis, we also noticed some interesting concepts connected to micro-credentials and their understanding and pointing out possible variations of terms and uses that can be linked to micro-credentials and their adoption in educational processes. For instance: “certifications of mastery of specific skills” [37], “education playlist” [33], “educational awards” [4], “educational credits” [12], “industry-aligned short units of learning” [4], “micro competencies” [56], “portable currency for professional learning” [35], “representation of an accomplishment” [7], or “tokens” [46]. This terminological variation shows that defining micro-credentials offers a heterogeneous conceptual landscape with sufficient resources for further investigation and development in this area, either in direct relation to the academic and educational certifications, as a central area of interest for developing these tools, but also in connection to an active and dynamic labour market, with its own needs for specific and targeted proofs of competencies and skills linked to job descriptions and qualifications.

13.4 Conclusions and Discussion The micro-credential conceptual landscape manifests a variety of formats and understandings since there are still unclear interconnections between the terminology used in traditional credentials and the new approaches for certifications [65]. Current literature shows us that creating a micro-credentials philosophy in higher education curriculum design does not only impose more coherent approaches in defining the concept and its uses [9], but also change how academic institutions favour active learning models and authentic assessment scenarios [39], offering a common ground for both “traditional” knowledge and labour market skills [66]. Moreover, these approaches can further be analysed in comparison with the European approach [17]

198

A. Cart, is, et al.

and with other policy-level documents that are relevant for developing a clear terminology. The paper does not propose going further from this approach, nor does it criticise the definitions and concepts that are proposed at that level, but offers a clear image of how the topic of micro-credentials is encountered in the scientific literature and how they are mirrored in different policy-level documents. It is a clear understanding that micro-credentials can have a measurable value [46], but not if they are continuously regarded only as “add-ons” and “nice-to-have” educational components to existing curricular constructs, but more like a macro-level strategic approach for reconsidering educational finalities in higher education [9, 20, 28]. More research is still needed to further understand and test the real presence and impact of micro-credentials in higher education curriculum, balancing a “trend-like” presence in academia [9, 40] and the discourse of employability skills in higher education practices [4]. The present paper did not proceed further into analysing how the current literature promotes or criticises micro-credentials, in relation to their potential opportunities and misuses. Still, an overarching analysis of the terminology used and how micro-credentials are linked with different parts of the entire academic certification process shows that decision-makers, higher education institutions, and labour market representatives need to focus on creating a more transparent and operational conceptual framework for building a micro-credentials philosophy in curricular innovation, based on prior experiences and practices in the sector. Further, data collection and research are needed to design, implement and assess high-quality micro-credentials [36] in higher education, especially in association with the growing demands of the labour market [9]. Our research represents a step forward in identifying and consolidating common understanding and uses for micro-credentials and new forms of certifications in academia, not only in relation to the European and international political perspectives, but also based on the existing research literature. The mapping process exposes a wide range of terms and concepts used for describing how micro-credentials are integrated into higher education strategies and practices, and what are the alternatives used for characterising these new approaches for certification. From definitions to tools and impacts, the range of topics and terms used for describing micro-credentials’ presence in higher education highlights that we are still in an emerging phase— either struggling to argue for umbrella concepts that encompass previous “known” tools and processes, or to present the innovative stance as a universal reply for all the challenges in the academia. Whereas both extremes have their own rights and wrongs, the growing trend of micro-credentials’ use in higher education forces all stakeholders to a more balanced approach, starting from what we also sought, a common understanding of the concept. Our future research will concentrate on the opportunities and challenges of integrating micro-credentials in higher education and, starting from the findings mentioned in this study, consider testing pilot processes for designing and implementing micro-credentials-based educational components at universities, linked to labour market needs and with curricular integration. Micro-credentials cannot be regarded as a universal panacea for solving all legislative and institutional barriers that block curricular innovation and paradigmatic changes in higher education [5], but some approaches can still create a proper setting

13 Conceptualising Micro-credentials in the Higher Education …

199

for micro-credentials design to really impact how universities respond to future societal demands. This is one of the directions in which micro-credentials can enhance what universities already do well and open new means for other forms of collaboration with the public society and the economic sector. Enriching the educational landscape—addressing modern skills and competencies and creating an integrated training framework that puts universities at the forefront of educational services, always considering rigour and quality—can benefit from further investigating what is the added value of a coherent approach to micro-credentials in the higher education sector.

References 1. Lim, C.L., Nair, P.K., Keppell, M., Hassan, N., Ayub, E.: Developing a framework for the university-wide implementation of micro-credentials and digital badges: a case study from a Malaysian Private University. In: 2018 IEEE 4th International Conference on Computer and Communications (ICCC), 1715–1719 (2018). https://doi.org/10.1109/CompComm.2018.878 0706 2. Maina, M.F., Ortiz, L.G., Mancini, F., Melo, M.M.: A micro-credentialing methodology for improved recognition of HE employability skills. Int. J. Educ. Technol. High. Educ. 19, 10 (2022). https://doi.org/10.1186/s41239-021-00315-5 3. Carey, K.L., Stefaniak, J.E.: An exploration of the utility of digital badging in higher education settings. Educat. Tech. Res. Dev. 66, 1211–1229 (2018). https://doi.org/10.1007/s11423-0189602-1 4. Wheelahan, L., Moodie, G.: Analysing micro-credentials in higher education: a Bernsteinian analysis. J. Curric. Stud. 53(2), 212–228 (2021). https://doi.org/10.1080/00220272.2021.188 7358 5. McGreal, R., Olcott, D., Jr.: A strategic reset: micro-credentials for higher education leaders. Smart Learn. Environ. 9, 9 (2022). https://doi.org/10.1186/s40561-022-00190-1 6. Clements, K., West, R.E., Hunsaker, E.: Getting Started with open badges and open microcredentials. Int. Rev. Res. Open Distributed Learn. 21(1), 154–172 (2020). https://doi.org/10. 19173/irrodl.v21i1.4529 7. Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., Knight, E.: Digital badges in education. Educ. Inf. Technol. 20, 403–410 (2015). https://doi.org/10.1007/s10639-013-9291-7 8. Elliott, R., Clayton, J., Iwata, J.: Exploring the use of micro-credentialing and digital badges in learning environments to encourage motivation to learn and achieve. In: Hegarty, B., McDonald, J., Loke, S.-K. (eds.) Rhetoric and Reality: Critical perspectives on educational technology. Proceedings Ascilite Dunedin 2014, 703–707 (2014). https://ascilite2014.otago.ac.nz/files/con cisepapers/276-Elliott.pdf 9. Kato, S., Galán-Muros, V., Weko, T.: The emergence of alternative credentials (Education Working Paper No. 216). OECD Publishing (2020). https://doi.org/10.1787/b741f39e-en 10. Jirgensons, M., Kapenieks, J.: Blockchain and the future of digital learning credential assessment and management. J. Teacher Educat. Sustain. 20(1), 145–156 (2018). https://doi.org/10. 2478/jtes-2018-0009 11. Fong, J., Janzow, P., Peck, K.: Demographic Shifts in Educational Demand and the Rise of Alternative Credentials. Pearson (2016) 12. Mugayitoglu, B., Borowczak, M., Burrows, A.: A University’s developmental framework: creating, implementing, and evaluating a K-12 teacher cybersecurity micro-credential course. Systemics, Cybernet. Informat. 19(2), 13–22 (2021). http://www.iiisci.org/Journal/pdv/sci/ pdfs/EB219EV21.pdf

200

A. Cart, is, et al.

13. International Council for Open and Distance Education: ICDE Report of the ICDE working group on The Present and Future of Alternative Digital Credentials (ADCS), 1– 54 (2019). https://www.icde.org/knowledge-hub/2019/4/10/the-present-and-future-of-altern ative-digital-credentials 14. Google Ngram Viewer. https://books.google.com/ngrams/graph?content=micro-credential& year_start=2000&year_end=2019&corpus=26&smoothing=3. Last accessed 16 Apr 2022 15. Tricco, A.C., Lillie, E., Zarin, W., O’Brien, K.K., et al.: PRISMA extension for scoping reviews (PRISMAScR): checklist and explanation. Ann. Intern. Med. 169(7), 467–473 (2018). https:// doi.org/10.7326/M18-0850 16. Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., Clarke, M., Devereaux, P.J., Kleijnen, J., Moher, D.: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J. Clin. Epidemiol. 62(10), e1–e34 (2009). https://doi.org/10.1016/j.jclinepi.2009. 06.006 17. European Commission: A European approach to micro-credentials. Output of the Microcredentials Higher Education Consultation Group: Final Report. Publications Office (2021). https://doi.org/10.2766/30863 18. European Commission: Commission Communication on a European strategy for universities. COM(2022) 16 final (2022). https://education.ec.europa.eu/document/commission-commun ication-on-a-european-strategy-for-universities. Last accessed 16 Apr 2022 19. Debiais-Sainton, V.: European approach to micro-credentials. EADTU Innovating Higher Education 2020 Bridging Event (I-HE2020), https://www2.slideshare.net/EADTU/ihe2020european-approach-to-microcredentials. Last accessed 16 Apr 2022 20. MICROBOL: Micro-credentials linked to the Bologna Key Commitments. Desk Research Report (2020). https://microcredentials.eu/wp-content/uploads/sites/20/2020/09/ MICROBOL-Desk-Research-Report.pdf. Last accessed 16 Apr 2022 21. Orr, D., Pupinis, M., Kirdulyt˙e, G.: Towards a European approach to micro-credentials: a study of practices and commonalities in offering micro-credentials in European higher education. Publications Office of the European Union (2020). https://data.europa.eu/doi/https://doi.org/ 10.2766/7338. Last accessed 16 Apr 2022 22. Sood, I., Padrón, C.L., Pirkkalainen, H., Camilleri, A.: D2.2 Future Impacts of Modularisation and MicroCredentials on European Higher Education and list of validated scenarios. MicroHE Consortium (2020). https://microcredentials.eu/wp-content/uploads/sites/20/2021/05/D2.2. Future-Impacts-of-Micro-Credentialling-on-European-Higher-Education.pdf. Last accessed 16 Apr 2022 23. MicroHE: Challenges and Opportunities of Micro-Credentials in Europe. Briefing Paper on the Award, Recognition, Portability and Accreditation of Micro-Credentials: an Investigation through Interviews with Key Stakeholders & Decision Makers. MicroHE Consortium (2019). https://microcredentials.eu/wp-content/uploads/sites/20/2021/05/D3.1-Interviews-with-KeyStakeholders-Decision-Makers-Overall-Summary-Report.pdf. Last accessed 16 Apr 2022 24. UNESCO: Digital credentialing: Implications for the recognition of learning across borders. UNESCO Education Sector (2018). https://unesdoc.unesco.org/ark:/48223/pf0000264428 25. EHEA: Rome Ministerial Communiqué (2020). http://www.ehea.info/Upload/Rome_Ministe rial_Communique.pdf, last accessed 2022/04/16. 26. International Labour Organisation: Report on employment in Africa (Re-Africa). Tackling the youth employment challenge. International Labour Office (2020). https://www.ilo.org/ wcmsp5/groups/public/---africa/---ro-abidjan/documents/publication/wcms_753300.pdf. Last accessed 16 Apr 2022 27. Brown, M., Mhichíl, M.N.G., Mac Lochlainn, C., Pirkkalainen, H., Wessels, O.: Paving the Road for the Micro-credentials Movement - ECIU University White Paper on MicroCredentials. ECIU University (2021). Zenodo. https://doi.org/10.5281/zenodo.4438507

13 Conceptualising Micro-credentials in the Higher Education …

201

28. Iucu, R., Ciolan, L., Nedelcu, A., Cartis, A.: Why micro-credentials should become educational “macro-policies” for defining the future European study programmes. University of Bucharest (2021). https://doi.org/10.5281/zenodo.6088135 29. Brown, M., Mhichíl, M.N.G., Beirne, E., Mac Lochlainn, C.: The global micro-credential landscape: charting a new credential ecology for lifelong learning. J.Learn. Develop. 8(2), 228–254 (2021). https://jl4d.org/index.php/ejl4d/article/view/525. Last accessed 16 Apr 2022 30. Commonwealth of Learning: Designing and Implementing Micro-credentials: A Guide for Practitioners (2019). http://oasis.col.org/handle/11599/3279. Last accessed 16 Apr 2022 31. Oliver, B.: Making micro-credentials work for learners, employers and providers. Deakin University (2019). https://wordpress-ms.deakin.edu.au/dteach/wp-content/uploads/sites/103/ 2019/08/Making-micro-credentials-work-Oliver-Deakin-2019-full-report.pdf?_ga=2.728 45779.1580040068.1625670555-368361593.1616148903, last accessed 2022/04/16. 32. Gallagher, S.R.: Educational Credentials Come of Age. Northeastern University (2018) 33. Jones-Schenk, J.: Alternative credentials for workforce development. J. Contin. Educ. Nurs. 49(10), 449–450 (2018). https://doi.org/10.3928/00220124-20180918-03 34. Ryerse, M.: Competency-Based micro-credentials are transforming professional learning. Getting Smart (2017). https://www.gettingsmart.com/2017/11/07/micro-credentials-transform ing-professional-learning/. Last accessed 16 Apr 2022 35. Berry, B., Airhart, K.M., Byrd, P.A.: Microcredentials: teacher learning transformed. Phi Delta Kappan (2016). https://doi.org/10.1177/0031721716677260 36. Burrows, A.C., Borowczak, M., Mugayitoglu, B.: Computer science beyond coding: partnering to create teacher cybersecurity microcredentials. Educat. Sci. 12(4), 1–13 (2022). https://doi. org/10.3390/educsci12010004 37. Hotaling, L., Van Sumeren, H.: The case for microcredentials for workforce preparation. Technical Note. Marine Technol. Soc. J. 56(1), 35–39 (2022). https://doi.org/10.4031/MTSJ. 56.1.5 38. Oliver, B.: Micro-credentials: a learner value framework. J. Teach. Learn. Graduate Employability, 12(1), 48–51 (2021). https://doi.org/10.21153/jtlge2021vol12no1art1456 39. Sokhanvar, Z., Salehi, K., Sokhanvar, F.: Advantages of authentic assessment for improving the learning experience and employability skills of higher education students: a systematic literature review. Stud. Educ. Eval. 70, 01030 (2021). https://doi.org/10.1016/j.stueduc.2021. 101030 40. Ward, R., Philips, O., Bowers, D., Crick, T., Davenport, J., Hanna, P., Hayes, A., Irons, A., Prickett, T.: Towards a 21st century personalised learning skills taxonomy. In: 2021 IEEE Global Engineering Education Conference (Educon 2021), pp. 344–354 (2021). https://doi. org/10.1109/EDUCON46332.2021.9453883 41. Zouri, M., Ferworn, A.: An ontology-based approach for curriculum mapping in higher education. In: 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC), pp. 141–147 (2021). https://doi.org/10.1109/CCWC51732.2021.9376163 42. Coyne, J., Hollas, T., Lane, M., Ellis, C.: Microcredentials: A Promising Professional Development Model for Teacher Leaders. International Journal of Innovation and Research in Educational Sciences, 6(5), 647–654 (2019). https://www.ijires.org/index.php/issues?view=public ation&task=show&id=539, last accessed 2022/04/16. 43. Sai, S. G., Unique, P., Vineesha, P.: A Blockchain Based Micro-Credentialing System. Deakin University (2019). https://doi.org/10.13140/RG.2.2.18403.20008 44. Fishman, B., Teasley, S., Cederquist, S.: Micro-Credentials as Evidence of College Readiness: Report of an NSF Workshop. University of Michigan (2018). https://hdl.handle.net/2027.42/ 143851, last accessed 2022/04/16. 45. Brown, D., Rhodes, D.E.: Show what you know. Phi Delta Kappan (2017). https://doi.org/10. 1177/0031721717708293

202

A. Cart, is, et al.

46. Grech, A., Camilleri, A.F.: Blockchain in Education. A. Inamorato dos Santos (Ed.). Publications Office of the European Union (2017). https://doi.org/10.2760/60649 47. Milligan, S., Kennedy, G.: To what degree? Alternative micro-credentialing in a digital age. In R. James, S. French, & P. Kelly (Eds.), Visions for Australian Tertiary Education, 41–53 (2017). https://melbourne-cshe.unimelb.edu.au/__data/assets/pdf_file/0006/226 3137/MCSHE-Visions-for-Aust-Ter-Ed-web2.pdf, last accessed 2022/04/16. 48. Hall-Ellis, S.D.: Stackable micro-credentials – a framework for the future. The Bottom Line 29(4), 233–236 (2016). https://doi.org/10.1108/BL-02-2016-0006 49. Lewis, M.J., Lodge, J.M.: Keep calm and credential on: linking learning, life and work practices in a complex world. In D. Ifenthaler, N. Bellin-Mularski & D.-K. Mah (Eds.), Foundation of Digital Badges and Micro-Credentials. Demonstrating and Recognizing Knowledge and Competencies, pp. 41–54. Springer (2016). https://doi.org/10.1007/978-3-319-15425-1_4 50. Gibb, S.: Soft skills assessment: theory development and the research agenda. Int. J. Lifelong Educ. 33, 455–471 (2014) 51. Joseph, M.A., Natarajan, J., Buckingham, J., Al Noumani, M.: Using digital badges to enhance nursing students’ attendance and motivation. Nurse Educ. Pract. 52, 103033 (2021). https:// doi.org/10.1016/j.nepr.2021.103033 52. Kishore, S., Chan, J., Muthupoltotage, U.P., Young, N., Sundaram, D.: Blockchain-based Micro-credentials: Design, Implementation, Evaluation and Adoption. Proceedings of the 54th Hawaii International Conference on System Sciences, 6846–6854 (2021). https://doi.org/10. 24251/HICSS.2021.821 53. McGovern, T.M., Gogan, J.L.: Can digital badging help universities flexibly support students and faculty during crises? A proposal and call for research. In: Proceedings of the 54th Hawaii International Conference on System Sciences, pp. 6339–6348 (2021). https://doi.org/10.24251/ HICSS.2021.765 54. Peisachovich, E.H., Dubrowski, A., Da Silva, C., Kapralos, B., Klein, J.E., Rahmanov, Z.: Using simulation-based methods to support demonstration of competencies required by microcredential courses. Cureus 13(8), e16908 (2021). https://doi.org/10.7759/cureus.16908 55. Gauthier, T.: The value of microcredentials: the employer’s perspective. J. Competency-Based Educat. 5(2), e01209 (2020). https://doi.org/10.1002/cbe2.1209 56. Song, Y.: Alignment of deliberate practice to micro-credentials in an introductory computer science course. In: 2019 IEEE Frontiers in Education Conference (FIE), pp.1–4 (2019). https:// doi.org/10.1109/FIE43999.2019.9028427 57. Daellenbach, K.: On carrot cake and marketing education: a perspective on balancing skills for employability. Austral. Marketi. J. (AMJ) 26(2), 172–179 (2018). https://doi.org/10.1016/ j.ausmj.2018.04.005 58. Powell, S., Leary, H., McLachlan, L., Brock, K.: Teachers collaboratively creating microcredentials for professional development. In: Kay, J., Luckin, R. (eds.) Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018, pp. 1157–1160 (2018). https://doi.org/10.22318/cscl2018. 1157 59. Gamrat, C., Toomey Zimmerman, H., Dudek, J., Peck, K.: Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program. Br. J. Edu. Technol. 45(6), 1136–1148 (2014). https://doi.org/10.1111/bjet.12200 60. Newby, T.J., Cheng, Z.: Instructional digital badges: effective learning tools. Education Tech. Research Dev. 68, 1053–1067 (2020). https://doi.org/10.1007/s11423-019-09719-7 61. Risquez, A., Cassidy, D., Ó Súilleabháin, G.: Badge of honour? An exploration of the use of digital badges to support a partnership approach to faculty development. Austral. J. Educat. Technol. 36(5), 18–29 (2020). https://doi.org/10.14742/ajet.6112 62. Watson, S.L., Yu, J.H., Alamri, H., Watson, W.R.: Preservice teachers’ technology integration attitude change in a course implementing digital badges. J. Technol. Teacher Educat. 28(1), 89–116 (2020). https://www.learntechlib.org/primary/p/211054/, last accessed 2022/04/16.

13 Conceptualising Micro-credentials in the Higher Education …

203

63. LaMagna, M.: Placing digital badges and micro-credentials in context. J. Electron. Resour. Librariansh. 29(4), 206–210 (2017). https://doi.org/10.1080/1941126X.2017.1378538 64. Lockley, A., Derryberry, A., West, D.: Drivers, affordances and challenges of digital badges. In: Ifenthaler, D., Bellin-Mularski, N., Mah, D.-K. (eds.) Foundation of digital badges and micro-credentials. Demonstrating and Recognizing Knowledge and Competencies, pp. 55–70. Springer (2016). https://doi.org/10.1007/978-3-319-15425-1_4 65. Stephen, R., Yi, J.: Businesses are rapidly adopting MOOC’s – Universities aren’t – what can be done?. Proceedings of 13th International Technology, Education and Development Conference (INTED2019), 914–922 (2019). https://doi.org/10.21125/inted.2019.0311 66. Kilsby, A., Goode, C.: Taking the College to the Company. Scope: (Teaching and Learning), Vol. 7, pp. 16–18 (2019). https://doi.org/10.34074/scop.4007006

Chapter 14

The Reaction of the University Ecosystem to the Pandemics in a Mid-East Country: The Case of IRAQ—A Compared Analysis of Students’ and Teachers’ Perceptions Alaa Alkhafaji, Haider Mshali, Marcello Passarelli, and Carlo Giovannella Abstract This paper presents an exploratory study on the reaction of the Iraqi university ecosystem to the Covid-19 pandemic, a learning ecosystem with no consolidated tradition in distance learning operating in a country where connectivity is granted mainly by a mobile phone infrastructure. The study analyses data collected from questionnaires filled in by 572 teachers and 2746 students belonging to more than 35 different universities and colleges located all over Iraq. The ecosystem was able to switch to distance learning in two weeks and to generate a reasonable level of satisfaction in the teachers and, even more, in the students, despite the problems that have been encountered. The influence of contextual and individual factors on the opinions and future intentions of both teachers and students has been investigated together with the causal network that puts in relation to such factors. The distance learning experience conducted during the first five months of lockdown induced a relevant change in the opinion about the nature of an educational experience, as well a desire to continue to experience distance and blended learning processes, in slightly more than 50% of the respondents. Future challenges are also highlighted. A. Alkhafaji (B) School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, UK e-mail: [email protected] H. Mshali The Ministry of Higher Education and Scientific Research, Studies, Planning and Follow-up Directorate, Baghdad, Iraq M. Passarelli ITD-CNR Genoa, Genoa, Italy M. Passarelli · C. Giovannella ASLERD, Vergata, Italy C. Giovannella University of Rome Tor, Vergata, Italy A. Alkhafaji Mustansiriyah University, Baghdad, Iraq © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2023 M. Dascalu et al. (eds.), Polyphonic Construction of Smart Learning Ecosystems, Smart Innovation, Systems and Technologies 908, https://doi.org/10.1007/978-981-19-5240-1_14

205

206

A. Alkhafaji et al.

14.1 Introduction As well known, the outbreak of Covid-19 represented a watershed for the world of education which as a whole was forced to experiment with modalities of distance learning. In fact, between the second half of February and the first half of March 2020, almost all educational institutions had to switch abruptly to online educational processes in a very short period of time [1] to contribute to social distancing. The shock was even stronger for countries like Iraq for which online learning represented an almost completely new experience, as evidenced by both the gray literature that can be found on the web and some initial investigations based mainly on interviews. As reported by some of these investigations [2, 3], Iraq started to experiment with e-learning quite late [4], even with respect to the other Arabic countries. In fact, due to the war, Iraq has experienced a delay in the development of many infrastructures including the electricity grid which is still somewhat unstable [4–7]. In 2010, UNESCO established Avicenna, an online learning environment [8] that in Iraq apparently remained at an early stage of development for a long time due to inadequate ICT infrastructure, limited technical support, and lack of ICT literacy [4, 6]. Among the first pilot studies, the realization of a virtual campus was implemented in the Basrah and Salahaddin universities in 2015, thanks to the support of the United Nations Educational Scientific and Cultural Organization and Iraq’s Ministry of Higher Education (MHE) [6, 7]. It was based on the use of virtual lectures and Google Classroom and involved post-graduate students. Only in mid2019, the outcomes of such pilot studies led the MHE to extend the experimentation to other universities. This situation appeared not to have changed in March 2020 [9] as confirmed by the gray literature and by a few comparative studies that considered also Iraq [10, 11]. Unfortunately, all such comparative studies involved a quite limited number of users and most of them were based on a few interviews. Because of this, we have conducted an empirical study on a large scale to collect teachers’ and students’ perceptions about the distance-learning experience with the aim to investigate how the Iraqi HE learning ecosystem reacted to the shock, to identify challenges faced throughout the pandemic time while performing educational activities via internet and to verify if such unexpected experience may have changed the students’ and teachers’ beliefs and expectations about technologies and online learning. A consistent number of Iraqi students (2746) and teachers (572) participated in the survey and filled in the questionnaire we proposed online. While a cross-country analysis of the teachers’ perceptions has been already published in another venue [12], here we present a study, with unique characteristics, that: (a) compares the opinions expressed by a consistent sample of two different categories of actors—students and teachers— belonging to the same learning ecosystem; (b) is focused on a virgin territory for what concerns the online learning; (c) thanks to the quantitative data collected, allows us to explore the relationships and causal dependence among many relevant factors, and thus their influence on future expectations and opportunities.

14 The Reaction of the University Ecosystem …

207

The rest of this paper is structured as follows. Section 14.2 provides an overview of the study design; Sects. 14.3 and 14.4 present the results and the analysis; Sect. 14.5 highlights challenges and implications and sets out a direction for future work.

14.2 The Study Design The extended study described in the following was conducted in Iraq, which could be regarded as a country characterized by a well-developed university system that, however, lacks experience with distance learning and in which the connection to the internet is assured mainly by the mobile phone network. A context, therefore, which differs substantially from those that one can find in Western countries [13] in which wide-band internet connectivity is assured for a large part of the population by optical fiber and in which telematic universities delivering online courses existed for many years. In particular, the purpose of this study is to compare the opinions of a large sample of both student and teacher populations of the Iraqi universities to investigate: (a) how an educational ecosystem operating in a cultural and infrastructural context like the Iraqi one reacted to such a rapid transition from the traditional education (f2f) to distance learning; (b) which among the factors considered by this investigation might have influenced teachers’ and students’ perceptions about technology-enhanced learning (TEL) and, as well, their expectations for the future. A mixed-methods approach was adopted with data being gathered using a questionnaire. In this paper, only quantitative data are discussed. The questionnaire was developed originally by ASLERD (the Association of Smart Learning Ecosystem and Regional Development) [13]. In Iraq, the questionnaire was made available approximately four months after the 2020 lockdown and just before the final exam of the first term. Students and teachers from different universities all around Iraq responded to the questionnaire.

14.2.1 Method The questionnaires adopted in this study consist of three sections, with a total of 79 questions in the case of the student questionnaire and 84 questions in the case of the teacher questionnaire (of course any comparison has been performed only on similar questions). Both questionnaires comprise multiple-choice questions, response scales (Likert-type 10-point response scales, scales ranging from −5 to 5 or 0–100% scales), and open-ended questions. The three sections of the questionnaire allowed us to investigate the role of four groups of numerical variables related to (a) technological context, (b) readiness for the transition to distance learning, (c) educational activities and operating conditions, and (d) future expectations and intentions. An Arabictranslated version was used. Two experts who speak both languages (English and Arabic) checked the quality of the translation and conducted a pilot test to evaluate

208

A. Alkhafaji et al.

the suitability of the questions for the Iraqi context. Ten participants took part voluntarily in the pilot test which helped to enhance the questions by pointing out the weaknesses of the translation. Different channels were used to recruit participants such as social media, emails and and electronic classrooms. Convenience sampling was adopted [19, 20]. The questionnaire, due to the lockdown and social distance measure, was made available as a Google form. Participation was completely voluntary and anonymous. Missing data were excluded from data analysis using pairwise deletion. A first simple descriptive analysis was performed by means of both SPSS and R to work out frequencies for the nominal data of multiple-choice questions, as well as means and t-test for the numeric data. R has also been used to perform a datadriven investigation on the direction of causality between the factors considered in this study.

14.2.2 Participants 2746 students responded to the student questionnaire; they were 52% female and 46% male (about 2% preferred not to answer). 96.9% of the respondents were affiliated with state universities; 82.2% were attending bachelor’s degree programs. As far as the participants’ age: 40% were younger than 22 years old, 45% were between 22 and 26 years old, 5% were between 27 and 30 years old, 5% were between 31 and 40 years old and 5% were older than 41 years. As far as the teachers, the survey was filled in by 572 university teachers (237 female, 330 male, 5 non-binary) employed mainly in public universities (99.3%). The mean age of respondents was 40 years. Participants reported teaching in 16 different scientific and humanistic cultural areas including humanities (19.2%), Mathematical, Physical, Chemical, and Natural Sciences (12.9%), and Educational Sciences (9.8%). Respondents were employed in more than 35 universities and colleges located all over Iraq.

14.3 Results First of all, we carried out descriptive and univariate analyses. For Likert-type response scales, we carried out a one-sample Wilcoxon test against the midpoint of the response scale for 10-point and −5 to 5 scales, and against 0 for the scales ranging from 0 to 100%. Means and the outcomes of the tests regarding the students’ answers are reported in Tables 14.3 and 14.4 (see Appendix), while those concerning the teachers’ answers have been already reported in Giovannella et al. [12]. The names of the variables were adopted from previous studies [12–14], with some amendments when needed.

14 The Reaction of the University Ecosystem …

209

Table 14.1 Comparison among students’ and teachers’ perceptions about the capability of the learning ecosystems to react, the operational conditions, and the features of the educational activities carried out Variable

Mean S

Mean T

Wilcoxon test

University Readiness to swap to online education (UR)

M = 7.23 [7.13, 7.33]

M = 6.95 [6.74, 7.15]

W = 801,594 p = 0.002

Technological Adequacy of Online Environments (TAOE)

M = 6.45 M = 5.91 W = 872,114 [6.35, 6.55], [5.70, 6.12], p < 0.001

Previous Experience in Online Learning (PEOL)

M = 5.69 [5.59, 5.80]

M = 4.55 [4.32, 4.78]

W = 872,114 p < 0.001

Teachers’ Technological Readiness (TTR)

M = 6.22 [6.11, 6.32]

M = 5.84 [5.65, 6.03]

W = 805,288 p < 0.001

Teachers’ Pedagogical Readiness (TPR)

M = 6.34 [6.24, 6.44]

M = 5.79 [5.59, 5.98]

W = 822,618 p < 0.001

Workload Increase (WI) %, tested against the baseline of 0

M = 0.49 [0.48, 0.50]

M = 0.57 [0.55, 0.59]

W = 615,338 p < 0.001

Students’/Teachers’ Time Management Capacity (TTMC) (scale −5, + 5)

M = 1.13 [1.00, 1.25]

M = 1.48 [1.25, 1.71]

W = 700,722 p = 0.16

Educational Activity: Lecture-Discussion (EALD) M = 0.37 (scale −5, + 5) [0.24, 0.49]

M = 0.94 [0.69, 1.18]

W = 603,788 p < 0.001

M = 0.51 [0.37, 0.63]

M = 1.40 [1.16, 1.65]

W = 555,430 p < 0.001

Educational Activity: Asynchronous-Synchronous M = 0.38 (EAAS) (scale −5, + 5) [0.26, 0.52]

M = 1.06 [0.81, 1.30]

W = 566,452 p < 0.001

Educational Activity: Individual-Collaborative (EAIC) (scale −5, + 5)

M = 0.62 [0.48, 0.75]

M = 1.33 [1.08, 1.57]

W = 582,435 p < 0.001

Reproducibility of Classroom Dynamics (RCD)

M = 5.88 [5.79, 5.98]

M = 6.08 [5.91, 6.24]

W = 667,616 p = 0.14

Easiness to Use On-Line Learning (EUOL)

M = 6.61 [6.50, 6.72]

M = 6.47 [6,27, 6.67]

W = 599,802 p = 0.05

Usefulness to On-Line Learning (UOL)

M = 6.17 [6.05, 6.28]

M = 6.35 [6,13, 6.57]

W = 552,854 p = 0.37

Degree of University e-Maturity (UeM)

M = 6.01 [5.90, 6.11]

M = 5.71 [5.52, 5.91]

W = 597,216 p = 0.007

Educational Activity: Transmissive-Interactive (EATI) (scale −5, + 5)

14.3.1 Participants As anticipated, data were collected in a context—the Iraqi one—characterized by connectivity based on the mobile phone network integrated with the use of wi-fi. Indeed 65% of the students and more than 50% of the teachers used wi-fi to connect to the internet; about 25% of both students and teachers used 3G, only 7% of the students and slightly more than 20% of teachers used optical fiber connectivity. As a logical consequence, 94% of the students and 77% of teachers reported mainly using mobile phones to perform activities over the internet. Most likely such widespread

210

A. Alkhafaji et al.

Table 14.2 Comparison among students’ and teachers’ perceptions about technologies and their expectations for the future Variable

Average S

Average T

Wilcoxon test

Sustainability of Online Education (SOE)

M = 5.63 [5.51, 5.75]

M = 5.87 [5.66, 6.08]

W = 603,350 p = 0.11

Change in the Idea of Educational Experience (CIEE)

M = 6.02 [5.91 6.14]

M = 6.28 [6.06, 6.49]

W = 557,982 p = 0.10

Improvement in the Attitude toward Technologies (IAT)

M = 6.50 M = 7.05 W = 522,952 [6.38, 6.61] [6.84, 7.26] p < 0.001

Improvements in Technological Skills (ITS)

M = 6.67 [6.56, 6.78]

M = 7.04 W = 541,330 [6.85, 7.23] p = 0.02

Intention to Work in Smart Working (IWSW/IWOL)

M = 5.92 [5.81, 6.04]

M = 6.33 W = 521,359 [6.12, 6.54] p = 0.004

Extent to which University should Rely on M = 6.11 Online Learning (UROL) [5.99, 6.23]

M = 6.51 W = 522,358 [6.30, 6.73] p = 0.007

use of smartphones can be explained by the need to use a personal device to perform learning activities and obviously students used the device they already own, instead of buying a new device (e.g., laptop), which they might not be able to afford [2, 10–12]. The transition to online learning represented a new experience for most of the actors involved, as evidenced by the rather low value of the average value of PEOL (Previous Experience in Online Learning): 5.69 for students and 4,55 for teachers. The difference between the average values could be justified by a greater familiarity of the students in searching over internet resources useful “to learn”. The novelty of the experience has certainly influenced the time needed to switch from f2f to online as well as the time to get used to the new conditions. Most of the students and the teachers needed at least a couple of weeks to get used to the online learning and become fully operative see Fig. 14.1. Two weeks is a time interval that has been detected in contexts—like the schools in western countries [14]—with a limited or no previous experience in e-learning, either due to limited technical promptness of the Institutions (UR) or to a limited technological and pedagogical preparedness (TTR and TPR) of the actors involved in the learning process. Actually, the opinions of the students are more positive with respect to those of the teachers (see Tables 14.1 and 14.2). In fact, the mean value assigned by the students to the Universities Readiness (UR) to switch to distance learning is 7.23 (6.95 for the teachers), which indicates that universities reacted as quickly as they could, whereas perceived Technological Adequacy of Online Environments (TAOE) was 6.45 (5.91 for the teachers). This indicates that universities were ready to change, but the technological infrastructure to deliver online learning (including applications and platforms) was considered not fully satisfactory, a criticality that may be attributed partially to the novelty of the experience for the Iraqi HE. This conclusion is confirmed also by the mean value attributed by the students to the Teachers’ Technological Readiness (TTR) and Teachers’ Pedagogical Readiness (TPR): 6.22 and 6.34, respectively (to be compared with 5.84 and 5.79 assigned by

14 The Reaction of the University Ecosystem …

211

Fig. 14.1 Time needed for students and teachers to get used to distance learning

the teachers). Teachers, thus, appeared not to be very familiar with distance learning and in need of training to develop the required skills to keep up the pace of the rapid transition from f2f to fully virtual educational processes and to perform learning activities over the internet. Nevertheless, in the end, despite the challenges they faced, teachers managed to provide learning services that could be considered good enough [12], so that the mean value assigned to the University e-Maturity (UeM) level was 6.01 (5.71 for the teachers). This value could be considered a satisfactory one with respect to a context affected by a lack of adequate infrastructures, software, and experience [2, 12–14]. To fully define the operational context, we also need to consider additional aspects such as the places used by students to participate in online activities: 96% of the respondents connect to the internet from home, but only 51% have the possibility to use a private room, the other 45% must connect from a shared one. The remaining ones connect from workspaces. The home setting generates problems for more than a quarter of the students: 29% meet difficulty concentrating and almost 25% consider the home environment not suitable to participate in an online learning process (see Fig. 14.2). In any case, the major complaint is for the limited bandwidth offered by the mobile infrastructure, underlined by almost 64% of the students (and more than 78% of the teachers). These outcomes clearly show that the infrastructure represents a systemic criticality that does not affect only the capacity of the institutions to react to the pandemic. Of course, as one may expect, and as we will see in the next subsection, the limitations of the infrastructures will unavoidably influence the nature and the quality of the learning activities.

212

A. Alkhafaji et al.

Fig. 14.2 Difficulties faced during distance learning

14.3.2 Educational Activities As one may expect teachers are more focused than students on the traditional teachers’ activities: delivering transmissive (50% vs. 30%) and interactive (40% vs. 23%) lessons, content production (22% vs. 18%) and sharing (41% vs. 28%), assign (36% vs. 27%), and carrying out (23% vs. 23%) exercises, assessment (23% vs. 18%), and managing the educational processes (20% vs. 10%). Interestingly, the students perceive the same intensity of the teachers in only a few activities: collaborative and team-working activities (32% vs. 31%), synchronous and asynchronous communication with teachers (28% vs. 30%), and socialization (13% vs.12%). This seems to indicate that during the social distancing imposed by

14 The Reaction of the University Ecosystem …

213

the pandemics the online activities have been used by students to maintain social contacts and interaction, also in terms of collaborative working. By comparing the percentage indicated by the teachers and those indicated by the students, other activities that have involved the students significantly are those regarding the execution of exercises, content production, and self-assessment. Regarding the latter, the preferred assessment approach has been the execution of online tests, while the less popular one has been the online interview. Probably this is due to the fact that online interviews, like the presentation of homework over the internet, are much more time-consuming and would have required a more powerful/stable/inexpensive internet connectivity. In terms of absolute percentage, of course, also delivering of lectures and content sharing represent a relevant part of the online activities. Both teachers and students do not feel that online learning has been used to personalize the learning process. Additionally, the diversification of the educational and methodological approaches is perceived as much less efficient by students than by teachers. Overall, it seems that the activities conducted online during distance learning tend to reproduce the standard f2f activities (e.g., transmissive lectures) although it comes out that, despite the infrastructural limitations, a large part of the activities involved synchronous interactions and collaboration, apparently in a larger percentage with respect to what has been detected in some western countries [12]. This conclusion is reinforced by the outcomes of Table 14.1 that clearly show how, despite the technological context and the limited experience, the teachers tried to implement with students collaborative, interactive, synchronous, and discussion-based (collaborative) activities: Lecture versus Discussion (EALD) (students M = 0.37, teachers M = 0.94), Transmissive versus Interactive (EATI) (students M = 0.51, teachers M = 1.40), Asynchronous versus Synchronous (EAAS) (students M = 0.38, teachers M = 1.06), Individual versus Collaborative (EAIC) (students M = 0.62, teachers M = 1.33). In our survey, we also investigated the intention and the capability of the online process to reproduce Classroom Dynamics (RCD). The detected means for this parameter are not so high (M = 5.88 for the students and M = 6.08 for the teachers) to indicate that online learning, in particular, in a technological context like the Iraqi one, is not capable to reproduce the classroom dynamic, and thus that process should be redesigned not only to counterbalance the infrastructural limitations, but also to take advantage of the new potentialities offered by the learning technologies. For sure, online learning is an activity that will require extra time and effort from both students and teachers. In fact, it is now well established [12–14] that online learning implies, at least in this phase, an increase in the working load. Accordingly, the perceived increase in working load, WI, was 49% for the students and 57% for the teachers. Despite the increase in WI both the categories recognized that online learning helped them better manage their own time (TTMC): M = + 1.13 for the students and M = + 1.48 for the teachers.

214

A. Alkhafaji et al.

14.3.3 Perceived Transformation and Future Expectations/Intentions Other aspects that have been investigated by the survey are the changes in the perception of technologies and technology augmented learning induced by the switch to online learning induced by the pandemics. From Table 14.1, it is quite clear that after only a few months during which students and teachers have experienced online learning their perceptions have changed quite a lot, more for teachers than for students, despite the latter having a more positive perception of the overall readiness of the context. The mean value associated with the Change in the Idea of Educational Experiences (CIEE) came out to be M = 6.02 for the students and M = 6.28 for the teachers, values not particularly high but well above the midpoint of the scale. These values should be associated with those observed for the variable Extent to which University should Rely on Online Learning (UROL), M = 6.11 for the students and M = 6.51 for the teachers. This change of perception with respect to the online learning activities, however, does not seem to support the perception of Sustainability of online education (SOE): students M = 5.63, teachers M = 5.87. Such average values of SOE should be probably ascribed, again, to the quality of the available infrastructure and software that are expected to be of higher quality to support sustainability. For most likely the same reason we have detected, in particular among the students a relatively low mean value of the Intention to Work in Smart Working (IWSW): students M = 5.92, teachers M = 6.33. Evidently, it is still considered a domain with uncertain perspectives in Iraq. Overall, however, the terrain appears to have been fertilized by the novel experience and the mean values of the variables Improvement in the Attitude toward Technologies (IAT; students M = 6.50, teachers M = 7.05) and Improvements in Technological Skills (ITS; students M = 6.67, teachers M = 7.04) let hope for the best. In particular, both teachers and students feel to have developed several digital skills: the ability to produce online digital content (also at a personal level) followed by digital media processing (videos, pictures, and sounds), and the capability to download and organize content and to use team-working environments. Overall, considering the technological maturity level of the context, it seems that online learning activities have triggered and initiated a potential innovation process. In fact, if we consider the preference for the future learning model, only 48% of students and 49% of teachers wish to come back to a full face-to-face process; the rest would prefer to switch toward full distance (26% of students and 18% of teachers) or blended learning (26% students and 33% of teachers). The possibilities for this innovation process to fully develop are partly related to the enhancement of technological infrastructures and partly to political will to support a cultural change in education. Coherently with what has been observed above, such a positive impression is triggered by the belief that, despite the infrastructural limitations, the technologies used during the pandemic could be very useful and suitable to deliver lectures, assign and carry on tasks/exercises, produce and share content, support communications and

14 The Reaction of the University Ecosystem …

215

collaborative team working. Students and, overall teachers, also believe that educational technologies may improve in many respects the learning processes. In particular, quality (64% for teachers and 34% for students), effectiveness (54% for teachers and 35% for students) and efficiency (48% for teachers and 31% for students), development of the own digital identity (57% for teachers and 26% for students), the interaction (52% for teachers and 33% for students), and the self-regulation (48% for teachers and 34% for students).

14.4 Casual Discovery and Network Analysis To better describe the landscape emerging from the previous section, we worked out the network of relations among the investigated factors and tried to infer the direction of causality for such associations. More specifically, we used first the PC algorithm [15] to infer the direction of causality in the graph (Sect. 4.1). Network analysis was also used to obtain a bird’seye view of both intensity and sign of the variables’ relations in undirected graphs (Sect. 4.2).

14.4.1 Perceived Transformation and Future Expectations/Intentions The search for the causal relations returned the pictures shown in Figs. 14.3 and 14.4. As in previous investigations [12–14], their interpretation should be considered tentatively due to the existence of possible relevant variables that have not been taken into account (hidden variables) and that may cause, for example, the impossibility to identify unambiguously the direction of all the relations. The two networks of Figs. 14.3 and 14.4 show not only similar groupings of factors (colored areas) and notable similarities, but also some differences. In the case of the students, the previous experience with online learning (PEOL) seems to assume a higher relevance than in the case of teachers and is capable to influence somehow the opinion on the technological responsiveness and preparedness of the learning ecosystem (light blue area). It also influences the perception of reproducibility of class dynamics (RCD) and the change in perception of the didactic process (CIEE). Very clear in both graphs, and expected, are the relationships between the factors that influence the readiness of the learning ecosystems (UR) and their e-maturity (UeM), see the light blue area. The centrality of the RDC node is much more evident in the teachers’ causal network, most likely because they are more interested than the students in the reproducibility of the class dynamics. This statement is supported also by the differences evidenced in the previous section concerning the perception of the activities accomplished at most during online learning. In the

216

A. Alkhafaji et al.

Fig. 14.3 Students: the causal structure that relates the main factors considered in this study. Red ovals identify local and main terminals of the causal network

Fig. 14.4 Teachers: the causal structure that relates the main factors considered in this study. Red ovals identify local and main terminals of the causal network

case of students, RDC still keeps a certain relevance but UeM, and TTR, influence directly the terminal factors included in the green and gray areas, see the Change in the Idea of Educational Experience (CIEE) and the Intention to Work in Smart Working (IWSW). In both networks, the increase of the working load (WI) influences the capacity to organize the own time (TTMC), and the latter is related to the characteristics of the didactic activities (light orange area). Moreover, in both networks, the

14 The Reaction of the University Ecosystem …

217

opinion on the Usefulness of On-Line Learning (UOL) and on the Easiness to Use On-Line Learning (EUOL) influence the extent to which University should Rely on Online Learning (UROL) but these two variables, UOL and EUOL—that are at the core of the TAM (Technology acceptance model) model [16]—are not essential. They represent, in fact, only two of the predictors of UROL, and the overall causal chain still remains in place also after bypassing them. In both the network, furthermore, UROL, CIEE and (IWSW/IWOL) act as the main terminals of the causal chain but the hierarchy is somewhat clearer in the case of the teachers’ network where the main terminal role played by CIEE is more clear, as well as the role of the Sustainability of Online Education (SOE) as a connecting knot.

14.4.2 Network Analysis Network analysis offers useful tools for visualizing complex webs of variable relationships; these include the plotting of least absolute shrinkage and selection operator (LASSO) regularized partial correlation networks [17]. Partial correlations measure the degree of association between two variables after controlling for all other variables being considered; as such, they are a useful measure of direct association. Using LASSO regularization, further aids in the interpretability of the network by only visualizing relatively strong associations and setting to 0 all weaker associations. Figures 14.5 and 14.6 show the LASSO-regularized networks for, respectively, students and teachers. In the figures, both the intensity and the sign of the associations among variables are shown. They provide us with the possibility of weighing the relevance of the relationships that emerged from the causal network. Apart from the high similarities between the two networks (Deltacon = 0.838 [18]) that confirm the picture emerging from the analysis of the causal networks,

Fig. 14.5 Student’s LASSO-regularized partialized network of the main factors considered in this study

218

A. Alkhafaji et al.

Fig. 14.6 Teachers’ LASSO-regularized partialized network of the main factors considered in this study

it is interesting to note that in both networks the increase of working load (WI) presents a negative correlation (red link) with the capacity to organize the own time (TTMC) and as well with the perception of Easiness to Use On-Line Learning (EUOL) technologies and environments. In the case of the students, weak negative correlations are also observed between WI and UROL or IWSW. In the case of teachers, on the other hand, WI presents a negative correlation, with medium intensity, also with TTR and CIEE to indicate that a lower technological preparedness tends to generate a higher working load and prevent a more positive attitude toward technology-enhanced education. Both the networks, moreover, are characterized by weak negative correlations between the future intention to switch to, or to use, blended learning (FBL) and the Teachers’ Pedagogical Readiness (TPR), the e-maturity of the learning environment (UeM), the previous experience with online learning (PEOL). These three variables emerge, thus as critical ones that should be counteracted in the future.

14.4.3 Model for Attitude to Get Engaged in Technological Innovation (MAETI) The networks shown in Figs. 14.4, 14.5, 14.6 and 14.7 confirm, also in the case of the students, the cluster organizations of the variables that were observed previously and lead to the formulation of a universal Model for Attitude to get Engaged in Technological Innovation (MAETI) [12]. The sketch of this model adapted to integrate the Iraqi students’ and the teachers’ views have been reported in Fig. 14.7. The main differences with respect to [12] are (a) the separated representation of the variables used in TAM; (b) the representation of the differences observed in

14 The Reaction of the University Ecosystem …

219

Fig. 14.7 A sketch of the universal Model for Attitude to get Engaged in Technological Innovation (MAETI). Solid lines: relationships observed either in the students’ and teachers’ causal networks; dashed solid lines: relationships observed only in the students’ causal network; dotted solid lines: relationships observed only in the teachers’ causal network

the paths that connect the different clusters of variables in the case of students’ and teachers’ causal networks. The overall picture does not change since the cluster of variables related to the Technological setting is always positioned at the start of the causal chain and the cluster of the variables related to the change in the perception of the educational process and the relationships between human and technologies, together with the cluster of variables related to the Future intentions, are always positioned at the end of the chain. Nevertheless, the paths that lead from the top to the bottom of the chain could be somewhat different for each category of the actors participating in the learning processes, as evidenced also by Sects. 3 and 4 that highlight the differences in the relevance attributed by students and teachers to the various aspects and factors investigated by our survey.

220

A. Alkhafaji et al.

14.5 Implications and Challenges—Discussion The survey used in this study allowed us to gather a considerable quantity of data related to the switch from f2f to online learning induced by the pandemic in a country, like Iraq, which can be considered almost virgin for what concerns e-learning. The considerable number of factors that have been investigated helped us to obtain a reasonable understanding of students’ and teachers’ perceptions, attitudes, and expectations with respect to online learning at the university level. The outcomes discussed in the previous sections show a relative prompt and efficient reaction to the context together with an overall positive perception by students and teachers. However, our investigation highlighted also several challenges that have been faced by either students and teachers during the first months of such experience and several aspects that should be considered, in the future, to design e-learning processes in Iraq and in countries with similar characteristics. Such challenges can be grouped under three general themes, which are: Infrastructure. It is very clear that in a quite large number of countries, like Iraq, connectivity to the internet is guaranteed by the mobile infrastructure. It is certainly enough to react to emergencies and assure continuity to the learning activities but the limitations in bandwidth were for sure the main challenge students and teachers faced in participating in online learning activities. The poor connectivity is, at present, the main problem in Iraq, either due to the network instability or, sometimes, to the unstable electrical power supply. The infrastructural conditions allowed the implementation of standard transmissive teaching processes but often prevented holding synchronous meetings or lectures. Instead, students had to receive videos of lectures to overcome the criticalities of the internet connection, which would cause students to lose direct communication with teachers and peers. To the infrastructural criticality, one has to also add the limitations to which are subjected the personal devices, mainly mobile phones. Again, the use of mobile phones allowed to assure educational continuity, but prevented the use of the full power of a large number of applications available on the web, including those that would support collaborative/cooperative learning. Physical learning context. Although the learning processes took place online, the actors had to stay in a physical place. Learning usually needs an environment that helps students to focus on the teaching activities such as a quiet place, which is not necessarily available while at home. Most students attended learning activities from home, often from a shared room, which is not considered the perfect environment for learning. Such a situation may have added extra workload on students to keep on track, which consequently needed more time for students to adapt. Given that, students faced an extra challenge, which is self-time management. Additionally, the infrastructural limitations prevent full interaction and communication between students and teachers. Interaction and even more collaborative/cooperative practices are very important to enhance engagement and produce familiarity with the topics and

14 The Reaction of the University Ecosystem …

221

develop competencies. With this respect, an important challenge has been the lack of funds that might have prevented the students to buy new devices such as laptops or tablets to perform learning activities and/or to ensure themselves a good/stable internet connection This is clearly an issue for the future. Pedagogical preparation. Another important challenge was the pedagogical skills of teachers to implement online learning processes. In fact, the rapid transition from f2f into fully virtual learning has left not enough time for teachers to enhance their skills in digital pedagogy. Additionally, in the Iraqi context, as distance learning is considered a new experience, this challenge has been magnified. The novelty and the lack of experience with online educational activities and the large choice of applications of interest generated also a sense of bewilderment in many teachers who have struggled to find the right ones to use to attract the students’ attention. In principle, this could lead to loss of engagement, a reduction of collaborative practices, and finally, causes a limited efficacy of the whole learning process. However, despite the lack of expertise, teachers managed to provide a learning service that could be considered sufficient in an emergency situation like the one provoked by the pandemic. Maybe for this reason, in the opinion of the Iraqi teachers, the relevance that should be given to Teacher Education in Digital Pedagogy (ITEDP) is relatively high, 6.62, but not so high as in other western countries [12]. No doubt that this is and remains an important issue for the future in order to support the adoption of technology-enhanced learning processes. Based on the discussion so far, it could be said that universities—teachers and students—managed to adapt to online learning after a short period of time, despite the several challenges they all faced. This allowed keeping the progress of the educational processes on track. With this respect, the experience with online learning could be considered successful, as confirmed, in particular, by the students’ perception. However, a big question arises here, how successful was this experience in terms of the quality of learning? Further work is certainly needed to measure the effectiveness of online learning, in particular, to analyze qualitative data (open-ended questions) to extract additional information on teachers’ and students’ opinions and perspectives, which might lead to extending the current list of challenges.

Appendix See Tables 14.3, 14.4, 14.5 and 14.6.

222

A. Alkhafaji et al.

Table 14.3 Students’ perception about the capability of the learning ecosystems to react, the operational conditions and the features of the educational activities carried out Variable

Mean

Wilcoxon test

University Readiness to swap to online education (UR)

M = 7.23 [7.13, 7.33]

(2579) V = 2,727,154, p < 0.001 Cohen’s d = 0.75

Technological Adequacy of Online Environments (TAOE)

M = 6.45 [6.35, 6.55]

(2548) V = 2,293,016, p < 0.001 Cohen’s d = 0.38

Previous Experience in Online Learning M = 5.69 (PEOL) [5.59, 5.80]

(2527) V = 1,743,746, p < 0.001 Cohen’s d = 0.07

M = 6.22 [6.11, 6.32]

(2574) V = 2,166,804, p < 0.001 Cohen’s d = 0.27

Teachers’ Pedagogical Readiness (TPR) M = 6.34 [6.24, 6.44]

(2553) V = 2,206,776, p < 0.001 Cohen’s d = 0.32

Workload Increase (WI) %, tested against the baseline of 0

M = 0.49 [0.48, 0.50]

(2554) V = 2,579,856, p < 0.001 Cohen’s d = 1.57

Students’ Time Management Capacity (TTMC) (scale −5, + 5)

M = 1.13 [1.00, 1.25]

(2552) V = 1,832,534, p < 0 .001 Cohen’s d = 0.35

Educational Activity: Lecture-Discussion (EALD) (scale −5, + 5)

M = 0.37 [0.24, 0.49]

(2447) V = 1,204,210, p < 0.001 Cohen’s d = 0.11

Educational Activity: Transmissive-Interactive (EATI) (scale −5, + 5)

M = 0.51 [0.37, 0.63]

(2425) V = 1,299,960, p < 0.001 Cohen’s d = 0.35

Educational Activity: Asynchronous-Synchronous (EAAS) (scale −5, + 5)

M = 0.38 [0.26, 0.52]

(2385) V = 1,161,732, p