Methodologies and Intelligent Systems for Technology Enhanced Learning, Workshops - 13th International Conference (Lecture Notes in Networks and Systems) 3031421337, 9783031421334

Citation preview

Lecture Notes in Networks and Systems 769

Zuzana Kubincová · Federica Caruso · Tae-eun Kim · Malinka Ivanova · Loreto Lancia · Maria Angela Pellegrino   Editors

Methodologies and Intelligent Systems for Technology Enhanced Learning, Workshops 13th International Conference

Lecture Notes in Networks and Systems

769

Series Editor Janusz Kacprzyk , Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

Advisory Editors Fernando Gomide, Department of Computer Engineering and Automation—DCA, School of Electrical and Computer Engineering—FEEC, University of Campinas— UNICAMP, São Paulo, Brazil Okyay Kaynak, Department of Electrical and Electronic Engineering, Bogazici University, Istanbul, Türkiye Derong Liu, Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, USA Institute of Automation, Chinese Academy of Sciences, Beijing, China Witold Pedrycz, Department of Electrical and Computer Engineering, University of Alberta, Alberta, Canada Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Marios M. Polycarpou, Department of Electrical and Computer Engineering, KIOS Research Center for Intelligent Systems and Networks, University of Cyprus, Nicosia, Cyprus Imre J. Rudas, Óbuda University, Budapest, Hungary Jun Wang, Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong

The series “Lecture Notes in Networks and Systems” publishes the latest developments in Networks and Systems—quickly, informally and with high quality. Original research reported in proceedings and post-proceedings represents the core of LNNS. Volumes published in LNNS embrace all aspects and subfields of, as well as new challenges in, Networks and Systems. The series contains proceedings and edited volumes in systems and networks, spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, Aerospace Systems, Automation, Manufacturing, Smart Grids, Nonlinear Systems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publication timeframe and the worldwide distribution and exposure which enable both a wide and rapid dissemination of research output. The series covers the theory, applications, and perspectives on the state of the art and future developments relevant to systems and networks, decision making, control, complex processes and related areas, as embedded in the fields of interdisciplinary and applied sciences, engineering, computer science, physics, economics, social, and life sciences, as well as the paradigms and methodologies behind them. Indexed by SCOPUS, INSPEC, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science. For proposals from Asia please contact Aninda Bose ([email protected]).

Zuzana Kubincová · Federica Caruso · Tae-eun Kim · Malinka Ivanova · Loreto Lancia · Maria Angela Pellegrino Editors

Methodologies and Intelligent Systems for Technology Enhanced Learning, Workshops - 13th International Conference

Editors Zuzana Kubincová Faculty of Mathematics, Physics and Informatics Comenius University Bratislava, Slovakia Tae-eun Kim Universitetet i Tromsø (UiT) – Norges arktiske universitet Tromsø, Norway Loreto Lancia Department of Life, Health and Environmental Sciences University of L’Aquila Coppito, Italy

Federica Caruso University of L’Aquila L’Aquila, Italy Malinka Ivanova Technical University of Sofia Sofia, Bulgaria Maria Angela Pellegrino Department of Computer Science University of Salerno Fisciano, Salerno, Italy

ISSN 2367-3370 ISSN 2367-3389 (electronic) Lecture Notes in Networks and Systems ISBN 978-3-031-42133-4 ISBN 978-3-031-42134-1 (eBook) https://doi.org/10.1007/978-3-031-42134-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023, corrected publication 2023, 2024 Chapter “Exploring Value and Ethical Dimensions of Disruptive Technologies for Learning and Teaching” is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecomm ons.org/licenses/by/4.0/). For further details see license information in the chapter. This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Preface

Education, science, research, and technology deployment in all areas of life are considered to be the basic pillars of the knowledge society. The International Conference in Methodologies and Intelligent Systems for Technology Enhanced Learning (mis4TEL) serves as a forum for experts from all these fields, including not only education, information technology, or computer science, but also disciplines such as psychology, medicine, and social sciences. It encourages multidisciplinary research and discussion on technology enhanced learning promoting new intelligent and creative solutions for formal as well as informal learning and all types of learners. In addition to technological solutions, the technology enhanced learning approach can be fostered by novel methods coming from different fields of research, and from diverse communities also including “fragile users”, like children, elderly people, or people with special needs. The annual appointment of mis4TEL established itself as a consolidated fertile forum where scholars and professionals from the international community, with a broad range of expertise in the TEL field, share results and compare experiences. The conference program also features four selected workshops which aim to provide participants with the opportunity to present and discuss novel research ideas on emerging topics complementing the main conference. In particular, the workshops focus on Integration of Emerging Technologies into Education and Training (ETELT), Interactive Environments and Emerging Technologies for eLearning (IEETeL), Technology Enhanced Learning in Nursing Education (NURSING), and Technology Enhanced Learning for Future Citizens (TEL4FC). This volume presents the papers that were accepted for these workshops of mis4TEL 2023. All papers underwent a peer-review selection: Each paper was assessed by three different reviewers, from an international panel of each workshop. A total of 38 quality papers, with authors coming from various countries, have been selected for the workshops and included in the present volume. This conference is organized by the LASI and Centro Algoritmi of the University of Minho (Portugal). We would like to thank all the contributing authors, the members of the Program Committee, the reviewers, the sponsors, and the Organizing Committee for their hard and highly valuable work. Thanks for your help—mis4TEL 2023 would not exist without your contribution. Zuzana Kubincová Federica Caruso Tae-eun Kim Malinka Ivanova Loreto Lancia Maria Angela Pellegrino

Organization of mis4TEL 2023

http://www.mis4tel-conference.net/ General Chairs Marcelo Milrad Nuno Otero

Linnaeus University, Sweden University of Greenwich, UK

Technical Program Chairs María Cruz Sánchez-Gómez Juan José Mena Dalila Durães Filippo Sciarrone

University of Salamanca, Spain University of Salamanca, Spain University of Minho, Portugal Universitas Mercatorum, Italy

Paper Chairs Claudio Álvarez Gómez Manuel Rodrigues

University of Los Andes, Chile University of Minho, Portugal

Workshop Chairs Zuzana Kubincová Federica Caruso

Comenius University in Bratislava, Slovakia University of L’Aquila, Italy

Steering Committee Representatives Pierpaolo Vittorini Rosella Gennari Tania Di Mascio Marco Temperini Fernando De la Prieta

University of L’aquila, Italy Free University of Bozen-Bolzano, Italy University of L’aquila, Italy Sapienza University of Rome, Italy University of Salamanca, Spain

viii

Organization of mis4TEL 2023

Local Organizing Committee Paulo Novais (Chair) José Manuel Machado (Co-chair) Hugo Peixoto Regina Sousa Pedro José Oliveira Francisco Marcondes Manuel Rodrigues Filipe Gonçalves Dalila Durães Sérgio Gonçalves

University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal University of Minho, Portugal

Organizing Committee Juan M. Corchado Rodríguez Fernando De la Prieta Sara Rodríguez González Javier Prieto Tejedor Ricardo S. Alonso Rincón Alfonso González Briones Pablo Chamoso Santos Javier Parra Liliana Durón Marta Plaza Hernández Belén Pérez Lancho Ana Belén Gil González Ana De Luis Reboredo Angélica González Arrieta Angel Luis Sánchez Lázaro Emilio S. Corchado Rodríguez Raúl López Beatriz Bellido María Alonso Yeray Mezquita Martín Sergio Márquez Andrea Gil Albano Carrera González

University of Salamanca and AIR Institute, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca and AIR Institute, Spain AIR Institute, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain University of Salamanca, Spain AIR Institute, Spain AIR Institute, Spain University of Salamanca, Spain AIR Institute, Spain

Workshop on Integration of Emerging Technologies into Education and Training (ETELT)

The rapid development of the Fourth Industrial Revolution, driven by emerging technologies such as Artificial Intelligence (AI), digitalization, robotics, and the Internet of things (IoT), is creating opportunities for unprecedented industrial growth, also having a profound and far-reaching impact to the field of education. Technologies such as Virtual Reality (VR), Augmented Reality (AR), and AI virtual assistants have already started to shape educational processes and are set to have an increasingly larger impact soon. Specifically, VR and AR provide 3D references to students and create immersive learning experience that make teaching and learning more engaging and interactive. They enable hands-on learning experiences and allow to visualize educational contexts, systems, or concepts in a more engaging way, which is particularly relevant for fields such as health care, engineering, and vocational education, where the students can simulate operations in a cost-effective, safe, and controlled environment. AI-enabled adaptive learning systems have also shown much potential in facilitating learner-centric educational experiences that are tailored to meet the pace, unique needs, and preferences of the individual students. They augment teachers’ capabilities by providing differentiated instructions and personalized learning paths according to the style and speed of each student, supporting self-regulated learning, and allowing for a more transparent understanding of the learning progress. Finally, the proliferation of mobile phones, handheld devices, and online learning platforms have greatly increased accessibility to education, making it possible for students to access materials and resources at anytime, anywhere, and at their own pace, maintaining the continuity of learning across technologies and settings. The integration of the abovementioned emerging technologies in a wide range of educational settings has diversified the learning environment and provided a more dynamic educational experience for learners to access and engage with educational materials in a flexible, interactive, and personalized way. However, despite such advantages, technology enhanced learning (TEL) has also raised many important issues which are yet to be solved. TEL raises concerns related to technology over-reliance, digital divide, cybersecurity, as well as issues related to professional skill development. It is essential to carefully consider the opportunities and challenges associated with TEL and address them proactively to maximize its benefits and minimize its risks in educational and training settings. This workshop is jointly proposed by five EU Horizon projects that have been recently funded by the Digital Europe program and under the topic “Integration of emerging new technologies into education and training” and focuses on current research trends, views, and results related to technology enhanced education and training by integrating emerging technologies. It aims to present and elaborate the state-of-the-art research on related methodologies and intelligent systems and discuss their potential applications in different learning and industrial contexts.

x

Workshop on Integration of Emerging Technologies into Education and Training

The five EU projects that propose this workshop are: • Augmented Intelligence for Pedagogically Sustained Training and Education (augMENTOR); • Electronic, Didactic, and Innovative Platform for Learning based On Multimedia Assets (e-DIPLOMA); • Design and Evaluation of Technological Support Tools to Empower Stakeholders in Digital Education (EMPOWER); • Extending Design Thinking with Emerging Digital Technologies (Exten (D.T)2); • Integrating Adaptive Learning in Maritime Simulator-Based Education and Training with Intelligent Learning System (i-MASTER). Each of these research projects has a specific scope, impact, and intended audience. By pooling resources, expertise, and knowledge together, this cluster of projects has the potential to produce well-rounded, comprehensive, and high-quality research on the integration of emerging technologies in TEL, which can lead to greater impact in the educational and training field.

Workshop Areas The research areas to be addressed in the workshop are: • Pedagogical deployment of learning technologies: Effective deployment of VR, AR, and AI-enabled learning systems requires technical expertise and a proper understanding of educational pedagogy. With the advent of emerging technologies, the role of teachers and the way teaching content is prepared, how we instruct, motivate, and engage students in the learning process are different from traditional classrooms. The pedagogical factors and instructional strategies need to be reconsidered along with technology development. • Human-centered approach to AI in education: AI-enabled learning systems should be designed to accommodate all students, regardless of their abilities or backgrounds, and it should also be designed and deployed responsibly and ethically in education considering the needs and well-being of students and teachers. There are many questions to be explored such as how AI-enabled learning systems can be designed to augment, rather than replace, the role of teachers in the learning process and how both students and teachers should be involved in the design and implementation process of these systems and how they impact on student’s skills development, employability needs, future careers, etc. • Policy recommendations regarding AI in education: Proper policy should be in place to ensure AI-enabled learning systems are used responsibly and effectively in educational settings. Research based policy recommendation is particularly useful as it provides data evidence, deep understanding of the issues being addressed, and a systematic and objective basis for policy considerations and decisions. Actionable recommendations for policymakers can be discussed during this workshop.

Workshop on Integration of Emerging Technologies into Education and Training

xi

• Emerging technologies and twenty-first-century competencies: While the research landscape provides much evidence on how mature digital tools contribute to the pedagogical design, the same does not apply when it comes to emerging technologies. Traditionally, instructional design using digital technologies favors mass instruction delivery, based on conventional forms of assessment, and aimed at developing knowledge and skills initially at a low level of competencies (conventional MOOCs are one such example). On the contrary, nowadays, the development of personalized learning, collaborative learning, and the achievement of higher-order thinking skills, what we call twenty-first-century competencies, remains a significant challenge. At the same time, it is of great interest to understand how learners work (emphasis on the process and not only on the outcomes), how educators orchestrate their classrooms, and what information they have access to in order to adapt/enhance/improve their orchestration. In addition to the above areas, we welcome works that report on the use of existing TEL platforms and applications and discuss best practices to be incorporated into learning technology design. Topics of interest include, but are not limited to, the followings: • • • • • • • • • • •

Online and blended teaching and training with the use of emerging technologies Pedagogical deployment of emerging technologies Emerging technologies and twenty-first-century skills and competencies Evaluation issues, experiences, and case studies Emerging technologies in web-based learning AI/ML-based platforms and applications for learning and training Digital assistants and intelligent chatbots in education Emerging technologies and new teaching methodologies Responsible research and innovation (RRI) and emerging technologies Application of emerging technologies in vocational education sector Ethical implications of using VR in class.

Organizing Committee Christothea Herodotou Marcelo Milrad Hans Joachim Schramm

Inmaculada Remolar Nikos Karacapilidis Marcos Fernández Tae-eun Kim

Open University, UK Linnaeus University, Sweden WU Vienna University of Economics and Business and External Lecturer at Copenhagen Business School, Austria Universitat Jaume I, Spain University of Patras, Greece University of Valencia, Spain University of Tromsø (UiT), Norway

xii

Workshop on Integration of Emerging Technologies into Education and Training

Program Committee Irene-Angelica Chounta Vassilis Komis Nikos Karacapilidis Margarida Romero Dimitris Tsakalidis Hans Joachim Schramm

University of Duisburg-Essen, Germany University of Patras, Greece University of Patras, Greece Université Côte d’Azur, France Novelcore, Greece WU Vienna University of Economics and Business, Austria Lokukaluge Prasad Channa Perera University of Tromsø (UiT), Norway Inmaculada Remolar Universitat Jaume I, Spain Marcelo Milrad Linnaeus University (LNU) Christothea.Herodotou Open University, UK Gerardo Herrera University of Valencia, Spain

Workshop on Interactive Environments and Emerging Technologies for eLearning (IEETeL)

eLearning is a dynamic research area reflecting on the current requirements of all participants in the educational process for innovative teaching and meaningful learning combining existing knowledge with future perspectives. The aim of IEETeL is to connect researchers, educators, and technology experts giving them an opportunity to share and discuss new solutions, trends, and realizations of eLearning environments and the adoption of emerging technologies in educational settings. These will draw the challenging problems in information gathering, processing, and usage; development of web-based and mobile services; building intelligent and social-oriented applications in support of flexible, personalized, adaptable, and in-demand learning. This forum invites research, technical papers, and report of work in progress, investigating eLearning environment development, information processing/management for education in local/small settings and in global/large scale ones, practices of implementation in education to facilitate high school and university students’ learning, and to help then train to become self-organized lifelong learners. (But we may consider proposals ranging over other topics in technology enhanced learning and education). • • • • • • • • • • • • • • • •

Assistive technologies 3D virtual environments Semantic web Adaptive and intuitive environments Personal learning environments Mobile learning environments and applications Augmented Reality implementation Intelligent and smart applications Visual aspects Internet of things Security and privacy Learning analytics Educational data mining Social-collaborative learning Peer learning and peer evaluation Assessment in education.

xiv

Workshop on Interactive Environments and Emerging Technologies for eLearning

Organizing Committee Malinka Ivanova Minoru Nakayama Marco Temperini

Technical University of Sofia, Bulgaria Tokyo Institute of Technology, Japan Sapienza University of Rome, Italy

Program Committee Alexander Mikroyannidis Andrea Sterbini Anna Rozeva Carla Limongelli Elvira Popescu Fernando Albuquerque Costa Fatima Sapundzhi Federica Caruso Filippo Sciarrone Galina Bogdanova Galena Pisoni Laurent Moccozet Luigi Laura Maiga Chang Maria De Marsico Maya Dimitrova Meglena Lazarova Petya Petkova Ricardo Queiros Roumiana Ilieva Svetozar Ilchev Slavi Georgiev Tsvetelina Petrova Valentina Terzieva Victoria Marin Zuzana Kubincová

The Open University, UK Sapienza University of Rome, Italy Technical University of Sofia, Bulgaria Roma Tre University, Italy University of Craiova, Romania University of Lisboa, Portugal South-West University “Neofit Rilski”, Bulgaria University of L’Aquila, Italy Roma Tre University, Italy Bulgarian Academy of Sciences, Bulgaria University of Trento, Italy University of Geneva, Switzerland Uninettuno University, Italy Athabasca University, Canada Sapienza University of Rome, Italy Bulgarian Academy of Sciences, Bulgaria Technical University of Sofia, Bulgaria Technical University of Sofia, Bulgaria Polytechnic of Porto, Portugal Technical University of Sofia, Bulgaria Bulgarian Academy of Sciences, Bulgaria University of Ruse “Angel Kanchev”, Bulgaria Technical University of Sofia, Bulgaria Bulgarian Academy of Sciences, Bulgaria University of Lleida, Spain Comenius University, Bratislava, Slovakia

Workshop on Technology Enhanced Learning in Nursing Education (NURSING)

Nurses are required to implement adequate clinical behaviors and possess a high-level of knowledge and skills to improve patients’ health outcomes and enhance community health-literacy levels. Current evidence highlights the potential power of technology-based educational systems, like simulation and blended learning models, in promoting better learning outcomes for students and professionals in the nursing field. This workshop aims to share the best available knowledge about the application of technology-based systems into undergraduate and graduate nursing educational programs, as well as in the continuing educational programs of healthcare workers. Workshop topics have been grouped into the following three main discussion areas. First, topics on education in nursing academic programs aim to discuss the effects of simulation and other technology-based systems on learning outcomes of nursing students and nurses, including ethical and legal aspects. Secondly, topics on community health educational programs aim to discuss the impact of technology in improving community health-literacy levels. Finally, the workshop intends to provide a complete overview of technology-based methods as useful tools to improve both learning and application of the nursing process in clinical settings.

Organizing Committee Loreto Lancia Cristina Petrucci Angelo Dante

Full Professor of Nursing at the University of L’Aquila, Italy Associate Professor of Nursing at the University of L’Aquila, Italy Assistant Professor of Nursing at the University of L’Aquila, Italy

Program Committee Pierpaolo Vittorini Valeria Caponnetto

University of L’Aquila, Italy University of L’Aquila, Italy

Workshop on Technology - Enhanced Learning for Future Citizens (TEL4FC)

Today’s youth will experience their future citizenship in a rapidly evolving knowledge society. K-12 kids must face crucial and technological challenges while addressing their educational needs and digital awareness. At its third edition, in 2023 TEL4FC workshop aims to make an opening for those researchers, educators, and technologists, who are involved in diverse knowledge areas, to promote among them a meaningful and interdisciplinary debate on computing education. Technology-enhanced learning is increasingly shaping the future of our society and people are boosting their own participation in democracy through digital architectures and the physical interaction to devices. Sharing and collaborating throughout platforms, networks, and interactive ecosystems and using different intelligences are decisive factors of a new approach to learning and teaching as well. Our world’s complexity is definitely permeating education and interlinking apparently different and separate fields: As researchers and educators, we have to facilitate students in shaping a sustainable citizenship in a connected and globalized world. The strategies to integrate diverse disciplines in a TEL framework are the aim of our workshop. It represents an opportunity for debate on how to increase the future citizens’ digital awareness and gradually setting up an ecological and sustainable mindset for all learners. The future of our society indeed depends on an engaged, informed, and critically thinking population. The workshop is meant to connect researchers, educators, and technologists involved in different and diverse areas, such as education, digital government, pedagogy, social and collaborative systems, cultural heritage, and ethics to promote interdisciplinary research around the citizenship and the role of technology enhanced learning in shaping the future of our society, through the construction of future citizens that are fully aware of potentials and risks. The workshop stimulates the submission of papers, of methodological, empirical or technological nature, about technology enhanced learning systems for the education of future citizens, including, but not limited to, the following fields: • • • • • • • • • • • •

Computing education Computational thinking and creative coding Data science (open data, knowledge graphs, data visualization, data exploitation) Media literacy and social learning, such as fake news detection Artificial Intelligence and machine learning Cultural heritage Ethics, privacy, (cyber)-security Social networks Cloud computing Virtual Reality Interaction with vocal assistants and bots Smart city and autonomous vehicles

xviii

Workshop on Technology - Enhanced Learning for Future Citizens (TEL4FC)

• Outreach programs (high school, universities, teachers development) • Digital government and digital participation • Social sciences.

Organizing Committee Maria Angela Pellegrino (Chair) Vittorio Scarano (Program Chair) Agnese Addone (Publicity Chair)

Università degli Studi di Salerno, Laboratorio CINI “Informatica&Scuola”, Italy Università degli Studi di Salerno, Laboratorio CINI “Informatica&Scuola”, Italy Università degli Studi di Salerno, Laboratorio CINI “Informatica&Scuola”, Italy

Program Committee Jerry Andriessen Pietro Boccadoro Serena Cangiano Françoise Détienne Karina Rodriguez Echavarria Rosella Gennari Lorenzo Guasti Beatrice Ligorio Alessandra Melonio Sébastien Nedjar Luca Tateo

Wise and Munro, The Netherlands Politecnico di Bari, Italy SUPSI, Switzerland Centre National de la Recherche Scientifique, France University of Brighton, UK University of Bozen, Italy INDIRE, Italy Università degli Studi di Bari, Italy University of Venice, Italy University of Aix-Marseille, France University of Oslo, Norway

Contents

Workshop on Integration of Emerging Technologies into Education and Training (ETELT) Capacity Building Across Higher Education and Rural Youth in WINnovators Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kai Pata, Kristi Jüristo, Matej Zapušek, Nathalie Leiba, Sanja Popovic Pantic, Vladan Devedzi´c, Sonja Radenkovic, Mirjana Devedzi´c, Marija Blagojevi´c, and Danijela Miloševi´c Combining Design Thinking with Emerging Technologies in K-12 Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marcelo Milrad, Christothea Herodotou, Marianthi Grizioti, Alisa Lincke, Carina Girvan, Sofia Papavlasopoulou, Sagun Shrestha, and Feiran Zhang Leveraging Cognitive and Social Engagement in Blended Learning Through an AI-Augmented Pedagogical Framework . . . . . . . . . . . . . . . . . . . . . . . . Andromachi Filippidi, Christophoros Karachristos, Konstantinos Lavidas, Vassilis Komis, and Nikos Karacapilidis Scenario Design, Data Measurement, and Analysis Approaches in Maritime Simulator Training: A Systematic Review . . . . . . . . . . . . . . . . . . . . . . Ziaul Haque Munim, Helene Krabbel, Per Haavardtun, Tae-Eun Kim, Morten Bustgaard, and Haakon Thorvaldsen Analysis of Creative Engagement in AI Tools in Education Based on the #PPai6 Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dea Puspita Septiani, Panos Kostakos, and Margarida Romero NLP-Assisted Educational Memory Game Experiment . . . . . . . . . . . . . . . . . . . . . . Viktória Burkus, Attila Kárpáti, and László Szécsi Datasets for Artificial Intelligence in Education: The Case of Children with Neurodevelopmental Disorders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marcos L. P. Bueno and Serge Thill Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention in Children with Neurodevelopmental Disorders . . . . . . . . . . . . . . . . . . . Cristina Costescu, Carmen David, Adrian Ros, an, Paula Ferreira, Aristides Ferreira, Lucia Vera, and Gerardo Herrera

3

15

28

39

48

59

70

78

xx

Contents

An Advanced Explainable and Interpretable ML-Based Framework for Educational Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ioannis E. Livieris, Nikos Karacapilidis, Georgios Domalis, and Dimitris Tsakalidis Development of an Immersive Virtual Reality System to Practice the Lumbar Puncture Manoeuvre . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . María Beatriz Villar-López, Águeda Gómez-Cambronero, Daniel Suarez, and Inmaculada Remolar

87

97

Exploring Value and Ethical Dimensions of Disruptive Technologies for Learning and Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Terje Väljataga, Kai Pata, Andrea Annus, Michelle Andrade Calisto, Agueda Gomez Cambronero, Elmar Eisemann, Athina Kasini, Ricardo Marroquim, Inmaculada Remolar, László Szécsi, Amir Zaidi, and Rubén García Vidal Workshop on Interactive Environments and Emerging Technologies for eLearning (IEETeL) Implicit Aspects of the Psychosocial Rehabilitation with a Humanoid Robot . . . 119 Maya Dimitrova, Virginia Ruiz Garate, Dan Withey, and Chris Harper Mobile Game Development Using Unity Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Fatima Sapundzhi, Anton Kitanov, Meglena Lazarova, and Slavi Georgiev Application of Artificial Neural Networks in Intelligent Tutoring: A Contemporary Glance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Tatyana Ivanova, Valentina Terzieva, and Malinka Ivanova Educators’ Support Through Predictive Analytics in an Assessment Process . . . 151 Malinka Ivanova, Petya Petkova, and Tsvetelina Petrova Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory Programming to Large Classes of Novice Students . . . . . . . . . . . . . . . . . . . . . . . . . 163 Philippe Weidmann, Milo Gianinazzi, and Laurent Moccozet Effectiveness of a “Nudge” for Online Discussion Participation About Attitude Toward Essay Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 Minoru Nakayama, Satoru Kikuchi, and Hiroh Yamamoto Design Considerations, Architecture and Implementation of a Wireless Sensor Network for Use in Smart Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Svetozar Ilchev

Contents

xxi

Physical and Digital Accessibility of Museums in Bulgaria: Problems and Innovative Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 Vesela Georgieva, Galina Bogdanova, and Mirena Todorova-Ekmekci Educational Technologies and Video Algorithms at Medical University – Varna, Bulgaria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Diana Dimitrova, Galina Bogdanova, Galya Georgieva-Tsaneva, and Evgeniya Gospodinova System Review and Requirements Analysis for Escape Classroom System . . . . . 212 Milen Petrov, Bozhidara Pachilova, Teodosia Hristodorova, and Adelina Aleksieva-Petrova A Mobile App Game Based on the Development and Design of a Puzzle Created for Educational Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 Fatima Sapundzhi, Anton Kitanov, Meglena Lazarova, and Slavi Georgiev Workshop on Technology Enhanced Learning in Nursing Education (NURSING) Health Service e-Health Platform: A Potential Tool for Nursing Practice and Patient Care . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Martina Cuomo, Sannino Antonio, Luigi Bruno, Assunta Napoleone, Vittorio Masotta, Cristina Petrucci, and Angelo Dante The Participatory Methodology Adopted to Develop an mHealth App as an Educational Tool to Promote Organizational Health Literacy at a Maternal and Child Health Hospital . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Raffaella Dobrina, Chiara De Vita, Cristina Petrucci, Laura Brunelli, Andrea Cassone, Livia Bicego, Luca Ronfani, Eva Orzan, Veronica Di Castro, Paola Di Rocco, Marzia Colautti, Patrizia Borzacchiello, Elisabetta Danielli, Tamara Stampalija, Mario Casolino, Anja Starec, Margherita Dal Cin, and Angelo Dante Factors Associated with Intensive and Critical Care Nursing Students’ Learning Gains Exposed to High-Fidelity Simulation Training . . . . . . . . . . . . . . . 252 Vittorio Masotta, Angelo Dante, Fabio Ferraiuolo, Francesca Ferretti, Valeria Caponnetto, Alessia Marcotullio, Luca Bertocchi, Francesco Camero, and Cristina Petrucci Community Healthcare and Electronic Nursing Documentation . . . . . . . . . . . . . . 261 Mariangela Vanalli

xxii

Contents

Virtual Gamification in Mental Health Nursing Education: An In-Depth Scoping Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 Mauro Parozzi, Stefano Terzoni, Sonia Lomuscio, Paolo Ferrara, and Anne Destrebecq 360-Degree Video for Cardiopulmonary Resuscitation (CPR) Knowledge: Preliminary Data of a Randomized Controlled Trial . . . . . . . . . . . . . . . . . . . . . . . . 280 Andrea Gazzelloni, Marco Sguanci, Michela Piredda, Cristina Calandrella, Gaetano Tieri, Simone Piga, Valentina Pizziconi, Giuliana D’Elpidio, Rosaria Alvaro, and Maria Grazia De Marinis Workshop on Technology - Enhanced Learning for Future Citizens (TEL4FC) Open Data Value Creation by High-School Learners via Data Stories . . . . . . . . . 291 Maria Anna Ambrosino and Vanja Annunziata Collaborative Cybersecurity Awareness Learning in Higher Education . . . . . . . . 303 Vittorio Scarano, Palmieri Giuseppina, Jerry Andriessen, and Mirjam Pardijs Digital Job Searching and Recruitment Platforms: A Semi-systematic Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Chiara Signore, Bice Della Piana, and Francesco Di Vincenzo Towards an Energy Social Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Stefano Converso, Ivana Veselinova, Gabriele Roselli, and Hamed Abbasi Mofrad Engagement in Open Data Workshops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 Alessia Antelmi Open Tools for Data Literacy Development at School. A Case of Systemic and Experience Design Approach to Civic Tech Education . . . . . . . . . . . . . . . . . . 343 Serena Cangiano Listening to Children and Adolescents with Special Needs During Covid-19 Pandemic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 Luca Tateo Engaging Learners in Familiarizing Themselves with Sensors and Actuators . . . 359 Mauro D’Angelo

Contents

xxiii

Using Videoconferencing Systems and Interactive Tools: Empirical French Investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 Maria Impedovo, Benjamin Ett, and Md Saiffudin Khalid Teaching Blockchain at School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378 Lorenzo Guasti, Alessandro Ferrini, and Gabriele Pieraccini Correction to: The Participatory Methodology Adopted to Develop an mHealth App as an Educational Tool to Promote Organizational Health Literacy at a Maternal and Child Health Hospital . . . . . . . . . . . . . . . . . . . . . . . . . . . Raffaella Dobrina, Chiara De Vita, Cristina Petrucci, Laura Brunelli, Andrea Cassone, Livia Bicego, Luca Ronfani, Eva Orzan, Veronica Di Castro, Paola Di Rocco, Marzia Colautti, Patrizia Borzacchiello, Elisabetta Danielli, Tamara Stampalija, Mario Casolino, Anja Starec, Margherita Dal Cin, and Angelo Dante Correction to: Exploring Value and Ethical Dimensions of Disruptive Technologies for Learning and Teaching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Terje Väljataga, Kai Pata, Andrea Annus, Michelle Andrade Calisto, Agueda Gomez Cambronero, Elmar Eisemann, Athina Kasini, Ricardo Marroquim, Inmaculada Remolar, László Szécsi, Amir Zaidi, and Rubén García Vidal

C1

C2

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

Workshop on Integration of Emerging Technologies into Education and Training (ETELT)

Capacity Building Across Higher Education and Rural Youth in WINnovators Space Kai Pata1(B) , Kristi Jüristo1 , Matej Zapušek2 , Nathalie Leiba3 , Sanja Popovic Pantic4 , Vladan Devedzi´c5,6 , Sonja Radenkovic5 , Mirjana Devedzi´c5 , Marija Blagojevi´c7 , and Danijela Miloševi´c7 1 School of Education, Tallinn University, Tallinn, Estonia

{kpata,kristi.juristo}@tlu.ee 2 Ljubljana University, Ljubljana, Slovenia [email protected] 3 VITECO, Catania, Italy [email protected] 4 Institute Mihajlo Pupin, Science and Technology Policy Research Center, Belgrade, Serbia [email protected] 5 University of Belgrade, Belgrade, Serbia [email protected] 6 Serbian Academy of Sciences and Arts, Belgrade, Serbia 7 Faculty of Technical Sciences, University of Kragujevac, Kragujevac, Serbia [email protected]

Abstract. This paper demonstrates the Design as a Hypothesis Framework for developing cross-university students and mentors, and rural youth (aged 18–30) and regional business ecosystems capacity building practice approaches to support sustainable development goals. We describe how a gamified learning and co-working WINnovators Space (https://winnovators-space.eu/) with e-learning materials for self-learning and mentored group work problem based challenges was developed to support university students’, mentors and the business partners’ engagement and building agency and capacity with regional rural young women. The Pilot study with the capacity building practice application validates the Design Hypothesis in three countries – Estonia (N = 35), Slovenia (N28), Serbia (N22) – involving young rural women, higher education students, academic and business mentors. Keywords: Capacity building · Higher education practice · Inclusive practice · Sustainability

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 3–14, 2023. https://doi.org/10.1007/978-3-031-42134-1_1

4

K. Pata et al.

1 Introduction 1.1 Capacity Building and Agency Development Across Higher Education and Communities Goals for Education 2030 [1], and sustainable action along the SDG 4 – providing lifelong learning for all, and SDG5 – considering gender equality in STEM education [2] require educational institutions to step out as actuating actors in the society. Boeren [2] recommended an interdependent approach that relates the development of individuals and the social structures around them at micro (e.g. socio-demographic and socio-economic factors, people’s attitudes, confidence, interests, and motivation to learn), meso (structure of educational offers, forms of practices), and macro levels (e.g. legal, regulatory, financial, political, ideological factors). In this paper we aim to demonstrate an approach where higher education acts for the society building the sustainable development capacity. Capacity concept denotes systemic readiness. Morgan [3] defines capacity as an emergent property in social ecosystems, an interaction effect that comes out of the dynamics involving a complex combination of attitudes, resources, strategies and skills, both tangible and intangible. Capacity as a state is an action potentiality of individuals within social and institutional contexts. It is a dynamic construct that has to be nourished in actions between different partners to solve challenges such as Education for Sustainable development Goals are. Our design research aims for developing new types of capacity building practices between universities and society. Capacity building requires Higher Education and Regional ecosystems to find new ways to orchestrate educational goals and formal education forms with the socioeconomic and socio-cultural community life expectations and with the informal education opportunities in regions. Blending formal and nonformal educational opportunities is one of the capacity building forms that Higher Education institutions (HEI) develop to offer flexible education with equal access. HEIs are increasingly demanded to be sustainable development actors, building partnerships with regional business ecosystems, for example, in advancing business and STEM competencies and overall active citizenship mentality in remote areas. Agency development of people is one central educational goal, because agency is a state that relates personal characteristics and contexts with the opportunities and problem solving challenges one has in life, as well as with the enabling situations or constraints [4]. Lifelong learning strategic documents that we list below have related the agency with the concept of active citizenship that one can learn in various formal, informal and nonformal ways. Competences for active citizens are incorporated into European agendas such as agenda “The future of education and skills: Education 2030” [5], the “UNESCO learning objectives for sustainable development” [6], the “Preparing our youth for an inclusive and sustainable world: The OECD PISA global competence framework” [7] and the GreenComp, “The European sustainability competence framework” [8]. In our design study we seek opportunities to blend formal HEI students’ studies with informal learning of young people in remote areas to develop their active citizenship and entrepreneurial, STEM and sustainability competences, as well as advancing a shared capacity between universities and regions.

Capacity Building Across Higher Education and Rural Youth

5

1.2 Research Problem The problem of this paper is the need to develop new capacity building approaches through which HEIs act out in the regional communities to promote informal learning for undermined young people who have fewer access to learning for STEM, sustainability, entrepreneurship. The research question for design study is: How can the universities grow the agency of higher education students to become change agents for the remote communities and help the undermined rural youth (18–30) to have equal access to entrepreneurial, stem and sustainability competencies in their regions?

2 Methods 2.1 Context of the Study To achieve sustainable development goals in education for all Education 2030: Incheon Declaration and Framework of Action [1] has stated that we need to promote equal access to education, developing flexible forms of learning that incorporate and blend formal and informal approaches, enabling for learners different learning paces and spaces. Our study was conducted in the frames of KA2 Erasmus + project WINnovators (2021– 2024) partnership aiming to explore the capacity building approaches in higher education teaching and learning that are directed to the regional communities, and particularly to young rural women in remote areas. The study builds a particular blended learning space, a learning community across higher education, regional rural ecosystems and business ecosystems, and explores it in action during the Pilots in three countries – Estonia, Serbia and Slovenia. 2.2 The Design Process In this study we applied a participatory design-based research method, aligned to software and interaction Design as a Hypothesis approach [9] that aims for designing tools for complex social systems where the iterative and hermeneutic design process consists of four partly overlapping phases: contextual inquiry, participatory design, product design, and production of solution as hypotheses. Our design hypothesis states that the WINnovators design aims for a new type of capacity building practice that unites universities and regions, and advances young people agency as active citizens competent to create STEM and sustainability related business ideas. Design as a Hypothesis approach means that we are experimental and analytic, but we try to keep ourselves open to serendipity that helps to be flexible in our design solutions. We acknowledge that our Pilots may result in some failures, and some of the outcomes can be different than initially expected. We followed the Design as a Hypothesis approach process stages: I. Contextual inquiry for defining the WINnovators context i) Benchmarking the trends in inclusive education for active citizenship;

6

K. Pata et al.

ii) Identifying the needs with an ethnographic exploration (observations and interviews with stakeholders in higher education, rural regions) about who-what-whywhere capacity building between higher education, rural ecosystems and businesses could contribute; II. Participatory design of Personas: iii) Design thinking and Persona development to represent the HEI learners as potential agents of change, and rural young women as community entrepreneurs and future leaders; III. Product conceptual design: iv) The design of a WINnovators Competence Framework for active citizenship; v) The design of gamified interaction principles for engagement between different stakeholders in informal learning scenarios; IV. Production of software and interaction as hypotheses: vi) The design of the eLearning course topics, and Teamwork Challenges; vii) The design of the interactive WINnovators Space https://winnovators-space.eu/ V. Testing and validating the hypothesis in Pilots viii) The Design is a formatively advanced process by a ‘community’ involvement: academics as the designers, higher education specialists and mentors; the students as learners, mentors, and change agents; the rural youth as learners, future entrepreneurs and active citizens; and the regional business ecosystem experts as mentors. ix) Evaluation: The usability survey of the Winnovator Space; the formative diaries tracking teams and country Cases; the competencies stored by Winnovator Space. 2.3 The Sample The Pilot study sample was formed in three countries (Estonia, Serbia, Slovenia) using the convenience sampling method (accessibility in the partner universities), and the snowball sampling approach involving students and rural young women through design stages. In three Piloting countries the sample of engaged participants is N = 85. The students, teachers and rural young women are testing out the designed learning practice in WINnovators Space. Estonia: youth work students, youth workers in rural areas, young rural women representing three target groups: young adults in unsatisfied life situations, young mothers and young unemployed adults and/or school drop-outs. Serbia: BADEN, the network of academic researchers from different universities, students from the computer science and business fields, the networked Serbian Association of Business Women, the female entrepreneurs, young rural women representing Romas. Slovenia: HEI teachers and students from the two-subject teacher study programmes Computer Science and Art Pedagogy. Cooperation with the Employment Service in the framework of the PUM-O project, which deals with young people who have not completed formal education, and with the Chamber of Commerce and Agriculture of Slovenia. Through these organisations, young rural women from vulnerable groups who have not completed their education were contacted.

Capacity Building Across Higher Education and Rural Youth

7

3 Results of the WINnovators Design As we are following the Design as a Hypothesis approach, in the following sections we will introduce step-by-step the results achieved in each design phase. 3.1 WINnovators’s Design Hypothesis About Capacity Building and Personal Agency This research addresses the research question: How can the universities grow the agency of higher education students to become change agents for the remote communities and help the undermined rural youth (18–30) to have equal access to entrepreneurial, stem and sustainability competencies in their regions? Our hypotheses consider two levels of impacts from learning as a WINnovator. Design hypothesis 1: The capacity between the different stakeholders can be built in problem solving groups where learning for STEAM, sustainability and entrepreneurial competencies is supported. We assert that working together in teams will grow the agency of HEI and rural Youth to become more self-development driven and community goods driven as change agents. They will gain competences that they can use in the future for common good. The indicators of capacity building are i) formed teams between HE students and rural young women, who pursue jointly for digital team challenges on STEM, sustainability related topics, ii) active citizenship competences individuals have gained through learning together. This hypothesis may be validated tracing teamwork and active citizenship competences in WINnovator Space, and using formative reports. Design hypothesis 2: At the institutional and regional level the joint capacity can be created between HEIs and the local stakeholder organisations. We assert that capacities as shared activity systems need to be built around shared teaching and problem solving practices across organisational borders and revising institutional regulatory frameworks for formal learning. The indicators of capacity building across HEIs and local communities are: i) established and durable support systems through which it is possible to recruit young rural women to the WINnovators learning activities provided jointly with HEI students as change agents; and ii) established coordinated support actions provided to young rural women by the local communities and the universities. This hypothesis may be validated tracing offline and online networking activities that the WINnovator Space can enhance, as well as we can use the formative reports. 3.2 WINnovator’s Space – a Shared Interaction Hub for Capacity Building The first prototype of WINnovator Space for interactive learning was born as a codesign result within the Consortium, engaging HEI experts and regional business and youth work ecosystems. We used the Persona approach to describe the needs of young people and students in the case of creating joint learning processes. Working closely with local partner organisations, we specified the target groups within the young women in vulnerable life situations, their specific characteristics and needs. Collected information and knowledge helped to develop the WINnovators e-learning materials and challenge modules that may correspond to the development of young people’s competences,

8

K. Pata et al.

offering self-fulfilment and a sense of achievement, as well as sufficient challenge and opportunities for experimentation. WINnovators’ Space (Fig. 1): • Supports the roles of mentors (from academia and business ecosystems), change agents (students from the university), and the WINnovators (young women in change). The interaction in the community is built around the design ideas as challenges that young women can try out with their supporters (mentors and change agents). • Provides online elearning content to achieve competences by individual learning or by learning in teams to solve problem challenges. WINnovator Space promotes elearning from self-learning short lessons. • WINnovator Space is also a community space in which teams can be formed around problem challenges that Rural Young Women see in their entrepreneurial visions. Team-working for problem challenges is promoted by team portfolios. Teams can upload files, pictures and videos about their problem solving journeys to the team portfolios. • Motivation to learn competences is managed by a gamified open badge provision system. The role shift in agency is measurable with the badge system that associates active citizenship competences with progress that we describe in the next section. Badges can be earned from individual elearning and for team activities. • Networking across different users is established by social media features, personal profiles.

Fig. 1. WINnovator Space provides interaction, elearning and teamwork for cross universitycommunity learning

3.3 WINnovators Space – a Gamified Motivation Space WINnovator Space is developed as a motivational space for capacity building between HEIs and the communities that would support the advancement in personal competencies and in the team and networking capabilities. To make this motivational space

Capacity Building Across Higher Education and Rural Youth

9

extrinsic the open badge [10] system was related with the WINnovators Space. For open badges a WINnovator competency framework was composed from several frameworks: Dig.comp 2.1 [11], GreenComp [8], Sustainability competencies [12], Learning for the future, Competencies in education for sustainable development [13], Entrepreneurial competencies [14], Innovation leadership competences [15], Innovation competencies [16] and Project management competencies [17]. The WINnovator competencies are grouped into the five dimensions that relate with the development of personal agency and group agency in the time of challenges: Learning to be; Learning to value, Learning to live and work together, Learning to comprehend and contribute and Learning to empower and lead. A leaderboard (Fig. 2) was integrated into the platform, to show the active users and the ones who earned the most badges.

Fig. 2. WINnovator Space leaderboard ranks achievements in five competence areas for becoming Winnovators - change agents in the communities

HEI students and young women who learn together can collect badges when they do individual learning or from group challenges. The advancement in competencies can be measured in action by people themselves, their team members (change agents), or by the mentors. In group work, badges are awarded for completing a task or for creating a product that must be first assessed by the instructor and are therefore awarded manually. ‘Learning to be’ WINnovators competencies are difficult to identify in others, therefore there is a possibility of awarding a limited number of badges to oneself. The accumulation of competencies enables learners to move up the rank: bronze, silver, and gold levels of being a WINinnovator - denoting the state of active citizenship competencies and agency. Finally, the system issues WINnovators certificates to users. 3.4 Testing the Capacity Building Hypothesis in WINnovators’ Pilots The trainings on the WINnovators platform are (see Fig. 1): 1. STEM/STEAM entrepreneurial communities for young women: General learning resources; 2. STEM/STEAM entrepreneurial communities for young women: Challenges. Within the first group of training, HEI students as well as young women in rural areas have the opportunity to gain competencies in the areas of project and teamwork management, development of websites, the time management, promotion of sustainable

10

K. Pata et al.

entrepreneurship, leadership, creative thinking, business creation plan, etc. The second group of training practice collection provides challenges in the field of using digital technologies in marketing, tourist services with gamified digital elements, development of digital learning communities, etc. For each lesson, an opportunity to earn specific badges is defined, through the presentation of the lesson or by taking a test. In proposed challenge learning modules the focus is on the collaboration among peers - students and young women working together, practice oriented learning with real-life connected activities and tangible outcomes or learning that can be transferred to young adults daily activities or other spheres of life. The role of the students in the initial phase of the Pilot is to establish contact with the young women and to advise them on the selection of appropriate learning units according to their needs and expectations. Within the online platform, teachers have formed subgroups for specific challenges and invited change agents (students) and WINnovators (rural young women) to participate. By presenting students with the same challenges as participants and allowing them to expand their participation with gamification elements in the form of badges, we aim to build a collegial relationship and sense of partnership between them. These connections can be sustained later as students move into their careers. Through networking, we want to foster collaboration in their local communities, where they can act as agents of change and promoters of education to other potential candidates. Different engagement approaches emerged at the Pilots to build capacity between HEIs and local communities and business ecosystems. Case: Estonia. The strategy to engage and support the participation of young women in vulnerable life situations in Estonia was through youth and community workers. Involving youth work organisations operating in formalised structures and working directly with youth at a municipality level was seen as an opportunity to design and offer activities that address the needs and fit into the local environment as well as correspond to the young adults’ everyday life situations, their interest and needs. Youth work organisations involved in the project are connected with HEI either through youth workers who are alumni of the youth work applied higher education or master programmes or serving as a traineeship base for youth work students. Previous collaboration created favourable conditions to build and keep alive a dynamical co-design and learning process between students and young rural women, between universities and the communities and between surrounding ecosystems. 2nd year youth work students were involved through a project based learning course with the aim of preparing students for project work as part of the daily activities of a youth worker, to find and use funding for youth projects, to support young people in the implementation of projects. Specific tasks of HEI students were connected with problem-based, collaborative and situated learning that took place through challenge modules in WINnovators Space. The contact with the target group of rural young women was based on trusting relationships between youth work students and young women who started to strengthen their sense of belonging in the community. Case: Serbia. The women entrepreneurs in rural areas in Serbia have not completed business schools and colleges, and have the problem of how to digitally transform their business and to enter from the local market to the electronic market. In the case developed in Serbia, the regional Winnovators learning ecosystem was established by the following

Capacity Building Across Higher Education and Rural Youth

11

interactions between the partners: Association of Business Women Serbia (ABWS) – HEI teachers: ABWS acted as the major source of contacts with young women from rural areas and provided the necessary link between the universities and the learners, HEI teachers – young rural women: the teachers were responsible for course development, presentation, maintenance and administration, as well as for fostering communication with young women; through their personal connections, the teachers attracted the young rural women to take the Winnovator courses. HEI teachers – HEI students (change agents): the teachers selected volunteering students and trained them to become change agents and carry out immediate communication with the young rural women. HEI students – young rural women: students contacted young women to take the WINnovators courses based on their personal connections and origin. Their age was similar to that of the prospective learners, which has proven helpful. The students initiated the promotion of the courses in rural areas, and gradually became the major contact points for the learners. HEI administration – HEI students: administration was asked to accept the students’ participation in the WINnovators activities as (part of) the mandatory final year project / field work, and also to issue certificates of completing extracurricular activities where appropriate. Case: Slovenia. In Slovenia, the organisations at the regional level were involved that are well established in the community, have a broad network of users, and operate within formalised organisational structures. The Employment Service, which runs a project training programme for young adults (PUM-O) who, for a variety of reasons, have not completed their formal education and want to acquire relevant skills to facilitate their entry into the labour market or their entrepreneurial path. The network of PUM-O centres is spread throughout the country, operates at the local level and can better identify and address people in their local environment and better adapt to their wants and needs. The course was promoted with the help of the consultants. A longer-term cooperation with the Chamber of Commerce and Agriculture of Slovenia was established, which is responsible at the national level for advising and promoting an economic and environmentally friendly agriculture, forestry and fishery. It has a very wide network of branches at the local level throughout the country and has contact with a large number of farmers, foresters and fishermen. Through its communication channels, young rural women were reached, who were among the most vulnerable groups due to their remoteness from centres and their lower level of education in digital skills and STEAM. HEI teachers recruited upper year students of education to participate in the project by introducing the project, its aims and objectives, and inviting them to participate. These students already acquire knowledge about working with vulnerable groups of people within the framework of the core pedagogical subjects and also have the relevant didactic knowledge that enables them to approach learning support more effectively, to find the appropriate interpretation of terms and concepts for the target audience, and to take into account the specifics of the subject didactics of the respective field, the psychology of learning, thus adapting better to the level of the learners. The teachers had an initial interview with the students, in which they explained in detail the process of the training and defined their role. They also informed them of their goal - to help the learner become a change agent in their local environment and spread knowledge and awareness about the importance of sustainable aspects of business. The weekly online

12

K. Pata et al.

meetings between the young rural women and the HEI students to promote the progress were coordinated by university mentors. All the Pilots started at Spring term 2023 and will last about 4 months. The next section presents the preliminary evaluation results about how WINnovators’ Design Hypothesis about capacity building and personal agency was achieved. After the pilots are finished the summative evaluation of the success indicators is conducted.

4 Formative Evaluation - Capacity Building Engagements Across Higher Education and Regional Youth WINnovators project has so far reached the formative Hypothesis validation phase where we will observe how our design concept was implemented in the WINnovator space in three Pilots. Two impacts were targeted: • the HEI students and rural young women should learn in problem based challenges and increase their competence as active citizens, • the capacity should be formed between stakeholder organisations. The indicators of the successful capacity building and learning for active citizen competences were observed at personal, team and country levels. Personal level: The 85 HEI students and the young women who participate in Pilot in Winnovator Space have completed a number of individual elearning courses and are currently working on the team challenges. 67% (N = 57) of the participants have been awarded with some of the Winnovators competences. Notable is that the most achievable have been the competences of Comprehending and contributing, Learning to be as a change agent, and the competences of Empowering and leading the others that all relate with active citizenship in the communities. The least awarded were the Learning to value competences (see Fig. 3).

Fig. 3. The overview of the Winnovator competences the participants achieved

The team level indicators: The Pilot is in the stage where 9 teams were established between HEI students, young rural women, and they have started to work on the challenges on the issues of sustainability footprint, digital promotion of business, and digital promotion of the communities. Reaching to the teamwork level using only online communication without personal meetings has been very difficult.

Capacity Building Across Higher Education and Rural Youth

13

The indicators of the capacity formation at the universities and the local communities level may be characterised as the following: The teams are supposed to work in the distant mode in WINnovators Space. The big challenge is that for regional young women the WINnovators learning activity is informal, and they do it from intrinsic interest. To keep them motivated and on track the local mediators are useful. In the Estonian case, the support comes from local youth workers, alumni of HEIs who can act as the mediating agents having trust relationships with local youth, and as alumni they are still connected with the university. In the Serbian case, the local mediators are business women from ABWS who mediate the youth to the BADEN network of HEI educators. In the Slovenian case the regional youth is motivated through a more formal approach created by their alignment to the Unemployment office and the Chamber of Commerce and Agriculture training programmes. HEIs in piloting countries have built partnerships with formal organisations (the Youth work centres, The Employment Service, the Chamber of Commerce and Agriculture, The Association of Business Women). We noticed that these organisations were good at helping to identify the people whom to reach out locally for educational interventions. However, these organisations did not have a good strategy to motivate the rural women to keep on learning. We found that the personal level partnerships between HEI students and the rural young women enabled them to establish the motivating bonds that could hold the young women in learning. We noticed that despite the networking features in WINnovators Space, the networking in learning teams was shifted to the other social media environments, using Facebook MSN, WhatsApp, Google drive and Zoom meetings. An important element in joint capacity building was considered developing the link between the formal and informal learning structures that differ by their motivational elements. To work with rural young women as change agents the students were assigned with HEI course tasks in two Pilots (Estonia, Slovenia), in Serbian pilot an extracurricular activity was offered for the students. All the HEI students joined in because they could collect credit points in the HEIs. For students course assessment provides an additional extrinsic motivation for participation. The young rural women, as well as the HEI students will receive in Winnovator space the open badges about their competences, as well as they can get final certificates from the system which they found important to advance in their career. To motivate individual and team learning the open badge system in the WINnovators Space enables monitoring the advancements at the leaderboard. In the end of the Pilot we will ask about the gamification elements usability with the survey. We have noticed that so far HEI students did rarely use their opportunity to request the badges for their team members.

5 Conclusion This study provided a preliminary view of how to build capacity and across university borders using a blended approach. The Piloting is ongoing until 2024. We will validate how to design digitally enhanced new learning practices that align to sustainable development SDG 4 – providing lifelong learning for all, and SDG5 – considering gender equality in STEM education, showing how HEIs could be promoters of educational inclusion in novel ways by doing capacity building with the communities.

14

K. Pata et al.

Acknowledgements. This research was supported by Erasmus+ project funding 2021-1-EE01KA220-HED-000032081.

References 1. Mundial, G.B., UNICEF.: Education 2030: Incheon declaration and framework for action: towards inclusive and equitable quality education and lifelong learning for all (2016) 2. Boeren, E.: Understanding sustainable development goal (SDG) 4 on “quality education” from micro, meso and macro perspectives. Int. Rev. Educ. 65(2), 277–294 (2019). https://doi. org/10.1007/s11159-019-09772-7 3. Morgan, P.: The concept of capacity. Eur. Centre Dev. Policy Manag. 1(19), 826–840 (2006) 4. Emirbayer, M., Mische, A.: What is agency? Am. J. Sociol. 103(4), 962–1023 (1998) 5. Howells, K.: The future of education and skills: education 2030: the future we want (2018) 6. Rieckmann, M.: Education for sustainable development goals: Learning objectives. Unesco Publishing (2017) 7. OECD.: Preparing our youth for an inclusive and sustainable world: The OECD PISA global competence framework (2018) 8. Bianchi, G., Pisiotis, U., Cabrera Giraldez, M.: GreenComp the European sustainability competence framework (No. JRC128040). Joint Research Centre (Seville site) (2022) 9. Leinonen, T., Toikkanen, T., Silfvast, K.: Software as hypothesis: research-based design methodology. In: Proceedings of the Tenth Anniversary Conference on Participatory Design 2008, pp. 61–70 (2008) 10. Jovanovic, J., Devedzic, V.: Open badges: Challenges and opportunities. In: Popescu, E., Lau, R.W.H., Pata, K., Leung, H., Laanpere, M. (eds.) ICWL 2014. LNCS, vol. 8613, pp. 56–65. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09635-3_6 11. Stephanie, C.G., Riina, V., Yves, P.: DigComp 2.1: the digital competence framework for citizens with eight proficiency levels and examples of use (2017) 12. Dzhengiz, T., Niesten, E.: Competences for environmental sustainability: a systematic review on the impact of absorptive capacity and capabilities. J. Bus. Ethics 162(4), 881–906 (2020) 13. ECE, U.: Learning for the future: Competences in Education for Sustainable Development. Geneva, United Nations Economic Commission for Europe, Steering Committee on Education for Sustainable Development (2011) 14. Kyguolien˙e, A., Švipas, L.: Personal entrepreneurial competencies of participants in experiential entrepreneurship education. Organizacij˛u vadyba: sisteminiai tyrimai 82, 37–51 (2019) 15. Vlok, A.: A leadership competency profile for innovation leaders in a science-based research and innovation organisation in South Africa. Procedia Soc. Behav. Sci. 41, 209–226 (2012) 16. Waychal, P.K.: A framework for developing innovation competencies. In: 2016 ASEE Annual Conference & Exposition (2016) 17. Dogbegah, R., Owusu-Manu, D., Omoteso, K.: A principal component analysis of project management competencies for the Ghanaian construction industry. Australas. J. Constr. Econ. Build. 11(1), 26–40 (2011)

Combining Design Thinking with Emerging Technologies in K-12 Education Marcelo Milrad1(B) , Christothea Herodotou2 , Marianthi Grizioti3 , Alisa Lincke1 , Carina Girvan4 , Sofia Papavlasopoulou5 , Sagun Shrestha2 , and Feiran Zhang5 1 Linnaeus University, Hus D, 2269D Växjö, Sweden {marcelo.milrad,alisa.lincke}@lnu.se 2 Open University’s Institute of Educational Technology, Walton Hall, Milton Keynes MK7 6AA, UK {christothea.herodotou,sagun.shrestha}@open.ac.uk 3 National and Kapodistrian University of Athens, 157 72 Athens, Greece [email protected] 4 The University of Dublin, College Green, Dublin 2, Ireland [email protected] 5 Norwegian University of Science and Technology, Høgskoleringen 1, 7034 Trondheim, Norway {spapav,feiran.zhang}@ntnu.no

Abstract. The use of emerging technologies such as Artificial Intelligence, Augmented Reality, and 3D printing are amongst the EU targeted actions for supporting the digital transformation of education. Yet, despite these technologies being accessible to education stakeholders, there is a lack of concrete pedagogy and teachers’ professional development for their meaningful integration into the current educational context. Extending Design Thinking with Emerging Digital Technologies – Exten (D.T.)2 - is a European funded project aiming to use emerging technologies to enhance the pedagogical value, sustainable digitization, and potential for wide deployment of Design Thinking (DT). DT is a promising pedagogical innovation, based on interdisciplinary co-creation, that can lead to sustainable educational innovation and development of students’ 21st century skills. In this paper, we describe the main components of the proposed educational transformation as developed during the first year of Exten (D.T.)2 including the theoretical foundations of the approach, the co-creation processes for meaningful engagement of teachers with educational innovation. Moreover, we present the educational technology tools (design, extensions, architecture) that are used in the project and the evaluation approach for capturing impact on improving students’ skills and competencies. Keywords: Design thinking · Emerging digital technologies · Digital transformation · XXI century skills

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 15–27, 2023. https://doi.org/10.1007/978-3-031-42134-1_2

16

M. Milrad et al.

1 Introduction While emerging technologies are rapidly evolving and diffused in our daily practices, their potential has not yet been harnessed to improve K-12 educational practice, enable students to develop the so-called 21st century skills - much needed in an ever-changing society, and create sustainable models of education bulletproof of future disasters (e.g., a pandemic) and flexible enough to achieve learning for all, anytime and anywhere. The project Extending Design Thinking with Emerging Digital Technologies - Exten (D.T.)2 aims to bring innovation in secondary education by combining design thinking and emerging digital technologies and provide an evidence-based paradigm of how teaching and learning could be sustainably transformed. Design Thinking (DT)consists of the following distinct, interconnected, iterative stages: empathizing with users, ideation and brainstorming, prototyping, testing and refinement of a solution/product. It has originated from industry [25] with a few implementations in education. Such implementations are accompanied by considerable challenges such as a lack of appropriate teachers’ training, constraints in resources and time, minimum integration with current curricula, limited evidence on developing creativity, fast convergence on a single idea, student confusion and frustration, and collaboration tensions [27]. To address these challenges, Exten (D.T.)2 aims to enhance DT pedagogy with emerging technologies and address the following Research Objectives (ROs): RO1: Design, implement and scale-up a transformative pedagogical intervention, supporting the implementation, monitoring and evaluation of DT projects extended with emerging technologies. RO2: Bring together different stakeholders to rethink the nature of emerging technologies for designing co-constructionist activities i.e., co-creating resources and technologies. RO3: Support Teacher Professional Development concerning competencies needed for the meaningful integration of DT projects at schools. RO4 Create a network of schools and organizations to collaborate on designing projects during and beyond the project timeframe. RO5: Develop a framework for stakeholders and policy makers including guidelines on how to set up, monitor and evaluate DT projects supported by the project’s emerging technologies. To illustrate the Exten (D.T.)2 approach we provide a concrete example of a learning scenario: A group of four students works on a DT educational project about “biodegradable, but attractive jewelry”. To understand the problem (why do we need environmentally biodegradable jewels?), they play a Tetris-like interactive (using gesture-based interaction techniques) game in which they sort with their body “falling” jewelry made of different materials (e.g. iron, wood, gold, plastic) into different categories that represent the time each material group needs to be degraded by the environment. Through an embodied experience, they realize that most of the jewels are made with slowly degradable material causing long term environmental pollution. While they play, an Authorable Learning Analytics (LA) component captures all data related to how students interacted with the game. The teacher has configured the LA component with necessary interaction indicators for real-time support to be available to students. In the

Combining Design Thinking with Emerging Technologies in K-12 Education

17

next DT stage, students use an online modeler to rapidly prototype 3D models of different jewelry. Students upload these digital models on a citizen science platform and ask other students, teachers, parents for feedback. When 3D models are finalized, students print them with a 3D printer that uses biodegradable filaments. This example has been presented to just illustrate the core ideas of our project and how design thinking and emerging technologies could be combined to support digital transformation processes in education. The remaining of the paper structured as follows, in the next section, we summarize how DT has been used in education and how emerging technologies could address some of the DT- related challenges. In Sect. 3, we detail how educational innovation could be facilitated through the meaningful engagement of teachers in the process of design and research and how tools such as the Activity plan template (Sect. 4) could scaffold research practices with teachers. In Sect. 5, we present the emerging technologies we plan to use to enhance DT, their architecture, and enhancements and in Sect. 6 give an overview of the evaluation approach of the project. Section 7 concludes the paper by presenting our final remarks.

2 Theoretical Foundations Related to DT and Emerging Technologies in Education DT is a dynamic process that emphasizes human-centered and user-oriented design and presents clear steps for collaboratively creating novel, sustainable and practical product solutions [3]. Prior studies using design thinking in education have highlighted the significance of DT in equipping younger generations with essential 21st-century skills to thrive in a rapidly evolving era. DT is perceived as a crucial educational resource and a highly esteemed way for preparing students for the demands of the 21st century [25]. Additionally, DT empowers students to adopt a user-centered approach to address wicked problems, which are complex, real-world, contested, and socio-scientific issues. This approach is particularly relevant in developing innovative solutions to sustainability challenges that require further experimentation [4]. Some educational concepts, such as maker education, constructionism, and design-based learning, have been used interchangeably or blended with DT. For instance, both maker education and DT share similar features such as ideation, making, and the fostering of 21st century skills. One study [37] reported that many practices had employed both approaches of DT and maker education in various educational settings, including after-school activities and school environments. A couple of case studies illustrate the use of these pedagogical concepts, such as the Fab Lab Oulu in Finland [20, 29]. DT is widely acknowledged as a crucial link in multidisciplinary education, particularly in STEAM subjects. Its application in K-12 education is diverse, with a strong emphasis on STEM and STEAM education in most DT programs [27]. Some of these programs employ the DT process to engage students in engineering problem-solving [26], teach physics concepts [32], and programming [13]. DT has also been integrated into other subjects such as design [7], geography [12], technology [11, 24], and multidisciplinary curricula [8, 22]. The promising impacts of DT in education include improvements in students’ selfefficacy [33] and interest in STEM subjects [31], as well as the development of 21stcentury skills. Moreover, education research is increasingly reporting on the opportunities for teaching and applying DT in digital contexts [9]. For instance, digital-enriched

18

M. Milrad et al.

environments offer prospects for fostering DT skills [14], while DT projects facilitated through computer-based environments have been shown to facilitate the learning of mathematical concepts and principles [6]. Specifically, computer-aided drawing and 3D printing can aid students’ comprehension of concepts, enable them to express their creativity, foster their motivation to learn, and improve their DT skills [15]. The effective utilization of technologies is acknowledged as a critical component of DT education and society in general, contributing to the digital education action plan (2021–2027). Although the integration of DT into school curricula, and the use of technologies offer many potential benefits, several challenges have been identified. One such challenge is that it can be difficult for instructors to discuss their activities and professional development [19]. The use of DT as a pedagogical approach, in this context, has the potential to enhance teachers’ technological pedagogical content knowledge in technology-integrated lesson design and 21st-century learning [17]. A finding that has emerged from reviewing relevant literature is that digital tools have only been used in specific stages of DT and mainly when teaching takes place online. For instance, no specific tool has been used to support the process of empathizing with users or no dedicated tools have been designed to track student progress and collaboration in real-time (learning analytics) and provide feedback as needed. Also, emerging technologies such as augmented reality motion sensors and virtual robotics have not yet been incorporated into DT education.

3 Actively Engaging Teachers with Educational Technology Innovation Amongst the key stakeholders of Exten(DT)2 are practitioners, that is secondary school teachers interested in developing and testing new approaches to teaching and learning using emerging digital technologies. The project team (researchers, academics, developers) aims to collaborate with teachers in two ways: (a) in a co-creation capacity, aiming to co-develop and test innovative teaching approaches, and (b) in a co-production, capacity exploring ways practitioners could be actively involved in the research process and the production of scientific knowledge. Co-creation refers to engaging participants (here teachers) in a process of defining and finding a solution to a problem [21], that is identify a topic they would like to examine and address with students using design thinking and emerging technologies. Teachers produce a set of educational materials for exploring, understanding, and finding workable solutions to a problem relevant to students’ needs and interests or of a broader scientific significance, such as a societal challenge (e.g. climate emergency, decline of biodiversity, misinformation and inter alia). Co-creating educational innovation with teachers addresses the need to develop design thinking approaches that are compatible with formal education and are grounded in student, teacher, and school needs [23] - teachers are defining the process of innovation drawing from their own expertise and experience. A co-creation approach can be a valuable experience for teachers and a means through which they can engage with leadership practices in an informal (or formal) capacity by showcasing agency to lead change and create conditions for improving learning (e.g., [10]). Co-production refers to secondary teachers being actively involved in the research process and the production of scientific knowledge, alongside the project team. This

Combining Design Thinking with Emerging Technologies in K-12 Education

19

encompasses co-defining how evaluation data should be collected to capture the impact of innovation on student learning, supporting the process of data collection in the classroom by allocating e.g. a survey and contributing to data analysis and interpretation also by getting engaged in learning analytics. These processes may result in joined dissemination outcomes such as co-authoring publications, co-presenting and co-disseminating project outcomes [35]. A co-produced approach is likely to be time-demanding for teachers [5] and thus a few of them may express interest in getting involved. Yet, when implemented well, co-produced research can empower and give voice to those with “lived experiences” – in our project, teachers who are experts in how teaching and learning takes place within a specific learning context. Research outcomes will have immediate impact on intended stakeholders as those have been involved in a proposed innovation [36]. Special attention should be given to issues of power and ethics to ensure that equitable relationships are developed between practitioners and researchers in terms of responsibilities, efforts, and benefits [34]. Overall, our approach to working with teachers is informed by participatory principles of research [30]. We seek to understand who our teachers are (what are their needs, requirements, skills, expectations, technology expertise etc.?) and what challenges and needs their schools and students face, co-decide the topics we are going to examine using proposed design thinking, enable flexibility and support through professional development activities, identify how teachers would like to be involved in research and dissemination processes and provide compensation where applicable. These participatory principles informed the organization and delivery of online and face-to-face workshops during which teachers receive training about design thinking, are given time to experiment and raise questions about emerging technologies, share their experiences of working in specific schools in regards to student needs, technology use etc., identify (and justify) a topic to be the focus of a design thinking project and come up with technology-based activities to support each phase of design thinking. This process was supported by the Activity Plan template, a tool developed to structure interactions and communication during the workshop implementation.

4 Scaffolding Teachers’ Engagement with DT and Technologies: The Activity Plan Template To support teachers in designing and organizing DT activities with technologies, we developed a strategic document that we called “Design Thinking Activity Plan Template”. It is a generic but well-structured template that identifies the critical elements, structure, and flow of a DT activity with emerging technologies that could be implemented in a school context. Unlike a traditional learning plan, the DT Activity Plan Template addresses the teacher’s personal pedagogy, beliefs, knowledge, reflections, and practice throughout its structure, serving as a tool for expressing and communicating personal pedagogical agenda. Previous research work in the field of education has shown that using the Activity Plan Template approach can support teachers to design and implement learning activities using new technologies [38]. Additionally, it can facilitate conversations between researchers and teachers for activity design, enhancing co-creation processes [18]. It

20

M. Milrad et al.

can be a valuable tool for researchers to evaluate empirical interventions as it provides a thorough document not only for the activity setting, but also for the teacher’s rationale behind the integration of technology with pedagogy for each activity. In the Exten (D.T.)2 project, we have developed a tailored version of an Activity Plan Template for use in DT enhanced with emerging technologies. It is expected to become a mediating artefact between the Exten (D.T.)2 researchers and the stakeholders interested in designing DT activities. The main pedagogical theory underlying the design of the DT Activity Plan Template is constructionism [28], where learners put concepts into use and generate powerful ideas through constructing and tinkering with digital artefacts with personal meaning. Through that view, in DT projects, students utilize technology as an expressive medium to experiment with, develop and exchange several personally meaningful artefacts. These artefacts evolve and change during the DT project and through them, children express ideas and meanings on the DT topic. In addition, the template puts emphasis on the social dimension of the co-construction process with technologies aiming to cultivate a specific learning attitude of sharing, discussing, and negotiating during the DT process [16]. The current version of the DT Activity Plan Template includes the following aspects structured under separate sections: the description of the Design Thinking project with reference to the different domains involved, the issue it concerns, and the targeted audiences; different types of learning objectives, duration of activities and necessary material; contextual information regarding space and characteristics of students; expected use of Exten (D.T.)2 Technologies as part of the whole DT process, rather than only the develop stage which is usually done in traditional DT approaches [27]; social orchestration of the activity (group or individual work, formation of groups, etc.); a description of the teaching and learning procedures structured in the different phases of DT methodology; expected student constructions; means of student evaluation and assessment. During the Exten (D.T.)2 project, the DT Activity Plan Template is expected to have multiple roles, rather than only that of a lesson plan. It is expected to be used as a) a tool for organizing and implementing a DT activity in the classroom or online with emerging technologies, b) a tool for designing and reflecting on activities as part of teachers’ professional development, c) a tool for evaluation of the learning and teaching practices designed for the interventions, as it provides the means to keep track of what has happened in a classroom or online, d) a tool to present the school activities to a wider audience in a structured way, as it provides metadata for different kinds of DT activities with students (e.g. age, technology used, final DT product, topic of DT project, related subjects) and enable peer learning amongst educators. The evaluation of project activities during the first year is expected to provide feedback on the efficiency of the current version of the DT Activity Plan Template that will inform its iteration in subsequent years.

5 The Digital Tools and the Architecture of the Exten (D.T.)2 Platform The Exten (D.T.)2 platform integrates a set of game-based educational tools such as ChoiCo, SorBET, and MaLT2 that have been previously developed. In addition, nQuire application allows learners to design a project or study for the last stage of DT learning

Combining Design Thinking with Emerging Technologies in K-12 Education

21

activities. All game-based educational tools provide two modes: the playing mode, where the players can play the default game, or a game designed by the teacher, or a game designed by the players; and the designing mode, where players can design their own games. The different games can be played collaboratively in groups of several players using one computer or can be played individually. The different game-based education tools are described in detail below. ChoiCo (Choices with Consequences) is an open-source, online authoring tool that provides an opportunity to play, design and modify choice-driven simulation games related to complex real-life issues. In these games, the player navigates through different map-based areas making choices that affect the game parameters (e.g. money, health, fun etc.). In the design mode, the user can modify a game or design a new one by using 3 interconnected computational affordances: a GIS map designer, an editable database, and block-based programming [18]. MaLT2 (MachineLab Turtleworlds2) is an open-source online tool of symbolic expression in mathematical activity by means of programming for the creation and tinkering of 3D dynamic graphical models. SorBET (Sorting Based on Educational Technology) affords the play and design of Tetris-like sorting games in which the player scores by ‘pushing’ elements falling off the top of the screen to drop into the right container according to the right category. ‘Pushing’ elements can be done by picking and dragging on a screen and will be extended to also include body movements interaction. The gameplay builds on quick decision-making, pattern recognition, and abstraction of the characteristics of falling objects. nQuire (https://nquire.org.uk/) is a web-based community and citizen science platform designed and maintained by the Open University UK. This tool will be used by students and teachers to support specific stages of the design thinking processes. For example, students could use nQuire to design studies with the aim of understanding the needs of the target group for which they are designing a solution for (this is the Empathize stage of design thinking). Students can work collaboratively or individually to design a study on nQuire. The educational tools are extended by some of the emerging technologies such as Augmented Reality (AR), 3D printing/ scanning, and virtual robotics (vRobotics). For example, the SorBET tool will be extended to support AR game playing and designing mode in the form of body interaction with the falling objects and voice control of the game settings (e.g. speed). The extension will enable students to engage with the game physically, in larger or smaller visual settings, and will support multiple people to simultaneously engage with the same game, supporting embodied collaborative learning experiences. In addition to these educational tools, the Exten (D.T.)2 platform contains the Authorable Learning Analytics (ALA) component and the Authorable Visualization Dashboard (AVD) component to leverage the digital implementation, monitoring, and assessment of the different learning and activities DT projects that will be implemented in the schools. The ALA component gathers data on student activity generated from the educational tools (described above). The component integrates high-level authoring tools, such as drag-and-drop UI tools, that enable different types of users (teachers, researchers) to

22

M. Milrad et al.

author: (a) which data to be captured for a learning activity and (b) when to provide feedback to students and what feedback for each activity, aiming to enhance students’ engagement with the Design Thinking stages. The AVD component will effectively analyze and display the collected data from the educational tools (described above). The component support high-level authoring tools for different types of users (teachers, researchers, and students). The users can create their own customizable visualization dashboards by drag-and-dropping widgets. Each widget has a configuration where the user can specify the widget’s style, colors, background, font size, and chart type (e.g., bar chart, pit chart, etc.)). In addition, some of the widgets will support real-time visualization of students’ performance in DT activities for teachers. The visualization for teachers and students is more explanatory, and for researchers is more exploratory. The high-level architecture of the Exten (D.T.)2 platform is shown in Fig. 1 and it consists of three main layers:

Fig. 1. High-level Exten (D.T.)2 platform architecture overview

Combining Design Thinking with Emerging Technologies in K-12 Education

23

• The Presentation layer is the web-based user interface (UI) for user authorization and accessing the game-based education tools, and other Exten (D.T.)2 services (ALA, AVD). • The Educational Tools Layer is the layer where the education tools are deployed. This layer provides seamless integration of another new educational tool to be added to the Exten (D.T.)2 platform. The nQuire platform will be deployed outside of the Exten (D.T.)2 platform but accessible only through the authentication in Exten (D.T.)2 platform. • The API Gateway is used to enable the interoperating software components in the system to combine with each other following a Software Oriented Architecture (SOA). • The Server component contains the database service with PostgreSQL database, other third-party services, and a business logic layer. • The Business logic layer implements the “business” logic of the system. That is any process or decision not related to data operations with the back-end DBMS. In the next section, we present the project evaluation plan, its core underlying ideas and its corresponding three cycles.

6 Evaluating the Use of Emerging Technologies and Design Thinking in Exten (D.T.)2 Exten (D.T.)2 evaluation aims to develop a critical understanding of the potential, opportunities, barriers, accessibility issues and risks of using ET in DT learning activities. It uses a participatory design-based research approach (DBR) [1], comprising three cycles. The first is exploratory. Given the dearth of knowledge at the outset of the Exten (D.T.)2 project and the need to develop an in-depth and contextualised understanding of the project technologies in action as part of DT learning activities, exploratory case studies are used to find out, at its simplest level, ‘what happens’. This same approach is taken during the co-creation of such learning activities and the early development of teacher professional development activities. Qualitative data, primarily from observations, reflections, and interviews, provide an opportunity to explore in-depth the interactions between participants and technology as well as with each other, as students engage in co-creation, teaching and learning activities. It provides an opportunity to consider not just what, but also how students learn, including subject specific knowledge, Design Thinking attitudes and 21st Century skills. To demonstrate the efficacy of DT and ET to develop skills and attitudes is a significant challenge in any intervention-based research study that does not extend to the scale of a randomized-control trial. Given that the context of each school intervention will differ significantly, from mode of delivery to subject, from school culture to the length and timing of such intervention, etc., there are many potentially confounding variables at work. Thus, at this stage the research team remains open to exploring several data collection approaches. One approach suggested in the literature is to avoid standardized testing and to focus instead on self-reported confidence [2]. In cycle 1 we pilot the use of an established instrument for 21st Century skills across contexts as well as a purposedesigned instrument in short surveys, for comparison with more qualitative approaches

24

M. Milrad et al.

including open survey questions, observations, interviews, and evidence from learning artefacts. Using constant comparative analysis, critical incidents and thematic analysis, findings of the Cycle 1 data analysis will be used to inform the development of tools, materials, pedagogy, and procedures across the ExtenDT2 project. Cycle 2 remains primarily qualitative, using an instrumental case study approach to explore the impact of project developments. However, it also provides an opportunity to pilot large-scale quantitative instruments, explore the input-output of the LA system and co-design evaluation tools with and for teachers. Again, findings from the analysis will be used to inform the development of tools and resources for the third and final cycle. Finally, Cycle 3 focuses on quantitative data to evaluate the efficacy of the final tools and pedagogic approaches developed through the first two years of the project. Drawing on evidence from largescale surveys and teacher’s toolkits across six countries, we will evaluate the impact of using DT with ET on teachers’ self-efficacy and students’ knowledge, attitudes, and skills. Throughout, one of the challenges within this research is that of ethics. While the basic research design which draws on established Social Science research methods does not raise any unusual ethical issues, the specific context of the research does the use and development of LA/AI within classrooms. Specifically, the intersection of disciplinary norms in education and computer science research ethics, together with the implementation of developing ET in real-world settings and pedagogic requirements and expectations of stakeholders, all within different cultural settings, results in a tangled set of issues. Emergent issues in Cycle 1 include tensions between stakeholder groups on the use of LA/AI in education; parental consent and student assent to participate in research activities; and balancing the pedagogic requirements of the teacher with the rights of the child/parent when exploring teachers’ decision making. While approaches such as value-sensitive design are used from the beginning, as the technology and its use develop over the lifetime of the project, the ethical entanglements will need to be continuously reviewed. Given the collaborative nature of the research, participatory ethics provides an opportunity for dialogue with stakeholders and reflective practice to guide ethical decision-making. Although the needs of participants may need to be balanced and may even generate tensions with institutional ethical review committees.

7 Conclusions and Final Remarks In this paper we have presented and discussed the theoretical foundations and initial development stages of the the Exten (D.T.)2 project. Exten (D.T.)2 is using emerging technologies to enhance the pedagogical value, sustainable digitization, and potential for wide deployment of DT in educational contexts with a focus on secondary education. We have described the main components of the proposed educational transformation as developed during the first year of Exten (D.T.)2 including the theoretical foundations of the approach, the co-creation processes for meaningful engagement of teachers with educational innovation. The educational technology tools (design, extensions, architecture) that are used in the project have been presented including an introduction to the authorable earning analytics and the authorable visualization dashboard. These components will help to leverage the digital implementation, monitoring, and assessment of DT

Combining Design Thinking with Emerging Technologies in K-12 Education

25

projects by teachers in schools and 21st century skills. Leverage the digital implementation, monitoring, and assessment of the different learning and activities DT projects that will be implemented in the schools. We finalized the paper by describing our evaluation approach for capturing impact on improving students’ skills and competencies. At the time of writing, we have started with the implementation of the first pilot projects to be deployed in secondary schools at six different European countries with more than 300 students. The next steps in the project comprise the analysis of the data collected during these pilots including also different aspects of the co-design of the pedagogical activities. The findings of these analysis will guide us in the coming phases of the project related to both pedagogical and technical aspects of developing and implementing the project at a bigger scale. Acknowledgement. The Extending Design Thinking with Emerging Digital Technologies EU project Grant agreement 101060231.

References 1. Barab, S., Squire, K.: Design-based research: putting a stake in the ground. J. Learn. Sci. 13(1), 1–14 (2004) 2. Bray, A., Byrne, P., O’Kelly, M.: A short instrument for measuring students’ confidence with ‘key skills’(sicks): development, validation and initial results. Thinking Skills and Creativity 37, 100700 (2020) 3. Brown, T.: Design thinking, harvard business review (2008) 4. Buhl, A., et al.: Design thinking for sustainability: why and how design thinking can foster sustainability-oriented innovation development. J. Clean. Prod. 231, 1248–1257 (2019). https://doi.org/10.1016/j.jclepro.2019.05.259 5. Campbell, H., Vanderhoven, D.: Knowledge that matters: realising the potential of coproduction (2016) 6. Chiu, J.L., et al.: WISEngineering: supporting precollege engineering design and mathematical understanding. Comput. Educ. 67, 142–155 (2013) 7. Christensen, K.S., Hjorth, M., Iversen, O.S., Smith, R.C.: Understanding design literacy in middle-school education: assessing students’ stances towards inquiry. Int. J. Technol. Des. Educ. 29, 633–654 (2019) 8. Cutumisu, M., Schwartz, D.L., Lou, N.M.: The relation between academic achievement and the spontaneous use of design-thinking strategies. Comput. Educ. 149, 103806 (2020) 9. Dragiˇcevi´c, N., Vladova, G., Ullrich, A.: Design thinking capabilities in the digital world: a bibliometric analysis of emerging trends. Front. Educ. 7, 1012478 (2023). https://doi.org/10. 3389/feduc.2022.1012478 10. Fairman, J.C., Mackenzie, S.V.: Spheres of teacher leadership action for learning. Prof. Dev. Educ. 38(2), 229–246 (2012) 11. Gennari, R., Melonio, A., Rizvi, M.: From children’s ideas to prototypes for the internet of things: a case study of cross-generational end-user design. Behav. Inf. Technol. 41(15), 3281–3300 (2022) 12. Goldman, S., Carroll, M., Royalty, A.: Destination, imagination & the fires within: Design thinking in a middle school classroom. In: Proceedings of the Seventh ACM Conference on Creativity and Cognition, pp. 371–372 (2009) 13. Gross, K., Gross, S.: Transformation: constructivism, design thinking, and elementary steam. Art Educ. 69(6), 36–43 (2016)

26

M. Milrad et al.

14. Hatzigianni, M., Stevenson, M., Falloon, G., Bower, M., Forbes, A.: Young children’s design thinking skills in makerspaces. Int. J. Child-Comput. Interact. 27, 100216 (2021) 15. Huang, C.Y., Wang, J.C.: Effectiveness of a three-dimensional-printing curriculum: developing and evaluating an elementary school design-oriented model course. Comput. Educ. 187, 104553 (2022) 16. Kafai, Y.B., Burke, Q.: Computational participation: teaching kids to create and connect through code. In: Rich, P.J., Hodges, C.B. (eds.) Emerging research, practice, and policy on computational thinking. ECTII, pp. 393–405. Springer, Cham (2017). https://doi.org/10. 1007/978-3-319-52691-1_24 17. Koh, J.H.L., Chai, C.S., Benjamin, W., Hong, H.Y.: Technological pedagogical content knowledge (tpack) and design thinking: a framework to support ICT lesson design for 21st century learning. Asia Pac. Educ. Res. 24, 535–543 (2015) 18. Kynigos, C., Grizioti, M., Gkreka, C.: Studying real-world societal problems in a stem context through robotics. arXiv preprint arXiv:1806.03245 (2018) 19. Landwehr Sydow, S., Åkerfeldt, A., Falk, P.: Becoming a maker pedagogue: exploring practices of making and developing a maker mindset for preschools. In: FabLearn Europe/MakeEd 2021-An International Conference on Computing, Design and Making in Education, pp. 1–10 (2021) 20. Laru, J., et al.: Designing seamless learning activities for school visitors in the context of fab lab oulu. In: Looi, C.-K., Wong, L.-H., Glahn, C., Cai, S. (eds.) seamless learning. LNET, pp. 153–169. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3071-1_8 21. Lee, J.J., Jaatinen, M., Salmi, A., Mattelmäki, T., Smeds, R., Holopainen, M.: Design choices framework for co-creation projects. International J. Des. 12(2) (2018) 22. Leinonen, T., Virnes, M., Hietala, I., Brinck, J.: 3d printing in the wild: adopting digital fabrication in elementary school education. Int. J. Art Des. Educ. 39(3), 600–615 (2020) 23. Li, T., Zhan, Z.: A systematic review on design thinking integrated learning in k-12 education. Appl. Sci. 12(16), 8077 (2022) 24. Lin, L., Shadiev, R., Hwang, W.Y., Shen, S.: From knowledge and skills to digital works: an application of design thinking in the information technology course. Thinking Skills and Creativity 36, 100646 (2020) 25. Matthews, J., Wrigley, C.: Design and design thinking in business and management higher education. J. Learn. Des. 10(1), 41–54 (2017) 26. Mentzer, N., Huffman, T., Thayer, H.: High school student modeling in the engineering design process. Int. J. Technol. Des. Educ. 24, 293–316 (2014) 27. Panke, S.: Design thinking in education: perspectives, opportunities and challenges. Open Educ. Stud. 1(1), 281–306 (2019) 28. Papert, S.A.: Mindstorms: Children, computers, and powerful ideas. Basic books (2020) 29. Pitkänen, K., Iwata, M., Laru, J.: Exploring technology-oriented fab lab facilitators’ role as educators in k-12 education: focus on scaffolding novice students’ learning in digital fabrication activities. Int. J. Child-Comput. Interact. 26, 100207 (2020) 30. Ross, L., et al.: Key practices for community engagement in research on mental health or substance abuse. Centre for Addiction and Mental Health (2015) 31. Schlegel, R.J., Chu, S.L., Chen, K., Deuermeyer, E., Christy, A.G., Quek, F.: Making in the classroom: longitudinal evidence of increases in self-efficacy and stem possible selves over time. Comput. Educ. 142, 103637 (2019) 32. Simeon, M.I., Samsudin, M.A., Yakob, N.: Effect of design thinking approach on students’ achievement in some selected physics concepts in the context of stem learning. Int. J. Technol. Des. Educ. 32(1), 185–212 (2020). https://doi.org/10.1007/s10798-020-09601-1 33. Tsai, M.J., Wang, C.Y.: Assessing young students’ design thinking disposition and its relationship with computer programming self-efficacy. J. Educ. Comput. Res. 59(3), 410–428 (2021)

Combining Design Thinking with Emerging Technologies in K-12 Education

27

34. UKRI: Co-production in research. https://www.ukri.org/about-us/policies-standards-anddata/good-research-resource-hub/research-co-production/ 35. Unertl, K.M., et al.: Integrating community-based participatory research and informatics approaches to improve the engagement and health of underserved populations. J. Am. Med. Inform. Assoc. 23(1), 60–73 (2016) 36. Vaughn, L.M., Jacquez, F.: Participatory research methods–choice points in the research process. J. Participatory Res. Meth. 1(1) (2020) 37. Veldhuis, A., et al.: The connected qualities of design thinking and maker education practices in early education: a narrative review. In: Fablearn Europe/MAKEED 2021-an International Conference on Computing, Design and Making in Education, pp. 1–10 (2021) 38. Yiannoutsou, N., Nikitopoulou, S., Kynigos, C., Gueorguiev, I., Fernandez, J.A.: Activity plan template: a mediating tool for supporting learning design with robotics. In: Merdan, M., Lepuschitz, W., Koppensteiner, G., Balogh, R. (eds.) Robotics in Education. AISC, vol. 457, pp. 3–13. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-42975-5_1

Leveraging Cognitive and Social Engagement in Blended Learning Through an AI-Augmented Pedagogical Framework Andromachi Filippidi(B) , Christophoros Karachristos, Konstantinos Lavidas , Vassilis Komis , and Nikos Karacapilidis University of Patras, 26504 Rio Patras, Greece [email protected]

Abstract. This paper proposes a new methodological approach for assessing and leveraging student engagement in a tertiary education course that builds on recent artificial intelligence advancements and the TPACK (Technological Pedagogical Content Knowledge) model. We describe how, through the design of a course that is based on Moodle and the utilization of different pedagogical and conceptual tools, we collect and process data aiming to enhance the course with techniques that may provide students and tutors with valuable insights and recommendations towards achieving better performance. Keywords: student engagement · emerging technologies · TPACK · Moodle

1 Introduction An important aspect of online learning is the use of a Learning Management System (LMS) to track and record diverse students’ activities. In this context, algorithms based on recent artificial intelligence (AI) advancements can analyze the large amounts of data collected, aiming to detect meaningful patterns and accordingly make predictions about students’ engagement levels. For instance, natural language processing techniques can be used to analyze students’ written responses to discussion prompts and assignments, and identify language patterns that are associated with higher levels of engagement; machine learning models can be used to create personalized feedback and recommendations to students, encouraging them to engage more actively in specific learning activities. Overall, the use of such techniques in assessing student engagement has the potential to provide valuable insights into the complex dynamics of student learning and inform the development of effective teaching strategies and interventions. This paper proposes a methodological approach for assessing and leveraging student engagement in a tertiary education course that builds on the above-mentioned advancements. Our aim is to describe how, through the design of a course that is based on Moodle and the utilization of different pedagogical and conceptual tools, we can collect and process data to enhance the course with AI-boosted techniques that may in turn provide students and tutors with valuable insights and recommendations towards achieving better performance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 28–38, 2023. https://doi.org/10.1007/978-3-031-42134-1_3

Leveraging Cognitive and Social Engagement in Blended Learning

29

2 Student Engagement In recent years, there has been a growing interest among researchers, practitioners, and policy makers in defining and measuring “student engagement”. The works of Astin on involvement theory [1], of Fredricks and her colleagues on the identification of the three dimensions of student engagement (behavioral, emotional, cognitive) [7], as well as those of Kahu and Nelson on sociocultural theories of engagement [11, 12] have significantly contributed to shaping our comprehension of the complex phenomenon of student engagement. Student engagement refers to the level of involvement, motivation, and participation that students demonstrate in their learning process [9]. Engaged students are active participants in their learning and they are more likely to be motivated, focused, and committed to their academic goals. According to [7], there are several dimensions to student engagement. Behavioral engagement refers to the observable actions and behaviors of students, such as participation in discussions, completion of assignments, and attendance. Another dimension is emotional engagement which refers to the emotional connection that students have to their learning and the learning environment. Cognitive engagement is a third dimension which refers to the mental effort and active processing that students put into their learning. We can also bring up more dimensions like the social engagement which refers to the relationships and social interactions that students have with their peers and teachers. The online and blended learning environments and the use of learning platforms like Moodle have added some extra characteristics in the different factors of student engagement [14]. Specifically, the behavioral engagement in an LMS includes factors such as logging into the system regularly, completing assignments on time, and participating in online discussions. The cognitive engagement dimension in an LMS includes factors such as reading and comprehending online materials, solving problems, and asking questions. The emotional engagement in a learning platform includes factors such as feeling motivated and interested in the course content, feeling supported by the instructor and peers, and experiencing a sense of community. The social engagement in an LMS includes factors such as participating in online discussions, collaborating on group projects, and seeking help from instructors or peers. By understanding and measuring these dimensions of student engagement within an LMS-supported learning environment, instructors can predict learners progress and design online courses that support and promote active, meaningful and connected learning experiences for all students.

3 The Pedagogical Framework of augMENTOR The augMENTOR project introduces an innovative Pedagogical Framework that has at its center the concept of technology-augmented educational activities that concern various types of stakeholders including students, teachers, policy makers, and educational leaders. The project advocates that a consistent organization of teaching process and learning cannot be achieved by solely focusing on the teacher, knowledge, or learner. Instead, the focus should be on the overall activity of the class or group of students or actors involved and on the data the participants produce. Our pedagogical framework is based on Activity Theory, according to which the proposed solution will involve the

30

A. Filippidi et al.

interactions between actors as defined in the teaching triangle and the physical and/or symbolic tools employed. The proposed pedagogical framework extends the Technology Enhanced Learning approach by integrating emerging technologies that significantly affect the involvement of stakeholders in the educational process by playing a dual role. Firstly, they are utilized as educational and cognitive tools embedded in educational practices. Secondly, AI and Educational Data Mining are used to analyze and adapt the teaching and learning process in real-time. This is achieved by providing feedback to teachers and students, creating suitable profiles, and offering materials and assistance accordingly. In the context of individual learning and teaching activities, emerging technologies have been investigated so far in a fragmented way; they are utilized either as teaching tools for certain parts of the curriculum, or for analyzing and providing feedback on the learning activity. In the literature, there is no framework that examines emerging technologies from a holistic point of view. As a result, it is crucial to generate new knowledge that will arise within the augMENTOR context. This new knowledge pertains to two primary levels. The Pedagogical Framework, at the macro-level, aims to expand the well-established TPACK (Technological Pedagogical Content Knowledge) model developed by Koehler and Mishra [13]. This model endeavors to identify the type of knowledge that teachers require for integrating technology in their teaching, while acknowledging the intricate and multifaceted nature of teacher knowledge [16]. TPACK (http://tpack.org) is perhaps the most widely used framework that studies technology integration in the classroom from a teacher’s perspective. However, in the current literature, technology is typically categorized as either conventional (e.g., projectors, etc.) or well-established modern technology (e.g., e-learning platforms, productivity software, etc.), or digital technologies related to particular subjects (simulations, mathematical software, etc.) or educational levels (early childhood, primary, etc.). Therefore, the main objective of this project is to include emerging technologies as a component of the framework and consequently expand TPACK to accommodate these emerging technologies. The creation of this model is a contribution to both basic and applied research, as it is grounded in research-based evidence (the augMENTOR project’s pilots) and facilitate the planning of training programs for pre-service and in-service teachers. At the micro-level, the proposed pedagogical framework is concentrated on Pedagogical Design in terms of technology-augmented educational scenarios. The difference between a technology-augmented educational scenario and a technology-rich (or technology-enhanced) educational scenario is that the first provides significantly more pedagogical and technological affordances to the students and teachers involved. The current literature provides us with well-grounded knowledge on how to design technologyrich educational scenarios, which refer to a set of teaching/learning activities carried out by teachers and students, employing appropriate teaching strategies to achieve specific learning outcomes while utilizing a suitable computing environment (educational software and/or hardware). These educational scenarios aim to teach and learn one or more key concepts of a subject through the existing curriculum in the school. To ensure a suitable and validated process for collecting data in an educational scenario (as shown in Fig. 1), a series of steps must be followed:

Leveraging Cognitive and Social Engagement in Blended Learning

31

• Define the learning objective of the educational scenario, including the title, class (e.g. tertiary education), knowledge areas involved, cognitive prerequisites, and associated cognitive, social, and other 21st century skills. • Define students’ ideas, representations, and possible difficulties in thinking about the subject matter, and associate them with cognitive, social, and other 21st century skills. • Specify the teaching objectives of the educational scenario in terms of the subject matter, use of Information and Communication Technologies (ICT), and learning process, and associate them with cognitive, social, and other 21st century skills. • Design the teaching material and required logistical infrastructure for the educational scenario, in alignment with the previous steps. • Implement appropriate classroom activities for the educational scenario, including teaching approaches and strategies, use of ICT in the learning process, and worksheets, based on learning profiles and students’ actual needs. • Evaluate both the student and scenario outcomes and make possible extensions to the scenario based on the actual needs of the participants.

Fig. 1. Micro-level: Technology – augmented Educational Scenarios (learning design)

4 Innovative Training Program for Pre-service Teachers To apply the augMENTOR solution, we redesign an innovative training course that aims to enhance the digital competence and confidence of pre-service candidate teachers. The course is titled “Information and Communication Technologies in Education” and is mandatory for pre-service teachers at the Department of Educational Sciences and Early Childhood Education, University of Patras, Greece. The aim of the course is to aid students acquire basic knowledge about the main approaches regarding the introduction and integration of ICT in the educational process, get acquainted with the main models of the introduction of these technologies into education, and develop basic competencies related to pedagogy and the educational use of basic computing and online applications and environments.

32

A. Filippidi et al.

At the beginning of the redesigning process, we conducted a requirements’ assessment to identify the different “new” needs of the target audience in terms of data to be recorded and analyzed in order to track students’ performance and offer each stakeholder (student, teacher etc.) with the appropriate insights. The second step was to revise the course syllabus, outline, and learning objectives to ensure that they are up-to-date and aligned with the identified learning needs and preferences. New topics were inserted for the digital upskilling of the students. The third step involved the development of learning materials and resources, including multimedia presentations, interactive simulations, and discussion forums, that are engaging, facilitate active learning and are more technology augmented in order to provide insights about the performance of the students. The fourth step was the selection of appropriate instructional strategies and methods, such as problem-based learning, case studies, and collaborative learning, that are suitable for online delivery. The course design was primarily influenced by the theoretical aspects of social constructivism. Additionally, the principles of blended learning were incorporated into the course design, based on those proposed in [10]. The course follows a problem-solving learning approach, as proposed in [4]. This step introduces the use of the pedagogical framework mentioned above. The next step is the integration of machine learning analytics tools and techniques into the course design to support the assessment of learning outcomes, track student progress, and provide personalized feedback and recommendations. This involves the selection of appropriate tools and techniques, such as learning analytics dashboards, predictive models, and natural language processing tools, and the integration of these tools into the learning management system. The final step involves the implementation and evaluation of the redesigned course, which includes testing the course materials, assessing student learning outcomes, and soliciting feedback from both students and instructors to identify areas for improvement. The evaluation results will be used to refine the course design and make necessary revisions to ensure the course is effective, engaging, and meets the learning needs and preferences of the target audience. 4.1 Structure and Delivery of the Course The course combines online and face-to-face lectures and includes a 13-week series of two-hour laboratory sessions. Each session focuses on a specific topic related to the course’s objectives, and a range of blended learning principles is used. The course is provided through a learning management system which is Moodle and Fig. 2 provides a prototype example of the course structure and interface. It is planned on a weekly basis, with each section focusing on a specific goal aligned with the course requirements and 21st century skills such as creativity, critical thinking, problem-solving, and collaboration. Every week, all students are required to attend a laboratory session at the university facilities and submit a personal report (assignment) at their own pace from home. The course material is organized according to each subject and remains available to students until the end of the semester. During the labs, students are engaged in problemsolving activities that require them to acquire the necessary skills and concepts to solve presented problems. Each week, relevant course materials, short activities, and lectures are made available to the students. A variety of resources, such as online materials in different formats (web pages, slides, animations, simulations, interactive hypermedia,

Leveraging Cognitive and Social Engagement in Blended Learning

33

Fig. 2. The course: Information and Communication Technologies in Education

encyclopedias, glossary exercises, and pdf documents), are provided to encourage students to tackle the given activities. In-person guidance is offered by teaching assistants during the labs, and the Moodle forum is used to answer students’ questions. The learning goals are introduced in detail at the beginning of each lab session to ensure students stay on track. The assignments in the course are focused on designing learning activities for early childhood education, using various educational software. The final deliverables comprised written essays, software files implementing the activity (such as concept maps), PowerPoint presentations, and completed evaluation rubrics. The final performance of the students is determined based on the evaluation of their assignments and their performance in the final exam, as outlined in [6]. The development of a course that incorporates all those technological and pedagogical features that are being developed and are necessary to apply the augMENTOR approach is imperative. Through a design-based research (DBR) methodological approach [15], we plan to identify the key elements of an innovative Pedagogical Framework that integrates pedagogical design and emerging technologies. DBR will be used in the evaluation of the proposed approach, with the aim to check if the modifications and adjustments made to the course are going well or have to be updated because they are not working as planned. It is noted here that DBR is a research methodology which aims to improve educational practices through systematic, flexible, and iterative review, analysis, design, development and implementation, based upon collaboration among researchers and practitioners in real-world settings.

34

A. Filippidi et al.

5 The augMENTOR Approach In our approach (Fig. 3), we attempt to gather, interpret, and analyze data to support students by giving them specific insights and recommendations and evaluate the aforementioned educational scenario. Based on Activity Theory Framework (ATF), each student must complete an activity per week so to achieve the object of each week. This goal (objective of the activity) concerns the course curriculum and specifically ICT in Education (to evaluate and intergrade concept maps in learning activities for kindergarten students) and 21st century skills (creativity and problem solving). Finally, the goal of each activity is being measured according to students’ performance. According to ATF analysis each activity consists of several tasks. The tasks which are available via Moodle can be an assignment, a course material for reading, a quiz, a forum, a discussion or every other activity or resource of Moodle. The tasks are composed by a number of actions that each participant has to do in order to complete them. These actions are different among the tasks and each action has states (values). The number of actions of each task are recorded to a database table which is called Moodle Data/Logs. A number of indicators is connected to each task. The indicators are of different types like social and cognitive indicators and creativity indicators. So, for a task that contains a quiz there will be a quiz cognitive indicator, for a forum task there will be a forum cognitive indicator and a forum social indicator, etc. Each indicator will be calculated according to a certain process. Based on these calculation processes, the augMENTOR solution will aid tutors in supporting students to accomplish the goals of each week.

Fig. 3. The augMENTOR approach.

Leveraging Cognitive and Social Engagement in Blended Learning

35

5.1 Data Resources and Engagement Indicators Both quantitative and qualitative methods will be used to collect the data produced by students. Quantitative data will be collected through log files (e.g. time devoted on selected tasks etc.) maintained by Moodle (e.g., quiz data, discussion data, assignment data), pre/post surveys (questionnaires to assess digital literacy and Technology, Pedagogy and Content Knowledge, and collect demographic data), digital artefacts created during training, and self-reflection activities (e.g. concept maps). Qualitative data will be collected via interviews with teachers/tutors and certain students who will be selected randomly at the beginning, mid, and end of their participation in a two-hour compulsory workshop session. Also, we plan to use rubrics for self-assessment, observations from the workshops and learning activities in portfolios/wiki/forum/written assignments, which will give the indicators for students’ performance. Educational resources, such as online eBooks, educational items in digital libraries, dedicated digitized materials, educational scenarios, and dedicated curricula, are also considered. The data will be captured in order to be analyzed so as to evaluate users’ actions and practices, and all actors’ attitudes and performance will be examined in relation to these actions and practices. This data will be presented in a visual format, such as graphs, to aid students in better comprehending their progress and assist tutors in providing appropriate guidance for each individual. Using this information, pre-service teachers will be provided with customized advice, suggestions for various learning paths, and additional complementary material to enhance their skills. The problem we face is that there does not exist a valid, commonly accepted, and broadly used framework for measuring the level of engagement of students in environments like Moodle. One common approach for student’s engagement in online environments is to measure different dimensions of engagement by using performance indicators, which are specific metrics that are used to evaluate the level of cognitive, social, and other types of engagement of students in a course or learning environment. In our case cognitive depth indicators measure the level of student involvement in activities that promote critical thinking and problem-solving. They assess the type of activity provided to the student and the degree to which the student demonstrates cognitive engagement in that activity. Social breadth indicators measure the extent to which students are given opportunities to interact with one another, build relationships, and engage in collaborative learning. They evaluate the range of communication channels available to students, such as discussion forums, chat rooms, and group projects. Creativity indicators measure the degree of creative engagement a learner can experience as a socio-cognitive process. Creativity indicators measure a student’s ability to think, imagine, and produce original and innovative ideas or solutions to problems. These indicators can include various aspects such as fluency, flexibility, originality, elaboration, risk-taking, imagination and sensitivity to problems. By developing these indicators in our solution, instructors and researchers can gain insights into the level of student engagement and identify areas where improvements can be made to promote deeper learning and a more fulfilling educational experience. The Moodle platform offers an integrated Learning Analytics tool suitable for building self-learning predictive machine learning models which is supported by a number of predefined indicators for social and cognitive engagement. These indicators are based

36

A. Filippidi et al.

on the theoretical framework of the Community of Inquiry [8] which defines cognitive presence and social presence as the two primary components of student engagement. However, the models constructed using these pre-established core indicators within the Moodle Analytics API exhibit a level of goodness and predictability that falls below the commonly acceptable standard [5]. This leads to machine learning models with low predictive capabilities, which may cause several false alarms about students’ performance or students’ risk of failure. The reason behind that poor performance of these built-in machine learning models is that the calculation of all core indicators is the same for all supported components (activities and resources of Moodle) and depend on the level of interaction of student with the component (viewed, submitted etc.) and not on the quality or correctness of the answer given. If we consider a quiz-type indicator, it implies that upon completing all questions, the indicator will receive the maximum value, regardless of whether the answers were incorrect, or the questions were not viewed. This approach to calculation appears illogical for quiz-type indicators, as the indicator value should be closely linked to the success function. 5.2 Redesigning and Building New Indicators The exploitation of indicators for our machine learning solution involves the evaluation of the performance of existing indicators that Moodle offers, and other researchers propose, the identification of their limitations, and the development of new indicators that better capture the relevant data features. The process of redesigning and building new indicators includes: • Definition of the problem: we will start by exactly defining the problem we are trying to solve and what kind of data we have. Understanding the nature of the data is important as it will inform the choice of indicators. • Evaluation of the existing indicators: we will review the performance of existing indicators and identify their strengths and limitations; this will help us understand which indicators are most useful and where improvements are needed. • Identification of new indicators: we will build new indicators that better capture the relevant features of the data; these may be derived by meaningful combinations of existing indicators or be completely new. • Testing and refinement of indicators: we will evaluate the performance of the new indicators and compare them to the existing ones; we will then refine the indicators based on the results of the evaluation. • Validation of the indicators: we will validate the indicators on new data to ensure that they generalize well and can be used in practice.

6 Conclusion In the context of the augMENTOR project, a team of educators and data scientists collaborates to develop a pedagogical framework that combines TPACK and machine learning techniques to improve student engagement. This framework is specifically designed for pre-service teachers who are preparing to enter the classroom and need to be equipped with the necessary skills to effectively use technology in their teaching. The proposed

Leveraging Cognitive and Social Engagement in Blended Learning

37

framework involves several key elements, including the use of TPACK to inform the selection of appropriate technologies for teaching specific content areas, and the use of machine learning techniques to assess student engagement in online courses. By combining these two approaches, we aim to create a holistic and data-driven approach to teaching that would help pre-service teachers to effectively engage their future students. The proposed pedagogical framework aims to be effective in improving student engagement. The use of TPACK will help pre-service teachers to select appropriate technologies for teaching specific content areas, while the machine learning algorithms will provide real-time feedback on student engagement (it is noted here that a technical description of these algorithms goes beyond the scope of this paper). This feedback will allow instructors to adjust their teaching strategies and provide additional support to students who need it. Overall, the proposed pedagogical framework offers a new approach to improving student engagement in online courses. This approach could be particularly useful for pre-service teachers who are preparing to enter the classroom and need to be equipped with the necessary skills to effectively use technology in their teaching. As a final note, we argue that this work has a series of limitations, which we plan to address through diverse research initiatives in the future. Specifically, a future work direction concerns the elaboration of additional learning aspects, such as metacognitive learning and affective learning, while also considering social motivation and the associated behavioral phenomena. Another research direction will focus on the meaningful association of the design concepts discussed in this paper with broadly established theoretical constructs (e.g. the Activity Theory). Acknowledgements. This work received funding from the Horizon Europe research and innovation programme under Grant Agreement No. 101061509, project augMENTOR (Augmented Intelligence for Pedagogically Sustained Training and Education).

References 1. Astin, A.W.: Student involvement: a developmental theory for higher education. J. Coll. Stud. Dev. 40(5), 518–529 (1999) 2. Bovill, C., Bulley, C.J.: A model of active student participation in curriculum design: exploring desirability and possibility. In: Rust, C. (ed.) Improving Student Learning Through Research and Scholarship: 20 Years of ISL, pp. 176–188. Oxford Centre for Staff and Learning Development, Oxford (2011) 3. Bovill, C., Cook-Sather, A., Felten, P., Millard, L., Moore-Cherry, N.: Addressing potential challenges in co-creating learning and teaching: overcoming resistance, navigating institutional norms, and ensuring inclusivity in student-staff partnerships. High. Educ. 71(2), 195–208 (2016) 4. Duffy, T., Kirkley, J.: Introduction: theory and practice in distance education. In: LearnerCentered Theory and Practice in Distance Education: Cases from Higher Education, pp. 3–16. Lawrence Erlbaum Associates, Inc., Mahwah, NJ (2004a) 5. Fauszt, T., Bognár, L., Sándor, Á.: Increasing the prediction power of moodle machine learning models with self-defined indicators. Int. J. Emerg. Technol. Learn. (iJET) 16(24), 23–39 (2021). https://doi.org/10.3991/ijet.v16i24.23923

38

A. Filippidi et al.

6. Filippidi, A., Tselios, N., Komis, V.: Impact of moodle usage practices on students’ performance in the context of a blended learning environment. In: Proceedings of Social Applications for Lifelong Learning Conference, 4–5 November 2010, Patra, Greece (2010) 7. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004). https://doi.org/10.3102/003465 43074001059 8. Garrison, D., Akyol, Z.: The community of inquiry theoretical framework. In: Moore, M.G. (ed.) Handbook of Distance Education, pp. 122–138. Routledge (2013). https://doi.org/10. 4324/9780203803738.ch7 9. Hu, Y.L., Ching, G.S., Chao, P.C.: Taiwan student engagement model: conceptual framework and overview of psychometric properties. Int. J. Res. Stud. Educ. 1, 69–90 (2012) 10. Jimoyiannis, A., Tsiotakis, P., Roussinos, D.: Social network analysis of students’ participation and presence in a community of educational blogging. Interact. Technol. Smart Educ. 10(1), 15–30 (2013) 11. Kahu, E.: Framing student engagement in higher education. Stud. High. Educ. 38, 758–773 (2013). https://doi.org/10.1080/03075079.2011.598505 12. Kahu, E.R., Nelson, K.: Student engagement in the educational interface: understanding the mechanisms of student success. High. Educ. Res. Dev. 37(1), 58–71 (2018). https://doi.org/ 10.1080/07294360.2017.1344197 13. Koehler, M.J., Mishra, P.: What Is technological pedagogical content knowledge? Contemp. Issues Technol. Teacher Educ. 9(1), 60–70 (2009) 14. Mahmoud, A., Gamal El-Din, S., Tharwat, G.: Students engagement in e-learning environments – a comprehensive model: measurement, evaluation, and recommendations. J. Al-Azhar Univ. Eng. Sector 16, 1171–1183 (2021). https://doi.org/10.21608/auej.2021.207676 15. Wang, F., Hannafin, M.J.: Design-based research and technology-enhanced learning environments. ETR&D 53, 5–23 (2005). https://doi.org/10.1007/BF02504682

Scenario Design, Data Measurement, and Analysis Approaches in Maritime Simulator Training: A Systematic Review Ziaul Haque Munim1(B) , Helene Krabbel1 , Per Haavardtun1 , Tae-Eun Kim2 , Morten Bustgaard1 , and Haakon Thorvaldsen1 1 Faculty of Technology, Natural, and Maritime Sciences, University of South-Eastern Norway,

3184 Horten, Norway [email protected] 2 Department of Technology and Safety, University of Tromsø (UiT) - The Arctic University of Norway, Tromsø, Norway

Abstract. Developing objective assessment approach in maritime simulator training can be a highly challenging task due to the complexity of simulating realistic scenarios, capturing relevant performance indicators and establishing good assessment protocols. This study provides a synthesis of simulation scenario contexts, data collection tools, and data analysis approaches in published simulator training studies. A systematic literature review (SLR) approach was followed for identifying the relevant studies for in-depth content analysis. The findings reveal that the reviewed studies focused on full-mission simulator-based assessment using collected data from various tools including surveys, eye-tracking, ECG, video or voice recording etc. The findings hold relevance in the development of learning analytics for facilitating objective assessment in maritime simulator training. Keywords: Navigation simulator · Navigation competence · Training assessment · Learning analytics

1 Introduction Objective assessment approaches in maritime simulator training and education have been consistently advocated for implementation [1]. While rapid technological advancements in simulation modalities have diversified the maritime learning approaches and delivery methods, the assessment aspect still mainly relies on human involvement. Instructors play an essential role in evaluating the learning performance and determining the competence and readiness of the learners. While the Standards of Training, Certification and Watchkeeping for Seafarers (STCW convention and codes) provides competence requirements and outlines assessment criteria, the specific assessment methods, scoring criteria and detailed performance standards may vary depending on the instructors and training institutions.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 39–47, 2023. https://doi.org/10.1007/978-3-031-42134-1_4

40

Z. H. Munim et al.

Recently, scholars have been investigating alternative assessment approaches in maritime simulator training and seek to explore innovative ways to evaluate seafarers’ performance and competence. For example, an automatic assessment approach was explored by ˙ Juszkiewicz and Zukowska [2]. The assessment approaches or tools need to be perceived as reliable and useful by the instructors and students. Munim and Kim [3] proposed a learning analytics dashboard (LAD) outline based on underlying pattern detection from simulator data. To develop a functioning LAD, data from multiple sources can be collected and analysed. The focus of assessment, data collection, and analysis approaches have to be aligned. Hence, the following research questions are addressed in this study: 1. What are the characteristics of the simulation scenarios in terms of context, duration, and level of complexity? 2. What are the data collection tools and approaches used in published maritime simulation training studies? 3. What are the common data analysis methods and approaches in the assessment of maritime simulator training?

2 Methodology Data for this review study was collected using a systematic approach. The scientific records database Scopus was used to identify relevant articles. Scopus is one of the largest databases and widely used in literature review studies, see for example Macke and Genari [4]. The literature search was conducted using Boolean expressions to identify relevant articles on various assessment approaches of maritime simulator training in March 2023. The search was limited to titles, abstracts, and keywords, and only journal articles written in English, which reverted to 348 potential records. The Boolean expression is reported in Table 1. After the initial search, articles published prior to 2010 (86) were not considered, leaving 262 articles for manual scrutiny. The abstracts of the records were screened to identify the studies that are highly relevant to the research questions of this study. Most of the studies (141) did either not focus on Maritime Education and Training (MET) at all or were focused on general simulator set-up without involving assessment approaches. Two papers were duplicate but not recognized as such initially due to minor inconsistencies in the title; 17 focused on the role of the instructor in MET, and not on the assessment of navigational competency and 18 on the engine department; 36 were related to autonomous shipping and 9 were on ports. After screening the abstracts, 36 Table 1. Literature search terms Search string 1: (“maritime” AND “simulat*” AND (“training” OR “education”)) OR Search string 2: (“maritime” OR “marine” OR “sea”) AND (“pilotage” OR “navigation*” OR “bridge” OR “seafarer”) AND (“education” OR “training”) AND (“simulator*” OR “simulation”)

Scenario Design, Data Measurement, and Analysis Approaches

41

studies were left which were read in full. Out of these, 13 were considered of high relevance. One article was added manually at this point. Hence a total of 14 articles were transcribed in a literature review matrix to compose the results presented in the next section.

3 Results A qualitative content analysis approach was adopted to answer the research questions. The 14 relevant studies were coded for content using a structured literature review matrix in Microsoft Excel. The matrix included coding of simulator use information, description of simulation scenario, exercise duration, key performance indicators (KPIs) for assessment, variables for assessment, and data collection and analysis approaches. A summary of the reviewed studies is presented in Table 2. Table 2. Summary of the reviewed studies Study

Data collection tool

Data frequency

Data analysis method

Software

[5]

Survey

10 min interval

Descriptive statistics, Spearman Rank Difference mean

Excel

[6]

Eye-tracking

Continuously, 100 Hz

Descriptive statistics, Kruskal Wallis test, Mann-Whitney U Test, Heat-map

Tobii Pro Lab Analyzer; SPSS

[7]

ECDIS

Every 10 s

Deviation calculation

Not reported, most likely Excel

[2]

SEA tool in the NA simulator, observation

SEA and review by experts

SEA program (automatic evaluation) and evaluation by qualified persons

[8]

EEG; Videos to label the events and match them

Continuously

SVM; EEG data processing; Evaluating the brain states using ML

C++ on Visual Studio; HTML and JavaScript

[9]

1. EEG 2. ECG 3. Eye-tracking 4.1 NASA TLX 4.2. Likert scale

1–3 continuously; Likert scale 4.1 after each exercise; transformed into a 4.2. Every 2 min continuous numerical function

Kubios HRV

(continued)

42

Z. H. Munim et al. Table 2. (continued)

Study

Data collection tool

Data frequency

Data analysis method

Software

[10]

Data from the simulation

NA

ANOVA; Pearson correlation

IBM SPSS

[11]

Cameras, microphones, eye tracking, simulator data storage

Continuously

Tobi Pro Glasses 2

Tobii Pro software development kit (SDK)

[12]

Revised NEO Personality Inventory, SARS questionnaires; ECG; Simulator data

5-min interval for ECG; The revised NEO Personality Inventory one week prior, SARS after the simulation

Hierarchical multiple regression; Pearson correlation; ANOVA: Turkey’s post hoc test

NA

[13]

Video recording

Continuously

Cataloguing to Excel, analytical search

Excel

[14]

SA: 15 questions; Decision making: questionnaire; Navigational performance: Simulator data

Simulator data: Continuously;

Mann–Whitney U test; Wilcoxon signed-rank test

NA

[15]

Observations from the Continuously instructors and participants

Safety compliance

NA

[16]

Empatica E4 band (wrist)

Body temp: 4 Hz; in Convolutional neural °C; EDA: 4 Hz, in µS; network BVP: 64 Hz; PPG; all signal are downsampled to 1 Hz for data analysis

NA

[17]

fNIRS data; Ship distance data from simulator

Continuously

NA

ANOVA, Artificial Neural Networks

3.1 Simulation Scenario Characteristics All the studies in the sample used full-mission simulators. Four studies reported the use of Kongsberg simulators [2, 5, 7, 16], while others did not report the simulator provider’s name. The simulation scenarios varied in terms of context, duration, and level of complexity. The majority of the simulation scenarios included anti-collision scenarios [2, 5, 6, 13, 14, 17]. Some focused on sub-sea gas compression installations [11], manoeuvring [8, 10], docking [7], berthing [9] and offloading operations [15]. Overall, simulation exercises included tasks related to navigation (situational awareness, Radar, ECDIS), watchkeeping, manoeuvring, docking/berthing/offloading

Scenario Design, Data Measurement, and Analysis Approaches

43

(dynamic positioning), collision avoidance (COLREG regulations), radio reporting, and decision making. Some of the sample studies described the simulation scenario context in detail. Simulation scenarios covered real voyages in the Strait of Gibraltar [5], the Oslofjord [7], the coast of Western Norway [11], and the Dover Strait in the English Channel [13]. Others focused more on the tasks, required activities, and/or phases of operations. For instance, the simulation training experiment in Orlandi et al., [10] included two phases: (1) developing four manoeuvring plans, and (2) executing the plans during voyage. ˙ Likewise, the task requirement in Juszkiewicz & Zukowska [2] included turning on and adjusting the ARPA to weather conditions, acquiring targets, assessing the situation, planning and executing an anticollision maneuver (if necessary), and also planning and executing a returning maneuver (if necessary). Studies that attempted to assess higher levels of navigation competency, usually included anti-collision, decision making [6, 17], and berthing operations [9, 10]. The length of simulation scenarios varied among the sample studies to a great extent. The shortest reported is 13 min [6] and longest ones lasted up to 02 h [9, 15]. The scenario length was about 60 min in several studies [8, 14, 16]. Most of the simulation scenarios can be divided into three to five phases. Typical phases are familiarization or baseline (5–15 min), operational tasks (5 to 10 min), duties or watchkeeping (15–30 min), crisis/emergency management or decision making (5–10 min), and feedback or debriefing (15–30 min). 3.2 Data Collection Tools and Approaches Based on the coding of information in the structured literature review matrix, a framework guiding the adoption of data collection tools and approaches for assessing various navigation competency KPIs is proposed in Fig. 1. In the sample of reviewed studies, five types of navigation competency KPIs are assessed: (1) navigation performance mainly through deviation from the planned route or course, (2) psychological and psychophysical factors, (3) situational awareness, (4) navigation rules and regulations, and (5) crisis management and decision making. The tools and approaches to data collection varied depending on the type of navigation competency KPIs. To assess (Type 1) navigation performance, data were collected mainly from the simulator itself (log data) [2, 7, 10, 14]; additionally, observation by the instructor was reported [2]. For assessing (Type 2) psychological and psychophysical factors such as workload, stress level, situation control, and level of difficulty, data were collected using eyetracking, Electroencephalogram (EEG), Electrocardiogram (ECG), self-reported likert scale surveys [8, 9], and functional near-infrared spectroscopy (fNIRS) [17]. While EEG and fNIRS are used for measuring brain activity, ECGs are used to record the electrical activity of the heart. Several studies focused on the assessment of (Type 3) situational awareness (SA), which has also been assessed using data from diverse tools and approaches. Atik [6] has used only eye-track data for SA assessment. In addition to eye-tracking and simulator data, Sanfilippo [11] used data from video recording cameras and voice recordings

44

Z. H. Munim et al.

from microphones for SA assessment. Türkistanli and Kuleyin [14] used survey questions multiple times at different phases of the simulation exercise. SA was assessed through variation in psychological and psychophysical measures as well using cardiovascular responses (CVR), heart rate variability (HRV), photoplethysmograph (PPG) sensor (measures blood volume pulse, HRV (can be also derived from blood volume pulse), infrared thermopile (reads peripheral skin temp), EDA sensor (measures the fluctuating changes in certain electrical properties of the skin), and 3-axis accelerometer (to capture motion-based activity) [12, 16]. The Empatica E4 (wrist) band was used for data collection by Xue et al., [16]. Saus et al., [12] used data from multiple sources such as the Situational Awareness Rating Scale (SARS), Revised NEO Personality Inventory (NEO PI-R), Ambulatory Monitoring System using 1 cm Ag/AgCl ECG electrodes, and simulator data. To assess (Type 4) navigation rules and regulations competency, video recording and observation were used [2, 13]. Finally, (Type 5) crisis management and decisionmaking were assessed using data collected through observations from the instructors and participants [15], and questionnaire surveys [14].

Fig. 1. A framework for data collection in the assessment of maritime simulator training

3.3 Data Analysis Methods and Approaches Data collected through various tools and approaches were mostly analysed using simpler statistical methods and widely used software. Descriptive analysis and mean comparison tests such as ANOVA, Kruskal-Wallis, Spearman Rank, and Mann-Whitney tests were

Scenario Design, Data Measurement, and Analysis Approaches

45

evident in experimental studies [5, 6, 10, 14, 17]. Use of correlation and multiple regressions were evident, too [12]. In addition to statistical analysis, heat map analysis was used for eye-tracking [6]. For navigation performance assessment using simulator data, deviations were calculated [2, 7]. For EEG, ECG, and fNIRS data analysis, machine learning approaches have been used, for instance, artificial neural networks (ANN) [17], convolutional neural network (CNN) [16], and support vector machine (SVM) [8]. In terms of software used, Tobii Pro was used when eye-tracking data was analysed [6, 11]. SPSS and Excel were mainly used for statistical analysis and deviation calculations [5, 10]. For machine learning applications, the software was not explicitly reported, but now-a-days ANN, CNN, and SVM can be used though many software packages including SPSS, Matlab, Stata, R, python etc. 3.4 Proposal for a Comprehensive Assessment Approach Since an inventory of data collection tools, analysis approaches, and how they are used in maritime simulator training have identified, a comprehensive assessment approach is proposed utilizing them in Fig. 2. For instance, to assess navigation performance, simulator data can be used which can be analysed using classification and/or clustering ML algorithms. Similarly, eye tracking can be used to assess situational awareness by heatmap analysis. This approach can be adopted to develop a LAD for students and instructor in MET. For assessment of five dimensions of MET, at least five different data sources are recommended, and should be analysed through at least five separate models or algorithms, but the outcomes need to be visualized in one dashboard.

Fig. 2. Proposed assessment approach integrating emerging technologies

46

Z. H. Munim et al.

4 Conclusions and Future Research This study provides a synthesis of maritime training simulation design characteristics, data collection and analysis tools and approaches in simulation training assessment. A systematic literature review approach was adopted to identify relevant published studies on the topic and analyse their contents. The findings reveal that the simulation scenarios varied in terms of context, duration, and level of difficulty. Data collection tools and approaches include simulator data (including ECDIS and Radar data), eye-tracking, questionnaire surveys, observations, EEG, ECG, fNIRS, video recording, and voice recording. Statistical analysis utilizing descriptive statistics and mean comparison tests were the most common analysis methods, while SPSS and Excel were the main analysis software tools. Based on the findings, a comprehensive assessment approach in maritime simulator training has been proposed. Since the reviewed studies only focused on full-mission simulation training, future research should focus more on other simulator modalities such as desktop, cloud and virtual reality. Future development of navigation simulator training might further benefit from the adaptation of the metaverse. Despite the use of wearable data collection tools in reviewed studies such as eye-tracker, fNIRS, EEG and ECG, their feasibility of use during regular training sessions needs to be assessed. Further, in terms of data analysis, a machine learning approach to data analysis is not evident in the sample of reviewed studies, which should be explored in future research. Funding. This research is funded by the European Union’s Horizon Europe research and innovation programme under grant agreement No 101060107.

References 1. Kobayashi, H.: Use of simulators in assessment, learning and teaching of mariners. WMU J. Marit. Aff. 4, 57–75 (2005) ˙ 2. Juszkiewicz, W., Zukowska, A.: The use of the K-sim polaris simulator in the process of automatic assessment of navigator competence in the aspect of anticollision activities. Appl. Sci. 13(2), 915 (2023) 3. Munim, Z.H., Kim, T.-E.: A review of learning analytics dashboard and a novel application in maritime simulator training. In: Proceedings of the AHFE Conference (2023) 4. Macke, J., Genari, D.: Systematic literature review on sustainable human resource management. J. Clean. Prod. 208, 806–815 (2019) 5. Jiménez, J.I.A.: Modelling the relationship between performance and ship-handling simulator. J. Marit. Res. 17(3), 68–73 (2020) 6. Atik, O.: Eye tracking for assessment of situational awareness in bridge resource management training. J. Eye Mov. Res. 12(3) (2019).https://doi.org/10.16910/jemr.12.3.7 7. Hjelmervik, K., Nazir, S., Myhrvold, A.: Simulator training for maritime complex tasks: an experimental study. WMU J. Marit. Aff. 17(1), 17–30 (2018) 8. Liu, Y., et al.: Psychophysiological evaluation of seafarers to improve training in maritime virtual simulator. Adv. Eng. Inform. 44, 101048 (2020) 9. Orlandi, L., Brooks, B.: Measuring mental workload and physiological reactions in marine pilots: building bridges towards redlines of performance. Appl. Ergon. 69, 74–92 (2018)

Scenario Design, Data Measurement, and Analysis Approaches

47

10. Orlandi, L., Brooks, B., Bowles, M.: A comparison of marine pilots’ planning and manoeuvring skills: uncovering mental models to assess shiphandling and explore expertise. J. Navig. 68(5), 897–914 (2015) 11. Sanfilippo, F.: A multi-sensor fusion framework for improving situational awareness in demanding maritime training. Reliab. Eng. Syst. Saf. 161, 12–24 (2017) 12. Saus, E.-R., Johnsen, B.H., Eid, J., Thayer, J.F.: Who benefits from simulator training: personality and heart rate variability in relation to situation awareness during navigation training. Comput. Hum. Behav. 28(4), 1262–1268 (2012) 13. Sellberg, C., Lundin, M., Säljö, R.: Assessment in the zone of proximal development: simulator-based competence tests and the dynamic evaluation of knowledge-in-action. Classroom Discourse 13(4), 365–385 (2022) 14. Türkistanli, T.T., Kuleyin, B.: Game-based learning for better decision-making: a collision prevention training for maritime transportation engineering students. Comput. Appl. Eng. Educ. 30(3), 917–933 (2022) 15. Wahl, A., Kongsvik, T., Antonsen, S.: Balancing safety I and safety II: learning to manage performance variability at sea using simulator-based training. Reliab. Eng. Syst. Saf. 195, 106698 (2020) 16. Xue, H., Batalden, B.-M., Sharma, P., Johansen, J.A., Prasad, D.K.: Biosignal-based driving skill classification using machine learning: a case study of maritime navigation. Appl. Sci. 11(20), 9765 (2021) 17. Fan, S., Yang, Z.: Towards objective human performance measurement for maritime safety: a new psychophysiological data-driven machine learning method. Reliab. Eng. Syst. Saf. 233, 109103 (2023)

Analysis of Creative Engagement in AI Tools in Education Based on the #PPai6 Framework Dea Puspita Septiani1

, Panos Kostakos2,3

, and Margarida Romero1(B)

1 Université Côte d’Azur, LINE, Nice, France [email protected], [email protected] 2 University of Oulu, Oulu, Finland [email protected] 3 Kaunas University of Technology, Kaunas, Lithuania

Abstract. Human creativity is a complex process that can be evaluated in a wide range of domains and tasks. The domain and task-specificity of human creativity challenge the process of designing AI-based tools to support teachers’ and learners’ creative engagement. In this study, we introduce the #PPai6 framework to look at the 21st-century skills that modern education needs to teach, with a focus on creativity and critical thinking. After introducing the #PPai6 framework, we analyze 41 studies using AI in education in order to identify the level of creative engagement they are able to support from primary education to Higher Education. The results show the most usual way of supporting learners’ creative engagement is through intelligent tutoring systems (ITS), which rely on the second level of creative engagement of the #PPai6 framework. In this second level, the AI tool shows adaptive behavior based on the learners’ interactions but does not engage the learners in creating new ideas or solutions. We analyze 13 cases where learners and teachers get help with their own creative processes, but only two cases of collective creativity are supported at the individual level. None of the AI tools in education supports collective creativity among teachers. Keywords: Artificial intelligence · creativity · creative engagement · education · 21st-century competencies

1 Introduction In the last five decades, AI tools have been developed for a variety of educational purposes. However, these tools were mostly used in limited situations, often for research purposes. The review by Feng and Law [1] on the studies developed during 2010–2019 has permitted two main educational technologies being supported by AI for educational purposes: Intelligent Tutoring Systems (ITS) and massive open online courses (MOOCs). The popularization of the use of AI tools for education has benefited over the past year from the public availability of the ChatGPT. In 2022, 242 papers were published on ChatGPT, and 1470 in the first three months of 2023. The media’s coverage of ChatGPT’s effects has made more people aware of how this tool could be used © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 48–58, 2023. https://doi.org/10.1007/978-3-031-42134-1_5

Analysis of Creative Engagement in AI Tools in Education

49

in education. However, it has also raised some concerns about the effects of this type of tool on academic integrity and on teachers’ and students’ ability to be creative in their educational activities. In this paper, we consider an emergent-based approach to technology-enhanced learning (TEL), in which AI tools are part of activity systems that are not only defined by the technology tools but also by how they mediate the human activity systems towards a certain goal [2, 3]. While some educators are concerned about how AI tools can hinder learners’ creative processes by delegating part of the process to AI tools, we should also consider how these tools can support AI-human collaboration for individual, group-based, and large-group collaboration. 1.1 Creativity as One of the 21st-Century Skills to Be Supported in AI-Human Learning Activities Creativity is now widely acknowledged as a crucial skill that distinguishes human labor from that performed by robots in an environment where automatization and artificial intelligence are having an increasing effect [4]–[6]. According to Florida [4], creativity is a factor in the social division of modern societies into “creative classes,” who establish professions where creativity is a decisive element in their sophisticated problem-solving activities, and “creative workers,” who do not. In postindustrial knowledge societies, different types of jobs increasingly depend on the “creative class” [4], and those who work in it “engage in complex problem-solving that involves a great deal of independent judgment and requires high levels of education or human capital” (p. 8). For this reason, supporting creative processes is essential in 21st-century education. AI tools should be part of the efforts to support human creativity development not only at the school, but also in creative professions. Teachers are among the professionals who face challenges related to the need to support learners’ 21st-century competencies as well as a better understanding of how AI technologies operate and can support the teaching and learning process. The emergence of generative AI tools such as ChatGPT has raised awareness among educators about the need to better understand AI and how to regulate or integrate their use for educational purposes. Creativity is one of the six key transversal competencies for the 21st century education #5c21 model [7], in which critical thinking is the ability to develop independent, critical thought in order to analyze ideas, knowledge, and processes based on a humanbased value system and judgment. While creativity can be considered either human or “artificial creativity” when an AI system develops useful, novel, and original ideas or artifacts [8]; critical thinking can only be performed by humans because this competency relies on human criteria and takes into account factors such as the cultural context and interpersonal relationships in a certain socio-cultural context [9]. In the Horizon AugMentor project [10], we aim to support learners’ creativity by developing a pedagogical framework permitting the use of LMS AI tools to support learners and teachers in their creative processes (Fig. 1).

50

D. P. Septiani et al.

Fig. 1. Five key competencies for the 21st-century education (#5c21)

1.2 The Limits of AI Tools for Supporting Cognitive Architecture to Support Creativity While creativity can manifest as either human or “artificial creativity” when an AI system generates useful, novel, and original ideas or artifacts, critical thinking is inherently human. It depends on human criteria and factors such as cultural context and interpersonal relationships within a specific socio-cultural environment. Human cognitive architecture allows for the incorporation of cultural context and interpersonal relationships within a specific socio-cultural environment. Presently, large language models (LLMs) like OpenAI’s ChatGPT are incapable of engaging in critical thinking and human-like creativity because they do not possess a cognitive architecture capable of self-reflection and metacognitive judgments. In short, as Chomsky explains [29], AI models like ChatGPT struggle to strike a balance between creativity and constraint, leading to excessive or minimal output with ethical ambiguity and linguistic inaccuracies, making their widespread acceptance a matter of amusement and desolation. The lack of a cognitive framework and the misalignment with human objectives, preferences, and ethical principles have motivated some GPT3 developers to devise a new framework known as Anthropic. A significant hurdle in the future advancement of conversational AI is establishing a cognitive architecture that could also support improved critical thinking. To address these limitations, OpenAI has also opted to develop a higher short memory (e.g., GPT4), which better supports the integration of socio-cultural prompts and creates results that are perceived by the end-user as more creative than prior releases. Despite their shortcomings, conversational AI models such as ChatGPT are becoming crucial enablers for Learner-Centered Instruction (LCI) and Tinker Learning. In a recent study [30], 41 students were given several assignments and a final project to complete, each designed to develop their skills in different areas. ChatGPT was used as a tool to enhance these assignments to analyze and understand datasets, generate insights and recommendations, create natural language descriptions of solutions, and analyze large amounts of text data. The feedback collected from the survey results suggests that the

Analysis of Creative Engagement in AI Tools in Education

51

teaching methods utilized in the course were positively received by the majority of the participating students. 1.3 Creative Engagement: Learners’ and Teachers’ Perspective Learners’ engagement is a requirement for correctly developing the learning activities designed by the teachers. Engagement is defined as the learners’ “involvement in learning activities in terms of attention, participation, effort, intensity, or persistence” [11, p. S16]. Creative engagement is a form of engagement that is not only cognitive but also in which the learner is a creative agent, producing (or making) generative acts or artifacts. Creative engagement as a teacher relies on generating new learning activities, while the learners’ creative engagement is related to the acts of artifacts developed by the learners. For example, the English-ABLE system [12] has the main pedagogical objective of supporting the learners’ engagement with three artificial agents based on the Open Students Model (OSM) which are designed to engage the learners’ in a teaching process of the OSM. Through the learners’ engagement with the OSM agents, they are expected to develop their English as a Second Language (ESL) competencies. The agents have been designed in a way to engage in different objectives, such as grammar feedback. We can consider the learners’ engagement in the English-ABLE system as creative because they are producing novel, useful, and original interactions with artificial agents. They are not only selecting pre-established answers (non-creative engagement), but they are also creatively engaged in the interaction with these three artificial agents. For supporting these creative engagements, the system is based on an “adaptive sequencing of activities and adaptive feedback mechanisms” [12, p. 382]. So, it’s important to model the learner, the activities, and the feedback system while keeping in mind the learner’s creative process for this particular task. The external regulation of the teachers can increase learners’ creative engagement. In the context of the English-ABLE solution, dynamic recommendations that enable teachers to make decisions about how to support the learners’ process support the teachers’ creative engagement in the supervision process.

2 #PPai6 Levels of Creative Engagement in AI for Education Technology alone is not sufficient for driving innovation or improvement in education [13, 14]. The learning activities are complex systems in which technological and cultural artifacts mediate individual and collective activity oriented towards a certain goal and are constrained by the socio-cultural and situational elements within which the learning activity is developed [15]. An important consequence of considering learning activities as an intertwined system in which there is an emergent use of technologies based on the learners’ motivations, the task constraints, and the situational aspects of the activity is that the same technology could be used in a very diverse way. For example, the Teachable Machine1 for creating machine learning models) can be used in a lecture in which the teacher shows an example of its use to a group of learners who are not engaged in the activity. In this context, the learners are “passive consumers” of the lecture on Teachable 1 https://teachablemachine.withgoogle.com/

52

D. P. Septiani et al.

Machine. But the teacher can also engage the learners in a group-based activity in which the children will create a model for improving the recycling system of their neighborhood in collaboration with the shop owners around the school. In this participatory solution co-creation, Teachable Machine could be used as a tool to integrate when tinkering with a solution that is possible to develop thanks to the participatory creative process engaged by the community participating in this objective. Teachable Machine is only a tool with the potential to support the creative engagement process, depending on the pedagogical scenario of the teaching activity proposed by the teacher. 2.1 From Passive Consumption to Participatory Content Co-Creation In Fig. 2 six levels of creative engagement with AI tools in education are identified, which are on a continuum from simple to complex, and reflect the degree of creative engagement a learner can experience as a socio-cognitive process: • Level 1: Passive consumer. The learner just consults AI-generated content with no creative engagement on their side. • Level 2: Interaction. The learner interacts with an AI system that adapts the feedback and activity progression based on the learners’ and task models. In this second level, the AI system adapts to the learner, but the learner isn’t doing anything creative; they’re just moving forward based on how the system is set up. • Level 3: Individual content creation. The learner can use creativity to put forth various suggestions for solutions or ideas that the AI system has not already predetermined. • Level 4: Collaborative content creation. A dyad or small group of learners are engaged in a joint creative activity to propose different ideas or solutions which are not predetermined by the AI system. E.g. A group of learners can engage in creating a poster to raise awareness about food waste in their school and do a joint brainstorming with ChatGPT before engaging in the final design of their poster. • Level 5: Participatory knowledge co-creation. A group of learners, in collaboration with other participants outside their learning group, engage in a creative participatory activity engaged in a complex problem-solving situation. • Level 6. Expansive learning supported by AI. In formative interventions supported by AI, participants’ agency may expand or transform problematic situations. AI tools can be used to help identify contradictions in complex problems and help generate concepts or artifacts to regulate conflicting stimuli and foster collective agency and action. AI tools can be used to assist in the modeling of activity systems as well as in the simulation of new actions, facilitating the expansive visualization process [16]. The third level of the #PPai6 is similar to the highest level of the Blooms’ taxonomy (creating). Nevertheless, the other levels of Blooms’ taxonomy are not applicable to the #PPai6 model because we do not assume that the passive consumption of AI content or the interaction with AI tools ensures the cognitive processes defined in Blooms’ taxonomy (remembering, understanding, applying, analyzing, evaluating). The first level of the ##PPai6 is similar to the “passive” level of Chi and Wylie’s ICAP framework [17]. The biggest difference between our model and the ICAP is the consideration of interactivity. While “interactive mode of engagement” is the highest level of cognitive engagement within the ICAP framework, our model considers “participatory knowledge

Analysis of Creative Engagement in AI Tools in Education

53

Fig. 2. Six levels of creative engagement in AI in education (##PPai6)

co-creation” and “expansive learning” as the most transformative AI-learner situations. Participatory knowledge co-creation engages the participants not only in an interactive and socio-constructive situation, but also engages learners in the identification, understanding, and problem-solving processes of a problematic situation within their learning or neighborhood community, linking the team-based co-creation process with a participatory process wherein a team of learners engages in their learning community in order to improve a real-world problem or value community initiatives [2, 18]. The first two levels do not engage the learners’ in a creative activity. In the first level, the learners use what is made available to them without any interaction. In the second level, learners interact with an AI system that responds to their actions based on a model of the learning task and a model of the learner that is built into the AI system. The AI tool has a predefined set of options that lead to interactions governed by a “programmed instruction” approach, harking back to Pressey’s teaching machine. ITS are the most common AI tools at this second level of creative engagement. The third level of creative engagement with AI tools in education is to get the learner to make texts, photos, or videos that are related to a certain learning moment or situation. Whereas the fourth and fifth levels of creative engagement with AI tools engage learners in a co-creation process that supports the knowledge construction process [19]. The fifth level gets students involved in finding a problem in their learning or neighborhood community, understanding it, and coming up with a solution. In this fifth level, the co-creation participatory process is oriented toward the community as well as real-life problemsolving [2]. Participatory and community-oriented (or based) knowledge co-creation values local community initiatives, promotes diversity, and regenerates intergenerational and intercultural links that are often missed in our current societies [20].

3 Methodology For the analysis of the six levels of creative engagement in AI in education, we revised the International Journal of Artificial Intelligence in Education (AIED) within the last three years. We selected all the papers integrating an empirical study focusing on the teacher, the learner or both. Based on these criteria, we analyzed 41 papers in order to identify the different levels of creative engagement, considering not only the learners’ and teachers’ perspectives on creative engagement but also the domain of application and the educational level of the study.

54

D. P. Septiani et al.

For papers integrating more than one AI solution, we evaluate them separately. For example, HOWARD and BioWorld AI-tools in the paper of Lajoie [21], we integrated two records in the analysis of creative engagement levels in order to report the specificities of these different AI solutions that were studied. Studies in which both learner and teacher activity are supported are also integrated as two separate entries in order to consider the level of creative engagement from each of these perspectives.

4 Results 4.1 Levels of Creative Engagement The assessment of the six levels of creative engagement across the 41 selected studies has revealed that the majority of creative engagement focuses on the learner perspective (n = 32), with only six instances of AI tools supporting teachers’ creative engagement (Fig. 3). In three studies (BioWorld & HOWARD Platform [21], MiWRITE [22], English-ABLE & CBAL [12]), the tool supported both the teacher and the learner’s creative engagement process.

Fig. 3. Number of papers (y-axis) according to the learners’ and teachers’ levels of creative engagement (x-axis).

Only one study has developed a solution in which there is no creative engagement from the learners. The study of Lawson, et al. [23] only looked at learners’ emotional expressions to animated instructions during a presentation. We can observe that the majority of studies (n = 21) support the second level of creative engagement, “interactive consuming”. In this level, the learner uses an AI tool to help with a well-defined learning activity, and the system adapts to the learner’s needs. Most of the systems in this context are self-identified as Intelligent Tutoring Systems (ITS). Individual creative engagement, the third level of the #PPai6 model, is the only level in which we can observe both the learner’s (n = 8) and teacher’s (n = 6) perspectives, which are in some cases supported independently, as in the case of Tuglet [24], Physics

Analysis of Creative Engagement in AI Tools in Education

55

Playground [25], C2STEM [26], TopoMath [27], etc. More interestingly, the BioWorld & HOWARD Platform [21], MiWRITE [22], English-ABLE and CBAL [12] tools support both the teachers and the learners in their creative processes. The fourth level, collective creative engagement, can be observed in two studies. The first one is the study of NoRILLA [28], a specialized Augmented Reality (AR) for STEM education where learners are encouraged to do an exploratory construction together. The learners’ using the AI tool NoRILLA are engaged in building blocks that should not fall during a strong motion. First, the learners advance individually, and once they have achieved a certain progression, they are able to engage in teams for an exploratory construction collaboratively with the tool. The second study is using the HOWARD Platform [21] where learners and instructors can simultaneously engage in monitoring and responding in a problem-based learning (PBL) environment in which discussion is encouraged. Participatory content co-creation, the fifth level of the #PPai6 framework, neither the sixth level (expansive learning) are not observed in any of the 41 studies revised.

5 Discussion AI tools can be used in a diversity of pedagogical scenarios with different degrees of creative engagement for the learner or the teacher. In this study, we have analyzed 41 studies in order to identify creative engagement with AI tools. The results show the majority of educational uses of AI are individual (n = 33 for the learners, n = 6 for the teachers), with only two uses to support the learners’ co-creativity. We can observe that most of the uses of AI tools do not engage learners creatively. Most of the studies engage learners in the second level of the #PPai6 framework, “interactive consuming”, through intelligent tutoring systems (ITS) which permit the system to adapt to the learners’ inputs based on the learner model and the learning task model integrated into the AI-tool. Developing a learner and task model is already a complex process that requires domainspecific expertise and computer modeling efforts. Computer-supported collaborative learning (CSCL) activities are more complex in relation to the group dynamics that are emerging not only at the individual level but at the collective level. The lack of standardization on these group dynamics ontologies and modes is one of the potential reasons why we observe a limited number of collaborative uses of AI tools in education. We can observe that some of the AI tools are Digital Game-Based Learning (DGBL) solutions. The characteristics of DGBL as systems engaging the learners’ by providing feedback and supporting actions towards a learning objective are aligned with the characteristics of AI tools in education, which share these two characteristics. Most DGBL solutions support the second level of creative engagement. Among the DGBL AI tools, two address STEM education, such as C2STEM [26] and NoRILLA [28]. None of the studies supports participatory content co-creation, the fifth and sixth level of the #PPai6 framework. We can consider not only the higher degree of complexity of AI systems but also their ability to support users whose’ behavior does not correspond to the teacher-and-learner model. In the context of Maison de l’intelligence artificielle (MIA) in Sophia Antipolis, the types of activities in which the learners are engaged correspond to the fifth level. The learners are engaged to co-create solutions addressing

56

D. P. Septiani et al.

different Sustainable Development Goals (SDGs) in collaboration with the AI experts in the MIA, their teachers, and other learners. The analysis of the levels of creative engagement will permit the AI community in education to consider the pedagogical integration of AI tools, considering the possibility of engaging teachers and learners in creative processes. Moreover, the results of the study contribute to the Horizon AugMENTOR Project by supporting researchers and computer engineers in their understanding of the different types of AI solutions that can better support 21st-century competencies. These results can permit educators to design the AI tools and their integration to better support the learners’ human creativity. Beyond the creative integration of AI tools in education, AI tools cannot be considered as creative as humans at the current stage. Consider a more robust cognitive architecture that will allow AI models to better simulate human-like cognition, allowing them to process cultural contexts, interpersonal relationships, and long-term memory more effectively. This would be particularly valuable for collaborative open-ended learning activities and CSCL pedagogical scenarios on the fifth level ( participatory content co-creation), where context and history play an essential role in decision-making and problem-solving processes. Incorporating a cognitive architecture in AI models would also help address challenges related to biases, ethics, and transparency, as it would allow AI systems to make decisions and judgments based on a more comprehensive understanding of the intergroup context and human values. Acknowledgments. This research is developed within the AugMENTOR project, funded by the HORIZON-CL2-2021 TRANSFORMATIONS-01-05 program.

References 1. Feng, S., Law, N.: Mapping artificial intelligence in education research: a network-based keyword analysis. Int. J. Artif. Intell. Educ. 31(2), 277–303 (2021). https://doi.org/10.1007/ s40593-021-00244-4 2. Isaac, G., Romero, M., Barma, S.: Understanding co-creativity in real-world problem solving in project-based learning in higher education. Rev. Int. CRIRES innover dans la tradition de Vygotsky 6(3), 86–99 (2022). https://doi.org/10.51657/ric.v6i2.51585 3. Engestrom, Y.: Activity theory as a framework for analyzing and redesigning work. Ergonomics 43(7), 960–974 (2000). https://doi.org/10.1080/001401300409143 4. Florida, R.L.: The Rise of the Creative Class: and how it’s Transforming Work, Leisure, Community and Everyday Life, Nachdr. Basic Books, New York, NY (2006) 5. De Bono, E.: Serious Creativity: Using the Power of Lateral Thinking to Create New Ideas. Penguin Random House, New York, NY, USA (2015) 6. Sternberg, R.J., Frensch, P.A.: Complex Problem Solving: Principles and Mechanisms. Psychology Press (2014) 7. Romero, M., Lille, B., Patino, A. (eds.): Usages créatifs du numérique pour l’apprentissage au XXIe siècle, vol. 1. Presses de l’Université du Québec, Québec (2017) 8. Moruzzi, C.: Measuring creativity: an account of natural and artificial creativity. Eur. J. Philos. Sci. 11(1), 1–20 (2020). https://doi.org/10.1007/s13194-020-00313-w 9. Romero, M.: “Développer la pensée informatique pour démystier l’intelligence artificielle”, 1024 – Bull. Société Inform. Fr. 12, 67–75 (2018)

Analysis of Creative Engagement in AI Tools in Education

57

10. augMENTOR: Augmented Intelligence for Pedagogically Sustained Training and Education (augMENTOR) (2023). https://cordis.europa.eu/project/id/101061509. Accessed 25 Mar 2023 11. Chiu, T.K.F.: Applying the self-determination theory (SDT) to explain student engagement in online learning during the COVID-19 pandemic. J. Res. Technol. Educ. 54(sup1), S14–S30 (2022). https://doi.org/10.1080/15391523.2021.1891998 12. Zapata-Rivera, D.: Open student modeling research and its connections to educational assessment. Int. J. Artif. Intell. Educ. 31(3), 380–396 (2020). https://doi.org/10.1007/s40593-02000206-2 13. Romero, M., Laferriere, T., Power, T.M.: The move is on! From the passive multimedia learner to the engaged co-creator. eLearn 2016(3), 1 (2016). https://doi.org/10.1145/2904374.289 3358 14. Tricot, A.: L’innovation pédagogique: Mythes et réalités (2017). http://www.cafepedag ogique.net/LEXPRESSO/Pages/2017/09/05092017Article636401937485284884.aspx. 05 Accessed 2019 15. Romero, M., Barma, S.: Analysing an interactive problem-solving task through the lens of double stimulation. Can. J. Learn. Technol. 48(1) (2022). https://doi.org/10.21432/cjlt28170 16. Romero, M., et al.: Creative engagement in #AIed: from the passive AI content consumer to expansive learning supported by AI (#PPai6). Creative Uses of Technology (2023). https://mar garidaromero.blog/2023/04/28/expansive-learning-supported-by-ai/. Accessed 19 May 2023 17. Chi, M.T.H., Wylie, R.: The ICAP framework: linking cognitive engagement to active learning outcomes. Educ. Psychol. 49(4), 219–243 (2014). https://doi.org/10.1080/00461520.2014. 965823 18. Lukyanenko, R., Parsons, J.: Extending participatory design principles to structured usergenerated content. In: Presented at the Scandinavian Conference on Information Systems, Oulu, Finland (2015) 19. Zhang, J., Scardamalia, M., Lamon, M., Messina, R., Reeve, R.: Socio-cognitive dynamics of knowledge building in the work of 9-and 10-year-olds. Educ. Technol. Res. Dev. 55(2), 117–145 (2007) 20. Cucinelli, G., Davidson, A.-L., Romero, M., Matheson, T.: Intergenerational learning through a participatory video game design workshop. J. Intergenerational Relatsh. 16(1–2), 146–165 (2018) 21. Lajoie, S.P.: Student modeling for individuals and groups: the BioWorld and HOWARD platforms. Int. J. Artif. Intell. Educ. 31(3), 460–475 (2020). https://doi.org/10.1007/s40593020-00219-x 22. Wilson, J., Huang, Y., Palermo, C., Beard, G., MacArthur, C.A.: Automated feedback and automated scoring in the elementary grades: usage, attitudes, and associations with writing outcomes in a districtwide implementation of MI write. Int. J. Artif. Intell. Educ. 31(2), 234–276 (2021). https://doi.org/10.1007/s40593-020-00236-w 23. Lawson, A.P., Mayer, R.E., Adamo-Villani, N., Benes, B., Lei, X., Cheng, J.: Do learners recognize and relate to the emotions displayed by virtual instructors? Int. J. Artif. Intell. Educ. 31(1), 134–153 (2021). https://doi.org/10.1007/s40593-021-00238-2 24. Käser, T., Schwartz, D.L.: Modeling and analyzing inquiry strategies in open-ended learning environments. Int. J. Artif. Intell. Educ. 30(3), 504–535 (2020). https://doi.org/10.1007/s40 593-020-00199-y 25. Shute, V.J., et al.: The design, development, and testing of learning supports for the physics playground game. Int. J. Artif. Intell. Educ. 31(3), 357–379 (2021). https://doi.org/10.1007/ s40593-020-00196-1

58

D. P. Septiani et al.

26. Hutchins, N.M., Biswas, G., Zhang, N., Snyder, C., Lédeczi, Á., Maróti, M.: Domain-specific modeling languages in computer-based learning environments: a systematic approach to support science learning through computational modeling. Int. J. Artif. Intell. Educ. 30(4), 537–580 (2020). https://doi.org/10.1007/s40593-020-00209-z 27. VanLehn, K., Banerjee, C., Milner, F., Wetzel, J.: Teaching algebraic model construction: a tutoring system, lessons learned and an evaluation. Int. J. Artif. Intell. Educ. 30(3), 459–480 (2020). https://doi.org/10.1007/s40593-020-00205-3 28. Yannier, N., Hudson, S.E., Koedinger, K.R.: Active learning is about more than hands-on: a mixed-reality AI system to support STEM education. Int. J. Artif. Intell. Educ. 30(1), 74–96 (2020). https://doi.org/10.1007/s40593-020-00194-3 29. Chomsky, N., Roberts, I., Watumull, J.: Noam Chomsky: The False Promise of ChatGPT. The New York Times (2023) 30. Tsai, Y.-C.: Empowering learner-centered instruction: integrating ChatGPT python API and tinker learning for enhanced creativity and problem-solving skills. In: Huang, Y.-M., Rocha, T. (eds.) Innovative Technologies and Learning: 6th International Conference, ICITL 2023, Porto, Portugal, August 28–30, 2023, Proceedings, pp. 531–541. Springer Nature Switzerland, Cham (2023). https://doi.org/10.1007/978-3-031-40113-8_52

NLP-Assisted Educational Memory Game Experiment Viktória Burkus(B) , Attila Kárpáti , and László Szécsi Department of Control Engineering and Information Technology, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Műegyetem rkp. 3., 1111 Budapest, Hungary [email protected]

Abstract. Memory card games are well-known for everyone. Their main goal is to find matching pairs among flipped cards by simultaneously turning two cards up. In each round information obtained and memorized from the previous round is used to predict matching pair positions. Whilst playing, a player’s thinking, memory, concentration, and attention skills can be improved, therefore, incorporating it into the learning process may be beneficial to the learner. In this research, we present a method for learning terminology from a scientific context by playing an AI-generated memory card game. We employ a general-purpose conversation chatbot, ChatGPT to generate the keyword-description pairs from a given scientific text. The primary purpose of our study is to evaluate the outcomes, with respect to their scientific accuracy and educational value. Consequently, we present a straightforward approach to constructing a game that facilitates the acquisition of knowledge by students in an enjoyable and playful way.

Keywords: Natural Language Processing Memory cards

1

· Educational chatbot ·

Introduction

Due to technological improvements, it is possible to effortlessly communicate with artificial intelligence. Futuristic visions that were only a dream are now becoming a reality thanks to the rapid development of Natural Language Processing (NLP) techniques. NLP aims to combine artificial intelligence and linguistics together in order to understand and process natural language, as well as generate natural language as output. Not only they are able to process input text and generate coherent, meaningful, natural text as output they also comprehend the meaning behind different parts of a text, as well as its connections to other parts and underlying semantics which is why their range of possible applications have expanded greatly. They can easily help in education on both supporting teaching and learning activities: they can check for syntactical and grammar mistakes on a given text, make short summaries of a given text, give c The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 59–69, 2023. https://doi.org/10.1007/978-3-031-42134-1_6

60

V. Burkus et al.

guidance to help learning different fields (for example mathematics, physics, and languages), or generate lesson notes and assessments automatically. However, it should be noted that human assessment is still necessary for its application, especially in education, as they might reply with inaccurate, misleading, or outof-date information. In this research we employ a general-purpose conversation chatbot, ChatGPT, to generate a memory game. A memory game is not an unfamiliar term for most people, a simple and effective way to train the mind. In order to be successful, one needs to memorize the figures on the flipped cards as well as their positions, thus, a participant’s thinking, memory, concentration, and attention skills can be improved. For learning a subject, cards can either contain a term or a description, or a term and a figure that indicates the corresponding term. In this way, it can be seen as a teaching tool that combines flash cards and memory cards together to make it easier for students to memorize key concepts of a given field. With AI, we can easily generate keyword-description pairs from any given scientific text. Our aim is not to evaluate the effectiveness of the memory card game as a teaching tool, but to analyse the accuracy and clarity of the pairing, and the mistakes the chatbot makes. These factors determine the usability of AI as a tool for teachers to create gamified teaching material.

2

Previous Work

One of the first significant milestones in the field can be attributed to the wellknown Alan Turing. His work, titled “Computing Machinery and Intelligence” [1] dates back to the 1950s s and investigates the question of whether machines are capable of thinking. The term “Turing Test” (also known as the “Imitation Game”) was introduced with the primary goal of testing if a machine is intelligent enough that its behaviour is difficult to distinguish from that of a human. The described “Imitation Game” starts with two participants who communicate with the third participant, the interrogator, in a way that the tones of voices have no influence on the outcome, preferably in writing or typing. Each participant has a class, or label, and the interrogator’s main objective is to correctly classify or label the participants. One participant’s goal is to deceive the interrogator in order to be identified in the other class, by convincing the interrogator that it belongs to another class, whereas the other participant’s objective is to be identified with a correct label. In Turing’s perspective, if a machine plays the Imitation Game and then passes with a human classification then it can be interpreted as proof that machines can “think”. Therefore, we can assess the significance of NPL methods. In order to trick the interrogator and pass the test, a machine needs to have outstanding natural language processing abilities. It must have a thorough comprehension of the input, the underlying reason, and general semantics. In 2014 a chatbot software called “Eugene Goostman” successfully misled the 33 percent of participating interrogators, therefore officially becoming the first artificial intelligence passing the Turing test. However, following its triumphant victory, a heated debate arose whether it genuinely passed the test or whether it was just smart software tricks used to hide its flaws and fool the judges [2].

NLP-Assisted Educational Memory Game Experiment

2.1

61

Large Language Models

In 2018, OpenAI released the Generative Pre-trained Transformer (GPT) [3], a large language model capable of processing and generating responses to incoming texts and assist in various language-related activities. Since then, many generations have appeared, including the fourth version that was released in March 2023. GPT models were pretrained using a vast corpus of data and were calibrated with reinforcement learning. In 2019, Google AI introduced BERT [4] that was pretrained on deep bidirectional representations from unlabeled text. With an improved pretraining method, RoBERTa [5] by Facebook AI in 2019 brings a solution which performs significantly better than BERT. They trained the model longer as well on longer sentences, used bigger batches, removed the next sentence prediction objective and also used dynamic masking. After introducing BERT, Google AI launched a method in 2020, namely XLNet [6], a generalized autoregressive pretraining method that aims to overcome the limitations of BERT. T5 was also introduced in 2020. The name T5 [7] pertains to “Text-to-Text Transfer Transformer”. This particular model can be used to perform various tasks such as abstractive summarization, which in case of encoder-only models is not applicable. In addition to GPT, OpenAI introduced InstructGPT [8]. With the help of fine-tuning the process with human feedback, they aim to show improvements in truthfulness and reductions in toxic output generations. A sibling model to InstructGPT, ChatGPT [3] was fine-tuned from a model in the GPT-3.5 series and was trained using RLHF (Reinforcement Learning from Human Feedback) in 2022. BLOOM [9], a decoder-only transformer multilingual language model was introduced in 2023 by BigScience. 2.2

Applications for Education Purposes

The potential implications, and restrictions of artificial intelligence in education are outlined in a number of papers: Hwang et al. [10] provide a great overview of opportunities and challenges of chatbots in educational sector. Holmes et al. [11] review the role of artificial intelligence in education. Tack et al. [12] test whether generative models, like GPT-3 can be good artificial teachers. Zhai [13] provide a study on the potential impact of ChatGPT. Hyangeun et al. [14] focus on reviewing conversational AIs in language educations. A study by Kung et al. [15] evaluates the accuracy of ChatGPT by testing its performance on questions from the USMLE. Abdelghani et al. [16] examine the advances in the natural language processing field and explore ways using GPT-3 to help children ask more and deeper questions in pedagogical context. Jia et al. [17] use BERT models to evaluate peer assessments by students. Language models also can be used to generate quizzes, flashcards exam questions: Nwafor et al. [18] use NLP for multiple-choice question generation. They use AI to extract keywords in a given material and were compared with manually selected keywords by the teacher. Dijkstra et al. [19] introduce a generator based on GPT-3 model, named EduQuiz to generate quizzes for any educational text. Bhat et al. [20] use T5 model to present a pipeline for generating and evaluating questions from text materials.

62

3

V. Burkus et al.

Outline of the System

The development of a memory card game intended for educational purposes, specifically aimed towards scientific terminology involves a series of distinct steps. Our approach can be seen in Fig. 1

Fig. 1. Data flow in our system. Phases that an AI could replace are highlighted in orange. Phases that can easily automated without AI are highlighted in green. Manual execution is denoted by the color blue. The figure’s upper side denotes the assembly portion, while the figure’s bottom portion indicates the actual playing part.

1. Collect text materials about subject: The teacher assembles pertinent data, such as information from reading material and scientific sources, that encapsulates the learning objective for a particular subject. The obtained text has to contain sufficient keywords to produce card game. The placement of key terms within the text should enable readers to find or assume their meanings. This step can be substituted by AI, as the large language models were trained with a considerable amount of teaching data and can produce textual outputs about a given subject with ease. 2. Identify key terms: The teacher identifies terminologies within the acquired text. In our research, this phase has been substituted with artificial intelligence. After uploading the materials, the artificial intelligence proceeds to identify terminology within the text.

NLP-Assisted Educational Memory Game Experiment

63

3. Gather descriptions for terms from text: Upon obtaining the keywords, teacher must find descriptions for each term while refraining from incorporating the term itself. The description must be concise, direct and accurate. This process can be quite time-consuming. Moreover, it is possible that certain keywords may lack definitions, or their respective definition may be obscure or concealed within the body of the text. Also, it is important to remove keywords lacking accompanying descriptions, or not relevant to the context through filtering procedures. The result of this step is a filtered list of termdescription pairs. In our system, this step is fully automated by AI. 4. Validate term-description pairs gathered from text: The teacher validates the output of the Gather descriptions for terms from text step. The description must correspond to its keyword. In order to ensure ease of recall, the description must have a clear and concise nature, while abstaining from embedding the keyword. As the AI-generated pairings may incorporate biased information, it must be performed manually. The output of this step indicates the success of the pairings. If it was not successful, then Identify key terms comes again. Else, the Assemble memory card game step follows. 5. Assemble memory card game: This step also can be fully automated, where the pairings turn into an actual game setup. Cards can be generated online with various tools. 6. Play game: Student plays with the generated game: One side of the card either has a keyword or a description, the other side of the card is blank or uniform for all cards. The cards are placed face down and spread out on a desk. It is a turn-based game, therefore, in each round a participant flips only two card. If the keyword matches with the description (Validate given key-description pair), then the player wins a point and can flip two more cards. If the flipped cards are not a match, then the next participant gets to play. The game ends when all pairs are found. 7. Validate given key-description pair: External source confirmation of the success of pairings during the game is needed. In case of identical figures this is trivial, but in case of key-description pairs, some sort of verification method is needed in the process. For example, a teacher verifies the output. This step can easily be automated without AI. If there is no card left, then the game ends. Else Play game is performed again by the next player.

4

Introducing NLP in the Process, a Case Study

Our solution is based on ChatGPT, which provides an effective and straightforward way to interact with an artificial intelligence. Our research also aims to assess if ChatGPT can generate a game that can be used for education purposes. For this case study, we iterate through the generation steps in Sect. 3. 4.1

Collect Text Materials About Subject

For the purpose of illustrating the procedure, the ensuing sentences were extracted from the Graphics Pipeline page of Wikipedia [21]. This particular

64

V. Burkus et al.

material presents many advantages with regard to our given context, consisting of various keyword-description pairs and a brief explanation of fundamental terminologies within the domain of Computer Graphics. Text: “A triangle is the most common geometric primitive of computer graphics. It is defined by its three vertices and a normal vector - the normal vector serves to indicate the front face of the triangle and is a vector that is perpendicular to the surface. The triangle may be provided with a color or with a texture (image “glued” on top of it). Triangles are preferred over rectangles because their three points always exist in a single plane. The world coordinate system is the coordinate system in which the virtual world is created. This should meet a few conditions for the following mathematics to be easily applicable: It must be a rectangular Cartesian coordinate system in which all axes are equally scaled. How the unit of the coordinate system is defined, is left to the developer. Whether, therefore, the unit vector of the system is to correspond in reality to one meter or an Ångström depends on the application.” From now on, we refer to this text as “Text”. 4.2

Identify Key Terms

Though manually searching for keywords within a given text may appear to be an uncomplicated task, it is also a laborious process that requires a significant amount of time. The subsequent input was given within the dialogue box of ChatGPT: Input: “I give you a text from Wikipedia. Can you please generate keyworddescription pairs from only this text? Please only include keywords that are present in the text. The text:“Text”.” Output: “Triangle, Geometric Primitive, Vertices, Normal Vector, Front Face, Color, Texture, Rectangles, World Coordinate System, Cartesian Coordinate System, Axes, Unit Vector, Unit of the Coordinate System, Developer, Mathematics” Although, ChatGPT did not generate descriptions at first, these keywords are in the text. There are also some keywords in the list that are not relevant or not specific to the subject: Mathematics, Developer. 4.3

Gather Descriptions for Terms from Text

Input: “Please generate the descriptions to the keywords too.” Output: – Keyword: Triangle; Description: “A geometric primitive commonly used in computer graphics, defined by its three vertices and a normal vector that serves to indicate the front face of the triangle and is perpendicular to the surface.” – Keyword: Vertex; Description: “A point in space that defines the corner of a polygon or a triangle in a 3D model.” – Keyword: Normal vector; Description: “A vector that is perpendicular to the surface of a polygon or a triangle, used to indicate the front face of the shape.”

NLP-Assisted Educational Memory Game Experiment

65

– Keyword: Color; Description: “A property that defines the appearance of a shape in terms of its hue, saturation, and brightness.” – Keyword: Texture; Description: “An image that is “glued” onto the surface of a polygon or a triangle to add detail and realism to a 3D model.” – Keyword: World coordinate system; Description: “The coordinate system in which a virtual world is created, usually a rectangular Cartesian coordinate system in which all axes are equally scaled.” – Keyword: Cartesian coordinate system; Description: “A coordinate system that uses three perpendicular axes to define a point in space.” – Keyword: Unit vector; Description: “A vector with a length of 1 that is used to define the direction of a line or a ray.” – Keyword: Meter; Description: “The basic unit of length in the International System of Units (SI), used to measure distance.” – Keyword: Ångström; Description: “A unit of length equal to 0.1 nm, commonly used in the measurement of atomic distances.” 4.4

Validate Term-Description Pairs Gathered from Text

It is clear that elements that were not part of the Text also appear between the received keyword-description pairs (Vertex). Also, some descriptions are not part of the text (Meter, Ångström) Input: Keyword-Description pairs. Output: False. 4.5

Identify Key Terms and Gather Descriptions for Terms from Text

Input: “Now generate the pairs with the following rules: generate the keyworddescription pairs from the given text, do not include outside knowledge and the description must not contain the keyword. Please only consider this text: Text” Output: The output of this step can be found in Sect. 4.6. 4.6

Validate Term-Description Pairs Gathered from Text

We conducted an examination of the correlations among the definitions of keywords from a professional perspective. Input: Keyword-Description pairs. Output: True. Evaluation: – Keyword: Triangle; Description: “A geometric primitive in computer graphics that is defined by its three vertices and a normal vector.” Evaluation: Slightly misleading. Triangles do have normal vectors, but those are not independent of their vertices. The original text immediately adds that the normal vector is relevant when front and back faces must be distinguished. This context is missing here.

66

V. Burkus et al.

– Keyword: Normal Vector; Description: “A vector that indicates the front face of a triangle and is perpendicular to its surface.” Evaluation: Correct. The term has a wider geometric meaning, but it is expected that it is not referred to here. – Keyword: Color; Description: “A feature that can be provided to a triangle, allowing it to be rendered with a specific hue.” Evaluation: Correct. – Keyword: Texture; Description: “An image that can be “glued” on top of a triangle, providing it with a specific pattern or appearance.” Evaluation: Correct. – Keyword: Rectangles; Description: “Geometric shapes that are not preferred in computer graphics because they require more points to define than triangles.” Evaluation: Too ambiguous. While the statement is true, it applies to any number of shapes that are not triangles, not only rectangles. – Keyword: World Coordinate System; Description: “The coordinate system used in creating a virtual world that must be a rectangular Cartesian coordinate system with equally scaled axes.” Evaluation: Somewhat ambiguous. The statement cites the text correctly, but the model coordinate system could also be construed to fit the definition. The main feature of the world coordinate system is that all object poses are specified in it. This is implicit in the text, and omitted from the Description. – Keyword: Cartesian Coordinate System; Description: “A coordinate system that defines a point in space using a combination of three numbers that represent its distance from three mutually perpendicular planes.” Evaluation: Correct in a 3D context. – Keyword: Unit Vector; Description: “A vector that has a magnitude of one and is used to define the scale of a coordinate system.” Evaluation: Correct. The formulation can be used to define would be better. – Keyword: Rectangular; Description: “A term used to describe a shape or coordinate system with right angles.” Evaluation: Correct. – Keyword: Cartesian; Description: “A term used to describe a coordinate system that uses three mutually perpendicular planes to define a point in space.” Evaluation: Correct, but it omits the notion of distance from the planes, which the Description for Keyword Cartesian Coordinate System included, making it less precise.

5

Discussion

The performed evaluation has yielded several conclusions. First, the description of a term may potentially be deceptive due to the exclusion of certain details present in the source material. Moreover, the information present in the output may be unclear or formulated imprecisely, which further compounds the difficulty in comprehending the intended meaning. It can be stated that ChatGPT has the capability to produce pairs of terms and their corresponding descriptions derived from a specified text. Nevertheless, these pairings can be considered as a preliminary step towards subsequent analysis and fixes performed by

NLP-Assisted Educational Memory Game Experiment

67

the teacher. The following steps can be deemed straightforward after the pairings have been adjusted. The development of online games wherein interested students can engage in competitive play using the matched pairings is a simple process. The appropriateness of the cards selected by the student can be quickly verified by teaching the specified pairings to a program beforehand. 5.1

Advantages and Disadvantages

In summary, the benefits of our approach: – The majority of time-consuming tasks (processing scientific text, putting together terminology and descriptions, assembling game, etc.) can be automated. – The output is not perfect, but the effort to correct the generated mistakes is negligible compared to creating everything manually. – The generated keywords were indeed among the most important terms in a given input. – Some of the descriptions contained misunderstandable phrases, but otherwise they were correct. – The process is easy to replicate. The disadvantages: – Reviewing of the created pairing is necessary. – Terms that were not in the input text may be included in the created keywords. – The generated descriptions may rely on scientific context that is not present in the original text, making them difficult to understand for students who are only becoming familiar with this subject area. During our experimenting we found that ChatGPT is a very versatile tool for a wide range of tasks. As a result of its extensive data-based training, it can provide both legitimate and in context, less valid answers as well. We needed to clarify the tasks in laying down rules. Evaluating the outcomes was a crucial step in the process. In overall, the approach presented in this study offers educators a fundamental framework for developing a game-based tool using the learning material, enabling students to engage in a fun and playful learning experience while comprehensively absorbing the subject matter. Acknowledgements. Disclaimer: This research was supported by the e-DIPLOMA project (101061424), funded by the European Union. The views expressed are those of the authors alone and do not reflect those of the European Union or the European Research Agency (REA). Neither the European Union nor the sponsoring authority can be held responsible for them.

68

V. Burkus et al.

References 1. Turing, A. M.: Computing machinery and intelligence. Mind LIX(236), 433–460 (1950). https://doi.org/10.1093/mind/LIX.236.433 2. Neufeld, E., Finnestad, S.: In defense of the turing test. AI Soc. 35, 819–827 (2020). https://doi.org/10.1007/s00146-020-00946-8 3. Kasneci, E., et al.: ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education. EdArXiv (2023). https://doi.org/10.35542/osf.io/ 5er8f 4. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv (2019). https://doi.org/ 10.48550/arXiv.1810.04805 5. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv (2019). https://doi.org/10.48550/arXiv.1907.11692 6. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. arXiv (2020) https://doi.org/10.48550/arXiv.1906.08237 7. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv (2020). https://doi.org/10.48550/arXiv.1910.10683 8. Ouyang, L., et al.: Training language models to follow instructions with human feedback. arXiv (2022). https://doi.org/10.48550/arXiv.2203.02155 9. Scao, T.L., et al.: BLOOM: a 176B-parameter open-access multilingual language model. arXiv (2023). https://doi.org/10.48550/arXiv.2211.05100 10. Hwang, G.-J., Chang, C.-Y.: A review of opportunities and challenges of chatbots in education. Interact. Learn. Environ. (2021). https://doi.org/10.1080/10494820. 2021.1952615 11. Holmes, W., Bialik, M., Fadel, C.: Artificial intelligence in education. Promise and Implications for Teaching and Learning (2019) 12. Tack, A., Piech, C.: The AI teacher test: measuring the pedagogical ability of blender and GPT-3 in educational dialogues (2022). https://doi.org/10.48550/ arXiv.2205.07540 13. Zhai, X.: ChatGPT user experience: implications for education (2022). https:// doi.org/10.2139/ssrn.4312418 14. Hyangeun, J., Insook, H., Yujung, K.: A systematic review of conversational AI in language education: focusing on the collaboration with human teachers. J. Res. Technol. Educ. 55(1), 48–63 (2023). https://doi.org/10.1080/15391523.2022. 2142873 15. Kung, T.H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., et al.: Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Dig. Health 2(2), e0000198 (2023). https://doi.org/10.1371/journal.pdig.0000198 16. Abdelghani, R., Wang, Y., Yuan, X., Wang, T., Sauz’eon, H., Oudeyer, P.: GPT3-driven pedagogical agents for training children’s curious question-asking skills. arXiv (2023). https://doi.org/10.48550/arXiv.2211.14228 17. Jia, Q., Cui, J., Xiao, Y., Liu, C., Rashid, P., Gehringer, E.F.: ALL-IN-ONE: multi-task learning BERT models for evaluating peer assessments. arXiv (2021). https://doi.org/10.48550/arXiv.2110.03895 18. Nwafor, C.A., Onyenwe, I.E.: An automated multiple-choice question generation using natural language processing techniques. Int. J. Nat. Lang. Comput. 10, 1–10 (2021). https://doi.org/10.48550/arXiv.2103.14757

NLP-Assisted Educational Memory Game Experiment

69

19. Dijkstra, R., Genç, Z., Kayal, S., Kamps, J.: Reading comprehension quiz generation using generative pre-trained transformers. In: The 23th International Conference on Artificial Intelligence in Education (2022) 20. Bhat, S., Nguyen, H.A., Moore, S., Stamper, J., Sakr, M., Nyberg, E.: Towards automated generation and evaluation of questions in educational domains. In: Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society, pp. 701–704 (2022). https://doi.org/ 10.5281/zenodo.6853085 21. Wikipedia: Graphics pipeline. https://en.wikipedia.org/wiki/Graphics_pipeline. Accessed 12 Apr 2023

Datasets for Artificial Intelligence in Education: The Case of Children with Neurodevelopmental Disorders Marcos L. P. Bueno(B) and Serge Thill Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Nijmegen, The Netherlands {marcos.depaulabueno,serge.thill}@donders.ru.nl

Abstract. Artificial intelligence has shown promise for supporting children with neurodevelopmental disorders (NDDs) in educational settings. For such vulnerable population, aspects such as emotion, communication, and motivation are very relevant, but also challenging to be modeled. In this work, we focus on the machine learning technology used in such scenarios, in particular the characteristics of datasets used for model training. We do this by analyzing recent papers on children with NDDs. This will give insight into existing trade-offs, such as data annotation involved in data collection, as well as automation aspects. We also analyze opportunities offered by the functionalities of ML models trained on such datasets. In addition, we point out limitations and future challenges to help advance the area. Keywords: machine learning · data annotation · neurodevelopmental disorders · children · education · human-computer interaction

1

Introduction

Recent years saw an emergence of artificial intelligence (AI) in all kinds of educational settings [4,5]. Processes such as teacher-student communication, interaction with peers, and learning of multiple educational subjects are central in education. For children with neurodevelopmental disorders (NDDs), such as autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD), difficulties around these processes are greatly amplified. This is because such students have impairments in several cognitive, emotional, and motivational factors [8]; for example, many autistic people have impairments in social skills, emotion regulation, and attention. Moreover, children often express themselves in different ways using creativity, nonverbal communication, etc. This makes the modeling by AI of such a vulnerable population a great challenge. AI technology, in particular machine learning (ML), has been notably helpful for children with NDDs in educational settings. This includes, e.g., automatic emotion recognition for personalized educational plans [10], recommendation of c The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 70–77, 2023. https://doi.org/10.1007/978-3-031-42134-1_7

Datasets for Artificial Intelligence in Education

71

teacher-student communication strategy [8], and assessment and improvement of skills via gamified tasks [7]. To that end, ML models have been built from different kinds of data, such as subjective, behavioral and physiological data. On the one hand, dealing with data of such different nature is a challenging multimodal problem, from data collection to model building. On the other hand, this allows one to understand the student’s cognitive, metacognitive, emotional, and motivational state [11]. This paper analyzes the opportunities offered by recent ML-based approaches for NDD children. We focus on two aspects: (i) the datasets used in the studies, and (ii) the capabilities of the ML models trained from such data. Typically, each study relies on different data, so we consider multiple aspects: what data is collected, who collects the data, how data annotation is used, the size of the data, and so on. We do this analysis based on descriptions of the datasets in the selected papers since actual access to these private datasets was not possible. While there exist reviews on AI for NDDs in education [1,6] that focus mostly on what the AI models cover (e.g. social skills) and the effects on NDD children, our focus is instead on how such AI models are constructed. Analyses such as ours exist in the context of neurotypical students in specific tasks, such as prediction of student performance and failure aiming to identify the most predictive features and the types of ML models used [3,14]. To our knowledge, however, there has been no review of ML datasets and models in the context of NDDs. The analysis of datasets is crucial to understand common trade-offs, such as the human effort employed in annotating the data and which processes can be automated. Moreover, our analysis will also indicate how practical different ML approaches can be, which gives insight into transferring such ideas to a new setting. In the end, we discuss future challenges to help advance the area. Since AI for NDD children in schools is still a somewhat emerging area, we extend our scope to studies that educational activities possibly outside a school setting (e.g. at home). The rest of this paper is organized as follows. In Sect. 2 we discuss the data characteristics of the selected studies, while the functionalities of ML models are discussed in Sect. 3. We discuss future challenges in Sect. 4, and conclude the paper in Sect. 5.

2

Analysis of Datasets

In this section, we analyze several characteristics of datasets used in recent studies on children with NDDs. We selected these papers since they are recent and cover different aspects of NDDs. We discuss the data collection and how humans were involved in this, e.g. by annotating variables. Note that our analysis is based on the data descriptions available in the papers, not on the actual datasets since they are private. We make a distinction in the type of ML paradigm used, which in this case is either a supervised learning or reinforcement learning approach. An overview of the datasets is shown in Table 1, while the data annotation and automation aspects are summarized in Table 2.

72

2.1

M. L. P. Bueno and S. Thill

Supervised Learning Studies

The goal of supervised learning is to predict a response variable given a set of predictors. It is a quite common ML approach given its conceptual simplicity, with well-established evaluation metrics. Normally, supervised learning starts with data collection followed by model training. After this, the model can be deployed, e.g. in a classroom. Table 1. Overview of studies and datasets. Type refers to the variable type. Supervised learning Study

NDD

Predictors (type)

Response (type)

Communication strategy [8]

ASD

Student’s individual characteristics and emotional state, teaching objective and type, context for teaching, teacher’s communication style (Categorical)

Student’s action (Categorical)

Emotion recognition [10]

Various

EEG signals (Time-series)

Student’s emotional state (Categorical)

Task switching [7]

ASD

Speech (Time-series)

Student’s performance in task (Categorical)

Reinforcement learning Study

NDD

State

Action

Motivator selection [13]

ASD

Antecedent event, context, time, challenging behavior and its cause, last unsuccessful motivator, motivator stats (Categorical)

Motivator (Categorical)

Communication Strategy. In [8] the goal is to model teacher-student communication strategies for autistic children in schools. Educational activities such as academic, social and pedagogic ones are considered, which can be done in small groups, individually, etc., and take place inside or outside the classroom. The main ML goal is to predict whether the student will write a sentence when asked to (which can be a full, partial, or no response). Several predictors are collected such as the teacher’s communication style, type of teaching, and the student’s emotional state during the interaction. As shown in Table 2, the student’s emotions and actions were annotated by humans in every interaction. Emotion Recognition. The work in [10] targets children with disabilities in schools, and aims to identify the student’s emotional state (sad, scared, happy, or calm). The approach also suggests an interface for instructors to create a management plan, if necessary. Electroencephalogram signals (EEG) of the student’s brain activity are recorded as predictors of the student’s emotional state.

Datasets for Artificial Intelligence in Education

73

Table 2. Data collection, including tasks by humans, involved in building the datasets. N: dataset size; n: number of subjects. The indicated human effort was explicitly described in the studies and we avoided making inferences when that was not the case (the same holds for automation aspects). Supervised learning Study

N

n

Human effort

Automation

Communication strategy [8]

5460 interactions

7

– Annotation of student’s response and emotions

–

Emotion recognition [10]

180 images per subject

5

– Dataset that links images to emotions (angry, sad, happy, calm)

Pre-existing dataset

Task switching [7]

12 sessions per 7 subject

–Use of Gamified tasks for mea- Automated suring competency speech-to-text – Annotation of student’s conversion performance as positive or negative

Reinforcement learning Motivator selection [13]

1231 steps 12 (598 episodes)

– Annotation of child’s behavior – and its cause – Annotation of child’s response (reward)

A ML model is trained to link EEGs and emotions. To this end, a dataset that associates images with emotions is used, as shown in Table 2. In this study, a pre-existing dataset was used (the international affective picture system (IAPS)), which results in less human effort by re-using data. Task Switching. In [7] task switching for computational thinking is considered for autistic adolescents. The goal is to identify the student’s performance (positive/negative) in four gamified tasks for assessing task switching. As predictors, they recorded the student’s speech (answers and utterances) in the tasks. As Table 2 indicates, the dataset instances were annotated as either positive or negative manually, by identifying examples of positive/negative performance in each gamified task [7]. However, the transcription of speech-to-text was automated. 2.2

Reinforcement Learning Studies

In reinforcement learning (RL) we model agents acting on an environment. The environment includes relevant information about the current state, e.g., student characteristics. As opposed to supervised learning, RL has the notion of action. The effect of an action on an environment is called a reward. Over time the agent learns how to better act, which is challenging since the actual effect of an action is uncertain.

74

M. L. P. Bueno and S. Thill

Motivator Selection. The study [13] takes place in the context o individual education programs for autistic children. The goal is to select the appropriate motivator when a disruptive behavior occurs. A RL approach is used, which models the dynamics of state–action–reward. In this case, the state represents the child’s behavior, an action represents a motivator (edible, sensory, activity, token, social, or choice), and a reward indicates the result of using that motivator. The RL system does not act autonomously here; instead, it supports the caregiver (e.g. a teacher or a therapist) by recommending a motivator when a behavior is identified by the caregiver. This ultimately puts the caregivers in control of which action to take. As Table 2 indicates, the caregiver annotates multiple pieces of information: the agent’s state (such as the student’s behavior and its cause) and the reward obtained by using the selected motivator.

3

Machine Learning Models

In this section, we discuss aspects of ML models of the previous studies, such as model training and the functionalities they provide. Table 3 provides a summary. The study on communication strategy [8] suggests that the main ML models are trained by using the educational interactions of all students together. Later on, they try to learn a model by adding previous interactions (i.e. autoregressions), indicating that for some types of ML models this can be beneficial. While the main models predict the student’s response, the paper also shows an alternative way of using the models to instead provide recommendations on the communication style (originally a predictor variable). To train a ML model for emotion recognition [10] , an image-emotion dataset is used, where each image is shown to the student. In this dataset, it is already known which emotion corresponds to the image, which allows the ML model to capture the EEG patterns for different emotions. The study suggests that a specific model is created for each student. After the model is trained, it can be used for predicting the emotional state of the student based on the observed EEG patterns. However, the performance of the models is unclear. In the task switching paper [7], speech-to-text is used on each gamified task, and datasets specific to each gamified task are created. After transformation to text, feature extraction is done to identify relevant textual features, such as n-grams, line length, etc., which allows for training ML models. There are two phases in the ML implementation: baseline sessions, in which models are trained; and intervention sessions, where the models are used for making predictions in real-time while the student does the tasks at home. As opposed to the other studies, the ML model for motivator selection [13] is built while the therapy intervention takes place. One major concern in this setting is ethical constraints. This means that the exploration by the RL agent is more limited (e.g. compared to game playing), which puts constraints on exploring all possible states, experiment repetition, among others. In practice, this means that the available data for the agent to learn will be limited. Moreover,

Datasets for Artificial Intelligence in Education

75

Table 3. Functionalities and details of machine learning models. Type models are RF: random forest; LR: logistic regression; GP: Gaussian process; SVM: support vector machine; kNN: k-nearest neighbors. Supervised learning Study

Functionalities

Type models

Best performance

Communication strategy [8]

– Prediction of student’s response RF, LR, GP F1 measure: (action) 0.757 (GP) – Recommendation of teacher-student communication strategy

Emotion recognition [10]

– Prediction of student’s emotional state

SVM, kNN

Unknown

Task switching [7]

– Prediction of student’s performance (positive/negative)

SVM

Accuracy: 0.761

RL

89.6% motivators worked

Reinforcement learning Motivator selection [13]

– Recommendation of a motivator

this RL system only provides suggestions as to which motivator to use since the final decision is made by the caregiver, who can also reject the suggestion. The performance of Table 3 indicates that when a motivator was selected, this led to a positive reward in 89.6% of the cases. The authors also considered when motivators were selected by the caregivers without the RL agent, which resulted in a performance of 45.5%.

4

Challenges

In this section we point out existing limitations and future challenges that can help advance the area, inspired by the studies analyzed. We provide considerations about the data and ML models. Datasets. One fundamental challenge for building useful ML models revolves around building datasets. For children with NDDs in educational contexts, we saw that datasets are built in different ways, where humans play a key role. This varies significantly, from manual labelling of data records (e.g. school activities [8]) to the usage of pre-existing datasets of image-emotions [10]. On the other hand, this indicates that processes such as data re-use and automation (e.g. the speech-to-text in [7]) are already in place. Processes such as data re-use and automation come with their own issues. Using data created in different situations raises several issues: privacy and ethical concerns on the data level (to a lesser extent if the data is public), and whether meaningful and accurate models can be created. On the other hand, proper data

76

M. L. P. Bueno and S. Thill

re-use and automation can not only reduce the (human) burden of creating a ML dataset, but also potentially enable creating even larger datasets. Another dataset issue is the limited size. Many studies rely on reasonablesized datasets, however based on a small or very small number of subjects. For example, a child can participate in many different activities over time [8]. This attempts to makes up for the small number of participants, but can still raise concerns on the representativeness of the sample, since records from the same subject are likely not independent from each other. Addressing this issue is very important to improve the effectiveness of ML models. ML Models. The deployment of ML models for vulnerable populations in educational settings is complex. In cases of data paucity, there can exist differences in the population where the ML model was trained on compared to the population where deployment happens (i.e. where the model is used). One example is when model training is done using pilot data and deployed on a different population (e.g. children and adolescents [7]). Such heterogeneity might lead to a deterioration of performance of ML models. This problem is complex to handle as it involves the study design. A related issue is when real-time data of audio, video, brain signals, and so on are collected and a shift in the data characteristics occurs at some point. While this can be problematic as well, in the ML community this problem is known as concept drift, for which several algorithms exist [9]. In terms of performance, the ML models based on supervised learning all achieved performance below 0.8 on different metrics, such as accuracy and F1 measure. While this is a reasonable performance, it is still far from perfect performance. In that sense, it is unclear whether the current performance is sufficient for the safe deployment of such approaches in classrooms involving students with NDDs. We also point out the feasibility of certain approaches in the context of children with NDDs. For example, deploying the ML models that use technologies such as EEGs [10], which might be intrusive for autistic people in classrooms, due to the sensory difficulties that many of them have. Although a positive usability evaluation was obtained when using EEGs, this was based on only a few subjects.

5

Conclusion

In this paper, we inspected the opportunities offered by ML models for children with NDDs in educational settings. In particular, we examined the datasets used in a selection of studies in terms of what data was collected, by whom and where, with a special focus on whether humans were involved (and to what extent). We saw that datasets are built in very different ways, but they all have the same issue of limited size. We see multiple possibilities for future work. We plan to include more studies, as well as extend the scope of our study by including, e.g., (affective) intelligent tutoring systems and systems based on unsupervised learning. Another direction

Datasets for Artificial Intelligence in Education

77

is to perform evaluations based on the datasets themselves, such as data quality [12] and readiness levels [2]. This would normally require access to the data itself, which seems more feasible for public datasets. Acknowledgements. This work is supported by the Horizon Europe project EMPOWER (GA No. 101060918).

References 1. Barua, P.D., et al.: Artificial intelligence enabled personalised assistive tools to enhance education of children with neurodevelopmental disorders - a review. Int. J. Environ. Res. Pub. Health 19(3), 1192 (2022) 2. Castelijns, L.A., Maas, Y., Vanschoren, J.: The ABC of data: a classifying framework for data readiness. In: Cellier, P., Driessens, K. (eds.) ECML PKDD 2019. CCIS, vol. 1167, pp. 3–16. Springer, Cham (2020). https://doi.org/10.1007/978-3030-43823-4_1 3. Chakrapani, P., Chitradevi, D.: Academic performance prediction using machine learning: a comprehensive & systematic review. In: 2022 International Conference on Electronic Systems and Intelligent Computing (ICESIC), pp. 335–340 (2022) 4. Chassignol, M., Khoroshavin, A., Klimova, A., Bilyatdinova, A.: Artificial intelligence trends in education: a narrative overview. Proc. Comp. Sci. 136, 16–24 (2018) 5. Chen, L., Chen, P., Lin, Z.: Artificial intelligence in education: a review. IEEE Access 8, 75264–75278 (2020) 6. Hopcan, S., Polat, E., Ozturk, M.E., Ozturk, L.: Artificial intelligence in special education: a systematic review. Interactive Learning Environments, pp. 1–19 (2022) 7. Ke, F., Moon, J., Sokolikj, Z.: Tracking representational flexibility development through speech data mining. In: 2020 IEEE Frontiers in Education Conference (FIE), pp. 1–4. IEEE (2020) 8. Lampos, V., Mintz, J., Qu, X.: An artificial intelligence approach for selecting effective teacher communication strategies in autism education. npj Sci. Learn. 6(1), 25 (2021). https://doi.org/10.1038/s41539-021-00102-x 9. Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31(12), 2346–2363 (2018) 10. Mehmood, R.M., Lee, H.J.: Towards building a computer aided education system for special students using wearable sensor technologies. Sensors 17(2), 317 (2017) 11. Molenaar, I., de Mooij, S., Azevedo, R., Bannert, M., Järvelä, S., Gašević, D.: Measuring self-regulated learning and the role of AI: five years of research using multimodal multichannel data. Comput. Hum. Behav. 139, 107540 (2023) 12. Pipino, L.L., Lee, Y.W., Wang, R.Y.: Data quality assessment. Commun. ACM 45(4), 211–218 (2002) 13. Siyam, N., Abdallah, S.: Toward automatic motivator selection for autism behavior intervention therapy. Universal Access in the Information Society, pp. 1–23 (2022) 14. Veloso, B., Barbosa, M.A., Faria, H., Marcondes, F.S., Durães, D., Novais, P.: A systematic review on student failure prediction. In: Kubincová, Z., Melonio, A., Durães, D., Rua Carneiro, D., Rizvi, M., Lancia, L. (eds.) Methodologies and Intelligent Systems for Technology Enhanced Learning, Workshops, 12th International Conference. MIS4TEL 2022. LNCS, vol. 538, pp. 43–52. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-20257-5_5

Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention in Children with Neurodevelopmental Disorders Cristina Costescu1 , Carmen David1 , Adrian Ros, an1 , Paula Ferreira2 , Aristides Ferreira3 , Lucia Vera4 , and Gerardo Herrera4(B) 1 Special Education Department, Babes-Bolyai University, Cluj-Napoca, Romania , 2 CICPSI, Faculty of Psychology, University of Lisbon, Lisbon, Portugal 3 Business Research Unit, Iscte - Instituto Universitário de Lisboa, Lisbon, Portugal 4 University Research Institute On Robotics and Information and Communication Technologies

(IRTIC), University of Valencia, Valencia, Spain [email protected]

Abstract. Children with neurodevelopmental disorders have executive function deficits that can interfere with their academic performance and school adaptation. The use of computerized training programs has proven to be useful to improve executive functions, as well as reading and math performance. We developed a digital game for assessing and improving sustained attention in children aged between 6 and 9 years with neurodevelopmental disorders. The game was developed based on a standardized task called Continuous Performance Test. This study aims to investigate teachers’ acceptability of the game and their perception and feedback in regard to different aspects of the game. Nine teachers from a special school in Romania participated in a focus group and completed the reflection sheets after playing the game. Our results showed that the participants really enjoyed playing the game and considered it appropriate for children with neurodevelopmental disorders. Findings provide important contributions to the domains of gamified psychological assessment in children with neurodevelopmental disorders. Keywords: digital game · sustained attention · acceptability study

1 Introduction Computer sciences are working increasingly interconnected with the social sciences and education fields, and in particular in the classroom context to support teachers and all the learning ecosystems. In this context, it is important to understand teachers’ perceptions of the cognitive gamified tasks used just for students with neurodevelopmental problems. Thus, this study aims to clarify some aspects of test development that may be considered when designing instruments for children with these cognitive problems. Attention and executive difficulties have been considered one of the main deficits of children with neurodevelopmental disorders, with significant difficulties in sustained attention, focus and attention, as well as attention shifting [1, 2]. Sustained attention is © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 78–86, 2023. https://doi.org/10.1007/978-3-031-42134-1_8

Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention

79

considered a core cognitive process, part of the executive functions, which allow individuals to control their thoughts, behaviour, and emotions to reach personal objectives [3]. In children, executive function deficits are mostly linked with poor academic performance, for example selective attention is linked with literacy and numeracy [4]. There are several studies showing how sustained attention, attention switching, and executive attention are linked with poor math performance in children with autism spectrum disorders [5–7]. Moreover, a previous meta-analysis which included 67 studies, showed a strong correlation between executive functions and academic performance in typically developing children aged between 3 and 18 years [8]. There are various studies suggesting that computerized attentional training programs can be beneficial for children with neurodevelopmental disorders (e.g., autism spectrum disorder, attention deficit and hyperactivity, intellectual disability, and specific learning disorders). Shalev and her colleagues developed the Computerized Progressive Attentional Training software (CPAT) [9], which was initially developed to assess and improve the ability to maintain the focus of attention in a given task, even if the task is monotonous. CPAT includes a computerized version of Rosvold et al. Continuous Performance Task (CPT) [10] which involved a series of tasks based on attention. The majority of the tasks used a series of stimuli presentation on the screen from which the participant had to identify the target stimuli as quickly as possible. A study conducted by Spaniol and her colleagues [11] showed improvements in children with autism aged between 6 and 10 using a computerized progressive attention training program (CPAT) after 8 weeks. They improved their performance in reading comprehension, work coping, and math. The meta-analysis conducted to quantify the effects of computerized or digital games on the cognitive skills of children with neurodevelopmental disorders suggests that digital game-based training improves cognitive functions and that the effect is also maintained at the follow-up [12]. Inspired by previous work, we developed a computerized task involving a long series of stimuli presented sequentially with participants being instructed to respond as fast as possible only when a pre-specified target is presented while withholding responses to other stimuli appearing on the screen. This task trains the ability to focus on some specific stimuli while ignoring possible distractors. In this sense, the current research seeks to contribute to the scientific domains of computer science, cognitive psychology and education. The analysis of features that are relevant to the development of appropriate cognitive tests for children with neurodevelopmental disorders is a very relevant contribution to various theoretical bodies associated with these scientific areas and to practice. 1.1 Sustained Attention Game The game is part of an educational platform, EMPOWER that aims to assess and improve executive functions (working memory, delayed gratification, cognitive flexibility, inhibition) and emotional skills (emotion naming, understanding and use of adaptative emotion regulation strategies). All the games have a common theme, an ecological farm and all the games are related. The sustained attention game has a 3D forest (see Fig. 1) as the background and the main task of the student is to press the spacebar whenever a specific mushroom appears in the forest. There is only one target at a time: a correct mushroom, and different distractors. The game has three main levels with three sub-levels

80

C. Costescu et al.

each, where we modify the distractors and the positions where the stimuli can appear, to increase the complexity of the task, as the child moves to a higher level. The game has 60 trials in each sub-level and measures the reaction time, the total correct answers, the total incorrect answers, and the correct and incorrect answers in each condition (target present and target not present). Depending on the level of the game, flowers, branches, butterflies, and other types of mushrooms appear as distractors. The distractors used and the position of the elements change depending on the sub-level. A higher sub-level implies a more difficult position for the target placement, as well as distractors and more similar-looking distractors to the target. At the end of the 60 trials, the student can see a “Well done” as feedback to know that the game has finished. Afterwards, the application shows the values of the recorded parameters for this task (see Fig. 2). The aim of the present study was to test teachers’ acceptability of the game task and to gather their feedback regarding the appropriateness of the game for children with neurodevelopmental disorders, the appropriateness of the colours and response time, as well as the difficulty of the game in different levels.

Fig. 1. The background of the sustained attention game. The levels and some of the distractors from the game

2 Methodology Participants The participants of the focus group were nine special education teachers from the School Centre for Inclusive Education located in Constanta, Romania, a public institution that provides educational services to children with special needs. The mission of “Maria Montessori” school is to encourage, support and improve the potential of their children

Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention

81

Fig. 2. Recorded parameters in the sustained attention game.

in partnership with their families to build their independent life skills for efficient social integration. All the teachers work with students from primary and secondary school. The focus group was online, and every member had the opportunity to complete the reflection sheet individually and express their opinions about the game. Resources The resources used were a focus-group interview script (with 13 questions), a reflection sheet, and the game itself. The objectives of the focus group interview were to assess participants perceptions about the integration of the technological tool in the teaching process, to acquire information about the adequacy of the characteristic of the game and to identify possible obstacles and problems in the implementation of the games with children with neurodevelopmental disorders. Participants filled in a reflection sheet while they played the game (see Fig. 3). This reflection sheet enabled them to register key information we required to acquire a first face and content validation of the game. The focus-group interview script included the following questions: 1. What do you think about integrating technological tools in the education process of children with neurodevelopmental disorders? 2. Have you ever used technology in your regular classes? Please give examples. 3. Please share with me your thoughts about the game and the notes you wrote down while playing the game. 4. Do you foresee any problems when assessing and training children with this game? If so, explain which. 5. Considering the children whom you work with, which are the characteristics (what type of children, in terms of age, diagnosis, cognitive abilities) that we should consider when recruiting children in our studies and who you think will benefit most from the game?

82

C. Costescu et al.

Fig. 3. Reflection sheet for the focus group with a task.

6. How useful do you consider the game, in terms of assessing and training executive functions? What other types of cognitive processes and behaviour do you believe that this training could help? 7. Do you think the game can predict these children’s academic performance, and/or social and emotional competencies? 8. How could you use the information extracted from the game in your everyday work? 9. What type of feedback do you want the game to produce? 10. How should feedback be given to the children through the game? 11. We are finishing our interview. Do you want to add something or is there any other relevant aspect you want to address? 12. You may have access to general data from this first study. If you are interested, you can provide us with your email contact. 13. We would like to thank you again for your availability and your contribution to the development of this project. We have reached the end of the interview. Procedure The teachers were informed about the objectives of our work, the relevance of the study and the expected results. Teachers’ collaboration was requested, and all the ethical guidelines were respected. This study also abides by The Ethical Principles for Medical Research Involving Human Subjects of the Declaration of Helsinki [13]. Each of the participants signed an informed consent and was informed about the confidentiality of their data. The focus group interview was audio recorded with the permission of the participants. The focus group was conducted by the two first authors of the paper, who developed the scripts together with the other researchers (authors of the present paper). Their background is in psychology or special education. Their training consisted in several meeting with all the researchers involved in the study in which they discussed the scripts and the methodology of the focus group (the introduction of the focus group

Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention

83

to the participants, the confidentiality and ethical aspects, the order of the questions, possible addition questions and the final remarks). The audio records of the focus group were transcribed, and each answer of the participants was evaluated. The analysis of the answers were analysed and the main recommendations for the games improvement were summarized in the results section. Participants gave their general opinion about the use of technology in the classroom and about their current practices. Afterwards, each teacher played the game on their computer. Prior to the focus group, teachers received a short tutorial about how to download and install the games. After playing each level of the sustained attention game, they completed the reflection sheet (see Fig. 2) with five major aspects to be specified: advantages of the games, disadvantages of the game, usefulness, enjoyment, and suggestions (these were open questions). Moreover, participants had to rate on a three-point scale from 1 - disagree, 2- agree more or less to 3- agree the following statements: i) the game reaches its objective, ii) the game has an appropriate range of difficulty, iii) the response time is adequate, and iv) the colours are appropriate.

3 Results Most of the teachers enjoyed playing the game and they considered it to be very engaging. Moreover, teacher consider the games to be very useful and help children improve their executive functions. They had a very positive attitude towards the use of technology in the teaching process. Moreover, they shared some ideas of good practice (e.g. types of technological applications that they use in their teaching process). In terms of the characteristics of the children who would benefit the most from the games, they mentioned children with good comprehension skills and with the ability to handle the tablet. There were some specific recommendations for improving the game. The first recommendation was to increase the dimension of the stimuli presented. For example, regarding the dimensions of the mushrooms, they mentioned that these were not visible enough for children. The second recommendation was to change the colour of the mushrooms, to be easier to distinguish from the background. The third recommendation was to include moving elements if possible (e.g., the path to move, the leaves of the trees to move, and the butterflies to fly). Also, another recommendation that was also reflected in the reflection sheet was about the colours of the game. The participants mentioned that vivid colours should be used to better distinguish the mushrooms from the other elements. Their answers to our reflection sheets are illustrated in Fig. 3 (Fig. 4). Out of the four questions from the reflection sheets, participants expressed their disagreement only regarding the one addressing the colours from the game. As for the appropriateness of the response time and the difficulty level they agreed that the game has good features for the children that they are working with.

84

C. Costescu et al.

Teachers responses 8 7 6 5 4 3 2 1 0 Reaches its objecves

Appropiate range of difficulty Disagree

Response me adequete

Agree more or less

Colours are appropriate

Agree

Fig. 4. Centralized responses from reflection sheet for focus group interview with a task

4 Conclusion The development of executive functions in children with neurodevelopmental disorders can help them perform better in school and better adapt to the school curriculum. Our study describes a digital game for assessing and training sustained attention that we developed as part of Horizon Europe EMPOWER project, which aims to develop an educational platform for children with neurodevelopmental disorders. Teachers’ attitudes toward the use of technological tools in the classroom are a great predictor of the actual use of the technology [14]. Since teachers frequently interact with children, they will have greater insight into the environments and stimuli that produce more favourable outcomes, as well as perceive task difficulty levels and appropriate response times. The results obtained in this study suggest an adequacy of the tasks related with the parameters under evaluation in the results section. Somehow they reinforce the importance of teacher feedback in the design and development of games for this target audience. Our findings suggest that teachers consider the games for executive function useful, easy to implement and that they are appropriate for the children that they work with. In this study, the importance of colour and its contrast in the scenarios presented should be greatly enhanced. This evidence meets the theoretical assumptions of the design science research Gestalt theory, which suggests the need to integrate both the interior and exterior modes in human-computer interaction (HCI) research [15]. Accordingly, our findings suggest the importance of designing IT environments that consider some of the specificities reported in this article (obtained from a sample with teachers), as a way to improve the game experience of children with neurodevelopmental disorders. As for the exterior modes, our study reinforces the importance of observation to capture more knowledge about the entire gamified platform. This study reinforces the need to continue to invest in educational games (e.g., [16]), and specifically, educational games

Mushroom Hunters: A Digital Game for Assessing and Training Sustained Attention

85

with good practice standards needed to develop rigorous game designs tested to solve human-centric problems [17], such as neurodevelopmental disorders. The current research has some limitations. Firstly, the small sample size does not allow extrapolation of the evidence obtained. It would be good to collect in future studies the background of the participants at least in terms of experience on working in special education. Also, the fact that the data were collected in only one country limits its generalizability. Nonetheless, we reinforce that this is a pilot study that aims to understand qualitatively the specificities of the use of technology among a very specific sample. Our data also reveal some saturation and repetition of content, suggesting that the sample size would be appropriate. In future studies, we suggest the adoption of a mixedmethods approach to include a design science research Gestalt approach as this mixedmethods choice was proven to increase student learning capabilities (c.f., [18]). Future studies should also consider a repeated measures methodological approach in order to evaluate more cautiously the progress and the development of the assessed students. Further work will include testing the developed games for usability with students with neurodevelopmental disorders. Technology usability should be a precondition for any technology-based investigations in autism, as it is something that may positively or negatively impact the magnitude of the intervention effect [19]. Once usability has been tested and the software functionality improved accordingly, the authors will run an intervention study to analyse the effectiveness of Mushroom Hunters as a tool to assess and train sustained attention in children with neurodevelopmental disorders. Acknowledgements. The research leading to these results is in the frame of the “EMPOWER. Design and evaluation of technological support tools to empower stakeholders in digital education” project, which has received funding from the European Union’s Horizon Europe programme under grant agreement No 101060918. Views and opinions expressed are however those of the authors(s) only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them.”

References 1. Chien, Y.L., Gau, S.F., Shang, C.Y., Chiu, Y.N., Tsai, W.C., Wu, Y.Y.: Visual memory and sustained attention impairment in youths with autism spectrum disorders. Psychol. Med. 45(11), 2263–2273 (2015) 2. Chien, Y.L., Gau, S.S.F., Chiu, Y.N., Tsai, W.C., Shang, C.Y., Wu, Y.Y.: Impaired sustained attention, focused attention, and vigilance in youths with autistic disorder and Asperger’s disorder. Res. Autism Spect. Disord. 8(7), 881–889 (2014) 3. Zelazo, P.D., Carlson, S.M.: Reconciling the context-dependency and domain-generality of executive function skills from a developmental systems perspective. J. Cognit. Dev., 1–19 (2022) 4. Stevens, C., Bavelier, D.: The role of selective attention on academic foundations: a cognitive neuroscience perspective. Dev. Cognit. Neurosci. 2, S30–S48 (2012) 5. Bull, R., Scerif, G.: Executive functioning as a predictor of children’s mathematics ability: inhibition, switching, and working memory. Dev. Neuropsychol. 19(3), 273–293 (2001) 6. May, T., Rinehart, N., Wilding, J., Cornish, K.: The role of attention in the academic attainment of children with autism spectrum disorder. J. Autism Dev. Disord. 43(9), 2147–2158 (2013)

86

C. Costescu et al.

7. May, T., Rinehart, N.J., Wilding, J., Cornish, K.: Attention and basic literacy and numeracy in children with autism spectrum disorder: a one-year follow-up study. Res. Autism Spect. Disord. 9, 193–201 (2015) 8. Jacob, R., Parkinson, J.: The potential for school-based interventions that target executive function to improve academic achievement: a review. Rev. Educ. Res. 85(4), 512–552 (2015) 9. Shalev, L., Tsal, Y., Mevorach, C.: Computerized progressive attentional training (CPAT) program: effective direct intervention for children with ADHD. Child Neuropsychol. 13(4), 382–388 (2007). https://doi.org/10.1080/09297040600770787 10. Rosvold, H.E., Mirsky, A.F., Sarason, I., Bransome, E.D., Beck, L.H.: A continuous performance test of brain damage. J. Consult. Psychol. 20, 343–350.1 (1956) 11. Spaniol, M.M., Shalev, L., Kossyvaki, L., Mevorach, C.: Attention training in autism as a potential approach to improving academic performance: a school-based pilot study. J. Autism Dev. Disord. 48, 592–610 (2018) 12. Ren, X., Wu, Q., Cui, N., Zhao, J., Bi, H.-Y.: Effectiveness of digital game-based training in children with neurodevelopmental disorders: a meta-analysis. Res. Dev. Disabil., 133, 104418 (2023) 13. World Medical Association: Ethical principles for medical research involving human subjects. Clin. Rev. Educ. 310(20), 2191–2194 (2013). https://www.wma.net/wp-content/upl oads/2016/11/DoH-Oct2013-JAMA.pdf 14. Tomczyk, S., Barth, S., Schmidt, S., Muehlan, H.: Utilizing health behaviour change and technology acceptance models to predict the adoption of COVID-19 contact tracing apps: cross-sectional survey study. J. Med. Internet Search 23(5), e25447 (2021) 15. Adam, M.T., Gregor, S., Hevner, A., Morana, S.: Design science research methods in humancomputer interaction projects. AIS Trans. Hum. Comput. Interact. 13(1), 1–11 (2021) 16. Shortall, R., Itten, A., van der Meer, M., Murukannaiah, P.K., Jonker, C.M.: Reason against the machine: Future directions for mass online deliberation (2021). arXiv. https://arxiv.org/ abs/2107.12711 17. Forni, D.: Horizon zero dawn: the educational influence of video games in counteracting gender stereotypes. Trans. Digit. Games Res. Assoc. 5(1), 77–105 (2020) 18. Mehta, A., Bond, J.L., Sankar, C.S.: Developing an inclusive education game using a design science research Gestalt method. AIS Trans. Hum. Comput. Interact. 14(4), 523–547 (2022) 19. Mazon, C., Fage, C., Sauzéon, H.: Effectiveness and usability of technology-based interventions for children and adolescents with ASD: a systematic review of reliability, consistency, generalization and durability related to the effects of intervention. Comput. Hum. Behav. 93 (2019). https://doi.org/10.1016/j.chb.2018.12.001

An Advanced Explainable and Interpretable ML-Based Framework for Educational Data Mining Ioannis E. Livieris1(B) , Nikos Karacapilidis2 , Georgios Domalis1 , and Dimitris Tsakalidis1 1

2

Novelcore, 10436 Athens, Greece {livieris,domalis,tsakalidis}@novelcore.eu Industrial Management and Information Systems Lab, University of Patras, 26504 Patras, Greece [email protected]

Abstract. During the last two decades, the adoption of machine learning techniques for addressing various challenging issues in the educational domain has gained much popularity. Nevertheless, there is still a lack of research on developing AI systems that focus on the interpretability and explainability of the associated models and algorithms, thus being able to present the data analysis results in a human understandable way. In this work, we propose a new explainable framework for predicting students’ performance, which provides accurate, reliable and interpretable results. Our framework builds on the recently proposed NGBoost algorithm for the development of an efficient prediction model, as well as on the LIME and SHAP methods for providing local and global explanations, respectively. The use cases presented in this paper demonstrate the applicability of our framework and give insights about the recommendations that can be provided to educators and students. Keywords: Educational data mining · machine learning · recommendation · explainability · NGBoost · LIME · SHAP

1 Introduction Educational Data Mining (EDM) is a field of research that builds on Machine Learning (ML) and other data analysis techniques to analyze educational data for gaining insights into the learning process. Its primary goal is to identify patterns, relationships and factors that affect the outcomes of the learning process, aiming to predict the learners’ future performance and accordingly use this information to improve educational practices and policies. EDM has been already applied in a wide range of educational contexts and has a great potential to enhance student learning, teacher effectiveness, as well as the overall institutional performance [7, 8, 14, 16, 18]. In recent years, there is an increasing trend in EDM to provide meaningful explanations to educators, administrators and students about how the underlying ML models arrive at their predictions or decisions. Explainability is considered important for a c The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 87–96, 2023. https://doi.org/10.1007/978-3-031-42134-1_9

88

I. E. Livieris et al.

variety of reasons. Firstly, as EDM models become more complex and powerful, it is difficult to understand how they make decisions or predictions. By ensuring that these ML models are transparent and explainable, educational stakeholders are able to better understand how they work and have more confidence in their results, thus leading to more informed decision-making. Secondly, explainability is important for ethical reasons; as ML models are used more frequently in education, it is of major importance to ensure that they are fair and unbiased, and that they do not perpetuate or amplify existing biases in the associated data. In the context under consideration, feature importance methods have been broadly used to measure the contribution of each feature on an EDM model’s predictions. Local Interpretable Model-agnostic Explanations [17] (LIME) and SHapley Additive exPlanations (SHAP) [15] are two permutation-based model-agnostic techniques, which probably constitute the most widely used feature importance methods [7, 8, 10, 16]. Both of them are able to randomly sample from the marginal distribution considering unrealistic instances that are not present in the training data. In simple words, they focus on extrapolating in the areas where the model was trained for measuring each features effects on the predictions. The major difference between these methods is that the former is based on a simple model for creating a local explanation around a prediction, while the latter is based on game theory to measure the magnitude of feature attributions. This work aims to contribute to the EDM field by proposing an advanced framework for predicting students’ performance, which ensures the provision of accurate, reliable and explainable predictions. Our primary goal is to develop an efficient model for predicting students’ performance and simultaneously provide human-interpretable explanations of individual predictions, which in turn give insights into how the model is making its decisions. Our prediction model is based on one of the most efficient ML algorithms, namely Natural Gradient Boosting for Probabilistic Prediction (NGBoost); as demonstrated in our experiments, NGBoost outperforms traditional state-of-the-art prediction algorithms. An attractive advantage of the proposed approach is that it is able to provide both local and global explainability, providing users with a complete picture as well as with human-interpretable insights about why a particular decision was made by the model. In simple words, it is able to provide a description of the model’s mechanisms and concepts, while simultaneously being able to explain each individual prediction made. This is achieved through the utilization of both the SHAP and the LIME methods, which ensure that the importance scores are fair and unbiased as well as a flexible, fast and reliable interpretation of each single prediction. Through two representative use cases, we also showcase the applicability of the proposed framework and the possible recommendations that could be provided to educators and students. The remainder of this paper is as follows: Sect. 2 presents a brief survey of recent studies concerning the application of ML models for predicting the students’ performance. Section 3 describes in detail the proposed framework, while Sect. 4 comments on the data used in this research. Section 5 reports on our experimental results and describes the two cases used for the application of the proposed framework. Finally, Sect. 6 sketches concluding remarks and proposes future research directions.

An Advanced Explainable and Interpretable ML-Based Framework

89

2 Related Work During the last two decades, the adoption of ML techniques for providing useful insights about the learning process and students’ behavior has gained much popularity. An interesting application concerns the successful prediction of students’ performance, especially of students that are at risk of failing, which has a significant impact to diverse educational stakeholders including teachers and students, but also the educational institute. In this context, the provision of accurate and explainable predictions is essential for effectively conducting the necessary pedagogical interventions to enhance students’ performance. A number of interesting related studies have been already proposed in recent years, whose findings and limitations are outlined below. Tampakas et al. [18] proposed a two-level classification scheme for predicting students’ graduation time within the first two years of their studies. The proposed scheme has two major features: (i) identification of students which are likely to fail to graduate, and (ii) prediction of students’ expected graduation time. The presented numerical experiments showed the superiority of the proposed approach compared to the traditional ones, and reveal that it is possible to accurately predict students’ graduation time within the first two years of their studies. However, this approach does not provide any explainable feedback about its predictions. Hue et al. [8] implemented personalized system intervention using a ML model to predict student performance and explained its predictions via a SHAP method. The proposed approach was evaluated in a self-paced, self-guided online learning system for college-level topics, which provided personalized interventions to enhance a learning behavior. A randomized controlled trial of 37 expert-system condition and 36 explanation condition participants showed that similar learning and topic-choosing behavior between conditions. Based on their findings, the authors stated that XAI-informed interventions facilitated student learning to a similar degree as expert-system interventions. However, a limitation of this work is the relatively small pilot and experimentation data size. Ramaswami et al. [16] presented a generic predictive model for identifying students at risk across a variety of courses in a blended learning environment. The numerical experiments included the evaluation of several state-of-the-art ML algorithm and showed that the CatBoost algorithm demonstrates the best overall performance. In this work, the authors used the SHAP method to estimate model behaviour without providing any recommendations or any explainable framework for assisting the education process. Guleria and Sood [7] proposed a new framework for students’ career counseling based on ML and AI techniques. White and black box models were trained on an educational dataset for predicting future students’ placement status (binary classification task). The numerical experiments include the performance evaluation of several stateof-the-art white and black box models. Additionally, the authors provided some global explainability insights using the SHAP method. Two limitations of this work are that the used dataset contains a limited scope of attributes and sample size and the fact that the proposed framework does not provide any local explainability feedback.

90

I. E. Livieris et al.

3 The Proposed Framework Aiming to overcome the above mentioned limitations, this work proposes an advanced framework for predicting students’ performance, which provides accurate, reliable and explainable predictions. The proposed framework builds on the recently proposed NGBoost algorithm for the development of an efficient prediction model. Contrary to previous works, our approach is able to provide both local and global explainability feedback about the predictions of the NGBoost model by exploting the LIME and SHAP importance methods, respectively. The proposed framework consists of two main components: the prediction model and the explainability modules. The former is used for predicting the students’ performance, while the latter for providing the reasoning behind the decisions and individual predictions of the model. Figure 1 provides a high-level overview of the proposed framework. Initially, the training data are preprocessed (invalid values removal, one-hot encoding of categorical variables and data imputation using 3NN) and transformed into a suitable form to be used as input to the ML algorithm for developing an efficient prediction model. Our goal is to develop a classifier with strong classification ability. For this reason, we selected the recently proposed NGBoost algorithm [4]. As soon as the prediction model is developed, it can be used for conducting predictions on new data; simultaneously, its predictions, along with the LIME and SHAP methods, are used for providing local and global explainability, respectively.

Fig. 1. The proposed framework

It is worth mentioning that the reason for selecting LIME for local explainability is that it probably constitutes the most flexible and widely utilized method for interpreting a single prediction [2]. Additionally, the reason for preferring the SHAP method for global explainability over NGBoost’s feature importance is that feature importance can be biased or unstable and is heavily depending on the specific method for its calculation [1]. On the other hand, SHAP is based on game theory, ensuring that the importance

An Advanced Explainable and Interpretable ML-Based Framework

91

scores are fair and unbiased [15]. In the following, we briefly present the main technologies on which the proposed framework is based, i.e. the NGBoost classifier, as well as the LIME and SHAP explainability methods. 3.1 NGBoost Algorithm NGBoost (Natural Gradient Boosting) [4] is a probabilistic boosting algorithm that can be used for classification and regression tasks. Its key innovation is the employment of natural gradient for performing gradient boosting by casting it as a problem of determining the parameters of a probability distribution. It is noted that ordinary gradients may be unsuitable for learning multi-parameter probability distributions (such as the normal distribution). On the other hand, the natural gradients with the use of training dynamics tends to be much more stable and robust, something that results in a better fitting process. The NGBoost algorithm consists of three components: (i) base learners, which uses weak ML models for making predictions of the input data and form the conditional probability, (ii) parametric probability distribution, which constitutes a conditional distribution and it is formed by an additive combination of base learners outputs, and (iii) scoring rule, which measures the quality of its probabilistic predictions and optimizes the ensemble-based model. In our experiments, NGBoost was implemented with decision trees as base learners for making the predictions of the input data and Negative Log Likelihood (NLL) as scoring rule. 3.2 Explainability Methods LIME (Local Interpretable Model-agnostic Explanations) [17] constitutes a modelagnostic and local interpretability method for explaining the predictions of any ML model. The rationale behind LIME is rooted in the need for transparency and interpretability in ML models, which is fundamental in many challenging real-world applications [2]. LIME has the advantage that it does not rely on any assumptions about the underlying model architecture or training process. In addition, it is highly transparent, which makes it easier to understand how the feature importances are calculated and how they contribute to the final prediction, flexible and computationally efficient in providing explanations. However, a limitations of LIME is that it does not provide a global view of the model’s behavior or insights into how the model makes decisions across the entire dataset. Therefore, it should be used in conjunction with other methods for model interpretability and transparency. SHAP (SHapley Additive exPlanations) [15] is a state-of-the-art explainability method [2], which measures the contribution of each data point to each feature value based on cooperative game theory (Shapley values). By doing so, it is able to deliver fine grain explanations and calculate the global feature contributions, including their direction. The key idea behind this method is that each feature’s contribution is the Shapley value, which provides information about the model’s performance if it was trained without that feature. However, a significant drawback of calculating the SHAP values is the computational cost, since the training time grows exponentially with the number of features.

92

I. E. Livieris et al.

4 Dataset For the purpose of this work, we utilized two real-world educational datasets. Specifically, DATASET1 includes information about 337 university students who attended an academic course using a Learning Content Management System (LMS) in a blended learning environment. This dataset was collected from the Department of Educational Sciences and Early Childhood Education, University of Patras, Greece, during the academic years 2007-2010. Each instance contains information related to the students’ perceptions about the Moodle LMS and their opinions about its educational value and usefulness, as well as information related to the students’ activity. Demographic values were not included due to the similar characteristics of all participants. In this dataset, students were classified upon whether they passed the lesson (“Pass”) or not (“Fail”). More information about this dataset can be found in [5]. DATASET2 contains data about 3716 students that attended courses of Mathematics of a secondary school (namely, the “Avgoulea-Linardatou” Microsoft Showcase School, Greece; data concern the time period 2007-2016). This dataset summarizes information about the students’ performance from the first two out of three semesters such as tests grades, final examination grades oral grades, as well as semester grades. Note that an academic year of a secondary school in Greece consists of three semesters. The students were classified according to their performance upon a four-level classification scheme, i.e. “Fail”, “Good”, “Very Good” and “Excellent” (more information is provided in [12])1 .

5 Experiments and Use Cases In this section, we report on the evaluation of the performance of the NGBoost algorithm that was adopted in the proposed framework against state-of-the-art ML classification algorithms. In addition, we present the explanations produced by our framework for two representative use cases. 5.1

Experimental Results

We have evaluated the performance of the NGBoost algorithm on two educational datasets against that of three state-of-the-art ML algorithms, namely Random-Forest [11], XGBoost [3] and LGBM [9]. Admittedly, these algorithms constitute the most efficient ones for handling classification tasks with tabular data [6]. In our original experiments, we included several algorithms such as neural networks, support vector machines and k-nearest neighbors; nevertheless, their performance was inferior to that of ensemble tree-based algorithms. The classification performance of all algorithms was evaluated using stratified 10fold cross-validation and the following performance metrics: Accuracy, Area under 1

It is noted that a detailed description of the features of both datasets, as well as their descriptive statistics and a complete exploratory data analysis, can be found in https://github.com/novelcore/A-new-explainable-and-interpretable-ML-based-frameworkfor-educational-data-mining.

An Advanced Explainable and Interpretable ML-Based Framework

93

Curve (AUC), Geometric Mean (GM), Precision and Recall. It is worth mentioning that the performance metrics AUC and GM, as well as the balance between Precision and Recall, present the information provided by a confusion matrix in compact form; therefore, they constitute the proper metrics to evaluate the classification ability of a prediction model [13]. The implementation code was written in Python 3.8, while the hyperparameters of all algorithms were optimized in order to achieve the best possible performance2 . Table 1 summarizes the performance of all classification algorithms on DATASET1 . As shown, NGBoost demonstrates the best overall classification performance for all performance metrics. Specifically, NGBoost reported the best Accuracy (71.2%), followed by Random-Forest (70.8%) and XGBoost (66.6%). Additionally, it reported the highest AUC (0.649) and GM (8.221) scores, while Random-Forest presented the second best performance. In contrast, LGBM was unable to distinguish between noise and rare cases, which resulted in poor performance. Table 2 presents the performance of all classification algorithms on DATASET2 . NGBoost reported the highest GM metric (45.24) and the best balance between Precision and Recall, which suggests that it presents the best overall performance. Additionally, XGBoost and Random-Forest reported the second best performance, while LGBM reported the worst performance for all performance metrics. From the above results, we can conclude that the NGBoost algorithm is able to develop the most accurate prediction model for both datasets used in this work. Table 1. Performance evaluation of classification algorithms on DATASET1 Algorithm

Accuracy AUC GM

Precision Recall

LGBM Random-Forest XGBoost NGBoost

60.4% 70.8% 66.6% 71.2%

0.581 0.605 0.592 0.608

0.633 0.641 0.640 0.649

8.034 8.094 8.156 8.221

0.633 0.641 0.640 0.648

Table 2. Performance evaluation of classification algorithms on DATASET2

2

Algorithm

Accuracy AUC GM

Precision Recall

LGBM Random-Forest XGBoost NGBoost

86.9% 89.4% 89.7% 90.0%

0.862 0.906 0.905 0.897

0.911 0.923 0.924 0.927

44.01 44.99 45 45.24

0.867 0.884 0.885 0.927

Additional information can be found in at https://github.com/novelcore/A-new-explainableand-interpretable-ML-based-framework-for-educational-data-mining.

94

5.2

I. E. Livieris et al.

Use Cases

This subsection reports on the application of the proposed framework on two use cases corresponding to DATASET1 and DATASET2 , aiming to demonstrate the usefulness of the feedback provided to educators and students. Figure 2 illustrates the output from the application of the proposed framework on the problem of identifying the university students who are at-risk of failing the examinations (DATASET1 ). The interpretation of Fig. 2(a) suggests that the three most important features for predicting if a university student will fail in the examinations are forum_view, course_view and Computer_at_home, while the features Perceived_Moodle_Usefulness, Perceived_Usefulness_assignment and Attitude_about_Moodle seem to have no effect in the model’s decisions. This implies that an educator should provide special attention to the number of times each student accesses the description and the basic material of each week’s laboratory session, accesses the forum section and if he/she owes a computer. In addition, the feedback for a student could be that he/she pays more attention to the basic material of each week’s laboratory. Local explainability focuses on understanding how the model made decisions for a single instance. For a specific student, the model predict that he/she will fail to the final course grade with probability 65% and the fact that this student has no computer at home has the highest impact on the model’s decision.

Fig. 2. Application of the proposed framework on DATASET1

Figure 3 presents the output from the application of the proposed framework on the problem of predicting the performance of high-school students at the final examinations (DATASET2 ). The global explainability component shows that the three features that influence most the model are Oral_A, Test2_A and Test1_A, while the three least important ones are Oral_B, Exam_A and Class. A possible recommendation to the educator could be that the students’ performance on the first semesters, especially their performance on oral examinations and on 15-minutes tests, seem to considerably affect the students’ performance on the final examination. The interpretation of Fig. 3(b) suggests that for a specific student, the model predicts that he/she will have “good” performance

An Advanced Explainable and Interpretable ML-Based Framework

95

on the final examinations, while the fact that his/her oral grade on the first semester is less than 13 (i.e. Oral_A = 12) has the highest impact on the model’s decision. Therefore, a recommendation that could be given to the student (as well as to the educator) is to improve his/her oral argumentation skills.

Fig. 3. Application of the proposed framework on DATASET2

6 Conclusions This work proposes an advanced explainable framework for predicting students’ performance, which provides accurate, reliable and interpretable predictions. The proposed framework is based on the recently developed NGBoost algorithm, as well as on the LIME and SHAP importance methods for providing local and global explainability, respectively. Our numerical experiments showed that NGBoost is able to outperform traditional state-of-the-art ML algorithms, hence providing empirical evidence that its adoption could lead to the development of an accurate prediction model. The adoption of SHAP ensures that the importance scores are fair and unbiased, while the adoption of LIME provides a flexible, fast and reliable technique for interpreting a single prediction. A limitation of this work is that the proposed framework was evaluated only on two real-world datasets, containing a limited number of attributes with the scope of predicting the students’ performance on the examinations. A future work direction is to consider and elaborate additional educational datasets, and accordingly evaluate the proposed framework across diverse challenging issues in the educational domain. Another direction for future research is the automatic generation of recommendations in text form based on the feedback provided by the proposed framework, the main objective being to provide more human-interpretable recommendations and enhance the students’ learning process. Acknowledgements. This work received funding from the Horizon Europe research and innovation programme under Grant Agreement No. 101061509, project augMENTOR (Augmented Intelligence for Pedagogically Sustained Training and Education). We would like to thank

96

I. E. Livieris et al.

the Department of Educational Sciences and Early Childhood Education, University of Patras, Greece, and the “Avgoulea-Linardatou” Microsoft Showcase School for providing us with the data used in this work.

References 1. Altmann, A., Tolo¸si, L., Sander, O., Lengauer, T.: Permutation importance: a corrected feature importance measure. Bioinformatics 26(10), 1340–1347 (2010) 2. Carvalho, D.V., Pereira, E.M., Cardoso, J.S.: Machine learning interpretability: a survey on methods and metrics. Electronics 8(8), 832 (2019) 3. Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016) 4. Duan, T., et al.: NGBoost: natural gradient boosting for probabilistic prediction. In: International Conference on Machine Learning, pp. 2690–2700. PMLR (2020) 5. Filippidi, A., Tselios, N., Komis, V.: Impact of Moodle usage practices on students’ performance in the context of a blended learning environment. In: Proceedings of Social Applications for Life Long Learning, pp. 2–7 (2010) 6. Grinsztajn, L., Oyallon, E., Varoquaux, G.: Why do tree-based models still outperform deep learning on tabular data? arXiv preprint arXiv:2207.08815 (2022) 7. Guleria, P., Sood, M.: Explainable AI and machine learning: performance evaluation and explainability of classifiers on educational data mining inspired career counseling. Educ. Inf. Technol. 28(1), 1081–1116 (2023) 8. Hur, P., Lee, H., Bhat, S., Bosch, N.: Using machine learning explainability methods to personalize interventions for students. International Educational Data Mining Society (2022) 9. Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems 30 (2017) 10. Knapiˇc, S., Malhi, A., Saluja, R., Främling, K.: Explainable artificial intelligence for human decision support system in the medical domain. Mach. Learn. Knowl. Extract. 3(3), 740–770 (2021) 11. Liaw, A., Wiener, M., et al.: Classification and regression by Random-Forest. R News 2(3), 18–22 (2002) 12. Livieris, I.E., Drakopoulou, K., Tampakas, V.T., Mikropoulos, T.A., Pintelas, P.: Predicting secondary school students’ performance utilizing a semi-supervised learning approach. J. Educ. Comput. Res. 57(2), 448–470 (2019) 13. Livieris, I.E., Kiriakidou, N., Stavroyiannis, S., Pintelas, P.: An advanced CNN-LSTM model for cryptocurrency forecasting. Electronics 10(3), 287 (2021) 14. Livieris, I.E., Kotsilieris, T., Tampakas, V., Pintelas, P.: Improving the evaluation process of students’ performance utilizing a decision support software. Neural Comput. Appl. 31, 1683–1694 (2019) 15. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30 (2017) 16. Ramaswami, G., Susnjak, T., Mathrani, A.: On developing generic models for predicting student outcomes in educational data mining. Big Data Cogn. Comput. 6(1), 6 (2022) 17. Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016) 18. Tampakas, V., Livieris, I.E., Pintelas, E., Karacapilidis, N., Pintelas, P.: Prediction of students’ graduation time using a two-level classification algorithm. In: Tsitouridou, M.A., Diniz, J., Mikropoulos, T.A. (eds.) TECH-EDU 2018. CCIS, vol. 993, pp. 553–565. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20954-4_42

Development of an Immersive Virtual Reality System to Practice the Lumbar Puncture Manoeuvre ´ Mar´ıa Beatriz Villar-L´ opez1 , Agueda G´ omez-Cambronero1(B) , 2 Daniel Suarez , and Inmaculada Remolar1 1

Institute of New Imaging Technologies, Universitat Jaume I, Av. Vicent Sos Baynat, s/n, 12071 Castell´ o de la Plana, Castellon, Spain [email protected] 2 Consultorio Auxiliar de Benej´ uzar, DS Orihuela, Spain Abstract. Training in medical practices requires materials, patients and a context that allows students to become familiar with the environment and tools. However, this is not always easy to achieve. Immersive virtual reality can be a suitable tool for moving from theoretical studies to practice in a real context. This paper describes the process of analysis, design and development of a virtual environment in which lumbar puncture can be performed. It is a project in development that combines medical research with the latest advances in extended reality to improve the training possibilities of medical students.

Keywords: virtual reality medical training

1

· immersive learning · lumbar puncture ·

Introduction

Lumbar puncture (LP), also known as a spinal tap, is a medical procedure that involves the insertion of a needle between two lumbar vertebrae in the lower back to remove a small amount of cerebrospinal fluid (CSF) for testing or to inject medication [16]. The procedure is typically performed to diagnose and monitor conditions that affect the brain and spinal cord, such as meningitis, encephalitis, multiple sclerosis, and certain cancers. It can also be used to relieve pressure on the brain or spinal cord caused by excess CSF or to administer chemotherapy drugs directly into the spinal cord. The common learning model for students who must learn this procedure is based on carrying out the practice under the supervision of an expert; however, this, in addition to exposing the patient, generates high levels of stress in the practitioners [10]. Virtual Reality (VR) provides an opportunity to experience close-to-reality situations in safe and immersive 3D environments. Studies have shown that VR can enhance technical skills and practical learning and is well-suited to support various learning methodologies, including experiential learning. VR can also improve student engagement and motivation, provide personalized learning experiences, and facilitate collaborative learning [1,2,13,14]. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  Z. Kubincov´ a et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 97–106, 2023. https://doi.org/10.1007/978-3-031-42134-1_10

98

M. B. Villar-L´ opez et al.

User-centered design (UCD) is an iterative design process that involves understanding the needs, goals, and preferences of the end-users and incorporating them into the design of a product or service [15]. The primary goal of UCD is to create designs that are intuitive, usable, and satisfying for the users. The UCD process typically involves several stages, such as user research, requirements gathering, prototyping, testing, and evaluation. The UCD approach emphasizes the importance of involving users throughout the design process to ensure that the final product meets their needs and expectations. By focusing on the users’ perspectives and experiences, UCD can result in products and services that are more effective, efficient, and enjoyable to use. This article presents the design of a virtual experience for spinal tap training developed following a UCD. The experience offers an immersive and realistic 3D environment for students to acquire spinal tap knowledge through close-toreality practice in a safe mode. The objective of this application is to provide a cost-effective and successful way for spinal tap training using the benefits of VR. The article is organized as follows: Sect. 2 provides a brief review of important VR applications in spinal tap manoeuvre and education. Section 3 describes the methodology, including the design process and implementation. Section 4 analyzes and discusses the obtained results, and Sect. 5 presents the conclusions and future work.

2

Literature Review

Several virtual haptic models of epidural injection simulators have been created to enhance the skills of medical students and professionals. Dang et al. [6] developed an epidural injection simulator in 2001, which demonstrated significant potential as a useful tool. In 2006, Dreifaldt [7] investigated the effectiveness of haptics as a training aid for medical doctors performing spinal anaesthetics. Additionally, in 2007, Groman et al. [9] developed a force feedback LP simulator, offering a safe method of training for students. Regarding VR simulators, in the field of medicine, there has been a growth for several years due to the ability they offer to gain experience in a safe and controlled environment. As early as 1994, Bostrom et al. [3] introduced the LP procedure with tactile feedback in a virtual simulator. In 2008, Kanumuri et al. [11] compare the effectiveness of using VR and computer-enhanced video-scopic training devices for complex pararoscopy tasks, concluding that the effectiveness was similar. Bott et al. [4], in the same year, evaluated a tool called virtX, computer-based training to work with X-ray imaging, showing that it reduces the time needed to perform this type of task. In some studies, such as the one presented by Farber et al. [8], the virtual environment emphasizes the specific puncture manoeuvre itself, demonstrating that the VR LP simulator provides an authentic tactile and visual experience of needle insertion, allowing for novel understandings of lumbar anatomy. Techniques with similar characteristics, such as acupuncture, have also been studied

VR to Practice the Lumbar Puncture Manoeuvre

99

using VR [5,12]. In some cases, force simulators through haptic gloves have been used. Despite the fact that all these approaches present a good acceptance of the technology and promising results, some aspects of students’ adherence to the systems, the UCD process, and the motivation to use the system still need further investigation.

3

Materials and Methods

A multidisciplinary team has followed an iterative process to carry out the design and implementation. The multidisciplinary team includes a medical doctor who supervises doctors in training and a professor who is an expert in the design and development of immersive VR experiences. Both are researchers in their respective fields. The team is completed by developers. The subsequent description outlines the tasks that have been performed. 3.1

Analysis

Several interviews were hold in order to establish the requirements of the developed application. A focus group was performed where doctors, health experts and developers define these requirements that have to be considered in the virtual environment. It was said that performing a LP requires proper training, skill, and knowledge of the anatomy and physiology of the spinal cord and surrounding structures. It is typically done to obtain CSF for diagnostic or therapeutic purposes. The requirements that have to be considered to train the LP procedure involves the following steps: – Patient positioning: Place the patient in a lateral decubitus position with the knees drawn up to the chest and the head bent forward, or in a seated position with the back arched and the chin tucked in. – Local anesthesia: Apply local anesthesia to the skin and subcutaneous tissue at the intended puncture site, usually at the L3-L4 or L4-L5 level. – Insertion of needle: Insert a sterile spinal needle through the anesthetized area and into the subarachnoid space, using the appropriate technique and angle. – Collection of CSF: Collect a sample of CSF for laboratory analysis or therapeutic purposes, and measure the opening pressure if indicated. – Needle removal and wound closure: Withdraw the needle slowly and carefully, and apply pressure to the puncture site to prevent CSF leakage. Cover the site with a sterile dressing and monitor the patient for any complications. It is essential to follow strict aseptic techniques and use appropriate equipment and supplies to minimize the risk of infection, bleeding, or nerve damage. It is also important to monitor the patient’s vital signs and neurological status before, during, and after the procedure.

100

3.2

M. B. Villar-L´ opez et al.

Design Process

The UCD approach has been employed to carry out the design process, which focuses on gaining a deep understanding of the end-users in order to meet their needs and validate the performance of the virtual environment design. The authors collaborated closely with Benejuzar Auxiliary Office, a health center specialized in performing LP and medical students from different faculties. During meetings with the parties involved, the training needs and issues faced by the teaching process were collected. Design Thinking (DT) was used to guide the UCD approach. DT is a problem-solving approach that involves empathizing with users, defining problems, ideating solutions, prototyping, and testing. The development of this experience took into account the five DT principles in the following way: Empathize: The user’s perspective, including students, teachers, and professionals, was considered in various meetings. The team also observed the LP maneouver through videos to gain a better understanding of the end-users’ needs. During this phase, the team gained a comprehensive understanding of the challenges encountered by doctors during their training in performing this maneouver. Specifically, they recognized the inherent difficulties associated with acquiring proficiency in invasive techniques, which are typically learned through the utilization of plastic models or real patients. Such practices have spatial and temporal limitations and present a multitude of challenges and complexities. Define: Specific and clear requirements were established to ensure that the development process aligned with the real needs of end-users. Based on the focus group insights, it became evident that acquiring feedback regarding the precise positioning of the vertebrae and ribs, while ensuring the sequential execution of the aforementioned steps, holds significant importance. Ideate: A variety of ideas were generated to tackle the specific challenge. The team members from different fields were encouraged to think creatively and share their ideas through brainstorming techniques. Prototyping: Incremental prototyping was utilized, introducing new models and functionalities at each stage of the design process. Firstly, the prototype allowed the injection of the anaesthesia into the patient through a needle. In the next interaction, the possibility of identifying the correct lumbar vertebrae to carry out the procedure was added. Thirdly, the prototype was augmented with the interaction to perform the lumbar puncture while monitoring the patient. Finally, other interactions necessary in LP maneouver were appended such as cleaning and disinfection of the spinal area and checking that each of the actions was performed in the correct order. All of these steps were implemented and resulted in a usable prototype without bugs.

VR to Practice the Lumbar Puncture Manoeuvre

101

Testing: The functionality of the application was assessed at every stage of development. Additionally, a study was designed to measure the usability, gameful perception, and potential adverse effects of the application on the users’ health. After the completion of the empathy, definition, and ideation phases, the development team began working on the implementation tasks, which resulted in the gradual development of an increasingly complete prototype. Subsequently, each iteration was tested by both the development team and the medical staff, who provided valuable feedback to inform the subsequent iterations. 3.3

Development

In order to create the virtual experimental learning, the Unity 3D game engine (version 2020.3.29f1, build 256) was utilized, along with the Open XR libraries for VR (including the Interaction XR Toolkit) to connect the development to the output device, the Oculus Quest 2 (build 258). The 3D scenes and elements were modeled using Blender, and the development was carried out using an Intel(R) Core(TM) i7-10700F CPU, including an Nvidia GeForce RTX 3070 Ti graphics card with 8 GB of memory (build 260). The implementation of the VR LP procedure includes the main steps. It involves inserting a needle into the spinal canal to collect cerebrospinal fluid for diagnostic or therapeutic purposes. There are some steps, as obtain the informed consent or monitor the patient after the procedure, that are not included in the project. Here are the general steps involved in performing a LP that have to be applied into the environment: – Step 1: Position the patient correctly: Have the patient lie on their side with their knees pulled up to their chest and their chin tucked down to their chest. – Step 2: Identify the site: Locate the site for needle insertion, which is typically between the third and fourth or the fourth and fifth lumbar vertebrae. Mark the site with a pen or marker. – Step 3: Clean the area: Clean the area with an antiseptic solution to reduce the risk of infection. – Step 4: Administer local anesthesia: Use a small needle to numb the skin and deeper tissues around the site. – Step 5: Insert the spinal needle: Insert the spinal needle slowly, aiming toward the umbilicus (belly button) while monitoring the patient for any signs of discomfort or neurological symptoms. – Step 6: Collect cerebrospinal fluid: When the needle reaches the subarachnoid space, cerebrospinal fluid should begin to drip out of the needle. Collect the desired amount of fluid, usually between 5 and 20 ml, depending on the reason for the LP. – Step 7: Remove the needle: Withdraw the needle slowly, and immediately apply pressure to the puncture site with a sterile gauze or bandage to prevent leakage of cerebrospinal fluid.

102

4

M. B. Villar-L´ opez et al.

Results

It is important to indicate that this work is in a preliminary phase. The design and development has been carried out and concluded. However at the time of writing, it is being evaluated by experts, who are preparing for experimentation with students. The scope of these results reaches the presentation of the final virtual environment, not including its validation results. The scenario where the training starts represents a medical office, where the user is tasked with performing a LP. The patient lies on a stretcher in a lateral decubitus position, while a nearby table holds the necessary instruments. A screen on the wall displays the necessary steps for the procedure. The user is free to move around and interact with the instruments. The screen updates messages as the correct steps are followed, while any errors are highlighted and the screen reminds the user of the missed step. The environment is designed to allow the user to correct her/his mistakes and resume the activity from the moment immediately before the error occurred. Figure 1 shows the context of the stage, including the table with the instruments, and the situation of the patient in the correct position. This relates to the first step of the procedure.

Fig. 1. Environment showing the patient and the instrument table.

The first action that the user must carry out is to follow steps two and three of the procedure, identifying the correct place and marking it (Fig. 2, left). The user palpates the area of the spine to find the appropriate location. When it is located, the HMD (Head-Mounted Display) controls vibrate to indicate that it has been found and then, this area has to be marked to highlight it. Then, the area where the puncture will be made has to be disinfected (Fig. 2, right). The next step (step 4) is to administer the anesthesia to the patient. To do this, he must first extract it from the container (Fig. 3, left) and then apply it to the marked area (Fig. 3, right). At this moment, the spinal needle must be inserted (Step 5), the user can visualize in real time where the needle is, so that the manoeuvre is as precise as possible, he also receives feedback in the form of vibrations if he does not puncture the spinal needle in the right place (Fig. 4).

VR to Practice the Lumbar Puncture Manoeuvre

103

Fig. 2. Example identifying and marking where the puncture will be made (left) and sanitising the area (right).

Fig. 3. Sample preparation of the anesthesia process (left) and its application (right).

Fig. 4. Introduction of the needle for LP.

104

M. B. Villar-L´ opez et al.

The last two steps (Steps 6 and 7) can be seen in Fig. 5, first the fluid is collected (left) and then the needle is slowly withdrawn (right).

Fig. 5. Obtaining the cerebrospinal fluid sample (left) and repositioning the stylet to finish the process (right).

Throughout this process the user is able to navigate the stage, review the instruments and interact with them. The system, in addition to allowing the steps to be practiced, allows the environment to be immersive and the student becomes familiar, not only with the procedure, but also with the context in which it is carried out. The user can run the simulation as many times as needed, review the steps and correct her/his actions. This history of attempts is stored in the user profile who performs the simulation, therefore teachers can review the progress and skills of the students who have performed these virtual practices.

5

Conclusions and Future Work

Traditional training methods can be expensive or difficult to reproduce, especially when it comes to medical simulation equipment. VR technology is a costeffective alternative that can be easily accessed from anywhere. This article has described the process of analysis, design and development of a virtual environment in which LP manoeuvre can be trained. It is a work in progress that hopes to make available to doctors or students a tool that allows them to become familiar with the procedure before moving on to actual practice. The work presents a safe and controlled environment that reproduces the usual scenario where this LP can be simulated. Also, VR developed application provides immediate feedback on the technique, allowing medical students to adjust and correct their technique in real-time. By practicing in a virtual reality environment, medical professionals can develop their skills without putting patients at risk. The immediate next step is the validation of the described environment with medical users, trying to make a comparison between the traditional methods and this new approach. The hypothesis is that the results of the comparison will show good acceptance with the technology and an improvement in learning.

VR to Practice the Lumbar Puncture Manoeuvre

105

This type of technology allows complicated procedures and manoeuvres in medical practice to be performed. VR scenarios can be customized to provide a variety of different scenarios, allowing medical professionals to practice a range of different techniques and approaches. Consequently, as future work, it is expected that other procedures such as intubation or laparoscopic procedures can be adapted. Acknowledgements. Research supported by the e-DIPLOMA, project number 101061424, funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or the European Research Executive Agency (REA). Neither the European Union nor the granting authority can be held responsible for them.

Research is also supported by the UJISABIO program, related to the project (UJISABIO22-AP07) “3D design and virtual reality applied to medical learning in emergencies and emergencies”.

References 1. dos Anjos, F.E.V., Rocha, L.A.O., da Silva, D.O., Pacheco, R.: Impacts of the application of virtual and augmented reality on teaching-learning processes in engineering courses: a systematic literature review about learning and satisfaction on students. Int. J. Virt. Person. Learn. Environ. (IJVPLE) 12(1), 1–19 (2022) 2. Araiza-Alba, P., Keane, T., Chen, W.S., Kaufman, J.: Immersive virtual reality as a tool to learn problem-solving skills. Comput. Educ. 164, 104121 (2021). https://doi.org/10.1016/j.compedu.2020.104121. https://www.sciencedirect.com/ science/article/pii/S0360131520303195 3. Bostrom, M., Singh, S.K., Wiley, C.W.: Design of an interactive lumbar puncture simulator with tactile feedback. In: Proceedings of IEEE Virtual Reality Annual International Symposium, pp. 280–286. IEEE (1993) 4. Bott, O., et al.: VirtX-evaluation of a computer-based training system for mobile C-arm systems in trauma and orthopedic surgery. Methods Inf. Med. 47(03), 270– 278 (2008) 5. Cheng, Z., Wang, H., Min, Y., Yan, Z., Hong, Z.T., Zhuang, T.: Preliminary study on force feedback of acupuncture in virtual reality based on the visible human. Zhongguo yi Liao qi xie za zhi= Chin. J. Med. Instrument. 31(1), 5–9 (2007) 6. Dang, T., Annaswamy, T.M., Srinivasan, M.A.: Development and evaluation of an epidural injection simulator with force feedback for medical training. In: Medicine Meets Virtual Reality 2001, pp. 97–102. IOS Press (2001) 7. Dreifaldt, U., Kulcsar, Z., Gallagher, P.: Exploring haptics as a tool to improve training of medical doctors in the procedure of spinal anaesthetics. Eurohaptics 2006 (2006) 8. F¨ arber, M., Hummel, F., Gerloff, C., Handels, H.: Virtual reality simulator for the training of lumbar punctures. Methods Inf. Med. 48(05), 493–501 (2009) 9. Gorman, P., Krummel, T., Webster, R., Smith, M., Hutchens, D.: A prototype haptic lumbar puncture simulator. In: Medicine meets virtual reality 2000, pp. 106–109. IOS Press (2000)

106

M. B. Villar-L´ opez et al.

10. Henriksen, M.J.V., Wienecke, T., Kristiansen, J., Park, Y.S., Ringsted, C., Konge, L.: Opinion and special articles: stress when performing the first lumbar puncture may compromise patient safety. Neurology 90(21), 981–987 (2018) 11. Kanumuri, P., Ganai, S., Wohaibi, E.M., Bush, R.W., Grow, D.R., Seymour, N.E.: Virtual reality and computer-enhanced training devices equally improve laparoscopic surgical skill in novices. JSLS: J. Soc. Laparoendoscopic Surg. 12(3), 219 (2008) 12. Leung, K.M., Heng, P.A., Sun, H., Wong, T.T.: A haptic needle manipulation simulator for chinese acupuncture. Studies in Health Technology and Informatics, pp. 187–189 (2003) 13. Pirker, J., Dengel, A., Holly, M., Safikhani, S.: Virtual reality in computer science education: A systematic review. In: VRST 2020: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology. VRST 2020, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/ 3385956.3418947 14. Radianti, J., Majchrzak, T.A., Fromm, J., Wohlgenannt, I.: A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput. Educ. 147, 103778 (2020). https:// doi.org/10.1016/j.compedu.2019.103778. https://www.sciencedirect.com/science/ article/pii/S0360131519303276 15. Schulze, A.N.: User-centered design for information professionals. J. Educ. Libr. Inf. Sci. 42, 116–122 (2001) 16. Wright, B.L., Lai, J.T., Sinclair, A.J.: Cerebrospinal fluid and lumbar puncture: a practical review. J. Neurol. 259, 1530–1545 (2012)

Exploring Value and Ethical Dimensions of Disruptive Technologies for Learning and Teaching Terje Väljataga1(B) , Kai Pata1 , Andrea Annus1 , Michelle Andrade Calisto2 , Agueda Gomez Cambronero2 , Elmar Eisemann3 , Athina Kasini4 , Ricardo Marroquim3 , Inmaculada Remolar2 , László Szécsi5 , Amir Zaidi3 , and Rubén García Vidal2 1 Tallinn University, Narva mnt 25, 10120 Tallinn, Estonia

[email protected]

2 Universitat Jaume I, Castello de la Plana, Comunitat Valenciana, Spain 3 Delft University of Technology, Mekelweg 5, 2628 CD Delft, Holland 4 Center for Social Innovation, 62 Rigenis Street, 1010 Nicosia, Cyprus 5 Budapest University of Technology and Economics, M˝uegyetem rkp. 3, Budapest

1111, Hungary

Abstract. Disruptive technology has become an integral part of our lives, and it has brought about a significant transformation in the way we interact, communicate, and share information, also in the field of education. Innovation in technology needs to be based on ethics and values of the intended result. As the use of disruptive technology continues to grow, so does the need to understand and consider ethical and value dimensions. How can disruptive technology be developed and used in an ethical way for learning and teaching? What are the values the development and implementation of disruptive technology for education should take into account? How to measure and evaluate values and ethical dimensions of disruptive technology for educational purposes? Are some of the important questions to address. This workshop paper presents a method for eliciting values and ethical dimensions of learning scenarios with disruptive technologies in vocational and higher education settings and illustrates its implementation in the context of the Horizon Europe e-DIPLOMA project. The workshop method, combining value cards and learning scenarios with disruptive technologies, was implemented in seven different countries. The preliminary results of the workshops are presented. The method has the potential to draw peoples’ attention to prospective value concerns and ethical aspects necessary for understanding and acknowledging the consequence of implementing disruptive technologies in education. Keywords: Disruptive Technologies · Value Elicitation · Ethics · Higher Education

The original version of this chapter was previously published non-open access. A Correction to this chapter is available at https://doi.org/10.1007/978-3-031-42134-1_40 © The Author(s) 2023, corrected publication 2024 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 107–116, 2023. https://doi.org/10.1007/978-3-031-42134-1_11

108

T. Väljataga et al.

1 Introduction Technology has been and will be constantly evolving and developed further disrupting all levels of society, institutions, existing processes, social relations, values and the nature of human cognition and experience (Hopster 2021). This opens up new possibilities for improving the quality of life and work, but also brings along a number of threats, questionable values and unexpected conditions. The rapid development of disruptive technology challenges ethics and values, creating a situation in which social and ethical norms often struggle to keep up with technological development (Kritikos 2018). Without a question, there is a need to reconsider ethical and value aspects of emerging technologies, some, which have recently arisen, not been under focus or even thought of in earlier times. Among some others, particularly sensitive towards technological developments is the field of education, especially learning and teaching with disruptive technologies. Disruptive technologies refer to an innovation that displaces an established technology transforming traditional approaches and significantly altering existing ways of learning and teaching, therefore, having a potential to change the current understanding of education. Opening a discussion on ethical and moral values of constantly emerging disruptive technologies that are used for educational purposes has of utmost relevance in the midst of ever growing smart algorithms, constantly developing machine learning, data sets of digital traces and big data analytics, artificial intelligence, smart sensors, etc. (WCO 2019). Consequently, it is important to raise the questions such as How can disruptive technology be developed and used in an ethical way for learning and teaching? What are the values the development and implementation of disruptive technology for education should take into account? How to measure and evaluate values, ethical dimensions and sustainability of disruptive technology for educational purposes?, but also provide evidence-based solutions. This workshop paper presents a method for eliciting values and ethical dimensions of learning scenarios with disruptive technologies and illustrates its implementation in the context of the Horizon Europe e-DIPLOMA project.

2 Background 2.1 Disruptive Technologies and Ethical Considerations Some emerging technologies are able to trigger profound changes and disrupt existing structures and norms, others not. The term “disruptive technology” was coined by C. Christensen, who used the term in the context of disruptive innovation theory in business (Christensen 1997). However, being criticized by many scholars (Tellis 2006), this understanding of disruptive technologies is not best suited for ethics because of its theory-laden conceptualisation (Hopster 2022). In recent years “digital disruption” as an emerging concept (Skog et al. 2018) has gained attention and has a potential in the context of ethics as it focuses on technology rather than innovation (Hopster 2022). According to Cambridge dictionary, to disrupt means “to prevent something, especially a system, process, or event, from continuing as usual or as expected”. Thus, disruption is usually perceived as a negative occurrence triggered by outside factors (Boucher et al. 2020). Schuelke-Leech (2018) makes a distinction based on the depth and scope of the disruption and classifies disruption as 1) first-order (local market disruptions, certain domains are the only ones experiencing change, while society as a whole is not affected;

Exploring Value and Ethical Dimensions of Disruptive Technologies

109

ethical issues and values are domain specific); 2) second-order (systematic disruptions at societal scale, technologies’ capacity to alter society and individuals). Accordingly, Hopster (2022) in turn divides technologies as disruptive based on their technological features or based on their societal impacts, creating different starting points for assessing ethical issues, but also bringing forth a different set of ethical concerns (Hopster 2022). Thus, according to Hopster (2022) there are two different ethical foci: (1) technology as an agent of social disruption, (2) technology entangled in social disruption. Table 1 below outlines the main differences between these two starting points. Table 1. Two ethical foci according to Hopster (2022). Technology as agent of social disruption

Technology entangled in social disruption

Primary assessment

disruptive potential of technologies

technological disruptions in society

Starting point of ethical inquiry

features of a disruptive technology technosocial disruptions

Conceptualisation of technologies

in terms of technological artifacts, in terms of technological or fields of R&D and their applications, or contexts of constitutive techniques implementation

Focus

responsibility of innovators and disruptors

societal response to disruptions

For ethical analysis of disruptive technologies, the aforementioned deviation plays a crucial role, however, evaluating disruptive technologies in the context of education, the focus is usually on technology as an agent of disruption. Nevertheless, depending on the emerging technology that is implemented for learning and teaching may have already been used in other contexts, thus widening its scope, but also its set of ethical concerns. 2.2 Values of Disruptive Technologies Innovation in technology needs to be based on the values of the intended result i.e., to make our lives better. Technology developers usually have some motives and promise certain values while balancing at the same time between expected values and ethical considerations. While not all emerging technologies will alter the target field or social landscape, some have the potential to significantly disrupt the status quo, reshape how people live and work. In recent years, values as well as ethics have gradually become part of the design process, building on methodologies, such as the value sensitive design approach, ethically aligned design, etc. (Kritikos 2018), which consider values as an important part and aspire to understand the values of the users, ensure that these values are carefully considered and implemented in the design of the technology (Knobel and Bowker 2011). However, there are many interpretations of how value is understood and what kind of values are in focus (be it an economic, human, social, value to the end-user, etc.) (Gilmore et al. 2008). Iversen et al. (2012) claim that “Values have a transcendental quality, guiding actions, attitudes, judgments and comparisons across specific objects and situations and beyond immediate goals to more long-term goals”

110

T. Väljataga et al.

(p. 143) or as stated by Borning and Muller (2012) “human values are simply defined as “what a person or group of people consider important in life” (p. 1125). As disruptive technologies continue to progress, there is a need for an increasing set of different values that guide the development and use of technologies. In general, technology can be used productively or destructively (Weinberg 2019). Incorporation of new technology without considering potential consequences can cause significant societal and cultural shifts and occasionally disrupt already established social and cultural norms (Kritikos 2018). Furthermore, it may challenge our traditional moral understandings and reshape our value assumptions and practices creating ambiguity around what is considered acceptable, normal and ethical behavior (Kritikos 2018). As a result, technology becomes a carrier and even a disruptor of values, compelling individuals and communities to adjust to technology rather than leveraging it to enhance human potential in accordance with societal objectives and customs (Weinberg 2019). Our decisions regarding technology, particularly when made without thoroughly considering their consequences, may contradict our fundamental goals, principles, and values. Media certainly plays its role here through its dominant messages as well as personal experiences with technologies that affect peoples’ vision and understanding of the ethics and values of new emerging technologies. Consequently, as individuals, families, communities, and societies, we must contemplate how we create and utilize technological instruments (Weinberg 2019). Furthermore, to guarantee that individuals have capacity to make wise technical decisions and technology is employed in a way that promotes both individual and societal well-being, technological assessment is necessary at all levels of society. These decisions are an expression of social, cultural, economic, political, ethical, and spiritual values (Weinberg 2019). Thus, value-centered design approaches and value elicitation methods are important tools to evaluate value and ethical dimensions of emerging disruptive technologies. 2.3 Value Elicitation Methods Eliciting values and ethical dimensions of technological disruption, various methods (value scenarios, in-depth interviews, workshops, etc.) have been utilised, each of them with their own advantages and disadvantages. The most common and widespread approach is using cards (Mora et al. 2017). Eliciting values and ethical dimensions of disruptive technologies, cards can facilitate conversation, stimulate the creative exploration of the design space, help the participants to reframe technical problems (Friedman and Hendry 2012), provoke reflective thinking, help participants to initiate and be focused in brainstorming sessions (Fedosov et al. 2019). Another method for eliciting values and considering ethical dimensions of potential activities with the disruptive technologies is to provide scenarios or design fictions (Cheon and Su 2018). Scenarios and design fictions provide a speculative space that helps to envision ways of using disruptive technologies and potential emerging value and ethical issues. They can help to focus attention on value tensions, and longer-term societal implications that might otherwise go unnoticed (Czeskis et al. 2010), support participants to understand the implicit future ethical and sustainability consequences of technology (Blythe 2017). In the context of the e-DIPLOMA project, the value cards were combined with the short description of learning and teaching scenarios supported by particular disruptive technologies.

Exploring Value and Ethical Dimensions of Disruptive Technologies

111

2.4 e-DIPLOMA Project e-DIPLOMA (Electronic, Didactive and Innovative Platform based on Multimedia Assets) is a Horizon Europe project (HORIZON-CL2-2021-TRANSFORMATIONS01) that will examine the current e-learning ecosystems by analysing the usefulness of disruptive technologies (Virtual Reality, Augmented Reality, Artificial Intelligence or Chatbots) to support education and training systems, specifically in practical distance learning; co-create methodology to determine which technologies are the most suitable for dealing with different types of educational methodologies; develop online platform that will integrate artificial intelligence, virtual and augmented reality, interactive technologies and gamification techniques. The project consortium consists of 9 partners, 14 associated partners from 8 different European countries. This project will overcome the weaknesses of the current e-learning by exploring the potential of disruptive technologies applied to the e-learning. It will go a step forward of the current state of art creating high quality content focused on experiential e-learning, an engaged learning process whereby students “learn by doing”. The project is mainly addressed to tertiary education or post-secondary education. It is based on experiential learning activities that raise the employability of the students.

3 Method 3.1 Value Elicitation Workshop in the Context of the e-DIPLOMA Project For evaluating values of disruptive technologies from an ethical and sustainability point of view a value-elicitation method together with the learning scenarios with disruptive technologies was developed. The value - elicitation method is a type of workshop with group interviews, which lasts usually around 2,5 h. The aim of the method is to elicit ethical values and sustainability aspects of disruptive technologies through different learning scenarios and assess potential benefits and vulnerabilities, which might arise from the use of disruptive technologies for learning and teaching. To support and direct workshop participants’ thinking and conceptualisation of learning scenarios with new disruptive technologies, value-elicitation cards were created (Mora et al. 2017). Every card has one value with its description. For instance, Privacy - The state of an agent, asset or system where it regulates its level of openness to external disturbances and relations to minimal or Productivity - The quality of agents or systems to efficiently transform inputs into useful outputs. The chosen set of values was drawn from the literature related to the use of disruptive technologies. A total of 45 value cards were created. Combining value cards with learning scenarios with disruptive technologies brings in the process playfulness and creativity (Lucero et al. 2016) to spark discussion, to expand participants’ minds about the existing values and give vocabulary for them to think and talk about ethics and sustainability aspects related to the particular learning scenario and technology. Learning scenarios were presented as short stories consisting of disruptive technology description, objectives and explanations of learning tasks with the particular technology (Fig. 1). For better understanding some visuals and pictures were added. Although fictional, the scenarios were grounded in actual products and learning events derived from and modified from different research studies in the field. For the e-DIPLOMA workshops following scenarios were presented to the workshop participants: 1. Virtual reality

112

T. Väljataga et al.

for the fire extinguisher training; 2. Supporting learning progress with AI in physics; 3. Cooking class in zoom with the augmented reality elements; 4. Telepresence robot in foreign language course. An example of the scenario is seen in Fig. 1.

Fig. 1. Example of the learning scenario with disruptive technology.

The research participants were divided into groups of 4–5 people. It was recommended that participants with different backgrounds (students, lecturers, educational technologists) form a group. The reason for these mixed groups is the different perspectives the specific target groups have regarding learning and teaching with disruptive technologies. In the workshop, each group received one learning scenario and a set of cards, read the scenario, sorted through cards, and then selected up to 8 cards that represent values in the technology presented in the learning scenario they were analyzing or should be there, but were missing. There are various ways for groups to approach the task, such as working collaboratively, which involves discussing all cards together, or working cooperatively, which entails dividing the cards among group members. Collaboration format was left for the groups to decide. At the end of this part of the workshop, participants had a selection of values that they thought should be / were present in the learning scenario. Next, groups filled in the worksheet for value descriptions. In each worksheet, the group formulated one selected value and 3–5 sentences how this value relates with the scenario (e.g. with interactions of people, interaction between people and the system, with algorithms, data, at society level). Teams submitted their responses about each selected value separately. Finally, reflective feedback between groups was carried out. Every team introduced orally within 5 min the scenario and explained how the values relate with this scenario. If some new value aspect emerged in the discussion, the team added it to the value analysis (in the worksheet).

Exploring Value and Ethical Dimensions of Disruptive Technologies

113

3.2 Data Collection and Analysis 7 countries (Spain, Netherlands, Italy, Hungary, Cyprus, Bulgaria, Estonia), the partners of the e-DIPLOMA project, organised their own workshops following the same guidelines and format as described above. The guidelines, value cards and learning scenarios were translated into local languages. Data from the partners was collected according to the learning scenarios, chosen value cards by the groups and group explanations related to the selected value card. All the value explanations were translated back to English for joint data analysis. For analysis, simple descriptive statistics as well as first-stage thematic analysis was used.

4 Preliminary Results of the International Value-Elicitation Workshops A total of 187 entries were created, of them 58 different values were presented (either from the pre-prepared value cards or new ones added by the participants) and discussed in different groups related to the four learning scenarios with disruptive technologies. According to the learning scenarios, more or less the same number of different cards were chosen, except for one scenario - Virtual reality (VR) for fire extinguisher training - which received 58 entries, the others between 41 and 46. Some trends of mentioning some values more often in specific countries could be observed with the values of Coercion, Accuracy and Accessibility, but due to the small sample size in the dataset we could not confirm country-specific differences in the values. The most often chosen value was Accessibility, selected 13 times for the three learning scenarios except for Supporting learning progress with AI in physics. The other more popular values with 7 entries were Accuracy, Flexibility, Responsibility, Surveillance, Sustainability, Trust, with 6 entries were Adaptability, Autonomy, Coercion, Connectivity and with 5 entries were Confidentiality, Consensus, Productivity, Satisfaction, Vulnerability. The rest of the values were selected less often by the participating groups. There was only one value - Flexibility chosen by the participating groups, which seemed to be relevant for all the four learning scenarios. The value dimensions that occurred in at least three scenarios (Accessibility, Connectivity, Vulnerability, Trust, Involvement, Autonomy, Control, Surveillance, Challenging, effectiveness, Productivity, Accuracy, Sustainability, and Satisfaction) indicate the value perspectives that meant most to people when they saw the learning scenarios with disruptive technologies. A comprehensive birds-eye view on the selected values related to the learning scenarios are demonstrated in Fig. 2. We also noticed that in the scenarios of Telepresence robots in foreign language course and Supporting learning progress with AI in physics were more concern-related values, such as Trust, Vulnerability, Equity, Fairness, and Autonomy. However, the negatively connotated values such as Confidentiality, Privacy, Coercion, Control and Surveillance were also perceived in regards to scenarios with augmented reality (AR) Cooking class in zoom with the augmented reality elements and virtual reality (VR) Virtual reality for the fire extinguisher training, and not only with the scenario with AI Supporting learning progress with AI in physics. The participants’ explanations and rationale for selected values varied. For instance, Disruption, usually evaluated as a negative occurrence, was

114

T. Väljataga et al.

Fig. 2. Example of the learning scenario with disruptive technology.

perceived also as a positive or driving forward value or with a negative connotation. For example, the most often selected value Accessibility was seen as a hindrance for students i.e. in the learning scenario “Cooking class in zoom with the augmented reality elements” one of the groups explained that “students are less accessible to teachers, making it harder to identify struggling students” or as a beneficial aspect, which is explained by one of the groups “It allows access to educational resources regardless of geographic distance and economic resources”. Yet another example is related to the value Confidentiality, which is often perceived as a threat with regard to disruptive technologies. The main argument is that “One cannot be sure who has access to the digital data in the system”. On the other hand, in the case of Supporting learning progress with AI (artificial intelligence) in physics learning scenario, one of the groups has presented it as a promising aspect “GoTrack data will only be displayed to the teacher via the teacher dashboard. The confidentiality of the discussion is guaranteed because it is available to the teacher. Thus, based on the provided examples, the value elicitation method allows to bring out participants’ critical perspectives of certain values and contradict them to the positive ones. The preliminary thematic analysis of participants’ explanations for every value resulted in themes as follows: risk, spatial quality, group, learning process, learner intrapersonal qualities, learning management, health/bodily reactions, cognitive effects, system’s capability, beneficial qualities of the environment, resources. The most often the participants pointed out Group related aspects (such as enabling collaboration, participation, connectedness, peer interaction, social learning, role distribution, etc.) as mainly positive features and potentially emerging Risks (such as control, personal data, reliability and monitoring issues, unwanted exposure, unsuitable for older learners, etc.) as negative consequences of the learning scenarios with disruptive technologies.

5 Conclusions Disruptive technologies bring along the need to take a deeper look at the ethical and value dimensions in order to make wise technical decisions and employ technology in a way that promotes both individual and societal well-being. We have presented a technique with some preliminary results to better and more efficiently elicit values and

Exploring Value and Ethical Dimensions of Disruptive Technologies

115

ethical dimensions related to disruptive technologies in educational settings. The method, combining value cards and learning scenarios with disruptive technologies, facilitates conversation, widens one’s thinking space and helps the participants to verbalize their perceived concerns as well as possibilities regarding the ethical and value dimensions of disruptive technologies in educational settings. The results point out the most important values and potential value spaces of different disruptive technologies perceived by different target groups, and provide a list of themes that occurred related to the selected values. Our next step is to look deeper into the value spaces of the disruptive technologies and look for patterns and connections of values and disruptive technologies. The outcome would help the developers, designers and policymakers to understand potentials and threats of disruptive technologies in educational settings. Acknowledgements. Research supported by the e-DIPLOMA, project number 101061424, funded by the European Union. Views and opinions expressed are, however, those of the authors only and do not necessarily reflect those of the European Union or the European Research Executive Agency (REA). Neither the European Union nor the granting authority can be held responsible for them.

References Blythe, M.: Research fiction: storytelling, plot and design. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 5400–5411 (2017). https://doi.org/ 10.1145/3025453.3026023 Borning, A., Muller, M.: Next steps for value sensitive design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1125–1134 (2012). https://doi.org/ 10.1145/2207676.2208560 Boucher, P., Bentzen, N., Latici, T., Madiega, T., Schmertzing, L., Szczepa´nski, M.: Disruption by technology: impacts on politics, economics and society. European Parliamentary Research Service, 2020. PE652.079 (2020). https://www.europarl.europa.eu/stoa/en/document/EPRS_I DA(2020)652079 Cheon, E., Su, N.M.: Futuristic autobiographies: weaving participant narratives to elicit values around robots. In: Proceedings of the 2018 ACM/IEEE International Conference on HumanRobot Interaction, pp. 388–397 (2018). https://doi.org/10.1145/3171221.3171244 Christensen, C.M.: The Innovator’s Dilemma: The Revolutionary Book that Will Change the Way You Do. Business. Harvard Business School Press, Boston (1997) Czeskis, A., et al.: Parenting from the pocket: value tensions and technical directions for secure and private parent-teen mobile safety. In: Proceedings of the Sixth Symposium on Usable Privacy and Security, pp. 1–15 (2010). https://doi.org/10.1145/1837110.1837130 Fedosov, A., Kitazaki, M., Odom, W., Langheinrich, M.: Sharing economy design cards. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2019). https://doi.org/10.1145/3290605.3300375 Friedman, B., Hendry, D.: The envisioning cards: a toolkit for catalyzing humanistic and technical imaginations. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1145–1148 (2012). https://doi.org/10.1145/2207676.2208562 Gilmore, D.J., Cockton, G., Churchill, E., Kujala, S., Henderson, A., Hammontree, M.: Values, value and worth: their relationship to HCI? In: Proceeding of the Twenty-Sixth Annual CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI 2008, p. 3933 (2008). https://doi.org/10.1145/1358628.1358960

116

T. Väljataga et al.

Hopster, J.: What are socially disruptive technologies? Technol. Soc. 67 (2021). https://doi.org/ 10.1016/j.techsoc.2021.101750 Hopster, J.: The ethics of disruptive technologies: towards a general framework. In: de Paz Santana, J.F., de la Iglesia, D.H., López Rivero, A.J. (eds.) DiTTEt 2021. AISC, vol. 1410, pp. 133–144. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-87687-6_14 Iversen, O.S., Leong, T.W., Wright, P., Gregory, J., Bowker, G.: Working with human values in design. In: Proceedings of the 12th Participatory Design Conference: Exploratory Papers, Workshop Descriptions, Industry Cases - Volume 2, pp. 143–144 (2012). https://doi.org/10. 1145/2348144.2348191 Knobel, C., Bowker, G.C.: Values in design. Commun. ACM 54(7), 26–28 (2011). https://doi.org/ 10.1145/1965724.1965735 Kritikos: What if technologies challenged our ethical norms? AT A GLANCE Scientific Foresight: What if? European Parliamentary Research Service (2018) Lucero, A., Dalsgaard, P., Halskov, K., Buur, J.: Designing with cards. In: Markopoulos, P., Martens, JB., Malins, J., Coninx, K., Liapis, A. (eds.) Collaboration in Creative Design, pp. 75– 95. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29155-0_5 Mora, S., Gianni, F., Divitini, M.: Tiles: a card-based ideation toolkit for the Internet of Things. In: Proceedings of the 2017 Conference on Designing Interactive Systems, pp. 587–598 (2017). https://doi.org/10.1145/3064663.3064699 Schuelke-Leech, B.: A Model for understanding the orders of magnitude of disruptive technologies. Technol. Forecast. Soc. Change 129, 261–274 (2018) Skog, D.A., Wimelius, H., Sandberg, J.: Digital disruption. Bus. Inf. Syst. Eng. 60(5), 431–437 (2018) Tellis, G.J.: Disruptive technology or visionary leadership? J. Prod. Innov. Manag. 23, 34–38 (2006) Utterback, J.M, Acee, H.J.: Disruptive technologies: an expanded view. Int. J. Innov. Manag. 9(1), 1–17 (2005) WCO: Study Report on Disruptive Technologies (2019). http://www.wcoomd.org/-/media/ wco/public/global/pdf/topics/facilitation/instruments-and-tools/tools/disruptive-technologies/ wco_disruptive_technologies_en.pdf?la=en Weinberg, M.: Technology, Values, and the Shaping of Social Reality. Bahai World, p. 5 (2019). https://bahai-library.com/weinberg_technology_values_reality

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Workshop on Interactive Environments and Emerging Technologies for eLearning (IEETeL)

Implicit Aspects of the Psychosocial Rehabilitation with a Humanoid Robot Maya Dimitrova1(B)

, Virginia Ruiz Garate2,3

, Dan Withey2 , and Chris Harper2

1 Institute of Robotics, Bulgarian Academy of Sciences, 1113 Sofia, Bulgaria

[email protected]

2 Bristol Robotics Lab, University of the West of England, Bristol BS34 8QZ, UK 3 Mondragon Unibertsitatea, 48014 Bilbao, Spain

Abstract. The paper presents an approach to understanding the emergent dynamics of human-robot interaction in the specific case of a humanoid robot taking the role of a co-therapist to the human therapist in psychosocial rehabilitation by focusing on the nonverbal communication in these sessions. The intention is to eliminate the negative effects of reactance to the robot by employing the SociBot platform, which has been designed with enhanced facial features. An empirical study is performed to investigate implicit aspects of perceiving faces of different modality. The results show that viewers assess differently the positive and negative features, which could be attributed to the presented faces – human, android or robotic (machine-looking). At the same time, no effect of type of face on the feature assessment process was observed. Some implications for using robots as co-therapists are briefly discussed as an emerging interactive technology in support of people with special needs. Keywords: Psychosocial Rehabilitation · Humanoid Robot · SociBot · Reactance · Uncanny Valley · Likert Scale · ANOVA

1 Introduction Psychosocial rehabilitation in the recent years, especially after a global pandemic, is gaining substantial significance with an increasing number of people experiencing feelings of fear and anxiety, e.g. [1]. From an ‘exotic’ treatment, which is not always affordable, it is transforming into a crucial vital necessity. Robotic solutions can help in bridging the affordability gap if being employed as entities, which implement professional skills to help the therapist in psychosocial rehabilitation, e.g. [2, 3]. In the process of communication with the robot co-therapist, the emergent interaction is structured by intelligent algorithms, thereby providing inherent control on the communication process for the aims of the successful therapy in a manner, similar to the proposed in [4]. The implementation challenges are still ahead since the humanoid robot - in its various appearances and with its diverse behaviours - is a radically novel complex stimulus for the cognitive system of the human (in some evolutionary sense). Novel experimental studies reveal the way the cognitive system discovers (or defines) © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 119–128, 2023. https://doi.org/10.1007/978-3-031-42134-1_12

120

M. Dimitrova et al.

similarities and differences of the perceived human professional and the humanoid robot (or android), designed to support this professional [5]. However, current studies provide still insufficient number of established facts about the way the cognitive system processes similarities and differences between a human and a humanoid robot [6, 7]. Extending such understanding via further studies can help in formulating novel approaches towards the self-analysis and self-improvement as a result of the counseling session [8], involving, in particular, a therapist, supported by a humanoid robot. In a broadest sense the robot is linked to the internet and can provide learning material as any MOOC type of university environment for assigning courses or course work to students, but also tasks, related to self-assessment or strategies for self-improvement. The specific aim of the present study is to explore the very basic level of acceptance of robots before implementing some verbal interactions. The research question is if a dramatic reaction to the robot is observed on some visceral level, or it is being accepted as a functional device designed to perform (efficiently) the designated social role. 1.1 A Humanoid Robot as a Co-therapist Co-therapy is a popular method to deal with psychosocial rehabilitation of families, groups, or individual psychotherapy [9, 10]. We focus on the potential to implement robots as co-therapists in psychosocial and pedagogical rehabilitation [11–14]. The model situation is counselling – a client seeks the help of a professional to cope with undesirable situations in their life by learning new social skills and strategies and reformulating their personal goals [15]. The co-therapeutic sessions include two actors - an active and a passive therapist. The active therapist is leading the session by engaging the client, asking questions, and directing the dialogue. Her/his role seems leading, but actually it is secondary. The passive therapist observes the session and takes notes on the entire process. Her/his role seems secondary, but in fact it is the leading role. In the subsequent analysis of the session and the client’s progress - the dominant role is of the passive therapist because she has been able to carefully observe the entire process and draw relevant conclusions [10]. In the present research it is proposed that the active actor is the robot co-therapist – a role for the humanoid robot that has not been explored to a sufficient extent yet. The overall research aim is to investigate the plausibility of the hypothesis that humanoid robots can perform various social roles successfully and support the humanprofessional in tasks like, for example, co-therapy. On the one hand - a lot of algorithmic knowledge in the form of rules to be performed by the help seeker/learner is employed in such scenarios and can be modelled via AI (e.g. a popular ChatGPT actor is being already around1 ). On the other – the situation complexity is still immensely high and requires substantial amount of creativity on the part of the human professional in most of the helping professions like counseling, social work, or education. Robots, programmed to imitate social roles, can bring co-therapy to a new level and better support the human professional and, at the same time, be entertaining and supportive to the help seeker - to a larger extent - due to the more sophisticated technology available today. However, the cognitive system of the human not always perceives the external stimuli as motivating to 1 https://openai.com/blog/chatgpt.

Implicit Aspects of the Psychosocial Rehabilitation

121

learn, especially when social skills are being involved in skills training situations [15]. Sometimes reactance is being observed, or a deeper visceral reaction like the so-called Uncanny Valley effect [16, 17]. The SociBot robot provides an appropriate platform for designing psychophysical experiments on human reaction to a variety of robotic behaviours on both visceral and cognitive levels [17]. The visceral level of reaction to a cognitive stimulus is similar to the reactance to the stimulus [16], but is characterized by response of body organs – stomach spasms, heartbeat, sweating, etc., whereas the reactance is felt as strong unwillingness to comply with the call of the stimulus. A substantial amount of the studies on robot acceptance by the user refer to humanoid robots with machine-like appearance such as Pepper [18] or NAO [19]. The expressed ‘friendliness’ by these robots is very well accepted by the users. Yet, the process of human-robot interaction (HRI) in general is being characterized by specificities such as the following: people do not typically focus on the robot face [20], gaze cuing by a robot is not spontaneous for the human [21], there is schematic or no lip reading in the human-robot dialogue [22], etc. In studies with a similar to the SociBot humanoid robot head designed via projecting the face features from an internal beamer, such as the one designed by Furhat robotics, the Wizard-of-Oz framework is being implemented [23]. We are using the deployed SociBot autonomous mode instead, signified by slight movement and eye blinking, following [6, 7]. We compared the viewer assessment of three ‘naturally’ moving faces (machine-like, android and human), when viewing videos, in a study, designed to understand the human visceral reaction underlying any further interaction – as presented in the next section. 1.2 Psychological Reactance to Advice Given by the Humanoid Robot SociBot A set of thorough studies of user acceptance of the SociBot revealed the user preference to a specific level of implemented social cues for better compliance with the advice given by the robot [6, 7]. The artificial agents were: a laptop, displaying written text, a SociBot with deployed minimal social cues, and a SociBot with a larger set of social cues, accompanying the verbal utterances of the robot. The minimal social cues were eye blinks and neutral face, whereas the larger set of social cues included head movements, facial expression, and tone of voice. The verbal utterances of the robot were coercive and could be perceived as frustrating. The two conditions were a) robot giving advice how to prepare a smoothie for the human (realistic, i.e. with maximum psychological involvement); or b) robot giving advice how to prepare a smoothie for an alien (unrealistic, i.e. with minimum psychological involvement). The main outcome from this study was defined by the total number of cases when the human complied with the robot advice and changed the recipe, depending on the experimental condition. It has turned out that the preferred robot case was SociBot with minimal social cues. The advice given by this robot was accepted most often and with the highest level of compliance. Therefore, the psychological reactance [24] towards the robot was minimal and the level of robot acceptance maximal when the robot displayed minimal social cues. This effect has justified our attempt to further investigate the acceptance of a minimal social cueing expressed by 2 types of robots and a human.

122

M. Dimitrova et al.

Our main hypothesis for the study is that the minimal social cueing of the SociBot robot, yet implementing realistic eye blinking and lip-synching, will demonstrate its appropriateness for maintaining dialogue with the person, seeking help, to a greater extent than, for example, a machine-looking robot with no ‘true’ eye blinking (just light flashing) and lip-synching. The results are only preliminary and the actual formal model of abstract robot feature representation has not been proposed yet. 1.3 Does the Uncanny Valley Effect Pose a Risk to Including a Humanoid Robot as a Co-therapist in Psychosocial Rehabilitation? The Uncanny Valley phenomenon [16] (and the related psychological reactance effect [6, 7, 24]) seems to be the most important obstacle to the implementation of humanoid robots in professions, which support human well-being via enhanced human-robot interactions, e.g. [17]. The nature in the phenomenon is in the sudden negative visceral reaction to an artificial object, which is designed to imitate the tissue and other attributes of a living creature/animal. As mentioned before, the psychological reactance can be considered a type of a visceral reaction with much lesser amplitude. The original reaction, investigated by the proposer of the theory behind the phenomenon Masahiro Mori [25] in 1970, was repulsion, experienced when shaking an artificial hand, believed initially to be real, but felt strange, cold, and unnatural. In the subsequent theoretical accounts, the Uncanny Valley effect was said to be due to experiencing “conflicting cues” [16] on different visceral and psychological levels. In the present study we included Likert scales to assess the degree, to which 3 positive and, respectively, 3 negative, characteristics can be associated with the presented faces. The aim was to initially assess the attitudes towards associating personality features to robotic faces, where strong assessment would signify reaction at a visceral level (especially in the case of negative features), and weak assessment (values around indifference) – reactions at a psychological level. Therefore – the indifference towards the presented robotic or human agents would be positive in terms of the possibility for endowing those agents with more pronounced social roles in a variety of professions, where mirroring the subjective state of the help seeker by a neutral personality is an asset.

2 The Experiment Sixteen participants took part in the experiment – 10 female and 6 male participants of age from 24 to 62 with normal and corrected to normal vision. The study was approved by the Ethics committee of IR-BAS. 2.1 Procedure The experimental procedure consisted of the presentation of brief videos of faces of 3 agents, named Roberta, Alice and Violina (Fig. 1). Roberta is a popular and familiar robot NAO, seen on the web or TV, performing the ‘autonomous life’ movement. The second video is SociBot called Alice with a projected female face in ‘autonomous life’

Implicit Aspects of the Psychosocial Rehabilitation

123

mode, too. Violina is a human actor, who agreed to participate. She was shown the autonomous life mode of the robots Roberta and Alice to get the idea on the type of presence she was supposed to perform in the video. Since the video fragments were brief – 3 s, the life mode was noticeable mainly by the blinks of these faces and some slight head movement.

Fig. 1. Short videos of faces in autonomous life mode for the 2 robotic faces Roberta and Alice, and a neutral natural expression of the human face Violina (from left to right).

An HP Pavilion 15.6 inch laptop was used for the stimulus presentation. The trial was in 3 phases. Phase 1 began by presenting a set of instructions on the screen, followed by a 10 s exposure of the photos of Roberta, Alice and Violina as in Fig. 1. Each face was presented for 10 s each and the participant was asked to name the presented photo in order to remember the name of the face. This was repeated twice. In phase 3 the participants were presented with the video of each agent in autonomous life mode for 15 s and asked to look at the face with no specific instruction. The intervals between the faces were 3 s of black screen with a cross in the middle. This was intended to remove any priming effect on face perception. Finally, the participants were presented with one of 6 Likert scales to assess the degree, to which each face can be associated with a personality trait. The traits were selected from the study of [26]. The faces were assessed along 3 positive and 3 negative dimensions in pseudo-random order. The positive dimensions were: Emotional stability, Sociability and (being) Trustworthy. The negative dimensions were: Aggression, Weirdness and Threat. The responses were made on a sheet of paper by circling the selected number of the Likert scale, presented on the respective slide. The participants were asked to rate the degree, to which the presented characteristic can be associated with the presented face from – 3 – ‘completely impossible to associate’, to +3 – ‘completely possible to associate’.

124

M. Dimitrova et al.

2.2 Results and Discussion The mean ratings of each feature, associated with each face, from 16 participants were processed by 2-way ANOVA (with replication) in Excel. The two factors were ‘type of face’ with 3 levels (Roberta, Alice, Violina) and ‘valence of the characteristic’ (positive – emotional stability, sociability, trustworthiness – and negative – aggression, weirdness, threat). The main result was the following: The factor ‘type of face’ did not reach significance level, F(2, 17) = 0.541, p = 0.596. The factor ‘valence’ reached significance level, F(1, 17) = 28.949, p = 0.000165. The interaction between these two factors was significant: F(2, 17) = 9,666, p = 0.003156. These effects can be seen in Figs. 2 and 3.

Fig. 2. Mean scores of positive features, attributed to different faces.

The present result is an important step towards understanding the ‘visceral’ reaction of the viewer towards 3 simultaneously present ‘live’ agents – a human and two different types of robots – machine-like and android. The type of face did not affect the ratings of the viewers. This is a very important outcome, supporting the assumption that robotic agents are being assessed and accepted as normal as the human agents in every-day situations. The presence of a robot, or a human, does not influence the objectivity of viewers’ assessment of these agents. The type of characteristic to be evaluated, however– positive or negative - influences the way the presented faces are being evaluated – actually, the magnitude of the positive scores is bigger (see Fig. 2), whereas the negative scores are being made in a more cautious way and are centered around the indifferent zero score (see Fig. 3). The observed interaction of both factors supports this observation. The human face is attributed the highest amount of positive features. The negative features are given negative marks – i.e. towards the “impossible to associate with” range. This observation validates the overall approach, since the silent human presence is the most preferred option.

Implicit Aspects of the Psychosocial Rehabilitation

125

Fig. 3. Mean scores of negative features, attributed to different faces.

The machine-looking NAO robot, which is especially designed to attract children’s attention, is very well accepted by the current viewers. The pattern of results supports the idea of specialization in endowing robots with functional roles – Roberta seems to the viewers quite sociable and emotionally stable, as well as least aggressive or threatening. The assessment of the android type of robot Alice is the most neutral. Viewers give Alice indifferent scores – around zero, except for one of the features – weirdness. The score is not quite high to invoke a negative visceral reaction, since the mean score of 1.69 is between ‘indifferent’ and ‘possibly associated with weirdness’. This robot is being seen by the viewers for the first time, therefore it is logical to assume it being weirder than the other and the human. It was established that the Uncanny Valley effect is being reduced with the increased familiarity of the robot after several repetitions [27]. Alice is being associated with emotional stability, which presumes it being appropriate for professional roles, requiring being neutral and, possibly, able to mirror the emotional state of the person, who seeks help. We conducted statistical analysis of scores given by female (10) vs. male (6) participants. No statistical difference was observed on any level of analysis, nor any interaction. Moreover, the correlation of the evaluation of the human face of the female and male participants was 0,75, whereas with robotics faces it was lower. Therefore, the distinction ‘human’ vs ‘non-human’ was an implicit reaction of the cognitive system to the complex face stimulus irrespective of the viewer gender. In overall, the participants tend to like the three stimuli, and are cautious in projecting negative traits to the robots. This is the main outcome of the present study favouring the idea of endowing humanoid robots with social skills to support the effort of the human professional in various jobs. Future work will explore whether machine looking robots can be better teachers, and android looking –better co-therapists.

126

M. Dimitrova et al.

We plan to explore the potential of employing the text-to-speech module of SociBot in co-therapy to help assess, communicate, reflect, remind, repeat, and mirror personal experiences to enhance the process of the so-called ‘self-controlled attitude shift’– and support the effort of the therapist. The respective verbal lines have been recorded for inclusion in the experimental design as a next step in the study on the appropriateness of implementation of humanoid robots as co-therapists in psychosocial rehabilitation.

3 Conclusions The paper presented an initial study of human assessment of characteristics, associated with ‘live’ faces for the purposes of including a humanoid robot as a co-therapist of the human therapist in psychosocial rehabilitation. The appropriateness of the SociBot platform, which has been designed with enhanced facial features in order to avoid the negative effects of reactance to the robot, was the main focus of the study. The statistical analysis of the data demonstrated that viewers assess differently the positive and negative features, which could be attributed to the presented faces – human, android or robotic (machine-looking). On the other hand, no effect of type of face on the feature assessment process was observed. This can be interpreted in favour of the idea of endowing professional roles to robots as co-therapists to the human therapist as an emerging interactive technology in support of people with special needs. Acknowledgement. The authors express their gratitude to actress Mrs. Violina VasilevaAlexandrova for participating in the video as the human agent, and Ms. Gagandeep Kaur for her help in the design of the stimulus. The research, leading to these results, has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 730994 (TERRINet), the Research Fund of Bulgaria for projects №. KP-06H42/4 (2020-2023), No. KP-06-PH57/28 (2021–2024) and OP Science and Education for Smart Growth (2014–2020) for project Competence Center “Intelligent mechatronic, eco- and energy saving systems and technologies” BG05M2OP001-1.002-0023.

References 1. Chaturvedi, S.K.: Covid-19, coronavirus and mental health rehabilitation at times of crisis. J. Psychosoc. Rehabil. Ment. Health 7, 1–2 (2020) 2. Utami, D., Bickmore, T.: Collaborative user responses in multiparty interaction with a couples counselor robot. In: 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 294–303 (2019) 3. Utami, D., Bickmore, T.W., Kruger, L.J.: A robotic couples counselor for promoting positive communication. In: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 248–255 (2017) 4. Jokinen, K., Wilcock, G.: Constructive interaction for talking about interesting topics. In: LREC, pp. 404–410 (2012) 5. Kelley, M.S., Noah, J.A., Zhang, X., Scassellati, B., Hirsch, J.: Comparison of human social brain activity during eye-contact with another human and a humanoid robot. Front. Robot. AI 7, 599581 (2021)

Implicit Aspects of the Psychosocial Rehabilitation

127

6. Ghazali, A.S., Ham, J., Barakova, E., Markopoulos, P.: The influence of social cues in persuasive social robots on psychological reactance and compliance. Comput. Hum. Behav. 87, 58–65 (2018) 7. Ghazali, A.S., Ham, J., Barakova, E., Markopoulos, P.: Assessing the effect of persuasive robots interactive social cues on users’ psychological reactance, liking, trusting beliefs and compliance. Adv. Robot. 33(7–8), 325–337 (2019) 8. Benson, L.A., McGinn, M.M., Christensen, A.: Common principles of couple therapy. Behav. Ther. 43(1), 25–35 (2012) 9. Hall, J., Taylor, K.: The emergence of Eric: co-therapy in the treatment of a family with a disabled child. Fam. Process 10(1), 85–96 (1971) 10. Rice, D.G., Fey, W.F., Kepecs, J.G.: Therapist experience and “style” as factors in co-therapy. Fam. Process 11(1), 1–2 (1972) 11. Barakova, E.I., Gillesen, J.C.C., Huskens, B.E.B.M., Lourens, T.: End-user programming architecture facilitates the uptake of robots in social therapies. Robot. Auton. Syst. 61(7), 704–713 (2013) 12. Ozaeta, L., Graña, M., Dimitrova, M., Krastev, A.: Child oriented storytelling with NAO robot in hospital environment: preliminary application results. Prob. Eng. Cybern. Robot. 69, 21–29 (2016). https://www.iict.bas.bg/pecr/2018/69/3-StorytellingNAOpaper.pdf 13. Ali, S., Mehmood, F., Ayaz, Y., Khan, M.J., Sadia, H., Nawaz, R.: Comparing the effectiveness of different reinforcement stimuli in a robotic therapy for children with ASD. IEEE Access 8, 13128–13137 (2020) 14. Crum, J., et al.: An approach to neuroimaging interpersonal interactions in mental health interventions. Biol. Psychiatry Cognit. Neurosci. Neuroimaging 7(7), 669–679 (2022) 15. Liberman, R.P., DeRisi, W.J., Mueser, K.T.: Social Skills Training for Psychiatric Patients. Pergamon Press, New York (1989) 16. Moore, R.: A Bayesian explanation of the ‘Uncanny Valley’ effect and related psychological phenomena. Sci. Rep. 2, 864 (2012) 17. Dimitrova, M. Wagatsuma, H.: Designing humanoid robots with novel roles and social abilities. Lovotics 3(112), 2 (2015). https://www.omicsonline.org/open-access/designing-hum anoid-robots-with-novel-roles-and-social-abilities-2090-9888-1000112.php?aid=67398 18. Ghit, a˘ , A.S, ., et al.: The AMIRO social robotics framework: deployment and evaluation on the pepper robot. Sensors 20(24), 7271 (2020) 19. Jokinen, K., Wilcock, G.: Multimodal open-domain conversations with the Nao robot. In: Mariani, J., Rosset, S., Garnier-Rizet, M., Devillers, L. (eds.) Natural Interaction with Robots, Knowbots and Smartphones, pp. 213–224. Springer, New York (2014). https://doi.org/10. 1007/978-1-4614-8280-2_19 20. Filippini, C., Perpetuini, D., Cardone, D., Merla, A.: Improving human–robot interaction by enhancing NAO robot awareness of human facial expression. Sensors 21(19), 6438 (2021) 21. Admoni, H., Bank, C., Tan, J., Toneva, M., Scassellati, B.: Robot gaze does not reflexively cue human attention. In: Proceedings 33rd Annual Conference of the Cognitive Science Society CogSci 2011, Austin TX USA, 2011, pp. 1983–1988 (2011) 22. Esfandbod, A., Nourbala, A., Rokhi, Z., Meghdari, A.F., Taheri, A., Alemi, M.: Design, manufacture, and acceptance evaluation of APO: a lip-syncing social robot developed for lip-reading training programs. Int. J. Soc. Robot., 1–15 (2022) 23. Moubayed, S.A., Skantze, G., Beskow, J.: The furhat back-projected humanoid head–lip reading, gaze and multi-party interaction. Int. J. Humanoid Robot. 10(01), 1350005 (2013) 24. Steindl, C., Jonas, E., Sittenthaler, S., Traut-Mattausch, E., Greenberg, J.: Understanding psychological reactance: new developments and findings. Zeitschrift für Psychologie 223(4), 205–214 (2015) 25. Mori, M.: The uncanny valley: the original essay by Masahiro Mori. IEEE Spectrum (1970)

128

M. Dimitrova et al.

26. Said, C.P., Sebe, N., Todorov, A.: Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces. Emotion 9(2), 260–264 (2009) 27. Burleigh, T.J., Schoenherr, J.R.: A reappraisal of the uncanny valley: categorical perception or frequency-based sensitization? Front. Psychol. 5, 1488 (2015)

Mobile Game Development Using Unity Engine Fatima Sapundzhi1

, Anton Kitanov1 , Meglena Lazarova2(B) , and Slavi Georgiev3,4

1 Department of Communication and Computer Engineering, Faculty of Engineering,

South-West University “Neofit Rilski”, 66 Ivan Mihailov Street, 2700 Blagoevgrad, Bulgaria [email protected] 2 Faculty of Applied Mathematics and Informatics, Technical University of Sofia, 8 Kliment Ohridski Blvd., 1000 Sofia, Bulgaria [email protected] 3 Institute of Mathematics and Infromatics, Bulgarian Academy of Sciences, 8 Acad. Georgi Bonchev Street, 1113 Sofia, Bulgaria [email protected] 4 Department of Applied Mathematics and Statistics, University of Ruse “Angel Kanchev”, 8 Studentska Street, 7004 Ruse, Bulgaria [email protected]

Abstract. Unity is a popular game development platform. Various industries are inspired by it and this can be a positive impact on the learning motivation, career growth and job opportunities. The aim of this paper is to develop and implement a 3D game application for education promoting through innovation and digital skills. The game consists of two parts. The first part of the developed application is a maze that the player must overcome. In this labyrinth the player has to collect a certain number of specific bags that contain pieces of the needed puzzle. The second part is a puzzle that the player begins to arrange. If the puzzle is arranged correctly, then the player can read an interesting fact about what is seen on the picture. The theme given on the picture is an educational theme and it main aim is to develop the educational and mental abilities of the playersp who may also be students. The game is suitable for a wide range of potential players as it can be interesting and educational for children, teenagers and even adults. Keywords: Game Development · Unity Game Engine · Image Processing and Visualization

1 Introduction With the rapid development of computer technology, the interest in many related industries is increasing [1]. Nowadays, the mobile phone is very popular as the trend is more and more people to play games on their mobile phones. A mobile game can be described as a simple smartphone game that includes graphics, interactions and controls [2–5]. There are different types of computer games: strategic, adventure, simulation, serious and others. In this paper, we present the methodology of developing a game that can be © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 129–138, 2023. https://doi.org/10.1007/978-3-031-42134-1_13

130

F. Sapundzhi et al.

used to support elementary students in e-learning or as modules supporting traditional learning. The developed application supports innovative game-based learning (GBL). Computer games have great potential in improving the educational process, present knowledge in an entertaining form, support the development of various cognitive skills, increase the internal motivation of learners by engaging in various game strategies [2, 6–8]. Popular game development platforms are Constructor (Scirra, Studio 414 The LightBulb) [9], GameMaker Studio (YoYo Games 2007) [10], Unreal Engine (Epic Games, Inc. 2004) [11], Unity Engine [12] and, etc. In the present article, we present the concepts, principles and methods of designing and developing a 3D game on the popular Unity Engine platform [13–15].

2 3D Game Development Using Unity Game Engine In this article, we present the methodology of developing a 3D game using Unity Engine version 2021.1.7f1 [11]. It is an integrated development environment (IDE) and a widespread technology for creating games for platforms such as Android, iOS, Mac OS, Windows, Linux, consoles, Augmented reality (AR), Virtual Reality (VR) and more. Unity provides various functions and features to create 2D (two-dimensional) or 3D (three-dimensional) games application, which can be web-based, desktop or console. The main languages that are integrated into the platform and can be used to program the games are C# and UnityScript. Some of the main features that Unity offers include a visual editor that makes it easy to create prototypes and game assets and also an ability to create rich, interactive 3D environments, physics and lighting simulations for realistic gameplay. The Unity user interface (UI) and Scene is shown on Fig. 1. The hand allows you to drag the view while the arrows allow you to move the objects in 3D space along three axes (X, Y, Z).

Fig. 1. Unity user interface and Scene View.

The Game View is rendered from the Camera(s) in the game and represents the final of the published game. The game consists of two parts:

Mobile Game Development Using Unity Engine

131

• The first part of the developed game is a maze that the player must overcome. In this labyrinth, the player has to collect a certain number of specific bags that contain pieces of a puzzle. The first level begins with overcoming a labyrinth. • The second part of the developed game is a puzzle. After overcoming the labyrinth, the player begins to arranging a small puzzle. After the puzzle is arranged, the player can read an interesting fact about what is seen on the picture. Usually the puzzle picture has an educationally theme. This turns into a circuit, where various mazes are overcome and then puzzles are arranged. The first step in developing the maze is to create the basic mechanics by using the character controller component (Fig. 2).

Fig. 2. Collider View and Directional Light object.

It allows to set the main characteristics of the object, controlled by the player. In our case, the object is a little man. The main characteristics of the object are presented in the following Table 1. Table 1. Main characteristics of the object. Properties

Functions

Step Offset

refers to climbing stairs, the larger the value, the higher the step can be climbed

Slope Limit

this is the final angle of the slope by which the object will be able to climb

Skin width

since two colliders can pass through each other, skin width intervenes here to prevent potentially jamming into another object. Low Skin Width can cause the character to get stuck (continued)

132

F. Sapundzhi et al. Table 1. (continued)

Properties

Functions

Min Move Distance the minimum distance that can be travelled at the start of the movement Center

the component creates a Capsule Collider must be set in a suitable way for the respective object, and it is possible to set values moving the collider from its center along the three axes of space

Radius and Height Determination of the length of the Capsule Collider’s radius and height

The next step is to write a script that uses the aforementioned component, and makes the object (in this case, the man) start to move under certain conditions and be controlled by the user. Then it follows the scene development (see Fig. 3). All objects for the specified scene are added here. The application/game is a set of scenes that are intended for different purposes such as the main menu, the individual levels/worlds, loading screens (loading screen) and others. Objects are added to the scene whose position, rotation and dimensions are controlled relatively to axes through the various tools or through the inspector. The “Player” tag has been added to the user-controlled object for easier recognition when creating various scripts that must interact with the player object. All objects on the scene are registered as Player Game Objects. The “Directional Light” object is used to illuminate the scene and relative to the specified direction the light illuminates at the specified angle. The “Main Camera” object is the main camera that visualizes the scene on the device screen. For the specific case it should track the player object (the man). This can be achieved through a hierarchical system, where the camera object is placed as a child of the man object. But in this case, it is not suitable, because when you rotate the man, the camera will rotate as well. For this reason, a script is added in order to track the position of the man and thus transmit the position to the camera. UI buttons and everything related to the user interface is added as a child object to the canvas. The canvas sets a persistent location on the user interface. The inspector offers configuration and sizing options for a specific screen relative to a given resolution. The selected resolution for the developed application is 800 × 600, but the program presents options to change it. In the puzzle level, the player must manipulate the objects presented on the screen. Each individual piece of the puzzle represents a separate game object that contains Box Collider and Rigid body components (Fig. 3). The main menu is the first scene added to the project and the first scene that loads when the game starts (Fig. 4). The menu consists of several buttons – Levels and Facts. What is more, here it is the exit button from the application (Exit). On initial launch of the game, the only accessible level is the first maze, and there is no access to facts (Table 2).

Mobile Game Development Using Unity Engine

133

Fig. 3. A view of the puzzle in development.

Fig. 4. Main menu view.

Figure 5 shows an example of a successfully arranged puzzle picture and information with interesting facts that game users can learn about it. Animations. Animations can be created both with external software and with Unity. After animations are added to the project, the Animator Controller component is required. It is added to the object that will use the animations. To get a transition from one animation to another, an Exit time or a certain condition is used, and the condition can be an activated trigger, a boolean, integer or float variable (see Fig. 6). On start-up, the Idle animation starts playing. This is the default animation for the man, it plays when he is at rest. It can run the jumping animation, the walking animation, or a repeating part of the jumping animation (“JumpLoop”), which is used when the man is in the air. As the corresponding transition requires the specified condition to be true. “Moves” and “is Grounded” are boolean variables and “Jumps” is a trigger. In order to perform functions for a certain frame of animation, from the animation window, the necessary animation is loaded and a so-called animation event can be placed on the corresponding frame, several such events can be added for one frame from the animation, each of which can perform a script function. For example, a sound is triggered when the jumping animation is performed.

134

F. Sapundzhi et al. Table 2. Pseudo code for the Main menu functions 1. Initialize GameObject array variable Levels 2. Initialize GameObject array variable FactButtons 3. Initialize GameObject array variable Facts 4. function Start(): 5. Set Time.timeScale to 1 6. end function 7. function LevelsClicked(): 8. for each GameObject obj in FactButtons: 9. Deactivate obj 10. for i=0 to value of the player preference variable L_Completed: 11. Activate Levels[i] 12. end for 13. end function 14. function FactsClicked(): 15. for each GameObject obj in Levels: 16. Deactivate obj 17. if the value of the player preference variable L_Completed > 1: 18. for i=0 to value of the player preference variable PuzzlesCompleted: 19. Activate FactButtons[i] 20. end for 21. end if 22. end function 23. function PlayLevel(LevelName): 24. Load the scene specified by LevelName in single mode 25. end function 26. function EiffelButtonClicked(OpenObj): 27. Activate OpenObj 28. end function 29. function Close(CloseObj): 30. Deactivate CloseObj 31. end function 32. function Exit(): 33. Quit the application 34. end function

Fig. 5. Arranged puzzle picture and information for it.

3 3D Model Creation Process and Add Sounds to the Game The 3D assets of the game were created and animated (the character) with Blender Version 2.8 [16]. It is a free and open-source 3D creation software that allows users to create a wide range of 3D models, animations, visual effects, and more. It is widely used

Mobile Game Development Using Unity Engine

135

Fig. 6. The block diagram of the animations and setting the animation event for the first frame.

in industries such as film and video game production, architecture, and product design. The method used to create the 3D assets with Blender is called hard-surface modeling. This technique involves using various tools and techniques in Blender to create precise and clean shapes, which typically involve a lot of straight edges and flat surfaces. 3.1 3D Model Creation Each model or mesh contains polygons (vertices in blender), edges, and faces. • Vertices: these are the points in 3D space that define the corners of a shape. They are represented by small dots in Blender and are connected by edges. • Edges: these are the lines that connect vertices and define the shape of an object. They are represented by straight lines in Blender. • Faces: these are the flat surfaces that make up an object. They are defined by the edges that connect their vertices and are represented by flat polygons in Blender. A face consists of 3 or more vertices connected together. As a rule, it is recommended that only quads (faces consisting of 4 vertices) are used to create models and in some cases tries (triangles, faces consisting of 3 vertices). Hard surface modeling is typically associated with creating high-fidelity objects with well-defined edges and surfaces, it can also be used to create low-poly characters which are used in our case. The main characteristics of the low-poly model are: • It is a 3D model that uses a relatively small number of polygons to define its shape and structure and “poly” refers to polygons, which are the flat, 2D shapes used to construct 3D models. • It is often used in real-time applications like video games, where performance is a concern. By using fewer polygons, the model can be rendered more quickly and efficiently, which can lead to smoother gameplay and faster load times. • These models can also be easier to create and work with than high-poly models, since they require less processing power and memory to manipulate. In the context of low-poly character modeling, hard surface modeling techniques can be used to create the basic structure and shape of the character’s body and clothing, while keeping the polygon count relatively low. This can make the model easier to animate and render, while still achieving a desired level of detail.

136

F. Sapundzhi et al.

Once the model of the character has been created, it needs to be rigged to a “skeleton” in order to animate it. A skeleton can be created by combining bones. A bone is essentially a virtual joint that can be positioned and rotated to create different poses and movements. The bones are connected together in a hierarchical structure, called an armature, which defines the overall shape and movement of the character. Rigging is the process of creating a system of bones and controls that can be used to animate a 3D model. When a 3D model is rigged, the bones are positioned and oriented to match the model’s anatomy, and controls are added to make it easier to pose and animate the character. The result is a rig, which is essentially a collection of bones and controls that allow the character to move and animate realistically. In Blender, rigging involves creating an armature and attaching it to a 3D model using a process called binding. Once the armature is bound to the model, the bones can be posed and animated using a variety of techniques, including keyframing, inverse kinematics, and constraints. On the Fig. 7 it can be seen the generated character with 5 different materials assigned: blonde for the hair; pink for the skin; green for the shirt; brown for the pants; black for the shoes. On the right side is the panel showing the keyframes that have been added for each bone as well as keyframes for other objects as well such as the camera and the light object.

Fig. 7. 3D player in Blender.

The final step is adding materials to our character. A material can include a variety of properties, such as colour, texture, reflectivity, and transparency, that determine how the object will look when it’s rendered. A block diagram describing how the movement of the man is performed in the game is presented on the Fig. 8. 3.2 Add Sounds to the Game The sounds in the game are assets from Unity’s asset store. The music has been created with Magix Music Maker – a digital audio workstation (DAW) software developed by Magix [17]. It allows users to create, edit, and produce their own music using a variety of virtual instruments, loops, and samples. The software includes various effects and mixing tools to enhance the final mix, such as equalizers, compressors, and reverb. There are many different ways of creating music with Music Maker. Such as: 1) Virtual instruments (synthesizers, drum machines, pianos and keyboards, basses and guitars), 2) Loops and samples, 3) Recording, 4) Effects, 5) Mixing. The sound pools are used to create the music of the game. Student learning is affected by game content, learning

Mobile Game Development Using Unity Engine

137

Fig. 8. A block diagram describing how the movement of the man in the game is carried out.

strategies, and motivation to participate. Using an educational game on mobile devices is more popular than using the game on computers and tablets [18, 19]. With the developed functionalities of the game, it can reach a wide range of potential players as it can be interesting and educational for the children, teenagers and adults. The name of the developed game is Puzzle Runner and it is available at Google Play Store. The user can download the mobile app versions for the Android and Microsoft Windows operating systems (OS). System requirements for the Android OS version are 7.1 or above. For PC it can be downloaded from the dev website (https://projec ttrmgs.blogspot.com/). System requirements for PC are Windows OS 7 or newer. The developed game is completely free. In further research, testing of the game prototype and its technological and pedagogical usability in a real learning environment will be carried out. The present research will benefit non-IT professionals who will be able to build educational video games. The game can be developed with new levels of complexity, requiring the player to assess situations, make decisions, critical thinking, teamwork, feedback, correcting answers, opportunity for new experiences, etc.

4 Conclusion The present study aims to describe the methodology for designing and developing a 3D game application using the Unity Engine. The developed game could be successfully used to promote education through innovation. In conclusion, with the development of educational technology, educational games are becoming more and more popular and are used to improve students’ motivation to learn and increase the learning effectiveness of

138

F. Sapundzhi et al.

students. The advantages of the game are that it offers a responsive design and the ability to play on screens with different resolutions and on different operating systems. It can be played on different devices – smartphones, tablets, personal computers. It is of great importance that the game is generally available to all without paying for its use. The user interface is intuitive and simple. The application allows easy extensibility and upgrade to newer versions. The chosen development platform Unity is widely popular, which ensures future support of the game, easy integration with other popular technologies and components. Acknowledgement. This publication is developed with the support of Project BG05M2OP0011.001-0004 UNITe, funded by the Operational Programme “Science and Education for Smart Growth”, co-funded by the European Union trough the European Structural and Investment Funds and Bulgarian National Science Fund under Project KP-06-M62/1 “Numerical deterministic, stochastic, machine and deep learning methods with applications in computational, quantitative, algorithmic finance, biomathematics, ecology and algebra” from 2022.

References 1. Jabangwe, R., Edison, H., Nguyen Duc, A.: Software engineering process models for mobile app development: a systematic literature review. J. Syst. Softw. 145, 98–111 (2018) 2. Papastergiou, M.: Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation. Comput. Educ. 52(1), 1–12 (2009) 3. Adams, E.: Fundamentals of Game Design, 3rd edn. Pearson Education, Inc., London (2014) 4. Salen, K., Zimmerman, E: Rules of Play - Game Design Fundamentals. MIT Press, Cambridge (2004). ISBN 0-262-24045-9 5. Fullerton, T., Swain, C., Hoffman, S.: Game Design Workshop: A Playcentric Approach to Creating Innovative Games, 2nd edn. Elsevier, Amsterdam (2008). ISBN 978-0-240-80974-8 6. Adams, E., Dormans, J.: Game Mechanics: Advanced Game Design. New Riders Games (2012). ISBN-13: 978-0-321-82027-3 7. Manning, J., Buttfield-Addison, P.: Mobile Game Development with Unity: Build Once, Deploy Anywhere, 1st edn. O’Reilly Media, Sebastopol (2017) 8. Nedyalkov, I., Stefanov, A., Georgiev, G.: E-learning on wireless telecommunication networks. In: IX National Conference with International Participation (ELECTRONICA), Sofia, Bulgaria, pp. 1–4 (2018) 9. Constructor. https://www.scirra.com/ 10. Game Maker Studio. https://www.yoyogames.com/gamemaker 11. Unreal Engine. https://www.unrealengine.com/en-US/what-is-unreal-engine-4 12. Unity. https://unity3d.com/ 13. Smith, M., Queiroz, C.: Unity 4.x Cookbook, Packt Publishing, Birmingham B3 2PB, UK (2013). ISBN 978-1-84969-042-3 14. Thorn, A.: Pro Unity Game Development with C#, 1st edn. Apress (2014). ISBN-13: 9781430267461 15. Ferrone, H.: Learning C# by Developing Games with Unity: Get to Grips with Coding in C# and Build Simple 3D Games in Unity 2022 From the Ground Up, 7th edn. (2022) 16. Blender. https://www.blender.org/ 17. MAGIX- Multimedia Software. https://www.magix.com/ 18. Sapundzhi, F., Mladenov, M.: An Android-based mobile application giving information for weather in real-time. Bul. Chem. Commun. 54, 89–91 (2022) 19. Mounir, S.E.A.: Software engineering for mobile applications, a survey on challenges and solutions. arXiv preprint arXiv:2301.00602 (2023)

Application of Artificial Neural Networks in Intelligent Tutoring: A Contemporary Glance Tatyana Ivanova1 , Valentina Terzieva2 , and Malinka Ivanova1(B) 1 Technical University of Sofia, blvd. Kl. Ohridski 8, Sofia, Bulgaria

{t.ivanova,m_ivanova}@tu-sofia.bg

2 Institute of Information and Communication Technologies, Bulgarian Academy of Sciences,

Bl. 2, Acad. G. Bonchev Street, 1113 Sofia, Bulgaria [email protected]

Abstract. Contemporary intelligent educational systems (IESs) and intelligent tutoring systems (ITSs) collect and process a large amount of data to deliver realtime learning, personalized according to the student’s needs. Such an approach includes a wide variety of machine learning algorithms, including artificial neural networks (ANNs), to model, analyze, and predict different issues in teaching and learning. This paper aims to summarize and discuss the current state regarding the utilization of ANNs in ITSs, comprising bibliometric analysis and a detailed review of scientific publications. A framework generalizing the main findings is created. Also, a classification model through a deep learning algorithm for predicting the personalized learning path is shown, which is characterized by high accuracy after the evaluation. Keywords: Artificial Neural Networks · Intelligent Education Systems · Intelligent Tutoring Systems · Personalized Learning

1 Introduction The continuous development of information technologies (IT) has led to digitizing the teaching-learning process. With the advancement of smart technologies in society, and in education in particular, new demands are placed for transformation and innovation to help overcome the shortcomings of the traditional learning model. The integration of artificial intelligence (AI), big data, artificial neural networks (ANNs), and other smart technologies in education is proving to be an efficient approach to transforming traditional eLearning systems into intelligent ones [1]. Intelligent education systems (IESs) combine the advantages of information technologies, technology-enhanced pedagogical methods, and digital learning content and scenarios to offer an alternative to the traditional teaching environment [2]. Thus, IES enables the teaching process to be dynamic and adaptive to the learner’s profile [3, 4] rather than static, while at the same time, real-time communication between students and teachers can occur. In this way, IES not only promotes personalized and more appropriate learning according to the needs of different students and the requirements of teachers but also improves teaching efficiency and facilitates knowledge acquisition and learning task performance [5]. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 139–150, 2023. https://doi.org/10.1007/978-3-031-42134-1_14

140

T. Ivanova et al.

Deep learning is one of the essential technologies that can help make education more personalized, adaptive, and efficient, leading to better learning outcomes for students. Technological advancement has increased the use of ANNs in intelligent education. The most prominent examples of these innovative applications are as follows [6]: (1) Recommendation systems: ANN can assist recommendation systems for educational content by analyzing students’ preferences, learning patterns, and performance data to suggest relevant and personalized learning resources; (2) Intelligent Tutoring Systems (ITS)s: ANN can support ITS to interact with students using natural language. These systems can identify students’ learning needs and provide personalized feedback, guidance, and help; (3) Student performance prediction: ANN can be trained to predict students’ performance based on various factors such as their previous grades, study habits, attendance, and learning activities. These predictions can identify students at risk of failing or dropping out and provide them with targeted support and resources; (4) Learning analytics: ANN can analyze large volumes of data generated by educational systems, including learning management systems, and assessment tools, among others. This data can help identify trends and patterns and develop insights to inform instructional design and pedagogical practices; (5) Adaptive learning: ANN can be used in adaptive learning systems that adjust the difficulty level of content based on the student’s performance. The network can analyze the student’s solutions to tasks and adjust the difficulty level of subsequent learning resources accordingly; (6) Automated grading: ANN can automatically grade assignments, quizzes, and exams. It can be trained on a large dataset of graded assignments to identify patterns and provide accurate and consistent grading; (7) Natural language processing: ANN can be used to develop natural language processing tools, such as chatbots or virtual assistants, which can assist students in learning by providing instant feedback answering questions, and helping them understand complex concepts; (8) Speech recognition: ANN can be used for speech recognition, allowing students to interact with educational software and devices using natural language. For example, an ANN can support recognizing spoken commands, answering questions, and providing feedback. Considering all the mentioned possible applications of ANNs, this paper investigates the utilization of ANNs-based techniques to facilitate intelligent tutoring and personalized learning. First, it presents a bibliometric and content analysis to explore the published scientific research about the implementation of ANNs to inform the importance of the considered issue. Then, it discusses the proposed predictive model in support of personalized learning path construction. At the end of the paper are conclusions.

2 Bibliometric View A combination of a bibliometric method and content analysis is applied in this work to investigate the published scientific production in the area of ANNs implementation in intelligent tutoring systems. The bibliometric approach allows to be outlined a comprehensive global picture and to be revealed future trends. Further, it leads to a better understanding of the most productive authors (with the origin country), the most preferred topics, and keywords. The utilized bibliometric data are derived from the abstracts and citation database Scopus considering the following two queries: (1) intelligent tutoring

Application of Artificial Neural Networks in Intelligent Tutoring

141

system and artificial neural network and (2) intelligent tutoring system and deep learning. The explored period is above a decade – from 2012 to February 2023. Then, the gathered bibliographic data are processed and analyzed by R software and using the Biblioshiny application [7]. The global picture regarding the key phrases intelligent tutoring system and artificial neural network is drawn through the found 33 documents. The countries of the most contributed authors are Brazil, Mexico, Hong Kong, USA, UK, India, and Greece. Some of the curves show a faster-increasing tendency in the number of published articles like these of Brazil, Hong Kong, and Greece. Others demonstrate constancy over a longer period, such as the scientific production of Mexico, USA, India and UK. The text mining of the papers’ abstracts gives the possibility for the construction of a co-occurrence network through bigrams to show the connection between paired terms (Fig. 1). Seven clusters are formed as they are presented in different colors. The biggest cluster organizes bigrams with the most important values like intelligent tutoring, tutoring systems, neural network, artificial intelligence, learning style, learning process, learning outcomes, expert systems, fuzzy logic, and social network. The second cluster includes bigrams related to neural networks, machine learning, data mining, mining techniques, and learning environment. The rest of the clusters are smaller and regard adaptive learning, knowledge tracing, and student knowledge in the third cluster; educational data and learning data in the fourth cluster; expert rules and proximal learning in the fifth cluster; students profile and students learning in the sixth cluster and intelligent systems in the seventh cluster. It seems that the crossing area between ITS and ANN involves the investigation of different problems concerning data mining, knowledge tracing, students’ learning profile, and intelligent systems.

Fig. 1. Co-occurrence network, constructed on the query intelligent tutoring system and artificial neural network

The dynamic of the ten most utilized authors’ keywords over the years is also explored. In a paper writing, the keywords are precisely selected by authors to reveal

142

T. Ivanova et al.

the key issues and the context of the discussed themes. Usage of exactly those keywords explains the researchers’ growing and evolving interests in contemporary real-world topics. Along with the keywords ITS and ANN, very often are also pointed out keywords such as affective computing, evaluation methodologies, fuzzy logic, mobile learning. Extracted trending topics after analysis of the papers’ abstracts in bigrams are the terms tutoring system and neural networks, students’ learning, learning progress, adaptive learning, knowledge tracing, learning styles, and machine learning. The aim of the second query is to explore the scientific production related to intelligent tutoring system and deep learning, the terms included in the abstract, title, and keywords of the papers. As a result, 117 documents are found in the period from 2012 to the 2023 year. The most productive authors are from the USA, China, India, Canada, Australia, the UK, and Hong Kong. The curve presenting the scientific production of the authors from the USA is steeper in comparison to the other curves. In the constructed co-occurrence network based on abstract bigrams are identified five different clusters as the bigger one includes terms like intelligent tutoring, deep learning, machine learning, learning algorithms, personalized feedback, learning models, natural language, data mining, and online courses (Fig. 2). The second one is formed around the terms: knowledge tracing, learning activities, students’ performance, students’ knowledge, students’ learning. The third cluster consists of the terms: artificial intelligence, learning analytics, educational data, question generation, learning environment. The fourth cluster is smaller, comprising the terms adaptive learning, reinforcement learning, and student learning. The last one includes the terms human-centric tasks and expert policy.

Fig. 2. Co-occurrence network, constructed on the query intelligent tutoring system and deep learning

Application of Artificial Neural Networks in Intelligent Tutoring

143

The most used terms in authors’ keywords are as follows: deep learning, ITS, knowledge tracing, machine learning, educational data mining, natural language processing, student’s modeling, and learning analytics. The bibliometric analysis shows the interest of the research society in utilization of ANNs as apparatus for facilitating and delivering more personalized and adaptive learning by supporting students’ profile modeling, tracing their progress, and observing assessment achievements.

3 The State of the Art Artificial neural networks could be used as tools for implicit knowledge modeling. An ANN is a collection of nodes connected by weighted links. Machine learning approaches are typically used to adjust the weights of the relations between nodes. At supervised or unsupervised machine learning the network is adapted to handle a task better. Learning involves adjusting the weights of the network to improve the accuracy of the output results. In this way, by machine learning algorithms and tutoring examples, ANNs – based knowledge models can automatically acquire knowledge from real domains and represent it in machine-processable formats. Artificial neural networks usually work well combined with other statistical and artificial intelligence techniques in adaptive and personalized eLearning. In the Linear Programming Intelligent Tutoring System [8], an approach that employs ANNs and expert systems to acquire knowledge for the learner model is used. Its main goal is determining the learner’s academic performance level to make a personalized offering of proper difficulty level in tutoring the linear programming course. The feed-forward back propagation algorithm trains ANNs using groups of learners’ data to predict their academic performance. An expert system is used to decide the proper difficulty level suitable for the expected academic performance of the learner. Intelligent Tutoring Systems capabilities propose one of the most successful architectures for integrating intelligent technologies, including supporting personalized and adaptive tutoring and learning. Integration of ANNs with AI methodologies has put together promising results in tutoring different groups of learners, including those having learning disabilities. An example is the ITS framework [9], where neural networks are used to identify the learning disabilities of children. The system [9] uses neural network classifiers to identify the students’ learning difficulties and fuzzy min-max neural network (FMNN) classification to determine learner profiles and propose or recommend the most suitable learner-centered content. Fuzzy sets also are used in supervised learning neural network classification for learner profiling. The experiments show encouraging accuracy of classification results based on the integration of fuzzy with the neural network in the ITS. Successful integration of the fuzzy logic method approach and the artificial neural network is proposed in [10], where it is used to support the calculation of learning outcomes. The combined usage of ANNs and fuzzy logic makes it possible to design an adaptive assessment process and adjust dynamic assessment rules to use by the lecturers/instructors for a precise, personalized assessment. The student’ s satisfaction is essential for successful learning. A model based on artificial neural networks that aim to identify factors that influence the student’ s satisfaction

144

T. Ivanova et al.

is presented in [10]. The model is designed to predict student satisfaction with eLearning, and researchers report a 92.2% correct classification rate based on experiments. Other significant conclusions are that different assessments are a powerful determinant of learning satisfaction. Monitoring the students’ emotions is very close to predicting satisfaction. A neural network model for recognizing students’ emotions is proposed in [11]. It is based on video images of their faces and uses the extracted information to classify the students’ individual and group emotions. It begins with performing the detection of the faces and drawing their features. Then the classification algorithm groups the faces with similar emotional looks. It uses machine learning and ANN to extract each selected person’s emotional features. Next, it performs aggregation by applying statistical functions and ANN-based classification. Predicting students’ learning performance is another critical issue for successful personalization mechanisms. To ensure dynamic personalization in virtual environments, in [12] is proposed a learning feature quantification method to convert the raw data from eLearning systems into sets of independent learning features. Then researchers use a convolutional neural network for learning performance prediction. Exploring the role of artificial intelligence in improving learners’ performance in eLearning systems is a prominent motivating research area that combines eLearning and traditional tutoring opportunities [13]. Personalized learning is one of the opportunities that help to increase the effectiveness of individual learning. An approach based on an artificial neural network to provide learners with the most suitable learning materials is proposed in [13]. It is based on extracting information about the learner’s knowledge background from the Web and using it as an input of the neural network. Classification possibilities of ANNs also were successfully used for automated determining the student’s learning style during tutoring [14] and then applying learning style– based personalization or recommendation of suitable content. The ANNs-based classification mechanism maps the learning content to learners having specific learning styles using self-organizing maps. Neural networks can also improve the learning of groups of learners with educational or psychological problems, such as dyslexia, dyscalculia, and autism. Learning platform for autism spectrum disorder children [15] uses reinforcement learning and specific types of ANN, such as Deep Convolution Neural Networks and Regional Convolution Neural Networks, to provide personalized education and assistance. Convolutional Neural Networks usually are used for recognizing students’ engagement in the environment of distance learning [16]. Experiment results show that the methods used have good performance and are the most suitable for real-world applications and deployments on mobile platforms for business education. Another type of ANN that proved its usefulness in eLearning is recurrent neural networks (RNNs). They are beneficial for processing sequential data and can selectively pass information across sequence steps. RNNs are the most useful for modeling input and/or output consisting of sequences of elements that are not independent. Recurrent neural networks, in conjunction with Hidden Markov models, can help to model the student learning sequences and to predict the future results of learning. It is the most

Application of Artificial Neural Networks in Intelligent Tutoring

145

important for the overall organization of the personalized educational process, including the generation and selection of learning paths. Learners’ concentration during learning is essential for successful learning. Evaluation of eLearners’ concentration using recurrent neural networks is performed in [17]. Deep learning methods and ANN were also used to automate assessment [18]. Assessment results give a possibility to shape a personalized learning path through the eLearning system. The relatively small amount of data in the form of available assessments was sufficient for training the needed neural network. The drawback is that specificity of each subject/course requires the preparation and training of a separate ANN. In an intelligent learning environment, the dynamic acquisition of a significant amount of information about learners is essential for the flexible organization of adaptive learning and tutoring. Determining the learning style or possible learning disabilities, for example, will be better done automatically in the first lessons of the course. Continuous monitoring of the learner’s performance is crucial for organizing the adaptive learning process. Prediction of learners’ success or problems, based on the acquiring information about each learner, is also vital for personalized learning path generation, selection, or resource recommendation. The obtained findings during the conducted bibliometric analysis and detailed papers’ review are sumarized in Fig. 3 as the utilization of ANNs in support of intelligent tutoring is categorized into three groups: ANNs-based applications for making predictions, for modeling and analysis, and in assistance of communication interface realization. The proposed framework shows the recently researched main topics and demonstrates some challenging problems that need further investigation.

Communication interface Natural language processing

Modeling and analysis Learner profile

Face recognition

Knowledge domain

Face expression

Learning content

Prediction Level of engagement

Learning performance

Emotional state

Learning difficulties

Assessment results Personalized learning path

Fig. 3. A framework regarding the usage of ANNs in support of ITS

146

T. Ivanova et al.

4 A Model for Predicting the Student’s Learning Path The performed extensive bibliometric analysis and literature review points out that the construction of personalized learning path is one of the challenging issues and an essential functional part of IESs and ITSs. Therefore, in this work, a model is proposed to predict every next learning object (LO) in a course, drawing the students’ learning paths. Five variables are considered: • Obtained points from an initial/previous task – The applied scale is from 70 to 100 points. The students who scored less than 70 points were considered to have failed their assignment; • Time for completing the current task – This variable can take two values: on time and late; • Achieved quality of the current task – The quality of the current task is evaluated through three values: good, very good, and excellent; • The student’s attitude toward the task performance – This variable is characterized by two values: positive and negative attitude; • Knowledge level of students – The students’ knowledge is evaluated considering three values: good, very good, and excellent. The delivered LOs in the learning path have four different complexities: LO1 has very high complexity, LO2 has high complexity, LO3 is characterized by average complexity, and LO4 has low complexity. The proposed solution presented in Fig. 4 gives possibility for the dynamic formation of the student’s learning path in a course.

LO1 LO1

…

Variables LO4

LO2

Variables

…

Variables LO3

Variables LO1

LO4

Variables

… LO4

Fig. 4. Predicting the student’s learning path

Figure 5 shows an illustrative example. When the student completes the initial task, it is predicted that the next task must be LO1, characterized by very high complexity. The prediction is made based on the five variables described above: the student received 95 points for the conductance of the task, submitted it on time, the quality of the performance was very good, the attitude towards the implementation of this LO is positive and the

Application of Artificial Neural Networks in Intelligent Tutoring

147

level of knowledge gained is excellent. After completing LO1, other worse results were obtained from the student: 90 points for the task completion, he/she was late with the assignment, the quality of the performance was very good, the attitude towards the implementation of this LO is positive and the level of knowledge gained is very good. Thus, the IES prognoses and delivers LO2, which is less complex than LO1.

Variables 95

On time Positive

Variables Very good

LO1

Excellent Predicted LO

Initial task

90

Late Positive

Very good

LO2

Very good Predicted LO

Performance of LO1

Fig. 5. An illustrative example

Let us imagine that the student has excelled grade so far and, for some reason, has performed worse on LO1, and then the system recommends him/her a LO2 of lower complexity. However, the student is not satisfied with his/her performance and the fact that the system predicted LO2 with lower complexity. In such a case, the system should allow the student to choose the predicted LO2 of lower complexity or the LO1 of higher complexity. This rule will contribute to maintaining the student’s motivation to learn, giving him the opportunity to control his/her learning process partially (Table 1). Table 1. An opportunity for the student partially to control his/her learning path

Learning path

Initial LO

Predicted LO1 Opportunity LO1 Predicted LO2 Opportunity LO2 Predicted LO3 Opportunity LO3 Predicted LO4

Result from taken learning path Highest Highest Higher Higher Average Average Low

To the training data in RapidMiner Studio, a deep learning algorithm is applied as the ANN consists of three hidden layers, respectively, with 50, 70, and 70 neurons. The ratio of training/testing data is 70% to 30%. The activation function is Rectifier. The model is evaluated following typical machine learning metrics like accuracy (94.74%), absolute error (0.0415), relative error (4.15%), and root mean squared error (0.1580). The results show that the model with high accuracy can predict the next LO considering

148

T. Ivanova et al.

how the students performed the current task and their previous marks. Figure 6 presents a chart plot comparing predicted LO and observation. It is seen that there is a problem with LO1 (with very high complexity), which in some cases, can be delivered to the student in the form of LO2 (with high complexity).

Fig. 6. Accuracy of the prediction

5 Conclusions The paper draws a contemporary glance at the application of ANNs in support of intelligent tutoring and intelligent educational systems. It is seen that machine and deep learning are used for purposes of modeling, analysis, and predicting the student’s profile, learning process, and teaching tasks. A framework that summarizes the findings gathered through a bibliometric approach and a detailed review of scientific publications is created to outline some challenging issues and essential research topics. The research reveals that one of the main problems for learning success in ITSs and IESs concerns providing students with the most suitable learning path in a course. A solution to this problem is proposed in the form of a predictive model showing the dynamically created learning path and considering five variables: points from an initial/previous task, time for task completing, quality of the current task, student’s attitude toward the task performance, and knowledge level. The predicted learning path could be implemented with an opportunity for the students to partially control it by selecting the level of LO complexity. Thus, the motivation to learn can be retained. The model is evaluated, and the results point out its high accuracy. Acknowledgments. This research is supported by the Bulgarian FNI fund through the project “Modeling and Research of Intelligent Educational Systems and Sensor Networks (ISOSeM)”, contract KP-06-H47/4 from 26.11.2020.

Application of Artificial Neural Networks in Intelligent Tutoring

149

References 1. Chen, L., Chen, P., Lin, Z.: Artificial intelligence in education: a review. IEEE Access 8, 75264–75278 (2020). https://doi.org/10.1109/ACCESS.2020.2988510 2. Hu, B.: Teaching quality evaluation research based on neural network for university physical education. In: 2017 International Conference on Smart Grid and Electrical Automation (ICSGEA), pp. 290–293. IEEE (2017). https://doi.org/10.1109/ICSGEA.2017.155 3. Bontchev, B., Vassileva, D.: Adaptive courseware design based on learner character. In: Proceedings of International Conference on Interactive Computer Aided Learning (ICL), pp. 1–8 (2009) 4. Bontchev, B., Antonova, A., Dankov, Y.: Educational video game design using personalized learning scenarios. In: Gervasi, O., et al. (eds.) ICCSA 2020. LNCS, vol. 12254, Part VI, pp. 829–845. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58817-5_59 5. Rico-Bautista, D., Medina-Cardenas, Y., Coronel-Rojas, L.A., Cuesta-Quintero, F., MaestreGongora, G., Guerrero, C.D.: Smart university: key factors for an artificial intelligence adoption model. In: García, M.V., Fernández-Peña, F., Gordón-Gallegos, C. (eds.) Advances and Applications in Computer Science, Electronics and Industrial Engineering. AISC, vol. 1307, pp. 153–166. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4565-2_10 6. Okewu, E., Adewole, P., Misra, S., Maskeliunas, R., Damasevicius, R.: Artificial neural networks for educational data mining in higher education: a systematic literature review. Appl. Artif. Intell. 35(13), 983–1021 (2021) 7. Aria, M., Cuccurullo, C.: bibliometrix: An R-tool for comprehensive science mapping analysis. J. Informetr. 11(4), 959–975 (2017) 8. Abu-Naser, S.S.: Predicting learners performance using artificial neural networks in linear programming intelligent tutoring system (2012) 9. Dutt, S., Ahuja, N.J., Kumar, M.: An intelligent tutoring system architecture based on fuzzy neural network (FNN) for special education of learning disabled learners. Educ. Inf. Technol. 27(2), 2613–2633 (2022) 10. Alnagar, D.K.F.: Using artificial neural network to predicted student satisfaction in e-learing. Am. J. Appl. Math. Stat. 8(3), 90–95 (2020) 11. Savchenko, A.V., Makarov, I.A.: Neural network model for video-based analysis of student’s emotions in eLearning. Opt. Mem. Neural Netw. 31(3), 237–244 (2022) 12. Wang, X., Wu, P., Liu, G., Huang, Q., Hu, X., Xu, H.: Learning performance prediction via convolutional GRU and explainable neural networks in eLearning environments. Computing 101, 587–604 (2019) 13. Azzi, I., Jeghal, A., Radouane, A., Yahyaouy, A., Tairi, H.: Approach based on artificial neural network to improve personalization in adaptive E-learning systems. In: Bhateja, V., Satapathy, S., Satori, H. (eds.) Embedded Systems and Artificial Intelligence. AISC, vol. 1076, pp. 463–474. Springer, Singapore (2020). https://doi.org/10.1007/978-981-15-09476_44 14. Cabada, R.Z., Estrada, M.L.B., García, C.A.R.: EDUCA: a web 2.0 authoring tool for developing adaptive and intelligent tutoring systems using a Kohonen network. Expert Syst. Appl. 38(8), 9522–9529 (2011) 15. Vijayan, S., Janmasree, C., Keerthana, L.B., Syla, A.: Framework for intelligent learning assistant platform based on cognitive computing for children with autism spectrum disorder. In: 2018 International CET Conference on Control, Communication, and Computing (IC4), pp. 361–365 (2018) 16. Naim, A.: ELearning engagement through convolution neural networks in business education. Eur. J. Innov. Nonform. Educ. 2(2), 497–501 (2022)

150

T. Ivanova et al.

17. Jeong, Y.S., Cho, N.W.: Evaluation of e-learners’ concentration using recurrent neural networks. J. Supercomput. 79(4), 4146–4163 (2023) 18. Cader, A.: The potential for the use of deep neural networks in e-learning student evaluation with new data augmentation method. In: Bittencourt, I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS, vol. 12164, Part II, pp. 37–42. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52240-7_7

Educators’ Support Through Predictive Analytics in an Assessment Process Malinka Ivanova1(B) , Petya Petkova2 , and Tsvetelina Petrova3 1 Department of Informatics, Faculty of Applied Mathematics and Informatics, Technical

University of Sofia, Blvd. Kl. Ohridski 8, Sofia 1797, Bulgaria [email protected] 2 Department of Electronics and Energy Engineering, Technical College - Sofia, Technical University of Sofia, Blvd. Kl. Ohridski 8, Sofia 1797, Bulgaria [email protected] 3 Department of Energy and Mechanical Engineering, Technical College - Sofia, Technical University of Sofia, Blvd. Kl. Ohridski 8, Sofia 1797, Bulgaria [email protected]

Abstract. The utilization of machine learning (ML) algorithms in the area of learning analytics is a contemporary approach for conducting classifications and predictions based on data collected in an educational process. In an assessment procedure also some prognoses could be done benefiting from ML. The aim of the paper is to present a ML-based methodology in support of educators when an assessment of digital objects has to be done. It is used for the students’ classification and prediction of their digital-models-driven learning progress and learning performance. The accuracy of the classifier (Artificial Neural Network) in different use cases is evaluated and is proven its high value. Keywords: machine learning · artificial neural networks · learning analytics · assessment · learning progress · learning performance · intelligent learning environment

1 Introduction Nowadays, different machine learning (ML) techniques are used for learning from data in order to grasp some patterns or to reveal anomalies. This is very useful for making predictions and analysis of facts, events or processes. Surveys like [1, 2] discuss the most popular machine learning algorithms and their applications in healthcare, security, manufacturing, Internet of things, autonomous vehicles, robotics and other industries. Recently, some scientific works showed increased interest in ML and its utilization for educational purposes. Most often, the research addresses problems related to prediction and improving students’ learning performance and their retention as well as issues concerning students’ grading and testing [3]. A contemporary learning environment proposes statistical summarization regarding the students’ activities and their achievements. This is possible due to features related to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 151–162, 2023. https://doi.org/10.1007/978-3-031-42134-1_15

152

M. Ivanova et al.

tracking students’ behavior, data collection and processing. The provided information is time-based when designates the duration of interaction with a certain learning object or needed time for the course completion, process-based when indicates the evolvement of students, or results-based when presents the students’ achievements both on a single task or a whole course. Such information is very beneficial for educators and their role of examiners who have to decide how to evaluate the students’ results and their final course mark formatting. Of course, the nature of the assessed task is important and this requires an individual approach to its evaluation. Each particular task is characterized by certain specifics and its conductance is evaluated according to explicitly defined criteria by the educator. A contemporary solution in an intelligent learning environment (ILE) would offer the possibility of automating this educators’ activity, reducing the implementation time and decreasing applied efforts. ML algorithms could advantage educators by equipping them with a toolkit for identifying the correct criteria for assessment and for predicting the students’ learning progress. The aim of the paper is to present a ML-driven methodology in support of the educators during the preparation of assessment criteria of digital models and according to them to predict the students’ progress and performance in a learning process.

2 Current State Continuous assessment is a very useful practice, allowing the provided to the students’ tasks to be assessed after their completion within a course. It gives a possibility for continuous monitoring of the students’ progress and when it is applicable some interventions from the educators’ side to be done. Furthermore, González et al. discuss the correlation between continuous assessment and final marks [4]. The application of ML techniques in e-Assessment is examined by different authors, seeing its potential to predict the students’ success or not. Morales et al. present models for predicting the students’ final score in two cases: when the variable grade is taken into account and when it is dismissed [5]. Some ML algorithms are applied like linear regression, logistic regression, artificial neural networks (ANNs), Bayesian networks as better learners are proved to be logistic regression and ANNs (multilayer perceptron). The authors conclude that the assessment tasks (1) should be tailored to the students’ profile, (2) to be suitable to the specifics of the final exam, and (3) the students’ learning performance in continuous assessment is connected to the final exam marks. Alsariera et al. investigate the importance of predictable students’ learning performance for supporting the educators’ work by surveying some ML models [6]. They found that the most used techniques are: linear regression, ANNs, decision tree, Naïve Bayes, support vector machine, K-nearest neighbor. Learning performance is considered in relation to the reached learning goals and mainly its prediction is based on predictors such as ages and final marks. Park et al. utilize ML together with explanatory item response theory (EIRT) to predict the students’ answers to items in an assessment as three scenarios of unrealized responses for existing and new items and responses, which are missed at existing items are examined [7]. Such an approach allows some personal information about how the students learn to be grasped and as main predictors to be considered outcomes and item responses. EIRT and ML techniques are evaluated as the conclusion

Educators’ Support Through Predictive Analytics

153

is that they are competitive with advantages for EIRT. Alruwais and Zakariah propose an approach for assessment the students’ knowledge and forecasting their learning performance through applying ML algorithms [8]. A set of characteristics important for classification of the students’ knowledge is identified and it includes learning behavior, knowledge level and exam marks. The students are categorized into four groups considering these characteristics. Seven classifiers are used and compared and with the highest accuracy is outlined to be Gradient Boosting Machine. The authors conclude that such an approach can facilitate teachers to find the weak point in the student‘s learning, improve it and increase the test’s success. Zhai et al. present a method for automatic evaluation of models, created by students, which is based on ML (convolutional neural networks) [9]. The description of models, written by the students, is also scored through analysis, which is based on natural language processing. The authors show the benefits of ML and its suitability for assessing drawn models and descriptions regarding these models. This review indicates for the increased interest of researchers in the assessment topic and how the students’ progress to be evaluated and predicted. Despite scientific achievements, there are still unexplored problems and challenging issues that need to be investigated. One of these problems is related to tasks assessment, which requires the students to create digital models with specific features. Another problem concerns the evaluation of the students’ learning performance based on the built digital models during a course. To reflect on these challenging points, a ML-based methodology is developed and presented in the next section.

3 Proposed Methodology The methodology comprises the following procedures and it is presented in Fig. 1: 1. Sets with criteria for tasks’ evaluation are prepared as the tasks are related to the creation of specific digital models. In the common case, the number of tasks in a course is n and the educator has to create n sets with criteria (Assessment scenario 1). The criteria reflect on specifics of digital models and steps/rules for their creation. When the tasks are similar and the digital objects to be developed by the students are similar, then the educator uses the same set of requirements to assess these similar tasks (Assessment scenario 2). Here, in scenario 1 and scenario 2, the educator is facilitated through the preparation of assessment templates with criteria. 2. Data for evaluation from the students’ digital models are gathered taking into account the prepared set/s with criteria. Datasets are labeled as predicting variables are students’ progress and learning performance. According to learning progress, the students are classified into three groups: least developed, developing and developed. According to their learning performance, three groups of students are formed: with high, average and low model-based learning performance. 3. ML algorithm is applied to predict the student’s group considering his/her learning progress and performance. ML model for solving classification problems requires the construction of an Artificial Neural Network (ANN). Evaluation of the predictive models is conducted.

154

M. Ivanova et al.

Data

Task 1 Evaluation template

gather-

Classifier

Predictive model

Model

evaluaStudents’

Data

Task 2 Evaluation

gather-

Classifier

Predictive model

classification • Learning progress •

template

Learning performance

Model

evalua-

… Task n Evaluation template

Data

gather-

Classifier

Predictive model

Model

evalua-

Assessment scenario 1 Students’ Task 1 ÷ n Evaluation template

Data

gather-

Classifier

Predictive model

classification • Learning progress

Model

evalua-

Assessment scenario 2

•

Learning performance

Fig. 1. Proposed methodology for predictive assessment

4 Experimentation and Results The proposed methodology is verified by three different use cases. In the first use case, four different tasks are assessed with one and the same set of criteria, because the created digital objects differed only in their complexity, but not in their nature. In the second use case, three tasks are assessed as two tasks are similar and the third one is different and requires a separate set of criteria. The third use case concerns the assessment of a project. 4.1 Use Case 1 Students for a specialty “Industrial Thermal Power Plants” study Computer-Aided Design (CAD) using the software CATIA STUDENT V5-6R2018. Students work individually during laboratory classes. The purpose is to put into practice knowledge from the lectures by applying continuous assessment approach. During the semester, the students

Educators’ Support Through Predictive Analytics

155

receive consecutively four different tasks that they have to perform under the supervision of the educator who consults/helps the students if there is a need. The first task is to draw a 2D model (sketch). The next three tasks are about building 3D models that become more complicated from task 2 to task 4. The educator assesses the progress and performance of every student using the following indicators: 1. Does the student follow the main rules while doing the task? - possible answers yes/no; 2. Does the student use complicated commands in the task? - possible answers yes/no; 3. Are there any missing elements in the digital model? - possible answers yes/no; 4. Are there any redundant elements in the model? - possible answers yes/no; 5. Can the student point out which are missing/redundant elements? - possible answers yes/no; 6. Does the student know how to add/remove missing/redundant elements? - possible answers yes/no; 7. Does the students know how to examine task performance step by step? - possible answers yes/no; 8. What is the score/assessment of the student about the particular task? - possible answers 3;4;5;6; 9. According to the teachers, in which group is the student? – possible answers least developed/ developing / developed; 10. According to the teacher, what is the learning performance of the student? – possible answers low learning performance /average learning performance /high learning performance. Datasets are prepared as included data responds to the indicators. In RapidMiner Studio a Deep Learning Algorithm (DL) is used to learn datasets and to do classification of the students into groups. The constructed ANN possesses two hidden layers with 50 neurons each. 70% of data are used for training and 30% for testing. The obtained models are evaluated regarding the received errors: Absolute error (AE), Relative error (RE), Root mean squared error (RMSE), Root relative squared error (RRSE) and Squared error (SE). Table 1 includes information about the accuracy of the classification model, which points out the group of students regarding their learning progress. Table 2 reports the accuracy of the model, which classifies the students into groups according to their learning performance. The next model prognoses the overall student’s learning performance and learning progress in a course considering his/her achievement in digital models development. It is created considering the discussed above datasets collected from the completion of the four tasks. The model is evaluated and it is characterized with very small errors: (1) at predicting the learning performance of the students: AE: 0.0202; RE: 2.02%, RMSE: 0.0823; RRSE: 0.3867; SE: 0.0068 and (2) at prognosis the students’ learning progress: AE: 0.0350, RE:3.50%, RMSE: 0.1472, RRSE: 0.7685, SE: 0.0217. Figure 2 presents a comparison of the models’ accuracy taking into consideration the students learning progress. The accuracy of models regarding the learning performance of the students is compared and presented in Fig. 3.

156

M. Ivanova et al. Table 1. Evaluation of the predictive model regarding the group of students’ progress

Parameter/Task

Task 1

Task 2

Task 3

Task 4

AE

0.0166

0.0060

0.1293

0.0063

RE

1.66%

0.60%

12.93%

0.63%

RMSE

0.0339

0.0096

0.1785

0.0113

RRSE

0.0815

0.0354

0.7787

0.0530

SE

0.0012

0.0001

0.0318

0.0001

Table 2. Evaluation of the predictive model regarding learning performance Parameter/Task

Task 1

AE

0.0125

Task 2 0.1318

Task 3 0.0057

Task 4 0.1635

RE

1.25%

13.18%

0.57%

16.35%

RMSE

0.0226

0.1669

0.0114

0.2346

RRSE

0.0543

0.6163

0.0499

1.1026

SE

0.0005

0.0279

0.0001

0.0550

Fig. 2. Comparison of models’ accuracy considering the students’ learning progress

4.2 Use Case 2 In the second use case the students with specialty “Informatics and Computer Sciences” during practical classes in the course “IT and privacy preservation” had to accomplish several assessment tasks as three of them are evaluated here. Two of the tasks are similar, but the third is different and this requires two different sets with evaluation criteria.

Educators’ Support Through Predictive Analytics

157

Fig. 3. Accuracy comparison of models regarding the students’ learning performance

The first and second tasks are oriented to the development of four digital models (for the first task)/three models (for the second task) for privacy preservation through the application of concrete algorithms with changing values of certain parameters. The criteria for these tasks‘ completion are the following: • Whether the four/three models are presented in the final report? – possible answers are yes/no; • Whether the usefulness of the four/three models is evaluated? - possible answers are yes/no; • Whether the risk assessment of the four/three models is done? - possible answers are yes/no; • Whether conclusions are written? - possible answers yes/no; • Whether the models are created in time? - possible answers are yes/no; • What is the score for completion of this task according to the educator? – possible answers are 3,4,5,6; • What is the group of a given student regarding his/her learning progress? – possible answers are: least developed, developing and developed; • What is the group of the student considering his/her learning performance? – possible answers are: in groups with high, average and low learning performance. In the third task, the students had to prepare a digital model and its improved version regarding privacy preservation through experimenting with another algorithm and the task is assessed taking into account the criteria: • Whether the dataset for the first model is added in the final report? - possible answers yes/no; • Whether the process workflow is presented? - possible answers yes/no; • Whether the model is created? - possible answers yes/no; • Whether the modified dataset is included? - possible answers yes/no; • Whether the modified model is created? - possible answers yes/no; • Whether conclusions are written? - possible answers yes/no; • Whether the task is accomplished in time? - possible answers yes/no;

158

M. Ivanova et al.

• What is the score for this task according to the educator’s opinion? – possible answers are 3,4,5,6; • What is the group of given student considering his/her learning progress? – possible answers are: least developed, developing and developed; • What is the group of the student according to his/her learning performance? – possible answers are: in group with high, average and low learning performance. In a similar way to use case one, datasets are prepared and used for learning from a Deep Learning Algorithm. The obtained accuracy of predictive models is very high, because of received small errors. Comparison of models’ accuracy at predicting the student’s group regarding his/her progress and learning performance is presented on Figs. 4 and 5.

Fig. 4. Models’ accuracy at predicting the group of students regarding their learning progress

4.3 Use Case 3 The third use case concerns the student’s evaluation process in first-year of bachelor‘s degree, specialty “E-Management”. The final grades evaluation is based on 1) presence and taking part in the education and training process and 2) four indicators calculated after completion of a final project and its defense. The evaluation process algorithm is as follows (Fig. 6): Every student is evaluated according to a scale for his/her presence and taking part in the lecture and seminars during the semester. That scale gives 25% of the final grade. The rest 75% are calculated from four basic quantitative indicators and two qualitative ones– student’s learning progress and learning performance, explain as follows: • Are the stylish requirements kept? – possible answers yes = 1/ no = 0; Indicator 1; • How much the project is relevant to the given topic –possible answers: (6.25 = 25%; 12.5 = 50%; 18:75 = 75%; 25 = 100%); Indicator 2;

Educators’ Support Through Predictive Analytics

159

Fig. 5. Comparison of models’ accuracy in predicting the students’ group considering their learning performance

Fig. 6. Students’ evaluation workflow in specialty “E-management”

• How much of the project content is used from the lecture materials – possible answers: 25 = 100%; 16.6 = 75–50%; 8.3 = > 50%; Indicator 3; • Does the student give in the project his personal added value – possible answers: yes = 25 / no = 0; Indicator 4; • The evaluation scale for the final grades is as follows (Table 3): • What is the group of a given student considering his/her learning progress? – possible answers are: least developed, developing and developed; • What is the group of the student according to his/her learning performance? – possible answers are: in groups with high, average and low learning performance. For solving a classification problem of prediction the students’ learning progress and learning performance already created machine learning model from the previous two use cases with a Deep Learning Algorithm was applied to a dataset constructed of

160

M. Ivanova et al. Table 3. Final project grades distribution

MAX points

Grade

0 - 24

poor (2)

25 - 38

average (3)

39 - 52

good (4)

53 - 66

very good (5)

67 - 80

excellent (6)

E-management course – number of students as examples and indicators as features. DL algorithm has two hidden layers, first with 30 and second with 50 neurons, Rectifier activation function and adaptive learning rate, epsilon = 1.0E-8 and rho = 0.99. The ML model’s accuracy is high 96%, because of the well-balanced dataset. Figures 7 and 8 disclose the comparison of predicted students’ Learning Progress (LProg) and Learning Performance (LP) evaluated by the AE, RE, RMSE, RRSE and SE.

Fig. 7. Comparison of students’ learning progress evaluation

5 Conclusions A ML-based methodology for facilitating the educators in the assessment procedure is proposed. It uses an ANN to predict the students’ learning progress and learning performance. Several datasets are prepared as each one corresponds to a given assessment task, which is related to construction of digital objects. Overall students’ progress in a course is also predicted considering their learning progress in the assessment tasks. The

Educators’ Support Through Predictive Analytics

161

Fig. 8. Comparison of students’ learning performance evaluation

accuracy of the predictive models is evaluated as the reached errors are very small. The accuracies of the models are compared as the optimal state is found according to the ANN structure and its training parameters. Since the errors obtained for all models are relatively small, it can be concluded that the students’ progress and learning performance can be predicted with great precision. With a timely analysis of these two parameters via the proposed ML algorithm the educators successfully could plan the improvements and their implementation in educational process in order to prevent student’s dropping out and increasing the quality and success rate in the higher education institution. Acknowledgments. This research is supported by the Bulgarian FNI fund through the project “Modeling and Research of Intelligent Educational Systems and Sensor Networks (ISOSeM)”, contract KP-06-H47/4 from 26.11.2020.

References 1. Das, K., Behera, R.N.: A survey on machine learning: concept, algorithms and applications. Int. J. Innovative Res. Comput. Commun. Eng. 5(2), 1301–1309 (2017). https://doi.org/10. 15680/IJIRCCE.2017.0502001 2. Kamepalli, S., Rao, B.S.: Recent applications of machine learning: a survey. Int. J. Innovative Technol. Exploring Eng. 8(6), 263–267 (2019) 3. Kuˇcak, D., Juriˇci´c, V., Ðambi´c, G.: Machine learning in education - a survey of current research trends. In: Katalinic, B. (ed.), Proceedings of the 29th DAAAM International Symposium, pp.0406–0410. Published by DAAAM International, Vienna (2018). ISBN 978–3–902734– 20–4, ISSN: 1726–9679, https://doi.org/10.2507/29th.daaam.proceedings.059 4. de la O González, M., Jareño, F., López, R.: Impact of students’ behavior on continuous assessment in higher education innovation. Eur. J. Soc. Sci. Res. 28(4), 498–507 (2015). https:// doi.org/10.1080/13511610.2015.1060882

162

M. Ivanova et al.

5. Morales, M., Salmerón, A., Maldonado, A.D., Masegosa, A.R., Rumí, R.: An empirical analysis of the impact of continuous assessment on the final exam mark. Mathematics 10, 3994 (2022). https://doi.org/10.3390/math10213994 6. Alsariera, Y.A., Baashar, Y., Alkawsi, G., Mustafa, A., Alkahtani, A.A., Ali, N.: Assessment and evaluation of different machine learning algorithms for predicting student performance. Comput. Intell. Neurosci. 2022, 1–11 (2022). https://doi.org/10.1155/2022/4151487 7. Park, J.Y., Dedja, K., Pliakos, K., et al.: Comparing the prediction performance of item response theory and machine learning methods on item responses for educational assessments. Behav Res (2022). https://doi.org/10.3758/s13428-022-01910-8 8. Alruwais, N., Zakariah, M.: Evaluating student knowledge assessment using machine learning techniques. Sustainability 15, 6229 (2023). https://doi.org/10.3390/su15076229 9. Zhai, X., He, P., Krajcik, J.: Applying machine learning to automatically assess scientific models. J. Res. Sci. Teach. 59(10), 1765–1794 (2022). https://doi.org/10.1002/tea.21773

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory Programming to Large Classes of Novice Students Philippe Weidmann, Milo Gianinazzi, and Laurent Moccozet(B) Computer Science Centre, University of Geneva, Geneva, Switzerland [email protected]

Abstract. Universities are increasingly introducing programming courses for beginners into their curricula as programming knowledge becomes a key skill in various fields which results in larger class sizes for programming teachers. Three key aspects of efficient programming education were identified: integrated development environments, collaboration through pair programming, and assessment with feedback methods. A novel platform designed for novice students was developed to embed an Integrated Development Environment (IDE) and live pair programming in a web browser. The platform was tested by 192 student volunteers from an introductory programming course in the first year of a bachelor’s degree in economics and management. The results indicate that it was useable and group work provided increased grades. The solution was developed mainly as a proof of concept but can be easily adapted to other courses. Keywords: Collaborative Coding · Pair Programming · Peer-to-peer · Programming Platform · Web Based Learning · Web IDE

1 Introduction Many universities recognize that programming knowledge has become a key skill in many fields [1]. For this reason, universities are beginning to introduce more courses for beginners into their curricula. These courses can help students to develop programming skills, which will give them a better understanding of how computers work and how they can be used to solve complex problems. These courses can help students develop their ability to think logically and analytically, which can be useful in many professional fields. Finally, these courses can also provide students with practical experience with computer development tools and technologies, which can help them in their future job [2]. These new courses lead to an increase in the number of students in beginner programming classes, while the number of teachers and available resources does not necessarily increase. This increase in the number of students implies an increase in the workload for the teachers, especially when grading the work submitted by the students, but also when © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 163–173, 2023. https://doi.org/10.1007/978-3-031-42134-1_16

164

P. Weidmann et al.

installing the various tools. The fact that the students can come from heterogeneous backgrounds with limited computer experience makes the task even more complicated for the teachers, who generally must dedicate more time to these new classes than to conventional computer training courses. The objective of the work described in this article is to develop a learning platform particularly oriented for large numbers of students new to programming that allows them to have a form of feedback and support that teachers can hardly give them in this context, by introducing peer learning through an adaptation of the pair programming method used in the professional environment. The developed platform is tested with students of an introductory programming course and their feedback on the use of the platform as well as a comparison of their individual and pair results are presented and discussed to evaluate the relevance of the proposed approach.

2 Related Work In the face of growing class sizes of novice students, Sim and Lau [3] recommend that teachers adopt or develop automated tools for teaching and learning. These tools should provide personalized feedback and reduce teacher workload. Another pedagogical feature is to ensure that the learning management system can also support collaborative learning. In addition, one aspect that is often overlooked with regard to novice students in programming outside of a computer science curriculum is the Integrated Development Environment (IDE) needed to program effectively. From this point of view, the use of online compilers is seen by some students as a solution when they do not have a computer to install an IDE but use a tablet or a smartphone simply connected to the internet, or when they find it difficult to install/configure different IDEs [4]. Based on these observations, three key aspects of effectively teaching programming courses for beginners in large classrooms were identified. The first one is the use of integrated development environments as fully integrated platforms to allow the teachers to identify their students’ needs. The second one is pair programming. In pairs, students learn more and answer questions between themselves before asking the teachers. Also, pair programming is used extensively in professional environments [5]. The last aspect is the assessment and feedback of the exercises. Automated grading is often done for programming classes. However, it is hard to implement for beginners. 2.1 Integrated Development Environment Programming courses in more advanced classes often involves the installation of an IDE but setting up such software in less technical classes often takes more time and forces the teacher to dedicate more time to the setup instead of the course. Also, students now come to classes with a large variety of devices and some of them are not compatible with advanced IDEs. Web-based IDEs can help solve configuration issues and bootstrap teaching for large classrooms. Some web-based platforms allow teachers to follow their classes and create

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory

165

exercises but many of them are oriented toward more professional use. Only some of them allow group collaboration [6]. A fully integrated platform has been developed by Patil et al. [7], it allows teachers to create exercises and follow their students’ progress. The whole course is done within the web-based platform. Students can compile and submit their code directly on the site. However, this platform doesn’t allow collaboration. Liao et al. [8] made a platform that allows students to access a remote environment to code in C online. The use of this platform eliminated the need to install an IDE on the student’s computer which can be a barrier to learning especially for beginners. 2.2 Collaboration Programming, in professional environments, is rarely done alone. Yet, in classes group work is still occasional. Goldman et al. [9] observed that professionals were more efficient when coding together using a live online IDE. Reproducing pair-programming techniques has been experienced in classes by Williams et al. [10]. Following their experiment, they recommend that pairs should be modified regularly to prevent dysfunctional pairs to carry on for a whole semester. They also note that it could be useful to only grant the right to do pair programming for students who had good grades in individual exercises to prevent mismatched pairs. Rahman et al. [11] made extensive use of online tools to support an introductory course to programming. Students were paired and had to recreate a game by filling in empty functions. Students also learned using interactive eBooks. The results obtained by the students who took this new course were higher compared to the old one. 2.3 Assessment and Feedback Manually correcting students becomes hard to maintain in larger classrooms. The easiest way to grade many coding exercises is to use automated tests. However, automated tests are hard to introduce to beginner classes because students write code in a way that enables the tests to run. This is often not feasible. Machine learning can be an approach to this problem. Such solution has already been tested to give an early warning to teachers if their students need help [12]. Using their model, the author can identify students in need of help but can also infer the final grade of the students. However, this method needs data to train the model. Another solution is peer-review, at the end of the exercise, students grade their counterparts using a set of criteria. This method allows students to learn while grading the others. However, grading can take a lot of time and students tend to give better grades than teachers [13]. Saikkonen et al. [14] developed a system that does fully automatic code. When doing automatic code assessment some considerations need to be done. For example, the code must be stopped if it is running in an infinite loop. The executable code also has to be ran in a secure environment to prevent running malicious code. Finally, students find that feedback is an important part of the learning process.

166

P. Weidmann et al.

3 Proposed Solution: Splitcode A web-based platform called splitcode was developed. The goal of this platform is to provide a fully integrated site for teachers and students. On this website, teachers can create courses and exercises. The students can take courses and resolve exercises entirely on the platform. It integrates a code editor and provides remote code compilation and execution. In this article we focus on the student part of the platform. Students solve the exercise in two steps. They must first craft a solution individually. Once they are satisfied with their solution, they can submit their attempt. The second part is done in pairs. Students collaborate in a live environment in an experience that is similar to writing a text document on a cloud platform. At the end of the group attempt, the code is submitted, and students answer 4 questions about the code of another group. 3.1 Student Registration The user flow for the student part was designed to be simple and not require any specific learning (Fig. 1). The goal is to be able to simply direct students to the platform without the need for any instruction. Students register on the platform using an email address and a password. Once they are logged in, students can select courses from the courses created by the teachers. After registering for a course users can see the exercises that they have to do with their associated deadline. For each attempt of the exercise, a countdown and a deadline are provided.

Fig. 1. Students’ user flow.

3.2 Individual Attempt The first attempt for an exercise is done alone. From the solo attempt view, students have access to a code editor (Fig. 2). The code editor is built using Monaco, this is the editor that powers VSCode, the IDE developed by Microsoft.

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory

167

The editor supports multiple languages with built-in auto-completion and can provide help when hovering elements of the code. The students also have access to the instructions for the exercise on the right. On top of the editor, a tab is provided with quick access to run the code and finish the current attempt. The goal of the top tab bar is to provide a similar interface as text editors which beginners students are more familiar with. When the student is done with his current attempt, he/she can send his/her code and proceed to the group attempt.

Fig. 2. The solo attempt interface.

3.3 Group Attempt After submitting the solo attempt, students can continue to the group attempt. For this attempt, students are matched in pairs. Since for the platform test, students will only complete one round of exercises, the pairs are simply created in the order of student turn-in. Thus, the first student to submit will have his or her pair assigned when the second student submits. If the student does not have an assigned pair at the time of submission, he/she can still start writing the shared code. If the student does not have a peer at the time of starting the attempt, the interface lets him know. Also, until the second student in the pair arrives, the first student cannot surrender for the whole group. Clearly, for repeated use by students, a more refined pair assignment strategy should be found. The interface has three elements (Fig. 3) On the top left, an editor with the users’ code from the solo attempt. On the top right, the other student’s code from his solo attempt in read-only. On the bottom, a centred panel with an editor to write the shared code.

168

P. Weidmann et al.

Every character written by a student in any of the panels is seen live by the other student as with any collaborative text editor. Students can see each other’s position with a label containing their initials when they are focusing on one of the editors. Students also have a live chat to communicate. Once they are done with their code, one of the students can submit the exercise for the whole group. 3.4 Code Review After submitting the group attempt, students must review the code of another group (Fig. 4). The questions asked for the review process can be dynamically modified for each exercise. The answers are given on a scale from one to five.

Fig. 3. The group attempt interface with initials in the corner of the editor focused by each student.

3.5 Architecture The splitcode architecture is based on a set of open-source technologies and frameworks (Fig. 5). The front end is a web app developed in JavaScript with the framework React and interfaced with the back end using a REST API. The back end is a PHP API developed with the framework Laravel. Clients are connected through web sockets to a NodeJS server. User data is stored in a SQL database (MariaDB). These components are placed behind a NGINX reverse-proxy. The platform was developed to be tested with a course that uses Scala as its teaching language. The Scala code is compiled on the server with sbt, the Scala build tool, and translated to JavaScript with the framework ScalaJS. The JavaScript code is then sent back to the client’s web browser for execution.

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory

169

Fig. 4. Code review interface. On top, the peer pair code to review. Below, the questions asked by the teacher.

Fig. 5. Splitcode architecture1

4 Experiment The platform was tested with volunteer students from an introductory programming course done in Scala. The experiment was done over two weeks: one week for the solo attempt and one week for the group attempt. In total, 192 students registered on the website. 183 of them finished the individual attempt. From all the individual attempts, 174 students completed the group attempt. Finally, 140 of the 174 students peer-reviewed the other groups (this part was not mandatory). The discontinuation rate for the first attempt is 4.7% and for the group attempt 4.9%. The automated grading process was the following: 1/6 for submitted code. 2/6 if the code compiles without errors. 3/6 if the code executes without crashing. 5/6 if the code produces a value near the expected result. 6/6 if the code produces the right value for the given input. 1 Source code available at https://github.com/PhilippeWeidmann/splitcode.

170

P. Weidmann et al.

Table 1. Time spent by each student in second. Students that didn’t participate (time spent = 0) are excluded. Individual Attempt

Group Attempt

Mean

687.8

726.2

Median

570

555

For the individual attempt, students obtained an average of 4.88/6. For the group attempt the average was higher, at 4.98/6. We can see that group attempts produced better grades. The platform also tracks the time spent by each student on the exercise for each attempt. On average time spent on individual attempts was lower than for group attempts. However, the median is lower for group attempts (Table 1). The delta between the time spent by the two students for the group attempt has been computed. On average the delta is 606 s with a median of 435 s. This shows that in general for a given group one student will work a lot more than the other. This issue can be eliminated by giving an early warning to the teacher while the students are still working on the exercise. After the experiment, students were asked to answer a survey on an external platform. A selection of the questions is discussed here. 111 students answered the survey. Students were asked if they had previous knowledge of code before taking this introductory course. 50 of the 111 students answered they had previous knowledge. This number was higher than expected, this can be explained by the fact that volunteers were more likely to participate in the trial because they had previous coding knowledge and were interested in this concept.

Fig. 6. Diagram of responses to the question of whether the method of carrying out programming exercises with splitcode appears to be a better alternative than the conventional method.

Of the 111 students, 7 of them used the platform on a tablet for the whole exercise. The others did it using a computer. This result shows that even though the platform was designed to be used with a computer, it is also usable with a tablet. A majority of the student answered that working with another student was beneficial to them (answer above 3 out of 5). Most students (83%) said that they either preferred or found no difference in doing the exercise first on their own and then in pairs, rather than working in the usual way, i.e.,

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory

171

only on their own (Fig. 6). 49.5% found the proposed approach to be a better alternative than the traditional approach and only 17% found the proposed approach to be no better. There was a high rate of neutral responses (answer 3). This high rate can perhaps be explained by the relatively low level of difficulty of the exercise which would not have allowed the students to appreciate a big difference and a greater added value with the usual approach. It would probably be interesting to cross-reference these responses with the results obtained in the exercise to identify whether there is a link between results and feelings. One can imagine, for example, that good programmers do not necessarily appreciate the pair programming step, because it does not bring them anything but extra work. Lastly, 102 students gave at least a 4 out of 5 for the ease of use of the platform (Fig. 7). The results show that students appreciated working in groups and that the tested platform was accessible to novice students.

Fig. 7. Chart of responses to the question of how easy it was to use splitcode.

5 Conclusion With increasing classes size, teachers need efficient tools to teach students coming from different backgrounds which can be non-technical. A platform was developed to provide a fully integrated environment for students to solve computer science exercises. On this platform, they have access to the instructions for the exercise and can write and test their code directly in an editor. The process of solving the programming exercise allows them to propose their own solution, then compare and discuss it with another student to produce a second solution in a partnership. They finish by evaluating the code of another student to whom they provide feedback by answering several questions on several criteria of the evaluated code. The scope of the tests is clearly limited and would have to be extended to several cycles of exercises throughout a semester of classes, but the first results obtained seem to indicate that the pedagogical approach underlying the platform’s operating structure offers an interesting proposition in the context of teaching programming to large numbers of students in non-computer science curricula.

6 Future Work The developed platform shows promising potential but was developed as a Minimal Viable Product. To be used in a real environment, security improvements need to be made when compiling code on the server. The automated grading was implemented

172

P. Weidmann et al.

only for the test exercise. However, it can easily be adapted and generalized for other exercises. A vocal chat could be added to enable easier communication between the students when they are coding together. For now, only Scala is supported but JS and other languages that can be executed directly in the browser could easily be added to the platform. It also seems possible to identify several indicators of students’ individual and global abilities and skills during the completion of these exercises. For example, the time spent solving the exercise in solo and in pairs. These indicators could be used to create a scorecard for each student so that they can see how they compare to the rest of the class. It might also be useful to communicate them to teachers so that they can monitor the progress of the class as a whole and identify students who are potentially in difficulty. Machine learning techniques like word embedding can be applied to the student’s answers so that students with alike answers can be matched together. The results of these analyses could be used to inform teachers and guide the creation of student pairs.

References 1. Camp, T., et al.: Generation CS: the growth of computer science. ACM Inroads 8(2), 44–50 (2017). https://doi.org/10.1145/3084362 2. Iqbal, S., Harsh, O.K.: A self review and external review model for teaching and assessing novice programmers. Int. J. Inf. Educ. Technol. 3(2), 120–123 (2013). https://doi.org/10.7763/ IJIET.2013.V3.247 3. Sim,T.Y., Lau, S.L.: Review on challenges and solutions in novice programming education. In: 2022 IEEE International Conference on Computing (ICOCO), pp. 55–61 (2022). https:// doi.org/10.1109/ICOCO56118.2022.10031657 4. Sinanaj, L., Ajdari, J., Hamiti, M., Zenuni, X.: A comparison between online compilers: a case study. In: 2022 11th Mediterranean Conference on Embedded Computing (MECO), pp. 1–6 (2022). https://doi.org/10.1109/MECO55406.2022.9797096 5. Sadowski,C., Söderberg, E., Church, L., Sipko, M., Bacchelli, A.: Modern code review: a case study at google. In: Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice, pp. 181–190. ACM, Gothenburg (2018). https:// doi.org/10.1145/3183519.3183525 6. Zinovieva, I.S., et al.: The use of online coding platforms as additional distance tools in programming education. J. Phys. Conf. Ser. 1840(1), 012029 (2021). https://doi.org/10.1088/ 1742-6596/1840/1/012029 7. Patil, M.S., Deore, S.N., Bisht, M.H.: Synergic coding system: an online coding platform. Int. J. Res. Appl. Sci. Eng. Technol. 10(11), 982–987 (2022). https://doi.org/10.22214/ijraset. 2022.47495 8. Liao, J., Chen, S., Xiong, H.: A cloud-based online coding platform for learning coding-related courses of computer science. ICIC Express Lett. Part B Appl. 8(1), 109–116 (2017) 9. Goldman,M., Little, G., Miller, R.C.: Real-time collaborative coding in a web IDE. In: Proceedings of the 24th annual ACM symposium on User interface software and technology UIST 2011, p. 155. ACM Press, Santa Barbara (2011). https://doi.org/10.1145/2047196.204 7215 10. Williams, L., Yang, K., Wiebe, E., Ferzli, M., Miller, C.: Pair programming in an introductory computer science course: initial results and recommendations. In: OOPSLA Educator’s Symposium. Seattle (2002)

Proposal for a Peer-to-Peer Coding Platform for Teaching Introductory

173

11. Rahman,M.M., Paudel, R., Sharker, M.H.: Effects of infusing interactive and collaborative learning to teach an introductory programming course. In: 2019 IEEE Frontiers in Education Conference (FIE), pp. 1–8. IEEE, Covington (2019). https://doi.org/10.1109/FIE43999.2019. 9028657 12. Ahadi,A., Lister, R., Haapala, H., Vihavainen, A.: Exploring machine learning methods to automatically identify students in need of assistance. In: Proceedings of the eleventh annual International Conference on International Computing Education Research, pp. 121–130. ACM, Omaha (2015). https://doi.org/10.1145/2787622.2787717 13. Hämäläinen, H., Hyyrynen, V., Ikonen, J., Porras, J.: Applying peer-review for programming assignments. Int J Inf Technol Secur 1, 3–17 (2011) 14. Saikkonen,R., Malmi, L., Korhonen, A.: Fully automatic assessment of programming exercises. In: Proceedings of the 6th annual conference on Innovation and technology in computer science education, in ITiCSE 2001, pp. 133–136. Association for Computing Machinery, New York (2001). https://doi.org/10.1145/377435.377666

Effectiveness of a “Nudge” for Online Discussion Participation About Attitude Toward Essay Writing Minoru Nakayama1(B) , Satoru Kikuchi2 , and Hiroh Yamamoto2 1

Tokyo Institute of Technology, Meguro, Tokyo 152-8552, Japan [email protected] 2 Shinshu University, Matsumoto, Nagano, Japan

Abstract. “Nudges” were used as incentives to promote greater online discussion activity. Their effectiveness was evaluated using text analysis of the characteristics of essays. The participants who were prompted to write comments were defined as people who posted only a single time. In order to evaluate their attitudes, essay texts were analysed as a form of learning performance. The length of essays of self-motivated participants and those who were prodded increased. The frequency of dependency pairs in the essay texts also increased along with the level of participation in online discussions. However, there were no differences in the lengths of essay comments, which remained similar between the three participation levels. The results of analysis of extracted term frequency using sentiment analysis show the same tendency for essay reports. The characteristics of essays of participants who were prompted are summarised. Keywords: Student’s Essay · Online discussion analysis · Fully online learning

1

· Nudge · Lexical

Introduction

In order for a society facing natural disasters to better adapt to changes caused by these disasters [1], an attempt to develop the critical thinking ability of youths on a large scale was made [9,10,13]. These improvements have been introduced in formal lectures and online courses over several years. Discussion stimulating activities may be an effective way to develop critical thinking attitudes and appropriate decision-making abilities [1,4,5,7,14]. As online discussion activity may be suitable for sharing information and exchanging differing opinions between participants, it may be a possible means to develop critical thinking skills [2,6]. However, students registered in these courses often hesitate to participate in optional online discussions. An appropriate invitation procedure, or an incentive such as a prompt was proposed to stimulate greater online discussion. Of course, a certain number of participants who understood the issues surrounding these c The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  Z. Kubincov´ a et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 174–181, 2023. https://doi.org/10.1007/978-3-031-42134-1_17

Effectiveness of a “Nudge” for Online Discussion Participation

175

societal problems and the natural human disinclination to participate in discussions actively joined the bulletin board discussions of these topics many times. As a result, participants from these two groups posted their opinions online during the course. If their posting behaviour and learning performance were comparable to self-motivated participants, the effect of stimulation may be sufficient for students who were encouraged to participate [15]. The results of analysis may provide encouragement more appropriate for the remaining participants. This paper confirms the effectiveness of promoting the joining of online discussions, such as using “Nudges”, which is based on a simple text analysis of online postings and submitted essays. The following topics will be addressed in the sections below. 1. The contribution of an invitation to participate in online discussions is examined using analysis of student’s posting behaviour during a course. 2. The effectiveness of joining online discussions by motivating participants to join is confirmed using simple text analysis of their essays, which are a form of critical thinking exercise. All analysis is based on a set of essay presentations which include online postings, reports, and comments, which are a part of the natural communication that occurs during fully online university courses which are currently being taught [3].

2

Method

All surveys were conducted during a bachelor level course at a Japanese university. The course target is the development of critical thinking ability in order to help mitigate the effects of natural disasters. The course was organised as a fully online course due to Covid-19 [9,10]. 2.1

Survey Course Setting

Course content to be learned each week was summarised in video clips, and was delivered using a learning management system (LMS). In addition to weekly tests, some essay writing was required, in order to confirm retention of course content and ensure the development of critical thinking ability. During the course, participation in discussion activities is emphasised. The course lecturer organised an online discussion board to facilitate discussion of the course topic by participants. Participation in online discussions was encouraged, but it needed to be optional for registered students. In order to encourage the joining of course discussions, participants could obtain additional marks which contributed to their final course mark. However, this reward feedback was not effective in stimulating online discussion. The lecturer tried prompting discussion using participation triggers. All online postings were recorded using the LMS. The posted messages were analysed after the course was completed.

176

M. Nakayama et al.

Level:2

Level:1

Level:0

Fig. 1. Comparing of text lengths (TextVolume) of submitted essays for two tasks using three levels of frequency of posting to the course online discussion board.

2.2

Essay Report and Comments

An essay report task was assigned to all participants in order to evaluate their attitude toward developing critical thinking ability and was to be marked as part of the overall assessment in the final grade. The valid number of essay submissions was 440. The report task was to consider a problem concerning human perception in daily life which was based on pictures taken by students. Another comment task was to summarise their recognition of the guest lecturer’s oral presentation, which consisted of providing information about the current communication resources used in the mitigation of natural disasters. These essays were submitted via the LMS platform, and the lecturer’s assessment scores were feedback sent to individual contributors via the LMS system. 2.3

Text Analysis of Essay Reports

All essays were presented in Japanese, and the essay texts were extracted from the documents. These were summarised as sets of plain text, and Japanese morphological analysis was applied to extract certain part of the texts. Dependency analysis of each sentence was conducted, and sentiment analysis was also conducted using a commercial language processing system [12]. In addition, simple analysis such as text length was conducted, as was done in previous studies [8,11].

3 3.1

Results Levels of Participation in Online Discussions

Ordinally, there were few discussants since course participants usually hesitated to join the online discussions. From the efforts of the lecturer, 127 out of 440

Effectiveness of a “Nudge” for Online Discussion Participation

bottun - press one’s affair - aware + unable photo - view environment - represent person - be interpretation - allow + unable necessity - suppose photo - take necessity - consider time - cost person - be many text characters - write anti-disaster - learn sight - view photo - view + like to sticker + paste interpretation - allow usage - aware + unable more than two - interpret signboard - view

177

Level:2

Fig. 2. Dependencies in report essays and their frequencies for three levels of online discussion participation.

Level:2

Fig. 3. Correspondence analysis for frequencies of dependency for three levels of online discussion participation.

students participated in the discussions. 29 percent of the registered participants joined the online discussions. However, when individual posting frequency was measured, 64 students posted only once. The remaining 63 students posted an average 3.8 times. The posting frequency of the two groups is completely different, though in comparing with participants who didn’t post, the participants who posted evaluated as proactive students. The former group (who posted once) might have joined in order to receive a reward for their performance. The length of messages posted by the two groups was comparable. In regard to posting frequency, participants were classified into levels 0, 1 or 2, representing no postings, 1 posting, or frequent postings, respectively. In regard to online discussion board activity, student’s participation in the course may differ. In order to evaluate the learning activity of online discussion participants during the course, the length of essays and comments submitted by all participants are compared using the three levels for clasifying online posting frequency. The participants who are proactive about online discussion may also make more considerable effort with their course assignments. The results are summarised in Fig. 1. Here, text volume shows the length of overall texts in essays. There are significant differences in the lengths of essays submitted for levels 0 and 2 (p < 0.01), but no significant difference for levels 0 and 1. Regarding the comments, there are no significant differences between the three levels. For essay reports, individual preferences are more freely expressed, and thus essay length may be longer for students who were encouraged to express themselves. As the comment task was simple for all students, comments are easily comparable. A detailed analysis of essay texts will be conducted in the sections which follow.

178

M. Nakayama et al.

information - obtain information - obtain + able disaster information - obtain early stage - obtain procedure - suppose necessity - consider disaster - occur disaster - happen I - reside person - be many disaster happening - obtain disaster - occurrence TV - switch on during disaster - obtain custom - be familiar ear - listen to TV - view information - obtain + unable procedure - consider information - get + able

Level:2

Level:2

Fig. 4. Dependencies in essay comments and their frequency by level of online discussion participation.

3.2

Fig. 5. Correspondence analysis for frequency of dependency for three levels of online discussion participation.

Dependency Analysis of Essays

Dependency analysis was applied to all essay texts, and frequent dependencies between the three levels of online discussion participation were extracted and compared. The frequencies for essay reports are summarised in Fig. 2. The figure shows the top 20 dependencies. There are some differences in frequency for pairs of dependency between the three levels. In particular, participants in level 2 used these dependencies more than the majority of participants in level 0, since they summarised longer reports, as Fig. 1 shows. Even the level 1 participants used some dependency pairs more frequently than level 0 participants did. Level 1 participants were encouraged to join online discussions after being prodded by the lecturer. In order to analyse the relationship between dependency pairs and levels of participation, correspondent analysis of the first and second major components was introduced, and is illustrated in Fig. 3. In the figure, a light coloured mark indicates a dependency pairs and a dark coloured mark indicates the values of the level of participation of each of the three groups. The size of the circle corresponds with the relative frequency of the dependency pairs. Some pairs of dependency are used selectively, and the plots of component pairs are diversified, as is shown in Fig. 3. Some specific pairs are placed in the neighbourhood of plots for groups of the same level, as they are representative of that group. Another set of dependency pairs in essay comments is summarised in Figs. 4 and 5. The frequencies of dependency pairs are comparable between levels of online discussion participation, as Fig. 4 shows. In an illustration of twodimensional features of these dependencies in Fig. 5, most features of dependencies are gathered around the feature mean for the level 0 group. Even the lengths of essay comments for the three levels are comparable, and there are no significant differences between the two figures.

Effectiveness of a “Nudge” for Online Discussion Participation human design place display decision signboard improvement operation comprehension interpretation text declaration setting presentation explanation unification place invention information photo

Level:2

information human

179

Level:2

disaster information

confirmation procedure utlisation actions during disaster application invention acquisition grasp SNS TV media information survey

use evacuation dispatch decision feature

Frequency

Fig. 6. Dependencies in essay comments and their frequency for three levels of online discussion participation.

3.3

Fig. 7. Correspondence analysis for frequency of dependency for three levels of online discussion participation.

Sentiment Analysis of Essays

Critical thinking attitude may reflect the differences in student’s positive and negative forms of presentation in sentences. Therefore, discussion essays may contain some negative sentences as criticism of some phenomena. These are extracted from essay texts using a text processing tool [12], and sentiment analysis of term frequencies in positive and negative sentences. The top 20 most frequent terms were extracted, and their frequencies summarised in Fig. 6 using the three levels of online discussion participation. The positive relative frequency shows the terms that are used in positive sentences, and the negative relative frequency shows the term that are used in negative sentences. Most terms for positive sentences are presented, but all terms for negative presentations are also noted. One third of the terms are used in many instances by the level 2 group, who posted online frequently. The results for the essay comments are summarised in Fig. 7. There are a few negative frequencies and a few differences in positive frequency between the three levels. The first three terms were frequently used for essay comments. Two of the three terms are common in the results for essays in Fig. 6. Since the frequencies of terms for negative presentations are small, most comments are summarised as positive impressions of a given topic. The possible reason may be the limitation of essay length or discussion points being used concerning a specific topic. A detailed analysis of the contents of the essays which were submitted will be the subject of our future work.

180

4

M. Nakayama et al.

Summary

The effectiveness of using “Nudges” to stimulate online discussions was examined, by comparing online posting activity and essay writing performance. The participants who were prodded are defined as those posting only once. Their attitude and performance were compared with those of self-motivated participants and participants who declined further online discussion. The following points were extracted. 1. During an online discussion, self-motivated participants posted an average of 3.8 messages, though “Nudged” participants provided only a single post. 2. Learning performance was evaluated using written essays. For the report essay task, self-motivated participants presented longer essays than those of other participants. The essay lengths of participants who were prodded is comparable to that of non-participants. 3. Frequencies of dependency pairs in essay reports submitted by participants were sorted into three levels and compared. The frequencies for students who were self-motivated, and for participants who were prompted, increased gradually. However, there were no significant differences in frequency of dependency for essay comments. 4. Also, frequently used terms extracted using sentiment analysis are different for report essays, but there are no differences for essay comments. The cause of the difference in essay characteristics should be examined in order to develop an appropriate set of procedural instructions, and thus an analysis of the essays which were submitted will be required. This will also be the subject of our future work. Acknowledgement. This research was partially supported by the Japan Society for the Promotion of Science (JSPS), KAKEN (21K18494: 2021–2023).

References 1. Attems, M.S., Thaler, T., Snel, K.A., Davids, P., Hartmann, T.: The influence of tailored risk communication on individual adaptive behaviour. Int. J. Disaster Risk Reduction 49, 1–9 (2015) 2. Cole, J., Ezziane, S., Watkins, C.: Rapid creation of an online discussion space (r/nipah) during a serious disease outbreak: observational study. JMIR Public Health Surveill. 5(4), 1–8 (2019) 3. Ferreira, M., Rolim, V., Mello, R.F., Lins, R.D., Chen, G., Gaˇsevi´c, D.: Towards automatic content analysis of social presence in transcripts of online discussions. In: Proceedings of LAK 2020, pp. 141–150 (2020) 4. H¨ oppner, C., Whittle, R., Br¨ undle, M., Buchecker, M.: Linking social capacities and risk communication in Europe: a gap between theory and practice? Nat. Hazards 64, 1753–1778 (2012) 5. Kawamoto, S., Nakayama, M., Saijo, M.: Using a scientific literacy cluster to determine participant attitudes in scientific events in Japan, and potential applications to improving science communication. JCOM 12(1), 1–12 (2013)

Effectiveness of a “Nudge” for Online Discussion Participation

181

6. Kusumi, T., Tanaka, Y.: A development of critical thinking ability during a class of English for specific purpose. In: Proceedings of JAEP Annual Meeting, pp. PF2–35 (2008) 7. Leh, A., Kremling, J., Nakayama, M.: Effects of the blog and discussion board on online teaching and learning. In: Proceedings of Society for Information Technology and Teacher Education International Conference, pp. 574–579 (2012) 8. Nakayama, M., Kikuchi, S., Uto, M., Yamamoto, H.: Evaluation of essays and comments for developing critical thinking ability during a university course. In: Proceedings of Psychology Learning Technology (PLT2022), pp. 1–16 (2022) 9. Nakayama, M., Kikuchi, S., Yamamoto, H.: Development of critical thinking disposition during a blended learning course. In: Proceedings of 19th European Conference on E-Learning, pp. 358–364. Berlin (2020) 10. Nakayama, M., Kikuchi, S., Yamamoto, H.: Development of critical thinking disposition using an online discussion board during a fully online course. In: Proceedings of 21st European Conference on e-Learning, pp. 295–301 (2022) 11. Nakayama, M., Kikuchi, S., Yamamoto, H.: Phrase features in essay report sentences for developing critical thinking ability in a fully online course. In: Proceedings of International Conference on Information Visualisation (iV2022), pp. 240–244 (2022) 12. NTT-Data Suuri: Text Data Mining Studio (2013) 13. Thomas, M., Klemm, C., Hutchins, B., Kaufman, S.: Emergency risk communication and sensemaking during smoke events: A survey of practitioners. Risk Anal. 43, 358–371 (2023) 14. Williams, H.T., McMurray, J.R., Kurz, T., Lambert, F.H.: Network analysis reveals open forums and echo chambers in social media discussions of climate change. Glob. Environ. Change 32, 126–138 (2015) 15. Yoo, J., Kim, J.: Can online discussion participation predict group project performance? investigating the roles of linguistic features and participation patterns. Int. J. Artif. Intell. Educ. 24, 8–32 (2014)

Design Considerations, Architecture and Implementation of a Wireless Sensor Network for Use in Smart Education Svetozar Ilchev(B) Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria [email protected]

Abstract. The paper identifies and analyzes some important aspects that pertain to the design and implementation of a wireless sensor network for smart education purposes. These aspects influence the technical, economic, and security-related characteristics of the network and its nodes. A suitable structure of the network nodes and an overall architecture of the network are proposed. An overview of their prototype implementation is briefly presented, which has both hardware and software parts. The implementation includes a custom communication protocol developed as part of our previous work in the industrial sector. It provides secure communication within the network and implements features such as packet integrity verification and retransmission of lost or damaged packets. The preliminary results of the conducted experiments are very promising and confirm that our network design is suitable for the application field. They also helped us identify some specific areas for future improvements that should increase the performance, usability and cost of the wireless sensor network and its nodes. Keywords: Wireless Sensor Networks · Smart Education · Internet of Things · Embedded Systems

1 Introduction The use of wireless sensor networks (WSNs) has become an increasingly popular trend in various use cases related to the automated gathering of data. This has been made possible by several technological and commercial advances that have enabled the design and development of new WSNs on a relatively low-cost basis as long as the design team has the necessary knowledge and experience. First, on the hardware side, relatively cheap and energy-efficient microprocessor units are available for purchase in small quantities (e.g. STM32, LPC microprocessors). Some of them integrate wireless communication transceivers (e.g. STM32Wx). New integrated circuits (IC) for energy-efficient wireless communication have been developed with new communication standards in mind (e.g. LoRa, NB-IoT). Custom communication solutions are also available (e.g. nRF24L01 + by Nordic Semiconductor). A large range of highly efficient power-delivery ICs for various applications are also available from manufacturers such as TI. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 182–191, 2023. https://doi.org/10.1007/978-3-031-42134-1_18

Design Considerations, Architecture and Implementation

183

Second, on the software side, several important open source design and development tools that are necessary for the creation of electronics products in general and WSNs in particular have matured to the point of becoming viable alternatives to expensive commercial solutions. Our design and implementation efforts presented in this paper have been made possible by the use of KiCad, GCC, Code::Blocks, LibOpenCM3 and several others. Third, the availability and cost of manufacturing services that are usually needed for the implementation of a sensor node have improved significantly even for small batches of less than a dozen units. In our case, these services involve the production of a printed circuit board (PCB) populated with all necessary electronic components and its assembly into a suitable sensor node enclosure. The considerations outlined above have influenced our decision to explore the design and implementation of a WSN for use in smart education. This is a specific use case, which merits additional investigation. For this purpose, a flexible WSN is needed that can be quickly adapted to a given classroom and added on top of whatever other devices and infrastructure are already present. Existing solutions did not have this kind of flexibility and we started the design and development of a new WSN, which is described in this paper. Subsequent paper sections include the brief presentation of related work and the discussion of important design considerations related to the architecture of the WSN. Next, the WSN implementation is discussed along with some experimental results. The paper concludes with the identification of a few future changes and enhancements that should lead to better usability of the WSN in our smart education context.

2 Related Work This section outlines some research related to WSNs and smart education. In [1], an overview of the use of innovative technologies such as the Internet of Things (IoT) in education is presented. The authors discuss achieving sustainable education assisted by IoT (sensors, wearable devices, etc.) from three distinct perspectives - school administrators, teachers, and students. The authors of [2] present a broad view of smart education part of which are smart learning environments. IoT and digital sensors are an important part of the architecture of these environments. In [3], sensors are used to assist teachers in improving the education process when performing hands-on activities. In [4], wearable sensors for acquiring electrocardiograms are used both to make students acquainted with the specifics of the sensors themselves and educate them on the analysis of generic electrocardiogram data. The authors of [5] present a large literature review of various sources that discuss using robotics, sensors and learning by doing in primary and secondary schools. In [6], the use of augmented reality and electroencephalographic sensors to provide feedback is researched to assess the student attention during learning. In [7], wearable sensors for evaluating users’ emotions are employed to assist educators in optimizing the learning process of challenged students. A classic use case of measuring physiological parameters via sensors in order to assess the effects of the educational process is discussed in [8]. The authors of [9] propose the creation of a sensor network for greenhouse monitoring and control, which is used in an educational context at a local elementary school. Another comprehensive review of the role of IoT devices in educational activities along with the relevant benefits and challenges is presented in [10]. The

184

S. Ilchev

authors of [11] explain some IoT concepts, definitions and technologies that are relevant to smart education and argue that IoT devices have great potential to increase the quality of the educational process and raise the efficiency of learning. In [12], the use of social robots for human interaction is evaluated with regard to education and learning. Their effectiveness, advantages, drawbacks and future potential are discussed. The authors of [13] discuss the importance of IoT technology for achieving better teaching efficiency in the field of education in general and natural sciences in particular. The authors of [14] present a comprehensive study with a focus on the progress of adopting IoT devices and wearables in various subfields of education such as medical and vocational education. In [15], the concept and possible applications of a smart classroom are introduced, some ethical implications are discussed and appropriate new regulations are recommended. A strong trend in education revolves around educational games and the gamification of the educational process, which makes it more engaging for students [16, 17]. WSNs can be an integral part of these efforts and gather data from the learning environment (e.g. temperature, humidity, lighting, sound, etc.), which can then be used to assess students’ actions and attention level and influence the teaching process accordingly. We also experiment with some applications of WSNs apart from the educational context discussed so far, which include the transmission of data in transportation networks [18], localization of mobile devices [19], gathering data from medical surgeries [20], conducting reliability evaluations [21, 22], etc.

3 Design Considerations and Architecture Smart education is a specific use case for WSNs and it has its own set of requirements, which have to be considered before designing the sensor units and the network itself. As education is multifaceted, one important requirement is that the WSN should be flexible. It should rely as little as possible on existing power and communication infrastructures and it should be able to incorporate various types of sensors. At this stage, smart education is still a relatively new concept and the technologies related to it such as WSNs are regarded as enhancements. They should not be too difficult to add on top of an existing classroom and should impose as few new requirements as possible on teachers and school administrators. A second requirement is related to the economic realities of most schools. WSNs for smart education should be relatively cheap and accessible for manufacturing and repair. A third requirement is the integration with existing cloud-based systems running on computer servers that will receive, store and evaluate the sensor data. Some educational institutions already have a system for storing educational materials, generating assessment tests, keeping track of students’ progress, etc. Ideally, the data from the WSN should be stored in this system and used to improve the teaching process. Last but not least, the WSN should incorporate some form of security so that the sensor data that pertain to the students in a given classroom are not made available to the wide public during the process of gathering and transmission to the cloud-based storage system. Considering these requirements, a few important design considerations for the sensor nodes and the corresponding WSN are identified. First, the sensor nodes should have a modular structure consisting of several distinct functional building blocks. The individual

Design Considerations, Architecture and Implementation

185

blocks should be replaceable with new blocks of the same type if repairs or modifications of the sensor node are needed. This type of modular structure makes the sensor nodes flexible and for a given classroom and given educational needs, a suitable combination of blocks can be chosen. The modular structure also enables flexible pricing - if larger amounts of funding are available, then more expensive blocks can be used and vice versa. This is useful as some sensors (e.g. sensors for measurement of the air quality) are more expensive than others (e.g. temperature sensors). Our sensor nodes follow the modular structure shown in Fig. 1. They consist of four distinct types of functional building blocks - a power supply block, a microprocessor block, one or more communication blocks and one or more sensor blocks. Sensor node Sensor

Communication Communication

Microprocessor ARM Cortex-Mx

Communication

Sensor Sensor

Power supply

Fig. 1. Modular structure of a sensor node.

The power supply block is connected to all other blocks and provides power to their electronic components. The output voltage of the block is in the range of 1.8V to 3.3V depending on the communication and sensors in use. Two types of this block are designed. The first type works with a nearly constant input voltage in the range 5V-24V DC. This block can draw and convert power from notebook or computer power supplies, various wall-mount adapters such as mobile phone chargers, lead-acid batteries and li-ion battery packs. Its main intended application areas are indoor classrooms and gymnasia. It is also suitable for use with the electrical outlets of trucks, vans, busses, RVs, etc. The second type is intended to continuously convert power from a small independent solar panel and store it in a supercapacitor, which then provides power to the other building blocks. This second power supply block is several times more complex and expensive than the first one and typically provides less total available power to the sensor node. Its main advantage is that the sensor node becomes independent of the existing power infrastructure and can be placed in open spaces or locations without any electrical wiring as long as there is enough available sunlight to charge the supercapacitor. One additional feature is that it is an ecologically conscious and relatively safe solution, which does not rely on chemical batteries. The microprocessor block is the core of the sensor node, which executes the firmware, controls the communication and handles the sensor data. It incorporates an ARM CortexM0, M3 or M4 microprocessor depending on the complexity of the data gathering tasks and the available energy. In most cases, a low-power Cortex-M0 is sufficient. The microprocessor has three important tasks. First, it must handle the gathering, processing and intermediary storage of sensor data. Second, it must ensure the timely transmission of the data to the cloud-based storage system. Third, if running on the second type of the

186

S. Ilchev

power supply block, the microprocessor must keep track of the available energy stored in the supercapacitor and conserve it, if necessary. The sensor node may include one or several communication blocks, each of which supports a given communication technology or interface. At this stage, experiments are conducted with several wireless communication technologies (LoRa, nRF24L01 + and WiFi) and several wired communication technologies (RS-232, RS-485, USB and Ethernet). Typically, most sensors need only one wireless low-energy communication block (e.g. LoRa or nRF24L01 +). They use this communication link to send the gathered sensor data either in real time or at regular time intervals to a dedicated node, which functions as a gateway. Besides the wireless communication block, this gateway node contains at least one other communication block, which is used to establish a connection either to a local computer (e.g. via USB, RS-232 or RS-485) or the Internet (e.g. via WiFi or Ethernet). The gateway may also function as a sensor node but its most important role is to relay the data from the WSN to the cloud-based system that stores this data permanently. Last but not least, the sensor node includes one or more sensor blocks, which measure physical parameters of the environment, e.g. temperature, humidity, lighting conditions, air quality, sound levels, presence of movement, etc. Some sensors are relatively cheap (e.g. temperature sensors) while other sensors may cost more than the rest of the sensor node (e.g. gas sensors). Different application scenarios need different sensors, so the sensor node must be flexible in this regard. The main difficulty in incorporating different types of sensors is related to the placement of the sensors relative to the sensor node enclosure. Sensors must have access to the environment outside the enclosure, which necessitates the presence of holes or slots or the use of transparent materials. After establishing the structure of the sensor nodes, a suitable architecture of the WSN must be decided on, which should also follow the design considerations outlined above. First, the architecture should take into consideration the capabilities of the relatively low-cost communication hardware and the energy limitations of the second type of power supply block. Second, the architecture of the WSN should facilitate the task of the gateway node to relay the data to a local computer or the Internet. Third, some form of security should be implemented to protect the data from unauthorized third parties. From an application point of view, the WSN should be capable of gathering sensor data from both a single classroom and a whole school building. For the moment, due to the aforementioned hardware and energy limitations, the WSN nodes use static addresses. New devices can be added without the need of reconfiguring the gateway or the neighboring nodes but each new device has to be configured with a suitable address before it can communicate with the rest of the WSN. The major benefit is that the nodes may make use of longer sleep intervals and conserve energy, as no dynamic addressing protocols have to be supported. This is clearly an area, which needs further analysis and improvement, which will be a focus of our future research. On the hardware level, the WSN permits the use of a full-mesh topology due to the specifics of the employed low-power radio links (LoRa and nRF24L01 +). On the logical level, the full mesh topology is restricted to a star topology (Fig. 2). The gateway node functions as a central hub, which provides an intermediary data storage and relays the data to the cloud-based storage system. It is assumed that the gateway node uses a

Design Considerations, Architecture and Implementation

187

power supply block of the first type with a suitable energy source. Thus, the sensor nodes may decide on appropriate sleep intervals and may be powered by low-energy sources. The star topology also makes the routing of the data very straightforward. This topology is possible because a LoRa radio link can typically cover the whole school building. nRF24L01 + radio links can cover a single classroom but need routing to reach a gateway in another classroom. The WSN uses a custom communication protocol of our design [23], which also supports routing, but our experiments at the moment focus on direct gateway-to-node connections. The protocol provides some degree of security and reliability of the communication via AES encryption, a customized handshaking mechanism, shared password authentication, packet integrity verification and retransmission of lost or damaged packets. The shared password authentication is a compromise as low-cost hardware is generally not suitable for asymmetric cryptography.

Fig. 2. WSN architecture

4 Implementation The prototype implementation of the WSN employs microcontroller units of our design that can function as a sensor node or a gateway (Fig. 3).

Fig. 3. Microcontroller unit that can function as a sensor node or a gateway

The microcontroller units can be powered by a solar panel and store the energy in a supercapacitor as described in the previous section. The microprocessor block is

188

S. Ilchev

equipped with a low-power Cortex-M3 microprocessor of the series STM32L1. Multiple sensors are attached to the extension connector - at the moment temperature, humidity, light intensity and sound can be measured. The connectivity to a computer system for the purposes of data forwarding is implemented via the RS-485 communication block and an external USB-to-RS-485 convertor. When connected via the RS-485 interface, power to the unit is delivered via the USB link instead of the solar panel. Admittedly, this microcontroller unit does not fully conform to the architectural guidelines of the previous section, as it was first developed and produced to serve data gathering purposes at outdoor locations without reference to smart education. Using it as a starting point to implement a WSN for smart education helped us define the improved structure of the nodes and the WSN presented in the previous section. The benefits of the unit in Fig. 3 is that it is close to what we want to have as a sensor node - it has suitable enclosure, includes several well-tested detachable sensors and uses a suitable communication protocol already tested in WSNs for industrial automation [23]. What the unit lacks is the full concept of detachable and replaceable building blocks. The design of its next version includes a communication connector for nRFL01 + transceivers (right now, we have hijacked part of the extension connector for sensor blocks for this purpose) and an Ethernet communication block with the WizNet W5500 integrated circuit, which is already designed and tested as part of another device. The nRF24L01 + radio link lowers the cost if the WSN needs to cover only a single classroom. The Ethernet communication block offers the possibility for a direct TCP/IP connection to a router without the involvement of a local computer. It is also beneficial to be able to swap the Cortex-M3 microprocessor for a Cortex-M0 of the series STM32L0 to lower the cost. In our current use case, only the gateway benefits from the use of the more powerful STM32L1. The STM32L0 is fully sufficient for the sensor nodes. The LoRa radio link can remain largely unchanged. The communication protocol offers the necessary functionality to transmit the data securely and reliably between the sensor nodes and the gateway. Its functionality was developed with industrial applications in mind but the preliminary experimental results in the next section already show that its features are also very useful for applications in the field of smart education.

5 Experimental Results In order to check the relevance of the concepts presented in the previous two sections, a small WSN consisting of several sensor nodes and one gateway was constructed. The microcontroller units shown in Fig. 3 were used for this purpose. The practical experiments gave us better insights into the needs and challenges that are characteristic for smart education. An important observation was that the LoRa communication has enough range to establish a direct connection to the gateway in all instances within a single large building. This means that there is no practical need for intermediary sensor data routing. The nodes can use a low-power microprocessor and all nodes besides the gateway may sleep for long periods of time. This was not the case with the nRF24L01 + radio link, which functioned reliably within a single large room but could not transmit data between rooms in all cases.

Design Considerations, Architecture and Implementation

189

Still, its low cost is a definite advantage especially if multiple dozens of locations within a single classroom (e.g. each student desk) need to be monitored. The solar panel power supply provided insufficient operating time indoors, which was to be expected. Part of our future work will include lowering the main operating voltage to values in the range 1.8V-2V to improve the power consumption of the nodes. The communication blocks (both LoRa and nRF24L01+) will have to be modified and the communication range will surely be reduced. Some older sensors such as SHT21 will also have to be replaced with newer alternatives but we do not expect much difficulty. The communication to the cloud via the RS-485 communication block was successful but this implementation is rather inconvenient, as it requires a connection to a local computer on the premises, which is usually not the server that stores the sensor data permanently. We feel that such an additional piece of equipment raises the long-term cost of the WSN unduly and we will provide and Ethernet communication block to the gateway for direct TCP/IP connectivity to the cloud-based data storage system. The sensor node microprocessors worked well and proved to have more than enough power to handle the data transmission over the custom communication protocol. The protocol that was borrowed from our previous industrial use cases functioned as intended. It provided a secure and reliable communication link between the sensors and the cloudbased data storage. This was to be expected, as industrial environments are usually more challenging with regard to radio communication than school buildings. A possible downside of the protocol is that we may have to adapt it to better integrate with different cloud-based storage systems in the field of smart education as they are different in comparison to the industry. In summary, we are very happy with the preliminary results of our prototype WSN. They largely proved our initial assumptions and convinced us that the design considerations and general architecture of the nodes and the WSN are viable in the context of smart education. The tests also provided us with several important aspects that need further work to improve the functioning and ease of use of the WSN.

6 Conclusions The paper discusses the design, implementation and preliminary testing of the nodes and the architecture of a WSN intended for use in applications related to smart education. We borrowed from our previous experience with industrial use cases to create and test a customized solution for this new application field. Our contributions encompass the identification of some important design considerations and the definition of a suitable architecture. We also created a prototype implementation of the nodes and the WSN and tested it to confirm its feasibility and gain some insights for future improvement. In accordance with the experimental results, our future work will include the improvement of the modularity of the nodes, the reduction of the power usage and the increased availability of communication blocks for direct TCP/IP connectivity to cloud systems.

190

S. Ilchev

Acknowledgements. This research is supported by the Bulgarian National Science Fund (FNI) through the project “Modeling and Research of Intelligent Educational Systems and Sensor Networks (ISOSeM)”, contract KP-06-H47/4 from 26.11.2020. This work is supported by the Bulgarian Ministry of Education and Science under the National Research Program “Smart crop production” approved by Decision of the Ministry Council №866/26.11.2020.

References 1. Zeeshan, K., Hämäläinen, T., Neittaanmäki, P.: Internet of Things for sustainable smart education: an overview. Sustainability 14(7), 4293 (2022). https://doi.org/10.3390/su1407 4293 2. Zhu, Z., Yu, M., Riezebos, P.: A research framework of smart education. Smart Learn. Environ. 3, 4 (2016). https://doi.org/10.1186/s40561-016-0026-2 3. Jimenez-Liso, M., Martinez-Chico, M., Avraamidou, L., Lucio-Villegas, R.: Scientific practices in teacher education: the interplay of sense, sensors, and emotions. Res. Sci. Technol. Educ. 39(1), 44–67 (2021). https://doi.org/10.1080/02635143.2019.1647158 4. Kanna, S., von Rosenberg, W., Goverdovsky, V., Constantinides, A., Mandic, D.: Bringing wearable sensors into the classroom: a participatory approach [SP education]. IEEE Signal Process. Mag. 35(3), 110–130 (2018). https://doi.org/10.1109/MSP.2018.2806418 5. Amo, D., Fox, P., Fonseca, D., Poyatos, C.: Systematic review on which analytics and learning methodologies are applied in primary and secondary education in the learning of robotics sensors. Sensors 21(1), 153 (2021). https://doi.org/10.3390/s21010153 6. Bos, A., et al.: Educational technology and its contributions in students’ focus and attention regarding augmented reality environments and the use of sensors. J. Educ. Comput. Res. 57(7), 1832–1848 (2019). https://doi.org/10.1177/0735633119854033 7. Mehmood, R., Lee, H.: Towards building a computer aided education system for special students using wearable sensor technologies. Sensors 17(2), 317 (2017). https://doi.org/10. 3390/s17020317 8. Zhang, W., Jing, S., Gui, L., Zhang, Y.: Physiological data acquisition system for education assessment using wireless sensor network. In: 2008 International Conference on Information Technology and Applications in Biomedicine, Shenzhen, China, pp. 471–473 (2008). https:// doi.org/10.1109/ITAB.2008.4570554 9. Meah, K., Forsyth, J., Moscola, J.: A smart sensor network for an automated urban greenhouse. In: 2019 International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), Dhaka, Bangladesh, pp. 23–27 (2019). https://doi.org/10.1109/ICREST.2019.864 4079 10. Kassab, M., DeFranco, J., Laplante, P.: A systematic literature review on Internet of Things in education: benefits and challenges. J. Comput. Assist. Learn. 36, 115–127 (2020). https:// doi.org/10.1111/jcal.12383 11. Abdel-Basset, M., Manogaran, G., Mohamed, M., Rushdy, E.: Internet of things in smart education environment: supportive framework in the decision-making process. Concurrency Computat. Pract. Exper. 31, e4515 (2019). https://doi.org/10.1002/cpe.4515 12. Pachidis, T., Vrochidou, E., Kaburlasos, V., Kostova, S., Bonkovi´c, M., Papi´c, V.: Social robotics in education: state-of-the-art and directions. In: Aspragathos, N., Koustoumpardis, P., Moulianitis, V. (eds.) Advances in Service and Industrial Robotics. RAAD 2018. Mechanisms and Machine Science, vol. 67. Springer, Cham (2018). https://doi.org/10.1007/978-3-03000232-9_72

Design Considerations, Architecture and Implementation

191

13. Ramlowat, D., Pattanayak, B.: Exploring the internet of things (IoT) in education: a review. In: Satapathy, S., Bhateja, V., Somanah, R., Yang, X.S., Senkerik, R. (eds.) Information Systems Design and Intelligent Applications. Advances in Intelligent Systems and Computing, vol. 863, Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-3338-5_23 14. Al-Emran, M., Malik, S., Al-Kabi, M.: A survey of internet of things (IoT) in education: opportunities and challenges. In: Hassanien, A., Bhatnagar, R., Khalifa, N., Taha, M. (eds.) Toward Social Internet of Things (SIoT): Enabling Technologies, Architectures and Applications. Studies in Computational Intelligence, vol. 846. Springer, Cham (2020). https://doi. org/10.1007/978-3-030-24513-9_12 15. Kwet, M., Prinsloo, P.: The ‘smart’ classroom: a new frontier in the age of the smart university. Teach. High. Educ. 25(4), 510–526 (2020). https://doi.org/10.1080/13562517.2020.1734922 16. Paunova-Hubenova, E., Terzieva, V., Dimitrov, S., Boneva, Y.: Integration of game-based teaching in Bulgarian schools – state of art. In: Proceedings of the 12th European Conference on Game-based Learning ECGBL 2018, Sophia Antipolis, France, pp. 516–525 (2018), ISSN 2049-0992 17. Savov, T., Terzieva, V., Todorova, K.: Computer vision and internet of things: attention system in educational context. In: Rachev, B., Smrikarov, A. (eds.) Proceedings of the 19th International Conference on Computer Systems and Technologies (CompSysTech 2018), pp. 171–177. ACM, New York (2018). https://doi.org/10.1145/3274005.3274014 18. Boneva, Y.: Optimization of car traffic flow on intersections regulated by traffic lights through the simulation environment AIMSUN. Acad. J. Mech. Transp. Commun., 16(2) (2018). ISSN 1312-3823 (print), ISSN 2367-6620 (online), Todor Kableshkov University of Transport, Bulgaria, pp. I-1–I-9 19. Alexandrov, A., Monov, V.: Method for indoor localization optimization of AoA based mobile devices. In: Proceedings of 12th Annual Meeting of the Bulgarian Section of SIAM BGSIAM 2017, 20–22 December 2017, Sofia, Bulgaria (2017). ISSN: 1313–3357 20. Ivanova, V., Bachvarov, D., Boneva, A.: An advanced robot system for diagnostic and therapeutics tasks with application in laparoscopic surgery. J. Comput. Eng. Inform. Technol. 7(2), 1–9 (2018), ISSN: 2324–9307 (Online), SciTechnol, London, United Kingdom. https://doi. org/10.4172/2324-9307.1000198 21. Djambazova, E.: A fault-tolerant real-time system with adjustable reliability. In: Proceedings of the 22nd International Conference on Computer Systems and Technologies (CompSysTech 2021), pp. 76–80. ACM (2021). https://doi.org/10.1145/3472410.3472415 22. Djambazova, E.: achieving system reliability using reliability adjustment. In: Proceedings of the 23rd International Conference on Computer Systems and Technologies (CompSysTech 2022), pp. 64–68. ACM (2022).https://doi.org/10.1145/3546118.3546129 23. Ilchev, S., Ilcheva, Z.: Internet-of-things communication protocol for low-cost devices in heterogeneous wireless networks. In: 18th International Conference on Computer Systems and Technologies (CompSysTech 2017), vol. 1369, pp. 272–279. ACM ICPS, Ruse (2017), ISBN: 978–1–4503–5234–5. https://doi.org/10.1145/3134302.3134329

Physical and Digital Accessibility of Museums in Bulgaria: Problems and Innovative Technologies Vesela Georgieva1 , Galina Bogdanova1

, and Mirena Todorova-Ekmekci2(B)

1 Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Acad. Georgi

Bonchev Str., Block 8, Sofia, Bulgaria 2 Institute of Ethnology and Folklore Studies with Ethnographic Museum, Bulgarian Academy

of Sciences, 6A Moskovska Str., Sofia, Bulgaria [email protected]

Abstract. The present article relates to the topic of physical and digital accessibility in the context of museums in Bulgaria, as well as the presentation of innovative digital technologies for accessibility to cultural heritage sites and the creation of a conceptual model of a digital passport for access and accessibility of museums in Bulgaria in terms of people with special needs. The paper shows research and certification of the accessibility of museums in Bulgaria for people with disabilities and impaired vision. The text covers the following: the Bulgarian museum context; the application of innovative technologies in museums and different examples; problems and possible solutions to the physical and digital accessibility of museums in Bulgaria; innovations for physical and digital accessibility of museums in Bulgaria, review of good practices for accessibility certification, development of a conceptual passport model for museum accessibility, using a comprised, adapted methodology. Keywords: Physical and digital accessibility · Passportization · Cultural heritage · Museums · Innovative digital technologies · Disabilities · Blind

1 Introduction According to Eurostat data on the European Council site of the EU, 87 million people in EU live with a disability in 2022, which is 1 of every 4 adults. Protecting their rights and improving the environment of everyday life of people with disabilities is a priority for the EU and that includes access to services, knowledge and culture. The Cultural Heritage Act of the Republic of Bulgaria requires equal access to museum spaces for people with special needs. Innovative technologies and following certain accessibility guidelines and criteria can be useful for overcoming accessibility problems and barriers. This scientific article examines some of the existing barriers and challenges related to the physical and digital accessibility of museums in Bulgaria, as well as the socialization and adaptation of innovative technologies for people with special needs. There are problems in different functional areas: management, communication, marketing, and advertising, while at the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 Z. Kubincová et al. (Eds.): MIS4TEL 2023, LNNS 769, pp. 192–199, 2023. https://doi.org/10.1007/978-3-031-42134-1_19

Physical and Digital Accessibility of Museums in Bulgaria

193

same time, they all function in a specific way in museum work. Gradual socialization and digitalization depend on traditions, financial, personnel and management capabilities. Modern information and communication technologies (ICT) and digitization contribute to the development of museum work in the context of the preservation, protection and promotion of cultural heritage and the creation of new opportunities for presentation and accessibility. The presentation of the report includes: innovative technologies, problems, and possible solutions of the physical and digital accessibility of museums in Bulgaria; innovations for physical and digital accessibility of museums in Bulgaria, as well as the development of a conceptual passport model for museum accessibility. The research focus for digital accessibility of cultural heritage sites is on visually impaired and blind people.

2 Usage of Innovative Technologies in Museums An important part of the modern development and modernization of museums is the use of innovative technologies and providing access to the cultural heritage of various target groups. Book [1] examines current topics and issues related to modern technologies and their application in museums, as well as the gap between formal education and training and real work. Over time, museum users are no longer just consumers of cultural products but become active subjects of cultural content. On the other hand, digital innovations help build the infrastructure that increases opportunities for exchange, accessibility and participation, and museums gradually begin to adapt to the new dynamic changes. In the modern world, museums are expected to present and socialize cultural heritage, both onsite and online, internally, and externally, locally, and internationally, including collections and various content, objects, and artifacts. Digital technologies are a very useful tool to achieve the aims of museums in the different functional areas of the work of museums and their personnel. For example, augmented reality and the inclusion of various interactive educational games can have a revolutionary role. Designing and offering immersive environments and applied games, in the context of museums is important as “gaming encourages active participation, user interaction and loyalty, before and after the visit [1]. By using various databases, museums can present their cultural content in an innovative way that is often more effective, more informative, easy to perceive, find, research, preserve and disseminate. By using 3D scanning it is now possible to access various high-level digital archives, which in turn allow quick and easy access to a vast range of information from museum collections. This is useful for all interested parties – the general population, organizers of exhibitions, related institutions, scientific researchers, associations, tourism-related parties, and others. Digital culture contributes to the modernization, socialization, dissemination, and promotion of museum cultural heritage, identified as one of the key highlights of the future. Sharing cultural heritage online also creates opportunities for development and growth, an indirect marketing tool, useful for audience expansion. The urbanization, globalization and dynamics of our modern world also affect museum cultural heritage usage. Modern users expect better tools for fast and easy digital, distanced presentation and accessibility. Examples of that are digital platforms and collections, applications, software, virtual tours, virtual reality and smart AI research and recreation tools, education tools, modern innovations and ICT.

194

V. Georgieva et al.

Digital Repositories. Digital technologies are used for accumulating, organizing, and providing fast and easy access to vast amounts of cultural information to local, national, international audiences and concerned parties. They are important for the digital transformation of museums and memory institutions and for providing access and accessibility of larger audiences to cultural heritage, including an increasing percentage of people with special needs and disabilities. Digital technologies, tools and accessibility standards are important for safeguarding, socializing, and promoting cultural heritage and the development of museums. Not just museums, but also local, national, and international digital platforms have started to gather and present in one place vast amounts of cultural heritage data and collections of cultural objects and elements. Often it is important for local museums to be mentioned or to show some part of their collections on such collective digital platforms like Europeana in order to attract more visitors and attention. Online presentation and promotion have become an important part of the socialization of tangible and intangible cultural heritage of countries, as people use search tools online in order to find what they are looking for. Digital Transformation and Virtual Museums. In the past, there was speculation and discussions in the museum community that physical museums might be diminished by virtual ones with a decrease in the on-the-sight real visits and experience of people in museums. But even though the number of virtual museums and exhibits has vastly increased around the world, in fact, the data from various research, like [2, 3] shows that virtual museums, collections and exhibitions do not reduce museum attendance. Virtual museums and exhibitions contribute to increased awareness of physical museums, reaching a wider audience, increased attendance, and enhanced museum experience and engagement [3–5]. Digitization of museum collections, virtual exhibits, and electronic resources is discussed and presented in detail in the book [2], which explores a wide variety of museums and musicological approaches worldwide. Whether virtual museums should be included in the definition of a museum is a controversial question posed by museum professionals and musicologists. In the same book, virtual museums, the hypothesis is that this type of museum [2, 3]. Virtual museums provide opportunities to access information and make an important contribution to society. It is also significant that the museums are perceived by their users as meaningful to them and provide opportunities for interaction with actual objects.

3 Problems and Solutions for Physical and Digital Accessibility 3.1 Some Problems and Research of the Accessibility of Bulgarian Museums The Cultural Heritage Act of the Republic of Bulgaria requires equal access to cultural heritage for all. To realize accessibility for people with disabilities, architectural, technological, and software solutions are needed, which depend on the type of disability (wheelchairs, elevators, ramps, marked paths, sound and light signaling, Braille inscriptions, screen readers, audio guides, magnifiers, glasses, etc. In 2015–2016, a team from the Institute of Mathematics and Informatics at the Bulgarian Academy of Sciences, together with volunteers from 3 groups of people with special needs: visual, physical, and cognitive difficulties, conducted field surveys of part

Physical and Digital Accessibility of Museums in Bulgaria

195

of the 100 national sites list in Bulgaria (over 100 sites). The research results showed physical inaccessibility for the majority of the national sites. One of the goals of the research was related to the perception of museums in an accessible way, working in favor of people with special needs and with the aim of evaluating museum accessibility and the use of modern assistive devices and technologies for accessibility of people with disabilities. Collected data from the field surveys of physical accessibility of the sights was analyzed and evaluated, in order to draw conclusions and recommendations for solutions towards better socialization and accessibility. A large part of the researched museum objects turned out to be completely inaccessible to people with mobility problems. Tactile accessibility for people with visual impairments was also researched and results showed a lack of such accessibility almost everywhere. Subsequently, in 2022, regional museums from the mentioned sites were tested for digital accessibility (sites, software applications, documents, etc.) for people with impaired or lost vision in several ways - by means of automated software (WAVE), by volunteers with visual problems, and the conclusion was that over 75% of the sites were largely inaccessible to people with visual disabilities. Most regional museum websites did not sufficiently meet the criteria for web accessibility [6, 7]. In some of the cases, solutions for physical accessibility and at the same time preservation of the uniqueness of the traditional museum and site look with old infrastructure cannot be found (Tsarevets - Veliko Tarnovo, Bulgaria, Regional Open Air Ethnographic Museum - ETARA - Gabrovo, Bulgaria, architectural and historical reserve “Bozhentsi” - Gabrovo region, Bulgaria, tombs from the Valley of the Thracian Kings in Kazanlashko, etc.). In these cases, alternative solutions were offered and sought depending on the disability (audio guides, photos, video films, 3D models, etc.). Almost all tested sites had no online audio guides to provide accessibility for people with physical and visual disabilities, but many of them had virtual tours. Concerning the issues related to digital accessibility from the perspective of museums and other memory institutions, it is important for them to consider the opportunities that digital new technologies, websites and software provide for better access and accessibility at all levels and for all users, including disabled people. Problems we found related to the topic are: lack of a methodological guide/s for digital accessibility for museum specialists; knowing in depth the essence of the problem and workers’ capacity building for improvement; digital access and usability for various groups of users of the museums in Bulgaria. There are both physical access issues in museums and issues concerning using digital new tools and technologies. Our observation and research results showed that the museums in Bulgaria have focused more on projects for developing physical facilities and helping tools for people with disabilities visiting the museums on sight. Less has been done for improving the digital accessibility of their cultural and historical heritage and many of the sites of museums that are being renewed and developed in recent years do not take as a priority in providing digital accessibility and functionality on the websites for people with different disabilities. Evaluating the digital accessibility of sites, as mentioned above, is difficult to do, mostly due to the need of using qualified specialists on the topic at the location as well as on the sites. Specialized testing software for accessibility can also be used, but mostly for basic digital accessibility, not in-depth testing, and analysis. It can be discussed

196

V. Georgieva et al.

if the museums currently have the necessary resources provided for the presentation of cultural heritage for people with special needs and disabilities, including the blind, deaf and mute, and those with physical disabilities and mobility difficulties. For each of the mentioned categories of disabilities, different and specific approaches for better accessibility are required. Due to the above-mentioned reasons and as a consequence of the public importance and functions of museums, it is necessary to ensure accessibility (physical and digital) to cultural artifacts and data for all types of museum audiences. Possible solutions include braille inscriptions, annotations, sound and tactile techniques to individual exhibits (for the blind); specialized visual and digital tools (for people with impaired or missing hearing); auxiliary ramps (for people with reduced mobility), etc. 3.2 Some Solutions for Physical and Digital Accessibility Digital Audio Guide System with Accessibility. In 2022 to 2023, a digital audio guide system with braille signs and QR codes was created as a joint project of our accessibility research team from the Institute of Mathematics and Informatics at the Bulgarian Academy of Sciences and the Regional Ethnographic Museum in the city of Plovdiv, Bulgaria. The project and the resulting digital-audio system with accessibility (digital and audio guide in the museum with additional cultural heritage information), was an example solution to some of the above-mentioned accessibility problems for museums. The digital website https://ethnograph.info/en/audio-guide/ also safeguards and presents information on local traditions and related to them historical objects and cultural artifacts via informative stories. The methodology of work can be used for other museums and memory institutions on a national and international scale and is included on a website ecosystem platform with methods, guidelines and good examples for digital accessibility: http://www.math.bas.bg/vt/ab/index-en.php. Digitalization, Socialization and Accessibility of Intangible Cultural Heritage. Museums in Bulgaria often have much more artifacts and information of tangible and intangible cultural heritage than they can exhibit in reality in the physical exhibition halls of the museums. Digital presentations and platforms like the abovementioned audio guide with traditional stories with multimedia and sound help for socializing and showing more cultural heritage to a wider audience on a national and international scale, including people with disabilities. Moreover, museums of the past often focus more on showing physical artifacts in static form behind glass displays, while digital technologies give better opportunities for preserving, presenting, understanding and enhancing the experience of local cultural traditions, rituals, context and way of life in an attractive and engaging way via vast categorized repositories with text, photo multimedia audio and video, digital screens, 3D models, virtual tours, augmented and virtual reality, smart games, etc. Accessibility Passportization Model with Related International Researches and Examples. A possible solution for assessing and overcoming the problems related to access and accessibility to museum objects and cultural historical knowledge in Bulgaria is the usage of a conceptual passport model for accessibility. The methods and good practices used in other countries to modernize cultural institutions and the need to

Physical and Digital Accessibility of Museums in Bulgaria

197

ensure and assess their accessibility for people with special needs are explored. Several international research papers, publications and developments on the topic of accessibility for people with special needs are considered in the following research for developing a passport model for accessibility for Bulgarian museums [6, 8]. Other valuable local and international research results, related to understanding and improving accessibility for people with special needs can also be found in the following papers [9–13]. The valuable experience of project ARCHES [14] helps European museums to become barrier-free with 3D art replicas, mobile phone apps, games and sign language video avatars. The Institute for Vocational Rehabilitation and Training of Personnel has also researched the accessibility of blind people [15] and afterward developed an accessibility passport for the visually impaired [16]. Passportization of social infrastructure and services in Russia is included in the state program for an accessible environment for 2011–2020 (Order of the Ministry of Culture of Russia dated November 16, 2015 No. 2800). In 2012, the UN Convention on the Rights of Persons with Disabilities was ratified. The passport shows to what extent the institution and its services are accessible to people with disabilities. There is a methodological guide for ensuring accessibility to objects and services for people with disabilities: prepared by the author’s team of the Interregional Resource Center “Accessible World” with the aim of spreading ideas, principles, and means for creating an accessible environment; describing the legal, organizational and methodological aspects of the certification of objects and services for assessing their accessibility and developing management solutions aimed at increasing accessibility indicators, both at the territorial level and at the level of institutions and organizations. Accessibility Passportization Model for Bulgarian Museums. Based on the abovementioned studies, results, guidelines and examples, we analyzed all and gathered the useful criteria and points for developing an adapted, appropriate methodology for certifying the accessibility of museums for people with disabilities in Bulgaria, suitable to the local context. We also prepared a questionnaire for the needs of the passport nation of museums, including the main criteria for digitalization and accessibility. A preliminary conceptual model of a digital accessibility passport of museums in Bulgaria (Regional Ethnographic Museum - Plovdiv) for people with special needs is presented. It consists of five main sections: (1) General information about the object (name, address, year of construction of the building, form of ownership, territorial affiliation, etc.); (2) Characteristics of the activity of the cultural institution (types of services, the form of service, age categories of visitors, planned capacity, etc.); (3) The state of accessibility at the site for people with disabilities and other persons with reduced mobility (route to the site, distance from public transport stops, presence of a pedestrian path, functionality when accessed, etc.); (4) Management decision - proposals for adapting the main structural elements of the site (recommendations for adapting the entrances of the buildings; adjacent territory; physical accessibility for people with disabilities); (5) Digital accessibility for people with disabilities. The next stages of our research include: testing and implementation of an accessibility passport in five sections; ensuring accessibility in a virtual environment; establishing

198

V. Georgieva et al.

good practices; obtaining information about the accessibility of museum objects; opportunities for improvement; passportization of museum accessibility for different types of disabilities at the national level.

4 Conclusion The article analyzes some of the challenges and existing barriers to the accessibility of museums in Bulgaria, as well as presents the application of modern technologies to improve the physical and digital accessibility of museums and their cultural heritage. The focus of the research is on digital accessibility for visually impaired and blind people. The text of the paper covers the Bulgarian museum context; the application of innovative technologies in museums; problems and possible solutions to the physical and digital accessibility of museums in Bulgaria; innovations for physical and digital accessibility of museums in Bulgaria, as well as the development of a conceptual passport model for museum accessibility. Documents related to practices used in museums for accessibility certification for different types of disabilities in different countries were reviewed. The article also proposes some possible solutions for accessibility and better presentation of cultural traditional heritage, such as digital audio guide system in museums with QR codes and braille signs, suitable for people with visual impairments and a conceptual model of a passport for the accessibility of museums in Bulgaria for people with special needs. A methodology for passportization of the accessibility of the museums in Bulgaria is offered, based on the researched methods. Acknowledgements. The research is financed under the project “Digital Accessibility for People with Special Needs: Methodology, Conceptual Models and Innovative Ecosystems”, grant contract No. KP-06-H42/4, dated 08.12.2020, by the National Science Fund of Bulgaria and the National program of the Ministry of education and culture “Young scientists and post-doctoral students 2” (206/07.04.2022).

References 1. Museum of the Future Insights and reflections from 10 international museums. Book of Mu.SA Consortium. http://www.project-musa.eu/. Accessed 05 May 2023 2. Latham, K.F.: Foundations of museum studies: evolving systems of knowledge (2014) 3. Styliani, S., Fotis, L., Kostas, K., Petros, P.: Virtual museums, a survey and some issues for consideration. J. Cult. Heritage 10, 520–528 (2009) 4. Caspan, S.S.: Virtual museums as digital storytellers for dissemination of built environment: possible narratives and outlooks for appealing and rich encounters with the past. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W5, 113–119 (2017) 5. Borisova. V.: Cultural heritage digitization and related intellectual property issues. J. Cult. Herit. 34, 145–150 (2018) 6. Todorov, T., Bogdanova, G., Todorova–Ekmekci, M.: Accessibility of Bulgarian regional museums websites. Int. J. Adv. Comput. Sci. Appl. 13(3), 404–408 (2022)

Physical and Digital Accessibility of Museums in Bulgaria

199

7. Todorova-Ekmekci, M., Todorov, T., Sotirova-Valkova, K.: Usage of innovative technologies and online media tools for digital presentation of cultural heritage in Bulgaria. Digital Presentation and Preservation of Cultural and Scientific Heritage, vol. 11, Sofia, Bulgaria: Institute of Mathematics and Informatics – BAS, pp. 303–308 (2021) 8. Bogdanova, G., Sabev, N., Tomov, Z., Ekmekci, M.: Physical and digital accessibility in museums in the new reality. In: 5th International Symposium on Multidisciplinary Studies and Innovative Technologies, Ankara, Turkey, pp. 404–408 (2021) 9. Baule, S.M.: Evaluating the accessibility of special education cooperative websites for individuals with disabilities. TechTrends 64(1), 50–56 (2019). https://doi.org/10.1007/s11528019-00421-2 10. Díaz-Rodríguez, N., Pisoni, G.: Accessible cultural heritage through explainable artificial intelligence. In: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, pp. 317–324 (2020) 11. Pisoni, G., Díaz-Rodríguez, N., Gijlers, H., Tonoll, L.: Human-Centered Artificial Intelligence for Designing Accessible Cultural Heritage. Appl. Sci. 11(2), 870 (2021) 12. Deffner, A., Psatha, E., Bogiantzidis, N., Mantas, N., Vlachaki, E., Polyxeni N.: Accessibility to culture and heritage: designing for all (2015) 13. Hayhoe, S., Carrisoza, H.G., Rix, J., Sheehy, K., Seale, J.: A survey of networked and Wi-Fi enabled practices to support disabled learners in museums. In: 2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 197–202 (2019) 14. Project ARCHES Homepage. http://www.arches-project.eu/. Accessed 05 May 2023 15. Collection of social networks and their opportunities for the visually impaired (materials of the XIX scientific-practical conference, 13/11/2019, Under the general editorship of prof. S.N. Vanshina/, Institute of Vocational Rehabilitation and Personnel Training, VOS “Reacomp”, p. 64 (2020). http://rehacomp.ru 16. Passport of accessibility developed in the Institute of Professional Rehabilitation and Staff Training. http://rehacomp.ru/wp-content/uploads/2020/02/Pasport-dostupnosti.pdf. Accessed 18 Aug 2022

Educational Technologies and Video Algorithms at Medical University – Varna, Bulgaria Diana Dimitrova1 , Galina Bogdanova2(B) , Galya Georgieva-Tsaneva3 and Evgeniya Gospodinova3

,

1 Medical University – Varna, Branch Veliko Tarnovo, Bulgaria

[email protected]

2 Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Sofia, Bulgaria

[email protected]

3 Institute of Robotics, Bulgarian Academy of Sciences, Sofia, Bulgaria

{galitsaneva,jenigospodinova}@abv.bg

Abstract. Video-learning (in the form of video films), when properly integrated into the curriculum, can make learning much more engaging and successful for medical and healthcare students. The aim of the present study was to compare the efficacy of learning practical skills (Holter placement) in Nursing students. The students were divided into two groups of 8 students each. The first group was given an algorithm on paper, and the second group - a specially made video film. The results showed that the acquisition of this medical manipulation was more effective in the students having access to the video film. 87.5% of the students who watched the film did well on the post-test, compared to 50% of the second group. When retesting after 14 days, the results show durability of the knowledge gained (